Fraud Bot

Ambient fraud detection agent for Warp. Runs daily to identify new and emerging fraud patterns.

Overview

This repo contains instructions and SQL queries that an AI agent uses to:

1. Run known fraud detection patterns against Metabase
2. Look for new/emerging fraud clusters
3. Report findings to Slack

Setup

Prerequisites: the Oz CLI (oz), authenticated with oz login (or WARP_API_KEY in CI). You bring your own data sources — a Metabase instance, a BigQuery warehouse matching SCHEMA.md, and Slack (optionally Front/Verisoul) tokens. This repo ships no credentials.

1. Point the schema config at your warehouse

Copy config/schema.json to config/schema.local.json (git-ignored) and set your project, datasets, tables, and Metabase database IDs. See SCHEMA.md for the column contract each logical table must satisfy.

2. Register secrets

Store these as team-scoped Oz agent secrets (injected as environment variables at runtime):

oz secret create --team METABASE_API_KEY   # prompts for the value
oz secret create --team SLACK_BOT_TOKEN
# Optional, for the appeals / verification skills:
oz secret create --team BOT_FRONT_API_KEY
oz secret create --team SLACK_COMMUNITY_BOT_TOKEN
oz secret create --team VERISOUL_API_KEY

Running locally instead of on Oz? Export the same names as environment variables — the scripts read them directly (e.g. export METABASE_API_KEY=...).

3. Create an environment

Create an Oz environment that includes this repo (optionally your own server / data-model repos for extra context), then find its ID:

oz environment list

4. Run

One-off cloud run:

oz agent run-cloud \
  --environment <ENV_ID> \
  --prompt "Read instructions.md and run daily fraud detection analysis"

Or run on a daily schedule:

oz schedule create \
  --name "Daily Fraud Detection" \
  --cron "0 9 * * *" \
  --environment <ENV_ID> \
  --prompt "Read instructions.md and run daily fraud detection analysis"

Structure

• Agent instructions (pick the cadence you want to run):
  • instructions.md — daily fraud report (24h window → CSV + Slack summary)
  • instructions_v2.md — in-depth investigation workflow (~8h cadence)
  • instructions_signup_spike.md — lightweight signup-spike monitor (every 30 min)
• queries/ - SQL queries for fraud detection
  • known_patterns.sql - Known fraud email/domain patterns (template)
  • domain_clustering.sql - Suspicious email domain analysis
  • velocity_detection.sql - High-speed credit burning
  • language_clustering.sql - Non-English language clusters
  • signup_bursts.sql - Rapid signup detection
  • prompt_similarity.sql - Coordinated prompt patterns
• data/ - Reference data (confirmed fraud accounts)
• config/schema.json - Logical→physical mapping for project, datasets, tables, and Metabase DB IDs
• SCHEMA.md - Schema/config layer docs and the column contract each logical table must satisfy
• scripts/schema.py - Loads the config and resolves schema placeholders in SQL
• scripts/ - Python helpers (api_client.py for Metabase/Slack, apply_front_tags.py, check_slack_community_activity.py) — Python standard library only at runtime

Skills

Two complementary skill formats ship with this repo; use whichever your agent runtime supports:

• .agents/skills/ — a granular suite, one directory per capability:
  • Detection / investigation: ip-cluster-analysis, prompt-injection-denylist, user-graph-traversal, user-convo-summary
  • Ban-appeal enrichment (optional Front / Slack / Verisoul integrations): front-appeals-inbox, slack-community-check, verisoul-lookup
• .warp/skills/fraud-checker/ — a single consolidated skill that mirrors the instructions_v2.md investigation workflow (with references/ schema + query notes). It intentionally overlaps with the instructions and queries above: prefer the .agents/skills/ suite for granular use, and treat fraud-checker as the all-in-one entry point.

Data Access

The agent queries Metabase using two logical databases, mapped to your physical Metabase database IDs in config/schema.json:

• warehouse — core user/account/AI-usage data (BigQuery)
• ugc — user-generated content / message bodies

Table and project names are also logical: queries reference placeholders like {{ tables.core_user_facts }} that resolve from config/schema.json. To point the bot at your own warehouse, edit that file (or a git-ignored config/schema.local.json) and satisfy the column contract in SCHEMA.md.

Example Fraud Patterns (illustrative)

Signals this toolkit looks for (adapt to whatever you observe in your own data):

• Faker-style emails: firstname_lastname123@example.com
• Windows OS almost exclusively
• Credits cluster at tier limits (150, 300, 500, 600)
• Model-forcing prompt-injection strings (fixed, non-natural text that pins a specific model)
• Many accounts referencing the same shared localhost endpoint
• Suspicious email domains (track your own confirmed-bad list)