Epic: deepfake-guard

[CodesTree]

Epic: deepfake-guard

Overview

DeepfakeGuard intercepts AI-cloned-voice-prompted transfers inside the TNG eWallet payment flow. When a user initiates a transfer above RM 200 to a new contact, they are invited to upload the audio/video that prompted the transfer. The system analyses the media and text in under 3 seconds via a multi-cloud AI pipeline and returns a Low / Caution / Block verdict with specific cue callouts. All intercept events are logged immutably.

Demo story (2 minutes): A user about to send RM 2,000 uploads a synthetic WhatsApp voice note → DeepfakeGuard returns Block in 1.3s → intercept overlay shows "AI voice detected · Urgency language · First-time recipient" → user taps "Call to verify" → dashboard shows aggregate RM value protected.

Architecture Decisions

Decision	Choice	Rationale
Audio model	garystafford/wav2vec2-deepfake-voice-detector (HuggingFace)	Pre-trained on ASVspoof corpus, no fine-tuning needed, >90% accuracy on TTS artifacts
Audio inference host	AWS SageMaker HuggingFace endpoint	Managed hosting, fast cold-start avoidance via pre-warm, IAM-gated
NLP scoring	AWS Bedrock (Claude Haiku)	Lowest latency Bedrock model, structured JSON output, no fine-tuning
Risk fusion	Alibaba Cloud PAI — XGBoost (3 features)	Satisfies multi-cloud requirement; XGBoost trivially trainable on synthetic data in <30 min
Backend runtime	FastAPI + Mangum adapter → AWS Lambda + API Gateway	Serverless, no idle cost, parallel SageMaker + Bedrock calls in one handler
Audit ledger	AWS QLDB	Cryptographically verifiable immutable log; satisfies BNM AML/CFT framing
Monitoring	Alibaba Cloud ARMS	Real-time latency + false positive proxy rate; ARMS dashboard is Scene 5 of the demo
Threat intel store	Alibaba Cloud Security Center + MaxCompute	Security Center for live signals; MaxCompute for batch analytics shown in pitch
Recipient risk	AWS Fraud Detector	Provides new-account / high-fan-in signal as a 4th input to the fusion model
Frontend	React (web)	Fastest to build and demo; no React Native device constraints
Audio privacy	Ephemeral processing only	PDPA 2010 compliance: no audio or voice embeddings persisted post-analysis

Technical Approach

Frontend Components

• TransferFlow — mock TNG transfer screen (amount, recipient, confirm). Threshold check client-side: if amount > RM 200 AND recipient has no prior transactions, show DeepfakeGuard prompt.
• DeepfakeGuardPrompt — optional audio upload (WAV/MP3/OGG/M4A ≤5MB) + text paste field + PDPA consent checkbox. Calls POST /analyze.
• InterceptOverlay — verdict card showing risk level badge, detected cue chips, "Call to verify" CTA, "Report scam" CTA, "Proceed anyway" secondary button.
• Dashboard — aggregate view: transfers blocked count, RM value protected, top signal types (synthetic data). Scene 5 of the judge demo.

Backend Services

• POST /analyze — accepts multipart/form-data with audio_file (optional), text (optional), transaction_metadata (JSON: amount, recipient_id, is_new_contact, device_changed). Orchestrates:
  1. SageMaker InvokeEndpoint (audio) + Bedrock InvokeModel (text) in parallel via asyncio.gather
  2. PAI EAS endpoint (risk fusion) with all scores
  3. Returns { risk_level, detected_signals, confidence, latency_ms }
• POST /report — accepts confirmed scam flag; writes QLDB document + Security Center custom threat signal.
• GET /dashboard — returns synthetic aggregate stats (pre-seeded in DynamoDB or hardcoded for demo).
• QLDB writer — called on every Block/Caution verdict: logs timestamp, anon user ID, risk level, signals, amount, user action.

Infrastructure

• SageMaker: HuggingFace model garystafford/wav2vec2-deepfake-voice-detector, instance ml.g4dn.xlarge, endpoint pre-warmed at H4. IAM role restricts invocation to Lambda execution role only.
• Bedrock: Claude Haiku (anthropic.claude-haiku-4-5-20251001), structured prompt returning { urgency_score, impersonation_cues[], risk_label }.
• PAI EAS: XGBoost model trained on 500-row synthetic dataset (audio_score, nlp_score, txn_metadata_score → risk_label). Training job at H8, EAS deployment at H10.
• QLDB: Ledger deepfake-guard-audit, table InterceptEvents.
• ARMS: Application monitoring dashboard tracking inference_latency_p95 and false_positive_proxy_rate metrics.
• MaxCompute: Table scam_signals receiving confirmed scam records from /report.
• Lambda: Python 3.12, 1024MB, 30s timeout. Deployed via AWS SAM or direct zip upload.
• API Gateway: HTTP API (not REST) for lower latency.

Implementation Strategy

Build in parallel streams starting at H2. Integration milestone at H14. Dry-run at H18. Code freeze H20.

Stream	Owner	Window	Depends on
ML pipeline (SageMaker endpoint + accuracy validation)	ML lead	H2–H14	—
Synthetic data generation (audio + transaction)	Float	H0–H4	—
Backend API (Lambda + SageMaker + Bedrock orchestration)	Backend lead	H2–H14	Synthetic data (H4)
Risk fusion (PAI XGBoost train + EAS deploy)	Integration lead	H8–H14	Synthetic data (H4)
Audit + monitoring (QLDB + ARMS + MaxCompute)	Integration lead	H8–H14	—
Frontend (React mock + intercept overlay + dashboard)	Frontend lead	H4–H16	Synthetic data (H4)
End-to-end integration	All	H14–H18	All above
Demo prep (dry-run, backup video, pitch deck)	Float + all	H16–H24	Integration complete

Task Breakdown Preview

#	Task	Parallel?	Conflicts with
001	Synthetic data generation — TTS scam audio (5 samples) + real audio negatives (5 samples) + mock transaction dataset	Yes	—
002	ML pipeline — download Wav2Vec2 model, deploy SageMaker HuggingFace endpoint, validate ≥85% TPR on synthetic audio set	Yes	—
003	Backend API — FastAPI + Mangum, POST /analyze with parallel SageMaker + Bedrock calls, POST /report, GET /dashboard, deploy to Lambda + API Gateway	Yes	—
004	Risk fusion — generate PAI training dataset, train XGBoost model on PAI, deploy to EAS, wire into POST /analyze	Yes	003 (integration point)
005	Audit + monitoring — QLDB ledger + InterceptEvents table, ARMS dashboard (latency + FP proxy), MaxCompute scam_signals table	Yes	—
006	Frontend — React TNG mock: TransferFlow, DeepfakeGuardPrompt (PDPA consent + upload), InterceptOverlay, Dashboard	Yes	001 (needs mock txn data)
007	End-to-end integration — wire frontend → backend → SageMaker + Bedrock + PAI + QLDB + ARMS, validate full 5-scene demo flow	No	all above
008	Demo prep — backup demo video (H16), full dry-run (H18), pitch deck finalisation, SC-05 rehearsal	No	007

Parallelization: Tasks 001–006 can all run concurrently. Task 007 gates on all of them. Task 008 gates on 007.

Dependencies

• HuggingFace Hub: garystafford/wav2vec2-deepfake-voice-detector weights available at H2
• ElevenLabs / Coqui TTS: synthetic scam audio generation before H4
• AWS account with SageMaker, Bedrock (Claude Haiku), Lambda, API Gateway, QLDB, Fraud Detector enabled
• Alibaba Cloud account with PAI, ARMS, Security Center, MaxCompute enabled
• Python 3.12, FastAPI, Mangum, boto3, alibabacloud-sdk

Success Criteria (Technical)

• SageMaker endpoint live and pre-warmed before judging window
• POST /analyze returns verdict in ≤3s end-to-end (measured in ARMS)
• Audio TPR ≥85% on the 5-sample synthetic scam audio set
• QLDB immutable log records a verifiable entry for every Block/Caution verdict
• PAI EAS endpoint substantively contributes to the risk verdict (not bypassed in demo)
• All 5 judge demo scenes execute without error in dry-run at H18
• Backup demo video recorded by H16

Estimated Effort

Stream	Effort	Risk
Synthetic data	2h	Low — TTS tools are fast
ML pipeline	8h	Medium — SageMaker cold-start + accuracy validation
Backend API	10h	Low — FastAPI + Lambda is well-understood
Risk fusion (PAI)	6h	Medium — PAI EAS setup time is variable
Audit + monitoring	4h	Low — QLDB + ARMS are straightforward
Frontend	10h	Low — React mock, no real TNG SDK
Integration	4h	High — multi-cloud wiring is the most failure-prone step
Demo prep	4h	Low

Total: ~48 person-hours across 5 people in 24 hours. Feasible with the parallel stream plan above.

Tasks Created

• 001.md - Synthetic Data Generation (parallel: true)
• 002.md - ML Pipeline — SageMaker Wav2Vec2 Endpoint (parallel: true)
• 003.md - Backend API — FastAPI Lambda Orchestration (parallel: true)
• 004.md - Risk Fusion — Alibaba Cloud PAI XGBoost (parallel: true)
• 005.md - Audit & Monitoring — QLDB, ARMS, MaxCompute (parallel: true)
• 006.md - Frontend — React TNG Mock UI (parallel: true)
• 007.md - End-to-End Integration (parallel: false)
• 008.md - Demo Prep — Dry Run, Backup Video, Pitch Deck (parallel: false)

Total tasks: 8
Parallel tasks: 6
Sequential tasks: 2
Estimated total effort: 48 person-hours

Total: ~48 person-hours across 5 people in 24 hours. Feasible with the parallel stream plan above.