Adversarial Testing Platform

A full-stack platform for adversarial robustness evaluation of image classifiers using IBM Adversarial Robustness Toolbox (ART) and Foolbox.

Architecture

adv-test-platform/
  backend/     FastAPI async REST API + RQ background workers
  frontend/    React 18 + TypeScript + Vite SPA
  docker-compose.yml

Services (docker-compose)

Service	Description	Port
db	PostgreSQL 16	5432
redis	Redis 7 (job queue + pub/sub)	6379
minio	MinIO S3-compatible object storage	9000 / 9001
api	FastAPI web server	8000
worker	RQ worker for attack jobs	—
frontend	React SPA served by Nginx	3000

Quick Start

# Clone / open the project
cd adv-test-platform

# Copy environment example
cp backend/.env.example backend/.env

# Build and start all services
docker compose up --build

# Open in browser
open http://localhost:3000
# API docs
open http://localhost:8000/docs
# MinIO console
open http://localhost:9001  (user: minioadmin / minioadmin)

Supported Attacks (13 total)

ID	Name	Category	Engine
fgsm	Fast Gradient Sign Method	gradient	art, foolbox
pgd_linf	PGD (L∞)	gradient	art, foolbox
pgd_l2	PGD (L2)	gradient	art, foolbox
deepfool	DeepFool	gradient	art, foolbox
cw_l2	Carlini & Wagner L2	gradient	art, foolbox
cw_linf	Carlini & Wagner L∞	gradient	art
hopskipjump	HopSkipJump	decision	art
boundary	Boundary Attack	decision	art, foolbox
square	Square Attack	score	art, foolbox
zoo	ZOO	score	art
auto_attack	AutoAttack	gradient	art
universal_perturbation	Universal Adversarial Perturbation	universal	art
jsma	JSMA	gradient	art

Supported Model Formats

Framework	Formats	Gradients
PyTorch	TorchScript (.pt) — trusted mode only	Yes
TensorFlow	Keras HDF5 (.h5), SavedModel	Yes
ONNX	.onnx	No
Black-Box	HTTP REST endpoint	No

Security

By default (TRUSTED_MODEL_MODE=false) the platform rejects Python pickle-based formats (.pkl, .pt, .pth) by checking both file extension and magic bytes. This prevents arbitrary code execution via malicious model files.

Set TRUSTED_MODEL_MODE=true to allow PyTorch TorchScript uploads from trusted sources.

API Endpoints

GET  /v1/catalog/frameworks
GET  /v1/catalog/attacks

POST   /v1/models              Upload model + metadata
POST   /v1/models/{id}/validate Forward-pass validation
GET    /v1/models/{id}
DELETE /v1/models/{id}

POST /v1/samples               Upload image
GET  /v1/samples/{id}

POST /v1/runs                  Create + queue run
GET  /v1/runs/{id}             Status
GET  /v1/runs/{id}/events      SSE stream (run.started / attack.progress / run.completed)
POST /v1/runs/{id}/cancel

GET /v1/runs/{id}/results           Per-attack metrics + robust accuracy curve
GET /v1/results/{result_id}/artifacts  Presigned URLs for images + heatmap

Running Tests

cd backend
pip install -r requirements.txt
pip install pytest pytest-asyncio httpx aiosqlite onnx

pytest tests/ -v

Development

Backend only

cd backend
uvicorn app.main:app --reload --port 8000
# In another terminal:
rq worker --url redis://localhost:6379/0

Frontend only

cd frontend
npm install
npm run dev

Environment Variables

Variable	Default	Description
DATABASE_URL	postgresql+asyncpg://user:password@localhost:5432/advtest	Postgres DSN
REDIS_URL	redis://localhost:6379/0	Redis URL
STORAGE_ENDPOINT	http://minio:9000	S3-compatible endpoint
STORAGE_BUCKET	adv-test-artifacts	Bucket name
STORAGE_ACCESS_KEY	minioadmin	S3 access key
STORAGE_SECRET_KEY	minioadmin	S3 secret key
SECRET_KEY	changeme	JWT signing key
TRUSTED_MODEL_MODE	false	Allow pickle-based model formats
DEBUG	false	Enable SQLAlchemy query logging
DEFAULT_TIMEOUT_SECONDS	900	Max job runtime
MAX_QUERIES	20000	Max model queries per job