Developer Infrastructure · CI, Test & Release Systems for GPU and Inference Workloads
I build the systems that let other engineers ship performance-critical GPU code with confidence:
release-blocking correctness gates, test harnesses that keep GPU kernels verifiable on CPU-only CI,
and the observability layer underneath LLM serving.
Contributing to vLLM — the inference engine behind a large share of production LLM serving.
| Pull request | What it does | Status |
|---|---|---|
#49366[Bugfix] |
Aria produces garbage output at tensor_parallel_size > 1. Root-caused a double all-reduce of shared-expert output: the MoERunner contract requires unreduced per-rank partial sums, but Aria built its MLP without reduce_results=False. Audited all nine other in-tree MoE models to establish the invariant and confirm Aria was the sole outlier. Fix is 3 lines. |
Open, under review |
#48999[Model] |
Adds MiniCPM-SALA — a 9.5B, 32-layer hybrid model (24 gated-linear "Lightning" attention layers + 8 GQA that switch to InfLLM-V2 block-sparse at long context). +1,630 lines, 34 CPU-only tests, greedy-token output identical to the HuggingFace reference on A100, TP=1/2/4 engine parity. | Open, under review |
Validation logs, compatibility matrix, and one-command reproduction scripts: vLLM-HybridAttn
Custom kernels are hard to test because the hardware is scarce and CI runners don't have it. My pattern across every kernel project:
Write a NumPy reference implementation first, make it the correctness oracle, then gate the kernel against it bit-exactly — and dispatch CPU/GPU so the suite runs everywhere.
The result is that contributors without a GPU can still verify correctness, and GPU scheduling stops being a source of flaky CI failures.
| Project | The gate | Measured |
|---|---|---|
| INT4 KV-Cache Quantization + Fused Flash-Attention CUDA · Triton · PyTorch |
48 tests run with no GPU, 50 on hardware; Triton port validated in CI on CPU runners via interpreter mode | 0.000% quantizer bin disagreement · attention MAE 3.1e-8 · 3.98× KV compression · 3.9× kernel speedup |
| CUDA Speculative Decoding Optimizer CUDA · PyTorch |
Six kernels each gated by bit-exact equality vs NumPy, including tie-breaking behavior | 31/31 on GPU · 29 on CPU-only runners |
| GPU Memory-Aware Request Scheduler CUDA · SLA scheduling |
Bit-exact gather → D2H → H2D → scatter round-trip gate | Swap-out 0.62 ms / swap-in 0.50 ms per 4 MB · admit rate 31.3% → 100% |
| PagedKV-Fusion CUDA · INT8 paged attention |
Reference-validated eviction scoring + quantized decode attention | Eviction 1.71 ms → 0.56 ms at 16K blocks (~3×) · ~50% KV memory |
- Real-Time Feature Store — a production-readiness audit that surfaced five real concurrency bugs, each reproduced with measurements and each given a permanent regression test. Parquet read-modify-write lost 174/200 records under 8 concurrent writers; every worker replica consumed every partition; online store applied last-write-wins by arrival order instead of
event_time. - AI Inference Observability Platform — FastAPI proxy exposing per-request TTFT / TBT / E2E for vLLM without forking the engine. P99 finalize overhead 46.1 µs → 20.1 µs (56%), SSE fast path 670 → 2,030 ops/sec. Publishes the proxy's own overhead cost alongside the wins.
- Agentic Verification Triage System — automated regression-failure triage: parses logs into failure signatures, clusters related failures, drafts prioritized bugs, runs a critic pass to suppress false positives. Includes tests asserting that generated drafts are labeled non-LLM and that unavailable seeds stay
nullrather than being fabricated. - KubeInfer — production Helm chart for multi-replica vLLM on Kubernetes: session-affinity routing for KV reuse, queue-depth autoscaling on
vllm:num_requests_waitingrather than CPU, NetworkPolicy, RBAC, 12 Prometheus alert rules, runbooks. - LLM-HDL-Bench — 46-module SystemVerilog benchmark requiring agreement from two independent tools (Yosys + Icarus) before a result counts. Reports 46/46 post-fix and 45/46 with the human fix reverted.
Because "seems better" and "is better" are different claims.
- A/B Test Guardrail Metrics — SRM gating, pre-registered metrics, Holm correction validated against the statsmodels reference. Applied to three real public experiments producing three different failure modes: a correctly-blocked regression, an honest "inconclusive," and a data-quality bug SRM alone could not detect.
- Toxic Content Prevalence & Intervention Impact — weighted difference-in-differences on 45.6M real Reddit comments. Reports the label-noise sensitivity that exceeds the point estimate, rather than burying it.
- Causal Inference Toolkit — DiD, Synthetic Control, and BSTS behind one API with ground-truth validation and placebo tests. Surfaces 69% cross-method disagreement instead of averaging incompatible answers.
Every metric on this profile is reproducible from a committed artifact in the repository it belongs to. Where something is unmeasured, the repo says so — several READMEs explicitly retract earlier claims that turned out to be simulated or leakage-inflated, and label pending work as pending rather than done.
AI Engineer, Developer Tooling & Semiconductor Automation at SwanTech AI M.S. Applied Data Science, University of San Diego · GPA 3.9
Open to developer-infrastructure and inference-systems roles · apatil@sandiego.edu · LinkedIn