Skip to content
View ashishki's full-sized avatar
🏠
Working from home
🏠
Working from home

Block or report ashishki

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Maximum 250 characters. Please don’t include any personal information such as legal names or email addresses. Markdown is supported. This note will only be visible to you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse
ashishki/README.md

Artem Shishkin

AI Systems Engineer focused on evaluation, reliability, and governed execution for LLM workflows.

I build AI backend systems where model and harness changes are evaluated, risky actions are controlled or reviewed, and cost, latency, and failure behavior remain observable.

Incoming MS in Financial Technology and Analytics at UT Dallas, Fall 2026.

Start here: Eval Ground Truth Lab · runnable reviewer demo · v0.2.1 evidence · known limits

Flagship: regression evaluation for LLM workflows

Eval Ground Truth Lab helps an AI engineer decide whether a model, prompt, tool, policy, or harness change is safe to release. It runs versioned datasets against baseline and candidate systems, applies deterministic quality, cost, and latency gates, and produces reviewable reports for CI and human approval.

Current maturity: released open-source alpha (v0.2.1) with local, synthetic, or curated evidence. v0.2.1 is an internal correctness/security release, not external-feedback maintenance evidence. It does not establish production traffic, customer adoption, or a production SLO.

System map

flowchart LR
    D[Versioned datasets] --> E[Eval Ground Truth Lab]
    G[gdev-agent\nreference workload] --> X[Workload adapters]
    T[Trader Risk Audit\nfinancial case] --> X
    X --> E
    C[Caller CI] --> Q[Reusable release gate Action]
    Q --> E
    R[Agent Runtime Grid\noptional failure proof] -. optional local path .-> E
    E --> P[Evidence pack\ngates, cost, latency, failures]
    P --> H[Human review\nand release decision]
    W[AI Workflow Playbook\ngovernance companion] -. contracts and receipts .-> E
Loading

The published reviewer umbrella provides the clean-checkout path across the locked Eval, gdev, and optional Runtime components. The gdev and sanitized Trader-to-Eval paths are reproduced local integrations; the reusable Action has a synthetic CI wiring proof. Dashed edges are optional relationships, not claims that a production platform, customer deployment, or production SLO exists.

Selected systems

System Role Best evidence Maturity
Eval Ground Truth Lab Regression evaluation and reusable CI release gates v0.2.1 exact-gate evidence Released alpha; local, synthetic, or curated evidence
Agent Runtime Grid Queue-backed execution with PostgreSQL lifecycle state, Redis Streams delivery, bounded retries, artifacts, and failure injection Evidence index Local reliability experiments
gdev-agent Governed support-workflow reference workload with approval, audit, cost controls, and PostgreSQL RLS Canonical Eval result Local reference workload; canonical quality gate currently fails
AI Workflow Playbook Independent governance and verification companion Guarantee maturity Mechanisms tested; adoption impact unproven

Engineering evidence

  • Versioned datasets and deterministic validators for reproducible regression gates.
  • Baseline/candidate comparison with explicit quality, cost, and latency thresholds, plus a least-privilege reusable Action v0.1.0.
  • PostgreSQL-authoritative job lifecycle state with Redis Streams delivery, bounded retries, cancellation, timeouts, and artifact integrity checks.
  • Human review and evidence indexes that separate local demonstrations, planned work, and external validation.
  • PostgreSQL owner/application-role separation with FORCE RLS and a clean-Compose cross-tenant denial proof for the reference workload.
  • A published one-command reviewer umbrella with exact component locks, tamper-checked evidence, readable previews, and an explicit pending-human-decision boundary.

Applied work

  • Trader Risk Audit applies deterministic post-trade policy checks to local or public-data examples. A history-preserving, path-purged standalone v0.2 candidate is verified locally, and Eval consumes its sanitized evidence through a documented adapter contract. The separate remote repository is not yet published. This is a case study, not a live trading system or financial advice.
  • Telegram Research Agent is a single-user research pipeline for source provenance, temporal threads, feedback, and bounded read-only assistance.

Current focus

  • Publish the verified standalone Trader v0.2 candidate when repository-creation access is available.
  • Validate one independently owned workflow adapter and collect two consented, structured feedback records.
  • Record one real independent umbrella review, then turn attributable external feedback into a bounded maintenance release and regression test.

Background

Before AI engineering, I worked as an international technical educator for Schwarzkopf Professional. I bring that training discipline to design reviews, technical writing, and cross-functional communication without treating it as a substitute for engineering evidence.

I begin the MS in Financial Technology and Analytics at UT Dallas in Fall 2026, with a growing focus on reliable AI systems for financial and document workflows.

Contact

LinkedIn · GitHub

Pinned Loading

  1. gdev-agent gdev-agent Public

    Local-first governed LLM support workflow with signed webhooks, guardrails, human approval, audit trails, cost controls, and deterministic evaluation paths.

    Python

  2. Agent-Runtime-Grid Agent-Runtime-Grid Public

    Local-first queue-backed runtime for reproducible AI job execution with Postgres lifecycle state, Redis Streams, bounded retries, artifacts, cost telemetry, and failure reports.

    Python

  3. Eval-Ground-Truth-Lab Eval-Ground-Truth-Lab Public

    Local-first regression evaluation for LLM and agent workflows with versioned datasets, deterministic validators, baseline comparison, cost and latency gates, and CI reports.

    Python

  4. trader-risk-audit trader-risk-audit Public

    Deterministic post-trade policy audit with explainable rule findings, evidence artifacts, and human-reviewed exceptions.

    Python