fix(storage): tolerate legacy trajectory shapes + harden reconcile#22
Merged
khaliqgant merged 5 commits intomainfrom Apr 13, 2026
Merged
fix(storage): tolerate legacy trajectory shapes + harden reconcile#22khaliqgant merged 5 commits intomainfrom
khaliqgant merged 5 commits intomainfrom
Conversation
Self-heals stale index.json when trajectory files are on disk but not tracked in the index. Walks active/ and completed/ (both flat root and YYYY-MM/ subdirs), adds missing entries, preserves existing entries. Ground-truth commit so downstream branches + worktrees start from a base that has reconcile + its tests. Schema loosening and further hardening stack on top in subsequent commits. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Relaxes on-read constraints so trajectories produced by external writers (notably the workforce workflow runner) can be loaded without manual migration, and closes the read/write validation asymmetry. **Schema loosening** - agents[].role: open string — accepts "workflow-runner", "specialist", and other domain-specific roles instead of the 3-value enum - Trajectory.id regex: /^traj_[a-z0-9_]+$/ — accepts legacy timestamp-hex ids like `traj_1775734701264_ba65c69b` - commits/filesChanged/tags default to [] when omitted - projectId is now optional - TrajectoryEventType gains a permissive TS fallback to match the zod side which already accepts unknown types **Save-path validation** FileStorage.save() now calls validateTrajectory() and throws on invalid input, closing the asymmetry that allowed writers to produce files the reader would later reject. **Reconcile hardening** - readTrajectoryFile returns a tagged ReadTrajectoryResult so callers can distinguish malformed JSON, schema violations, and IO errors. A readTrajectoryOrNull wrapper preserves ergonomics for get()/getActive(). - reconcileIndex walks completed/ recursively (flat root + YYYY-MM + any nesting), remains additive (preserves existing entries), and returns a structured ReconcileSummary. - Summary is logged as a single `[trajectories] reconciled N/M, ...` warning only when something was added or skipped. - get()'s manual completed/ search also checks the flat root now. **Fixture-based real-data tests** - tests/fixtures/workforce-trajectories/ has two hand-redacted fixtures: one in the flat `completed/` root, one in `completed/2026-04/`. - tests/storage/reconcile-real-data.test.ts locks down the legacy contract with 8 assertions covering both layouts, legacy role values, legacy id shapes, default arrays, ReconcileSummary counts, and malformed/schema-violation counting. Regressions fail at vitest layer (~100ms) instead of E2E layer (~8 min). **Workflow prompt hardening** workflows/fix-trajectory-schema/02-implement.ts gains a vitest-baseline step and a HARD CONSTRAINTS block in the impl-trajectories worker prompt that explicitly forbids editing src/storage/file.ts reconcile internals or the fixture tests. E2E gate now asserts `reconciled N/N` matches files-on-disk instead of a hardcoded N ≥ 11 (workforce data is mostly status=abandoned, so compact count is not a proxy for reconcile correctness). vitest pass count must be monotonically non-decreasing after the worker's edits. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
khaliqgant
added a commit
to AgentWorkforce/relay
that referenced
this pull request
Apr 13, 2026
The SDK's WorkflowTrajectory is a standalone writer that emits
trajectory JSON without using the `agent-trajectories` library. Its
output diverged from the canonical layout in two visible ways:
1. Completed files landed in a flat `completed/` root instead of
`completed/YYYY-MM/{id}.json`, forcing downstream readers to grow
legacy-layout fallbacks.
2. Top-level `commits`, `filesChanged`, and `tags` arrays were omitted,
even though the canonical schema declares them. Readers had to
default them at load time.
These are now aligned:
- `moveToCompleted()` computes a `YYYY-MM` bucket from
`completedAt` (falling back to `startedAt`) and writes into
`completed/<bucket>/{id}.json`.
- `TrajectoryFile` interface declares `commits: string[]`,
`filesChanged: string[]`, `tags: string[]`, and `start()` initializes
them to empty arrays.
Canonical output is defense-in-depth — `agent-trajectories` PR #22
(AgentWorkforce/trajectories#22) makes the reader tolerant of the prior
shapes, so this change is not urgent. It does let the reader shed those
fallbacks over time.
Test coverage:
- Existing `readCompletedTrajectoryFile` helper now walks completed/
recursively so tests don't hardcode the bucket name.
- New: asserts the completed file path has exactly one `YYYY-MM`
intermediate directory.
- New: asserts `commits`/`filesChanged`/`tags` are populated as [] on
`start()`.
All 29 tests in `workflow-trajectory.test.ts` pass. Diff scoped to
two files; unrelated full-suite failures (verification.test.ts,
step-executor, etc.) are pre-existing and out of scope.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This was referenced Apr 13, 2026
Closed
The lead agent in the fix-trajectory-schema/02-implement.ts open-prs step was reporting completion via relay channel messages, which the output_contains verification check does not watch — it only reads the agent's PTY stdout. Result: the PR got opened successfully but the workflow timed out on verification and reported "failed" anyway. Fix: have the lead write PR URL(s) to PR_URLS.txt as the durable completion artifact, and gate verification on file_exists. Channel messages and stdout prints are both optional for logging, but the file on disk is the canonical signal. Also fixes strict-TS issues in the same file that were tolerated at runtime but flagged by tsc --strict: - exit_code verifications now include explicit value: "0" - onEvent handler uses "stepName" in e narrowing instead of e.step Same verification trap bit the unify-sdk-trajectory-writer workflow during PR #732's run — the lead successfully opened the PR but the workflow reported failed because the completion signal was only in relay messages. The PR was real; the failure was cosmetic. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
Devin review on PR #22 caught two regex sites that parse trajectory ids but were not updated when the id regex was loosened in schema.ts/id.ts: 1. src/core/trailers.ts parseTrajectoryFromMessage used `traj_[a-z0-9]+` (no underscore). The end-anchor ($) means legacy `traj_<timestamp>_<hex>` ids fail to match entirely and the parser returns null -- commits linked to legacy trajectories would have no trailer recognized. 2. src/core/trailers.ts generateHookScript emitted `traj_[a-z0-9]*`, which with `grep -o` silently truncates legacy ids at the first internal underscore. For traj_1775734701264_ba65c69b this would emit `traj_1775734701264` into the commit trailer -- wrong id, no failure signal. Both character classes now include underscore to stay in sync with schema.ts and id.ts. Tests added to lock the regression: - parseTrajectoryFromMessage covers the legacy timestamp-hex shape both as a solo trailer and mixed with other trailers. - generateHookScript string-asserts the corrected character class and explicitly rejects the old one. - A behavioral test writes a fixture json with a legacy id, runs the hook's exact grep pipeline through real sh, and asserts the output is the full id, not a truncated prefix. 234 tests pass (up from 230: +4 new). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
khaliqgant
added a commit
to AgentWorkforce/relay
that referenced
this pull request
Apr 13, 2026
The unification in 95eafc4 relied on looser types from agent-trajectories (role as open string, expanded TrajectoryEventType, relaxed id regex). Those types only existed locally on fix/reconcile-stale-index when the worker implemented the PR, so CI (which installs from npm) hit 5 TS errors against the published 0.5.3. agent-trajectories@0.5.4 is now published with those looser types (AgentWorkforce/trajectories#22). Bumping: - packages/sdk/package.json: ^0.5.3 -> ^0.5.4 - root package.json (devDependencies): ^0.4.1 -> ^0.5.4 - package-lock.json regenerated via `npm install --package-lock-only` Verified locally: - `npm run build` in packages/sdk: clean tsc -p tsconfig.build.json - `npx vitest run src/__tests__/workflow-trajectory.test.ts`: 31 passed This should turn CI green on PR #732. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
khaliqgant
added a commit
to AgentWorkforce/relay
that referenced
this pull request
Apr 13, 2026
* refactor(sdk): unify WorkflowTrajectory on agent-trajectories SDK
Collapse the hand-rolled trajectory writer in packages/sdk/src/workflows/
trajectory.ts onto the agent-trajectories SDK. The SDK writer was a parallel
implementation of the same on-disk format, with its own types, filesystem I/O,
and chapter bookkeeping — a steady source of drift against the canonical
reader/writer in agent-trajectories.
Why:
- Two implementations of the same trajectory schema means two places to fix
bugs, two places to evolve the schema, and inevitable skew over time.
- The SDK duplicated type shadows, mkdir/writeFile/rename plumbing, and
chapter open/close state that agent-trajectories already owns.
- One source of truth for trajectory persistence lets downstream tooling
(viewer, reconcile, compaction) trust a single format.
What changed (trajectory.ts: 779 → 314 lines, -465):
- Deleted: local Chapter/TrajectoryAgent/TrajectoryEvent type shadows,
hand-rolled flush()/moveToCompleted() filesystem I/O, manual chapter state
tracking, and ID generation.
- Kept: public WorkflowTrajectory API, workflow event shapes, chapter title
semantics, failure classification / diagnosis / synthesis helpers, and the
non-blocking guard() boundary (all workflow semantics are preserved).
- Added: agent-trajectories as a runtime dependency; the adapter now drives
TrajectoryBuilder / core functions + FileStorage directly.
- Tests: workflow-trajectory.test.ts updated for the new persistence path;
31/31 passing.
Chosen approach is the hybrid (B-leaning) plan from
workflows/unify-sdk-trajectory-writer/DESIGN.md — keep Proposal B's semantic
conservatism (public API, event strings, non-blocking boundary) but take
Proposal A's deletion targets (type shadows, hand-rolled I/O).
Verification — all 5 gates green:
1. structural: imports agent-trajectories ✓
2. quantitative: 314 ≤ 400 line budget ✓
3. build: tsc -p tsconfig.build.json ✓
4. unit: 31/31 vitest tests passing ✓
5. 80→100 round-trip: trajectory created, flushed, moved to completed,
re-read via FileStorage — id/title/status/chapters/agents all match ✓
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore(sdk): bump agent-trajectories to ^0.5.4
The unification in 95eafc4 relied on looser types from
agent-trajectories (role as open string, expanded TrajectoryEventType,
relaxed id regex). Those types only existed locally on fix/reconcile-stale-index
when the worker implemented the PR, so CI (which installs from npm)
hit 5 TS errors against the published 0.5.3.
agent-trajectories@0.5.4 is now published with those looser types
(AgentWorkforce/trajectories#22). Bumping:
- packages/sdk/package.json: ^0.5.3 -> ^0.5.4
- root package.json (devDependencies): ^0.4.1 -> ^0.5.4
- package-lock.json regenerated via `npm install --package-lock-only`
Verified locally:
- `npm run build` in packages/sdk: clean tsc -p tsconfig.build.json
- `npx vitest run src/__tests__/workflow-trajectory.test.ts`: 31 passed
This should turn CI green on PR #732.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore(sdk): move agent-trajectories to root dependencies
The previous bump in 5411666 placed agent-trajectories@^0.5.4 in the
root devDependencies, which tripped scripts/audit-bundled-deps.mjs in
CI:
❌ MISSING DEPENDENCIES
"agent-trajectories": "^0.5.4",
// used by: sdk
@agent-relay/sdk is a bundled sub-package inside the root agent-relay
tarball. When a user runs `npm install -g agent-relay`, only the root
`dependencies` (not `devDependencies`) are installed as runtime deps.
Since the SDK now imports agent-trajectories at runtime (the unified
WorkflowTrajectory wraps TrajectoryClient from agent-trajectories/sdk),
it must be a runtime dep of the root, not a dev dep.
Move agent-trajectories from devDependencies to dependencies and
regenerate the lock file.
Local audit now passes:
✅ All bundled package dependencies are properly hoisted.
Bundled packages: 9
External dependencies found: 15
Hoisted to root: 15
Missing from root: 0
Build + unit tests still green (31/31 in workflow-trajectory.test.ts).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes
trail compact --allreporting "No trajectories found" in the workforce repo even though the directory has 10+ completed trajectory JSON files. Two layers addressed in one stacked branch:FileStorage.reconcileIndex()self-heals staleindex.jsonon everyinitialize(), walkingactive/andcompleted/recursively (flat root +YYYY-MM/+ any nesting). Additive — preserves existing entries."workflow-runner"/"specialist", legacytraj_<ts>_<hex>ids, and omits optional fields. Schema now accepts all of these on read.Key changes
src/core/schema.ts):role→ open string; trajectory id regex allows underscores;commits/filesChanged/tagsdefault to[];projectIdoptionalFileStorage.save): callsvalidateTrajectory()and throws on invalid input, closing the historical read/write asymmetryreadTrajectoryFile): returns{ok: true, trajectory}or{ok: false, reason: 'malformed_json' | 'schema_violation' | 'io_error'}. Callers pick their policyreconcileIndex()returns aReconcileSummary(scanned/added/alreadyIndexed/skippedMalformedJson/skippedSchemaViolation/skippedIoError) and emits a single structured[trajectories] reconciled N/M, ...logtests/storage/reconcile-real-data.test.ts+tests/fixtures/workforce-trajectories/): 8 assertions using hand-redacted real workforce data committed as fixtures. Future refactors that break legacy data compatibility fail in ~100ms at vitest layer, not 8+ minutes at an E2E gateworkflows/fix-trajectory-schema/02-implement.ts): adds avitest-baselinestep and a HARD CONSTRAINTS block forbidding workers from modifying reconcile internals or fixture tests. E2E gate now assertsreconciled N/Nmatches files-on-disk instead of a hardcoded threshold (workforce data is mostlyabandoned, whichcompact --allcorrectly filters out — compact count is not a proxy for reconcile correctness)Test plan
npm run typecheck— cleannpx vitest run— 230 passed (up from 222; +8 fixture tests)npm run build— cleanagent-relay run --dry-run workflows/fix-trajectory-schema/02-implement.ts— validates clean (9 steps, 7 waves, 0 errors)Stacks on
Two commits:
a387b5c feat(storage): reconcile index from disk on FileStorage.initialize— ground-truth reconcile from earlier session, committed so downstream branches/worktrees start from a correct baseeee1a44 feat(storage): tolerate legacy trajectory shapes + harden reconcile— everything else🤖 Generated with Claude Code