Skip to content

feat: add orchestrator pipeline, TopicRegistry, and CI improvements#59

Merged
behinddwalls merged 1 commit into
mainfrom
preetam/more-queues
Feb 24, 2026
Merged

feat: add orchestrator pipeline, TopicRegistry, and CI improvements#59
behinddwalls merged 1 commit into
mainfrom
preetam/more-queues

Conversation

@behinddwalls
Copy link
Copy Markdown
Collaborator

@behinddwalls behinddwalls commented Feb 24, 2026

Summary

  • TopicRegistry: Centralizes queue topic configuration, decoupling controllers from subscription details. Controllers declare their topic and consumer group; the registry provides the queue and subscription config.
  • Orchestrator pipeline: Adds request controller with publisher for forwarding messages to the next pipeline stage (request → batch → speculation → build → merge).
  • Consumer refactor: Accepts TopicRegistry instead of raw queue.Queue + subscriber name. Removes SubscriptionConfig() from the Controller interface.
  • Server startup simplification: Removes retry loops from gateway and orchestrator servers — relies on Docker Compose healthchecks (--wait) to ensure dependencies are ready before services start.
  • CI caching fix: Caches bazel-disk-cache and bazel-repo-cache (matching .bazelrc paths) with per-job keys to avoid concurrent save races. Previous setup cached the wrong directory and all jobs raced on a single key.
  • Auto-tail container logs: Streams Docker container logs to stderr in real-time during integration/e2e tests via ComposeStack.tailLogs(). Removes the now-redundant .github/actions/logs action.
  • Test output streaming: All test targets use --test_output=streamed for real-time output in both local and CI runs.
  • Documentation consolidation: Merges PROJECT_STRUCTURE.md into CLAUDE.md, updates TESTING.md for new ComposeStack behavior.

Test plan

@behinddwalls behinddwalls force-pushed the preetam/more-queues branch 3 times, most recently from 8f1b8a8 to ca20b81 Compare February 24, 2026 05:48
@behinddwalls behinddwalls changed the title feat(orchestrator): add complete pipeline controllers feat: add orchestrator pipeline, TopicRegistry, and CI improvements Feb 24, 2026
@behinddwalls behinddwalls marked this pull request as ready for review February 24, 2026 05:50
@behinddwalls behinddwalls requested review from a team and sbalabanov as code owners February 24, 2026 05:50
Comment thread core/consumer/registry.go Outdated
Comment thread core/consumer/registry.go Outdated
Comment thread core/consumer/registry.go Outdated
- Add TopicRegistry to centralize queue topic configuration and
  decouple controllers from subscription details
- Refactor consumer to accept TopicRegistry instead of raw queue +
  subscriber name
- Add request controller to orchestrator pipeline with publisher for
  forwarding to next stage
- Generate gomock mocks for consumer, queue, publisher, and subscriber
  interfaces
- Simplify server startup by removing retry loops (rely on Docker
  Compose healthchecks via --wait)
- Auto-tail container logs to stderr during integration/e2e tests
- Fix CI Bazel caching: use disk-cache + repo-cache from .bazelrc,
  per-job cache keys to avoid save races
- Stream test output in real-time (--test_output=streamed)
- Remove redundant .github/actions/logs (logs now stream automatically)
- Consolidate PROJECT_STRUCTURE.md into CLAUDE.md

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@behinddwalls behinddwalls merged commit adbb3b8 into main Feb 24, 2026
8 checks passed
@behinddwalls behinddwalls deleted the preetam/more-queues branch June 2, 2026 18:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants