Skip to content

fix(monitor): keep regions visible before scheduled runs#11

Open
pavelm wants to merge 1 commit into
mainfrom
fix/initialize-monitor-metrics
Open

fix(monitor): keep regions visible before scheduled runs#11
pavelm wants to merge 1 commit into
mainfrom
fix/initialize-monitor-metrics

Conversation

@pavelm

@pavelm pavelm commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

Summary

Freshly deployed regional workers now remain visible to Prometheus and Grafana while waiting for their staggered benchmark slot. Startup publishes zero-valued attempt and failure counter series for every configured provider and network, preserving both the nonce-safe schedule and real counter semantics. This prevents region counts and dashboard variables from disappearing for up to roughly 70 minutes after a worker restart, while also giving the first completed batch a valid counter baseline.

Validation

  • bun test — 329 passed, 0 failed, 768 assertions
  • bunx tsc --noEmit
  • git diff --check

Production scrape labels are attached externally and were not available locally. After deployment, count(count by (region) (txe_bench_attempts_total{measurement_epoch="alchemy-status-v2",environment="prod",network="base-mainnet"})) should report all four workers without waiting for their benchmark batches to finish.


Compound Engineering

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant