Skip to content

feat: make Prometheus remote-write endpoint optional - #26

Merged
cbermudez97 merged 1 commit into
mainfrom
feat/allow-empty-prometheus
Jul 21, 2026
Merged

feat: make Prometheus remote-write endpoint optional#26
cbermudez97 merged 1 commit into
mainfrom
feat/allow-empty-prometheus

Conversation

@cbermudez97

Copy link
Copy Markdown
Contributor

Summary

The full benchmark workflow does not require a Prometheus remote-write endpoint — k6 already writes a summary.json, and the runner has a complete path for building per-client/per-method metrics from it. Until now the CLI forced Prometheus by defaulting --prometheus to http://localhost:9090, rejecting an empty value, and constructing PrometheusRW unconditionally.

This PR makes Prometheus optional and disabled by default.

Behavior

--prometheus value k6 remote-write Client metrics source
empty (new default) not enabled summary.json
a URL (opt-in) enabled, as before Prometheus query, summary.json fills gaps

Changes

  • runner/cmd/benchmark.go--prometheus defaults to empty; removed the --prometheus is required guard; PrometheusRW is only built when an endpoint is provided (cfg.Outputs stays non-nil).
  • runner/metrics/results_collection.goCollectClientsMetrics routes to a new summary-only collection path when Prometheus is not configured (instead of returning "no outputs configured"), filling every client/method pair from summary.json via the existing fallback. Extracted shared newClientsMetricsSkeleton and finalizeClientMetrics helpers so both paths behave identically after data load.
  • runner/generator/k6_generator.go — defensive nil-Outputs guard; k6 already skipped the experimental-prometheus-rw output when PrometheusRW was nil.
  • runner/analyzer/performance_analyzer.go — fix a latent 0/0 = NaN in throughput averaging for a client with no methods. The summary-only path can now surface an empty-methods client (e.g. a node returning zero successful requests), and the NaN broke JSON export.
  • README.md — documented Prometheus as optional/disabled-by-default across usage examples, the migration table, and the metrics section.
  • rpc-calls/eth_getStorageAt-mainnet-latest.jsonl — dropped an invalid entry with a non-hex "0xasdf" storage slot.

Tests / verification

  • Added runner/metrics/results_collection_test.go covering CollectClientsMetrics with both nil Outputs and nil PrometheusRW, asserting metrics come from summary.json and totals are finalized.
  • go build, go vet, and package tests pass (the pre-existing TestP99DataFlow requires a live PostgreSQL and is unrelated).
  • Ran a real benchmark with no --prometheus: no required-flag error, k6 ran without remote-write, the summary.json fallback engaged, results.json was written, and no NaN in output. Opt-in run with --prometheus http://localhost:9090 is unchanged.

The full benchmark workflow does not require a Prometheus remote-write
endpoint: k6 already writes a summary.json, and the runner has a complete
path for building per-client/per-method metrics from that file. Previously
the CLI forced Prometheus by defaulting --prometheus to http://localhost:9090
and rejecting an empty value, and by constructing PrometheusRW unconditionally.

Prometheus is now optional and disabled by default:

- --prometheus defaults to empty; the empty-check guard is removed. Passing
  an endpoint opts into remote-write and post-run PromQL queries.
- cfg.Outputs.PrometheusRW is only built when an endpoint is provided.
- CollectClientsMetrics routes to a new summary-only path that fills every
  client/method pair from summary.json when Prometheus is not configured,
  instead of returning "no outputs configured". Skeleton building and totals
  finalization are shared between the Prometheus and summary-only paths.
- Fix a 0/0 = NaN in the performance analyzer for a client with no methods,
  which the summary-only path can now surface and which broke JSON export.

Also drop an invalid eth_getStorageAt entry (non-hex "0xasdf" storage slot)
from the mainnet rpc-calls fixture.
@cbermudez97
cbermudez97 merged commit 89c65c7 into main Jul 21, 2026
4 checks passed
@cbermudez97
cbermudez97 deleted the feat/allow-empty-prometheus branch July 21, 2026 18:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant