feat: make Prometheus remote-write endpoint optional - #26
Merged
Conversation
The full benchmark workflow does not require a Prometheus remote-write endpoint: k6 already writes a summary.json, and the runner has a complete path for building per-client/per-method metrics from that file. Previously the CLI forced Prometheus by defaulting --prometheus to http://localhost:9090 and rejecting an empty value, and by constructing PrometheusRW unconditionally. Prometheus is now optional and disabled by default: - --prometheus defaults to empty; the empty-check guard is removed. Passing an endpoint opts into remote-write and post-run PromQL queries. - cfg.Outputs.PrometheusRW is only built when an endpoint is provided. - CollectClientsMetrics routes to a new summary-only path that fills every client/method pair from summary.json when Prometheus is not configured, instead of returning "no outputs configured". Skeleton building and totals finalization are shared between the Prometheus and summary-only paths. - Fix a 0/0 = NaN in the performance analyzer for a client with no methods, which the summary-only path can now surface and which broke JSON export. Also drop an invalid eth_getStorageAt entry (non-hex "0xasdf" storage slot) from the mainnet rpc-calls fixture.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The full benchmark workflow does not require a Prometheus remote-write endpoint — k6 already writes a
summary.json, and the runner has a complete path for building per-client/per-method metrics from it. Until now the CLI forced Prometheus by defaulting--prometheustohttp://localhost:9090, rejecting an empty value, and constructingPrometheusRWunconditionally.This PR makes Prometheus optional and disabled by default.
Behavior
--prometheusvaluesummary.jsonsummary.jsonfills gapsChanges
runner/cmd/benchmark.go—--prometheusdefaults to empty; removed the--prometheus is requiredguard;PrometheusRWis only built when an endpoint is provided (cfg.Outputsstays non-nil).runner/metrics/results_collection.go—CollectClientsMetricsroutes to a new summary-only collection path when Prometheus is not configured (instead of returning"no outputs configured"), filling every client/method pair fromsummary.jsonvia the existing fallback. Extracted sharednewClientsMetricsSkeletonandfinalizeClientMetricshelpers so both paths behave identically after data load.runner/generator/k6_generator.go— defensive nil-Outputsguard; k6 already skipped theexperimental-prometheus-rwoutput whenPrometheusRWwas nil.runner/analyzer/performance_analyzer.go— fix a latent0/0 = NaNin throughput averaging for a client with no methods. The summary-only path can now surface an empty-methods client (e.g. a node returning zero successful requests), and theNaNbroke JSON export.README.md— documented Prometheus as optional/disabled-by-default across usage examples, the migration table, and the metrics section.rpc-calls/eth_getStorageAt-mainnet-latest.jsonl— dropped an invalid entry with a non-hex"0xasdf"storage slot.Tests / verification
runner/metrics/results_collection_test.gocoveringCollectClientsMetricswith both nilOutputsand nilPrometheusRW, asserting metrics come fromsummary.jsonand totals are finalized.go build,go vet, and package tests pass (the pre-existingTestP99DataFlowrequires a live PostgreSQL and is unrelated).--prometheus: no required-flag error, k6 ran without remote-write, the summary.json fallback engaged,results.jsonwas written, and noNaNin output. Opt-in run with--prometheus http://localhost:9090is unchanged.