rpc-client: per-provider/method request metrics (counted by batch element)#493
Closed
elina-chertova wants to merge 1 commit into
Closed
rpc-client: per-provider/method request metrics (counted by batch element)#493elina-chertova wants to merge 1 commit into
elina-chertova wants to merge 1 commit into
Conversation
…ment)
Instrument at the RpcClient level so EVERY SDK path (dump, processor, data-service) gets
per-upstream visibility from one place. Provider/chain identity come from constructor options
(set by the caller from its infra/deployment config — the canonical 'name', NOT derived from the
URL host, which can vary per provider). Calls are tallied by BATCH ELEMENT (a JSON-RPC batch is
one HTTP call but N billable items) and split by outcome, keyed by method — so a Prometheus
exporter can emit {provider, chain, method, outcome} and reconcile against provider CU bills.
Exposed via getMetrics() (RpcMetrics.methods) rather than a prom-client Counter inside rpc-client:
keeps this low-level lib dependency-free and avoids duplicating the per-CLI exporters that already
read getMetrics(). The exporter side (PrometheusServer label emission) is a follow-up.
requestsServed (HTTP-call count) is unchanged for back-compat; methods{} is additive.
elina-chertova
force-pushed
the
metrics/rpc-per-provider-method
branch
from
2f5bcbe to
f52dcc6
Compare
Contributor
|
this pr targets open-beta branch which is not maintained anymore. closing in favour of #511 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add per-upstream request metrics at the
RpcClientlevel, so every SDK ingest path (dump, processor, data-service) can see how many requests we send to each provider — broken down by network, method and outcome — from our own metrics.Why
Today we can't reconcile our request volume against a provider's dashboard:
sqd_chain_rpc_requests_served_total{url}is emitted only by the dump CLI → archive-only; the live / data-service path has no per-provider counter.So we can't answer "how many requests / which methods did we send to provider X on chain Y", nor catch over-usage spikes from our side. Instrumenting at
RpcClient— the transport all paths share — closes the blind spot in one place.What changed (
util/rpc-client/src/client.ts)RpcClientOptionsgainsprovider?andchain?— the canonical identity, set by the caller from its deployment/infra config. Deliberately not derived from the URL host (a provider can use several hosts, e.g.direct.drpc.organdlb.drpc.live).success/failure, keyed by method.getMetrics()(RpcMetrics.methods, plusprovider/chain).Design decisions
getMetrics(), not aprom-clientCounter insiderpc-client— keeps this low-level lib dependency-free and avoids duplicating the per-CLI exporters that already readgetMetrics(). An exporter then emitssqd_rpc_upstream_requests_total{provider, chain, method, outcome}.sqd_rpc_batch_size_*).Not in this PR (follow-ups)
PrometheusServers (dump-cli, processor-tools) + data-service emit the labeled counter fromgetMetrics().methods.provider/chainfrom config.Compatibility
requestsServed(HTTP-call count) unchanged;methods{}is purely additive.Testing
Couldn't run
tscin the authoring environment (rush deps not installed) — please confirm withrush build --to @subsquid/rpc-client.