client/resource_group: add client-side demand observability

[okJiang]

What problem does this PR solve?

Issue Number: ref #10488, ref #10581

Resource control observability on the client side does not show the difference between real demand and post-throttling consumption. It is also hard to tell whether pressure is local to one TiDB instance or spread across the whole resource group.

What is changed and how does it work?

This PR adds client-side metrics and slow-path logs so we can explain demand, limiter state, and per-instance pressure before requests are throttled.

client/resource_group: add client-side demand observability

It includes:

• a pre-throttling demand_ru_per_sec metric collected before Reserve()
• client-side balance, fill rate, burst limit, average RU/s, throttled state, and consumption breakdown metrics
• slow-path structured logs for degraded mode, slow token requests, long waits, and throttling
• cleanup coverage so stale resource-group labels are removed correctly

Check List

Tests

• Unit test
• Manual test (add detailed scripts or steps below)

Manual test:

• Verified in a local tiup playground setup with 3 PD, 2 TiDB, and 3 TiKV.
• Created a low-fill-rate resource group and confirmed Grafana shows demand_ru_per_sec > fill_rate, higher client wait, and throttling on the hot TiDB instance.
• Confirmed stale metrics are cleaned when the resource group is removed.

Code changes

• Has the configuration change

Side effects

• Increased code complexity

Related changes

• PR to update pingcap/docs/pingcap/docs-cn:
• PR to update pingcap/tiup:
• Need to cherry-pick to the release branch

Release note

Add client-side resource control metrics for pre-throttling demand, limiter state, and per-instance RU consumption breakdowns.

[ti-chi-bot]

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

[ti-chi-bot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign yisaer for approval. For more information see the Code Review Process.
Please ensure that each of them provides their approval before proceeding.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: b083421f-cf8b-457d-be30-b200efce46f6

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}