Design client-side caching for list_tools/list_prompts/list_resources

[maxisbey]

Summary

Every list_tools(), list_prompts(), and list_resources() call makes a fresh network request. For high-traffic use cases (gateways, proxies, multi-tenant deployments), this creates significant redundant load on upstream MCP servers.

The SDK should explore offering opt-in client-side caching for these list operations, with invalidation driven by list_changed notifications (see #2107).

Open Questions

• Where does the cache live? In-memory on the Client/Session, or pluggable (Redis, etc.)?
• What is the cache key? Per-session? Per-server URL?
• What is the default TTL, if any, independent of list_changed notifications?
• Should this be a feature of Client, ClientSession, or a separate component?
• How does this interact with stateless HTTP mode where there is no persistent session to receive notifications?

[gspeter-max]

Design Question: Storage Backend Approach

Before implementing, I'd like to get feedback on the caching approach. The issue asks "Where does the cache live? In-memory on the Client/Session, or pluggable (Redis, etc.)?"

I've analyzed three options:

---

Option A: In-Memory Only

Pros:

• ✅ Simple implementation (~100 lines)
• ✅ No external dependencies
• ✅ Solves ~80% of use cases (simple scripts, single client instances)
• ✅ Backward compatible (opt-in via parameter)

Cons:

• ❌ Doesn't solve the gateway/proxy multi-instance use case mentioned in the issue
• ❌ Each Client instance has its own cache (no sharing)

---

Option B: Redis Only

Pros:

• ✅ Solves gateway/proxy use case (shared cache across instances)
• ✅ Cache survives process restarts
• ✅ Suitable for high-traffic deployments

Cons:

• ❌ Requires Redis dependency for all users
• ❌ Overkill for simple scripts
• ❌ Adds operational complexity

---

Option C: Both (Pluggable Interface)

Pros:

• ✅ Flexible - users choose what they need
• ✅ In-memory as default (backward compatible)
• ✅ Redis optional for advanced use cases
• ✅ Solves all use cases

Cons:

• ❌ More complex (requires interface + 2 implementations)
• ❌ More code to maintain
• ❌ Larger PR to review

---

My Recommendation

Start with Option A (In-Memory Only) for this issue:

• Addresses the core problem (redundant network calls)
• Simple, easy to review
• Can extend with pluggable storage in a follow-up issue if there's demand

Questions for maintainers:

1. Which approach do you prefer?
2. Is the gateway/proxy use case (shared cache) a hard requirement for this issue?
3. Should we make it pluggable from the start, or add Redis later when needed?

cc: @maxisbey

[martimfasantos]

This is very helpful and aligns well with what we’ve been seeing. We’re also exploring client-side caching, and having a cache on the client to avoid making an HTTP request for every list_tools() call would be a big improvement.

Any indication of when this might be prioritized?