Skip to content

BREAKING(cache): add unified Cache#7082

Open
tomas-zijdemans wants to merge 18 commits intodenoland:mainfrom
tomas-zijdemans:cache
Open

BREAKING(cache): add unified Cache#7082
tomas-zijdemans wants to merge 18 commits intodenoland:mainfrom
tomas-zijdemans:cache

Conversation

@tomas-zijdemans
Copy link
Copy Markdown
Contributor

@tomas-zijdemans tomas-zijdemans commented Apr 2, 2026

NOTE: Builds on this PR

This PR replaces LruCache and TtlCache with a single Cache<K, V> that unifies LRU eviction, TTL expiration, stale-while-revalidate, and load-through (getOrLoad).

The existing caches extend Map, so inherited methods bypass LRU/TTL logic entirely. Both subclass Map, making it structurally impossible to compose LRU with TTL.

Cache<K, V> uses composition over inheritance: it owns a Map for storage and delegates to IndexedHeap for deadline-ordered expiration. Mode is determined by options, not class choice, and a discriminated union on CacheOptions makes illegal combinations compile-time errors.

Beyond fixing the structural issues, this introduces capabilities that didn't exist before:

  • Combined LRU + TTL in a single cache
  • getOrLoad(key, loader) with automatic in-flight deduplication
  • Stale-while-revalidate with background refresh, error handling, and configurable soft/hard TTL thresholds
  • Per-entry TTL overrides and sliding expiration
  • Single heap-driven timer replacing per-entry setTimeouts
  • Hit/miss/eviction stats, typed ejection reasons, and Symbol.dispose

Since @std/cache is still experimental (0.2.2), this is the right time to make a clean break before stabilization locks things in.

Bonus: The new Cache has significant performance improvements for real-life workflows (e.g. in my testing, write-heavy workloads (set, eviction) are 4-22x faster. hot-key reads are 55x faster thanks to a linked-list)

@codecov
Copy link
Copy Markdown

codecov bot commented Apr 2, 2026

Codecov Report

❌ Patch coverage is 99.84917% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 94.47%. Comparing base (a3a6ef6) to head (b4e74c5).

Files with missing lines Patch % Lines
cache/cache.ts 99.79% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #7082      +/-   ##
==========================================
+ Coverage   94.43%   94.47%   +0.04%     
==========================================
  Files         630      630              
  Lines       50566    51008     +442     
  Branches     8969     9067      +98     
==========================================
+ Hits        47750    48192     +442     
  Misses       2247     2247              
  Partials      569      569              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@bartlomieju
Copy link
Copy Markdown
Member

I'll hold on reviewing this one until #7076 lands.

@tomas-zijdemans tomas-zijdemans changed the title feat(cache): add unified Cache BREAKING(cache): add unified Cache Apr 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants