Skip to content

feat(feature-flags): TTL evaluation cache with per-tenant invalidation#650

Open
almuslavish wants to merge 1 commit into
CredenceOrg:mainfrom
almuslavish:feat/feature-flag-eval-cache
Open

feat(feature-flags): TTL evaluation cache with per-tenant invalidation#650
almuslavish wants to merge 1 commit into
CredenceOrg:mainfrom
almuslavish:feat/feature-flag-eval-cache

Conversation

@almuslavish

Copy link
Copy Markdown

Summary

Closes #639

Adds an in-process TTL evaluation cache to FeatureFlagService so that hot-path isEnabled calls avoid repeated backing-store reads.

Changes

src/services/featureFlags/index.ts

  • TtlCache<T> – lightweight Map-based cache with per-entry TTL and LRU-style max-size eviction (no new dependencies).
  • FeatureFlagService – wraps a FeatureFlagStore with:
    • isEnabled(flagKey, tenantId, userId?) – cached; key is tenantId:flagKey[:userId] (cross-tenant leakage is impossible by construction)
    • updateFlag – calls store.updateFlag then bustFlag before returning (correctness over speed)
    • setOverride – calls store.setOverride then evicts the user-scoped cache entry
    • bustFlag(flagKey, tenantId) – evicts all entries under the tenantId:flagKey prefix; callable from the invalidation bus (src/cache/invalidationBus.ts)
    • clearCache() – full flush (shutdown / testing)
  • Metrics (prom-client Counters with tenant_id + flag_key labels):
    • feature_flag_cache_hits_total
    • feature_flag_cache_misses_total
    • feature_flag_cache_invalidations_total
  • Cache options: ttlMs (default 30 s), maxSize (default 1 000), disabled flag

src/services/featureFlags/index.test.ts

Vitest suite covering:

  • Cache miss → store fetch → cache hit (no second store call)
  • TTL expiry (fake timers): re-fetches after TTL, serves from cache before boundary
  • Two-tenant isolation: separate entries, busting one tenant doesn't affect the other
  • updateFlag invalidates base and user-scoped entries
  • setOverride invalidates the specific user entry; override added after a cached miss is honoured immediately
  • bustFlag evicts all scoped entries; no metric emitted when nothing was evicted
  • User-level override evaluation (override wins over flag default; falls back when absent)
  • Disabled cache: every call goes to store
  • maxSize eviction: oldest entry is dropped when capacity is reached
  • Pass-through admin methods: getFlag, listFlags, listFlagsWithOverrides

src/observability/index.ts

Re-exports featureFlagCacheHits, featureFlagCacheMisses, featureFlagCacheInvalidations.

Testing

npm run test -- featureFlags
npm run lint
npm run build

Notes

  • lru-cache is not in package.json; the implementation uses a plain Map which satisfies the same requirements with zero new dependencies.
  • Invalidation happens before the mutating method returns, satisfying the "correctness over speed" requirement.

Adds an in-process TtlCache layer to FeatureFlagService.isEnabled with
update-driven invalidation and hit/miss/invalidation metrics.

- TtlCache: Map-based, configurable ttlMs + maxSize, no external deps
- Cache key is tenant-scoped (tenantId:flagKey[:userId]) — cross-tenant
  leakage is impossible
- updateFlag and setOverride invalidate affected entries before returning
- bustFlag(flagKey, tenantId) exposed for the invalidation bus
- prom-client Counters: feature_flag_cache_{hits,misses,invalidations}_total
- Metrics re-exported from src/observability/index.ts
- Tests: cache-hit, TTL-expiry, two-tenant isolation, invalidation-on-update,
  setOverride invalidation, bustFlag, disabled cache, maxSize eviction,
  override evaluation

Resolves CredenceOrg#639
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add a feature-flag evaluation cache with TTL and per-tenant invalidation in services/featureFlags

1 participant