feat(skill): skills management manager, cache and client#861
Conversation
Hand-written high-level skills API mirroring the prompt manager:
- SkillManager (get with caching + stale-while-revalidate + fallback +
retry, create, update labels, list, delete)
- LangfuseSkillCache with TTL and prefix invalidation
- SkillClient wrapper with {{variable}} compilation of the instructions
body (single client; skills have no chat/text split)
- wired onto LangfuseClient as .skill plus getSkill/createSkill/updateSkill
- unit tests for cache TTL and client compilation
Depends on the auto-generated @langfuse/core skills client
(api-spec-bot regeneration from the Fern contract in langfuse/langfuse).
|
Someone is attempting to deploy a commit to the langfuse Team on Vercel. A member of the Team first needs to authorize it. |
| for (const key of this._cache.keys()) { | ||
| if (key.startsWith(skillName)) { | ||
| this._cache.delete(key); | ||
| } | ||
| } |
There was a problem hiding this comment.
Cache invalidation prefix collision:
startsWith(skillName) without a separator will incorrectly evict entries for any skill whose name begins with the same characters. For example, calling invalidate("foo") will also delete the cached entries for a skill named "foobar" because "foobar-label:production".startsWith("foo") is true. The key format always appends - before the version/label segment, so anchoring the prefix check to skillName + "-" prevents these false invalidations. Note: the identical issue is present in LangfusePromptCache.invalidate.
| for (const key of this._cache.keys()) { | |
| if (key.startsWith(skillName)) { | |
| this._cache.delete(key); | |
| } | |
| } | |
| for (const key of this._cache.keys()) { | |
| if (key.startsWith(skillName + "-")) { | |
| this._cache.delete(key); | |
| } | |
| } |
Prompt To Fix With AI
This is a comment left during a code review.
Path: packages/client/src/skill/skillCache.ts
Line: 85-89
Comment:
Cache invalidation prefix collision: `startsWith(skillName)` without a separator will incorrectly evict entries for any skill whose name begins with the same characters. For example, calling `invalidate("foo")` will also delete the cached entries for a skill named `"foobar"` because `"foobar-label:production".startsWith("foo")` is `true`. The key format always appends `-` before the version/label segment, so anchoring the prefix check to `skillName + "-"` prevents these false invalidations. Note: the identical issue is present in `LangfusePromptCache.invalidate`.
```suggestion
for (const key of this._cache.keys()) {
if (key.startsWith(skillName + "-")) {
this._cache.delete(key);
}
}
```
How can I resolve this? If you propose a fix, please make it concise.| version: options?.version, | ||
| }); | ||
| const cachedSkill = this.cache.getIncludingExpired(cacheKey); | ||
| if (!cachedSkill || options?.cacheTtlSeconds === 0) { |
There was a problem hiding this comment.
cacheTtlSeconds: 0 still writes to cache: When cacheTtlSeconds is explicitly 0 (caching disabled), fetchSkillAndUpdateCache is still called, which stores the result with TTL 0. This creates an immediately-expired entry. On the very next get() call that does not pass cacheTtlSeconds: 0, getIncludingExpired will find this stale entry and trigger the stale-while-revalidate path, returning stale data rather than fetching synchronously. The same behaviour exists in PromptManager, but it may be surprising to callers who used cacheTtlSeconds: 0 expecting a complete bypass.
Prompt To Fix With AI
This is a comment left during a code review.
Path: packages/client/src/skill/skillManager.ts
Line: 228
Comment:
**`cacheTtlSeconds: 0` still writes to cache**: When `cacheTtlSeconds` is explicitly `0` (caching disabled), `fetchSkillAndUpdateCache` is still called, which stores the result with TTL 0. This creates an immediately-expired entry. On the very next `get()` call that does *not* pass `cacheTtlSeconds: 0`, `getIncludingExpired` will find this stale entry and trigger the stale-while-revalidate path, returning stale data rather than fetching synchronously. The same behaviour exists in `PromptManager`, but it may be surprising to callers who used `cacheTtlSeconds: 0` expecting a complete bypass.
How can I resolve this? If you propose a fix, please make it concise.LangfuseSkillCache.invalidate matched on startsWith(skillName), which could
evict entries for a different skill whose name shares the prefix (e.g.
invalidate('foo') dropping 'foobar-*'). Anchor to skillName + '-'.
abd49e7 to
602b604
Compare
Skills management (JS SDK)
Hand-written high-level skills API mirroring the prompt manager:
SkillManager—get(caching + stale-while-revalidate + fallback + retry),create,update(labels),list,deleteLangfuseSkillCache(TTL + prefix invalidation)SkillClientwith{{variable}}compilation of theinstructionsbody (single client — skills have no chat/text split)LangfuseClientas.skill+getSkill/createSkill/updateSkillDepends on the auto-generated
@langfuse/coreskills client from theapi-spec-botregeneration of the Fern contract in langfuse/langfuse (langfuse/langfuse#14838). Typechecks/builds once that client lands.Greptile Summary
This PR adds a hand-written
SkillManager/SkillClient/LangfuseSkillCachelayer for skills management, closely mirroring the existingPromptManagerimplementation. It wires the manager ontoLangfuseClientas.skillwith convenience aliases (getSkill,createSkill,updateSkill), and ships unit tests for cache TTL behaviour and Mustache template compilation.get(TTL cache + stale-while-revalidate + fallback + retry),create,update(labels),list, anddelete, all following the same patterns asPromptManager.compile()helper for{{variable}}substitution via Mustache (HTML-escaping disabled).invalidate()method useskey.startsWith(skillName)without a-separator, which is a logic error copied fromLangfusePromptCache: skill names that are a prefix of another skill name (e.g.,"foo"vs"foobar") will cause false cache evictions.Confidence Score: 3/5
The skill manager logic is sound and closely tracks the existing prompt manager. One real defect in the cache invalidation logic — using a bare prefix match without a separator — can silently evict unrelated skills from cache in applications where skill names share a common prefix.
The invalidation method deletes every cache entry whose key starts with the supplied skill name, but the key format always places a '-' immediately after the name. Without anchoring the check to skillName + '-', a skill named 'foo' incorrectly evicts all cached entries for 'foobar' when either is deleted or its labels are updated. Any application with skills following a naming convention (e.g., 'agent', 'agent-v2', 'agent-v3') would experience unexpected cache misses and extra API calls. The same defect exists verbatim in LangfusePromptCache and was carried forward unchanged here.
packages/client/src/skill/skillCache.ts — the invalidate method's prefix match needs the '-' separator fix. The same one-line change is also worth applying to packages/client/src/prompt/promptCache.ts.
Sequence Diagram
%%{init: {'theme': 'neutral'}}%% sequenceDiagram participant Caller participant LangfuseClient participant SkillManager participant LangfuseSkillCache participant LangfuseAPIClient Caller->>LangfuseClient: getSkill("my-skill", options) LangfuseClient->>SkillManager: get("my-skill", options) SkillManager->>LangfuseSkillCache: "createKey({name, version, label})" LangfuseSkillCache-->>SkillManager: cacheKey SkillManager->>LangfuseSkillCache: getIncludingExpired(cacheKey) alt "Cache miss or cacheTtlSeconds === 0" LangfuseSkillCache-->>SkillManager: null SkillManager->>LangfuseAPIClient: "skills.get(name, {version, label}, opts)" LangfuseAPIClient-->>SkillManager: Skill data SkillManager->>LangfuseSkillCache: set(cacheKey, SkillClient, ttl) SkillManager-->>LangfuseClient: SkillClient else Cache hit (fresh) LangfuseSkillCache-->>SkillManager: CacheItem (not expired) SkillManager-->>LangfuseClient: SkillClient (from cache) else Cache hit (stale) LangfuseSkillCache-->>SkillManager: CacheItem (expired) SkillManager->>LangfuseSkillCache: isRefreshing(cacheKey)? LangfuseSkillCache-->>SkillManager: false SkillManager->>LangfuseAPIClient: skills.get(...) [background] SkillManager->>LangfuseSkillCache: addRefreshingPromise(cacheKey, promise) SkillManager-->>LangfuseClient: SkillClient (stale, refreshing in background) end LangfuseClient-->>Caller: SkillClient%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%% sequenceDiagram participant Caller participant LangfuseClient participant SkillManager participant LangfuseSkillCache participant LangfuseAPIClient Caller->>LangfuseClient: getSkill("my-skill", options) LangfuseClient->>SkillManager: get("my-skill", options) SkillManager->>LangfuseSkillCache: "createKey({name, version, label})" LangfuseSkillCache-->>SkillManager: cacheKey SkillManager->>LangfuseSkillCache: getIncludingExpired(cacheKey) alt "Cache miss or cacheTtlSeconds === 0" LangfuseSkillCache-->>SkillManager: null SkillManager->>LangfuseAPIClient: "skills.get(name, {version, label}, opts)" LangfuseAPIClient-->>SkillManager: Skill data SkillManager->>LangfuseSkillCache: set(cacheKey, SkillClient, ttl) SkillManager-->>LangfuseClient: SkillClient else Cache hit (fresh) LangfuseSkillCache-->>SkillManager: CacheItem (not expired) SkillManager-->>LangfuseClient: SkillClient (from cache) else Cache hit (stale) LangfuseSkillCache-->>SkillManager: CacheItem (expired) SkillManager->>LangfuseSkillCache: isRefreshing(cacheKey)? LangfuseSkillCache-->>SkillManager: false SkillManager->>LangfuseAPIClient: skills.get(...) [background] SkillManager->>LangfuseSkillCache: addRefreshingPromise(cacheKey, promise) SkillManager-->>LangfuseClient: SkillClient (stale, refreshing in background) end LangfuseClient-->>Caller: SkillClientPrompt To Fix All With AI
Reviews (1): Last reviewed commit: "feat(skill): add skills management manag..." | Re-trigger Greptile