Skip to content

feat: model-aware safety buffer for context compaction (closes #2)#5

Merged
Blankll merged 1 commit into
masterfrom
feat/model-aware-safety-buffer
Jun 17, 2026
Merged

feat: model-aware safety buffer for context compaction (closes #2)#5
Blankll merged 1 commit into
masterfrom
feat/model-aware-safety-buffer

Conversation

@Blankll

@Blankll Blankll commented Jun 17, 2026

Copy link
Copy Markdown
Member

What

Implements #2 — replaces the flat compaction safety buffer and ratio with tiered, model-aware functions.

Changes

Safety buffer

  • Removed flat SAFETY_BUFFER_TOKENS = 13,000
  • Added safety_buffer_for_window(context_window) -> usize:
    • ≥ 200K → 8,000
    • ≥ 32K → 4,000
    • < 32K → 2,000

Compaction ratio

  • Removed flat DEFAULT_COMPACT_RATIO = 0.75
  • Added compact_ratio_for_window(context_window) -> f64:
    • ≥ 100K → 0.85 (less aggressive summarization)
    • < 100K → 0.75

API change

  • compact_trigger_threshold(&ModelSpec) — now model-aware, computes usable_window() internally

Impact

  • Large windows (GPT-4.1 1M+) use more capacity before compacting
  • Small windows (Ollama 8K) no longer trigger at near-zero tokens (was 1 → now 4,144)
  • No downstream breakage: crate not yet published, all callers updated

Verification

  • ✅ Build: green
  • ✅ Tests: 62/62 pass (8 new)
  • ✅ Clippy + fmt: clean
  • ✅ Oracle peer-reviewed

Replace flat SAFETY_BUFFER_TOKENS (13,000) with tiered
safety_buffer_for_window() that scales proportionally:
  >= 200K -> 8,000 | >= 32K -> 4,000 | < 32K -> 2,000

Replace flat DEFAULT_COMPACT_RATIO (0.75) with tiered
compact_ratio_for_window():
  >= 100K -> 0.85 (less aggressive) | < 100K -> 0.75

compact_trigger_threshold now takes &ModelSpec instead of
raw capacity, computing usable_window() internally.

Closes #2
@Blankll Blankll merged commit a2795e0 into master Jun 17, 2026
12 checks passed
@Blankll Blankll deleted the feat/model-aware-safety-buffer branch June 17, 2026 17:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant