Skip to content

Increase default max-tokens from 1000 to 8192 #204

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
alexkroman merged 1 commit into from
Jun 16, 2026
Merged

Increase default max-tokens from 1000 to 8192 #204

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion aai_cli/core/llm.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,10 @@
# The LLM Gateway is OpenAI-compatible, so we talk to it through the OpenAI SDK
# pointed at the active environment's gateway base (see _client / code_gen).
DEFAULT_MODEL = "claude-haiku-4-5-20251001"
DEFAULT_MAX_TOKENS = 1000
# Generous ceiling so long reduces/summaries aren't clipped mid-sentence; the
# gateway only bills tokens actually generated, so a high cap costs nothing on
# short replies. Override per-call with --max-tokens.
DEFAULT_MAX_TOKENS = 8192

# Exact tag the gateway substitutes with a transcript's text when `transcript_id`
# is supplied. Must be exactly "{{ transcript }}" (spaces included).
Expand Down
2 changes: 1 addition & 1 deletion tests/__snapshots__/test_snapshots_help_history.ambr
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,7 @@
│ transcript. │
│ --model TEXT LLM Gateway model │
│ [default: claude-haiku-4-5-20251001] │
│ --max-tokens INTEGER Max tokens [default: 1000] │
│ --max-tokens INTEGER Max tokens [default: 8192] │
╰──────────────────────────────────────────────────────────────────────────────╯

Examples
Expand Down
16 changes: 8 additions & 8 deletions tests/__snapshots__/test_snapshots_help_run.ambr
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@
│ [default: │
│ claude-haiku-4-5-20251001] │
│ --max-tokens INTEGER RANGE [x>=1] Max tokens per reply │
│ [default: 1000] │
│ [default: 8192] │
│ --llm-config TEXT Set any LLM Gateway request field │
│ as KEY=VALUE (repeatable) │
╰──────────────────────────────────────────────────────────────────────────────╯
Expand Down Expand Up @@ -314,7 +314,7 @@
│ --model TEXT LLM Gateway model for --llm │
│ [default: claude-haiku-4-5-20251001] │
│ --max-tokens INTEGER Max tokens for the --llm selection reply │
│ [default: 1000] │
│ [default: 8192] │
╰──────────────────────────────────────────────────────────────────────────────╯

Examples
Expand Down Expand Up @@ -461,7 +461,7 @@
│ utterances │
│ [default: claude-haiku-4-5-20251001] │
│ --max-tokens INTEGER Max tokens per utterance translation │
│ [default: 1000] │
│ [default: 8192] │
╰──────────────────────────────────────────────────────────────────────────────╯

Examples
Expand Down Expand Up @@ -563,7 +563,7 @@
│ previous one's output. │
│ --model TEXT LLM Gateway model │
│ [default: claude-haiku-4-5-20251001] │
│ --max-tokens INTEGER Max tokens [default: 1000] │
│ --max-tokens INTEGER Max tokens [default: 8192] │
╰──────────────────────────────────────────────────────────────────────────────╯

Examples
Expand Down Expand Up @@ -621,7 +621,7 @@
│ answer, pipe-friendly) or │
│ json │
│ --max-tokens INTEGER RANGE [x>=1] Max tokens to generate │
│ [default: 1000] │
│ [default: 8192] │
│ --config TEXT Set any extra gateway request │
│ field: KEY=VALUE, repeatable │
│ (e.g. --config │
Expand Down Expand Up @@ -856,7 +856,7 @@
│ --model TEXT LLM Gateway model │
│ [default: │
│ claude-haiku-4-5-20251001] │
│ --max-tokens INTEGER RANGE [x>=1] Max tokens [default: 1000] │
│ --max-tokens INTEGER RANGE [x>=1] Max tokens [default: 8192] │
╰──────────────────────────────────────────────────────────────────────────────╯
╭─ Advanced ───────────────────────────────────────────────────────────────────╮
│ --config KEY=VALUE Set any StreamingParameters field as │
Expand Down Expand Up @@ -1066,7 +1066,7 @@
│ transcript. │
│ --model TEXT LLM Gateway model │
│ [default: claude-haiku-4-5-20251001] │
│ --max-tokens INTEGER Max tokens [default: 1000] │
│ --max-tokens INTEGER Max tokens [default: 8192] │
╰──────────────────────────────────────────────────────────────────────────────╯

Examples
Expand Down Expand Up @@ -1251,7 +1251,7 @@
│ transcript. │
│ --model TEXT LLM Gateway model │
│ [default: claude-haiku-4-5-20251001] │
│ --max-tokens INTEGER Max tokens [default: 1000] │
│ --max-tokens INTEGER Max tokens [default: 8192] │
╰──────────────────────────────────────────────────────────────────────────────╯

Examples
Expand Down
Loading