diff --git a/bin/gstack-config b/bin/gstack-config index 0cec75b6a5..dd85b12761 100755 --- a/bin/gstack-config +++ b/bin/gstack-config @@ -48,6 +48,17 @@ CONFIG_HEADER='# gstack configuration — edit freely, changes take effect on ne # # continuous — auto-commit after each significant change # # with WIP: prefix + [gstack-context] body # +# ─── Outside voice (plan reviews) ──────────────────────────────────── +# outside_voice_llm_model: grok-4-fast +# # Used when `codex` is unavailable or errors. If the `llm` +# # CLI (datasette/llm) is installed AND this model is +# # registered (e.g. via `llm install llm-xai`), skills +# # invoke `llm -m ...` as the next fallback before +# # the Claude subagent. Set to empty string to skip the +# # llm step entirely. +# # Default: grok-4-fast (per Grok reasoning-mode caveat — +# # grok-4-latest can hang 2-5 minutes on big prompts). +# # checkpoint_push: false # true = push WIP commits to remote as you go # # false = keep WIP commits local only (default) # # Pushing can trigger CI/deploy hooks — opt in carefully. @@ -107,6 +118,7 @@ lookup_default() { cross_project_learnings) echo "" ;; # intentionally empty → unset triggers first-time prompt artifacts_sync_mode) echo "off" ;; artifacts_sync_mode_prompted) echo "false" ;; + outside_voice_llm_model) echo "grok-4-fast" ;; *) echo "" ;; esac } diff --git a/office-hours/SKILL.md b/office-hours/SKILL.md index b8b6fe1f9c..793895fee4 100644 --- a/office-hours/SKILL.md +++ b/office-hours/SKILL.md @@ -1265,7 +1265,9 @@ codex exec "$(cat "$CODEX_PROMPT_FILE")" -C "$_REPO_ROOT" -s read-only -c 'model Use a 5-minute timeout (`timeout: 300000`). After the command completes, read stderr: ```bash cat "$TMPERR_OH" -rm -f "$TMPERR_OH" "$CODEX_PROMPT_FILE" +rm -f "$TMPERR_OH" +# NOTE: keep $CODEX_PROMPT_FILE alive — the llm fallback (next step) reuses it. +# Final cleanup of $CODEX_PROMPT_FILE happens after the whole outside-voice chain. ``` **Error handling:** All errors are non-blocking — second opinion is a quality enhancement, not a prerequisite. @@ -1273,9 +1275,52 @@ rm -f "$TMPERR_OH" "$CODEX_PROMPT_FILE" - **Timeout:** "Codex timed out after 5 minutes." Fall back to Claude subagent. - **Empty response:** "Codex returned no response." Fall back to Claude subagent. -On any Codex error, fall back to the Claude subagent below. +On any Codex error, try the `llm` CLI next, then fall back to the Claude subagent. -**If CODEX_NOT_AVAILABLE (or Codex errored):** +**If CODEX_NOT_AVAILABLE (or Codex errored), try `llm` CLI:** + +The `llm` CLI (datasette/llm) supports Grok and other non-OpenAI models — when configured, it provides genuine cross-model independence (different model family than Claude). The skill skips this step if `outside_voice_llm_model` is unset or the model isn't registered with `llm`. + +```bash +LLM_MODEL=$(~/.claude/skills/gstack/bin/gstack-config get outside_voice_llm_model 2>/dev/null) +if [ -n "$LLM_MODEL" ] && command -v llm >/dev/null 2>&1 && llm models 2>/dev/null | grep -q "$LLM_MODEL\$"; then + echo "LLM_AVAILABLE: $LLM_MODEL" +else + echo "LLM_NOT_AVAILABLE" +fi +``` + +**If LLM_AVAILABLE:** + +Reuse the second-opinion prompt assembled for Codex (read from `$CODEX_PROMPT_FILE`). Invoke llm with the configured model: + +```bash +TMPERR_LLM=$(mktemp /tmp/llm-oh-err-XXXXXXXX) +llm -m "$LLM_MODEL" "$(cat "$CODEX_PROMPT_FILE")" 2>"$TMPERR_LLM" +``` + +Use a 5-minute timeout (`timeout: 300000`). After completion, read stderr: +```bash +cat "$TMPERR_LLM" && rm -f "$TMPERR_LLM" "$CODEX_PROMPT_FILE" +``` + +Present the full output verbatim: + +``` +SECOND OPINION (llm, $LLM_MODEL): +════════════════════════════════════════════════════════════ + +════════════════════════════════════════════════════════════ +``` + +**Error handling:** All errors are non-blocking — fall back to the Claude subagent. +- Auth failure (stderr contains "key", "auth", "401", "API key"): "llm key not configured. Run \`llm keys set \` to set it up." Fall back to Claude subagent. +- Timeout: "llm timed out after 5 minutes." Fall back to Claude subagent. +- Empty response: "llm returned no response." Fall back to Claude subagent. + +On any llm error, fall back to the Claude subagent below. + +**If LLM_NOT_AVAILABLE (or llm errored, or both Codex and llm failed):** Dispatch via the Agent tool. The subagent has fresh context — genuine independence. @@ -1295,6 +1340,14 @@ SECOND OPINION (Codex): ════════════════════════════════════════════════════════════ ``` +If llm ran: +``` +SECOND OPINION (llm, $LLM_MODEL): +════════════════════════════════════════════════════════════ + +════════════════════════════════════════════════════════════ +``` + If Claude subagent ran: ``` SECOND OPINION (Claude subagent): @@ -1303,6 +1356,11 @@ SECOND OPINION (Claude subagent): ════════════════════════════════════════════════════════════ ``` +Final cleanup (runs regardless of which path produced output): +```bash +rm -f "$CODEX_PROMPT_FILE" +``` + 5. **Cross-model synthesis:** After presenting the second opinion output, provide 3-5 bullet synthesis: - Where Claude agrees with the second opinion - Where Claude disagrees and why diff --git a/plan-ceo-review/SKILL.md b/plan-ceo-review/SKILL.md index a0b24ef99a..001eb68abe 100644 --- a/plan-ceo-review/SKILL.md +++ b/plan-ceo-review/SKILL.md @@ -1680,9 +1680,52 @@ CODEX SAYS (plan review — outside voice): - Timeout: "Codex timed out after 5 minutes." - Empty response: "Codex returned no response." -On any Codex error, fall back to the Claude adversarial subagent. +On any Codex error, try the `llm` CLI next, then fall back to the Claude subagent. -**If CODEX_NOT_AVAILABLE (or Codex errored):** +**If CODEX_NOT_AVAILABLE (or Codex errored), try `llm` CLI:** + +The `llm` CLI (datasette/llm) supports Grok and other non-OpenAI models — when configured, it provides genuine cross-model independence (different model family than Claude). The skill skips this step if `outside_voice_llm_model` is unset or the model isn't registered with `llm`. + +```bash +LLM_MODEL=$(~/.claude/skills/gstack/bin/gstack-config get outside_voice_llm_model 2>/dev/null) +if [ -n "$LLM_MODEL" ] && command -v llm >/dev/null 2>&1 && llm models 2>/dev/null | grep -q "$LLM_MODEL\$"; then + echo "LLM_AVAILABLE: $LLM_MODEL" +else + echo "LLM_NOT_AVAILABLE" +fi +``` + +**If LLM_AVAILABLE:** + +Reuse the plan review prompt assembled for Codex. Invoke llm with the configured model: + +```bash +TMPERR_LLM=$(mktemp /tmp/llm-planreview-XXXXXXXX) +llm -m "$LLM_MODEL" "" 2>"$TMPERR_LLM" +``` + +Use a 5-minute timeout (`timeout: 300000`). After completion, read stderr: +```bash +cat "$TMPERR_LLM" && rm -f "$TMPERR_LLM" +``` + +Present the full output verbatim: + +``` +OUTSIDE VOICE (llm, $LLM_MODEL): +════════════════════════════════════════════════════════════ + +════════════════════════════════════════════════════════════ +``` + +**Error handling:** All errors are non-blocking — fall back to the Claude subagent. +- Auth failure (stderr contains "key", "auth", "401", "API key"): "llm key not configured for the model's provider. Run \`llm keys set \` to set it up." +- Timeout: "llm timed out after 5 minutes." +- Empty response: "llm returned no response." + +On any llm error, fall back to the Claude subagent below. + +**If LLM_NOT_AVAILABLE (or llm errored, or both Codex and llm failed):** Dispatch via the Agent tool. The subagent has fresh context — genuine independence. diff --git a/plan-devex-review/SKILL.md b/plan-devex-review/SKILL.md index 371d07a758..cce6fb3325 100644 --- a/plan-devex-review/SKILL.md +++ b/plan-devex-review/SKILL.md @@ -1652,9 +1652,52 @@ CODEX SAYS (plan review — outside voice): - Timeout: "Codex timed out after 5 minutes." - Empty response: "Codex returned no response." -On any Codex error, fall back to the Claude adversarial subagent. +On any Codex error, try the `llm` CLI next, then fall back to the Claude subagent. -**If CODEX_NOT_AVAILABLE (or Codex errored):** +**If CODEX_NOT_AVAILABLE (or Codex errored), try `llm` CLI:** + +The `llm` CLI (datasette/llm) supports Grok and other non-OpenAI models — when configured, it provides genuine cross-model independence (different model family than Claude). The skill skips this step if `outside_voice_llm_model` is unset or the model isn't registered with `llm`. + +```bash +LLM_MODEL=$(~/.claude/skills/gstack/bin/gstack-config get outside_voice_llm_model 2>/dev/null) +if [ -n "$LLM_MODEL" ] && command -v llm >/dev/null 2>&1 && llm models 2>/dev/null | grep -q "$LLM_MODEL\$"; then + echo "LLM_AVAILABLE: $LLM_MODEL" +else + echo "LLM_NOT_AVAILABLE" +fi +``` + +**If LLM_AVAILABLE:** + +Reuse the plan review prompt assembled for Codex. Invoke llm with the configured model: + +```bash +TMPERR_LLM=$(mktemp /tmp/llm-planreview-XXXXXXXX) +llm -m "$LLM_MODEL" "" 2>"$TMPERR_LLM" +``` + +Use a 5-minute timeout (`timeout: 300000`). After completion, read stderr: +```bash +cat "$TMPERR_LLM" && rm -f "$TMPERR_LLM" +``` + +Present the full output verbatim: + +``` +OUTSIDE VOICE (llm, $LLM_MODEL): +════════════════════════════════════════════════════════════ + +════════════════════════════════════════════════════════════ +``` + +**Error handling:** All errors are non-blocking — fall back to the Claude subagent. +- Auth failure (stderr contains "key", "auth", "401", "API key"): "llm key not configured for the model's provider. Run \`llm keys set \` to set it up." +- Timeout: "llm timed out after 5 minutes." +- Empty response: "llm returned no response." + +On any llm error, fall back to the Claude subagent below. + +**If LLM_NOT_AVAILABLE (or llm errored, or both Codex and llm failed):** Dispatch via the Agent tool. The subagent has fresh context — genuine independence. diff --git a/plan-eng-review/SKILL.md b/plan-eng-review/SKILL.md index 925daab131..c25dbfb4fb 100644 --- a/plan-eng-review/SKILL.md +++ b/plan-eng-review/SKILL.md @@ -1281,9 +1281,52 @@ CODEX SAYS (plan review — outside voice): - Timeout: "Codex timed out after 5 minutes." - Empty response: "Codex returned no response." -On any Codex error, fall back to the Claude adversarial subagent. +On any Codex error, try the `llm` CLI next, then fall back to the Claude subagent. -**If CODEX_NOT_AVAILABLE (or Codex errored):** +**If CODEX_NOT_AVAILABLE (or Codex errored), try `llm` CLI:** + +The `llm` CLI (datasette/llm) supports Grok and other non-OpenAI models — when configured, it provides genuine cross-model independence (different model family than Claude). The skill skips this step if `outside_voice_llm_model` is unset or the model isn't registered with `llm`. + +```bash +LLM_MODEL=$(~/.claude/skills/gstack/bin/gstack-config get outside_voice_llm_model 2>/dev/null) +if [ -n "$LLM_MODEL" ] && command -v llm >/dev/null 2>&1 && llm models 2>/dev/null | grep -q "$LLM_MODEL\$"; then + echo "LLM_AVAILABLE: $LLM_MODEL" +else + echo "LLM_NOT_AVAILABLE" +fi +``` + +**If LLM_AVAILABLE:** + +Reuse the plan review prompt assembled for Codex. Invoke llm with the configured model: + +```bash +TMPERR_LLM=$(mktemp /tmp/llm-planreview-XXXXXXXX) +llm -m "$LLM_MODEL" "" 2>"$TMPERR_LLM" +``` + +Use a 5-minute timeout (`timeout: 300000`). After completion, read stderr: +```bash +cat "$TMPERR_LLM" && rm -f "$TMPERR_LLM" +``` + +Present the full output verbatim: + +``` +OUTSIDE VOICE (llm, $LLM_MODEL): +════════════════════════════════════════════════════════════ + +════════════════════════════════════════════════════════════ +``` + +**Error handling:** All errors are non-blocking — fall back to the Claude subagent. +- Auth failure (stderr contains "key", "auth", "401", "API key"): "llm key not configured for the model's provider. Run \`llm keys set \` to set it up." +- Timeout: "llm timed out after 5 minutes." +- Empty response: "llm returned no response." + +On any llm error, fall back to the Claude subagent below. + +**If LLM_NOT_AVAILABLE (or llm errored, or both Codex and llm failed):** Dispatch via the Agent tool. The subagent has fresh context — genuine independence. diff --git a/scripts/resolvers/review.ts b/scripts/resolvers/review.ts index 0c7cb8230f..76a54c4842 100644 --- a/scripts/resolvers/review.ts +++ b/scripts/resolvers/review.ts @@ -357,7 +357,9 @@ codex exec "$(cat "$CODEX_PROMPT_FILE")" -C "$_REPO_ROOT" -s read-only -c 'model Use a 5-minute timeout (\`timeout: 300000\`). After the command completes, read stderr: \`\`\`bash cat "$TMPERR_OH" -rm -f "$TMPERR_OH" "$CODEX_PROMPT_FILE" +rm -f "$TMPERR_OH" +# NOTE: keep \$CODEX_PROMPT_FILE alive — the llm fallback (next step) reuses it. +# Final cleanup of \$CODEX_PROMPT_FILE happens after the whole outside-voice chain. \`\`\` **Error handling:** All errors are non-blocking — second opinion is a quality enhancement, not a prerequisite. @@ -365,9 +367,52 @@ rm -f "$TMPERR_OH" "$CODEX_PROMPT_FILE" - **Timeout:** "Codex timed out after 5 minutes." Fall back to Claude subagent. - **Empty response:** "Codex returned no response." Fall back to Claude subagent. -On any Codex error, fall back to the Claude subagent below. +On any Codex error, try the \`llm\` CLI next, then fall back to the Claude subagent. -**If CODEX_NOT_AVAILABLE (or Codex errored):** +**If CODEX_NOT_AVAILABLE (or Codex errored), try \`llm\` CLI:** + +The \`llm\` CLI (datasette/llm) supports Grok and other non-OpenAI models — when configured, it provides genuine cross-model independence (different model family than Claude). The skill skips this step if \`outside_voice_llm_model\` is unset or the model isn't registered with \`llm\`. + +\`\`\`bash +LLM_MODEL=$(~/.claude/skills/gstack/bin/gstack-config get outside_voice_llm_model 2>/dev/null) +if [ -n "$LLM_MODEL" ] && command -v llm >/dev/null 2>&1 && llm models 2>/dev/null | grep -q "$LLM_MODEL\\$"; then + echo "LLM_AVAILABLE: $LLM_MODEL" +else + echo "LLM_NOT_AVAILABLE" +fi +\`\`\` + +**If LLM_AVAILABLE:** + +Reuse the second-opinion prompt assembled for Codex (read from \`$CODEX_PROMPT_FILE\`). Invoke llm with the configured model: + +\`\`\`bash +TMPERR_LLM=$(mktemp /tmp/llm-oh-err-XXXXXXXX) +llm -m "$LLM_MODEL" "$(cat "$CODEX_PROMPT_FILE")" 2>"$TMPERR_LLM" +\`\`\` + +Use a 5-minute timeout (\`timeout: 300000\`). After completion, read stderr: +\`\`\`bash +cat "$TMPERR_LLM" && rm -f "$TMPERR_LLM" "$CODEX_PROMPT_FILE" +\`\`\` + +Present the full output verbatim: + +\`\`\` +SECOND OPINION (llm, \$LLM_MODEL): +════════════════════════════════════════════════════════════ + +════════════════════════════════════════════════════════════ +\`\`\` + +**Error handling:** All errors are non-blocking — fall back to the Claude subagent. +- Auth failure (stderr contains "key", "auth", "401", "API key"): "llm key not configured. Run \\\`llm keys set \\\` to set it up." Fall back to Claude subagent. +- Timeout: "llm timed out after 5 minutes." Fall back to Claude subagent. +- Empty response: "llm returned no response." Fall back to Claude subagent. + +On any llm error, fall back to the Claude subagent below. + +**If LLM_NOT_AVAILABLE (or llm errored, or both Codex and llm failed):** Dispatch via the Agent tool. The subagent has fresh context — genuine independence. @@ -387,6 +432,14 @@ SECOND OPINION (Codex): ════════════════════════════════════════════════════════════ \`\`\` +If llm ran: +\`\`\` +SECOND OPINION (llm, \$LLM_MODEL): +════════════════════════════════════════════════════════════ + +════════════════════════════════════════════════════════════ +\`\`\` + If Claude subagent ran: \`\`\` SECOND OPINION (Claude subagent): @@ -395,6 +448,11 @@ SECOND OPINION (Claude subagent): ════════════════════════════════════════════════════════════ \`\`\` +Final cleanup (runs regardless of which path produced output): +\`\`\`bash +rm -f "$CODEX_PROMPT_FILE" +\`\`\` + 5. **Cross-model synthesis:** After presenting the second opinion output, provide 3-5 bullet synthesis: - Where Claude agrees with the second opinion - Where Claude disagrees and why @@ -667,9 +725,52 @@ CODEX SAYS (plan review — outside voice): - Timeout: "Codex timed out after 5 minutes." - Empty response: "Codex returned no response." -On any Codex error, fall back to the Claude adversarial subagent. +On any Codex error, try the \`llm\` CLI next, then fall back to the Claude subagent. + +**If CODEX_NOT_AVAILABLE (or Codex errored), try \`llm\` CLI:** + +The \`llm\` CLI (datasette/llm) supports Grok and other non-OpenAI models — when configured, it provides genuine cross-model independence (different model family than Claude). The skill skips this step if \`outside_voice_llm_model\` is unset or the model isn't registered with \`llm\`. + +\`\`\`bash +LLM_MODEL=$(~/.claude/skills/gstack/bin/gstack-config get outside_voice_llm_model 2>/dev/null) +if [ -n "$LLM_MODEL" ] && command -v llm >/dev/null 2>&1 && llm models 2>/dev/null | grep -q "$LLM_MODEL\\$"; then + echo "LLM_AVAILABLE: $LLM_MODEL" +else + echo "LLM_NOT_AVAILABLE" +fi +\`\`\` + +**If LLM_AVAILABLE:** + +Reuse the plan review prompt assembled for Codex. Invoke llm with the configured model: + +\`\`\`bash +TMPERR_LLM=$(mktemp /tmp/llm-planreview-XXXXXXXX) +llm -m "$LLM_MODEL" "" 2>"$TMPERR_LLM" +\`\`\` + +Use a 5-minute timeout (\`timeout: 300000\`). After completion, read stderr: +\`\`\`bash +cat "$TMPERR_LLM" && rm -f "$TMPERR_LLM" +\`\`\` + +Present the full output verbatim: + +\`\`\` +OUTSIDE VOICE (llm, \$LLM_MODEL): +════════════════════════════════════════════════════════════ + +════════════════════════════════════════════════════════════ +\`\`\` + +**Error handling:** All errors are non-blocking — fall back to the Claude subagent. +- Auth failure (stderr contains "key", "auth", "401", "API key"): "llm key not configured for the model's provider. Run \\\`llm keys set \\\` to set it up." +- Timeout: "llm timed out after 5 minutes." +- Empty response: "llm returned no response." + +On any llm error, fall back to the Claude subagent below. -**If CODEX_NOT_AVAILABLE (or Codex errored):** +**If LLM_NOT_AVAILABLE (or llm errored, or both Codex and llm failed):** Dispatch via the Agent tool. The subagent has fresh context — genuine independence.