[claude-code-user-docs-review] Claude Code User Documentation Review - 2026-04-14 #26220
Closed
Replies: 1 comment
-
|
This discussion has been marked as outdated by Claude Code User Documentation Review. A newer discussion is available at Discussion #26433. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Executive Summary
I reviewed the gh-aw documentation from the perspective of a developer who uses Claude Code as their primary AI assistant and has no GitHub Copilot access. The documentation has improved substantially: authentication, tools, CLI commands, and engine configuration are all well-covered for non-Copilot users. However, several Copilot-centric patterns persist — most notably in architectural diagrams, the workflow-authoring guide, and the
gh aw initdescription — that create friction and confusion for Claude Code users evaluating this tool.Key Finding: Claude Code users can successfully use gh-aw, but they will encounter several documentation gaps that require them to work past Copilot-centric framing before reaching the Claude-specific instructions they need.
Persona Context
I reviewed this documentation as a developer who:
Question 1: Onboarding Experience
Can a Claude Code user understand and get started with gh-aw?
Generally yes, but with friction. The Quick Start prerequisites explicitly list "Anthropic Claude" alongside Copilot and Codex — this is the first thing a new user reads and it correctly sets expectations. The
add-wizardinteractive command also presents engine selection early. However, subsequent documentation sections revert to Copilot-centric framing that could confuse or discourage a non-Copilot user.Specific Issues Found:
docs/src/content/docs/setup/creating-workflows.mdxline 20–21: The GitHub Web Interface authoring section opens with "If you have access to GitHub Copilot, you can create and edit Agentic Workflows directly from the Web Interface." This explicitly gates a major authoring path on Copilot with no alternative offered for Claude users. A new user without Copilot would feel this entire section doesn't apply to them.docs/src/content/docs/setup/creating-workflows.mdxlines 104–128: Thegh aw initsection explains the command primarily in terms of Copilot Chat benefits: registering the/agent agentic-workflowsdispatcher in Copilot Chat, enabling workflow authoring via Copilot on github.com. Claude users runninggh aw initwon't get these benefits, and the docs don't explain what they do get (.gitattributessetup, VS Code settings).docs/src/content/docs/introduction/architecture.mdxline 181 and 252: The AWF firewall diagram showsCOPILOT["Copilot CLI"]as the sole entry in the "AI Agent Process" box. The MCP Gateway diagram labels the agent container as"Copilot CLI + MCP client". These diagrams apply to ALL engines but mislead users into thinking only Copilot is supported at the infrastructure level.Recommended Fixes:
gh aw initsection to describe the full set of benefits for all users, and note explicitly which parts (the dispatcher agent) are Copilot-specific.COPILOT["Copilot CLI"]withAGENT["AI Engine\n(Copilot CLI, Claude Code, etc.)"]and fix the MCP Gateway label similarly.Question 2: Inaccessible Features for Non-Copilot Users
What features or steps don't work without Copilot?
Features That Require Copilot:
creating-workflows.mdxline 20) — requires Copilot chat on github.com/agent agentic-workflowsdispatcher (creating-workflows.mdxline 123) — Copilot Chat specificmax-continuations— Copilot-only engine feature (well documented inengines.mdcomparison table)engine.agentcustom agent files — Copilot-only (well documented)assign-to-copilotsafe output — explicitly assigns to Copilot coding agent (clearly named, no confusion)COPILOT_GITHUB_TOKENwith GitHub App — not supported (PAT only); documented clearlyFeatures That Work Without Copilot:
engine: claude,engine: codex, orengine: geminiedit:,bash:,github:,web-fetch:,playwright:,cache-memory:,repo-memory:,agentic-workflows:,qmd:— completely engine-agnosticgh aw compile,gh aw run,gh aw logs,gh aw audit, etc.)max-turns— Claude-only but a positive feature for Claude users, not a restrictionMissing Documentation:
gh aw init.Question 3: Documentation Gaps and Assumptions
Where does the documentation assume Copilot usage?
Copilot-Centric Language Found In:
docs/src/content/docs/setup/creating-workflows.mdxline 20docs/src/content/docs/setup/creating-workflows.mdxline 118gh aw initbenefits described only as Copilot Chat featuresdocs/src/content/docs/introduction/architecture.mdxline 181COPILOT["Copilot CLI"]in the AI Agent boxdocs/src/content/docs/introduction/architecture.mdxline 252docs/src/content/docs/reference/auth.mdxline 125engines.md#available-coding-agents— an anchor that doesn't lead to Claude+MCP-specific configuration.Missing Alternative Instructions:
engine:in a workflow defaults to Copilot (requiringCOPILOT_GITHUB_TOKEN) — a silent gotcha for users who copy-paste example workflows.Severity-Categorized Findings
🚫 Critical Blockers (Score: 1/10)
No true blockers exist. A Claude Code user who carefully reads the documentation can get started. The authentication page is excellent and covers Claude setup in full detail.
Obstacle 1: Architecture Diagrams Show Only "Copilot CLI"
Impact: Confusion about whether Claude/Codex actually work at the infrastructure level
Current State:
architecture.mdxhas two Mermaid diagrams that label engine-agnostic containers with "Copilot CLI":COPILOT["Copilot CLI"]inside the "AI Agent Process" subgraph"Agent container\nCopilot CLI + MCP client\n172.30.0.20"Why It's Problematic: The architecture page is often consulted by users evaluating security and compatibility. Seeing only "Copilot CLI" in the infrastructure diagrams signals that the entire infrastructure is Copilot-specific, undermining confidence that Claude will work.
Suggested Fix: Replace "Copilot CLI" labels with "AI Engine (Copilot/Claude/Codex)" in both diagrams.
Affected Files:
docs/src/content/docs/introduction/architecture.mdxObstacle 2: Creating Workflows Guide is Copilot-Gated at Entry Point
Impact: Claude Code users feel the "official" workflow authoring path is unavailable to them
Current State: The GitHub Web Interface authoring section (the first method shown) opens with: "If you have access to GitHub Copilot, you can create and edit Agentic Workflows directly from the Web Interface." No equivalent web-based authoring path for Claude users is offered. The
gh aw initsection describes the dispatcher agent as: "registers theagentic-workflowscustom agent in GitHub Copilot and enables workflow authoring via/agent agentic-workflowsin Copilot Chat on github.com". Non-Copilot users would get no authoring UX fromgh aw initand would not understand what value they receive from running it.Why It's Problematic: A Claude user reading this guide would conclude: (a) web authoring requires Copilot, (b)
gh aw initdoesn't benefit them. Both impressions are only partially true —gh aw initdoes set up.gitattributesand VS Code settings — but these benefits are buried.Suggested Fix:
gh aw initsection, add a paragraph: "Even without Copilot,gh aw initsets up.gitattributes, VS Code settings, and MCP integration so your local Claude Code or Codex agent can author workflows."Affected Files:
docs/src/content/docs/setup/creating-workflows.mdxObstacle 3: Broken Cross-Reference for Claude + GitHub MCP Additional Config
Impact: Users following auth docs to understand Claude + GitHub MCP requirements end up on a page that doesn't answer their question
Current State:
auth.mdxline 125 states: "See also AI Engines for additional configuration needed when using Claude with GitHub MCP." The same pattern repeats for Codex (line 167) and Gemini (line 185). The anchor#available-coding-agentsresolves to the comparison table at the top of engines.md, which doesn't have a Claude+GitHub MCP-specific configuration section.Why It's Problematic: Users following this cross-reference expecting to find "additional configuration for Claude+MCP" land on a generic engine table. The actual additional configuration (needing a PAT for remote mode) is in
github-tools.md, not linked from here.Suggested Fix: Either remove the "See also" if there's nothing Claude-specific in engines.md, or link directly to the relevant
github-tools.mdsection on remote mode authentication.Affected Files:
docs/src/content/docs/reference/auth.mdxObstacle 4: No Warning That Omitting `engine:` Defaults to Copilot
Impact: Claude users copying example workflows without an
engine:field will get confusing authentication errorsCurrent State: engines.md states "Copilot CLI is the default —
engine:can be omitted when using Copilot." The quick-start and other guides don't prominently warn that copy-pasting workflow snippets without settingengine: claudewill requireCOPILOT_GITHUB_TOKENand fail with cryptic errors for Claude users.Why It's Problematic: Many documentation examples omit
engine:. A Claude Code user who tries one of these examples will see a Copilot authentication failure with no clear indication of what went wrong.Suggested Fix: Add a tip box in quick-start.mdx and the quick reference in how-they-work.mdx: "The default engine is Copilot. If you're using Claude, always add
engine: claudeto your workflow frontmatter (or usegh aw new --engine claudeto auto-populate it)."Affected Files:
docs/src/content/docs/setup/quick-start.mdx,docs/src/content/docs/introduction/how-they-work.mdx💡 Minor Confusion Points (Score: 3/10)
auth.mdxshowsgh aw secrets set COPILOT_GITHUB_TOKENas the only CLI example in the "Adding secrets using the CLI" section before showing the table. File:docs/src/content/docs/reference/auth.mdxengine: copilot— The network configuration example inarchitecture.mdxline 228 usesengine: copilotas the only example. Could easily showengine: claudeor omit the engine field.secrets bootstrap --engineonly lists copilot/claude/codex — Gemini is missing from--engineoption list in CLI docs:gh aw secrets bootstrap --engine copilot. Possible stale docs.Engine Comparison Analysis
Available Engines
Based on my review, gh-aw supports four engines:
copilot(default),claude,codex, andgemini.Documentation Quality by Engine
Rating Scale: ⭐⭐⭐⭐⭐ (Excellent) to ⭐ (Poor/Missing)
Notes:
CLAUDE_CODE_OAUTH_TOKENis NOT supported is uniquely helpful for Claude Code users..github/workflows/(vs. ~93 Copilot) — a meaningful body of examples.Tool Availability Analysis
Tools Review
Engine-Agnostic Tools (all engines):
edit:— file editingbash:— shell commandsgithub:— GitHub API (all toolsets)web-fetch:— HTTP fetchingplaywright:— browser automationcache-memory:— persistent memoryrepo-memory:— repository-scoped memoryagentic-workflows:— workflow introspectionqmd:— vector searchmcp-servers:)Engine-Specific Features:
max-turns— Claude only (advantage, not limitation)max-continuations— Copilot onlyengine.agent— Copilot onlyweb-search:— Codex opt-in; all others via MCPConclusion: Tools are almost entirely engine-agnostic. The Claude engine has no missing tools relative to Copilot.
Authentication Requirements
Current Documentation Status
Quick Start guide covers authentication for:
CLAUDE_CODE_OAUTH_TOKENis NOT supported noteCODEX_API_KEY/OPENAI_API_KEYfallback chain explainedClaude-Specific Auth Notes (Positive Findings)
The auth page handles Claude Code users exceptionally well:
This is exactly the kind of Claude-aware documentation that prevents frustrating auth failures.
Secret Names Reference
COPILOT_GITHUB_TOKENANTHROPIC_API_KEYOPENAI_API_KEYCODEX_API_KEYGEMINI_API_KEYExample Workflow Analysis
Workflow Count by Engine
Quality of Examples
Claude Examples: 45 workflows including
audit-workflows.md,deep-report.md,scout.md,go-fan.md,security-compliance.md, and this very workflow (claude-code-user-docs-review.md). These demonstrate that Claude workflows are production-grade and well-used in this repository.Coverage: Claude examples cover scheduling, issue analysis, code auditing, safe outputs, and multi-turn operations. Sufficient for a new user to understand what's possible.
Gap: No "hello world" Claude-specific example in the documentation site itself — examples live in
.github/workflows/but aren't featured on a docs page dedicated to Claude engine usage.Recommended Actions
Priority 1: Critical Documentation Fixes
docs/src/content/docs/introduction/architecture.mdxlines 181, 252docs/src/content/docs/reference/auth.mdxlines 125, 167, 185engine:defaults to Copilot. Files:docs/src/content/docs/setup/quick-start.mdx,docs/src/content/docs/introduction/how-they-work.mdxPriority 2: Major Improvements
gh aw initsection for non-Copilot users — Document what valuegh aw initprovides for Claude Code users (.gitattributes, VS Code settings, MCP tools). File:docs/src/content/docs/setup/creating-workflows.mdxline 102+docs/src/content/docs/setup/creating-workflows.mdxline 20+Priority 3: Nice-to-Have Enhancements
max-turnscontrol, API key billing, Claude Code familiarity) to help users make an informed choice.engine: claudeto more documentation examples — When showing workflow frontmatter snippets in reference docs, alternate the engine used rather than always defaulting to Copilot.Positive Findings
What Works Well for Claude Code Users
ANTHROPIC_API_KEYdocumented in full with a unique, Claude-Code-aware note thatCLAUDE_CODE_OAUTH_TOKENis not supportedgh aw new --engine claudegenerates correctly-configured templatesgh aw secrets bootstrap --engine claudeprovides Claude-specific secret validationmax-turnsis a Claude-only advantage clearly documentedadd-wizardengine selection makes first-run experience smooth regardless of engine choiceConclusion
Can Claude Code Users Successfully Adopt gh-aw?
Answer: Yes, with moderate effort
Reasoning: The core gh-aw feature set works equally well with Claude. Authentication documentation is strong and Claude-aware. The CLI, tools, and safe-outputs system are fully engine-agnostic. A Claude Code user who works through the quick start and reads the auth page will have a functional setup.
The friction comes from Copilot-centric framing in architectural diagrams, the creating-workflows guide's web interface section, the
gh aw initdescription, and a few misleading cross-references. These are papercuts, not walls — but they cumulatively signal to a Claude Code user that they are a second-class user of this tool, which isn't accurate.Overall Assessment Score: 7/10
Next Steps
architecture.mdx(fast, high-impact)auth.mdx(fast fix)quick-start.mdx(fast, high-value for new users)creating-workflows.mdxto neutralize Copilot-centric authoring framing (medium effort)Appendix: Files Reviewed
Complete List of Documentation Files Analyzed
README.mddocs/src/content/docs/setup/quick-start.mdxdocs/src/content/docs/setup/creating-workflows.mdxdocs/src/content/docs/setup/cli.mddocs/src/content/docs/introduction/overview.mdxdocs/src/content/docs/introduction/how-they-work.mdxdocs/src/content/docs/introduction/architecture.mdxdocs/src/content/docs/reference/tools.mddocs/src/content/docs/reference/engines.mddocs/src/content/docs/reference/auth.mdxdocs/src/content/docs/reference/github-tools.md.github/workflows/*.md(sampled 10+ Claude and Copilot workflows)References:
Workflow: claude-code-user-docs-review | Engine: claude (eating our own dog food 🐕)
Beta Was this translation helpful? Give feedback.
All reactions