-
Notifications
You must be signed in to change notification settings - Fork 163
feat: add @aicoach chat participant for AI Engineer Coach data access #16
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
mc5eamus
wants to merge
29
commits into
main
Choose a base branch
from
feat/chat-with-my-data
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+1,318
−6
Open
Changes from all commits
Commits
Show all changes
29 commits
Select commit
Hold shift + click to select a range
64fd7da
feat: add @aicoach chat participant for AI Engineer Coach data access
f276f18
fix: sync lockfile and cspell for CI verify job
Copilot 383b028
style: use American spelling in chat prompts
Copilot 76e5426
style: normalize analyze spelling in manifest metadata
Copilot eaa0f1e
feat: implement capped conversation history management in chat partic…
eb65e0f
feat: update chat participant tools and implement gated reporting for…
a0d205a
feat: update credit usage tool to conditionally handle token reportin…
d870e0c
Potential fix for pull request finding
mc5eamus af3e2dd
fix: use structural guards for chat history turns
Copilot 6257f27
docs: correct chat tool count
Copilot ab42c17
fix: avoid mutating antipattern cache order
Copilot 32062bc
feat: add @aicoach chat participant for AI Engineer Coach data access
d414d32
fix: sync lockfile and cspell for CI verify job
Copilot c39b3fb
style: use American spelling in chat prompts
Copilot 4593be5
style: normalize analyze spelling in manifest metadata
Copilot b382b7d
feat: implement capped conversation history management in chat partic…
04c24f8
feat: update chat participant tools and implement gated reporting for…
050a4d3
feat: update credit usage tool to conditionally handle token reportin…
4e9cae9
Potential fix for pull request finding
mc5eamus 942a2a0
fix: use structural guards for chat history turns
Copilot e6a5762
docs: correct chat tool count
Copilot 739f985
fix: avoid mutating antipattern cache order
Copilot fee63cb
Merge branch 'feat/chat-with-my-data' of https://github.com/microsoft…
8669b3d
docs: note chat sidebar prerequisite
Copilot df7203c
Potential fix for pull request finding
mc5eamus c0b4bcd
Potential fix for pull request finding
mc5eamus 6ae71de
Harden system prompt against tool-output prompt injection
Copilot 3442dce
fix: validate sessions paging input for MCP tool
Copilot 703ebb1
Merge origin/main into feat/chat-with-my-data
Copilot File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,95 @@ | ||
| --- | ||
| title: "Chat Participant" | ||
| weight: 30 | ||
| description: "Conversational access to all coaching data via @aicoach in VS Code chat" | ||
| --- | ||
|
|
||
| # Chat Participant | ||
|
|
||
| The `@aicoach` chat participant gives you conversational access to all AI Engineer Coach data directly in the VS Code chat panel. Ask questions in natural language and get data-driven coaching responses without leaving your editor. | ||
|
|
||
| ## Getting Started | ||
|
|
||
| Before using `@aicoach`, open the AI Engineer Coach sidebar at least once to load your session data. | ||
|
|
||
|
mc5eamus marked this conversation as resolved.
|
||
| > **Note:** When you use `@aicoach`, your question and the results returned by AI Engineer Coach tools are sent to the selected VS Code chat model so it can synthesize a response. The underlying coaching data is gathered locally, but the final chat answer is not produced purely through local processing. | ||
|
|
||
| Type `@aicoach` in any VS Code chat panel followed by your question: | ||
|
|
||
| ``` | ||
| @aicoach how am I doing this week? | ||
| ``` | ||
|
|
||
| The participant is sticky — once invoked, follow-up messages in the same thread continue the conversation without needing to type `@aicoach` again. | ||
|
|
||
|
mc5eamus marked this conversation as resolved.
|
||
| ## Slash Commands | ||
|
|
||
| | Command | Description | Default prompt | | ||
| |---|---|---| | ||
| | `/summary` | Quick usage overview | Highlights strengths and top areas to improve | | ||
| | `/improve` | Improvement recommendations | Top 3 things to improve with specific actions | | ||
| | `/compare` | Tool comparison | Compare AI coding tools and their effectiveness | | ||
| | `/flow` | Flow & focus analysis | Deep work patterns and best productivity hours | | ||
|
|
||
| Use a slash command with no additional text to get the default analysis, or add your own question: | ||
|
|
||
| ``` | ||
| @aicoach /flow Am I more productive in the morning or afternoon? | ||
| ``` | ||
|
|
||
| ## Available Tools | ||
|
|
||
| The participant has access to 12 backend tools that it selects automatically based on your question: | ||
|
|
||
| | Tool | Domain | What it returns | | ||
| |---|---|---| | ||
| | `aiEngineerCoach_summary` | Observe | Session counts, recommendations, top anti-patterns | | ||
| | `aiEngineerCoach_activity` | Observe | Daily requests, LOC, sessions, and harness breakdown | | ||
| | `aiEngineerCoach_credits` | Measure | AI credit usage, per-model breakdown, daily trend, and costly requests | | ||
| | `aiEngineerCoach_codeProduction` | Measure | AI vs user LOC, language breakdown, workspace distribution | | ||
| | `aiEngineerCoach_flow` | Measure | Deep work scores, best hours, follow-up latency | | ||
| | `aiEngineerCoach_patterns` | Improve | Anti-patterns and practice recommendations with severity | | ||
| | `aiEngineerCoach_insights` | Improve | Learning velocity, intent classification, prompt maturity | | ||
| | `aiEngineerCoach_wellbeing` | Improve | Work-life balance score, time distribution, burnout risk | | ||
| | `aiEngineerCoach_workflows` | Improve | Repeated workflow clusters with automation suggestions | | ||
| | `aiEngineerCoach_harnessComparison` | Observe | Side-by-side tool comparison: sessions, LOC, cancel rates | | ||
| | `aiEngineerCoach_sessions` | Observe | Browse or search individual sessions by ID or keyword | | ||
| | `aiEngineerCoach_contextHealth` | Improve | Context utilization, compaction, config health, and instruction quality | | ||
|
|
||
|
mc5eamus marked this conversation as resolved.
|
||
| All tools accept optional `fromDate`, `toDate`, `workspaceId`, and `harness` filters. The participant resolves relative time references ("last week", "past month") automatically. | ||
|
|
||
| ## How It Works | ||
|
|
||
| The participant runs an **agentic loop** that: | ||
|
|
||
| 1. Sends your question along with a coaching persona and tool-selection heuristics to the language model | ||
| 2. The model decides which tools to call based on your intent | ||
| 3. Tool results are fed back into the conversation for the model to synthesize | ||
| 4. The model may call additional tools if needed (up to 8 rounds) | ||
| 5. A final, synthesized coaching response is streamed back to you | ||
|
mc5eamus marked this conversation as resolved.
|
||
|
|
||
| This means a single question like "compare my productivity this week vs last week" can trigger multiple tool calls (activity, flow, code production) and produce a unified answer. | ||
|
|
||
| ## Example Conversations | ||
|
|
||
| **Broad check-in:** | ||
| ``` | ||
| @aicoach Give me a quick health check | ||
| ``` | ||
| → Calls `summary`, returns practice scores, session count, top anti-pattern, and a suggested next step. | ||
|
|
||
| **Specific investigation:** | ||
| ``` | ||
| @aicoach Why is my prompt quality score dropping? | ||
| ``` | ||
| → Calls `patterns` with recent date range, surfaces the specific anti-patterns driving the score down with example prompts from your sessions. | ||
|
|
||
| ## Follow-ups | ||
|
|
||
| After each response, the participant suggests follow-up prompts to guide deeper analysis: | ||
|
|
||
| - **Improve** — "What should I improve next?" | ||
| - **Compare tools** — "Compare my AI tools" | ||
| - **Flow state** — "How is my focus & flow?" | ||
|
|
||
| Click any follow-up to continue the conversation without typing. | ||
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.