|
1 | 1 | # Refresh capability matrix |
2 | 2 |
|
3 | 3 | The machine-readable source of truth is |
4 | | -[`capabilities.json`](./capabilities.json). A repository is **report-only** |
5 | | -unless every required capability is confirmed, an administrator records |
6 | | -approval evidence, and `writeAutomationAllowed` is explicitly set to `true`. |
| 4 | +[`capabilities.json`](./capabilities.json). |
7 | 5 |
|
8 | | -| Repository | Current mode | Capability status | Administrator approval | |
| 6 | +- Buildathon stays **report-only** until every same-repository write and dispatch |
| 7 | + capability is confirmed, approval evidence is recorded, and |
| 8 | + `writeAutomationAllowed` is explicitly set to `true`. |
| 9 | +- The two external sample repositories are intentionally **report-only** and do |
| 10 | + not have an automated write-enablement path under this strategy. |
| 11 | + |
| 12 | +| Repository | Role | Current mode | Capability status | |
9 | 13 | |---|---|---|---| |
10 | | -| `Azure-Samples/JavaScript-AI-Buildathon` | Report-only | Blocked | Missing | |
11 | | -| `Azure-Samples/serverless-chat-langchainjs` | Report-only | Blocked | Missing | |
12 | | -| `Azure-Samples/mcp-agent-langchainjs` | Report-only | Blocked | Missing | |
| 14 | +| `Azure-Samples/JavaScript-AI-Buildathon` | Automation host | Report-only | Blocked | |
| 15 | +| `Azure-Samples/serverless-chat-langchainjs` | External evidence | Report-only | Policy | |
| 16 | +| `Azure-Samples/mcp-agent-langchainjs` | External evidence | Report-only | Policy | |
13 | 17 |
|
14 | | -## Required confirmation |
| 18 | +## Buildathon confirmation |
15 | 19 |
|
16 | | -For each repository, an administrator must confirm: |
| 20 | +Before Buildathon write automation is enabled, confirm: |
17 | 21 |
|
18 | | -1. The dedicated GitHub App is installed for that repository. |
19 | | -2. Copilot Code Review is enabled and an AI-credit budget is available. |
20 | | -3. `refresh/**` branches may be created by the App. |
21 | | -4. The main-branch ruleset requires pull requests, CODEOWNER approval, |
22 | | - conversation resolution, and the repository's known CI checks. |
23 | | -5. The App cannot bypass the ruleset, push to `main`, force-push, delete a |
24 | | - protected branch, or merge. |
25 | | -6. `REFRESH_APPROVER_LOGIN` is set to the exact approved GitHub login. |
26 | | -7. `AGENTIC_REFRESH_APP_ID` and `AGENTIC_REFRESH_APP_PRIVATE_KEY` are configured |
27 | | - without exposing their values in issues, logs, or repository files. |
| 22 | +1. Copilot inference works with `copilot-requests: write` and the organization |
| 23 | + has an available AI-credit budget. |
| 24 | +2. `GITHUB_TOKEN` can create a Buildathon issue through a bounded safe output. |
| 25 | +3. `GITHUB_TOKEN` can create and update only `refresh/**` branches and open a |
| 26 | + Buildathon pull request through bounded safe outputs. |
| 27 | +4. Explicit workflow dispatch produces every required check for the current head |
| 28 | + SHA after token-created PRs and subsequent automated updates. |
| 29 | +5. Copilot Code Review can be explicitly requested for the current head SHA. |
| 30 | +6. The main ruleset requires pull requests, CODEOWNER approval, conversation |
| 31 | + resolution, and the known required checks. |
| 32 | +7. The workflow cannot push to `main`, merge, force-push, delete branches, or |
| 33 | + write to another repository. |
| 34 | +8. `REFRESH_APPROVER_LOGIN` is set to the exact approved GitHub login. |
28 | 35 |
|
29 | | -Record the approver login, UTC timestamp, and a settings or approval reference |
30 | | -in `capabilities.json`. Do not replace an unknown capability with an assumption. |
| 36 | +Record the approver login, UTC timestamp, and trial evidence in |
| 37 | +`capabilities.json`. Do not replace an unknown capability with an assumption. |
31 | 38 |
|
32 | 39 | ## Audit findings |
33 | 40 |
|
34 | 41 | The SSO-authorized keyring token completed a read-only audit on July 23, 2026. |
35 | 42 |
|
36 | | -- Buildathon: Julia has repository administration; Copilot Code Review is |
37 | | - active; refresh labels and `REFRESH_APPROVER_LOGIN` are configured. Active |
38 | | - branch protection requires one approval and conversation resolution. Disabled |
39 | | - ruleset `19622585` is prepared to require the Phase 0 checks, one CODEOWNER |
40 | | - approval, approval after the latest push, conversation resolution, and to |
41 | | - block deletion and force-push after this Phase 0 change lands. A separate |
42 | | - update-restriction template permits only Julia, through a pull request, to |
43 | | - update `main`; the App is not a bypass actor. GitHub returned HTTP 404 when |
44 | | - that user-bypass ruleset was created through the REST API, so an administrator |
45 | | - must confirm or create it in repository settings. The App, AI-credit budget |
46 | | - confirmation, ruleset activation, and explicit App permission for |
47 | | - `refresh/**` branches are missing. |
48 | | -- Serverless Chat: Julia has write but not administration; two active rulesets |
49 | | - and Copilot Code Review are visible. An administrator must confirm the |
50 | | - rulesets, budget, App installation, and `refresh/**` branch policy. |
51 | | -- MCP Agent: Julia has write but not administration; Copilot Code Review is |
52 | | - visible, but no qualifying ruleset or classic main-branch protection was |
53 | | - visible. An administrator must establish the complete governance boundary. |
54 | | - |
55 | | -Julia cannot access the Azure-Samples organization settings needed to create an |
56 | | -organization-owned App. An organization owner must complete that action. |
| 43 | +- Buildathon: Copilot Code Review is active; refresh labels and |
| 44 | + `REFRESH_APPROVER_LOGIN` are configured. Active branch protection requires one |
| 45 | + approval and conversation resolution. Disabled ruleset `19622585` is prepared |
| 46 | + to require the Phase 0 checks, CODEOWNER approval, approval after the latest |
| 47 | + push, conversation resolution, and to block deletion and force-push. A |
| 48 | + separate human-only update template still requires administrator confirmation |
| 49 | + because the user-bypass REST request returned HTTP 404. |
| 50 | +- Serverless Chat and MCP Agent remain external evidence sources only. No |
| 51 | + workflow in Buildathon may write to them. |
57 | 52 |
|
58 | | -## Organization-owner handoff |
| 53 | +The unavailable organization-owned GitHub App is no longer a blocker or planned |
| 54 | +dependency. Built-in-token event suppression and repository write permissions |
| 55 | +are the new technical gates. |
59 | 56 |
|
60 | | -Ask an Azure-Samples owner to: |
| 57 | +## Activation sequence |
61 | 58 |
|
62 | | -1. Create an organization-owned **Azure Samples Agentic Refresh** GitHub App |
63 | | - with the repository permissions in `github-app.json` and no organization |
64 | | - permissions. |
65 | | -2. Disable webhooks and restrict installation to the Azure-Samples account. |
66 | | -3. Install it only on `Azure-Samples/JavaScript-AI-Buildathon`. |
67 | | -4. Provide the App ID through repository variable |
68 | | - `AGENTIC_REFRESH_APP_ID`. |
69 | | -5. Store the generated PEM private key directly in repository secret |
70 | | - `AGENTIC_REFRESH_APP_PRIVATE_KEY`; never send it through chat or an issue. |
71 | | -6. Confirm that Copilot Code Review has an available AI-credit budget. |
72 | | -7. Confirm that the App may create `refresh/**` branches but cannot bypass |
73 | | - ruleset `19622585`, write to `main`, or merge. |
74 | | -8. After the Phase 0 pull request merges and both validation checks have run |
75 | | - successfully, activate ruleset `19622585` and the human-only merge ruleset. |
76 | | -9. Confirm that App installation tokens are available only to the pinned |
77 | | - deterministic safe-output action and never to an agent or generic shell step. |
| 59 | +1. Merge the Phase 0 pull request. |
| 60 | +2. Confirm or create the human-only merge ruleset in repository settings. |
| 61 | +3. Activate ruleset `19622585` and the human-only merge ruleset. |
| 62 | +4. Run read-only discovery trials. |
| 63 | +5. Run the bounded `GITHUB_TOKEN` issue, branch, PR, explicit-dispatch, and |
| 64 | + Copilot-review spike. |
| 65 | +6. Enable issue-only reporting if issue writes pass. |
| 66 | +7. Enable Buildathon implementation only after every write/review gate passes. |
78 | 67 |
|
79 | | -The external repositories stay report-only. Their App installation requires a |
80 | | -separate capability approval after a repository administrator confirms each |
81 | | -ruleset, CI, CODEOWNER, branch, and budget requirement. |
| 68 | +External CodeTour findings remain manual handoffs regardless of Buildathon |
| 69 | +capability results. |
0 commit comments