Replace Hetzner-platform PATs (GH_ORG_VAR_TOKEN + RUNNER_PAT) with a single GitHub App

[fabapp2]

Why

The Hetzner runner platform currently depends on two fine-grained PATs, both of which have repeatedly broken runs over the last few days for the same class of reasons (wrong scope picked at creation, expired, not re-approved after edit, etc.):

Secret	Used by	Required permission	Recent failures
GH_ORG_VAR_TOKEN	hetzner-snapshot-bake.yml (writes vars.HETZNER_RUNNER_SNAPSHOT_ID after each bake)	Organization → Variables: Read and write	run 27287833311, run 27296295604
RUNNER_PAT	hetzner-runner.yml (registers each Hetzner VM as a self-hosted runner via Cyclenerd's action) — used by every caller repo	Repository → Administration: Read and write (on every repo that uses Hetzner runners)	promptlm-app run 27355655316

The shared failure modes (all reproduced in the linked runs):

• Annual expiration. Fine-grained PATs max out at 1 year. When either token expires, the platform silently breaks.
• Tied to a human. If the token's owner leaves the org or loses the required org role, the platform breaks.
• Easy to misconfigure at creation. The PAT creation flow defaults every permission to Read-only; picking "Read and write" requires an extra click per permission. Easy to miss for Administration AND for Variables.
• Org-admin re-approval required after edits. Changing a fine-grained PAT's permissions invalidates the prior approval; the platform can break days later when nobody remembers.
• Multiplied across repos. RUNNER_PAT needs Administration: RW on every caller repo. As more repos opt in, the scope of the PAT grows, and the audit blast radius grows with it.

A single GitHub App, installed on the org, eliminates all four failure modes for both tokens at once.

What to do

1. Create a new GitHub App under the promptLM org. Name suggestion: Hetzner Runner Platform.
   • Organization permissions:
     • Variables: Read and write
     • Administration: Read and write (for the rare case the platform itself needs to do org-level repo admin — optional, can be dropped if not needed)
     • Self-hosted runners: Read and write (if needed by Cyclenerd's action at the org level)
   • Repository permissions:
     • Administration: Read and write — required to register self-hosted runners
     • Metadata: Read (default)
   • Webhook: disabled.
   • Where can this GitHub App be installed? Only on this account (promptLM org).
2. Install the App on the promptLM org. Grant access to all repositories (so every caller of hetzner-runner.yml is covered without per-repo install steps), or to a curated repo list if you'd prefer least-privilege.
3. Store credentials as org-level Actions secrets/variables:
   • BAKE_APP_ID → Variable, not Secret. App IDs aren't sensitive. Visibility: all repos.
   • BAKE_APP_PRIVATE_KEY → Secret. The App's PEM. Visibility: all repos (so every caller of hetzner-runner.yml can mint a token).
4. Update .github/workflows/hetzner-snapshot-bake.yml. Replace the GH_ORG_VAR_TOKEN-using step:

   - name: Get app token
     id: app
     uses: actions/create-github-app-token@v2  # pin to commit SHA per oss-checks pattern
     with:
       app-id: ${{ vars.BAKE_APP_ID }}
       private-key: ${{ secrets.BAKE_APP_PRIVATE_KEY }}
       owner: promptLM
   
   - name: Update org variable HETZNER_RUNNER_SNAPSHOT_ID
     env:
       GH_TOKEN: ${{ steps.app.outputs.token }}
       VAR_NAME: HETZNER_RUNNER_SNAPSHOT_ID
     run: |
       # ... existing probe + branch logic stays as-is

5. Update .github/workflows/hetzner-runner.yml. Replace the RUNNER_PAT-using steps in both provision and teardown:

   - name: Get app token
     id: app
     uses: actions/create-github-app-token@v2  # pin to commit SHA
     with:
       app-id: ${{ vars.BAKE_APP_ID }}
       private-key: ${{ secrets.BAKE_APP_PRIVATE_KEY }}
       owner: ${{ github.repository_owner }}
       repositories: ${{ github.event.repository.name }}
   
   - id: runner
     uses: Cyclenerd/hcloud-github-runner@...
     with:
       mode: create
       github_token: ${{ steps.app.outputs.token }}
       hcloud_token: ${{ secrets.HCLOUD_TOKEN }}
       # ...

   secrets: block at the workflow_call level can drop RUNNER_PAT (becomes optional during transition, removed after).
6. Caller repos that inlined the 3-job pattern (e.g. promptlm-app PR #383) also swap RUNNER_PAT for the App-token step. Document this in docs/hetzner-runners.md.
7. Once the new flow has run green at least once from .github's bake AND from one caller repo's runner, retire the PATs:
   • Delete org secrets GH_ORG_VAR_TOKEN and RUNNER_PAT.
   • Revoke the PATs at https://github.com/settings/tokens.
8. Update the comment block at the top of both workflows (currently documents the PATs) and the Org-level secrets table in docs/hetzner-runners.md to describe the App-token setup instead.

Acceptance criteria

• GitHub App created and installed on promptLM with the permissions listed above and no extras.
• hetzner-snapshot-bake.yml uses actions/create-github-app-token (pinned to a commit SHA per the existing oss-checks pinning pattern) instead of GH_ORG_VAR_TOKEN.
• hetzner-runner.yml uses actions/create-github-app-token (same pin pattern) instead of RUNNER_PAT, scoped to the calling repo only.
• One successful manual workflow_dispatch run of the bake on main proves the variable is updated end-to-end via the App.
• One successful end-to-end run of hetzner-runner.yml from a caller repo (e.g. promptlm-app acceptance) proves runner registration and teardown via the App.
• Old GH_ORG_VAR_TOKEN and RUNNER_PAT org secrets removed; PATs revoked.
• Workflow header comments and docs/hetzner-runners.md updated to reflect the new auth model.
• Caller-repo migration documented (one-line note in docs/hetzner-runners.md Quick start section).

Out of scope

• Migrating non-Hetzner-platform token usage. If RUNNER_PAT or GH_ORG_VAR_TOKEN is referenced from any workflow outside the Hetzner platform, those callers will need separate migration PRs — but as of writing, no such callers exist.

Refs

• Failure modes that motivated this:
  • GH_ORG_VAR_TOKEN: #33, runs 27287833311, 27296295604
  • RUNNER_PAT: promptlm-app PR #383, run 27355655316
• actions/create-github-app-token: https://github.com/actions/create-github-app-token
• Platform docs: https://github.com/promptLM/.github/blob/main/docs/hetzner-runners.md