Skip to content

doc(skills): monthly-newsletter — collect data via git, not the REST API#28

Merged
fredbi merged 2 commits into
masterfrom
claude/skill-git-data-collection
Jul 6, 2026
Merged

doc(skills): monthly-newsletter — collect data via git, not the REST API#28
fredbi merged 2 commits into
masterfrom
claude/skill-git-data-collection

Conversation

@fredbi

@fredbi fredbi commented Jul 6, 2026

Copy link
Copy Markdown
Member

Updates the monthly-newsletter skill's data-collection method to match how the routine actually has to run in the scheduled cloud sandbox.

Why

The routine's original method assumed the GitHub REST API (gh api, gh repo list). In the scheduled sandbox the egress proxy binds the session to a single repository, so every api.github.com, github.com, and codeload.github.com request returns 403 — including unauthenticated public reads and the org-listing/GraphQL endpoints gh repo list relies on. A token doesn't change this; the block is at the network layer. (This is what caused the June run to first fail before being retried via the approach below.)

What the proxy does allow — and what the skill now uses

  • git smart-HTTP (git clone, git ls-remote, git fetch) against any public repo and the fork
  • raw.githubusercontent.com single-file reads

Changes to the skill

  • Enumerate repos from the union of go-openapi/* module paths in a few go.mod files (via raw) plus a known set of non-module infra/tooling repos, validated with git ls-remote.
  • Collect commits and releases from shallow --shallow-since --no-checkout --filter=blob:none clones and their tags — no Releases/Search API.
  • Contributor handles recovered from NNN+login@users.noreply.github.com emails and Signed-off-by:/Co-authored-by: trailers (git has no .author.login); omit anyone whose handle can't be derived rather than guessing.
  • Publishing note: documents that the gh api …/contents/… fork path is blocked and that plain git push yields an unsigned, proxy-attributed commit; the working path is the in-scope MCP server creating a verified, maintainer-authored commit on a claude/monthly-<YM> branch + same-repo PR.

Verified end-to-end by the June 2026 report run (#27).

🤖 Generated with Claude Code


Generated by Claude Code

fredbi and others added 2 commits July 6, 2026 22:25
… API

The scheduled cloud routine's egress is bound to a single repo, so every
api.github.com / github.com / codeload request returns 403 — including
unauthenticated public reads and the org-listing endpoints gh repo list uses.
Rewrite the data-collection method to use the git smart-HTTP protocol (clone,
ls-remote) and raw.githubusercontent.com, which the proxy does allow:

- enumerate repos from go.mod unions + known infra, validated with git ls-remote
- collect commits/releases from shallow --shallow-since clones and tags
- recover contributor handles from noreply emails and commit trailers (no .author.login)
- document the MCP Contents-API publish path (no git push: unsigned/proxy-attributed)

Signed-off-by: Frédéric BIDON <fredbi@yahoo.com>
Co-Authored-By: Claude <noreply@anthropic.com>
Post-mortem fix for the June run, which led with a "security" theme (a few
commits) and filtered away the month's actual centre of gravity: 367 codescan
commits (61% of the month) sweeping the go-swagger issue backlog into tests
after the codescan extraction.

- New "Finding the month's story" section: rank repos by load-bearing commit
  share; the top repo(s) MUST be characterized; a repo >~15-20% of commits may
  never be dismissed as noise even when chore:/test:-prefixed; investigate
  dominant chore:/test: clusters (their (#NNN) refs reveal backlog sweeps)
  instead of filtering them; connect the causal chain and get the timing right.
- Data collection: exclude non-load-bearing bot authors (dependabot,
  bot-go-openapi/go-openapi-bot, github-actions) and automated
  contributors/prepare-release commits from the counts that drive the narrative.

Signed-off-by: Frédéric BIDON <fredbi@yahoo.com>
Co-Authored-By: Claude <noreply@anthropic.com>
@fredbi fredbi merged commit c6db84c into master Jul 6, 2026
3 of 4 checks passed
@fredbi fredbi deleted the claude/skill-git-data-collection branch July 6, 2026 20:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant