Skip to content

[Labelling Health] Labelling Health Report — 2026-05-24 #441

@mnkiefer

Description

@mnkiefer

Summary

Overall status: Mixed (labelling pipeline operational; correction loop stalled)

The Label Discussions workflow is running successfully every day, but no daily summary issues were parsed, so reviewed/changed counts cannot be directly measured. The correction backlog has not changed in over 7 weeks: 333 of 335 signals remain open, no new signals have been created since 2026-04-18, the Labelling Correction Collector has not run in the last 30 days, and the Labelling Correction Feedback workflow has been skipped on every run.


Key Metrics

Metric Last 7 days Prev 7 days
Discussions reviewed Unknown — no parsed daily summaries Unknown
Label changes applied Unknown — no parsed daily summaries Unknown
Change rate (changed / reviewed) N/A N/A
Label Discussions runs 7 / 7 (all ✅ success) 7 / 7 (all ✅ success)
Labelling Correction Collector runs 0 0
Labelling Correction Feedback runs 6 (all ⏭ skipped) 7 (all ⏭ skipped)
Open correction signals 333 333
Correction signals created (last 7d / 30d) 0 / 0

Note on reviewed/changed counts: No daily summary issues were returned in the data. The Label Discussions workflow ran successfully on all 7 days last week, but without parsed summaries the actual discussion throughput cannot be confirmed. Treat pipeline health as inferred from run success only.


Correction Pressure

The open correction backlog contains 333 signals covering 9 discussion categories. Three categories account for roughly 70% of open signals:

Category Open signals Dominant label cluster
Copilot Conversations 110 Copilot (54), Copilot in GitHub (42), bug (9)
GitHub Education 71 GitHub Education (38), bug (8), Verification Help & Guidance (7)
Other Feature Feedback, Questions, & Ideas 52 Other Features and Feedback (20), Profile (10), Code Search and Navigation (8)
Apps, API and Webhooks 21 Apps API and Webhooks (10), API (6)
Mobile 14 Mobile (10)

By event type: 197 signals are labeled corrections (a label was added) vs 136 unlabeled (a label was removed). This means the dominant pressure is on labels being added that the auto-labeller did not assign — particularly Copilot, Copilot in GitHub, and GitHub Education.

All correction pressure is concentrated into 4 open parent intake issues (Batch 02, created 2026-04-08). The signals are not spread organically — they originate from a single batch run and have been sitting unaddressed since.

Top category + label clusters (open signals)
Count Category Latest label
54 Copilot Conversations Copilot
42 Copilot Conversations Copilot in GitHub
38 GitHub Education GitHub Education
20 Other Feature Feedback, Questions, & Ideas Other Features and Feedback
10 Apps, API and Webhooks Apps API and Webhooks
10 Other Feature Feedback, Questions, & Ideas Profile
9 Copilot Conversations bug
9 Mobile Mobile
8 Other Feature Feedback, Questions, & Ideas Code Search and Navigation
8 GitHub Education bug
8 Discussions Discussions
7 GitHub Education Verification Help & Guidance
6 GitHub Education Benefit Activation/Waiting
6 Apps, API and Webhooks API
Highest-pressure individual open signals (by correction count)
Signal Discussion Correction count Category Latest label
#114 #147 19 Other Feature Feedback, Questions, & Ideas Code Search and Navigation
#69 #185 17 GitHub Education question
#63 #195 17 Other Feature Feedback, Questions, & Ideas Profile
#55 #189 17 GitHub Education GitHub Education Benefits
#51 #186 17 Other Feature Feedback, Questions, & Ideas Code Search and Navigation

Open Instruction Debt

The correction backlog is not shrinking — it is stale.

  • 333 open signals remain from a batch created between 2026-03-31 and 2026-04-18.
  • The last signal closure was on 2026-04-03 (2 signals closed), ≥51 days ago.
  • The oldest open signal (#47) is 45 days old.
  • 4 open parent intake issues (Batch 02, all created 2026-04-08) group the backlog. Batch 01 is the only closed parent.
  • The Labelling Correction Collector has run 0 times in the last 30 days — no new signals are being generated.
  • The Labelling Correction Feedback workflow has been skipped on every run for at least 14 days, meaning the feedback loop that should act on signals to improve .github/instructions/community-discussion-labeling.md is completely inactive.

The backlog appears structurally stalled, not just slow. Without the Collector running, no new signals enter the system, and without the Feedback workflow acting, existing signals are not closing.

Open parent intake issues
Issue Title Created
#50 [Labelling Correction Intake] Batch 02 2026-04-08
#49 [Labelling Correction Intake] Batch 02 2026-04-08
#46 [Labelling Correction Intake] Batch 02 2026-04-08
#45 [Labelling Correction Intake] Batch 02 2026-04-08

Recommendations

  1. Investigate why Labelling Correction Collector has not run in 30+ days. This is the most critical gap: without the Collector, no new correction signals enter the system and the feedback loop cannot close. Check the workflow's trigger conditions, schedule, and any disabling configuration.

  2. Investigate why Labelling Correction Feedback is being skipped on every run. All 13 runs in the last two weeks returned skipped. Determine whether a guard condition is preventing feedback processing and whether the 333 open signals are within its scope.

  3. Prioritize .github/instructions/community-discussion-labeling.md updates for the top two category clusters: Copilot Conversations and GitHub Education. These account for 181 of 333 open signals (~54%). The dominant patterns — Copilot, Copilot in GitHub, and GitHub Education labels being added — suggest the instructions likely need clearer rules for distinguishing feature-specific labels from general platform labels in these categories.

  4. Triage and close or batch-process the 4 open Batch 02 parent intake issues. With 333 signals unresolved for 45+ days, a manual review of the highest-pressure signals (correction count ≥ 10) would identify the most impactful instruction gaps without waiting for the automated feedback loop to resume.


References

Recent Label Discussions workflow runs (last 7 days)
Date Conclusion Run
2026-05-23 ✅ success §26343278873
2026-05-22 ✅ success §26311705585
2026-05-21 ✅ success §26253217435
2026-05-20 ✅ success §26190383401
2026-05-19 ✅ success §26125292964
2026-05-18 ✅ success §26060063772
2026-05-17 ✅ success §26002404998
Recent Labelling Correction Feedback runs (last 7 days)
Date Conclusion Run
2026-05-23 ⏭ skipped §26322472294
2026-05-22 ⏭ skipped §26267043354
2026-05-21 ⏭ skipped §26203969785

§26343278873 · §26322472294 · §26311705585


Report generated: 2026-05-24T03:34Z · Data window: 2026-04-24 – 2026-05-24 · Workflow run: 26350885997

Generated by Labelling Health Report · ● 334.3K ·

  • expires on Jun 23, 2026, 3:38 AM UTC

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions