Skip to content

[Labelling Health] Labelling Health Report — 2026-05-27 #444

@mnkiefer

Description

@mnkiefer

Summary

Status: Mixed — pipeline is active but correction backlog is stagnant and feedback loop is broken.

The Label Discussions workflow ran successfully every day last week and over the last 30 days. However, no auto-labelling daily summary issues were parsed, so reviewed/changed counts are unavailable. More critically, the Labelling Correction Feedback workflow has been skipping every run for the entire 30-day window, meaning no correction signals are being resolved. The backlog of 333 open correction signals has not moved in 7+ weeks.


Key Metrics

Metric Value
Discussions reviewed — last 7 days ⚠️ Unknown (no parsed daily summary issues)
Label changes applied — last 7 days ⚠️ Unknown (no parsed daily summary issues)
Change rate — last 7 days ⚠️ Cannot compute
Label Discussions runs — last 7 days ✅ 7 / 7 succeeded
Labelling Correction Collector runs — last 7 days ⚠️ 0 (not found in data)
Labelling Correction Feedback runs — last 7 days 🔴 7 / 7 skipped
Open correction signals 🔴 333
New correction signals — last 7 days 0
New correction signals — last 30 days 0
Signals closed — last 7 days 0

Note: No daily summary issues were available in the data (auto_labelling_summaries is empty). Reviewed and changed counts cannot be estimated from this report cycle. This is a data gap that should be investigated.


Correction Pressure

The 333 open correction signals are concentrated in three categories:

Category Open Signals
Copilot Conversations 110 (33%)
GitHub Education 71 (21%)
Other Feature Feedback, Questions & Ideas 52 (16%)
Apps, API and Webhooks 21 (6%)
Mobile 14 (4%)
All others 65 (19%)

The highest-pressure label clusters by category+label pairing:

  • Copilot Conversations → Copilot: 54 signals
  • Copilot Conversations → Copilot in GitHub: 42 signals
  • GitHub Education → GitHub Education: 38 signals
  • Other Feature Feedback → Other Features and Feedback: 20 signals

The unlabeled event type accounts for 136 of 333 open signals (41%), suggesting the auto-labeller is frequently removing labels that trusted correctors want kept, or applying labels that are subsequently removed.

The five highest-correction individual signals (correction_count ≥ 17) all fall in Other Feature Feedback and GitHub Education categories, pointing to chronic disagreements between the auto-labeller and trusted correctors in those areas.

All 333 signals are grouped under 4 open parent intake issues (all "Batch 02", created 2026-04-08). Pressure is not spread across many batches — it is entirely concentrated in this single unresolved batch.

Open correction signal detail by category
Category Count
Copilot Conversations 110
GitHub Education 71
Other Feature Feedback, Questions, & Ideas 52
Apps, API and Webhooks 21
Mobile 14
Discussions 13
Actions 10
Repositories 7
Pull Requests 6
Discover: GitHub Best Practices 4
Projects and Issues 3
Codespaces 3
Packages 3
GitHub Learn 3
Accessibility 3
Code Security 3
A Welcome to GitHub 2
npm 2
New to GitHub 2
Enterprise 1

Top 5 signals by correction count:

Signal Category Current Label Corrections
#114 Other Feature Feedback... Code Search and Navigation 19
#69 GitHub Education question 17
#63 Other Feature Feedback... Profile 17
#55 GitHub Education GitHub Education Benefits 17
#51 Other Feature Feedback... Code Search and Navigation 17

Open Instruction Debt

The correction backlog is growing in age, not shrinking. No signals were created or closed in the last 30 days, and the Labelling Correction Feedback workflow has been skipping every run for the full 30-day observation window. This means no automated resolution is occurring.

  • Oldest open signal: #47 — 48 days old
  • Open parent intake issues: 4 (all "Batch 02", all 48 days old)
  • Closed parent intake issues: 1 ("Batch 01")
  • Signal closure rate: 2 closed out of 335 total (0.6%)

The backlog appears stale and unactionable in its current state. The feedback loop that would normally resolve signals is entirely inactive. Without intervention, the backlog will continue aging with no resolution.

Parent intake issue status
Issue Title State Age
#50 [Labelling Correction Intake] Batch 02 🔴 open 48 days
#49 [Labelling Correction Intake] Batch 02 🔴 open 48 days
#46 [Labelling Correction Intake] Batch 02 🔴 open 48 days
#45 [Labelling Correction Intake] Batch 02 🔴 open 48 days
#26 [Labelling Correction Intake] Batch 01 ✅ closed 56 days

Recommendations

  1. Diagnose why Labelling Correction Feedback always skips. This workflow has returned skipped on every one of its 28 runs over the last 30 days. Until this is fixed, no correction signals can be resolved automatically. Inspect the workflow's trigger conditions and any gate checks that may be causing the skip.

  2. Investigate the missing Labelling Correction Collector runs. No collector runs appear in the 30-day data. If this workflow is not running, no new signals will be generated going forward. Verify the workflow schedule and trigger configuration.

  3. Triage Copilot Conversations labelling rules in .github/instructions/community-discussion-labeling.md. With 110 open signals — 54 resolving to Copilot and 42 to Copilot in GitHub — there is likely ambiguity in how the auto-labeller distinguishes these two labels. Clarifying the instruction boundary between them could reduce future correction pressure.

  4. Manually review and close the 4 open Batch 02 parent intake issues (#45, #46, #49, #50) after triaging their sub-issues. A 48-day-old batch with 333 unresolved sub-issues is a signal that the intake-to-resolution pipeline needs an explicit triage cadence, not just automated tooling.


References

Recent workflow runs

Label Discussions (last 7 days — all succeeded)

Run Date Conclusion
§83 2026-05-26 ✅ success
§82 2026-05-25 ✅ success
§81 2026-05-24 ✅ success
§80 2026-05-23 ✅ success
§79 2026-05-22 ✅ success
§78 2026-05-21 ✅ success
§77 2026-05-20 ✅ success

Labelling Correction Feedback (last 7 days — all skipped)

Runs #415#421 (2026-05-20 through 2026-05-26): all skipped.

Labelling Correction Collector: No runs found in the 30-day window.

§83 · §82 · §81

Generated by Labelling Health Report · ● 413.1K ·

  • expires on Jun 26, 2026, 3:38 AM UTC

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions