Skip to content

[Labelling Health] Labelling Health Report — 2026-05-25 #442

@mnkiefer

Description

@mnkiefer

Summary

Overall status: Mixed / Regressing

The auto-labelling pipeline (Label Discussions) is running reliably with 7 consecutive successful runs over each of the last two 7-day windows. However, the correction infrastructure is effectively broken: the Labelling Correction Collector has not run at all in the last 30 days, Labelling Correction Feedback has skipped on every one of its 28 runs in the same period, and 333 correction signals remain open — all untouched for more than 46 days. No daily summary issues were available, so reviewed and changed counts cannot be reported.


Key Metrics

Metric Value
Discussions reviewed — last 7 days Unknown (no summary issues parsed)
Label changes applied — last 7 days Unknown (no summary issues parsed)
Change rate — last 7 days Unknown
Correction-Collector runs — last 7 days 0 (workflow absent from all 30-day data)
Labelling Correction Feedback runs — last 7 days 6 (all skipped)
Label Discussions runs — last 7 days 7 / 7 successful
Open correction signals 333
Correction signals created — last 7 days 0
Correction signals created — last 30 days 0
Oldest open correction signal 46 days (issue #47, created 2026-04-08)

⚠️ auto_labelling_summaries was empty — no daily summary issues were discovered or parsed. Reviewed/changed counts are not available for either 7-day window. The Label Discussions workflow is confirmed running and succeeding, but its output volume is opaque.


Correction Pressure

All 333 open signals were created in a single burst during weeks 14–15 of 2026 (April 8–18). No new signals have been ingested since, consistent with the Labelling Correction Collector not running.

The top labels appearing across open signals are:

Label Open signals
Copilot 56
Copilot in GitHub 42
GitHub Education 38
bug 37
Other Features and Feedback 20
question 14
Apps API and Webhooks 10
Profile 10
Mobile 10
Product Feedback 9

Copilot-related labels (Copilot + Copilot in GitHub) account for 98 signals (≈29%), making them the highest-pressure cluster. The bug type-label adds another 37 signals that may indicate systematic misclassification of support requests as bugs.

Signals are spread across 116 unique discussions, with an average of 2.9 signals per discussion (max 5). The relatively even spread suggests that the pressure is not driven by a handful of hot discussions, but reflects broad category ambiguity across the label taxonomy.

All open signals belong to Batch 02, concentrated in 4 open parent intake issues (#45, #46, #49, #50), all created on 2026-04-08.

Open correction signal label breakdown (full)
Label Count
Copilot 56
Copilot in GitHub 42
GitHub Education 38
bug 37
Other Features and Feedback 20
question 14
Apps API and Webhooks 10
Profile 10
Mobile 10
Product Feedback 9
Code Search and Navigation 8
Discussions 8
Verification Help & Guidance 7
Actions 7
Benefit Activation/Waiting 6
API 6
Repositories 5
ARC (Actions Runner Controller) 5
Best Practices 4
Pull Requests 4
Evidence and Image Capture 3
Issues 2
VS Code 2
Codespaces 2
Packages 2
GitHub Learn 2
iOS 2
Lists 2
GitHub Education Benefits 2
(miscellaneous singles) 7

Open Instruction Debt

The correction backlog is stagnant and large. Of 335 total signals ever created:

  • 333 are open — 99.4% unresolved
  • 2 were closed (both on 2026-04-03, predating the current batch)
  • 0 signals were created in the last 30 days — new signal ingestion has stopped entirely

The 4 open Batch 02 parent intake issues (#45, #46, #49, #50) have remained open since April 8, with no visible triage activity. Batch 01 (#26) is the only closed parent, and its signals were resolved only partially.

At the current resolution rate of ~0 signals/day, the backlog is not actionable without intervention. The root cause is the Labelling Correction Collector workflow being completely absent — it has produced zero runs in the observable 30-day window.

Open correction signal parent intake issues
Issue Title State Created
#50 [Labelling Correction Intake] Batch 02 open 2026-04-08
#49 [Labelling Correction Intake] Batch 02 open 2026-04-08
#46 [Labelling Correction Intake] Batch 02 open 2026-04-08
#45 [Labelling Correction Intake] Batch 02 open 2026-04-08
#26 [Labelling Correction Intake] Batch 01 closed 2026-03-31

Recommendations

  1. Restore the Labelling Correction Collector workflow. This is the most urgent action. The workflow has not run in at least 30 days, which means no new trusted correction signals are being captured. Investigate whether the workflow was disabled, has a broken trigger, or is failing silently outside the observable window.

  2. Investigate why Labelling Correction Feedback always skips. All 28 runs over 30 days returned skipped. If the skip condition depends on the Collector having recent output, fixing the Collector may resolve this automatically. If not, review the skip condition logic independently.

  3. Triage the Copilot and bug label clusters in .github/instructions/community-discussion-labeling.md. With 98 Copilot-related open signals and 37 bug-type signals, the labelling instructions likely need sharper disambiguation rules — e.g., explicit guidance on when Copilot vs. Copilot in GitHub applies, and when a discussion should be classified as bug vs. a product feedback or support label.

  4. Restore daily summary issue creation or surface output from Label Discussions. The auto-labelling pipeline is running daily but its reviewed/changed counts are invisible — no summary issues were found. Without this signal, it is impossible to track labelling volume or change rate. Verify that daily summary issues are being created and are discoverable by this health report workflow.


References

Recent Label Discussions workflow runs (last 7 days)
Run Date Status
§81 2026-05-24 ✅ success
§80 2026-05-23 ✅ success
§79 2026-05-22 ✅ success
§78 2026-05-21 ✅ success
§77 2026-05-20 ✅ success
§76 2026-05-19 ✅ success
§75 2026-05-18 ✅ success

§81 · §80 · §79

Generated by Labelling Health Report · ● 365.5K ·

  • expires on Jun 24, 2026, 3:39 AM UTC

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions