[Labelling Health] Labelling Health Report — 2026-05-23

[mnkiefer]

Summary

Overall status: Mixed — auto-labelling pipeline is operational but the correction pipeline is fully stalled.

The Label Discussions workflow ran successfully every day last week (7/7 runs), indicating active discussion processing. However, the Labelling Correction Collector has zero runs in the last 30 days, no new correction signals have been created since 2026-04-18, and 333 of 335 correction signals remain open. The backlog is not shrinking.

---

Key Metrics

Metric	Last 7 Days	Prev 7 Days
Discussions reviewed	N/A — no parsed summary issues	N/A
Label changes applied	N/A — no parsed summary issues	N/A
Change rate (changed / reviewed)	Not calculable	Not calculable
Label Discussions runs (success)	7 / 7	7
Labelling Correction Collector runs	0	0
Labelling Correction Feedback runs (skipped)	6 / 6	7
Open correction signals	333	—
New correction signals last 7 days	0	0
New correction signals last 30 days	0	0

⚠️ No daily summary issues were available to parse. Reviewed and changed counts are unavailable; auto-labelling throughput cannot be confirmed beyond "runs succeeded."

⚠️ The Labelling Correction Collector has not run in the last 30 days. This is the primary signal-ingestion workflow; its absence explains why no new correction signals have appeared.

---

Correction Pressure

All 333 open correction signals were created between 2026-03-31 and 2026-04-18 (Batch 02). No new signals have been ingested since. The backlog represents 851 total correction events across 333 discussions.

The highest-pressure categories by open-signal count:

Category	Open Signals
Copilot Conversations	110
GitHub Education	71
Other Feature Feedback, Questions & Ideas	52
Apps, API and Webhooks	21
Mobile	14
Discussions	13
Actions	10

The most-corrected labels (by latest_label on open signals):

Label	Signals
Copilot	56
Copilot in GitHub	42
GitHub Education	38
bug	37
Other Features and Feedback	20

Event type split: 197 labeled vs 136 unlabeled events — meaning the system is adding wrong labels slightly more often than it is failing to add correct ones.

100 signals have correction_count ≥ 2, indicating repeated human intervention on the same discussions. The top 5 most-corrected signals:

Signal	Discussion	correction_count	Category
#114	#147	19	Other Feature Feedback
#69	#185	17	GitHub Education
#63	#195	17	Other Feature Feedback
#55	#189	17	GitHub Education
#51	#186	17	Other Feature Feedback

All 333 open signals are concentrated in 4 open parent intake issues (Batch 02: #45, #46, #49, #50). No signals are spread outside these batches.

Open correction signal breakdown (top 15 by correction_count)

Issue	Discussion	correction_count	Category	Latest Label
#114	#147	19	Other Feature Feedback, Questions & Ideas	Code Search and Navigation
#69	#185	17	GitHub Education	question
#63	#195	17	Other Feature Feedback, Questions & Ideas	Profile
#55	#189	17	GitHub Education	GitHub Education Benefits
#51	#186	17	Other Feature Feedback, Questions & Ideas	Code Search and Navigation
#109	#151	16	Copilot Conversations	Copilot
#75	#175	15	GitHub Education	GitHub Education
#118	#148	14	Apps, API and Webhooks	Apps API and Webhooks
#77	#170	14	Copilot Conversations	Copilot
#73	#184	14	Other Feature Feedback, Questions & Ideas	Copilot
#108	#152	12	Copilot Conversations	Copilot
#74	#176	12	Copilot Conversations	Copilot
#57	#187	11	GitHub Education	GitHub Education
#79	#167	11	Other Feature Feedback, Questions & Ideas	Profile
#83	#163	11	Copilot Conversations	Copilot

---

Open Instruction Debt

The correction backlog is steady (not shrinking). With the Correction Collector not running, the queue is frozen rather than growing or resolving.

• Oldest open correction signal: #47, created 2026-04-08 — 44 days old
• Open parent intake issues: 4 (all Batch 02; Batch 01 is the only closed one)
• Closed signals: 2 of 335 (< 1%)

The backlog does not look actionable in its current state: no workflow is collecting new corrections and no process is closing resolved signals. The Correction Feedback runs are all skipped, which is consistent with no new signals arriving as triggers. Until the Collector resumes, the backlog will remain frozen and diverge further from the live labelling state.

---

Recommendations

1. Investigate and restore the Labelling Correction Collector workflow. It has not run in at least 30 days. This is the root cause of the frozen correction pipeline. Check for disabled schedules, failed permissions, or quota exhaustion.

2. Review .github/instructions/community-discussion-labeling.md for the three highest-pressure clusters: Copilot/Copilot-in-GitHub label confusion, GitHub Education signal patterns, and the bug vs. question vs. Other Features and Feedback ambiguity. The 100 signals with correction_count ≥ 2 represent confirmed instruction gaps.

3. Triage the 4 open Batch 02 parent intake issues (#45, #46, #49, #50). Determine whether they represent distinct rule-update opportunities or can be consolidated. The signals under them predate the current labelling instruction version and may partly be stale.

4. Enable daily summary issue publishing for Label Discussions runs (or verify the parsing pipeline). Without readable summary issues, throughput metrics (reviewed count, change rate) are completely invisible, making health reporting a lagging indicator.

---

References

Recent Label Discussions workflow runs (last 7 days)

• §79 — 2026-05-22 — success
• §78 — 2026-05-21 — success
• §77 — 2026-05-20 — success
• §76 — 2026-05-19 — success
• §75 — 2026-05-18 — success
• §74 — 2026-05-17 — success
• §73 — 2026-05-16 — success

Recent Labelling Correction Feedback runs (last 7 days, all skipped)

• §417 — 2026-05-22 — skipped
• §416 — 2026-05-21 — skipped
• §415 — 2026-05-20 — skipped
• §414 — 2026-05-18 — skipped
• §413 — 2026-05-17 — skipped
• §412 — 2026-05-16 — skipped

§79 · §78 · §77

---

Generated automatically by the Labelling Health workflow on 2026-05-23T03:34:00Z.

Generated by Labelling Health Report · ● 384.8K · ◷

• expires on Jun 22, 2026, 3:37 AM UTC

[mnkiefer]

This issue is being closed as outdated. A newer issue has been created: #441

View newer issue

---

This action was performed automatically by the Labelling Health Report workflow.