Add Copilot CLI session logs as a mining source#5
Merged
Conversation
Lift the text-emission loop out of the role-check block and use an early continue. Behavior is unchanged for Codex and Claude logs; this only reshapes the control flow so a new log format can be added with a small diff. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Read ~/.copilot/session-state/*/events.jsonl alongside Codex and Claude logs. Copilot events use type 'user.message' with the typed text in data.content; system-injected steering messages (data.source == 'system') are skipped so only prose you actually typed is mined. - SOURCES gains a 'copilot' root; --source accepts 'copilot'; auto scans it - user_messages parses the Copilot event shape; Codex/Claude handling is unchanged (verified against a real log: 3395 of 4007 events are system injections, correctly dropped) - docstring and README note the new source Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot logs are all named events.jsonl under a per-session directory, so os.path.basename collapsed every block header to '===== events.jsonl =====' and sessions became indistinguishable in the corpus. Prefix the parent directory: the session id for Copilot, the project slug for Claude, the day for Codex. Cosmetic change to the session separator only; extraction, redaction, and chunk boundaries are unaffected. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds Copilot CLI session logs as a mining source, alongside Codex and Claude.
Reads
~/.copilot/session-state/*/events.jsonl.Related #3
How
Copilot events use
type: "user.message"with the typed text indata.content. System-injected steering messages (data.source == "system")are skipped, so only prose the user actually typed is mined. The raw
data.contentis read, notdata.transformedContent, which carries datetimeand system-reminder wrappers.
Commits (atomic)
user_messagesto a guard-clause shape. No behavior change forCodex or Claude; it only reshapes control flow so a new format is a small diff.
SOURCESroot, parser branch,--source copilot,auto-scan, docstring, README.
events.jsonl, so the bare basename collapsed every block header to the samestring. This also gives Claude blocks their project slug.
Verification
user.messageevents are systeminjections, all correctly dropped; typed messages are kept.
Notes
Comprehensive in-process unit tests and 100% coverage follow in a stacked PR #6