diff --git a/AGENTS.md b/AGENTS.md index 22bae3e..a4f0806 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -74,6 +74,7 @@ LLM이 판단한 것은 `generated_by`, `info_class`, `semantic_judgement_status - 기존 사용자 변경을 되돌리지 않는다. - 무관한 파일을 정리한다는 이유로 삭제하지 않는다. - 대량 이동, 삭제, DB 변형, 라우팅/프롬프트/answer_mode 대변경은 먼저 문서화하거나 결재를 받는다. +- 새 발주서를 제안하거나 확정하면 반드시 `Administrative_Reform_1/04_Orders/`에 발주서 문서로 남기고, `04_Orders/README.md`와 필요한 실행 기록을 갱신해 송련과 외부 에이전트가 추적 가능하게 한다. - 테스트 결과와 실패 원인을 실행 기록에 남긴다. - `git add .` 같은 무차별 staging은 쓰지 않는다. diff --git a/Administrative_Reform_1/00_Philosophy/Answer_Basis_Mode_And_Evidence_Role_Philosophy_2026_06_26.md b/Administrative_Reform_1/00_Philosophy/Answer_Basis_Mode_And_Evidence_Role_Philosophy_2026_06_26.md index 19892a3..d96909a 100644 --- a/Administrative_Reform_1/00_Philosophy/Answer_Basis_Mode_And_Evidence_Role_Philosophy_2026_06_26.md +++ b/Administrative_Reform_1/00_Philosophy/Answer_Basis_Mode_And_Evidence_Role_Philosophy_2026_06_26.md @@ -141,6 +141,81 @@ current_user_utterance_evidence 이 경우 `"전진"`은 내부 문서 근거가 아니라 현재 사용자 발화 근거다. +## 2026-06-29 추가 메모: 메타정보 판단의 장기 위치 + +장기적으로 보면 메타정보 판단은 턴의 한 지점에만 묶일 일이 아니다. + +턴 맨 앞에서는 다음 질문이 필요해진다. + +```text +이번 입력은 무엇을 근거로 이해해야 하는가? +최근 대화인가? +현재 사용자 발화인가? +문서 검색인가? +실행 상태인가? +``` + +턴 중간에서는 다음 질문이 필요해진다. + +```text +지금 수집된 자료는 절대정보인가, 상대정보인가, 혼합정보인가? +이 자료를 다음 노드에 원문으로 넘겨야 하는가, 장부로 넘겨야 하는가, 요약 후보로 넘겨야 하는가? +``` + +턴 끝에서는 다음 질문이 필요해진다. + +```text +최종 답변은 어떤 정보 등급의 말하기 태도를 가져야 하는가? +절대정보 중심으로 좁혀야 하는가? +상대 해석을 허용해도 되는가? +여러 source bundle을 엮은 혼합정보로 한계를 드러내야 하는가? +``` + +즉 메타정보 판단은 장기적으로 2번 node 하나의 좁은 후처리만이 아니라, 송련 전체 런타임의 여러 지점에서 반복적으로 필요해질 수 있다. + +하지만 지금 당장 이 전체 구조를 열면 범위가 너무 커진다. + +따라서 단기 구현 경계는 다음처럼 둔다. + +```text +이번 단계에서는 node_2가 node_3에게 답변 태도만 정해 준다. +``` + +여기서 말하는 답변 태도는 다음 세 갈래 정도로 좁힌다. + +```text +absolute_first +relative_allowed +mixed_or_uncertain +``` + +이 경계 안에서는 node_2가 "이번 턴은 절대정보 중심으로 답해야 한다"라고 판단할 수 있다. + +그 경우 node_3에게는 요약이나 감상보다 원문, count, trace/data 장부, document material packet 같은 검증 가능한 재료를 우선 공급하는 방향이 자연스럽다. + +반대로 node_2가 상대정보 또는 혼합정보를 허용하는 답변 태도를 고른다면, 장기적으로는 L3, 5, 또는 별도 요약 노드가 source가 붙은 요약 카드나 문서별 근거 카드를 만들어 node_3에게 줄 수 있다. + +단, 이 요약은 code가 의미를 대신 판단한 절대정보가 아니다. +요약은 LLM이 생성한 상대정보 또는 혼합정보로 남겨야 하며, 반드시 다음을 드러내야 한다. + +```text +generated_by +info_class +semantic_judgement_status +source_data_ids +source_trace_ids +``` + +중요한 결론: + +```text +장기적으로는 턴 전/중/후 어디서든 메타정보 판단이 필요할 수 있다. +하지만 지금은 node_2 -> node_3 답변 태도 전달까지만 구현 대상으로 삼는다. +``` + +이 제한은 후퇴가 아니다. +먼저 사용자-facing 최종 답변의 태도를 안정시킨 뒤, 나중에 기억 요약, L3 문서별 요약, 5 기억 압축, node_4 검토 루프와 함께 더 넓은 메타정보 판단 배치를 다시 설계하기 위한 안전한 순서다. + ## 다음 MVP 후보 다음 MVP는 구현 전에 별도 발주서로 좁혀야 한다. diff --git a/Administrative_Reform_1/00_Philosophy/Night_Government_August_Materials_From_Main_Project_2026_07_02.md b/Administrative_Reform_1/00_Philosophy/Night_Government_August_Materials_From_Main_Project_2026_07_02.md new file mode 100644 index 0000000..d17fafa --- /dev/null +++ b/Administrative_Reform_1/00_Philosophy/Night_Government_August_Materials_From_Main_Project_2026_07_02.md @@ -0,0 +1,459 @@ +# Night Government August Materials From Main Project 2026-07-02 + +**계층**: 00 철학 +**상태**: 본점 정밀 탐사 기반 미승격 설계 재료 +**권한**: 실행 권한 없음. 향후 발주서, 유지 체계, R-loop/Night Government 지도 문서로 승격 심사한다. + +## 1. 이 문서의 목적 + +이 문서는 SongRyeon 본점의 심야정부, Sleep Stack, CoreEgo, local council, D10 trace 문서를 탐사한 뒤, SongRyeon Core가 8월까지 완성할 심야정부에 가져올 재료를 Core식으로 정리한 문서다. + +본점의 결론을 그대로 베끼지 않는다. + +본점에서 가져올 것은 다음이다. + +```text +밤이 낮의 경험을 정리한다. +원본과 파생 정보를 분리한다. +정책/요약/가이드는 반드시 근거 주소를 가진다. +후보/자문/승인/무효화 상태를 섞지 않는다. +R-loop가 길을 찾을 수 있게 그래프 기억을 정리한다. +``` + +Core에서 버릴 것은 다음이다. + +```text +한 번에 모든 정부/의회/정책 그래프를 열기. +밤 산출물이 낮 route/tool/final answer 권한으로 바로 승격되기. +저가치 추측을 durable graph node로 굳히기. +private/public, candidate/approved, source/advisory 구분 없이 섞기. +진행 장부 없이 500개 넘는 파일을 한 번에 요약하기. +``` + +## 2. 본점 탐사 근거 + +읽은 본점 문서와 코드 표면: + +- `ANIMA_SLEEP_STACK_V1.md` +- `ANIMA_SLEEP_STACK_V2.md` +- `ANIMA_V4_MIDNIGHT_GOVERNMENT_PROPOSAL.md` +- `ANIMA_MIDNIGHT_DMN_REORG10A_INVENTORY_2026_05_26.md` +- `D2_DAY_NIGHT_STATE_SEPARATION_AND_ZERO_SUPPLY_AUDIT_2026_06_19.md` +- `D3_NIGHT_COREEGO_SOURCE_AUTHORITY_BOUNDARY_AUDIT_2026_06_19.md` +- `D7_NIGHT_MEMORY_PROMOTION_AND_GRAPH_PRIVACY_AUDIT_2026_06_19.md` +- `D8_MIDNIGHT_GOVERNMENT_PROPORTIONAL_COUNCIL_SCOPE_AUDIT_2026_06_19.md` +- `D9_COREEGO_LOCAL_COUNCIL_PROPORTIONAL_IDENTITY_AUDIT_2026_06_19.md` +- `D10_NIGHT_COREEGO_TRACEABILITY_REPLAY_REQUIREMENTS_2026_06_19.md` +- `GOVERNOR_MIDNIGHT_COREEGO_PROPORTIONAL_COUNCIL_SYNTHESIS_2026_06_19.md` +- `ANIMA_MIDNIGHT_COREEGO_TRACE_REPLAY_FIXTURE_REPORT_2026_06_20.md` +- `Core/_archive_v3_midnight/midnight_reflection.py` +- `Core/_archive_v3_midnight/midnight/rem_governor.py` + +본점의 큰 흐름은 다음이었다. + +```text +Dream / TurnProcess / PhaseSnapshot +-> REMPlan / REMGovernor +-> Coverage / Supply planning / Review +-> Placement / Policy fruit +-> BranchGrowth / BranchDigest +-> DreamHint / RoutePolicy / ToolDoctrine / TacticalThought +``` + +Core는 이 흐름을 지금 당장 전부 열지 않는다. + +Core는 먼저 다음 좁은 골격만 가져온다. + +```text +Raw source +-> source version +-> leaf summary +-> source kind bundle summary +-> time bundle summary +-> CoreEgo guide summary +-> R-loop guide / traversal material +``` + +## 3. 본점에서 배운 가장 중요한 실패 방지선 + +본점 문서들이 반복해서 말한 경계는 하나다. + +```text +밤 산출물은 유용하지만, 낮의 명령권이 아니다. +``` + +따라서 Core 심야정부 산출물의 기본 상태는 다음이어야 한다. + +| 산출물 | 기본 지위 | 낮 루프 권한 | +| --- | --- | --- | +| leaf summary | 근거 달린 relative/mixed 요약 | 직접 route/tool/final 권한 없음 | +| source kind bundle summary | 여러 원본 묶음에 대한 mixed 요약 | R-loop 탐색 안내 가능 | +| time bundle summary | 특정 관측 시간 묶음에 대한 mixed 요약 | R-loop 탐색 안내 가능 | +| CoreEgo guide summary | 그래프 탐색 가이드 | R-loop 안내 가능, 낮 route 명령 금지 | +| invalidation ledger | 절대정보 상태 장부 | 현재 근거 사용 가능 여부 표시 | +| night job ledger | 절대정보 작업 장부 | 재개/진행/실패 표시 | + +밤이 만든 요약은 사용자가 바로 믿어야 하는 최종 진실이 아니다. + +요약은 source bundle에서 파생된 정보이고, 사용 시점마다 다음을 드러내야 한다. + +```text +generated_by +info_class +semantic_judgement_status +source_graph_node_ids +source_data_ids +source_trace_ids +validity_status +summary_depth +``` + +## 4. Core가 이미 가진 기반 + +Core는 본점처럼 처음부터 새 기억 DB를 만들 필요가 없다. + +이미 있는 기반: + +- `TraceStore` +- `DataStore` +- `TurnStateCapsule` +- source manifest +- graph memory builder +- graph vessel adapter +- Neo4j readback/inspect +- source version lineage +- summary invalidation ledger +- source leaf summary worker + +따라서 Core 심야정부의 출발점은 새 `MemoryRecord`가 아니다. + +출발점은 다음이다. + +```text +이미 존재하는 trace/data/capsule/source manifest 좌표를 +그래프 DB의 raw/source node로 올리고, +그 위에 요약 node를 별도로 붙인다. +``` + +## 5. 8월 심야정부의 기본 재료 + +8월 완성판을 위해 필요한 재료 후보는 다음이다. + +### 5.1 NightJobLedger + +심야정부 작업 장부다. + +절대정보로 기록할 것: + +- job id +- target graph node id +- target source version hash +- planned action +- status: `pending`, `running`, `written`, `failed`, `superseded_before_run`, `skipped_no_text` +- started_at / finished_at +- llm call id +- output graph node id +- failure reason +- next resume cursor + +목적: + +```text +심야정부가 중간에 끊겨도 어디까지 했는지 알고 다시 시작한다. +``` + +### 5.2 SourceVersionLineage + +동적 원본의 버전 계보 장부다. + +코드/문서처럼 변할 수 있는 데이터는 같은 path라도 같은 원본이 아니다. + +기준: + +```text +source kind + path + observed_at + source_last_modified_at + content hash +``` + +새 버전이 생기면 예전 버전을 삭제하지 않는다. + +예전 버전에서 파생된 요약은 현재 근거로 쓰지 못하도록 무효화한다. + +### 5.3 SummaryInvalidationLedger + +요약 무효화 장부다. + +규칙: + +```text +pending summary target이 실행 전에 새 버전으로 대체되면 요약하지 않는다. +완료된 옛 summary는 삭제하지 않고 invalidated로 바꾼다. +상위 summary는 하위 active summary fingerprint가 바뀌면 stale이 된다. +``` + +핵심은 삭제가 아니라 상태 변경이다. + +### 5.4 LeafSummaryGraphNode + +원본 하나에 직접 대응하는 요약 노드다. + +분류: + +```text +source leaf 1개 -> summary 1개 +info_class = relative +``` + +예: + +```text +RawSource: songryeon_core/tools/code_tools.py +<- SUMMARY_OF +LeafSummary: code_tools.py 요약 +``` + +이 요약은 원본 하나에 대응하므로 상대정보다. + +### 5.5 SourceKindBundleSummary + +같은 종류의 원본 묶음에 대한 요약이다. + +예: + +```text +internal_document 묶음 +source_code_file 묶음 +conversation_turn 묶음 +``` + +분류: + +```text +여러 raw source 또는 여러 leaf summary 묶음 -> mixed +``` + +목적: + +```text +R-loop가 처음부터 500개 leaf를 보지 않고, source kind별 지도부터 본다. +``` + +### 5.6 TimeBundleSummary + +같은 관측/수집 시간에 묶인 source kind bundle들을 다시 묶은 요약이다. + +분류: + +```text +여러 source kind bundle 또는 여러 summary node 묶음 -> mixed +``` + +목적: + +```text +특정 시점에 그래프 DB에 무엇이 들어왔는지 큰 흐름을 잡는다. +``` + +### 5.7 CoreEgoGuideSummary + +CoreEgo가 R-loop에게 줄 그래프 탐색 안내 요약이다. + +기본 지위: + +```text +R-loop guide material +advisory +not day-route authority +not final-answer evidence +``` + +필수로 드러낼 것: + +- 어떤 시간축 entry가 있는가 +- source kind별 요약이 어디 있는가 +- active summary depth 범위 +- invalidated/stale summary가 있는가 +- R-loop가 먼저 볼 만한 entry는 무엇인가 +- 사용하면 안 되는 stale/private/candidate material은 무엇인가 + +## 6. 처리 흐름 + +8월 심야정부의 권장 처리 순서는 다음이다. + +```text +1. source/capsule/trace를 관측한다. +2. 새 원본 또는 변경된 원본만 raw source version으로 올린다. +3. 같은 내용이면 새 요약 작업을 만들지 않는다. +4. pending 작업이 새 버전으로 superseded되면 요약하지 않고 닫는다. +5. 바뀐 raw source leaf를 하나씩 요약한다. +6. leaf summary가 충분히 생기면 source kind bundle summary를 만든다. +7. source kind bundle summary가 생기면 time bundle summary를 만든다. +8. time bundle summary와 graph snapshot을 보고 CoreEgoGuideSummary를 만든다. +9. R-loop는 CoreEgo guide에서 시작해서 필요한 농도까지 내려간다. +``` + +중요: + +```text +요약 작업은 한 번에 전부 끝내야 하는 일이 아니다. +한 개씩 저장하고, 진행률을 기록하고, 중단되면 재개한다. +``` + +## 7. 시간축과 의미축 + +Core의 초기 graph memory는 시간축을 기본 골격으로 삼는다. + +이유: + +- 시간은 모든 사건에 붙을 수 있다. +- 코드가 비교적 안정적으로 확정할 수 있다. +- 무작위성과 정렬성을 동시에 가진다. +- R-loop가 처음 길을 잃지 않게 해준다. + +그러나 장기적으로는 의미축도 필요하다. + +다만 의미축은 처음부터 전부 자동 생성하지 않는다. + +의미축은 다음 조건에서 수요 기반으로 만든다. + +- R-loop가 시간축만으로 자주 탐색 실패한다. +- 특정 주제 질문이 반복된다. +- CoreEgo 직속 시간축 entry가 너무 많아져 R1 부담이 커진다. +- 사용자가 시간보다 주제 기준 검색을 요구한다. + +초기 결론: + +```text +시간축은 기본 골격. +의미축은 수요 기반 확장. +``` + +## 8. 메타정보 분류 + +심야정부의 정보 분류는 다음처럼 잠근다. + +| 대상 | 분류 | +| --- | --- | +| raw source node | absolute | +| source version lineage | absolute | +| job ledger | absolute | +| invalidation ledger | absolute | +| leaf summary, source 1개 대응 | relative | +| source kind bundle summary | mixed | +| time bundle summary | mixed | +| CoreEgo guide summary | mixed/advisory | +| R-loop selected path reason | selected source 수에 따라 relative 또는 mixed | + +핵심: + +```text +하나에 대응하면 relative. +여러 source bundle에 근거하면 mixed. +코드가 확인 가능한 상태/좌표/횟수/해시는 absolute. +``` + +## 9. trace/replay 요구 + +본점 D10 문서에서 Core가 반드시 가져와야 하는 원칙: + +```text +좋아 보이는 설계만으로 live insertion 하지 않는다. +trace/replay로 경계가 지켜졌는지 보여야 한다. +``` + +심야정부/R-loop에서 최소로 보여야 할 trace row: + +- producer family: night, graph builder, R-loop, L-loop, node_0, node_2 등 +- material kind: raw source, leaf summary, bundle summary, guide, candidate, invalidation +- authority status: raw, relative, mixed, advisory, candidate, active, stale, invalidated +- privacy scope: private, public-safe, unknown, mixed-risk +- source graph node ids +- consumer role +- consumer use: route consideration, graph traversal, answer material, debug only +- promotion status +- final answer 사용 여부 + +`TRACE_INSUFFICIENT`는 실패가 아니라 정직한 차단 상태다. + +## 10. 8월 완성판까지의 추천 단계 + +### Phase A. 진행 장부와 재개성 + +가장 먼저 해야 한다. + +- one-by-one 처리 +- progressive save +- progress inspect +- resume cursor +- superseded pending skip +- failed job 재시도 경계 + +### Phase B. leaf summary 안정화 + +- 새로 들어왔거나 바뀐 코드/문서 leaf만 요약 +- 원문 하나당 요약 하나 +- relative 정보로 기록 +- 원본 없는/빈 파일은 `skipped_no_text`로 닫기 + +### Phase C. source kind bundle summary + +- internal document끼리 묶기 +- source code file끼리 묶기 +- conversation turn끼리 묶기 +- 서로 다른 data kind는 섞지 않기 + +### Phase D. time bundle summary + +- 같은 ingest/observation batch의 source kind bundle들을 시간 묶음으로 정리 +- summary depth와 source counts를 계산 + +### Phase E. CoreEgo guide summary + +- R-loop가 볼 첫 안내판 생성 +- active/stale/invalidated 상태 분리 +- source kind와 time bundle entry를 사람이 읽을 수 있게 정리 + +### Phase F. R-loop read path + +- R1이 guide를 보고 목표/예산/농도 설정 +- R2가 entry 선택 +- R3가 충분성/농도/가지 문제 판단 +- 0이 R-loop trace와 읽은 graph node를 downstream에 전달 + +### Phase G. 수요 기반 의미축 + +나중에 연다. + +- 반복 탐색 실패 +- 반복 주제 질문 +- R1 context 초과 +- 시간축 과밀화 + +위 조건이 관측되기 전에는 의미축을 자동 대량 생성하지 않는다. + +## 11. 8월 전까지 금지할 것 + +아직 하지 않는다. + +- 밤 산출물이 낮 route/tool/answer mode를 직접 결정하게 하기 +- DreamHint/SecondDream류를 최종 답변 근거로 바로 쓰기 +- candidate/local/proportional material을 approved identity로 승격하기 +- private SongRyeon material을 public-safe로 묵시 승격하기 +- 의미축 자동 대량 생성 +- 저가치 추측 노드 durable 저장 +- 작업 장부 없는 대량 LLM 요약 +- 실패한 요약을 성공한 요약처럼 숨기기 +- 원본을 잘라서 중간 청크만 요약하고 전체 원본 요약처럼 표시하기 + +## 12. 한 줄 결론 + +8월의 Core 심야정부는 "밤에 똑똑한 말을 많이 쓰는 장치"가 아니다. + +8월의 Core 심야정부는 다음이어야 한다. + +```text +원본을 버리지 않고, +버전과 무효화를 기록하고, +요약을 source bundle에 매달고, +R-loop가 길을 찾을 수 있는 계층 지도를 만드는 +그래프 기억 행정부. +``` diff --git a/Administrative_Reform_1/00_Philosophy/Night_Government_Graph_Memory_Philosophy_2026_06_30.md b/Administrative_Reform_1/00_Philosophy/Night_Government_Graph_Memory_Philosophy_2026_06_30.md new file mode 100644 index 0000000..f77704b --- /dev/null +++ b/Administrative_Reform_1/00_Philosophy/Night_Government_Graph_Memory_Philosophy_2026_06_30.md @@ -0,0 +1,415 @@ +# Night Government Graph Memory Philosophy 2026-06-30 + +**계층**: 00 철학 +**상태**: 미승격 설계 철학 +**권한**: 실행 권한 없음. 향후 유지 체계, 지도, 발주서로 승격 심사한다. + +## 1. 배경 + +외부 채팅방에서 구현된 심야정부 MVP는 제거되었다. + +제거 이유는 단순히 코드 품질 문제가 아니라, 송련 Core가 이미 가진 기억 기반과 맞지 않았기 때문이다. + +송련 Core에는 이미 다음 기반이 있다. + +```text +TraceStore +DataStore +TurnStateCapsule +ZeroState +MemoryPacket +``` + +따라서 심야정부는 별도 `MemoryRecord` 저장소를 새로 발명하기보다, 기존 trace/capsule/data 좌표를 그래프 DB에 올리고 그 위에 출처가 분명한 상대/혼합정보를 쌓는 방향이어야 한다. + +## 2. 핵심 원칙 + +심야정부의 첫 목표는 "기억을 새로 상상해서 쓰는 것"이 아니다. + +첫 목표는 다음과 같다. + +```text +기존 trace/capsule/data를 잃지 않고 +그래프 DB에 절대정보 좌표로 정리하고 +그 위에 LLM 요약을 출처 달린 상대/혼합정보로 연결한다. +``` + +즉 심야정부는 원본 기억을 대체하지 않는다. + +심야정부는 원본 기억을 그래프 구조로 재배치하고, 요약 노드를 별도로 붙인다. + +## 3. 용어 구분 + +앞으로 "노드"라는 말을 두 의미로 섞지 않는다. + +```text +GraphNode +-> 그래프 DB에 저장되는 데이터 객체 + +NightWorker +-> 심야정부에서 실행되는 LLM 작업자 +``` + +예: + +- `RawCapsuleGraphNode`: 한 턴 캡슐 또는 그 안의 원본 좌표를 나타내는 그래프 노드. +- `RawBundleGraphNode`: 여러 raw leaf를 토큰 예산 안에서 묶은 그래프 노드. +- `SummaryGraphNode`: LLM이 만든 요약/판단 그래프 노드. +- `CoreEgoGuideWorker`: R루프 안내 정보를 생성하는 LLM 호출 작업자. + +그래프 DB의 노드와 LLM 실행 노드를 혼동하면, 저장 구조와 실행 책임이 흐려진다. + +## 4. Raw Leaf와 Raw Bundle + +최하위 원본 단위는 raw leaf로 본다. + +예: + +- 한 턴의 `TurnStateCapsule` +- 특정 trace/data record 묶음 +- 내부 문서 원문 +- 코드 파일 원문 + +raw leaf는 절대정보에 가깝다. + +코드가 확인할 수 있는 것은 다음이다. + +- 존재 여부 +- `turn_id` +- `trace_id` +- `data_id` +- 파일 경로 +- 문자 수 +- 생성 시각 +- 원본 종류 + +raw leaf 여러 개를 묶은 `RawBundleGraphNode`도 의미 요약이 아니다. + +`RawBundleGraphNode`는 다음과 같은 절대정보적 묶음 좌표다. + +```text +이 raw leaf 묶음이 있다. +이 묶음은 이 source id 목록에 기대고 있다. +이 묶음은 이 시간 범위에 속한다. +이 묶음은 이 토큰 예산 안에서 구성되었다. +``` + +## 5. 원문 절단 금지 + +토큰 예산 때문에 계층을 나눌 때 원문을 중간에서 자르지 않는다. + +기본 규칙: + +```text +원문을 자르지 않는다. +토큰 예산을 넘으면 leaf 단위로 bundle을 나눈다. +예산에 걸치는 마지막 leaf는 빼거나 다음 bundle로 넘긴다. +``` + +최하단 bundle worker는 임시 요약을 보지 않는다. + +최하단 `NightWorker`는 가능한 한 raw leaf 원문들을 직접 본다. + +이유: + +1. 첫 요약 이전의 정보 손실을 줄인다. +2. 요약의 요약 오류가 초반부터 전파되는 것을 막는다. +3. 첫 LLM 산출물이 어떤 원문 source bundle을 직접 봤는지 분명히 남길 수 있다. + +## 6. 요약은 원본 속성이 아니라 별도 노드다 + +LLM이 만든 요약을 raw node의 속성으로 바로 박지 않는다. + +더 안전한 구조는 다음이다. + +```text +RawGraphNode +<- SUMMARY_OF +SummaryGraphNode +``` + +원본 노드는 절대정보 좌표로 깨끗하게 남긴다. + +요약 노드는 별도 graph node로 만들고, 다음 필드를 가진다. + +```text +info_class +generated_by +semantic_judgement_status +source_graph_node_ids +source_trace_ids +source_data_ids +source_bundle_kind +review_status +``` + +이 구조는 나중에 요약이 틀렸을 때 원본을 오염시키지 않고 요약 노드만 폐기하거나 재생성할 수 있게 한다. + +## 7. 상대정보와 혼합정보 분류 + +심야정부의 LLM 산출물은 다음 기준으로 분류한다. + +```text +원문 leaf 1개를 직접 보고 만든 요약/판단 +-> relative + +원문 leaf 여러 개를 직접 보고 만든 요약/판단 +-> mixed + +여러 summary node를 보고 만든 상위 요약/판단 +-> mixed + +대화, 내부 문서, 코드처럼 서로 다른 data kind가 섞인 source bundle +-> mixed +``` + +따라서 현실적으로 한 턴만 처리하는 특수 상황이 아니라면, 첫 `NightWorker` 산출물도 대부분 `mixed`일 가능성이 높다. + +중요한 점: + +```text +mixed는 출처가 흐린 정보가 아니다. +mixed는 여러 절대정보 묶음에 근거했다는 사실을 드러내는 정보다. +``` + +혼합정보는 반드시 source bundle을 가진다. + +## 8. 요약 계층 계산 + +심야정부의 summary node에는 계층 계산 필드가 필요하다. + +필수 후보: + +```text +summary_depth +source_depth_min +source_depth_max +source_leaf_count +source_summary_count +source_bundle_kind +``` + +계산 원칙: + +```text +raw leaf node: +depth = 0 + +raw leaf를 직접 보고 만든 첫 summary: +summary_depth = 1 + +summary node들을 보고 만든 상위 summary: +summary_depth = max(source_depths) + 1 +``` + +`source_leaf_count`는 위로 올라가도 누적한다. + +예: + +```text +대화 원문 8턴 -> depth 1 mixed summary +대화 원문 8턴 -> depth 1 mixed summary +두 summary를 다시 묶음 -> depth 2 mixed summary +``` + +이때 depth 2 summary는 다음을 기록한다. + +```text +summary_depth = 2 +source_leaf_count = 16 +source_summary_count = 2 +source_depth_min = 1 +source_depth_max = 1 +``` + +이 인프라가 있어야 나중에 R루프가 다음을 구분할 수 있다. + +- 원문에 가까운 기억인가? +- 요약의 요약인가? +- 몇 단계 압축됐나? +- 너무 추상화된 기억인가? +- R1이 직접 쓰기에는 위험한 고압축 정보인가? + +## 8.1 동적 원본 변경과 파생 요약 무효화 + +코드 파일, 내부 문서, 진행 중 발주서처럼 나중에 바뀔 수 있는 원본은 정적 기억으로 취급하지 않는다. + +동적 원본은 다음처럼 관측 시점과 내용 해시를 가진 versioned raw source로 본다. + +```text +source identity: +- source kind +- path +- observed_at +- ingested_at +- source_last_modified_at +- content hash +``` + +같은 path의 동적 원본이라도 content hash 또는 관측 시점이 달라지면 새 raw source version으로 기록한다. + +기존 raw source node와 그 위에 붙은 LLM summary node는 삭제하지 않는다. + +대신 새 관측 결과가 기존 동적 원본의 변경을 확인하면, 기존 원본 version에서 파생된 모든 LLM summary node는 현재 기준으로 무효화 처리한다. + +무효화 후보 필드: + +```text +validity_status = invalidated_by_source_change +invalidated_at = ... +invalidated_by_source_graph_node_id = ... +invalidated_reason_code = source_content_changed +superseded_by_source_graph_node_id = ... +``` + +무효화의 의미: + +```text +과거에 어떤 원본을 보고 어떤 판단을 했는지는 보존한다. +하지만 바뀐 원본 기준으로 그 summary를 현재 사실처럼 쓰면 안 된다. +``` + +무효화는 삭제가 아니다. + +무효화는 추적 가능한 상태 변경이다. + +따라서 R루프와 심야정부 worker는 summary node를 사용할 때 다음을 확인해야 한다. + +```text +1. source_graph_node_ids가 현재 유효한가? +2. summary node 자체가 invalidated 상태인가? +3. summary_depth가 너무 높은가? +4. 동적 원본 변경 이후 재요약이 필요한가? +``` + +원칙: + +```text +원본은 보존한다. +요약도 보존한다. +다만 동적 원본 변경 이후의 파생 요약은 현재 답변 근거로 쓰지 못하게 상태를 바꾼다. +``` + +이 정책은 외부 기억 시스템의 일반적인 invalidation 발상과 닮았을 수 있으나, 송련 Core에서는 외부 구현을 권위로 삼지 않는다. + +송련 Core의 기준은 다음이다. + +```text +모든 상대/혼합 정보는 절대정보 source bundle에서 파생된다. +source bundle이 바뀌면 파생 정보의 현재 유효성도 재검토해야 한다. +``` + +## 9. CoreEgo 연결: 초기에는 시간축만 + +초기 CoreEgo 직속 그래프 연결은 시간축만 사용한다. + +이유: + +- 시간, 턴, trace, data 좌표는 코드가 비교적 안정적으로 확정할 수 있다. +- 의미 분류는 LLM의 상대/혼합 판단이므로 처음부터 CoreEgo 직속 구조로 쓰면 위험하다. +- 먼저 시간축으로 작고 정직하게 시작해야 한다. + +초기 구조: + +```text +CoreEgo + -> TimeAxis + -> TimeBundle_2026_06_30_session_001 + -> RawCapsule_turn_0001 + -> RawCapsule_turn_0002 + -> Summary_time_bundle_001 +``` + +의미축은 나중에 R1의 부담이 실제로 커졌을 때 도입한다. + +도입 조건 후보: + +- R1 context가 시간축 guide만으로 자주 초과된다. +- R루프가 시간축에서 반복적으로 탐색 실패한다. +- 사용자 질문이 시간보다 주제 기준 검색을 요구하는 비율이 높아진다. +- graph node 수가 많아져 CoreEgo 직속 시간축만으로는 entry selection 비용이 커진다. + +## 10. 장기 의미축 구상 + +장기적으로는 다음 구조를 병행할 수 있다. + +```text +CoreEgo + -> TimeAxis + -> TimeBundle_... + -> SemanticAxis + -> Topic_최근기억시스템 + -> Topic_L루프검색구조 + -> Topic_메타정보정책 +``` + +의미축 도입 후 R1은 기본적으로 의미축을 우선 열람하고, 필요할 때 시간축으로 역추적할 수 있다. + +다만 의미축은 코드가 몰래 만들면 안 된다. + +의미 분류 필수 필드 후보: + +```text +topic_label +topic_assignment_generated_by +topic_assignment_info_class +topic_source_graph_node_ids +topic_confidence +topic_review_status +``` + +의미축은 강력하지만 위험하다. + +따라서 초기 MVP에서는 열지 않는다. + +## 11. 지금 당장 하지 않는 것 + +이 문서는 구현 발주서가 아니다. + +지금 당장 하지 않는다. + +- 그래프 DB 연결 +- 의미축 CoreEgo 연결 +- R루프 구현 +- 자동 장기기억 승격 +- 4 승인 없는 summary 사용 +- 외부 DB record를 기존 capsule과 별도로 새로 발명하기 +- 모든 노드에 CoreEgo 최소정보 배분하기 + +## 12. 임시 결론 + +심야정부는 기억 요약기가 아니라 그래프 기억 정리기여야 한다. + +첫 MVP의 정신은 다음이다. + +```text +TurnStateCapsule/TraceStore/DataStore를 source로 삼는다. +raw graph node는 절대정보 좌표로 만든다. +summary는 별도 node로 만들고 source edge를 단다. +첫 LLM worker는 원문 leaf bundle을 직접 본다. +대부분의 첫 summary는 mixed다. +summary depth를 계산한다. +CoreEgo 직속 연결은 처음에는 시간축만 쓴다. +의미축은 R1 부담이 실제로 관측된 뒤 연다. +``` + +## 13. 2026-07-02 본점 정밀 탐사 이후 추가 재료 + +본점의 Sleep Stack, Midnight Government, CoreEgo, local council, D10 trace 계열 문서를 다시 읽은 뒤, Core의 8월 심야정부 재료는 별도 철학 문서로 분리했다. + +연결 문서: + +```text +Administrative_Reform_1/00_Philosophy/Night_Government_August_Materials_From_Main_Project_2026_07_02.md +``` + +핵심 보강: + +```text +심야정부는 낮의 명령권이 아니다. +심야정부는 source/version/summary/validity/R-guide를 정리한다. +작업 장부 없이 대량 LLM 요약을 돌리지 않는다. +pending 작업이 새 버전으로 대체되면 요약하지 않고 superseded로 닫는다. +완료된 옛 요약은 삭제하지 않고 invalidated로 남긴다. +시간축은 기본 골격이고, 의미축은 R-loop 수요가 생긴 뒤 연다. +``` diff --git a/Administrative_Reform_1/00_Philosophy/Qwen_Model_Vs_SongRyeon_Runtime_Experiment_Philosophy_2026_06_29.md b/Administrative_Reform_1/00_Philosophy/Qwen_Model_Vs_SongRyeon_Runtime_Experiment_Philosophy_2026_06_29.md new file mode 100644 index 0000000..fa0641d --- /dev/null +++ b/Administrative_Reform_1/00_Philosophy/Qwen_Model_Vs_SongRyeon_Runtime_Experiment_Philosophy_2026_06_29.md @@ -0,0 +1,188 @@ +# Qwen Model Vs SongRyeon Runtime Experiment Philosophy + +## 상태 + +미승격 철학 메모. + +이 문서는 즉시 구현 명령이 아니다. +Qwen 14B 단독 성능과 SongRyeon Core 런타임 구조가 더하는 가치를 비교하기 위한 실험 설계 기록이다. + +## 배경 + +사용자는 아버지에게 다음 질문을 받았다. + +```text +이 답변이 qwen3:14b 자체 성능으로 나온 것인가, +아니면 SongRyeon Core 구조를 잘 만들어서 나온 것인가? +``` + +이 질문은 단순한 모델 품질 평가가 아니다. +SongRyeon Core가 추구하는 핵심 가치, 즉 LLM 답변을 더 똑똑하게 만드는 것뿐 아니라 근거, 한계, 출처, 실행 경로를 더 정직하게 드러내는 구조가 실제로 유효한지 묻는 질문이다. + +## 핵심 관점 + +비교해야 할 것은 "누가 더 멋진 문장을 쓰는가"가 아니다. + +비교해야 할 것은 다음이다. + +1. 같은 질문에서 근거 없는 일반론이 얼마나 줄어드는가. +2. 확인한 문서와 확인하지 못한 문서를 구분하는가. +3. "읽은 문서 기준"이라는 한계를 스스로 드러내는가. +4. 문서 수, 검색 상태, 실패 상태 같은 절대정보를 임의로 꾸미지 않는가. +5. 코드 변경 제안, 테스트 전략, 회귀 위험 탐지, 설계 문서 추적, 실행 기록 관리 같은 사용자의 평가 축을 빠뜨리지 않는가. +6. 최종 답변 뒤에 어떤 구조가 그 답변을 만들었는지 추적 가능한가. + +## 비교군 + +### A. Qwen 14B 단독 + +Qwen 14B에 동일한 질문만 준다. + +목적: + +- 모델 날것의 일반 추론 능력을 확인한다. +- 문서 검색, trace, node_2 answer basis, L3 요약, node_4 검사가 없을 때 답변이 어떻게 흐르는지 본다. + +예상 위험: + +- 그럴듯한 일반론으로 흐를 수 있다. +- 실제 SongRyeon Core 내부 문서 기준이 아니라 보편적 소프트웨어 공학 상식으로 답할 수 있다. +- 근거 한계가 흐릿할 수 있다. + +### B. Qwen 14B + 문서 원문 수동 제공 + +Qwen 14B에 동일한 질문과 함께 SongRyeon이 읽은 문서 일부 또는 동일 문서 원문을 직접 붙여 준다. + +목적: + +- 모델에게 컨텍스트만 충분히 주면 SongRyeon 구조 없이도 비슷한 답이 나오는지 확인한다. +- "검색/노드 구조"가 아니라 "문서 제공량"만으로 좋아지는 부분을 분리한다. + +예상 위험: + +- 답변 품질은 올라갈 수 있지만 어떤 문서가 어떤 판단의 근거인지 흐려질 수 있다. +- count, trace, 실패 상태, partial 상태 같은 런타임 사실을 스스로 관리하지 못할 수 있다. + +### C. SongRyeon Core 전체 런타임 + +동일한 질문을 SongRyeon Core `qwen-turn` 또는 live runtime으로 실행한다. + +포함되는 구조: + +- node_0 memory/material supply +- node_1 routing +- L loop search/read/revision +- L3 achievement/recheck +- node_2 answer basis mode +- L3 per-document summary +- node_3 final report +- node_4 gatekeeper +- terminal/runtime trace + +목적: + +- SongRyeon 구조가 Qwen 14B의 답변을 어떻게 제한, 보강, 추적 가능하게 만드는지 확인한다. +- 단순 문장 품질보다 근거 정직성, 실패 노출, 한계 표시, 추적성을 본다. + +예상 장점: + +- 느리지만 근거와 한계를 더 잘 드러낼 수 있다. +- "LLM이 쓴 의미 판단"과 "code가 확정한 절대정보"를 분리할 수 있다. +- node_4가 일부 과잉 주장이나 count mismatch를 막을 수 있다. + +예상 위험: + +- 구조가 무거워서 응답 시간이 길 수 있다. +- L loop가 질문 목적에 맞는 문서를 잘 못 고르면 partial 상태가 길어질 수 있다. +- L3 요약 대체 정책이 원문 세부를 과하게 줄일 수 있다. + +## 고정 질문 후보 + +```text +SongRyeon Core를 숙련 개발자의 실무 보조 도구로 쓴다고 가정하고, +현재 구조에서 실제 개발 업무에 도움이 될 만한 기능과 아직 위험한 기능을 내부 문서 기준으로 나눠줘. + +코드 변경 제안, 테스트 전략, 회귀 위험 탐지, 설계 문서 추적, 실행 기록 관리 관점에서 평가해줘. + +확인한 근거와 한계를 분리해서 말해줘. +``` + +## 평가 기준 + +### 근거 정직성 + +- 읽은 문서 기준이라고 말하는가. +- 확인하지 않은 것을 확인한 것처럼 말하지 않는가. +- "모든 문서", "전혀 없다", "부재한다" 같은 강한 단정을 조심하는가. + +### 구조 추적성 + +- 답변이 어떤 근거에서 나왔는지 추적 가능한가. +- 문서 수, 검색 후보 수, read_doc 수, 실패 상태를 확인할 수 있는가. +- 결과를 나중에 다시 감사할 수 있는가. + +### 실무 유용성 + +- 숙련 개발자가 실제 업무에서 쓸 만한 판단을 주는가. +- 코드 변경 제안, 테스트 전략, 회귀 위험 탐지, 설계 문서 추적, 실행 기록 관리 축을 실제로 다루는가. +- 위험한 기능과 이미 쓸 만한 기능을 구분하는가. + +### 한계 표시 + +- partial, failed, not_enough_evidence 같은 상태를 답변 태도에 반영하는가. +- 문서 검색 성공과 질문 목표 달성을 구분하는가. +- 문서가 있어도 질문의 핵심 근거가 부족할 수 있음을 드러내는가. + +## 결과 기록 위치 + +실제 실험 결과는 5번 계층에 남긴다. + +후보 파일명: + +```text +Administrative_Reform_1/05_Execution_Records/qwen_vs_songryeon_runtime_value_experiment_2026_06_29_001.md +``` + +기록할 항목: + +```text +실험 일시: +실행 환경: +모델: +고정 질문: + +A. Qwen 14B 단독 결과 요약: +B. Qwen 14B + 문서 원문 수동 제공 결과 요약: +C. SongRyeon Core 전체 런타임 결과 요약: + +비교 평가: +- 근거 정직성: +- 구조 추적성: +- 실무 유용성: +- 한계 표시: +- 속도/비용: + +잠정 결론: +후속 발주 후보: +``` + +## 주의 + +이 실험은 Qwen 14B의 객관적 벤치마크가 아니다. +또한 SongRyeon Core가 항상 더 좋은 답을 만든다는 증명도 아니다. + +이 실험의 목적은 한 가지다. + +```text +같은 모델을 그냥 쓰는 것과, +SongRyeon Core의 trace/source/count/gate 구조 안에서 쓰는 것이 +답변의 정직성, 추적성, 실무 안정성을 얼마나 다르게 만드는지 확인한다. +``` + +## 하지 말아야 할 것 + +- 한 번의 실험 결과로 모델 우열을 단정하지 않는다. +- 문장이 더 길거나 그럴듯하다는 이유만으로 더 좋은 답이라고 판단하지 않는다. +- SongRyeon Core가 읽지 않은 문서까지 읽은 것처럼 평가하지 않는다. +- code가 LLM 의미 판단을 대신했다고 포장하지 않는다. +- 실패 상태를 숨기고 성공한 실험처럼 기록하지 않는다. diff --git a/Administrative_Reform_1/00_Philosophy/README.md b/Administrative_Reform_1/00_Philosophy/README.md index c5cce64..6dc7f29 100644 --- a/Administrative_Reform_1/00_Philosophy/README.md +++ b/Administrative_Reform_1/00_Philosophy/README.md @@ -13,3 +13,7 @@ - [Raw_Memory_Window_And_Node5_Compression_Philosophy_2026_06_26.md](Raw_Memory_Window_And_Node5_Compression_Philosophy_2026_06_26.md): 최근 원문 기억 최대 8턴, 최소 3턴 보장, 9턴 진입 시 4턴 압축 후보 분리, 0/5/4 역할 분담을 정리한 미승격 기억 철학 문서. - [Answer_Basis_Mode_And_Evidence_Role_Philosophy_2026_06_26.md](Answer_Basis_Mode_And_Evidence_Role_Philosophy_2026_06_26.md): 기억은 전달되지만 node_3가 문서/최근기억/현재발화 중 무엇을 주근거로 삼아야 하는지 정하지 못하는 문제와 `answer_basis_mode` 다음 MVP 후보를 정리한 미승격 철학 문서. - [L_Loop_Dynamic_Budget_Management_Philosophy_2026_06_27.md](L_Loop_Dynamic_Budget_Management_Philosophy_2026_06_27.md): L3 partial 이후 예산 부족 신호, code budget policy 승인, context packing 예산을 어떻게 장기적으로 분리할지 정리한 미승격 철학 문서. +- [Qwen_Model_Vs_SongRyeon_Runtime_Experiment_Philosophy_2026_06_29.md](Qwen_Model_Vs_SongRyeon_Runtime_Experiment_Philosophy_2026_06_29.md): Qwen 14B 단독 성능과 SongRyeon Core 런타임 구조가 더하는 가치를 비교하기 위한 A/B/C 실험 설계 철학 문서. +- [Night_Government_Graph_Memory_Philosophy_2026_06_30.md](Night_Government_Graph_Memory_Philosophy_2026_06_30.md): 외부 채팅방산 심야정부 MVP 제거 이후, TurnStateCapsule/TraceStore/DataStore 기반 graph memory, raw/summary node 분리, summary depth, 시간축 CoreEgo 연결 우선 원칙을 정리한 미승격 철학 문서. +- [Night_Government_August_Materials_From_Main_Project_2026_07_02.md](Night_Government_August_Materials_From_Main_Project_2026_07_02.md): 본점 Sleep Stack/Midnight/CoreEgo/D10 trace 문서를 정밀 탐사한 뒤, 8월 Core 심야정부를 위한 job ledger, source version, invalidation, leaf/source-kind/time-bundle/CoreEgo guide summary 재료를 정리한 미승격 철학 문서. +- [R_Loop_Graph_Guide_Philosophy_2026_06_30.md](R_Loop_Graph_Guide_Philosophy_2026_06_30.md): L루프의 목표/후보/예산/continuation 패턴을 R루프의 CoreEgo graph traversal, 정보 농도, 가지 전환, RLoopGraphGuidePacket 구상으로 옮긴 미승격 철학 문서. diff --git a/Administrative_Reform_1/00_Philosophy/R_Loop_Graph_Guide_Philosophy_2026_06_30.md b/Administrative_Reform_1/00_Philosophy/R_Loop_Graph_Guide_Philosophy_2026_06_30.md new file mode 100644 index 0000000..bea63a4 --- /dev/null +++ b/Administrative_Reform_1/00_Philosophy/R_Loop_Graph_Guide_Philosophy_2026_06_30.md @@ -0,0 +1,371 @@ +# R Loop Graph Guide Philosophy 2026-06-30 + +**계층**: 00 철학 +**상태**: 미승격 설계 철학 +**권한**: 실행 권한 없음. 향후 유지 체계, 지도, 발주서로 승격 심사한다. + +## 1. 배경 + +R루프를 지금 구상하는 이유는, 심야정부의 첫 MVP 목표가 "완성된 장기기억"이 아니라 R루프의 그래프 DB 길찾기 안내이기 때문이다. + +본점의 CoreEgo 구상은 모든 송련 노드가 봐야 할 최소 정보를 생성하고 배분하는 더 큰 목표를 가졌다. + +SongRyeon Core에서는 그 큰 목표를 뒤로 미룬다. + +현재 MVP 감각은 더 좁다. + +```text +CoreEgo는 우선 R루프가 그래프 DB에서 길을 잃지 않게 하는 안내 정보를 만든다. +``` + +## 2. L루프에서 배운 패턴 + +L루프의 성능이 좋아진 이유는 "그냥 검색을 더 시켰기 때문"이 아니다. + +L루프는 다음 장부를 갖게 되면서 강해졌다. + +```text +목표 +예산 +검색 후보 +읽은 문서 +안 읽은 후보 +남은 예산 +성공/부분성공/실패 판단 +다음에 할 수 있는 행동 +``` + +현재 L루프의 큰 흐름은 다음과 같다. + +```text +L1: 목표와 필요 근거, 예산 요구 설정 +-> L2: 검색어/도구/읽을 대상 선택 +-> tool 실행 +-> L3: 목표 달성 여부와 근거 충분성 판단 +-> continuation controller: + 충분하면 stop + 부족하고 예산 남으면 L2 revision + 예산 없으면 partial/fail로 닫음 +-> 0이 L return summary를 만들어 다음 노드에 전달 +``` + +R루프도 이 패턴을 그대로 가져가되, 검색 대상이 문서가 아니라 그래프 DB가 된다. + +## 3. R루프의 기본 목표 + +R루프는 범용 대화 루프가 아니다. + +R루프는 다음 역할을 위한 미래 루프다. + +```text +CoreEgo +정체성 +장기 기억 +그래프 DB 탐색 +심야정부가 만든 기억 계층 읽기 +``` + +초기 R루프의 목표는 다음 하나로 좁힌다. + +```text +CoreEgo에 직속 연결된 그래프 노드들에서 시작해, +사용자 질문에 필요한 정보 농도까지 안전하게 내려간다. +``` + +## 4. R1, R2, R3 역할 구상 + +### R1: Graph Goal And Budget Planner + +R1은 0이 공급한 CoreEgo guide와 현재 사용자 질문을 보고 그래프 탐색 목표를 세운다. + +R1이 정해야 할 것: + +```text +graph_search_goal +required_information_granularity +allowed_summary_depth +max_traversal_depth +max_branch_switches +max_node_reads +max_context_tokens +stop_condition +``` + +R1은 다음 질문에 답해야 한다. + +```text +이번 질문은 원문에 가까운 정보가 필요한가? +낮은 농도의 summary로 충분한가? +높은 농도의 mixed summary도 쓸 수 있는가? +몇 계층까지 파도 되는가? +예산을 어디까지 쓸 것인가? +``` + +R1의 목표/예산 판단은 LLM 판단이면 상대/혼합정보다. + +코드는 예산 숫자와 schema만 검증한다. + +### R2: Graph Entry Or Child Selector + +R2는 R1의 목표를 보고 탐색할 그래프 노드 하나를 고른다. + +초기 MVP에서는 CoreEgo 직속 시간축 노드 중 하나를 고르는 역할이 중심이다. + +나중에 deeper 탐색 중에는 선택된 상위 노드의 하위 노드 목록을 보고 하나를 고른다. + +R2가 고르는 것: + +```text +selected_graph_node_id +selection_scope +selection_reason +expected_information_granularity +expected_source_kind +``` + +R2의 선택 이유는 의미 판단이다. + +따라서 code fallback이 "이 노드가 의미상 맞다"고 대신 판단하면 안 된다. + +### R3: Graph Node Inspector And Sufficiency Judge + +R3는 R2가 고른 그래프 노드를 검사한다. + +R3가 봐야 할 것: + +```text +선택된 노드가 raw leaf인가? +bundle인가? +summary인가? +summary라면 depth가 몇인가? +하위 노드가 있는가? +현재 정보 농도로 답할 수 있는가? +더 아래로 내려가야 하는가? +애초에 다른 가지를 골라야 하는가? +``` + +R3는 두 문제를 구분해야 한다. + +```text +농도 문제: +같은 가지는 맞지만 정보가 너무 압축되어 있다. +-> 더 아래로 판다. + +가지 문제: +애초에 선택한 그래프 노드가 목표와 맞지 않는다. +-> 다른 entry 또는 sibling branch로 돌아간다. +``` + +이 구분이 없으면 R루프는 같은 곳만 계속 파거나, 조금 부족하다는 이유로 엉뚱한 곳으로 튈 수 있다. + +## 5. 정보 농도 + +사용자 표현의 "농도"는 R루프에서 중요한 개념이다. + +정식 후보 용어: + +```text +information_granularity +evidence_density +summary_depth +``` + +초기 감각: + +```text +낮은 농도 +-> 원문에 가까움 +-> 구체적 +-> 토큰을 많이 먹음 + +중간 농도 +-> 한 묶음 요약 +-> 적당히 압축됨 + +높은 농도 +-> 요약의 요약 +-> 빠르지만 세부 근거 약함 +``` + +사용자가 구체적인 것을 물으면 R루프는 낮은 농도 정보나 원본 데이터를 향해 내려가야 한다. + +사용자가 큰 흐름, 방향, 전략을 물으면 더 높은 농도의 mixed summary에서 멈출 수 있다. + +R3의 핵심 질문: + +```text +지금 농도의 정보로 사용자 질문에 답해도 되는가? +``` + +## 6. R Continuation Controller + +R3 이후에는 controller가 반복 여부를 결정한다. + +controller는 의미 판단을 새로 하지 않는다. + +controller가 보는 것은 구조화된 R3 결과와 예산 숫자다. + +후보 상태: + +```text +stop_sufficient +continue_deeper +continue_switch_branch +stop_budget_exhausted +stop_no_actionable_path +stop_failed_final +``` + +의미: + +- `stop_sufficient`: 현재 노드/요약 농도로 충분하다. +- `continue_deeper`: 같은 가지는 맞지만 더 낮은 농도 정보가 필요하다. +- `continue_switch_branch`: 현재 가지가 목표와 맞지 않아 다른 entry 또는 sibling을 봐야 한다. +- `stop_budget_exhausted`: 더 볼 수 있지만 예산이 없다. +- `stop_no_actionable_path`: 하위 노드도 없고 다른 선택지도 없다. +- `stop_failed_final`: 반복 한계에 닿았다. + +## 7. 0의 역할 + +R루프에서도 0은 중요하다. + +L루프에서 0이 잘하는 일은 "의미 판단"이 아니라 다음이다. + +```text +루프가 만든 trace/data 좌표를 다음 노드가 잃어버리지 않게 봉투에 담는다. +``` + +R루프에서도 0은 다음을 해야 한다. + +- R루프 시작 전에 CoreEgo guide packet을 공급한다. +- R1/R2/R3/controller가 만든 frame ID를 보존한다. +- 읽은 graph node 목록을 보존한다. +- 선택하지 않은 후보 목록도 필요하면 보존한다. +- R return summary를 만들어 1 또는 downstream node가 이해할 수 있게 한다. + +0은 그래프 의미 판단을 대신하지 않는다. + +0은 좌표, trace, source, budget, status를 잃지 않게 전달한다. + +## 8. RLoopGraphGuidePacket + +심야정부/CoreEgo 측에서 R루프에 줄 첫 안내 정보는 `RLoopGraphGuidePacket`으로 볼 수 있다. + +역할: + +```text +R루프가 그래프 DB를 탐색할 때 어디서 시작하고, +어떤 종류의 노드를 조심하고, +어떤 depth의 요약을 어떻게 볼지 알려주는 안내판. +``` + +후보 필드: + +```text +graph_snapshot_id +target_consumer = R_LOOP +available_entry_nodes +node_kind_counts +data_kind_counts +summary_depth_range +source_leaf_count_range +high_value_entry_node_ids +risky_or_unreviewed_node_ids +recommended_traversal_hints +forbidden_shortcuts +source_graph_node_ids +generated_by +info_class +semantic_judgement_status +``` + +주의: + +- `available_entry_nodes`, count, depth range는 code가 확정 가능한 절대정보다. +- `recommended_traversal_hints`는 의미 판단이므로 LLM이 만들어야 한다. +- guide packet은 최종 진실이 아니라 탐색 안내다. + +## 9. 초기 CoreEgo MVP 범위 + +본점의 CoreEgo 전체 목표는 아직 열지 않는다. + +아직 하지 않는 것: + +- 모든 노드에 공통 최소 정보 배분 +- 장기 자아 요약 +- 전역 세계관 생성 +- 자동 행동 계획 +- 의미축 CoreEgo 직속 연결 + +이번 철학이 가리키는 초기 MVP: + +```text +심야정부가 그래프 DB를 정리한다. +CoreEgoGuideWorker가 시간축 graph snapshot을 본다. +RLoopGraphGuidePacket을 만든다. +R루프는 그 guide를 보고 그래프 탐색을 시작한다. +``` + +## 10. R루프와 L루프의 대응 + +```text +L1: 문서 검색 목표/예산 +R1: 그래프 탐색 목표/예산 + +L2: 검색어/도구/문서 후보 선택 +R2: CoreEgo entry 또는 하위 graph node 선택 + +L3: 검색 결과와 읽은 문서가 목표에 충분한지 판단 +R3: 선택 노드의 정보 농도와 하위 구조가 목표에 충분한지 판단 + +L continuation: 더 검색/읽기 또는 멈춤 +R continuation: 더 깊게/다른 가지/멈춤 + +L return summary: 0이 다음 노드에 전달 +R return summary: 0이 1 또는 downstream에 전달 +``` + +## 11. 지금 당장 하지 않는 것 + +이 문서는 발주서가 아니다. + +지금 당장 하지 않는다. + +- R루프 구현 +- 그래프 DB 연결 +- CoreEgo full implementation +- 의미축 graph hierarchy +- R1/R2/R3 prompt 작성 +- 자동 graph traversal +- 0의 graph memory packet 구현 + +먼저 이 철학을 기준으로 R루프 프레임/상태기계 설계 감사를 해야 한다. + +## 12. 임시 결론 + +R루프는 L루프의 검색 반복 구조를 그래프 탐색 반복 구조로 바꾸는 것이다. + +L루프가 다음을 한다면: + +```text +검색 후보 -> 문서 읽기 -> 충분성 판단 -> 더 검색/읽기 +``` + +R루프는 다음을 한다. + +```text +CoreEgo entry 후보 -> graph node 열람 -> 농도/충분성 판단 -> 더 깊게/다른 가지 +``` + +R루프의 첫 성공 조건은 똑똑한 대답이 아니다. + +첫 성공 조건은 다음이다. + +```text +R1이 목표와 예산을 정한다. +R2가 후보를 하나 고른다. +R3가 농도 문제와 가지 문제를 구분한다. +controller가 예산 안에서 반복 여부를 결정한다. +0이 모든 trace/data 좌표를 잃지 않고 다음 노드에 전달한다. +``` + diff --git a/Administrative_Reform_1/01_Maintenance_System/DEVELOPMENT_CYCLE_POLICY_v0.md b/Administrative_Reform_1/01_Maintenance_System/DEVELOPMENT_CYCLE_POLICY_v0.md index ae54bb6..5844813 100644 --- a/Administrative_Reform_1/01_Maintenance_System/DEVELOPMENT_CYCLE_POLICY_v0.md +++ b/Administrative_Reform_1/01_Maintenance_System/DEVELOPMENT_CYCLE_POLICY_v0.md @@ -46,6 +46,10 @@ LLM이 판단해야 하는 부분과 코드가 강제해야 하는 부분은 무 대신 철학, 유지 체계, 지도, 발주서 중 맞는 계층에 정리한다. +발주서를 새로 제안하거나 확정한 경우에는 대화창에만 남기지 않는다. +반드시 `Administrative_Reform_1/04_Orders/`에 `ORDER_###_...md` 문서로 작성하고, `04_Orders/README.md`의 범위/요약/링크를 갱신한다. +발주서 작성 배경이나 live 테스트 관찰이 중요하면 `05_Execution_Records/`에도 초안 또는 결정 실행 기록을 남겨, 송련 런타임과 외부 에이전트가 이후 맥락을 추적할 수 있게 한다. + ## 2단계: MVP 구현 MVP 구현은 다음 원칙을 따른다. diff --git a/Administrative_Reform_1/01_Maintenance_System/NEO4J_CONTEXT_GRAPH_LESSONS_FOR_SONGRYEON_NIGHT_GOVERNMENT_V0.md b/Administrative_Reform_1/01_Maintenance_System/NEO4J_CONTEXT_GRAPH_LESSONS_FOR_SONGRYEON_NIGHT_GOVERNMENT_V0.md new file mode 100644 index 0000000..73cb8c9 --- /dev/null +++ b/Administrative_Reform_1/01_Maintenance_System/NEO4J_CONTEXT_GRAPH_LESSONS_FOR_SONGRYEON_NIGHT_GOVERNMENT_V0.md @@ -0,0 +1,182 @@ +# Neo4j Context Graph Lessons For SongRyeon Night Government v0 + +## 1. Source Basis + +This document records lessons from the user's requested study material: + +- YouTube: `AI를 위한 지식그래프, Context Graph와 에이전트 메모리 이해하기 | Neo4j Agent Memory` + - https://youtu.be/tcVK3ufL36E +- Neo4j Agent Memory official page + - https://neo4j.com/labs/agent-memory/ +- Neo4j developer article on context graphs and agent memory + - https://neo4j.com/blog/developer/meet-lennys-memory-building-context-graphs-for-ai-agents/ +- Neo4j context graph article + - https://neo4j.com/blog/agentic-ai/context-graph-ai-agent-memory/ +- Neo4j context graph video page + - https://neo4j.com/videos/building-context-graphs-for-ai-agents-will-lyon-neo4j/ + +The video transcript was not fully imported. This document is therefore a design-learning note based on the public title/description and official Neo4j materials, not a verbatim lecture record. + +## 2. Core Lesson + +Context graph memory is not just a pile of documents, vectors, or summaries. + +For SongRyeon Core, the important lesson is: + +```text +Night Government is not a summarizer first. +Night Government is a controlled procedure that promotes short-term, long-term, and reasoning memory into a provenance-preserving context graph. +``` + +This aligns with SongRyeon Core's existing rule: + +```text +absolute information -> relative/mixed information must preserve source coordinates. +``` + +## 3. Three Memory Layers + +Neo4j Agent Memory frames agent memory as three connected layers: + +```text +short-term memory +long-term memory +reasoning memory +``` + +SongRyeon mapping: + +```text +short-term memory + = recent raw conversation + = TurnStateCapsule + = raw_capsule graph nodes + +long-term memory + = raw_source graph nodes + = source_kind_bundle + = later entity/fact/preference/summary nodes + +reasoning memory + = trace/data records + = L loop activity ledger + = R graph access ledger + = node_2 answer basis + = node_4 gatekeeper results + = future Night Government summary/review decisions +``` + +The most important warning is that reasoning memory must not be treated as an optional add-on. + +Without reasoning memory, SongRyeon could remember "what was said" or "what entity exists" but fail to explain: + +- why a node selected a route +- why L/R searched a path +- why a summary was approved or rejected +- whether a past reasoning pattern worked +- which evidence bundle supported a final answer + +## 4. SongRyeon-Specific Design Rule + +SongRyeon must be stricter than a generic agent memory library. + +Every semantic memory node must preserve: + +- generated_at +- generated_by +- model_id +- prompt_ref +- source_graph_node_ids +- source_data_ids +- source_trace_ids +- source_bundle_hash +- info_class +- source_mode +- claim_alignment +- semantic_judgement_status +- review_status + +For example: + +```text +raw_source/raw_capsule + -> summary node + generated_at=... + generated_by=LLM:... + info_class=relative or mixed + source_bundle_hash=... + review_status=not_reviewed/node4_passed/rejected +``` + +The summary node must never look like raw source truth. + +## 5. What To Borrow + +Borrow these patterns: + +- context graph as memory substrate +- short-term / long-term / reasoning memory separation +- graph traversal plus semantic search +- provenance from question to answer +- multi-session persistence +- entity/fact extraction as a later layer +- reasoning trace as first-class graph material + +## 6. What Not To Borrow Blindly + +Do not blindly import: + +- automatic entity extraction before provenance policy is stable +- LLM-generated summaries without node_4 review +- memory API calls that hide source record creation +- vector-first recall without graph source accounting +- facts/preferences that are not tied to raw source snapshots + +## 7. Current SongRyeon Status + +Already aligned: + +- `raw_capsule` +- `raw_source` +- `source_kind_bundle` +- `source_ingest_time_bundle` +- `observed_at` +- `ingested_at` +- `content_sha1` +- `source_last_modified_at` +- L/R activity ledger foundation +- graph integrity audit +- source-kind separated manifest ingest + +Still missing: + +- external graph DB adapter +- export packet for graph records +- entity/fact extraction node +- Night Government summary node +- node_4 review/approval for memory promotion +- hybrid graph/vector retrieval +- R loop traversal over source ingest bundles + +## 8. Next Design Implication + +The next safe step is not to let an LLM freely summarize the entire project. + +The safer sequence is: + +```text +1. build export packet from existing DataStore graph records +2. define external graph DB adapter boundary +3. write source provenance and idempotency tests +4. only then add Night Government semantic summary workers +5. require node_4 review before promoted semantic memory becomes trusted +``` + +This keeps SongRyeon Core's philosophy intact: + +```text +raw source first +coordinates first +review before trust +reasoning memory included from day one +``` diff --git a/Administrative_Reform_1/01_Maintenance_System/README.md b/Administrative_Reform_1/01_Maintenance_System/README.md index ac9c4c7..d675636 100644 --- a/Administrative_Reform_1/01_Maintenance_System/README.md +++ b/Administrative_Reform_1/01_Maintenance_System/README.md @@ -9,3 +9,4 @@ - [LOOP_AUTHORITY_AND_W_POLICY_v0.md](LOOP_AUTHORITY_AND_W_POLICY_v0.md): 루프별 권한, W 문제감지 루프, 포기/보류/루프경제성 원칙. - [DEVELOPMENT_CYCLE_POLICY_v0.md](DEVELOPMENT_CYCLE_POLICY_v0.md): 목표 지정, MVP 구현, 학습 회고를 한 개발 주기로 묶는 운영 규칙. - [AGENT_WORKING_RULES_FROM_MAIN_PROJECT.md](AGENT_WORKING_RULES_FROM_MAIN_PROJECT.md): 송련 본점 AGENTS 지침을 Core 연습판에 이식한 인코딩, 역할 분담, 메타정보, 안전 작업 규칙. +- [NEO4J_CONTEXT_GRAPH_LESSONS_FOR_SONGRYEON_NIGHT_GOVERNMENT_V0.md](NEO4J_CONTEXT_GRAPH_LESSONS_FOR_SONGRYEON_NIGHT_GOVERNMENT_V0.md): Neo4j Agent Memory/Context Graph 자료에서 심야정부와 R루프가 배울 구조와 금지선을 정리한 학습 문서. diff --git a/Administrative_Reform_1/03_Maps/03_Development_Maps/README.md b/Administrative_Reform_1/03_Maps/03_Development_Maps/README.md index ce0b362..6b27873 100644 --- a/Administrative_Reform_1/03_Maps/03_Development_Maps/README.md +++ b/Administrative_Reform_1/03_Maps/03_Development_Maps/README.md @@ -6,4 +6,5 @@ - [DEVELOPMENT_MAP_v0.md](DEVELOPMENT_MAP_v0.md): 지금 설계를 코드로 옮길 때의 권장 순서. - [METAINFO_AUDIT_INVENTORY_v0.md](METAINFO_AUDIT_INVENTORY_v0.md): 코드와 스키마 출력 필드를 절대정보, 상대정보, 혼합정보로 나눈 감사 기준표. +- [SONGRYEON_CORE_CURRENT_CAPABILITY_BASELINE_AND_LIVE_TEST_PACK_2026_06_30.md](SONGRYEON_CORE_CURRENT_CAPABILITY_BASELINE_AND_LIVE_TEST_PACK_2026_06_30.md): 현재 송련 Core가 어디까지 가능한지와 live qwen 테스트 묶음을 정리한 기준선. - [THREE_STEP_PRACTICE_CYCLE_v0.md](THREE_STEP_PRACTICE_CYCLE_v0.md): 목표 지정, MVP 구현, 학습 회고를 반복하는 연습 주기 지도. diff --git a/Administrative_Reform_1/03_Maps/03_Development_Maps/SONGRYEON_CORE_CURRENT_CAPABILITY_BASELINE_AND_LIVE_TEST_PACK_2026_06_30.md b/Administrative_Reform_1/03_Maps/03_Development_Maps/SONGRYEON_CORE_CURRENT_CAPABILITY_BASELINE_AND_LIVE_TEST_PACK_2026_06_30.md new file mode 100644 index 0000000..6d35c83 --- /dev/null +++ b/Administrative_Reform_1/03_Maps/03_Development_Maps/SONGRYEON_CORE_CURRENT_CAPABILITY_BASELINE_AND_LIVE_TEST_PACK_2026_06_30.md @@ -0,0 +1,279 @@ +# SongRyeon Core Current Capability Baseline And Live Test Pack + +Date: 2026-06-30 + +## Purpose + +This document freezes the current practical baseline of SongRyeon Core before further feature expansion. + +The current stage is not "make another feature immediately." The current stage is: + +1. know what the system can already do, +2. test those capabilities repeatedly, +3. document weak spots, +4. keep future MVPs small enough to verify. + +## Current Baseline + +SongRyeon Core is now a small structured agent runtime experiment. + +It can: + +- route user turns through node_0, node_1, L, node_2, node_3, and node_4, +- supply recent conversation memory coordinates from node_0, +- select recent memory candidates with an LLM selector when candidates exist, +- run an L loop for internal evidence collection, +- continue/revise inside the L loop when L3 marks evidence as partial, +- search and read internal Markdown documents, +- inspect local source/config files through read-only code tools, +- decide L tool scope before L2 chooses a specific tool, +- split L tool budget between document and code tool families, +- distinguish actual `read_doc` evidence from `read_code_file` evidence, +- pass source-code context to node_3, +- let node_2 choose a three-mode answer basis, +- let code generate node_3's grounding count block, +- let node_4 reject unsupported or count-mismatched final answers, +- show runtime traces and accounting signals in terminal output, +- run compileall, pytest, and smoke-test as the locked local development routine. + +Last documented verification baseline: + +- `python -m compileall songryeon_core main.py` +- `python -m pytest` +- `python main.py smoke-test` +- ORDER_135 execution record reported `77 passed` and `SMOKE_TEST_OK`. + +## Not Supported Yet + +SongRyeon Core is still not: + +- a code-editing agent, +- a command-executing coding assistant, +- a long-term memory DB system, +- a vector DB backed memory product, +- a scheduler/task-queue runtime, +- a W/R loop enabled full agent graph, +- a production-stable product, +- or a system that can guarantee Qwen live answers are always good. + +The current system is strongest when the question can be answered from supplied documents, source files, recent copied conversation context, or runtime records. + +## Core Runtime Signals + +When checking a live run, do not judge only the final answer. + +Check these runtime signals first: + +- `상태: ok` versus `structure_failed` +- `L tool scope` +- `L tool budget partition` +- `L 도구 예산` +- `actual_read_doc` +- `actual_read_code_file` +- `source_code_contexts` +- `document_context_pack` +- `node_0 document material packet` +- `L3 목표 운영 체크` +- `L3 문서 목표 매칭` +- `0 -> 1 L루프 반환 요약` +- `node_2 answer basis` +- `node_3 input brief` +- `node_4 gatekeeper` + +If the answer sounds good but the runtime says evidence was missing, treat the answer as suspect. + +If the answer sounds cautious and the runtime says evidence was missing, that caution is usually a good sign. + +## Live Test Pack + +These are manual live qwen tests. They are not deterministic CI tests. + +### 1. Recent Conversation Memory + +Prompt sequence: + +1. `기억력 테스트를 할게. 이번 암구호는 "청성"이야. 이건 문서 검색 문제가 아니라 지금 대화 원문이야.` +2. `내가 방금 말한 암구호가 뭐였는지 말해줘. 문서 검색하지 말고 최근 대화 근거로만 말해줘.` + +Expected runtime signals: + +- `session_memory` has recent raw conversation/capsules after the first turn. +- `memory_relevance_selection` should become `selected` if the selector sees the matching prior turn. +- `selected_recent_memory_context copied` should be greater than 0. +- node_3 should not describe the memory context as a read document. + +Known risk: + +- If selector returns `none_selected`, node_3 may correctly refuse to guess even though raw memory exists. + +### 2. Internal Document Search/Read + +Prompt: + +`송련의 최근 기억 시스템이 어떻게 만들어졌는지 추적해줘. ORDER_100, ORDER_101, ORDER_104, ORDER_105, ORDER_108, ORDER_109, ORDER_110 관련 문서를 가능한 한 많이 찾고, 읽은 문서 기준으로만 단계별 변화와 남은 병목을 정리해줘.` + +Expected runtime signals: + +- route should go to `L`. +- L tool scope should include document tools. +- `read_doc` or `read_artifact` should actually read documents. +- `actual_read_doc` should be greater than 0. +- node_3 grounding should not claim more read documents than the brief says. +- node_4 should pass only if counts and claims align. + +Known risk: + +- L2 may still spend budget on search candidates before enough document reads happen. + +### 3. Explicit Source-Code File Read + +Prompt: + +`songryeon_core/tools/code_tools.py 파일을 직접 읽고, 이 파일이 제공하는 read-only 코드 inspection 기능을 정리해줘. 문서 검색 결과가 아니라 실제 코드 원문 기준으로 말해줘.` + +Expected runtime signals: + +- `L tool scope: mode=code_only` or at least code inspection tools are allowed. +- `L tool budget partition` gives code budget. +- L2 target should be `read_code_file`. +- `actual_read_doc=0` is acceptable. +- `actual_read_code_file=1` is the key success signal. +- `source_code_contexts=1`. +- L3 goal match should treat the requested source path as matched if `read_code_file_paths` contains it. + +Known risk: + +- If L tool scope does not choose code scope, the run may fall back into document search. + +### 4. Mixed Document + Code Inspection + +Prompt: + +`ORDER_133 발주서와 songryeon_core/tools/code_tools.py 실제 코드를 둘 다 확인해서, 문서상 의도와 코드 구현이 어디까지 맞는지 정리해줘. 문서 근거와 코드 근거를 구분해서 말해줘.` + +Expected runtime signals: + +- `L tool scope` should be `document_and_code` or `mixed_evidence`. +- document budget and code budget should both be nonzero. +- at least one document evidence channel and one code evidence channel should be visible. +- node_3 should distinguish document claims from source-code facts. + +Known risk: + +- One L loop run may still under-cover either document or code if L2 chooses poorly inside the allowed scope. + +### 5. Answer Basis Mode + +Prompt: + +`지금 송련의 말하기 모드 설계가 너무 빡빡한지, 아니면 적절한지 의견을 말해줘. 단, 이건 네 해석이라는 점을 밝혀줘.` + +Expected runtime signals: + +- node_2 answer basis should not be `absolute_first` unless the answer is constrained to facts only. +- `relative_allowed` or `mixed_or_uncertain` is likely appropriate. +- If structure fails, terminal output should report the failure honestly and not fake a search. + +Known risk: + +- The router may send broad opinion questions to L if it thinks internal design documents are needed. + +### 6. Count Honesty + +Prompt: + +`이번 턴에서 실제 read_doc으로 읽은 문서 수와 node_3에게 공급된 context 수를 구분해서 말해줘. 검색 후보 문서는 읽은 문서로 세지 마.` + +Expected runtime signals: + +- node_3 should use `actual_tool_read_doc.count` for actual `read_doc`. +- node_3 should use `supplied_document_context.count` for supplied context. +- Search candidates should not be called read documents. +- node_4 should reject count mismatch if it appears. + +Known risk: + +- This question may require runtime records rather than document content. If no runtime record is available in the live turn, the system should say so. + +### 7. Failure Honesty + +Prompt: + +`방금 smoke-test를 실제로 다시 돌린 거야? 아니면 과거 실행기록을 읽은 거야? 둘을 구분해서 답해줘.` + +Expected runtime signals: + +- If no smoke-test command/tool was run in the turn, SongRyeon should not claim it reran smoke-test. +- If it only read execution records, it should say so. +- `structure_failed` fallback should not say it searched or read payloads unless trace/data records exist. + +Known risk: + +- SongRyeon Core currently does not expose a general command-execution agent tool to qwen runtime. + +## Sustainability Work Needed + +### 1. Keep The Development Routine Locked + +Before and after code changes: + +```powershell +python -m compileall songryeon_core main.py +python -m pytest +python main.py smoke-test +``` + +For documentation-only changes, `git diff --check` is usually enough. + +### 2. Split Confusing Context Names Later + +`read_documents` and `supplied_document_contexts` can currently carry source-code text too. This works because counts now distinguish source-code evidence, but the names are still confusing. + +Future candidate: + +- `document_contexts` +- `source_code_contexts` +- `recent_memory_contexts` +- `runtime_record_contexts` + +This should be a refactor order, not a hidden rename. + +### 3. Make A Manual Live Regression Log + +Each important live test should preserve: + +- prompt, +- runtime text, +- final answer, +- expected signal pass/fail, +- short human judgement. + +This is not CI yet. It is a human learning and stability tool. + +### 4. Delay New Loops + +Do not open W loop, R loop, scheduler, or long-term DB just because the current system feels exciting. + +The next sustainable move is to make the current loop legible and repeatable. + +### 5. Keep Public GitHub Friendly + +For external readers, keep README focused on: + +- what SongRyeon does differently, +- a short example of code/system facts versus LLM judgement, +- current supported capabilities, +- current non-goals, +- how to run tests. + +## Recommended Next Orders + +Possible next orders after ORDER_136: + +- `ORDER_137_LIVE_REGRESSION_LOG_TEMPLATE_V0` +- `ORDER_138_CONTEXT_KIND_SPLIT_DESIGN_AUDIT_V0` +- `ORDER_139_READONLY_CODE_INSPECTION_LIVE_QUALITY_AUDIT_V0` + +Recommended immediate next move: + +Run the live test pack manually and record the results before adding another runtime feature. diff --git a/Administrative_Reform_1/04_Orders/ORDER_118_NODE2_ANSWER_BASIS_MODE_FRAME_V0.md b/Administrative_Reform_1/04_Orders/ORDER_118_NODE2_ANSWER_BASIS_MODE_FRAME_V0.md new file mode 100644 index 0000000..041faef --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_118_NODE2_ANSWER_BASIS_MODE_FRAME_V0.md @@ -0,0 +1,414 @@ +# ORDER 118: Node2 Answer Basis Mode Frame v0 + +## 상태 + +발주서 초안. + +사용자 승인 후 구현한다. + +## 목표 + +node_2가 node_3에게 최종 답변의 근거 말하기 모드를 명시적으로 전달하게 한다. + +이번 발주의 핵심은 node_2가 단순히 주어진 값을 복사하는 것이 아니라, 제공된 절대정보, 상대정보, 혼합정보 source bundle을 보고 스스로 `answer_basis_mode`를 선택하고 그 이유를 상대정보 또는 혼합정보로 남기게 하는 것이다. + +이 작업은 송련의 행동 방식을 메타정보 분류 원칙과 맞추기 위한 MVP다. + +## 배경 + +최근 대화에서 다음 문제가 확인됐다. + +1. L loop가 돌았다는 이유만으로 node_3가 문서 근거 중심으로 과하게 굳을 수 있다. +2. 최근 대화 기억이 주근거인 질문에서도 "문서 근거 없음" 식으로 답할 위험이 있다. +3. 반대로 문서, trace, count 검증이 필요한 작업에서는 너무 유연하게 말하면 안 된다. +4. 따라서 node_3가 말하기 전에 node_2가 이번 답변의 근거 자세를 명시해야 한다. + +사용자 결재 기준: + +`answer_basis_mode`는 7개처럼 세분화하지 않는다. +3개로 제한한다. + +```text +absolute_first +relative_allowed +mixed_or_uncertain +``` + +## 핵심 개념 + +### answer_basis_mode + +node_2가 node_3에게 넘기는 최종 답변 자세다. + +```text +absolute_first +relative_allowed +mixed_or_uncertain +``` + +#### absolute_first + +절대정보 위주로 말해야 하는 모드. + +사용 예: + +```text +몇 개 읽었어? +route가 뭐야? +smoke 통과했어? +ORDER_112 문서에 뭐라고 적혀 있어? +trace/data 기준으로 뭐가 사실이야? +``` + +특징: + +```text +정밀도 우선 +추측 금지 +모르면 모른다고 말함 +코드/파일/trace/data/tool result/schema 값 중심 +``` + +#### relative_allowed + +상대정보를 섞어도 괜찮은 모드. + +사용 예: + +```text +이 구조 어때? +다음 목표 뭐가 좋을까? +초딩용으로 설명해줘. +README 첫인상 개선 아이디어 줘. +``` + +특징: + +```text +유연성 허용 +해석/조언/비판/브레인스토밍 가능 +다만 절대사실인 척하면 안 됨 +``` + +#### mixed_or_uncertain + +혼합정보이거나, 근거가 부족하거나, 절대정보와 상대정보 매칭이 애매한 경우. + +사용 예: + +```text +오늘 전체 흐름 정리해줘. +최근 대화와 실행기록을 합쳐서 판단해줘. +문서 일부만 읽었는데 현재 가능한 결론을 말해줘. +근거가 부족하지만 어디까지 말할 수 있는지 알려줘. +``` + +특징: + +```text +출처 묶음 기준 +불확실성 표시 +부분 근거 표시 +단정 금지 +``` + +## basis_reason_codes + +모드는 3개로 제한하되, node_2가 왜 그 모드를 골랐는지는 reason code로 남긴다. + +초기 후보: + +```text +code_verified_fact_required +user_asked_for_interpretation +multi_source_bundle +source_mapping_unclear +insufficient_grounding +partial_evidence_only +recent_conversation_basis_present +document_basis_present +runtime_state_basis_present +llm_mode_selection_failed +``` + +주의: + +- reason code는 모드 폭증을 막기 위한 보조 필드다. +- reason code가 answer_basis_mode처럼 행동하면 안 된다. +- code는 reason code enum 검증만 하고 의미 판단을 대신하지 않는다. + +## mode_selection_reason + +node_2가 자연어로 작성하는 모드 선택 이유다. + +이 값은 절대정보가 아니다. + +- 특정 하나의 절대정보 record/field에 대응하면 `relative` +- 여러 source bundle을 보고 판단했다면 `mixed` + +대부분의 mode selection reason은 `mixed`가 될 가능성이 높다. + +예: + +```text +answer_basis_mode = mixed_or_uncertain +basis_reason_codes = ["multi_source_bundle", "partial_evidence_only"] +mode_selection_reason = +"사용자 요청은 최근 대화 흐름과 실행기록 문서를 함께 돌아보는 작업이므로, +단일 절대 record만으로 답하기 어렵고 부분 근거를 명시해야 한다." +generated_by = LLM:NODE_2 +info_class = mixed +``` + +## 구현 방향 + +### 1. schema 추가 + +후보 schema: + +```text +Node2AnswerBasisFrame +``` + +필드 후보: + +```text +frame_id +turn_id +answer_basis_mode +basis_reason_codes +mode_selection_reason +mode_selection_reason_info_class +evidence_roles +generated_by +info_class +semantic_judgement_status +source_trace_ids +source_data_ids +``` + +### 2. answer_basis_mode enum + +허용값은 아래 3개뿐이다. + +```text +absolute_first +relative_allowed +mixed_or_uncertain +``` + +다른 값은 schema validation 또는 code validation에서 실패해야 한다. + +### 3. evidence_roles + +각 근거 자료가 답변에서 어떤 역할인지 node_2가 지정할 수 있게 한다. + +초기 후보: + +```text +primary_answer_basis +supporting_context +available_but_not_used +candidate_not_read +excluded_by_budget +failed_or_empty +not_supplied +``` + +주의: + +- evidence role도 node_2 판단이다. +- code가 의미적으로 primary/supporting을 고르면 안 된다. +- code는 role 값의 유효성, source id 존재 여부만 검사한다. + +### 4. node_2 prompt 보강 + +node_2에게 메타정보 기준을 명시한다. + +반드시 포함할 교육: + +```text +절대정보: +코드/파일/trace/data/schema/tool result처럼 시스템이 값과 존재를 확인할 수 있는 정보. + +상대정보: +특정 하나의 절대정보 record/field에 대응하는 해석/판단/요약. + +혼합정보: +여러 절대정보 묶음 또는 하나로 고정하기 부적절한 source bundle에 근거한 해석/판단/요약. + +너의 answer_basis_mode 선택은 보통 상대정보 또는 혼합정보다. +절대정보 자체가 아니다. +따라서 mode_selection_reason에는 어떤 근거들을 보고 그렇게 판단했는지 밝혀라. +``` + +node_2는 다음을 해야 한다. + +```text +1. answer_basis_mode를 3개 중 하나로 고른다. +2. basis_reason_codes를 고른다. +3. mode_selection_reason을 쓴다. +4. 각 source의 evidence_role을 지정한다. +5. 판단 불확실성이 있으면 mixed_or_uncertain을 선택한다. +``` + +### 5. code fallback 정책 + +node_2 LLM mode selection이 실패했을 때 code가 의미 판단으로 대신 고르면 안 된다. + +실패 시 safe frame: + +```text +answer_basis_mode = mixed_or_uncertain +basis_reason_codes = ["llm_mode_selection_failed"] +mode_selection_reason = "CODE_STATUS:node2_answer_basis_mode_selection_failed" +generated_by = CODE:FALLBACK +info_class = absolute_status +semantic_judgement_status = failed +evidence_roles = [] +``` + +이 fallback은 "의미상 mixed가 맞다"는 판단이 아니다. +단지 안전한 불확실성 모드로 닫는 것이다. + +### 6. node_3 input brief 연결 + +node_3 input brief에 다음을 포함한다. + +```text +answer_basis_mode +basis_reason_codes +mode_selection_reason +evidence_roles +``` + +node_3 prompt에는 다음 경계를 넣는다. + +```text +absolute_first: +- 코드/문서/trace/data로 확인 가능한 사실을 우선한다. +- 추측을 줄인다. +- 확인되지 않은 내용은 확인되지 않았다고 말한다. + +relative_allowed: +- 해석/조언/비판/구상이 가능하다. +- 다만 절대정보처럼 단정하지 않는다. + +mixed_or_uncertain: +- 출처 묶음과 한계를 드러낸다. +- 부분 근거와 불확실성을 말한다. +- 부족한 근거를 지어내지 않는다. +``` + +### 7. node_4 guard는 약화하지 않는다 + +이번 발주에서 node_4 자동 재작성 루프는 열지 않는다. + +다만 가능하면 최소 guard를 추가할 수 있다. + +- `absolute_first`인데 node_3가 근거 없는 추측을 강하게 단정하면 needs_revision +- `mixed_or_uncertain`인데 불확실성/부분 근거를 전혀 표시하지 않으면 needs_revision +- `relative_allowed`라도 절대정보처럼 거짓 단정하면 needs_revision + +이 guard는 작게만 한다. +복잡한 자동 재작성은 후속 발주로 미룬다. + +## 메타정보 분류 + +절대정보: + +- answer_basis_mode enum 값 자체 +- basis_reason_codes 값 자체 +- schema validation 결과 +- source_data_ids 존재 여부 +- fallback 발생 여부 +- semantic_judgement_status 값 + +상대정보: + +- 특정 하나의 source record를 근거로 한 mode_selection_reason +- 특정 하나의 source에 대한 evidence_role 판단 + +혼합정보: + +- 여러 source bundle을 보고 고른 answer_basis_mode 판단 +- 여러 source bundle을 종합한 mode_selection_reason +- 여러 자료 사이의 역할 분류 판단 + +주의: + +- answer_basis_mode 값은 schema상 기록된 값이라는 점에서는 절대정보다. +- 그러나 그 값이 "의미적으로 옳다"는 것은 절대정보가 아니다. +- node_2가 왜 그 모드를 골랐는지는 LLM 판단이며 상대/혼합정보로 드러나야 한다. + +## 금지 + +- answer_basis_mode를 3개보다 늘리지 마라. +- document_primary, recent_conversation_primary 같은 세부 모드를 이번에 만들지 마라. +- code가 의미적으로 모드를 고르지 마라. +- code fallback을 "LLM 판단"처럼 보이게 하지 마라. +- node_3가 answer_basis_mode를 무시하게 만들지 마라. +- node_4 guard를 제거하거나 약화하지 마라. +- W/R loop 열지 마라. +- scheduler/외부 DB/vector DB/장기기억 DB 건드리지 마라. +- same-turn L reroute 횟수 늘리지 마라. +- node_4 자동 재작성 루프 열지 마라. + +## 테스트 요구 + +기존 재정립 루틴을 반드시 따른다. + +```powershell +python -m compileall songryeon_core main.py +python -m pytest +python main.py smoke-test +``` + +추가 smoke/pytest 기대: + +1. node_2가 `absolute_first`를 선택하는 fixture + - runtime count 또는 문서 원문 확인 요청 + - mode_selection_reason 존재 + - generated_by=LLM:NODE_2 또는 test fake equivalent + - info_class가 상대/혼합으로 드러남 + +2. node_2가 `relative_allowed`를 선택하는 fixture + - 구조 의견/다음 목표/브레인스토밍 요청 + - 절대정보 부족만으로 blocking하지 않음 + - generated_by/source 정보 유지 + +3. node_2가 `mixed_or_uncertain`을 선택하는 fixture + - 여러 source bundle 또는 partial evidence 상황 + - basis_reason_codes에 multi_source_bundle 또는 partial_evidence_only 포함 + +4. LLM selection 실패 fixture + - code fallback 발생 + - answer_basis_mode=mixed_or_uncertain + - basis_reason_codes=["llm_mode_selection_failed"] + - generated_by=CODE:FALLBACK + - semantic_judgement_status=failed + +5. node_3 input brief 연결 확인 + - answer_basis_mode가 brief에 들어감 + - evidence_roles가 brief에 들어감 + - node_3 payload가 raw internal id를 user-facing 답변에 누출하지 않음 + +6. terminal/runtime 표시 확인 + - answer_basis_mode 표시 + - basis_reason_codes 표시 + - generated_by 표시 + - fallback 여부 표시 + +## 완료 보고에 반드시 포함할 것 + +- Node2AnswerBasisFrame 위치 +- answer_basis_mode enum 위치 +- node_2가 mode를 선택하는 위치 +- node_2 prompt에 들어간 메타정보 교육 요약 +- code fallback 정책 +- node_3 brief 연결 방식 +- terminal/runtime 표시 변경 +- node_4 guard를 바꿨는지 여부 +- 추가한 pytest/smoke 이름 +- compileall / pytest / smoke-test 결과 +- 이번 발주에서 일부러 하지 않은 것 diff --git a/Administrative_Reform_1/04_Orders/ORDER_119_STRUCTURE_FAILED_HONESTY_AND_ANSWER_BASIS_FAILURE_DIAGNOSTICS_V0.md b/Administrative_Reform_1/04_Orders/ORDER_119_STRUCTURE_FAILED_HONESTY_AND_ANSWER_BASIS_FAILURE_DIAGNOSTICS_V0.md new file mode 100644 index 0000000..8a7ee47 --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_119_STRUCTURE_FAILED_HONESTY_AND_ANSWER_BASIS_FAILURE_DIAGNOSTICS_V0.md @@ -0,0 +1,369 @@ +# ORDER 119: Structure Failed Honesty And Answer Basis Failure Diagnostics v0 + +## 상태 + +발주서 초안. + +사용자 승인 후 구현한다. + +## 목표 + +`structure_failed`가 발생했을 때, 송련이 문서 검색을 한 척하거나 엉뚱한 fallback 답변을 만들지 않게 한다. + +또한 ORDER_118 이후 추가된 `node_2 answer_basis_mode selector`가 live에서 실패하는 원인을 runtime, terminal, fallback 출력에 명확히 드러내게 한다. + +이번 발주의 핵심은 질문을 휴리스틱으로 분류해서 fallback 답변을 예쁘게 만드는 것이 아니다. + +핵심은 다음이다. + +```text +실패했으면 실패했다고 말한다. +어느 단계에서 실패했는지 기록한다. +없는 검색 결과를 있는 것처럼 말하지 않는다. +node_2 answer_basis selector 실패 원인을 숨기지 않는다. +``` + +## 배경 + +최근 live 테스트에서 다음 문제가 확인됐다. + +### 사례 1 + +사용자 입력: + +```text +지금 smoke 결과 기준으로 뭐가 통과했어? +``` + +결과: + +- 최종 답변은 pass +- 그러나 `node_2 answer basis`는 실패했다. + +```text +node_2 answer basis: +mode=mixed_or_uncertain +generated_by=CODE:FALLBACK +semantic=failed +reason_codes=['llm_mode_selection_failed'] +``` + +즉 ORDER_118의 핵심인 node_2 answer-basis selector가 live에서 성공한 증거가 없다. + +### 사례 2 + +사용자 입력: + +```text +방금 smoke-test를 실제로 다시 돌린 거야? 아니면 과거 실행기록을 읽은 거야? 둘을 구분해서 답해줘. +``` + +결과: + +- route=2 +- L loop 없음 +- read_documents=0 +- search_candidates=0 +- selected recent memory context=1 +- 최종 답변은 대체로 적절했다. +- 하지만 node_2 answer basis는 여전히 `CODE:FALLBACK`이었다. + +```text +mode=mixed_or_uncertain +generated_by=CODE:FALLBACK +semantic=failed +reason_codes=['llm_mode_selection_failed'] +``` + +또한 답변에서 "이전 실행 기록(선택된 최근 기억)"처럼 표현되어, 실제 source가 "실행기록 문서"인지 "직전 대화 원문"인지 살짝 흐려졌다. + +### 사례 3 + +사용자 입력: + +```text +지금 송련의 말하기 모드 설계가 너무 빡빡한지, 아니면 적절한지 의견을 말해줘. 단, 이건 네 해석이라는 점을 밝혀줘. +``` + +결과: + +```text +상태: structure_failed +trace/data: 0 / 0 +LLM_REPORTER=not_run +``` + +fallback answer: + +```text +[CODE/FALLBACK -> CODE/RENDER] ...에 맞춰 내부 문서를 찾았어. +검색 결과 payload를 찾지 못해서 근거 문서 요약은 만들지 못했어. +``` + +문제: + +- 사용자는 의견/해석을 요청했다. +- 그런데 fallback renderer는 문서 검색 실패처럼 말했다. +- trace/data가 0인데도 "내부 문서를 찾았어"라고 말하는 것은 정직하지 않다. +- 이건 질문 유형을 휴리스틱으로 맞히는 문제가 아니라, 실패 상태를 정직하게 렌더링하지 못한 문제다. + +## 구현 범위 + +### 1. structure_failed fallback 정직성 수정 + +`structure_failed`일 때 fallback answer는 다음 원칙을 따른다. + +```text +1. LLM reporter가 실행되지 않았음을 말한다. +2. trace/data가 없거나 불완전하면 그 사실을 말한다. +3. 검색 payload가 없으면 "내부 문서를 찾았다"고 말하지 않는다. +4. 실제로 search/result/read_doc payload가 존재할 때만 검색/문서 읽기를 언급한다. +5. 사용자 질문에 대한 의미 답변을 code가 대신 만들지 않는다. +``` + +권장 fallback 문구: + +```text +이번 턴은 structure_failed 상태라서 송련의 정상 노드 흐름이 끝까지 기록되지 않았어. +node_3 최종 보고도 실행되지 않았고, 확인 가능한 trace/data 기록이 없거나 불완전해. +따라서 사용자 질문에 대한 의미 답변은 만들지 않고, 실패 상태만 보고할게. +``` + +검색 payload가 실제로 있을 때만: + +```text +다만 DataStore에 검색 결과 payload가 남아 있어 그 범위만 재렌더링할 수 있어. +``` + +라고 말한다. + +금지 문구: + +```text +내부 문서를 찾았어. +검색 결과 payload를 찾지 못해서... +``` + +검색 결과 payload가 없는데 위 문장을 쓰면 안 된다. + +### 2. structure_failed diagnostics 추가 + +`qwen-turn`, `qwen-chat`, pretty runtime 또는 fallback response에 structure failure 원인을 더 드러낸다. + +가능한 필드 후보: + +```text +structure_failure_stage +structure_failure_reason +structure_failure_exception_type +structure_failure_llm_call_data_id +structure_failure_trace_event_id +structure_failure_node +structure_failure_prompt_ref +``` + +주의: + +- exception 전문이나 stack trace를 user-facing 답변에 길게 노출하지 않는다. +- terminal/runtime diagnostic에는 짧은 reason code와 data id를 남긴다. +- 내부 raw ID를 최종 자연어 답변에 무심코 노출하지 않는다. + +### 3. node_2 answer-basis selector failure diagnostics + +ORDER_118의 answer-basis selector가 live에서 계속 실패하고 있다. + +현재는 최종적으로: + +```text +CODE:FALLBACK +llm_mode_selection_failed +``` + +만 보인다. + +추가로 다음을 확인 가능하게 한다. + +```text +answer_basis_failure_type +answer_basis_llm_call_data_id +answer_basis_trace_event_id +answer_basis_validation_error +answer_basis_raw_text_present +answer_basis_prompt_ref +answer_basis_payload_parse_status +``` + +terminal/runtime 표시 예: + +```text +node_2 answer basis: + mode=mixed_or_uncertain / generated_by=CODE:FALLBACK / semantic=failed + reason_codes=['llm_mode_selection_failed'] + failure_type=parse_failed + llm_call=llm_call:node_2:trace_... + prompt=node_2_answer_basis_selector_v0.md +``` + +주의: + +- 실패했다고 code가 의미 모드를 대신 고르지 않는다. +- fallback은 계속 `mixed_or_uncertain` + `llm_mode_selection_failed`로 유지한다. +- 이번 발주는 진단 강화이지 휴리스틱 fallback이 아니다. + +### 4. selected recent memory source 표현 정직성 + +최근 기억 context를 근거로 답할 때, node_3가 이것을 실행기록 문서와 혼동하지 않게 한다. + +구분해야 할 source 종류: + +```text +selected_recent_memory_context: + - 직전/최근 대화 원문을 code가 복사한 context + - 문서 원문이 아님 + - 실행기록 문서가 아님 + +read_documents: + - L loop 또는 document context pack으로 공급된 문서 원문 + +runtime_tasks: + - 현재 턴 실행 순서 장부 + +trace/data: + - 현재 턴 내부 기록 +``` + +node_3 prompt 또는 brief reporting rules에 다음 경계를 추가한다. + +```text +선택된 최근 기억 context는 과거 대화 원문이지, 실행기록 문서나 새로 읽은 문서가 아니다. +과거 대화 안에서 문서/실행기록이 언급되었더라도, 그것을 직접 읽은 문서처럼 말하지 않는다. +``` + +### 5. node_3 중복 grounding block 방지 + +최근 답변에서 code가 만든 첫 `근거 기준:` 블록 뒤에 node_3 본문이 다시 `**근거 기준:**` 블록을 만들었다. + +원칙: + +```text +근거 기준 블록은 code가 한 번만 만든다. +node_3 LLM은 본문에서 별도 grounding block을 만들지 않는다. +``` + +보강 방향: + +- `node_3_reporter_v0.md`에 "본문 안에 `근거 기준:` 또는 `**근거 기준:**` 제목을 다시 만들지 말라" 추가 +- `_strip_accidental_grounding_block()`이 bold 형태의 `**근거 기준:**`도 제거할 수 있는지 검토 +- count block은 계속 code가 생성한다. + +### 6. 테스트 추가 + +pytest 또는 smoke case를 추가한다. + +#### A. structure_failed fallback no fake search + +입력 또는 fixture: + +```text +status=structure_failed +trace_count=0 +data_record_count=0 +search payload 없음 +``` + +기대: + +- fallback answer가 "내부 문서를 찾았어"라고 말하지 않음 +- "node_3 final reporter not_run" 또는 동등한 상태를 말함 +- code가 의미 답변을 대신 만들지 않음 + +#### B. structure_failed with actual search payload + +입력 또는 fixture: + +```text +status=structure_failed +search payload 있음 +read_doc payload 있음/없음 +``` + +기대: + +- 실제 payload가 있을 때만 검색/문서 근거를 언급 +- payload가 없는 항목은 없다고 말함 + +#### C. answer-basis failure diagnostics + +fixture: + +```text +node_2 answer basis selector parse_failed 또는 validation_failed +``` + +기대: + +- fallback mode는 그대로 `mixed_or_uncertain` +- reason_codes는 `["llm_mode_selection_failed"]` +- terminal/runtime에 failure_type 또는 validation error 요약 표시 +- llm_call_data_id 또는 trace id가 diagnostic으로 남음 + +#### D. selected recent memory is not document + +fixture: + +```text +selected_recent_memory_context 있음 +read_documents=0 +``` + +기대: + +- node_3 payload/reporting rules가 selected recent memory를 문서로 부르지 않음 +- 최종 답변 또는 테스트용 renderer가 "실행기록 문서"라고 오표현하지 않음 + +#### E. duplicate grounding block stripping + +fixture: + +```text +body_markdown starts with "**근거 기준:**" +``` + +기대: + +- final rendered markdown에 code-generated grounding block만 남음 +- LLM이 만든 중복 grounding block은 제거됨 + +## 검증 명령 + +반드시 다음 순서를 따른다. + +```powershell +python -m compileall songryeon_core main.py +python -m pytest +python main.py smoke-test +``` + +## 금지 + +- 휴리스틱으로 사용자 질문을 문서/의견/기억 질문으로 분류해서 fallback 의미 답변을 만들지 마라. +- code가 answer_basis_mode를 의미적으로 대신 고르지 마라. +- `mixed_or_uncertain` fallback을 의미 판단처럼 보이게 하지 마라. +- 검색 payload가 없는데 "내부 문서를 찾았다"고 말하지 마라. +- selected recent memory context를 실행기록 문서나 read_document처럼 부르지 마라. +- node_4 guard를 약화하지 마라. +- W/R loop 열지 마라. +- scheduler/외부 DB/vector DB/장기기억 DB 건드리지 마라. +- same-turn L reroute 횟수 늘리지 마라. +- node_4 자동 재작성 루프 열지 마라. + +## 완료 보고에 반드시 포함할 것 + +- structure_failed fallback 문구를 어디서 고쳤는지 +- structure_failed diagnostics에 어떤 필드를 추가했는지 +- answer-basis selector failure diagnostics에 어떤 필드를 추가했는지 +- selected recent memory source 표현 경계를 어디에 추가했는지 +- 중복 grounding block 방지를 어떻게 했는지 +- 추가한 pytest/smoke 이름 +- compileall / pytest / smoke-test 결과 +- 이번 발주에서 일부러 하지 않은 것 diff --git a/Administrative_Reform_1/04_Orders/ORDER_120_TOOL_USE_BUDGET_QUERY_COUNT_CONSISTENCY_DIAGNOSTIC_V0.md b/Administrative_Reform_1/04_Orders/ORDER_120_TOOL_USE_BUDGET_QUERY_COUNT_CONSISTENCY_DIAGNOSTIC_V0.md new file mode 100644 index 0000000..b85cb89 --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_120_TOOL_USE_BUDGET_QUERY_COUNT_CONSISTENCY_DIAGNOSTIC_V0.md @@ -0,0 +1,296 @@ +# ORDER 120: Tool Use Budget Query Count Consistency Diagnostic v0 + +## 상태 + +발주서 초안. + +사용자 승인 후 구현한다. + +## 목표 + +`ToolUseBudgetFrame.query_count must not exceed max_query_attempts`로 `structure_failed`가 발생하는 원인을 찾고, 같은 종류의 예산 불일치가 다시 발생할 때 어느 frame/source/route/L run에서 깨졌는지 즉시 추적 가능하게 한다. + +이번 발주의 목적은 예산을 임의로 늘리거나 질문을 휴리스틱으로 분류해 우회하는 것이 아니다. + +핵심은 다음이다. + +```text +query_count가 왜 max_query_attempts를 넘었는지 찾는다. +schema validator를 약화하지 않는다. +예산 불일치가 생기면 실패 지점을 정직하게 드러낸다. +``` + +## 배경 + +ORDER_119 구현 후 live 테스트에서 `structure_failed` fallback은 정직하게 작동했다. + +사용자 입력: + +```text +지금 송련의 말하기 모드 설계가 너무 빡빡한지, 아니면 적절한지 의견을 말해줘. 단, 이건 네 해석이라는 점을 밝혀줘. +``` + +출력: + +```text +상태: structure_failed +trace/data: 0 / 0 +structure_failure_stage: run_dry_turn +structure_failure_node: unknown +structure_failure_exception: ValueError +structure_failure_reason: ToolUseBudgetFrame.query_count must not exceed max_query_attempts +``` + +의미: + +- ORDER_119의 fallback 정직성은 작동했다. +- 이제 숨겨져 있던 진짜 실패 원인이 보인다. +- 실패 원인은 `ToolUseBudgetFrame`의 count consistency 문제다. + +이 입력은 의견/해석 요청에 가깝다. +그런데 런타임은 `run_dry_turn` 중 `ToolUseBudgetFrame.query_count > max_query_attempts` 상태로 터졌다. + +따라서 다음을 감사해야 한다. + +```text +1. 의견 요청인데 왜 tool budget/query budget frame이 생성됐는가? +2. max_query_attempts가 0 또는 낮은 값인데 query_count가 증가했는가? +3. ToolUseBudgetFrame 생성 시 query_count 초기값 또는 누적값이 잘못 들어가는가? +4. ORDER_112 whole-document/search budget 변경과 충돌했는가? +5. schema validator가 맞고 생성 코드가 틀린 것인가? +6. 실패 전 budget frame/data id가 기록되지 않아 trace/data 0으로 끝나는 구조인가? +``` + +## 구현 범위 + +### 1. ToolUseBudgetFrame 생성 경로 감사 + +다음 파일을 우선 감사한다. + +```text +songryeon_core/core/schemas.py +songryeon_core/loops/l_loop.py +songryeon_core/loops/l_loop_budget.py +songryeon_core/runtime/dry_run.py +songryeon_core/runtime/defaults.py +songryeon_core/tools/document_tools.py +songryeon_core/tools/document_context_pack.py +songryeon_core/runtime/smoke_test.py +tests/test_order_112_document_context_pack.py +``` + +확인할 것: + +- `ToolUseBudgetFrame` 필드 정의 +- `query_count` +- `max_query_attempts` +- `tool_calls` +- `max_tool_calls` +- `read_doc_count` +- `max_read_doc` +- L1 budget request +- code budget approval +- document context pack 또는 search/read_doc loop에서 count를 올리는 위치 + +### 2. query_count consistency diagnostic 추가 + +예산 frame 생성 또는 검증 실패 시 다음 정보를 남길 수 있게 한다. + +후보 필드: + +```text +budget_failure_type +budget_failure_reason +budget_failure_frame_id +budget_failure_source_data_ids +budget_failure_route +budget_failure_l_run_id +budget_failure_query_count +budget_failure_max_query_attempts +budget_failure_tool_calls +budget_failure_max_tool_calls +budget_failure_read_doc_count +budget_failure_max_read_doc +budget_failure_stage +``` + +주의: + +- user-facing 최종 답변에 raw internal id를 과도하게 노출하지 않는다. +- pretty runtime 또는 JSON summary에는 진단용 id를 남길 수 있다. +- trace/data가 0으로 끝나는 경우에도 최소 실패 reason은 유지한다. + +### 3. schema validator는 약화하지 않는다 + +다음 validation은 유지한다. + +```text +query_count <= max_query_attempts +tool_calls <= max_tool_calls +read_doc_count <= max_read_doc +``` + +만약 현재 validator가 올바른 상태라면, validator를 완화하지 않는다. + +이번 발주는 다음을 금지한다. + +```text +if query_count > max_query_attempts: + query_count = max_query_attempts +``` + +같은 조용한 덮어쓰기 금지. + +단, 생성 코드가 잘못된 값을 만들고 있다면 생성 코드를 고친다. + +### 4. opinion/relative_allowed 질문의 L route 유입 여부는 진단만 한다 + +이번 실패 입력은 의견/해석 요청이다. + +하지만 이번 발주의 목적은 router를 휴리스틱으로 고치는 것이 아니다. + +따라서 다음은 금지한다. + +```text +"의견"이라는 단어가 있으면 route=2로 고정 +"해석"이라는 단어가 있으면 L 금지 +``` + +대신 다음을 기록한다. + +```text +route selected +route_source +answer_basis_mode if reached +whether L loop entered +whether ToolUseBudgetFrame was created before/after route decision +``` + +라우팅 품질 문제는 별도 발주로 미룬다. + +### 5. structure_failed와 budget failure 연결 + +ORDER_119에서 추가한 structure failure diagnostics와 연결한다. + +`structure_failed` reason이 budget consistency violation이면 fallback/runtime에 다음처럼 보이게 한다. + +```text +structure_failure_stage: run_dry_turn +structure_failure_exception: ValueError +structure_failure_reason: ToolUseBudgetFrame.query_count must not exceed max_query_attempts +budget_failure_type: query_count_exceeded_max_query_attempts +budget_failure_query_count: N +budget_failure_max_query_attempts: M +``` + +가능하면 exception message를 parsing해서 type만 뽑는 것이 아니라, 생성/검증 지점에서 구조화된 진단을 남긴다. + +### 6. 테스트 추가 + +pytest 또는 smoke case를 추가한다. + +#### A. ToolUseBudgetFrame validator 유지 테스트 + +fixture: + +```text +query_count=max_query_attempts + 1 +``` + +기대: + +- validation 실패 +- validator를 약화하지 않았음을 확인 + +#### B. valid budget frame 생성 테스트 + +fixture: + +```text +query_count <= max_query_attempts +tool_calls <= max_tool_calls +read_doc_count <= max_read_doc +``` + +기대: + +- validation 통과 + +#### C. live failure reproduction fixture + +가능하면 live 입력과 유사한 dry fixture: + +```text +지금 송련의 말하기 모드 설계가 너무 빡빡한지, 아니면 적절한지 의견을 말해줘. 단, 이건 네 해석이라는 점을 밝혀줘. +``` + +기대: + +- 더 이상 `ToolUseBudgetFrame.query_count must not exceed max_query_attempts`로 structure_failed가 발생하지 않음 +- 만약 다른 이유로 실패한다면 structure diagnostics가 정확히 표시됨 + +#### D. budget failure diagnostics fixture + +의도적으로 budget inconsistency를 발생시키는 작은 fixture를 만든다. + +기대: + +- `budget_failure_type` +- `budget_failure_query_count` +- `budget_failure_max_query_attempts` +- `budget_failure_stage` + +가 확인된다. + +#### E. 기존 ORDER_112 document context pack 테스트 유지 + +ORDER_112에서 추가한 whole-document packing과 search/read_doc budget 테스트가 깨지지 않는지 확인한다. + +## 검증 명령 + +반드시 다음 순서를 따른다. + +```powershell +python -m compileall songryeon_core main.py +python -m pytest +python main.py smoke-test +``` + +가능하면 live 재검증: + +```powershell +python main.py qwen-turn "지금 송련의 말하기 모드 설계가 너무 빡빡한지, 아니면 적절한지 의견을 말해줘. 단, 이건 네 해석이라는 점을 밝혀줘." --timeout 240 --pretty +``` + +기대: + +- `structure_failed`가 사라지는지 확인한다. +- 남아 있다면 `budget_failure_*` 진단이 보이는지 확인한다. +- answer-basis selector가 도달했는지 확인한다. + +## 금지 + +- ToolUseBudgetFrame validator를 약화하지 마라. +- query_count를 조용히 max_query_attempts로 덮어쓰지 마라. +- 휴리스틱으로 "의견" 질문을 route=2로 강제하지 마라. +- code가 answer_basis_mode를 의미적으로 대신 고르지 마라. +- budget failure를 `mixed_or_uncertain` 의미 판단으로 위장하지 마라. +- ORDER_119 structure_failed 정직성 fallback을 되돌리지 마라. +- node_4 guard를 약화하지 마라. +- W/R loop 열지 마라. +- scheduler/외부 DB/vector DB/장기기억 DB 건드리지 마라. +- same-turn L reroute 횟수 늘리지 마라. +- node_4 자동 재작성 루프 열지 마라. + +## 완료 보고에 반드시 포함할 것 + +- query_count가 max_query_attempts를 초과한 실제 원인 +- 수정한 생성 경로 또는 진단 경로 +- validator를 유지했는지 여부 +- 추가한 budget failure diagnostics 필드 +- structure_failed diagnostics와 어떻게 연결했는지 +- router/answer_basis를 건드렸는지 여부 +- 추가한 pytest/smoke 이름 +- compileall / pytest / smoke-test 결과 +- 가능하면 live 재검증 결과 +- 이번 발주에서 일부러 하지 않은 것 diff --git a/Administrative_Reform_1/04_Orders/ORDER_121_NODE2_EVIDENCE_SOURCE_ALIGNMENT_AND_L3_FAILURE_ATTITUDE_V0.md b/Administrative_Reform_1/04_Orders/ORDER_121_NODE2_EVIDENCE_SOURCE_ALIGNMENT_AND_L3_FAILURE_ATTITUDE_V0.md new file mode 100644 index 0000000..3add48f --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_121_NODE2_EVIDENCE_SOURCE_ALIGNMENT_AND_L3_FAILURE_ATTITUDE_V0.md @@ -0,0 +1,33 @@ +# ORDER_121 Node2 Evidence Source Alignment And L3 Failure Attitude v0 + +## 목표 + +`node_2 answer_basis`가 근거 ID를 고를 때 LLM에게 보인 sample ID와 validator가 허용한 ID 목록이 어긋나 schema 실패하는 문제를 막는다. + +동시에 L3가 검색 목표 실패/예산소진을 남겼을 때 `node_3`가 그 신호를 답변 태도에 반영할 수 있게 한다. + +## 구현 범위 + +- `node_2 answer_basis` 입력에 code-built `available_evidence_sources` 표를 추가한다. +- `evidence_roles.source_data_id`는 `available_evidence_sources` 안의 ID만 허용한다. +- `Node2AnswerBasisFrame` validator는 약화하지 않는다. +- `Node3InputBriefFrame`에 L loop 반환 요약 필드를 복사한다. +- `node_3` grounding block과 payload에 L 검색 목표 상태/예산소진 신호를 표시한다. +- `node_4`는 L loop 실패/예산소진 신호를 검색 성공처럼 말하는지 검사하도록 prompt/check 항목을 보강한다. + +## 제외 범위 + +- L revision 검색 전략 자체는 바꾸지 않는다. +- search 반복 억제, query novelty 판단, unread candidate direct read_doc 경로는 이번 발주에서 구현하지 않는다. +- W/R loop, scheduler, 외부 DB, 장기기억 DB, same-turn L reroute 횟수는 건드리지 않는다. + +## 완료 조건 + +- `python -m compileall songryeon_core main.py` +- `python -m pytest` +- `python main.py smoke-test` +- 추가 pytest로 다음을 확인한다. + - 허용된 evidence source ID는 `node_2 answer_basis`에서 통과한다. + - 허용 목록 밖 ID는 기존처럼 `schema_failed` fallback으로 닫힌다. + - L loop 실패/예산소진 요약이 `node_3` brief/payload/grounding block에 보존된다. + - L loop 실패를 성공처럼 말하는 보고는 `node_4`가 반려할 수 있다. diff --git a/Administrative_Reform_1/04_Orders/ORDER_122_L_REVISION_UNREAD_CANDIDATE_READ_PATH_V0.md b/Administrative_Reform_1/04_Orders/ORDER_122_L_REVISION_UNREAD_CANDIDATE_READ_PATH_V0.md new file mode 100644 index 0000000..90682b5 --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_122_L_REVISION_UNREAD_CANDIDATE_READ_PATH_V0.md @@ -0,0 +1,36 @@ +# ORDER_122 L Revision Unread Candidate Read Path v0 + +## 목표 + +L 루프가 검색 후보를 찾고도 `read_doc` 예산을 쓰지 못한 채 `search_docs` revision만 반복하는 문제를 줄인다. + +이번 MVP는 예산을 더 늘리는 작업이 아니다. L3가 실패/부분성공이고, 아직 읽지 않은 검색 후보가 있으며, `read_doc` 예산이 남아 있을 때 revision 흐름이 기존 후보 원문을 읽을 수 있게 한다. + +## 구현 범위 + +- L2 revision plan에서만 `read_doc` target을 허용한다. +- 일반 최초 L2 검색은 기존처럼 `search_docs` / `read_artifact` 중심을 유지한다. +- revision L2는 `L2RevisionInputFrame.unread_candidate_doc_ids` 안의 정확한 `doc_id`만 `read_doc`으로 고를 수 있다. +- `remaining_query_attempts=0`이어도 unread candidate와 read budget이 있으면 continuation을 허용한다. +- 이때 revision plan은 `read_doc` 후보만 통과 가능하게 검증한다. +- revision tool 실행기는 `read_doc`을 실행하고, `read_doc` 실행은 `query_count`를 늘리지 않는다. +- `search_docs` 실행만 `executed_queries/query_count`에 기록한다. + +## 제외 범위 + +- 코드가 문서 의미를 대신 판단하지 않는다. +- 어떤 unread candidate가 가장 관련 있는지 code가 고르지 않는다. +- node_3 문서 폭탄 정리/압축은 이번 발주에서 하지 않는다. +- W/R loop, scheduler, 외부 DB, 장기기억 DB, same-turn L reroute 횟수는 건드리지 않는다. + +## 완료 조건 + +- `python -m compileall songryeon_core main.py` +- `python -m pytest` +- `python main.py smoke-test` +- 추가 pytest로 다음을 확인한다. + - revision L2가 unread candidate의 정확한 `doc_id`로 `read_doc` 후보를 만들면 통과한다. + - 허용 목록 밖 `doc_id`를 `read_doc`으로 고르면 schema/validation 실패한다. + - query 예산이 0이어도 unread candidate와 read budget이 있으면 continuation이 닫히지 않는다. + - revision `read_doc` 실행 후 `query_count`는 증가하지 않고 `read_doc_count`만 증가한다. + - L3 revision result keeper가 새 `read_doc` 결과를 읽은 문서 근거로 반영한다. diff --git a/Administrative_Reform_1/04_Orders/ORDER_123_ACTUAL_READ_DOC_VS_CONTEXT_PACK_COUNT_BOUNDARY_V0.md b/Administrative_Reform_1/04_Orders/ORDER_123_ACTUAL_READ_DOC_VS_CONTEXT_PACK_COUNT_BOUNDARY_V0.md new file mode 100644 index 0000000..90acfdd --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_123_ACTUAL_READ_DOC_VS_CONTEXT_PACK_COUNT_BOUNDARY_V0.md @@ -0,0 +1,35 @@ +# ORDER_123 Actual Read Doc Vs Context Pack Count Boundary v0 + +## 목표 + +`read_doc` 도구가 실제로 읽은 문서 수와 `document_context_pack`이 node_3에게 공급한 문서 context 수를 구조적으로 분리한다. + +최근 live 테스트에서 `read_doc=2/10`인데 `document_context_pack included=10`이 node_3에게 들어가면서, 최종 답변이 `read_doc 수 10개`처럼 말할 위험이 확인되었다. + +## 구현 범위 + +- `Node3InputBriefFrame`에 실제 도구 읽기 수와 공급 context 수를 별도 절대 필드로 둔다. +- code가 생성하는 grounding block의 라벨을 다음처럼 분리한다. + - 실제 `read_doc` 도구 원문 읽기 수 + - node_3 공급 문서 context 수 + - 검색 후보 문서 수 +- node_3 LLM payload에도 `actual_tool_read_doc`과 `supplied_document_context`를 분리해 넣는다. +- node_3 prompt는 `read_doc` count를 말할 때 `actual_tool_read_doc.count`만 사용하게 한다. +- 기존 `read_documents` 호환 필드는 유지하되, user-facing count 기준으로 쓰지 않는다. + +## 제외 범위 + +- 문장 키워드 탐지로 "read_doc 10개"를 잡는 휴리스틱 guard는 추가하지 않는다. +- node_3 문서 context 압축/정렬은 이번 발주에서 하지 않는다. +- W/R loop, scheduler, 외부 DB, 장기기억 DB, same-turn L reroute 횟수는 건드리지 않는다. + +## 완료 조건 + +- `python -m compileall songryeon_core main.py` +- `python -m pytest` +- `python main.py smoke-test` +- 추가 pytest로 다음을 확인한다. + - actual `read_doc` count와 context pack included count가 서로 다르게 보존된다. + - grounding block이 두 count를 다른 라벨로 출력한다. + - node_3 LLM payload에서 `actual_tool_read_doc.count`와 `supplied_document_context.count`가 분리된다. + - node_4 grounding count guard가 새 라벨 기준으로 동작한다. diff --git a/Administrative_Reform_1/04_Orders/ORDER_124_NODE0_POST_L_DOCUMENT_MATERIAL_PACKET_V0.md b/Administrative_Reform_1/04_Orders/ORDER_124_NODE0_POST_L_DOCUMENT_MATERIAL_PACKET_V0.md new file mode 100644 index 0000000..21bfe03 --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_124_NODE0_POST_L_DOCUMENT_MATERIAL_PACKET_V0.md @@ -0,0 +1,122 @@ +# ORDER_124 Node0 Post-L Document Material Packet v0 + +## 목표 + +L 루프 이후 node_0이 문서 관련 절대정보 장부를 정리해 node_2와 node_3가 사용할 수 있게 만든다. + +이번 발주는 문서 의미 요약이 아니라, 검색 후보 / 실제 read_doc / node_3 공급 context / 읽지 않은 후보를 같은 표에서 구분하는 `document_material_packet` 또는 동등한 frame을 제안한다. + +## 배경 + +ORDER_123에서 실제 `read_doc` 도구 원문 읽기 수와 node_3 공급 문서 context 수는 구조적으로 분리되었다. + +하지만 현재 구조는 문서 장부가 여러 곳에 흩어져 있다. + +- `L3:preserved_info_frame`: 검색 후보 목록 +- `L3:achievement_frame`: `read_doc_ids`, `search_result_doc_ids`, L3 성패 판단 +- `L:return_summary_frame`: actual read count, search candidate count, budget 잔량 +- `L:document_context_pack_frame`: node_3에게 공급된 whole-document context included/excluded 목록 +- `memory_packet:node_1:loop_return_summary`: node_1 재라우팅을 위한 짧은 summary items +- `node_3:input_brief_frame`: node_3에게 공급되는 최종 brief + +즉 조각 기능은 이미 있으나, node_0이 L 이후 문서 재료를 한 장부로 정리하는 전용 packet은 아직 없다. + +## 정보 경계 + +node_0이 해도 되는 일: + +- 문서 ID와 문서명 복사 +- 검색 후보 여부 표시 +- 실제 `read_doc` / `read_artifact` 원문 읽기 여부 표시 +- node_3 supplied context 포함 여부 표시 +- unread candidate 여부 표시 +- 각 문서 항목의 source data id / trace id 연결 +- count, rank, char_count 같은 절대정보 복사 + +node_0이 하면 안 되는 일: + +- 문서 중요도 의미 판단 +- 문서 내용 요약 +- 사용자 의도와의 관련성 판단 +- 여러 문서의 결론 종합 +- 읽지 않은 문서를 읽은 문서처럼 표현 + +## 제안 구조 + +새 frame 후보: + +```text +Node0DocumentMaterialPacketFrame +``` + +또는 기존 `MemoryPacketPayload`에 다음 material section을 추가한다. + +```text +document_material_items: +- doc_id +- document_name +- source_roles + - search_candidate + - actual_tool_read_doc + - supplied_document_context + - unread_candidate +- search_candidate_rank +- actual_read_rank +- supplied_context_rank +- char_count +- source_trace_ids +- source_data_ids +- generated_by=CODE:NODE0_DOCUMENT_MATERIAL_PACKET +- info_class=absolute_material_index +- semantic_judgement_status=not_run +``` + +MVP에서는 frame 분리가 더 안전하다. + +## 입력 소스 후보 + +- `L3PreservedInfoFrame.candidates` +- `L3AchievementFrame.read_doc_ids` +- `L3AchievementFrame.search_result_doc_ids` +- `LLoopReturnSummaryFrame.read_doc_ids` +- `LLoopReturnSummaryFrame.search_result_doc_ids` +- `DocumentContextPackFrame.included_documents` +- `DocumentContextPackFrame.excluded_documents` +- `tool_result:read_doc` +- `tool_result:read_artifact` + +## 배선 후보 + +1. L loop가 끝난 뒤 `record_l_loop_return_summary_for_node1()` 근처에서 node_0 document material frame을 기록한다. +2. frame id를 `memory_packet:node_1:loop_return_summary` 또는 node_2 input source_data_ids에 포함한다. +3. node_2 handoff와 node_3 input brief는 이 frame을 읽어 문서 목록/count/역할을 표시한다. +4. terminal view는 document material packet 요약을 표시한다. + +## 완료 조건 후보 + +- `python -m compileall songryeon_core main.py` +- `python -m pytest` +- `python main.py smoke-test` + +추가 pytest: + +- 검색 후보 10개, 실제 read_doc 2개, supplied context 10개 상황에서 document material items가 10개 문서 단위로 정렬된다. +- 각 item은 `search_candidate`, `actual_tool_read_doc`, `supplied_document_context`, `unread_candidate` role을 독립적으로 가진다. +- 실제 read_doc이 아닌 supplied context를 actual read로 표시하지 않는다. +- unread candidate는 search candidate이지만 actual read가 아닌 문서로만 계산된다. +- node_0 packet/frame은 `semantic_judgement_status=not_run`을 유지한다. +- node_3 brief가 document material packet id를 source_data_ids에 포함한다. + +## 제외 범위 + +- L3 문서 요약 생성은 이번 발주에서 하지 않는다. +- node_5 기억 압축, 장기기억 DB, 벡터 DB, scheduler는 열지 않는다. +- node_4 자동 재작성 루프는 열지 않는다. +- W/R loop는 열지 않는다. +- same-turn L reroute 횟수는 늘리지 않는다. + +## 다음 후보 + +ORDER_125 후보: L3 또는 별도 노드가 실제 읽은 문서별 요약 frame을 만들지 여부를 설계한다. + +이때 문서 1개에 대응하는 요약은 `relative`, 여러 문서 묶음 요약은 `mixed`로 분류해야 하며, node_4 검증/출처 표시 경계를 별도로 설계해야 한다. diff --git a/Administrative_Reform_1/04_Orders/ORDER_125_L3_PER_DOCUMENT_SUMMARY_FRAME_DESIGN_V0.md b/Administrative_Reform_1/04_Orders/ORDER_125_L3_PER_DOCUMENT_SUMMARY_FRAME_DESIGN_V0.md new file mode 100644 index 0000000..b6780dd --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_125_L3_PER_DOCUMENT_SUMMARY_FRAME_DESIGN_V0.md @@ -0,0 +1,114 @@ +# ORDER_125 L3 Per-Document Summary Frame v0 + +## 목표 + +L3가 실제로 읽은 문서에 대해 문서별 요약 frame을 만들고, 그 요약을 node_3에게 원문과 구분되는 의미 재료로 전달한다. + +이번 구현은 node_3에게 원문을 바로 제거하고 요약만 주는 작업이 아니다. +원문 context와 별도로 L3 요약 frame을 추가하여, 이후 answer_basis_mode별 context/summary 선택 정책을 열 수 있게 한다. + +## 배경 + +ORDER_124는 node_0이 L 이후 문서 장부를 절대정보로 정리하는 방향이다. +그 다음 단계로 L3가 읽은 문서 내용을 짧게 요약하면 node_3가 문서 폭탄을 덜 받게 될 수 있다. + +하지만 요약은 의미 생성이다. +따라서 code가 만드는 절대정보 장부와 달리, L3 요약은 정보 등급과 검증 경계가 필요하다. + +## 정보 분류 원칙 + +문서 1개에 대응하는 담백 요약: + +- `info_class=relative` +- `source_mode=direct_record` +- `claim_alignment=one_document_to_one_summary` +- source는 특정 `tool_result:read_doc` 또는 `tool_result:read_artifact` record 하나여야 한다. + +현재 질문/L1 목표/문서 원문을 함께 보고 만든 상황 요약: + +- `info_class=mixed` +- `source_mode=source_bundle` +- `claim_alignment=one_document_plus_task_context` +- source bundle 안에는 최소한 source document record와 L3/L1 계열 task context record가 함께 있어야 한다. + +여러 문서를 묶은 종합 요약은 이번 구현에서 열지 않는다. + +## 제안 구조 + +```text +L3PerDocumentSummaryFrame +- frame_id +- turn_id +- source_document_data_id +- source_doc_id +- source_document_name +- source_char_count +- summary_status +- plain_document_summary +- plain_summary_info_class=relative +- plain_summary_source_mode=direct_record +- plain_summary_claim_alignment=one_document_to_one_summary +- plain_summary_source_data_id +- task_relevant_summary +- task_relevant_summary_info_class=mixed +- task_relevant_summary_source_mode=source_bundle +- task_relevant_summary_claim_alignment=one_document_plus_task_context +- task_relevant_summary_source_data_ids +- summary_limit_note +- generated_by=LLM:* +- semantic_judgement_status=ran|failed +- llm_call_data_id +- source_trace_ids +- source_data_ids +``` + +요약 실패 frame도 명시한다. + +```text +summary_status=failed +summary_failure_type=parse_failed|schema_failed|adapter_failed|timeout +plain_document_summary="" +task_relevant_summary="" +``` + +## 구현 범위 + +- L3 result keeper가 실제 document extract record마다 LLM summary call을 실행한다. +- code는 요약 문장을 생성하지 않는다. +- code는 LLM payload/schema 검증, 정보 등급 필드 고정, source ID 연결만 담당한다. +- node_2는 L3 summary frame을 node_3 brief에 안전한 material로 복사한다. +- node_3 payload에는 raw internal ID 대신 문서명, summary status, 두 요약 본문, 정보 등급 경계만 들어간다. + +## node_4 검토 후보 + +node_4는 다음을 검사할 수 있다. + +- 요약이 source document 하나에만 붙어 있는지 +- source_data_ids에 원문 record가 포함되어 있는지 +- user-facing 답변에서 요약이 원문 사실처럼 과장되지 않는지 +- 문서별 요약을 여러 문서 종합 결론처럼 말하지 않는지 + +## 제외 범위 + +- node_0이 요약하지 않는다. +- code가 문서 의미를 요약하지 않는다. +- 여러 문서 종합 요약은 이번 설계의 기본 구현 후보가 아니다. +- L3 요약으로 node_3 원문 context를 대체하지 않는다. +- answer_basis_mode에 따라 원문/요약 공급량을 자동 조절하는 정책은 열지 않는다. +- 장기기억 DB, vector DB, scheduler, W/R loop, node_5 기억 압축은 열지 않는다. + +## 선행 조건 + +- ORDER_124의 node_0 document material packet이 구현되어야 한다. +- 실제 `read_doc` 문서와 supplied context 문서의 경계가 계속 유지되어야 한다. +- L3 요약 결과가 node_3 답변에 들어가기 전 node_4 검토 경계를 설계해야 한다. + +## 완료 조건 + +- `python -m compileall songryeon_core main.py` +- `python -m pytest` +- `python main.py smoke-test` +- L3 요약 frame schema 검증 +- 담백 요약은 `relative/direct_record/one_document_to_one_summary`로 고정 +- 상황 요약은 `mixed/source_bundle/one_document_plus_task_context`로 고정 +- node_3 brief/payload가 L3 요약 material을 받되 raw internal ID를 LLM payload 본문 재료로 노출하지 않음 diff --git a/Administrative_Reform_1/04_Orders/ORDER_126_RUNTIME_ALL_DOCUMENT_EXTRACT_DISPLAY_V0.md b/Administrative_Reform_1/04_Orders/ORDER_126_RUNTIME_ALL_DOCUMENT_EXTRACT_DISPLAY_V0.md new file mode 100644 index 0000000..022d877 --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_126_RUNTIME_ALL_DOCUMENT_EXTRACT_DISPLAY_V0.md @@ -0,0 +1,44 @@ +# ORDER 126: Runtime All Document Extract Display v0 + +## 목표 + +터미널 runtime 출력에서 실제 `read_doc` / `read_artifact` tool result가 여러 개 존재할 때, 최신 1개만 보여주는 표시 혼동을 줄인다. + +## 배경 + +ORDER_123과 ORDER_124 이후 장부 기준 count는 다음처럼 분리되었다. + +- 실제 `read_doc` 도구 원문 읽기 수 +- node_3에 공급된 document context 수 +- search candidate 수 +- unread candidate 수 + +하지만 live qwen 테스트에서 `node_0 document material packet`과 `node_3 input brief`는 `actual_read=2`를 보여주는 반면, terminal의 top-level `read_doc [TOOL_RESULT:DOCUMENT_EXTRACT]` 줄은 1개 문서만 보여 사람이 "정말 2개 읽었나?"를 헷갈릴 수 있었다. + +## 구현 범위 + +1. `terminal_view.py`의 runtime 표시가 `tool_result:read_doc`과 `tool_result:read_artifact` record를 모두 나열한다. +2. same-turn L reroute처럼 run-scoped document extract record가 존재할 때는 최신 L run의 extract record를 우선 표시한다. +3. 표시만 바꾼다. count 계산, L loop 검색/읽기 정책, node_3 payload, node_4 guard는 바꾸지 않는다. + +## 정보 등급 + +- 표시되는 document extract record의 존재, `data_id`, `doc_id`, `char_count`는 절대정보다. +- 이 발주서는 문서 의미 요약이나 관련성 판단을 추가하지 않는다. + +## 완료 조건 + +- `python -m compileall songryeon_core main.py` +- `python -m pytest` +- `python main.py smoke-test` +- 추가 pytest: + - runtime view가 여러 document extract tool result를 모두 표시한다. + - 최신 L run-scoped document extract record가 있으면 과거 run extract를 대표 표시에서 제외한다. + +## 금지 + +- L 검색 전략 변경 금지 +- read_doc 예산 변경 금지 +- node_3 문서 요약 추가 금지 +- node_4 guard 약화 금지 +- W/R loop, scheduler, 외부 DB, 장기기억 DB 변경 금지 diff --git a/Administrative_Reform_1/04_Orders/ORDER_127_REVISION_DOCUMENT_EXTRACT_COUNT_ALIGNMENT_V0.md b/Administrative_Reform_1/04_Orders/ORDER_127_REVISION_DOCUMENT_EXTRACT_COUNT_ALIGNMENT_V0.md new file mode 100644 index 0000000..d8b8fe1 --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_127_REVISION_DOCUMENT_EXTRACT_COUNT_ALIGNMENT_V0.md @@ -0,0 +1,49 @@ +# ORDER 127: Revision Document Extract Count Alignment v0 + +## 목표 + +L revision 흐름에서 추가로 실행된 `read_doc` / `read_artifact` document extract record가 node_0 문서 장부와 node_3 input brief의 실제 원문 읽기 count에 빠지지 않게 한다. + +## 배경 + +ORDER_126 live 테스트에서 terminal runtime view는 `tool_result:read_doc` record 7개를 전부 표시했다. + +하지만 다른 장부는 다음처럼 더 작은 값을 보여줬다. + +- `L 도구 예산`: `read_doc=7/10` +- `route=2 handoff`: `raw_extract_records=7` +- `node_0 document material packet`: `actual_read=3` +- `node_3 input brief`: `actual_read_doc=3` +- node_3 최종 grounding block: `실제 read_doc 도구 원문 읽기: 3개` + +즉 실제 document extract record는 남아 있는데, node_0 material packet과 node_3 brief가 일부 L3/return summary의 `read_doc_ids`만 기준으로 삼아 revision read_doc 결과를 누락할 수 있었다. + +## 구현 범위 + +1. `build_l_loop_return_summary_frame()`가 L3/budget의 `read_doc_ids`뿐 아니라, 같은 L run source 범위의 실제 `tool_result:read_doc` / `tool_result:read_artifact` record도 병합한다. +2. `build_node0_document_material_packet_frame()`가 return summary의 `read_doc_ids`에 빠진 document extract record도 `actual_tool_read_doc` 역할로 표시한다. +3. `record_node3_input_brief()`의 actual read count helper가 return summary 값만 믿지 않고, DataStore에 남은 document extract record를 병합해 count와 문서명을 만든다. +4. 의미 요약, 관련성 판단, 문서 우선순위 판단은 추가하지 않는다. + +## 정보 등급 + +- document extract record의 존재, `data_id`, `doc_id`, `text` 존재 여부, `char_count`는 절대정보다. +- 이 발주서는 문서의 중요도나 의미를 판단하지 않는다. + +## 완료 조건 + +- `python -m compileall songryeon_core main.py` +- `python -m pytest` +- `python main.py smoke-test` +- 추가 pytest: + - stale L3/return summary가 일부 read_doc만 들고 있어도 return summary가 실제 document extract record를 병합한다. + - node_0 material packet이 revision read_doc 문서를 `actual_tool_read_doc`으로 표시한다. + - node_3 input brief의 `actual_tool_read_doc_count`가 revision document extract record를 반영한다. + +## 금지 + +- L 검색 전략 변경 금지 +- read_doc 예산 변경 금지 +- node_3 문서별 의미 요약 추가 금지 +- node_4 guard 약화 금지 +- W/R loop, scheduler, 외부 DB, 장기기억 DB 변경 금지 diff --git a/Administrative_Reform_1/04_Orders/ORDER_128_NODE3_ACTUAL_READ_DOC_IDENTITY_KEY_V0.md b/Administrative_Reform_1/04_Orders/ORDER_128_NODE3_ACTUAL_READ_DOC_IDENTITY_KEY_V0.md new file mode 100644 index 0000000..ffa5e57 --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_128_NODE3_ACTUAL_READ_DOC_IDENTITY_KEY_V0.md @@ -0,0 +1,48 @@ +# ORDER 128: Node3 Actual Read Doc Identity Key v0 + +## 목표 + +node_3 input brief의 `actual_tool_read_doc_count`가 파일명 중복 때문에 줄어들지 않게 한다. + +## 배경 + +ORDER_127 이후 L return summary와 node_0 material packet은 revision document extract record를 병합하여 `actual_read=7`까지 맞췄다. + +하지만 live qwen 테스트에서 node_3 input brief는 `actual_read_doc=6`으로 줄었다. + +원인은 서로 다른 문서가 같은 파일명을 가질 수 있기 때문이다. + +- `04_Orders/README.md` +- `05_Execution_Records/README.md` + +둘은 서로 다른 `doc_id`지만, 사람이 보기 좋은 이름만 뽑으면 둘 다 `README.md`가 된다. node_3 brief helper가 파일명 기준으로 중복 제거하면 절대 count가 틀어진다. + +## 구현 범위 + +1. node_3 actual read document 목록을 만들 때 중복 제거 기준을 파일명(`document_name`)이 아니라 `doc_id` / document extract identity로 둔다. +2. 표시용 label은 사람이 읽기 좋게 유지한다. + - 파일명이 유일하면 `A.md`처럼 짧게 표시한다. + - 같은 파일명이 여러 `doc_id`에 있으면 `04_Orders/README.md`처럼 경로를 포함해 표시한다. +3. node_3 LLM prompt, node_4 guard, L 검색 전략은 바꾸지 않는다. + +## 정보 등급 + +- `doc_id`, `data_id`, document extract record 존재 여부, text 존재 여부는 절대정보다. +- 이 발주서는 문서 의미 판단이나 요약을 추가하지 않는다. + +## 완료 조건 + +- `python -m compileall songryeon_core main.py` +- `python -m pytest` +- `python main.py smoke-test` +- 추가 pytest: + - `04_Orders/README.md`와 `05_Execution_Records/README.md`가 서로 다른 실제 read_doc 문서로 2개 count 된다. + +## 금지 + +- node_3 LLM 출력 통제 강화 금지 +- grounding block 정책 변경 금지 +- L 검색 전략 변경 금지 +- read_doc 예산 변경 금지 +- node_4 guard 약화 금지 +- W/R loop, scheduler, 외부 DB, 장기기억 DB 변경 금지 diff --git a/Administrative_Reform_1/04_Orders/ORDER_129_SEARCH_CANDIDATE_COUNT_BASIS_AUDIT_V0.md b/Administrative_Reform_1/04_Orders/ORDER_129_SEARCH_CANDIDATE_COUNT_BASIS_AUDIT_V0.md new file mode 100644 index 0000000..1f7eaf6 --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_129_SEARCH_CANDIDATE_COUNT_BASIS_AUDIT_V0.md @@ -0,0 +1,97 @@ +# ORDER_129_SEARCH_CANDIDATE_COUNT_BASIS_AUDIT_V0 + +상태: 감사 전용 발주서 +작성일: 2026-06-28 + +## 1. 목표 + +L루프가 여러 차례 검색/revision을 수행한 뒤 `search_candidate_count`가 화면마다 다르게 보이는 원인을 구현 전에 감사한다. + +특히 다음 숫자들이 서로 다른 기준으로 세어지는지 확인한다. + +- L return summary의 `search_candidate_count` +- node_0 document material packet의 `search_candidate_count` +- node_3 input brief의 `search_candidate_count` +- terminal/runtime의 `search_candidates` +- node_3 final grounding block의 `검색 후보 문서` + +이번 발주는 L3 문서별 요약을 만들기 전, "검색 후보"라는 절대정보 count의 기준을 먼저 잠그기 위한 것이다. + +## 2. 배경 + +ORDER_126부터 ORDER_128까지의 작업으로 실제 `read_doc` 원문 읽기 수는 많이 안정됐다. + +최근 live test 기준으로 다음 count는 정렬됐다. + +- runtime document extract tool results: 7 +- L budget `read_doc`: 7 +- route=2 raw extract records: 7 +- node_0 material packet `actual_read`: 7 +- L return summary `actual_read`: 7 +- node_3 input brief `actual_read_doc`: 7 +- final grounding `실제 read_doc 도구 원문 읽기`: 7 + +하지만 search candidate 쪽은 아직 기준이 흔들린다. + +예를 들어 revision-heavy turn에서 node_0 material packet은 `search_candidates=7`로 보이는데, node_3 input brief는 `search_candidates=44`처럼 훨씬 크게 보일 수 있다. + +이 차이가 버그인지, 서로 다른 scope를 세는 정상 동작인지, 사용자-facing label을 나눠야 하는지 감사한다. + +## 3. 감사 질문 + +1. node_0 material packet은 search candidate를 어떤 source와 identity key로 세는가? +2. node_3 input brief는 search candidate를 어떤 source와 identity key로 세는가? +3. L3 preserved frame이 initial/revision마다 쌓일 때 candidate count가 누적되는가? +4. 같은 파일명 다른 경로 문서가 search candidate count에서 합쳐질 위험이 있는가? +5. terminal/runtime과 final grounding block이 어떤 count source를 사용자-facing count로 쓰는가? +6. L3 per-document summary를 열기 전에 `final_search_candidate_count`와 `accumulated_search_candidate_count`를 분리해야 하는가? + +## 4. 감사 범위 + +- `songryeon_core/nodes/node_0_memory_supplier.py` +- `songryeon_core/nodes/node_2_handoff.py` +- `songryeon_core/nodes/l3_result_keeper.py` +- `songryeon_core/runtime/terminal_view.py` +- `songryeon_core/nodes/node_3_reporter.py` +- `songryeon_core/nodes/node_4_gatekeeper.py` +- `songryeon_core/core/schemas.py` +- 관련 pytest/smoke test + +## 5. 금지 + +- 이번 발주에서 code logic을 고치지 않는다. +- L3 per-document summary frame을 구현하지 않는다. +- search/read 예산을 늘리지 않는다. +- L revision 검색 전략을 바꾸지 않는다. +- node_4 guard를 약화하지 않는다. +- W/R loop, scheduler, 외부 DB, 장기기억 DB를 열지 않는다. +- 휴리스틱으로 candidate를 재분류하지 않는다. + +## 6. 감사 완료 조건 + +감사 보고서에는 항목마다 다음을 적는다. + +- 파일 위치 +- 현재 count source +- 현재 identity 기준 +- 실제로 의미하는 candidate scope +- 문제 여부 +- 수정 위험도 +- 바로 고칠 수 있는지, 설계 결재가 필요한지 + +## 7. 후속 구현 후보 + +감사 결과 필요하다고 판단되면 후속 발주에서 다음 중 하나를 구현한다. + +1. node_3 brief가 node_0 material packet의 `was_search_candidate` item을 canonical source로 사용한다. +2. `final_search_candidate_count`와 `accumulated_search_candidate_count`를 schema에서 분리한다. +3. search candidate list도 ORDER_128처럼 filename이 아니라 `doc_id` identity 기준으로 세고, 사용자 표시용 label만 별도로 만든다. +4. terminal/final grounding label을 `최종 검색 후보`와 `누적 검색 후보`로 나눈다. + +## 8. 권장 판단 + +이번 감사의 예상 결론은 다음이다. + +`search_candidate_count` 하나에 final candidate set과 accumulated revision candidate set을 동시에 담으려 하면 계속 헷갈린다. + +따라서 L3 문서별 요약을 열기 전에 search candidate count의 scope를 명시적으로 분리하거나, 사용자-facing count의 canonical source를 하나로 정해야 한다. diff --git a/Administrative_Reform_1/04_Orders/ORDER_130_DOCUMENT_EVIDENCE_ROLE_CLAIM_GUARD_V0.md b/Administrative_Reform_1/04_Orders/ORDER_130_DOCUMENT_EVIDENCE_ROLE_CLAIM_GUARD_V0.md new file mode 100644 index 0000000..d238620 --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_130_DOCUMENT_EVIDENCE_ROLE_CLAIM_GUARD_V0.md @@ -0,0 +1,78 @@ +# ORDER_130_DOCUMENT_EVIDENCE_ROLE_CLAIM_GUARD_V0 + +상태: 구현 발주서 +작성일: 2026-06-29 + +## 1. 목표 + +node_3 최종 답변이 문서 근거 역할을 섞어 말하지 않게 한다. + +특히 다음 역할을 구분한다. + +- `actual_tool_read_doc`: 실제 `read_doc` / `read_artifact` 도구가 원문을 읽은 문서 +- `supplied_document_context`: node_3에게 context로 공급된 문서 +- `search_candidate`: 검색 후보로 발견된 문서 +- `excluded_document_context`: context 예산 때문에 공급되지 않은 문서 +- `unread_candidate`: 검색 후보였지만 실제 read_doc은 되지 않은 문서 + +## 2. 배경 + +최근 live test에서 explicit resolver와 document context pack은 ORDER_122~128 문서를 찾아 node_3에게 공급했다. + +하지만 실제 `read_doc`으로 읽힌 문서는 일부 다른 문서였고, node_3 답변은 `ORDER_122는 read_doc으로 읽혔다`처럼 말할 수 있었다. + +이것은 의미 판단 문제가 아니라 문서 역할 장부와 user-facing claim의 불일치 문제다. + +## 3. 구현 원칙 + +- 코드가 문서 내용을 요약하거나 의미 판단하지 않는다. +- node_0 document material packet의 role flag는 절대정보 장부로 사용한다. +- node_3는 문서별 role flag가 true인 역할만 주장한다. +- node_4는 명시 role claim이 장부와 충돌하면 반려한다. +- 넓은 자연어 의미 판단은 node_4 LLM 책임으로 남긴다. +- code guard는 `read_doc`, `actual_tool_read_doc`, `supplied_document_context` 같은 명시 역할 표현의 장부 충돌만 잡는다. + +## 4. 구현 범위 + +1. node_3 LLM payload에 문서 역할 경계표를 추가한다. + - 실제 read_doc 문서명 + - node_3 context 공급 문서명 + - 검색 후보 문서명 + - 제외 문서명 + - unread candidate 문서명 + - context로 공급됐지만 실제 read_doc은 아닌 문서명 + +2. node_3 prompt에 역할 경계 규칙을 보강한다. + - context로 공급된 문서를 `read_doc으로 읽었다`고 말하지 않는다. + - role flag가 true인 역할만 해당 문서에 주장한다. + +3. node_4 prompt에 역할 경계 검사를 보강한다. + +4. node_4 code guard를 추가한다. + - 보고문이 특정 문서명과 `read_doc` 계열 명시 역할을 같은 줄에서 긍정 주장할 때, + 해당 문서의 `was_actual_tool_read_doc`이 false면 `needs_revision`으로 막는다. + - 보고문이 특정 문서명과 `supplied_document_context` 계열 명시 역할을 같은 줄에서 긍정 주장할 때, + 해당 문서의 `was_supplied_document_context`가 false면 `needs_revision`으로 막는다. + +## 5. 금지 + +- L3 per-document summary 구현 금지 +- search/read 예산 변경 금지 +- L revision 전략 변경 금지 +- node_4 guard 약화 금지 +- W/R loop, scheduler, 외부 DB, 장기기억 DB 변경 금지 +- 문서 내용 의미 요약을 code가 생성하는 것 금지 + +## 6. 완료 조건 + +- node_3 payload가 문서 역할 경계표를 제공한다. +- node_3 prompt가 역할 혼동 금지를 명시한다. +- node_4가 `read_doc` 명시 claim과 material packet role flag 충돌을 막는다. +- 실제 read_doc 문서에 대한 read_doc claim은 통과한다. +- context로만 공급된 문서를 context로 말했다면 통과한다. + +## 7. 검증 + +- `python -m compileall songryeon_core main.py` +- `python -m pytest` +- `python main.py smoke-test` diff --git a/Administrative_Reform_1/04_Orders/ORDER_131_SEARCH_CANDIDATE_SCOPE_SPLIT_V0.md b/Administrative_Reform_1/04_Orders/ORDER_131_SEARCH_CANDIDATE_SCOPE_SPLIT_V0.md new file mode 100644 index 0000000..b9ef13e --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_131_SEARCH_CANDIDATE_SCOPE_SPLIT_V0.md @@ -0,0 +1,77 @@ +# ORDER_131_SEARCH_CANDIDATE_SCOPE_SPLIT_V0 + +상태: 구현 발주서 +작성일: 2026-06-29 +선행 감사: `ORDER_129_SEARCH_CANDIDATE_COUNT_BASIS_AUDIT_V0` + +## 1. 목표 + +`search_candidate_count`가 화면마다 다른 뜻으로 보이는 문제를 줄인다. + +현재는 다음 두 범위가 같은 이름으로 섞일 수 있다. + +- 최종/최신 L3 return summary 기준 검색 후보 +- L3 initial/revision preserved frame 전체 누적 검색 후보 + +이번 발주는 이 둘을 분리한다. + +## 2. 구현 방향 + +`Node3InputBriefFrame`에 다음 필드를 추가한다. + +- `final_search_candidate_count` +- `final_search_candidate_documents` +- `accumulated_search_candidate_count` +- `accumulated_search_candidate_documents` + +호환용 기존 필드인 `search_candidate_count`와 `search_candidate_documents`는 유지하되, 의미를 `final_search_candidate_*` alias로 고정한다. + +## 3. 기준 + +최종 후보: + +- node_0 document material packet의 `was_search_candidate=true` item 기준 +- material packet이 없으면 L return summary의 `search_result_doc_ids` fallback +- identity 기준은 `doc_id` + +누적 후보: + +- L3 preserved info frame 전체의 `candidates[].doc_id` 기준 +- initial/revision preserved frame을 모두 포함 +- identity 기준은 `doc_id` + +표시용 문서명은 내부 ID를 그대로 노출하지 않되, 같은 파일명이 여러 경로에 있으면 전체 path label로 구분한다. + +## 4. 변경 범위 + +- `songryeon_core/core/schemas.py` +- `songryeon_core/nodes/node_2_handoff.py` +- `songryeon_core/nodes/node_3_reporter.py` +- `songryeon_core/nodes/node_4_gatekeeper.py` +- `songryeon_core/runtime/terminal_view.py` +- `songryeon_core/runtime/smoke_test.py` +- 관련 pytest + +## 5. 금지 + +- L3 per-document summary 구현 금지 +- 검색/read 예산 변경 금지 +- L revision 전략 변경 금지 +- node_4 guard 약화 금지 +- W/R loop, scheduler, 외부 DB, 장기기억 DB 변경 금지 +- 휴리스틱으로 검색 후보를 재분류하는 것 금지 + +## 6. 완료 조건 + +- node_3 brief가 최종 후보와 누적 후보 count를 둘 다 가진다. +- legacy `search_candidate_count`는 final count와 일치한다. +- node_3 grounding block이 최종/누적 검색 후보 count를 분리 표시한다. +- node_4 count guard가 두 count를 각각 검증한다. +- terminal runtime이 두 count를 구분 표시한다. +- 같은 파일명 다른 doc_id 후보가 누적 후보 count에서 collapse되지 않는다. + +## 7. 검증 + +- `python -m compileall songryeon_core main.py` +- `python -m pytest` +- `python main.py smoke-test` diff --git a/Administrative_Reform_1/04_Orders/ORDER_132_NODE2_ANSWER_BASIS_MATERIAL_DELIVERY_POLICY_V0.md b/Administrative_Reform_1/04_Orders/ORDER_132_NODE2_ANSWER_BASIS_MATERIAL_DELIVERY_POLICY_V0.md new file mode 100644 index 0000000..6497809 --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_132_NODE2_ANSWER_BASIS_MATERIAL_DELIVERY_POLICY_V0.md @@ -0,0 +1,178 @@ +# ORDER_132 Node2 Answer Basis Material Delivery Policy v0 + +## 목표 + +node_2가 고른 `answer_basis_mode`를 바탕으로, node_3가 원문 문서 context와 L3 문서별 요약을 어떤 태도로 사용할지 명시적인 material delivery policy로 전달한다. + +이번 발주는 `absolute_first`가 아닌 답변 모드에서 L3 요약이 node_3 LLM 입력의 원문 text를 대체하게 한다. +단, 원문 `read_doc`/`read_artifact` record는 DataStore에 그대로 남긴다. +대체 범위는 node_3 LLM payload의 raw document text에 한정한다. + +## 배경 + +ORDER_125에서 L3는 실제 읽은 문서마다 두 종류의 요약을 만들 수 있게 되었다. + +- `plain_document_summary`: 문서 하나에 직접 대응하는 `relative` 요약 +- `task_relevant_summary`: 현재 질문/L1 목표와 문서 원문 source bundle에 근거한 `mixed` 요약 + +이제 다음 문제는 node_3가 원문과 요약을 언제 어떻게 써야 하는지다. +특히 `answer_basis_mode=absolute_first`인 질문에서는 원문/trace/data 같은 확인 가능한 재료를 우선해야 한다. +반대로 `relative_allowed`나 `mixed_or_uncertain`에서는 문서 폭탄을 줄이기 위해 L3 요약을 node_3 LLM 입력의 주 문서 재료로 사용한다. + +## 핵심 원칙 + +- code는 “어떤 문서 내용이 의미적으로 중요한지”를 판단하지 않는다. +- node_2 LLM이 고른 `answer_basis_mode`는 의미 판단이다. +- code는 이미 선택된 mode를 기반으로 정해진 정책 mapping만 적용한다. +- 정책 mapping은 숨은 휴리스틱이 아니라 명시적 발주/스키마/trace로 남긴다. +- L3 요약은 원문이 아니라 LLM이 만든 의미 재료다. +- 원문을 쓰는 답변과 요약을 쓰는 답변은 사용자-facing 답변에서 경계를 드러내야 한다. +- 원문 record를 삭제하거나 변형하지 않는다. +- 원문 대체는 node_3 LLM payload에서만 일어난다. +- L3 요약이 없으면 code가 요약을 만들지 않고, raw fallback 정책을 명시한다. + +## 제안 schema + +```text +Node3MaterialDeliveryPolicyFrame +- frame_id +- turn_id +- answer_basis_frame_id +- answer_basis_mode +- material_delivery_mode +- raw_document_policy +- l3_summary_policy +- uncertainty_policy +- policy_reason_code +- llm_raw_document_text_count +- llm_l3_summary_context_count +- raw_context_replaced_by_summary_count +- generated_by=CODE:ANSWER_BASIS_MATERIAL_POLICY +- info_class=absolute_policy_decision +- semantic_judgement_status=not_run +- source_trace_ids +- source_data_ids +``` + +## 정책 mapping v0 + +### 1. `answer_basis_mode=absolute_first` + +```text +material_delivery_mode=raw_document_primary +raw_document_policy=preserve_supplied_raw_context +l3_summary_policy=auxiliary_only +uncertainty_policy=do_not_replace_raw_with_summary +policy_reason_code=absolute_first_requires_checkable_material +llm_raw_document_text_count=supplied_document_context_count +raw_context_replaced_by_summary_count=0 +``` + +의미: + +- node_3는 공급된 원문 context를 우선 근거로 사용한다. +- L3 요약은 안내/보조 재료일 뿐, 원문을 대체하지 않는다. +- 요약만 보고 원문을 확인한 것처럼 말하지 않는다. + +### 2. `answer_basis_mode=relative_allowed` + +```text +material_delivery_mode=l3_summary_replaces_raw_context +raw_document_policy=omit_raw_text_from_llm_payload +l3_summary_policy=replace_raw_context_with_labeled_l3_summary +uncertainty_policy=keep_summary_boundary_visible +policy_reason_code=relative_allowed_uses_l3_summary_to_reduce_context_volume +llm_raw_document_text_count=0 +raw_context_replaced_by_summary_count=supplied_document_context_count +``` + +의미: + +- node_3 LLM payload에는 원문 text를 넣지 않고 L3 요약을 넣는다. +- L3 요약을 사용할 때는 “요약 재료”라는 경계를 유지한다. +- 해석/조언/구상은 가능하지만 code fact처럼 말하지 않는다. + +### 3. `answer_basis_mode=mixed_or_uncertain` + +```text +material_delivery_mode=l3_summary_replaces_raw_context_with_uncertainty +raw_document_policy=omit_raw_text_from_llm_payload +l3_summary_policy=replace_raw_context_with_labeled_l3_summary_and_limits +uncertainty_policy=surface_partial_or_bundle_based_grounding +policy_reason_code=mixed_or_uncertain_uses_l3_summary_with_limit_visibility +llm_raw_document_text_count=0 +raw_context_replaced_by_summary_count=supplied_document_context_count +``` + +의미: + +- node_3는 출처 묶음, 요약 한계, 부족한 근거를 분명히 말한다. +- `task_relevant_summary`는 mixed 정보이므로 source bundle 기반 요약이라고 취급한다. +- 원문/요약/검색 목표 실패 신호를 섞어 성공처럼 말하지 않는다. + +### 4. L3 요약이 없는 비-절대정보 모드 + +```text +material_delivery_mode=raw_document_fallback_no_l3_summary +raw_document_policy=preserve_raw_context_because_l3_summary_missing +l3_summary_policy=unavailable +uncertainty_policy=expose_summary_absence +policy_reason_code=l3_summary_unavailable_cannot_replace_raw_context +llm_raw_document_text_count=supplied_document_context_count +raw_context_replaced_by_summary_count=0 +``` + +의미: + +- code는 요약을 대신 만들지 않는다. +- L3 요약이 없기 때문에 원문 text를 node_3 LLM payload에 유지한다. +- 이 fallback은 의미 판단이 아니라 summary availability에 따른 절대 정책 분기다. + +## 구현 방향 + +1. `Node3MaterialDeliveryPolicyFrame` schema를 추가한다. +2. node_2 handoff 또는 node_3 brief 생성 단계에서 `answer_basis_mode`를 읽어 정책 frame을 만든다. +3. policy frame은 별도 DataStore record로 저장한다. +4. `Node3InputBriefFrame`에 policy 요약 필드를 추가한다. +5. `node3_brief_llm_payload()`에 safe policy payload를 넣는다. +6. `node_3_reporter_v0.md`에 다음 경계를 추가한다. + - absolute_first에서는 원문 context를 우선한다. + - relative_allowed에서는 L3 요약이 원문 text payload를 대체한다. + - mixed_or_uncertain에서는 L3 요약이 원문 text payload를 대체하되 요약/출처 묶음/불확실성을 더 분명히 드러낸다. +7. terminal view에 material delivery policy를 표시한다. + +## 테스트 계획 + +- `python -m compileall songryeon_core main.py` +- `python -m pytest` +- `python main.py smoke-test` + +추가 pytest: + +- `absolute_first`이면 `raw_document_primary` policy가 생성된다. +- `relative_allowed`이고 L3 요약이 있으면 `l3_summary_replaces_raw_context` policy가 생성된다. +- `mixed_or_uncertain`이고 L3 요약이 있으면 `l3_summary_replaces_raw_context_with_uncertainty` policy가 생성된다. +- 비-절대정보 모드인데 L3 요약이 없으면 `raw_document_fallback_no_l3_summary` policy가 생성된다. +- policy frame은 `generated_by=CODE:ANSWER_BASIS_MATERIAL_POLICY`, `semantic_judgement_status=not_run`이다. +- node_3 brief/payload가 policy를 받되 raw internal ID를 노출하지 않는다. +- L3 요약이 있어도 `absolute_first` policy에서는 원문 대체로 표시되지 않는다. +- L3 요약이 있는 비-절대정보 모드에서는 node_3 LLM payload의 `supplied_document_contexts`/`read_documents`에 원문 `text`가 들어가지 않는다. +- DataStore의 원문 document extract record는 삭제되지 않는다. + +## 제외 범위 + +- 원문 context packing 예산을 줄이지 않는다. +- code가 문서 중요도나 관련성을 판단하지 않는다. +- node_0 요약, node_5 기억 압축, 장기기억 DB, vector DB, scheduler는 열지 않는다. +- W/R loop와 same-turn L reroute 횟수는 건드리지 않는다. +- node_4 자동 재작성 루프는 열지 않는다. + +## 완료 조건 + +- policy frame이 DataStore에 남는다. +- node_3 brief와 payload에서 material delivery policy를 확인할 수 있다. +- 비-절대정보 모드에서 L3 요약이 있으면 node_3 LLM payload의 원문 text가 생략된다. +- 원문 DataStore record는 그대로 유지된다. +- terminal runtime view에서 policy mode를 확인할 수 있다. +- 기존 ORDER_125 L3 summary frame 동작을 깨지 않는다. +- compileall / pytest / smoke-test가 모두 통과한다. diff --git a/Administrative_Reform_1/04_Orders/ORDER_133_CODEBASE_READONLY_INSPECTION_MVP_V0.md b/Administrative_Reform_1/04_Orders/ORDER_133_CODEBASE_READONLY_INSPECTION_MVP_V0.md new file mode 100644 index 0000000..cb00c4a --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_133_CODEBASE_READONLY_INSPECTION_MVP_V0.md @@ -0,0 +1,157 @@ +# ORDER_133_CODEBASE_READONLY_INSPECTION_MVP_V0 + +## 상태 + +발주서. + +## 목표 + +SongRyeon Core가 내부 문서만이 아니라 실제 코드 파일 구조도 읽을 수 있게 하는 첫 read-only MVP를 구현한다. + +이번 작업의 목표는 "코딩하는 에이전트"가 아니다. +목표는 "코드베이스를 읽고, 어떤 파일을 실제로 확인했는지 근거와 한계를 드러내며 설명할 수 있는 에이전트"의 최소 기반이다. + +## 배경 + +최근 live 테스트에서 사용자가 다음과 같이 물었다. + +```text +그럼 네가 코딩에 얼마나 쓸모를 발휘할 수 있을지 가늠하게 알아서 네 코드 파일이 어떻게 구성됐는지 최대한 자세히 알아와봐 +``` + +송련은 이 질문을 L로 라우팅했지만, 실제 코드 파일을 직접 읽지 못하고 내부 문서와 발주서 중심으로 답하려 했다. +node_4가 근거 불일치를 감지해 최종 답변을 막은 것은 좋은 실패였지만, 코딩 보조로 발전하려면 코드 파일 자체를 읽는 read-only 도구가 필요하다. + +## 구현 범위 + +### In Scope + +1. read-only code inspection tool 추가. + - `list_code_files` + - `search_code` + - `read_code_file` + +2. 각 도구는 코드 원문 또는 코드 파일 목록에서 확인 가능한 절대정보만 반환한다. + - 파일 경로 + - 확장자 + - 크기 + - 라인 수 + - 검색 match 위치 + - 읽은 코드 원문 + - truncation 여부 + +3. 도구 결과는 기존 TraceStore/DataStore tool result 흐름을 사용한다. + +4. 도구는 read-only로만 등록한다. + +5. L2가 코드 구조 질문에서 코드 도구를 선택할 수 있게 최소 prompt/schema/tool registry를 확장한다. + +6. smoke/pytest는 다음을 확인한다. + - 코드 도구가 workspace 내부 파일만 읽는다. + - path traversal을 차단한다. + - `search_code`가 검색 결과를 절대정보로 반환한다. + - `read_code_file`이 실제 원문과 line count를 반환한다. + - 코드 도구는 파일 수정 기능을 갖지 않는다. + +### Out Of Scope + +- 파일 수정 +- patch 생성 +- apply_patch 자동 실행 +- 테스트 자동 실행 계획 생성 +- 코드 리뷰 의미 판단 자동화 +- dependency graph 고급 분석 +- AST 기반 구조 분석 +- W/R loop +- scheduler +- 외부 DB/vector DB +- 장기기억 DB +- same-turn L reroute 횟수 변경 + +## 정보 분류 + +### 절대정보 + +code tool이 직접 확인한 값. + +예: + +```text +file_path +extension +size_bytes +line_count +match_line +match_text +read_text +truncated +``` + +### 상대정보 + +특정 코드 파일 하나에 대해 LLM이 만드는 의미 해석. + +예: + +```text +이 파일은 LLM runtime 설정을 담당하는 것으로 보인다. +``` + +### 혼합정보 + +여러 코드 파일, 문서, trace, 실행 기록을 함께 보고 만드는 판단. + +예: + +```text +현재 코드베이스는 문서 검색 런타임에는 강하지만 실제 코드 수정 에이전트로는 아직 미완성이다. +``` + +## 금지선 + +코드 도구는 절대 파일을 수정하지 않는다. + +코드는 다음을 하면 안 된다. + +```text +"이 파일은 중요하다" 같은 의미 판단 +"이 구조가 좋다/나쁘다" 같은 평가 +"이 파일을 수정해야 한다" 같은 제안 +``` + +그런 판단은 node/L3/node_2/node_3 같은 LLM 판단 경계에서 source를 드러내고 수행한다. + +## 완료 조건 + +1. `python -m compileall songryeon_core main.py` +2. `python -m pytest` +3. `python main.py smoke-test` + +추가 기대: + +- 코드 파일 구조 질문에서 L2가 `search_code` 또는 `read_code_file`을 선택할 수 있다. +- 코드 도구 result가 TraceStore/DataStore에 남는다. +- runtime 출력이나 node_3 brief에는 적어도 코드 도구가 실행됐다는 사실이 보존된다. +- 최종 답변이 코드 파일을 읽은 척하려면 실제 code tool result가 있어야 한다. + +## 후속 후보 + +ORDER_134 후보: + +```text +CODEBASE_STRUCTURE_PACKET_TO_NODE3_BRIEF +``` + +ORDER_135 후보: + +```text +CODE_REVIEW_READONLY_RISK_FRAME +``` + +ORDER_136 이후 후보: + +```text +PROPOSED_PATCH_FRAME_WITHOUT_APPLYING +``` + +실제 코드 수정 에이전트는 read-only 코드 구조 파악과 코드 리뷰 프레임이 안정된 뒤에 연다. diff --git a/Administrative_Reform_1/04_Orders/ORDER_134_L_TOOL_SCOPE_AND_BUDGET_PARTITION_V0.md b/Administrative_Reform_1/04_Orders/ORDER_134_L_TOOL_SCOPE_AND_BUDGET_PARTITION_V0.md new file mode 100644 index 0000000..f2ec58b --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_134_L_TOOL_SCOPE_AND_BUDGET_PARTITION_V0.md @@ -0,0 +1,143 @@ +# ORDER_134_L_TOOL_SCOPE_AND_BUDGET_PARTITION_V0 + +## Goal + +Make the L loop decide its evidence/tool scope before L2 chooses individual tools. + +This order exists because ORDER_133 added read-only code inspection tools, but a live test showed that L2 kept using document search only. The fix must not be keyword heuristics such as "if user says code, use code tools." Instead, the L loop must first record an explicit tool-scope contract, then allocate budgets by allowed tool group, and only then allow L2 to choose tools inside that contract. + +## Problem + +Current L2 receives a flat tool catalog and is asked to choose a target tool directly. In mixed requests such as "read ORDER_133 and actual source code," L2 may keep selecting `search_docs` even though code tools exist. + +That is not a tool implementation failure. It is a missing planning layer: + +- L1 describes the evidence goal. +- L2 chooses one tool/query. +- But no frame says which evidence worlds are open this turn. +- No frame divides budget between document tools and code inspection tools. +- Therefore L2 can over-focus on one tool family. + +## Required MVP + +Add a small L tool scope stage after L1 and before L2 query planning. + +### New frame + +Create `LToolScopeFrame` with at least: + +- `frame_id` +- `turn_id` +- `tool_scope_mode` + - `document_only` + - `code_only` + - `document_and_code` + - `runtime_trace_only` + - `mixed_evidence` +- `allowed_tool_groups` + - `document_tools` + - `code_inspection_tools` + - `runtime_record_tools` +- `required_materials` + - `order_document` + - `source_code_file` + - `code_search_result` + - `runtime_trace` + - `execution_record` + - `project_document` +- `scope_reason` +- `scope_reason_info_class` +- `generated_by` +- `info_class` +- `semantic_judgement_status` +- `source_trace_ids` +- `source_data_ids` + +### Responsibility boundary + +- LLM chooses the scope. +- Code validates enum values and allowed tool groups. +- Code must not decide from user keywords which tool group is semantically needed. +- If the scope LLM fails or is absent, code may use a compatibility fallback, but it must label it as `CODE:FALLBACK`, `semantic_judgement_status=failed`, and must not pretend it is a semantic decision. + +### Tool catalog filtering + +After `LToolScopeFrame` is recorded: + +- L2 receives only tools allowed by the selected tool groups. +- `document_tools` allow existing document tools. +- `code_inspection_tools` allow `list_code_files`, `search_code`, `read_code_file`. +- `runtime_record_tools` is reserved for later and should not add new tools in this MVP. + +### Budget partition + +Add a code-built budget partition record from the approved L budget and the tool scope: + +- `document_tool_call_budget` +- `document_query_budget` +- `document_read_budget` +- `code_tool_call_budget` +- `code_query_budget` +- `code_read_budget` + +For this MVP, simple explicit partition policy is acceptable: + +- `document_only`: all approved budget to document group. +- `code_only`: all approved budget to code group. +- `document_and_code`: reserve at least 1 code query/read/tool call when possible, and at least 1 document query/read/tool call when possible. +- `mixed_evidence`: split conservatively between document and code when both groups are allowed. +- `runtime_trace_only`: no new external/document/code tools beyond existing runtime records in this MVP. + +This partition is a policy decision derived from the explicit scope frame, not hidden semantic classification. + +### L2 contract + +L2 query planner input must include: + +- `l_tool_scope` +- `budget_partition` +- filtered `available_tools` + +The prompt must say: + +- choose tools only from the supplied `available_tools`; +- if both `order_document` and `source_code_file` are required, create candidates that cover both document and code tools when available; +- do not claim a tool ran before it actually runs. + +## Tests + +Required verification: + +- `python -m compileall songryeon_core main.py` +- `python -m pytest` +- `python main.py smoke-test` + +Add pytest coverage: + +- scope LLM can select `document_and_code` and the frame validates. +- allowed tool catalog is filtered to document + code inspection tools. +- L2 planner receives filtered tools and scope payload. +- L2 candidate outside the allowed scope is rejected or filtered. +- budget partition for `document_and_code` reserves nonzero document and code budgets when global budget allows. +- fallback scope is explicit `CODE:FALLBACK` with failed semantic status. + +## Out Of Scope + +- No keyword heuristics. +- No code editing tools. +- No patch generation/application. +- No automatic test-running tool inside SongRyeon runtime. +- No W/R loop. +- No scheduler. +- No external DB/vector DB/long-term memory DB. +- No same-turn L reroute policy change. +- No runtime trace tool implementation yet. + +## Completion Report Must Include + +- where `LToolScopeFrame` is created; +- how tool groups map to allowed tools; +- how budget partition is recorded; +- what L2 receives; +- what happens on scope LLM failure; +- compileall / pytest / smoke-test results. diff --git a/Administrative_Reform_1/04_Orders/ORDER_135_CODE_EVIDENCE_ACCOUNTING_AND_L3_SUCCESS_BOUNDARY_V0.md b/Administrative_Reform_1/04_Orders/ORDER_135_CODE_EVIDENCE_ACCOUNTING_AND_L3_SUCCESS_BOUNDARY_V0.md new file mode 100644 index 0000000..6e57c85 --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_135_CODE_EVIDENCE_ACCOUNTING_AND_L3_SUCCESS_BOUNDARY_V0.md @@ -0,0 +1,80 @@ +# ORDER_135_CODE_EVIDENCE_ACCOUNTING_AND_L3_SUCCESS_BOUNDARY_V0 + +## Goal + +Fix the boundary where `read_code_file` succeeds but downstream accounting still treats the turn as failed because it only counts `read_doc`. + +ORDER_133 opened read-only code inspection tools. ORDER_134 made L choose a tool scope before L2 tool selection. A live test then showed the next bottleneck: + +- L tool scope selected `code_only`. +- L2 selected `read_code_file`. +- The source file was available to node_3 as context. +- But L3 and downstream count fields still centered on `read_doc`, so the run was described as if no original evidence had been read. + +This order separates document evidence from source-code evidence without turning code into a hidden semantic judge. + +## Required MVP + +### L3 Accounting + +Add source-code evidence fields to `L3AchievementFrame`: + +- `read_code_file_paths` +- `actual_read_code_file_count` + +L3 should still keep `read_doc_ids` for document reads. It must not rename source-code evidence into document evidence. + +### Goal Match Boundary + +If the requested exact artifact/source path matches a successful `read_code_file` path, L3 may mark the specific file match as `matched`. + +This is a path/record match, not semantic code understanding. + +### L1 Minimum Evidence Guard + +The minimum read requirement must not be checked only against `read_doc` when the tool scope/material is source-code evidence. + +For source-code turns, successful `read_code_file` records count as source evidence for the minimum evidence guard. + +### Return Summary And Node_3 Brief + +Carry source-code counts downstream: + +- `LLoopReturnSummaryFrame.actual_read_code_file_count` +- `LLoopReturnSummaryFrame.read_code_file_paths` +- `Node3InputBriefFrame.actual_tool_read_code_file_count` +- `Node3InputBriefFrame.actual_tool_read_code_file_paths` +- `Node3InputBriefFrame.supplied_source_code_context_count` + +The node_3 grounding block should show document reads and source-code reads separately. + +### Prompt/Runtime Boundary + +Update node_3 prompt/runtime wording so: + +- `read_doc` means document tool reads only. +- `read_code_file` means source-code file reads only. +- supplied source-code context can be used as copied source text. +- source-code context must not be called a document search result. + +## Non-Goals + +- No code editing tools. +- No command execution tools. +- No AST/dependency graph analysis. +- No keyword heuristic that decides code/document scope. +- No W/R loop, scheduler, external DB, vector DB, or long-term memory DB changes. +- No same-turn L reroute policy change. + +## Verification + +- `python -m compileall songryeon_core main.py` +- `python -m pytest` +- `python main.py smoke-test` + +Add focused tests proving: + +- Successful `read_code_file` is counted separately from `read_doc`. +- L3 can mark an exact requested source file as matched when the read path matches. +- L loop return summary carries source-code evidence counts. +- node_3 grounding block shows code reads separately from document reads. diff --git a/Administrative_Reform_1/04_Orders/ORDER_136_CURRENT_CAPABILITY_BASELINE_AND_LIVE_TEST_PACK_V0.md b/Administrative_Reform_1/04_Orders/ORDER_136_CURRENT_CAPABILITY_BASELINE_AND_LIVE_TEST_PACK_V0.md new file mode 100644 index 0000000..4f5d60f --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_136_CURRENT_CAPABILITY_BASELINE_AND_LIVE_TEST_PACK_V0.md @@ -0,0 +1,89 @@ +# ORDER_136_CURRENT_CAPABILITY_BASELINE_AND_LIVE_TEST_PACK_V0 + +## Goal + +Freeze the current SongRyeon Core capability baseline before the next feature expansion. + +The project has recently added recent-memory supply, L loop continuation, node_2 answer-basis mode, L3 summaries, document material accounting, read-only code inspection, L tool scope, budget partitioning, and source-code evidence accounting. This is now powerful enough that the next risk is not a missing feature but losing track of what the system can actually do. + +ORDER_136 documents the current baseline and defines a small live test pack that can be reused before future MVP work. + +## Problem + +SongRyeon Core now has many safety and evidence-accounting layers: + +- node_0 memory supply +- node_1 routing +- L loop document/code evidence collection +- L3 achievement judgement +- node_2 metainfo boundary and answer-basis mode +- node_3 code-supplied grounding block +- node_4 gatekeeper +- runtime/terminal renderer +- pytest and smoke-test baseline + +Without a baseline map, future work may accidentally: + +- retest the wrong capability, +- confuse document reads with code reads, +- treat a live qwen answer as proof when the runtime signals say otherwise, +- reopen W/R/scheduler/long-term DB work too early, +- or keep adding features before the current system is understandable to humans and external reviewers. + +## Required Documentation + +Create a capability baseline document under: + +`Administrative_Reform_1/03_Maps/03_Development_Maps/` + +The document must include: + +- current verified capability groups, +- what is deliberately not supported yet, +- live test prompts, +- expected runtime signals for each prompt, +- known weak spots, +- next sustainable work candidates. + +## Live Test Pack Requirements + +The live test pack should cover: + +1. recent conversation memory, +2. internal document search/read, +3. explicit source-code file read, +4. mixed document + code inspection, +5. answer-basis mode behavior, +6. count honesty, +7. failure honesty. + +The test pack is manual for now. It should not pretend to be a deterministic CI suite. + +## Non-Goals + +- No new runtime feature. +- No W loop. +- No R loop. +- No scheduler. +- No external DB/vector DB/long-term memory DB. +- No code editing agent tools. +- No new heuristic classifier. +- No hidden keyword routing policy. + +## Completion Conditions + +- ORDER_136 file is added. +- Capability baseline/live test pack document is added. +- `04_Orders/README.md` is updated. +- `03_Development_Maps/README.md` is updated. +- Execution record is added. + +## Verification + +Documentation-only change. + +Recommended check: + +```powershell +git diff --check +``` diff --git a/Administrative_Reform_1/04_Orders/ORDER_137_SOURCE_CODE_CONTEXT_SUMMARY_COVERAGE_GUARD_V0.md b/Administrative_Reform_1/04_Orders/ORDER_137_SOURCE_CODE_CONTEXT_SUMMARY_COVERAGE_GUARD_V0.md new file mode 100644 index 0000000..33272c8 --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_137_SOURCE_CODE_CONTEXT_SUMMARY_COVERAGE_GUARD_V0.md @@ -0,0 +1,64 @@ +# ORDER_137_SOURCE_CODE_CONTEXT_SUMMARY_COVERAGE_GUARD_V0 + +## 1. 목표 + +`read_code_file`로 source/config 파일 원문을 읽는 경로는 열렸지만, node_3가 최종 답변에서 파일 안의 공개 함수나 상수 같은 핵심 구조를 빠뜨릴 수 있다. + +이번 MVP는 code가 읽힌 source-code 원문에서 문법적으로 확인 가능한 outline을 만들고, node_3에게 coverage checklist로 전달한다. + +## 2. 배경 + +최근 live 테스트에서 `songryeon_core/tools/code_tools.py`는 정상적으로 직접 읽혔다. + +- L scope: `code_only` +- budget partition: `document=0 / code=18` +- 실제 도구: `read_code_file` +- L3: `achieved`, `goal_match_status=matched` +- node_3 brief: `actual_read_code_file=1`, `source_code_contexts=1` + +하지만 최종 답변은 `read_code_file` 중심으로 설명했고, 같은 파일의 공개 기능인 `list_code_files`, `search_code` 등을 충분히 다루지 못했다. + +이는 검색/읽기 실패가 아니라, 읽힌 코드 원문의 구조를 node_3가 답변 coverage 기준으로 안정적으로 보지 못한 문제다. + +## 3. 구현 범위 + +1. `read_code_file` 결과 원문에서 source-code outline을 생성한다. + - Python 파일은 `ast.parse`로 top-level 구조를 읽는다. + - 함수, async 함수, class, 대문자 상수 assign을 기록한다. + - 이름, 종류, line number, public/private 여부, docstring 존재 여부만 기록한다. + +2. outline은 절대정보로 취급한다. + - code는 함수의 의미를 요약하지 않는다. + - code는 이름과 문법 위치만 기록한다. + - 의미 설명은 node_3 LLM이 source text와 outline을 보고 작성한다. + +3. `Node3InputBriefFrame`에 source-code outline 목록을 추가한다. + - `source_code_outlines` + - outline count는 grounding block과 runtime view에 표시한다. + +4. node_3 prompt를 보강한다. + - source-code 파일의 기능을 설명할 때 outline의 public top-level function 목록을 coverage checklist로 사용한다. + - 함수 이름만 보고 동작을 단정하지 말고, supplied source text를 근거로 설명한다. + - outline은 코드 문법 장부이며 의미 요약이 아님을 명시한다. + +5. node_4 guard, L routing, tool budget, W/R loop, scheduler, 외부 DB는 건드리지 않는다. + +## 4. 금지 + +- 질문 문자열 기반 휴리스틱 추가 금지. +- code가 함수 의미를 대신 요약하는 것 금지. +- read_code_file을 read_doc으로 섞어 세는 것 금지. +- L loop 예산 확대 금지. +- W loop, R loop, scheduler, 외부 DB, 장기기억 DB 변경 금지. + +## 5. 완료 조건 + +1. `python -m compileall songryeon_core main.py` +2. `python -m pytest` +3. `python main.py smoke-test` +4. 추가 테스트: + - `code_tools.py`를 `read_code_file`로 읽으면 source-code outline이 생성된다. + - outline public function 목록에 `list_code_files`, `search_code`, `read_code_file`이 포함된다. + - outline 상수 목록에 `DEFAULT_CODE_FILE_EXTENSIONS`, `DEFAULT_IGNORED_DIR_NAMES`가 포함된다. + - node_3 LLM payload에 raw internal data id 없이 outline coverage checklist가 들어간다. + - grounding block과 runtime view에 source-code outline count가 표시된다. diff --git a/Administrative_Reform_1/04_Orders/ORDER_138_INTEGRATION_BASELINE_AND_DIRTY_WORKTREE_RECONCILIATION_V0.md b/Administrative_Reform_1/04_Orders/ORDER_138_INTEGRATION_BASELINE_AND_DIRTY_WORKTREE_RECONCILIATION_V0.md new file mode 100644 index 0000000..82abc06 --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_138_INTEGRATION_BASELINE_AND_DIRTY_WORKTREE_RECONCILIATION_V0.md @@ -0,0 +1,77 @@ +# ORDER_138_INTEGRATION_BASELINE_AND_DIRTY_WORKTREE_RECONCILIATION_V0 + +## Status Note + +2026-06-30 기준, 이 문서에 기록된 심야정부 MVP(`songryeon_core/night_government`, `night-*` CLI, 전용 pytest)는 이후 감사에서 송련 Core의 TurnStateCapsule/TraceStore/ZeroState 기반 기억 체계와 맞지 않는 provisional 기능으로 판단되어 제거되었다. + +이 문서는 과거 기준선 기록으로만 남긴다. 심야정부를 다시 설계할 경우, 여기의 JSONL `MemoryRecord` 구조를 이어받지 말고 송련의 기존 trace/capsule/metainfo 원칙에서 다시 시작한다. + +## 1. 목표 + +ORDER_133 이후 codebase inspection, L tool scope, code evidence accounting, source-code outline, 그리고 심야정부 MVP가 빠르게 들어왔다. + +이번 발주는 새 기능을 추가하지 않고, 현재 작업트리 상태를 기준선으로 묶어 다음 개발자가 길을 잃지 않게 한다. + +## 2. 배경 + +최근 개발 흐름은 다음과 같다. + +- ORDER_133: read-only codebase inspection 도구 추가 +- ORDER_134: L tool scope와 도구군별 예산 분배 추가 +- ORDER_135: `read_code_file`을 `read_doc`과 분리해 source-code evidence로 인정 +- ORDER_136: 현재 capability baseline과 live test pack 문서화 +- ORDER_137: `read_code_file` source-code outline을 node_3 coverage checklist로 공급 +- 심야정부 MVP: JSONL 기반 외부 기억 DB와 active memory packet CLI 추가 + +심야정부 MVP는 검증을 통과했지만, 정식 발주서 없이 먼저 구현되어 문서-현실 불일치가 생겼다. + +## 3. 이번 발주 범위 + +1. 현재 작업트리의 기능 확장 상태를 감사한다. +2. ORDER/Execution Records README가 최신 상태인지 확인한다. +3. 전체 검증 명령을 다시 실행해 통합 기준선을 기록한다. +4. 심야정부 MVP를 다음과 같이 분류한다. + - 현재 상태: provisional external memory DB MVP + - 검증 상태: local tests, full pytest, smoke-test 통과 + - 아직 아님: 송련식 메타정보 provenance DB +5. 다음 발주 후보를 분리한다. + - 심야정부를 `info_class`, `source_mode`, `claim_alignment`, `semantic_judgement_status`, `source_data_ids/source_trace_ids` 기반으로 정렬하는 별도 발주 + - schema/test 파일 비대화 추가 분해 발주 + +## 4. 금지 + +- 새 runtime 기능 추가 금지. +- 심야정부 active packet을 0 기억공급관에 자동 주입하지 않는다. +- 외부 DB를 SQLite/Neo4j로 교체하지 않는다. +- 기존 사용자/다른 채팅방 변경을 되돌리지 않는다. +- 대량 리팩터링을 이번 발주에 끼워 넣지 않는다. + +## 5. 완료 조건 + +1. `python -m compileall songryeon_core main.py` +2. `python -m pytest` +3. `python main.py smoke-test` +4. `git diff --check` +5. 실행 기록에 다음을 남긴다. + - 현재 dirty worktree 요약 + - ORDER_133~137 및 심야정부 MVP 상태 + - 전체 검증 결과 + - 다음 위험과 다음 발주 후보 + +## 6. 다음 발주 후보 + +### ORDER_139_NIGHT_GOVERNMENT_METAINFO_ALIGNMENT_V0 + +심야정부 `MemoryRecord`와 `MemoryActivationItem`을 송련 메타정보 원칙에 맞춘다. + +필수 방향: + +- `memory_role`과 `info_class`를 분리한다. +- relative/mixed memory record는 orphan 금지. +- relative는 단일 source record/field가 필요하다. +- mixed는 source bundle이 필요하다. +- activation item은 현재 턴에 들어올 때 사용 권한과 한계를 표시한다. + +### Schema/Test Refoundation Follow-up + +`schemas.py`, `smoke_test.py`, ORDER별 pytest helper가 다시 커지고 있으므로, 다음 큰 기능 전 추가 분해를 검토한다. diff --git a/Administrative_Reform_1/04_Orders/ORDER_139_GRAPH_MEMORY_FOUNDATION_AND_RLOOP_GUIDE_PACKET_V0.md b/Administrative_Reform_1/04_Orders/ORDER_139_GRAPH_MEMORY_FOUNDATION_AND_RLOOP_GUIDE_PACKET_V0.md new file mode 100644 index 0000000..23798d9 --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_139_GRAPH_MEMORY_FOUNDATION_AND_RLOOP_GUIDE_PACKET_V0.md @@ -0,0 +1,364 @@ +# ORDER_139_GRAPH_MEMORY_FOUNDATION_AND_RLOOP_GUIDE_PACKET_V0 + +## 1. 목표 + +심야정부/R루프 구현의 첫 삽을 뜬다. + +이번 발주는 완성된 외부 그래프 DB나 완성된 R루프가 아니다. + +목표는 더 작다. + +```text +TurnStateCapsule/TraceStore/DataStore 기반 raw graph node 문법을 만들고, +CoreEgo 시간축 entry를 구성하고, +R루프가 나중에 읽을 RLoopGraphGuidePacket을 생성한다. +``` + +즉 이번 작업의 성공은 "송련이 장기기억을 완성했다"가 아니라 다음이다. + +```text +기존 capsule/trace/data 좌표를 잃지 않고 graph memory foundation으로 올릴 수 있다. +CoreEgo 직속 연결은 시간축만 쓴다. +R루프용 graph guide packet이 생성되고 provenance가 남는다. +``` + +## 2. 반드시 먼저 읽을 문서 + +구현 전 아래 문서를 읽고 시작한다. + +1. `AGENTS.md` +2. `Administrative_Reform_1/01_Maintenance_System/SCHEMA_METAINFO_POLICY_v0.md` +3. `Administrative_Reform_1/00_Philosophy/Night_Government_Graph_Memory_Philosophy_2026_06_30.md` +4. `Administrative_Reform_1/00_Philosophy/R_Loop_Graph_Guide_Philosophy_2026_06_30.md` +5. `Administrative_Reform_1/05_Execution_Records/night_government_mvp_removed_2026_06_30_001.md` +6. `Administrative_Reform_1/04_Orders/ORDER_138_INTEGRATION_BASELINE_AND_DIRTY_WORKTREE_RECONCILIATION_V0.md` + +특히 `night_government` 제거 기록을 반드시 읽는다. + +이전 실패를 반복하지 않는다. + +## 3. 배경 판단 + +외부 채팅방산 심야정부 MVP는 제거되었다. + +이유: + +- 기존 TurnStateCapsule/TraceStore/DataStore를 우선 사용하지 않았다. +- 별도 `MemoryRecord` 저장소를 만들었다. +- relative/mixed 정보의 source provenance가 송련 기준으로 부족했다. +- 0 기억공급관과 연결되지 않은 독립 active packet을 만들었다. + +따라서 이번 ORDER_139는 "새 기억장"을 만들지 않는다. + +기존 송련 runtime이 이미 만든 절대정보 좌표를 graph memory 문법으로 정리한다. + +## 4. 구현 범위 + +### 4.1 Graph Memory Schema + +최소 schema/frame을 추가한다. + +후보 이름: + +```text +GraphMemoryNodeFrame +GraphMemoryEdgeFrame +GraphMemorySnapshotFrame +CoreEgoTimeAxisFrame +RLoopGraphGuidePacketFrame +``` + +정확한 이름은 코드베이스 패턴에 맞춰 조정해도 된다. + +단, 아래 개념은 반드시 표현한다. + +#### Graph node kind + +```text +raw_capsule +raw_bundle +summary +core_ego +time_axis +time_bundle +``` + +이번 발주에서 `summary`는 schema/계산 필드만 열어도 된다. + +LLM 요약 생성은 하지 않는다. + +#### Graph edge kind + +```text +CONTAINS +CHILD_OF_TIME_AXIS +SOURCE_OF +SUMMARY_OF +``` + +이번 MVP에서 실제로 필요한 최소 edge만 만들어도 된다. + +### 4.2 Raw Capsule Graph Node 생성 + +최근 `TurnStateCapsule` 또는 테스트용 capsule 입력을 받아 raw graph node를 만든다. + +원칙: + +- 새 memory text를 발명하지 않는다. +- 기존 capsule의 `turn_id`, trace count, movement count, user/final trace anchor 등 절대정보 좌표만 담는다. +- node id는 deterministic/idempotent 해야 한다. +- 같은 capsule을 두 번 ingest해도 중복 raw node를 만들지 않는다. + +예상 node id 후보: + +```text +graph:raw_capsule:{turn_id} +``` + +### 4.3 CoreEgo 시간축 연결 + +초기 CoreEgo 직속 연결은 시간축만 쓴다. + +의미축은 만들지 않는다. + +최소 구조: + +```text +graph:core_ego:root + -> graph:axis:time + -> graph:time_bundle:{session_or_batch_id} + -> graph:raw_capsule:{turn_id} +``` + +시간축 bundle은 코드가 확정 가능한 절대정보로 만든다. + +사용자 의도/주제/의미 유사도 기준으로 묶지 않는다. + +### 4.4 원문 절단 금지와 토큰 예산 + +이번 MVP는 원문 전문을 반드시 graph node에 넣을 필요는 없다. + +다만 bundle 구성 정책은 아래 원칙을 반영한다. + +```text +원문을 중간에서 자르지 않는다. +토큰/문자 예산을 넘으면 leaf 단위로 bundle을 나눈다. +예산에 걸치는 마지막 leaf는 다음 bundle로 넘기거나 제외한다. +``` + +초기 테스트에서는 문자 수 기반 예산을 써도 된다. + +단, 이것을 "의미 판단"처럼 표시하지 않는다. + +### 4.5 Summary Depth 계산 인프라 + +이번 발주에서 LLM summary를 만들지는 않는다. + +하지만 summary depth 계산에 필요한 필드는 schema에 둔다. + +후보 필드: + +```text +summary_depth +source_depth_min +source_depth_max +source_leaf_count +source_summary_count +source_bundle_kind +``` + +raw leaf node는 다음과 같이 본다. + +```text +summary_depth = 0 +source_leaf_count = 1 +source_summary_count = 0 +``` + +raw bundle node는 summary가 아니므로 의미 요약으로 취급하지 않는다. + +### 4.6 RLoopGraphGuidePacket 생성 + +그래프 snapshot을 기반으로 R루프용 guide packet을 만든다. + +이번 MVP의 guide packet은 code-generated absolute/status 중심이어야 한다. + +필수 후보 필드: + +```text +graph_snapshot_id +target_consumer = R_LOOP +available_entry_nodes +node_kind_counts +data_kind_counts +summary_depth_range +source_leaf_count_range +risky_or_unreviewed_node_ids +generated_by +info_class +semantic_judgement_status +source_graph_node_ids +source_data_ids +source_trace_ids +``` + +이번 MVP에서는 `recommended_traversal_hints`를 LLM으로 만들지 않는다. + +LLM hint는 다음 발주 후보로 남긴다. + +따라서 guide packet은 다음처럼 표시한다. + +```text +generated_by = CODE:GRAPH_MEMORY_GUIDE_BUILDER +info_class = absolute +semantic_judgement_status = not_run +recommended_traversal_hints_status = not_run +``` + +### 4.7 0 기억공급관 연결은 최소 또는 보류 + +이번 발주에서 0이 실제 qwen-turn/qwen-chat 흐름에 guide packet을 자동 주입하는 것은 필수 아님. + +허용 범위: + +- dry-run/smoke/test helper에서 guide packet 생성 확인 +- DataStore record로 보존 +- runtime summary에 count/status만 노출 + +금지: + +- node_1 route 판단에 바로 사용 +- node_3 최종 답변에 바로 사용 +- R루프를 실제 route로 열기 + +## 5. 정보 분류 원칙 + +코드가 쓸 수 있는 것: + +- node id +- edge id +- source id +- count +- existence +- graph kind +- created_at +- depth number +- token/char budget status +- schema pass/fail + +코드가 쓰면 안 되는 것: + +- 이 기억이 사용자에게 중요하다는 의미 판단 +- 이 capsule이 어떤 주제라는 판단 +- 이 graph node가 어떤 질문에 유용하다는 판단 +- 요약/해석/의도/감정 + +이번 발주에서 LLM 의미 판단은 기본 `not_run`이다. + +## 6. 금지 + +- `songryeon_core/night_government` 패키지를 되살리지 않는다. +- `MemoryRecord`, `NightGovernmentPacket`, `MemoryActivationItem` 구조를 재사용하지 않는다. +- 새 독립 기억장 JSONL DB를 만들지 않는다. +- 외부 Neo4j/SQLite/vector DB 연결을 이번 발주에서 열지 않는다. +- 의미축 CoreEgo 연결을 만들지 않는다. +- R1/R2/R3 LLM loop를 구현하지 않는다. +- R route를 node_1에 연결하지 않는다. +- W loop/scheduler/장기기억 자동 승격을 열지 않는다. +- summary를 raw node 속성에 덮어쓰지 않는다. +- 코드가 의미 요약을 쓰지 않는다. + +## 7. 파일 후보 + +구현자는 실제 코드 구조를 읽고 조정한다. + +후보: + +```text +songryeon_core/core/schemas.py +songryeon_core/core/schema_parts/ +songryeon_core/core/data_store.py +songryeon_core/nodes/node_0_memory_supplier.py +songryeon_core/runtime/dry_run.py +songryeon_core/runtime/terminal_view.py +songryeon_core/runtime/smoke_test.py +tests/test_order_139_graph_memory_foundation.py +``` + +schema가 비대해질 경우 `schema_parts`에 새 모듈로 분리하고 compatibility layer를 유지한다. + +## 8. 체크포인트 루틴 + +작업 전: + +```powershell +git status --short --branch +python -m compileall songryeon_core main.py +python -m pytest +python main.py smoke-test +``` + +작업 후: + +```powershell +python -m compileall songryeon_core main.py +python -m pytest +python main.py smoke-test +git diff --check +``` + +전체 pytest/smoke가 오래 걸리더라도 이번 작업은 기반 작업이므로 생략하지 않는다. + +## 9. 추가 테스트 요구 + +pytest를 추가한다. + +필수 테스트: + +1. 같은 `TurnStateCapsule`을 두 번 graph ingest해도 raw capsule node가 중복 생성되지 않는다. +2. raw capsule graph node는 capsule의 절대정보 좌표만 담고 의미 요약을 담지 않는다. +3. CoreEgo root -> TimeAxis -> TimeBundle -> RawCapsule edge가 생성된다. +4. 의미축/semantic topic node가 생성되지 않는다. +5. raw node의 `summary_depth=0`, `source_leaf_count=1`, `source_summary_count=0`이 보존된다. +6. RLoopGraphGuidePacket이 entry node 목록/count/depth range/source id를 담는다. +7. RLoopGraphGuidePacket의 LLM hint는 `not_run`이다. +8. guide packet이 node_1 routing이나 node_3 answer에 자동 주입되지 않는다. +9. runtime/smoke summary에 graph memory guide 생성 상태가 노출된다. + +## 10. 완료 보고에 반드시 포함할 것 + +완료 보고에는 다음을 적는다. + +- Graph memory frame/schema를 어디에 만들었는지 +- raw capsule node idempotency를 어떻게 보장했는지 +- CoreEgo 시간축 edge 구조가 어떻게 생겼는지 +- summary depth/source count 계산이 어디에 있는지 +- RLoopGraphGuidePacket이 어디서 생성되는지 +- LLM 의미 판단이 `not_run`으로 남아 있는지 +- 금지 항목을 열지 않았는지 +- compileall/pytest/smoke/diff-check 결과 + +## 11. 다음 발주 후보 + +ORDER_140 후보: + +```text +R1/R2/R3 frame-only state machine audit +``` + +ORDER_141 후보: + +```text +CoreEgoGuideWorker LLM traversal hint generation +``` + +ORDER_142 후보: + +```text +External graph DB adapter boundary +``` + +외부 DB 연결은 ORDER_139에서 하지 않는다. + +먼저 graph memory 문법과 guide packet이 송련 메타정보 원칙을 지키는지 잠근다. + diff --git a/Administrative_Reform_1/04_Orders/ORDER_140_R_LOOP_FRAME_ONLY_STATE_MACHINE_AUDIT_V0_CANDIDATE.md b/Administrative_Reform_1/04_Orders/ORDER_140_R_LOOP_FRAME_ONLY_STATE_MACHINE_AUDIT_V0_CANDIDATE.md new file mode 100644 index 0000000..e0a7f8c --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_140_R_LOOP_FRAME_ONLY_STATE_MACHINE_AUDIT_V0_CANDIDATE.md @@ -0,0 +1,178 @@ +# ORDER_140_R_LOOP_FRAME_ONLY_STATE_MACHINE_AUDIT_V0_CANDIDATE + +## Candidate Status + +이 문서는 후보 발주서로 작성되었고, 2026-06-30 사용자 승인 후 frame-only 구현으로 승격되었다. + +구현 기록: + +- `Administrative_Reform_1/05_Execution_Records/order_140_r_loop_frame_state_machine_2026_06_30_001.md` + +초기 후보 상태의 금지선은 계속 유지한다. + +이 발주서는 R루프를 실제 route로 여는 작업이 아니다. + +## 1. 목표 + +R루프의 R1/R2/R3/controller/return summary를 코드로 바로 구현하기 전에, 필요한 frame과 상태기계만 설계하고 감사한다. + +이번 후보 발주의 핵심은 다음이다. + +```text +R루프를 실행하지 않는다. +R루프가 실행된다면 어떤 frame과 상태 전이가 필요한지 먼저 고정한다. +``` + +## 2. 선행 조건 + +- ORDER_139가 완료되어 graph memory foundation과 RLoopGraphGuidePacket의 기본 형태가 존재해야 한다. +- `Administrative_Reform_1/00_Philosophy/R_Loop_Graph_Guide_Philosophy_2026_06_30.md`를 읽어야 한다. +- `Administrative_Reform_1/00_Philosophy/Night_Government_Graph_Memory_Philosophy_2026_06_30.md`를 읽어야 한다. + +## 3. 제안 frame + +후보 frame: + +```text +R1GraphGoalFrame +RLoopBudgetFrame +R2GraphNodeSelectionFrame +R3GraphInspectionFrame +RLoopContinuationFrame +RLoopReturnSummaryFrame +``` + +이번 후보 발주가 구현으로 승격되더라도, 처음에는 dataclass/schema와 validator/fake test까지만 다룬다. + +## 4. R1 역할 + +R1은 graph guide와 사용자 질문을 보고 탐색 목표와 예산을 정한다. + +후보 필드: + +```text +graph_search_goal +required_information_granularity +allowed_summary_depth +max_traversal_depth +max_branch_switches +max_node_reads +max_context_tokens +stop_condition +source_graph_guide_packet_id +source_data_ids +source_trace_ids +generated_by +info_class +semantic_judgement_status +``` + +R1의 목표/예산 이유가 LLM 산출이면 relative/mixed다. + +코드는 R1의 의미 판단을 대신하지 않는다. + +## 5. R2 역할 + +R2는 후보 graph node 중 하나를 고른다. + +후보 필드: + +```text +selection_scope +available_graph_node_ids +selected_graph_node_id +selection_reason +expected_information_granularity +expected_source_kind +source_r1_goal_frame_id +generated_by +info_class +semantic_judgement_status +``` + +R2가 선택할 수 있는 node id는 code가 제공한 `available_graph_node_ids` 안으로 제한한다. + +허용 목록 밖 ID는 schema 실패다. + +## 6. R3 역할 + +R3는 선택된 graph node를 검사한다. + +후보 필드: + +```text +inspected_graph_node_id +node_kind +child_node_count +child_node_ids +summary_depth +source_leaf_count +current_information_granularity +sufficiency_status +granularity_problem_status +branch_problem_status +recommended_next_action +inspection_reason +source_r2_selection_frame_id +generated_by +info_class +semantic_judgement_status +``` + +중요 구분: + +```text +농도 문제 +-> 같은 가지는 맞지만 더 낮은 농도 정보가 필요하다. + +가지 문제 +-> 애초에 선택한 가지가 맞지 않는다. +``` + +## 7. Controller 상태 후보 + +```text +stop_sufficient +continue_deeper +continue_switch_branch +stop_budget_exhausted +stop_no_actionable_path +stop_failed_final +``` + +controller는 의미 판단을 새로 하지 않는다. + +controller는 R3의 구조화 status와 예산 숫자만 본다. + +## 8. 금지 + +- R route를 node_1에 연결하지 않는다. +- 실제 graph DB traversal을 자동 실행하지 않는다. +- R1/R2/R3 LLM prompt를 구현하지 않는다. +- 의미축 CoreEgo를 만들지 않는다. +- node_3 최종 답변에 R 결과를 주입하지 않는다. +- code가 graph node 선택 이유를 의미적으로 작성하지 않는다. + +## 9. 테스트 후보 + +구현으로 승격될 경우 테스트: + +1. R2 selection은 허용된 graph node id만 통과한다. +2. 허용 목록 밖 graph node id는 validator가 거부한다. +3. R3는 raw leaf, bundle, summary node kind를 구조적으로 구분한다. +4. `continue_deeper`와 `continue_switch_branch`가 서로 다른 status로 남는다. +5. budget exhausted 시 controller가 loop를 닫는다. +6. 모든 LLM 의미 판단 필드는 LLM이 없으면 `not_run` 또는 `failed`로 남는다. + +## 10. 완료 조건 후보 + +아직 구현 금지. + +승격 시 완료 조건: + +```powershell +python -m compileall songryeon_core main.py +python -m pytest +python main.py smoke-test +git diff --check +``` diff --git a/Administrative_Reform_1/04_Orders/ORDER_141_CORE_EGO_GUIDE_WORKER_LLM_HINTS_V0_CANDIDATE.md b/Administrative_Reform_1/04_Orders/ORDER_141_CORE_EGO_GUIDE_WORKER_LLM_HINTS_V0_CANDIDATE.md new file mode 100644 index 0000000..d91855e --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_141_CORE_EGO_GUIDE_WORKER_LLM_HINTS_V0_CANDIDATE.md @@ -0,0 +1,109 @@ +# ORDER_141_CORE_EGO_GUIDE_WORKER_LLM_HINTS_V0 + +## Status + +구현 완료. + +ORDER_139와 ORDER_140 결과 위에서 구현한다. + +실행 기록: + +- `Administrative_Reform_1/05_Execution_Records/order_141_core_ego_guide_worker_hints_2026_06_30_001.md` + +## 1. 목표 + +RLoopGraphGuidePacket의 code-generated graph snapshot 위에 LLM 기반 traversal hint를 추가하는 후보 발주다. + +이번 후보의 핵심: + +```text +code는 graph count/entry/depth/source를 쓴다. +LLM은 그 snapshot을 보고 R루프가 어디서 시작하면 좋을지 hint를 쓴다. +hint는 mixed 정보로 source bundle과 함께 기록한다. +``` + +## 2. 선행 조건 + +- ORDER_139 완료. +- RLoopGraphGuidePacket이 code-generated absolute/status 정보로 생성되어야 한다. +- ORDER_140에서 R루프 frame/state machine이 감사되어야 한다. + +## 3. LLM이 쓸 수 있는 것 + +LLM Guide Worker 후보 출력: + +```text +recommended_entry_node_ids +avoid_entry_node_ids +traversal_strategy_hint +reason_summary +risk_notes +expected_depth_policy +source_graph_node_ids +source_data_ids +generated_by +info_class = mixed +semantic_judgement_status = ran +``` + +이 정보는 최종 진실이 아니다. + +R루프가 graph DB를 탐색할 때 참고하는 안내판이다. + +## 4. code가 쓸 수 있는 것 + +code는 다음만 쓴다. + +- graph node count +- edge count +- node kind count +- depth range +- source leaf count range +- available entry node id 목록 +- schema validation status +- LLM call trace/data id + +code는 추천 이유를 쓰지 않는다. + +## 5. 검증 경계 + +LLM hint는 다음 검증을 통과해야 한다. + +- 추천 entry id가 available entry node 목록 안에 있다. +- source graph node id가 실제 snapshot 안에 있다. +- hint가 빈 문자열이 아니다. +- generated_by가 LLM임을 드러낸다. +- info_class는 mixed다. +- semantic_judgement_status는 ran 또는 failed다. + +실패 시 code fallback으로 추천을 만들지 않는다. + +실패 상태만 기록한다. + +## 6. 금지 + +- R route를 열지 않는다. +- R1/R2/R3를 실제 실행하지 않는다. +- LLM hint를 절대정보처럼 표시하지 않는다. +- code fallback으로 "이 노드가 좋아 보인다"는 의미 판단을 만들지 않는다. +- node_3 최종 답변에 바로 사용하지 않는다. +- 의미축 graph hierarchy를 만들지 않는다. + +## 7. 테스트 후보 + +1. LLM이 허용된 entry node id를 추천하면 pass. +2. 허용 목록 밖 entry node id를 추천하면 schema 실패. +3. LLM 실패 시 recommendation은 비고, status만 failed로 남는다. +4. hint는 `info_class=mixed`와 source bundle을 가진다. +5. RLoopGraphGuidePacket의 code-generated count와 LLM hint가 분리된다. + +## 8. 완료 조건 + +구현 후 다음을 통과해야 한다. + +```powershell +python -m compileall songryeon_core main.py +python -m pytest +python main.py smoke-test +git diff --check +``` diff --git a/Administrative_Reform_1/04_Orders/ORDER_142_EXTERNAL_GRAPH_DB_ADAPTER_BOUNDARY_V0_CANDIDATE.md b/Administrative_Reform_1/04_Orders/ORDER_142_EXTERNAL_GRAPH_DB_ADAPTER_BOUNDARY_V0_CANDIDATE.md new file mode 100644 index 0000000..41d2cc3 --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_142_EXTERNAL_GRAPH_DB_ADAPTER_BOUNDARY_V0_CANDIDATE.md @@ -0,0 +1,102 @@ +# ORDER_142_EXTERNAL_GRAPH_DB_ADAPTER_BOUNDARY_V0_CANDIDATE + +## Candidate Status + +이 문서는 후보 발주서로 작성되었고, 2026-06-30 사용자 승인 후 adapter boundary 구현으로 승격되었다. + +구현 기록: + +- `Administrative_Reform_1/05_Execution_Records/order_142_external_graph_db_adapter_boundary_2026_06_30_001.md` + +외부 DB를 바로 붙이는 발주가 아니다. + +## 1. 목표 + +외부 graph DB를 붙이기 전에 adapter boundary를 설계한다. + +핵심: + +```text +송련 내부 graph frame/schema가 먼저다. +외부 DB는 그 frame을 저장/조회하는 adapter일 뿐이다. +``` + +이번 후보 발주의 목적은 Neo4j/SQLite/vector DB 중 하나를 바로 고르는 것이 아니라, 어떤 DB를 붙여도 송련 메타정보 원칙이 깨지지 않게 boundary를 정하는 것이다. + +## 2. 선행 조건 + +- ORDER_139 완료. +- GraphMemoryNodeFrame/Edge/Snapshot의 최소 schema가 존재. +- graph node idempotency가 테스트로 잠겨 있어야 한다. + +## 3. Adapter 후보 인터페이스 + +후보: + +```text +GraphMemoryStoreProtocol +InMemoryGraphMemoryStore +JsonlGraphMemoryStore +ExternalGraphMemoryStoreAdapter +``` + +필수 method 후보: + +```text +upsert_node(node_frame) +upsert_edge(edge_frame) +get_node(node_id) +list_children(node_id) +list_core_ego_entries(axis) +snapshot() +``` + +adapter는 의미 판단을 하지 않는다. + +adapter는 저장/조회만 한다. + +## 4. 금지 + +- Neo4j/SQLite/vector DB 의존성을 바로 추가하지 않는다. +- 의미 검색을 바로 구현하지 않는다. +- embedding 기반 retrieval을 열지 않는다. +- graph DB 안에서 relative/mixed source provenance를 생략하지 않는다. +- 외부 DB record가 내부 schema를 우회하게 하지 않는다. + +## 5. 메타정보 원칙 + +외부 DB에 저장되는 모든 relative/mixed node는 다음을 보존해야 한다. + +```text +info_class +generated_by +semantic_judgement_status +source_graph_node_ids +source_trace_ids +source_data_ids +source_bundle_kind +review_status +``` + +외부 DB가 이 필드를 표현하지 못하면, 그 DB는 첫 adapter 대상으로 부적합하다. + +## 6. 테스트 후보 + +1. in-memory store에 node/edge를 upsert하고 다시 조회할 수 있다. +2. 같은 node id를 두 번 upsert해도 중복되지 않는다. +3. source provenance 필드가 round-trip 된다. +4. external adapter를 쓰지 않아도 전체 smoke가 통과한다. +5. adapter가 의미 분류나 topic label을 생성하지 않는다. + +## 7. 완료 조건 후보 + +아직 구현 금지. + +승격 시: + +```powershell +python -m compileall songryeon_core main.py +python -m pytest +python main.py smoke-test +git diff --check +``` diff --git a/Administrative_Reform_1/04_Orders/ORDER_143_R_LOOP_NODE0_MEMORY_PACKET_HANDOFF_V0_CANDIDATE.md b/Administrative_Reform_1/04_Orders/ORDER_143_R_LOOP_NODE0_MEMORY_PACKET_HANDOFF_V0_CANDIDATE.md new file mode 100644 index 0000000..7489891 --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_143_R_LOOP_NODE0_MEMORY_PACKET_HANDOFF_V0_CANDIDATE.md @@ -0,0 +1,118 @@ +# ORDER_143_R_LOOP_NODE0_MEMORY_PACKET_HANDOFF_V0 + +## Status + +구현 완료. + +ORDER_139~142 결과 위에서 구현한다. + +실행 기록: + +- `Administrative_Reform_1/05_Execution_Records/order_143_r_loop_node0_memory_packet_handoff_2026_06_30_001.md` + +## 1. 목표 + +0 기억공급관이 R루프에 graph guide와 graph source 좌표를 안전하게 넘기는 handoff packet을 설계한다. + +이번 후보는 R루프 실행 자체가 아니다. + +핵심: + +```text +0은 R루프에게 graph memory guide 좌표를 공급한다. +0은 의미 판단을 하지 않는다. +0은 R루프 trace/data 좌표를 downstream에 잃지 않게 전달한다. +``` + +## 2. 선행 조건 + +- ORDER_139 완료. +- RLoopGraphGuidePacket이 존재. +- ORDER_140 frame audit 완료. +- 가능하면 ORDER_142 adapter boundary 확정. + +## 3. 후보 packet + +후보 이름: + +```text +MemoryPacketForRLoop +RLoopMemoryHandoffPacket +Node0RLoopGraphGuidePacket +``` + +후보 필드: + +```text +packet_id +target = R_LOOP +mode = graph_guide_handoff +graph_snapshot_id +r_loop_graph_guide_packet_id +available_entry_node_ids +node_kind_counts +summary_depth_range +source_graph_node_ids +source_data_ids +source_trace_ids +generated_by = CODE:node_0_memory_supplier +info_class = absolute +semantic_judgement_status = not_run +``` + +## 4. 0의 책임 + +0이 할 수 있는 것: + +- guide packet id를 복사한다. +- graph snapshot id를 복사한다. +- entry node id 목록을 복사한다. +- count/depth/status를 복사한다. +- source ids를 보존한다. + +0이 하면 안 되는 것: + +- 어떤 graph node가 의미상 적합한지 고른다. +- 사용자의 의도에 맞는 topic을 만든다. +- R1/R2/R3 판단을 대신한다. +- final answer에 graph memory 내용을 직접 말한다. + +## 5. Runtime 표시 후보 + +terminal/runtime에는 다음 정도만 표시한다. + +```text +r_loop_graph_guide: status=available|missing +entry_nodes=N +summary_depth_range=min..max +semantic_hint_status=not_run|ran|failed +``` + +내부 raw graph id를 사용자 최종 답변에 노출하지 않는다. + +## 6. 금지 + +- node_1 route=R 연결 금지. +- R루프 자동 실행 금지. +- node_3 답변에 graph memory 자동 주입 금지. +- 0이 graph relevance 판단 금지. +- 의미축 graph hierarchy 생성 금지. + +## 7. 테스트 후보 + +1. 0이 RLoopGraphGuidePacket id를 handoff packet에 보존한다. +2. source graph/data/trace id가 누락되지 않는다. +3. 0의 packet은 `semantic_judgement_status=not_run`이다. +4. guide packet이 없으면 missing 상태로 닫고 code fallback 의미 판단을 만들지 않는다. +5. terminal runtime은 count/status만 표시한다. + +## 8. 완료 조건 + +구현 후 다음을 통과해야 한다. + +```powershell +python -m compileall songryeon_core main.py +python -m pytest +python main.py smoke-test +git diff --check +``` diff --git a/Administrative_Reform_1/04_Orders/ORDER_144_R_ROUTE_DRY_RUN_ONLY_V0_CANDIDATE.md b/Administrative_Reform_1/04_Orders/ORDER_144_R_ROUTE_DRY_RUN_ONLY_V0_CANDIDATE.md new file mode 100644 index 0000000..23fbd67 --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_144_R_ROUTE_DRY_RUN_ONLY_V0_CANDIDATE.md @@ -0,0 +1,95 @@ +# ORDER_144_R_ROUTE_DRY_RUN_ONLY_V0 + +## Status + +구현 완료. + +ORDER_139~143 결과 위에서 구현한다. + +이 발주는 Qwen live R route가 아니다. + +실행 기록: + +- `Administrative_Reform_1/05_Execution_Records/order_144_r_route_dry_run_only_2026_06_30_001.md` + +## 1. 목표 + +R route를 실제 qwen-chat에 여는 것이 아니라, dry-run/fake adapter 환경에서만 R루프 골격을 좁게 실행해 본다. + +핵심: + +```text +route=R을 live runtime에 열지 않는다. +dry-run/fake-turn에서만 R1/R2/R3 frame 흐름을 검증한다. +``` + +## 2. 선행 조건 + +- ORDER_139 graph memory foundation 완료. +- ORDER_140 R frame/state machine 설계 완료. +- ORDER_143 node_0 R handoff packet 완료. + +## 3. dry-run 흐름 후보 + +```text +0 -> R guide handoff +R1 fake goal frame +R2 fake selection frame +R3 fake inspection frame +R continuation controller +R return summary +1 receives R return summary +``` + +이 흐름은 fake adapter와 deterministic fixture만 사용한다. + +LLM 호출은 하지 않는다. + +## 4. R return summary 후보 + +필드 후보: + +```text +r_loop_task_status +selected_entry_node_ids +inspected_graph_node_ids +final_information_granularity +summary_depth_used +continuation_status +budget_status +source_graph_node_ids +source_data_ids +source_trace_ids +generated_by +info_class +semantic_judgement_status +``` + +## 5. route 연결 경계 + +이번 후보 발주가 승격되더라도 다음은 금지한다. + +- node_1 LLM/router가 실제 route=R을 고르게 하지 않는다. +- qwen-chat에서 R루프를 실행하지 않는다. +- node_3 final answer에 R 결과를 주입하지 않는다. +- R 결과를 stable CoreEgo truth로 표시하지 않는다. + +## 6. 테스트 후보 + +1. dry-run fixture에서 R handoff -> R1 -> R2 -> R3 -> return summary가 생성된다. +2. fake R2는 허용된 graph node id만 선택한다. +3. fake R3의 `continue_deeper` 결과가 controller에 보존된다. +4. 예산 소진 시 `stop_budget_exhausted`로 닫힌다. +5. qwen-turn/qwen-chat 기본 route에는 R이 열리지 않는다. +6. terminal runtime은 dry-run R 상태를 code/fake로 표시한다. + +## 7. 완료 조건 + +구현 후 다음을 통과해야 한다. + +```powershell +python -m compileall songryeon_core main.py +python -m pytest +python main.py smoke-test +git diff --check +``` diff --git a/Administrative_Reform_1/04_Orders/ORDER_145_R_LOOP_PRE_LIVE_ROUTE_AUDIT_BASELINE_V0.md b/Administrative_Reform_1/04_Orders/ORDER_145_R_LOOP_PRE_LIVE_ROUTE_AUDIT_BASELINE_V0.md new file mode 100644 index 0000000..1a1c439 --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_145_R_LOOP_PRE_LIVE_ROUTE_AUDIT_BASELINE_V0.md @@ -0,0 +1,89 @@ +# ORDER_145_R_LOOP_PRE_LIVE_ROUTE_AUDIT_BASELINE_V0 + +## Status + +구현 완료. + +후속 변경: + +- ORDER_146 이후에도 기본 route=R은 닫혀 있다. +- 단, 명시 실험 플래그가 있을 때만 `route=R`이 조건부로 허용될 수 있다. + +실행 기록: + +- `Administrative_Reform_1/05_Execution_Records/order_145_r_loop_pre_live_route_audit_baseline_2026_07_01_001.md` + +## 1. 목표 + +ORDER_139~144로 만들어진 graph memory/R loop 골격이 아직 live route가 아니라는 사실을 감사하고 테스트 기준선으로 잠근다. + +핵심: + +```text +R은 지금 dry-run opt-in skeleton이다. +qwen-chat/live node_1 route=R은 아직 열지 않는다. +``` + +## 2. 현재 코드 사실 + +- `RoutingDecisionFrame` validator는 route를 `L` 또는 `2`로만 허용한다. +- node_1 LLM router payload도 route를 `L` 또는 `2`로만 허용한다. +- `run_dry_turn()` 기본값은 `enable_r_route_dry_run=False`다. +- `enable_r_route_dry_run=True`일 때만 deterministic R1/R2/R3/budget/continuation/return summary frame이 생성된다. +- R dry-run frame은 `generated_by=CODE:R_LOOP_DRY_RUN_ONLY`, `semantic_judgement_status=not_run`으로 닫혀 있다. +- R1/R2/R3 placeholder frame은 아직 LLM 판단을 실행하지 않은 `mixed/not_run`이다. +- R budget/continuation/return summary 같은 code control frame은 `absolute/not_run`이다. +- R output은 node_1 routing decision이나 node_3 final report source로 자동 주입되지 않는다. + +## 3. 감사 질문 + +1. 현재 node_1이 live route=R을 선택할 수 있는가? +2. R dry-run skeleton이 기본 턴에서 몰래 실행되는가? +3. R dry-run output이 node_1/node_3 근거로 섞이는가? +4. R frame이 LLM 판단이나 graph DB 진실처럼 보이는가? +5. 다음 단계에서 live R route를 열기 전에 어떤 경계가 남아 있는가? + +## 4. 이번 작업 범위 + +- ORDER_145 문서화. +- R pre-live boundary 전용 pytest 추가. +- 실행 기록 추가. +- `04_Orders/README.md`, `05_Execution_Records/README.md` 갱신. + +## 5. 금지 + +- node_1 route set에 `R`을 추가하지 않는다. +- qwen-chat/live runtime에서 R route를 실행하지 않는다. +- R1/R2/R3 LLM prompt를 새로 만들지 않는다. +- node_3 final answer에 R 결과를 주입하지 않는다. +- external DB/Neo4j 연결을 열지 않는다. +- semantic axis graph를 만들지 않는다. +- code가 graph 의미 판단을 대신 생성하지 않는다. + +## 6. 테스트 기준 + +1. 기본 `RoutingDecisionFrame(route="R")`는 validator에서 실패해야 한다. +2. node_1 LLM router payload가 `route="R"`을 내면 실패해야 한다. +3. 기본 `run_dry_turn()`은 R dry-run output을 만들지 않아야 한다. +4. `run_dry_turn(enable_r_route_dry_run=True)`에서만 R dry-run output이 생겨야 한다. +5. R dry-run output은 `CODE:R_LOOP_DRY_RUN_ONLY` / `not_run`이어야 한다. +6. R budget/continuation/return summary control frame은 `absolute/not_run`이어야 한다. +7. R dry-run output은 node_1 routing decision 또는 node_3 report의 `source_data_ids`에 섞이지 않아야 한다. + +## 7. 완료 조건 + +```powershell +python -m compileall songryeon_core main.py +python -m pytest +python main.py smoke-test +git diff --check +``` + +## 8. 다음 후보 + +ORDER_145 이후 live R route를 열려면 먼저 별도 발주서에서 다음을 정해야 한다. + +- node_1이 어떤 조건에서 route=R을 허용할지 +- R route가 node_3 답변에 언제, 어떤 형태로 연결될지 +- R1/R2/R3가 LLM 판단을 할 때 상대/혼합정보를 어떻게 기록할지 +- R loop 실패/부분성공을 node_3 태도에 어떻게 전달할지 diff --git a/Administrative_Reform_1/04_Orders/ORDER_146_R_ROUTE_EXPERIMENTAL_GATE_AND_ALLOWED_ROUTE_SET_V0.md b/Administrative_Reform_1/04_Orders/ORDER_146_R_ROUTE_EXPERIMENTAL_GATE_AND_ALLOWED_ROUTE_SET_V0.md new file mode 100644 index 0000000..ad52395 --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_146_R_ROUTE_EXPERIMENTAL_GATE_AND_ALLOWED_ROUTE_SET_V0.md @@ -0,0 +1,78 @@ +# ORDER_146_R_ROUTE_EXPERIMENTAL_GATE_AND_ALLOWED_ROUTE_SET_V0 + +## Status + +구현 완료. + +실행 기록: + +- `Administrative_Reform_1/05_Execution_Records/order_146_r_route_experimental_gate_2026_07_01_001.md` + +## 1. 목표 + +ORDER_145 기준선 위에서 live R route를 바로 일반 개방하지 않고, 명시 실험 플래그 뒤에서만 node_1 LLM이 `route=R`을 고를 수 있게 한다. + +핵심: + +```text +기본 route set은 L/2 유지. +--enable-r-route-experimental이 있을 때만 allowed_routes에 R 추가. +code는 R을 의미적으로 고르지 않는다. +R 선택은 node_1 LLM decision으로만 기록한다. +``` + +## 2. 구현 범위 + +- node_1 LLM router input payload의 `allowed_routes`를 조건부로 확장한다. +- `RoutingDecisionFrame(route="R")`는 다음 조건을 모두 만족할 때만 통과한다. + - `policy_flag=enable_r_route_experimental` + - `route_source=LLM:*` + - `llm_routing_status=ran` + - `route_rule_id=llm_router` + - `expected_next_0_mode=r_loop_graph_guide_handoff` +- `run_dry_turn(..., enable_r_route_experimental=True)`에서 node_1이 R을 고르면 experimental R skeleton을 실행한다. +- R experimental skeleton은 `R:experimental:*` ID와 `generated_by=CODE:R_ROUTE_EXPERIMENTAL_GATE`로 기록한다. +- R experimental skeleton 이후에는 code가 `route=2`로 닫아 node_2/node_3 경계로 보낸다. +- CLI에는 `--enable-r-route-experimental` 플래그를 추가한다. + +## 3. 메타정보 경계 + +- R route 선택 자체는 node_1 LLM 판단이다. +- R route 허용 여부는 code 정책이다. +- R skeleton 실행 frame은 code-generated 구조 점검이다. +- R skeleton의 R1/R2/R3 placeholder는 아직 `semantic_judgement_status=not_run`이다. +- R budget/continuation/return summary는 code가 확인 가능한 제어 장부다. + +## 4. 금지 + +- 기본 qwen-chat에서 route=R을 열지 않는다. +- 키워드/휴리스틱으로 code가 R을 고르지 않는다. +- R1/R2/R3 LLM prompt를 새로 만들지 않는다. +- external DB/Neo4j 연결을 열지 않는다. +- semantic graph axis를 만들지 않는다. +- node_3가 R 결과를 완전한 장기기억 탐색 성공으로 말하게 하지 않는다. + +## 5. 테스트 기준 + +1. 실험 플래그가 없으면 node_1 LLM payload `route=R`은 실패하거나 fallback으로 닫힌다. +2. 실험 플래그가 있으면 `allowed_routes`에 R이 들어가고 `route=R` payload가 통과한다. +3. `RoutingDecisionFrame(route=R)`은 policy flag/LLM source/R handoff mode 없이는 실패한다. +4. experimental R이 실행되면 `route:R -> R:experimental:* -> route:2`로 닫힌다. +5. terminal은 dry-run R과 experimental R을 구분해서 표시한다. + +## 6. 완료 조건 + +```powershell +python -m compileall songryeon_core main.py +python -m pytest +python main.py smoke-test +git diff --check +``` + +## 7. 다음 후보 + +ORDER_147에서는 다음 중 하나를 결재해야 한다. + +- R1/R2/R3 LLM화를 시작할지 +- R return summary를 node_3 brief에 구조화해 전달할지 +- R route live test pack을 먼저 만들지 diff --git a/Administrative_Reform_1/04_Orders/ORDER_147_R_RESULT_TO_NODE3_BRIEF_V0.md b/Administrative_Reform_1/04_Orders/ORDER_147_R_RESULT_TO_NODE3_BRIEF_V0.md new file mode 100644 index 0000000..b22476d --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_147_R_RESULT_TO_NODE3_BRIEF_V0.md @@ -0,0 +1,72 @@ +# ORDER_147_R_RESULT_TO_NODE3_BRIEF_V0 + +## Status + +구현 완료. + +실행 기록: + +- `Administrative_Reform_1/05_Execution_Records/order_147_r_result_to_node3_brief_2026_07_01_001.md` + +## 1. 목표 + +ORDER_146의 experimental R route skeleton 결과를 node_3 input brief에 구조화해서 전달한다. + +핵심: + +```text +R route skeleton이 실행됐다는 절대 장부는 node_3가 볼 수 있게 한다. +하지만 node_3가 이를 완전한 graph memory 탐색 성공으로 말하지 못하게 한다. +``` + +## 2. 구현 범위 + +- `Node3RLoopResultMaterial`을 추가한다. +- `Node3InputBriefFrame.r_loop_result_material`에 최신 `R:experimental:return_summary_frame` 요약을 복사한다. +- node_3 LLM payload에 safe R result summary를 넣는다. +- node_3 grounding block에 R 탐색 실험 상태를 표시한다. +- runtime terminal view가 node_3 brief 안의 R result material을 표시한다. +- node_3 prompt에 R skeleton/partial 결과의 사용자-facing 경계를 추가한다. + +## 3. 메타정보 경계 + +- R return summary material은 code가 DataStore record에서 복사한 절대 상태 장부다. +- `generated_by=CODE:R_ROUTE_EXPERIMENTAL_GATE` +- `info_class=absolute` +- `semantic_judgement_status=not_run` +- `attitude_hint`는 skeleton/partial 결과를 성공처럼 말하지 않게 하는 표시다. + +## 4. 금지 + +- R1/R2/R3 LLM node를 열지 않는다. +- R loop 자동 반복을 열지 않는다. +- external DB/Neo4j 연결을 열지 않는다. +- 의미축 graph node를 만들지 않는다. +- node_3가 R skeleton을 완전한 장기기억 탐색 성공으로 표현하게 하지 않는다. +- code가 R 결과의 의미 판단을 대신 만들지 않는다. + +## 5. 테스트 기준 + +1. experimental R route가 실행되면 node_3 input brief에 `r_loop_result_material`이 들어간다. +2. 해당 material의 source data id는 `R:experimental:return_summary_frame`이다. +3. material은 `generated_by=CODE:R_ROUTE_EXPERIMENTAL_GATE`, `info_class=absolute`, `semantic_judgement_status=not_run`을 유지한다. +4. node_3 grounding block은 R 탐색 실험 상태와 한계를 표시한다. +5. node_3 LLM payload는 raw internal ID 대신 safe boundary만 제공한다. +6. terminal은 node_3 brief 안의 R result material을 표시한다. + +## 6. 완료 조건 + +```powershell +python -m compileall songryeon_core main.py +python -m pytest +python main.py smoke-test +git diff --check +``` + +## 7. 다음 후보 + +ORDER_148에서는 다음 중 하나를 결재한다. + +- R live test pack 작성 +- R1/R2/R3 frame-only state machine 실제 구현 +- R result를 node_0 memory packet으로 회수하는 handoff diff --git a/Administrative_Reform_1/04_Orders/ORDER_148_GRAPH_NEXT_AND_TURN_ACCESS_LEDGER_V0.md b/Administrative_Reform_1/04_Orders/ORDER_148_GRAPH_NEXT_AND_TURN_ACCESS_LEDGER_V0.md new file mode 100644 index 0000000..e5cae03 --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_148_GRAPH_NEXT_AND_TURN_ACCESS_LEDGER_V0.md @@ -0,0 +1,102 @@ +# ORDER 148: Graph NEXT And Turn Access Ledger v0 + +## 1. Goal + +`TurnStateCapsule` 기반 graph memory에서 시간순 raw capsule 탐색과 R 루프 graph 열람 이력 추적을 작게 잠근다. + +이번 발주는 두 가지를 한다. + +1. 시간순으로 인접한 `raw_capsule` graph node 사이에 `NEXT` edge를 만든다. +2. R skeleton이 어떤 graph node를 후보로 보고, 선택하고, 검사했는지를 `TurnGraphAccessLedgerFrame`으로 남긴다. + +## 2. Background + +ORDER_139 이후 Core에는 다음 구조가 있다. + +```text +graph:core_ego:root + -> graph:axis:time + -> graph:time_bundle:{batch_id} + -> graph:raw_capsule:{turn_id} +``` + +하지만 raw capsule끼리 시간순으로 바로 이어지지 않아, 인접 턴을 볼 때 상위 time bundle을 거쳐 다시 내려가야 한다. + +또한 ORDER_140~147 이후 R skeleton에는 graph node 선택/검사 결과가 생겼지만, 해당 턴이 어떤 graph node를 보았는지 본점의 `used_sources`/`REFERENCES_MEMO` 같은 장부로 승격되지는 않았다. + +## 3. Scope + +### 3.1 NEXT edge + +- `GraphMemoryEdgeFrame.edge_kind`에 `NEXT`를 추가한다. +- `build_graph_memory_snapshot_from_capsules()`는 dedupe된 capsule 입력 순서 기준으로 인접 raw capsule node를 `NEXT` edge로 연결한다. +- `NEXT`는 의미 판단이 아니라 입력 순서와 `turn_id` 좌표에 근거한 절대정보다. +- raw 원본이나 capsule payload는 변경하지 않는다. + +### 3.2 Turn graph access ledger + +새 schema를 추가한다. + +```text +TurnGraphAccessLedgerFrame +``` + +최소 필드: + +- `frame_id` +- `turn_id` +- `turn_capsule_graph_node_id` +- `candidate_graph_node_ids` +- `selected_graph_node_ids` +- `inspected_graph_node_ids` +- `read_graph_node_ids` +- `used_as_answer_source_graph_node_ids` +- `access_records` +- `source_trace_ids` +- `source_data_ids` +- `generated_by=CODE:GRAPH_ACCESS_LEDGER` +- `info_class=absolute` +- `semantic_judgement_status=not_run` + +이번 MVP에서 `read_graph_node_ids`와 `used_as_answer_source_graph_node_ids`는 빈 배열일 수 있다. 아직 R route 결과가 최종 답변 근거로 쓰이는 단계가 아니기 때문이다. + +### 3.3 R skeleton connection + +- `run_r_loop_dry_run_skeleton()`이 R2/R3/return summary 결과에서 ledger를 생성한다. +- ledger는 DataStore에 `graph_memory:turn_access_ledger_frame`로 기록한다. +- ledger는 code가 이미 가진 graph node id를 복사할 뿐, 관련성/중요도/정답성을 판단하지 않는다. + +## 4. Non-goals + +- 외부 DB/Neo4j 연결을 열지 않는다. +- 의미축 graph node를 만들지 않는다. +- R1/R2/R3 LLM 판단을 새로 열지 않는다. +- live R route 정책을 확장하지 않는다. +- node_3 최종 답변에 R 결과를 새로 주입하지 않는다. +- graph node 원문 요약이나 기억 요약을 만들지 않는다. + +## 5. Test Plan + +필수 검증: + +```powershell +python -m compileall songryeon_core main.py +python -m pytest +python main.py smoke-test +``` + +추가 pytest: + +- raw capsule 3개를 graph memory로 만들면 `NEXT` edge 2개가 생긴다. +- `NEXT` edge는 dedupe된 capsule 순서를 따른다. +- `TurnGraphAccessLedgerFrame`은 `generated_by=CODE:GRAPH_ACCESS_LEDGER`, `info_class=absolute`, `semantic_judgement_status=not_run`을 지킨다. +- R dry-run skeleton은 selected/inspected/source graph node id를 ledger에 보존한다. +- ledger는 의미 판단 텍스트를 만들지 않는다. + +## 6. Completion Report Must Include + +- `NEXT` edge가 어디서 생성되는지 +- `TurnGraphAccessLedgerFrame`이 어디서 생성/기록되는지 +- 어떤 값이 후보/선택/검사 graph node로 들어가는지 +- 아직 하지 않은 것 +- compileall / pytest / smoke-test 결과 diff --git a/Administrative_Reform_1/04_Orders/ORDER_149_R_GRAPH_TRAVERSAL_CANDIDATE_SURFACE_V0.md b/Administrative_Reform_1/04_Orders/ORDER_149_R_GRAPH_TRAVERSAL_CANDIDATE_SURFACE_V0.md new file mode 100644 index 0000000..844208c --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_149_R_GRAPH_TRAVERSAL_CANDIDATE_SURFACE_V0.md @@ -0,0 +1,112 @@ +# ORDER 149: R Graph Traversal Candidate Surface v0 + +## 1. Goal + +R loop가 다음 graph node로 이동하기 전에, code가 확인 가능한 graph 좌표만 모아 `RGraphTraversalCandidateSurfaceFrame`으로 정리한다. + +ORDER_148은 R skeleton이 어떤 graph node를 보았는지 `TurnGraphAccessLedgerFrame`으로 남겼다. ORDER_149는 그 다음 단계로, R3가 검사한 graph node에서 이동 가능한 후보를 명시적인 후보판으로 만든다. + +## 2. Background + +현재 R skeleton은 다음 frame을 만든다. + +- `R2GraphNodeSelectionFrame` +- `R3GraphInspectionFrame` +- `RLoopContinuationFrame` +- `RLoopReturnSummaryFrame` +- `TurnGraphAccessLedgerFrame` + +그러나 R3 이후 다음 R2가 볼 수 있는 후보 목록이 별도 frame으로 정리되어 있지 않다. + +ORDER_148에서 raw capsule 사이 `NEXT` edge가 추가되었으므로, 이제 code는 다음 후보를 절대정보로 모을 수 있다. + +- inspected node의 child/source graph nodes +- inspected node의 outgoing `NEXT` target +- inspected node의 incoming `NEXT` source + +## 3. Scope + +### 3.1 New schema + +Add: + +```text +RGraphTraversalCandidateSurfaceFrame +``` + +Minimum fields: + +- `frame_id` +- `source_r3_inspection_frame_id` +- `inspected_graph_node_id` +- `child_candidate_node_ids` +- `next_candidate_node_ids` +- `previous_candidate_node_ids` +- `candidate_graph_node_ids` +- `candidate_records` +- `candidate_count` +- `source_data_ids` +- `source_trace_ids` +- `generated_by=CODE:R_GRAPH_TRAVERSAL_CANDIDATE_SURFACE` +- `info_class=absolute` +- `semantic_judgement_status=not_run` + +### 3.2 Candidate sources + +Code may copy only these coordinates: + +- `R3GraphInspectionFrame.child_node_ids` +- DataStore graph edge payloads where: + - `edge_kind=NEXT` + - `from_node_id == inspected_graph_node_id` + - `to_node_id == inspected_graph_node_id` + +Candidate records must include: + +- `candidate_node_id` +- `relation` +- `source_frame_id` or `source_edge_id` +- `source_field` + +### 3.3 R skeleton connection + +- `run_r_loop_dry_run_skeleton()` creates and records this frame after R3 inspection. +- The access ledger includes this candidate surface as an additional `candidate_seen` source. +- R2 selection behavior is not changed in this order. + +## 4. Non-goals + +- Do not implement multi-step R traversal. +- Do not let R2 choose from the new candidate surface yet. +- Do not add R LLM traversal. +- Do not open live R route beyond existing experimental gate. +- Do not add semantic relevance ranking. +- Do not connect Neo4j or external DB. +- Do not summarize graph node contents. + +## 5. Test Plan + +Required commands: + +```powershell +python -m compileall songryeon_core main.py +python -m pytest +python main.py smoke-test +``` + +Additional pytest: + +- Candidate surface copies R3 child candidates. +- Candidate surface copies outgoing `NEXT` target. +- Candidate surface copies incoming `NEXT` source as previous candidate. +- Candidate surface is code-generated absolute information. +- R skeleton records the candidate surface to DataStore. +- Access ledger includes candidate records sourced from the candidate surface. + +## 6. Completion Report Must Include + +- where the candidate surface schema is defined +- where the candidate surface is generated +- which graph relations are copied +- what was deliberately not implemented +- compileall / pytest / smoke-test result diff --git a/Administrative_Reform_1/04_Orders/ORDER_150_R_LOOP_MULTI_STEP_TRAVERSAL_DRY_RUN_V0.md b/Administrative_Reform_1/04_Orders/ORDER_150_R_LOOP_MULTI_STEP_TRAVERSAL_DRY_RUN_V0.md new file mode 100644 index 0000000..55a1b84 --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_150_R_LOOP_MULTI_STEP_TRAVERSAL_DRY_RUN_V0.md @@ -0,0 +1,98 @@ +# ORDER 150: R Loop Multi-Step Traversal Dry-Run v0 + +## 1. Goal + +ORDER_149에서 만든 `RGraphTraversalCandidateSurfaceFrame`을 R dry-run skeleton이 실제로 한 번 이상 소비하게 만든다. + +이번 발주는 live R route 확장이 아니라, dry-run/fake 환경에서만 graph traversal frame chain이 예산 안에서 여러 step 이어질 수 있는지 검증하는 작업이다. + +## 2. Background + +현재 R skeleton은 다음 흐름을 가진다. + +```text +R1 goal +R budget +R2 select entry node +R3 inspect selected node +candidate surface +continuation +return summary +access ledger +``` + +그러나 continuation이 `continue_deeper`를 내도 실제 다음 R2/R3 step은 열리지 않는다. + +ORDER_150은 다음을 확인한다. + +```text +R3 says continue_deeper +candidate surface has candidate nodes +budget remains +-> dry-run code selects the next candidate coordinate +-> R3 inspects it +-> repeat within narrow cap +``` + +## 3. Scope + +### 3.1 Multi-step dry-run only + +- `run_r_loop_dry_run_skeleton()` 안에서만 multi-step traversal을 연다. +- qwen-chat/live R route 일반 개방은 하지 않는다. +- R LLM selector는 만들지 않는다. + +### 3.2 Candidate consumption + +- R2 step 1은 기존처럼 handoff entry node를 선택한다. +- R2 step 2+는 직전 `RGraphTraversalCandidateSurfaceFrame.candidate_graph_node_ids` 안에서만 선택한다. +- 이번 dry-run의 deterministic selection은 의미 판단이 아니라 wiring verification이다. +- selection reason은 `CODE_STATUS:*`로 명시한다. + +### 3.3 Budget semantics + +- entry node를 읽는 것은 traversal depth 0으로 본다. +- child edge를 따라 한 번 내려가면 traversal depth 1이다. +- `max_traversal_depth=2`면 `time_axis -> time_bundle -> raw_capsule`까지 확인 가능하다. +- node read count는 실제 inspected graph node 수를 따른다. + +### 3.4 Summary and ledger + +- return summary는 모든 selected/inspected graph node id를 누적한다. +- access ledger는 모든 candidate/selected/inspected graph node id를 누적한다. +- candidate surface frame도 step별로 기록한다. + +## 4. Non-goals + +- 외부 DB/Neo4j 연결을 열지 않는다. +- R LLM traversal을 만들지 않는다. +- live R route policy를 넓히지 않는다. +- node_3 답변 본문에 graph traversal 내용을 새로 주입하지 않는다. +- graph node 원문 요약/의미 요약을 만들지 않는다. +- semantic relevance ranking을 만들지 않는다. + +## 5. Test Plan + +Required commands: + +```powershell +python -m compileall songryeon_core main.py +python -m pytest +python main.py smoke-test +``` + +Additional pytest: + +- default R dry-run reaches raw capsule through candidate surface within depth 2. +- step 2+ R2 selection only chooses IDs from previous candidate surface. +- final return summary accumulates selected/inspected graph nodes. +- access ledger accumulates candidate/selected/inspected graph nodes across steps. +- force budget exhausted still stops after first step. + +## 6. Completion Report Must Include + +- how many R traversal steps dry-run executes +- how candidate surface is consumed +- how budget usage is counted +- what remains unopened +- compileall / pytest / smoke-test result diff --git a/Administrative_Reform_1/04_Orders/ORDER_151_L_LOOP_ACTIVITY_LEDGER_FOR_GRAPH_BACKUP_V0.md b/Administrative_Reform_1/04_Orders/ORDER_151_L_LOOP_ACTIVITY_LEDGER_FOR_GRAPH_BACKUP_V0.md new file mode 100644 index 0000000..c620ef4 --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_151_L_LOOP_ACTIVITY_LEDGER_FOR_GRAPH_BACKUP_V0.md @@ -0,0 +1,97 @@ +# ORDER 151: L Loop Activity Ledger For Graph Backup v0 + +## 1. Goal + +L루프가 한 턴에서 만든 주요 산출물과 문서/코드 근거 좌표를 `LLoopActivityLedgerFrame`으로 한 번 더 묶는다. + +이 작업은 L 검색 전략을 바꾸지 않는다. 의미 판단, 요약, 그래프 DB 연결도 하지 않는다. +목표는 나중에 `graph:raw_capsule:{turn_id}`와 L 활동 산출물을 안전하게 연결할 수 있는 절대정보 장부 표면을 만드는 것이다. + +## 2. Background + +ORDER_148~150으로 R루프에는 다음 기반이 생겼다. + +- raw capsule 사이 `NEXT` edge +- R graph traversal candidate surface +- R turn graph access ledger +- multi-step R dry-run traversal + +반면 L루프는 `LLoopResult`, `LLoopReturnSummaryFrame`, `Node0DocumentMaterialPacketFrame` 등 원재료는 충분히 남지만, "이번 L이 어떤 DataStore record를 만들고 어떤 문서/코드 좌표를 남겼는가"를 한 장으로 모은 전용 ledger는 없다. + +## 3. Scope + +추가한다. + +- `LLoopActivityLedgerFrame` +- `record_l_loop_activity_ledger()` +- `run_dry_turn()` 배선 +- runtime 표시 +- smoke/pytest 검증 + +열지 않는다. + +- L 검색 전략 변경 +- L3 의미 판단 변경 +- R live route 확장 +- 외부 DB/Neo4j 연결 +- 그래프 노드/edge 실제 연결 +- summary layer + +## 4. Frame Contract + +`LLoopActivityLedgerFrame`은 code-generated absolute ledger다. + +필수 경계: + +- `generated_by=CODE:L_LOOP_ACTIVITY_LEDGER` +- `info_class=absolute` +- `semantic_judgement_status=not_run` +- `turn_capsule_graph_node_id=graph:raw_capsule:{turn_id}` + +주요 필드: + +- `run_frame_data_ids` +- `goal_data_ids` +- `query_plan_data_ids` +- `query_data_ids` +- `control_data_ids` +- `tool_result_data_ids` +- `tool_budget_data_ids` +- `continuation_data_ids` +- `revision_*_data_ids` +- `preserved_data_ids` +- `achievement_data_ids` +- `return_summary_frame_id` +- `document_material_packet_frame_id` +- `output_data_ids` +- `search_candidate_doc_ids` +- `read_doc_ids` +- `read_code_file_paths` +- `activity_records` +- `source_trace_ids` +- `source_data_ids` + +## 5. Rules + +- code는 기존 record ID와 count만 복사한다. +- code는 "중요한 문서", "관련 문서", "성공한 검색" 같은 의미 판단을 새로 만들지 않는다. +- `activity_records`는 `stage`, `data_id`, `source_field`만 담는다. +- 모든 activity data id는 `source_data_ids`에 포함되어야 한다. +- count는 대응 list 길이와 일치해야 한다. + +## 6. Test Plan + +```powershell +python -m compileall songryeon_core main.py +python -m pytest tests/test_order_151_l_loop_activity_ledger.py +python -m pytest +python main.py smoke-test +``` + +## 7. Done Criteria + +- dry turn에서 `L:activity_ledger_frame`이 DataStore에 기록된다. +- ledger는 `L:return_summary_frame`과 `node_0:document_material_packet_frame`을 source로 가진다. +- ledger는 read_doc/search candidate/code file 좌표를 count와 함께 보존한다. +- ledger의 `semantic_judgement_status`는 `not_run`이다. +- smoke-test가 ledger 존재와 주요 source 연결을 확인한다. diff --git a/Administrative_Reform_1/04_Orders/ORDER_152_RAW_CAPSULE_TO_ACTIVITY_LEDGER_GRAPH_LINK_V0.md b/Administrative_Reform_1/04_Orders/ORDER_152_RAW_CAPSULE_TO_ACTIVITY_LEDGER_GRAPH_LINK_V0.md new file mode 100644 index 0000000..2759abe --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_152_RAW_CAPSULE_TO_ACTIVITY_LEDGER_GRAPH_LINK_V0.md @@ -0,0 +1,91 @@ +# ORDER 152: Raw Capsule To Activity Ledger Graph Link v0 + +## 1. Goal + +`graph:raw_capsule:{turn_id}`가 같은 턴의 L/R 활동 장부를 graph-memory 표면에서 찾을 수 있게 만든다. + +ORDER_151까지는 L/R 활동 장부가 DataStore record로 존재하지만, raw capsule graph node와 graph node/edge로 직접 연결되지는 않았다. +이번 MVP는 활동 장부를 의미 요약하지 않고, code-generated graph node/edge와 link frame으로만 연결한다. + +## 2. Scope + +추가한다. + +- `activity_ledger` graph node kind +- `HAS_ACTIVITY_LEDGER` graph edge kind +- `TurnActivityGraphLinkFrame` +- `record_turn_activity_graph_links()` +- dry-run 배선 +- runtime/smoke/pytest 검증 + +열지 않는다. + +- 외부 DB/Neo4j 연결 +- summary layer +- L/R 의미 판단 +- R live route 확장 +- raw capsule payload 재작성 +- node_3 답변 주입 + +## 3. Graph Shape + +```text +graph:raw_capsule:{turn_id} + -[HAS_ACTIVITY_LEDGER]-> +graph:activity_ledger:{ledger_data_id} +``` + +`activity_ledger` node는 원본 활동 장부 DataStore record를 graph-memory에서 찾기 위한 좌표 노드다. +LLM 요약이나 중요도 판단을 담지 않는다. + +## 4. Link Frame + +`TurnActivityGraphLinkFrame` + +- `frame_id` +- `turn_id` +- `turn_capsule_graph_node_id` +- `l_loop_activity_ledger_data_ids` +- `r_graph_access_ledger_data_ids` +- `activity_ledger_graph_node_ids` +- `activity_ledger_graph_edge_ids` +- `link_records` +- `source_trace_ids` +- `source_data_ids` +- `generated_by=CODE:TURN_ACTIVITY_GRAPH_LINK_BUILDER` +- `info_class=absolute` +- `semantic_judgement_status=not_run` + +`link_records`는 다음 값만 담는다. + +- `activity_kind` +- `ledger_data_id` +- `graph_node_id` +- `edge_id` +- `source_field` + +## 5. Rules + +- code는 기존 L/R ledger record의 존재와 ID만 복사한다. +- code는 어떤 ledger가 더 중요한지 판단하지 않는다. +- raw capsule node와 ledger node 사이 edge는 결정론적 ID를 가진다. +- 같은 payload를 재기록하면 중복 생성하지 않는다. +- link frame count는 list 길이와 일치해야 한다. + +## 6. Test Plan + +```powershell +python -m compileall songryeon_core main.py +python -m pytest tests/test_order_152_raw_capsule_activity_graph_link.py +python -m pytest +python main.py smoke-test +git diff --check +``` + +## 7. Done Criteria + +- dry turn에서 `graph:turn_activity_graph_link:{turn_id}`가 기록된다. +- L ledger용 `activity_ledger` graph node가 기록된다. +- R dry-run이 켜진 경우 R access ledger용 `activity_ledger` graph node도 기록된다. +- `HAS_ACTIVITY_LEDGER` edge가 raw capsule node에서 activity ledger node로 이어진다. +- 모든 새 frame/node/edge는 `info_class=absolute`, `semantic_judgement_status=not_run`을 지킨다. diff --git a/Administrative_Reform_1/04_Orders/ORDER_153_GRAPH_MEMORY_EXPORT_INTEGRITY_AUDIT_AND_R_EXPERIMENTAL_SOURCE_RECORDING_V0.md b/Administrative_Reform_1/04_Orders/ORDER_153_GRAPH_MEMORY_EXPORT_INTEGRITY_AUDIT_AND_R_EXPERIMENTAL_SOURCE_RECORDING_V0.md new file mode 100644 index 0000000..edc6a11 --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_153_GRAPH_MEMORY_EXPORT_INTEGRITY_AUDIT_AND_R_EXPERIMENTAL_SOURCE_RECORDING_V0.md @@ -0,0 +1,80 @@ +# ORDER 153: Graph Memory Export Integrity Audit And R Experimental Source Recording v0 + +## 1. Goal + +외부 graph DB/Neo4j/Vessel 연결 전에 graph-memory DataStore 기록의 참조 무결성을 잠근다. + +이번 발주는 외부 DB 연결이 아니라, 외부 DB로 내보내기 전에 다음 두 가지를 보장하는 좁은 MVP다. + +1. R experimental route가 참조하는 graph snapshot/guide/node/edge 좌표가 DataStore에 실제 record로 남는다. +2. graph/rloop source reference가 DataStore 안에서 dangling 상태인지 검사하는 read-only audit helper가 생긴다. + +## 2. Background + +ORDER_152 감사에서 다음 문제가 확인됐다. + +```text +run_dry_turn(..., enable_r_route_experimental=True) +r_route_experimental_graph_data_ids=[] +missing_graph_or_rloop_count=14 +``` + +즉 R experimental branch는 `build_graph_memory_snapshot_from_capsules(...)`로 graph frame을 메모리에서 만들지만, 그 snapshot/guide/node/edge를 DataStore record로 보존하지 않은 채 R handoff와 R ledger source로 인용한다. + +이는 외부 DB export 전에 위험하다. + +## 3. Scope + +### 3.1 R experimental source recording + +- `decision.route == "R"` experimental branch에서 R handoff 전에 graph source records를 DataStore에 기록한다. +- 고정 ID인 `graph:core_ego:root`, `graph:axis:time`, root->axis shared edge는 final turn graph recording과 payload 충돌 가능성이 있으므로 직접 재기록하지 않는다. +- batch-specific snapshot/guide/time_bundle/raw/edge records는 기록한다. +- 기록된 graph data ids를 `r_route_experimental_graph_data_ids`와 node movement output에 반영한다. + +### 3.2 Integrity audit helper + +- DataStore를 읽어서 graph/rloop source reference가 missing인지 검사한다. +- code는 누락을 자동 보정하지 않는다. +- helper는 read-only다. +- 검사 대상은 최소한 다음이다. + - graph/rloop prefix를 가진 `source_data_ids` + - `graph_memory:edge:*`의 `from_node_id`, `to_node_id` + +### 3.3 Tests + +- ORDER_152 감사에서 재현된 R experimental dangling graph refs가 사라졌는지 확인한다. +- 일부러 missing graph/rloop ref를 가진 DataStore를 만들면 audit helper가 보고하는지 확인한다. +- default dry-run graph integrity가 pass인지 확인한다. + +## 4. Non-goals + +이번 발주는 다음을 열지 않는다. + +- Neo4j/Vessel 실제 연결 +- graph DB export 실행 +- semantic axis +- R LLM selector +- R live route 확대 +- R traversal이 `HAS_ACTIVITY_LEDGER`를 따라가는 기능 +- L/R summary generation +- node_3 답변 주입 변경 + +## 5. Test Plan + +Required commands: + +```powershell +python -m compileall songryeon_core main.py +python -m pytest tests/test_order_153_graph_memory_integrity.py +python -m pytest +python main.py smoke-test +git diff --check +``` + +## 6. Completion Report Must Include + +- R experimental dangling source ref가 어떻게 사라졌는지 +- integrity audit helper가 무엇을 검사하는지 +- 외부 DB/Neo4j를 아직 열지 않았는지 +- compileall / pytest / smoke-test 결과 diff --git a/Administrative_Reform_1/04_Orders/ORDER_154_FAST_TEST_GATE_V0.md b/Administrative_Reform_1/04_Orders/ORDER_154_FAST_TEST_GATE_V0.md new file mode 100644 index 0000000..9aeda2a --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_154_FAST_TEST_GATE_V0.md @@ -0,0 +1,59 @@ +# ORDER 154: Fast Test Gate v0 + +## 1. Goal + +전체 pytest와 smoke-test가 오래 걸리는 상황에서, 개발 중 빠르게 빨간불을 볼 수 있는 CLI test gate를 추가한다. + +이번 발주는 전체 검증을 약화하지 않는다. `python -m pytest`와 `python main.py smoke-test`는 최종 검증으로 유지한다. + +## 2. Problem + +최근 기준: + +```text +python -m pytest -q -> 147 passed, 약 14분 33초 +python main.py smoke-test -> 수 분 단위 +``` + +이 속도는 큰 변경을 잠글 때는 괜찮지만, 작은 수정 직후 매번 돌리기에는 느리다. + +## 3. Scope + +추가한다. + +- `python main.py fast-test` +- `python main.py fast-test --profile core` +- `python main.py fast-test --profile graph` +- `python main.py fast-test --dry-run` + +프로필: + +- `core`: compileall + import/schema compatibility pytest +- `graph`: compileall + current graph/R/L activity ledger boundary pytest + +## 4. Non-goals + +이번 발주는 다음을 하지 않는다. + +- 전체 pytest 기본 동작 변경 +- smoke-test 삭제/약화 +- slow marker로 기존 테스트를 숨김 +- CI 기준선 변경 +- 테스트 의미를 느슨하게 바꿈 + +## 5. Test Plan + +```powershell +python -m compileall songryeon_core main.py +python -m pytest tests/test_order_154_fast_test_gate.py tests/test_import_baseline.py -q +python main.py fast-test --dry-run +python main.py fast-test --profile core +git diff --check +``` + +큰 발주 완료 시에는 여전히 다음을 돌린다. + +```powershell +python -m pytest +python main.py smoke-test +``` diff --git a/Administrative_Reform_1/04_Orders/ORDER_155_GRAPH_SOURCE_KIND_SEPARATED_INGEST_FOUNDATION_V0.md b/Administrative_Reform_1/04_Orders/ORDER_155_GRAPH_SOURCE_KIND_SEPARATED_INGEST_FOUNDATION_V0.md new file mode 100644 index 0000000..dcd4f41 --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_155_GRAPH_SOURCE_KIND_SEPARATED_INGEST_FOUNDATION_V0.md @@ -0,0 +1,78 @@ +# ORDER 155: Graph Source Kind Separated Ingest Foundation v0 + +## 1. Goal + +심야정부가 대화 턴만이 아니라 내부 문서와 코드 파일도 다룰 수 있게, source kind별로 분리된 graph source ingest 기반을 만든다. + +이번 발주는 요약이나 의미 판단이 아니라 원본 좌표를 graph memory 표면에 올리는 작업이다. + +## 2. Core Rule + +데이터 종류는 처음부터 섞지 않는다. + +예: + +```text +internal_document bundle + -> raw internal document source nodes + +source_code_file bundle + -> raw source code file nodes + +external_project_file bundle + -> raw external project file nodes +``` + +서로 다른 source kind를 같은 bundle에 넣지 않는다. + +## 3. Source Kinds + +초기 source kind: + +- `internal_document` +- `source_code_file` +- `external_project_file` + +대화 턴은 기존 `raw_capsule` 경로를 계속 사용한다. 이번 발주는 대화 턴 ingest를 새로 만들지 않는다. + +## 4. What Code May Write + +Code may write absolute file/source coordinates: + +- source kind +- file path +- file suffix +- file exists +- char count +- deterministic source file data id +- deterministic graph node id +- source-kind bundle node id +- CONTAINS edge id + +Code must not write: + +- semantic topic +- summary +- importance score +- relevance judgment +- meaning cluster + +## 5. Non-goals + +This order does not open: + +- Neo4j/Vessel connection +- night-government summary worker +- semantic axis +- R LLM selector +- R traversal over the new source kind bundles +- node_3 answer injection + +## 6. Test Plan + +```powershell +python -m compileall songryeon_core main.py +python -m pytest tests/test_order_155_graph_source_kind_ingest.py tests/test_import_baseline.py -q +python main.py fast-test --profile graph +git diff --check +``` diff --git a/Administrative_Reform_1/04_Orders/ORDER_156_GRAPH_SOURCE_OBSERVATION_TIME_AND_CORE_EGO_LINK_V0.md b/Administrative_Reform_1/04_Orders/ORDER_156_GRAPH_SOURCE_OBSERVATION_TIME_AND_CORE_EGO_LINK_V0.md new file mode 100644 index 0000000..d5b9101 --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_156_GRAPH_SOURCE_OBSERVATION_TIME_AND_CORE_EGO_LINK_V0.md @@ -0,0 +1,89 @@ +# ORDER 156: Graph Source Observation Time And CoreEgo Link v0 + +## 1. Goal + +ORDER 155의 source-kind separated ingest를 한 단계 잠근다. + +내부 문서와 코드 파일은 대화 턴과 달리 계속 바뀌거나 삭제될 수 있다. 따라서 graph raw source node는 "현재 파일의 영원한 진실"이 아니라, 특정 시각에 특정 경로에서 관측한 source snapshot coordinate임을 payload 자체에 기록해야 한다. + +또한 source kind bundle은 R loop가 CoreEgo 시간축에서 발견할 수 있도록 CoreEgo time axis 아래에 연결되어야 한다. + +## 2. Required Absolute Fields + +Raw source file metadata and raw source graph nodes must record: + +- `observed_at` +- `ingested_at` +- `source_last_modified_at` +- `exists_at_ingest` +- `content_sha1` + +Meanings: + +- `observed_at`: code observed the filesystem state at this time. +- `ingested_at`: code recorded the observation into graph/DataStore at this time. +- `source_last_modified_at`: OS file last modified timestamp at observation time. +- `exists_at_ingest`: whether the file existed when code observed it. +- `content_sha1`: content fingerprint for that observation. + +## 3. CoreEgo Link + +Source ingest must create this shape when a CoreEgo time axis already exists in DataStore: + +```text +graph:core_ego:root + -> graph:axis:time + -> graph:source_ingest_time_bundle:{batch_id} + -> graph:source_kind_bundle:{batch_id}:{source_kind} + -> graph:raw_source:{source_kind}:{observation_digest} +``` + +The source ingest code must not silently invent a detached CoreEgo root if the root/time axis is missing. Missing CoreEgo time axis should fail loudly, because otherwise downstream R traversal would see an inconsistent graph surface. + +## 4. What Code May Write + +Code may write only absolute observation coordinates: + +- source kind +- normalized path +- file name +- suffix +- exists-at-ingest flag +- char count +- source last modified timestamp +- observed/ingested timestamp +- content SHA-1 +- deterministic graph/data IDs for that observation +- CoreEgo time-axis graph edges +- source ingest snapshot/guide packet coordinates + +## 5. What Code Must Not Write + +Code must not write: + +- semantic topic +- summary +- importance score +- relevance judgment +- meaning cluster +- current-truth claim that the file still has the same content later + +## 6. Non-goals + +This order does not open: + +- Neo4j/Vessel connection +- night-government summary worker +- semantic axis +- R LLM selector +- automatic source path discovery +- node_3 answer injection + +## 7. Test Plan + +```powershell +python -m compileall songryeon_core main.py +python -m pytest tests/test_order_155_graph_source_kind_ingest.py tests/test_order_156_graph_source_observation_time_and_core_link.py tests/test_import_baseline.py -q +python main.py fast-test --profile graph +git diff --check +``` diff --git a/Administrative_Reform_1/04_Orders/ORDER_157_SONGRYEON_CORE_SOURCE_INGEST_MANIFEST_V0.md b/Administrative_Reform_1/04_Orders/ORDER_157_SONGRYEON_CORE_SOURCE_INGEST_MANIFEST_V0.md new file mode 100644 index 0000000..a39921f --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_157_SONGRYEON_CORE_SOURCE_INGEST_MANIFEST_V0.md @@ -0,0 +1,78 @@ +# ORDER 157: SongRyeon Core Source Ingest Manifest v0 + +## 1. Goal + +ORDER 155~156에서 만든 source-kind separated graph ingest를 SongRyeon Core 자체에 적용할 수 있는 명시 manifest/batch runner를 만든다. + +이번 발주는 의미 판단이 아니라, 송련 코어 내부 문서/코드/test 파일을 source kind별로 분리하고 관측 시각이 붙은 graph source snapshot으로 올리는 작업이다. + +## 2. Approved Policy + +사용자 결재: + +- 내부 문서/코드 원문 snapshot 저장 허용. +- `tests/**/*.py`도 `source_code_file`로 ingest한다. +- glob manifest는 허용한다. 단, glob은 의미 추측이 아니라 명시 정책이다. +- 명시 경로가 missing이면 실패한다. +- glob pattern 결과가 비어 있거나, 과거 파일이 사라져 glob에 잡히지 않는 경우는 이번 MVP에서 삭제 감지로 다루지 않는다. + +## 3. Manifest Policy + +Initial SongRyeon Core source manifest: + +```text +internal_document: + explicit: + - AGENTS.md + - README.md + globs: + - Administrative_Reform_1/**/*.md + +source_code_file: + explicit: + - main.py + globs: + - songryeon_core/**/*.py + - tests/**/*.py +``` + +`external_project_file` is not auto-ingested in this order. External project files must be provided explicitly by a later caller/order. + +## 4. Text Snapshot Boundary + +For SongRyeon Core internal documents and source code files, code may store a raw text snapshot as absolute copied source content. + +Text snapshot records must include: + +- source kind +- path +- observed_at +- ingested_at +- content_sha1 +- char_count +- text +- generated_by +- info_class=`absolute_copied_source` +- semantic_judgement_status=`not_run` + +Text snapshot records are raw copied content, not summaries. + +## 5. What Code Must Not Do + +Code must not: + +- classify files by semantic guessing +- summarize files +- assign importance or relevance +- create semantic axis +- create Neo4j/Vessel records +- inject source ingest material into node_3 answers + +## 6. Test Plan + +```powershell +python -m compileall songryeon_core main.py +python -m pytest tests/test_order_157_songryeon_core_source_manifest.py tests/test_order_155_graph_source_kind_ingest.py tests/test_order_156_graph_source_observation_time_and_core_link.py tests/test_import_baseline.py -q +python main.py fast-test --profile graph +git diff --check +``` diff --git a/Administrative_Reform_1/04_Orders/ORDER_158_GRAPH_MEMORY_EXPORT_PACKET_V0.md b/Administrative_Reform_1/04_Orders/ORDER_158_GRAPH_MEMORY_EXPORT_PACKET_V0.md new file mode 100644 index 0000000..6ae9a28 --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_158_GRAPH_MEMORY_EXPORT_PACKET_V0.md @@ -0,0 +1,78 @@ +# ORDER_158_GRAPH_MEMORY_EXPORT_PACKET_V0 + +## 목표 + +외부 Vessel/Neo4j 어댑터를 만들기 전에, 현재 `DataStore`에 쌓인 graph memory / graph source 기록 중 외부 DB로 내보낼 수 있는 기록 목록을 code-generated absolute export packet으로 고정한다. + +이번 발주는 외부 DB write가 아니라 export 직전의 “수출 명세서”를 만드는 작업이다. + +## 배경 + +ORDER_139부터 ORDER_157까지 진행하면서 송련 Core는 다음 절대정보 기반을 갖추었다. + +- `TurnStateCapsule` 기반 raw capsule graph node +- CoreEgo -> time axis -> time bundle -> raw capsule 구조 +- source-kind separated ingest +- source observation timestamp +- SongRyeon Core 내부 문서/source code manifest +- graph integrity audit + +하지만 아직 Vessel/Neo4j로 실제 기록을 보내기 전에 어떤 record를 보낼지, 어떤 source trace에 근거하는지, graph integrity가 어떤 상태인지 한 번에 묶은 export packet이 없다. + +## 구현 범위 + +1. 새 builder를 추가한다. + - 후보 파일: `songryeon_core/core/graph_memory_export.py` + +2. `DataStore`에서 다음 record를 수집한다. + - `graph_memory:node:*` + - `graph_memory:edge:*` + - `graph_memory:snapshot` + - `graph_memory:core_ego_time_axis_frame` + - `graph_memory:rloop_guide_packet` + - `graph_source:file_metadata` + - `graph_source:file_text_snapshot` + - `graph_source:source_kind_ingest_frame` + - `graph_source:songryeon_core_source_manifest_frame` + +3. export packet에는 다음을 남긴다. + - packet id + - batch id + - target adapter name: `songryeon-neo4j-vessel` + - external write status: `not_run` + - included data ids + - category별 data ids + - source trace ids + - data type counts + - graph integrity summary + - `generated_by=CODE:GRAPH_MEMORY_EXPORT_PACKET_BUILDER` + - `info_class=absolute` + - `semantic_judgement_status=not_run` + +4. export packet 자체도 `DataStore`에 기록한다. + - data type: `graph_memory:export_packet` + - trace actor: `graph_memory_export_packet_builder` + +5. graph fast-test profile에 ORDER_158 테스트를 포함한다. + +## 금지 + +- 외부 DB에 실제 write하지 않는다. +- Neo4j driver를 추가하지 않는다. +- Vessel adapter를 구현하지 않는다. +- LLM 요약, 상대정보, 혼합정보를 만들지 않는다. +- graph integrity guard를 약화하지 않는다. +- 의미축, R live route, scheduler, 장기기억 DB를 열지 않는다. + +## 완료 조건 + +- `python -m compileall songryeon_core main.py` +- `python -m pytest tests/test_order_158_graph_memory_export_packet.py -q` +- `python main.py fast-test --profile graph` +- 가능하면 `python -m pytest`, `python main.py smoke-test` + +## 기대 결과 + +ORDER_158 이후에는 외부 DB 어댑터가 바로 DataStore 전체를 뒤지는 것이 아니라, 먼저 code가 만든 export packet을 읽고 그 목록만 Vessel/Neo4j write 대상으로 삼을 수 있다. + +즉, “외부 DB에 뭘 썼는지 모르겠다”가 아니라 “이 packet에 적힌 절대정보 record만 내보낸다”는 경계가 생긴다. diff --git a/Administrative_Reform_1/04_Orders/ORDER_159_VESSEL_ADAPTER_WRITE_PLAN_BOUNDARY_V0.md b/Administrative_Reform_1/04_Orders/ORDER_159_VESSEL_ADAPTER_WRITE_PLAN_BOUNDARY_V0.md new file mode 100644 index 0000000..12992dc --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_159_VESSEL_ADAPTER_WRITE_PLAN_BOUNDARY_V0.md @@ -0,0 +1,83 @@ +# ORDER_159_VESSEL_ADAPTER_WRITE_PLAN_BOUNDARY_V0 + +## 목표 + +ORDER_158의 `GraphMemoryExportPacket`을 입력으로 받아, 외부 Vessel/Neo4j에 실제로 쓰기 전에 code-generated absolute `GraphVesselWritePlan`을 만든다. + +이번 발주는 “DB write”가 아니라 “DB write 직전 계획서”다. + +## 배경 + +ORDER_142에서 `songryeon-neo4j-vessel` 이름과 graph store boundary가 예약되었다. + +ORDER_158에서 DataStore 안 graph/source record를 export packet으로 묶었다. + +다음 단계에서 바로 Neo4j driver를 붙이면 다음 위험이 있다. + +- 어떤 record를 썼는지 추적이 흐려질 수 있다. +- node/edge/source payload write 순서가 암묵화될 수 있다. +- graph integrity 실패 상태에서도 외부 write가 시도될 수 있다. +- 외부 DB adapter가 의미 판단 필드를 몰래 추가할 수 있다. + +따라서 먼저 export packet을 읽고 write operation 목록만 생성하는 boundary를 둔다. + +## 구현 범위 + +1. 새 모듈을 추가한다. + - 후보 파일: `songryeon_core/core/graph_vessel_adapter.py` + +2. 새 plan을 만든다. + - `GraphVesselWritePlan` + - `GraphVesselWriteOperation` + - data type: `graph_vessel:write_plan` + - generator: `CODE:GRAPH_VESSEL_ADAPTER_BOUNDARY` + +3. write operation 종류를 좁게 고정한다. + - `upsert_source_payload` + - `upsert_graph_node` + - `upsert_graph_edge` + - `upsert_support_record` + +4. operation 순서를 code가 고정한다. + - source payload + - graph node + - graph edge + - support record + +5. graph integrity가 실패한 export packet은 외부 write plan을 `blocked_integrity_failed`로 닫는다. + - operation list는 비운다. + - 실패 이유와 integrity summary는 plan에 남긴다. + +6. plan은 `DataStore`와 `TraceStore`에 기록한다. + +## 메타정보 경계 + +- `generated_by=CODE:GRAPH_VESSEL_ADAPTER_BOUNDARY` +- `info_class=absolute` +- `semantic_judgement_status=not_run` +- `external_write_status=not_run` + +이번 작업은 record ID, data type, operation order, integrity status 같은 code-checkable absolute 정보만 생성한다. + +## 금지 + +- Neo4j driver 추가 금지 +- 외부 DB 연결 금지 +- 실제 write 금지 +- LLM 요약/상대정보/혼합정보 생성 금지 +- 의미축 생성 금지 +- R live route 변경 금지 +- scheduler/장기기억 DB 변경 금지 + +## 완료 조건 + +- `python -m compileall songryeon_core main.py` +- `python -m pytest tests/test_order_159_vessel_adapter_boundary.py -q` +- `python main.py fast-test --profile graph` +- 가능하면 `python -m pytest`, `python main.py smoke-test` + +## 기대 결과 + +ORDER_159 이후에는 외부 DB adapter가 export packet을 바로 해석해 임의로 쓰는 것이 아니라, 먼저 write plan을 만들고 그 plan에 적힌 operation만 수행하는 구조로 갈 수 있다. + +즉 다음 ORDER에서 Neo4j/Vessel write를 열더라도 “무엇을 어떤 순서로 쓸지”가 이미 DataStore/TraceStore에 남는다. diff --git a/Administrative_Reform_1/04_Orders/ORDER_160_LOCAL_VESSEL_NEO4J_FIRST_WRITE_V0.md b/Administrative_Reform_1/04_Orders/ORDER_160_LOCAL_VESSEL_NEO4J_FIRST_WRITE_V0.md new file mode 100644 index 0000000..cf03680 --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_160_LOCAL_VESSEL_NEO4J_FIRST_WRITE_V0.md @@ -0,0 +1,82 @@ +# ORDER_160_LOCAL_VESSEL_NEO4J_FIRST_WRITE_V0 + +## 목표 + +로컬 데스크탑 Neo4j/Vessel에 최초 write를 수행한다. + +단, 입력은 ORDER_159의 `GraphVesselWritePlan`으로 제한하고, qwen-chat 기본 실행이나 기존 송련 브레인에는 연결하지 않는다. + +## 배경 + +현재까지의 흐름은 다음처럼 잠겼다. + +1. DataStore에 graph/source record가 쌓인다. +2. ORDER_158이 export packet을 만든다. +3. ORDER_159가 write plan을 만든다. +4. ORDER_160에서 처음으로 로컬 Vessel에 write한다. + +이번 작업은 “외부 DB를 아무렇게나 붙이는 작업”이 아니라, 이미 생성된 write plan에 적힌 operation만 실행하는 수동 opt-in write다. + +## 로컬 원칙 + +- Vessel은 로컬 데스크탑 환경에서만 접속한다. +- cloud DB, remote URI 자동 업로드는 금지한다. +- 기존 송련 브레인과 섞지 않는다. +- 실험 DB/namespace는 다음 값을 사용한다. + - service name: `songryeon-neo4j-vessel` + - database name: `songryeon_vessel` + - namespace: `songryeon_core_graph_v0` + +## 구현 범위 + +1. Neo4j writer 모듈을 추가한다. + - 후보 파일: `songryeon_core/core/graph_vessel_neo4j.py` + +2. writer 입력은 `GraphVesselWritePlan`으로 제한한다. + - `plan_status=ready_to_write` + - `external_write_status=not_run` + - `graph_integrity_status=passed` + - target adapter: `songryeon-neo4j-vessel` + +3. writer 출력 result frame을 만든다. + - data type: `graph_vessel:neo4j_write_result` + - generated_by: `CODE:GRAPH_VESSEL_NEO4J_WRITER` + - info_class: `absolute` + - semantic_judgement_status: `not_run` + +4. Neo4j 미연결/설정 누락/driver 누락은 정직하게 닫는다. + - `adapter_unavailable` + - `neo4j_driver_missing` + - `neo4j_config_missing` + - `neo4j_write_failed` + +5. 수동 CLI를 추가한다. + - 후보 명령: `python main.py vessel-first-write` + - 이 명령은 작은 fixture graph를 만들고 export packet -> write plan -> Neo4j writer 순서로 실행한다. + +## 금지 + +- qwen-chat 자동 연결 금지 +- R live route 자동 연결 금지 +- 기존 송련 브레인 DB와 혼합 금지 +- DataStore 전체를 무차별 write 금지 +- LLM 요약/상대정보/혼합정보 생성 금지 +- embedding/vector 검색 금지 +- 의미축 생성 금지 +- 기존 DataStore 삭제/변형 금지 + +## 완료 조건 + +- Neo4j writer 단위 테스트 통과 +- Neo4j 미설정 시 `adapter_unavailable`으로 정직하게 닫힘 +- fake Neo4j driver 기준 실제 operation write 흐름 검증 +- `python -m compileall songryeon_core main.py` +- `python -m pytest tests/test_order_160_local_vessel_neo4j_writer.py -q` +- `python main.py fast-test --profile graph` +- 가능하면 `python -m pytest`, `python main.py smoke-test` + +## 기대 결과 + +ORDER_160 이후에는 송련 Core가 “로컬 그래프 DB에 첫 write를 수행할 수 있는 상태”가 된다. + +단, 실제 제품 흐름은 아직 아니다. 이번 단계는 수동 opt-in local Vessel write까지만 연다. diff --git a/Administrative_Reform_1/04_Orders/ORDER_161_VESSEL_DISPLAY_VOCABULARY_V0.md b/Administrative_Reform_1/04_Orders/ORDER_161_VESSEL_DISPLAY_VOCABULARY_V0.md new file mode 100644 index 0000000..93a0e12 --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_161_VESSEL_DISPLAY_VOCABULARY_V0.md @@ -0,0 +1,92 @@ +# ORDER_161_VESSEL_DISPLAY_VOCABULARY_V0 + +## 목표 + +Neo4j Browser에서 사람이 보는 node label, relationship type, display properties를 더 이해하기 쉬운 graph vocabulary로 바꾼다. + +내부 `data_id`, provenance, payload, schema는 유지한다. + +## 배경 + +ORDER_160으로 로컬 Neo4j/Vessel first write가 성공했다. + +하지만 Neo4j 화면에 보이는 이름은 다음처럼 너무 기술적이다. + +- `SongRyeonRecord` +- `SongRyeonGraphNode` +- `SongRyeonGraphEdgeRecord` +- `SongRyeonSupportRecord` +- `SONGRYEON_GRAPH_EDGE` + +이 이름들은 안전하지만 사람이 그래프를 훑을 때 직관적이지 않다. + +## 구현 범위 + +1. base label을 `VesselRecord`로 바꾼다. + +2. graph node payload에 따라 display label을 추가한다. + - `core_ego` -> `CoreEgo` + - `time_axis` -> `TimeAxis` + - `time_bundle` -> `TimeBundle` + - `raw_capsule` -> `RawCapsule` + - `raw_source` -> `RawSource` + - `source_kind_bundle` -> `SourceKindBundle` + - `source_ingest_time_bundle` -> `SourceIngestBundle` + - fallback -> `GraphMemoryNode` + +3. support record는 더 구체적인 label로 나눈다. + - `graph_memory:snapshot` -> `GraphMemorySnapshot` + - `graph_memory:rloop_guide_packet` -> `RLoopGuide` + - `graph_memory:core_ego_time_axis_frame` -> `GraphMemoryIndex` + - source manifest / source ingest frame -> `GraphMemoryIndex` + +4. source payload는 `GraphMemorySource` label을 사용한다. + +5. edge record는 `GraphMemoryEdgeRecord` label을 사용한다. + +6. relationship type을 사람이 읽기 쉬운 이름으로 바꾼다. + - CoreEgo -> TimeAxis: `HAS_AXIS` + - TimeAxis -> TimeBundle: `HAS_BUNDLE` + - TimeBundle -> RawCapsule: `CONTAINS_MEMORY` + - RawCapsule -> RawCapsule: `NEXT` + - SourceIngestBundle -> SourceKindBundle: `HAS_SOURCE_KIND` + - SourceKindBundle -> RawSource: `CONTAINS_SOURCE` + - fallback: `CONTAINS` + +7. display properties를 추가한다. + - `display_name` + - `display_kind` + - `display_label` + - relationship에는 `display_relationship_type` + +## 기존 실험 데이터 처리 + +ORDER_160 first-write fixture는 작다. + +새 writer는 기존 old label을 제거하고 새 display label을 붙일 수 있게 한다. 기존 old `SONGRYEON_GRAPH_EDGE` 관계는 같은 `edge_id`/namespace 기준으로 삭제한 뒤 새 relationship type으로 재생성한다. + +## 금지 + +- 내부 `data_id` 변경 금지 +- source/provenance/payload 삭제 금지 +- 의미 요약/LLM 판단 추가 금지 +- qwen-chat/R live route 자동 연결 금지 +- 기존 송련 브레인과 혼합 금지 + +## 완료 조건 + +- `python -m compileall songryeon_core main.py` +- `python -m pytest tests/test_order_160_local_vessel_neo4j_writer.py -q` +- `python main.py fast-test --profile graph` +- 가능하면 `python -m pytest`, `python main.py smoke-test` + +## 기대 결과 + +Neo4j Browser에서 다음처럼 볼 수 있어야 한다. + +```cypher +MATCH path = (core:CoreEgo)-[:HAS_AXIS]->(:TimeAxis)-[:HAS_BUNDLE]->(:TimeBundle)-[:CONTAINS_MEMORY]->(:RawCapsule) +RETURN path; +``` + +즉, 송련 내부 ID는 그대로 유지하되 사람이 보는 그래프 이름은 읽기 쉬워진다. diff --git a/Administrative_Reform_1/04_Orders/ORDER_162_VESSEL_READBACK_VERIFICATION_V0.md b/Administrative_Reform_1/04_Orders/ORDER_162_VESSEL_READBACK_VERIFICATION_V0.md new file mode 100644 index 0000000..901c0ac --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_162_VESSEL_READBACK_VERIFICATION_V0.md @@ -0,0 +1,72 @@ +# ORDER 162: Vessel Readback Verification v0 + +## 상태 + +구현 대상. + +## 목표 + +`vessel-first-write`로 Neo4j Vessel에 기록한 그래프를 코드가 다시 읽어, 다음 경로가 실제로 존재하는지 절대정보로 확인한다. + +```text +CoreEgo -> HAS_AXIS -> TimeAxis -> HAS_BUNDLE -> TimeBundle -> CONTAINS_MEMORY -> RawCapsule +``` + +## 배경 + +ORDER 160은 로컬 Neo4j Vessel에 첫 쓰기를 열었다. +ORDER 161은 사람이 보기 쉬운 label/relationship/display vocabulary를 정리했다. +하지만 아직 “쓴 뒤에 코드가 같은 vocabulary로 다시 읽을 수 있는지”를 확인하는 좁은 검증 명령이 없다. + +## 구현 범위 + +- `vessel-readback` CLI를 추가한다. +- Neo4j에서 다음 수량을 읽는다. + - VesselRecord node count + - Vessel relationship count + - CoreEgo / TimeAxis / TimeBundle / RawCapsule count + - HAS_AXIS / HAS_BUNDLE / CONTAINS_MEMORY count + - CoreEgo -> TimeAxis -> TimeBundle -> RawCapsule path count + - 필수 provenance property 누락 count +- readback 결과를 `GraphVesselNeo4jReadbackResultFrame`으로 DataStore에 남긴다. +- `generated_by=CODE:GRAPH_VESSEL_NEO4J_READBACK_VERIFIER` +- `info_class=absolute` +- `semantic_judgement_status=not_run` + +## 금지 + +- Neo4j 데이터를 변경하지 않는다. +- R route, R1/R2/R3 자동 실행을 열지 않는다. +- node_1/node_2/node_3 답변 흐름에 자동 주입하지 않는다. +- LLM 요약, 중요도, 관련성 판단을 생성하지 않는다. +- 의미축 graph node를 만들지 않는다. + +## 완료 조건 + +- `python -m compileall songryeon_core main.py` +- `python -m pytest tests/test_order_162_vessel_readback_verification.py -q` +- `python main.py fast-test --profile graph` +- `python -m pytest` +- `python main.py smoke-test` + +## 수동 확인 명령 + +Neo4j 비밀번호가 환경변수 또는 `--password`로 설정된 터미널에서 다음을 실행한다. + +```powershell +python main.py vessel-readback --database neo4j +``` + +기대값: + +- `readback_status=passed` +- `core_path_exists=true` +- `core_path_count>=1` +- `core_ego_count>=1` +- `time_axis_count>=1` +- `time_bundle_count>=1` +- `raw_capsule_count>=1` + +## 다음 단계 후보 + +ORDER 162가 통과하면, 다음 후보는 R loop 또는 수동 graph-inspect 명령이 이 readback 결과를 이용해 CoreEgo에서 시간축 후보를 열람하는 단계다. diff --git a/Administrative_Reform_1/04_Orders/ORDER_163_VESSEL_INSPECT_MANUAL_WALK_V0.md b/Administrative_Reform_1/04_Orders/ORDER_163_VESSEL_INSPECT_MANUAL_WALK_V0.md new file mode 100644 index 0000000..86fc5ae --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_163_VESSEL_INSPECT_MANUAL_WALK_V0.md @@ -0,0 +1,78 @@ +# ORDER 163: Vessel Inspect Manual Walk v0 + +## 상태 + +구현 대상. + +## 목표 + +Neo4j Vessel에 들어간 graph memory를 사람이 볼 수 있는 tree로 펼쳐 확인하는 읽기 전용 CLI를 만든다. + +```powershell +python main.py vessel-inspect --database neo4j +``` + +## 배경 + +ORDER 160은 Neo4j Vessel 첫 쓰기를 열었다. +ORDER 161은 사람이 보기 쉬운 label/relationship vocabulary를 정리했다. +ORDER 162는 다음 경로가 실제로 존재하는지 readback으로 확인했다. + +```text +CoreEgo -> HAS_AXIS -> TimeAxis -> HAS_BUNDLE -> TimeBundle -> CONTAINS_MEMORY -> RawCapsule +``` + +하지만 아직 사용자가 터미널에서 “CoreEgo 아래 무엇이 있는지”를 한눈에 펼쳐볼 수 없다. + +## 구현 범위 + +- `vessel-inspect` CLI를 추가한다. +- Neo4j에서 CoreEgo -> TimeAxis -> TimeBundle -> RawCapsule 경로를 읽는다. +- 사람이 볼 수 있는 `tree_lines` / `tree_text`를 만든다. +- 결과를 `GraphVesselNeo4jInspectResultFrame`으로 DataStore에 남긴다. +- 결과는 다음으로 고정한다. + - `generated_by=CODE:GRAPH_VESSEL_NEO4J_INSPECTOR` + - `info_class=absolute` + - `semantic_judgement_status=not_run` + +## 출력 예시 + +```text +CoreEgo [graph:core_ego:root] + HAS_AXIS -> Time Axis [graph:axis:time] + HAS_BUNDLE -> Time Bundle [graph:time_bundle:manual_vessel_first_write:core] + CONTAINS_MEMORY -> Raw Capsule: turn_vessel_first_write_0001:previous [graph:raw_capsule:turn_vessel_first_write_0001:previous] +``` + +## 금지 + +- Neo4j 데이터를 변경하지 않는다. +- R route/R1/R2/R3 자동 실행을 열지 않는다. +- node_1/node_2/node_3 답변 흐름에 자동 주입하지 않는다. +- LLM 요약, 중요도, 관련성 판단을 만들지 않는다. +- 의미축 graph node를 만들지 않는다. + +## 완료 조건 + +- `python -m compileall songryeon_core main.py` +- `python -m pytest tests/test_order_163_vessel_inspect_manual_walk.py -q` +- `python main.py fast-test --profile graph` +- `python main.py smoke-test` + +## 수동 확인 명령 + +Neo4j 비밀번호가 환경변수 또는 `--password`로 설정된 터미널에서 실행한다. + +```powershell +python main.py vessel-inspect --database neo4j --format text +``` + +기대값: + +- `inspect_status=passed` +- `inspected_path_count>=1` +- tree 안에 `CoreEgo`, `Time Axis`, `Time Bundle`, `Raw Capsule`이 표시된다. + +## 다음 단계 후보 + +ORDER 163이 통과하면 다음 후보는 R loop 또는 별도 graph-walk 명령이 inspect 결과를 이용해 시간축 후보를 선택하는 단계다. diff --git a/Administrative_Reform_1/04_Orders/ORDER_164_DYNAMIC_SOURCE_VERSION_LINEAGE_AND_SUMMARY_INVALIDATION_LEDGER_V0.md b/Administrative_Reform_1/04_Orders/ORDER_164_DYNAMIC_SOURCE_VERSION_LINEAGE_AND_SUMMARY_INVALIDATION_LEDGER_V0.md new file mode 100644 index 0000000..413b63c --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_164_DYNAMIC_SOURCE_VERSION_LINEAGE_AND_SUMMARY_INVALIDATION_LEDGER_V0.md @@ -0,0 +1,59 @@ +# ORDER_164 Dynamic Source Version Lineage And Summary Invalidation Ledger v0 + +## 목표 + +동적 원본 파일이 바뀌었을 때, 기존 raw source graph node와 그 원본에서 파생된 요약/혼합정보를 지우지 않고 추적 가능하게 무효화할 수 있는 최소 장부를 만든다. + +이번 발주는 LLM 요약 생성이 아니라 code-generated absolute infrastructure다. + +## 배경 + +현재 graph source ingest는 파일 경로, source kind, observed_at, ingested_at, source_last_modified_at, content_sha1, raw_source graph node를 저장한다. + +하지만 같은 파일이 나중에 다시 관측됐을 때 다음 질문에 바로 답하기 어렵다. + +- 이 raw source는 같은 파일의 몇 번째 관측본인가? +- 내용이 그대로 다시 관측된 것인가, 실제로 바뀐 것인가? +- 예전 관측본에 기대어 만든 LLM summary가 있다면 이제 current answer 근거로 써도 되는가? + +## 구현 범위 + +1. SourceVersionLineageFrame을 추가한다. + - 같은 source_kind + path를 하나의 source identity로 묶는다. + - 관측된 raw_source graph node들을 observed_at 순서로 나열한다. + - active source는 최신 관측본으로 둔다. + - content_sha1이 바뀌면 lineage_status=content_changed로 기록한다. + - 참고: ORDER_165 이후 content hash가 같은 재관측은 source version lineage 상태가 아니라 observation ledger의 unchanged 기록으로 분리한다. + +2. SummaryInvalidationLedgerFrame을 추가한다. + - changed lineage의 superseded raw_source를 근거로 하는 summary node를 찾는다. + - 해당 summary를 삭제하지 않고 invalidation record로 표시한다. + - invalidated_reason_code=source_content_changed를 사용한다. + +3. graph source ingest 이후에도 이 장부를 code가 만들 수 있게 한다. + - code는 의미 요약을 만들지 않는다. + - code는 어떤 요약이 낡았는지 source_graph_node_ids와 content_sha1 변화만으로 판단한다. + +4. export packet / Vessel write plan이 이 장부 record를 support record로 다룰 수 있게 한다. + +## 금지 + +- LLM 요약 생성 금지. +- summary node 본문 변경/삭제 금지. +- 기존 raw_source graph node 삭제 금지. +- Neo4j에 직접 semantic summary를 쓰는 기능 금지. +- R route, R1/R2/R3 탐색 정책 변경 금지. +- source 내용 유사도/키워드 휴리스틱 추가 금지. + +## 완료 조건 + +- `python -m compileall songryeon_core main.py` +- `python -m pytest tests/test_order_164_source_version_lineage_and_invalidation.py` +- 관련 source ingest/export pytest 통과 +- `git diff --check` + +## 기대 효과 + +나중에 심야정부가 raw source에서 상대/혼합 요약을 만들더라도, 그 요약이 어느 원본 버전에 기대고 있는지 추적할 수 있다. + +동적 원본이 바뀌면 예전 요약은 지우지 않고 `invalidated_by_source_change` 계열 장부로 current answer 근거에서 제외할 수 있다. diff --git a/Administrative_Reform_1/04_Orders/ORDER_165_SAME_CONTENT_REOBSERVE_OBSERVATION_LEDGER_V0.md b/Administrative_Reform_1/04_Orders/ORDER_165_SAME_CONTENT_REOBSERVE_OBSERVATION_LEDGER_V0.md new file mode 100644 index 0000000..0842017 --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_165_SAME_CONTENT_REOBSERVE_OBSERVATION_LEDGER_V0.md @@ -0,0 +1,51 @@ +# ORDER_165 Same-Content Reobserve Observation Ledger v0 + +## 목표 + +동적 원본 파일을 다시 봤는데 내용 hash가 같을 때 새 `RawSource` graph node를 만들지 않고, "검사했지만 unchanged였다"는 관측 장부만 남긴다. + +## 배경 + +ORDER_164는 source version lineage에 `same_content_reobserved` 상태를 두었다. 이 방식은 감사에는 친절하지만, 같은 파일/같은 내용이 관측 시각만 다르다는 이유로 graph raw source version이 계속 늘어날 위험이 있다. + +그래프 메모리에서는 "내용이 같은 원본"은 같은 원본 버전으로 유지하고, 관측 시각은 별도 장부로 빼는 편이 더 단순하다. + +## 구현 방향 + +1. `SourceVersionLineageFrame.lineage_status`에서 `same_content_reobserved`를 제거한다. + - 허용 상태는 `single_version`, `content_changed`만 둔다. + +2. `raw_source` graph node identity에서 `observed_at`을 제외한다. + - 기준은 `source_kind + path + content_sha1`이다. + - 같은 파일/같은 내용이면 기존 `raw_source` node를 재사용한다. + - 파일 내용이 바뀌면 content hash가 달라지므로 새 `raw_source` node가 생긴다. + +3. `SourceObservationLedgerFrame`을 추가한다. + - 현재 ingest batch의 source file observation들을 기록한다. + - observation status는 `new_source_version`, `unchanged`, `content_changed` 중 하나다. + - unchanged는 새 raw source version을 만들지 않는다. + +4. summary invalidation은 content_changed일 때만 작동한다. + - unchanged observation은 기존 summary를 무효화하지 않는다. + +## 금지 + +- LLM 요약 생성 금지. +- raw source나 summary record 삭제 금지. +- unchanged 파일에 대해 새 raw source version을 만드는 fallback 금지. +- source 의미 유사도/키워드 휴리스틱 금지. +- R route/R traversal 정책 변경 금지. +- Neo4j live write 자동 실행 변경 금지. + +## 완료 조건 + +- `python -m compileall songryeon_core main.py` +- `python -m pytest tests/test_order_164_source_version_lineage_and_invalidation.py tests/test_order_165_same_content_observation_ledger.py -q` +- 관련 graph/source/export fast-test 통과 +- `git diff --check` + +## 기대 효과 + +동적 원본 파일을 자주 검사해도 같은 내용의 raw source graph node가 불필요하게 늘지 않는다. + +대신 "언제 다시 검사했고, 그때 바뀌었는지/안 바뀌었는지"는 observation ledger로 추적할 수 있다. diff --git a/Administrative_Reform_1/04_Orders/ORDER_166_NIGHT_TIME_BUNDLE_SUMMARY_NODE_V0.md b/Administrative_Reform_1/04_Orders/ORDER_166_NIGHT_TIME_BUNDLE_SUMMARY_NODE_V0.md new file mode 100644 index 0000000..8b650b5 --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_166_NIGHT_TIME_BUNDLE_SUMMARY_NODE_V0.md @@ -0,0 +1,143 @@ +# ORDER 166: Night TimeBundle Summary Node v0 + +## 1. Goal + +심야정부가 `TimeBundle` graph node 하나를 대상으로 LLM 요약 graph node를 생성할 수 있는 최소 MVP를 만든다. + +이번 발주의 핵심은 원본 `TimeBundle` payload를 수정하지 않고, 별도 `SummaryGraphNode`를 만들어 `SUMMARY_OF` edge로 연결하는 것이다. + +```text +TimeBundle + <- SUMMARY_OF +SummaryGraphNode +``` + +## 2. Background + +철학 문서 기준: + +- raw/time bundle graph node는 code-generated absolute coordinate다. +- LLM 요약은 원본 속성이 아니라 별도 summary node다. +- raw leaf 1개를 직접 보고 만든 요약은 `relative`다. +- raw leaf 여러 개 또는 bundle을 보고 만든 요약은 `mixed`다. +- summary node는 source bundle, generated_by, info_class, semantic_judgement_status, summary_depth를 드러내야 한다. + +현재 Core에는 이미 다음 기반이 있다. + +- `CoreEgo -> TimeAxis -> TimeBundle -> RawCapsule` +- `GraphMemoryNodeFrame` absolute lock +- `SUMMARY_OF` edge kind +- summary invalidation ledger foundation +- graph export packet / Vessel write plan / Neo4j writer + +## 3. Scope + +### 3.1 New summary frame + +Add a dedicated summary frame for LLM-generated graph summaries. + +Required fields: + +- `node_id` +- `node_kind=summary` +- `target_graph_node_id` +- `target_node_kind` +- `summary_text` +- `summary_depth` +- `source_depth_min` +- `source_depth_max` +- `source_leaf_count` +- `source_summary_count` +- `source_bundle_kind` +- `source_graph_node_ids` +- `source_trace_ids` +- `source_data_ids` +- `validity_status` +- `review_status` +- `llm_call_data_id` +- `llm_trace_event_id` +- `prompt_ref` +- `generated_by` +- `info_class` +- `semantic_judgement_status` + +### 3.2 Classification rule + +Code may classify only by source cardinality and target structure, not by semantic content. + +```text +source_leaf_count == 1 and len(source_graph_node_ids) == 1 +-> info_class=relative + +otherwise +-> info_class=mixed +``` + +If an LLM payload supplies an incompatible `info_class`, validation fails. Code does not silently correct the semantic text. + +### 3.3 LLM boundary + +LLM writes: + +- `summary_text` + +Code writes/checks: + +- target node coordinates +- source ids +- summary depth/count fields +- info_class classification by source cardinality +- validity/review status +- schema and allowed ID validation + +If adapter is missing or fails, no fake summary is created. A failed summary frame may be recorded with empty `summary_text`, `semantic_judgement_status=failed`, and failure diagnostics. + +### 3.4 Storage/export + +The summary is recorded as `graph_memory:node:summary`. + +The `SUMMARY_OF` edge is recorded as `graph_memory:edge:SUMMARY_OF`. + +Graph export and Vessel write plan should include the summary node and summary edge through existing graph node/edge export behavior. + +## 4. Non-goals + +- Do not mutate `TimeBundle` payload. +- Do not weaken `GraphMemoryNodeFrame` absolute lock. +- Do not open semantic axis. +- Do not auto-feed summary into R loop answers. +- Do not open R1/R2/R3 live LLM traversal. +- Do not add node_4 approval loop yet. +- Do not delete or overwrite old summary nodes. +- Do not make code write semantic summary text. + +## 5. Test Plan + +Required: + +```powershell +python -m compileall songryeon_core main.py +python -m pytest tests/test_order_166_night_time_bundle_summary_node.py +python main.py fast-test --profile graph +git diff --check +``` + +Additional test expectations: + +- multi-leaf `TimeBundle` summary is `mixed`. +- single-leaf `TimeBundle` summary may be `relative`. +- incompatible LLM `info_class` fails and records failed status without fake summary text. +- original `TimeBundle` remains `absolute/not_run`. +- summary node has `summary_depth=1` for raw leaf bundle sources. +- `SUMMARY_OF` edge points from summary node to target time bundle. +- export packet includes summary node and summary edge. +- Vessel write plan treats summary node as graph node operation, not support record. + +## 6. Completion Report Must Include + +- where summary frame/schema was added +- how `relative` vs `mixed` is decided +- how failed LLM summary is handled +- how original TimeBundle stays clean +- export/write-plan behavior +- test results diff --git a/Administrative_Reform_1/04_Orders/ORDER_167_NIGHT_SUMMARY_NAMING_CLARITY_V0.md b/Administrative_Reform_1/04_Orders/ORDER_167_NIGHT_SUMMARY_NAMING_CLARITY_V0.md new file mode 100644 index 0000000..ea0fe35 --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_167_NIGHT_SUMMARY_NAMING_CLARITY_V0.md @@ -0,0 +1,62 @@ +# ORDER 167: Night Summary Naming Clarity v0 + +## 1. Goal + +ORDER 166에서 추가한 심야정부 TimeBundle summary worker의 이름을 더 사람이 이해하기 쉬운 표준 이름으로 정리한다. + +현재 이름: + +```text +night_time_bundle_summary_worker +run_night_time_bundle_summary_worker +night_time_bundle_summary_worker_v0.md +``` + +새 표준 이름: + +```text +night_summarize_time_bundle +run_night_summarize_time_bundle +night_summarize_time_bundle_v0.md +``` + +## 2. Naming Rule + +앞으로 심야정부 작업자는 다음 패턴을 우선한다. + +```text +night__ +``` + +예: + +- `night_summarize_time_bundle` +- `night_summarize_source_kind_bundle` +- `night_review_summary_node` + +이름 안에서 `worker`는 내부 구현 설명으로만 쓰고, 사용자-facing / 호출-facing 이름에서는 줄인다. + +## 3. Scope + +- 새 표준 함수 `run_night_summarize_time_bundle()`을 추가한다. +- 기존 `run_night_time_bundle_summary_worker()`는 compatibility alias로 유지한다. +- 새 prompt ref `songryeon_core/prompts/night_summarize_time_bundle_v0.md`를 사용한다. +- node id와 data type도 새 이름을 기준으로 기록한다. +- 기존 schema명 `NightTimeBundleSummaryFrame`은 유지한다. + +## 4. Non-goals + +- summary schema 구조 변경 없음. +- LLM summary 정책 변경 없음. +- R loop 자동 사용 없음. +- Neo4j/Vessel write 정책 변경 없음. +- source kind bundle summary worker 구현 없음. + +## 5. Test Plan + +```powershell +python -m compileall songryeon_core main.py +python -m pytest tests/test_order_166_night_time_bundle_summary_node.py -q +python main.py fast-test --profile graph +git diff --check +``` diff --git a/Administrative_Reform_1/04_Orders/ORDER_168_NIGHT_SUMMARIZE_CHANGED_SOURCE_LEAVES_V0.md b/Administrative_Reform_1/04_Orders/ORDER_168_NIGHT_SUMMARIZE_CHANGED_SOURCE_LEAVES_V0.md new file mode 100644 index 0000000..de25688 --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_168_NIGHT_SUMMARIZE_CHANGED_SOURCE_LEAVES_V0.md @@ -0,0 +1,104 @@ +# ORDER 168: Night Summarize Changed Source Leaves v0 + +## 1. Goal + +새로 들어왔거나 내용이 바뀐 코드/문서 `raw_source` leaf를 전부 1:1로 LLM 요약한다. + +이번 발주의 핵심: + +```text +RawSource leaf 하나 + <- SUMMARY_OF +SummaryGraphNode 하나 +``` + +즉 source kind bundle을 먼저 요약하지 않는다. + +원본 파일 leaf와 바로 1:1 대응되는 첫 summary node를 만든다. + +## 2. Source Selection Rule + +Code selects source leaves only from `SourceObservationLedgerFrame`. + +요약 대상: + +- `observation_status=new_source_version` +- `observation_status=content_changed` + +요약 제외: + +- `observation_status=unchanged` + +이 선택은 의미 판단이 아니다. + +새 raw source version이 생겼는지, 기존 source content가 바뀌었는지는 code가 content hash와 observation ledger로 확인한 절대정보다. + +## 3. Text Availability Rule + +LLM summary는 원문 text snapshot이 있는 raw source에만 실행한다. + +If a raw source has no text snapshot: + +```text +summary_status=skipped_no_text_snapshot +semantic_judgement_status=not_run +``` + +If a raw source text snapshot is empty: + +```text +summary_status=skipped_empty_text +semantic_judgement_status=not_run +``` + +Code must not invent a summary for missing/empty text. + +## 4. Metainfo Rule + +Each successful summary is directly grounded in exactly one raw source leaf. + +Therefore: + +```text +info_class=relative +source_mode=single_source +claim_alignment=single_absolute_record +``` + +Failed LLM attempts remain `relative/failed` because the attempted semantic output was supposed to be a one-source summary. + +Skipped no-text cases are code-checkable status records: + +```text +info_class=absolute +semantic_judgement_status=not_run +``` + +## 5. Scope + +- Add `NightSourceLeafSummaryFrame`. +- Add `run_night_summarize_source_leaf()`. +- Add `run_night_summarize_changed_source_leaves()`. +- Record successful summaries as `graph_memory:node:summary`. +- Record successful edges as `graph_memory:edge:SUMMARY_OF`. +- Record skipped/failed attempts as `node_output:night_summarize_source_leaf_frame`. +- Include new test in graph fast-test profile. + +## 6. Non-goals + +- Do not summarize unchanged source observations. +- Do not summarize source kind bundles. +- Do not summarize conversation TimeBundles here. +- Do not open semantic axis. +- Do not auto-feed summaries into R loop or node_3. +- Do not make code write semantic summary text. +- Do not delete or overwrite old summaries. + +## 7. Test Plan + +```powershell +python -m compileall songryeon_core main.py +python -m pytest tests/test_order_168_night_summarize_changed_source_leaves.py -q +python main.py fast-test --profile graph +git diff --check +``` diff --git a/Administrative_Reform_1/04_Orders/ORDER_169_NIGHT_CHANGED_SOURCE_SUMMARY_CLI_V0.md b/Administrative_Reform_1/04_Orders/ORDER_169_NIGHT_CHANGED_SOURCE_SUMMARY_CLI_V0.md new file mode 100644 index 0000000..333ad03 --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_169_NIGHT_CHANGED_SOURCE_SUMMARY_CLI_V0.md @@ -0,0 +1,60 @@ +# ORDER 169: Night Changed Source Summary CLI v0 + +## 1. Goal + +ORDER_168의 changed source leaf summary engine을 사람이 터미널에서 한 번에 실행할 수 있게 한다. + +목표 실행 흐름: + +```text +SongRyeon Core source manifest +-> graph source ingest with text snapshots +-> SourceObservationLedgerFrame +-> summarize new/content_changed raw_source leaves +-> graph export packet +-> Vessel write plan +-> optional local Neo4j write +``` + +## 2. Command + +Add a manual opt-in CLI command: + +```powershell +python main.py night-summarize-changed-sources --root . --llm-mode qwen +``` + +Optional Neo4j write: + +```powershell +python main.py night-summarize-changed-sources --root . --llm-mode qwen --write-vessel +``` + +## 3. Rules + +- Code selects source leaves only from `SourceObservationLedgerFrame`. +- Summarize only `new_source_version` and `content_changed`. +- Do not summarize `unchanged`. +- Use raw source text snapshots as the LLM input. +- Successful summaries remain `relative` because each summary maps to exactly one raw source leaf. +- Missing or empty text snapshots must produce skipped status records, not fake summary graph nodes. +- Neo4j write must remain explicit opt-in. + +## 4. Non-goals + +- Do not create semantic axis. +- Do not run R loop. +- Do not feed summaries into node_3 automatically. +- Do not summarize source kind bundles. +- Do not summarize conversation TimeBundles in this command. +- Do not delete or overwrite previous summaries. +- Do not make code write semantic summary text. + +## 5. Test Plan + +```powershell +python -m compileall songryeon_core main.py +python -m pytest tests/test_order_169_night_changed_source_summary_cli.py -q +python main.py fast-test --profile graph +git diff --check +``` diff --git a/Administrative_Reform_1/04_Orders/ORDER_170_NIGHT_CHANGED_SOURCE_ONE_AT_A_TIME_RUNNER_V0.md b/Administrative_Reform_1/04_Orders/ORDER_170_NIGHT_CHANGED_SOURCE_ONE_AT_A_TIME_RUNNER_V0.md new file mode 100644 index 0000000..59d1686 --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_170_NIGHT_CHANGED_SOURCE_ONE_AT_A_TIME_RUNNER_V0.md @@ -0,0 +1,75 @@ +# ORDER 170: Night Changed Source One-At-A-Time Runner v0 + +## 1. Goal + +`night-summarize-changed-sources`가 변경된 코드/문서 leaf를 한 번에 전부 요약하지 않고, 명시 옵션을 켰을 때 한 실행에 source leaf 하나만 요약하게 만든다. + +목표는 긴 심야정부 실행을 다음처럼 바꾸는 것이다. + +```text +첫 실행: +source manifest ingest +-> SourceObservationLedgerFrame +-> changed source leaf queue 고정 +-> queue 첫 leaf 1개 요약 +-> 저장 + +다음 실행: +기존 queue 읽기 +-> 아직 처리 안 된 다음 leaf 1개 요약 +-> 저장 +``` + +## 2. Why + +ORDER_169는 사람이 한 번에 심야 source 요약을 돌릴 수 있게 만들었지만, 변경 leaf가 수백 개이면 Qwen 호출이 길게 이어지고 중간 중단/재개/진척 확인이 어렵다. + +이번 발주는 성능 최적화가 아니라 실행 안정성 MVP다. 한 번에 하나씩 처리하면 사용자는 중간에 멈추거나 다시 실행해도 어디까지 진행됐는지 확인할 수 있다. + +## 3. Required Behavior + +- 기본 `night-summarize-changed-sources` 동작은 유지한다. +- 새 옵션 `--one-at-a-time`을 켰을 때만 한 실행에 source leaf 1개만 요약한다. +- one-at-a-time 최초 실행은 `SourceObservationLedgerFrame`에서 `new_source_version` / `content_changed` leaf 목록을 절대정보 queue로 고정한다. +- 이후 실행은 새 관측으로 남은 대상을 잃지 않고 기존 queue를 읽어 다음 미처리 leaf를 고른다. +- 처리 완료 여부는 code가 기존 summary node/frame 존재 여부로 계산한다. +- LLM은 여전히 leaf 원문 하나의 요약문만 쓴다. +- code는 요약 의미를 쓰지 않는다. +- queue/progress 값은 `generated_by=CODE:*`, `info_class=absolute`, `semantic_judgement_status=not_run`으로 둔다. +- optional Neo4j write는 기존처럼 명시 `--write-vessel`이 있을 때만 실행한다. + +## 4. Non-goals + +- R loop를 열지 않는다. +- semantic axis를 만들지 않는다. +- source kind bundle / time bundle 상위 요약은 만들지 않는다. +- 실패 leaf 자동 재시도 정책을 만들지 않는다. +- 이전 summary를 삭제하거나 덮어쓰지 않는다. +- DataStore record를 in-place update하지 않는다. + +## 5. Command + +```powershell +python main.py night-summarize-changed-sources --root . --llm-mode qwen --one-at-a-time +``` + +Neo4j까지 매 step 반영: + +```powershell +python main.py night-summarize-changed-sources --root . --llm-mode qwen --one-at-a-time --write-vessel +``` + +새 queue를 강제로 시작해야 하면 새 `--batch-id`를 명시한다. + +```powershell +python main.py night-summarize-changed-sources --root . --llm-mode qwen --one-at-a-time --batch-id night_sources_2026_07_02_manual_002 +``` + +## 6. Test Plan + +```powershell +python -m compileall songryeon_core main.py +python -m pytest tests/test_order_170_night_changed_source_one_at_a_time.py -q +python -m pytest tests/test_order_169_night_changed_source_summary_cli.py -q +git diff --check +``` diff --git a/Administrative_Reform_1/04_Orders/ORDER_171_NIGHT_TOKEN_BUDGET_LAYER_SUMMARY_V0.md b/Administrative_Reform_1/04_Orders/ORDER_171_NIGHT_TOKEN_BUDGET_LAYER_SUMMARY_V0.md new file mode 100644 index 0000000..ed02f1a --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_171_NIGHT_TOKEN_BUDGET_LAYER_SUMMARY_V0.md @@ -0,0 +1,67 @@ +# ORDER 171: Night Token Budget Layer Summary v0 + +## 1. Goal + +심야정부가 source leaf 요약을 만든 뒤, 그 요약들을 날짜 단위가 아니라 예산 단위로 묶어 상위 summary layer를 만들게 한다. + +이번 MVP의 목표 흐름: + +```text +source leaf summary nodes +-> code-generated token budget summary bundle queue +-> one bundle graph node +-> LLM summary graph node attached to that bundle +-> optional Vessel write +``` + +## 2. Important v0 Budget Boundary + +정확한 tokenizer adapter는 아직 없다. + +따라서 v0은 토큰 예산 구조를 여는 발주서이지만, 실제 계량 단위는 code가 확정 가능한 `summary_text` 문자 수(`char_budget`)로 둔다. 이 값을 토큰 수라고 숨기지 않는다. + +나중에 tokenizer가 추가되면 queue/bundle 구조는 유지하고 budget calculator만 교체한다. + +## 3. Required Behavior + +- leaf summary graph node 중 `summary_status=ran`, `validity_status=active`, `data_kind=source_leaf_summary`인 것만 대상으로 삼는다. +- code가 summary text 길이를 세고, `max_bundle_chars` 안에 들어가도록 summary node들을 순서대로 묶는다. +- source summary 하나가 budget보다 크면 원문을 자르지 않고 단독 bundle로 둔다. +- queue는 code-generated absolute record로 남긴다. +- 실행은 한 번에 한 bundle만 처리한다. +- bundle graph node는 원본 summary node들을 `CONTAINS` edge로 가진다. +- LLM은 bundle 안의 summary text들을 보고 상위 요약문만 쓴다. +- 여러 summary에 근거한 상위 요약이므로 성공한 summary는 `info_class=mixed`다. +- code는 의미 요약문을 쓰지 않는다. +- 원본 leaf summary node는 삭제/수정하지 않는다. +- optional Vessel/Neo4j write는 명시 `--write-vessel`이 있을 때만 한다. + +## 4. Non-goals + +- 정확 tokenizer 구현은 하지 않는다. +- semantic axis는 만들지 않는다. +- R loop live route는 열지 않는다. +- CoreEgo 직속 의미축 재편성은 하지 않는다. +- 모든 layer를 한 번에 끝까지 재귀 요약하지 않는다. +- failed bundle 자동 재시도 정책은 만들지 않는다. + +## 5. Command + +```powershell +python main.py night-summarize-token-layer --store-dir .songryeon_core_cache/night_changed_sources --llm-mode qwen --max-bundle-chars 8000 +``` + +Neo4j까지 매 step 반영: + +```powershell +python main.py night-summarize-token-layer --llm-mode qwen --max-bundle-chars 8000 --write-vessel +``` + +## 6. Test Plan + +```powershell +python -m compileall songryeon_core main.py +python -m pytest tests/test_order_171_night_token_budget_layer_summary.py -q +python main.py fast-test --profile graph +git diff --check +``` diff --git a/Administrative_Reform_1/04_Orders/ORDER_172_NIGHT_TOKEN_LAYER_AUTO_REDUCE_UNTIL_CONTEXT_BUDGET_V0.md b/Administrative_Reform_1/04_Orders/ORDER_172_NIGHT_TOKEN_LAYER_AUTO_REDUCE_UNTIL_CONTEXT_BUDGET_V0.md new file mode 100644 index 0000000..039418d --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_172_NIGHT_TOKEN_LAYER_AUTO_REDUCE_UNTIL_CONTEXT_BUDGET_V0.md @@ -0,0 +1,50 @@ +# ORDER 172: Night Token Layer Auto Reduce Until Context Budget v0 + +## 1. Goal + +ORDER_171의 token-budget layer summary를 사람이 수십 번 반복 실행하지 않아도 되게 한다. + +목표는 "몇 계층까지"를 고정 숫자로 정하는 것이 아니라, 현재 활성 summary layer 전체량이 target context budget 이하가 될 때까지 계층 요약을 자동으로 조금씩 진행하는 것이다. + +## 2. Command + +```powershell +python main.py night-summarize-token-layer --llm-mode qwen --max-bundle-chars 8000 --until-context-budget --target-context-chars 12000 --max-layer-depth 5 --max-steps 5 +``` + +Neo4j까지 반영: + +```powershell +python main.py night-summarize-token-layer --llm-mode qwen --max-bundle-chars 8000 --until-context-budget --target-context-chars 12000 --max-layer-depth 5 --max-steps 5 --write-vessel +``` + +## 3. Rules + +- 기본 one-bundle-at-a-time 동작은 유지한다. +- `--until-context-budget`을 켰을 때만 자동 반복한다. +- 자동 반복은 한 번에 무제한으로 돌지 않는다. +- `--max-steps`로 이번 실행에서 처리할 최대 bundle 수를 제한한다. +- layer 2 queue가 미완료이면 먼저 layer 2를 계속 처리한다. +- layer N이 완료된 뒤 그 layer의 성공 summary 전체 문자 수가 `target_context_chars` 이하이면 멈춘다. +- 아직 크면 layer N+1 queue를 만든다. +- `max_layer_depth`에 도달하면 더 깊게 가지 않고 멈춘다. +- v0 budget 단위는 tokenizer가 아니라 code-counted characters다. +- 실패 bundle은 다음 layer 재료로 쓰지 않는다. +- code는 의미 요약을 쓰지 않는다. + +## 4. Non-goals + +- 정확 tokenizer adapter는 만들지 않는다. +- semantic axis는 만들지 않는다. +- R loop live route는 열지 않는다. +- failed bundle retry policy는 만들지 않는다. +- 원본/기존 summary node를 삭제하거나 덮어쓰지 않는다. + +## 5. Test Plan + +```powershell +python -m compileall songryeon_core main.py +python -m pytest tests/test_order_172_night_token_layer_auto_reduce.py -q +python main.py fast-test --profile graph +git diff --check +``` diff --git a/Administrative_Reform_1/04_Orders/ORDER_173_NIGHT_CHECKPOINTED_LONG_RUNNER_V0.md b/Administrative_Reform_1/04_Orders/ORDER_173_NIGHT_CHECKPOINTED_LONG_RUNNER_V0.md new file mode 100644 index 0000000..3d34eeb --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_173_NIGHT_CHECKPOINTED_LONG_RUNNER_V0.md @@ -0,0 +1,59 @@ +# ORDER 173: Night Checkpointed Long Runner v0 + +## 1. Goal + +심야정부 token layer 요약을 사람이 수동으로 여러 번 치지 않아도 되게 한다. + +단, "무식한 한 방"이 아니라 다음 성질을 가진 긴 실행 runner로 만든다. + +```text +loop: + 다음 pending bundle 1개 처리 + DataStore/TraceStore 저장 + progress JSONL 기록 + 선택 정책에 따라 Vessel write + failure/runtime/max_steps 조건 확인 +``` + +## 2. Required Behavior + +- 기존 `--until-context-budget` 자동 reducer를 유지한다. +- 매 step 뒤 DataStore/TraceStore checkpoint를 남긴다. +- 매 step 진행 상황을 JSONL로 남길 수 있게 한다. +- `--max-runtime-minutes`로 이번 실행의 시간 상한을 둘 수 있게 한다. +- `--stop-on-failure` 기본값으로 실패 bundle이 나오면 멈춘다. +- Vessel write는 다음 모드를 지원한다. + - `none` + - `every-step` + - `at-end` +- 기존 `--write-vessel`은 호환상 `every-step`과 같은 의미로 둔다. +- 중간에 죽어도 재실행 시 queue와 기존 summary node/frame을 보고 이어간다. + +## 3. Non-goals + +- 정확 tokenizer adapter는 만들지 않는다. +- semantic axis는 만들지 않는다. +- R loop live route는 열지 않는다. +- failed bundle retry policy는 만들지 않는다. +- 원본/기존 summary node를 삭제하거나 덮어쓰지 않는다. + +## 4. Command + +```powershell +python main.py night-summarize-token-layer --llm-mode qwen --max-bundle-chars 8000 --until-context-budget --target-context-chars 12000 --max-layer-depth 5 --max-steps 999 --max-runtime-minutes 60 --vessel-write-mode at-end +``` + +기존 방식: + +```powershell +python main.py night-summarize-token-layer --llm-mode qwen --until-context-budget --max-steps 5 --write-vessel +``` + +## 5. Test Plan + +```powershell +python -m compileall songryeon_core main.py +python -m pytest tests/test_order_173_night_checkpointed_long_runner.py -q +python main.py fast-test --profile graph +git diff --check +``` diff --git a/Administrative_Reform_1/04_Orders/ORDER_174_VESSEL_INSPECT_SUMMARY_LAYER_VIEW_V0.md b/Administrative_Reform_1/04_Orders/ORDER_174_VESSEL_INSPECT_SUMMARY_LAYER_VIEW_V0.md new file mode 100644 index 0000000..5346aa1 --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_174_VESSEL_INSPECT_SUMMARY_LAYER_VIEW_V0.md @@ -0,0 +1,44 @@ +# ORDER 174: Vessel Inspect Summary Layer View v0 + +## 1. Goal + +Neo4j Vessel에 저장된 심야정부 요약 노드가 실제로 얼마나 생성됐고, 어떤 계층/종류로 쌓였는지 사람이 바로 볼 수 있게 한다. + +현재 `vessel-inspect --format text`는 `CoreEgo -> Time Axis -> Time Bundle -> Raw Capsule` 기본 경로만 보여준다. +그 결과 source leaf summary나 token budget bundle summary가 Neo4j에 들어갔어도 화면에서는 보이지 않는다. + +## 2. Required Behavior + +- `vessel-inspect`가 read-only로 `SummaryGraphNode`를 조회한다. +- 다음 절대정보를 결과 payload와 text renderer에 표시한다. + - summary node 총수 + - active / invalidated summary 수 + - `data_kind`별 summary 수 + - `summary_depth`별 summary 수 + - sample summary node 목록 +- sample에는 다음 필드를 보여준다. + - summary data id + - data kind + - summary depth + - info class + - target graph node id / display name + - summary text preview +- 코드는 새 요약이나 의미 판단을 만들지 않는다. +- Neo4j에 이미 저장된 node property와 `payload_json`만 읽는다. + +## 3. Non-goals + +- R loop route를 열지 않는다. +- semantic axis를 만들지 않는다. +- 요약 품질을 평가하지 않는다. +- summary를 새로 생성하거나 수정하지 않는다. +- graph DB schema를 파괴적으로 바꾸지 않는다. + +## 4. Test Plan + +```powershell +python -m compileall songryeon_core main.py +python -m pytest tests/test_order_163_vessel_inspect_manual_walk.py tests/test_order_174_vessel_inspect_summary_layer_view.py -q +python main.py fast-test --profile graph +git diff --check +``` diff --git a/Administrative_Reform_1/04_Orders/ORDER_175_VESSEL_BACKED_R_GRAPH_READ_PACKET_V0.md b/Administrative_Reform_1/04_Orders/ORDER_175_VESSEL_BACKED_R_GRAPH_READ_PACKET_V0.md new file mode 100644 index 0000000..d922efd --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_175_VESSEL_BACKED_R_GRAPH_READ_PACKET_V0.md @@ -0,0 +1,57 @@ +# ORDER 175: Vessel-Backed R Graph Read Packet v0 + +## 1. Goal + +R루프가 실제 Neo4j Vessel 그래프를 바로 의미 판단하지 않고, 먼저 읽기 전용 후보 봉투로 받아볼 수 있게 한다. + +현재 R루프 frame/state machine과 dry-run 구조는 존재하지만, 실제 Vessel 그래프의 최신 summary/node 후보를 R루프 입력 후보로 안전하게 읽는 얇은 경계가 부족하다. + +이번 발주는 R1/R2/R3 LLM 선택을 열기 전, `CoreEgo -> Time Axis` 주변의 진입 후보와 active summary 후보를 절대정보 패킷으로 고정하는 MVP다. + +## 2. Required Behavior + +- 새 CLI `vessel-r-read-packet`을 추가한다. +- Neo4j Vessel을 read-only로 조회한다. +- 다음 entry 후보를 읽는다. + - `TimeBundle` + - `SourceIngestBundle` +- 다음 summary 후보를 읽는다. + - `SummaryGraphNode` + - `payload_json.summary_status == "ran"` + - `payload_json.validity_status == "active"` +- invalidated summary, failed summary, text 없는 skipped summary는 R 후보에서 제외한다. +- packet에는 다음 절대정보를 남긴다. + - entry 후보 수 + - active summary 후보 수 + - scan한 summary 수 + - 제외된 summary 수 + - summary `data_kind`별 수 + - summary `summary_depth`별 수 + - 후보 node id, target id, summary text, source id +- `generated_by=CODE:R_LOOP_VESSEL_READ_PACKET_BUILDER` +- `info_class=absolute` +- `semantic_judgement_status=not_run` + +## 3. Non-goals + +- R route를 기본 live route로 열지 않는다. +- R1/R2/R3 LLM을 호출하지 않는다. +- 어떤 그래프 후보가 사용자 질문에 관련 있는지 code가 판단하지 않는다. +- semantic axis를 만들지 않는다. +- Neo4j에 새 노드/관계를 쓰지 않는다. +- summary를 생성/수정/무효화하지 않는다. + +## 4. Test Plan + +```powershell +python -m compileall songryeon_core main.py +python -m pytest tests/test_order_175_vessel_backed_r_read_packet.py -q +python main.py fast-test --profile graph +git diff --check +``` + +수동 확인: + +```powershell +python main.py vessel-r-read-packet --database neo4j --format text +``` diff --git a/Administrative_Reform_1/04_Orders/ORDER_176_VESSEL_R_ONE_STEP_TRAVERSAL_V0.md b/Administrative_Reform_1/04_Orders/ORDER_176_VESSEL_R_ONE_STEP_TRAVERSAL_V0.md new file mode 100644 index 0000000..68c6aef --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_176_VESSEL_R_ONE_STEP_TRAVERSAL_V0.md @@ -0,0 +1,56 @@ +# ORDER 176: Vessel R One-Step Traversal v0 + +## 1. Goal + +ORDER_175에서 만든 `RLoopVesselReadPacketFrame`을 R루프가 실제로 한 번 읽고, LLM이 후보 하나를 선택한 뒤, code가 그 선택을 검증하는 최소 R 탐색 MVP를 만든다. + +이번 발주는 full R loop가 아니다. +목표는 Neo4j Vessel에 저장된 entry/summary 후보를 R1/R2/R3가 한 단계만 다뤄보고, 선택과 검사 결과를 trace/DataStore에 남기는 것이다. + +## 2. Required Behavior + +- 새 CLI `vessel-r-one-step`을 추가한다. +- 실행 흐름은 다음 순서로 제한한다. + - `vessel-r-read-packet`과 같은 read-only packet 생성 + - R1: graph search goal / desired granularity 판단 + - code: one-step budget frame 생성 + - R2: packet 안의 `available_graph_node_ids` 중 하나 선택 + - code: R2 선택 ID가 packet 안에 있는지 검증 + - R3: 선택된 candidate record를 보고 sufficiency / granularity / branch 판단 + - code: continuation / return summary / one-step result frame 기록 +- R2가 packet 밖 ID를 고르면 schema failure로 닫는다. +- R3가 child node id를 invent해도 code는 사용하지 않는다. + - child 좌표는 packet 안의 `source_graph_node_ids` / `target_graph_node_id`만 사용한다. +- adapter가 없으면 선택 fallback을 만들지 않고 실패 frame으로 닫는다. +- 기본 live qwen-chat route=R은 열지 않는다. + +## 3. Metainfo Boundary + +- R1/R2/R3의 goal/selection/inspection reason은 LLM 의미 판단이므로 `info_class=mixed`, `semantic_judgement_status=ran`으로 기록한다. +- code가 만든 budget/continuation/return summary/result frame은 구조화된 절대정보이므로 `info_class=absolute`, `semantic_judgement_status=not_run`으로 기록한다. +- code는 candidate relevance, sufficiency meaning, branch meaning을 대신 판단하지 않는다. +- code는 ID 존재 여부, 예산, packet 내부 좌표 복사만 맡는다. + +## 4. Non-goals + +- R route를 기본 live route로 열지 않는다. +- multi-step R traversal을 실제 Vessel DB에 연결하지 않는다. +- R 결과를 node_3 최종 답변에 자동 주입하지 않는다. +- semantic axis를 만들지 않는다. +- Neo4j에 새 노드/관계를 쓰지 않는다. +- summary 생성/수정/무효화를 하지 않는다. + +## 5. Test Plan + +```powershell +python -m compileall songryeon_core main.py +python -m pytest tests/test_order_176_vessel_r_one_step_traversal.py -q +python main.py fast-test --profile graph +git diff --check +``` + +수동 확인: + +```powershell +python main.py vessel-r-one-step "송련 Core의 그래프 기억 구조를 한 단계만 탐색해줘" --database neo4j --llm-mode fake --format text +``` diff --git a/Administrative_Reform_1/04_Orders/ORDER_177_R1_CANDIDATE_TEXT_BLINDNESS_V0.md b/Administrative_Reform_1/04_Orders/ORDER_177_R1_CANDIDATE_TEXT_BLINDNESS_V0.md new file mode 100644 index 0000000..1141a1c --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_177_R1_CANDIDATE_TEXT_BLINDNESS_V0.md @@ -0,0 +1,53 @@ +# ORDER 177: R1 Candidate Text Blindness v0 + +## 1. Goal + +R1이 후보 summary 본문이나 개별 후보 내용에 끌려 사용자 질문 밖으로 목표가 새는 문제를 막는다. + +R1은 L1처럼 목표 설정자여야 한다. +따라서 R1은 사용자 질문과 Vessel packet의 절대 통계/count/kind/depth만 보고 graph search goal을 세우고, 개별 후보 본문은 R2/R3 단계에서만 보게 한다. + +## 2. Required Behavior + +- R1 input에서 다음을 제거한다. + - `summary_text` + - `summary_text_preview` + - `summary_candidate_samples` + - `entry_candidate_samples` + - 개별 후보 node id 목록 +- R1 input에는 다음만 남긴다. + - user question + - read packet id + - entry candidate count + - summary candidate count + - summary count by data kind + - summary count by depth + - one-step policy/budget + - R1은 후보 본문을 보지 않는다는 policy note +- R2 input은 기존처럼 후보 목록과 summary text를 볼 수 있다. +- R3 input은 R2가 선택한 후보 하나의 record만 볼 수 있다. +- R1 output validator는 user question의 핵심 anchor가 사라지는 것을 막기 위한 최소 구조 검사를 추가한다. + - 휴리스틱 의미 판단이 아니라, R1 goal이 user question의 의미 있는 토큰 중 하나도 포함하지 않으면 schema failure로 닫는다. + +## 3. Non-goals + +- R route를 기본 live route로 열지 않는다. +- R2/R3 선택 품질을 이번에 크게 바꾸지 않는다. +- multi-step Vessel traversal을 열지 않는다. +- Neo4j write를 수행하지 않는다. +- summary 생성/수정/무효화를 하지 않는다. + +## 4. Test Plan + +```powershell +python -m compileall songryeon_core main.py +python -m pytest tests/test_order_176_vessel_r_one_step_traversal.py tests/test_order_177_r1_candidate_text_blindness.py -q +python main.py fast-test --profile graph +git diff --check +``` + +수동 확인: + +```powershell +python main.py vessel-r-one-step "송련 Core의 그래프 기억 구조에서 source summary와 token layer summary가 어떻게 이어지는지 한 단계만 골라봐" --database neo4j --llm-mode qwen --timeout 120 --format text +``` diff --git a/Administrative_Reform_1/04_Orders/ORDER_178_R_VESSEL_CANDIDATE_LAYER_SURFACE_V0.md b/Administrative_Reform_1/04_Orders/ORDER_178_R_VESSEL_CANDIDATE_LAYER_SURFACE_V0.md new file mode 100644 index 0000000..7c7db14 --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_178_R_VESSEL_CANDIDATE_LAYER_SURFACE_V0.md @@ -0,0 +1,78 @@ +# ORDER 178: R Vessel Candidate Layer Surface v0 + +## Status + +Proposed and approved for immediate implementation on 2026-07-02. + +## Problem + +ORDER 177 removed candidate text from R1, so R1 now behaves like a goal setter instead of reading the whole candidate board. + +The next live test showed a remaining bottleneck: R2 still receives a flat candidate list. If the first 50 active summaries are dominated by one kind of node, R2 can pick an unrelated first-looking candidate even when the user asks about another layer. + +This is not a semantic-routing problem for code to solve. It is a candidate presentation problem. + +## Goal + +Build a code-generated absolute "candidate layer surface" for Vessel-backed R one-step traversal. + +R2 should first see a shelf/table-of-contents view of candidate groups, then select one candidate inside a chosen shelf. + +## Scope + +- Add a candidate layer surface frame built only from existing packet fields. +- Group candidates by explicit absolute fields: + - entry candidate kind + - summary data kind + - summary depth + - info class +- Update R2 input so it receives: + - available surface IDs + - surface records with counts + - candidate records grouped by surface +- Require R2 to output: + - selected_surface_id + - selected_graph_node_id +- Validate that: + - selected_surface_id exists + - selected_graph_node_id belongs to the selected surface +- Adjust the Vessel read packet candidate limit policy so active summary candidates are selected across data-kind/depth groups instead of taking the first flat 50. + +## Non-Goals + +- Do not make code decide which surface is semantically relevant. +- Do not add a multi-step R loop. +- Do not add meaning-axis CoreEgo links. +- Do not change Neo4j schema shape. +- Do not weaken R1/R2/R3 schema validation. +- Do not connect R route to live node_1/node_2/node_3 flow. + +## Metadata Boundary + +The candidate layer surface is absolute information: + +- generated_by: `CODE:R_LOOP_VESSEL_CANDIDATE_LAYER_SURFACE_BUILDER` +- info_class: `absolute` +- semantic_judgement_status: `not_run` + +R2's surface and node choice remains LLM semantic judgment: + +- generated_by: `LLM:*:R2_vessel_node_selector` +- info_class: `mixed` +- semantic_judgement_status: `ran` + +## Completion Criteria + +- `python -m compileall songryeon_core main.py` +- `python -m pytest tests/test_order_175_vessel_backed_r_read_packet.py tests/test_order_176_vessel_r_one_step_traversal.py tests/test_order_177_r1_candidate_text_blindness.py tests/test_order_178_r_vessel_candidate_layer_surface.py -q` +- `python main.py fast-test --profile graph` +- `python main.py smoke-test` + +## Expected Tests + +- Candidate surface groups packet candidates by explicit absolute fields. +- Surface frame does not contain summary text. +- R2 payload is grouped by surface and no longer exposes flat `summary_candidate_records`. +- R2 fails if it selects a graph node outside the selected surface. +- Read packet limiting keeps multiple summary kinds/depths visible when one kind has many records. + diff --git a/Administrative_Reform_1/04_Orders/ORDER_179_R2_VESSEL_SELECTION_ID_DISAMBIGUATION_V0.md b/Administrative_Reform_1/04_Orders/ORDER_179_R2_VESSEL_SELECTION_ID_DISAMBIGUATION_V0.md new file mode 100644 index 0000000..5c5af42 --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_179_R2_VESSEL_SELECTION_ID_DISAMBIGUATION_V0.md @@ -0,0 +1,68 @@ +# ORDER 179: R2 Vessel Selection ID Disambiguation v0 + +## Status + +Proposed after the first live ORDER_178 Qwen test on 2026-07-02. + +## Trigger + +Live command: + +```powershell +python main.py vessel-r-one-step "송련 Core의 그래프 기억 구조에서 source summary와 token layer summary가 어떻게 이어지는지 한 단계만 골라봐" --database neo4j --llm-mode qwen --timeout 120 --format text +``` + +Observed result: + +- `read_packet_status=passed` +- `entry_candidate_count=3` +- `summary_candidate_count=50` +- `failure_stage=R2` +- `failure_type=schema_failed` +- `failure_reason=R2 selected_graph_node_id must be in available_graph_node_ids` + +## Problem + +ORDER_178 added candidate layer surfaces, but R2 can still confuse selection IDs with explanatory/source IDs inside candidate records. + +Most likely confusion: + +- should select: `graph_node_id` +- may have selected instead: `target_graph_node_id`, `source_graph_node_ids`, or other source/target IDs + +This is not a reason to weaken validation. It is a payload clarity and diagnostics issue. + +## Goal + +Make R2's selectable ID field unambiguous and make failed R2 payloads visible enough to debug. + +## Scope + +- R2 candidate records expose `graph_node_id` as the only direct selectable graph node field. +- R2 candidate records do not expose `target_graph_node_id` or `source_graph_node_ids`. +- R2 prompt explicitly says: + - choose `selected_surface_id` from `available_surface_ids` + - choose `selected_graph_node_id` by copying a candidate record's `graph_node_id` + - do not use target/source/data IDs as selected graph node IDs +- Failed R2 schema validation records/prints a compact payload summary: + - selected surface ID + - selected graph node ID + - whether the surface ID was allowed + - whether the graph node ID was globally allowed + - whether the graph node ID belonged to the selected surface + +## Non-Goals + +- Do not allow invalid selected graph node IDs. +- Do not let code choose a semantic fallback candidate. +- Do not add two-stage R2a/R2b yet. +- Do not open multi-step R traversal. +- Do not change Neo4j schema or write behavior. + +## Completion Criteria + +- `python -m compileall songryeon_core main.py` +- `python -m pytest tests/test_order_176_vessel_r_one_step_traversal.py tests/test_order_177_r1_candidate_text_blindness.py tests/test_order_178_r_vessel_candidate_layer_surface.py tests/test_order_179_r2_vessel_selection_id_disambiguation.py -q` +- `python main.py fast-test --profile graph` +- `git diff --check` + diff --git a/Administrative_Reform_1/04_Orders/ORDER_180_R2_PROMPT_EXAMPLE_ID_REMOVAL_V0.md b/Administrative_Reform_1/04_Orders/ORDER_180_R2_PROMPT_EXAMPLE_ID_REMOVAL_V0.md new file mode 100644 index 0000000..a438559 --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_180_R2_PROMPT_EXAMPLE_ID_REMOVAL_V0.md @@ -0,0 +1,47 @@ +# ORDER 180: R2 Prompt Example ID Removal v0 + +## Status + +Proposed and approved for immediate implementation after the second ORDER_178/179 live Qwen test on 2026-07-02. + +## Trigger + +Live result: + +```text +failure_stage: R2 +failure_type: schema_failed +failure_reason: R2 selected_graph_node_id must be in available_graph_node_ids +failure_payload_summary: { + 'selected_surface_id': 'surface:summary:data:source_leaf_summary:depth:1:info:relative', + 'selected_graph_node_id': 'graph:summary:source_leaf:example', + 'selected_surface_id_in_available': True, + 'selected_graph_node_id_in_available': False, + 'selected_graph_node_id_in_selected_surface': False +} +``` + +## Diagnosis + +R2 copied the prompt's sample `selected_graph_node_id` value instead of copying a runtime candidate record's `graph_node_id`. + +This is prompt example leakage, not a graph database read failure. + +## Goal + +Remove concrete fake graph IDs from the R2 prompt so the model cannot copy a plausible-looking example ID as if it were a real candidate. + +## Scope + +- Remove the JSON code block with concrete example IDs from `r2_vessel_node_selector_v0.md`. +- Replace it with a key contract written as field requirements. +- Keep the R2 validator strict. +- Add a regression test that the prompt does not contain the leaked example ID. + +## Non-Goals + +- Do not weaken R2 schema validation. +- Do not make code select a fallback candidate. +- Do not add R2a/R2b two-stage traversal yet. +- Do not change Neo4j data. + diff --git a/Administrative_Reform_1/04_Orders/ORDER_181_R2_OFFICIAL_SELECTION_REF_MAP_V0.md b/Administrative_Reform_1/04_Orders/ORDER_181_R2_OFFICIAL_SELECTION_REF_MAP_V0.md new file mode 100644 index 0000000..698523b --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_181_R2_OFFICIAL_SELECTION_REF_MAP_V0.md @@ -0,0 +1,55 @@ +# ORDER 181: R2 Official Selection Ref Map v0 + +## Status + +Proposed after the third live R one-step Qwen test on 2026-07-02. + +## Trigger + +Live R2 failure: + +```text +failure_stage: R2 +failure_type: schema_failed +failure_reason: R2 selected_surface_id must be in available_surface_ids +failure_payload_summary: { + 'selected_surface_id': 'surface_003', + 'selected_graph_node_id': 'node_073', + 'selected_surface_id_in_available': False, + 'selected_graph_node_id_in_available': False +} +``` + +## Diagnosis + +After fake example IDs were removed, Qwen still avoided copying long graph IDs and invented short labels like `surface_003` and `node_073`. + +The structural issue is that R2 should not have to copy long Neo4j graph IDs at all. + +## Goal + +Provide official short selection refs generated by code: + +- `surface_ref`, for example `surface_001` +- `node_ref`, for example `node_001` + +R2 selects refs. Code maps refs back to real graph IDs and records actual IDs in existing R frames. + +## Metadata Boundary + +The ref map is absolute information: + +- generated by code from the read packet and candidate surface +- no semantic ranking +- no fallback candidate selection + +R2 still performs semantic selection. + +## Non-Goals + +- Do not accept arbitrary invented refs. +- Do not weaken validation. +- Do not open R2a/R2b. +- Do not open multi-step R traversal. +- Do not change Neo4j data. + diff --git a/Administrative_Reform_1/04_Orders/ORDER_182_R_CORE_EGO_START_SELECTION_SURFACE_V0.md b/Administrative_Reform_1/04_Orders/ORDER_182_R_CORE_EGO_START_SELECTION_SURFACE_V0.md new file mode 100644 index 0000000..99c488a --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_182_R_CORE_EGO_START_SELECTION_SURFACE_V0.md @@ -0,0 +1,56 @@ +# ORDER 182: R CoreEgo Start Selection Surface v0 + +## Status + +Approved by the user on 2026-07-02 for immediate small implementation. + +## Trigger + +Live R one-step tests no longer failed on invented IDs after ORDER_181, but R2 still received too many deep candidates too early. + +Observed shape: + +- `entry_candidate_count=3` +- `summary_candidate_count=50` +- R2 could reason over source leaf summaries and token layer summaries in the first step. + +This contradicted the intended traversal model: + +```text +CoreEgo -> Time Axis -> Time Bundle / Source Ingest Bundle -> lower graph nodes +``` + +R2 should not start by seeing many leaf summaries. R2 should first choose the next graph entry point from CoreEgo's direct entry layer, and R3 should inspect that selected node. + +## Goal + +Make the first R one-step selection surface CoreEgo-started: + +- R2 first view is entry/root candidates only. +- If the read packet contains a `TimeAxis` entry candidate, prefer that as the first CoreEgo direct child surface. +- If no `TimeAxis` entry exists, fall back to existing entry bundle candidates. +- Summary candidates remain in the read packet for later use, but are not exposed in the first R2 selection payload. + +## Metadata Boundary + +The first-step surface is absolute information: + +- code groups existing read-packet entries; +- code does not semantically choose the best candidate; +- R2 still performs the semantic selection among the visible official refs. + +## Non-Goals + +- Do not introduce R2a/R2b or A/B style selection. +- Do not open full multi-step R traversal yet. +- Do not delete summary candidates from the read packet. +- Do not weaken R2 ref validation. +- Do not change node_1 routing, node_3 answer generation, W/R scheduler policy, or external DB schema beyond read packet visibility. + +## Verification Targets + +- R2 payload hides `summary_text`, `summary_node_id`, and summary data kinds in the first view. +- R2 payload keeps official `surface_ref` / `node_ref` selection. +- TimeAxis candidates are recognized as `time_axis`. +- If TimeAxis is present, first surface exposes TimeAxis only. +- If TimeAxis is absent, entry bundles remain available as fallback. diff --git a/Administrative_Reform_1/04_Orders/ORDER_183_R_VESSEL_HIERARCHICAL_CHILD_CANDIDATE_SURFACE_V0.md b/Administrative_Reform_1/04_Orders/ORDER_183_R_VESSEL_HIERARCHICAL_CHILD_CANDIDATE_SURFACE_V0.md new file mode 100644 index 0000000..426dbbe --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_183_R_VESSEL_HIERARCHICAL_CHILD_CANDIDATE_SURFACE_V0.md @@ -0,0 +1,60 @@ +# ORDER 183: R Vessel Hierarchical Child Candidate Surface v0 + +## Status + +Approved by the user on 2026-07-02 for immediate implementation. + +## Trigger + +After ORDER_182, R2 no longer sees 50 leaf summaries at the first CoreEgo step. However, the actual R Vessel read flow still needs a hierarchy-aware next layer: + +```text +CoreEgo -> TimeAxis -> SourceIngestBundle -> SourceKindBundle -> RawSource / Summary +``` + +The graph already contains lower-level source and summary nodes, but the one-step Vessel R flow did not yet expose the selected node's direct children as a structured downstream candidate surface. + +## Goal + +Add a minimal hierarchy read path: + +- Read packet records enough metadata to infer child candidates from graph node payloads and edges. +- When R2 selects a node, code builds direct child candidate records for that selected node. +- R3 receives those child candidates as context. +- After R3, code records an `RGraphTraversalCandidateSurfaceFrame` using the existing R loop schema. +- The candidate surface is absolute information and can be used by a later full multi-step R traversal. + +## Intended Reading Order + +This order does not make R2 recursively loop yet. It prepares the exact next-layer surface: + +```text +R2 selects current node +R3 inspects current node + child candidates +code records child candidate surface +continuation can truthfully say whether deeper traversal is possible +``` + +## Metadata Boundary + +- Code may copy graph IDs, child IDs, node kinds, counts, and display names. +- Code must not decide which child is semantically best. +- R2/R3 semantic decisions remain LLM-generated mixed information. +- The child candidate surface is code-generated absolute information. + +## Non-Goals + +- Do not open full automatic multi-step R traversal yet. +- Do not introduce R2a/R2b or A/B selection formats. +- Do not delete or rewrite graph nodes. +- Do not weaken R2 official ref validation. +- Do not make code choose a semantic child candidate. +- Do not connect R loop output to node_3 final answer yet. + +## Verification Targets + +- TimeAxis selection exposes time/source ingest bundles as child candidates. +- SourceIngestBundle can expose SourceKindBundle children when present in the read packet. +- Token budget summary bundle can expose its source summary children through `source_graph_node_ids`. +- R3 child IDs are copied from code-built hierarchy records, not invented by the LLM. +- The existing `RGraphTraversalCandidateSurfaceFrame` validator is used. diff --git a/Administrative_Reform_1/04_Orders/ORDER_184_R_VESSEL_MULTI_STEP_TRAVERSAL_MVP_V0.md b/Administrative_Reform_1/04_Orders/ORDER_184_R_VESSEL_MULTI_STEP_TRAVERSAL_MVP_V0.md new file mode 100644 index 0000000..8a19ead --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_184_R_VESSEL_MULTI_STEP_TRAVERSAL_MVP_V0.md @@ -0,0 +1,71 @@ +# ORDER 184: R Vessel Multi-Step Traversal MVP v0 + +## Status + +Approved by the user on 2026-07-02 for overnight implementation. + +## Trigger + +ORDER_182 and ORDER_183 prepared the pieces: + +- ORDER_182: R2 first view starts from CoreEgo's entry layer instead of leaf summary 폭탄. +- ORDER_183: after R3 inspection, code can build a direct child candidate surface. + +The remaining gap is that the child candidate surface is not yet fed back into R2. R still performs only one selection step. + +## Goal + +Implement a standalone R Vessel multi-step traversal MVP: + +```text +R1 goal once +-> R2 selects from current candidate surface +-> R3 inspects selected node +-> code builds child candidate surface +-> if R3 recommends deeper and budget remains, feed that surface back to R2 +-> stop on sufficient / budget exhausted / no actionable path / failure +``` + +## CLI Target + +Add a new command: + +```powershell +python main.py vessel-r-traverse "질문" --database neo4j --llm-mode fake --format text +``` + +## Metadata Boundary + +- Code may build candidate surfaces, copy graph IDs, enforce budgets, and record path frames. +- Code must not semantically choose the best child candidate. +- R2 remains responsible for semantic node selection among code-supplied official refs. +- R3 remains responsible for sufficiency/granularity/branch judgment. +- Traversal result frame is code-generated absolute information summarizing recorded frames and statuses. + +## Non-Goals + +- Do not connect R route to node_1 yet. +- Do not inject R result into node_0 memory packets yet. +- Do not connect R result to node_3 final answer yet. +- Do not add R2a/R2b or A/B selection formats. +- Do not mutate Neo4j graph structure. +- Do not weaken R2 official ref validation. + +## Initial Policy + +Use explicit fixed budgets for this MVP: + +- `max_traversal_depth=4` +- `max_node_reads=4` +- `max_branch_switches=0` +- `max_context_tokens=8000` + +These are policy constants, not hidden heuristics. + +## Verification Targets + +- Fake traversal can descend across at least two R2/R3 steps. +- Each step records R2, R3, continuation, and candidate surface frames. +- The traversal path is preserved in a code-generated result frame. +- The command renders step path and final status in text mode. +- Graph fast-test includes the new traversal tests. diff --git a/Administrative_Reform_1/04_Orders/ORDER_185_R_TERMINAL_MATERIAL_BUDGET_AND_EARLY_STOP_GUARD_V0.md b/Administrative_Reform_1/04_Orders/ORDER_185_R_TERMINAL_MATERIAL_BUDGET_AND_EARLY_STOP_GUARD_V0.md new file mode 100644 index 0000000..fea5f67 --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_185_R_TERMINAL_MATERIAL_BUDGET_AND_EARLY_STOP_GUARD_V0.md @@ -0,0 +1,80 @@ +# ORDER 185: R Terminal Material Budget And Early Stop Guard v0 + +## Status + +Approved by the user on 2026-07-03 for immediate implementation. + +## Trigger + +Live `vessel-r-traverse` Qwen test returned: + +```text +step_count: 1 +final_graph_node_id: graph:axis:time +final_sufficiency_status: sufficient +final_continuation_status: stop_sufficient +``` + +The runtime completed structurally, but R3 treated `Time Axis` as sufficient even though no terminal material had been inspected. + +## Goal + +Prevent R Vessel traversal from stopping as sufficient before it has inspected at least one terminal material node. + +For this MVP, terminal material means a graph candidate that directly carries usable material for downstream answer/evidence work: + +- raw source node +- raw capsule node +- active summary node with summary text + +Axis and bundle nodes are traversal scaffolding, not terminal material. + +## Policy + +Initial code policy: + +```text +min_terminal_material_reads = 1 +``` + +If all of the following are true: + +- R3 says `sufficient` or recommends `stop` +- terminal material seen count is still below the policy minimum +- the inspected node has code-supplied child candidates +- traversal budget still allows another node read + +then code must not accept `stop_sufficient`. + +Instead, the continuation frame should stay within the existing schema and use: + +```text +continuation_status=continue_deeper +continuation_reason_code=CODE_STATUS:r_loop_terminal_material_not_seen +next_target_node=R2 +``` + +## Metadata Boundary + +- Code may count whether a selected record is terminal material. +- Code may enforce the minimum terminal material count. +- Code must not decide whether the selected material semantically answers the user. +- R2 still chooses among supplied official refs. +- R3 still judges sufficiency/granularity/branch choice. +- The guard only blocks a structurally premature stop. + +## Non-Goals + +- Do not connect R route to node_1 yet. +- Do not connect R result to node_0 or node_3 yet. +- Do not mutate Neo4j. +- Do not create semantic-axis traversal. +- Do not make R1 truly decide budgets yet. +- Do not add hidden text/keyword heuristics. + +## Verification Targets + +- If R3 marks `graph:axis:time` sufficient while child candidates exist and no terminal material has been seen, traversal continues. +- Traversal can still stop after inspecting one terminal summary. +- Result frame records terminal material counts and early-stop guard trigger count. +- Existing ORDER_184 traversal path tests still pass. diff --git a/Administrative_Reform_1/04_Orders/ORDER_186_R_VESSEL_EXACT_CHILD_RECORD_EXPANSION_V0.md b/Administrative_Reform_1/04_Orders/ORDER_186_R_VESSEL_EXACT_CHILD_RECORD_EXPANSION_V0.md new file mode 100644 index 0000000..5adc94c --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_186_R_VESSEL_EXACT_CHILD_RECORD_EXPANSION_V0.md @@ -0,0 +1,72 @@ +# ORDER 186: R Vessel Exact Child Record Expansion v0 + +## Status + +Approved by the user on 2026-07-03 for immediate implementation. + +## Trigger + +Live `vessel-r-traverse` reached: + +```text +step 1: graph:axis:time +step 2: graph:source_ingest_time_bundle:... +final_continuation_status=stop_no_actionable_path +terminal_material_seen_count=0 +``` + +Audit showed that Neo4j already had the intended structure: + +```text +SourceIngestBundle -> SourceKindBundle -> RawSource -> SummaryGraphNode / TokenBudgetSummaryBundle +``` + +But the R Vessel read packet only used the first bounded entry rows. In the observed packet order, `SourceKindBundle` records appeared after many raw source rows, so the source ingest node had child IDs that were real in Neo4j but absent from the packet candidate map. + +## Goal + +When an already visible entry candidate has exact child graph node IDs, include those child records in the same R Vessel read packet even if they appear outside the base row limit. + +This is a structural record expansion, not semantic ranking. + +## Policy + +- Keep the base entry row limit. +- Add exact child records only when their IDs are already present in the graph record itself or in `parent_graph_node_ids`. +- Follow exact child links for a small bounded depth: + +```text +R_LOOP_VESSEL_EXACT_CHILD_EXPANSION_MAX_DEPTH = 2 +``` + +- Use the same numeric `limit` as the maximum additional expansion budget. +- Record the expansion facts in `RLoopVesselReadPacketFrame`: + - `exact_child_expansion_policy_id` + - `base_entry_candidate_count` + - `exact_child_expanded_entry_count` + - `exact_child_expanded_node_ids` + - `exact_child_expansion_truncated` + +## Metadata Boundary + +- Code may follow exact graph IDs and count what it added. +- Code may not decide which child is semantically more relevant. +- R2 still chooses from official refs. +- R3 still judges sufficiency and whether to go deeper. +- This order does not add keyword matching, vector similarity, or hidden ranking. + +## Non-Goals + +- Do not increase default R route scope by simply dumping all graph records into R2. +- Do not connect R route to normal node_1/node_2/node_3 flow. +- Do not mutate Neo4j. +- Do not create semantic-axis traversal. +- Do not change R1 budget policy. +- Do not add hidden text heuristics. + +## Verification Targets + +- A `SourceKindBundle` outside the base packet limit is included when a visible `SourceIngestBundle` points to it by exact child ID. +- The packet records base count and exact child expansion count. +- Multi-step traversal can descend through the expanded child record to terminal summary material. +- Existing R Vessel read/traverse tests remain green. diff --git a/Administrative_Reform_1/04_Orders/ORDER_187_R_VESSEL_SUMMARY_LAYER_BEFORE_RAW_V0.md b/Administrative_Reform_1/04_Orders/ORDER_187_R_VESSEL_SUMMARY_LAYER_BEFORE_RAW_V0.md new file mode 100644 index 0000000..b54cb26 --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_187_R_VESSEL_SUMMARY_LAYER_BEFORE_RAW_V0.md @@ -0,0 +1,69 @@ +# ORDER 187: R Vessel Summary Layer Before Raw v0 + +## Status + +Approved by the user on 2026-07-03 for immediate implementation. + +## Trigger + +After ORDER 186, live R traversal successfully descended: + +```text +TimeAxis -> SourceIngestBundle -> SourceKindBundle -> RawSource +``` + +This proved the graph path was visible, but it also showed that R could jump from a source-kind bundle directly into raw source material even when active summary-layer material already existed. + +## Goal + +Make R traversal prefer existing summary-layer material before raw source material under a `SourceKindBundle`. + +Target behavior: + +```text +SourceKindBundle +-> token_budget_bundle_summary, if it structurally covers that source kind's raw children +-> source_leaf_summary, if token layer is absent +-> RawSource, only if no active summary-layer child exists +``` + +## Policy + +This is a structural source-bundle policy, not semantic ranking. + +Code may: + +- read the exact raw child IDs under a `SourceKindBundle` +- inspect active summary candidate source IDs +- expose summaries that explicitly include those raw child IDs in their `source_data_ids`, `source_graph_node_ids`, or `target_graph_node_id` +- prefer token-budget bundle summaries over leaf summaries when both are structurally available + +Code must not: + +- judge which summary is semantically best +- infer relevance from text +- use keyword matching or vector similarity +- hide raw source forever + +## Metadata Boundary + +- `SourceKindBundle` and `RawSource` links are absolute graph structure. +- `SummaryGraphNode` text remains LLM-generated relative/mixed information. +- Code only chooses which layer to expose first based on explicit source IDs. +- R2 still chooses from official refs. +- R3 still decides sufficiency and whether to continue. + +## Non-Goals + +- Do not connect R route to the normal node_1/node_2/node_3 live flow. +- Do not mutate Neo4j. +- Do not create semantic-axis traversal. +- Do not summarize new material. +- Do not add hidden heuristics. + +## Verification Targets + +- If a token-budget summary covers raw source children under a source kind, traversal selects the token summary before raw. +- If token layer is absent but source-leaf summary exists, traversal selects the leaf summary before raw. +- Raw remains available as fallback when no summary layer is present. +- Existing R hierarchy and traversal tests remain green. diff --git a/Administrative_Reform_1/04_Orders/ORDER_188_R_VESSEL_RAW_ORIGINAL_READ_CAP_V0.md b/Administrative_Reform_1/04_Orders/ORDER_188_R_VESSEL_RAW_ORIGINAL_READ_CAP_V0.md new file mode 100644 index 0000000..f54824d --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_188_R_VESSEL_RAW_ORIGINAL_READ_CAP_V0.md @@ -0,0 +1,64 @@ +# ORDER 188: R Vessel Raw Original Read Cap v0 + +## Status + +Approved by the user on 2026-07-03 for immediate implementation. + +## Trigger + +After ORDER 187, R traversal now prefers summary layers before raw source material. The next safety boundary is to prevent R traversal from opening too many raw originals in a single traversal. + +User requirement: + +```text +원본은 최대 5회만 열람할 수 있도록 해 +``` + +## Goal + +Limit raw original material inspection during one R Vessel traversal to at most 5. + +For this MVP, raw original material means: + +- `RawSource` +- `RawCapsule` + +Summary nodes do not count as raw original material, even if their summaries are about raw material. + +## Policy + +```text +R_TRAVERSE_MAX_RAW_ORIGINAL_MATERIAL_READS = 5 +``` + +The 5th raw original inspection is allowed. If R3 asks to continue after the cap is reached, code closes traversal with: + +```text +continuation_status=stop_budget_exhausted +continuation_reason_code=CODE_STATUS:r_loop_raw_original_read_cap_reached +next_target_node=return_summary +``` + +## Metadata Boundary + +- Code may count raw original inspections. +- Code may stop traversal when the raw original read cap is reached. +- Code must not semantically judge whether raw original content was useful. +- R2 still chooses among official refs. +- R3 still judges sufficiency and desired next action until the cap blocks further traversal. + +## Non-Goals + +- Do not change summary-layer priority. +- Do not increase traversal defaults. +- Do not connect R route to node_1/node_2/node_3 live flow. +- Do not mutate Neo4j. +- Do not add semantic-axis traversal. +- Do not add keyword or text heuristics. + +## Verification Targets + +- R traversal stops after 5 raw original inspections if it would otherwise continue. +- The 6th raw source is not selected. +- Summary-layer traversal does not increment raw original read count. +- Result frame records raw original count, max cap, and cap reached status. diff --git a/Administrative_Reform_1/04_Orders/ORDER_189_R_TRAVERSE_LIVE_AUDIT_V0.md b/Administrative_Reform_1/04_Orders/ORDER_189_R_TRAVERSE_LIVE_AUDIT_V0.md new file mode 100644 index 0000000..fdbc4fa --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_189_R_TRAVERSE_LIVE_AUDIT_V0.md @@ -0,0 +1,136 @@ +# ORDER 189: R Traverse Live Audit v0 + +## Status + +Approved by the user on 2026-07-03 for immediate audit. + +## Trigger + +ORDER 184-188 made R Vessel traversal able to start from CoreEgo, descend through graph layers, prefer summary layers before raw originals, and cap raw original material inspection at 5. + +The next risk is not missing another large feature. The next risk is believing the R loop is useful before live traversal behavior is checked. + +## Goal + +Audit whether current R Vessel traversal can safely and usefully inspect the Vessel graph in live/manual tests. + +This order is an audit order, not a feature expansion order. + +## Questions To Answer + +1. Does R traversal start from the CoreEgo/time-axis surface instead of seeing every summary at once? +2. Does R traversal descend through graph layers in a readable order? +3. Does R traversal prefer summary material before raw source material? +4. Does R traversal open raw originals only when traversal requires it? +5. Does the raw original material cap stay visible and enforced? +6. Does the final result distinguish: + - summary-only inspection + - raw-original inspection + - partial traversal + - budget/cap stop +7. Does the output provide enough information for the next MVP without pretending to be a full user-facing answer? + +## Expected Traversal Shape + +The expected high-level path is: + +```text +CoreEgo +-> Time Axis +-> Time Bundle or Source Ingest Bundle +-> Source Kind Bundle +-> Token Budget Bundle / Summary Layer +-> Source Leaf Summary +-> RawSource only if needed +``` + +This order does not require every test to reach every layer. It requires the runtime to make the reached layer, stop reason, and material type clear. + +## Live Test Pack + +Run at least one fake-adapter test and, if Qwen is available, at least two Qwen tests. + +### Test 1: Structure Walk + +```powershell +python main.py vessel-r-traverse "송련 Core의 그래프 기억 구조를 CoreEgo에서 시작해서 한 단계씩 내려가며 설명 가능한 만큼만 탐색해줘" --database neo4j --llm-mode fake --format text +``` + +Expected: + +- read packet passes. +- traversal starts from graph entry layer. +- no raw original material is opened unnecessarily. + +### Test 2: Summary Layer Link + +```powershell +python main.py vessel-r-traverse "송련 Core의 그래프 기억 구조에서 source summary와 token layer summary가 어떻게 이어지는지 계층적으로 탐색해줘" --database neo4j --llm-mode qwen --timeout 180 --format text +``` + +Expected: + +- traversal does not jump straight to arbitrary leaf summaries. +- token-budget or summary layer appears before raw originals. +- if traversal ends partial, the stop reason is clear. + +### Test 3: Code/Document Separation + +```powershell +python main.py vessel-r-traverse "송련 Core 그래프 기억에서 코드 파일과 내부 문서가 서로 어떻게 분리되어 저장됐는지 계층적으로 확인해줘" --database neo4j --llm-mode qwen --timeout 180 --format text +``` + +Expected: + +- source kind separation is visible. +- R2/R3 does not claim raw text was read unless raw material count confirms it. + +### Test 4: Raw Cap Stress + +```powershell +python main.py vessel-r-traverse "가능하면 원본까지 내려가되, 원본을 너무 많이 열지 말고 어떤 지점에서 멈추는지 확인해줘" --database neo4j --llm-mode qwen --timeout 180 --format text +``` + +Expected: + +- raw original material count is visible. +- raw original material count never exceeds 5. +- if cap is reached, reason uses `CODE_STATUS:r_loop_raw_original_read_cap_reached`. + +## Audit Report Template + +For each live test, record: + +- command +- status +- step_count +- final_graph_node_id +- final_sufficiency_status +- final_continuation_status +- r_loop_task_status +- terminal_material_seen_count +- raw_original_material_seen_count +- max_raw_original_material_count +- raw_original_read_cap_reached +- whether traversal path is understandable +- whether the result is pass, partial, or fail +- observed bottleneck + +## Non-Goals + +- Do not connect R traversal to normal qwen-chat routing. +- Do not make node_1 choose route=R by default. +- Do not change R1/R2/R3 prompt policy unless the audit proves a narrow defect. +- Do not add semantic-axis traversal. +- Do not add graph write behavior. +- Do not increase raw original cap. +- Do not replace summaries with raw originals. +- Do not add keyword heuristics. + +## Completion Criteria + +- ORDER 189 is documented. +- At least fake traversal is run and recorded. +- If Neo4j/Qwen are available, Qwen live traversal is run and recorded. +- The audit report states the next MVP candidate based on observed behavior. + diff --git a/Administrative_Reform_1/04_Orders/ORDER_190_R_VESSEL_TOKEN_SUMMARY_DEEPER_CHILD_EXPANSION_V0.md b/Administrative_Reform_1/04_Orders/ORDER_190_R_VESSEL_TOKEN_SUMMARY_DEEPER_CHILD_EXPANSION_V0.md new file mode 100644 index 0000000..cfc2720 --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_190_R_VESSEL_TOKEN_SUMMARY_DEEPER_CHILD_EXPANSION_V0.md @@ -0,0 +1,82 @@ +# ORDER 190: R Vessel Token Summary Deeper Child Expansion v0 + +## Status + +Approved by the user on 2026-07-03 for immediate implementation. + +## Trigger + +ORDER 189 live audit showed that R traversal now starts high and reaches token-budget summary material, but Qwen often wants to continue deeper after inspecting a token summary. + +Observed bottleneck: + +```text +CoreEgo -> Time Axis -> Source Ingest Bundle -> Source Kind Bundle -> Token Summary +``` + +At the token summary step, R traversal stopped as partial because the default traversal budget ended and the lower summary children were not reliably available as readable candidate records. + +## Goal + +Let R traversal continue from a token-budget summary into its lower summary children when R3 asks for `deeper`. + +This is still a read-only traversal improvement. It does not connect R traversal to normal qwen-chat answers. + +## Key Rules + +- Token summary children must be copied from code-visible source IDs. +- Code may follow `graph:summary:*` source IDs and copy matching active summary records. +- Code must not semantically decide which child is relevant. +- R2 still chooses among official refs. +- R3 still judges whether the selected node is sufficient. +- Raw originals remain capped by ORDER 188. + +## Implementation Requirements + +1. Expand R read packet summary candidates. + - Keep the original balanced summary candidate window. + - Add active summary records referenced by selected summary candidates' `source_graph_node_ids` or `source_data_ids`. + - Limit summary child expansion by explicit policy. + +2. Prefer higher summary layers first. + - Token-budget summaries should be visible before source leaf summaries in the base summary window. + - Among summaries, higher `summary_depth` is a higher-level summary and should be considered first. + +3. Token summary child surface. + - When the selected node is a `token_budget_bundle_summary`, its next child candidate surface should prefer `graph:summary:*` children. + - Do not expose unresolved token bundle IDs as first-choice material when readable summary children exist. + - If a token summary has no summary children, do not fall through to unreadable bundle IDs as if they were useful material. + +4. Traversal budget. + - Increase default multi-step R traversal depth/node-read budget from 4 to 6. + - This allows: + +```text +Time Axis +-> Source Ingest Bundle +-> Source Kind Bundle +-> Token Summary +-> Lower Token Summary / Source Leaf Summary +``` + +5. Runtime visibility. + - R read packet output should show base summary candidate count and summary child expansion count. + +## Non-Goals + +- Do not route normal user turns to R by default. +- Do not connect R result to node_3 yet. +- Do not add semantic-axis traversal. +- Do not add branch comparison logic beyond exposing lower child surfaces. +- Do not increase raw original read cap. +- Do not write new graph nodes. +- Do not add keyword heuristics. + +## Verification Targets + +- A token summary that references a lower summary record has that child summary copied into the R read packet. +- R traversal can continue from parent token summary to lower token summary. +- Token summary traversal does not count as raw original material. +- Existing R traversal tests still pass. +- `fast-test --profile graph` still passes. + diff --git a/Administrative_Reform_1/04_Orders/ORDER_191_R2_SOURCE_INGEST_BRANCH_SELECTION_STABILITY_V0.md b/Administrative_Reform_1/04_Orders/ORDER_191_R2_SOURCE_INGEST_BRANCH_SELECTION_STABILITY_V0.md new file mode 100644 index 0000000..2ab820a --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_191_R2_SOURCE_INGEST_BRANCH_SELECTION_STABILITY_V0.md @@ -0,0 +1,94 @@ +# ORDER 191: R2 Source Ingest Branch Selection Stability v0 + +## Status + +Approved by the user on 2026-07-03 for immediate implementation. + +## Trigger + +ORDER 190 opened lower summary traversal under token-budget summary nodes. Live probing then showed a different bottleneck: + +```text +R2 sometimes selects a plain TimeBundle branch when the user is asking about source/token summary structure. +``` + +This is not a raw-original overread issue. It is a branch-signage issue at the graph traversal surface. + +## Goal + +Make the R2 candidate surface distinguish structural branch roles so that source/code/document graph questions can select the source-ingest path more reliably. + +This order must not make code choose semantic relevance. Code only labels graph branches by explicit structural node kind. + +## Structural Branch Roles + +R candidate surfaces may expose: + +```text +graph_axis +source_material_ingest +source_material_leaf +summary_memory +conversation_time_memory +unknown_graph_branch +``` + +Meaning: + +- `graph_axis`: top-level graph axis entry. +- `source_material_ingest`: source ingest / source kind branch for code, internal documents, and source material. +- `source_material_leaf`: raw source or low-level source material. +- `summary_memory`: existing summary graph node material. +- `conversation_time_memory`: conversation/time bundle branch. +- `unknown_graph_branch`: structurally unclassified graph branch. + +## Implementation Requirements + +1. Add `branch_role` to candidate layer surface records. +2. Add `branch_role` to R2 visible surface records. +3. Add `branch_role` to R2 visible candidate records. +4. Sort child surfaces structurally: + - source ingest/source kind branch before conversation time bundle branch + - summary layers before generic/unknown lower material + - conversation time memory after source material branches +5. Update R2 prompt: + - `branch_role` is a code-supplied structural label. + - R2 still performs semantic choice. + - If the goal is about source summaries, token summaries, source kinds, code files, internal documents, or graph ingest structure, R2 should prefer a compatible source/material branch when present. + - If the goal is about past conversation turns or time memory, R2 should prefer conversation/time memory when present. + +## Metadata Boundary + +Code may: + +- classify graph node kind into a structural `branch_role` +- sort surfaces by structural role +- expose branch roles to R2 + +Code must not: + +- decide that a branch is semantically relevant to the user's question +- force R2 to select source ingest +- use keyword fallback or hidden heuristics +- hide available time-bundle branches + +R2 choice remains LLM-generated mixed/semantic judgment. + +## Non-Goals + +- Do not connect R traversal to normal qwen-chat route. +- Do not connect R result to node_3. +- Do not add branch comparison traversal. +- Do not add semantic-axis traversal. +- Do not increase raw original cap. +- Do not write graph nodes. +- Do not add keyword heuristics. + +## Verification Targets + +- TimeAxis child surface exposes both `source_material_ingest` and `conversation_time_memory`. +- R2 payload carries branch roles on both surface records and candidate records. +- The structurally ordered surface presents source ingest before time bundle. +- Existing R traversal tests still pass. +- Live Qwen source/token-summary traversal should be less likely to enter TimeBundle accidentally. + diff --git a/Administrative_Reform_1/04_Orders/ORDER_192_R1_USER_QUESTION_ANCHOR_COPY_V0.md b/Administrative_Reform_1/04_Orders/ORDER_192_R1_USER_QUESTION_ANCHOR_COPY_V0.md new file mode 100644 index 0000000..44b03f9 --- /dev/null +++ b/Administrative_Reform_1/04_Orders/ORDER_192_R1_USER_QUESTION_ANCHOR_COPY_V0.md @@ -0,0 +1,177 @@ +# ORDER 192: R1 User Question Anchor Copy v0 + +## Status + +Prepared on 2026-07-03 as the next R traversal stability order. + +Implementation should proceed after confirming the live test failure pattern remains: + +```text +failure_stage: R1 +failure_type: schema_failed +failure_reason: R1 graph_search_goal must preserve a user question anchor +``` + +## Trigger + +After ORDER 191, R2 branch signage became more stable, but a live Vessel R traversal failed before R2: + +```text +status: R_LOOP_VESSEL_TRAVERSE_NOT_PASSED +failure_stage: R1 +failure_type: schema_failed +failure_reason: R1 graph_search_goal must preserve a user question anchor +``` + +This means the graph traversal did not fail because R2 chose the wrong branch. It failed because R1's generated goal did not satisfy the current user-question-anchor validator. + +## Problem + +The current R1 validator asks whether `graph_search_goal` preserves a user-question anchor. + +That guard is valuable because R1 must not drift away from the user's request. However, the current implementation can fail on Korean questions because the validator extracts simple normalized anchor tokens from the user question and then checks whether the generated English/Korean goal contains one of those tokens. + +This creates a fragile boundary: + +- The user question may be Korean. +- R1 may paraphrase the goal naturally in Korean. +- The validator may expect an English/ASCII token such as `source` or `token`. +- R1 can be semantically aligned but still fail schema validation. + +## Goal + +Replace fragile goal-text anchor matching with explicit code-supplied anchor copying. + +R1 should receive an official user-question anchor from code and copy it into its output frame. The validator should then check the copied anchor exactly. + +This keeps the anti-drift guard while avoiding Korean/English token mismatch. + +## Implementation Requirements + +1. Add a code-generated user question anchor to the R1 input payload. + +Suggested payload fields: + +```json +{ + "user_question_anchor": { + "anchor_id": "r1_user_question_anchor:", + "source_field": "user_question", + "copy_required": true + } +} +``` + +2. Extend the R1 output contract. + +Suggested output field: + +```json +{ + "user_question_anchor_id": "r1_user_question_anchor:" +} +``` + +3. Update `R1GraphGoalFrame` if needed. + +The frame should preserve the copied anchor as absolute linkage: + +```text +user_question_anchor_id +``` + +This field is not a semantic judgment. It is a code-supplied ID copied by the LLM and validated by code. + +4. Update R1 prompt. + +R1 must be told: + +- Copy `user_question_anchor.anchor_id` exactly into `user_question_anchor_id`. +- Do not translate it. +- Do not invent it. +- Keep `graph_search_goal` human-readable. +- The goal may be Korean. + +5. Update R1 validator. + +Preferred rule: + +```text +payload.user_question_anchor_id == input.user_question_anchor.anchor_id +``` + +The existing `_shares_user_question_anchor(...)` text-token guard should be removed from the R1 live path or demoted to a legacy/internal helper only if tests still need it. + +6. Preserve metadata boundary. + +Code may: + +- generate the anchor ID from the user question +- require exact anchor copy +- record validation failure if the anchor is missing or wrong + +Code must not: + +- judge semantic relevance of the R1 goal +- parse Korean meaning with keyword heuristics +- silently rewrite R1's goal +- create a code fallback graph goal pretending to be LLM judgment + +## Non-Goals + +- Do not change R2 selection policy. +- Do not change R3 sufficiency policy. +- Do not add keyword heuristics. +- Do not connect R traversal to normal qwen-chat route. +- Do not write new graph memory nodes. +- Do not increase raw original read cap. +- Do not change Vessel read packet candidate budgets. + +## Test Plan + +Run: + +```powershell +python -m compileall songryeon_core main.py +python -m pytest -q tests/test_order_192_r1_user_question_anchor_copy.py +python main.py fast-test --profile graph +``` + +Add tests: + +1. R1 input payload includes `user_question_anchor.anchor_id`. +2. R1 output with matching `user_question_anchor_id` passes even when `graph_search_goal` is Korean and does not copy English/ASCII words from the user question. +3. R1 output with missing anchor fails schema validation. +4. R1 output with invented/wrong anchor fails schema validation. +5. Existing ORDER 177 R1 candidate text blindness remains true. +6. Existing ORDER 191 R2 branch role tests remain true. + +Manual live test after implementation: + +```powershell +Set-ExecutionPolicy -Scope Process -ExecutionPolicy Bypass +. .\.env.vessel.local.ps1 +python main.py vessel-r-traverse "송련 Core의 그래프 기억 구조에서 소스 요약과 토큰 묶음 요약이 어떻게 이어지는지 계층적으로 탐색해줘. 과거 대화 기억 가지가 아니라 코드/문서 소스 가지를 우선 보고, 시간축에서 시작해서 어떤 묶음을 거쳐 내려가는지 말해줘." --database neo4j --llm-mode qwen --timeout 180 --format text +``` + +Expected: + +- The run should no longer fail at `failure_stage=R1` due to anchor preservation. +- If it fails later, the later failure should be recorded separately. + +## Expected Result + +R1 keeps a hard source link to the user's question without depending on fragile Korean/English token matching. + +In elementary terms: + +```text +Before: +R1 had to repeat a magic word from the user's sentence. + +After: +Code gives R1 a numbered ticket. +R1 copies the ticket. +Code checks the ticket. +``` + diff --git a/Administrative_Reform_1/04_Orders/README.md b/Administrative_Reform_1/04_Orders/README.md index 0c87f86..02162f7 100644 --- a/Administrative_Reform_1/04_Orders/README.md +++ b/Administrative_Reform_1/04_Orders/README.md @@ -2,7 +2,7 @@ 발주서는 개발 지도에서 내려온 실무 계획서다. -현재 정식 발주서는 `ORDER_001`부터 `ORDER_117`까지 있다. +현재 정식 발주서는 `ORDER_001`부터 `ORDER_192`까지 있다. `ORDER_066`부터 `ORDER_075`까지는 메타정보 관리법을 실제 런타임과 LLM 노드 배선에 적용하기 위한 복구 로드맵이다. @@ -74,6 +74,88 @@ `ORDER_117`은 compileall / pytest / smoke-test를 개발 루틴과 CI에 고정하는 발주서다. +`ORDER_118`은 node_2가 node_3에게 `absolute_first`, `relative_allowed`, `mixed_or_uncertain` 3종 답변 근거 모드를 선택 이유와 함께 전달하도록 하는 발주서다. + +`ORDER_119`는 `structure_failed` fallback이 검색하지 않은 문서를 찾은 척하지 않게 하고, node_2 answer-basis selector 실패 원인을 runtime에 드러내는 정직성/진단 발주서다. + +`ORDER_120`은 `ToolUseBudgetFrame.query_count`가 `max_query_attempts`를 초과해 `structure_failed`가 발생하는 예산 count 불일치 원인을 찾고 진단 정보를 남기는 발주서다. + +`ORDER_121`은 node_2 answer-basis 근거 ID 허용 목록을 정렬하고, L3 실패/예산소진 신호가 node_3 답변 태도에 드러나게 하는 발주서다. + +`ORDER_122`는 L revision 흐름에서 unread search candidate를 `read_doc`으로 읽을 수 있게 하여, 검색 후보를 찾고도 원문 읽기를 적게 하는 병목을 줄이는 발주서다. + +`ORDER_123`은 실제 `read_doc` 도구 원문 읽기 수와 node_3 공급 문서 context 수를 구조적으로 분리해, 최종 답변이 두 count를 섞지 않게 하는 발주서다. + +`ORDER_124`는 L 이후 node_0이 검색 후보 / 실제 read_doc / node_3 공급 context / unread 후보를 문서별 절대정보 장부로 정리해 node_2와 node_3에 공급하는 발주서다. + +`ORDER_125`는 L3가 실제 읽은 문서별 요약 frame을 만들고, 담백 문서 요약(relative)과 상황 맞춤 요약(mixed)을 구분해 node_3에 전달하는 발주서다. + +`ORDER_126`은 terminal runtime view가 여러 `read_doc` / `read_artifact` document extract tool result를 최신 1개로 접지 않고 모두 표시하게 하는 발주서다. + +`ORDER_127`은 L revision에서 추가로 실행된 document extract record가 node_0 material packet과 node_3 actual read count에 누락되지 않게 병합하는 발주서다. + +`ORDER_128`은 node_3 actual read document count가 같은 파일명 문서를 하나로 합쳐 절대 count를 틀리지 않도록 `doc_id` identity 기준으로 정렬하는 발주서다. + +`ORDER_129`는 L3 문서별 요약을 열기 전에 node_0 material packet과 node_3 input brief의 search candidate count가 서로 다른 범위/identity 기준으로 세어지는지 감사하는 발주서다. + +`ORDER_130`은 node_3가 실제 read_doc, node_3 context 공급, search candidate, excluded/unread candidate 역할을 섞어 말하지 않게 하고 node_4가 명시 역할 claim 충돌을 막는 발주서다. + +`ORDER_131`은 `search_candidate_count`를 최종 검색 후보와 L3 preserved frame 누적 검색 후보로 분리해, 같은 숫자 이름이 서로 다른 범위를 가리키지 않게 하는 발주서다. + +`ORDER_132`는 node_2의 `answer_basis_mode`를 바탕으로 node_3가 원문 문서 context와 L3 문서별 요약을 어떤 태도로 사용할지 명시적인 material delivery policy로 전달하는 발주서다. + +`ORDER_133`은 송련이 내부 문서뿐 아니라 실제 source/config 파일 구조도 읽기 전용으로 검사할 수 있게 하는 codebase inspection MVP 발주서다. + +`ORDER_134`는 L2가 도구를 바로 고르기 전에 L루프가 먼저 tool scope와 도구군별 예산 분배를 명시 frame으로 확정하게 하는 발주서다. + +`ORDER_135`는 `read_code_file` 성공을 `read_doc`과 분리해 L3, return summary, node_3 grounding에서 source-code evidence로 인정하게 하는 발주서다. + +`ORDER_136`은 새 기능 확장 전에 현재 capability baseline과 live qwen 테스트 묶음을 문서화해, 다음 MVP가 기존 가능 범위를 잃지 않게 하는 발주서다. + +`ORDER_137`은 `read_code_file`로 읽은 source-code 원문에서 code가 문법적 outline을 만들고, node_3가 공개 함수/상수 coverage를 빠뜨리지 않게 하는 발주서다. + +`ORDER_138`은 ORDER_133 이후 빠르게 들어온 code inspection 계열과 심야정부 MVP를 새 기능 추가 없이 통합 기준선으로 묶고, dirty worktree/문서-현실 불일치를 정리하는 발주서다. + +`ORDER_139`는 `TurnStateCapsule`을 graph memory raw node로 옮기고 CoreEgo time axis와 RLoopGraphGuidePacket을 code-generated absolute 정보로 만드는 발주서다. + +`ORDER_140`은 R route를 열기 전에 R1/R2/R3/budget/continuation/return summary frame과 state-machine helper만 감사하는 발주서다. + +`ORDER_141`은 code-generated RLoopGraphGuidePacket 위에 CoreEgo guide worker LLM traversal hint를 별도 mixed frame으로 기록하는 발주서다. + +`ORDER_142`는 외부 그래프 DB 연결 전에 `songryeon-neo4j-vessel` 이름과 graph memory store adapter boundary를 예약하는 발주서다. + +`ORDER_143`은 node_0이 R_LOOP에게 graph guide 좌표를 넘기는 `RLoopMemoryHandoffPacketFrame`을 만들고 runtime에 status/count만 표시하게 하는 발주서다. + +`ORDER_144`는 qwen/live route=R을 열지 않고 dry-run opt-in fixture에서만 R1/R2/R3/continuation/return summary frame 골격을 실행해 보는 발주서다. + +`ORDER_145`는 live route=R을 열기 전에 현재 R skeleton이 dry-run opt-in 상태로 닫혀 있음을 문서와 테스트 기준선으로 감사하는 발주서다. + +`ORDER_146`은 기본 route set은 L/2로 유지하되, 명시 실험 플래그가 있을 때만 node_1 LLM이 `route=R`을 고르고 `R:experimental:*` skeleton을 실행할 수 있게 하는 발주서다. + +`ORDER_147`부터 `ORDER_161`까지는 R 결과 전달, graph traversal, activity ledger, graph source ingest, export packet, Vessel write plan boundary, local Neo4j first write, Vessel display vocabulary를 순서대로 잠그는 그래프 메모리/심야정부 전초 작업이다. + +`ORDER_162`부터 `ORDER_165`까지는 Vessel readback/inspect와 동적 원본 변경 시 source version lineage, observation ledger, summary invalidation ledger를 남기는 외부 그래프 DB 안전장치 작업이다. + +`ORDER_166`은 원본 `TimeBundle` graph node를 오염시키지 않고, LLM이 만든 심야정부 요약을 별도 `SummaryGraphNode`와 `SUMMARY_OF` edge로 붙이는 첫 summary node MVP 발주서다. + +`ORDER_167`은 ORDER_166의 심야정부 TimeBundle 요약 worker 이름을 `night_summarize_time_bundle` 계열로 정리하고, 기존 이름은 호환 alias로 남기는 명명 정리 발주서다. + +`ORDER_168`은 새로 관측됐거나 내용이 바뀐 코드/문서 `raw_source` leaf를 원문 text snapshot과 1:1로 연결해 LLM 요약 `SummaryGraphNode`를 붙이는 발주서다. + +`ORDER_169`는 ORDER_168의 source leaf 요약 엔진을 터미널에서 한 번에 실행하는 `night-summarize-changed-sources` CLI로 묶는 발주서다. + +`ORDER_170`은 `night-summarize-changed-sources --one-at-a-time` 모드로 변경 source leaf 요약 대상을 queue로 고정하고 한 실행에 하나씩만 요약해 재개 가능하게 만드는 발주서다. + +`ORDER_171`은 source leaf summary node들을 날짜가 아니라 예산 단위로 묶어, 한 실행에 하나의 token-budget summary bundle을 만들고 상위 mixed summary node를 붙이는 발주서다. + +`ORDER_172`는 token-budget summary layer를 사람이 반복 실행하지 않아도 되게, target context budget 이하가 될 때까지 `max_steps`와 `max_layer_depth` 안에서 자동으로 계층 요약을 진행하는 발주서다. + +`ORDER_173`은 ORDER_172 자동 reducer를 긴 실행으로 쓸 수 있게 progress JSONL, runtime limit, failure stop, Vessel write mode를 더한 checkpointed long runner 발주서다. + +`ORDER_174`는 Neo4j Vessel inspect가 summary/layer 노드를 read-only로 보여주게 하는 발주서다. 기본 CoreEgo 시간축 경로뿐 아니라 summary 수, depth, data_kind, sample preview를 확인한다. + +`ORDER_175`부터 `ORDER_192`까지는 Vessel-backed R read packet, R1/R2/R3 one-step traversal, 계층 surface, multi-step traversal, summary-before-raw 정책, raw original cap, R traverse live audit, token summary 하위 summary expansion, R2 branch role surface 안정화, R1 user question anchor copy로 이어지는 R루프 실전성 검증 작업이다. + ## 임시 발주서 - [TMP Auto Hunt Orders 2026-06-21](TMP_Auto_Hunt_2026_06_21/README.md): 지금까지의 철학 정리본을 바탕으로 만든 자동 사냥용 후보 발주서 묶음. @@ -172,3 +254,78 @@ - [ORDER 115: Schema Module Split With Compatibility Layer v0](ORDER_115_SCHEMA_MODULE_SPLIT_COMPAT_LAYER_V0.md) - [ORDER 116: Smoke Test Decomposition To Pytest v0](ORDER_116_SMOKE_TEST_DECOMPOSITION_TO_PYTEST_V0.md) - [ORDER 117: CI And Development Routine Lock v0](ORDER_117_CI_AND_DEVELOPMENT_ROUTINE_LOCK_V0.md) +- [ORDER 118: Node2 Answer Basis Mode Frame v0](ORDER_118_NODE2_ANSWER_BASIS_MODE_FRAME_V0.md) +- [ORDER 119: Structure Failed Honesty And Answer Basis Failure Diagnostics v0](ORDER_119_STRUCTURE_FAILED_HONESTY_AND_ANSWER_BASIS_FAILURE_DIAGNOSTICS_V0.md) +- [ORDER 120: Tool Use Budget Query Count Consistency Diagnostic v0](ORDER_120_TOOL_USE_BUDGET_QUERY_COUNT_CONSISTENCY_DIAGNOSTIC_V0.md) +- [ORDER 121: Node2 Evidence Source Alignment And L3 Failure Attitude v0](ORDER_121_NODE2_EVIDENCE_SOURCE_ALIGNMENT_AND_L3_FAILURE_ATTITUDE_V0.md) +- [ORDER 122: L Revision Unread Candidate Read Path v0](ORDER_122_L_REVISION_UNREAD_CANDIDATE_READ_PATH_V0.md) +- [ORDER 123: Actual Read Doc Vs Context Pack Count Boundary v0](ORDER_123_ACTUAL_READ_DOC_VS_CONTEXT_PACK_COUNT_BOUNDARY_V0.md) +- [ORDER 124: Node0 Post-L Document Material Packet v0](ORDER_124_NODE0_POST_L_DOCUMENT_MATERIAL_PACKET_V0.md) +- [ORDER 125: L3 Per-Document Summary Frame v0](ORDER_125_L3_PER_DOCUMENT_SUMMARY_FRAME_DESIGN_V0.md) +- [ORDER 126: Runtime All Document Extract Display v0](ORDER_126_RUNTIME_ALL_DOCUMENT_EXTRACT_DISPLAY_V0.md) +- [ORDER 127: Revision Document Extract Count Alignment v0](ORDER_127_REVISION_DOCUMENT_EXTRACT_COUNT_ALIGNMENT_V0.md) +- [ORDER 128: Node3 Actual Read Doc Identity Key v0](ORDER_128_NODE3_ACTUAL_READ_DOC_IDENTITY_KEY_V0.md) +- [ORDER 129: Search Candidate Count Basis Audit v0](ORDER_129_SEARCH_CANDIDATE_COUNT_BASIS_AUDIT_V0.md) +- [ORDER 130: Document Evidence Role Claim Guard v0](ORDER_130_DOCUMENT_EVIDENCE_ROLE_CLAIM_GUARD_V0.md) +- [ORDER 131: Search Candidate Scope Split v0](ORDER_131_SEARCH_CANDIDATE_SCOPE_SPLIT_V0.md) +- [ORDER 132: Node2 Answer Basis Material Delivery Policy v0](ORDER_132_NODE2_ANSWER_BASIS_MATERIAL_DELIVERY_POLICY_V0.md) +- [ORDER 133: Codebase Readonly Inspection MVP v0](ORDER_133_CODEBASE_READONLY_INSPECTION_MVP_V0.md) +- [ORDER 134: L Tool Scope And Budget Partition v0](ORDER_134_L_TOOL_SCOPE_AND_BUDGET_PARTITION_V0.md) +- [ORDER 135: Code Evidence Accounting And L3 Success Boundary v0](ORDER_135_CODE_EVIDENCE_ACCOUNTING_AND_L3_SUCCESS_BOUNDARY_V0.md) +- [ORDER 136: Current Capability Baseline And Live Test Pack v0](ORDER_136_CURRENT_CAPABILITY_BASELINE_AND_LIVE_TEST_PACK_V0.md) +- [ORDER 137: Source Code Context Summary Coverage Guard v0](ORDER_137_SOURCE_CODE_CONTEXT_SUMMARY_COVERAGE_GUARD_V0.md) +- [ORDER 138: Integration Baseline And Dirty Worktree Reconciliation v0](ORDER_138_INTEGRATION_BASELINE_AND_DIRTY_WORKTREE_RECONCILIATION_V0.md) +- [ORDER 139: Graph Memory Foundation And RLoop Guide Packet v0](ORDER_139_GRAPH_MEMORY_FOUNDATION_AND_RLOOP_GUIDE_PACKET_V0.md) +- [ORDER 140: R Loop Frame-Only State Machine Audit v0](ORDER_140_R_LOOP_FRAME_ONLY_STATE_MACHINE_AUDIT_V0_CANDIDATE.md) +- [ORDER 141: CoreEgo Guide Worker LLM Hints v0](ORDER_141_CORE_EGO_GUIDE_WORKER_LLM_HINTS_V0_CANDIDATE.md) +- [ORDER 142: External Graph DB Adapter Boundary v0](ORDER_142_EXTERNAL_GRAPH_DB_ADAPTER_BOUNDARY_V0_CANDIDATE.md) +- [ORDER 143: R Loop Node0 Memory Packet Handoff v0](ORDER_143_R_LOOP_NODE0_MEMORY_PACKET_HANDOFF_V0_CANDIDATE.md) +- [ORDER 144: R Route Dry-Run Only v0](ORDER_144_R_ROUTE_DRY_RUN_ONLY_V0_CANDIDATE.md) +- [ORDER 145: R Loop Pre-Live Route Audit Baseline v0](ORDER_145_R_LOOP_PRE_LIVE_ROUTE_AUDIT_BASELINE_V0.md) +- [ORDER 146: R Route Experimental Gate And Allowed Route Set v0](ORDER_146_R_ROUTE_EXPERIMENTAL_GATE_AND_ALLOWED_ROUTE_SET_V0.md) +- [ORDER 147: R Result To Node3 Brief v0](ORDER_147_R_RESULT_TO_NODE3_BRIEF_V0.md) +- [ORDER 148: Graph NEXT And Turn Access Ledger v0](ORDER_148_GRAPH_NEXT_AND_TURN_ACCESS_LEDGER_V0.md) +- [ORDER 149: R Graph Traversal Candidate Surface v0](ORDER_149_R_GRAPH_TRAVERSAL_CANDIDATE_SURFACE_V0.md) +- [ORDER 150: R Loop Multi-Step Traversal Dry-Run v0](ORDER_150_R_LOOP_MULTI_STEP_TRAVERSAL_DRY_RUN_V0.md) +- [ORDER 151: L Loop Activity Ledger For Graph Backup v0](ORDER_151_L_LOOP_ACTIVITY_LEDGER_FOR_GRAPH_BACKUP_V0.md) +- [ORDER 152: Raw Capsule To Activity Ledger Graph Link v0](ORDER_152_RAW_CAPSULE_TO_ACTIVITY_LEDGER_GRAPH_LINK_V0.md) +- [ORDER 153: Graph Memory Export Integrity Audit And R Experimental Source Recording v0](ORDER_153_GRAPH_MEMORY_EXPORT_INTEGRITY_AUDIT_AND_R_EXPERIMENTAL_SOURCE_RECORDING_V0.md) +- [ORDER 154: Fast Test Gate v0](ORDER_154_FAST_TEST_GATE_V0.md) +- [ORDER 155: Graph Source Kind Separated Ingest Foundation v0](ORDER_155_GRAPH_SOURCE_KIND_SEPARATED_INGEST_FOUNDATION_V0.md) +- [ORDER 156: Graph Source Observation Time And CoreEgo Link v0](ORDER_156_GRAPH_SOURCE_OBSERVATION_TIME_AND_CORE_EGO_LINK_V0.md) +- [ORDER 157: SongRyeon Core Source Ingest Manifest v0](ORDER_157_SONGRYEON_CORE_SOURCE_INGEST_MANIFEST_V0.md) +- [ORDER 158: Graph Memory Export Packet v0](ORDER_158_GRAPH_MEMORY_EXPORT_PACKET_V0.md) +- [ORDER 159: Vessel Adapter Write Plan Boundary v0](ORDER_159_VESSEL_ADAPTER_WRITE_PLAN_BOUNDARY_V0.md) +- [ORDER 160: Local Vessel Neo4j First Write v0](ORDER_160_LOCAL_VESSEL_NEO4J_FIRST_WRITE_V0.md) +- [ORDER 161: Vessel Display Vocabulary v0](ORDER_161_VESSEL_DISPLAY_VOCABULARY_V0.md) +- [ORDER 162: Vessel Readback Verification v0](ORDER_162_VESSEL_READBACK_VERIFICATION_V0.md) +- [ORDER 163: Vessel Inspect Manual Walk v0](ORDER_163_VESSEL_INSPECT_MANUAL_WALK_V0.md) +- [ORDER 164: Dynamic Source Version Lineage And Summary Invalidation Ledger v0](ORDER_164_DYNAMIC_SOURCE_VERSION_LINEAGE_AND_SUMMARY_INVALIDATION_LEDGER_V0.md) +- [ORDER 165: Same-Content Reobserve Observation Ledger v0](ORDER_165_SAME_CONTENT_REOBSERVE_OBSERVATION_LEDGER_V0.md) +- [ORDER 166: Night TimeBundle Summary Node v0](ORDER_166_NIGHT_TIME_BUNDLE_SUMMARY_NODE_V0.md) +- [ORDER 167: Night Summary Naming Clarity v0](ORDER_167_NIGHT_SUMMARY_NAMING_CLARITY_V0.md) +- [ORDER 168: Night Summarize Changed Source Leaves v0](ORDER_168_NIGHT_SUMMARIZE_CHANGED_SOURCE_LEAVES_V0.md) +- [ORDER 169: Night Changed Source Summary CLI v0](ORDER_169_NIGHT_CHANGED_SOURCE_SUMMARY_CLI_V0.md) +- [ORDER 170: Night Changed Source One-At-A-Time Runner v0](ORDER_170_NIGHT_CHANGED_SOURCE_ONE_AT_A_TIME_RUNNER_V0.md) +- [ORDER 171: Night Token Budget Layer Summary v0](ORDER_171_NIGHT_TOKEN_BUDGET_LAYER_SUMMARY_V0.md) +- [ORDER 172: Night Token Layer Auto Reduce Until Context Budget v0](ORDER_172_NIGHT_TOKEN_LAYER_AUTO_REDUCE_UNTIL_CONTEXT_BUDGET_V0.md) +- [ORDER 173: Night Checkpointed Long Runner v0](ORDER_173_NIGHT_CHECKPOINTED_LONG_RUNNER_V0.md) +- [ORDER 174: Vessel Inspect Summary Layer View v0](ORDER_174_VESSEL_INSPECT_SUMMARY_LAYER_VIEW_V0.md) +- [ORDER 175: Vessel-Backed R Graph Read Packet v0](ORDER_175_VESSEL_BACKED_R_GRAPH_READ_PACKET_V0.md) +- [ORDER 176: Vessel R One-Step Traversal v0](ORDER_176_VESSEL_R_ONE_STEP_TRAVERSAL_V0.md) +- [ORDER 177: R1 Candidate Text Blindness v0](ORDER_177_R1_CANDIDATE_TEXT_BLINDNESS_V0.md) +- [ORDER 178: R Vessel Candidate Layer Surface v0](ORDER_178_R_VESSEL_CANDIDATE_LAYER_SURFACE_V0.md) +- [ORDER 179: R2 Vessel Selection ID Disambiguation v0](ORDER_179_R2_VESSEL_SELECTION_ID_DISAMBIGUATION_V0.md) +- [ORDER 180: R2 Prompt Example ID Removal v0](ORDER_180_R2_PROMPT_EXAMPLE_ID_REMOVAL_V0.md) +- [ORDER 181: R2 Official Selection Ref Map v0](ORDER_181_R2_OFFICIAL_SELECTION_REF_MAP_V0.md) +- [ORDER 182: R CoreEgo Start Selection Surface v0](ORDER_182_R_CORE_EGO_START_SELECTION_SURFACE_V0.md) +- [ORDER 183: R Vessel Hierarchical Child Candidate Surface v0](ORDER_183_R_VESSEL_HIERARCHICAL_CHILD_CANDIDATE_SURFACE_V0.md) +- [ORDER 184: R Vessel Multi-Step Traversal MVP v0](ORDER_184_R_VESSEL_MULTI_STEP_TRAVERSAL_MVP_V0.md) +- [ORDER 185: R Terminal Material Budget And Early Stop Guard v0](ORDER_185_R_TERMINAL_MATERIAL_BUDGET_AND_EARLY_STOP_GUARD_V0.md) +- [ORDER 186: R Vessel Exact Child Record Expansion v0](ORDER_186_R_VESSEL_EXACT_CHILD_RECORD_EXPANSION_V0.md) +- [ORDER 187: R Vessel Summary Layer Before Raw v0](ORDER_187_R_VESSEL_SUMMARY_LAYER_BEFORE_RAW_V0.md) +- [ORDER 188: R Vessel Raw Original Read Cap v0](ORDER_188_R_VESSEL_RAW_ORIGINAL_READ_CAP_V0.md) +- [ORDER 189: R Traverse Live Audit v0](ORDER_189_R_TRAVERSE_LIVE_AUDIT_V0.md) +- [ORDER 190: R Vessel Token Summary Deeper Child Expansion v0](ORDER_190_R_VESSEL_TOKEN_SUMMARY_DEEPER_CHILD_EXPANSION_V0.md) +- [ORDER 191: R2 Source Ingest Branch Selection Stability v0](ORDER_191_R2_SOURCE_INGEST_BRANCH_SELECTION_STABILITY_V0.md) +- [ORDER 192: R1 User Question Anchor Copy v0](ORDER_192_R1_USER_QUESTION_ANCHOR_COPY_V0.md) diff --git a/Administrative_Reform_1/05_Execution_Records/README.md b/Administrative_Reform_1/05_Execution_Records/README.md index 86b3cb2..a585fed 100644 --- a/Administrative_Reform_1/05_Execution_Records/README.md +++ b/Administrative_Reform_1/05_Execution_Records/README.md @@ -100,3 +100,87 @@ - [order_115_schema_module_split_compat_layer_2026_06_27_001.md](order_115_schema_module_split_compat_layer_2026_06_27_001.md) - [order_116_smoke_test_decomposition_to_pytest_2026_06_27_001.md](order_116_smoke_test_decomposition_to_pytest_2026_06_27_001.md) - [order_117_ci_and_development_routine_lock_2026_06_27_001.md](order_117_ci_and_development_routine_lock_2026_06_27_001.md) +- [order_118_node2_answer_basis_mode_frame_implementation_2026_06_27_001.md](order_118_node2_answer_basis_mode_frame_implementation_2026_06_27_001.md) +- [order_121_node2_evidence_source_alignment_and_l3_failure_attitude_2026_06_28_001.md](order_121_node2_evidence_source_alignment_and_l3_failure_attitude_2026_06_28_001.md) +- [order_122_l_revision_unread_candidate_read_path_2026_06_28_001.md](order_122_l_revision_unread_candidate_read_path_2026_06_28_001.md) +- [order_123_actual_read_doc_vs_context_pack_count_boundary_2026_06_28_001.md](order_123_actual_read_doc_vs_context_pack_count_boundary_2026_06_28_001.md) +- [order_124_node0_post_l_document_material_packet_order_draft_2026_06_28_001.md](order_124_node0_post_l_document_material_packet_order_draft_2026_06_28_001.md) +- [order_124_node0_post_l_document_material_packet_implementation_2026_06_28_001.md](order_124_node0_post_l_document_material_packet_implementation_2026_06_28_001.md) +- [order_125_l3_per_document_summary_frame_design_order_draft_2026_06_28_001.md](order_125_l3_per_document_summary_frame_design_order_draft_2026_06_28_001.md) +- [order_126_runtime_all_document_extract_display_2026_06_28_001.md](order_126_runtime_all_document_extract_display_2026_06_28_001.md) +- [order_127_revision_document_extract_count_alignment_2026_06_28_001.md](order_127_revision_document_extract_count_alignment_2026_06_28_001.md) +- [order_128_node3_actual_read_doc_identity_key_2026_06_28_001.md](order_128_node3_actual_read_doc_identity_key_2026_06_28_001.md) +- [order_129_search_candidate_count_basis_audit_2026_06_28_001.md](order_129_search_candidate_count_basis_audit_2026_06_28_001.md) +- [order_130_document_evidence_role_claim_guard_2026_06_29_001.md](order_130_document_evidence_role_claim_guard_2026_06_29_001.md) +- [order_131_search_candidate_scope_split_2026_06_29_001.md](order_131_search_candidate_scope_split_2026_06_29_001.md) +- [answer_basis_material_policy_philosophy_2026_06_29_001.md](answer_basis_material_policy_philosophy_2026_06_29_001.md) +- [order_125_l3_per_document_summary_frame_implementation_2026_06_29_001.md](order_125_l3_per_document_summary_frame_implementation_2026_06_29_001.md) +- [order_132_node2_answer_basis_material_delivery_policy_order_draft_2026_06_29_001.md](order_132_node2_answer_basis_material_delivery_policy_order_draft_2026_06_29_001.md) +- [order_132_node2_answer_basis_material_delivery_policy_implementation_2026_06_29_001.md](order_132_node2_answer_basis_material_delivery_policy_implementation_2026_06_29_001.md) +- [qwen_vs_songryeon_runtime_value_experiment_2026_06_29_001.md](qwen_vs_songryeon_runtime_value_experiment_2026_06_29_001.md) +- [order_133_codebase_readonly_inspection_mvp_2026_06_30_001.md](order_133_codebase_readonly_inspection_mvp_2026_06_30_001.md) +- [order_134_l_tool_scope_and_budget_partition_2026_06_30_001.md](order_134_l_tool_scope_and_budget_partition_2026_06_30_001.md) +- [order_135_code_evidence_accounting_2026_06_30_001.md](order_135_code_evidence_accounting_2026_06_30_001.md) +- [order_136_current_capability_baseline_and_live_test_pack_2026_06_30_001.md](order_136_current_capability_baseline_and_live_test_pack_2026_06_30_001.md) +- [order_137_source_code_context_summary_coverage_guard_2026_06_30_001.md](order_137_source_code_context_summary_coverage_guard_2026_06_30_001.md) +- [order_138_integration_baseline_dirty_worktree_reconciliation_2026_06_30_001.md](order_138_integration_baseline_dirty_worktree_reconciliation_2026_06_30_001.md) +- [night_government_mvp_removed_2026_06_30_001.md](night_government_mvp_removed_2026_06_30_001.md) +- [order_139_graph_memory_foundation_order_draft_2026_06_30_001.md](order_139_graph_memory_foundation_order_draft_2026_06_30_001.md) +- [order_139_graph_memory_foundation_2026_06_30_001.md](order_139_graph_memory_foundation_2026_06_30_001.md) +- [order_140_144_candidate_order_batch_2026_06_30_001.md](order_140_144_candidate_order_batch_2026_06_30_001.md) +- [order_140_r_loop_frame_state_machine_2026_06_30_001.md](order_140_r_loop_frame_state_machine_2026_06_30_001.md) +- [order_141_core_ego_guide_worker_hints_2026_06_30_001.md](order_141_core_ego_guide_worker_hints_2026_06_30_001.md) +- [order_142_external_graph_db_adapter_boundary_2026_06_30_001.md](order_142_external_graph_db_adapter_boundary_2026_06_30_001.md) +- [order_143_r_loop_node0_memory_packet_handoff_2026_06_30_001.md](order_143_r_loop_node0_memory_packet_handoff_2026_06_30_001.md) +- [order_144_r_route_dry_run_only_2026_06_30_001.md](order_144_r_route_dry_run_only_2026_06_30_001.md) +- [order_145_r_loop_pre_live_route_audit_baseline_2026_07_01_001.md](order_145_r_loop_pre_live_route_audit_baseline_2026_07_01_001.md) +- [order_146_r_route_experimental_gate_2026_07_01_001.md](order_146_r_route_experimental_gate_2026_07_01_001.md) +- [order_147_r_result_to_node3_brief_2026_07_01_001.md](order_147_r_result_to_node3_brief_2026_07_01_001.md) +- [order_148_graph_next_and_turn_access_ledger_2026_07_01_001.md](order_148_graph_next_and_turn_access_ledger_2026_07_01_001.md) +- [order_149_r_graph_traversal_candidate_surface_2026_07_01_001.md](order_149_r_graph_traversal_candidate_surface_2026_07_01_001.md) +- [order_150_r_loop_multi_step_traversal_dry_run_2026_07_01_001.md](order_150_r_loop_multi_step_traversal_dry_run_2026_07_01_001.md) +- [order_151_l_loop_activity_ledger_for_graph_backup_2026_07_01_001.md](order_151_l_loop_activity_ledger_for_graph_backup_2026_07_01_001.md) +- [order_152_raw_capsule_to_activity_ledger_graph_link_2026_07_01_001.md](order_152_raw_capsule_to_activity_ledger_graph_link_2026_07_01_001.md) +- [graph_memory_activity_ledger_pre_external_db_audit_2026_07_01_001.md](graph_memory_activity_ledger_pre_external_db_audit_2026_07_01_001.md) +- [order_153_graph_memory_integrity_and_r_experimental_source_recording_2026_07_01_001.md](order_153_graph_memory_integrity_and_r_experimental_source_recording_2026_07_01_001.md) +- [order_154_fast_test_gate_2026_07_01_001.md](order_154_fast_test_gate_2026_07_01_001.md) +- [order_155_graph_source_kind_separated_ingest_2026_07_01_001.md](order_155_graph_source_kind_separated_ingest_2026_07_01_001.md) +- [order_156_graph_source_observation_time_and_core_link_2026_07_01_001.md](order_156_graph_source_observation_time_and_core_link_2026_07_01_001.md) +- [order_157_songryeon_core_source_ingest_manifest_2026_07_01_001.md](order_157_songryeon_core_source_ingest_manifest_2026_07_01_001.md) +- [neo4j_context_graph_lessons_for_night_government_2026_07_01_001.md](neo4j_context_graph_lessons_for_night_government_2026_07_01_001.md) +- [order_158_graph_memory_export_packet_2026_07_01_001.md](order_158_graph_memory_export_packet_2026_07_01_001.md) +- [order_159_vessel_adapter_write_plan_boundary_2026_07_01_001.md](order_159_vessel_adapter_write_plan_boundary_2026_07_01_001.md) +- [order_160_local_vessel_neo4j_first_write_2026_07_01_001.md](order_160_local_vessel_neo4j_first_write_2026_07_01_001.md) +- [order_161_vessel_display_vocabulary_2026_07_01_001.md](order_161_vessel_display_vocabulary_2026_07_01_001.md) +- [order_162_vessel_readback_verification_2026_07_01_001.md](order_162_vessel_readback_verification_2026_07_01_001.md) +- [order_163_vessel_inspect_manual_walk_2026_07_01_001.md](order_163_vessel_inspect_manual_walk_2026_07_01_001.md) +- [order_164_dynamic_source_lineage_invalidation_2026_07_02_001.md](order_164_dynamic_source_lineage_invalidation_2026_07_02_001.md) +- [order_165_same_content_observation_ledger_2026_07_02_001.md](order_165_same_content_observation_ledger_2026_07_02_001.md) +- [order_166_night_time_bundle_summary_node_2026_07_02_001.md](order_166_night_time_bundle_summary_node_2026_07_02_001.md) +- [order_167_night_summary_naming_clarity_2026_07_02_001.md](order_167_night_summary_naming_clarity_2026_07_02_001.md) +- [order_168_night_summarize_changed_source_leaves_2026_07_02_001.md](order_168_night_summarize_changed_source_leaves_2026_07_02_001.md) +- [order_169_night_changed_source_summary_cli_2026_07_02_001.md](order_169_night_changed_source_summary_cli_2026_07_02_001.md) +- [night_government_august_materials_from_main_project_audit_2026_07_02_001.md](night_government_august_materials_from_main_project_audit_2026_07_02_001.md) +- [order_170_night_changed_source_one_at_a_time_runner_2026_07_02_001.md](order_170_night_changed_source_one_at_a_time_runner_2026_07_02_001.md) +- [order_171_night_token_budget_layer_summary_2026_07_02_001.md](order_171_night_token_budget_layer_summary_2026_07_02_001.md) +- [order_172_night_token_layer_auto_reduce_2026_07_02_001.md](order_172_night_token_layer_auto_reduce_2026_07_02_001.md) +- [order_173_night_checkpointed_long_runner_2026_07_02_001.md](order_173_night_checkpointed_long_runner_2026_07_02_001.md) +- [order_174_vessel_inspect_summary_layer_view_2026_07_02_001.md](order_174_vessel_inspect_summary_layer_view_2026_07_02_001.md) +- [order_175_vessel_backed_r_graph_read_packet_2026_07_02_001.md](order_175_vessel_backed_r_graph_read_packet_2026_07_02_001.md) +- [order_176_vessel_r_one_step_traversal_2026_07_02_001.md](order_176_vessel_r_one_step_traversal_2026_07_02_001.md) +- [order_177_r1_candidate_text_blindness_2026_07_02_001.md](order_177_r1_candidate_text_blindness_2026_07_02_001.md) +- [order_178_r_vessel_candidate_layer_surface_2026_07_02_001.md](order_178_r_vessel_candidate_layer_surface_2026_07_02_001.md) +- [order_179_r2_vessel_selection_id_disambiguation_2026_07_02_001.md](order_179_r2_vessel_selection_id_disambiguation_2026_07_02_001.md) +- [order_180_r2_prompt_example_id_removal_2026_07_02_001.md](order_180_r2_prompt_example_id_removal_2026_07_02_001.md) +- [order_181_r2_official_selection_ref_map_2026_07_02_001.md](order_181_r2_official_selection_ref_map_2026_07_02_001.md) +- [order_182_r_core_ego_start_selection_surface_2026_07_02_001.md](order_182_r_core_ego_start_selection_surface_2026_07_02_001.md) +- [order_183_r_vessel_hierarchical_child_candidate_surface_2026_07_02_001.md](order_183_r_vessel_hierarchical_child_candidate_surface_2026_07_02_001.md) +- [order_184_r_vessel_multi_step_traversal_mvp_2026_07_02_001.md](order_184_r_vessel_multi_step_traversal_mvp_2026_07_02_001.md) +- [order_185_r_terminal_material_guard_2026_07_03_001.md](order_185_r_terminal_material_guard_2026_07_03_001.md) +- [order_186_r_vessel_exact_child_record_expansion_2026_07_03_001.md](order_186_r_vessel_exact_child_record_expansion_2026_07_03_001.md) +- [order_187_r_vessel_summary_layer_before_raw_2026_07_03_001.md](order_187_r_vessel_summary_layer_before_raw_2026_07_03_001.md) +- [order_188_r_vessel_raw_original_read_cap_2026_07_03_001.md](order_188_r_vessel_raw_original_read_cap_2026_07_03_001.md) +- [order_189_r_traverse_live_audit_2026_07_03_001.md](order_189_r_traverse_live_audit_2026_07_03_001.md) +- [order_190_r_vessel_token_summary_deeper_child_expansion_2026_07_03_001.md](order_190_r_vessel_token_summary_deeper_child_expansion_2026_07_03_001.md) +- [order_191_r2_source_ingest_branch_selection_stability_2026_07_03_001.md](order_191_r2_source_ingest_branch_selection_stability_2026_07_03_001.md) +- [order_192_r1_user_question_anchor_copy_2026_07_03_001.md](order_192_r1_user_question_anchor_copy_2026_07_03_001.md) diff --git a/Administrative_Reform_1/05_Execution_Records/answer_basis_material_policy_philosophy_2026_06_29_001.md b/Administrative_Reform_1/05_Execution_Records/answer_basis_material_policy_philosophy_2026_06_29_001.md new file mode 100644 index 0000000..2308486 --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/answer_basis_material_policy_philosophy_2026_06_29_001.md @@ -0,0 +1,44 @@ +# answer_basis_material_policy_philosophy_2026_06_29_001 + +작성일: 2026-06-29 +대상 문서: `Administrative_Reform_1/00_Philosophy/Answer_Basis_Mode_And_Evidence_Role_Philosophy_2026_06_26.md` + +## 1. 기록 목적 + +사용자가 제안한 장기 구조 감각을 철학 문서에 반영했다. + +핵심 제안: + +- 장기적으로 메타정보 판단은 턴 앞, 턴 중간, 턴 끝 모두에서 필요해질 수 있다. +- 하지만 지금 당장 전체 구조를 열지 않는다. +- 단기적으로는 node_2가 node_3에게 답변 태도를 정해 주는 선에서만 다룬다. + +## 2. 반영 내용 + +철학 문서에 `2026-06-29 추가 메모: 메타정보 판단의 장기 위치` 섹션을 추가했다. + +추가한 경계: + +- `absolute_first`일 때는 원문, count, trace/data 장부, document material packet 같은 검증 가능한 재료를 우선한다. +- `relative_allowed` 또는 `mixed_or_uncertain`일 때는 장기적으로 L3, 5, 또는 별도 요약 노드가 source가 붙은 요약 카드/문서별 근거 카드를 만들 수 있다. +- 단, 요약은 code의 절대정보가 아니라 LLM이 생성한 상대정보 또는 혼합정보로 보존해야 한다. +- 지금 구현 범위는 node_2 -> node_3 답변 태도 전달까지만 둔다. + +## 3. 하지 않은 것 + +- 새 발주서 작성 없음 +- 코드 변경 없음 +- 테스트 실행 없음 +- L3 per-document summary 구현 없음 +- node_5 기억 압축 구현 없음 +- 전역 메타정보 판단 루프 구현 없음 + +## 4. 후속 후보 + +나중에 다음 주제를 별도 발주서로 좁힐 수 있다. + +- answer material policy +- absolute_first일 때 raw evidence 우선 공급 +- relative/mixed일 때 source-attached summary card 공급 +- L3 문서별 요약과 node_4 검토 연결 +- node_5 기억 압축 요약과 node_4 승인 연결 diff --git a/Administrative_Reform_1/05_Execution_Records/graph_memory_activity_ledger_pre_external_db_audit_2026_07_01_001.md b/Administrative_Reform_1/05_Execution_Records/graph_memory_activity_ledger_pre_external_db_audit_2026_07_01_001.md new file mode 100644 index 0000000..7727d5f --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/graph_memory_activity_ledger_pre_external_db_audit_2026_07_01_001.md @@ -0,0 +1,266 @@ +# Graph Memory / Activity Ledger Pre-External-DB Audit + +Date: 2026-07-01 + +## Scope + +Audited the current graph-memory and activity-ledger path before opening a real external graph DB. + +Primary scope: + +- ORDER 139 graph memory foundation +- ORDER 148 NEXT edge and R graph access ledger +- ORDER 149 R traversal candidate surface +- ORDER 150 R multi-step dry-run traversal +- ORDER 151 L loop activity ledger +- ORDER 152 raw capsule to activity ledger graph link + +This audit did not patch code. + +## Baseline + +Before this audit, the latest committed implementation had passed: + +```powershell +python -m compileall songryeon_core main.py +python -m pytest +python main.py smoke-test +``` + +Observed committed baseline: + +```text +pytest: 143 passed +smoke-test: SMOKE_TEST_OK +latest commit: 4c0ac86 Add raw capsule activity graph links +``` + +## What Looks Healthy + +### 1. ORDER 152 default graph link is structurally sound + +Default dry-turn records: + +- `graph:turn_activity_graph_link:turn_dry_001` +- `graph:activity_ledger:L:activity_ledger_frame` +- `graph:edge:has_activity_ledger:graph:raw_capsule:turn_dry_001:graph:activity_ledger:L:activity_ledger_frame` + +The frame uses: + +- `generated_by=CODE:TURN_ACTIVITY_GRAPH_LINK_BUILDER` +- `info_class=absolute` +- `semantic_judgement_status=not_run` + +The graph activity link layer is therefore not pretending to be an LLM summary or semantic memory. + +### 2. Graph source refs are valid in default + R dry-run path + +Custom audit check: + +```text +run_dry_turn(enable_r_route_dry_run=True) +graph_missing_source_refs=[] +missing_edge_endpoints=[] +turn_activity_counts=1 L ledger / 1 R ledger / 2 nodes / 2 edges +``` + +This means the default graph-memory records and the dry-run R access ledger can be followed through DataStore for the tested path. + +### 3. Same-turn L run 2 activity ledger is linked + +Custom audit check with same-turn L reroute policy enabled: + +```text +l_loop_activity_ledger_data_ids=[ + "L:activity_ledger_frame", + "L:run:0002:L:activity_ledger_frame" +] +link_l_ledgers=[ + "L:activity_ledger_frame", + "L:run:0002:L:activity_ledger_frame" +] +has_run2_activity_node=True +``` + +So ORDER 152 can link multiple L activity ledgers for the same turn. + +### 4. External DB is still unopened + +The graph DB boundary reserves names only: + +- service: `songryeon-neo4j-vessel` +- database: `songryeon_vessel` +- namespace: `songryeon_core_graph_v0` + +No real Neo4j/graph DB write path has been opened yet. + +## Findings + +### P1. Experimental R route can create dangling graph source refs + +Location: + +- `songryeon_core/runtime/dry_run.py` + +Observed path: + +- `decision.route == "R"` +- code calls `build_graph_memory_snapshot_from_capsules(...)` +- but does not record that graph snapshot/guide/nodes/edges to DataStore +- then R handoff and R frames cite those IDs in `source_data_ids` + +Audit reproduction: + +```text +run_dry_turn(..., enable_r_route_experimental=True) +r_route_experimental_graph_data_ids=[] +missing_graph_or_rloop_count=14 +``` + +Examples of missing refs: + +```text +rloop:graph_guide:graph:snapshot:turn_dry_001:r_route_experimental +graph:snapshot:turn_dry_001:r_route_experimental +graph:time_bundle:turn_dry_001:r_route_experimental +graph:edge:child_of_time_axis:graph:axis:time:graph:time_bundle:turn_dry_001:r_route_experimental +``` + +Why this matters: + +The R experimental route can produce trace/data records whose declared graph sources cannot be dereferenced from DataStore. This is risky before external DB export or live R routing. + +Recommended fix: + +Record the experimental R graph build before creating the R handoff. Prefer reusing `record_graph_memory_for_capsules(...)` or extracting a helper that records an already-built graph memory result. Then fill `r_route_experimental_graph_data_ids` with the recorded node/edge/snapshot/guide ids. + +Do not fix by silently dropping source refs. + +### P2. R1/R2/R3 dry-run frames use `mixed` while semantic judgement is `not_run` + +Location: + +- `songryeon_core/core/schema_parts/r_loop.py` +- `songryeon_core/loops/r_loop_dry_run.py` + +Observed: + +- `R1GraphGoalFrame.info_class = "mixed"` +- `R2GraphNodeSelectionFrame.info_class = "mixed"` +- `R3GraphInspectionFrame.info_class = "mixed"` +- generated by code in dry-run skeleton +- `semantic_judgement_status="not_run"` + +Why this matters: + +The current dry-run skeleton is documented as wiring verification, not semantic ranking. However, `mixed` normally means a multi-source semantic/interpretive judgment. These frames currently look more like explicit code policy/status frames than true mixed-information LLM judgments. + +Risk: + +Low for current dry-run tests, medium before live R route. If this shape is copied into live R, code may appear to have made a mixed semantic decision. + +Recommended fix: + +Do not add hidden heuristics. Either: + +- keep these frames clearly dry-run-only and document the exception, or +- split code-policy fields into an absolute/absolute-policy-decision frame, leaving true mixed R selection to a later R LLM frame. + +### P2. Activity ledger graph nodes are not in the current graph snapshot/guide packet + +Location: + +- `songryeon_core/core/graph_memory.py` +- `songryeon_core/core/turn_activity_graph_links.py` + +Observed: + +`activity_ledger` nodes and `HAS_ACTIVITY_LEDGER` edges are recorded after the main `GraphMemorySnapshotFrame` and `RLoopGraphGuidePacketFrame` are built. + +Audit check: + +```text +snapshot_has_activity_nodes=[] +snapshot_has_activity_edges=[] +``` + +Why this matters: + +If a future external DB export only exports the snapshot/guide graph ids, it will miss activity ledger nodes and edges. + +Recommended fix: + +Before real graph DB export, define an export packet or export query that collects all `graph_memory:node:*` and `graph_memory:edge:*` DataStore records, not only `GraphMemorySnapshotFrame.graph_node_ids` and `graph_edge_ids`. + +### P3. R traversal cannot yet walk `HAS_ACTIVITY_LEDGER` + +Location: + +- `songryeon_core/core/schema_parts/r_loop.py` +- `songryeon_core/loops/r_loop_dry_run.py` + +Observed: + +R traversal candidate relations are currently: + +```text +child +next +previous +``` + +`HAS_ACTIVITY_LEDGER` is a graph edge kind, but not a candidate relation surfaced to R traversal. + +Why this matters: + +ORDER 152 makes activity ledgers graph-linkable and exportable, but R cannot yet intentionally traverse from a raw capsule into its L/R activity ledger. + +Recommended fix: + +Not urgent. Add later only when R is meant to inspect activity ledgers. This should be an explicit order, because it changes R's reading surface. + +### P2. DataStore graph records lack a global graph integrity validator + +Location: + +- `songryeon_core/core/schema_parts/graph_memory.py` +- `songryeon_core/core/graph_memory_store.py` +- `songryeon_core/core/turn_activity_graph_links.py` + +Observed: + +Per-frame validators check shape and metainfo. `InMemoryGraphMemoryStore.upsert_edge()` checks endpoint existence, but raw DataStore recording can still hold a graph edge whose endpoint records are missing unless each call path does its own ordering and checks. + +Why this matters: + +External DB export needs a global invariant: + +- every `graph_memory:edge:*` source/target node exists +- every graph frame `source_data_ids` graph/rloop ref exists or is explicitly marked external +- snapshot/guide/export packet lists match the exported record set + +Recommended fix: + +Add a read-only graph integrity audit helper before Neo4j export. It should report missing refs rather than auto-repairing them. + +## Next Recommended Order + +Recommended next order before real Neo4j work: + +```text +ORDER_153_GRAPH_MEMORY_EXPORT_INTEGRITY_AUDIT_AND_R_EXPERIMENTAL_SOURCE_RECORDING_V0 +``` + +Narrow scope: + +1. Record R experimental graph snapshot/guide/node/edge records before R handoff. +2. Add a read-only graph-memory integrity audit helper. +3. Add tests that fail on dangling graph/rloop source refs. +4. Do not open Neo4j yet. +5. Do not add semantic axis, R LLM selector, or activity-ledger traversal yet. + +## Bottom Line + +The current graph-memory foundation is usable as a local DataStore prototype. + +It is not yet clean enough to export blindly to a graph DB because the experimental R route can reference graph IDs that were built in memory but not recorded, and because snapshot-only export would miss activity ledger graph links. diff --git a/Administrative_Reform_1/05_Execution_Records/neo4j_context_graph_lessons_for_night_government_2026_07_01_001.md b/Administrative_Reform_1/05_Execution_Records/neo4j_context_graph_lessons_for_night_government_2026_07_01_001.md new file mode 100644 index 0000000..e193546 --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/neo4j_context_graph_lessons_for_night_government_2026_07_01_001.md @@ -0,0 +1,42 @@ +# Neo4j Context Graph Lessons For Night Government - Execution Record 2026-07-01 001 + +## Scope + +Created a maintenance-system learning document from the user's requested Neo4j Agent Memory / Context Graph video and official Neo4j materials. + +## Added + +- `Administrative_Reform_1/01_Maintenance_System/NEO4J_CONTEXT_GRAPH_LESSONS_FOR_SONGRYEON_NIGHT_GOVERNMENT_V0.md` + +## Key Conclusion + +Night Government should not be designed as a free summarizer. + +It should be a controlled procedure that promotes: + +- short-term memory +- long-term memory +- reasoning memory + +into a provenance-preserving context graph. + +## SongRyeon-Specific Boundary + +The document emphasizes that SongRyeon Core must be stricter than a generic memory API: + +- semantic memory must keep source graph/data/trace IDs +- summary nodes must record generated time/model/prompt/source bundle hash +- reasoning memory must be first-class +- node_4 review should gate trusted semantic promotion + +## Verification + +No code behavior changed. + +Checked: + +```powershell +git status --short +``` + +Documentation only. diff --git a/Administrative_Reform_1/05_Execution_Records/night_government_august_materials_from_main_project_audit_2026_07_02_001.md b/Administrative_Reform_1/05_Execution_Records/night_government_august_materials_from_main_project_audit_2026_07_02_001.md new file mode 100644 index 0000000..3f30d49 --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/night_government_august_materials_from_main_project_audit_2026_07_02_001.md @@ -0,0 +1,50 @@ +# Night Government August Materials From Main Project Audit 2026-07-02 001 + +## 작업 목적 + +사용자 요청에 따라 SongRyeon 본점의 심야정부 관련 문서와 코드 흔적을 정밀 탐사하고, SongRyeon Core의 8월 심야정부 완성에 필요한 재료를 철학 문서로 정리했다. + +## 읽은 본점 근거 + +- `SongRyeon_Project/AGENTS.md` +- `SongRyeon_Project/ANIMA_DOCS_INDEX.md` +- `SongRyeon_Project/Administrative_Reform_1/00_Philosophy/ANIMA_SLEEP_STACK_V1.md` +- `SongRyeon_Project/Administrative_Reform_1/00_Philosophy/ANIMA_SLEEP_STACK_V2.md` +- `SongRyeon_Project/Administrative_Reform_1/00_Philosophy/ANIMA_V4_MIDNIGHT_GOVERNMENT_PROPOSAL.md` +- `SongRyeon_Project/Administrative_Reform_1/03_Maps/01_Function_Maps/ANIMA_MIDNIGHT_DMN_REORG10A_INVENTORY_2026_05_26.md` +- `SongRyeon_Project/Administrative_Reform_2/02_Department_Studies/D2_DAY_NIGHT_STATE_SEPARATION_AND_ZERO_SUPPLY_AUDIT_2026_06_19.md` +- `SongRyeon_Project/Administrative_Reform_2/02_Department_Studies/D3_NIGHT_COREEGO_SOURCE_AUTHORITY_BOUNDARY_AUDIT_2026_06_19.md` +- `SongRyeon_Project/Administrative_Reform_2/02_Department_Studies/D7_NIGHT_MEMORY_PROMOTION_AND_GRAPH_PRIVACY_AUDIT_2026_06_19.md` +- `SongRyeon_Project/Administrative_Reform_2/02_Department_Studies/D8_MIDNIGHT_GOVERNMENT_PROPORTIONAL_COUNCIL_SCOPE_AUDIT_2026_06_19.md` +- `SongRyeon_Project/Administrative_Reform_2/02_Department_Studies/D9_COREEGO_LOCAL_COUNCIL_PROPORTIONAL_IDENTITY_AUDIT_2026_06_19.md` +- `SongRyeon_Project/Administrative_Reform_2/02_Department_Studies/D10_NIGHT_COREEGO_TRACEABILITY_REPLAY_REQUIREMENTS_2026_06_19.md` +- `SongRyeon_Project/Administrative_Reform_2/02_Department_Studies/GOVERNOR_MIDNIGHT_COREEGO_PROPORTIONAL_COUNCIL_SYNTHESIS_2026_06_19.md` +- `SongRyeon_Project/Administrative_Reform_2/03_V5_Constitution_Materials/ANIMA_MIDNIGHT_COREEGO_TRACE_REPLAY_FIXTURE_REPORT_2026_06_20.md` +- `SongRyeon_Project/Core/_archive_v3_midnight/midnight_reflection.py` +- `SongRyeon_Project/Core/_archive_v3_midnight/midnight/rem_governor.py` + +## Core에 반영한 철학 문서 + +- `Administrative_Reform_1/00_Philosophy/Night_Government_August_Materials_From_Main_Project_2026_07_02.md` +- `Administrative_Reform_1/00_Philosophy/Night_Government_Graph_Memory_Philosophy_2026_06_30.md` +- `Administrative_Reform_1/00_Philosophy/README.md` + +## 정리한 핵심 + +- 본점의 Sleep Stack은 밤이 낮의 정책/도구/기억 사용을 준비한다는 큰 비전을 제공한다. +- Core는 이 비전을 당장 route/tool/final authority로 열지 않고, graph memory의 source/version/summary/validity/R-guide 정리로 좁힌다. +- 8월 Core 심야정부의 핵심 재료는 `NightJobLedger`, `SourceVersionLineage`, `SummaryInvalidationLedger`, `LeafSummaryGraphNode`, `SourceKindBundleSummary`, `TimeBundleSummary`, `CoreEgoGuideSummary`다. +- 밤 산출물은 기본적으로 advisory/candidate/summary material이며, 낮의 route/tool/answer mode/final evidence 권한을 직접 갖지 않는다. +- dynamic source가 바뀌면 완료된 옛 summary는 삭제하지 않고 invalidated로 남기며, pending summary는 `superseded_before_run`으로 닫고 LLM 호출을 하지 않는다. +- 시간축은 기본 골격으로 유지하고, 의미축은 R-loop 수요가 실제로 관측된 뒤 demand-driven으로 연다. + +## 검증 + +문서 작업만 수행했다. + +실행 예정 검증: + +```powershell +git diff --check +``` + diff --git a/Administrative_Reform_1/05_Execution_Records/night_government_mvp_removed_2026_06_30_001.md b/Administrative_Reform_1/05_Execution_Records/night_government_mvp_removed_2026_06_30_001.md new file mode 100644 index 0000000..d277042 --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/night_government_mvp_removed_2026_06_30_001.md @@ -0,0 +1,56 @@ +# night_government_mvp_removed_2026_06_30_001 + +## 1. 작업 요약 + +외부 채팅방에서 먼저 구현된 심야정부 MVP를 송련 Core 런타임에서 제거했다. + +제거 판단: + +- `MemoryRecord`는 기존 TurnStateCapsule/TraceStore/DataStore 좌표를 재사용하지 않는 별도 JSONL 기억 카드였다. +- `NightGovernmentPacket`은 송련의 0 기억공급관이나 캡슐 계층과 연결되지 않은 독립 active packet이었다. +- `MemoryActivationItem`은 기존 메타정보 출처 체계와 정렬되지 않은 별도 활성화 항목이었다. +- 따라서 현재 상태로는 송련 Core의 "절대정보에서 상대/혼합정보가 파생된다"는 원칙을 강화하기보다, 중복 기억 저장소를 만들 위험이 컸다. + +## 2. 제거한 것 + +- `songryeon_core/night_government/__init__.py` +- `songryeon_core/night_government/runtime.py` +- `songryeon_core/night_government/schemas.py` +- `songryeon_core/night_government/store.py` +- `tests/test_night_government_mvp.py` +- `main.py`의 `night-ingest`, `night-run`, `night-active` CLI + +## 3. 남긴 것 + +- ORDER_138과 관련 실행 기록은 과거 감사 자료로 남겼다. +- 단, 두 문서 상단에 심야정부 MVP가 제거되었고 재설계 기준으로 쓰면 안 된다는 상태 표시를 추가했다. + +## 4. 향후 원칙 + +심야정부나 외부 DB 기억 체계를 다시 열 경우, 다음 조건을 먼저 만족해야 한다. + +- 기존 TurnStateCapsule/TraceStore/DataStore 좌표를 우선 사용한다. +- relative/mixed 정보는 orphan으로 저장하지 않는다. +- 외부 DB record는 `info_class`, `generated_by`, `semantic_judgement_status`, `source_trace_ids`, `source_data_ids` 같은 송련 메타정보 경계를 먼저 갖춘다. +- 0 기억공급관과의 연결은 별도 발주와 smoke-test를 둔다. + +## 5. 검증 결과 + +다음 명령을 실행했다. + +```powershell +python -m compileall songryeon_core main.py +python -m pytest +python main.py smoke-test +``` + +결과: + +- `compileall`: 통과 +- 전체 `pytest`: `77 passed` +- `smoke-test`: `SMOKE_TEST_OK` + +참고: + +- 제거 전 전체 pytest는 심야정부 전용 테스트 2개를 포함해 `79 passed`였다. +- 제거 후 `tests/test_night_government_mvp.py`를 삭제했기 때문에 전체 pytest 수가 `77`로 줄어든 것은 의도된 결과다. diff --git a/Administrative_Reform_1/05_Execution_Records/order_112_explicit_artifact_priority_and_whole_document_packing_2026_06_27_001.md b/Administrative_Reform_1/05_Execution_Records/order_112_explicit_artifact_priority_and_whole_document_packing_2026_06_27_001.md new file mode 100644 index 0000000..fdf5156 --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_112_explicit_artifact_priority_and_whole_document_packing_2026_06_27_001.md @@ -0,0 +1,158 @@ +# ORDER_112 explicit artifact priority and whole-document packing 실행 기록 + +## 목적 + +사용자 입력에 `ORDER_100` 같은 명시 문서 reference가 있을 때 embedding search 후보보다 먼저 원문 ORDER 문서를 direct resolve하고, node_3에 공급되는 문서 context를 whole-document char budget 기준으로 packing했다. + +## 구현 위치 + +새 파일: + +```text +songryeon_core/tools/document_context_pack.py +tests/test_order_112_document_context_pack.py +``` + +주요 변경: + +```text +songryeon_core/core/schemas.py +songryeon_core/tools/document_tools.py +songryeon_core/loops/l_loop.py +songryeon_core/nodes/node_2_handoff.py +songryeon_core/runtime/terminal_view.py +songryeon_core/runtime/defaults.py +songryeon_core/runtime/dry_run.py +songryeon_core/runtime/user_turn.py +main.py +songryeon_core/prompts/node_3_reporter_v0.md +``` + +## 동작 + +추출: + +```text +extract_explicit_artifact_references(user_text) +``` + +direct resolve: + +```text +read_artifact(root, artifact_ref) +ExplicitArtifactReferenceFrame +``` + +`ORDER_###` bare reference는 `04_Orders/` 아래 원문 ORDER 파일명의 prefix로만 매칭한다. +실행기록, README, digest, summary는 bare ORDER 원문 대체 후보로 쓰지 않는다. + +context packing: + +```text +DocumentContextPackFrame +max_document_context_chars +budget_unit=chars +whole_document_only=true +strict_rank_order=true +``` + +rank order: + +```text +explicit unique references in user order +-> embedding search preserved unique doc_id candidates +``` + +budget 초과 문서는 `excluded_due_to_context_budget`가 되고, strict rank order 때문에 그 뒤 후보는 `excluded_after_strict_rank_cutoff`가 된다. + +## node_3 연결 + +`node_2_handoff.record_node3_input_brief()`는 `DocumentContextPackFrame`이 있으면 included document만 `read_documents`에 넣는다. + +excluded document는 `excluded_document_contexts`로만 전달한다. +이 목록은 읽은 문서 count에 들어가지 않는다. + +## runtime 표시 + +pretty runtime에 다음을 표시한다. + +```text +explicit_artifact_refs +document_context_pack included/excluded/budget/cutoff +route=2 handoff document_context_pack counts +node_3 input brief excluded_contexts +``` + +## 테스트 + +추가 pytest: + +```text +tests/test_order_112_document_context_pack.py +``` + +검증한 내용: + +```text +explicit ORDER reference extraction +unique ORDER direct resolve and rank priority +ambiguous ORDER reference no arbitrary selection +whole-document packing without mid-document cut +excluded_due_to_context_budget +excluded_after_strict_rank_cutoff +node_3 read_documents count == included count +excluded document not counted as read +terminal view displays explicit resolve and context pack +``` + +## 검증 결과 + +실행: + +```powershell +python -m compileall songryeon_core main.py +python -m pytest +python main.py smoke-test +``` + +결과: + +```text +compileall passed +pytest: 14 passed in 243.25s +smoke-test passed: SMOKE_TEST_OK +``` + +중간 실패: + +```text +pytest first full run failed with NameError: query_data_ids is not defined +``` + +원인: + +```text +DocumentContextPackFrame source_data_ids 조립에서 존재하지 않는 로컬 이름을 사용했다. +``` + +조치: + +```text +l2_query_data_id와 revision_query_data_ids를 명시적으로 사용하도록 수정했다. +``` + +## 일부러 하지 않은 것 + +이번 작업에서 하지 않은 것: + +- W loop 열기 +- R loop 열기 +- scheduler 변경 +- 외부 DB/vector DB/장기기억 DB 변경 +- same-turn L reroute 횟수 확대 +- node_4 자동 재작성 루프 +- tokenizer 기반 정확 토큰 계산 +- 동적 document context budget 자동 증액 +- 코드가 의미적으로 중요한 문서를 고르는 정책 +- excluded 문서를 node_3 read document count에 포함 +- 문서 중간을 잘라 넣고 전체 문서처럼 표시 diff --git a/Administrative_Reform_1/05_Execution_Records/order_118_answer_basis_mode_order_draft_2026_06_27_001.md b/Administrative_Reform_1/05_Execution_Records/order_118_answer_basis_mode_order_draft_2026_06_27_001.md new file mode 100644 index 0000000..1663b93 --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_118_answer_basis_mode_order_draft_2026_06_27_001.md @@ -0,0 +1,32 @@ +# Execution Record: ORDER 118 Answer Basis Mode Order Draft 2026-06-27 001 + +## 목적 + +node_2가 node_3에게 답변 근거 말하기 모드를 전달하는 후속 발주서를 문서화했다. + +## 배경 + +최근 대화에서 사용자는 node_2의 말하기 모드를 7개 세부 모드가 아니라 3개 상위 모드로 제한하겠다고 결재했다. + +결재된 3개 모드: + +```text +absolute_first +relative_allowed +mixed_or_uncertain +``` + +이 구조는 송련의 행동 방식을 절대정보, 상대정보, 혼합정보 분류 원칙과 맞추기 위한 것이다. + +## 변경 + +- `Administrative_Reform_1/04_Orders/ORDER_118_NODE2_ANSWER_BASIS_MODE_FRAME_V0.md` 추가 +- `Administrative_Reform_1/04_Orders/README.md`의 정식 발주서 범위를 `ORDER_118`까지 확장 +- `ORDER_118` 요약과 링크 추가 + +## 비고 + +이번 작업은 문서화만 수행했다. +코드 구현, schema 변경, prompt 변경, smoke/pytest 추가는 하지 않았다. + +현재 작업트리에는 별도 진행 중인 ORDER_112 구현 변경이 존재하므로, 이번 기록은 ORDER_118 문서화 범위만 다룬다. diff --git a/Administrative_Reform_1/05_Execution_Records/order_118_node2_answer_basis_mode_frame_implementation_2026_06_27_001.md b/Administrative_Reform_1/05_Execution_Records/order_118_node2_answer_basis_mode_frame_implementation_2026_06_27_001.md new file mode 100644 index 0000000..76ae5fe --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_118_node2_answer_basis_mode_frame_implementation_2026_06_27_001.md @@ -0,0 +1,149 @@ +# Execution Record: ORDER 118 Node2 Answer Basis Mode Frame Implementation 2026-06-27 001 + +## 목적 + +`ORDER_118_NODE2_ANSWER_BASIS_MODE_FRAME_V0.md`에 따라 node_2가 node_3에게 최종 답변의 근거 말하기 모드(`answer_basis_mode`)를 명시적으로 전달하도록 MVP를 구현했다. + +## 구현 요약 + +- `Node2AnswerBasisFrame`을 추가했다. +- `answer_basis_mode` enum은 아래 3개로 고정했다. + +```text +absolute_first +relative_allowed +mixed_or_uncertain +``` + +- node_2 answer-basis selector prompt를 추가했다. +- node_2가 LLM 선택에 성공하면 `generated_by=LLM:*`, `semantic_judgement_status=ran`으로 frame을 기록한다. +- node_2 LLM 선택이 실패하거나 adapter가 없으면 code가 의미 판단을 대신하지 않고 안전 fallback frame을 기록한다. +- node_3 input brief와 node_3 LLM payload에 answer-basis 정보를 연결했다. +- node_3 reporter prompt와 grounding block에 answer-basis 경계를 반영했다. +- node_4 gatekeeper prompt와 checks에 answer-basis 모드 검사를 추가했다. +- terminal/runtime view와 JSON summary에 answer-basis 상태를 표시했다. + +## 주요 위치 + +- `Node2AnswerBasisFrame`: `songryeon_core/core/schemas.py` +- `answer_basis_mode` enum: `songryeon_core/core/schemas.py`의 `ANSWER_BASIS_MODES` +- node_2 mode 선택: `songryeon_core/nodes/node_2_metainfo_boundary.py`의 `run_node2_answer_basis_selection` +- node_2 prompt: `songryeon_core/prompts/node_2_answer_basis_selector_v0.md` +- node_3 brief 연결: `songryeon_core/nodes/node_2_handoff.py`의 `record_node3_input_brief`, `node3_brief_llm_payload` +- node_3 reporter 연결: `songryeon_core/nodes/node_3_reporter.py` +- node_4 guard prompt/check 연결: `songryeon_core/prompts/node_4_gatekeeper_v0.md`, `songryeon_core/nodes/node_4_gatekeeper.py` +- terminal/runtime 표시: `songryeon_core/runtime/terminal_view.py`, `songryeon_core/runtime/dry_run.py`, `main.py`, `songryeon_core/runtime/user_turn.py` + +## code fallback 정책 + +LLM selection 실패 또는 adapter 미공급 시: + +```text +answer_basis_mode = mixed_or_uncertain +basis_reason_codes = ["llm_mode_selection_failed"] +mode_selection_reason = "CODE_STATUS:node2_answer_basis_mode_selection_failed" +generated_by = CODE:FALLBACK +info_class = absolute_status +semantic_judgement_status = failed +evidence_roles = [] +``` + +이 fallback은 의미상 mixed가 맞다는 판단이 아니라, 판단 실패를 불확실성 모드로 닫는 안전 상태다. + +## node_2 prompt 교육 요약 + +새 prompt는 다음 경계를 교육한다. + +- 절대정보: 코드/파일/trace/data/schema/tool result처럼 시스템이 값과 존재를 확인할 수 있는 정보 +- 상대정보: 특정 하나의 절대정보 record/field에 대응하는 해석/판단/요약 +- 혼합정보: 여러 절대정보 묶음 또는 하나로 고정하기 부적절한 source bundle에 근거한 해석/판단/요약 +- answer-basis 선택은 보통 상대정보 또는 혼합정보이며, 절대정보 자체가 아님 +- 불확실하면 `mixed_or_uncertain`을 선택 + +## node_3 brief 연결 방식 + +`Node3InputBriefFrame`에 다음 필드를 추가했다. + +```text +answer_basis_frame_id +answer_basis_mode +basis_reason_codes +mode_selection_reason +mode_selection_reason_info_class +evidence_roles +answer_basis_generated_by +answer_basis_info_class +answer_basis_semantic_judgement_status +``` + +node_3 LLM payload에는 `answer_basis` object로 전달한다. +`evidence_roles`의 source는 raw internal ID 대신 안전한 source label로 바꿔 전달한다. + +## node_4 변경 여부 + +node_4 guard를 제거하거나 약화하지 않았다. + +추가한 것은 작다. + +- prompt에 answer-basis별 검사 기준 추가 +- node_4 입력 checks에 answer-basis 자세 확인 항목 추가 + +자동 재작성 루프는 열지 않았다. + +## 추가 테스트 + +추가한 pytest: + +- `tests/test_order_118_answer_basis.py::test_node2_answer_basis_absolute_first_fixture` +- `tests/test_order_118_answer_basis.py::test_node2_answer_basis_relative_allowed_fixture` +- `tests/test_order_118_answer_basis.py::test_node2_answer_basis_mixed_or_uncertain_fixture` +- `tests/test_order_118_answer_basis.py::test_node2_answer_basis_llm_failure_uses_code_fallback` +- `tests/test_order_118_answer_basis.py::test_node3_brief_receives_answer_basis_without_raw_id_leak_in_llm_payload` +- `tests/test_order_118_answer_basis.py::test_runtime_view_displays_answer_basis_fallback` + +확장한 smoke: + +- `run_smoke_tests()` 필수 data id에 `node_2:answer_basis_frame` 추가 +- `_check_route2_handoff_and_brief()`에서 answer-basis frame과 node_3 brief 보존 검사 추가 +- same-turn L 2회차 downstream scope smoke에 scoped answer-basis frame 검사 추가 +- smoke-test 결과 JSON에 answer-basis 표시 추가 + +## 검증 결과 + +```powershell +python -m compileall songryeon_core main.py +``` + +결과: 통과. + +```powershell +python -m pytest +``` + +결과: 20 passed in 247.35s. + +```powershell +python main.py smoke-test +``` + +결과: `SMOKE_TEST_OK`. + +확인된 smoke answer-basis 표시: + +```text +node2_answer_basis_mode = mixed_or_uncertain +node2_answer_basis_reason_codes = ["llm_mode_selection_failed"] +node2_answer_basis_generated_by = CODE:FALLBACK +node2_answer_basis_semantic = failed +``` + +## 일부러 하지 않은 것 + +- `answer_basis_mode`를 3개보다 늘리지 않았다. +- `document_primary`, `recent_conversation_primary` 같은 세부 모드를 만들지 않았다. +- code가 의미적으로 모드를 고르도록 만들지 않았다. +- scheduler, 외부 DB, vector DB, 장기기억 DB를 건드리지 않았다. +- W/R loop를 열지 않았다. +- same-turn L reroute 횟수를 늘리지 않았다. +- node_4 자동 재작성 루프를 열지 않았다. +- node_4 기존 guard를 약화하지 않았다. diff --git a/Administrative_Reform_1/05_Execution_Records/order_119_structure_failed_honesty_implementation_2026_06_27_001.md b/Administrative_Reform_1/05_Execution_Records/order_119_structure_failed_honesty_implementation_2026_06_27_001.md new file mode 100644 index 0000000..637cca7 --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_119_structure_failed_honesty_implementation_2026_06_27_001.md @@ -0,0 +1,88 @@ +# Execution Record: ORDER 119 Structure Failed Honesty Implementation 2026-06-27 001 + +## 목적 + +`structure_failed` 상태에서 검색/문서 근거를 가진 것처럼 말하는 fallback 답변을 막고, node_2 answer-basis selector 실패 원인을 runtime과 test에서 확인 가능하게 했다. + +## 변경 요약 + +- `structure_failed` fallback 답변을 별도 렌더링 경로로 분리했다. +- 검색 또는 read_doc payload가 실제로 있을 때만 해당 payload를 언급하도록 했다. +- node_2 answer-basis fallback frame에 실패 진단 필드를 추가했다. +- runtime/CLI 요약에 answer-basis 실패 진단을 표시하도록 했다. +- selected recent memory context가 실행기록 문서나 read_document가 아니라는 경계를 node_3 prompt와 brief payload에 추가했다. +- `**근거 기준:**` 형태의 중복 grounding block도 code assembly에서 제거하도록 했다. + +## 추가한 진단 필드 + +### structure_failed + +- `structure_failure_stage` +- `structure_failure_reason` +- `structure_failure_exception_type` +- `structure_failure_llm_call_data_id` +- `structure_failure_trace_event_id` +- `structure_failure_node` +- `structure_failure_prompt_ref` + +### node_2 answer basis + +- `answer_basis_failure_type` +- `answer_basis_llm_call_data_id` +- `answer_basis_trace_event_id` +- `answer_basis_validation_error` +- `answer_basis_raw_text_present` +- `answer_basis_prompt_ref` +- `answer_basis_payload_parse_status` + +## 추가한 테스트 + +- `tests/test_order_119_structure_failed_honesty.py::test_structure_failed_fallback_no_fake_search` +- `tests/test_order_119_structure_failed_honesty.py::test_structure_failed_mentions_actual_search_payload_only_when_present` +- `tests/test_order_119_structure_failed_honesty.py::test_answer_basis_failure_diagnostics_are_recorded_and_rendered` +- `tests/test_order_119_structure_failed_honesty.py::test_selected_recent_memory_payload_boundary_is_not_document` +- `tests/test_order_119_structure_failed_honesty.py::test_bold_duplicate_grounding_block_is_stripped` + +## 검증 + +```powershell +python -m compileall songryeon_core main.py +``` + +결과: 통과. + +```powershell +python -m pytest +``` + +1차 결과: 124초 제한에서 timeout. 출력 없이 종료되어 테스트 실패라기보다 실행 시간 제한 문제로 판단했다. + +재실행: + +```powershell +python -m pytest +``` + +결과: 25 passed in 260.85s. + +```powershell +python main.py smoke-test +``` + +결과: `SMOKE_TEST_OK`. + +추가 확인: + +```powershell +python -m pytest tests/test_order_119_structure_failed_honesty.py +``` + +결과: 5 passed in 0.07s. + +## 비범위 + +- 사용자 질문을 휴리스틱으로 분류해 fallback 의미 답변을 만들지 않았다. +- answer_basis_mode를 code가 의미적으로 대신 고르지 않았다. +- node_4 guard를 약화하지 않았다. +- W/R loop, scheduler, 외부 DB, vector DB, 장기기억 DB를 건드리지 않았다. +- same-turn L reroute 횟수나 node_4 자동 재작성 루프를 늘리지 않았다. diff --git a/Administrative_Reform_1/05_Execution_Records/order_119_structure_failed_honesty_order_draft_2026_06_27_001.md b/Administrative_Reform_1/05_Execution_Records/order_119_structure_failed_honesty_order_draft_2026_06_27_001.md new file mode 100644 index 0000000..597be2a --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_119_structure_failed_honesty_order_draft_2026_06_27_001.md @@ -0,0 +1,60 @@ +# Execution Record: ORDER 119 Structure Failed Honesty Order Draft 2026-06-27 001 + +## 목적 + +ORDER_118 live 테스트 중 발견된 `structure_failed` fallback 정직성 문제와 node_2 answer-basis selector 실패 진단 부족 문제를 후속 발주서로 문서화했다. + +## 관찰한 문제 + +### node_2 answer-basis selector live 실패 + +다음 질문들은 node_2 answer-basis selector가 의미 있는 모드를 고르는지 보기 위한 live 테스트였다. + +```text +지금 smoke 결과 기준으로 뭐가 통과했어? +방금 smoke-test를 실제로 다시 돌린 거야? 아니면 과거 실행기록을 읽은 거야? 둘을 구분해서 답해줘. +``` + +두 경우 모두 최종 답변은 생성됐지만, answer-basis frame은 다음 fallback 상태였다. + +```text +answer_basis_mode = mixed_or_uncertain +generated_by = CODE:FALLBACK +semantic_judgement_status = failed +basis_reason_codes = ["llm_mode_selection_failed"] +``` + +따라서 ORDER_118의 live 성공 여부를 판단하려면 failure type, llm call id, validation error 등 진단 정보가 더 필요하다. + +### structure_failed fallback 오표현 + +다음 질문은 상대/해석 허용 모드 테스트였다. + +```text +지금 송련의 말하기 모드 설계가 너무 빡빡한지, 아니면 적절한지 의견을 말해줘. 단, 이건 네 해석이라는 점을 밝혀줘. +``` + +결과는 `structure_failed`, `trace/data: 0 / 0`, `LLM_REPORTER=not_run`이었다. + +그런데 fallback answer는 내부 문서를 찾은 것처럼 표현했다. + +```text +내부 문서를 찾았어. +검색 결과 payload를 찾지 못해서 근거 문서 요약은 만들지 못했어. +``` + +검색 payload가 없고 trace/data가 0인 상태에서 이 표현은 정직하지 않다. + +## 문서화 변경 + +- `Administrative_Reform_1/04_Orders/ORDER_119_STRUCTURE_FAILED_HONESTY_AND_ANSWER_BASIS_FAILURE_DIAGNOSTICS_V0.md` 추가 +- `Administrative_Reform_1/04_Orders/README.md`의 정식 발주서 범위를 `ORDER_119`까지 확장 +- `ORDER_119` 요약과 링크 추가 + +## 비범위 + +이번 작업은 문서화만 수행했다. + +코드 구현, schema 변경, prompt 변경, smoke/pytest 추가는 하지 않았다. + +현재 작업트리에는 ORDER_112와 ORDER_118 관련 구현 변경이 이미 존재하므로, 이번 기록은 ORDER_119 문서화 범위만 다룬다. diff --git a/Administrative_Reform_1/05_Execution_Records/order_120_tool_use_budget_query_count_diagnostic_implementation_2026_06_27_001.md b/Administrative_Reform_1/05_Execution_Records/order_120_tool_use_budget_query_count_diagnostic_implementation_2026_06_27_001.md new file mode 100644 index 0000000..724d799 --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_120_tool_use_budget_query_count_diagnostic_implementation_2026_06_27_001.md @@ -0,0 +1,120 @@ +# Execution Record: ORDER 120 Tool Use Budget Query Count Diagnostic Implementation 2026-06-27 001 + +## 목적 + +`ToolUseBudgetFrame.query_count must not exceed max_query_attempts` 구조 실패의 실제 원인을 확인하고, 같은 예산 불일치가 다시 생길 때 어느 budget frame, route, L run, count에서 깨졌는지 드러나게 했다. + +## 실제 원인 + +Qwen live 재현에서 강제 L route를 켰을 때 다음 진단이 확인됐다. + +```text +budget_failure_type=query_count_exceeded_max_query_attempts +budget_failure_frame_id=tool_budget:turn_dry_001:0016 +budget_failure_source_data_ids=[L2:revision_query_frame:0008, ...] +budget_failure_query_count=9 +budget_failure_max_query_attempts=8 +``` + +원인은 초기 search query가 아니라 L3 이후 continuation/revision 경로였다. + +`record_l_loop_continuation_decision()`은 query 예산이 소진됐더라도 `read_doc` 예산과 unread candidate가 남아 있으면 `continue`를 허용할 수 있었다. 하지만 현재 구현된 continuation 행동은 unread candidate를 바로 `read_doc`하는 경로가 아니라 새 `L2 revision query`를 만든 뒤 tool attempt를 실행하는 경로뿐이다. 따라서 query 예산 8/8 상태에서도 revision query 0008이 추가 실행되며 `query_count=9`가 만들어졌다. + +## 변경 요약 + +- `BudgetConsistencyError`를 추가해 `ToolUseBudgetFrame` 검증 실패에 구조화된 budget diagnostics를 붙였다. +- `record_tool_use_budget_frame()`이 validator 실패를 조용히 보정하지 않고, frame 값을 기반으로 budget failure field를 만들어 예외에 싣도록 했다. +- `run_qwen_user_turn()`의 `structure_failed` 진단이 `budget_diagnostics`를 top-level response에 병합하도록 했다. +- pretty runtime과 `structure_failed` fallback answer에 budget failure type/count/stage를 표시하도록 했다. +- L continuation 정책을 수정했다. 현재 read_doc 직행 continuation 경로가 없으므로 `remaining_query_attempts <= 0`이면 read_doc 예산이 남아 있어도 `stop_budget_exhausted`로 닫는다. + +## 추가한 budget failure diagnostics 필드 + +- `budget_failure_type` +- `budget_failure_reason` +- `budget_failure_frame_id` +- `budget_failure_source_data_ids` +- `budget_failure_route` +- `budget_failure_l_run_id` +- `budget_failure_query_count` +- `budget_failure_max_query_attempts` +- `budget_failure_tool_calls` +- `budget_failure_max_tool_calls` +- `budget_failure_read_doc_count` +- `budget_failure_max_read_doc` +- `budget_failure_stage` + +## 추가한 테스트 + +- `tests/test_order_120_tool_budget_diagnostics.py::test_tool_use_budget_validator_keeps_query_count_limit` +- `tests/test_order_120_tool_budget_diagnostics.py::test_valid_tool_use_budget_frame_records_successfully` +- `tests/test_order_120_tool_budget_diagnostics.py::test_budget_failure_diagnostics_expose_query_count_mismatch` +- `tests/test_order_120_tool_budget_diagnostics.py::test_structure_failed_renderer_includes_budget_failure_diagnostics` +- `tests/test_order_120_tool_budget_diagnostics.py::test_continuation_stops_when_query_budget_exhausted_even_with_unread_candidates` +- `tests/test_order_120_tool_budget_diagnostics.py::test_live_like_opinion_input_no_budget_query_count_structure_failure` + +## 검증 + +```powershell +python -m pytest tests/test_order_120_tool_budget_diagnostics.py +``` + +결과: 6 passed in 11.69s. + +```powershell +python -m compileall songryeon_core main.py +``` + +결과: 통과. + +```powershell +python -m pytest +``` + +1차 결과: 424초 제한에서 timeout. + +재실행 결과: 31 passed in 628.60s. + +```powershell +python main.py smoke-test +``` + +1차 결과: 424초 제한에서 timeout. + +재실행 결과: `SMOKE_TEST_OK`. + +## live 확인 + +### 재현 + +다음 명령으로 budget failure 진단이 확인됐다. + +```powershell +python main.py qwen-turn "지금 송련의 말하기 모드 설계가 너무 빡빡한지, 아니면 적절한지 의견을 말해줘. 단, 이건 네 해석이라는 점을 밝혀줘." --timeout 240 --pretty --force-l +``` + +결과: + +```text +status=structure_failed +structure_failure_exception=BudgetConsistencyError +budget_failure_type=query_count_exceeded_max_query_attempts +budget_failure_query_count=9 +budget_failure_max_query_attempts=8 +budget_failure_source_data_ids includes L2:revision_query_frame:0008 +``` + +### 사후 재검증 + +continuation 정책 수정 후 같은 live 명령과 축소 budget live 명령을 다시 시도했으나 Qwen 응답 시간이 길어져 각각 304초/244초 timeout으로 종료됐다. 따라서 post-fix Qwen force-L live 완료 여부는 이번 기록에서 확정하지 않는다. + +비강제 live는 수정 전 한 차례 `status=ok`, `route=2`, `answer_basis=LLM:qwen3:14b`로 통과했지만, 수정 후 단독 재시도는 304초 timeout이었다. + +## 유지한 것 + +- `ToolUseBudgetFrame` validator의 `query_count <= max_query_attempts`, `tool_call_count <= max_tool_calls`, `read_doc_count <= max_read_doc_calls` 조건은 유지했다. +- query_count를 max 값으로 조용히 덮어쓰지 않았다. +- router와 answer_basis mode 선택 정책은 건드리지 않았다. +- node_4 guard를 약화하지 않았다. +- W/R loop, scheduler, 외부 DB/vector DB/장기기억 DB를 건드리지 않았다. +- same-turn L reroute 횟수와 node_4 자동 재작성 루프를 늘리지 않았다. diff --git a/Administrative_Reform_1/05_Execution_Records/order_120_tool_use_budget_query_count_diagnostic_order_draft_2026_06_27_001.md b/Administrative_Reform_1/05_Execution_Records/order_120_tool_use_budget_query_count_diagnostic_order_draft_2026_06_27_001.md new file mode 100644 index 0000000..7b76067 --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_120_tool_use_budget_query_count_diagnostic_order_draft_2026_06_27_001.md @@ -0,0 +1,56 @@ +# Execution Record: ORDER 120 Tool Use Budget Query Count Diagnostic Order Draft 2026-06-27 001 + +## 목적 + +ORDER_119 구현 후 live 테스트에서 드러난 `ToolUseBudgetFrame.query_count must not exceed max_query_attempts` 실패를 후속 발주서로 문서화했다. + +## 관찰한 live 실패 + +사용자 입력: + +```text +지금 송련의 말하기 모드 설계가 너무 빡빡한지, 아니면 적절한지 의견을 말해줘. 단, 이건 네 해석이라는 점을 밝혀줘. +``` + +ORDER_119 이후 출력은 더 이상 가짜 검색 fallback을 만들지 않았고, 다음 진단을 정직하게 표시했다. + +```text +상태: structure_failed +trace/data: 0 / 0 +structure_failure_stage: run_dry_turn +structure_failure_node: unknown +structure_failure_exception: ValueError +structure_failure_reason: ToolUseBudgetFrame.query_count must not exceed max_query_attempts +``` + +## 판단 + +ORDER_119는 실패 정직성 측면에서 작동했다. + +새로 드러난 문제는 `ToolUseBudgetFrame`의 query count consistency 문제다. + +이번 문제는 질문을 휴리스틱으로 분류해 우회할 문제가 아니다. + +확인해야 할 핵심: + +```text +1. 왜 query_count가 max_query_attempts를 초과했는가? +2. budget frame 생성 코드가 잘못된 값을 만들었는가? +3. ORDER_112 document context/search budget 변경과 충돌했는가? +4. schema validator는 유지되어야 하는가? +5. 실패 시 budget frame/source/route/L run 진단이 충분히 남는가? +``` + +## 문서화 변경 + +- `Administrative_Reform_1/04_Orders/ORDER_120_TOOL_USE_BUDGET_QUERY_COUNT_CONSISTENCY_DIAGNOSTIC_V0.md` 추가 +- `Administrative_Reform_1/04_Orders/README.md`의 정식 발주서 범위를 `ORDER_120`까지 확장 +- `ORDER_120` 요약과 링크 추가 + +## 비범위 + +이번 작업은 문서화만 수행했다. + +코드 구현, schema 변경, runtime 변경, prompt 변경, smoke/pytest 추가는 하지 않았다. + +현재 작업트리에는 ORDER_112, ORDER_118, ORDER_119 관련 구현 변경이 이미 존재하므로, 이번 기록은 ORDER_120 문서화 범위만 다룬다. diff --git a/Administrative_Reform_1/05_Execution_Records/order_121_node2_evidence_source_alignment_and_l3_failure_attitude_2026_06_28_001.md b/Administrative_Reform_1/05_Execution_Records/order_121_node2_evidence_source_alignment_and_l3_failure_attitude_2026_06_28_001.md new file mode 100644 index 0000000..7968cee --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_121_node2_evidence_source_alignment_and_l3_failure_attitude_2026_06_28_001.md @@ -0,0 +1,36 @@ +# ORDER_121 실행 기록 - node_2 evidence source 정렬과 L3 실패 태도 전달 + +## 작업 일시 + +2026-06-28 + +## 변경 요약 + +- `node_2 answer_basis` LLM 입력에 `available_evidence_sources` 표를 추가했다. +- `evidence_roles.source_data_id` 검증 기준을 `available_evidence_sources`와 같은 ID 목록으로 맞췄다. +- `Node2AnswerBasisFrame` validator는 약화하지 않았다. +- `Node3InputBriefFrame`에 L loop 반환 요약 필드를 추가했다. +- `node_3` grounding block과 LLM payload에 L 검색 목표 상태/실패/예산소진 신호를 전달했다. +- `node_4` prompt와 검사 목록에 L loop 실패/예산소진 신호를 검색 성공처럼 말하는지 확인하는 항목을 추가했다. + +## 의도적으로 하지 않은 것 + +- L revision 검색 전략을 바꾸지 않았다. +- search 반복 억제, query novelty 판단, unread candidate direct read_doc 경로를 열지 않았다. +- W/R loop, scheduler, 외부 DB, 장기기억 DB, same-turn L reroute 횟수를 건드리지 않았다. + +## 검증 + +- `python -m compileall songryeon_core main.py` 통과. +- `python -m pytest tests/test_order_121_answer_basis_and_l3_attitude.py` 통과: 4 passed. +- `python -m pytest tests/test_order_118_answer_basis.py tests/test_order_119_structure_failed_honesty.py tests/test_order_120_tool_budget_diagnostics.py` 통과: 17 passed. +- `python -m pytest tests/test_smoke_baseline.py -q` 통과: 1 passed. +- `python -m pytest -q` 통과: 35 passed. +- `python main.py smoke-test` 통과: `SMOKE_TEST_OK`. +- `git diff --check` 통과. + +## 남은 위험 + +- L 검색 뺑뺑이 자체는 아직 해결하지 않았다. +- 이번 작업은 L3 실패/예산소진 신호가 downstream 답변 태도에 숨지 않게 하는 잠금이다. +- 실제 Qwen live에서 `node_2 answer_basis`가 새 `available_evidence_sources`를 안정적으로 따르는지는 별도 live 테스트가 필요하다. diff --git a/Administrative_Reform_1/05_Execution_Records/order_122_l_revision_unread_candidate_read_path_2026_06_28_001.md b/Administrative_Reform_1/05_Execution_Records/order_122_l_revision_unread_candidate_read_path_2026_06_28_001.md new file mode 100644 index 0000000..0dd0e76 --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_122_l_revision_unread_candidate_read_path_2026_06_28_001.md @@ -0,0 +1,32 @@ +# ORDER_122 L Revision Unread Candidate Read Path Implementation 2026-06-28 001 + +## 작업 요약 + +L revision 흐름에서 `search_docs` 후보를 이미 확보했지만 `query` 예산이 소진된 경우, unread candidate를 `read_doc`으로 읽을 수 있는 경로를 열었다. + +이번 변경은 예산을 늘리지 않았다. `read_doc`은 `query_count`를 증가시키지 않고, `read_doc_ids/read_doc_count`만 증가하도록 분리했다. + +## 핵심 변경 + +- `L2RevisionInputFrame`에 `unread_candidate_doc_ids`를 추가해 revision L2가 읽을 수 있는 후보 ID를 절대정보로 받게 했다. +- 일반 L2는 기존처럼 `search_docs` / `read_artifact`만 허용하고, revision L2에서만 `read_doc` target을 허용했다. +- revision `read_doc` 후보는 `unread_candidate_doc_ids` 안의 정확한 `doc_id`일 때만 통과한다. +- `remaining_query_attempts=0`이면 revision plan은 `read_doc` 후보만 통과한다. +- continuation controller는 query 예산이 0이어도 unread candidate와 read budget이 있으면 `continue`로 열어 둔다. +- revision tool attempt는 `read_doc`을 실제 실행하고, `search_docs`만 `executed_queries/query_count`에 기록한다. + +## 검증 + +- `python -m compileall songryeon_core main.py` 통과 +- `python -m pytest tests/test_order_122_l_revision_read_doc_path.py -q` 통과: 4 passed +- `python -m pytest tests/test_order_120_tool_budget_diagnostics.py tests/test_order_121_answer_basis_and_l3_attitude.py -q` 통과: 11 passed +- `python -m pytest -q` 통과: 40 passed +- `python main.py smoke-test` 통과: `SMOKE_TEST_OK` +- `python main.py qwen-turn "...ORDER_100...ORDER_110..." --timeout 180 --pretty` 통과: `status=ok` + - 이 live 입력은 explicit artifact/context pack 경로로 L3가 바로 `achieved` 처리되어 ORDER_122 revision read path 자체를 직접 밟지는 않았다. + +## 남은 위험 + +- 실제 Qwen live에서는 revision L2가 unread candidate 중 어떤 문서를 읽을지 여전히 LLM 판단에 달려 있다. +- 이번 작업은 `node_3`가 많이 받은 문서를 정리/압축하는 문제를 해결하지 않는다. +- query novelty 판단, 검색 반복 억제, 문서 context pack 정리는 다음 발주 후보로 남긴다. diff --git a/Administrative_Reform_1/05_Execution_Records/order_123_actual_read_doc_vs_context_pack_count_boundary_2026_06_28_001.md b/Administrative_Reform_1/05_Execution_Records/order_123_actual_read_doc_vs_context_pack_count_boundary_2026_06_28_001.md new file mode 100644 index 0000000..7d5ba5b --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_123_actual_read_doc_vs_context_pack_count_boundary_2026_06_28_001.md @@ -0,0 +1,64 @@ +# ORDER_123 Actual Read Doc Vs Context Pack Count Boundary 실행 기록 2026-06-28 001 + +## 배경 + +live 테스트에서 `L 도구 예산: read_doc=2/10`과 `document_context_pack included=10`이 동시에 나타났다. +이후 node_3 최종 답변이 context pack 공급 문서 수를 실제 `read_doc` 도구 읽기 수처럼 말할 위험이 확인되었다. + +## 구현 판단 + +이번 수정은 문장 키워드 탐지 휴리스틱을 추가하지 않았다. +대신 `Node3InputBriefFrame` 안에서 다음 절대 count를 구조적으로 분리했다. + +- `actual_tool_read_doc_count`: L return summary 또는 실제 도구 extract record 기준의 도구 원문 읽기 수 +- `actual_tool_read_doc_documents`: 실제 도구 원문 읽기 문서명 목록 +- `supplied_document_context_count`: node_3에게 본문 context로 공급된 문서 수 + +## 변경 요약 + +- `Node3InputBriefFrame`에 실제 read_doc 계열 도구 읽기 count와 node_3 공급 context count를 별도 필드로 추가했다. +- node_3 grounding block을 다음처럼 분리했다. + - 실제 read_doc 도구 원문 읽기 + - node_3 공급 문서 context + - 검색 후보 문서 + - 현재 턴 실행 순서 자료 +- node_3 LLM payload에 `actual_tool_read_doc`와 `supplied_document_context`를 분리해서 넣었다. +- 기존 `read_documents` payload는 호환 alias로 유지하되, `legacy_alias=supplied_document_context`를 붙였다. +- node_3 prompt는 실제 read_doc 수를 말할 때 `actual_tool_read_doc.count`만 쓰도록 경계를 강화했다. +- terminal runtime view는 node_3 input brief를 `actual_read_doc`와 `supplied_contexts`로 분리 표시한다. +- node_4 grounding count guard는 새 라벨 기준으로 count mismatch를 검사한다. + +## 테스트 + +추가 pytest: + +- `tests/test_order_123_actual_read_doc_vs_context_pack_count.py` + +확인한 사항: + +- actual `read_doc` count와 supplied context count가 서로 다르게 보존된다. +- grounding block이 두 count를 다른 라벨로 출력한다. +- node_3 LLM payload에서 `actual_tool_read_doc.count`와 `supplied_document_context.count`가 분리된다. +- `supplied_document_context_count`가 `read_documents` 길이와 다르면 schema validation이 실패한다. +- node_4 count guard는 context count가 아니라 actual read_doc count를 기준으로 mismatch를 잡는다. + +검증 결과: + +- `python -m compileall songryeon_core main.py`: 통과 +- `python -m pytest`: `45 passed` +- `python main.py smoke-test`: `SMOKE_TEST_OK` +- live qwen-turn 재현 테스트: `status=ok` + +live qwen-turn 확인값: + +- runtime `node_3 input brief`: `actual_read_doc=2 / supplied_contexts=10 / search_candidates=10` +- 최종 grounding block: `실제 read_doc 도구 원문 읽기: 2개` +- 최종 grounding block: `node_3 공급 문서 context: 10개` +- 본문 `read_doc 수`: 2개로 출력 +- 본문 `node_3 공급 문서 context 수`: 10개로 출력 + +## 제외한 것 + +- `read_doc`라는 단어가 포함된 문장을 탐지해 막는 휴리스틱 guard는 추가하지 않았다. +- node_3 문서 context 압축/정렬은 하지 않았다. +- W/R loop, scheduler, 외부 DB, 장기기억 DB, same-turn L reroute 횟수는 변경하지 않았다. diff --git a/Administrative_Reform_1/05_Execution_Records/order_124_node0_post_l_document_material_packet_implementation_2026_06_28_001.md b/Administrative_Reform_1/05_Execution_Records/order_124_node0_post_l_document_material_packet_implementation_2026_06_28_001.md new file mode 100644 index 0000000..78e2fa3 --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_124_node0_post_l_document_material_packet_implementation_2026_06_28_001.md @@ -0,0 +1,70 @@ +# ORDER_124 Node0 Post-L Document Material Packet 구현 기록 2026-06-28 001 + +## 구현 요약 + +L 이후 node_0이 문서별 장부를 만드는 `Node0DocumentMaterialPacketFrame`을 추가했다. + +이 frame은 의미 요약을 만들지 않고, 기존 구조화 record에서 다음 절대정보만 복사/정렬한다. + +- 검색 후보 여부 +- 실제 `read_doc` / `read_artifact` 도구 원문 읽기 여부 +- node_3 supplied document context 포함 여부 +- document context pack 제외 여부 +- unread candidate 여부 +- 문서별 rank/count/source id + +## 핵심 변경 + +- `songryeon_core/core/schemas.py` + - `Node0DocumentMaterialItem` + - `Node0DocumentMaterialPacketFrame` + - `validate_node0_document_material_packet_frame` +- `songryeon_core/nodes/node_0_memory_supplier.py` + - `record_node0_document_material_packet` + - `build_node0_document_material_packet_frame` + - L loop return summary 기록 시 document material packet도 함께 기록 + - loop return memory packet에 `document_material_packet_status` item 추가 +- `songryeon_core/runtime/dry_run.py` + - node_2 input source 범위에 document material frame id 추가 +- `songryeon_core/nodes/node_2_handoff.py` + - route=2 handoff와 node_3 input brief가 document material packet을 읽고 보존 + - node_3 LLM payload에 raw internal id 없이 문서명/역할/rank/count만 전달 +- `songryeon_core/runtime/terminal_view.py` + - runtime에 `node_0 document material packet` 섹션 표시 + - node_3 input brief에 `document_materials` count 표시 +- `songryeon_core/prompts/node_3_reporter_v0.md` + - document material packet은 의미 요약이 아니라 code-built document ledger임을 명시 + +## 테스트 + +추가 pytest: + +- `tests/test_order_124_node0_document_material_packet.py` + +확인한 사항: + +- 검색 후보 3개, 실제 read_doc 2개, supplied context 3개 상황에서 문서별 item 3개가 만들어진다. +- unread candidate는 search candidate이면서 actual read가 아닌 문서만 잡힌다. +- node_0 document material packet은 `semantic_judgement_status=not_run`이다. +- route=2 handoff가 document material packet id/count를 보존한다. +- node_3 brief와 LLM payload가 document material ledger를 받는다. +- node_3 LLM payload의 material items에는 raw `source_data_ids`가 직접 들어가지 않는다. + +## 검증 결과 + +- `python -m compileall songryeon_core main.py`: 통과 +- `python -m pytest`: `48 passed` +- `python main.py smoke-test`: `SMOKE_TEST_OK` +- `python main.py fake-turn "문서 장부 확인: 검색 후보, 실제 read_doc, 공급 context, unread 후보를 구분해줘" --pretty`: 통과 + +fake-turn 확인값: + +- runtime `node_0 document material packet`: `items=12 / search_candidates=12 / actual_read=2 / supplied_contexts=11 / unread_candidates=10` +- runtime `node_3 input brief`: `document_materials=12` +- node_3 brief 내부 material packet: `items=12 / unread_candidates=10` + +## 제외한 것 + +- L3 문서 요약은 구현하지 않았다. +- node_0은 문서 중요도/관련성/요약 의미 판단을 만들지 않는다. +- 장기기억 DB, vector DB, scheduler, W/R loop, node_5 압축, node_4 자동 재작성 루프는 열지 않았다. diff --git a/Administrative_Reform_1/05_Execution_Records/order_124_node0_post_l_document_material_packet_order_draft_2026_06_28_001.md b/Administrative_Reform_1/05_Execution_Records/order_124_node0_post_l_document_material_packet_order_draft_2026_06_28_001.md new file mode 100644 index 0000000..af4338b --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_124_node0_post_l_document_material_packet_order_draft_2026_06_28_001.md @@ -0,0 +1,26 @@ +# ORDER_124 Node0 Post-L Document Material Packet 발주서 초안 기록 2026-06-28 001 + +## 작성 이유 + +ORDER_123 이후 실제 `read_doc` 수와 node_3 공급 context 수는 분리되었다. +다음 병목은 문서 재료가 여러 frame에 흩어져 있어 node_2/node_3가 문서별 역할을 한눈에 보기 어렵다는 점이다. + +사용자와 논의한 결론은 L 이후 node_0이 문서 장부를 정리하는 방향이다. +단, node_0은 의미 요약이나 중요도 판단을 하지 않고 절대정보 좌표와 역할만 정리한다. + +## 생성 문서 + +- `Administrative_Reform_1/04_Orders/ORDER_124_NODE0_POST_L_DOCUMENT_MATERIAL_PACKET_V0.md` + +## 상태 + +- 상태: 발주서 초안 작성 +- 구현: 아직 하지 않음 +- 테스트: 아직 하지 않음 + +## 핵심 경계 + +- node_0은 문서 사서/장부 담당이다. +- node_0은 문서 내용을 요약하지 않는다. +- node_0은 관련성/중요도 의미 판단을 하지 않는다. +- L3 문서 요약은 ORDER_125 후보로 분리한다. diff --git a/Administrative_Reform_1/05_Execution_Records/order_125_l3_per_document_summary_frame_design_order_draft_2026_06_28_001.md b/Administrative_Reform_1/05_Execution_Records/order_125_l3_per_document_summary_frame_design_order_draft_2026_06_28_001.md new file mode 100644 index 0000000..a135eef --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_125_l3_per_document_summary_frame_design_order_draft_2026_06_28_001.md @@ -0,0 +1,18 @@ +# ORDER_125 L3 Per-Document Summary Frame Design 발주서 초안 기록 2026-06-28 001 + +## 작성 이유 + +사용자가 L3가 읽은 문서들을 그때마다 요약하게 할 수 있는지 물었다. +감사 결과 L3는 이미 `read_document_previews`를 목표 달성 판단 재료로 받지만, 공식 문서 요약 frame은 아직 없다. + +요약은 LLM 의미 생성이므로 ORDER_124의 절대정보 문서 장부와 분리해야 한다. + +## 생성 문서 + +- `Administrative_Reform_1/04_Orders/ORDER_125_L3_PER_DOCUMENT_SUMMARY_FRAME_DESIGN_V0.md` + +## 상태 + +- 상태: 발주서 초안 작성 +- 구현: 하지 않음 +- 이유: L3 문서 요약은 relative/mixed 정보 경계를 열기 때문에 ORDER_124 구현 뒤 별도 결재가 필요하다. diff --git a/Administrative_Reform_1/05_Execution_Records/order_125_l3_per_document_summary_frame_implementation_2026_06_29_001.md b/Administrative_Reform_1/05_Execution_Records/order_125_l3_per_document_summary_frame_implementation_2026_06_29_001.md new file mode 100644 index 0000000..b0f6591 --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_125_l3_per_document_summary_frame_implementation_2026_06_29_001.md @@ -0,0 +1,54 @@ +# ORDER_125 L3 Per-Document Summary Frame Implementation 2026-06-29 001 + +## 작업 범위 + +- `ORDER_125`를 설계 전용 문서에서 구현 발주로 보강했다. +- L3가 실제 document extract record마다 문서별 요약 frame을 만들 수 있게 했다. +- 한 frame 안에 두 종류의 요약을 분리했다. + - `plain_document_summary`: 문서 하나에 직접 대응하는 `relative/direct_record/one_document_to_one_summary` + - `task_relevant_summary`: 현재 질문/L1 목표/문서 원문 source bundle에 근거한 `mixed/source_bundle/one_document_plus_task_context` +- code는 요약 문장을 생성하지 않고, LLM payload/schema 검증과 source ID 연결만 수행한다. + +## 주요 변경 파일 + +- `Administrative_Reform_1/04_Orders/ORDER_125_L3_PER_DOCUMENT_SUMMARY_FRAME_DESIGN_V0.md` +- `Administrative_Reform_1/04_Orders/README.md` +- `songryeon_core/core/schemas.py` +- `songryeon_core/nodes/l3_result_keeper.py` +- `songryeon_core/nodes/node_2_handoff.py` +- `songryeon_core/prompts/l3_per_document_summary_v0.md` +- `songryeon_core/prompts/node_3_reporter_v0.md` +- `songryeon_core/loops/l_loop.py` +- `songryeon_core/runtime/terminal_view.py` +- `tests/test_order_125_l3_per_document_summary_frame.py` + +## 검증한 경계 + +- 담백 요약이 `mixed`로 찍히면 schema validation이 실패한다. +- 상황 요약은 source bundle data ID가 2개 이상이어야 한다. +- L3 result keeper가 실제 read document extract record에서 summary frame을 기록한다. +- node_3 brief와 LLM payload는 L3 summary material을 받되 raw internal source ID를 payload item에 노출하지 않는다. +- L3 summary frame은 원문 context를 대체하지 않고 별도 의미 재료로만 전달된다. + +## 검증 명령 + +```powershell +python -m compileall songryeon_core main.py +python -m pytest +python main.py smoke-test +``` + +## 결과 + +- `python -m compileall songryeon_core main.py`: 통과 +- `python -m pytest`: `63 passed` +- `python main.py smoke-test`: `SMOKE_TEST_OK` + +## 제외한 것 + +- node_0 요약 기능은 만들지 않았다. +- code가 문서 의미를 요약하지 않았다. +- 여러 문서 종합 요약은 만들지 않았다. +- L3 요약으로 node_3 원문 context를 자동 대체하지 않았다. +- answer_basis_mode에 따른 원문/요약 공급량 조절 정책은 열지 않았다. +- W/R loop, scheduler, vector DB, 외부 DB, 장기기억 DB, node_5 기억 압축은 건드리지 않았다. diff --git a/Administrative_Reform_1/05_Execution_Records/order_126_runtime_all_document_extract_display_2026_06_28_001.md b/Administrative_Reform_1/05_Execution_Records/order_126_runtime_all_document_extract_display_2026_06_28_001.md new file mode 100644 index 0000000..24eb2ca --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_126_runtime_all_document_extract_display_2026_06_28_001.md @@ -0,0 +1,40 @@ +# ORDER 126 Runtime All Document Extract Display 실행 기록 + +일시: 2026-06-28 + +## 배경 + +ORDER_124 live qwen 테스트에서 `node_0 document material packet`과 `node_3 input brief`는 실제 원문 읽기 수를 `actual_read=2`로 보여줬다. + +그러나 terminal top-level `read_doc [TOOL_RESULT:DOCUMENT_EXTRACT]` 표시가 최신 document extract record 1개만 보여, 사람이 runtime 화면만 볼 때 실제 읽은 문서 수를 헷갈릴 수 있었다. + +## 변경 + +- `songryeon_core/runtime/terminal_view.py` + - `_latest_document_extract_record()` 1개 표시 대신 `_document_extract_records_for_display()` 목록 표시를 추가했다. + - `tool_result:read_doc`과 `tool_result:read_artifact` record를 모두 runtime view에 표시한다. + - 최신 L run-scoped document extract record가 있으면 최신 run record를 우선 표시한다. + - run-scoped `read_artifact`도 도구명을 `read_artifact`로 표시하도록 `_document_extract_tool_name()`을 추가했다. + +- `tests/test_order_126_runtime_document_extract_display.py` + - 여러 document extract record가 모두 표시되는지 확인한다. + - 최신 L run-scoped document extract가 있으면 과거 run record가 대표 표시에서 제외되는지 확인한다. + +## 하지 않은 것 + +- L 검색 전략은 바꾸지 않았다. +- `read_doc` 예산은 바꾸지 않았다. +- node_3 문서 요약은 추가하지 않았다. +- node_4 guard는 약화하지 않았다. +- W/R loop, scheduler, 외부 DB, 장기기억 DB는 건드리지 않았다. + +## 검증 + +- `python -m compileall songryeon_core main.py` + - 통과 +- `python -m pytest tests/test_order_126_runtime_document_extract_display.py` + - 2 passed +- `python -m pytest` + - 50 passed +- `python main.py smoke-test` + - `SMOKE_TEST_OK` diff --git a/Administrative_Reform_1/05_Execution_Records/order_127_revision_document_extract_count_alignment_2026_06_28_001.md b/Administrative_Reform_1/05_Execution_Records/order_127_revision_document_extract_count_alignment_2026_06_28_001.md new file mode 100644 index 0000000..da5ee23 --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_127_revision_document_extract_count_alignment_2026_06_28_001.md @@ -0,0 +1,50 @@ +# ORDER 127 Revision Document Extract Count Alignment 실행 기록 + +일시: 2026-06-28 + +## 배경 + +ORDER_126 live 테스트에서 terminal runtime view는 실제 `tool_result:read_doc` record 7개를 모두 표시했다. + +그러나 node_0 material packet과 node_3 input brief는 `actual_read=3` / `actual_read_doc=3`으로 표시했다. route=2 handoff와 tool budget은 7개 record를 보고 있었기 때문에, revision read_doc 결과 일부가 downstream count 장부에 합류하지 않는 문제가 드러났다. + +## 원인 + +- `build_l_loop_return_summary_frame()`는 L3/budget의 `read_doc_ids`를 우선 기준으로 삼았다. +- `build_node0_document_material_packet_frame()`도 return summary/L3의 `read_doc_ids`에 있는 문서만 `actual_tool_read_doc`으로 표시했다. +- node_3 brief의 actual read helper도 return summary count가 있으면 그 값을 우선 사용했다. + +따라서 revision에서 추가 실행된 `tool_result:read_doc` / `tool_result:read_artifact` record가 DataStore에는 남아도, L3/return summary의 `read_doc_ids`에 반영되지 않으면 node_0 material packet과 node_3 brief count에서 빠질 수 있었다. + +## 변경 + +- `songryeon_core/nodes/node_0_memory_supplier.py` + - 실제 document extract record를 읽는 `_document_extract_sources()` / `_document_extract_doc_ids()` helper를 추가했다. + - `build_l_loop_return_summary_frame()`가 L3/budget의 `read_doc_ids`와 실제 document extract record의 `doc_id`를 병합한다. + - `build_node0_document_material_packet_frame()`가 return summary의 `read_doc_ids`에 빠진 document extract record도 `actual_tool_read_doc` 역할로 표시한다. + +- `songryeon_core/nodes/node_2_handoff.py` + - node_3 actual read helper가 return summary의 문서명과 DataStore document extract record 문서명을 병합한다. + - actual read count는 병합된 문서명이 있으면 그 길이를 우선 사용한다. + +- `tests/test_order_127_revision_document_extract_count_alignment.py` + - stale L3/return summary + 추가 revision read_doc record 상황을 재현하는 pytest를 추가했다. + +## 하지 않은 것 + +- L 검색 전략은 바꾸지 않았다. +- `read_doc` 예산은 바꾸지 않았다. +- node_3 문서별 의미 요약은 추가하지 않았다. +- node_4 guard는 약화하지 않았다. +- W/R loop, scheduler, 외부 DB, 장기기억 DB는 건드리지 않았다. + +## 검증 + +- `python -m compileall songryeon_core main.py` + - 통과 +- `python -m pytest tests/test_order_127_revision_document_extract_count_alignment.py` + - 3 passed +- `python -m pytest` + - 53 passed +- `python main.py smoke-test` + - `SMOKE_TEST_OK` diff --git a/Administrative_Reform_1/05_Execution_Records/order_128_node3_actual_read_doc_identity_key_2026_06_28_001.md b/Administrative_Reform_1/05_Execution_Records/order_128_node3_actual_read_doc_identity_key_2026_06_28_001.md new file mode 100644 index 0000000..0c115da --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_128_node3_actual_read_doc_identity_key_2026_06_28_001.md @@ -0,0 +1,49 @@ +# ORDER 128 Node3 Actual Read Doc Identity Key 실행 기록 + +일시: 2026-06-28 + +## 배경 + +ORDER_127 이후 runtime, L budget, route=2 handoff, node_0 material packet, L return summary는 실제 read_doc count를 7개로 맞췄다. + +하지만 live qwen 테스트에서 node_3 input brief와 최종 grounding count는 6개로 줄었다. + +## 원인 + +node_3 actual read document 목록을 만들 때 document extract record의 `doc_id`를 파일명으로 줄인 뒤 중복 제거했다. + +예를 들어 아래 두 문서는 서로 다른 문서다. + +- `04_Orders/README.md` +- `05_Execution_Records/README.md` + +하지만 파일명만 쓰면 둘 다 `README.md`가 되어 하나로 합쳐질 수 있었다. + +## 변경 + +- `songryeon_core/nodes/node_2_handoff.py` + - `_actual_tool_read_doc_documents()`가 중복 제거 기준을 파일명 대신 `doc_id` 또는 document extract record identity로 잡게 했다. + - 표시 label은 파일명이 유일하면 짧은 파일명을 쓰고, 같은 파일명이 여러 `doc_id`에 있으면 경로 포함 label을 쓴다. + - `_read_doc_display_labels()` helper를 추가했다. + +- `tests/test_order_128_node3_actual_read_doc_identity_key.py` + - `04_Orders/README.md`와 `05_Execution_Records/README.md`가 서로 다른 실제 read_doc 문서로 2개 count 되는지 확인한다. + +## 하지 않은 것 + +- node_3 LLM prompt는 바꾸지 않았다. +- grounding block 정책은 바꾸지 않았다. +- node_4 guard는 약화하지 않았다. +- L 검색 전략과 read_doc 예산은 바꾸지 않았다. +- W/R loop, scheduler, 외부 DB, 장기기억 DB는 건드리지 않았다. + +## 검증 + +- `python -m compileall songryeon_core main.py` + - 통과 +- `python -m pytest tests/test_order_127_revision_document_extract_count_alignment.py tests/test_order_128_node3_actual_read_doc_identity_key.py` + - 4 passed +- `python -m pytest` + - 54 passed +- `python main.py smoke-test` + - `SMOKE_TEST_OK` diff --git a/Administrative_Reform_1/05_Execution_Records/order_129_search_candidate_count_basis_audit_2026_06_28_001.md b/Administrative_Reform_1/05_Execution_Records/order_129_search_candidate_count_basis_audit_2026_06_28_001.md new file mode 100644 index 0000000..6e79630 --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_129_search_candidate_count_basis_audit_2026_06_28_001.md @@ -0,0 +1,251 @@ +# order_129_search_candidate_count_basis_audit_2026_06_28_001 + +작성일: 2026-06-28 +관련 발주서: `Administrative_Reform_1/04_Orders/ORDER_129_SEARCH_CANDIDATE_COUNT_BASIS_AUDIT_V0.md` + +## 1. 감사 범위 + +이번 기록은 구현 없이 search candidate count 기준만 감사한다. + +확인한 파일: + +- `songryeon_core/nodes/node_0_memory_supplier.py` +- `songryeon_core/nodes/node_2_handoff.py` +- `songryeon_core/nodes/l3_result_keeper.py` +- `songryeon_core/runtime/terminal_view.py` +- `songryeon_core/core/schemas.py` +- `Administrative_Reform_1/04_Orders/ORDER_124_NODE0_POST_L_DOCUMENT_MATERIAL_PACKET_V0.md` +- `Administrative_Reform_1/04_Orders/ORDER_128_NODE3_ACTUAL_READ_DOC_IDENTITY_KEY_V0.md` + +## 2. 결론 요약 + +`read_doc` count 쪽은 ORDER_128 이후 doc_id identity 기준으로 많이 정렬됐다. + +반면 `search_candidate_count`는 현재 같은 이름 아래 두 가지 범위가 섞일 수 있다. + +- node_0 material packet / L return summary: 최신 또는 최종 L3 achievement/return summary가 들고 온 `search_result_doc_ids` 중심 +- node_3 input brief: namespace 안의 모든 `node_output:L3...preserved_info_frame` record 후보를 훑은 누적 후보 중심 + +따라서 revision이 많은 턴에서는 node_0은 `search_candidates=7`, node_3 brief는 `search_candidates=44`처럼 다르게 보일 수 있다. + +이것은 단순히 "숫자가 틀렸다"기보다, 같은 label이 서로 다른 scope를 가리키는 상태에 가깝다. + +## 3. 항목별 감사 + +### 3.1 node_0 material packet + +파일 위치: + +- `songryeon_core/nodes/node_0_memory_supplier.py` + +현재 count source: + +- `build_node0_document_material_packet_frame()`은 `return_summary_payload.get("search_result_doc_ids")`를 먼저 사용한다. +- 없으면 최신 L3 payload의 `search_result_doc_ids`를 fallback으로 사용한다. + +현재 identity 기준: + +- `item_map`의 key는 `doc_id`다. +- filename이 아니라 전체 `doc_id` 기준으로 material item을 만든다. + +실제로 의미하는 candidate scope: + +- 최종 L return summary 또는 최신 L3 achievement가 보존한 검색 후보 문서 집합에 가깝다. +- 모든 revision preserved frame의 누적 후보라기보다는 최종 요약 후보에 가깝다. + +문제 여부: + +- 자체 기준은 비교적 안전하다. +- 다만 사용자-facing label이 node_3의 `search_candidate_count`와 같으면 혼동이 생긴다. + +수정 위험도: + +- 낮음에서 중간. +- node_0 자체보다 downstream label/field 분리가 더 중요하다. + +바로 고쳐도 되는지: + +- 구현 전 설계 결재 권장. +- `final_search_candidate_count`라는 명시적 이름으로 분리하는 것이 안전하다. + +### 3.2 L loop return summary + +파일 위치: + +- `songryeon_core/nodes/node_0_memory_supplier.py` + +현재 count source: + +- `build_l_loop_return_summary_frame()`은 최신 L3 achievement payload의 `search_result_doc_ids` 길이를 사용한다. +- 없으면 `candidate_count`를 fallback으로 사용한다. + +현재 identity 기준: + +- `search_result_doc_ids` list에 들어온 doc_id 기준이다. + +실제로 의미하는 candidate scope: + +- 최신 L3 achievement 기준의 검색 후보 수다. + +문제 여부: + +- return summary 자체는 "최신/최종 L3 기준"으로 보면 일관적이다. +- 하지만 accumulated candidate와 구분하는 field name은 없다. + +수정 위험도: + +- 중간. +- L3 achievement schema와 node_0 material packet, terminal label이 함께 움직여야 한다. + +바로 고쳐도 되는지: + +- 설계 결재 권장. +- 최소 구현은 기존 field를 유지하고 새 field를 추가하는 방향이 안전하다. + +### 3.3 node_3 input brief + +파일 위치: + +- `songryeon_core/nodes/node_2_handoff.py` + +현재 count source: + +- `_search_candidate_documents()`가 DataStore 전체 record 중 namespace에 속한 `node_output:L3...preserved_info_frame`을 모두 훑는다. +- 각 preserved frame의 `candidates`에서 `doc_id`를 꺼낸다. + +현재 identity 기준: + +- `doc_id`를 `_document_name(doc_id)`로 바꾼 뒤 `_unique_strings(names)`로 중복 제거한다. +- 즉 count identity가 doc_id가 아니라 표시용 document name이다. + +실제로 의미하는 candidate scope: + +- initial L3 preserved frame과 revision L3 preserved frame을 모두 포함한 누적 후보 문서명 집합에 가깝다. + +문제 여부: + +- revision이 많으면 node_0 material packet보다 크게 나올 수 있다. +- 같은 파일명 다른 경로 문서는 하나로 접힐 수 있다. +- ORDER_128에서 actual read_doc에 대해 해결한 identity 문제가 search candidate 쪽에는 아직 남아 있다. + +수정 위험도: + +- 중간. +- node_3 prompt, final grounding block, node_4 count guard와 연결되어 있어 label 변경이 필요하다. + +바로 고쳐도 되는지: + +- 설계 결재 권장. +- `accumulated_search_candidate_count`와 `final_search_candidate_count`를 분리할지 먼저 결정해야 한다. + +### 3.4 terminal/runtime display + +파일 위치: + +- `songryeon_core/runtime/terminal_view.py` + +현재 count source: + +- node_3 input brief 표시에서는 `node3_brief["search_candidate_documents"]` list 길이를 세어 `search_candidates`로 출력한다. + +현재 identity 기준: + +- node_3 brief가 만든 document name list 기준이다. + +실제로 의미하는 candidate scope: + +- node_3 brief의 누적 preserved frame 기반 후보 문서명 수다. + +문제 여부: + +- terminal에 node_0 material packet의 `search_candidate_count`도 따로 보이면, 같은 `search_candidates` label 아래 다른 숫자가 같이 나타날 수 있다. + +수정 위험도: + +- 낮음. +- 표시 label만 바꾸는 것은 쉽지만, schema 의미 정리 없이 표시만 바꾸면 근본 해결은 아니다. + +바로 고쳐도 되는지: + +- 후속 구현에서 schema field와 함께 고치는 것이 낫다. + +### 3.5 node_3 final grounding block + +파일 위치: + +- `songryeon_core/nodes/node_3_reporter.py` + +현재 count source: + +- `brief_frame.search_candidate_count` + +현재 identity 기준: + +- node_3 input brief가 채운 count 기준이다. + +실제로 의미하는 candidate scope: + +- 현재 구현상 누적 L3 preserved 후보 문서명 count에 가깝다. + +문제 여부: + +- 사용자는 이 숫자를 node_0 material packet의 최종 검색 후보 수와 같은 뜻으로 이해할 수 있다. +- final answer에서 `검색 후보 문서: 44개`가 나오면 "최종 후보가 44개였다"로 오해될 수 있다. + +수정 위험도: + +- 중간. +- node_4 grounding count guard가 이 count를 비교하므로, field 추가/label 변경 시 test 보강이 필요하다. + +바로 고쳐도 되는지: + +- 설계 결재 후 후속 발주에서 고치는 것이 안전하다. + +## 4. 원인 정리 + +원인은 크게 두 가지다. + +1. scope 불일치 + - node_0은 최종/최신 L3 summary 계열을 본다. + - node_3는 모든 L3 preserved frame 후보를 본다. + +2. identity 불일치 + - node_0은 doc_id 기준이다. + - node_3 search candidate list는 document name 기준으로 접는다. + +## 5. 추천 후속 작업 + +후속 구현은 L3 per-document summary보다 먼저 하는 것을 권장한다. + +권장 방향: + +- 기존 `search_candidate_count`의 의미를 바로 바꾸지 말고, 새 field를 추가한다. +- 후보 field: + - `final_search_candidate_count` + - `final_search_candidate_documents` + - `accumulated_search_candidate_count` + - `accumulated_search_candidate_documents` +- node_0 material packet은 final/canonical candidate source로 유지한다. +- node_3 brief는 누적 후보를 보존하되, final grounding block에는 label을 명확히 한다. +- search candidate identity는 doc_id 기준으로 세고, 사용자 표시용 label은 별도 생성한다. + +## 6. 이번에 하지 않은 것 + +- code logic 수정 없음 +- schema field 추가 없음 +- L3 per-document summary 구현 없음 +- prompt 수정 없음 +- 테스트 추가 없음 +- live qwen 재실행 없음 + +## 7. 검증 + +이번 작업은 감사/문서화 전용이라 compileall, pytest, smoke-test는 실행하지 않았다. + +다음 구현 발주에서 최소 검증은 다음이 필요하다. + +- `python -m compileall songryeon_core main.py` +- `python -m pytest` +- `python main.py smoke-test` +- initial/revision preserved frame이 함께 있을 때 final candidate count와 accumulated candidate count가 분리되는 pytest +- 같은 파일명 다른 doc_id candidate가 collapse되지 않는 pytest diff --git a/Administrative_Reform_1/05_Execution_Records/order_130_document_evidence_role_claim_guard_2026_06_29_001.md b/Administrative_Reform_1/05_Execution_Records/order_130_document_evidence_role_claim_guard_2026_06_29_001.md new file mode 100644 index 0000000..17889dd --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_130_document_evidence_role_claim_guard_2026_06_29_001.md @@ -0,0 +1,112 @@ +# order_130_document_evidence_role_claim_guard_2026_06_29_001 + +작성일: 2026-06-29 +관련 발주서: `Administrative_Reform_1/04_Orders/ORDER_130_DOCUMENT_EVIDENCE_ROLE_CLAIM_GUARD_V0.md` + +## 1. 목적 + +node_3 최종 답변이 문서 역할을 섞어 말하지 않도록 잠갔다. + +최근 live trace에서 explicit resolver와 document context pack은 ORDER_122~128 문서를 찾아 node_3에게 공급했지만, 실제 `read_doc`으로 읽힌 문서는 달랐다. 그런데 node_3는 `ORDER_122는 read_doc으로 읽혔다`처럼 말했고 node_4가 이를 통과시켰다. + +이번 작업은 이 문제를 막기 위한 작은 정직성 패치다. + +## 2. 변경 파일 + +- `songryeon_core/nodes/node_2_handoff.py` +- `songryeon_core/nodes/node_4_gatekeeper.py` +- `songryeon_core/prompts/node_3_reporter_v0.md` +- `songryeon_core/prompts/node_4_gatekeeper_v0.md` +- `tests/test_order_130_document_evidence_role_claim_guard.py` +- `Administrative_Reform_1/04_Orders/ORDER_130_DOCUMENT_EVIDENCE_ROLE_CLAIM_GUARD_V0.md` +- `Administrative_Reform_1/04_Orders/README.md` + +## 3. 구현 내용 + +### 3.1 node_3 payload 역할 경계표 추가 + +`node3_brief_llm_payload()`에 `document_evidence_role_boundaries`를 추가했다. + +포함한 항목: + +- `actual_tool_read_doc_document_names` +- `supplied_context_document_names` +- `search_candidate_document_names` +- `excluded_context_document_names` +- `unread_candidate_document_names` +- `supplied_but_not_actual_read_doc_document_names` + +이 정보는 문서별 역할 장부에서 code가 복사한 절대정보다. + +### 3.2 node_3 reporting rule/prompt 보강 + +node_3에게 다음 경계를 명시했다. + +- 문서별 role flag가 true인 역할만 해당 문서에 주장한다. +- node_3 context로 공급된 문서라도 `was_actual_tool_read_doc=false`이면 `read_doc`으로 읽었다고 말하지 않는다. +- `document_context_pack`으로 공급된 문서는 usable context일 수 있지만, 실제 `read_doc` tool read와 다르다. + +### 3.3 node_4 prompt 보강 + +node_4가 `document_material_packet.items`와 `document_evidence_role_boundaries`를 문서 역할 장부로 보게 했다. + +### 3.4 node_4 code guard 추가 + +`_document_evidence_role_code_guard()`를 추가했다. + +검사하는 것: + +- 보고문 한 줄에 특정 문서명과 `read_doc` / `actual_tool_read_doc` 계열 명시 역할 주장이 함께 있고, + 해당 문서의 `was_actual_tool_read_doc`가 false이면 `needs_revision`. +- 보고문 한 줄에 특정 문서명과 `supplied_document_context` / context 공급 계열 명시 역할 주장이 함께 있고, + 해당 문서의 `was_supplied_document_context`가 false이면 `needs_revision`. + +reason code: + +- `CODE_STATUS:document_evidence_role_claim_mismatch` + +contradiction 예: + +- `read_doc_claim_without_actual_tool_read_doc:ORDER_122.md` + +## 4. 의도적으로 하지 않은 것 + +- L3 per-document summary 구현 없음 +- search/read 예산 변경 없음 +- L revision 전략 변경 없음 +- node_4 guard 약화 없음 +- W/R loop, scheduler, 외부 DB, 장기기억 DB 변경 없음 +- code가 문서 의미 요약 생성하지 않음 + +## 5. 테스트 + +추가 pytest: + +- `test_node3_payload_exposes_document_role_boundaries` +- `test_node4_blocks_read_doc_claim_for_supplied_only_document` +- `test_node4_allows_context_claim_for_supplied_only_document` +- `test_node4_allows_read_doc_claim_for_actual_read_document` + +검증 결과: + +- `python -m pytest tests/test_order_130_document_evidence_role_claim_guard.py` + - 4 passed +- `python -m compileall songryeon_core main.py` + - 통과 +- `python -m pytest` + - 58 passed in 344.83s +- `python main.py smoke-test` + - 첫 실행은 출력 없이 exit 1로 종료되어 stderr/stdout 병합 재실행 + - 재실행 결과 `SMOKE_TEST_OK` + +## 6. 남은 위험 + +이번 code guard는 넓은 자연어 의미 판단을 대신하지 않는다. + +명시적으로 `read_doc` / `actual_tool_read_doc` / `supplied_document_context` 같은 역할 표현을 쓰는 경우의 장부 충돌만 막는다. 더 애매한 자연어 표현은 node_4 LLM 판단과 후속 설계 대상이다. + +다음 후보: + +- ORDER_129 구현: final search candidate count와 accumulated search candidate count 분리 +- L3 per-document summary 설계 재개 +- node_4의 문서 역할 claim 검사 coverage 확대 diff --git a/Administrative_Reform_1/05_Execution_Records/order_131_search_candidate_scope_split_2026_06_29_001.md b/Administrative_Reform_1/05_Execution_Records/order_131_search_candidate_scope_split_2026_06_29_001.md new file mode 100644 index 0000000..4f3c944 --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_131_search_candidate_scope_split_2026_06_29_001.md @@ -0,0 +1,122 @@ +# order_131_search_candidate_scope_split_2026_06_29_001 + +작성일: 2026-06-29 +관련 발주서: `Administrative_Reform_1/04_Orders/ORDER_131_SEARCH_CANDIDATE_SCOPE_SPLIT_V0.md` +선행 감사: `Administrative_Reform_1/04_Orders/ORDER_129_SEARCH_CANDIDATE_COUNT_BASIS_AUDIT_V0.md` + +## 1. 목적 + +`search_candidate_count`가 최종 후보와 누적 후보를 섞어 표시하던 문제를 분리했다. + +기존 감사 결론: + +- node_0 material packet / L return summary는 최신/최종 L3 기준 검색 후보에 가깝다. +- node_3 input brief는 L3 preserved frame 전체를 훑어 누적 후보에 가깝게 세고 있었다. +- 같은 파일명 다른 경로 후보가 document name 기준으로 collapse될 위험도 있었다. + +## 2. 변경 내용 + +### 2.1 Node3InputBriefFrame 필드 추가 + +추가 필드: + +- `final_search_candidate_count` +- `final_search_candidate_documents` +- `accumulated_search_candidate_count` +- `accumulated_search_candidate_documents` + +호환용 기존 필드: + +- `search_candidate_count` +- `search_candidate_documents` + +기존 필드는 유지하되, 이제 `final_search_candidate_*`와 같은 값이어야 한다. + +### 2.2 count source 분리 + +최종 후보: + +- node_0 document material packet의 `was_search_candidate=true` item 기준 +- material packet이 없으면 L return summary의 `search_result_doc_ids` fallback +- identity 기준은 `doc_id` + +누적 후보: + +- L3 preserved info frame 전체의 `candidates[].doc_id` +- initial/revision preserved frame 전체 포함 +- identity 기준은 `doc_id` + +표시용 label은 같은 파일명이 여러 경로에 있으면 전체 path로 구분한다. + +### 2.3 node_3 grounding block 변경 + +기존: + +- `검색 후보 문서: N개` + +변경: + +- `검색 후보 문서(최종): N개` +- `검색 후보 문서(누적): N개` + +### 2.4 node_4 count guard 변경 + +node_4 code grounding count guard가 최종/누적 검색 후보 count를 각각 검증한다. + +### 2.5 terminal view 변경 + +node_3 input brief runtime 표시가 다음처럼 분리된다. + +- `search_candidates_final` +- `search_candidates_accumulated` + +### 2.6 node_3 LLM payload 변경 + +`search_candidate_scope`를 추가했다. + +포함 항목: + +- `final_search_candidate` +- `accumulated_search_candidate` +- `legacy_search_candidate_count_is` +- `boundary` + +## 3. 테스트 + +추가 pytest: + +- `tests/test_order_131_search_candidate_scope_split.py` + +검증한 것: + +- final 후보와 accumulated 후보가 다른 source에서 온다. +- legacy `search_candidate_count`는 final 후보 count와 같다. +- 같은 파일명 다른 doc_id 후보가 accumulated 후보에서 collapse되지 않는다. +- material packet이 없으면 final 후보가 L return summary `search_result_doc_ids`로 fallback된다. +- node_3 grounding block이 최종/누적 count를 분리 표시한다. + +## 4. 검증 결과 + +- `python -m pytest tests/test_order_131_search_candidate_scope_split.py tests/test_order_123_actual_read_doc_vs_context_pack_count.py tests/test_order_130_document_evidence_role_claim_guard.py` + - 11 passed +- `python -m compileall songryeon_core main.py` + - 통과 +- `python -m pytest` + - 60 passed in 307.86s +- `python main.py smoke-test` + - `SMOKE_TEST_OK` + +## 5. 하지 않은 것 + +- L3 per-document summary 구현 없음 +- 검색/read 예산 변경 없음 +- L revision 전략 변경 없음 +- node_4 guard 약화 없음 +- W/R loop, scheduler, 외부 DB, 장기기억 DB 변경 없음 +- 휴리스틱으로 검색 후보 재분류 없음 + +## 6. 남은 위험 + +이번 작업은 candidate count scope를 분리했지만, L3가 어떤 후보를 더 읽을지 고르는 전략은 바꾸지 않았다. + +다음 후보는 L3 per-document summary 재개 또는 node_3 context 폭탄 정리다. diff --git a/Administrative_Reform_1/05_Execution_Records/order_132_node2_answer_basis_material_delivery_policy_implementation_2026_06_29_001.md b/Administrative_Reform_1/05_Execution_Records/order_132_node2_answer_basis_material_delivery_policy_implementation_2026_06_29_001.md new file mode 100644 index 0000000..1fd83a1 --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_132_node2_answer_basis_material_delivery_policy_implementation_2026_06_29_001.md @@ -0,0 +1,65 @@ +# ORDER_132 Node2 Answer Basis Material Delivery Policy Implementation 2026-06-29 001 + +## 작업 범위 + +- ORDER_132를 `요약 보조`가 아니라 `비-절대정보 모드에서 L3 요약이 node_3 LLM 입력의 원문 text를 대체`하는 정책으로 수정했다. +- 원문 document extract record는 DataStore에 그대로 보존하고, node_3 LLM payload에서만 raw text를 생략한다. +- `Node3MaterialDeliveryPolicyFrame`을 추가했다. +- `Node3InputBriefFrame`에 material delivery policy 요약 필드와 LLM payload 기준 count를 추가했다. +- node_2 handoff에서 policy frame을 별도 DataStore record로 저장한 뒤 node_3 brief에 연결했다. + +## 정책 mapping + +- `absolute_first` -> `raw_document_primary` + - 원문 text를 node_3 LLM payload에 유지한다. + - L3 요약은 보조 재료다. +- `relative_allowed` + L3 summary 있음 -> `l3_summary_replaces_raw_context` + - 원문 text를 node_3 LLM payload에서 생략한다. + - L3 요약을 labeled summary material로 사용한다. +- `mixed_or_uncertain` + L3 summary 있음 -> `l3_summary_replaces_raw_context_with_uncertainty` + - 원문 text를 node_3 LLM payload에서 생략한다. + - source-bundle/summary limit를 더 드러내게 한다. +- 비-절대정보 모드 + L3 summary 없음 -> `raw_document_fallback_no_l3_summary` + - code가 요약을 대신 만들지 않고 원문 text를 유지한다. + +## 주요 변경 파일 + +- `Administrative_Reform_1/04_Orders/ORDER_132_NODE2_ANSWER_BASIS_MATERIAL_DELIVERY_POLICY_V0.md` +- `songryeon_core/core/schemas.py` +- `songryeon_core/nodes/node_2_handoff.py` +- `songryeon_core/nodes/node_3_reporter.py` +- `songryeon_core/prompts/node_3_reporter_v0.md` +- `songryeon_core/runtime/terminal_view.py` +- `tests/test_order_132_material_delivery_policy.py` + +## 검증한 경계 + +- 비-절대정보 모드에서 L3 summary가 있으면 `supplied_document_contexts`와 legacy `read_documents` payload에 raw `text`가 들어가지 않는다. +- raw document text가 payload에서 빠져도 DataStore의 원문 `tool_result:read_doc` payload는 유지된다. +- `absolute_first`에서는 L3 summary가 있어도 raw text가 유지된다. +- L3 summary가 없으면 비-절대정보 모드라도 raw fallback으로 닫힌다. +- grounding block에 `node_3 LLM 원문 text`와 `L3 문서별 요약 재료` count가 표시된다. +- terminal view에 material delivery policy가 표시된다. + +## 검증 명령 + +```powershell +python -m compileall songryeon_core main.py +python -m pytest +python main.py smoke-test +git diff --check +``` + +## 결과 + +- `python -m compileall songryeon_core main.py`: 통과 +- `python -m pytest`: `67 passed` +- `python main.py smoke-test`: `SMOKE_TEST_OK` + +## 제외한 것 + +- 원문 DataStore record를 삭제하거나 변형하지 않았다. +- code가 문서 중요도나 관련성을 판단하지 않았다. +- node_0 요약, node_5 기억 압축, 장기기억 DB, vector DB, scheduler는 열지 않았다. +- W/R loop와 same-turn L reroute 횟수는 건드리지 않았다. +- node_4 자동 재작성 루프는 열지 않았다. diff --git a/Administrative_Reform_1/05_Execution_Records/order_132_node2_answer_basis_material_delivery_policy_order_draft_2026_06_29_001.md b/Administrative_Reform_1/05_Execution_Records/order_132_node2_answer_basis_material_delivery_policy_order_draft_2026_06_29_001.md new file mode 100644 index 0000000..bd68918 --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_132_node2_answer_basis_material_delivery_policy_order_draft_2026_06_29_001.md @@ -0,0 +1,35 @@ +# ORDER_132 Node2 Answer Basis Material Delivery Policy Order Draft 2026-06-29 001 + +## 작성 배경 + +ORDER_125에서 L3 문서별 요약 frame이 구현되었다. +다음 단계로 node_3가 원문 문서 context와 L3 요약을 언제 어떤 태도로 사용할지 정해야 한다. + +사용자 아이디어: + +- node_2가 `absolute_first`에 가까운 판단을 하면 node_3는 원문을 우선 봐야 한다. +- 더 유연한 답변에서는 L3 요약이 문서 폭탄을 줄이는 보조 재료가 될 수 있다. +- 장기적으로는 턴 앞/중간/뒤 모든 단계에서 메타정보 판단이 필요할 수 있지만, 지금은 node_2 -> node_3 답변 재료 전달 정책까지만 연다. + +## 작성 내용 + +- 새 발주서 `ORDER_132_NODE2_ANSWER_BASIS_MATERIAL_DELIVERY_POLICY_V0.md`를 추가했다. +- `Administrative_Reform_1/04_Orders/README.md`의 정식 발주서 범위를 `ORDER_132`까지 갱신했다. +- ORDER_132는 구현 전 발주서이며, 이번 기록에서는 코드 변경을 수행하지 않았다. + +## 핵심 방향 + +- `answer_basis_mode=absolute_first` -> `raw_document_primary` +- `answer_basis_mode=relative_allowed` -> `summary_assisted` +- `answer_basis_mode=mixed_or_uncertain` -> `uncertainty_summary_assisted` + +## 제외한 것 + +- L3 요약으로 원문 context를 자동 대체하지 않았다. +- node_3 context packing 정책을 바꾸지 않았다. +- code가 문서 관련성이나 중요도를 판단하게 만들지 않았다. +- node_0 요약, node_5 기억 압축, 장기기억 DB, vector DB, scheduler, W/R loop는 열지 않았다. + +## 검증 + +- 문서 작성 작업이므로 compileall/pytest/smoke-test는 실행하지 않았다. diff --git a/Administrative_Reform_1/05_Execution_Records/order_133_codebase_readonly_inspection_mvp_2026_06_30_001.md b/Administrative_Reform_1/05_Execution_Records/order_133_codebase_readonly_inspection_mvp_2026_06_30_001.md new file mode 100644 index 0000000..cb8e3eb --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_133_codebase_readonly_inspection_mvp_2026_06_30_001.md @@ -0,0 +1,77 @@ +# ORDER_133 Codebase Readonly Inspection MVP Implementation + +Date: 2026-06-30 + +## Scope + +ORDER_133 implemented the first code-structure reading MVP. + +This is not a coding agent loop. It only gives the L loop read-only inspection tools for the local workspace: + +- `list_code_files` +- `search_code` +- `read_code_file` + +## Core Changes + +- Added `songryeon_core/tools/code_tools.py`. + - Lists source/config files under the workspace. + - Searches source/config files by literal substring. + - Reads one workspace-relative source/config file. + - Rejects absolute paths, path traversal outside the workspace, ignored cache/venv paths, directories, missing files, and unsupported extensions. + +- Extended `build_document_tool_registry()`. + - The tool catalog now includes 7 read-only tools: existing document tools plus the 3 code inspection tools. + - `code_root` is optional and defaults to the current working directory. + +- Extended L2 query schema and prompts. + - Added `code_file_list`, `code_search`, and `code_file_read` query modes. + - Added `list_code_files`, `search_code`, and `read_code_file` as allowed L2 targets. + - Prompt boundary says these are read-only inspection tools and must not propose edits, patches, file writes, tests, or command execution. + +- Extended L loop execution. + - Initial L2 code tool choices now execute the selected code tool instead of falling through to `search_docs`. + - Revision tool attempts can also execute code inspection tools when the revision query frame selects them. + +- Extended tool result distillation. + - `list_code_files` and `search_code` are distilled into small candidate-like previews. + - `read_code_file` is distilled into an excerpt-like preview. + - These are copied file/line/path facts, not semantic code review. + +- Extended node_3 handoff. + - Successful `read_code_file` records are copied into node_3 supplied context as source-code text. + - Search/list code results alone do not become raw source context. + +- Protected document context packing. + - `document_context_pack` now excludes candidates that are not readable Markdown documents instead of crashing when L3 preserved candidates include source-code paths. + - New exclusion reason: `excluded_not_readable_markdown_document`. + +## Verification + +- `python -m compileall songryeon_core main.py` + - passed + +- `python -m pytest tests/test_order_133_codebase_readonly_inspection.py` + - 4 passed + +- `python -m pytest` + - 71 passed + +- `python main.py smoke-test` + - `SMOKE_TEST_OK` + - observed `tool_catalog_count=7` + +## Deliberately Not Done + +- No file editing tools. +- No patch generation or application. +- No automatic test execution as a SongRyeon runtime tool. +- No AST/dependency graph analysis. +- No W/R loop, scheduler, external DB, vector DB, or long-term memory DB changes. +- No same-turn L reroute policy change. + +## Remaining Risk + +- Code search is literal substring search, not semantic code understanding. +- Code context currently reuses node_3 supplied context plumbing, so future work may need a separate `source_code_contexts` field if code/document counts must be separated more strictly. +- L3 achievement language still evolved from document evidence language, so later audits should check whether code-inspection turns are described as code evidence rather than document evidence. diff --git a/Administrative_Reform_1/05_Execution_Records/order_134_l_tool_scope_and_budget_partition_2026_06_30_001.md b/Administrative_Reform_1/05_Execution_Records/order_134_l_tool_scope_and_budget_partition_2026_06_30_001.md new file mode 100644 index 0000000..8cf99ac --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_134_l_tool_scope_and_budget_partition_2026_06_30_001.md @@ -0,0 +1,79 @@ +# ORDER_134 L Tool Scope And Budget Partition Implementation + +Date: 2026-06-30 + +## Scope + +ORDER_134 adds an explicit L tool-scope stage before L2 query planning. + +This was implemented after ORDER_133 added read-only code inspection tools, but live testing showed that L2 could still keep using document search only. The fix is not a keyword heuristic. L now records an explicit scope frame first, then code filters the tool catalog and builds a budget partition from that recorded scope. + +## Core Changes + +- Added `LToolScopeFrame`. + - `tool_scope_mode` is constrained to `document_only`, `code_only`, `document_and_code`, `runtime_trace_only`, or `mixed_evidence`. + - `allowed_tool_groups` is constrained to explicit tool groups. + - LLM success is recorded as `info_class=mixed` and `semantic_judgement_status=ran`. + - Missing/failed LLM scope planning is recorded as `CODE:FALLBACK`, `info_class=absolute_status`, and `semantic_judgement_status=failed`. + +- Added `LToolBudgetPartitionFrame`. + - Code builds document/code budget slices from the approved L budget and the recorded scope. + - Code does not decide semantic relevance from user keywords. + - The budget partition is an absolute policy decision derived from explicit frame values. + +- Added `songryeon_core/nodes/l_tool_scope.py`. + - Runs the scope planner. + - Records fallback status honestly. + - Filters the tool catalog by allowed tool groups. + - Builds the budget partition frame. + +- Added `songryeon_core/prompts/l_tool_scope_planner_v0.md`. + - Asks the LLM to choose the evidence/tool world before L2 selects a specific tool. + - Distinguishes document tools, code inspection tools, and future runtime record tools. + +- Updated L2 query planning. + - `run_l2_query_planner()` and `run_l2_revision_query_planner()` now receive `l_tool_scope`, `budget_partition`, and a filtered tool catalog. + - L2 candidates outside the filtered allowed tools fail schema validation. + - Revision L2 retains compatibility with ORDER_122 by allowing default `read_doc` when no explicit tool catalog is supplied. + +- Updated L loop wiring. + - The L loop now runs scope planning after L1 and before L2. + - The L loop passes the filtered catalog and budget partition to initial and revision L2 planning. + - Fallback L2 tool selection respects the filtered catalog. + +- Updated runtime output. + - Terminal output now displays L tool scope and L budget partition records. + +- Updated ORDER_133 code inspection tests. + - Code-inspection behavior now uses an explicit `code_only` scope adapter. + - This preserves the ORDER_134 rule that code tools are opened by a recorded scope, not by hidden keyword matching. + +## Verification + +- `python -m compileall songryeon_core main.py` + - passed + +- `python -m pytest tests/test_order_122_l_revision_read_doc_path.py tests/test_order_134_l_tool_scope_budget_partition.py` + - 9 passed + +- `python -m pytest` + - 76 passed + +- `python main.py smoke-test` + - `SMOKE_TEST_OK` + - observed `tool_catalog_count=7` + +## Deliberately Not Done + +- No user-keyword routing heuristic. +- No code editing tools. +- No automatic test execution as a SongRyeon runtime tool. +- No runtime trace tool implementation for `runtime_record_tools`. +- No W/R loop, scheduler, external DB, vector DB, or long-term memory DB changes. +- No same-turn L reroute policy change. + +## Remaining Risk + +- `runtime_trace_only` is reserved by schema, but no runtime-record tool exists yet. +- Live Qwen behavior still depends on whether the L scope planner chooses `code_only` or `document_and_code` when the user asks for source inspection. +- The scope frame opens the correct tool world structurally; it does not guarantee the model will choose the best exact file or query inside that world. diff --git a/Administrative_Reform_1/05_Execution_Records/order_135_code_evidence_accounting_2026_06_30_001.md b/Administrative_Reform_1/05_Execution_Records/order_135_code_evidence_accounting_2026_06_30_001.md new file mode 100644 index 0000000..294203a --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_135_code_evidence_accounting_2026_06_30_001.md @@ -0,0 +1,70 @@ +# ORDER_135 Code Evidence Accounting Implementation + +Date: 2026-06-30 + +## Scope + +ORDER_135 fixes the downstream accounting gap where `read_code_file` could succeed but L3/node_3 still reported the turn as if no original evidence had been read. + +This implementation keeps document evidence and source-code evidence separate. It does not rename code reads into `read_doc`. + +## Core Changes + +- Added source-code evidence fields. + - `L3AchievementFrame.read_code_file_paths` + - `L3AchievementFrame.actual_read_code_file_count` + - `LLoopReturnSummaryFrame.read_code_file_paths` + - `LLoopReturnSummaryFrame.actual_read_code_file_count` + - `Node3InputBriefFrame.actual_tool_read_code_file_paths` + - `Node3InputBriefFrame.actual_tool_read_code_file_count` + - `Node3InputBriefFrame.supplied_source_code_context_count` + +- Updated L3 goal-match logic. + - Successful `read_code_file` records are scanned as source-code evidence. + - If the requested path matches a read source file path, the goal match can be `matched`. + - The match reason is recorded as `CODE_STATUS:requested_source_code_file_read_code_file_matched`. + +- Updated L1 minimum evidence guard. + - Source-code evidence can satisfy the minimum evidence count when the L tool scope/material expects source-code evidence. + - Document reads remain counted through `read_doc_ids`. + +- Updated L return summary and node_2 handoff. + - Return summary carries source-code read count/path. + - route=2 handoff now distinguishes document extract counts from code extract counts. + - `missing_l_evidence_result` is not added when successful code evidence exists. + +- Updated node_3 brief and final grounding block. + - Grounding now displays `read_doc` count and `read_code_file` count separately. + - Source-code context count is visible separately from total supplied context. + +- Updated prompts. + - L3 prompt now treats `read_code_file_count` and `read_code_file_paths` as source/config read evidence. + - node_3 prompt now tells the reporter not to describe `read_code_file` as `read_doc`. + +## Verification + +- `python -m pytest tests/test_order_133_codebase_readonly_inspection.py tests/test_order_134_l_tool_scope_budget_partition.py tests/test_order_135_code_evidence_accounting.py` + - 10 passed + +- `python -m pytest` + - 77 passed + +- `python -m compileall songryeon_core main.py` + - passed + +- `python main.py smoke-test` + - `SMOKE_TEST_OK` + +## Deliberately Not Done + +- No code editing tools. +- No command execution tools. +- No AST/dependency graph analysis. +- No keyword heuristic for deciding code/document scope. +- No W/R loop, scheduler, external DB, vector DB, or long-term memory DB changes. +- No same-turn L reroute policy change. + +## Remaining Risk + +- Source-code context still travels through the legacy `read_documents`/`supplied_document_contexts` payload shape, although counts now distinguish it. +- Future work may split node_3 context into separate `document_contexts` and `source_code_contexts` fields if the mixed payload name keeps confusing the model or users. diff --git a/Administrative_Reform_1/05_Execution_Records/order_136_current_capability_baseline_and_live_test_pack_2026_06_30_001.md b/Administrative_Reform_1/05_Execution_Records/order_136_current_capability_baseline_and_live_test_pack_2026_06_30_001.md new file mode 100644 index 0000000..f64a9e2 --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_136_current_capability_baseline_and_live_test_pack_2026_06_30_001.md @@ -0,0 +1,61 @@ +# ORDER_136 Current Capability Baseline And Live Test Pack Documentation + +Date: 2026-06-30 + +## Scope + +ORDER_136 is a documentation-only stabilization order. + +The purpose was to stop feature drift after ORDER_133-135 and record what SongRyeon Core can currently do, what it cannot do yet, and which live qwen prompts should be used to test the current baseline. + +## Added Documents + +- `Administrative_Reform_1/04_Orders/ORDER_136_CURRENT_CAPABILITY_BASELINE_AND_LIVE_TEST_PACK_V0.md` +- `Administrative_Reform_1/03_Maps/03_Development_Maps/SONGRYEON_CORE_CURRENT_CAPABILITY_BASELINE_AND_LIVE_TEST_PACK_2026_06_30.md` + +## Updated Indexes + +- `Administrative_Reform_1/04_Orders/README.md` + - Updated formal order range to ORDER_001 through ORDER_136. + - Added ORDER_136 summary and link. + +- `Administrative_Reform_1/03_Maps/03_Development_Maps/README.md` + - Added current capability baseline/live test pack link. + +## Baseline Captured + +The new map records: + +- recent conversation memory, +- internal document search/read, +- read-only source/config code inspection, +- L tool scope and budget partition, +- document evidence versus source-code evidence accounting, +- node_2 answer-basis modes, +- node_3 code-supplied grounding block, +- node_4 gatekeeping, +- runtime signals to inspect before trusting a final answer, +- seven manual live qwen tests, +- sustainability work needed before larger feature expansion. + +## Verification + +Documentation-only change. + +Recommended check: + +- `git diff --check` + +## Deliberately Not Done + +- No runtime code changes. +- No test code changes. +- No W/R loop. +- No scheduler. +- No external DB/vector DB/long-term memory DB. +- No code editing tools. +- No new heuristic classifier. + +## Next Recommendation + +Run the live test pack manually and save the observed runtime/final answer results before adding another feature. diff --git a/Administrative_Reform_1/05_Execution_Records/order_137_source_code_context_summary_coverage_guard_2026_06_30_001.md b/Administrative_Reform_1/05_Execution_Records/order_137_source_code_context_summary_coverage_guard_2026_06_30_001.md new file mode 100644 index 0000000..d166f16 --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_137_source_code_context_summary_coverage_guard_2026_06_30_001.md @@ -0,0 +1,59 @@ +# order_137_source_code_context_summary_coverage_guard_2026_06_30_001 + +## 1. 작업 요약 + +ORDER_137을 구현했다. + +`read_code_file`로 source-code 원문을 읽은 뒤 node_3가 공개 함수/상수 coverage를 빠뜨리지 않도록, code가 문법적으로 확인 가능한 source-code outline을 만들어 `Node3InputBriefFrame`과 node_3 LLM payload에 공급한다. + +## 2. 변경 파일 + +- `Administrative_Reform_1/04_Orders/ORDER_137_SOURCE_CODE_CONTEXT_SUMMARY_COVERAGE_GUARD_V0.md` +- `Administrative_Reform_1/04_Orders/README.md` +- `songryeon_core/core/schemas.py` +- `songryeon_core/nodes/node_2_handoff.py` +- `songryeon_core/nodes/node_3_reporter.py` +- `songryeon_core/prompts/node_3_reporter_v0.md` +- `songryeon_core/runtime/terminal_view.py` +- `tests/test_order_135_code_evidence_accounting.py` + +## 3. 핵심 구현 + +- `Node3SourceCodeSymbol`과 `Node3SourceCodeOutline`을 추가했다. +- `read_code_file` result payload가 source-code context로 공급되면 Python `ast.parse`로 top-level outline을 생성한다. +- outline에는 함수, async 함수, class, 대문자 상수 assign의 이름/종류/line number/public 여부/docstring 존재 여부만 기록한다. +- code는 함수 의미를 요약하지 않는다. 의미 설명은 node_3가 supplied source text와 outline을 함께 보고 작성한다. +- node_3 LLM payload에는 `source_code_outlines`를 safe payload로 넣고 raw internal data id는 노출하지 않는다. +- grounding block과 terminal runtime view에 source-code outline count를 표시한다. +- node_3 prompt에 source-code outline을 coverage checklist로 쓰되, 함수명만 보고 동작을 단정하지 말라는 경계를 추가했다. + +## 4. 검증 + +- `python -m compileall songryeon_core main.py` + - 통과 +- `python -m pytest tests/test_order_135_code_evidence_accounting.py` + - `1 passed` +- `python -m pytest` + - `77 passed` +- `python main.py smoke-test` + - `SMOKE_TEST_OK` + +## 5. 확인한 값 + +- `songryeon_core/tools/code_tools.py`를 `read_code_file`로 읽는 테스트에서 source-code outline이 1개 생성됐다. +- public function checklist에 다음 이름이 포함됐다. + - `list_code_files` + - `search_code` + - `read_code_file` +- top-level symbol 목록에 다음 상수가 포함됐다. + - `DEFAULT_CODE_FILE_EXTENSIONS` + - `DEFAULT_IGNORED_DIR_NAMES` +- node_3 LLM payload의 outline item에는 `source_data_id`를 넣지 않았다. +- grounding block에 `source-code 구조 목록: 1개`가 표시된다. + +## 6. 일부러 하지 않은 것 + +- 질문 문자열 기반 휴리스틱을 추가하지 않았다. +- code가 source-code 기능 의미를 대신 요약하지 않았다. +- read_code_file count를 read_doc count와 섞지 않았다. +- L loop budget, W/R loop, scheduler, 외부 DB, 장기기억 DB는 건드리지 않았다. diff --git a/Administrative_Reform_1/05_Execution_Records/order_138_integration_baseline_dirty_worktree_reconciliation_2026_06_30_001.md b/Administrative_Reform_1/05_Execution_Records/order_138_integration_baseline_dirty_worktree_reconciliation_2026_06_30_001.md new file mode 100644 index 0000000..4dc2240 --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_138_integration_baseline_dirty_worktree_reconciliation_2026_06_30_001.md @@ -0,0 +1,110 @@ +# order_138_integration_baseline_dirty_worktree_reconciliation_2026_06_30_001 + +## Status Note + +2026-06-30 이후 감사와 사용자 결재에 따라, 이 기록에 포함된 심야정부 MVP(`songryeon_core/night_government`, `night-*` CLI, `tests/test_night_government_mvp.py`)는 송련 Core의 기존 TurnStateCapsule/TraceStore/ZeroState 기반 기억 체계와 맞지 않는 외부 provisional 구현으로 판단되어 제거되었다. + +이 기록은 과거 기준선 감사 자료로만 남기며, 심야정부 재설계의 구현 기준으로 사용하지 않는다. + +## 1. 작업 요약 + +ORDER_138을 작성하고 현재 작업트리 기준선을 감사했다. + +이번 작업은 새 runtime 기능 구현이 아니라, ORDER_133~137 및 심야정부 MVP가 빠르게 들어온 뒤 현재 상태를 추적 가능하게 묶는 정리 작업이다. + +## 2. 현재 확인한 기능 상태 + +### ORDER_133~137 + +- ORDER_133: read-only codebase inspection MVP가 들어와 `code_tools.py` 기반 코드 파일 목록/검색/읽기 도구를 제공한다. +- ORDER_134: L loop가 L2 도구 선택 전에 tool scope와 도구군별 예산 분배를 만든다. +- ORDER_135: `read_code_file` 결과가 `read_doc`과 분리되어 L3/node_3에서 source-code evidence로 인정된다. +- ORDER_136: current capability baseline과 live test pack이 문서화되었다. +- ORDER_137: `read_code_file` 원문에서 source-code outline을 만들어 node_3 coverage checklist로 공급한다. + +### 심야정부 MVP + +현재 심야정부 MVP는 다음 파일로 들어와 있다. + +- `songryeon_core/night_government/schemas.py` +- `songryeon_core/night_government/store.py` +- `songryeon_core/night_government/runtime.py` +- `tests/test_night_government_mvp.py` +- `main.py`의 `night-ingest`, `night-run`, `night-active` CLI + +검증 결과, 심야정부 MVP 자체는 작동한다. + +다만 현재 상태는 `memory_role` 기반 JSONL 외부 기억장 MVP다. +아직 송련식 메타정보 provenance DB로 정렬되지는 않았다. + +부족한 필드/경계: + +- `info_class` +- `generated_by` +- `source_mode` +- `claim_alignment` +- `semantic_judgement_status` +- `source_data_ids` +- `source_trace_ids` + +따라서 심야정부는 다음 단계에서 메타정보 정렬 발주가 필요하다. + +## 3. 작업트리 상태 + +`git status --short` 기준으로 작업트리는 여전히 매우 dirty하다. + +특징: + +- 기존 tracked 파일 다수가 수정되어 있다. +- ORDER_118~138 문서 다수가 아직 untracked 상태다. +- ORDER별 테스트 파일 다수가 아직 untracked 상태다. +- 심야정부 새 패키지도 아직 untracked 상태다. +- 이번 작업에서는 사용자/다른 채팅방 변경을 되돌리지 않았다. + +특히 심야정부 MVP는 구현과 검증은 되었지만, 정식 발주 문서 없이 먼저 들어온 기능이므로 ORDER_138에서 provisional 상태로 기록했다. + +## 4. 검증 결과 + +다음 명령을 실행했다. + +```powershell +python -m compileall songryeon_core main.py +python -m pytest tests/test_night_government_mvp.py -q +python -m pytest +python main.py smoke-test +git diff --check +``` + +결과: + +- `compileall`: 통과 +- `tests/test_night_government_mvp.py`: `2 passed` +- 전체 `pytest`: `79 passed` +- `smoke-test`: `SMOKE_TEST_OK` +- `git diff --check`: 통과 + +## 5. 남은 위험 + +1. 심야정부 MVP는 아직 0 기억공급관과 연결되지 않았다. +2. 심야정부 record는 아직 송련 메타정보 정책의 `absolute/relative/mixed` 분류를 강제하지 않는다. +3. 상대/혼합 기억 record가 절대정보 source 없이 저장되는 것을 막는 validator가 아직 없다. +4. `schemas.py`, `smoke_test.py`, 일부 node 파일이 다시 커지고 있다. +5. 작업트리가 매우 dirty하므로, 다음 기능 확장 전 commit/PR 단위 정리 또는 최소 staging 기준 합의가 필요하다. + +## 6. 다음 권장 발주 + +### ORDER_139_NIGHT_GOVERNMENT_METAINFO_ALIGNMENT_V0 + +심야정부 record를 송련 메타정보 원칙에 맞춘다. + +핵심: + +- `memory_role`과 `info_class` 분리 +- relative는 단일 source 필요 +- mixed는 source bundle 필요 +- orphan relative/mixed record 금지 +- active packet이 다음 턴에 들어올 때 사용 권한과 한계 표시 + +### Schema/Test 추가 정리 + +외부 DB를 더 키우기 전에 schema/test 비대화의 2차 분해를 검토한다. diff --git a/Administrative_Reform_1/05_Execution_Records/order_139_graph_memory_foundation_2026_06_30_001.md b/Administrative_Reform_1/05_Execution_Records/order_139_graph_memory_foundation_2026_06_30_001.md new file mode 100644 index 0000000..a88466d --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_139_graph_memory_foundation_2026_06_30_001.md @@ -0,0 +1,97 @@ +# order_139_graph_memory_foundation_2026_06_30_001 + +## 1. 작업 요약 + +ORDER_139에 따라 TurnStateCapsule 기반 graph memory foundation과 RLoopGraphGuidePacket v0를 추가했다. + +이번 작업은 외부 graph DB, 심야정부 worker, R route, R1/R2/R3 loop를 열지 않았다. + +구현 범위: + +- `GraphMemoryNodeFrame`, `GraphMemoryEdgeFrame`, `GraphMemorySnapshotFrame`, `CoreEgoTimeAxisFrame`, `RLoopGraphGuidePacketFrame` schema 추가. +- `TurnStateCapsule` 좌표에서 `graph:raw_capsule:{turn_id}` raw node를 deterministic/idempotent하게 생성. +- `graph:core_ego:root -> graph:axis:time -> graph:time_bundle:{batch_id} -> graph:raw_capsule:{turn_id}` edge 구조 생성. +- raw node의 `summary_depth=0`, `source_leaf_count=1`, `source_summary_count=0` 보존. +- snapshot 기반 `RLoopGraphGuidePacketFrame` 생성 및 DataStore 보존. +- dry-run/smoke/runtime summary에 graph memory guide count/status 노출. + +## 2. 읽은 기준 문서 + +- `AGENTS.md` +- `Administrative_Reform_1/01_Maintenance_System/SCHEMA_METAINFO_POLICY_v0.md` +- `Administrative_Reform_1/00_Philosophy/Night_Government_Graph_Memory_Philosophy_2026_06_30.md` +- `Administrative_Reform_1/00_Philosophy/R_Loop_Graph_Guide_Philosophy_2026_06_30.md` +- `Administrative_Reform_1/05_Execution_Records/night_government_mvp_removed_2026_06_30_001.md` +- `Administrative_Reform_1/04_Orders/ORDER_138_INTEGRATION_BASELINE_AND_DIRTY_WORKTREE_RECONCILIATION_V0.md` + +## 3. 구현 위치 + +- Schema: + - `songryeon_core/core/schema_parts/graph_memory.py` + - `songryeon_core/core/schema_parts/__init__.py` + - `songryeon_core/core/schemas.py` +- Builder/recording: + - `songryeon_core/core/graph_memory.py` +- Runtime exposure: + - `songryeon_core/runtime/dry_run.py` + - `songryeon_core/runtime/terminal_view.py` + - `songryeon_core/runtime/smoke_test.py` +- Tests: + - `tests/test_order_139_graph_memory_foundation.py` + +## 4. 메타정보 경계 + +Raw capsule node는 의미 기억 text를 만들지 않는다. + +담는 값: + +- node id +- source turn id +- trace count +- movement count +- user/final trace anchor +- source trace ids +- summary/source count 계산 필드 +- char budget status + +RLoopGraphGuidePacket은 다음 상태로 고정했다. + +```text +generated_by = CODE:GRAPH_MEMORY_GUIDE_BUILDER +info_class = absolute +semantic_judgement_status = not_run +recommended_traversal_hints_status = not_run +``` + +LLM traversal hint와 의미축은 생성하지 않았다. + +## 5. 금지 항목 확인 + +- `songryeon_core/night_government` 패키지 재생성 없음. +- `MemoryRecord`, `NightGovernmentPacket`, `MemoryActivationItem` 재사용 없음. +- 독립 JSONL memory DB 생성 없음. +- Neo4j/SQLite/vector DB 연결 없음. +- CoreEgo 의미축 생성 없음. +- R1/R2/R3 loop 구현 없음. +- node_1 R route 연결 없음. +- node_3 최종 답변에 graph guide 자동 주입 없음. +- summary를 raw node 속성에 덮어쓰기 없음. + +## 6. 검증 결과 + +작업 전 체크: + +- `git status --short --branch`: clean, `codex/integration-night-baseline-20260630` +- `python -m compileall songryeon_core main.py`: 통과 +- `python -m pytest`: 120초 제한에서는 timeout +- `python main.py smoke-test`: 120초 제한에서는 timeout + +작업 후 체크: + +- `python -m compileall songryeon_core main.py`: 통과 +- `python -m pytest tests/test_order_139_graph_memory_foundation.py`: `6 passed` +- `python -m pytest`: `83 passed in 339.37s` +- `python main.py smoke-test`: `SMOKE_TEST_OK` +- `git diff --check`: 통과 + +작업 전 timeout은 실패로 판정하지 않고, 전체 pytest/smoke가 120초보다 오래 걸리는 기준선 특성으로 기록한다. diff --git a/Administrative_Reform_1/05_Execution_Records/order_139_graph_memory_foundation_order_draft_2026_06_30_001.md b/Administrative_Reform_1/05_Execution_Records/order_139_graph_memory_foundation_order_draft_2026_06_30_001.md new file mode 100644 index 0000000..694f50d --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_139_graph_memory_foundation_order_draft_2026_06_30_001.md @@ -0,0 +1,25 @@ +# order_139_graph_memory_foundation_order_draft_2026_06_30_001 + +## 1. 작업 요약 + +심야정부 graph memory와 R루프 graph guide 구상을 ORDER_139 발주서로 문서화했다. + +이번 기록은 구현 기록이 아니다. + +## 2. 작성한 발주서 + +- `Administrative_Reform_1/04_Orders/ORDER_139_GRAPH_MEMORY_FOUNDATION_AND_RLOOP_GUIDE_PACKET_V0.md` + +## 3. 발주서 핵심 + +- 기존 TurnStateCapsule/TraceStore/DataStore를 우선 사용한다. +- raw capsule graph node와 CoreEgo 시간축 연결을 만든다. +- summary depth/source count 계산 인프라를 둔다. +- RLoopGraphGuidePacket을 code-generated absolute/status 중심으로 생성한다. +- LLM traversal hint, 의미축 CoreEgo, 실제 R루프 route, 외부 DB 연결은 열지 않는다. + +## 4. 검증 + +문서 작성만 수행했다. + +`git diff --check`만 확인한다. diff --git a/Administrative_Reform_1/05_Execution_Records/order_140_144_candidate_order_batch_2026_06_30_001.md b/Administrative_Reform_1/05_Execution_Records/order_140_144_candidate_order_batch_2026_06_30_001.md new file mode 100644 index 0000000..447e532 --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_140_144_candidate_order_batch_2026_06_30_001.md @@ -0,0 +1,42 @@ +# order_140_144_candidate_order_batch_2026_06_30_001 + +## 1. 작업 요약 + +ORDER_139 이후 예상되는 R루프/심야정부 graph memory 로드맵을 후보 발주서로 미리 작성했다. + +이번 작업은 구현이 아니다. + +## 2. 작성한 후보 발주서 + +- `ORDER_140_R_LOOP_FRAME_ONLY_STATE_MACHINE_AUDIT_V0_CANDIDATE.md` +- `ORDER_141_CORE_EGO_GUIDE_WORKER_LLM_HINTS_V0_CANDIDATE.md` +- `ORDER_142_EXTERNAL_GRAPH_DB_ADAPTER_BOUNDARY_V0_CANDIDATE.md` +- `ORDER_143_R_LOOP_NODE0_MEMORY_PACKET_HANDOFF_V0_CANDIDATE.md` +- `ORDER_144_R_ROUTE_DRY_RUN_ONLY_V0_CANDIDATE.md` + +## 3. 공통 상태 + +모든 문서는 후보 발주서다. + +사용자 별도 결재와 선행 ORDER 완료 전 구현 금지로 표시했다. + +## 4. 의도 + +밤샘 개발 중에도 "다음 수"가 흥분에 밀려 범위 초과되지 않도록, 구현 순서와 금지선을 미리 문서화했다. + +로드맵: + +```text +ORDER_139: graph memory foundation + RLoopGraphGuidePacket +ORDER_140 candidate: R frame/state machine audit +ORDER_141 candidate: CoreEgoGuideWorker LLM hints +ORDER_142 candidate: external graph DB adapter boundary +ORDER_143 candidate: node_0 -> R handoff packet +ORDER_144 candidate: dry-run only R route skeleton +``` + +## 5. 검증 + +문서 작성만 수행했다. + +`git diff --check`만 확인한다. diff --git a/Administrative_Reform_1/05_Execution_Records/order_140_r_loop_frame_state_machine_2026_06_30_001.md b/Administrative_Reform_1/05_Execution_Records/order_140_r_loop_frame_state_machine_2026_06_30_001.md new file mode 100644 index 0000000..bac4e67 --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_140_r_loop_frame_state_machine_2026_06_30_001.md @@ -0,0 +1,67 @@ +# order_140_r_loop_frame_state_machine_2026_06_30_001 + +## 1. 작업 요약 + +ORDER_140 후보 발주를 사용자 승인에 따라 frame-only state machine 구현으로 승격했다. + +이번 작업은 R루프를 실제 route로 열지 않는다. + +구현 범위는 R1/R2/R3/controller/return summary frame schema와 deterministic continuation decision helper, 그리고 pytest다. + +## 2. 핵심 변경 + +- `songryeon_core/core/schema_parts/r_loop.py` + - `R1GraphGoalFrame` + - `RLoopBudgetFrame` + - `R2GraphNodeSelectionFrame` + - `R3GraphInspectionFrame` + - `RLoopContinuationFrame` + - `RLoopReturnSummaryFrame` + - 각 validator 추가 +- `songryeon_core/core/r_loop_state_machine.py` + - R3 구조화 상태와 budget 숫자만 보고 continuation status를 결정하는 helper 추가 +- `songryeon_core/core/schemas.py`, `schema_parts/__init__.py` + - 기존 import 호환 surface에 R loop frame 연결 +- `tests/test_order_140_r_loop_frame_state_machine.py` + - 허용 graph node id 검증, deeper/switch 분리, budget exhausted, not_run 경계 테스트 추가 + +## 3. 금지선 + +- node_1 route=R 연결 없음. +- qwen-turn/qwen-chat에서 R루프 실행 없음. +- R1/R2/R3 LLM prompt 없음. +- 외부 graph DB 연결 없음. +- 의미축 CoreEgo 연결 없음. +- node_3 final answer에 R 결과 주입 없음. +- code가 graph node 선택 이유나 inspection reason을 의미적으로 작성하지 않음. + +## 4. 검증 + +다음 명령을 실행했다. + +```powershell +python -m compileall songryeon_core main.py +python -m pytest tests/test_order_140_r_loop_frame_state_machine.py +python -m pytest tests/test_order_139_graph_memory_foundation.py tests/test_order_140_r_loop_frame_state_machine.py +python -m pytest +python main.py smoke-test +git diff --check +``` + +결과: + +- `compileall`: 통과 +- ORDER_140 단독 pytest: `6 passed` +- ORDER_139+140 묶음 pytest: `12 passed` +- 전체 pytest: `89 passed` +- smoke-test: `SMOKE_TEST_OK` +- `git diff --check`: 통과 + +## 5. 확인된 경계 + +- R2 selection은 허용된 graph node id만 통과한다. +- 허용 목록 밖 graph node id는 validator가 거부한다. +- R3는 raw capsule, time bundle, summary node kind를 구조적으로 구분한다. +- `continue_deeper`와 `continue_switch_branch`는 다른 continuation status로 남는다. +- 예산 소진 시 `stop_budget_exhausted`로 닫힌다. +- schema-only/fake frame의 semantic judgement는 `not_run`으로 유지된다. diff --git a/Administrative_Reform_1/05_Execution_Records/order_141_core_ego_guide_worker_hints_2026_06_30_001.md b/Administrative_Reform_1/05_Execution_Records/order_141_core_ego_guide_worker_hints_2026_06_30_001.md new file mode 100644 index 0000000..b94b696 --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_141_core_ego_guide_worker_hints_2026_06_30_001.md @@ -0,0 +1,69 @@ +# ORDER_141 CoreEgo Guide Worker LLM Hints Implementation + +Date: 2026-06-30 + +## Scope + +Implemented ORDER_141 as a narrow LLM hint layer over the code-generated `RLoopGraphGuidePacketFrame`. + +## Changed Files + +- `songryeon_core/core/schema_parts/graph_memory.py` + - Added `CoreEgoGuideWorkerHintFrame`. + - Added validator for available entry IDs, source graph node IDs, source bundle metadata, and failure frames. +- `songryeon_core/core/schema_parts/__init__.py` + - Re-exported the new frame and validator. +- `songryeon_core/core/schemas.py` + - Preserved compatibility import surface for the new frame and validator. +- `songryeon_core/nodes/core_ego_guide_worker.py` + - Added `run_core_ego_guide_worker_hint`. + - Records input, LLM call, and hint frame in TraceStore/DataStore. + - Does not make code fallback recommendations on LLM failure. +- `songryeon_core/prompts/core_ego_guide_worker_v0.md` + - Added JSON-only traversal hint prompt. +- `tests/test_order_141_core_ego_guide_worker_hints.py` + - Added success, schema failure, parse failure, direct validator, and separation tests. + +## Metainfo Boundary + +- `RLoopGraphGuidePacketFrame` remains code-generated absolute information. +- `CoreEgoGuideWorkerHintFrame` is LLM-generated mixed information because it interprets a graph guide source bundle. +- Failed hint frames preserve failure state with empty recommendations. +- Code does not choose a replacement graph entry when the LLM output fails. + +## Explicit Non-Goals + +- Did not open R route. +- Did not execute R1/R2/R3. +- Did not connect Neo4j or any external DB. +- Did not create semantic axis hierarchy. +- Did not inject the hint into node_1/node_3 user-facing answer flow. + +## Verification + +Focused verification passed: + +```powershell +python -m compileall songryeon_core main.py +python -m pytest tests/test_order_141_core_ego_guide_worker_hints.py -q +``` + +Graph/R focused bundle passed: + +```powershell +python -m pytest tests/test_order_139_graph_memory_foundation.py tests/test_order_140_r_loop_frame_state_machine.py tests/test_order_141_core_ego_guide_worker_hints.py tests/test_order_142_graph_memory_store_boundary.py -q +``` + +Full verification passed: + +```powershell +python -m pytest +python main.py smoke-test +git diff --check +``` + +Result: + +- `python -m pytest`: `101 passed` +- `python main.py smoke-test`: `SMOKE_TEST_OK` +- `git diff --check`: passed diff --git a/Administrative_Reform_1/05_Execution_Records/order_142_external_graph_db_adapter_boundary_2026_06_30_001.md b/Administrative_Reform_1/05_Execution_Records/order_142_external_graph_db_adapter_boundary_2026_06_30_001.md new file mode 100644 index 0000000..5d85bbc --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_142_external_graph_db_adapter_boundary_2026_06_30_001.md @@ -0,0 +1,67 @@ +# order_142_external_graph_db_adapter_boundary_2026_06_30_001 + +## 1. 작업 요약 + +ORDER_142 후보 발주를 사용자 승인에 따라 external graph DB adapter boundary 구현으로 승격했다. + +이번 작업은 Neo4j/SQLite/vector DB를 실제 연결하지 않는다. + +송련 내부 `GraphMemoryNodeFrame`, `GraphMemoryEdgeFrame`, `GraphMemorySnapshotFrame`을 저장/조회하는 protocol과 in-memory adapter를 추가해 외부 DB 연결 전 경계를 잠갔다. + +## 2. 핵심 변경 + +- `songryeon_core/core/graph_memory_store.py` + - `GraphMemoryStoreProtocol` + - `InMemoryGraphMemoryStore` + - Vessel 이름 상수: + - `SONGRYEON_VESSEL_SERVICE_NAME = "songryeon-neo4j-vessel"` + - `SONGRYEON_VESSEL_DATABASE_NAME = "songryeon_vessel"` + - `SONGRYEON_GRAPH_NAMESPACE = "songryeon_core_graph_v0"` +- `tests/test_order_142_graph_memory_store_boundary.py` + - node/edge upsert와 조회 + - same-payload idempotent upsert + - same-id different-payload collision rejection + - source provenance round-trip + - snapshot count round-trip + - semantic topic/embedding field 미생성 + - Vessel 이름 예약 확인 + +## 3. 금지선 + +- Neo4j driver/dependency 추가 없음. +- SQLite/vector DB 추가 없음. +- 의미 검색 없음. +- embedding retrieval 없음. +- semantic topic/meaning cluster 생성 없음. +- 외부 DB record가 내부 graph memory schema를 우회하는 경로 없음. + +## 4. 검증 + +다음 명령을 실행했다. + +```powershell +python -m compileall songryeon_core main.py +python -m pytest tests/test_order_142_graph_memory_store_boundary.py +python -m pytest tests/test_order_139_graph_memory_foundation.py tests/test_order_140_r_loop_frame_state_machine.py tests/test_order_142_graph_memory_store_boundary.py +python -m pytest +python main.py smoke-test +git diff --check +``` + +결과: + +- `compileall`: 통과 +- ORDER_142 단독 pytest: `7 passed` +- ORDER_139/140/142 graph/R-loop 묶음 pytest: `19 passed` +- 전체 pytest: `96 passed` +- smoke-test: `SMOKE_TEST_OK` +- `git diff --check`: 통과 + +## 5. 확인된 경계 + +- in-memory store에 graph node/edge를 upsert하고 다시 조회할 수 있다. +- 같은 node/edge payload를 다시 upsert해도 중복되지 않는다. +- 같은 ID에 다른 payload가 들어오면 collision으로 거부한다. +- `source_graph_node_ids`, `source_trace_ids`, `source_data_ids`가 round-trip 된다. +- `songryeon-neo4j-vessel`, `songryeon_vessel`, `songryeon_core_graph_v0` 이름은 예약했지만 실제 Neo4j 연결은 열지 않았다. +- adapter는 semantic topic, meaning cluster, embedding id를 생성하지 않는다. diff --git a/Administrative_Reform_1/05_Execution_Records/order_143_r_loop_node0_memory_packet_handoff_2026_06_30_001.md b/Administrative_Reform_1/05_Execution_Records/order_143_r_loop_node0_memory_packet_handoff_2026_06_30_001.md new file mode 100644 index 0000000..9cd1105 --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_143_r_loop_node0_memory_packet_handoff_2026_06_30_001.md @@ -0,0 +1,75 @@ +# ORDER_143 R Loop Node0 Memory Packet Handoff Implementation + +Date: 2026-06-30 + +## Scope + +Implemented a node_0 absolute handoff packet for future R loop graph traversal. + +This work does not execute R route or R1/R2/R3. It only records the graph guide coordinates that R loop code can read later. + +## Changed Files + +- `songryeon_core/core/schema_parts/graph_memory.py` + - Added `RLoopMemoryHandoffPacketFrame`. + - Added validation for target/mode/status/source IDs and metainfo boundary. +- `songryeon_core/core/schema_parts/__init__.py` + - Re-exported the new frame and validator. +- `songryeon_core/core/schemas.py` + - Preserved compatibility import surface. +- `songryeon_core/nodes/node_0_memory_supplier.py` + - Added builder and recorder for node_0 R loop graph guide handoff. +- `songryeon_core/runtime/dry_run.py` + - Records the handoff packet after graph memory guide creation. + - Adds summary fields for runtime/smoke. +- `songryeon_core/runtime/terminal_view.py` + - Displays R loop handoff status/count/depth/hint status only. +- `songryeon_core/runtime/smoke_test.py` + - Extends graph memory smoke to verify handoff preservation and no node_1/node_3 injection. +- `tests/test_order_143_r_loop_node0_memory_handoff.py` + - Adds focused pytest coverage. + +## Metainfo Boundary + +- `RLoopMemoryHandoffPacketFrame.info_class = absolute` +- `generated_by = CODE:node_0_memory_supplier` +- `semantic_judgement_status = not_run` +- The frame copies graph snapshot ID, guide packet ID, entry IDs, counts, depth ranges, source graph node IDs, source trace IDs, and source data IDs. +- Missing guide status closes as `packet_status=missing` without code semantic fallback. + +## Explicit Non-Goals + +- Did not open node_1 route=R. +- Did not execute R1/R2/R3. +- Did not create semantic graph hierarchy. +- Did not connect Neo4j or any external DB. +- Did not inject graph guide or R handoff into node_3 final answer. + +## Verification + +Focused verification passed: + +```powershell +python -m compileall songryeon_core main.py +python -m pytest tests/test_order_143_r_loop_node0_memory_handoff.py -q +``` + +Graph/R focused bundle passed: + +```powershell +python -m pytest tests/test_order_139_graph_memory_foundation.py tests/test_order_140_r_loop_frame_state_machine.py tests/test_order_141_core_ego_guide_worker_hints.py tests/test_order_142_graph_memory_store_boundary.py tests/test_order_143_r_loop_node0_memory_handoff.py -q +``` + +Full verification passed: + +```powershell +python -m pytest +python main.py smoke-test +git diff --check +``` + +Result: + +- `python -m pytest`: `106 passed` +- `python main.py smoke-test`: `SMOKE_TEST_OK` +- `git diff --check`: passed before checkpoint diff --git a/Administrative_Reform_1/05_Execution_Records/order_144_r_route_dry_run_only_2026_06_30_001.md b/Administrative_Reform_1/05_Execution_Records/order_144_r_route_dry_run_only_2026_06_30_001.md new file mode 100644 index 0000000..40408ed --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_144_r_route_dry_run_only_2026_06_30_001.md @@ -0,0 +1,83 @@ +# ORDER_144 R Route Dry-Run Only Implementation + +Date: 2026-06-30 + +## Scope + +Implemented a deterministic, opt-in R-loop dry-run skeleton. + +This is not a live Qwen route=R feature. It runs only when `run_dry_turn(enable_r_route_dry_run=True)` is explicitly requested. + +## Changed Files + +- `songryeon_core/loops/r_loop_dry_run.py` + - Added deterministic R1/R budget/R2/R3/continuation/return summary skeleton. + - Uses no LLM calls. + - Uses node_0 R loop handoff packet and graph node DataStore records. +- `songryeon_core/runtime/dry_run.py` + - Added `enable_r_route_dry_run` and `r_route_dry_run_force_budget_exhausted` flags. + - Records R dry-run summary fields only when enabled. +- `songryeon_core/runtime/terminal_view.py` + - Displays R dry-run skeleton status only when frames exist. +- `songryeon_core/runtime/smoke_test.py` + - Added R dry-run only smoke. +- `tests/test_order_144_r_route_dry_run_only.py` + - Added focused tests for default closed state, enabled skeleton, R2 allowed IDs, budget exhaustion, terminal display, and no node_1/node_3 injection. + +## Runtime Boundary + +- Default `run_dry_turn()` keeps R dry-run disabled. +- Qwen/live route=R is not opened. +- node_1 router route set is not expanded. +- node_3 final answer does not receive R frames. +- R dry-run frames use `generated_by=CODE:R_LOOP_DRY_RUN_ONLY`. +- R dry-run semantic status remains `not_run`. + +## Dry-Run Flow + +```text +node_0 R handoff packet +-> R1GraphGoalFrame +-> RLoopBudgetFrame +-> R2GraphNodeSelectionFrame +-> R3GraphInspectionFrame +-> RLoopContinuationFrame +-> RLoopReturnSummaryFrame +``` + +## Explicit Non-Goals + +- Did not open node_1 route=R in qwen-chat. +- Did not add R-loop LLM prompts. +- Did not connect external DB traversal. +- Did not create semantic axis hierarchy. +- Did not inject R output into final answer. + +## Verification + +Focused verification passed: + +```powershell +python -m compileall songryeon_core main.py +python -m pytest tests/test_order_144_r_route_dry_run_only.py -q +``` + +Graph/R focused bundle passed: + +```powershell +python -m pytest tests/test_order_139_graph_memory_foundation.py tests/test_order_140_r_loop_frame_state_machine.py tests/test_order_141_core_ego_guide_worker_hints.py tests/test_order_142_graph_memory_store_boundary.py tests/test_order_143_r_loop_node0_memory_handoff.py tests/test_order_144_r_route_dry_run_only.py -q +``` + +Full verification passed: + +```powershell +python -m pytest +python main.py smoke-test +git diff --check +``` + +Result: + +- `python -m pytest`: `112 passed` +- `python main.py smoke-test`: `SMOKE_TEST_OK` +- `git diff --check`: passed before checkpoint diff --git a/Administrative_Reform_1/05_Execution_Records/order_145_r_loop_pre_live_route_audit_baseline_2026_07_01_001.md b/Administrative_Reform_1/05_Execution_Records/order_145_r_loop_pre_live_route_audit_baseline_2026_07_01_001.md new file mode 100644 index 0000000..8494fa3 --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_145_r_loop_pre_live_route_audit_baseline_2026_07_01_001.md @@ -0,0 +1,85 @@ +# ORDER_145 R Loop Pre-Live Route Audit Baseline + +Date: 2026-07-01 + +## Scope + +Audited and locked the current R-loop boundary before opening any live route=R path. + +This work did not add new runtime behavior. It added documentation and focused tests proving that R remains a dry-run-only skeleton unless explicitly enabled by `run_dry_turn(enable_r_route_dry_run=True)`. + +Follow-up note: + +- ORDER_146 later added an explicit experimental route=R gate. +- The ORDER_145 baseline remains true for default runtime: route=R is still closed unless a later explicit experimental flag is enabled. + +## Findings + +- `RoutingDecisionFrame` currently accepts only `L` and `2`. +- node_1 LLM router payload currently accepts only `L` and `2`. +- Default `run_dry_turn()` does not create `R_loop_return_summary_frame`. +- Opt-in `run_dry_turn(enable_r_route_dry_run=True)` creates deterministic R dry-run frames. +- R dry-run frames remain code-generated non-semantic records: + - `generated_by=CODE:R_LOOP_DRY_RUN_ONLY` + - `semantic_judgement_status=not_run` +- R1/R2/R3 placeholder frames remain `mixed/not_run`. +- R budget/continuation/return summary control frames remain `absolute/not_run`. +- R dry-run output is not injected into node_1 routing decision or node_3 report source ids. + +## Changed Files + +- `Administrative_Reform_1/04_Orders/ORDER_145_R_LOOP_PRE_LIVE_ROUTE_AUDIT_BASELINE_V0.md` + - Added the pre-live R route audit order. +- `Administrative_Reform_1/04_Orders/README.md` + - Registered ORDER_145. +- `Administrative_Reform_1/05_Execution_Records/order_145_r_loop_pre_live_route_audit_baseline_2026_07_01_001.md` + - Added this execution record. +- `Administrative_Reform_1/05_Execution_Records/README.md` + - Registered this execution record. +- `tests/test_order_145_r_loop_pre_live_route_baseline.py` + - Added focused boundary tests. + +## Explicit Non-Goals + +- Did not open node_1 route=R. +- Did not add R route to qwen-chat/live runtime. +- Did not create R1/R2/R3 LLM prompts. +- Did not inject R output into node_3 final answer. +- Did not connect Neo4j or another external graph DB. +- Did not create semantic graph axis. +- Did not change existing graph memory generation semantics. + +## Verification + +Verification passed: + +```powershell +python -m compileall songryeon_core main.py +python -m pytest tests/test_order_145_r_loop_pre_live_route_baseline.py -q +python -m pytest +python main.py smoke-test +git diff --check +``` + +Result: + +- `python -m compileall songryeon_core main.py`: passed +- `python -m pytest tests/test_order_145_r_loop_pre_live_route_baseline.py -q`: `7 passed` +- `python -m pytest`: `119 passed` +- `python main.py smoke-test`: `SMOKE_TEST_OK` +- `git diff --check`: passed + +## Baseline Conclusion + +R live route remains closed. + +The current state is safe to describe as: + +```text +graph memory foundation exists +node_0 R handoff packet exists +R dry-run skeleton exists +live qwen route=R does not exist yet +``` + +The next order may design a live R route gate, but it must explicitly change the node_1 route set and downstream answer boundary instead of relying on the dry-run skeleton. diff --git a/Administrative_Reform_1/05_Execution_Records/order_146_r_route_experimental_gate_2026_07_01_001.md b/Administrative_Reform_1/05_Execution_Records/order_146_r_route_experimental_gate_2026_07_01_001.md new file mode 100644 index 0000000..88cef56 --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_146_r_route_experimental_gate_2026_07_01_001.md @@ -0,0 +1,91 @@ +# ORDER_146 R Route Experimental Gate Implementation + +Date: 2026-07-01 + +## Scope + +Added an explicit experimental gate for node_1 LLM route=R. + +This is not a general live R route opening. It only allows R when the runtime caller explicitly sets `enable_r_route_experimental=True` or the CLI flag `--enable-r-route-experimental`. + +## Changed Files + +- `songryeon_core/core/schemas.py` + - Added route=R validation conditions for the experimental policy flag. +- `songryeon_core/nodes/node_1_router.py` + - Added conditional `allowed_routes` expansion and R payload validation. +- `songryeon_core/runtime/dry_run.py` + - Added `enable_r_route_experimental`. + - Runs `R:experimental:*` skeleton when node_1 LLM selects R. + - Closes back to `route:2` after the experimental R summary. +- `songryeon_core/loops/r_loop_dry_run.py` + - Parameterized the deterministic R skeleton frame label/generator. + - Added fallback R3 inspection from handoff coordinates when graph node records are not yet in DataStore. +- `songryeon_core/nodes/node_0_memory_supplier.py` + - Allowed custom R handoff packet ids to avoid collision with the normal end-of-turn handoff. +- `songryeon_core/nodes/node_2_handoff.py` + - Displays R experimental route path distinctly. +- `songryeon_core/runtime/terminal_view.py` + - Distinguishes experimental R skeleton from dry-run R skeleton. +- `songryeon_core/runtime/user_turn.py` + - Exposes experimental R status in turn summaries. +- `main.py` + - Added `--enable-r-route-experimental`. +- `songryeon_core/prompts/node_1_router_v0.md` + - Replaced hardcoded L/2 rule with payload-driven `allowed_routes`. +- `tests/test_order_146_r_route_experimental_gate.py` + - Added focused tests. + +## Boundary + +- code does not choose R by keyword. +- node_1 LLM may choose R only when the explicit experimental gate is open. +- route=R frame must reveal: + - `policy_flag=enable_r_route_experimental` + - `route_source=LLM:*` + - `llm_routing_status=ran` + - `expected_next_0_mode=r_loop_graph_guide_handoff` +- experimental R records use `generated_by=CODE:R_ROUTE_EXPERIMENTAL_GATE`. +- experimental R closes to `route=2`; it does not create an open-ended R loop. + +## Verification + +Verification passed: + +```powershell +python -m compileall songryeon_core main.py +python -m pytest tests/test_order_146_r_route_experimental_gate.py -q +python -m pytest tests/test_order_145_r_loop_pre_live_route_baseline.py tests/test_order_146_r_route_experimental_gate.py -q +python -m pytest +python main.py smoke-test +git diff --check +``` + +Result: + +- `python -m compileall songryeon_core main.py`: passed +- `python -m pytest tests/test_order_146_r_route_experimental_gate.py -q`: `6 passed` +- `python -m pytest tests/test_order_145_r_loop_pre_live_route_baseline.py tests/test_order_146_r_route_experimental_gate.py -q`: `13 passed` +- `python -m pytest`: `125 passed` +- `python main.py smoke-test`: `SMOKE_TEST_OK` +- `git diff --check`: passed + +Additional manual parser check: + +```powershell +python main.py fake-turn "R 실험 플래그 CLI 확인" --enable-r-route-experimental --pretty +``` + +The command ran successfully. The default fake adapter still chose route=2, which is expected because ORDER_146 does not add a keyword heuristic that forces R. + +## Baseline Conclusion + +The system now has three distinct R states: + +```text +default runtime: route=R closed +dry-run fixture: enable_r_route_dry_run=True creates R:dry_run:* skeleton +experimental gate: enable_r_route_experimental=True allows node_1 LLM to choose route=R and creates R:experimental:* skeleton before closing to route=2 +``` + +This is still not a full R loop. It is a controlled experimental gate and one-pass skeleton. diff --git a/Administrative_Reform_1/05_Execution_Records/order_147_r_result_to_node3_brief_2026_07_01_001.md b/Administrative_Reform_1/05_Execution_Records/order_147_r_result_to_node3_brief_2026_07_01_001.md new file mode 100644 index 0000000..41cc541 --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_147_r_result_to_node3_brief_2026_07_01_001.md @@ -0,0 +1,63 @@ +# ORDER_147 R Result To Node3 Brief Implementation + +Date: 2026-07-01 + +## Scope + +Threaded the experimental R route return summary into node_3 input brief as an explicit, limited, code-copied status material. + +This does not open a full R loop. It only lets downstream reporting see that the experimental R skeleton ran and how limited the result was. + +## Changed Files + +- `songryeon_core/core/schemas.py` + - Added `Node3RLoopResultMaterial`. + - Added `Node3InputBriefFrame.r_loop_result_material`. + - Added validation for R result material source inclusion and metainfo boundary. +- `songryeon_core/nodes/node_2_handoff.py` + - Copies the latest `node_output:R_loop_return_summary_frame` record into node_3 brief. + - Adds a safe `r_loop_result` section to the node_3 LLM payload. +- `songryeon_core/nodes/node_3_reporter.py` + - Adds R route experimental status to the code-supplied grounding block. + - Keeps skeleton/partial R results from being presented as graph memory traversal success. +- `songryeon_core/prompts/node_3_reporter_v0.md` + - Adds the R result boundary for node_3 LLM prose. +- `songryeon_core/runtime/terminal_view.py` + - Displays R result material inside node_3 brief. +- `tests/test_order_147_r_result_to_node3_brief.py` + - Adds focused tests. + +## Boundary + +- R result material is copied from DataStore by code. +- It remains `generated_by=CODE:R_ROUTE_EXPERIMENTAL_GATE`. +- It remains `info_class=absolute`. +- It remains `semantic_judgement_status=not_run`. +- It does not imply R1/R2/R3 semantic traversal ran. +- It does not imply graph memory traversal fully answered the user goal. + +## Verification + +Verification passed: + +```powershell +python -m compileall songryeon_core main.py +python -m pytest tests/test_order_147_r_result_to_node3_brief.py -q +python -m pytest tests/test_order_146_r_route_experimental_gate.py tests/test_order_147_r_result_to_node3_brief.py -q +python -m pytest +python main.py smoke-test +git diff --check +``` + +Result: + +- `python -m compileall songryeon_core main.py`: passed +- `python -m pytest tests/test_order_147_r_result_to_node3_brief.py -q`: `4 passed` +- `python -m pytest tests/test_order_146_r_route_experimental_gate.py tests/test_order_147_r_result_to_node3_brief.py -q`: `10 passed` +- `python -m pytest`: `129 passed` +- `python main.py smoke-test`: `SMOKE_TEST_OK` +- `git diff --check`: passed + +## Baseline Conclusion + +ORDER_147 makes ORDER_146's experimental R result visible to node_3 without inflating it into a finished R loop. diff --git a/Administrative_Reform_1/05_Execution_Records/order_148_graph_next_and_turn_access_ledger_2026_07_01_001.md b/Administrative_Reform_1/05_Execution_Records/order_148_graph_next_and_turn_access_ledger_2026_07_01_001.md new file mode 100644 index 0000000..e30c13e --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_148_graph_next_and_turn_access_ledger_2026_07_01_001.md @@ -0,0 +1,86 @@ +# ORDER 148 Graph NEXT And Turn Access Ledger Implementation + +Date: 2026-07-01 + +## Summary + +Implemented ORDER_148 as a narrow graph-memory provenance step. + +This change adds: + +- time-adjacent `NEXT` edges between deduped raw capsule graph nodes +- `TurnGraphAccessLedgerFrame` for per-turn graph access coordinates +- R skeleton DataStore recording for graph access ledger frames +- runtime display for graph access ledger counts + +## Code Changes + +- `songryeon_core/core/schema_parts/graph_memory.py` + - Added `NEXT` to `GRAPH_MEMORY_EDGE_KINDS`. + - Added `TurnGraphAccessLedgerFrame`. + - Added `validate_turn_graph_access_ledger_frame`. + - Added `GRAPH_ACCESS_LEDGER_CODE_GENERATOR=CODE:GRAPH_ACCESS_LEDGER`. + +- `songryeon_core/core/schemas.py` + - Re-exported the new schema and validator through the compatibility layer. + +- `songryeon_core/core/schema_parts/__init__.py` + - Re-exported the new schema and validator through the split-schema package. + +- `songryeon_core/core/graph_memory.py` + - `build_graph_memory_snapshot_from_capsules()` now creates `NEXT` edges between adjacent deduped raw capsule nodes. + +- `songryeon_core/loops/r_loop_dry_run.py` + - `run_r_loop_dry_run_skeleton()` now builds and records a `TurnGraphAccessLedgerFrame`. + - The ledger copies R2/R3 graph coordinates: + - R2 available nodes -> `candidate_graph_node_ids` + - R2 selected node -> `selected_graph_node_ids` + - R3 inspected node -> `inspected_graph_node_ids` + - R3 child nodes -> additional `candidate_graph_node_ids` + +- `songryeon_core/runtime/dry_run.py` + - Added result keys for R dry-run / experimental access ledger ids and counts. + +- `songryeon_core/runtime/terminal_view.py` + - Added `R graph access ledger` runtime display block. + +- `tests/test_order_148_graph_next_and_access_ledger.py` + - Added NEXT edge and access ledger tests. + +## Guardrails Preserved + +- No external DB / Neo4j connection was opened. +- No semantic axis was created. +- No R LLM traversal was added. +- No live R route policy expansion was added. +- No graph summary or memory summary was generated. +- `TurnGraphAccessLedgerFrame` is code-generated absolute information: + - `generated_by=CODE:GRAPH_ACCESS_LEDGER` + - `info_class=absolute` + - `semantic_judgement_status=not_run` + +## Verification + +Passed: + +```powershell +python -m compileall songryeon_core main.py +python -m pytest +python main.py smoke-test +``` + +Pytest result: + +```text +131 passed in 516.56s +``` + +Smoke result: + +```text +SMOKE_TEST_OK +``` + +## Remaining Risk + +The ledger records R skeleton graph coordinates, but live R traversal is still gated and limited. Future work should decide how external graph DB reads map to `read_graph_node_ids` and how final answers mark `used_as_answer_source_graph_node_ids`. diff --git a/Administrative_Reform_1/05_Execution_Records/order_149_r_graph_traversal_candidate_surface_2026_07_01_001.md b/Administrative_Reform_1/05_Execution_Records/order_149_r_graph_traversal_candidate_surface_2026_07_01_001.md new file mode 100644 index 0000000..716b0c8 --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_149_r_graph_traversal_candidate_surface_2026_07_01_001.md @@ -0,0 +1,80 @@ +# ORDER 149 R Graph Traversal Candidate Surface Implementation + +Date: 2026-07-01 + +## Summary + +Implemented ORDER_149 as a narrow R graph traversal candidate surface. + +The new frame gives future R2 traversal a code-generated candidate board without letting code rank semantic relevance. + +## Code Changes + +- `songryeon_core/core/schema_parts/r_loop.py` + - Added `RGraphTraversalCandidateSurfaceFrame`. + - Added `validate_r_graph_traversal_candidate_surface_frame`. + - Added `R_GRAPH_TRAVERSAL_CANDIDATE_SURFACE_GENERATOR`. + - Added allowed candidate relations: `child`, `next`, `previous`. + +- `songryeon_core/core/schemas.py` + - Re-exported the new frame and validator through the compatibility layer. + +- `songryeon_core/core/schema_parts/__init__.py` + - Re-exported the new frame and validator through the split-schema package. + +- `songryeon_core/loops/r_loop_dry_run.py` + - Builds candidate surface after R3 inspection. + - Copies only code-checkable graph coordinates: + - `R3.child_node_ids` -> `child_candidate_node_ids` + - outgoing `NEXT` edge targets -> `next_candidate_node_ids` + - incoming `NEXT` edge sources -> `previous_candidate_node_ids` + - Records the frame as `node_output:R_graph_traversal_candidate_surface_frame`. + - Feeds candidate surface ids into `RLoopReturnSummaryFrame` and `TurnGraphAccessLedgerFrame`. + +- `songryeon_core/runtime/dry_run.py` + - Added result keys for candidate surface id/count and next/previous counts. + +- `songryeon_core/runtime/terminal_view.py` + - Added `R graph traversal candidates` runtime display. + +- `tests/test_order_149_r_graph_traversal_candidate_surface.py` + - Added child candidate and NEXT/previous candidate tests. + +## Guardrails Preserved + +- R2 selection behavior was not changed. +- No multi-step R traversal was opened. +- No R LLM traversal was added. +- No live R route expansion was added. +- No external DB / Neo4j connection was opened. +- No semantic ranking or relevance judgement was added. +- Candidate surface is code-generated absolute information: + - `generated_by=CODE:R_GRAPH_TRAVERSAL_CANDIDATE_SURFACE` + - `info_class=absolute` + - `semantic_judgement_status=not_run` + +## Verification + +Passed: + +```powershell +python -m compileall songryeon_core main.py +python -m pytest +python main.py smoke-test +``` + +Pytest result: + +```text +133 passed in 484.63s +``` + +Smoke result: + +```text +SMOKE_TEST_OK +``` + +## Remaining Risk + +The candidate surface is now recorded, but R2 does not yet consume it for multi-step traversal. The next candidate order should decide how R2 receives this surface and how budgeted traversal repeats without opening live R route broadly. diff --git a/Administrative_Reform_1/05_Execution_Records/order_150_r_loop_multi_step_traversal_dry_run_2026_07_01_001.md b/Administrative_Reform_1/05_Execution_Records/order_150_r_loop_multi_step_traversal_dry_run_2026_07_01_001.md new file mode 100644 index 0000000..43eb6d1 --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_150_r_loop_multi_step_traversal_dry_run_2026_07_01_001.md @@ -0,0 +1,91 @@ +# ORDER 150 R Loop Multi-Step Traversal Dry-Run Implementation + +Date: 2026-07-01 + +## Summary + +Implemented ORDER_150 as a dry-run-only multi-step R traversal. + +The R skeleton now consumes `RGraphTraversalCandidateSurfaceFrame` and can move from: + +```text +graph:axis:time + -> graph:time_bundle:{turn_id} + -> graph:raw_capsule:{turn_id} +``` + +within the existing dry-run fixture. + +## Code Changes + +- `songryeon_core/loops/r_loop_dry_run.py` + - `run_r_loop_dry_run_skeleton()` now loops across candidate surfaces while continuation says `R2` and budget remains. + - Added step-indexed frame ids: + - `R:...:budget_frame:0001` + - `R2:...:graph_node_selection_frame:0001` + - `R3:...:graph_inspection_frame:0001` + - `R:...:graph_traversal_candidate_surface_frame:0001` + - `R:...:continuation_frame:0001` + - R2 step 2+ selects only from the previous candidate surface coordinate list. + - Return summary and access ledger now accumulate all selected/inspected/candidate graph node ids. + - Added optional in-memory graph payload views for experimental R route so it can inspect graph nodes without permanently recording provisional graph nodes into DataStore. + +- `songryeon_core/runtime/dry_run.py` + - Added `r_route_dry_run_traversal_step_count`. + - Experimental R route now passes build-only graph payloads into the skeleton instead of persisting provisional graph nodes. + +- `tests/test_order_150_r_loop_multi_step_traversal_dry_run.py` + - Added tests for candidate-surface consumption, depth counting, access ledger accumulation, and forced budget exhaustion. + +## Budget Semantics + +- Entry node inspection uses traversal depth `0`. +- Moving through one graph edge uses traversal depth `1`. +- With `max_traversal_depth=2`, dry-run can reach `raw_capsule` from `time_axis`. +- Node read count follows inspected graph node count. + +## Guardrails Preserved + +- No live R route expansion. +- No R LLM traversal. +- No external DB / Neo4j connection. +- No semantic ranking. +- No graph summarization. +- No node_3 answer behavior expansion. + +## Verification + +Passed: + +```powershell +python -m compileall songryeon_core main.py +python -m pytest +python main.py smoke-test +``` + +Pytest result: + +```text +137 passed in 535.04s +``` + +Smoke result: + +```text +SMOKE_TEST_OK +``` + +## Observed Runtime Baseline + +Default R dry-run now reports: + +```text +r_route_dry_run_status=sufficient +r_route_dry_run_continuation=stop_sufficient +``` + +This replaces the previous single-step baseline where dry-run stopped at `continue_deeper`. + +## Remaining Risk + +R traversal is still deterministic dry-run wiring. It proves that graph traversal frames, candidate surfaces, budgets, summaries, and ledgers can chain across multiple steps. It does not yet prove semantic graph search quality. diff --git a/Administrative_Reform_1/05_Execution_Records/order_151_l_loop_activity_ledger_for_graph_backup_2026_07_01_001.md b/Administrative_Reform_1/05_Execution_Records/order_151_l_loop_activity_ledger_for_graph_backup_2026_07_01_001.md new file mode 100644 index 0000000..f3049d1 --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_151_l_loop_activity_ledger_for_graph_backup_2026_07_01_001.md @@ -0,0 +1,77 @@ +# ORDER 151: L Loop Activity Ledger For Graph Backup - Execution Record 2026-07-01-001 + +## Summary + +Implemented `LLoopActivityLedgerFrame` as a code-generated absolute ledger for one L loop run. + +The ledger does not change L routing, search, read, L3 judgment, R loop behavior, or graph DB behavior. +It copies existing L run output IDs and document/code coordinate lists so a later graph-memory step can connect +`graph:raw_capsule:{turn_id}` to the L activity records from that turn. + +## Files Changed + +- `songryeon_core/core/schema_parts/loop_activity.py` +- `songryeon_core/core/schema_parts/__init__.py` +- `songryeon_core/core/schemas.py` +- `songryeon_core/loops/l_loop_namespace.py` +- `songryeon_core/loops/l_loop_activity_ledger.py` +- `songryeon_core/runtime/dry_run.py` +- `songryeon_core/runtime/terminal_view.py` +- `songryeon_core/runtime/smoke_test.py` +- `tests/test_import_baseline.py` +- `tests/test_order_151_l_loop_activity_ledger.py` +- `Administrative_Reform_1/04_Orders/ORDER_151_L_LOOP_ACTIVITY_LEDGER_FOR_GRAPH_BACKUP_V0.md` +- `Administrative_Reform_1/04_Orders/README.md` + +## Implemented Contract + +- DataStore record: `L:activity_ledger_frame` +- Data type: `loop_activity:l_loop_activity_ledger_frame` +- `generated_by=CODE:L_LOOP_ACTIVITY_LEDGER` +- `info_class=absolute` +- `semantic_judgement_status=not_run` +- Graph anchor: `turn_capsule_graph_node_id=graph:raw_capsule:{turn_id}` + +The frame indexes: + +- L run/goal/query/control/tool/budget/continuation/revision/L3 output IDs +- L return summary frame ID +- node_0 document material packet frame ID +- search candidate document IDs +- actual `read_doc` document IDs +- `read_code_file` paths +- activity records with `stage`, `data_id`, and `source_field` + +## Verification + +```powershell +python -m compileall songryeon_core main.py +# passed + +python -m pytest tests/test_order_151_l_loop_activity_ledger.py tests/test_import_baseline.py -q +# 4 passed in 18.29s + +python -m pytest +# 140 passed in 542.90s + +python main.py smoke-test +# SMOKE_TEST_OK +``` + +Smoke added/confirmed: + +- `l_loop_activity_ledger_outputs=33` +- `l_loop_activity_ledger_tool_results=3` +- `l_loop_activity_ledger_read_doc=2` + +## Not Done + +- No graph DB/Neo4j connection. +- No graph edge from raw capsule to L ledger yet. +- No L search/read strategy change. +- No L3 prompt or semantic judgment change. +- No node_3 answer behavior change. + +## Next Candidate + +ORDER_152 should connect `graph:raw_capsule:{turn_id}` to L/R activity ledger records as code-generated graph edges or graph source coordinates. diff --git a/Administrative_Reform_1/05_Execution_Records/order_152_raw_capsule_to_activity_ledger_graph_link_2026_07_01_001.md b/Administrative_Reform_1/05_Execution_Records/order_152_raw_capsule_to_activity_ledger_graph_link_2026_07_01_001.md new file mode 100644 index 0000000..7eea67e --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_152_raw_capsule_to_activity_ledger_graph_link_2026_07_01_001.md @@ -0,0 +1,71 @@ +# ORDER 152 Raw Capsule To Activity Ledger Graph Link - Execution Record + +Date: 2026-07-01 + +## Goal + +Raw capsule graph nodes should preserve traceable graph links to the turn activity ledgers produced by L loop and R graph traversal runtime work. + +This is not a semantic memory expansion. The goal is a code-generated absolute graph bookkeeping layer: + +- raw capsule node -> L loop activity ledger +- raw capsule node -> R graph access ledger + +## Implemented + +- Added `TurnActivityGraphLinkFrame`. +- Added graph node kind `activity_ledger`. +- Added graph edge kind `HAS_ACTIVITY_LEDGER`. +- Added `songryeon_core/core/turn_activity_graph_links.py`. +- Connected dry-run output to record turn activity graph links after L/R ledger generation. +- Added terminal/runtime display for turn activity graph links. +- Added smoke coverage for the default L activity ledger link. +- Added pytest coverage for L ledger links, R access ledger links, and validator rejection. + +## Boundaries + +The implementation did not open: + +- semantic graph axis +- external graph DB +- live R routing +- node_3 automatic use of graph ledger links +- L/R loop behavior changes +- LLM-generated memory summaries + +The new link frame is generated by code and uses: + +- `generated_by=CODE:TURN_ACTIVITY_GRAPH_LINK_BUILDER` +- `info_class=absolute` +- `semantic_judgement_status=not_run` + +## Verification + +Commands passed: + +```powershell +python -m compileall songryeon_core main.py +python -m pytest tests/test_order_152_raw_capsule_activity_graph_link.py tests/test_import_baseline.py -q +python -m pytest +python main.py smoke-test +``` + +Observed smoke values: + +```text +status=SMOKE_TEST_OK +turn_activity_graph_link_l_ledgers=1 +turn_activity_graph_link_r_ledgers=0 +turn_activity_graph_link_nodes=1 +turn_activity_graph_link_edges=1 +``` + +Full pytest result: + +```text +143 passed +``` + +## Remaining Risk + +The graph link layer is still in DataStore/runtime memory only. It is ready for later graph DB export work, but no external DB persistence has been added in this order. diff --git a/Administrative_Reform_1/05_Execution_Records/order_153_graph_memory_integrity_and_r_experimental_source_recording_2026_07_01_001.md b/Administrative_Reform_1/05_Execution_Records/order_153_graph_memory_integrity_and_r_experimental_source_recording_2026_07_01_001.md new file mode 100644 index 0000000..5ae1942 --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_153_graph_memory_integrity_and_r_experimental_source_recording_2026_07_01_001.md @@ -0,0 +1,87 @@ +# ORDER 153 Graph Memory Integrity And R Experimental Source Recording - Execution Record + +Date: 2026-07-01 + +## Goal + +Before opening a real external graph DB, make graph-memory references auditable and remove the dangling source refs found in the experimental R route. + +## Implemented + +### R experimental graph source recording + +The experimental R route now records graph source records before the R handoff is created. + +Recorded source ids include batch-specific graph records such as: + +- `graph:snapshot:turn_dry_001:r_route_experimental` +- `rloop:graph_guide:graph:snapshot:turn_dry_001:r_route_experimental` +- `graph:time_bundle:turn_dry_001:r_route_experimental` + +Shared graph ids that can collide with the final turn graph payload are intentionally not re-recorded in that early branch: + +- `graph:core_ego:root` +- `graph:axis:time` +- `graph:edge:contains:graph:core_ego:root:graph:axis:time` + +Those shared ids are still recorded by the normal final turn graph-memory build. + +### Read-only integrity audit helper + +Added `songryeon_core/core/graph_memory_integrity.py`. + +The helper checks: + +- graph/rloop-prefixed `source_data_ids` +- `graph_memory:edge:*` `from_node_id` +- `graph_memory:edge:*` `to_node_id` + +It reports missing references but does not auto-repair them. + +## Tests Added + +Added `tests/test_order_153_graph_memory_integrity.py`. + +Covered cases: + +- default dry turn graph-memory integrity passes +- R experimental route records graph source ids and integrity passes +- missing graph/rloop `source_data_ids` are reported +- missing graph edge endpoints are reported + +## Verification + +Commands passed: + +```powershell +python -m compileall songryeon_core main.py +python -m pytest tests/test_order_153_graph_memory_integrity.py -q +python -m pytest tests/test_order_146_r_route_experimental_gate.py tests/test_order_147_r_result_to_node3_brief.py tests/test_order_153_graph_memory_integrity.py -q +python -m pytest -q +python main.py smoke-test +``` + +Observed results: + +```text +ORDER_153 pytest: 4 passed +R route focused pytest: 14 passed +full pytest: 147 passed in 873.18s +smoke-test: SMOKE_TEST_OK +``` + +## Not Opened + +This order did not open: + +- Neo4j/Vessel connection +- graph DB export execution +- semantic graph axis +- R LLM selector +- live R route expansion +- R traversal over `HAS_ACTIVITY_LEDGER` +- node_3 answer injection changes + +## Remaining Risk + +`R1/R2/R3` dry-run frames still use code-generated wiring decisions in a dry-run skeleton. Before live R traversal is expanded, their metainfo class should be reviewed separately so code-policy wiring does not look like LLM semantic judgment. diff --git a/Administrative_Reform_1/05_Execution_Records/order_154_fast_test_gate_2026_07_01_001.md b/Administrative_Reform_1/05_Execution_Records/order_154_fast_test_gate_2026_07_01_001.md new file mode 100644 index 0000000..78dbdce --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_154_fast_test_gate_2026_07_01_001.md @@ -0,0 +1,61 @@ +# ORDER 154 Fast Test Gate - Execution Record + +Date: 2026-07-01 + +## Goal + +Add a fast local test gate for development loops without weakening the full pytest/smoke-test baseline. + +## Implemented + +Added CLI: + +```powershell +python main.py fast-test +python main.py fast-test --profile core +python main.py fast-test --profile graph +python main.py fast-test --dry-run +``` + +Profiles: + +- `core`: compileall + import/schema compatibility pytest +- `graph`: compileall + current graph/R activity ledger boundary pytest + +The implementation lives in: + +- `songryeon_core/runtime/fast_test.py` +- `main.py` + +## Verification + +Commands passed: + +```powershell +python -m compileall songryeon_core main.py +python -m pytest tests/test_order_154_fast_test_gate.py tests/test_import_baseline.py -q +python main.py fast-test --dry-run +python main.py fast-test --profile core +python main.py fast-test +``` + +Observed: + +```text +ORDER_154 pytest: 5 passed +fast-test --profile core: FAST_TEST_OK, about 0.9s +fast-test default graph profile: FAST_TEST_OK, 20 passed, about 46.7s +``` + +## Boundary + +This does not replace final validation. + +Large or risky changes should still run: + +```powershell +python -m pytest +python main.py smoke-test +``` + +No test was hidden with a slow marker, and CI/smoke-test policy was not weakened. diff --git a/Administrative_Reform_1/05_Execution_Records/order_155_graph_source_kind_separated_ingest_2026_07_01_001.md b/Administrative_Reform_1/05_Execution_Records/order_155_graph_source_kind_separated_ingest_2026_07_01_001.md new file mode 100644 index 0000000..b3cac6d --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_155_graph_source_kind_separated_ingest_2026_07_01_001.md @@ -0,0 +1,74 @@ +# ORDER 155 Graph Source Kind Separated Ingest - Execution Record 2026-07-01 001 + +## Scope + +Implemented the first graph source ingest foundation for non-conversation materials. + +This order keeps source kinds separated before any later night-government summary or R traversal work. + +Initial source kinds: + +- `internal_document` +- `source_code_file` +- `external_project_file` + +Conversation turns still use the existing `raw_capsule` graph path. + +## Changes + +- Added `songryeon_core/core/graph_source_ingest.py`. +- Added graph node kinds: + - `raw_source` + - `source_kind_bundle` +- Added validation for raw source leaves and source-kind bundle child counts. +- Added source-kind-separated ingest tests in `tests/test_order_155_graph_source_kind_ingest.py`. +- Added the new module to the import baseline. +- Added the ORDER 155 test to `fast-test --profile graph`. +- Documented the order in `Administrative_Reform_1/04_Orders/ORDER_155_GRAPH_SOURCE_KIND_SEPARATED_INGEST_FOUNDATION_V0.md`. + +## Data Boundary + +The code writes only absolute source coordinates: + +- source kind +- normalized path +- file name +- suffix +- exists flag +- char count +- content SHA-1 +- deterministic source file data ID +- deterministic raw source graph node ID +- source-kind bundle node ID +- `CONTAINS` edge ID + +The code does not write: + +- semantic topic +- summary +- importance score +- relevance judgment +- meaning cluster + +## Verification + +```powershell +python -m compileall songryeon_core main.py +# passed + +python -m pytest tests/test_order_155_graph_source_kind_ingest.py tests/test_import_baseline.py -q +# 7 passed + +python main.py fast-test --profile graph +# FAST_TEST_OK +# pytest:graph: 26 passed +``` + +## Not Opened + +- Neo4j/Vessel connection +- night-government summary worker +- semantic axis +- R LLM selector over these bundles +- node_3 answer injection +- conversation ingest replacement diff --git a/Administrative_Reform_1/05_Execution_Records/order_156_graph_source_observation_time_and_core_link_2026_07_01_001.md b/Administrative_Reform_1/05_Execution_Records/order_156_graph_source_observation_time_and_core_link_2026_07_01_001.md new file mode 100644 index 0000000..d3456cf --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_156_graph_source_observation_time_and_core_link_2026_07_01_001.md @@ -0,0 +1,82 @@ +# ORDER 156 Graph Source Observation Time And CoreEgo Link - Execution Record 2026-07-01 001 + +## Scope + +Implemented observation-time metadata and CoreEgo time-axis linkage for source-kind separated graph ingest. + +This makes raw source graph nodes represent a specific filesystem observation, not a timeless claim about the current file. + +## Changes + +- Added graph node kind: + - `source_ingest_time_bundle` +- Added optional observation fields to `GraphMemoryNodeFrame`: + - `observed_at` + - `ingested_at` + - `source_last_modified_at` + - `exists_at_ingest` + - `content_sha1` +- Required those fields for `raw_source` graph nodes. +- Required `observed_at` and `ingested_at` for `source_kind_bundle` and `source_ingest_time_bundle` nodes. +- Updated `record_graph_source_kind_ingest()` to require existing CoreEgo root/time-axis graph records. +- Added graph path: + +```text +graph:core_ego:root + -> graph:axis:time + -> graph:source_ingest_time_bundle:{batch_id} + -> graph:source_kind_bundle:{batch_id}:{source_kind} + -> graph:raw_source:{source_kind}:{observation_digest} +``` + +- Added source-ingest graph snapshot and RLoop guide packet records for the source ingest surface. +- Added observation time into source file data IDs, so the same file/content observed at different times becomes distinct raw source snapshot coordinates. +- Added ORDER 156 pytest coverage and included it in `fast-test --profile graph`. + +## Absolute Information Boundary + +Code writes only observation coordinates: + +- source kind +- normalized path +- path name +- suffix +- exists-at-ingest flag +- char count +- source last modified timestamp +- observed/ingested timestamp +- content SHA-1 +- graph/data IDs +- CoreEgo time-axis graph edges +- source ingest snapshot/guide packet coordinates + +Code does not write summaries, semantic topics, importance scores, relevance judgments, or current-truth claims about future file state. + +## Verification + +```powershell +python -m compileall songryeon_core main.py +# passed + +python -m pytest tests/test_order_155_graph_source_kind_ingest.py tests/test_order_156_graph_source_observation_time_and_core_link.py tests/test_import_baseline.py -q +# 12 passed + +python main.py fast-test --profile graph +# FAST_TEST_OK +# pytest:graph: 31 passed + +python -m pytest +# 162 passed in 582.76s + +python main.py smoke-test +# SMOKE_TEST_OK +``` + +## Not Opened + +- Neo4j/Vessel connection +- night-government summary worker +- semantic axis +- R LLM selector over source ingest bundles +- automatic source path discovery +- node_3 answer injection diff --git a/Administrative_Reform_1/05_Execution_Records/order_157_songryeon_core_source_ingest_manifest_2026_07_01_001.md b/Administrative_Reform_1/05_Execution_Records/order_157_songryeon_core_source_ingest_manifest_2026_07_01_001.md new file mode 100644 index 0000000..4bb1594 --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_157_songryeon_core_source_ingest_manifest_2026_07_01_001.md @@ -0,0 +1,89 @@ +# ORDER 157 SongRyeon Core Source Ingest Manifest - Execution Record 2026-07-01 001 + +## Scope + +Implemented a manifest-based source ingest runner for SongRyeon Core internal documents and source code files. + +This order applies the ORDER 155~156 graph source ingest foundation to the project itself without semantic file classification. + +## Approved Policy Applied + +- Internal document/code raw text snapshots are stored. +- `tests/**/*.py` is included as `source_code_file`. +- Glob manifest is allowed only as explicit policy, not semantic guessing. +- Missing explicit paths fail. +- Missing glob matches are not treated as deletion detection in this MVP. + +## Manifest Policy + +```text +internal_document: + explicit: + - AGENTS.md + - README.md + globs: + - Administrative_Reform_1/**/*.md + +source_code_file: + explicit: + - main.py + globs: + - songryeon_core/**/*.py + - tests/**/*.py +``` + +`external_project_file` is not auto-ingested in this order. + +## Code Changes + +- Added `songryeon_core/core/songryeon_source_manifest.py`. +- Added optional text snapshot support to `record_graph_source_kind_ingest()`. +- Added text snapshot DataRecord type: + - `graph_source:file_text_snapshot` +- Text snapshot payloads use: + - `info_class=absolute_copied_source` + - `semantic_judgement_status=not_run` +- Added ORDER 157 pytest coverage. +- Added the ORDER 157 test to `fast-test --profile graph`. +- Added the new manifest module to the import baseline. + +## Current Repository Manifest Check + +Manual manifest resolution against the current repository root: + +```text +internal_document: 364 +source_code_file: 135 +``` + +This was a manifest resolution check only, not a full current-repo source text ingest export. + +## Verification + +```powershell +python -m compileall songryeon_core main.py +# passed + +python -m pytest tests/test_order_155_graph_source_kind_ingest.py tests/test_order_156_graph_source_observation_time_and_core_link.py tests/test_order_157_songryeon_core_source_manifest.py tests/test_import_baseline.py -q +# 17 passed + +python main.py fast-test --profile graph +# FAST_TEST_OK +# pytest:graph: 36 passed + +python -m pytest +# 167 passed in 584.42s + +python main.py smoke-test +# SMOKE_TEST_OK +``` + +## Not Opened + +- Neo4j/Vessel connection +- semantic axis +- night-government summary worker +- automatic semantic file selection +- external project file auto-discovery +- R LLM selector over source ingest bundles +- node_3 answer injection diff --git a/Administrative_Reform_1/05_Execution_Records/order_158_graph_memory_export_packet_2026_07_01_001.md b/Administrative_Reform_1/05_Execution_Records/order_158_graph_memory_export_packet_2026_07_01_001.md new file mode 100644 index 0000000..9e3381a --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_158_graph_memory_export_packet_2026_07_01_001.md @@ -0,0 +1,70 @@ +# ORDER_158 Graph Memory Export Packet 실행 기록 + +## 작업 일시 + +2026-07-01 + +## 목적 + +외부 Vessel/Neo4j write 전에 `DataStore`에 존재하는 graph memory / graph source record 목록을 code-generated absolute export packet으로 묶었다. + +## 구현 내용 + +- `songryeon_core/core/graph_memory_export.py` 추가 + - `GraphMemoryExportPacket` + - `record_graph_memory_export_packet` + - `build_graph_memory_export_packet` + - `graph_memory_export_packet_id` +- export packet data type 추가 + - `graph_memory:export_packet` +- target adapter name은 `songryeon-neo4j-vessel`로 기록 +- `external_write_status=not_run`으로 외부 DB write 미실행을 명시 +- `audit_graph_memory_integrity()` 결과를 packet에 포함 +- graph fast-test profile에 ORDER_158 pytest 포함 +- import baseline에 `songryeon_core.core.graph_memory_export` 추가 + +## 수집 대상 + +- `graph_memory:node:*` +- `graph_memory:edge:*` +- `graph_memory:snapshot` +- `graph_memory:core_ego_time_axis_frame` +- `graph_memory:rloop_guide_packet` +- `graph_source:file_metadata` +- `graph_source:file_text_snapshot` +- `graph_source:source_kind_ingest_frame` +- `graph_source:songryeon_core_source_manifest_frame` + +## 메타정보 경계 + +- `generated_by=CODE:GRAPH_MEMORY_EXPORT_PACKET_BUILDER` +- `info_class=absolute` +- `semantic_judgement_status=not_run` + +이번 작업은 record 목록, count, source trace id, integrity status 같은 code-checkable absolute 정보만 생성한다. + +## 하지 않은 것 + +- Neo4j driver 추가 없음 +- Vessel adapter 구현 없음 +- 외부 DB write 없음 +- LLM 요약/상대정보/혼합정보 생성 없음 +- 의미축 생성 없음 +- R live route 변경 없음 +- scheduler/장기기억 DB 변경 없음 + +## 검증 결과 + +- `python -m compileall songryeon_core main.py`: 통과 +- `python -m pytest tests/test_order_158_graph_memory_export_packet.py -q`: 4 passed +- `python -m pytest tests/test_import_baseline.py tests/test_order_154_fast_test_gate.py -q`: 5 passed +- `python main.py fast-test --profile graph`: FAST_TEST_OK, 40 passed +- `python -m pytest`: 171 passed +- `python main.py smoke-test`: SMOKE_TEST_OK + +## 확인한 경계 + +- 같은 batch/timestamp로 export packet을 다시 기록하면 기존 packet payload와 충돌하지 않고 existing으로 처리된다. +- export packet 자체는 graph/source export 대상에 다시 포함하지 않는다. +- 일반 `node_output:*` record는 export packet included list에서 제외된다. +- graph integrity 실패는 `graph_integrity_status=failed`로 packet에 기록되며, 외부 write는 여전히 `not_run`이다. diff --git a/Administrative_Reform_1/05_Execution_Records/order_159_vessel_adapter_write_plan_boundary_2026_07_01_001.md b/Administrative_Reform_1/05_Execution_Records/order_159_vessel_adapter_write_plan_boundary_2026_07_01_001.md new file mode 100644 index 0000000..45868ad --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_159_vessel_adapter_write_plan_boundary_2026_07_01_001.md @@ -0,0 +1,75 @@ +# ORDER_159 Vessel Adapter Write Plan Boundary 실행 기록 + +## 작업 일시 + +2026-07-01 + +## 목적 + +ORDER_158의 `GraphMemoryExportPacket`을 입력으로 받아 외부 Vessel/Neo4j write 직전의 `GraphVesselWritePlan`을 생성했다. + +이번 작업은 실제 외부 DB write가 아니라 write operation 목록을 code-generated absolute 정보로 고정하는 단계다. + +## 구현 내용 + +- `songryeon_core/core/graph_vessel_adapter.py` 추가 + - `GraphVesselWritePlan` + - `GraphVesselWriteOperation` + - `build_graph_vessel_write_plan` + - `record_graph_vessel_write_plan` +- write plan data type 추가 + - `graph_vessel:write_plan` +- operation 종류 고정 + - `upsert_source_payload` + - `upsert_graph_node` + - `upsert_graph_edge` + - `upsert_support_record` +- operation 순서 고정 + - source payload -> graph node -> graph edge -> support record +- target adapter는 `songryeon-neo4j-vessel`만 허용 +- graph integrity가 실패한 export packet은 `blocked_integrity_failed`로 닫고 operation list를 비운다. +- graph fast-test profile에 ORDER_159 pytest 포함 + +## 메타정보 경계 + +- `generated_by=CODE:GRAPH_VESSEL_ADAPTER_BOUNDARY` +- `info_class=absolute` +- `semantic_judgement_status=not_run` +- `external_write_status=not_run` + +이번 작업은 record ID, data type, operation order, target adapter name, integrity status 같은 code-checkable absolute 정보만 생성한다. + +## 하지 않은 것 + +- Neo4j driver 추가 없음 +- 외부 DB 연결 없음 +- 실제 write 없음 +- LLM 요약/상대정보/혼합정보 생성 없음 +- 의미축 생성 없음 +- R live route 변경 없음 +- scheduler/장기기억 DB 변경 없음 + +## 검증 결과 + +- `python -m compileall songryeon_core main.py`: 통과 +- `python -m pytest tests/test_order_159_vessel_adapter_boundary.py -q`: 5 passed +- `python -m pytest tests/test_import_baseline.py tests/test_order_154_fast_test_gate.py -q`: 5 passed +- `python main.py fast-test --profile graph`: FAST_TEST_OK, 45 passed +- `python main.py smoke-test`: SMOKE_TEST_OK +- `python -m pytest`: 176 passed + +## 검증 중 메모 + +처음에 `python -m pytest`와 `python main.py smoke-test`를 동시에 실행했을 때 Windows cache file write 경합으로 `.songryeon_core_cache/document_memory_indexes/*`에서 `PermissionError`가 1회 발생했다. + +이후 `python main.py smoke-test`는 통과했고, `python -m pytest`를 단독으로 재실행하자 176개 테스트가 모두 통과했다. + +따라서 ORDER_159 기능 실패가 아니라 병렬 캐시 쓰기 경합으로 기록한다. + +## 확인한 경계 + +- write plan은 `external_write_status=not_run`이며 실제 외부 DB write를 수행하지 않는다. +- target adapter는 `songryeon-neo4j-vessel`만 허용한다. +- graph integrity 실패 export packet은 `blocked_integrity_failed`로 닫고 operation list를 비운다. +- operation order는 source payload -> graph node -> graph edge -> support record 순서로 고정된다. +- write plan은 의미/요약/embedding/topic 필드를 생성하지 않는다. diff --git a/Administrative_Reform_1/05_Execution_Records/order_160_local_vessel_neo4j_first_write_2026_07_01_001.md b/Administrative_Reform_1/05_Execution_Records/order_160_local_vessel_neo4j_first_write_2026_07_01_001.md new file mode 100644 index 0000000..dba1321 --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_160_local_vessel_neo4j_first_write_2026_07_01_001.md @@ -0,0 +1,91 @@ +# ORDER_160 Local Vessel Neo4j First Write 실행 기록 + +## 작업 일시 + +2026-07-01 + +## 목적 + +ORDER_159 `GraphVesselWritePlan`을 입력으로 받아 로컬 Neo4j/Vessel에 수동 opt-in first write를 수행할 수 있는 writer와 CLI를 추가했다. + +## 구현 내용 + +- `songryeon_core/core/graph_vessel_neo4j.py` 추가 + - `GraphVesselNeo4jConfig` + - `GraphVesselNeo4jWriteResultFrame` + - `write_graph_vessel_plan_to_neo4j` + - `record_graph_vessel_neo4j_write_result` +- `songryeon_core/runtime/graph_vessel_first_write.py` 추가 + - tiny fixture graph 생성 + - export packet 생성 + - write plan 생성 + - Neo4j writer 실행 +- `main.py`에 수동 명령 추가 + - `python main.py vessel-first-write` +- graph fast-test profile에 ORDER_160 pytest 포함 + +## 로컬 Vessel 경계 + +- target adapter: `songryeon-neo4j-vessel` +- database default: `songryeon_vessel` +- namespace: `songryeon_core_graph_v0` +- 기본 접속 URI: `bolt://localhost:7687` + +## 메타정보 경계 + +- `generated_by=CODE:GRAPH_VESSEL_NEO4J_WRITER` +- `info_class=absolute` +- `semantic_judgement_status=not_run` + +Neo4j writer는 write result/status/count만 생성하며 의미 요약이나 topic/embedding 필드를 생성하지 않는다. + +## 하지 않은 것 + +- qwen-chat 자동 연결 없음 +- R live route 자동 연결 없음 +- 기존 송련 브레인 DB와 혼합 없음 +- DataStore 전체 무차별 write 없음 +- LLM 요약/상대정보/혼합정보 생성 없음 +- embedding/vector 검색 없음 +- 의미축 생성 없음 +- 기존 DataStore 삭제/변형 없음 + +## 검증 결과 + +- `python -m compileall songryeon_core main.py`: 통과 +- `python -m pytest tests/test_order_160_local_vessel_neo4j_writer.py -q`: 7 passed +- `python -m pytest tests/test_import_baseline.py tests/test_order_154_fast_test_gate.py -q`: 5 passed +- `python main.py fast-test --profile graph`: FAST_TEST_OK, 52 passed +- `python -m pytest`: 183 passed +- `python main.py smoke-test`: SMOKE_TEST_OK + +## 로컬 실행 확인 + +`python main.py vessel-first-write`를 실행했다. + +결과: + +- `status=VESSEL_FIRST_WRITE_NOT_WRITTEN` +- `write_status=adapter_unavailable` +- `failure_type=neo4j_config_missing` +- `failure_reason=Neo4j password is not configured. Set SONGRYEON_NEO4J_PASSWORD or pass --password.` +- `operation_count=10` +- `attempted_operation_count=0` +- `written_operation_count=0` + +현재 환경 확인: + +- Python `neo4j` driver는 설치되어 있다. +- `neo4j` CLI command는 발견되지 않았다. +- `localhost:7474`, `localhost:7687` listening connection은 발견되지 않았다. +- `SONGRYEON_*` Neo4j 환경변수는 발견되지 않았다. + +따라서 ORDER_160 writer/CLI는 구현되었지만, 실제 로컬 Neo4j 서버/비밀번호가 준비되지 않아 first write는 아직 수행되지 않았다. + +## 확인한 경계 + +- writer 입력은 `GraphVesselWritePlan`으로 제한된다. +- `plan_status=ready_to_write`, `external_write_status=not_run`, `graph_integrity_status=passed`, target adapter 일치 조건을 검사한다. +- Neo4j 설정 누락은 `adapter_unavailable`으로 정직하게 닫힌다. +- fake Neo4j driver 기준 write operation이 실행되고 result frame이 `written`으로 기록된다. +- writer는 의미/요약/topic/embedding 필드를 생성하지 않는다. diff --git a/Administrative_Reform_1/05_Execution_Records/order_161_vessel_display_vocabulary_2026_07_01_001.md b/Administrative_Reform_1/05_Execution_Records/order_161_vessel_display_vocabulary_2026_07_01_001.md new file mode 100644 index 0000000..964c471 --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_161_vessel_display_vocabulary_2026_07_01_001.md @@ -0,0 +1,80 @@ +# ORDER_161 Vessel Display Vocabulary 실행 기록 + +## 작업 일시 + +2026-07-01 + +## 목적 + +Neo4j Browser에서 사람이 보는 node label, relationship type, display properties를 더 읽기 쉬운 이름으로 바꾸었다. + +내부 `data_id`, payload, provenance, schema는 유지했다. + +## 구현 내용 + +- `songryeon_core/core/graph_vessel_neo4j.py`의 Neo4j write vocabulary 변경 +- base label 변경 + - 기존: `SongRyeonRecord` + - 신규: `VesselRecord` +- graph node display label 추가 + - `CoreEgo` + - `TimeAxis` + - `TimeBundle` + - `RawCapsule` + - `RawSource` + - `SourceKindBundle` + - `SourceIngestBundle` + - fallback: `GraphMemoryNode` +- support/display record label 변경 + - `GraphMemorySnapshot` + - `RLoopGuide` + - `GraphMemoryIndex` + - `GraphMemorySource` + - `GraphMemoryEdgeRecord` +- relationship type 변경 + - `HAS_AXIS` + - `HAS_BUNDLE` + - `CONTAINS_MEMORY` + - `NEXT` + - `HAS_SOURCE_KIND` + - `CONTAINS_SOURCE` + - fallback: `CONTAINS` +- display properties 추가 + - `display_name` + - `display_kind` + - `display_label` + - relationship `display_relationship_type` + +## 기존 실험 데이터 처리 + +새 writer는 같은 `data_id`/namespace 노드에 새 label을 붙이고 기존 old `SongRyeon*` label을 제거한다. + +같은 `edge_id`/namespace의 기존 old `SONGRYEON_GRAPH_EDGE` 관계는 삭제한 뒤 새 relationship type으로 재생성한다. + +## 메타정보 경계 + +- 내부 `data_id` 유지 +- `source_data_ids`, `source_trace_ids`, `payload_json` 유지 +- LLM 요약/상대정보/혼합정보 생성 없음 +- qwen-chat/R live route 자동 연결 없음 + +## 검증 결과 + +- `python -m compileall songryeon_core main.py`: 통과 +- `python -m pytest tests/test_order_160_local_vessel_neo4j_writer.py -q`: 8 passed +- `python main.py fast-test --profile graph`: FAST_TEST_OK, 53 passed +- `python -m pytest`: 184 passed +- `python main.py smoke-test`: SMOKE_TEST_OK + +## 로컬 실행 메모 + +현재 Codex 실행 터미널에는 Neo4j password 환경변수가 없어서 `python main.py vessel-first-write --database neo4j`는 `adapter_unavailable / neo4j_config_missing`으로 닫혔다. + +사용자가 Neo4j password 환경변수를 설정한 터미널에서 같은 명령을 다시 실행하면 기존 old `SongRyeon*` label이 제거되고 새 display vocabulary가 적용된다. + +예상 확인 쿼리: + +```cypher +MATCH path = (core:CoreEgo)-[:HAS_AXIS]->(:TimeAxis)-[:HAS_BUNDLE]->(:TimeBundle)-[:CONTAINS_MEMORY]->(:RawCapsule) +RETURN path; +``` diff --git a/Administrative_Reform_1/05_Execution_Records/order_162_vessel_readback_verification_2026_07_01_001.md b/Administrative_Reform_1/05_Execution_Records/order_162_vessel_readback_verification_2026_07_01_001.md new file mode 100644 index 0000000..5abe4c8 --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_162_vessel_readback_verification_2026_07_01_001.md @@ -0,0 +1,79 @@ +# ORDER 162: Vessel Readback Verification 실행 기록 + +## 날짜 + +2026-07-01 + +## 변경 요약 + +Neo4j Vessel에 기록된 graph memory가 사람이 보는 label/relationship vocabulary로 다시 읽히는지 확인하는 read-only verifier를 추가했다. + +추가된 핵심 파일: + +- `songryeon_core/core/graph_vessel_readback.py` +- `songryeon_core/runtime/graph_vessel_readback.py` +- `tests/test_order_162_vessel_readback_verification.py` + +추가된 CLI: + +```powershell +python main.py vessel-readback --database neo4j +``` + +## 확인 대상 경로 + +```text +CoreEgo -> HAS_AXIS -> TimeAxis -> HAS_BUNDLE -> TimeBundle -> CONTAINS_MEMORY -> RawCapsule +``` + +## 기록되는 절대정보 + +- `readback_status` +- `core_path_exists` +- `core_path_count` +- `vessel_record_count` +- `vessel_relationship_count` +- `core_ego_count` +- `time_axis_count` +- `time_bundle_count` +- `raw_capsule_count` +- `has_axis_count` +- `has_bundle_count` +- `contains_memory_count` +- `required_property_missing_count` + +결과 frame은 다음으로 고정했다. + +- `generated_by=CODE:GRAPH_VESSEL_NEO4J_READBACK_VERIFIER` +- `info_class=absolute` +- `semantic_judgement_status=not_run` + +## 수동 CLI 확인 + +Codex 실행 환경에는 Neo4j password 환경변수가 설정되어 있지 않아 다음 결과가 나왔다. + +```text +readback_status=adapter_unavailable +failure_type=neo4j_config_missing +failure_reason=Neo4j password is not configured. +``` + +이는 실패를 성공처럼 숨기지 않는 정상 동작이다. 사용자가 비밀번호 환경변수를 설정한 VS Code 터미널에서는 아래 명령으로 실제 readback을 확인하면 된다. + +```powershell +python main.py vessel-readback --database neo4j +``` + +## 검증 + +- `python -m compileall songryeon_core main.py`: 통과 +- `python -m pytest tests/test_order_162_vessel_readback_verification.py -q`: 8 passed +- `python main.py fast-test --profile graph`: FAST_TEST_OK, 61 passed +- `python main.py smoke-test`: SMOKE_TEST_OK +- `python -m pytest -q`: 192 passed in 1030.04s + +## 남은 위험 + +- 현재 Codex 셸에는 Neo4j 비밀번호가 없어 실제 로컬 DB readback pass는 사용자 터미널에서 확인해야 한다. +- `python -m pytest`는 smoke 전체를 포함해 약 17분이 걸린다. 빠른 회귀 확인은 `python main.py fast-test --profile graph`를 우선 사용한다. +- 이번 작업은 readback verification까지만이며, R route/R1/R2/R3가 Vessel을 자동 열람하는 기능은 아직 열지 않았다. diff --git a/Administrative_Reform_1/05_Execution_Records/order_163_vessel_inspect_manual_walk_2026_07_01_001.md b/Administrative_Reform_1/05_Execution_Records/order_163_vessel_inspect_manual_walk_2026_07_01_001.md new file mode 100644 index 0000000..14e9249 --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_163_vessel_inspect_manual_walk_2026_07_01_001.md @@ -0,0 +1,79 @@ +# ORDER 163: Vessel Inspect Manual Walk 실행 기록 + +## 날짜 + +2026-07-01 + +## 변경 요약 + +Neo4j Vessel에 기록된 graph memory를 읽기 전용으로 펼쳐보는 `vessel-inspect` CLI를 추가했다. + +추가된 핵심 파일: + +- `songryeon_core/core/graph_vessel_inspect.py` +- `songryeon_core/runtime/graph_vessel_inspect.py` +- `tests/test_order_163_vessel_inspect_manual_walk.py` + +추가된 CLI: + +```powershell +python main.py vessel-inspect --database neo4j +python main.py vessel-inspect --database neo4j --format text +``` + +## 확인 대상 경로 + +```text +CoreEgo -> HAS_AXIS -> TimeAxis -> HAS_BUNDLE -> TimeBundle -> CONTAINS_MEMORY -> RawCapsule +``` + +## 출력 형태 + +`GraphVesselNeo4jInspectResultFrame`은 다음을 기록한다. + +- `inspect_status` +- `inspected_path_count` +- `core_count` +- `time_axis_count` +- `time_bundle_count` +- `raw_capsule_count` +- `path_items` +- `tree_lines` +- `source_data_ids` +- `source_trace_ids` + +결과 frame은 다음으로 고정했다. + +- `generated_by=CODE:GRAPH_VESSEL_NEO4J_INSPECTOR` +- `info_class=absolute` +- `semantic_judgement_status=not_run` + +## 수동 CLI 확인 + +Codex 실행 환경에는 Neo4j password 환경변수가 설정되어 있지 않아 다음 결과가 나왔다. + +```text +status: VESSEL_INSPECT_NOT_PASSED +inspect_status: adapter_unavailable +failure_type: neo4j_config_missing +failure_reason: Neo4j password is not configured. +``` + +이는 실패를 성공처럼 숨기지 않는 정상 동작이다. 사용자가 비밀번호 환경변수를 설정한 VS Code 터미널에서는 아래 명령으로 실제 inspect를 확인하면 된다. + +```powershell +python main.py vessel-inspect --database neo4j --format text +``` + +## 검증 + +- `python -m compileall songryeon_core main.py`: 통과 +- `python -m pytest tests/test_order_163_vessel_inspect_manual_walk.py -q`: 8 passed +- `python main.py fast-test --profile graph`: FAST_TEST_OK, 69 passed +- `python main.py smoke-test`: SMOKE_TEST_OK + +## 남은 위험 + +- 현재 Codex 셸에는 Neo4j 비밀번호가 없어 실제 로컬 DB inspect pass는 사용자 터미널에서 확인해야 한다. +- 이번 작업은 수동 tree inspect까지만이며, R route/R1/R2/R3가 Vessel inspect 결과를 자동으로 사용하는 기능은 아직 열지 않았다. +- `python -m pytest` 전체는 최근 기준 약 17분이 걸리므로 이번 작업에서는 graph fast-test와 smoke-test를 우선 검증으로 사용했다. diff --git a/Administrative_Reform_1/05_Execution_Records/order_164_dynamic_source_lineage_invalidation_2026_07_02_001.md b/Administrative_Reform_1/05_Execution_Records/order_164_dynamic_source_lineage_invalidation_2026_07_02_001.md new file mode 100644 index 0000000..21c09b8 --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_164_dynamic_source_lineage_invalidation_2026_07_02_001.md @@ -0,0 +1,55 @@ +# ORDER_164 Dynamic Source Version Lineage And Summary Invalidation Ledger 실행 기록 + +## 작업 요약 + +동적 원본 파일이 바뀌었을 때 기존 raw source와 파생 summary를 삭제하지 않고 추적할 수 있도록, code-generated absolute 장부를 추가했다. + +## 핵심 변경 + +- `SourceVersionLineageFrame` 추가 + - 같은 `source_kind + path`를 stable source identity로 묶는다. + - 관측된 raw source graph node를 `observed_at` 순서로 기록한다. + - 최신 raw source를 active version으로 표시한다. + - content hash가 바뀌면 `lineage_status=content_changed`로 표시한다. + +- `SummaryInvalidationLedgerFrame` 추가 + - content-changed lineage의 superseded raw source를 근거로 하는 summary node를 찾는다. + - summary record를 수정/삭제하지 않고 별도 invalidation record를 남긴다. + - 무효화 reason은 `source_content_changed`, validity status는 `invalidated_by_source_change`로 고정했다. + +- `graph_source_ingest` 이후 lineage/ledger frame을 자동 기록한다. + - source ingest 결과에 lineage frame id와 summary invalidation ledger id를 포함한다. + - 같은 batch/source 재기록은 기존 idempotency 규칙을 유지한다. + +- export/Vessel boundary에 lineage/ledger record를 support record로 포함했다. + +## 정보 분류 + +- source lineage: `generated_by=CODE:SOURCE_VERSION_LINEAGE_BUILDER`, `info_class=absolute`, `semantic_judgement_status=not_run` +- invalidation ledger: `generated_by=CODE:SUMMARY_INVALIDATION_LEDGER_BUILDER`, `info_class=absolute`, `semantic_judgement_status=not_run` + +이번 작업에서 code가 의미 요약을 만들지 않았다. code는 source graph node id, file metadata, content hash 변화, summary의 source_graph_node_ids만 검사했다. + +## 검증 + +- `python -m compileall songryeon_core main.py`: 통과 +- `python -m pytest tests/test_order_164_source_version_lineage_and_invalidation.py -q`: 3 passed +- `python -m pytest tests/test_order_155_graph_source_kind_ingest.py tests/test_order_156_graph_source_observation_time_and_core_link.py tests/test_order_157_songryeon_core_source_manifest.py tests/test_order_158_graph_memory_export_packet.py tests/test_order_159_vessel_adapter_boundary.py tests/test_order_160_local_vessel_neo4j_writer.py -q`: 33 passed +- `python -m pytest tests/test_schema_split_compat.py -q`: 2 passed +- `python -m pytest tests/test_order_154_fast_test_gate.py -q`: 4 passed +- `python main.py fast-test --profile graph`: FAST_TEST_OK, 72 passed +- `git diff --check`: 통과 + +## 일부러 하지 않은 것 + +- LLM summary 생성 없음. +- summary node payload 수정/삭제 없음. +- raw_source node 삭제 없음. +- R route/R traversal 정책 변경 없음. +- Neo4j live write 자동 실행 변경 없음. +- source 의미 유사도/키워드 휴리스틱 추가 없음. + +## 남은 위험 + +- 실제 심야정부 LLM summary node가 생기면, 그 summary가 `source_graph_node_ids`로 원본 raw source를 정확히 가리키도록 별도 발주가 필요하다. +- 현재 ledger는 summary record를 직접 mutate하지 않는다. downstream이 current answer 근거에서 invalidated summary를 제외하도록 읽는 정책은 후속 발주 대상이다. diff --git a/Administrative_Reform_1/05_Execution_Records/order_165_same_content_observation_ledger_2026_07_02_001.md b/Administrative_Reform_1/05_Execution_Records/order_165_same_content_observation_ledger_2026_07_02_001.md new file mode 100644 index 0000000..1bf9694 --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_165_same_content_observation_ledger_2026_07_02_001.md @@ -0,0 +1,55 @@ +# ORDER_165 Same-Content Reobserve Observation Ledger 실행 기록 + +## 작업 요약 + +같은 source file을 다시 관측했지만 content hash가 같을 때, 새 `RawSource` graph node를 만들지 않고 observation ledger에 `unchanged`로 기록하도록 바꿨다. + +## 핵심 변경 + +- `SourceVersionLineageFrame.lineage_status`에서 same-content 재관측 상태를 제거했다. + - 현재 lineage status는 `single_version`, `content_changed`만 허용한다. + +- `raw_source` graph node ID 기준에서 `observed_at`을 제외했다. + - 기준: `source_kind + path + content_sha1` + - 같은 파일/같은 내용이면 기존 raw source node를 재사용한다. + - 파일 내용이 바뀌면 content hash가 달라져 새 raw source node가 생긴다. + +- `SourceObservationLedgerFrame`을 추가했다. + - 현재 ingest batch에서 관측한 source file metadata를 기록한다. + - observation status: + - `new_source_version` + - `unchanged` + - `content_changed` + - unchanged는 summary invalidation을 일으키지 않는다. + +- export/Vessel boundary에 observation ledger를 support record로 포함했다. + +## 정보 분류 + +- source observation ledger: `generated_by=CODE:SOURCE_OBSERVATION_LEDGER_BUILDER`, `info_class=absolute`, `semantic_judgement_status=not_run` +- source lineage: `generated_by=CODE:SOURCE_VERSION_LINEAGE_BUILDER`, `info_class=absolute`, `semantic_judgement_status=not_run` +- summary invalidation ledger: `generated_by=CODE:SUMMARY_INVALIDATION_LEDGER_BUILDER`, `info_class=absolute`, `semantic_judgement_status=not_run` + +이번 작업에서 code는 원본 의미를 요약하거나 해석하지 않았다. content hash와 source coordinate만 사용했다. + +## 검증 + +- `python -m compileall songryeon_core main.py`: 통과 +- `python -m pytest tests/test_order_164_source_version_lineage_and_invalidation.py tests/test_order_165_same_content_observation_ledger.py tests/test_order_156_graph_source_observation_time_and_core_link.py -q`: 10 passed +- `python -m pytest tests/test_order_154_fast_test_gate.py tests/test_schema_split_compat.py -q`: 6 passed +- `python -m pytest tests/test_order_155_graph_source_kind_ingest.py tests/test_order_157_songryeon_core_source_manifest.py tests/test_order_158_graph_memory_export_packet.py tests/test_order_159_vessel_adapter_boundary.py tests/test_order_160_local_vessel_neo4j_writer.py -q`: 28 passed +- `python main.py fast-test --profile graph`: FAST_TEST_OK, 74 passed +- `git diff --check`: 통과 + +## 일부러 하지 않은 것 + +- LLM summary 생성 없음. +- raw source/summary record 삭제 없음. +- unchanged source observation을 이유로 summary 무효화하지 않음. +- R route/R traversal 정책 변경 없음. +- Neo4j live write 자동 실행 변경 없음. + +## 남은 위험 + +- 현재 source ingest batch node/source kind bundle/snapshot은 여전히 batch 실행 사실을 남긴다. 즉 unchanged source라도 "검사 실행 장부"는 남는다. +- downstream R loop가 invalidated summary를 자동 제외하는 정책은 아직 후속 발주 대상이다. diff --git a/Administrative_Reform_1/05_Execution_Records/order_166_night_time_bundle_summary_node_2026_07_02_001.md b/Administrative_Reform_1/05_Execution_Records/order_166_night_time_bundle_summary_node_2026_07_02_001.md new file mode 100644 index 0000000..f7d38c0 --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_166_night_time_bundle_summary_node_2026_07_02_001.md @@ -0,0 +1,152 @@ +# order_166_night_time_bundle_summary_node_2026_07_02_001 + +## Summary + +ORDER 166을 구현했다. + +이번 작업은 심야정부가 `TimeBundle` graph node 하나를 대상으로 LLM summary node를 만들 수 있게 하는 첫 MVP다. + +핵심 원칙: + +```text +원본 TimeBundle은 code-generated absolute 좌표로 그대로 둔다. +LLM 요약은 별도 SummaryGraphNode로 만든다. +SummaryGraphNode -> TimeBundle 방향의 SUMMARY_OF edge를 붙인다. +``` + +## Changed Files + +- `Administrative_Reform_1/04_Orders/ORDER_166_NIGHT_TIME_BUNDLE_SUMMARY_NODE_V0.md` +- `Administrative_Reform_1/04_Orders/README.md` +- `songryeon_core/core/schema_parts/graph_memory.py` +- `songryeon_core/core/schema_parts/__init__.py` +- `songryeon_core/core/schemas.py` +- `songryeon_core/core/graph_vessel_neo4j.py` +- `songryeon_core/nodes/night_time_bundle_summary_worker.py` +- `songryeon_core/prompts/night_time_bundle_summary_worker_v0.md` +- `songryeon_core/runtime/fast_test.py` +- `tests/test_order_166_night_time_bundle_summary_node.py` + +## Implementation Notes + +Added `NightTimeBundleSummaryFrame`. + +The frame records: + +- `summary_graph_node_id` +- `target_graph_node_id` +- `summary_text` +- `summary_status` +- `failure_type` +- `summary_depth` +- `source_leaf_count` +- `source_summary_count` +- `source_graph_node_ids` +- `validity_status` +- `review_status` +- `llm_call_data_id` +- `generated_by` +- `info_class` +- `semantic_judgement_status` + +The summary worker is `songryeon_core/nodes/night_time_bundle_summary_worker.py`. + +On success it records: + +- `graph_memory:node:summary` +- `graph_memory:edge:SUMMARY_OF` + +On adapter/schema/parse failure it records only: + +- `node_output:night_time_bundle_summary_frame` + +It does not create a fake graph summary node on failure. + +## Metainfo Classification + +Code decides `info_class` only from source cardinality. + +```text +source_leaf_count == 1 +and source_summary_count == 0 +and len(source_graph_node_ids) == 1 +-> relative + +otherwise +-> mixed +``` + +Code does not write `summary_text`. + +LLM writes `summary_text`. + +If the LLM supplies an incompatible `info_class`, the payload is rejected and a failed frame is recorded. + +## Vessel / Export Behavior + +Existing export behavior already includes `graph_memory:node:*` and `graph_memory:edge:*`. + +Therefore successful summary records are included as graph node and graph edge export items. + +The Vessel write plan treats: + +- summary node as `upsert_graph_node` +- `SUMMARY_OF` edge as `upsert_graph_edge` + +Neo4j display vocabulary now gives summary nodes the `SummaryGraphNode` label and a readable display name. + +## Deliberately Not Opened + +- R loop automatic use of the summary node +- live R1/R2/R3 LLM traversal +- semantic axis +- node_4 approval loop for graph summary +- mutation of original `TimeBundle` +- weakening `GraphMemoryNodeFrame` absolute lock +- code-generated semantic summary text + +## Verification + +```powershell +python -m compileall songryeon_core main.py +``` + +Passed. + +```powershell +python -m pytest tests/test_order_166_night_time_bundle_summary_node.py -q +``` + +Passed: `4 passed`. + +```powershell +python main.py fast-test --profile graph +``` + +Passed: `FAST_TEST_OK`, graph pytest `78 passed`. + +```powershell +python -m pytest -q +``` + +Passed after increasing timeout: `209 passed in 686.63s`. + +First full pytest attempt timed out at about 424 seconds before completion, so it was rerun with a longer limit. + +```powershell +python main.py smoke-test +``` + +Passed: `SMOKE_TEST_OK`. + +```powershell +git diff --check +``` + +Passed. + +## Remaining Risk + +Current `RawCapsule` graph nodes mainly expose trace/capsule coordinates, not full previous conversation text. + +Therefore this MVP proves the summary node/edge/schema/export path, but rich conversation-content summarization still needs a later order that supplies original text or approved source snapshots to the night worker. diff --git a/Administrative_Reform_1/05_Execution_Records/order_167_night_summary_naming_clarity_2026_07_02_001.md b/Administrative_Reform_1/05_Execution_Records/order_167_night_summary_naming_clarity_2026_07_02_001.md new file mode 100644 index 0000000..d926c5a --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_167_night_summary_naming_clarity_2026_07_02_001.md @@ -0,0 +1,112 @@ +# order_167_night_summary_naming_clarity_2026_07_02_001 + +## Summary + +ORDER 167을 구현했다. + +ORDER 166에서 추가한 심야정부 TimeBundle summary worker의 표준 이름을 더 읽기 쉬운 형태로 정리했다. + +새 표준 이름: + +```text +night_summarize_time_bundle +run_night_summarize_time_bundle +songryeon_core/prompts/night_summarize_time_bundle_v0.md +``` + +기존 이름은 compatibility wrapper로 유지했다. + +## Changed Files + +- `Administrative_Reform_1/04_Orders/ORDER_167_NIGHT_SUMMARY_NAMING_CLARITY_V0.md` +- `Administrative_Reform_1/04_Orders/README.md` +- `songryeon_core/core/schema_parts/graph_memory.py` +- `songryeon_core/nodes/night_summarize_time_bundle.py` +- `songryeon_core/nodes/night_time_bundle_summary_worker.py` +- `songryeon_core/prompts/night_summarize_time_bundle_v0.md` +- `tests/test_order_166_night_time_bundle_summary_node.py` + +## Naming Result + +Old implementation module was moved to: + +```text +songryeon_core/nodes/night_summarize_time_bundle.py +``` + +Old module now exists only as a compatibility import surface: + +```text +songryeon_core/nodes/night_time_bundle_summary_worker.py +``` + +New code should call: + +```python +run_night_summarize_time_bundle(...) +``` + +Old code may still call: + +```python +run_night_time_bundle_summary_worker(...) +``` + +## Data Naming + +New node id: + +```text +night_summarize_time_bundle +``` + +New prompt ref: + +```text +songryeon_core/prompts/night_summarize_time_bundle_v0.md +``` + +Failed frame data type: + +```text +node_output:night_summarize_time_bundle_frame +``` + +Successful graph summary node and edge data types are unchanged: + +```text +graph_memory:node:summary +graph_memory:edge:SUMMARY_OF +``` + +## Deliberately Not Changed + +- `NightTimeBundleSummaryFrame` schema name remains unchanged. +- Existing graph summary node IDs remain compatible. +- Summary generation policy is unchanged. +- R loop usage remains unopened. +- Source kind bundle summary worker is not implemented here. + +## Verification + +```powershell +python -m compileall songryeon_core main.py +``` + +Passed. + +```powershell +python -m pytest tests/test_order_166_night_time_bundle_summary_node.py -q +``` + +Passed: `4 passed`. + +```powershell +python main.py fast-test --profile graph +``` + +Passed: `FAST_TEST_OK`, graph pytest `78 passed`. + +## Remaining Note + +The old names still appear in compatibility wrappers and historical execution records. That is intentional. They should not be used as the primary names for new work. diff --git a/Administrative_Reform_1/05_Execution_Records/order_168_night_summarize_changed_source_leaves_2026_07_02_001.md b/Administrative_Reform_1/05_Execution_Records/order_168_night_summarize_changed_source_leaves_2026_07_02_001.md new file mode 100644 index 0000000..c5d1c5e --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_168_night_summarize_changed_source_leaves_2026_07_02_001.md @@ -0,0 +1,102 @@ +# ORDER 168 Night Summarize Changed Source Leaves - Execution Record + +## Summary + +Implemented ORDER_168_NIGHT_SUMMARIZE_CHANGED_SOURCE_LEAVES_V0. + +The MVP summarizes newly observed or changed code/document `raw_source` leaves one-to-one. + +Successful result shape: + +```text +raw_source leaf <- SUMMARY_OF <- source leaf SummaryGraphNode +``` + +## Key Changes + +- Added `NightSourceLeafSummaryFrame`. +- Added `songryeon_core/nodes/night_summarize_source_leaf.py`. +- Added `songryeon_core/prompts/night_summarize_source_leaf_v0.md`. +- Added `run_night_summarize_source_leaf(...)`. +- Added `run_night_summarize_changed_source_leaves(...)`. +- Added ORDER_168 graph fast-test coverage. + +## Selection Rule + +Code reads `SourceObservationLedgerFrame`. + +Summarized: + +- `observation_status=new_source_version` +- `observation_status=content_changed` + +Not summarized: + +- `observation_status=unchanged` + +This selection is code-checkable absolute information based on source observation records. + +## Text Snapshot Rule + +If the raw source has a text snapshot and non-empty text, the LLM summary runs. + +If the raw source has no text snapshot: + +```text +summary_status=skipped_no_text_snapshot +info_class=absolute +semantic_judgement_status=not_run +``` + +If the text snapshot is empty: + +```text +summary_status=skipped_empty_text +info_class=absolute +semantic_judgement_status=not_run +``` + +Code does not invent semantic summary text. + +## Metainfo + +Successful source leaf summaries are: + +```text +info_class=relative +source_mode=single_source +claim_alignment=single_absolute_record +semantic_judgement_status=ran +generated_by=LLM:*:night_summarize_source_leaf +``` + +Reason: + +One LLM summary is grounded in exactly one `raw_source` graph node and its copied text snapshot. + +## Validation + +Passed: + +```powershell +python -m compileall songryeon_core main.py +python -m pytest tests/test_order_168_night_summarize_changed_source_leaves.py -q +python main.py fast-test --profile graph +``` + +Observed: + +```text +ORDER_168 pytest: 6 passed +graph fast-test: FAST_TEST_OK, 84 passed +``` + +## Non-goals Preserved + +- Did not summarize unchanged observations. +- Did not summarize source kind bundles. +- Did not summarize conversation TimeBundles in this order. +- Did not create semantic axis. +- Did not auto-feed summaries into R loop or node_3. +- Did not delete or overwrite old summaries. +- Did not make code write semantic summary text. diff --git a/Administrative_Reform_1/05_Execution_Records/order_169_night_changed_source_summary_cli_2026_07_02_001.md b/Administrative_Reform_1/05_Execution_Records/order_169_night_changed_source_summary_cli_2026_07_02_001.md new file mode 100644 index 0000000..726205b --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_169_night_changed_source_summary_cli_2026_07_02_001.md @@ -0,0 +1,85 @@ +# ORDER 169 Night Changed Source Summary CLI - Execution Record + +## Summary + +Implemented ORDER_169_NIGHT_CHANGED_SOURCE_SUMMARY_CLI_V0. + +Added a manual opt-in CLI command: + +```powershell +python main.py night-summarize-changed-sources --root . --llm-mode qwen +``` + +Optional Vessel write: + +```powershell +python main.py night-summarize-changed-sources --root . --llm-mode qwen --write-vessel +``` + +## Runtime Flow + +```text +SongRyeon Core source manifest +-> graph source ingest with text snapshots +-> SourceObservationLedgerFrame +-> summarize new/content_changed raw_source leaves +-> graph export packet +-> Vessel write plan +-> optional Neo4j write +``` + +## Local Cache + +The command stores trace/data cache under: + +```text +.songryeon_core_cache/night_changed_sources/ +``` + +This is required so the next run can compare source observations and skip `unchanged` files. + +Without this cache, every fresh process would treat every file as `new_source_version`. + +## Metainfo + +Successful source leaf summaries remain: + +```text +info_class=relative +source_mode=single_source +claim_alignment=single_absolute_record +semantic_judgement_status=ran +``` + +Reason: + +Each summary is grounded in exactly one `raw_source` graph leaf and its copied text snapshot. + +The batch command summary itself is code-generated absolute runtime status. + +## Safety + +- Neo4j write is explicit opt-in via `--write-vessel`. +- Default `--llm-mode` is `off`. +- `--llm-mode fake` is deterministic test mode. +- `--llm-mode qwen` uses the normal Qwen adapter/runtime config. +- The command does not feed summaries into R loop or node_3. +- The command does not create semantic axis. +- The command does not overwrite old summaries. + +## Validation + +Passed: + +```powershell +python -m compileall songryeon_core main.py +python -m pytest tests/test_order_169_night_changed_source_summary_cli.py -q +python main.py fast-test --profile graph +``` + +Observed: + +```text +ORDER_169 pytest: 4 passed +graph fast-test: FAST_TEST_OK, 88 passed +``` diff --git a/Administrative_Reform_1/05_Execution_Records/order_170_night_changed_source_one_at_a_time_runner_2026_07_02_001.md b/Administrative_Reform_1/05_Execution_Records/order_170_night_changed_source_one_at_a_time_runner_2026_07_02_001.md new file mode 100644 index 0000000..c66e940 --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_170_night_changed_source_one_at_a_time_runner_2026_07_02_001.md @@ -0,0 +1,55 @@ +# ORDER 170 Execution Record: Night Changed Source One-At-A-Time Runner + +## Summary + +Implemented `night-summarize-changed-sources --one-at-a-time`. + +The default ORDER_169 all-at-once mode remains unchanged. The new one-at-a-time mode records a code-generated queue of `new_source_version` / `content_changed` raw source leaves and processes exactly one unprocessed leaf per command run. + +## Key Changes + +- Added order document: + - `Administrative_Reform_1/04_Orders/ORDER_170_NIGHT_CHANGED_SOURCE_ONE_AT_A_TIME_RUNNER_V0.md` +- Updated order index: + - `Administrative_Reform_1/04_Orders/README.md` +- Added source leaf selection helper: + - `songryeon_core/nodes/night_summarize_source_leaf.py` +- Added one-at-a-time runtime queue and step runner: + - `songryeon_core/runtime/night_changed_source_summary.py` +- Added CLI option: + - `python main.py night-summarize-changed-sources --one-at-a-time` +- Added tests: + - `tests/test_order_170_night_changed_source_one_at_a_time.py` + +## Design Notes + +- The queue is `info_class=absolute` and `semantic_judgement_status=not_run`. +- The queue is not an LLM summary. It is a code-generated list of graph leaf IDs selected from `SourceObservationLedgerFrame`. +- The LLM still writes only the summary text for one raw source leaf at a time. +- DataStore records are not updated in place. Progress is computed from existing summary node/frame records. +- Re-running the same one-at-a-time `batch_id` reads the existing queue instead of re-observing all source files, so unprocessed targets do not disappear as `unchanged`. +- Optional Vessel/Neo4j write remains explicit through `--write-vessel`. + +## Verification + +```powershell +python -m compileall songryeon_core main.py +# passed + +python -m pytest tests/test_order_170_night_changed_source_one_at_a_time.py tests/test_order_169_night_changed_source_summary_cli.py -q +# 7 passed + +python main.py fast-test --profile graph +# FAST_TEST_OK, 91 passed + +git diff --check +# passed +``` + +## Not Done + +- No semantic axis. +- No R loop activation. +- No source kind bundle or source ingest time bundle summary. +- No automatic failed-leaf retry policy. +- No deletion or overwrite of older summary records. diff --git a/Administrative_Reform_1/05_Execution_Records/order_171_night_token_budget_layer_summary_2026_07_02_001.md b/Administrative_Reform_1/05_Execution_Records/order_171_night_token_budget_layer_summary_2026_07_02_001.md new file mode 100644 index 0000000..33a4cd3 --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_171_night_token_budget_layer_summary_2026_07_02_001.md @@ -0,0 +1,59 @@ +# ORDER 171 Execution Record: Night Token Budget Layer Summary + +## Summary + +Implemented `night-summarize-token-layer`. + +This command reads existing active source leaf summary graph nodes, builds a code-generated budget queue, records one token-budget summary bundle graph node, and asks the LLM to summarize that one bundle. + +## Key Changes + +- Added order: + - `Administrative_Reform_1/04_Orders/ORDER_171_NIGHT_TOKEN_BUDGET_LAYER_SUMMARY_V0.md` +- Added prompt: + - `songryeon_core/prompts/night_summarize_token_budget_bundle_v0.md` +- Added worker: + - `songryeon_core/nodes/night_summarize_token_budget_bundle.py` +- Added runtime CLI backend: + - `songryeon_core/runtime/night_token_budget_layer_summary.py` +- Added CLI command: + - `python main.py night-summarize-token-layer` +- Added test: + - `tests/test_order_171_night_token_budget_layer_summary.py` +- Added graph fast-test coverage: + - `songryeon_core/runtime/fast_test.py` + +## Design Notes + +- v0 budget unit is explicitly `characters`, not hidden tokenizer output. +- The queue is code-generated absolute information. +- The bundle node is `graph_memory:node:token_budget_summary_bundle`. +- The bundle node contains source leaf summary nodes through `CONTAINS` edges. +- The successful upper summary is `info_class=mixed`. +- The original leaf summary nodes are not edited or deleted. +- One execution processes at most one bundle. +- Optional Neo4j write remains explicit through `--write-vessel`. + +## Verification + +```powershell +python -m compileall songryeon_core main.py +# passed + +python -m pytest tests/test_order_171_night_token_budget_layer_summary.py -q +# 3 passed + +python -m pytest tests/test_order_169_night_changed_source_summary_cli.py tests/test_order_170_night_changed_source_one_at_a_time.py tests/test_order_171_night_token_budget_layer_summary.py -q +# 10 passed + +python main.py fast-test --profile graph +# FAST_TEST_OK, 94 passed +``` + +## Not Done + +- No exact tokenizer adapter. +- No semantic axis. +- No R loop live route. +- No recursive all-layer summarization. +- No automatic failed-bundle retry policy. diff --git a/Administrative_Reform_1/05_Execution_Records/order_172_night_token_layer_auto_reduce_2026_07_02_001.md b/Administrative_Reform_1/05_Execution_Records/order_172_night_token_layer_auto_reduce_2026_07_02_001.md new file mode 100644 index 0000000..955e242 --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_172_night_token_layer_auto_reduce_2026_07_02_001.md @@ -0,0 +1,57 @@ +# ORDER 172 Execution Record: Night Token Layer Auto Reduce + +## Summary + +Implemented automatic token-layer reduction for `night-summarize-token-layer`. + +The command can now process more than one bundle in a bounded run and continue building higher summary layers until the current layer fits the target context budget, reaches `max_steps`, or reaches `max_layer_depth`. + +## Key Changes + +- Added order: + - `Administrative_Reform_1/04_Orders/ORDER_172_NIGHT_TOKEN_LAYER_AUTO_REDUCE_UNTIL_CONTEXT_BUDGET_V0.md` +- Updated CLI: + - `--until-context-budget` + - `--target-context-chars` + - `--max-layer-depth` + - `--max-steps` +- Extended runtime: + - `songryeon_core/runtime/night_token_budget_layer_summary.py` +- Added tests: + - `tests/test_order_172_night_token_layer_auto_reduce.py` +- Added graph fast-test coverage: + - `songryeon_core/runtime/fast_test.py` + +## Behavior + +- Default one-bundle-at-a-time behavior remains unchanged. +- Auto mode first completes any incomplete layer queue. +- Once a layer is complete, code sums that layer's successful summary text character counts. +- If the layer total is within `target_context_chars`, auto mode stops. +- If the layer is still too large and depth allows, code creates the next layer queue. +- Each step still processes only one bundle and saves after the step. +- v0 budget unit remains explicit `characters`, not hidden tokenizer output. + +## Verification + +```powershell +python -m compileall songryeon_core main.py +# passed + +python -m pytest tests/test_order_172_night_token_layer_auto_reduce.py -q +# 3 passed + +python -m pytest tests/test_order_171_night_token_budget_layer_summary.py tests/test_order_172_night_token_layer_auto_reduce.py -q +# 6 passed + +python main.py fast-test --profile graph +# FAST_TEST_OK, 97 passed +``` + +## Not Done + +- No exact tokenizer adapter. +- No semantic axis. +- No R loop live route. +- No failed-bundle retry policy. +- No deletion or overwrite of existing summary nodes. diff --git a/Administrative_Reform_1/05_Execution_Records/order_173_night_checkpointed_long_runner_2026_07_02_001.md b/Administrative_Reform_1/05_Execution_Records/order_173_night_checkpointed_long_runner_2026_07_02_001.md new file mode 100644 index 0000000..a7d9ceb --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_173_night_checkpointed_long_runner_2026_07_02_001.md @@ -0,0 +1,70 @@ +# ORDER 173 Execution Record: Night Checkpointed Long Runner v0 + +## 변경 요약 + +- `night-summarize-token-layer --until-context-budget` 자동 reducer에 checkpointed long-runner 옵션을 추가했다. +- 매 step 이후 `TraceStore` / `DataStore` 저장은 기존 구조를 유지하고, 추가로 progress JSONL을 남기게 했다. +- `--max-runtime-minutes`로 실행 시간 상한을 둘 수 있게 했다. +- `--vessel-write-mode none|every-step|at-end`를 추가했다. +- 기존 `--write-vessel`은 호환상 `every-step`으로 해석한다. +- 실패 bundle을 만나면 기본적으로 멈추는 `stop_on_failure=True` 정책을 추가했고, CLI에서는 `--no-stop-on-failure`로 끌 수 있게 했다. + +## 주요 파일 + +- `songryeon_core/runtime/night_token_budget_layer_summary.py` +- `main.py` +- `songryeon_core/runtime/fast_test.py` +- `tests/test_order_173_night_checkpointed_long_runner.py` +- `Administrative_Reform_1/04_Orders/ORDER_173_NIGHT_CHECKPOINTED_LONG_RUNNER_V0.md` +- `Administrative_Reform_1/04_Orders/README.md` + +## 구현 경계 + +- 새 요약 의미 판단은 추가하지 않았다. +- 정확 tokenizer는 만들지 않았다. +- semantic axis, R live route, failed bundle retry policy는 열지 않았다. +- 원본 graph node나 기존 summary node를 삭제/덮어쓰기 하지 않았다. + +## 검증 + +```powershell +python -m compileall songryeon_core main.py +``` + +통과. + +```powershell +python -m pytest tests/test_order_173_night_checkpointed_long_runner.py -q +``` + +`3 passed`. + +```powershell +python -m pytest tests/test_order_172_night_token_layer_auto_reduce.py tests/test_order_173_night_checkpointed_long_runner.py -q +``` + +`6 passed`. + +```powershell +git diff --check +``` + +통과. + +```powershell +python main.py fast-test --profile graph +``` + +`FAST_TEST_OK`, graph profile `100 passed`. + +```powershell +python main.py smoke-test +``` + +`SMOKE_TEST_OK`. + +```powershell +python -m pytest -q +``` + +604초 제한에서 timeout. 이번 ORDER_173 단독 테스트, ORDER_172+173 결합 테스트, graph fast-test, smoke-test는 통과했지만 전체 pytest 장시간 실행은 별도 재확인 대상이다. diff --git a/Administrative_Reform_1/05_Execution_Records/order_174_vessel_inspect_summary_layer_view_2026_07_02_001.md b/Administrative_Reform_1/05_Execution_Records/order_174_vessel_inspect_summary_layer_view_2026_07_02_001.md new file mode 100644 index 0000000..09b8f20 --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_174_vessel_inspect_summary_layer_view_2026_07_02_001.md @@ -0,0 +1,56 @@ +# ORDER 174 Execution Record: Vessel Inspect Summary Layer View v0 + +## 변경 요약 + +- `vessel-inspect`가 Neo4j Vessel의 `SummaryGraphNode`를 read-only로 조회하게 했다. +- 기존 `CoreEgo -> Time Axis -> Time Bundle -> Raw Capsule` tree 출력은 유지했다. +- inspect 결과 payload와 text renderer에 다음 절대정보를 추가했다. + - `summary_count` + - `active_summary_count` + - `invalidated_summary_count` + - `summary_count_by_data_kind` + - `summary_count_by_depth` + - `summary_sample_items` + - `summary_lines` +- summary sample은 Neo4j node property와 `payload_json`에서 읽은 값만 표시한다. +- 새 요약 생성, 요약 품질 평가, semantic axis, R live route는 열지 않았다. + +## 주요 파일 + +- `songryeon_core/core/graph_vessel_inspect.py` +- `songryeon_core/runtime/graph_vessel_inspect.py` +- `songryeon_core/runtime/fast_test.py` +- `tests/test_order_163_vessel_inspect_manual_walk.py` +- `tests/test_order_174_vessel_inspect_summary_layer_view.py` +- `Administrative_Reform_1/04_Orders/ORDER_174_VESSEL_INSPECT_SUMMARY_LAYER_VIEW_V0.md` +- `Administrative_Reform_1/04_Orders/README.md` + +## 검증 + +```powershell +python -m compileall songryeon_core main.py +``` + +통과. + +```powershell +python -m pytest tests/test_order_163_vessel_inspect_manual_walk.py tests/test_order_174_vessel_inspect_summary_layer_view.py -q +``` + +`10 passed`. + +```powershell +python main.py fast-test --profile graph +``` + +`FAST_TEST_OK`, graph profile `102 passed`. + +## 사용 명령 + +Neo4j env가 설정된 PowerShell에서: + +```powershell +python main.py vessel-inspect --database neo4j --format text +``` + +이제 출력에는 기본 시간축 tree 아래에 `Summary samples` 섹션이 함께 표시된다. diff --git a/Administrative_Reform_1/05_Execution_Records/order_175_vessel_backed_r_graph_read_packet_2026_07_02_001.md b/Administrative_Reform_1/05_Execution_Records/order_175_vessel_backed_r_graph_read_packet_2026_07_02_001.md new file mode 100644 index 0000000..e854584 --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_175_vessel_backed_r_graph_read_packet_2026_07_02_001.md @@ -0,0 +1,60 @@ +# ORDER 175 Execution Record: Vessel-Backed R Graph Read Packet + +## Summary + +ORDER_175를 구현했다. + +R루프가 실제 Neo4j Vessel 그래프를 바로 의미 판단하지 않고 읽기 전용 후보 봉투로 받을 수 있도록 `vessel-r-read-packet` CLI와 `RLoopVesselReadPacketFrame` 생성 경계를 추가했다. + +## Changed Files + +- `songryeon_core/core/r_loop_vessel_read_packet.py` +- `songryeon_core/runtime/r_loop_vessel_read_packet.py` +- `main.py` +- `songryeon_core/runtime/fast_test.py` +- `tests/test_order_175_vessel_backed_r_read_packet.py` +- `Administrative_Reform_1/04_Orders/ORDER_175_VESSEL_BACKED_R_GRAPH_READ_PACKET_V0.md` +- `Administrative_Reform_1/04_Orders/README.md` + +## Behavior + +- `CoreEgo -> TimeAxis` 아래의 `TimeBundle` / `SourceIngestBundle`을 R entry 후보로 읽는다. +- `SummaryGraphNode` 중 `summary_status=ran`이고 `validity_status=active`인 summary만 R summary 후보로 읽는다. +- invalidated summary와 failed/skipped summary는 R 후보에서 제외한다. +- summary text는 Neo4j에 저장된 `payload_json.summary_text`를 복사한다. +- code는 관련성, 중요도, 의미 선택을 판단하지 않는다. +- packet은 `generated_by=CODE:R_LOOP_VESSEL_READ_PACKET_BUILDER`, `info_class=absolute`, `semantic_judgement_status=not_run`으로 고정한다. + +## Verification + +```powershell +python -m compileall songryeon_core main.py +``` + +통과. + +```powershell +python -m pytest tests/test_order_175_vessel_backed_r_read_packet.py -q +``` + +결과: `6 passed in 0.12s` + +```powershell +python main.py fast-test --profile graph +``` + +결과: `FAST_TEST_OK`, `108 passed in 52.51s` + +```powershell +git diff --check +``` + +통과. + +## Non-goals Preserved + +- R route를 기본 live route로 열지 않았다. +- R1/R2/R3 LLM을 호출하지 않았다. +- semantic axis를 만들지 않았다. +- Neo4j에 새 노드/관계를 쓰지 않았다. +- summary를 새로 생성/수정/무효화하지 않았다. diff --git a/Administrative_Reform_1/05_Execution_Records/order_176_vessel_r_one_step_traversal_2026_07_02_001.md b/Administrative_Reform_1/05_Execution_Records/order_176_vessel_r_one_step_traversal_2026_07_02_001.md new file mode 100644 index 0000000..efc90fa --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_176_vessel_r_one_step_traversal_2026_07_02_001.md @@ -0,0 +1,93 @@ +# ORDER 176 Execution Record: Vessel R One-Step Traversal + +## Summary + +ORDER_176을 구현했다. + +ORDER_175의 `RLoopVesselReadPacketFrame`을 입력으로 받아 R1/R2/R3가 한 단계만 Vessel 후보를 탐색하는 실험용 R one-step traversal을 추가했다. + +## Changed Files + +- `songryeon_core/prompts/r1_vessel_goal_setter_v0.md` +- `songryeon_core/prompts/r2_vessel_node_selector_v0.md` +- `songryeon_core/prompts/r3_vessel_inspector_v0.md` +- `songryeon_core/loops/r_loop_vessel_one_step.py` +- `songryeon_core/runtime/r_loop_vessel_one_step.py` +- `main.py` +- `songryeon_core/runtime/fast_test.py` +- `tests/test_order_176_vessel_r_one_step_traversal.py` +- `Administrative_Reform_1/04_Orders/ORDER_176_VESSEL_R_ONE_STEP_TRAVERSAL_V0.md` +- `Administrative_Reform_1/04_Orders/README.md` + +## Behavior + +- 새 CLI `vessel-r-one-step`을 추가했다. +- 실행 흐름은 다음과 같다. + - read-only Vessel packet 생성 + - R1 LLM goal/granularity 판단 + - code one-step budget 생성 + - R2 LLM candidate 선택 + - code가 선택 ID가 packet 안에 있는지 검증 + - R3 LLM sufficiency/granularity/branch 판단 + - code continuation/return summary/result frame 기록 +- R2가 packet 밖 ID를 고르면 schema failure로 닫는다. +- R3가 child node ID를 invent해도 code는 사용하지 않고 packet 안의 `source_graph_node_ids` / `target_graph_node_id`만 복사한다. +- adapter가 없으면 선택 fallback을 만들지 않고 `adapter_missing` 실패 frame으로 닫는다. + +## Metainfo Boundary + +- R1/R2/R3의 판단 frame은 `generated_by=LLM:*`, `info_class=mixed`, `semantic_judgement_status=ran`이다. +- budget/continuation/return summary/result frame은 code 생성 절대정보이며 `semantic_judgement_status=not_run`이다. +- code는 후보 관련성, 충분성, branch 의미 판단을 대신하지 않는다. + +## Verification + +```powershell +python -m compileall songryeon_core main.py +``` + +통과. + +```powershell +python -m pytest tests/test_order_176_vessel_r_one_step_traversal.py -q +``` + +결과: `6 passed in 0.15s` + +```powershell +python main.py fast-test --profile graph +``` + +결과: `FAST_TEST_OK`, `114 passed in 51.19s` + +```powershell +python main.py smoke-test +``` + +결과: `SMOKE_TEST_OK` + +```powershell +git diff --check +``` + +통과. + +## Manual CLI Note + +Codex 셸에서 아래 명령을 추가로 시도했다. + +```powershell +python main.py vessel-r-one-step "송련 Core의 그래프 기억 구조를 한 단계만 탐색해줘" --llm-mode fake --format text --allow-no-auth +``` + +이 셸에는 사용자의 Neo4j env가 로드되어 있지 않아 기본 `bolt://localhost:7687`로 연결을 시도했고, 로컬 Neo4j 연결 거부로 `read_packet` 단계에서 실패했다. +이는 ORDER_176 deterministic 테스트 실패가 아니라 수동 CLI 실행 환경 문제다. +사용자 PowerShell에서 `.env.vessel.local.ps1`을 로드한 뒤 `--database neo4j`로 다시 실행하면 실제 Vessel에 대해 확인할 수 있다. + +## Non-goals Preserved + +- 기본 live `qwen-chat` route=R은 열지 않았다. +- multi-step Vessel R traversal은 열지 않았다. +- R 결과를 node_3 최종 답변에 자동 주입하지 않았다. +- semantic axis를 만들지 않았다. +- Neo4j write는 수행하지 않았다. diff --git a/Administrative_Reform_1/05_Execution_Records/order_177_r1_candidate_text_blindness_2026_07_02_001.md b/Administrative_Reform_1/05_Execution_Records/order_177_r1_candidate_text_blindness_2026_07_02_001.md new file mode 100644 index 0000000..649e77a --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_177_r1_candidate_text_blindness_2026_07_02_001.md @@ -0,0 +1,72 @@ +# ORDER 177 Execution Record: R1 Candidate Text Blindness + +## Summary + +ORDER_177을 구현했다. + +R1이 Vessel 후보 summary 본문이나 개별 후보 샘플에 끌려 사용자 질문 밖으로 graph search goal이 새는 문제를 줄이기 위해, R1 input을 packet-level count/kind/depth/budget 중심으로 제한했다. + +## Changed Files + +- `songryeon_core/loops/r_loop_vessel_one_step.py` +- `songryeon_core/runtime/fast_test.py` +- `tests/test_order_177_r1_candidate_text_blindness.py` +- `Administrative_Reform_1/04_Orders/ORDER_177_R1_CANDIDATE_TEXT_BLINDNESS_V0.md` +- `Administrative_Reform_1/04_Orders/README.md` + +## Behavior + +- R1 input에서 candidate sample과 summary text를 제거했다. +- R1은 다음만 본다. + - user question + - read packet id + - entry candidate count + - summary candidate count + - summary count by data kind + - summary count by depth + - one-step policy + - candidate text visibility policy note +- R2는 기존처럼 후보 목록과 summary text를 볼 수 있다. +- R3는 R2가 선택한 후보 하나의 record를 볼 수 있다. +- R1 graph_search_goal이 user question의 명시 ASCII anchor를 전혀 보존하지 않으면 schema failure로 닫는다. +- 이 anchor 검사는 의미 판단이 아니라 사용자 질문에 명시된 영문/숫자 토큰을 완전히 버리는 drift를 막는 최소 구조 검사다. + +## Verification + +```powershell +python -m compileall songryeon_core main.py +``` + +통과. + +```powershell +python -m pytest tests/test_order_176_vessel_r_one_step_traversal.py tests/test_order_177_r1_candidate_text_blindness.py -q +``` + +결과: `9 passed in 0.15s` + +```powershell +python main.py fast-test --profile graph +``` + +결과: `FAST_TEST_OK`, `117 passed in 51.07s` + +```powershell +python main.py smoke-test +``` + +결과: `SMOKE_TEST_OK` + +```powershell +git diff --check +``` + +통과. + +## Non-goals Preserved + +- R route를 기본 live route로 열지 않았다. +- R2/R3의 선택/충분성 품질을 크게 바꾸지 않았다. +- multi-step Vessel traversal을 열지 않았다. +- Neo4j write를 수행하지 않았다. +- summary 생성/수정/무효화를 하지 않았다. diff --git a/Administrative_Reform_1/05_Execution_Records/order_178_r_vessel_candidate_layer_surface_2026_07_02_001.md b/Administrative_Reform_1/05_Execution_Records/order_178_r_vessel_candidate_layer_surface_2026_07_02_001.md new file mode 100644 index 0000000..1ed5519 --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_178_r_vessel_candidate_layer_surface_2026_07_02_001.md @@ -0,0 +1,85 @@ +# ORDER 178 Execution Record: R Vessel Candidate Layer Surface + +## Summary + +ORDER_178을 구현했다. + +R1 후보 텍스트 차단 이후에도 R2가 flat 50개 후보판을 직접 받아 첫 후보에 끌리는 문제가 남아 있었다. 이번 패치는 R2 앞에 code-generated absolute candidate layer surface를 추가해, R2가 먼저 surface/table-of-contents shelf를 고른 뒤 그 안의 graph node를 고르게 했다. + +## Changed Files + +- `songryeon_core/core/r_loop_vessel_read_packet.py` +- `songryeon_core/loops/r_loop_vessel_one_step.py` +- `songryeon_core/prompts/r2_vessel_node_selector_v0.md` +- `songryeon_core/runtime/fast_test.py` +- `tests/test_order_176_vessel_r_one_step_traversal.py` +- `tests/test_order_177_r1_candidate_text_blindness.py` +- `tests/test_order_178_r_vessel_candidate_layer_surface.py` +- `Administrative_Reform_1/04_Orders/ORDER_178_R_VESSEL_CANDIDATE_LAYER_SURFACE_V0.md` +- `Administrative_Reform_1/04_Orders/README.md` + +## Behavior + +- `RLoopVesselCandidateLayerSurfaceFrame`을 추가했다. + - generated_by: `CODE:R_LOOP_VESSEL_CANDIDATE_LAYER_SURFACE_BUILDER` + - info_class: `absolute` + - semantic_judgement_status: `not_run` +- surface frame은 다음 absolute fields만 기준으로 후보를 묶는다. + - entry candidate kind + - summary data kind + - summary depth + - info class +- surface frame 자체에는 `summary_text`를 넣지 않는다. +- R2 input은 flat `summary_candidate_records`/`entry_candidate_records` 대신: + - `candidate_layer_surface_frame` + - `available_surface_ids` + - `candidate_records_by_surface` + 를 받는다. +- R2 output은 이제 `selected_surface_id`와 `selected_graph_node_id`를 함께 내야 한다. +- code validator는 `selected_graph_node_id`가 selected surface 안에 없으면 schema failure로 닫는다. +- Vessel read packet은 active summaries를 flat first-N으로 자르지 않고, data-kind/depth/info-class group을 round-robin으로 담는다. + +## Metadata Boundary + +Code는 surface grouping과 ID membership만 확정한다. 어떤 surface 또는 candidate가 질문에 의미상 맞는지는 R2 LLM 판단으로 남겼다. + +## Verification + +```powershell +python -m compileall songryeon_core main.py +``` + +통과. + +```powershell +python -m pytest tests/test_order_175_vessel_backed_r_read_packet.py tests/test_order_176_vessel_r_one_step_traversal.py tests/test_order_177_r1_candidate_text_blindness.py tests/test_order_178_r_vessel_candidate_layer_surface.py -q +``` + +결과: `19 passed in 0.24s` + +```powershell +python main.py fast-test --profile graph +``` + +결과: `FAST_TEST_OK`, `121 passed in 68.27s` + +```powershell +python main.py smoke-test +``` + +결과: `SMOKE_TEST_OK` + +```powershell +git diff --check +``` + +통과. + +## Non-goals Preserved + +- R route를 기본 live route로 열지 않았다. +- multi-step R traversal을 열지 않았다. +- code가 semantic surface relevance를 판단하지 않았다. +- Neo4j schema/write shape를 바꾸지 않았다. +- night summary 생성/무효화 로직을 건드리지 않았다. + diff --git a/Administrative_Reform_1/05_Execution_Records/order_179_r2_vessel_selection_id_disambiguation_2026_07_02_001.md b/Administrative_Reform_1/05_Execution_Records/order_179_r2_vessel_selection_id_disambiguation_2026_07_02_001.md new file mode 100644 index 0000000..650f556 --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_179_r2_vessel_selection_id_disambiguation_2026_07_02_001.md @@ -0,0 +1,69 @@ +# ORDER 179 Execution Record: R2 Vessel Selection ID Disambiguation + +## Summary + +ORDER_179를 구현했다. + +ORDER_178 live Qwen test에서 R2가 schema failure로 닫혔다. 실패 원인은 `selected_graph_node_id`가 `available_graph_node_ids` 안에 없다는 것이었다. 이는 R2가 선택해야 하는 summary/entry candidate ID와 설명용 target/source ID를 혼동했을 가능성이 높다. + +## Changed Files + +- `songryeon_core/loops/r_loop_vessel_one_step.py` +- `songryeon_core/runtime/r_loop_vessel_one_step.py` +- `songryeon_core/prompts/r2_vessel_node_selector_v0.md` +- `songryeon_core/runtime/fast_test.py` +- `tests/test_order_179_r2_vessel_selection_id_disambiguation.py` +- `Administrative_Reform_1/04_Orders/ORDER_179_R2_VESSEL_SELECTION_ID_DISAMBIGUATION_V0.md` +- `Administrative_Reform_1/04_Orders/README.md` + +## Behavior + +- R2 candidate records now expose `graph_node_id` as the direct selectable ID. +- R2 candidate records no longer expose: + - `target_graph_node_id` + - `source_graph_node_ids` + - `summary_node_id` + - `candidate_node_id` +- R2 prompt now says to copy `selected_graph_node_id` from candidate record `graph_node_id`. +- R2 prompt now explicitly says not to use target/source/data IDs as selected graph node IDs. +- Failed R2 schema validation now stores and renders `failure_payload_summary`: + - selected surface ID + - selected graph node ID + - whether selected surface ID was allowed + - whether selected graph node ID was globally allowed + - whether selected graph node ID belonged to the selected surface + +## Verification + +```powershell +python -m compileall songryeon_core main.py +``` + +통과. + +```powershell +python -m pytest tests/test_order_176_vessel_r_one_step_traversal.py tests/test_order_177_r1_candidate_text_blindness.py tests/test_order_178_r_vessel_candidate_layer_surface.py tests/test_order_179_r2_vessel_selection_id_disambiguation.py -q +``` + +결과: `15 passed in 0.22s` + +```powershell +python main.py fast-test --profile graph +``` + +결과: `FAST_TEST_OK`, `123 passed in 62.29s` + +```powershell +git diff --check +``` + +통과. + +## Non-goals Preserved + +- R2 validator를 약화하지 않았다. +- invalid selected graph node ID를 허용하지 않았다. +- code가 semantic fallback candidate를 고르지 않았다. +- R2a/R2b two-stage selector는 아직 열지 않았다. +- multi-step R traversal은 열지 않았다. + diff --git a/Administrative_Reform_1/05_Execution_Records/order_180_r2_prompt_example_id_removal_2026_07_02_001.md b/Administrative_Reform_1/05_Execution_Records/order_180_r2_prompt_example_id_removal_2026_07_02_001.md new file mode 100644 index 0000000..7107582 --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_180_r2_prompt_example_id_removal_2026_07_02_001.md @@ -0,0 +1,56 @@ +# ORDER 180 Execution Record: R2 Prompt Example ID Removal + +## Summary + +ORDER_180을 구현했다. + +ORDER_179 이후 live Qwen test에서 R2가 prompt sample에 있던 `graph:summary:source_leaf:example`을 그대로 `selected_graph_node_id`로 출력했다. 이는 runtime 후보 선택 실패라기보다 prompt example ID leakage였다. + +## Changed Files + +- `songryeon_core/prompts/r2_vessel_node_selector_v0.md` +- `songryeon_core/runtime/fast_test.py` +- `tests/test_order_180_r2_prompt_example_id_removal.py` +- `Administrative_Reform_1/04_Orders/ORDER_180_R2_PROMPT_EXAMPLE_ID_REMOVAL_V0.md` +- `Administrative_Reform_1/04_Orders/README.md` + +## Behavior + +- R2 prompt에서 concrete fake graph ID가 들어간 JSON code block을 제거했다. +- R2 prompt는 이제 output key contract만 설명한다. +- R2 prompt는 `selected_graph_node_id`를 runtime candidate record의 `graph_node_id`에서 복사하라고 말한다. +- R2 prompt는 selectable ID가 runtime input payload에만 있다고 명시한다. + +## Verification + +```powershell +python -m compileall songryeon_core main.py +``` + +통과. + +```powershell +python -m pytest tests/test_order_176_vessel_r_one_step_traversal.py tests/test_order_177_r1_candidate_text_blindness.py tests/test_order_178_r_vessel_candidate_layer_surface.py tests/test_order_179_r2_vessel_selection_id_disambiguation.py tests/test_order_180_r2_prompt_example_id_removal.py -q +``` + +결과: `16 passed in 0.23s` + +```powershell +python main.py fast-test --profile graph +``` + +결과: `FAST_TEST_OK`, `124 passed in 50.91s` + +```powershell +git diff --check +``` + +통과. + +## Non-goals Preserved + +- R2 validator를 약화하지 않았다. +- code semantic fallback selection을 만들지 않았다. +- R2a/R2b two-stage selector는 아직 열지 않았다. +- Neo4j data/write shape를 바꾸지 않았다. + diff --git a/Administrative_Reform_1/05_Execution_Records/order_181_r2_official_selection_ref_map_2026_07_02_001.md b/Administrative_Reform_1/05_Execution_Records/order_181_r2_official_selection_ref_map_2026_07_02_001.md new file mode 100644 index 0000000..82c579b --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_181_r2_official_selection_ref_map_2026_07_02_001.md @@ -0,0 +1,76 @@ +# ORDER 181 Execution Record: R2 Official Selection Ref Map + +## Summary + +ORDER_181을 구현했다. + +ORDER_180 이후 live Qwen test에서 R2가 긴 runtime ID 대신 `surface_003`, `node_073` 같은 짧은 label을 임의 생성했다. 이는 LLM이 긴 graph ID를 직접 복사하는 구조가 취약하다는 신호였다. + +이번 패치는 code가 공식 선택 ref를 제공하고, R2는 ref만 선택하게 만들었다. + +## Changed Files + +- `songryeon_core/loops/r_loop_vessel_one_step.py` +- `songryeon_core/prompts/r2_vessel_node_selector_v0.md` +- `songryeon_core/runtime/fast_test.py` +- `tests/test_order_176_vessel_r_one_step_traversal.py` +- `tests/test_order_177_r1_candidate_text_blindness.py` +- `tests/test_order_178_r_vessel_candidate_layer_surface.py` +- `tests/test_order_179_r2_vessel_selection_id_disambiguation.py` +- `tests/test_order_180_r2_prompt_example_id_removal.py` +- `tests/test_order_181_r2_official_selection_ref_map.py` +- `Administrative_Reform_1/04_Orders/ORDER_181_R2_OFFICIAL_SELECTION_REF_MAP_V0.md` +- `Administrative_Reform_1/04_Orders/README.md` + +## Behavior + +- R2 input now exposes official short refs: + - `available_surface_refs` + - `candidate_layer_surface_ref_records` + - `candidate_records_by_surface_ref` +- R2 output now uses: + - `selected_surface_ref` + - `selected_node_ref` +- Actual graph IDs are hidden from R2 candidate records. +- Code maps official refs back to actual `surface_id` and `graph_node_id`. +- Existing R2/R3 frames still store actual graph IDs after validation. +- Arbitrary invented refs still fail validation. + +## Metadata Boundary + +The ref map is code-generated absolute information. It does not rank surfaces or candidates semantically. R2 still makes the semantic selection; code only validates official ref membership and translates refs to actual IDs. + +## Verification + +```powershell +python -m compileall songryeon_core main.py +``` + +통과. + +```powershell +python -m pytest tests/test_order_176_vessel_r_one_step_traversal.py tests/test_order_177_r1_candidate_text_blindness.py tests/test_order_178_r_vessel_candidate_layer_surface.py tests/test_order_179_r2_vessel_selection_id_disambiguation.py tests/test_order_180_r2_prompt_example_id_removal.py tests/test_order_181_r2_official_selection_ref_map.py -q +``` + +결과: `18 passed in 0.32s` + +```powershell +python main.py fast-test --profile graph +``` + +결과: `FAST_TEST_OK`, `126 passed in 50.51s` + +```powershell +git diff --check +``` + +통과. + +## Non-goals Preserved + +- Arbitrary `surface_003`/`node_073` style invented refs are not accepted unless code supplied them. +- R2 validator was not weakened. +- Code does not choose a semantic fallback. +- R2a/R2b and multi-step R traversal remain unopened. +- Neo4j data/write shape was not changed. + diff --git a/Administrative_Reform_1/05_Execution_Records/order_182_r_core_ego_start_selection_surface_2026_07_02_001.md b/Administrative_Reform_1/05_Execution_Records/order_182_r_core_ego_start_selection_surface_2026_07_02_001.md new file mode 100644 index 0000000..4602d8f --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_182_r_core_ego_start_selection_surface_2026_07_02_001.md @@ -0,0 +1,72 @@ +# ORDER 182 Execution Record: R CoreEgo Start Selection Surface + +## Summary + +ORDER_182를 구현했다. + +ORDER_181로 R2의 공식 ref 선택 문제는 해결됐지만, live R one-step에서 R2가 여전히 첫 단계부터 source leaf summary와 token layer summary 후보를 넓게 보고 있었다. 이는 사용자가 의도한 `CoreEgo -> Time Axis -> Bundle -> Leaf/Summary` 하향 탐색과 맞지 않았다. + +이번 패치는 첫 R2 선택 화면을 CoreEgo 시작 entry 후보로 제한했다. + +## Changed Files + +- `songryeon_core/core/r_loop_vessel_read_packet.py` +- `songryeon_core/loops/r_loop_vessel_one_step.py` +- `songryeon_core/prompts/r2_vessel_node_selector_v0.md` +- `songryeon_core/runtime/fast_test.py` +- `tests/test_order_176_vessel_r_one_step_traversal.py` +- `tests/test_order_177_r1_candidate_text_blindness.py` +- `tests/test_order_178_r_vessel_candidate_layer_surface.py` +- `tests/test_order_179_r2_vessel_selection_id_disambiguation.py` +- `tests/test_order_181_r2_official_selection_ref_map.py` +- `tests/test_order_182_r_core_ego_start_surface.py` +- `Administrative_Reform_1/04_Orders/ORDER_182_R_CORE_EGO_START_SELECTION_SURFACE_V0.md` +- `Administrative_Reform_1/04_Orders/README.md` + +## Behavior + +- R read packet now recognizes `TimeAxis` entry rows as `candidate_kind=time_axis`. +- Neo4j entry read query includes the `TimeAxis` row before TimeBundle/SourceIngestBundle rows. +- First R2 candidate surface now uses CoreEgo-start entry candidates only. +- If a TimeAxis candidate exists, the first R2 surface exposes TimeAxis only. +- If no TimeAxis candidate exists, first R2 surface falls back to existing entry bundle candidates. +- Summary candidates remain in the read packet, but first R2 payload does not expose `summary_text`, `summary_node_id`, `source_leaf_summary`, or `token_budget_bundle_summary`. +- R2 still selects official refs only; code maps refs to graph IDs after validation. + +## Metadata Boundary + +The first-step surface is code-generated absolute information. It does not decide which graph branch is semantically best. R2 still performs semantic selection over visible official refs. + +## Verification + +```powershell +python -m pytest tests/test_order_175_vessel_backed_r_read_packet.py tests/test_order_176_vessel_r_one_step_traversal.py tests/test_order_177_r1_candidate_text_blindness.py tests/test_order_178_r_vessel_candidate_layer_surface.py tests/test_order_179_r2_vessel_selection_id_disambiguation.py tests/test_order_180_r2_prompt_example_id_removal.py tests/test_order_181_r2_official_selection_ref_map.py tests/test_order_182_r_core_ego_start_surface.py -q +``` + +결과: `26 passed in 0.35s` + +```powershell +python -m compileall songryeon_core main.py +``` + +통과. + +```powershell +python main.py fast-test --profile graph +``` + +결과: `FAST_TEST_OK`, `128 passed in 60.09s` + +```powershell +git diff --check +``` + +통과. + +## Non-goals Preserved + +- R2a/R2b 또는 A/B 선택 구조를 만들지 않았다. +- full multi-step R traversal은 열지 않았다. +- summary 후보를 read packet에서 삭제하지 않았다. +- R2 validator를 약화하지 않았다. +- node_1 routing, node_3 answer, W/R scheduler, 외부 DB write schema는 바꾸지 않았다. diff --git a/Administrative_Reform_1/05_Execution_Records/order_183_r_vessel_hierarchical_child_candidate_surface_2026_07_02_001.md b/Administrative_Reform_1/05_Execution_Records/order_183_r_vessel_hierarchical_child_candidate_surface_2026_07_02_001.md new file mode 100644 index 0000000..e8d9f4c --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_183_r_vessel_hierarchical_child_candidate_surface_2026_07_02_001.md @@ -0,0 +1,78 @@ +# ORDER 183 Execution Record: R Vessel Hierarchical Child Candidate Surface + +## Summary + +ORDER_183을 구현했다. + +ORDER_182로 R2 첫 화면은 CoreEgo entry layer만 보게 되었지만, 선택한 노드 아래의 다음 계층 후보를 구조화해서 남기는 장부가 Vessel one-step에는 없었다. 이번 패치는 선택된 Vessel graph node의 direct child candidates를 code가 복사하고, 기존 R loop schema인 `RGraphTraversalCandidateSurfaceFrame`으로 기록하게 만든다. + +## Changed Files + +- `songryeon_core/core/r_loop_vessel_read_packet.py` +- `songryeon_core/loops/r_loop_vessel_one_step.py` +- `songryeon_core/prompts/r3_vessel_inspector_v0.md` +- `songryeon_core/runtime/r_loop_vessel_one_step.py` +- `songryeon_core/runtime/fast_test.py` +- `tests/test_order_183_r_vessel_hierarchical_child_surface.py` +- `Administrative_Reform_1/04_Orders/ORDER_183_R_VESSEL_HIERARCHICAL_CHILD_CANDIDATE_SURFACE_V0.md` +- `Administrative_Reform_1/04_Orders/README.md` + +## Behavior + +- R Vessel read packet entry candidates now preserve: + - `source_graph_node_ids` + - `parent_graph_node_ids` +- Neo4j read query can include: + - `TimeAxis` + - `TimeBundle` + - `SourceIngestBundle` + - `SourceKindBundle` + - `RawSource` + - `token_budget_summary_bundle` +- When R2 selects a node, code derives `hierarchy_child_node_ids` and `hierarchy_child_candidate_records`. +- R3 input now includes: + - `hierarchy_child_candidate_records` + - `hierarchy_child_candidate_count` + - `hierarchy_read_policy` +- After R3, code records `RGraphTraversalCandidateSurfaceFrame` with child candidate IDs. +- Runtime text output shows `hierarchy_child_candidate_count` and up to 20 child node IDs. + +## Metadata Boundary + +The hierarchy child candidate surface is absolute information. Code copies graph IDs and relationships from read packet records and graph payloads. It does not choose which child is semantically best. + +R3 may judge whether the selected node is sufficient or whether deeper traversal is needed. That remains LLM-generated mixed information. + +## Verification + +```powershell +python -m pytest tests/test_order_183_r_vessel_hierarchical_child_surface.py -q +``` + +결과: `3 passed in 0.13s` + +```powershell +python -m pytest tests/test_order_175_vessel_backed_r_read_packet.py tests/test_order_176_vessel_r_one_step_traversal.py tests/test_order_177_r1_candidate_text_blindness.py tests/test_order_178_r_vessel_candidate_layer_surface.py tests/test_order_179_r2_vessel_selection_id_disambiguation.py tests/test_order_180_r2_prompt_example_id_removal.py tests/test_order_181_r2_official_selection_ref_map.py tests/test_order_182_r_core_ego_start_surface.py tests/test_order_183_r_vessel_hierarchical_child_surface.py -q +``` + +결과: `29 passed in 0.36s` + +```powershell +python -m compileall songryeon_core main.py +``` + +통과. + +```powershell +python main.py fast-test --profile graph +``` + +결과: `FAST_TEST_OK`, `131 passed in 54.65s` + +## Non-goals Preserved + +- Full automatic multi-step R traversal은 열지 않았다. +- R2a/R2b 또는 A/B 선택 구조는 만들지 않았다. +- Code가 semantic child choice를 대신하지 않았다. +- R2 official ref validation은 약화하지 않았다. +- node_3 최종 답변 연결은 열지 않았다. diff --git a/Administrative_Reform_1/05_Execution_Records/order_184_r_vessel_multi_step_traversal_mvp_2026_07_02_001.md b/Administrative_Reform_1/05_Execution_Records/order_184_r_vessel_multi_step_traversal_mvp_2026_07_02_001.md new file mode 100644 index 0000000..0cd9dea --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_184_r_vessel_multi_step_traversal_mvp_2026_07_02_001.md @@ -0,0 +1,73 @@ +# ORDER 184 R Vessel Multi-Step Traversal MVP 실행 기록 + +## 결론 + +ORDER_184를 구현했다. + +R Vessel 탐색은 이제 독립 CLI에서 다음 흐름을 반복할 수 있다. + +```text +R1 목표 설정 +-> R2 현재 후보 surface에서 공식 ref 선택 +-> R3 선택 노드 검사 +-> code가 하위 후보 surface 생성 +-> budget/충분성에 따라 R2로 재진입 또는 중단 +``` + +## 구현 내용 + +- `run_r_loop_vessel_traverse(...)` 추가. +- `RLoopVesselTraverseResultFrame` 추가. +- `RLoopVesselTraverseFakeLLMAdapter` 추가. +- `vessel-r-traverse` CLI 추가. +- traversal text renderer 추가. +- R1/R2/R3 prompt의 one-step 전용 표현을 traversal policy 기준 표현으로 조정. +- `fast-test --profile graph` 대상에 ORDER_184 pytest 추가. + +## 메타정보 경계 + +- code가 생성하는 것: + - candidate layer surface + - child candidate surface + - traversal budget + - continuation record + - traversal result frame +- LLM이 판단하는 것: + - R1 graph search goal + - R2 candidate selection + - R3 sufficiency/granularity/branch judgment +- code는 의미적으로 “어떤 후보가 좋다”를 고르지 않는다. +- Neo4j graph mutation은 하지 않았다. + +## 검증 + +```powershell +python -m pytest tests/test_order_184_r_vessel_multi_step_traversal.py -q +# 2 passed + +python -m compileall songryeon_core main.py +# passed + +python main.py vessel-r-traverse --help +# command registered + +python main.py fast-test --profile graph --skip-compileall +# FAST_TEST_OK +# 133 passed + +python -m pytest -q +# 264 passed in 753.54s + +python main.py smoke-test +# SMOKE_TEST_OK + +git diff --check +# passed +``` + +## 남은 범위 + +- 아직 node_1 route=R live field loop에는 연결하지 않았다. +- 아직 R 결과를 node_0 memory packet이나 node_3 final answer에 자동 주입하지 않았다. +- 아직 Neo4j에 R traversal trace를 백업하지 않는다. +- Qwen live traversal 품질은 별도 수동 테스트 대상이다. diff --git a/Administrative_Reform_1/05_Execution_Records/order_185_r_terminal_material_guard_2026_07_03_001.md b/Administrative_Reform_1/05_Execution_Records/order_185_r_terminal_material_guard_2026_07_03_001.md new file mode 100644 index 0000000..4072b87 --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_185_r_terminal_material_guard_2026_07_03_001.md @@ -0,0 +1,78 @@ +# ORDER 185 R Terminal Material Budget And Early Stop Guard 실행 기록 + +## 결론 + +ORDER_185를 구현했다. + +R Vessel multi-step traversal은 이제 R3가 `sufficient`를 내더라도, terminal material을 아직 하나도 보지 않았고 하위 후보와 예산이 남아 있으면 `stop_sufficient`를 그대로 수용하지 않는다. + +## 핵심 변경 + +- `R_TRAVERSE_MIN_TERMINAL_MATERIAL_READS = 1` 정책 추가. +- terminal material 판정 추가: + - 텍스트가 있는 summary node + - raw source node + - raw capsule node +- `RLoopVesselTraverseResultFrame`에 다음 절대 카운트 추가: + - `terminal_material_seen_count` + - `min_terminal_material_count` + - `early_stop_guard_trigger_count` +- 조기 종료 guard 추가: + - R3가 sufficient/stop을 내도 + - terminal material count가 최소치보다 낮고 + - child candidate가 있고 + - 예산이 남아 있으면 + - continuation을 `continue_deeper`로 기록한다. +- guard reason: + +```text +CODE_STATUS:r_loop_terminal_material_not_seen +``` + +## 메타정보 경계 + +code는 답변 의미의 충분성을 판단하지 않았다. + +code가 한 일은 다음 절대정보 확인뿐이다. + +- 현재까지 terminal material을 몇 개 봤는지 +- 현재 노드에 child candidate가 있는지 +- traversal budget이 남아 있는지 + +R2의 후보 선택과 R3의 충분성 판단은 그대로 LLM 책임으로 남겼다. + +## 검증 + +```powershell +python -m pytest tests/test_order_184_r_vessel_multi_step_traversal.py tests/test_order_185_r_terminal_material_guard.py -q +# 3 passed + +python -m compileall songryeon_core main.py +# passed + +python main.py fast-test --profile graph --skip-compileall +# FAST_TEST_OK +# 134 passed + +python main.py smoke-test +# SMOKE_TEST_OK + +git diff --check +# passed +``` + +## 수동 CLI 확인 + +Codex 작업 셸에서는 Neo4j 비밀번호 환경변수가 설정되어 있지 않아 다음 명령은 read packet 단계에서 `neo4j_config_missing`으로 닫혔다. + +```powershell +python main.py vessel-r-traverse "송련 Core의 그래프 기억 구조에서 source summary와 token layer summary가 어떻게 이어지는지 계층적으로 탐색해줘" --database neo4j --llm-mode fake --format text +``` + +다만 새 출력 필드(`terminal_material_seen_count`, `min_terminal_material_count`, `early_stop_guard_trigger_count`)는 renderer에 노출되는 것을 확인했다. + +## 남은 범위 + +- live Qwen `vessel-r-traverse` 재실행으로 실제 조기 종료 개선을 확인해야 한다. +- 아직 R route를 node_1 field loop에 연결하지 않았다. +- 아직 R 결과를 node_0 memory packet이나 node_3 final answer에 자동 공급하지 않았다. diff --git a/Administrative_Reform_1/05_Execution_Records/order_186_r_vessel_exact_child_record_expansion_2026_07_03_001.md b/Administrative_Reform_1/05_Execution_Records/order_186_r_vessel_exact_child_record_expansion_2026_07_03_001.md new file mode 100644 index 0000000..01a01b6 --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_186_r_vessel_exact_child_record_expansion_2026_07_03_001.md @@ -0,0 +1,74 @@ +# ORDER 186 Execution Record: R Vessel Exact Child Record Expansion + +## Date + +2026-07-03 + +## Scope + +Implemented a structural R Vessel read packet expansion so that exact child graph records referenced by visible parent candidates are included in the packet even when they appear outside the base entry row limit. + +## Changed Files + +- `songryeon_core/core/r_loop_vessel_read_packet.py` +- `songryeon_core/runtime/r_loop_vessel_read_packet.py` +- `songryeon_core/runtime/fast_test.py` +- `tests/test_order_186_r_vessel_exact_child_expansion.py` +- `Administrative_Reform_1/04_Orders/ORDER_186_R_VESSEL_EXACT_CHILD_RECORD_EXPANSION_V0.md` +- `Administrative_Reform_1/04_Orders/README.md` +- `Administrative_Reform_1/05_Execution_Records/README.md` + +## Implementation Notes + +- Added `R_LOOP_VESSEL_EXACT_CHILD_EXPANSION_V0`. +- `RLoopVesselReadPacketFrame` now records: + - `exact_child_expansion_policy_id` + - `base_entry_candidate_count` + - `exact_child_expanded_entry_count` + - `exact_child_expanded_node_ids` + - `exact_child_expansion_truncated` +- The packet builder keeps the base `limit`, then follows exact child IDs from visible entry candidates: + - payload `source_graph_node_ids` + - record `parent_graph_node_ids` +- Expansion is bounded by: + - max depth: 2 + - max additional records: the same numeric `limit` + +## Boundary + +This change does not rank, summarize, or semantically select graph nodes. Code only follows explicit graph IDs and records the result as absolute information. + +R2 still chooses from official refs, and R3 still evaluates sufficiency/depth. + +## Verification + +Passed: + +```text +python -m compileall songryeon_core main.py +python -m pytest -q tests/test_order_186_r_vessel_exact_child_expansion.py +python -m pytest -q tests/test_order_175_vessel_backed_r_read_packet.py tests/test_order_183_r_vessel_hierarchical_child_surface.py tests/test_order_184_r_vessel_multi_step_traversal.py tests/test_order_185_r_terminal_material_guard.py tests/test_order_186_r_vessel_exact_child_expansion.py +python main.py fast-test --profile graph --skip-compileall +python main.py fast-test --profile graph +python main.py smoke-test +python main.py vessel-r-read-packet --database neo4j --format text +python main.py vessel-r-traverse "송련 Core의 그래프 기억 구조에서 source summary와 token layer summary가 어떻게 이어지는지 계층적으로 탐색해줘" --database neo4j --llm-mode fake --format text +python main.py vessel-r-traverse "송련 Core의 그래프 기억 구조에서 source summary와 token layer summary가 어떻게 이어지는지 계층적으로 탐색해줘" --database neo4j --llm-mode qwen --timeout 180 --format text +``` + +Observed: + +```text +tests/test_order_186_r_vessel_exact_child_expansion.py: 2 passed +ORDER 175/183/184/185/186 bundle: 14 passed +fast-test graph: 136 passed +fast-test graph with compileall: 136 passed +smoke-test: SMOKE_TEST_OK +vessel-r-read-packet: entry_candidate_count=100, base_entry_candidate_count=50, exact_child_expanded_entry_count=50, exact_child_expansion_truncated=True +vessel-r-traverse fake: TimeAxis -> SourceIngestBundle -> SourceKindBundle -> RawSource, terminal_material_seen_count=1 +vessel-r-traverse qwen: TimeAxis -> SourceIngestBundle -> SourceKindBundle -> RawSource, terminal_material_seen_count=1 +``` + +## Remaining Risk + +This unblocks exact child visibility, but it does not yet make R traversal semantically sharp. If a bundle has many children, R2 still needs better layer-level and budget-level selection policies in later orders. diff --git a/Administrative_Reform_1/05_Execution_Records/order_187_r_vessel_summary_layer_before_raw_2026_07_03_001.md b/Administrative_Reform_1/05_Execution_Records/order_187_r_vessel_summary_layer_before_raw_2026_07_03_001.md new file mode 100644 index 0000000..dac28f8 --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_187_r_vessel_summary_layer_before_raw_2026_07_03_001.md @@ -0,0 +1,69 @@ +# ORDER 187 Execution Record: R Vessel Summary Layer Before Raw + +## Date + +2026-07-03 + +## Scope + +Implemented a structural R traversal policy so that `SourceKindBundle` exposes active summary-layer children before direct raw source children. + +## Changed Files + +- `songryeon_core/loops/r_loop_vessel_one_step.py` +- `songryeon_core/runtime/fast_test.py` +- `tests/test_order_187_r_vessel_summary_layer_before_raw.py` +- `Administrative_Reform_1/04_Orders/ORDER_187_R_VESSEL_SUMMARY_LAYER_BEFORE_RAW_V0.md` +- `Administrative_Reform_1/04_Orders/README.md` +- `Administrative_Reform_1/05_Execution_Records/README.md` + +## Implementation Notes + +- Added source-kind summary-layer child selection inside R Vessel hierarchy child generation. +- For a selected `SourceKindBundle`, code gathers raw child IDs from: + - selected record `source_graph_node_ids` + - entry records whose `parent_graph_node_ids` include the selected source-kind node +- Code then finds active summary candidate records that explicitly cover those raw IDs through: + - `source_data_ids` + - `source_graph_node_ids` + - `target_graph_node_id` +- Priority is structural: + - `token_budget_bundle_summary` + - then `source_leaf_summary` + - then fallback to raw source children +- `hierarchy_child_node_ids` now acts as the official code-built child list. When present, it prevents raw `source_graph_node_ids` from being appended back into the R3 child list. + +## Boundary + +No semantic relevance judgement was added. Code does not inspect summary text or choose the best summary by meaning. + +R2 still chooses from official refs. R3 still judges sufficiency. + +## Verification + +Passed: + +```text +python -m compileall songryeon_core main.py +python -m pytest -q tests/test_order_187_r_vessel_summary_layer_before_raw.py +python -m pytest -q tests/test_order_183_r_vessel_hierarchical_child_surface.py tests/test_order_184_r_vessel_multi_step_traversal.py tests/test_order_185_r_terminal_material_guard.py tests/test_order_186_r_vessel_exact_child_expansion.py +python main.py fast-test --profile graph +python main.py smoke-test +python main.py vessel-r-traverse "송련 Core의 그래프 기억 구조에서 source summary와 token layer summary가 어떻게 이어지는지 계층적으로 탐색해줘" --database neo4j --llm-mode fake --format text +python main.py vessel-r-traverse "송련 Core의 그래프 기억 구조에서 source summary와 token layer summary가 어떻게 이어지는지 계층적으로 탐색해줘" --database neo4j --llm-mode qwen --timeout 180 --format text +``` + +Observed: + +```text +ORDER 187 tests: 2 passed +ORDER 183/184/185/186 bundle: 8 passed +fast-test graph: 138 passed +smoke-test: SMOKE_TEST_OK +vessel-r-traverse fake: TimeAxis -> SourceIngestBundle -> SourceKindBundle -> token_budget_bundle_summary, terminal_material_seen_count=1, stop_sufficient +vessel-r-traverse qwen: TimeAxis -> SourceIngestBundle -> SourceKindBundle -> token_budget_bundle_summary, terminal_material_seen_count=1, stop_budget_exhausted because R3 wanted deeper after the token summary +``` + +## Remaining Risk + +This only changes the first layer exposed under `SourceKindBundle`. It does not yet build a full semantic-axis navigator, nor does it make R1 truly set traversal budgets. diff --git a/Administrative_Reform_1/05_Execution_Records/order_188_r_vessel_raw_original_read_cap_2026_07_03_001.md b/Administrative_Reform_1/05_Execution_Records/order_188_r_vessel_raw_original_read_cap_2026_07_03_001.md new file mode 100644 index 0000000..d25443d --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_188_r_vessel_raw_original_read_cap_2026_07_03_001.md @@ -0,0 +1,70 @@ +# ORDER 188 Execution Record: R Vessel Raw Original Read Cap + +## Date + +2026-07-03 + +## Scope + +Implemented a hard cap for raw original material inspection in R Vessel traversal. + +## Changed Files + +- `songryeon_core/loops/r_loop_vessel_one_step.py` +- `songryeon_core/runtime/r_loop_vessel_one_step.py` +- `songryeon_core/runtime/fast_test.py` +- `tests/test_order_188_r_vessel_raw_original_cap.py` +- `Administrative_Reform_1/04_Orders/ORDER_188_R_VESSEL_RAW_ORIGINAL_READ_CAP_V0.md` +- `Administrative_Reform_1/04_Orders/README.md` +- `Administrative_Reform_1/05_Execution_Records/README.md` + +## Implementation Notes + +- Added `R_TRAVERSE_MAX_RAW_ORIGINAL_MATERIAL_READS = 5`. +- Added result frame fields: + - `raw_original_material_seen_count` + - `max_raw_original_material_count` + - `raw_original_read_cap_reached` +- Raw original material is counted only for `RawSource` and `RawCapsule`. +- Summary nodes are not counted as raw original reads. +- If R3 wants to continue after 5 raw original reads, code returns: + +```text +continuation_status=stop_budget_exhausted +continuation_reason_code=CODE_STATUS:r_loop_raw_original_read_cap_reached +``` + +## Boundary + +This is an absolute budget guard. Code counts raw-original inspections but does not judge their semantic value. + +## Verification + +Passed: + +```text +python -m compileall songryeon_core main.py +python -m pytest -q tests/test_order_188_r_vessel_raw_original_cap.py +python -m pytest -q tests/test_order_184_r_vessel_multi_step_traversal.py tests/test_order_185_r_terminal_material_guard.py tests/test_order_186_r_vessel_exact_child_expansion.py tests/test_order_187_r_vessel_summary_layer_before_raw.py tests/test_order_188_r_vessel_raw_original_cap.py +python main.py fast-test --profile graph +python main.py smoke-test +git diff --check +``` + +Observed: + +```text +ORDER 188 tests: 2 passed +ORDER 184-188 focused tests: 9 passed +graph fast-test: FAST_TEST_OK, 140 passed +smoke-test: SMOKE_TEST_OK +diff check: clean +``` + +During verification, graph fast-test initially caught a misplaced `max_raw_original_material_reads` +check in the one-step R runner. The one-step runner does not own this cap, so the stray check was +removed and graph fast-test was rerun successfully. + +## Remaining Risk + +This cap is enforced inside current R Vessel traversal. If future R implementations add a separate raw-fetch tool outside this traversal path, the same cap must be mirrored there. diff --git a/Administrative_Reform_1/05_Execution_Records/order_189_r_traverse_live_audit_2026_07_03_001.md b/Administrative_Reform_1/05_Execution_Records/order_189_r_traverse_live_audit_2026_07_03_001.md new file mode 100644 index 0000000..9cedd9a --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_189_r_traverse_live_audit_2026_07_03_001.md @@ -0,0 +1,199 @@ +# ORDER 189 Execution Record: R Traverse Live Audit + +## Date + +2026-07-03 + +## Scope + +Created the ORDER 189 audit order and prepared live/manual R Vessel traversal checks. + +## Changed Files + +- `Administrative_Reform_1/04_Orders/ORDER_189_R_TRAVERSE_LIVE_AUDIT_V0.md` +- `Administrative_Reform_1/04_Orders/README.md` +- `Administrative_Reform_1/05_Execution_Records/README.md` +- `Administrative_Reform_1/05_Execution_Records/order_189_r_traverse_live_audit_2026_07_03_001.md` + +## Audit Intent + +This audit checks whether R Vessel traversal is ready for the next integration step. + +The audit focuses on: + +- CoreEgo/time-axis start +- hierarchical descent +- summary-before-raw behavior +- raw original cap visibility +- partial/failure stop honesty +- whether the next MVP should be integration, prompt repair, or traversal structure repair + +## Verification Log + +### Baseline + +```text +python -m compileall songryeon_core main.py +``` + +Result: + +```text +passed +``` + +### Vessel Readback + +```text +python main.py vessel-readback --database neo4j +``` + +Result: + +```text +status=VESSEL_READBACK_OK +readback_status=passed +vessel_record_count=4611 +vessel_relationship_count=1747 +core_ego_count=1 +time_axis_count=1 +time_bundle_count=2 +raw_capsule_count=2 +``` + +### Fake Traversal: Structure Walk + +```text +python main.py vessel-r-traverse "송련 Core의 그래프 기억 구조를 CoreEgo에서 시작해서 한 단계씩 내려가며 설명 가능한 만큼만 탐색해줘" --database neo4j --llm-mode fake --format text +``` + +Result: + +```text +status=R_LOOP_VESSEL_TRAVERSE_OK +step_count=4 +final_graph_node_id=graph:summary:token_budget_bundle:08a835f99e68e01e:night_token_budget_layer_active +final_sufficiency_status=sufficient +final_continuation_status=stop_sufficient +r_loop_task_status=sufficient +terminal_material_seen_count=1 +raw_original_material_seen_count=0 +max_raw_original_material_count=5 +raw_original_read_cap_reached=False +``` + +Observed path: + +```text +graph:axis:time +-> graph:source_ingest_time_bundle:night_changed_sources_2026_07_02T14_02_36_242082:source_manifest +-> graph:source_kind_bundle:night_changed_sources_2026_07_02T14_02_36_242082:source_manifest:internal_document +-> graph:summary:token_budget_bundle:08a835f99e68e01e:night_token_budget_layer_active +``` + +Interpretation: + +- CoreEgo/time-axis start path is working. +- Summary-before-raw is working. +- Fake adapter stops after one terminal summary. +- Raw originals were not opened. + +### Qwen Traversal: Summary Layer Link + +```text +python main.py vessel-r-traverse "송련 Core의 그래프 기억 구조에서 source summary와 token layer summary가 어떻게 이어지는지 계층적으로 탐색해줘" --database neo4j --llm-mode qwen --timeout 180 --format text +``` + +Result: + +```text +status=R_LOOP_VESSEL_TRAVERSE_OK +step_count=4 +final_graph_node_id=graph:summary:token_budget_bundle:08a835f99e68e01e:night_token_budget_layer_active +final_sufficiency_status=insufficient +final_continuation_status=stop_budget_exhausted +r_loop_task_status=partial +terminal_material_seen_count=1 +raw_original_material_seen_count=0 +max_raw_original_material_count=5 +raw_original_read_cap_reached=False +``` + +Observed: + +- Qwen descended through the same high-level hierarchy. +- Qwen wanted to continue beyond the token-budget summary. +- Traversal stopped because the current traversal budget ended, not because raw cap was reached. +- Raw originals were not opened. + +### Qwen Traversal: Code/Document Separation + +```text +python main.py vessel-r-traverse "송련 Core 그래프 기억에서 코드 파일과 내부 문서가 서로 어떻게 분리되어 저장됐는지 계층적으로 확인해줘" --database neo4j --llm-mode qwen --timeout 180 --format text +``` + +Result: + +```text +status=R_LOOP_VESSEL_TRAVERSE_OK +step_count=4 +final_graph_node_id=graph:summary:token_budget_bundle:a5f9e552fe7ff1cf:night_token_budget_layer_active_layer_03 +final_sufficiency_status=insufficient +final_continuation_status=stop_budget_exhausted +r_loop_task_status=partial +terminal_material_seen_count=1 +raw_original_material_seen_count=0 +max_raw_original_material_count=5 +raw_original_read_cap_reached=False +``` + +Observed: + +- Qwen again followed the hierarchy instead of seeing every leaf summary at once. +- Qwen selected `internal_document` source kind in this run. +- Qwen did not yet inspect both code and internal-document branches in one traversal. +- The bottleneck is traversal budget/deeper branch handling, not raw-original overread. + +## Audit Conclusion + +Current R traversal is healthy enough to keep developing, but not ready to be wired into normal user-facing answers yet. + +What works: + +- Neo4j Vessel readback passes. +- CoreEgo/time-axis entry path exists. +- R traversal starts high and descends. +- Summary-before-raw behavior is visible. +- Raw original read count stays at 0 in these tests. +- Raw original cap is visible in runtime output. + +Current bottleneck: + +- Qwen reaches a token-budget summary and often wants to continue deeper. +- The traversal currently stops partial when traversal budget ends. +- For questions that require comparing branches, R traversal needs a structured next step before node_3 integration. + +## Recommended Next MVP + +Do not connect R traversal to node_3 yet. + +Recommended next order: + +```text +ORDER_190_R_TRAVERSE_DEEPER_FROM_TOKEN_SUMMARY_OR_BRANCH_COMPARE_V0 +``` + +Goal: + +- When R3 says a token-budget summary is insufficient and wants `deeper`, expose the exact children under that summary or the appropriate lower summary/leaf layer. +- For branch comparison questions, preserve enough information for R2 to switch from one source-kind branch to another without pretending one branch answered the whole question. + +## Initial Expected Next MVP + +If live traversal passes at least fake traversal and one Qwen traversal, the next likely MVP is: + +```text +R traversal result -> node_3 brief material packet +``` + +If live traversal fails because R2/R3 cannot choose valid child nodes, the next MVP should stay inside R traversal and repair the candidate surface before any node_3 integration. diff --git a/Administrative_Reform_1/05_Execution_Records/order_190_r_vessel_token_summary_deeper_child_expansion_2026_07_03_001.md b/Administrative_Reform_1/05_Execution_Records/order_190_r_vessel_token_summary_deeper_child_expansion_2026_07_03_001.md new file mode 100644 index 0000000..9700faa --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_190_r_vessel_token_summary_deeper_child_expansion_2026_07_03_001.md @@ -0,0 +1,181 @@ +# ORDER 190 Execution Record: R Vessel Token Summary Deeper Child Expansion + +## Date + +2026-07-03 + +## Scope + +Implemented lower-summary expansion for token-budget summary traversal in the R Vessel loop. + +## Changed Files + +- `songryeon_core/core/r_loop_vessel_read_packet.py` +- `songryeon_core/loops/r_loop_vessel_one_step.py` +- `songryeon_core/runtime/r_loop_vessel_read_packet.py` +- `songryeon_core/runtime/fast_test.py` +- `tests/test_order_190_r_vessel_token_summary_deeper_child_expansion.py` +- `Administrative_Reform_1/04_Orders/ORDER_190_R_VESSEL_TOKEN_SUMMARY_DEEPER_CHILD_EXPANSION_V0.md` +- `Administrative_Reform_1/04_Orders/README.md` +- `Administrative_Reform_1/05_Execution_Records/README.md` + +## Implementation Notes + +- Added `R_LOOP_VESSEL_SUMMARY_CHILD_EXPANSION_V0`. +- R read packet now keeps: + - `base_summary_candidate_count` + - `summary_child_expanded_count` + - `summary_child_expanded_node_ids` + - `summary_child_expansion_truncated` +- Summary candidate balancing now prefers higher-level token summaries before lower source leaf summaries. +- Multi-step R traversal default budget changed: + +```text +R_TRAVERSE_MAX_TRAVERSAL_DEPTH: 4 -> 6 +R_TRAVERSE_MAX_NODE_READS: 4 -> 6 +``` + +- Token-budget summary child traversal now exposes `graph:summary:*` children first. +- Token-budget summary traversal no longer falls through to unreadable token bundle IDs when no summary child record is available. +- ORDER 188 raw original cap remains unchanged: + +```text +R_TRAVERSE_MAX_RAW_ORIGINAL_MATERIAL_READS = 5 +``` + +## Metadata Boundary + +This patch is code-generated absolute structure handling. + +Code does: + +- count and copy active summary records by source ID +- sort summary layers by explicit structural policy +- stop exposing unresolved bundle IDs as readable material + +Code does not: + +- decide semantic relevance of a summary child +- summarize content +- write graph nodes +- route user turns to R + +R2/R3 still perform LLM selection and sufficiency judgment. + +## Verification + +Passed: + +```text +python -m compileall songryeon_core main.py +python -m pytest -q tests/test_order_190_r_vessel_token_summary_deeper_child_expansion.py +python -m pytest -q tests/test_order_175_vessel_backed_r_read_packet.py tests/test_order_184_r_vessel_multi_step_traversal.py tests/test_order_185_r_terminal_material_guard.py tests/test_order_186_r_vessel_exact_child_expansion.py tests/test_order_187_r_vessel_summary_layer_before_raw.py tests/test_order_188_r_vessel_raw_original_cap.py +python main.py fast-test --profile graph +python main.py smoke-test +git diff --check +``` + +Observed: + +```text +ORDER 190 focused tests: 2 passed +ORDER 175/184-188 related tests: 15 passed +graph fast-test: FAST_TEST_OK, 142 passed +smoke-test: SMOKE_TEST_OK +diff check: clean +``` + +## Live Vessel Checks + +### Read Packet + +```text +python main.py vessel-r-read-packet --database neo4j --limit 50 --format text +``` + +Observed: + +```text +status=R_LOOP_VESSEL_READ_PACKET_OK +entry_candidate_count=100 +base_entry_candidate_count=50 +exact_child_expanded_entry_count=50 +exact_child_expansion_truncated=True +summary_candidate_count=100 +base_summary_candidate_count=50 +summary_child_expanded_count=50 +summary_child_expansion_truncated=True +total_summary_scanned_count=598 +summary_count_by_data_kind={'source_leaf_summary': 68, 'token_budget_bundle_summary': 32} +summary_count_by_depth={'1': 68, '2': 26, '3': 6} +``` + +Interpretation: + +- Summary-child expansion is active in the live Neo4j read packet. +- The packet now carries additional lower summary records needed for deeper traversal. + +### Qwen Normal Traversal + +```text +python main.py vessel-r-traverse "송련 Core의 그래프 기억 구조에서 source summary와 token layer summary가 어떻게 이어지는지 계층적으로 탐색해줘" --database neo4j --llm-mode qwen --timeout 180 --format text +``` + +Observed: + +```text +status=R_LOOP_VESSEL_TRAVERSE_OK +step_count=4 +final_graph_node_id=graph:summary:token_budget_bundle:08a835f99e68e01e:night_token_budget_layer_active +final_sufficiency_status=sufficient +final_continuation_status=stop_sufficient +r_loop_task_status=sufficient +raw_original_material_seen_count=0 +raw_original_read_cap_reached=False +``` + +Interpretation: + +- The previous ORDER 189 partial/budget-exhausted behavior did not repeat for this normal traversal. +- Qwen accepted the token summary as sufficient and stopped honestly. + +### Qwen Forced-Deeper Probe + +```text +python main.py vessel-r-traverse "송련 Core 그래프에서 token summary가 충분해 보여도 그 아래 lower summary child가 실제로 있는지 한 단계 더 내려가서 확인해줘" --database neo4j --llm-mode qwen --timeout 180 --format text +``` + +Observed: + +```text +status=R_LOOP_VESSEL_TRAVERSE_OK +step_count=2 +final_graph_node_id=graph:time_bundle:night_changed_sources_2026_07_02T14_02_36_242082:core +final_sufficiency_status=insufficient +final_continuation_status=stop_no_actionable_path +r_loop_task_status=partial +raw_original_material_seen_count=0 +raw_original_read_cap_reached=False +``` + +Interpretation: + +- Qwen chose the time-bundle branch instead of the source-ingest/source-kind branch. +- This is not a raw-original overread problem. +- This identifies the next bottleneck: branch choice stability when the user asks for a specific graph path. + +## Remaining Risk + +Branch comparison is still not implemented. If the user asks for code-vs-document comparison, R can descend one branch more safely, but it still does not yet guarantee multi-branch traversal before returning. + +## Recommended Next MVP + +```text +ORDER_191_R2_SOURCE_INGEST_BRANCH_SELECTION_STABILITY_V0 +``` + +Goal: + +- When the user asks about source summaries, token summaries, source kinds, or code/document separation, the R2 selection surface should keep source-ingest/source-kind branches distinguishable from old conversation time bundles. +- Do not use keyword heuristics. +- Prefer a structural surface policy: source-ingest/source-kind branch surfaces should be explicit and selectable as graph branches, while plain conversation `TimeBundle` should not accidentally intercept source-ingest questions. diff --git a/Administrative_Reform_1/05_Execution_Records/order_191_r2_source_ingest_branch_selection_stability_2026_07_03_001.md b/Administrative_Reform_1/05_Execution_Records/order_191_r2_source_ingest_branch_selection_stability_2026_07_03_001.md new file mode 100644 index 0000000..597dbc9 --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_191_r2_source_ingest_branch_selection_stability_2026_07_03_001.md @@ -0,0 +1,107 @@ +# ORDER 191 Execution Record: R2 Source Ingest Branch Selection Stability + +## Date + +2026-07-03 + +## Scope + +Implemented structural branch role labels for R2 Vessel candidate surfaces. + +## Changed Files + +- `songryeon_core/loops/r_loop_vessel_one_step.py` +- `songryeon_core/prompts/r2_vessel_node_selector_v0.md` +- `songryeon_core/runtime/fast_test.py` +- `tests/test_order_191_r2_branch_role_surface_stability.py` +- `Administrative_Reform_1/04_Orders/ORDER_191_R2_SOURCE_INGEST_BRANCH_SELECTION_STABILITY_V0.md` +- `Administrative_Reform_1/04_Orders/README.md` +- `Administrative_Reform_1/05_Execution_Records/README.md` +- `Administrative_Reform_1/05_Execution_Records/order_191_r2_source_ingest_branch_selection_stability_2026_07_03_001.md` + +## Implementation Notes + +Added structural `branch_role` labels: + +```text +graph_axis +source_material_ingest +source_material_leaf +summary_memory +conversation_time_memory +unknown_graph_branch +``` + +Branch roles are attached to: + +- candidate layer surface records +- R2 visible surface records +- R2 visible candidate records + +Structural surface order now keeps source-material ingest surfaces before conversation time memory surfaces. This helps R2 see the source/code/document path before plain time-bundle paths. + +## Metadata Boundary + +This patch is structural labeling, not semantic routing. + +Code does: + +- map explicit graph node kinds to structural branch roles +- expose those roles to R2 +- sort surfaces by branch role + +Code does not: + +- decide that a branch is relevant to the user question +- force source-ingest selection +- hide TimeBundle candidates +- add keyword heuristics + +R2 still chooses one official surface ref and one official node ref as an LLM judgment. + +## Verification + +Passed: + +```powershell +python -m compileall songryeon_core main.py +python -m pytest -q tests/test_order_191_r2_branch_role_surface_stability.py tests/test_order_183_r_vessel_hierarchical_child_surface.py tests/test_order_184_r_vessel_multi_step_traversal.py tests/test_order_187_r_vessel_summary_layer_before_raw.py tests/test_order_190_r_vessel_token_summary_deeper_child_expansion.py +python main.py fast-test --profile graph +python main.py smoke-test +git diff --check +``` + +Observed results: + +```text +compileall: passed +focused/related R tests: 11 passed +graph fast-test: FAST_TEST_OK, 144 passed +smoke-test: SMOKE_TEST_OK +git diff --check: passed +``` + +Manual fake traversal with Vessel env configured: + +```powershell +Set-ExecutionPolicy -Scope Process -ExecutionPolicy Bypass +. .\.env.vessel.local.ps1 +python main.py vessel-r-traverse "송련 Core의 그래프 기억 구조에서 source summary와 token layer summary가 어떻게 이어지는지 계층적으로 탐색해줘" --database neo4j --llm-mode fake --format text +``` + +Observed path: + +```text +step 1: graph:axis:time +step 2: graph:source_ingest_time_bundle:night_changed_sources_2026_07_02T14_02_36_242082:source_manifest +step 3: graph:source_kind_bundle:night_changed_sources_2026_07_02T14_02_36_242082:source_manifest:internal_document +step 4: graph:summary:token_budget_bundle:05689be7afe978fc:night_token_budget_layer_active_layer_03 +``` + +This confirms the deterministic traversal can prefer the source/material branch before plain conversation time memory for the tested source-summary/token-summary question. + +Live Qwen traversal was attempted with the same source-summary/token-summary style question, but the local model call did not complete inside the 180 second runtime timeout / 240 second shell window. This is recorded as a live-adapter availability/performance limit, not as a schema or validator failure. + +## Remaining Risk + +Branch comparison remains unimplemented. ORDER 191 improves first branch choice, but it does not yet make R traverse multiple branches before returning. diff --git a/Administrative_Reform_1/05_Execution_Records/order_192_r1_user_question_anchor_copy_2026_07_03_001.md b/Administrative_Reform_1/05_Execution_Records/order_192_r1_user_question_anchor_copy_2026_07_03_001.md new file mode 100644 index 0000000..d12d2d5 --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_192_r1_user_question_anchor_copy_2026_07_03_001.md @@ -0,0 +1,151 @@ +# ORDER 192 Execution Record: R1 User Question Anchor Copy + +## Date + +2026-07-03 + +## Scope + +Implemented explicit code-supplied user question anchor copying for R1 Vessel traversal. + +## Trigger + +Before this patch, a Korean live R traversal failed at R1: + +```text +status: R_LOOP_VESSEL_TRAVERSE_NOT_PASSED +failure_stage: R1 +failure_type: schema_failed +failure_reason: R1 graph_search_goal must preserve a user question anchor +``` + +A manual pre-patch test with an explicit `anchor_source_token_path` string succeeded, proving that R1/R2/R3 traversal could work when the anchor guard was satisfied. The fragile part was the text-token anchor check, not the graph traversal path itself. + +## Changed Files + +- `songryeon_core/core/schema_parts/r_loop.py` +- `songryeon_core/loops/r_loop_vessel_one_step.py` +- `songryeon_core/prompts/r1_vessel_goal_setter_v0.md` +- `songryeon_core/runtime/fast_test.py` +- `tests/test_order_176_vessel_r_one_step_traversal.py` +- `tests/test_order_177_r1_candidate_text_blindness.py` +- `tests/test_order_178_r_vessel_candidate_layer_surface.py` +- `tests/test_order_179_r2_vessel_selection_id_disambiguation.py` +- `tests/test_order_181_r2_official_selection_ref_map.py` +- `tests/test_order_182_r_core_ego_start_surface.py` +- `tests/test_order_183_r_vessel_hierarchical_child_surface.py` +- `tests/test_order_185_r_terminal_material_guard.py` +- `tests/test_order_190_r_vessel_token_summary_deeper_child_expansion.py` +- `tests/test_order_191_r2_branch_role_surface_stability.py` +- `tests/test_order_192_r1_user_question_anchor_copy.py` +- `Administrative_Reform_1/04_Orders/ORDER_192_R1_USER_QUESTION_ANCHOR_COPY_V0.md` +- `Administrative_Reform_1/04_Orders/README.md` +- `Administrative_Reform_1/05_Execution_Records/README.md` +- `Administrative_Reform_1/05_Execution_Records/order_192_r1_user_question_anchor_copy_2026_07_03_001.md` + +## Implementation Notes + +R1 input payload now includes: + +```json +{ + "user_question_anchor": { + "anchor_id": "r1_user_question_anchor:", + "source_field": "user_question", + "copy_required": true + } +} +``` + +R1 output now includes: + +```json +{ + "user_question_anchor_id": "r1_user_question_anchor:" +} +``` + +`R1GraphGoalFrame` preserves `user_question_anchor_id`. + +R1 validation now checks exact copied anchor identity: + +```text +payload.user_question_anchor_id == expected user question anchor +``` + +The previous live R1 text-token guard `_shares_user_question_anchor(...)` is no longer used for R1 payload validation. + +## Metadata Boundary + +Code does: + +- generate a deterministic anchor ID from the user question +- give that anchor ID to R1 +- require R1 to copy the anchor ID exactly +- reject missing or invented anchor IDs + +Code does not: + +- judge whether R1's graph goal is semantically correct +- parse Korean meaning with keyword heuristics +- rewrite R1's graph goal +- create a code fallback goal pretending to be R1's judgment + +The copied anchor is absolute linkage. R1's graph goal remains LLM-generated mixed/semantic material. + +## Verification + +Passed: + +```powershell +python -m compileall songryeon_core main.py +python -m pytest -q tests/test_order_192_r1_user_question_anchor_copy.py tests/test_order_177_r1_candidate_text_blindness.py tests/test_order_191_r2_branch_role_surface_stability.py +python -m pytest -q tests/test_order_176_vessel_r_one_step_traversal.py tests/test_order_177_r1_candidate_text_blindness.py tests/test_order_178_r_vessel_candidate_layer_surface.py tests/test_order_179_r2_vessel_selection_id_disambiguation.py tests/test_order_181_r2_official_selection_ref_map.py tests/test_order_182_r_core_ego_start_surface.py tests/test_order_183_r_vessel_hierarchical_child_surface.py tests/test_order_184_r_vessel_multi_step_traversal.py tests/test_order_185_r_terminal_material_guard.py tests/test_order_187_r_vessel_summary_layer_before_raw.py tests/test_order_188_r_vessel_raw_original_cap.py tests/test_order_190_r_vessel_token_summary_deeper_child_expansion.py tests/test_order_191_r2_branch_role_surface_stability.py tests/test_order_192_r1_user_question_anchor_copy.py +python main.py fast-test --profile graph +python -m pytest +python main.py smoke-test +git diff --check +``` + +Observed results: + +```text +compileall: passed +focused ORDER 192 / related tests: 9 passed +R traversal related tests: 37 passed +graph fast-test: FAST_TEST_OK, 148 passed +full pytest: 279 passed +smoke-test: SMOKE_TEST_OK +git diff --check: passed +``` + +Live Qwen Vessel traversal after implementation: + +```powershell +Set-ExecutionPolicy -Scope Process -ExecutionPolicy Bypass +. .\.env.vessel.local.ps1 +python main.py vessel-r-traverse "송련 Core의 그래프 기억 구조에서 소스 요약과 토큰 묶음 요약이 어떻게 이어지는지 계층적으로 탐색해줘. 과거 대화 기억 가지가 아니라 코드/문서 소스 가지를 우선 보고, 시간축에서 시작해서 어떤 묶음을 거쳐 내려가는지 말해줘." --database neo4j --llm-mode qwen --timeout 180 --format text +``` + +Observed: + +```text +status: R_LOOP_VESSEL_TRAVERSE_OK +traverse_status: completed +step_count: 4 +final_sufficiency_status: sufficient +r_loop_task_status: sufficient + +step 1: graph:axis:time +step 2: graph:source_ingest_time_bundle:night_changed_sources_2026_07_02T14_02_36_242082:source_manifest +step 3: graph:source_kind_bundle:night_changed_sources_2026_07_02T14_02_36_242082:source_manifest:internal_document +step 4: graph:summary:token_budget_bundle:08a835f99e68e01e:night_token_budget_layer_active +``` + +This confirms that the same Korean live test no longer fails at R1 due to the anchor guard. + +## Remaining Risk + +R1 anchor copy only proves source linkage. It does not prove that R1's natural-language goal is semantically ideal. + +R traversal can now reach token-budget summary material more reliably, but the next quality bottleneck is still how R3 decides whether a summary is sufficient or whether it should descend further. diff --git a/Administrative_Reform_1/05_Execution_Records/order_documentation_rule_2026_06_27_001.md b/Administrative_Reform_1/05_Execution_Records/order_documentation_rule_2026_06_27_001.md new file mode 100644 index 0000000..461692f --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/order_documentation_rule_2026_06_27_001.md @@ -0,0 +1,27 @@ +# Execution Record: Order Documentation Rule 2026-06-27 001 + +## 목적 + +사용자가 "다음부터 모든 발주서는 전부 문서화를 해서 송련이나 외부 에이전트가 추적 가능하게 할 수 있도록 해"라고 지시했다. + +이에 따라 발주서가 대화창에만 남지 않도록 유지 규칙을 갱신했다. + +## 변경 + +- `AGENTS.md`의 안전한 작업 규칙에 발주서 문서화 의무를 추가했다. +- `Administrative_Reform_1/01_Maintenance_System/DEVELOPMENT_CYCLE_POLICY_v0.md`의 목표 지정 단계에 발주서 문서화 절차를 추가했다. + +## 새 규칙 요약 + +새 발주서를 제안하거나 확정하면 다음을 수행한다. + +```text +1. Administrative_Reform_1/04_Orders/ 에 ORDER 문서로 저장한다. +2. 04_Orders/README.md의 범위, 요약, 링크를 갱신한다. +3. 배경이나 live 테스트 관찰이 중요하면 05_Execution_Records/ 에 실행 기록도 남긴다. +``` + +## 비고 + +이번 작업은 운영 규칙 문서화만 수행했다. +코드 변경, schema 변경, runtime 변경, 테스트 실행은 하지 않았다. diff --git a/Administrative_Reform_1/05_Execution_Records/qwen_vs_songryeon_runtime_value_experiment_2026_06_29_001.md b/Administrative_Reform_1/05_Execution_Records/qwen_vs_songryeon_runtime_value_experiment_2026_06_29_001.md new file mode 100644 index 0000000..cc47b60 --- /dev/null +++ b/Administrative_Reform_1/05_Execution_Records/qwen_vs_songryeon_runtime_value_experiment_2026_06_29_001.md @@ -0,0 +1,273 @@ +# qwen_vs_songryeon_runtime_value_experiment_2026_06_29_001 + +작성일: 2026-06-29 + +## 1. 실험 목적 + +Qwen 14B 단독 응답과 SongRyeon Core 런타임 구조를 거친 응답을 비교했다. + +핵심 질문: + +```text +이 답변이 qwen3:14b 자체 성능으로 나온 것인가, +아니면 SongRyeon Core 구조를 잘 만들어서 나온 것인가? +``` + +이번 실험은 모델 우열 벤치마크가 아니다. +같은 Qwen 14B를 쓰더라도 문서 제공 방식, trace/source/count/gate 구조가 답변의 정직성, 추적성, 한계 표시를 얼마나 바꾸는지 확인하기 위한 비교 실험이다. + +철학 문서: + +```text +Administrative_Reform_1/00_Philosophy/Qwen_Model_Vs_SongRyeon_Runtime_Experiment_Philosophy_2026_06_29.md +``` + +## 2. 고정 질문 + +```text +SongRyeon Core를 숙련 개발자의 실무 보조 도구로 쓴다고 가정하고, +현재 구조에서 실제 개발 업무에 도움이 될 만한 기능과 아직 위험한 기능을 내부 문서 기준으로 나눠줘. + +코드 변경 제안, 테스트 전략, 회귀 위험 탐지, 설계 문서 추적, 실행 기록 관리 관점에서 평가해줘. + +확인한 근거와 한계를 분리해서 말해줘. +``` + +## 3. 실행 환경 + +- 모델: `qwen3:14b` +- 실행 방식 A/B/B-control: Ollama HTTP API `/api/generate` +- 실행 방식 C: `python main.py qwen-turn` +- 온도: A/B/B-control `temperature=0` +- 주의: PowerShell에서 Ollama API에 한글 prompt를 보낼 때 UTF-8 byte body를 사용했다. 최초 문자열 body 호출은 한글이 깨진 듯한 응답을 만들어 실험 결과에서 제외했다. + +## 4. A군: Qwen 14B 단독 + +### 실행 + +문서 원문 없이 고정 질문만 Qwen 14B에 전달했다. + +### 관찰 + +Qwen은 문장 구성은 잘했지만, SongRyeon Core 내부에 실제로 존재한다고 확인되지 않은 내부 보고서, 실무 사례, 수치를 만들어냈다. + +예시: + +```text +"코드 품질 개선을 위한 AI 기반 리팩토링 도구" (2023년 10월 내부 보고서) +GitHub Actions에서 자동 리뷰 시 30%의 중복 코드 감소 기록 +AI 기반 회귀 분석 도구의 정확도 85% 달성 +``` + +### 평가 + +- 근거 정직성: 낮음 +- 구조 추적성: 없음 +- 실무 유용성: 일반론 수준에서는 있음 +- 한계 표시: 약함 + +### 잠정 결론 + +Qwen 14B 단독은 그럴듯한 실무 평가 문서를 만들 수 있지만, 내부 문서 기준이라는 조건을 만족하지 못했다. +특히 "내부 문서 기준"이라는 문구를 받았음에도 실제 제공된 문서가 없으면 가짜 내부 근거를 만들 수 있다. + +## 5. B군: Qwen 14B + 수동 문서 묶음 + +### 제공 문서 + +다음 문서 원문을 수동으로 prompt에 붙였다. + +```text +AGENTS.md +README.md +Administrative_Reform_1/01_Maintenance_System/AGENT_WORKING_RULES_FROM_MAIN_PROJECT.md +Administrative_Reform_1/01_Maintenance_System/ADMIN_RULES_v0.md +Administrative_Reform_1/01_Maintenance_System/DEVELOPMENT_CYCLE_POLICY_v0.md +Administrative_Reform_1/04_Orders/ORDER_117_CI_AND_DEVELOPMENT_ROUTINE_LOCK_V0.md +Administrative_Reform_1/04_Orders/ORDER_125_L3_PER_DOCUMENT_SUMMARY_FRAME_DESIGN_V0.md +Administrative_Reform_1/04_Orders/ORDER_132_NODE2_ANSWER_BASIS_MATERIAL_DELIVERY_POLICY_V0.md +``` + +### 관찰 + +가짜 내부 보고서와 임의 수치가 사라졌다. +답변은 문서 기반 개발 루틴, compileall/pytest/smoke-test, 실행 기록, 발주서 문서화, L3 요약/재료 전달 정책을 근거로 평가했다. + +대표 요약: + +```text +SongRyeon Core는 명시적인 문서 기반 개발, 체계적인 테스트 계층, +실행 기록 관리 등 개발 업무에 도움이 되는 기능을 제공한다. +그러나 자동화된 코드 변경 제안, LLM 기반 테스트 자동화, +회귀 위험 탐지 도구, 설계 문서 추적 자동화, 실행 기록 분석 도구는 제공되지 않는다. +``` + +### 평가 + +- 근거 정직성: A보다 크게 개선 +- 구조 추적성: 문서명 수준에서는 있음 +- 실무 유용성: 높음 +- 한계 표시: 있음 + +### 잠정 결론 + +Qwen 14B는 충분한 원문 문서를 직접 받으면 꽤 안정적인 평가를 낸다. +다만 source ID, trace, read_doc count, partial/failure 상태, node_4 검사는 없으므로 런타임 추적성은 SongRyeon보다 약하다. + +## 6. B-control: Qwen 14B + C군이 실제 읽은 문서 4개 + +### 목적 + +B군과 C군의 문서 묶음이 달랐기 때문에, C군이 실제 read_doc한 문서 4개를 Qwen 14B에 직접 제공하는 보정 비교를 추가했다. + +### 제공 문서 + +```text +Administrative_Reform_1/05_Execution_Records/tool_result_distillation_2026_06_22_001.md +Administrative_Reform_1/00_Philosophy/Qwen_Model_Vs_SongRyeon_Runtime_Experiment_Philosophy_2026_06_29.md +Administrative_Reform_1/01_Maintenance_System/AGENT_WORKING_RULES_FROM_MAIN_PROJECT.md +Administrative_Reform_1/04_Orders/ORDER_061_DOCUMENT_MEMORY_INDEX_V2.md +``` + +### 관찰 + +B-control도 A보다 훨씬 안정적이었다. +문서 추적, tool result distillation, smoke-test, document memory index 같은 근거를 사용했다. + +다만 일부 표현은 여전히 조심해야 한다. +예를 들어 제공 문서에 있는 구현 설명을 바탕으로 실제 코드 파일 또는 현재 구현 상태까지 강하게 확장해서 말할 위험이 있다. + +### 평가 + +- 근거 정직성: A보다 높음 +- 구조 추적성: 문서명 수준에서는 있음 +- 실무 유용성: 중간 이상 +- 한계 표시: 있음 + +### 잠정 결론 + +같은 문서를 직접 주면 Qwen 14B도 상당히 좋은 답변을 낸다. +하지만 C군처럼 read_doc count, context pack, L3 partial, budget exhausted, node_4 pass 같은 런타임 상태를 스스로 생성하거나 검증하지는 않는다. + +## 7. C군: SongRyeon Core 전체 런타임 + +### 실행 명령 + +```powershell +python main.py qwen-turn "SongRyeon Core를 숙련 개발자의 실무 보조 도구로 쓴다고 가정하고, 현재 구조에서 실제 개발 업무에 도움이 될 만한 기능과 아직 위험한 기능을 내부 문서 기준으로 나눠줘. 코드 변경 제안, 테스트 전략, 회귀 위험 탐지, 설계 문서 추적, 실행 기록 관리 관점에서 평가해줘. 확인한 근거와 한계를 분리해서 말해줘." --timeout 180 --pretty --export "Administrative_Reform_1\05_Execution_Records\runtime_runs\qwen_vs_songryeon_runtime_value_experiment_c_2026_06_29.json" +``` + +### 저장된 런타임 기록 + +```text +Administrative_Reform_1/05_Execution_Records/runtime_runs/qwen_vs_songryeon_runtime_value_experiment_c_2026_06_29.json +``` + +### 주요 런타임 사실 + +```text +status=ok +trace/data=176 / 215 +actual_l_runs=1 +l_internal_revision=present +read_doc=4 +document_context_pack included=4 / excluded=39 +search_candidates_final=10 +search_candidates_accumulated=43 +L tool budget=tool_calls 13/18, query_attempts 8/8, read_doc 4/10 +L loop result=partial / budget_exhausted / semantic=partial +node_2 answer_basis_mode=mixed_or_uncertain +node_3 LLM raw text=0 +node_3 L3 summaries=2 +material_delivery_policy=l3_summary_replaces_raw_context_with_uncertainty +node_4 gatekeeper=pass +``` + +### C군 답변 특징 + +송련은 다음을 답변 맨 앞에 명시했다. + +```text +실제 read_doc 도구 원문 읽기: 4개 +node_3 공급 문서 context: 4개 +node_3 LLM 원문 text: 0개 +L3 문서별 요약 재료: 2개 +L 검색 목표 상태: partial / budget_exhausted / semantic=partial +답변 근거 자세: 혼합/불확실성 표시 +재료 전달 정책: L3 요약 대체/불확실성 표시 +``` + +최종 답변은 코드 변경 제안, 설계 문서 추적, 실행 기록 관리를 도움이 되는 기능으로 분류했고, 회귀 위험 탐지와 테스트 전략 부족을 위험 또는 한계로 분류했다. + +### 평가 + +- 근거 정직성: 높음 +- 구조 추적성: 매우 높음 +- 실무 유용성: 중간 이상 +- 한계 표시: 높음 +- 비용/속도: 가장 무거움 + +### 잠정 결론 + +SongRyeon Core는 Qwen 14B보다 "더 많이 아는 모델"이 아니다. +하지만 같은 Qwen 14B를 다음 구조 안에 넣어 답변을 더 정직하게 만든다. + +```text +검색/읽기 기록 +count 고정 +L3 partial/failure 상태 +node_2 answer_basis_mode +L3 요약 대체 정책 +node_4 gatekeeper +runtime trace/export +``` + +이번 C군의 가장 큰 장점은 대답이 완벽해서가 아니라, 대답이 불완전하다는 사실까지 함께 보고했다는 점이다. + +## 8. 비교 요약 + +| 항목 | A: Qwen 단독 | B: Qwen+수동 문서 | B-control: Qwen+C문서 | C: SongRyeon | +| --- | --- | --- | --- | --- | +| 근거 정직성 | 낮음 | 높음 | 중간~높음 | 높음 | +| 가짜 내부 근거 | 많음 | 거의 없음 | 적음 | node_4 검사를 거침 | +| 문서 수/count 고정 | 없음 | 없음 | 없음 | 있음 | +| 실패/partial 표시 | 없음 | prompt 의존 | prompt 의존 | 런타임에서 표시 | +| trace/source 추적 | 없음 | 문서명 수준 | 문서명 수준 | trace/data 수준 | +| 실무 판단 품질 | 일반론 | 좋음 | 중간 이상 | 한계 포함 실무 판단 | +| 속도 | 빠름 | 빠름 | 빠름 | 느림 | + +## 9. 중요한 한계 + +이번 실험에는 다음 한계가 있다. + +1. C군은 새로 작성한 철학 문서 `Qwen_Model_Vs_SongRyeon_Runtime_Experiment_Philosophy_2026_06_29.md`를 실제 read_doc했다. 따라서 C군 답변은 실험 설계 문서 자체의 영향을 받았다. +2. B군은 사람이 고른 문서 묶음이고, C군은 L loop가 고른 문서 묶음이다. 그래서 B와 C는 검색 능력만 분리해 비교한 완전 통제 실험은 아니다. +3. B-control은 C군이 읽은 문서를 직접 제공했지만, SongRyeon의 node_2/node_3/node_4 구조는 포함하지 않았다. +4. 단일 질문 1회 실험이므로 모델/런타임 우열을 일반화할 수 없다. + +## 10. 잠정 결론 + +아버지에게 설명할 수 있는 가장 정확한 결론은 다음이다. + +```text +Qwen 14B도 문서를 충분히 주면 꽤 좋은 답을 한다. +하지만 Qwen 14B 단독은 내부 근거를 지어낼 위험이 크다. +SongRyeon Core의 가치는 Qwen을 더 똑똑한 모델로 바꾸는 데 있지 않고, +Qwen의 판단을 trace, source, count, failure state, gate 구조 안에 넣어 +근거와 한계를 더 정직하게 드러내는 데 있다. +``` + +이번 실험 기준으로 SongRyeon Core의 실질적 장점은 다음이다. + +```text +빠른 답변 품질이 아니라, +답변이 어디서 왔고 어디까지 믿을 수 있는지 보여주는 런타임 구조. +``` + +## 11. 후속 후보 + +1. 같은 질문을 3회 반복해 검색 문서 선택 안정성을 본다. +2. B-control과 C군의 문서 묶음을 더 엄격히 동일하게 맞춘다. +3. C군에서 실험 철학 문서가 검색에 끼어드는 영향을 줄이기 위해, 평가 대상 문서 세트를 명시 지정하는 모드를 검토한다. +4. node_4가 "제공 문서에 없음"과 "프로젝트에 없음"을 구분하도록 과잉 부재 단정 guard를 추가할지 검토한다. +5. 숙련 개발자 실무 보조 평가 전용 테스트 문구 세트를 만든다. diff --git a/main.py b/main.py index 6d363bd..44de9bf 100644 --- a/main.py +++ b/main.py @@ -6,7 +6,35 @@ from pathlib import Path from songryeon_core.runtime.dry_run import run_dry_turn +from songryeon_core.runtime.fast_test import run_fast_tests +from songryeon_core.runtime.graph_vessel_first_write import run_local_vessel_first_write +from songryeon_core.runtime.graph_vessel_inspect import ( + render_vessel_inspect_text, + run_local_vessel_inspect, +) +from songryeon_core.runtime.graph_vessel_readback import run_local_vessel_readback from songryeon_core.runtime.l_loop_smoke import run_qwen_l_loop_smoke +from songryeon_core.runtime.night_changed_source_summary import ( + DEFAULT_NIGHT_CHANGED_SOURCE_STORE_DIR, + run_night_changed_source_summary, +) +from songryeon_core.runtime.night_token_budget_layer_summary import ( + DEFAULT_NIGHT_TOKEN_BUDGET_MAX_LAYER_DEPTH, + DEFAULT_NIGHT_TOKEN_BUDGET_MAX_STEPS, + DEFAULT_NIGHT_TOKEN_BUDGET_LAYER_MAX_BUNDLE_CHARS, + DEFAULT_NIGHT_TOKEN_BUDGET_TARGET_CONTEXT_CHARS, + run_night_token_budget_layer_summary, +) +from songryeon_core.runtime.r_loop_vessel_read_packet import ( + render_r_loop_vessel_read_packet_text, + run_local_r_loop_vessel_read_packet, +) +from songryeon_core.runtime.r_loop_vessel_one_step import ( + render_r_loop_vessel_one_step_text, + render_r_loop_vessel_traverse_text, + run_local_r_loop_vessel_one_step, + run_local_r_loop_vessel_traverse, +) from songryeon_core.runtime.replay import replay_run from songryeon_core.runtime.smoke_test import run_smoke_tests from songryeon_core.runtime.terminal_view import render_pretty_turn @@ -19,6 +47,7 @@ store_chat_turn_result, ) from songryeon_core.runtime.defaults import ( + DEFAULT_MAX_DOCUMENT_CONTEXT_CHARS, DEFAULT_MAX_INPUT_CHARS, DEFAULT_MAX_QUERY_ATTEMPTS, DEFAULT_MAX_READ_DOC_CALLS, @@ -91,6 +120,200 @@ def main() -> None: # smoke-test는 "지금 기준선이 깨졌는가?"를 빠르게 확인하는 자동 점검이다. subparsers.add_parser("smoke-test") + fast_test_parser = subparsers.add_parser("fast-test") + fast_test_parser.add_argument("--profile", choices=["core", "graph"], default="graph") + fast_test_parser.add_argument("--skip-compileall", action="store_true") + fast_test_parser.add_argument("--dry-run", action="store_true") + + vessel_first_write_parser = subparsers.add_parser("vessel-first-write") + vessel_first_write_parser.add_argument("--root", default=".") + vessel_first_write_parser.add_argument("--batch-id", default="manual_vessel_first_write") + vessel_first_write_parser.add_argument("--turn-id", default="turn_vessel_first_write_0001") + vessel_first_write_parser.add_argument("--uri", default=None) + vessel_first_write_parser.add_argument("--user", default=None) + vessel_first_write_parser.add_argument("--password", default=None) + vessel_first_write_parser.add_argument("--database", default=None) + vessel_first_write_parser.add_argument("--allow-no-auth", action="store_true") + vessel_first_write_parser.add_argument("--include-source-manifest", action="store_true") + vessel_first_write_parser.add_argument("--store-text-snapshots", action="store_true") + + vessel_readback_parser = subparsers.add_parser("vessel-readback") + vessel_readback_parser.add_argument("--batch-id", default="manual_vessel_readback") + vessel_readback_parser.add_argument("--turn-id", default="turn_vessel_readback_0001") + vessel_readback_parser.add_argument("--uri", default=None) + vessel_readback_parser.add_argument("--user", default=None) + vessel_readback_parser.add_argument("--password", default=None) + vessel_readback_parser.add_argument("--database", default=None) + vessel_readback_parser.add_argument("--allow-no-auth", action="store_true") + + vessel_inspect_parser = subparsers.add_parser("vessel-inspect") + vessel_inspect_parser.add_argument("--batch-id", default="manual_vessel_inspect") + vessel_inspect_parser.add_argument("--turn-id", default="turn_vessel_inspect_0001") + vessel_inspect_parser.add_argument("--uri", default=None) + vessel_inspect_parser.add_argument("--user", default=None) + vessel_inspect_parser.add_argument("--password", default=None) + vessel_inspect_parser.add_argument("--database", default=None) + vessel_inspect_parser.add_argument("--allow-no-auth", action="store_true") + vessel_inspect_parser.add_argument("--limit", type=int, default=50) + vessel_inspect_parser.add_argument("--format", choices=["json", "text"], default="json") + + r_loop_vessel_read_packet_parser = subparsers.add_parser("vessel-r-read-packet") + r_loop_vessel_read_packet_parser.add_argument( + "--batch-id", + default="manual_r_loop_vessel_read_packet", + ) + r_loop_vessel_read_packet_parser.add_argument( + "--turn-id", + default="turn_r_loop_vessel_read_packet_0001", + ) + r_loop_vessel_read_packet_parser.add_argument("--uri", default=None) + r_loop_vessel_read_packet_parser.add_argument("--user", default=None) + r_loop_vessel_read_packet_parser.add_argument("--password", default=None) + r_loop_vessel_read_packet_parser.add_argument("--database", default=None) + r_loop_vessel_read_packet_parser.add_argument("--allow-no-auth", action="store_true") + r_loop_vessel_read_packet_parser.add_argument("--limit", type=int, default=50) + r_loop_vessel_read_packet_parser.add_argument( + "--format", + choices=["json", "text"], + default="json", + ) + + r_loop_vessel_one_step_parser = subparsers.add_parser("vessel-r-one-step") + r_loop_vessel_one_step_parser.add_argument("user_question") + r_loop_vessel_one_step_parser.add_argument( + "--batch-id", + default="manual_r_loop_vessel_one_step", + ) + r_loop_vessel_one_step_parser.add_argument( + "--turn-id", + default="turn_r_loop_vessel_one_step_0001", + ) + r_loop_vessel_one_step_parser.add_argument("--uri", default=None) + r_loop_vessel_one_step_parser.add_argument("--user", default=None) + r_loop_vessel_one_step_parser.add_argument("--password", default=None) + r_loop_vessel_one_step_parser.add_argument("--database", default=None) + r_loop_vessel_one_step_parser.add_argument("--allow-no-auth", action="store_true") + r_loop_vessel_one_step_parser.add_argument("--limit", type=int, default=50) + r_loop_vessel_one_step_parser.add_argument( + "--llm-mode", + choices=["off", "fake", "qwen"], + default="fake", + ) + r_loop_vessel_one_step_parser.add_argument("--endpoint", default=None) + r_loop_vessel_one_step_parser.add_argument("--model-id", default=None) + r_loop_vessel_one_step_parser.add_argument("--timeout", type=int, default=None) + r_loop_vessel_one_step_parser.add_argument( + "--format", + choices=["json", "text"], + default="json", + ) + + r_loop_vessel_traverse_parser = subparsers.add_parser("vessel-r-traverse") + r_loop_vessel_traverse_parser.add_argument("user_question") + r_loop_vessel_traverse_parser.add_argument( + "--batch-id", + default="manual_r_loop_vessel_traverse", + ) + r_loop_vessel_traverse_parser.add_argument( + "--turn-id", + default="turn_r_loop_vessel_traverse_0001", + ) + r_loop_vessel_traverse_parser.add_argument("--uri", default=None) + r_loop_vessel_traverse_parser.add_argument("--user", default=None) + r_loop_vessel_traverse_parser.add_argument("--password", default=None) + r_loop_vessel_traverse_parser.add_argument("--database", default=None) + r_loop_vessel_traverse_parser.add_argument("--allow-no-auth", action="store_true") + r_loop_vessel_traverse_parser.add_argument("--limit", type=int, default=50) + r_loop_vessel_traverse_parser.add_argument( + "--llm-mode", + choices=["off", "fake", "qwen"], + default="fake", + ) + r_loop_vessel_traverse_parser.add_argument("--endpoint", default=None) + r_loop_vessel_traverse_parser.add_argument("--model-id", default=None) + r_loop_vessel_traverse_parser.add_argument("--timeout", type=int, default=None) + r_loop_vessel_traverse_parser.add_argument( + "--format", + choices=["json", "text"], + default="json", + ) + + night_changed_sources_parser = subparsers.add_parser( + "night-summarize-changed-sources" + ) + night_changed_sources_parser.add_argument("--root", default=".") + night_changed_sources_parser.add_argument( + "--store-dir", + default=DEFAULT_NIGHT_CHANGED_SOURCE_STORE_DIR, + ) + night_changed_sources_parser.add_argument("--batch-id", default=None) + night_changed_sources_parser.add_argument("--turn-id", default=None) + night_changed_sources_parser.add_argument( + "--llm-mode", + choices=["off", "fake", "qwen"], + default="off", + ) + night_changed_sources_parser.add_argument("--endpoint", default=None) + night_changed_sources_parser.add_argument("--model-id", default=None) + night_changed_sources_parser.add_argument("--timeout", type=int, default=None) + night_changed_sources_parser.add_argument("--one-at-a-time", action="store_true") + night_changed_sources_parser.add_argument("--write-vessel", action="store_true") + night_changed_sources_parser.add_argument("--uri", default=None) + night_changed_sources_parser.add_argument("--user", default=None) + night_changed_sources_parser.add_argument("--password", default=None) + night_changed_sources_parser.add_argument("--database", default=None) + night_changed_sources_parser.add_argument("--allow-no-auth", action="store_true") + + night_token_layer_parser = subparsers.add_parser("night-summarize-token-layer") + night_token_layer_parser.add_argument( + "--store-dir", + default=DEFAULT_NIGHT_CHANGED_SOURCE_STORE_DIR, + ) + night_token_layer_parser.add_argument("--batch-id", default=None) + night_token_layer_parser.add_argument("--turn-id", default=None) + night_token_layer_parser.add_argument( + "--max-bundle-chars", + type=int, + default=DEFAULT_NIGHT_TOKEN_BUDGET_LAYER_MAX_BUNDLE_CHARS, + ) + night_token_layer_parser.add_argument( + "--llm-mode", + choices=["off", "fake", "qwen"], + default="off", + ) + night_token_layer_parser.add_argument("--endpoint", default=None) + night_token_layer_parser.add_argument("--model-id", default=None) + night_token_layer_parser.add_argument("--timeout", type=int, default=None) + night_token_layer_parser.add_argument("--until-context-budget", action="store_true") + night_token_layer_parser.add_argument( + "--target-context-chars", + type=int, + default=DEFAULT_NIGHT_TOKEN_BUDGET_TARGET_CONTEXT_CHARS, + ) + night_token_layer_parser.add_argument( + "--max-layer-depth", + type=int, + default=DEFAULT_NIGHT_TOKEN_BUDGET_MAX_LAYER_DEPTH, + ) + night_token_layer_parser.add_argument( + "--max-steps", + type=int, + default=DEFAULT_NIGHT_TOKEN_BUDGET_MAX_STEPS, + ) + night_token_layer_parser.add_argument("--max-runtime-minutes", type=float, default=None) + night_token_layer_parser.add_argument("--progress-jsonl", default=None) + night_token_layer_parser.add_argument( + "--vessel-write-mode", + choices=["none", "every-step", "at-end"], + default=None, + ) + night_token_layer_parser.add_argument("--no-stop-on-failure", action="store_true") + night_token_layer_parser.add_argument("--write-vessel", action="store_true") + night_token_layer_parser.add_argument("--uri", default=None) + night_token_layer_parser.add_argument("--user", default=None) + night_token_layer_parser.add_argument("--password", default=None) + night_token_layer_parser.add_argument("--database", default=None) + night_token_layer_parser.add_argument("--allow-no-auth", action="store_true") args = parser.parse_args() @@ -138,10 +361,12 @@ def main() -> None: max_query_candidates=args.max_query_candidates, max_read_doc_calls=args.max_read_doc_calls, max_input_chars=args.max_input_chars, + max_document_context_chars=args.max_document_context_chars, include_data_records=args.pretty, force_l_route=args.force_l, same_turn_l_reroute_enabled=args.same_turn_l_reroute, max_l_runs_per_turn=args.max_l_runs_per_turn, + enable_r_route_experimental=args.enable_r_route_experimental, live_trace=args.live_trace, ) if args.pretty: @@ -161,10 +386,12 @@ def main() -> None: max_query_candidates=args.max_query_candidates, max_read_doc_calls=args.max_read_doc_calls, max_input_chars=args.max_input_chars, + max_document_context_chars=args.max_document_context_chars, include_data_records=args.pretty, force_l_route=args.force_l, same_turn_l_reroute_enabled=args.same_turn_l_reroute, max_l_runs_per_turn=args.max_l_runs_per_turn, + enable_r_route_experimental=args.enable_r_route_experimental, live_trace=args.live_trace, ) if args.pretty: @@ -175,6 +402,179 @@ def main() -> None: _run_qwen_chat(args) elif args.command == "smoke-test": print(json.dumps(run_smoke_tests(), ensure_ascii=False, indent=2)) + elif args.command == "fast-test": + result = run_fast_tests( + profile=args.profile, + skip_compileall=args.skip_compileall, + dry_run=args.dry_run, + ) + print(json.dumps(result, ensure_ascii=False, indent=2)) + if result["status"] == "FAST_TEST_FAILED": + raise SystemExit(1) + elif args.command == "vessel-first-write": + result = run_local_vessel_first_write( + root_path=args.root, + batch_id=args.batch_id, + turn_id=args.turn_id, + uri=args.uri, + user=args.user, + password=args.password, + database=args.database, + allow_no_auth=args.allow_no_auth, + include_source_manifest=args.include_source_manifest, + store_text_snapshots=args.store_text_snapshots, + ) + print(json.dumps(result, ensure_ascii=False, indent=2)) + if result["write_status"] == "write_failed": + raise SystemExit(1) + elif args.command == "vessel-readback": + result = run_local_vessel_readback( + batch_id=args.batch_id, + turn_id=args.turn_id, + uri=args.uri, + user=args.user, + password=args.password, + database=args.database, + allow_no_auth=args.allow_no_auth, + ) + print(json.dumps(result, ensure_ascii=False, indent=2)) + if result["readback_status"] in {"read_failed", "failed"}: + raise SystemExit(1) + elif args.command == "vessel-inspect": + result = run_local_vessel_inspect( + batch_id=args.batch_id, + turn_id=args.turn_id, + uri=args.uri, + user=args.user, + password=args.password, + database=args.database, + allow_no_auth=args.allow_no_auth, + limit=args.limit, + ) + if args.format == "text": + print(render_vessel_inspect_text(result)) + else: + print(json.dumps(result, ensure_ascii=False, indent=2)) + if result["inspect_status"] == "read_failed": + raise SystemExit(1) + elif args.command == "vessel-r-read-packet": + result = run_local_r_loop_vessel_read_packet( + batch_id=args.batch_id, + turn_id=args.turn_id, + uri=args.uri, + user=args.user, + password=args.password, + database=args.database, + allow_no_auth=args.allow_no_auth, + limit=args.limit, + ) + if args.format == "text": + print(render_r_loop_vessel_read_packet_text(result)) + else: + print(json.dumps(result, ensure_ascii=False, indent=2)) + if result["read_status"] == "read_failed": + raise SystemExit(1) + elif args.command == "vessel-r-one-step": + result = run_local_r_loop_vessel_one_step( + user_question=args.user_question, + batch_id=args.batch_id, + turn_id=args.turn_id, + uri=args.uri, + user=args.user, + password=args.password, + database=args.database, + allow_no_auth=args.allow_no_auth, + limit=args.limit, + llm_mode=args.llm_mode, + endpoint=args.endpoint, + model_id=args.model_id, + timeout_seconds=args.timeout, + ) + if args.format == "text": + print(render_r_loop_vessel_one_step_text(result)) + else: + print(json.dumps(result, ensure_ascii=False, indent=2)) + if result["one_step_status"] == "failed": + raise SystemExit(1) + elif args.command == "vessel-r-traverse": + result = run_local_r_loop_vessel_traverse( + user_question=args.user_question, + batch_id=args.batch_id, + turn_id=args.turn_id, + uri=args.uri, + user=args.user, + password=args.password, + database=args.database, + allow_no_auth=args.allow_no_auth, + limit=args.limit, + llm_mode=args.llm_mode, + endpoint=args.endpoint, + model_id=args.model_id, + timeout_seconds=args.timeout, + ) + if args.format == "text": + print(render_r_loop_vessel_traverse_text(result)) + else: + print(json.dumps(result, ensure_ascii=False, indent=2)) + if result["traverse_status"] == "failed": + raise SystemExit(1) + elif args.command == "night-summarize-changed-sources": + result = run_night_changed_source_summary( + root_path=args.root, + store_dir=args.store_dir, + batch_id=args.batch_id, + turn_id=args.turn_id, + llm_mode=args.llm_mode, + endpoint=args.endpoint, + model_id=args.model_id, + timeout_seconds=args.timeout, + one_at_a_time=args.one_at_a_time, + write_vessel=args.write_vessel, + uri=args.uri, + user=args.user, + password=args.password, + database=args.database, + allow_no_auth=args.allow_no_auth, + ) + print(json.dumps(result, ensure_ascii=False, indent=2)) + if ( + result.get("write_result") is not None + and isinstance(result.get("write_result"), dict) + and result["write_result"].get("write_status") == "write_failed" + ): + raise SystemExit(1) + elif args.command == "night-summarize-token-layer": + result = run_night_token_budget_layer_summary( + store_dir=args.store_dir, + batch_id=args.batch_id, + turn_id=args.turn_id, + max_bundle_chars=args.max_bundle_chars, + llm_mode=args.llm_mode, + endpoint=args.endpoint, + model_id=args.model_id, + timeout_seconds=args.timeout, + until_context_budget=args.until_context_budget, + target_context_chars=args.target_context_chars, + max_layer_depth=args.max_layer_depth, + max_steps=args.max_steps, + max_runtime_minutes=args.max_runtime_minutes, + progress_jsonl=args.progress_jsonl, + vessel_write_mode=args.vessel_write_mode, + stop_on_failure=not args.no_stop_on_failure, + write_vessel=args.write_vessel, + uri=args.uri, + user=args.user, + password=args.password, + database=args.database, + allow_no_auth=args.allow_no_auth, + ) + print(json.dumps(result, ensure_ascii=False, indent=2)) + if ( + result.get("write_result") is not None + and isinstance(result.get("write_result"), dict) + and result["write_result"].get("write_status") == "write_failed" + ): + raise SystemExit(1) def _add_turn_runtime_args(parser: argparse.ArgumentParser, *, include_qwen_args: bool) -> None: @@ -189,9 +589,15 @@ def _add_turn_runtime_args(parser: argparse.ArgumentParser, *, include_qwen_args parser.add_argument("--max-query-candidates", type=int, default=None) parser.add_argument("--max-read-doc-calls", type=int, default=DEFAULT_MAX_READ_DOC_CALLS) parser.add_argument("--max-input-chars", type=int, default=DEFAULT_MAX_INPUT_CHARS) + parser.add_argument( + "--max-document-context-chars", + type=int, + default=DEFAULT_MAX_DOCUMENT_CONTEXT_CHARS, + ) parser.add_argument("--force-l", action="store_true") parser.add_argument("--same-turn-l-reroute", action="store_true") parser.add_argument("--max-l-runs-per-turn", type=int, default=1) + parser.add_argument("--enable-r-route-experimental", action="store_true") parser.add_argument("--live-trace", action="store_true") if include_qwen_args: parser.add_argument("--endpoint", default=None) @@ -241,10 +647,12 @@ def _run_qwen_chat(args: argparse.Namespace) -> None: max_query_candidates=args.max_query_candidates, max_read_doc_calls=args.max_read_doc_calls, max_input_chars=args.max_input_chars, + max_document_context_chars=args.max_document_context_chars, include_data_records=True, force_l_route=args.force_l, same_turn_l_reroute_enabled=args.same_turn_l_reroute, max_l_runs_per_turn=args.max_l_runs_per_turn, + enable_r_route_experimental=args.enable_r_route_experimental, recent_raw_conversation=session_memory.recent_raw_conversation, previous_turn_capsules=session_memory.previous_turn_capsules, live_trace=args.live_trace, @@ -291,6 +699,7 @@ def _summary(result: dict[str, object]) -> dict[str, object]: "l_loop_budget_plan_count": result.get("l_loop_budget_plan_count"), "search_top_k": result.get("search_top_k"), "max_query_attempts": result.get("max_query_attempts"), + "max_document_context_chars": result.get("max_document_context_chars"), "l_loop_final_decision": result.get("l_loop_final_decision"), "l_loop_final_continuation_status": result.get("l_loop_final_continuation_status"), "l_loop_continuation_count": result.get("l_loop_continuation_count"), @@ -309,6 +718,14 @@ def _summary(result: dict[str, object]) -> dict[str, object]: "node1_llm_routing_failed_count": result.get("node1_llm_routing_failed_count"), "node1_router_fallback_count": result.get("node1_router_fallback_count"), "node1_router_fallback_policy": result.get("node1_router_fallback_policy"), + "r_route_experimental_enabled": result.get("r_route_experimental_enabled"), + "r_route_experimental_status": result.get("r_route_experimental_status"), + "r_route_experimental_return_summary_id": result.get( + "r_route_experimental_return_summary_id" + ), + "r_route_experimental_close_route_id": result.get( + "r_route_experimental_close_route_id" + ), "recent_memory_relevance_selection_status": result.get( "recent_memory_relevance_selection_status" ), @@ -338,6 +755,22 @@ def _summary(result: dict[str, object]) -> dict[str, object]: "node4_unsupported_recent_memory_claim_count": result.get( "node4_unsupported_recent_memory_claim_count" ), + "node2_answer_basis_mode": result.get("node2_answer_basis_mode"), + "node2_answer_basis_generated_by": result.get("node2_answer_basis_generated_by"), + "node2_answer_basis_reason_codes": result.get("node2_answer_basis_reason_codes"), + "node2_answer_basis_semantic_judgement_status": result.get( + "node2_answer_basis_semantic_judgement_status" + ), + "node2_answer_basis_failure_type": result.get("node2_answer_basis_failure_type"), + "node2_answer_basis_llm_call_data_id": result.get( + "node2_answer_basis_llm_call_data_id" + ), + "node2_answer_basis_trace_event_id": result.get( + "node2_answer_basis_trace_event_id" + ), + "node2_answer_basis_payload_parse_status": result.get( + "node2_answer_basis_payload_parse_status" + ), "export_dir": result.get("export_dir"), } diff --git a/songryeon_core/core/graph_memory.py b/songryeon_core/core/graph_memory.py new file mode 100644 index 0000000..ad44c5a --- /dev/null +++ b/songryeon_core/core/graph_memory.py @@ -0,0 +1,578 @@ +from __future__ import annotations + +import json +from dataclasses import asdict, dataclass + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.schemas import ( + CoreEgoTimeAxisFrame, + GraphMemoryEdgeFrame, + GraphMemoryNodeFrame, + GraphMemorySnapshotFrame, + RLoopGraphGuidePacketFrame, + TurnStateCapsule, + validate_core_ego_time_axis_frame, + validate_graph_memory_edge_frame, + validate_graph_memory_node_frame, + validate_graph_memory_snapshot_frame, + validate_rloop_graph_guide_packet_frame, +) +from songryeon_core.core.trace_store import TraceStore + + +CORE_EGO_ROOT_NODE_ID = "graph:core_ego:root" +TIME_AXIS_NODE_ID = "graph:axis:time" +TIME_BUNDLE_CHAR_BUDGET_POLICY_ID = "TIME_BUNDLE_CHAR_BUDGET_LEAF_POLICY_V0" +DEFAULT_TIME_BUNDLE_CHAR_BUDGET = 12000 + + +@dataclass +class GraphMemoryBuildResult: + nodes: list[GraphMemoryNodeFrame] + edges: list[GraphMemoryEdgeFrame] + core_ego_time_axis: CoreEgoTimeAxisFrame + snapshot: GraphMemorySnapshotFrame + guide_packet: RLoopGraphGuidePacketFrame + + +@dataclass +class RecordedGraphMemoryResult: + build: GraphMemoryBuildResult + trace_event_id: str + created_data_ids: list[str] + existing_data_ids: list[str] + + +@dataclass +class _CapsuleBundle: + capsules: list[TurnStateCapsule] + source_char_count: int + char_budget_status: str + + +def raw_capsule_graph_node_id(turn_id: str) -> str: + return f"graph:raw_capsule:{turn_id}" + + +def time_bundle_graph_node_id(batch_id: str, *, index: int, total: int) -> str: + if total == 1: + return f"graph:time_bundle:{batch_id}" + return f"graph:time_bundle:{batch_id}:{index:03d}" + + +def graph_memory_snapshot_id(batch_id: str) -> str: + return f"graph:snapshot:{batch_id}" + + +def rloop_graph_guide_packet_id(snapshot_id: str) -> str: + return f"rloop:graph_guide:{snapshot_id}" + + +def build_graph_memory_snapshot_from_capsules( + *, + capsules: list[TurnStateCapsule], + batch_id: str, + source_data_ids: list[str] | None = None, + char_budget: int = DEFAULT_TIME_BUNDLE_CHAR_BUDGET, +) -> GraphMemoryBuildResult: + """Build a graph-memory snapshot from TurnStateCapsule coordinate fields.""" + + if not batch_id: + raise ValueError("batch_id must not be empty") + if char_budget <= 0: + raise ValueError("char_budget must be positive") + + unique_capsules = _dedupe_capsules_by_turn_id(capsules) + raw_nodes = [_build_raw_capsule_node(capsule) for capsule in unique_capsules] + for node in raw_nodes: + validate_graph_memory_node_frame(node) + + raw_nodes_by_turn_id = { + node.source_turn_id: node + for node in raw_nodes + if node.source_turn_id is not None + } + bundles = _partition_capsules_by_char_budget(unique_capsules, char_budget=char_budget) + total_bundles = len(bundles) + time_bundle_nodes: list[GraphMemoryNodeFrame] = [] + for index, bundle in enumerate(bundles, start=1): + bundle_raw_nodes = [ + raw_nodes_by_turn_id[capsule.turn_id] + for capsule in bundle.capsules + if capsule.turn_id in raw_nodes_by_turn_id + ] + node = GraphMemoryNodeFrame( + node_id=time_bundle_graph_node_id(batch_id, index=index, total=total_bundles), + node_kind="time_bundle", + data_kind="time_bundle", + summary_depth=0, + source_depth_min=0, + source_depth_max=0, + source_leaf_count=len(bundle_raw_nodes), + source_summary_count=0, + source_bundle_kind="time_bundle", + bundle_policy_id=TIME_BUNDLE_CHAR_BUDGET_POLICY_ID, + char_budget=char_budget, + source_char_count=bundle.source_char_count, + char_budget_status=bundle.char_budget_status, + source_graph_node_ids=[node.node_id for node in bundle_raw_nodes], + source_trace_ids=_unique_strings( + [ + trace_id + for raw_node in bundle_raw_nodes + for trace_id in raw_node.source_trace_ids + ] + ), + source_data_ids=[], + ) + validate_graph_memory_node_frame(node) + time_bundle_nodes.append(node) + + source_leaf_count = len(raw_nodes) + time_axis_node = GraphMemoryNodeFrame( + node_id=TIME_AXIS_NODE_ID, + node_kind="time_axis", + data_kind="time_axis", + summary_depth=0, + source_depth_min=0, + source_depth_max=0, + source_leaf_count=source_leaf_count, + source_summary_count=0, + source_bundle_kind="time_axis", + source_graph_node_ids=[node.node_id for node in time_bundle_nodes], + source_trace_ids=_unique_strings( + [ + trace_id + for node in time_bundle_nodes + for trace_id in node.source_trace_ids + ] + ), + ) + validate_graph_memory_node_frame(time_axis_node) + core_ego_node = GraphMemoryNodeFrame( + node_id=CORE_EGO_ROOT_NODE_ID, + node_kind="core_ego", + data_kind="core_ego_root", + summary_depth=0, + source_depth_min=0, + source_depth_max=0, + source_leaf_count=source_leaf_count, + source_summary_count=0, + source_bundle_kind="core_ego_time_axis_root", + source_graph_node_ids=[time_axis_node.node_id], + source_trace_ids=time_axis_node.source_trace_ids, + ) + validate_graph_memory_node_frame(core_ego_node) + + nodes = [core_ego_node, time_axis_node, *time_bundle_nodes, *raw_nodes] + edges = _build_core_ego_time_edges( + core_ego_node=core_ego_node, + time_axis_node=time_axis_node, + time_bundle_nodes=time_bundle_nodes, + ) + raw_nodes_by_id = {node.node_id: node for node in raw_nodes} + for bundle_node in time_bundle_nodes: + for raw_node_id in bundle_node.source_graph_node_ids: + raw_node = raw_nodes_by_id[raw_node_id] + edges.append( + _build_edge( + edge_kind="CONTAINS", + from_node_id=bundle_node.node_id, + to_node_id=raw_node.node_id, + source_trace_ids=raw_node.source_trace_ids, + ) + ) + for previous_raw_node, next_raw_node in zip(raw_nodes, raw_nodes[1:]): + edges.append( + _build_edge( + edge_kind="NEXT", + from_node_id=previous_raw_node.node_id, + to_node_id=next_raw_node.node_id, + source_trace_ids=_unique_strings( + [ + *previous_raw_node.source_trace_ids, + *next_raw_node.source_trace_ids, + ] + ), + ) + ) + for edge in edges: + validate_graph_memory_edge_frame(edge) + + graph_node_ids = [node.node_id for node in nodes] + graph_edge_ids = [edge.edge_id for edge in edges] + node_kind_counts = _count_by(nodes, "node_kind") + edge_kind_counts = _count_by(edges, "edge_kind") + data_kind_counts = _count_by(nodes, "data_kind") + source_trace_ids = _unique_strings( + [trace_id for node in nodes for trace_id in node.source_trace_ids] + ) + snapshot = GraphMemorySnapshotFrame( + snapshot_id=graph_memory_snapshot_id(batch_id), + batch_id=batch_id, + root_node_id=CORE_EGO_ROOT_NODE_ID, + time_axis_node_id=TIME_AXIS_NODE_ID, + graph_node_ids=graph_node_ids, + graph_edge_ids=graph_edge_ids, + node_kind_counts=node_kind_counts, + edge_kind_counts=edge_kind_counts, + data_kind_counts=data_kind_counts, + source_graph_node_ids=graph_node_ids, + source_trace_ids=source_trace_ids, + source_data_ids=_unique_strings([*graph_node_ids, *graph_edge_ids, *(source_data_ids or [])]), + ) + validate_graph_memory_snapshot_frame(snapshot) + + core_ego_time_axis = CoreEgoTimeAxisFrame( + frame_id=f"graph:core_ego_time_axis:{batch_id}", + batch_id=batch_id, + core_ego_node_id=CORE_EGO_ROOT_NODE_ID, + time_axis_node_id=TIME_AXIS_NODE_ID, + time_bundle_node_ids=[node.node_id for node in time_bundle_nodes], + raw_capsule_node_ids=[node.node_id for node in raw_nodes], + edge_ids=graph_edge_ids, + source_graph_node_ids=graph_node_ids, + source_trace_ids=source_trace_ids, + source_data_ids=[snapshot.snapshot_id], + ) + validate_core_ego_time_axis_frame(core_ego_time_axis) + + guide_packet = _build_rloop_guide_packet( + snapshot=snapshot, + nodes=nodes, + source_data_ids=[snapshot.snapshot_id, *graph_node_ids, *graph_edge_ids], + ) + validate_rloop_graph_guide_packet_frame(guide_packet) + + return GraphMemoryBuildResult( + nodes=nodes, + edges=edges, + core_ego_time_axis=core_ego_time_axis, + snapshot=snapshot, + guide_packet=guide_packet, + ) + + +def record_graph_memory_for_capsules( + *, + trace_store: TraceStore, + data_store: DataStore, + turn_id: str, + capsules: list[TurnStateCapsule], + batch_id: str, + source_data_ids: list[str] | None = None, + char_budget: int = DEFAULT_TIME_BUNDLE_CHAR_BUDGET, +) -> RecordedGraphMemoryResult: + """Record graph-memory node, edge, snapshot, and R loop guide frames in DataStore.""" + + build = build_graph_memory_snapshot_from_capsules( + capsules=capsules, + batch_id=batch_id, + source_data_ids=source_data_ids, + char_budget=char_budget, + ) + event = trace_store.create_event( + turn_id=turn_id, + actor="graph_memory_builder", + event_type="node_output", + input_ref=build.snapshot.source_trace_ids, + output_ref=[build.snapshot.snapshot_id, build.guide_packet.packet_id], + schema_status="passed", + ) + + created_data_ids: list[str] = [] + existing_data_ids: list[str] = [] + for node in build.nodes: + _record_payload_if_missing( + data_store=data_store, + data_id=node.node_id, + data_type=f"graph_memory:node:{node.node_kind}", + payload=asdict(node), + created_at=event.timestamp, + source_trace_id=event.event_id, + created_data_ids=created_data_ids, + existing_data_ids=existing_data_ids, + ) + for edge in build.edges: + _record_payload_if_missing( + data_store=data_store, + data_id=edge.edge_id, + data_type=f"graph_memory:edge:{edge.edge_kind}", + payload=asdict(edge), + created_at=event.timestamp, + source_trace_id=event.event_id, + created_data_ids=created_data_ids, + existing_data_ids=existing_data_ids, + ) + _record_payload_if_missing( + data_store=data_store, + data_id=build.core_ego_time_axis.frame_id, + data_type="graph_memory:core_ego_time_axis_frame", + payload=asdict(build.core_ego_time_axis), + created_at=event.timestamp, + source_trace_id=event.event_id, + created_data_ids=created_data_ids, + existing_data_ids=existing_data_ids, + ) + _record_payload_if_missing( + data_store=data_store, + data_id=build.snapshot.snapshot_id, + data_type="graph_memory:snapshot", + payload=asdict(build.snapshot), + created_at=event.timestamp, + source_trace_id=event.event_id, + created_data_ids=created_data_ids, + existing_data_ids=existing_data_ids, + ) + _record_payload_if_missing( + data_store=data_store, + data_id=build.guide_packet.packet_id, + data_type="graph_memory:rloop_guide_packet", + payload=asdict(build.guide_packet), + created_at=event.timestamp, + source_trace_id=event.event_id, + created_data_ids=created_data_ids, + existing_data_ids=existing_data_ids, + ) + + return RecordedGraphMemoryResult( + build=build, + trace_event_id=event.event_id, + created_data_ids=created_data_ids, + existing_data_ids=existing_data_ids, + ) + + +def _build_raw_capsule_node(capsule: TurnStateCapsule) -> GraphMemoryNodeFrame: + if not capsule.turn_id: + raise ValueError("TurnStateCapsule.turn_id must not be empty") + source_trace_ids = _unique_strings(capsule.trace_event_ids) + user_input_trace_id = ( + capsule.user_input_trace_id + if capsule.user_input_trace_id in source_trace_ids + else None + ) + final_response_trace_id = ( + capsule.final_response_trace_id + if capsule.final_response_trace_id in source_trace_ids + else None + ) + return GraphMemoryNodeFrame( + node_id=raw_capsule_graph_node_id(capsule.turn_id), + node_kind="raw_capsule", + data_kind="turn_state_capsule", + source_turn_id=capsule.turn_id, + trace_count=len(capsule.trace_event_ids), + movement_count=len(capsule.node_movements), + user_input_trace_id=user_input_trace_id, + final_response_trace_id=final_response_trace_id, + summary_depth=0, + source_depth_min=0, + source_depth_max=0, + source_leaf_count=1, + source_summary_count=0, + source_bundle_kind="raw_leaf", + source_char_count=_capsule_coordinate_char_count(capsule), + source_graph_node_ids=[], + source_trace_ids=source_trace_ids, + source_data_ids=[], + ) + + +def _partition_capsules_by_char_budget( + capsules: list[TurnStateCapsule], + *, + char_budget: int, +) -> list[_CapsuleBundle]: + if not capsules: + return [_CapsuleBundle(capsules=[], source_char_count=0, char_budget_status="within_budget")] + + bundles: list[_CapsuleBundle] = [] + current: list[TurnStateCapsule] = [] + current_chars = 0 + current_status = "within_budget" + for capsule in capsules: + capsule_chars = _capsule_coordinate_char_count(capsule) + if current and current_chars + capsule_chars > char_budget: + bundles.append( + _CapsuleBundle( + capsules=current, + source_char_count=current_chars, + char_budget_status=current_status, + ) + ) + current = [] + current_chars = 0 + current_status = "within_budget" + if not current and capsule_chars > char_budget: + bundles.append( + _CapsuleBundle( + capsules=[capsule], + source_char_count=capsule_chars, + char_budget_status="leaf_exceeds_budget_kept_whole", + ) + ) + continue + current.append(capsule) + current_chars += capsule_chars + if current: + bundles.append( + _CapsuleBundle( + capsules=current, + source_char_count=current_chars, + char_budget_status=current_status, + ) + ) + return bundles + + +def _build_core_ego_time_edges( + *, + core_ego_node: GraphMemoryNodeFrame, + time_axis_node: GraphMemoryNodeFrame, + time_bundle_nodes: list[GraphMemoryNodeFrame], +) -> list[GraphMemoryEdgeFrame]: + edges = [ + _build_edge( + edge_kind="CONTAINS", + from_node_id=core_ego_node.node_id, + to_node_id=time_axis_node.node_id, + source_trace_ids=time_axis_node.source_trace_ids, + ) + ] + for bundle_node in time_bundle_nodes: + edges.append( + _build_edge( + edge_kind="CHILD_OF_TIME_AXIS", + from_node_id=time_axis_node.node_id, + to_node_id=bundle_node.node_id, + source_trace_ids=bundle_node.source_trace_ids, + ) + ) + return edges + + +def _build_edge( + *, + edge_kind: str, + from_node_id: str, + to_node_id: str, + source_trace_ids: list[str], +) -> GraphMemoryEdgeFrame: + return GraphMemoryEdgeFrame( + edge_id=f"graph:edge:{edge_kind.lower()}:{from_node_id}:{to_node_id}", + edge_kind=edge_kind, + from_node_id=from_node_id, + to_node_id=to_node_id, + source_graph_node_ids=[from_node_id, to_node_id], + source_trace_ids=source_trace_ids, + source_data_ids=[from_node_id, to_node_id], + ) + + +def _build_rloop_guide_packet( + *, + snapshot: GraphMemorySnapshotFrame, + nodes: list[GraphMemoryNodeFrame], + source_data_ids: list[str], +) -> RLoopGraphGuidePacketFrame: + summary_depths = [node.summary_depth for node in nodes] + source_leaf_counts = [node.source_leaf_count for node in nodes] + risky_or_unreviewed_node_ids = [ + node.node_id + for node in nodes + if node.node_kind == "summary" or node.semantic_judgement_status != "not_run" + ] + return RLoopGraphGuidePacketFrame( + packet_id=rloop_graph_guide_packet_id(snapshot.snapshot_id), + graph_snapshot_id=snapshot.snapshot_id, + target_consumer="R_LOOP", + available_entry_nodes=[snapshot.time_axis_node_id], + node_kind_counts=dict(snapshot.node_kind_counts), + data_kind_counts=dict(snapshot.data_kind_counts), + summary_depth_range=_int_range(summary_depths), + source_leaf_count_range=_int_range(source_leaf_counts), + risky_or_unreviewed_node_ids=risky_or_unreviewed_node_ids, + recommended_traversal_hints=[], + recommended_traversal_hints_status="not_run", + source_graph_node_ids=list(snapshot.graph_node_ids), + source_trace_ids=list(snapshot.source_trace_ids), + source_data_ids=_unique_strings(source_data_ids), + generated_by="CODE:GRAPH_MEMORY_GUIDE_BUILDER", + info_class="absolute", + semantic_judgement_status="not_run", + ) + + +def _record_payload_if_missing( + *, + data_store: DataStore, + data_id: str, + data_type: str, + payload: dict[str, object], + created_at: str, + source_trace_id: str, + created_data_ids: list[str], + existing_data_ids: list[str], +) -> None: + existing = data_store.get_record(data_id) + if existing is not None: + if existing.data_type != data_type: + raise ValueError(f"graph memory data_id collision with different type: {data_id}") + if existing.payload != payload: + raise ValueError(f"graph memory data_id collision with different payload: {data_id}") + existing_data_ids.append(data_id) + return + data_store.create_record( + data_id=data_id, + data_type=data_type, + exists=True, + created_at=created_at, + source_trace_id=source_trace_id, + payload=payload, + ) + created_data_ids.append(data_id) + + +def _dedupe_capsules_by_turn_id(capsules: list[TurnStateCapsule]) -> list[TurnStateCapsule]: + seen: set[str] = set() + unique_capsules: list[TurnStateCapsule] = [] + for capsule in capsules: + if not capsule.turn_id: + raise ValueError("TurnStateCapsule.turn_id must not be empty") + if capsule.turn_id in seen: + continue + seen.add(capsule.turn_id) + unique_capsules.append(capsule) + return unique_capsules + + +def _capsule_coordinate_char_count(capsule: TurnStateCapsule) -> int: + return len(json.dumps(asdict(capsule), ensure_ascii=False, sort_keys=True)) + + +def _count_by(items: list[object], attr_name: str) -> dict[str, int]: + counts: dict[str, int] = {} + for item in items: + value = getattr(item, attr_name) + if not isinstance(value, str): + continue + counts[value] = counts.get(value, 0) + 1 + return counts + + +def _int_range(values: list[int]) -> list[int]: + if not values: + return [0, 0] + return [min(values), max(values)] + + +def _unique_strings(values: list[str | None]) -> list[str]: + seen: set[str] = set() + result: list[str] = [] + for value in values: + if not value or value in seen: + continue + seen.add(value) + result.append(value) + return result diff --git a/songryeon_core/core/graph_memory_export.py b/songryeon_core/core/graph_memory_export.py new file mode 100644 index 0000000..9bdb3d2 --- /dev/null +++ b/songryeon_core/core/graph_memory_export.py @@ -0,0 +1,294 @@ +from __future__ import annotations + +from dataclasses import asdict, dataclass +from datetime import datetime + +from songryeon_core.core.data_store import DataRecord, DataStore +from songryeon_core.core.graph_memory_integrity import audit_graph_memory_integrity +from songryeon_core.core.trace_store import TraceStore + + +GRAPH_MEMORY_EXPORT_PACKET_DATA_TYPE = "graph_memory:export_packet" +GRAPH_MEMORY_EXPORT_PACKET_GENERATOR = "CODE:GRAPH_MEMORY_EXPORT_PACKET_BUILDER" +GRAPH_MEMORY_EXPORT_PACKET_POLICY_ID = "GRAPH_MEMORY_EXPORT_PACKET_V0" +GRAPH_MEMORY_EXPORT_TARGET_ADAPTER = "songryeon-neo4j-vessel" + +GRAPH_MEMORY_NODE_DATA_TYPE_PREFIX = "graph_memory:node:" +GRAPH_MEMORY_EDGE_DATA_TYPE_PREFIX = "graph_memory:edge:" + +_GRAPH_MEMORY_DATA_TYPES = { + "graph_memory:core_ego_time_axis_frame", + "graph_memory:rloop_guide_packet", + "graph_memory:snapshot", +} +_GRAPH_SOURCE_DATA_TYPES = { + "graph_source:file_metadata", + "graph_source:file_text_snapshot", + "graph_source:source_kind_ingest_frame", + "graph_source:songryeon_core_source_manifest_frame", + "graph_source:source_version_lineage_frame", + "graph_source:source_observation_ledger_frame", + "graph_source:summary_invalidation_ledger_frame", +} + + +@dataclass(frozen=True) +class GraphMemoryExportPacket: + packet_id: str + batch_id: str + created_at: str + policy_id: str + target_adapter_name: str + external_write_status: str + graph_integrity_status: str + graph_integrity_summary: dict[str, object] + graph_node_data_ids: list[str] + graph_edge_data_ids: list[str] + graph_snapshot_data_ids: list[str] + core_ego_time_axis_frame_ids: list[str] + rloop_guide_packet_data_ids: list[str] + source_file_metadata_data_ids: list[str] + source_text_snapshot_data_ids: list[str] + source_ingest_frame_data_ids: list[str] + source_manifest_frame_data_ids: list[str] + source_version_lineage_frame_data_ids: list[str] + source_observation_ledger_frame_data_ids: list[str] + summary_invalidation_ledger_frame_data_ids: list[str] + included_data_ids: list[str] + source_trace_ids: list[str] + data_type_counts: dict[str, int] + generated_by: str = GRAPH_MEMORY_EXPORT_PACKET_GENERATOR + info_class: str = "absolute" + semantic_judgement_status: str = "not_run" + + +@dataclass(frozen=True) +class RecordedGraphMemoryExportPacket: + packet: GraphMemoryExportPacket + trace_event_id: str + created_data_ids: list[str] + existing_data_ids: list[str] + + +def graph_memory_export_packet_id(batch_id: str) -> str: + if not batch_id: + raise ValueError("batch_id must not be empty") + return f"graph_memory:export_packet:{batch_id}" + + +def build_graph_memory_export_packet( + *, + data_store: DataStore, + batch_id: str, + created_at: str | None = None, +) -> GraphMemoryExportPacket: + """Build a code-only packet that lists graph/source records ready for export.""" + + timestamp = created_at or _now_iso() + graph_node_data_ids: list[str] = [] + graph_edge_data_ids: list[str] = [] + graph_snapshot_data_ids: list[str] = [] + core_ego_time_axis_frame_ids: list[str] = [] + rloop_guide_packet_data_ids: list[str] = [] + source_file_metadata_data_ids: list[str] = [] + source_text_snapshot_data_ids: list[str] = [] + source_ingest_frame_data_ids: list[str] = [] + source_manifest_frame_data_ids: list[str] = [] + source_version_lineage_frame_data_ids: list[str] = [] + source_observation_ledger_frame_data_ids: list[str] = [] + summary_invalidation_ledger_frame_data_ids: list[str] = [] + included_records: list[DataRecord] = [] + + for record in data_store.list_records(): + data_type = record.data_type + if data_type.startswith(GRAPH_MEMORY_NODE_DATA_TYPE_PREFIX): + graph_node_data_ids.append(record.data_id) + included_records.append(record) + elif data_type.startswith(GRAPH_MEMORY_EDGE_DATA_TYPE_PREFIX): + graph_edge_data_ids.append(record.data_id) + included_records.append(record) + elif data_type == "graph_memory:snapshot": + graph_snapshot_data_ids.append(record.data_id) + included_records.append(record) + elif data_type == "graph_memory:core_ego_time_axis_frame": + core_ego_time_axis_frame_ids.append(record.data_id) + included_records.append(record) + elif data_type == "graph_memory:rloop_guide_packet": + rloop_guide_packet_data_ids.append(record.data_id) + included_records.append(record) + elif data_type == "graph_source:file_metadata": + source_file_metadata_data_ids.append(record.data_id) + included_records.append(record) + elif data_type == "graph_source:file_text_snapshot": + source_text_snapshot_data_ids.append(record.data_id) + included_records.append(record) + elif data_type == "graph_source:source_kind_ingest_frame": + source_ingest_frame_data_ids.append(record.data_id) + included_records.append(record) + elif data_type == "graph_source:songryeon_core_source_manifest_frame": + source_manifest_frame_data_ids.append(record.data_id) + included_records.append(record) + elif data_type == "graph_source:source_version_lineage_frame": + source_version_lineage_frame_data_ids.append(record.data_id) + included_records.append(record) + elif data_type == "graph_source:source_observation_ledger_frame": + source_observation_ledger_frame_data_ids.append(record.data_id) + included_records.append(record) + elif data_type == "graph_source:summary_invalidation_ledger_frame": + summary_invalidation_ledger_frame_data_ids.append(record.data_id) + included_records.append(record) + + integrity_report = audit_graph_memory_integrity(DataStore(included_records)) + packet = GraphMemoryExportPacket( + packet_id=graph_memory_export_packet_id(batch_id), + batch_id=batch_id, + created_at=timestamp, + policy_id=GRAPH_MEMORY_EXPORT_PACKET_POLICY_ID, + target_adapter_name=GRAPH_MEMORY_EXPORT_TARGET_ADAPTER, + external_write_status="not_run", + graph_integrity_status="passed" if integrity_report.passed else "failed", + graph_integrity_summary=integrity_report.to_summary(), + graph_node_data_ids=graph_node_data_ids, + graph_edge_data_ids=graph_edge_data_ids, + graph_snapshot_data_ids=graph_snapshot_data_ids, + core_ego_time_axis_frame_ids=core_ego_time_axis_frame_ids, + rloop_guide_packet_data_ids=rloop_guide_packet_data_ids, + source_file_metadata_data_ids=source_file_metadata_data_ids, + source_text_snapshot_data_ids=source_text_snapshot_data_ids, + source_ingest_frame_data_ids=source_ingest_frame_data_ids, + source_manifest_frame_data_ids=source_manifest_frame_data_ids, + source_version_lineage_frame_data_ids=source_version_lineage_frame_data_ids, + source_observation_ledger_frame_data_ids=source_observation_ledger_frame_data_ids, + summary_invalidation_ledger_frame_data_ids=summary_invalidation_ledger_frame_data_ids, + included_data_ids=_unique_strings([record.data_id for record in included_records]), + source_trace_ids=_unique_strings( + [record.source_trace_id for record in included_records] + ), + data_type_counts=_count_by_data_type(included_records), + ) + _validate_export_packet(packet) + return packet + + +def record_graph_memory_export_packet( + *, + trace_store: TraceStore, + data_store: DataStore, + turn_id: str, + batch_id: str, + created_at: str | None = None, +) -> RecordedGraphMemoryExportPacket: + """Record the export packet in DataStore without writing to any external DB.""" + + packet = build_graph_memory_export_packet( + data_store=data_store, + batch_id=batch_id, + created_at=created_at, + ) + event = trace_store.create_event( + turn_id=turn_id, + actor="graph_memory_export_packet_builder", + event_type="node_output", + timestamp=packet.created_at, + input_ref=packet.source_trace_ids, + output_ref=[packet.packet_id], + schema_status="passed", + ) + created_data_ids: list[str] = [] + existing_data_ids: list[str] = [] + _record_payload_if_missing( + data_store=data_store, + data_id=packet.packet_id, + data_type=GRAPH_MEMORY_EXPORT_PACKET_DATA_TYPE, + payload=asdict(packet), + created_at=event.timestamp, + source_trace_id=event.event_id, + created_data_ids=created_data_ids, + existing_data_ids=existing_data_ids, + ) + return RecordedGraphMemoryExportPacket( + packet=packet, + trace_event_id=event.event_id, + created_data_ids=created_data_ids, + existing_data_ids=existing_data_ids, + ) + + +def _validate_export_packet(packet: GraphMemoryExportPacket) -> None: + if packet.generated_by != GRAPH_MEMORY_EXPORT_PACKET_GENERATOR: + raise ValueError("GraphMemoryExportPacket.generated_by must be code builder") + if packet.info_class != "absolute": + raise ValueError("GraphMemoryExportPacket.info_class must be absolute") + if packet.semantic_judgement_status != "not_run": + raise ValueError("GraphMemoryExportPacket.semantic_judgement_status must be not_run") + if packet.external_write_status != "not_run": + raise ValueError("GraphMemoryExportPacket.external_write_status must be not_run") + if packet.graph_integrity_status not in {"passed", "failed"}: + raise ValueError("GraphMemoryExportPacket.graph_integrity_status is invalid") + if not packet.included_data_ids: + raise ValueError("GraphMemoryExportPacket.included_data_ids must not be empty") + + +def _record_payload_if_missing( + *, + data_store: DataStore, + data_id: str, + data_type: str, + payload: dict[str, object], + created_at: str, + source_trace_id: str, + created_data_ids: list[str], + existing_data_ids: list[str], +) -> None: + existing = data_store.get_record(data_id) + if existing is not None: + if existing.data_type != data_type: + raise ValueError(f"graph export data_id collision with different type: {data_id}") + if existing.payload != payload: + raise ValueError(f"graph export data_id collision with different payload: {data_id}") + existing_data_ids.append(data_id) + return + data_store.create_record( + data_id=data_id, + data_type=data_type, + exists=True, + created_at=created_at, + source_trace_id=source_trace_id, + payload=payload, + ) + created_data_ids.append(data_id) + + +def _count_by_data_type(records: list[DataRecord]) -> dict[str, int]: + counts: dict[str, int] = {} + for record in records: + counts[record.data_type] = counts.get(record.data_type, 0) + 1 + return counts + + +def _unique_strings(values: list[str | None]) -> list[str]: + seen: set[str] = set() + result: list[str] = [] + for value in values: + if not value or value in seen: + continue + seen.add(value) + result.append(value) + return result + + +def _now_iso() -> str: + return datetime.now().isoformat(timespec="seconds") + + +__all__ = [ + "GRAPH_MEMORY_EXPORT_PACKET_DATA_TYPE", + "GRAPH_MEMORY_EXPORT_PACKET_GENERATOR", + "GRAPH_MEMORY_EXPORT_PACKET_POLICY_ID", + "GRAPH_MEMORY_EXPORT_TARGET_ADAPTER", + "GraphMemoryExportPacket", + "RecordedGraphMemoryExportPacket", + "build_graph_memory_export_packet", + "graph_memory_export_packet_id", + "record_graph_memory_export_packet", +] diff --git a/songryeon_core/core/graph_memory_integrity.py b/songryeon_core/core/graph_memory_integrity.py new file mode 100644 index 0000000..006682f --- /dev/null +++ b/songryeon_core/core/graph_memory_integrity.py @@ -0,0 +1,113 @@ +from __future__ import annotations + +from dataclasses import asdict, dataclass + +from songryeon_core.core.data_store import DataStore + + +GRAPH_MEMORY_REFERENCE_PREFIXES = ("graph:", "rloop:") +GRAPH_MEMORY_EDGE_DATA_TYPE_PREFIX = "graph_memory:edge:" + + +@dataclass(frozen=True) +class GraphMemoryIntegrityIssue: + record_data_id: str + record_data_type: str + field_name: str + missing_ref: str + + +@dataclass(frozen=True) +class GraphMemoryIntegrityReport: + checked_record_count: int + checked_source_ref_count: int + checked_edge_count: int + missing_source_refs: list[GraphMemoryIntegrityIssue] + missing_edge_endpoints: list[GraphMemoryIntegrityIssue] + + @property + def passed(self) -> bool: + return not self.missing_source_refs and not self.missing_edge_endpoints + + def to_summary(self) -> dict[str, object]: + return { + "passed": self.passed, + "checked_record_count": self.checked_record_count, + "checked_source_ref_count": self.checked_source_ref_count, + "checked_edge_count": self.checked_edge_count, + "missing_source_ref_count": len(self.missing_source_refs), + "missing_edge_endpoint_count": len(self.missing_edge_endpoints), + "missing_source_refs": [asdict(issue) for issue in self.missing_source_refs], + "missing_edge_endpoints": [asdict(issue) for issue in self.missing_edge_endpoints], + } + + +def audit_graph_memory_integrity(data_store: DataStore) -> GraphMemoryIntegrityReport: + """Read-only graph/rloop reference audit for pre-export DataStore records.""" + + records = data_store.list_records() + record_ids = {record.data_id for record in records} + checked_record_count = 0 + checked_source_ref_count = 0 + checked_edge_count = 0 + missing_source_refs: list[GraphMemoryIntegrityIssue] = [] + missing_edge_endpoints: list[GraphMemoryIntegrityIssue] = [] + + for record in records: + payload = record.payload + if not isinstance(payload, dict): + continue + checked_record_count += 1 + + source_data_ids = payload.get("source_data_ids") + if isinstance(source_data_ids, list): + for index, ref in enumerate(source_data_ids): + if not _is_graph_memory_ref(ref): + continue + checked_source_ref_count += 1 + if ref not in record_ids: + missing_source_refs.append( + GraphMemoryIntegrityIssue( + record_data_id=record.data_id, + record_data_type=record.data_type, + field_name=f"source_data_ids[{index}]", + missing_ref=ref, + ) + ) + + if record.data_type.startswith(GRAPH_MEMORY_EDGE_DATA_TYPE_PREFIX): + checked_edge_count += 1 + for field_name in ("from_node_id", "to_node_id"): + ref = payload.get(field_name) + if not _is_graph_memory_ref(ref): + continue + if ref not in record_ids: + missing_edge_endpoints.append( + GraphMemoryIntegrityIssue( + record_data_id=record.data_id, + record_data_type=record.data_type, + field_name=field_name, + missing_ref=ref, + ) + ) + + return GraphMemoryIntegrityReport( + checked_record_count=checked_record_count, + checked_source_ref_count=checked_source_ref_count, + checked_edge_count=checked_edge_count, + missing_source_refs=missing_source_refs, + missing_edge_endpoints=missing_edge_endpoints, + ) + + +def _is_graph_memory_ref(value: object) -> bool: + return isinstance(value, str) and value.startswith(GRAPH_MEMORY_REFERENCE_PREFIXES) + + +__all__ = [ + "GRAPH_MEMORY_EDGE_DATA_TYPE_PREFIX", + "GRAPH_MEMORY_REFERENCE_PREFIXES", + "GraphMemoryIntegrityIssue", + "GraphMemoryIntegrityReport", + "audit_graph_memory_integrity", +] diff --git a/songryeon_core/core/graph_memory_store.py b/songryeon_core/core/graph_memory_store.py new file mode 100644 index 0000000..8d8b3fc --- /dev/null +++ b/songryeon_core/core/graph_memory_store.py @@ -0,0 +1,185 @@ +from __future__ import annotations + +from dataclasses import asdict +from typing import Protocol + +from songryeon_core.core.graph_memory import CORE_EGO_ROOT_NODE_ID, TIME_AXIS_NODE_ID +from songryeon_core.core.schemas import ( + GraphMemoryEdgeFrame, + GraphMemoryNodeFrame, + GraphMemorySnapshotFrame, + validate_graph_memory_edge_frame, + validate_graph_memory_node_frame, + validate_graph_memory_snapshot_frame, +) + + +SONGRYEON_VESSEL_SERVICE_NAME = "songryeon-neo4j-vessel" +SONGRYEON_VESSEL_DATABASE_NAME = "songryeon_vessel" +SONGRYEON_GRAPH_NAMESPACE = "songryeon_core_graph_v0" + + +class GraphMemoryStoreProtocol(Protocol): + """Storage boundary for graph memory frames. + + Implementations may use memory, JSONL, Neo4j, or another DB later, but they + must accept and return SongRyeon graph-memory frames without inventing + semantic fields. + """ + + def upsert_node(self, node: GraphMemoryNodeFrame) -> str: + ... + + def upsert_edge(self, edge: GraphMemoryEdgeFrame) -> str: + ... + + def get_node(self, node_id: str) -> GraphMemoryNodeFrame | None: + ... + + def get_edge(self, edge_id: str) -> GraphMemoryEdgeFrame | None: + ... + + def list_children(self, node_id: str, *, edge_kind: str | None = None) -> list[GraphMemoryNodeFrame]: + ... + + def list_core_ego_entries(self, *, axis: str = "time") -> list[GraphMemoryNodeFrame]: + ... + + def snapshot(self, *, snapshot_id: str, batch_id: str) -> GraphMemorySnapshotFrame: + ... + + +class InMemoryGraphMemoryStore: + """In-memory graph-memory adapter used to lock the external DB boundary.""" + + def __init__(self) -> None: + self._nodes: dict[str, GraphMemoryNodeFrame] = {} + self._edges: dict[str, GraphMemoryEdgeFrame] = {} + + def upsert_node(self, node: GraphMemoryNodeFrame) -> str: + validate_graph_memory_node_frame(node) + existing = self._nodes.get(node.node_id) + if existing is not None: + if asdict(existing) != asdict(node): + raise ValueError(f"graph memory node collision with different payload: {node.node_id}") + return node.node_id + self._nodes[node.node_id] = _copy_node(node) + return node.node_id + + def upsert_edge(self, edge: GraphMemoryEdgeFrame) -> str: + validate_graph_memory_edge_frame(edge) + if edge.from_node_id not in self._nodes: + raise ValueError(f"edge source node is missing: {edge.from_node_id}") + if edge.to_node_id not in self._nodes: + raise ValueError(f"edge target node is missing: {edge.to_node_id}") + existing = self._edges.get(edge.edge_id) + if existing is not None: + if asdict(existing) != asdict(edge): + raise ValueError(f"graph memory edge collision with different payload: {edge.edge_id}") + return edge.edge_id + self._edges[edge.edge_id] = _copy_edge(edge) + return edge.edge_id + + def get_node(self, node_id: str) -> GraphMemoryNodeFrame | None: + node = self._nodes.get(node_id) + return _copy_node(node) if node is not None else None + + def get_edge(self, edge_id: str) -> GraphMemoryEdgeFrame | None: + edge = self._edges.get(edge_id) + return _copy_edge(edge) if edge is not None else None + + def list_children( + self, + node_id: str, + *, + edge_kind: str | None = None, + ) -> list[GraphMemoryNodeFrame]: + children: list[GraphMemoryNodeFrame] = [] + for edge in self._edges.values(): + if edge.from_node_id != node_id: + continue + if edge_kind is not None and edge.edge_kind != edge_kind: + continue + child = self._nodes.get(edge.to_node_id) + if child is not None: + children.append(_copy_node(child)) + return children + + def list_core_ego_entries(self, *, axis: str = "time") -> list[GraphMemoryNodeFrame]: + if axis != "time": + return [] + entries: list[GraphMemoryNodeFrame] = [] + for child in self.list_children(CORE_EGO_ROOT_NODE_ID): + if child.node_id == TIME_AXIS_NODE_ID: + entries.append(child) + return entries + + def snapshot(self, *, snapshot_id: str, batch_id: str) -> GraphMemorySnapshotFrame: + if not snapshot_id: + raise ValueError("snapshot_id must not be empty") + if not batch_id: + raise ValueError("batch_id must not be empty") + graph_node_ids = list(self._nodes.keys()) + graph_edge_ids = list(self._edges.keys()) + source_trace_ids = _unique_strings( + [trace_id for node in self._nodes.values() for trace_id in node.source_trace_ids] + ) + snapshot = GraphMemorySnapshotFrame( + snapshot_id=snapshot_id, + batch_id=batch_id, + root_node_id=CORE_EGO_ROOT_NODE_ID, + time_axis_node_id=TIME_AXIS_NODE_ID, + graph_node_ids=graph_node_ids, + graph_edge_ids=graph_edge_ids, + node_kind_counts=_count_by(self._nodes.values(), "node_kind"), + edge_kind_counts=_count_by(self._edges.values(), "edge_kind"), + data_kind_counts=_count_by(self._nodes.values(), "data_kind"), + source_graph_node_ids=list(graph_node_ids), + source_trace_ids=source_trace_ids, + source_data_ids=_unique_strings([*graph_node_ids, *graph_edge_ids]), + ) + validate_graph_memory_snapshot_frame(snapshot) + return snapshot + + def node_count(self) -> int: + return len(self._nodes) + + def edge_count(self) -> int: + return len(self._edges) + + +def _copy_node(node: GraphMemoryNodeFrame) -> GraphMemoryNodeFrame: + return GraphMemoryNodeFrame(**asdict(node)) + + +def _copy_edge(edge: GraphMemoryEdgeFrame) -> GraphMemoryEdgeFrame: + return GraphMemoryEdgeFrame(**asdict(edge)) + + +def _count_by(items: object, attr_name: str) -> dict[str, int]: + counts: dict[str, int] = {} + for item in items: + value = getattr(item, attr_name) + if isinstance(value, str): + counts[value] = counts.get(value, 0) + 1 + return counts + + +def _unique_strings(values: list[str | None]) -> list[str]: + seen: set[str] = set() + result: list[str] = [] + for value in values: + if not value or value in seen: + continue + seen.add(value) + result.append(value) + return result + + +__all__ = [ + "GraphMemoryStoreProtocol", + "InMemoryGraphMemoryStore", + "SONGRYEON_GRAPH_NAMESPACE", + "SONGRYEON_VESSEL_DATABASE_NAME", + "SONGRYEON_VESSEL_SERVICE_NAME", +] diff --git a/songryeon_core/core/graph_source_ingest.py b/songryeon_core/core/graph_source_ingest.py new file mode 100644 index 0000000..8c3f8a0 --- /dev/null +++ b/songryeon_core/core/graph_source_ingest.py @@ -0,0 +1,945 @@ +from __future__ import annotations + +import hashlib +from dataclasses import asdict, dataclass +from datetime import datetime +from pathlib import Path + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.graph_memory import ( + CORE_EGO_ROOT_NODE_ID, + TIME_AXIS_NODE_ID, + rloop_graph_guide_packet_id, +) +from songryeon_core.core.schemas import ( + GRAPH_MEMORY_CODE_GENERATOR, + RLOOP_GUIDE_CODE_GENERATOR, + GraphMemoryEdgeFrame, + GraphMemoryNodeFrame, + GraphMemorySnapshotFrame, + RLoopGraphGuidePacketFrame, + validate_graph_memory_edge_frame, + validate_graph_memory_node_frame, + validate_graph_memory_snapshot_frame, + validate_rloop_graph_guide_packet_frame, +) +from songryeon_core.core.source_version_lineage import ( + record_source_version_lineage_and_summary_invalidation, +) +from songryeon_core.core.trace_store import TraceStore + + +GRAPH_SOURCE_KINDS = { + "internal_document", + "source_code_file", + "external_project_file", +} +GRAPH_SOURCE_KIND_INGEST_GENERATOR = "CODE:GRAPH_SOURCE_KIND_INGEST" +GRAPH_SOURCE_KIND_BUNDLE_POLICY_ID = "SOURCE_KIND_SEPARATED_INGEST_V0" +GRAPH_SOURCE_OBSERVATION_POLICY_ID = "SOURCE_OBSERVATION_SNAPSHOT_V0" +GRAPH_SOURCE_FILE_DATA_TYPE = "graph_source:file_metadata" +GRAPH_SOURCE_TEXT_SNAPSHOT_DATA_TYPE = "graph_source:file_text_snapshot" +GRAPH_SOURCE_INGEST_FRAME_DATA_TYPE = "graph_source:source_kind_ingest_frame" + + +@dataclass(frozen=True) +class GraphSourceKindIngestResult: + frame_id: str + trace_event_id: str + source_file_data_ids: list[str] + source_text_snapshot_data_ids: list[str] + source_ingest_time_bundle_node_id: str + raw_source_node_ids: list[str] + source_kind_bundle_node_ids: list[str] + graph_edge_ids: list[str] + graph_snapshot_id: str + rloop_graph_guide_packet_id: str + source_version_lineage_frame_ids: list[str] + source_observation_ledger_frame_id: str + source_observation_status_counts: dict[str, int] + summary_invalidation_ledger_frame_id: str + invalidated_summary_node_ids: list[str] + source_kind_counts: dict[str, int] + created_data_ids: list[str] + existing_data_ids: list[str] + + +@dataclass(frozen=True) +class _PreparedSourceFile: + source_kind: str + path: str + path_name: str + suffix: str + char_count: int + content_sha1: str + text: str + text_snapshot_data_id: str | None + observed_at: str + ingested_at: str + source_last_modified_at: str + exists_at_ingest: bool + source_file_data_id: str + raw_source_node_id: str + + +def record_graph_source_kind_ingest( + *, + trace_store: TraceStore, + data_store: DataStore, + turn_id: str, + batch_id: str, + source_paths_by_kind: dict[str, list[str | Path]], + input_ref: list[str] | None = None, + observed_at: str | None = None, + ingested_at: str | None = None, + store_text_snapshots: bool = False, +) -> GraphSourceKindIngestResult: + """Record source-kind-separated file coordinates as graph memory leaves.""" + + if not turn_id: + raise ValueError("turn_id must not be empty") + if not batch_id: + raise ValueError("batch_id must not be empty") + + observation_time = observed_at or _now_iso() + ingest_time = ingested_at or observation_time + + prepared_by_kind = _prepare_source_files_by_kind( + source_paths_by_kind, + observed_at=observation_time, + ingested_at=ingest_time, + store_text_snapshots=store_text_snapshots, + ) + _require_core_ego_time_axis(data_store) + raw_nodes: list[GraphMemoryNodeFrame] = [] + bundle_nodes: list[GraphMemoryNodeFrame] = [] + edges: list[GraphMemoryEdgeFrame] = [] + source_file_data_ids: list[str] = [] + source_text_snapshot_data_ids: list[str] = [] + source_kind_counts: dict[str, int] = {} + + for source_kind in sorted(prepared_by_kind): + prepared_files = prepared_by_kind[source_kind] + if not prepared_files: + continue + source_kind_counts[source_kind] = len(prepared_files) + source_file_data_ids.extend( + prepared.source_file_data_id for prepared in prepared_files + ) + source_text_snapshot_data_ids.extend( + prepared.text_snapshot_data_id + for prepared in prepared_files + if prepared.text_snapshot_data_id is not None + ) + + kind_raw_nodes = [ + _build_raw_source_node(prepared, data_store=data_store) + for prepared in prepared_files + ] + for node in kind_raw_nodes: + validate_graph_memory_node_frame(node) + raw_nodes.extend(kind_raw_nodes) + + bundle_node = _build_source_kind_bundle_node( + batch_id=batch_id, + source_kind=source_kind, + raw_nodes=kind_raw_nodes, + observed_at=observation_time, + ingested_at=ingest_time, + ) + validate_graph_memory_node_frame(bundle_node) + bundle_nodes.append(bundle_node) + + for raw_node in kind_raw_nodes: + edge = _build_contains_edge( + bundle_node=bundle_node, + raw_node=raw_node, + ) + validate_graph_memory_edge_frame(edge) + edges.append(edge) + + frame_id = graph_source_kind_ingest_frame_id(batch_id) + source_ingest_time_bundle_node = _build_source_ingest_time_bundle_node( + batch_id=batch_id, + source_kind_bundle_nodes=bundle_nodes, + observed_at=observation_time, + ingested_at=ingest_time, + ) + validate_graph_memory_node_frame(source_ingest_time_bundle_node) + edges.append( + _build_time_axis_edge( + source_ingest_time_bundle_node=source_ingest_time_bundle_node, + ) + ) + for bundle_node in bundle_nodes: + edges.append( + _build_source_ingest_bundle_edge( + source_ingest_time_bundle_node=source_ingest_time_bundle_node, + source_kind_bundle_node=bundle_node, + ) + ) + for edge in edges: + validate_graph_memory_edge_frame(edge) + + raw_source_node_ids = [node.node_id for node in raw_nodes] + source_kind_bundle_node_ids = [node.node_id for node in bundle_nodes] + source_ingest_time_bundle_node_id = source_ingest_time_bundle_node.node_id + graph_edge_ids = [edge.edge_id for edge in edges] + graph_snapshot = _build_source_ingest_snapshot( + batch_id=batch_id, + source_ingest_time_bundle_node=source_ingest_time_bundle_node, + source_kind_bundle_nodes=bundle_nodes, + raw_nodes=raw_nodes, + edges=edges, + source_data_ids=[*source_file_data_ids, *source_text_snapshot_data_ids], + ) + validate_graph_memory_snapshot_frame(graph_snapshot) + guide_packet = _build_source_ingest_guide_packet( + snapshot=graph_snapshot, + nodes=[source_ingest_time_bundle_node, *bundle_nodes, *raw_nodes], + source_data_ids=[ + graph_snapshot.snapshot_id, + source_ingest_time_bundle_node_id, + *source_kind_bundle_node_ids, + *raw_source_node_ids, + *graph_edge_ids, + *source_file_data_ids, + *source_text_snapshot_data_ids, + ], + ) + validate_rloop_graph_guide_packet_frame(guide_packet) + + event = trace_store.create_event( + turn_id=turn_id, + actor="graph_source_kind_ingest", + event_type="node_output", + timestamp=ingest_time, + input_ref=input_ref or [], + output_ref=[ + frame_id, + graph_snapshot.snapshot_id, + guide_packet.packet_id, + source_ingest_time_bundle_node_id, + *source_kind_bundle_node_ids, + *raw_source_node_ids, + ], + schema_status="passed", + ) + + created_data_ids: list[str] = [] + existing_data_ids: list[str] = [] + timestamp = event.timestamp + + for prepared in [ + prepared + for prepared_files in prepared_by_kind.values() + for prepared in prepared_files + ]: + _record_payload_if_missing( + data_store=data_store, + data_id=prepared.source_file_data_id, + data_type=GRAPH_SOURCE_FILE_DATA_TYPE, + payload=_source_file_payload(prepared), + created_at=timestamp, + source_trace_id=event.event_id, + created_data_ids=created_data_ids, + existing_data_ids=existing_data_ids, + ) + if prepared.text_snapshot_data_id is not None: + _record_payload_if_missing( + data_store=data_store, + data_id=prepared.text_snapshot_data_id, + data_type=GRAPH_SOURCE_TEXT_SNAPSHOT_DATA_TYPE, + payload=_source_text_snapshot_payload(prepared), + created_at=timestamp, + source_trace_id=event.event_id, + created_data_ids=created_data_ids, + existing_data_ids=existing_data_ids, + ) + + for node in [source_ingest_time_bundle_node, *bundle_nodes, *raw_nodes]: + _record_payload_if_missing( + data_store=data_store, + data_id=node.node_id, + data_type=f"graph_memory:node:{node.node_kind}", + payload=asdict(node), + created_at=timestamp, + source_trace_id=event.event_id, + created_data_ids=created_data_ids, + existing_data_ids=existing_data_ids, + ) + + for edge in edges: + _record_payload_if_missing( + data_store=data_store, + data_id=edge.edge_id, + data_type=f"graph_memory:edge:{edge.edge_kind}", + payload=asdict(edge), + created_at=timestamp, + source_trace_id=event.event_id, + created_data_ids=created_data_ids, + existing_data_ids=existing_data_ids, + ) + + _record_payload_if_missing( + data_store=data_store, + data_id=graph_snapshot.snapshot_id, + data_type="graph_memory:snapshot", + payload=asdict(graph_snapshot), + created_at=timestamp, + source_trace_id=event.event_id, + created_data_ids=created_data_ids, + existing_data_ids=existing_data_ids, + ) + _record_payload_if_missing( + data_store=data_store, + data_id=guide_packet.packet_id, + data_type="graph_memory:rloop_guide_packet", + payload=asdict(guide_packet), + created_at=timestamp, + source_trace_id=event.event_id, + created_data_ids=created_data_ids, + existing_data_ids=existing_data_ids, + ) + + frame_payload = { + "frame_id": frame_id, + "batch_id": batch_id, + "observed_at": observation_time, + "ingested_at": ingest_time, + "core_ego_time_axis_link_status": "linked", + "source_ingest_time_bundle_node_id": source_ingest_time_bundle_node_id, + "graph_snapshot_id": graph_snapshot.snapshot_id, + "rloop_graph_guide_packet_id": guide_packet.packet_id, + "source_kind_counts": source_kind_counts, + "source_file_data_ids": source_file_data_ids, + "source_text_snapshot_data_ids": source_text_snapshot_data_ids, + "text_snapshot_status": "stored" if store_text_snapshots else "not_stored", + "raw_source_node_ids": raw_source_node_ids, + "source_kind_bundle_node_ids": source_kind_bundle_node_ids, + "graph_edge_ids": graph_edge_ids, + "source_graph_node_ids": [ + source_ingest_time_bundle_node_id, + *source_kind_bundle_node_ids, + *raw_source_node_ids, + ], + "source_trace_ids": input_ref or [], + "source_data_ids": [ + *source_kind_bundle_node_ids, + *raw_source_node_ids, + *graph_edge_ids, + *source_file_data_ids, + *source_text_snapshot_data_ids, + ], + "generated_by": GRAPH_SOURCE_KIND_INGEST_GENERATOR, + "info_class": "absolute", + "semantic_judgement_status": "not_run", + } + _record_payload_if_missing( + data_store=data_store, + data_id=frame_id, + data_type=GRAPH_SOURCE_INGEST_FRAME_DATA_TYPE, + payload=frame_payload, + created_at=timestamp, + source_trace_id=event.event_id, + created_data_ids=created_data_ids, + existing_data_ids=existing_data_ids, + ) + + lineage_result = record_source_version_lineage_and_summary_invalidation( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + batch_id=batch_id, + input_ref=[event.event_id], + observed_source_file_data_ids=source_file_data_ids, + created_at=timestamp, + ) + created_data_ids.extend(lineage_result.created_data_ids) + existing_data_ids.extend(lineage_result.existing_data_ids) + + return GraphSourceKindIngestResult( + frame_id=frame_id, + trace_event_id=event.event_id, + source_file_data_ids=source_file_data_ids, + source_text_snapshot_data_ids=source_text_snapshot_data_ids, + source_ingest_time_bundle_node_id=source_ingest_time_bundle_node_id, + raw_source_node_ids=raw_source_node_ids, + source_kind_bundle_node_ids=source_kind_bundle_node_ids, + graph_edge_ids=graph_edge_ids, + graph_snapshot_id=graph_snapshot.snapshot_id, + rloop_graph_guide_packet_id=guide_packet.packet_id, + source_version_lineage_frame_ids=[ + frame.frame_id for frame in lineage_result.lineage_frames + ], + source_observation_ledger_frame_id=lineage_result.observation_ledger.frame_id, + source_observation_status_counts=dict( + lineage_result.observation_ledger.observation_status_counts + ), + summary_invalidation_ledger_frame_id=lineage_result.invalidation_ledger.frame_id, + invalidated_summary_node_ids=list( + lineage_result.invalidation_ledger.invalidated_summary_node_ids + ), + source_kind_counts=source_kind_counts, + created_data_ids=created_data_ids, + existing_data_ids=existing_data_ids, + ) + + +def graph_source_kind_ingest_frame_id(batch_id: str) -> str: + return f"graph_source:source_kind_ingest:{batch_id}" + + +def graph_source_ingest_snapshot_id(batch_id: str) -> str: + return f"graph:snapshot:{batch_id}:source_ingest" + + +def source_ingest_time_bundle_graph_node_id(batch_id: str) -> str: + return f"graph:source_ingest_time_bundle:{batch_id}" + + +def source_kind_bundle_graph_node_id(batch_id: str, source_kind: str) -> str: + _validate_source_kind(source_kind) + return f"graph:source_kind_bundle:{batch_id}:{source_kind}" + + +def source_file_data_id( + *, + source_kind: str, + path: str | Path, + content_sha1: str, + observed_at: str, +) -> str: + return f"source_file:{source_kind}:{_source_digest(source_kind, path, content_sha1, observed_at)}" + + +def source_text_snapshot_data_id( + *, + source_kind: str, + path: str | Path, + content_sha1: str, + observed_at: str, +) -> str: + return f"source_text:{source_kind}:{_source_digest(source_kind, path, content_sha1, observed_at)}" + + +def raw_source_graph_node_id( + *, + source_kind: str, + path: str | Path, + content_sha1: str, + observed_at: str, +) -> str: + _ = observed_at + return f"graph:raw_source:{source_kind}:{_source_version_digest(source_kind, path, content_sha1)}" + + +def _prepare_source_files_by_kind( + source_paths_by_kind: dict[str, list[str | Path]], + *, + observed_at: str, + ingested_at: str, + store_text_snapshots: bool, +) -> dict[str, list[_PreparedSourceFile]]: + prepared_by_kind: dict[str, list[_PreparedSourceFile]] = {} + for source_kind, paths in source_paths_by_kind.items(): + _validate_source_kind(source_kind) + prepared_files: list[_PreparedSourceFile] = [] + seen_paths: set[str] = set() + for raw_path in paths: + path = Path(raw_path).resolve() + normalized_path = path.as_posix() + if normalized_path in seen_paths: + continue + seen_paths.add(normalized_path) + if not path.exists(): + raise FileNotFoundError(normalized_path) + if not path.is_file(): + raise ValueError(f"source path must be a file: {normalized_path}") + source_last_modified_at = datetime.fromtimestamp( + path.stat().st_mtime + ).isoformat(timespec="seconds") + text = path.read_text(encoding="utf-8") + content_sha1 = hashlib.sha1(text.encode("utf-8")).hexdigest() + text_snapshot_id = ( + source_text_snapshot_data_id( + source_kind=source_kind, + path=path, + content_sha1=content_sha1, + observed_at=observed_at, + ) + if store_text_snapshots + else None + ) + prepared_files.append( + _PreparedSourceFile( + source_kind=source_kind, + path=normalized_path, + path_name=path.name, + suffix=path.suffix, + char_count=len(text), + content_sha1=content_sha1, + text=text, + text_snapshot_data_id=text_snapshot_id, + observed_at=observed_at, + ingested_at=ingested_at, + source_last_modified_at=source_last_modified_at, + exists_at_ingest=True, + source_file_data_id=source_file_data_id( + source_kind=source_kind, + path=path, + content_sha1=content_sha1, + observed_at=observed_at, + ), + raw_source_node_id=raw_source_graph_node_id( + source_kind=source_kind, + path=path, + content_sha1=content_sha1, + observed_at=observed_at, + ), + ) + ) + prepared_by_kind[source_kind] = prepared_files + return prepared_by_kind + + +def _build_raw_source_node( + prepared: _PreparedSourceFile, + *, + data_store: DataStore, +) -> GraphMemoryNodeFrame: + existing = data_store.get_record(prepared.raw_source_node_id) + if existing is not None: + if existing.data_type != "graph_memory:node:raw_source": + raise ValueError( + f"raw source data_id collision with different type: {prepared.raw_source_node_id}" + ) + if not isinstance(existing.payload, dict): + raise TypeError("existing raw source payload must be dict") + return GraphMemoryNodeFrame(**existing.payload) + return GraphMemoryNodeFrame( + node_id=prepared.raw_source_node_id, + node_kind="raw_source", + data_kind=prepared.source_kind, + summary_depth=0, + source_depth_min=0, + source_depth_max=0, + source_leaf_count=1, + source_summary_count=0, + source_bundle_kind=prepared.source_kind, + bundle_policy_id=GRAPH_SOURCE_KIND_BUNDLE_POLICY_ID, + source_char_count=prepared.char_count, + observed_at=prepared.observed_at, + ingested_at=prepared.ingested_at, + source_last_modified_at=prepared.source_last_modified_at, + exists_at_ingest=prepared.exists_at_ingest, + content_sha1=prepared.content_sha1, + source_graph_node_ids=[], + source_trace_ids=[], + source_data_ids=_unique_strings( + [ + prepared.source_file_data_id, + prepared.text_snapshot_data_id, + ] + ), + generated_by=GRAPH_MEMORY_CODE_GENERATOR, + info_class="absolute", + semantic_judgement_status="not_run", + ) + + +def _build_source_kind_bundle_node( + *, + batch_id: str, + source_kind: str, + raw_nodes: list[GraphMemoryNodeFrame], + observed_at: str, + ingested_at: str, +) -> GraphMemoryNodeFrame: + return GraphMemoryNodeFrame( + node_id=source_kind_bundle_graph_node_id(batch_id, source_kind), + node_kind="source_kind_bundle", + data_kind=f"{source_kind}_bundle", + summary_depth=0, + source_depth_min=0, + source_depth_max=0, + source_leaf_count=len(raw_nodes), + source_summary_count=0, + source_bundle_kind=source_kind, + bundle_policy_id=GRAPH_SOURCE_KIND_BUNDLE_POLICY_ID, + source_char_count=sum(node.source_char_count for node in raw_nodes), + observed_at=observed_at, + ingested_at=ingested_at, + source_graph_node_ids=[node.node_id for node in raw_nodes], + source_trace_ids=[], + source_data_ids=[ + source_data_id + for node in raw_nodes + for source_data_id in node.source_data_ids + ], + generated_by=GRAPH_MEMORY_CODE_GENERATOR, + info_class="absolute", + semantic_judgement_status="not_run", + ) + + +def _build_source_ingest_time_bundle_node( + *, + batch_id: str, + source_kind_bundle_nodes: list[GraphMemoryNodeFrame], + observed_at: str, + ingested_at: str, +) -> GraphMemoryNodeFrame: + return GraphMemoryNodeFrame( + node_id=source_ingest_time_bundle_graph_node_id(batch_id), + node_kind="source_ingest_time_bundle", + data_kind="source_ingest_time_bundle", + summary_depth=0, + source_depth_min=0, + source_depth_max=0, + source_leaf_count=sum(node.source_leaf_count for node in source_kind_bundle_nodes), + source_summary_count=0, + source_bundle_kind="source_ingest_time_bundle", + bundle_policy_id=GRAPH_SOURCE_KIND_BUNDLE_POLICY_ID, + source_char_count=sum(node.source_char_count for node in source_kind_bundle_nodes), + observed_at=observed_at, + ingested_at=ingested_at, + source_graph_node_ids=[node.node_id for node in source_kind_bundle_nodes], + source_trace_ids=[], + source_data_ids=[ + source_data_id + for node in source_kind_bundle_nodes + for source_data_id in node.source_data_ids + ], + generated_by=GRAPH_MEMORY_CODE_GENERATOR, + info_class="absolute", + semantic_judgement_status="not_run", + ) + + +def _build_contains_edge( + *, + bundle_node: GraphMemoryNodeFrame, + raw_node: GraphMemoryNodeFrame, +) -> GraphMemoryEdgeFrame: + return GraphMemoryEdgeFrame( + edge_id=f"graph:edge:contains:{bundle_node.node_id}:{raw_node.node_id}", + edge_kind="CONTAINS", + from_node_id=bundle_node.node_id, + to_node_id=raw_node.node_id, + source_graph_node_ids=[bundle_node.node_id, raw_node.node_id], + source_trace_ids=[], + source_data_ids=[ + bundle_node.node_id, + raw_node.node_id, + *raw_node.source_data_ids, + ], + generated_by=GRAPH_MEMORY_CODE_GENERATOR, + info_class="absolute", + semantic_judgement_status="not_run", + ) + + +def _build_time_axis_edge( + *, + source_ingest_time_bundle_node: GraphMemoryNodeFrame, +) -> GraphMemoryEdgeFrame: + return GraphMemoryEdgeFrame( + edge_id=( + f"graph:edge:child_of_time_axis:" + f"{TIME_AXIS_NODE_ID}:{source_ingest_time_bundle_node.node_id}" + ), + edge_kind="CHILD_OF_TIME_AXIS", + from_node_id=TIME_AXIS_NODE_ID, + to_node_id=source_ingest_time_bundle_node.node_id, + source_graph_node_ids=[ + TIME_AXIS_NODE_ID, + source_ingest_time_bundle_node.node_id, + ], + source_trace_ids=[], + source_data_ids=[ + TIME_AXIS_NODE_ID, + source_ingest_time_bundle_node.node_id, + ], + generated_by=GRAPH_MEMORY_CODE_GENERATOR, + info_class="absolute", + semantic_judgement_status="not_run", + ) + + +def _build_source_ingest_bundle_edge( + *, + source_ingest_time_bundle_node: GraphMemoryNodeFrame, + source_kind_bundle_node: GraphMemoryNodeFrame, +) -> GraphMemoryEdgeFrame: + return GraphMemoryEdgeFrame( + edge_id=( + f"graph:edge:contains:" + f"{source_ingest_time_bundle_node.node_id}:{source_kind_bundle_node.node_id}" + ), + edge_kind="CONTAINS", + from_node_id=source_ingest_time_bundle_node.node_id, + to_node_id=source_kind_bundle_node.node_id, + source_graph_node_ids=[ + source_ingest_time_bundle_node.node_id, + source_kind_bundle_node.node_id, + ], + source_trace_ids=[], + source_data_ids=[ + source_ingest_time_bundle_node.node_id, + source_kind_bundle_node.node_id, + *source_kind_bundle_node.source_data_ids, + ], + generated_by=GRAPH_MEMORY_CODE_GENERATOR, + info_class="absolute", + semantic_judgement_status="not_run", + ) + + +def _build_source_ingest_snapshot( + *, + batch_id: str, + source_ingest_time_bundle_node: GraphMemoryNodeFrame, + source_kind_bundle_nodes: list[GraphMemoryNodeFrame], + raw_nodes: list[GraphMemoryNodeFrame], + edges: list[GraphMemoryEdgeFrame], + source_data_ids: list[str], +) -> GraphMemorySnapshotFrame: + graph_node_ids = [ + CORE_EGO_ROOT_NODE_ID, + TIME_AXIS_NODE_ID, + source_ingest_time_bundle_node.node_id, + *[node.node_id for node in source_kind_bundle_nodes], + *[node.node_id for node in raw_nodes], + ] + graph_edge_ids = [edge.edge_id for edge in edges] + nodes_for_counts = [ + source_ingest_time_bundle_node, + *source_kind_bundle_nodes, + *raw_nodes, + ] + node_kind_counts = _count_by_node_kind(nodes_for_counts) + node_kind_counts["core_ego"] = 1 + node_kind_counts["time_axis"] = 1 + data_kind_counts = _count_by_data_kind(nodes_for_counts) + data_kind_counts["core_ego_root"] = 1 + data_kind_counts["time_axis"] = 1 + return GraphMemorySnapshotFrame( + snapshot_id=graph_source_ingest_snapshot_id(batch_id), + batch_id=batch_id, + root_node_id=CORE_EGO_ROOT_NODE_ID, + time_axis_node_id=TIME_AXIS_NODE_ID, + graph_node_ids=graph_node_ids, + graph_edge_ids=graph_edge_ids, + node_kind_counts=node_kind_counts, + edge_kind_counts=_count_edges_by_kind(edges), + data_kind_counts=data_kind_counts, + source_graph_node_ids=graph_node_ids, + source_trace_ids=[], + source_data_ids=_unique_strings( + [ + *graph_node_ids, + *graph_edge_ids, + *source_data_ids, + ] + ), + generated_by=GRAPH_MEMORY_CODE_GENERATOR, + info_class="absolute", + semantic_judgement_status="not_run", + ) + + +def _build_source_ingest_guide_packet( + *, + snapshot: GraphMemorySnapshotFrame, + nodes: list[GraphMemoryNodeFrame], + source_data_ids: list[str], +) -> RLoopGraphGuidePacketFrame: + summary_depths = [node.summary_depth for node in nodes] + source_leaf_counts = [node.source_leaf_count for node in nodes] + return RLoopGraphGuidePacketFrame( + packet_id=rloop_graph_guide_packet_id(snapshot.snapshot_id), + graph_snapshot_id=snapshot.snapshot_id, + target_consumer="R_LOOP", + available_entry_nodes=[TIME_AXIS_NODE_ID], + node_kind_counts=dict(snapshot.node_kind_counts), + data_kind_counts=dict(snapshot.data_kind_counts), + summary_depth_range=_int_range(summary_depths), + source_leaf_count_range=_int_range(source_leaf_counts), + risky_or_unreviewed_node_ids=[], + recommended_traversal_hints=[], + recommended_traversal_hints_status="not_run", + source_graph_node_ids=list(snapshot.graph_node_ids), + source_trace_ids=[], + source_data_ids=_unique_strings(source_data_ids), + generated_by=RLOOP_GUIDE_CODE_GENERATOR, + info_class="absolute", + semantic_judgement_status="not_run", + ) + + +def _source_file_payload(prepared: _PreparedSourceFile) -> dict[str, object]: + return { + "source_kind": prepared.source_kind, + "path": prepared.path, + "path_name": prepared.path_name, + "suffix": prepared.suffix, + "exists": True, + "exists_at_ingest": prepared.exists_at_ingest, + "char_count": prepared.char_count, + "observed_at": prepared.observed_at, + "ingested_at": prepared.ingested_at, + "source_last_modified_at": prepared.source_last_modified_at, + "content_sha1": prepared.content_sha1, + "generated_by": GRAPH_SOURCE_KIND_INGEST_GENERATOR, + "info_class": "absolute", + "semantic_judgement_status": "not_run", + } + + +def _source_text_snapshot_payload(prepared: _PreparedSourceFile) -> dict[str, object]: + return { + "source_kind": prepared.source_kind, + "path": prepared.path, + "observed_at": prepared.observed_at, + "ingested_at": prepared.ingested_at, + "content_sha1": prepared.content_sha1, + "char_count": prepared.char_count, + "text": prepared.text, + "generated_by": GRAPH_SOURCE_KIND_INGEST_GENERATOR, + "info_class": "absolute_copied_source", + "semantic_judgement_status": "not_run", + } + + +def _record_payload_if_missing( + *, + data_store: DataStore, + data_id: str, + data_type: str, + payload: dict[str, object], + created_at: str, + source_trace_id: str, + created_data_ids: list[str], + existing_data_ids: list[str], +) -> None: + existing = data_store.get_record(data_id) + if existing is not None: + if existing.data_type != data_type: + raise ValueError(f"graph source data_id collision with different type: {data_id}") + if existing.payload != payload: + raise ValueError(f"graph source data_id collision with different payload: {data_id}") + existing_data_ids.append(data_id) + return + data_store.create_record( + data_id=data_id, + data_type=data_type, + exists=True, + created_at=created_at, + source_trace_id=source_trace_id, + payload=payload, + ) + created_data_ids.append(data_id) + + +def _source_digest( + source_kind: str, + path: str | Path, + content_sha1: str, + observed_at: str, +) -> str: + _validate_source_kind(source_kind) + normalized_path = Path(path).resolve().as_posix() + digest_source = f"{source_kind}:{normalized_path}:{content_sha1}:{observed_at}" + return hashlib.sha1(digest_source.encode("utf-8")).hexdigest()[:16] + + +def _source_version_digest( + source_kind: str, + path: str | Path, + content_sha1: str, +) -> str: + _validate_source_kind(source_kind) + normalized_path = Path(path).resolve().as_posix() + digest_source = f"{source_kind}:{normalized_path}:{content_sha1}" + return hashlib.sha1(digest_source.encode("utf-8")).hexdigest()[:16] + + +def _require_core_ego_time_axis(data_store: DataStore) -> None: + missing = [ + data_id + for data_id in (CORE_EGO_ROOT_NODE_ID, TIME_AXIS_NODE_ID) + if data_store.get_record(data_id) is None + ] + if missing: + raise ValueError( + "source ingest requires existing CoreEgo time axis graph records: " + + ", ".join(missing) + ) + + +def _count_by_node_kind(nodes: list[GraphMemoryNodeFrame]) -> dict[str, int]: + counts: dict[str, int] = {} + for node in nodes: + counts[node.node_kind] = counts.get(node.node_kind, 0) + 1 + return counts + + +def _count_by_data_kind(nodes: list[GraphMemoryNodeFrame]) -> dict[str, int]: + counts: dict[str, int] = {} + for node in nodes: + counts[node.data_kind] = counts.get(node.data_kind, 0) + 1 + return counts + + +def _count_edges_by_kind(edges: list[GraphMemoryEdgeFrame]) -> dict[str, int]: + counts: dict[str, int] = {} + for edge in edges: + counts[edge.edge_kind] = counts.get(edge.edge_kind, 0) + 1 + return counts + + +def _int_range(values: list[int]) -> list[int]: + if not values: + return [0, 0] + return [min(values), max(values)] + + +def _unique_strings(values: list[str | None]) -> list[str]: + seen: set[str] = set() + result: list[str] = [] + for value in values: + if not value or value in seen: + continue + seen.add(value) + result.append(value) + return result + + +def _now_iso() -> str: + return datetime.now().isoformat(timespec="seconds") + + +def _validate_source_kind(source_kind: str) -> None: + if source_kind not in GRAPH_SOURCE_KINDS: + raise ValueError(f"unknown graph source kind: {source_kind}") + + +__all__ = [ + "GRAPH_SOURCE_FILE_DATA_TYPE", + "GRAPH_SOURCE_INGEST_FRAME_DATA_TYPE", + "GRAPH_SOURCE_KINDS", + "GRAPH_SOURCE_KIND_BUNDLE_POLICY_ID", + "GRAPH_SOURCE_KIND_INGEST_GENERATOR", + "GRAPH_SOURCE_OBSERVATION_POLICY_ID", + "GRAPH_SOURCE_TEXT_SNAPSHOT_DATA_TYPE", + "GraphSourceKindIngestResult", + "graph_source_ingest_snapshot_id", + "graph_source_kind_ingest_frame_id", + "raw_source_graph_node_id", + "record_graph_source_kind_ingest", + "source_file_data_id", + "source_ingest_time_bundle_graph_node_id", + "source_kind_bundle_graph_node_id", + "source_text_snapshot_data_id", +] diff --git a/songryeon_core/core/graph_vessel_adapter.py b/songryeon_core/core/graph_vessel_adapter.py new file mode 100644 index 0000000..3c9db66 --- /dev/null +++ b/songryeon_core/core/graph_vessel_adapter.py @@ -0,0 +1,383 @@ +from __future__ import annotations + +from dataclasses import asdict, dataclass +from datetime import datetime + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.graph_memory_export import ( + GRAPH_MEMORY_EXPORT_PACKET_DATA_TYPE, +) +from songryeon_core.core.graph_memory_store import ( + SONGRYEON_GRAPH_NAMESPACE, + SONGRYEON_VESSEL_DATABASE_NAME, + SONGRYEON_VESSEL_SERVICE_NAME, +) +from songryeon_core.core.trace_store import TraceStore + + +GRAPH_VESSEL_WRITE_PLAN_DATA_TYPE = "graph_vessel:write_plan" +GRAPH_VESSEL_ADAPTER_BOUNDARY_GENERATOR = "CODE:GRAPH_VESSEL_ADAPTER_BOUNDARY" +GRAPH_VESSEL_WRITE_PLAN_POLICY_ID = "GRAPH_VESSEL_WRITE_PLAN_BOUNDARY_V0" +GRAPH_VESSEL_WRITE_PLAN_SCHEMA_NAME = "GraphVesselWritePlan" +GRAPH_VESSEL_WRITE_PLAN_STATUSES = { + "ready_to_write", + "blocked_integrity_failed", +} +GRAPH_VESSEL_WRITE_OPERATION_KINDS = { + "upsert_source_payload", + "upsert_graph_node", + "upsert_graph_edge", + "upsert_support_record", +} + + +@dataclass(frozen=True) +class GraphVesselWriteOperation: + operation_id: str + operation_index: int + operation_kind: str + source_data_id: str + source_data_type: str + external_write_status: str = "not_run" + + +@dataclass(frozen=True) +class GraphVesselWritePlan: + plan_id: str + export_packet_id: str + export_packet_batch_id: str + created_at: str + policy_id: str + target_adapter_name: str + vessel_database_name: str + graph_namespace: str + plan_status: str + block_reason: str | None + external_write_status: str + operations: list[GraphVesselWriteOperation] + operation_count: int + operation_count_by_kind: dict[str, int] + graph_integrity_status: str + graph_integrity_summary: dict[str, object] + source_data_ids: list[str] + source_trace_ids: list[str] + generated_by: str = GRAPH_VESSEL_ADAPTER_BOUNDARY_GENERATOR + info_class: str = "absolute" + semantic_judgement_status: str = "not_run" + schema_name: str = GRAPH_VESSEL_WRITE_PLAN_SCHEMA_NAME + + +@dataclass(frozen=True) +class RecordedGraphVesselWritePlan: + plan: GraphVesselWritePlan + trace_event_id: str + created_data_ids: list[str] + existing_data_ids: list[str] + + +def graph_vessel_write_plan_id(batch_id: str) -> str: + if not batch_id: + raise ValueError("batch_id must not be empty") + return f"graph_vessel:write_plan:{batch_id}" + + +def build_graph_vessel_write_plan( + *, + data_store: DataStore, + export_packet_id: str, + created_at: str | None = None, +) -> GraphVesselWritePlan: + """Build a no-write Vessel adapter plan from a graph memory export packet.""" + + packet_record = data_store.require_record(export_packet_id) + if packet_record.data_type != GRAPH_MEMORY_EXPORT_PACKET_DATA_TYPE: + raise ValueError(f"expected graph memory export packet: {export_packet_id}") + if not isinstance(packet_record.payload, dict): + raise TypeError("graph memory export packet payload must be dict") + + packet = packet_record.payload + batch_id = _require_str(packet, "batch_id") + target_adapter_name = _require_str(packet, "target_adapter_name") + if target_adapter_name != SONGRYEON_VESSEL_SERVICE_NAME: + raise ValueError(f"unexpected target adapter name: {target_adapter_name}") + + plan_id = graph_vessel_write_plan_id(batch_id) + timestamp = created_at or _now_iso() + graph_integrity_status = _require_str(packet, "graph_integrity_status") + graph_integrity_summary = _require_dict(packet, "graph_integrity_summary") + source_trace_ids = _string_list(packet.get("source_trace_ids")) + + if graph_integrity_status != "passed": + plan = GraphVesselWritePlan( + plan_id=plan_id, + export_packet_id=export_packet_id, + export_packet_batch_id=batch_id, + created_at=timestamp, + policy_id=GRAPH_VESSEL_WRITE_PLAN_POLICY_ID, + target_adapter_name=target_adapter_name, + vessel_database_name=SONGRYEON_VESSEL_DATABASE_NAME, + graph_namespace=SONGRYEON_GRAPH_NAMESPACE, + plan_status="blocked_integrity_failed", + block_reason="CODE_STATUS:graph_integrity_failed", + external_write_status="not_run", + operations=[], + operation_count=0, + operation_count_by_kind={}, + graph_integrity_status=graph_integrity_status, + graph_integrity_summary=graph_integrity_summary, + source_data_ids=[export_packet_id], + source_trace_ids=source_trace_ids, + ) + _validate_write_plan(plan) + return plan + + operations = _build_write_operations( + data_store=data_store, + plan_id=plan_id, + source_payload_data_ids=[ + *_string_list(packet.get("source_file_metadata_data_ids")), + *_string_list(packet.get("source_text_snapshot_data_ids")), + ], + graph_node_data_ids=_string_list(packet.get("graph_node_data_ids")), + graph_edge_data_ids=_string_list(packet.get("graph_edge_data_ids")), + support_record_data_ids=[ + *_string_list(packet.get("core_ego_time_axis_frame_ids")), + *_string_list(packet.get("graph_snapshot_data_ids")), + *_string_list(packet.get("rloop_guide_packet_data_ids")), + *_string_list(packet.get("source_ingest_frame_data_ids")), + *_string_list(packet.get("source_manifest_frame_data_ids")), + *_string_list(packet.get("source_version_lineage_frame_data_ids")), + *_string_list(packet.get("source_observation_ledger_frame_data_ids")), + *_string_list(packet.get("summary_invalidation_ledger_frame_data_ids")), + ], + ) + plan = GraphVesselWritePlan( + plan_id=plan_id, + export_packet_id=export_packet_id, + export_packet_batch_id=batch_id, + created_at=timestamp, + policy_id=GRAPH_VESSEL_WRITE_PLAN_POLICY_ID, + target_adapter_name=target_adapter_name, + vessel_database_name=SONGRYEON_VESSEL_DATABASE_NAME, + graph_namespace=SONGRYEON_GRAPH_NAMESPACE, + plan_status="ready_to_write", + block_reason=None, + external_write_status="not_run", + operations=operations, + operation_count=len(operations), + operation_count_by_kind=_count_operations_by_kind(operations), + graph_integrity_status=graph_integrity_status, + graph_integrity_summary=graph_integrity_summary, + source_data_ids=_unique_strings( + [export_packet_id, *[operation.source_data_id for operation in operations]] + ), + source_trace_ids=source_trace_ids, + ) + _validate_write_plan(plan) + return plan + + +def record_graph_vessel_write_plan( + *, + trace_store: TraceStore, + data_store: DataStore, + turn_id: str, + export_packet_id: str, + created_at: str | None = None, +) -> RecordedGraphVesselWritePlan: + """Record a no-write Vessel adapter plan in TraceStore/DataStore.""" + + plan = build_graph_vessel_write_plan( + data_store=data_store, + export_packet_id=export_packet_id, + created_at=created_at, + ) + event = trace_store.create_event( + turn_id=turn_id, + actor="graph_vessel_adapter_boundary", + event_type="node_output", + timestamp=plan.created_at, + input_ref=plan.source_trace_ids, + output_ref=[plan.plan_id], + schema_status="passed", + ) + created_data_ids: list[str] = [] + existing_data_ids: list[str] = [] + _record_payload_if_missing( + data_store=data_store, + data_id=plan.plan_id, + data_type=GRAPH_VESSEL_WRITE_PLAN_DATA_TYPE, + payload=asdict(plan), + created_at=event.timestamp, + source_trace_id=event.event_id, + created_data_ids=created_data_ids, + existing_data_ids=existing_data_ids, + ) + return RecordedGraphVesselWritePlan( + plan=plan, + trace_event_id=event.event_id, + created_data_ids=created_data_ids, + existing_data_ids=existing_data_ids, + ) + + +def _build_write_operations( + *, + data_store: DataStore, + plan_id: str, + source_payload_data_ids: list[str], + graph_node_data_ids: list[str], + graph_edge_data_ids: list[str], + support_record_data_ids: list[str], +) -> list[GraphVesselWriteOperation]: + operations: list[GraphVesselWriteOperation] = [] + for operation_kind, data_ids in [ + ("upsert_source_payload", source_payload_data_ids), + ("upsert_graph_node", graph_node_data_ids), + ("upsert_graph_edge", graph_edge_data_ids), + ("upsert_support_record", support_record_data_ids), + ]: + for data_id in _unique_strings(data_ids): + record = data_store.require_record(data_id) + operation_index = len(operations) + 1 + operation = GraphVesselWriteOperation( + operation_id=f"{plan_id}:op:{operation_index:04d}", + operation_index=operation_index, + operation_kind=operation_kind, + source_data_id=record.data_id, + source_data_type=record.data_type, + ) + _validate_write_operation(operation) + operations.append(operation) + return operations + + +def _validate_write_plan(plan: GraphVesselWritePlan) -> None: + if plan.generated_by != GRAPH_VESSEL_ADAPTER_BOUNDARY_GENERATOR: + raise ValueError("GraphVesselWritePlan.generated_by must be code boundary") + if plan.info_class != "absolute": + raise ValueError("GraphVesselWritePlan.info_class must be absolute") + if plan.semantic_judgement_status != "not_run": + raise ValueError("GraphVesselWritePlan.semantic_judgement_status must be not_run") + if plan.external_write_status != "not_run": + raise ValueError("GraphVesselWritePlan.external_write_status must be not_run") + if plan.plan_status not in GRAPH_VESSEL_WRITE_PLAN_STATUSES: + raise ValueError("GraphVesselWritePlan.plan_status is invalid") + if plan.target_adapter_name != SONGRYEON_VESSEL_SERVICE_NAME: + raise ValueError("GraphVesselWritePlan.target_adapter_name is invalid") + if plan.vessel_database_name != SONGRYEON_VESSEL_DATABASE_NAME: + raise ValueError("GraphVesselWritePlan.vessel_database_name is invalid") + if plan.graph_namespace != SONGRYEON_GRAPH_NAMESPACE: + raise ValueError("GraphVesselWritePlan.graph_namespace is invalid") + if plan.plan_status == "ready_to_write" and not plan.operations: + raise ValueError("ready_to_write plan must include operations") + if plan.plan_status == "blocked_integrity_failed" and plan.operations: + raise ValueError("blocked write plan must not include operations") + if plan.operation_count != len(plan.operations): + raise ValueError("GraphVesselWritePlan.operation_count mismatch") + for operation in plan.operations: + _validate_write_operation(operation) + + +def _validate_write_operation(operation: GraphVesselWriteOperation) -> None: + if not operation.operation_id: + raise ValueError("GraphVesselWriteOperation.operation_id must not be empty") + if operation.operation_index <= 0: + raise ValueError("GraphVesselWriteOperation.operation_index must be positive") + if operation.operation_kind not in GRAPH_VESSEL_WRITE_OPERATION_KINDS: + raise ValueError("GraphVesselWriteOperation.operation_kind is invalid") + if not operation.source_data_id: + raise ValueError("GraphVesselWriteOperation.source_data_id must not be empty") + if not operation.source_data_type: + raise ValueError("GraphVesselWriteOperation.source_data_type must not be empty") + if operation.external_write_status != "not_run": + raise ValueError("GraphVesselWriteOperation.external_write_status must be not_run") + + +def _record_payload_if_missing( + *, + data_store: DataStore, + data_id: str, + data_type: str, + payload: dict[str, object], + created_at: str, + source_trace_id: str, + created_data_ids: list[str], + existing_data_ids: list[str], +) -> None: + existing = data_store.get_record(data_id) + if existing is not None: + if existing.data_type != data_type: + raise ValueError(f"graph vessel plan data_id collision with different type: {data_id}") + if existing.payload != payload: + raise ValueError(f"graph vessel plan data_id collision with different payload: {data_id}") + existing_data_ids.append(data_id) + return + data_store.create_record( + data_id=data_id, + data_type=data_type, + exists=True, + created_at=created_at, + source_trace_id=source_trace_id, + payload=payload, + ) + created_data_ids.append(data_id) + + +def _count_operations_by_kind( + operations: list[GraphVesselWriteOperation], +) -> dict[str, int]: + counts: dict[str, int] = {} + for operation in operations: + counts[operation.operation_kind] = counts.get(operation.operation_kind, 0) + 1 + return counts + + +def _require_str(payload: dict[str, object], field_name: str) -> str: + value = payload.get(field_name) + if not isinstance(value, str) or not value: + raise ValueError(f"{field_name} must be a non-empty string") + return value + + +def _require_dict(payload: dict[str, object], field_name: str) -> dict[str, object]: + value = payload.get(field_name) + if not isinstance(value, dict): + raise ValueError(f"{field_name} must be a dict") + return value + + +def _string_list(value: object) -> list[str]: + if not isinstance(value, list): + return [] + return [item for item in value if isinstance(item, str) and item] + + +def _unique_strings(values: list[str | None]) -> list[str]: + seen: set[str] = set() + result: list[str] = [] + for value in values: + if not value or value in seen: + continue + seen.add(value) + result.append(value) + return result + + +def _now_iso() -> str: + return datetime.now().isoformat(timespec="seconds") + + +__all__ = [ + "GRAPH_VESSEL_ADAPTER_BOUNDARY_GENERATOR", + "GRAPH_VESSEL_WRITE_OPERATION_KINDS", + "GRAPH_VESSEL_WRITE_PLAN_DATA_TYPE", + "GRAPH_VESSEL_WRITE_PLAN_POLICY_ID", + "GRAPH_VESSEL_WRITE_PLAN_STATUSES", + "GraphVesselWriteOperation", + "GraphVesselWritePlan", + "RecordedGraphVesselWritePlan", + "build_graph_vessel_write_plan", + "graph_vessel_write_plan_id", + "record_graph_vessel_write_plan", +] diff --git a/songryeon_core/core/graph_vessel_inspect.py b/songryeon_core/core/graph_vessel_inspect.py new file mode 100644 index 0000000..2005f32 --- /dev/null +++ b/songryeon_core/core/graph_vessel_inspect.py @@ -0,0 +1,771 @@ +from __future__ import annotations + +import json +from dataclasses import asdict, dataclass +from datetime import datetime +from typing import Any, Callable + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.graph_memory_store import ( + SONGRYEON_GRAPH_NAMESPACE, + SONGRYEON_VESSEL_SERVICE_NAME, +) +from songryeon_core.core.graph_vessel_neo4j import GraphVesselNeo4jConfig +from songryeon_core.core.trace_store import TraceStore + + +GRAPH_VESSEL_NEO4J_INSPECT_RESULT_DATA_TYPE = "graph_vessel:neo4j_inspect_result" +GRAPH_VESSEL_NEO4J_INSPECT_GENERATOR = "CODE:GRAPH_VESSEL_NEO4J_INSPECTOR" +GRAPH_VESSEL_NEO4J_INSPECT_POLICY_ID = "VESSEL_INSPECT_MANUAL_WALK_V0" +GRAPH_VESSEL_NEO4J_INSPECT_SCHEMA_NAME = "GraphVesselNeo4jInspectResultFrame" +GRAPH_VESSEL_NEO4J_INSPECT_STATUSES = { + "passed", + "empty", + "adapter_unavailable", + "read_failed", +} + +Neo4jDriverFactory = Callable[..., Any] + + +@dataclass(frozen=True) +class GraphVesselNeo4jInspectPathItem: + core_data_id: str + core_display_name: str + axis_data_id: str + axis_display_name: str + bundle_data_id: str | None + bundle_display_name: str | None + bundle_created_at: str | None + bundle_written_at: str | None + raw_capsule_data_id: str | None + raw_capsule_display_name: str | None + raw_capsule_created_at: str | None + raw_capsule_written_at: str | None + raw_capsule_source_trace_id: str | None + + +@dataclass(frozen=True) +class GraphVesselNeo4jInspectSummarySampleItem: + summary_data_id: str + summary_display_name: str + data_kind: str | None + summary_depth: int | None + summary_status: str | None + validity_status: str | None + review_status: str | None + target_graph_node_id: str | None + target_display_name: str | None + target_node_kind: str | None + source_leaf_count: int | None + source_summary_count: int | None + info_class: str | None + generated_by: str | None + summary_text_preview: str + + +@dataclass(frozen=True) +class GraphVesselNeo4jInspectResultFrame: + result_id: str + created_at: str + policy_id: str + target_adapter_name: str + vessel_database_name: str + graph_namespace: str + inspect_status: str + failure_type: str | None + failure_reason: str | None + inspected_path_count: int + core_count: int | None + time_axis_count: int | None + time_bundle_count: int | None + raw_capsule_count: int | None + summary_count: int | None + active_summary_count: int | None + invalidated_summary_count: int | None + summary_count_by_data_kind: dict[str, int] + summary_count_by_depth: dict[str, int] + summary_sample_items: list[dict[str, object]] + summary_lines: list[str] + path_items: list[dict[str, object]] + tree_lines: list[str] + source_data_ids: list[str] + source_trace_ids: list[str] + generated_by: str = GRAPH_VESSEL_NEO4J_INSPECT_GENERATOR + info_class: str = "absolute" + semantic_judgement_status: str = "not_run" + schema_name: str = GRAPH_VESSEL_NEO4J_INSPECT_SCHEMA_NAME + + +@dataclass(frozen=True) +class RecordedGraphVesselNeo4jInspectResult: + result: GraphVesselNeo4jInspectResultFrame + trace_event_id: str + created_data_ids: list[str] + existing_data_ids: list[str] + + +def graph_vessel_neo4j_inspect_result_id(batch_id: str) -> str: + if not batch_id: + raise ValueError("batch_id must not be empty") + return f"graph_vessel:neo4j_inspect_result:{_stable_suffix(batch_id)}" + + +def inspect_graph_vessel_from_neo4j( + *, + config: GraphVesselNeo4jConfig, + batch_id: str = "manual_vessel_inspect", + limit: int = 50, + created_at: str | None = None, + driver_factory: Neo4jDriverFactory | None = None, +) -> GraphVesselNeo4jInspectResultFrame: + timestamp = created_at or _now_iso() + safe_limit = max(1, min(limit, 500)) + base = _InspectBase( + result_id=graph_vessel_neo4j_inspect_result_id(batch_id), + created_at=timestamp, + target_adapter_name=SONGRYEON_VESSEL_SERVICE_NAME, + vessel_database_name=config.database, + graph_namespace=SONGRYEON_GRAPH_NAMESPACE, + ) + + config_failure = _config_failure(config) + if config_failure is not None: + failure_type, failure_reason = config_failure + return _make_result( + base=base, + inspect_status="adapter_unavailable", + failure_type=failure_type, + failure_reason=failure_reason, + counts=_empty_counts(), + path_items=[], + ) + + driver_factory = driver_factory or _load_neo4j_driver_factory() + if driver_factory is None: + return _make_result( + base=base, + inspect_status="adapter_unavailable", + failure_type="neo4j_driver_missing", + failure_reason="Python neo4j package is not installed.", + counts=_empty_counts(), + path_items=[], + ) + + try: + auth = None if config.allow_no_auth and config.password is None else (config.user, config.password) + driver = driver_factory(config.uri, auth=auth) + try: + with driver.session(database=config.database) as session: + ( + counts, + path_items, + summary_items, + summary_count_by_data_kind, + summary_count_by_depth, + ) = session.execute_read( + _execute_inspect, + SONGRYEON_GRAPH_NAMESPACE, + safe_limit, + ) + finally: + close = getattr(driver, "close", None) + if callable(close): + close() + except Exception as exc: + return _make_result( + base=base, + inspect_status="read_failed", + failure_type="neo4j_read_failed", + failure_reason=f"{type(exc).__name__}: {exc}", + counts=_empty_counts(), + path_items=[], + ) + + return _make_result( + base=base, + inspect_status="passed" if path_items else "empty", + failure_type=None if path_items else "no_inspectable_path", + failure_reason=None if path_items else "No CoreEgo -> TimeAxis path was found.", + counts=counts, + path_items=path_items, + summary_items=summary_items, + summary_count_by_data_kind=summary_count_by_data_kind, + summary_count_by_depth=summary_count_by_depth, + ) + + +def record_graph_vessel_neo4j_inspect_result( + *, + trace_store: TraceStore, + data_store: DataStore, + turn_id: str, + config: GraphVesselNeo4jConfig, + batch_id: str = "manual_vessel_inspect", + limit: int = 50, + created_at: str | None = None, + driver_factory: Neo4jDriverFactory | None = None, +) -> RecordedGraphVesselNeo4jInspectResult: + result = inspect_graph_vessel_from_neo4j( + config=config, + batch_id=batch_id, + limit=limit, + created_at=created_at, + driver_factory=driver_factory, + ) + event = trace_store.create_event( + turn_id=turn_id, + actor="graph_vessel_neo4j_inspector", + event_type="node_output", + timestamp=result.created_at, + input_ref=[], + output_ref=[result.result_id], + schema_status="passed", + ) + created_data_ids: list[str] = [] + existing_data_ids: list[str] = [] + _record_payload_if_missing( + data_store=data_store, + data_id=result.result_id, + data_type=GRAPH_VESSEL_NEO4J_INSPECT_RESULT_DATA_TYPE, + payload=asdict(result), + created_at=event.timestamp, + source_trace_id=event.event_id, + created_data_ids=created_data_ids, + existing_data_ids=existing_data_ids, + ) + return RecordedGraphVesselNeo4jInspectResult( + result=result, + trace_event_id=event.event_id, + created_data_ids=created_data_ids, + existing_data_ids=existing_data_ids, + ) + + +@dataclass(frozen=True) +class _InspectBase: + result_id: str + created_at: str + target_adapter_name: str + vessel_database_name: str + graph_namespace: str + + +@dataclass(frozen=True) +class _InspectCounts: + core_count: int | None + time_axis_count: int | None + time_bundle_count: int | None + raw_capsule_count: int | None + summary_count: int | None + active_summary_count: int | None + invalidated_summary_count: int | None + + +def _make_result( + *, + base: _InspectBase, + inspect_status: str, + failure_type: str | None, + failure_reason: str | None, + counts: _InspectCounts, + path_items: list[GraphVesselNeo4jInspectPathItem], + summary_items: list[GraphVesselNeo4jInspectSummarySampleItem] | None = None, + summary_count_by_data_kind: dict[str, int] | None = None, + summary_count_by_depth: dict[str, int] | None = None, +) -> GraphVesselNeo4jInspectResultFrame: + path_item_payloads = [asdict(item) for item in path_items] + summary_items = summary_items or [] + summary_item_payloads = [asdict(item) for item in summary_items] + source_data_ids = _unique_strings( + [ + value + for item in path_items + for value in ( + item.core_data_id, + item.axis_data_id, + item.bundle_data_id, + item.raw_capsule_data_id, + ) + ] + + [item.summary_data_id for item in summary_items] + + [item.target_graph_node_id for item in summary_items] + ) + source_trace_ids = _unique_strings( + [item.raw_capsule_source_trace_id for item in path_items] + ) + result = GraphVesselNeo4jInspectResultFrame( + result_id=base.result_id, + created_at=base.created_at, + policy_id=GRAPH_VESSEL_NEO4J_INSPECT_POLICY_ID, + target_adapter_name=base.target_adapter_name, + vessel_database_name=base.vessel_database_name, + graph_namespace=base.graph_namespace, + inspect_status=inspect_status, + failure_type=failure_type, + failure_reason=failure_reason, + inspected_path_count=len(path_items), + core_count=counts.core_count, + time_axis_count=counts.time_axis_count, + time_bundle_count=counts.time_bundle_count, + raw_capsule_count=counts.raw_capsule_count, + summary_count=counts.summary_count, + active_summary_count=counts.active_summary_count, + invalidated_summary_count=counts.invalidated_summary_count, + summary_count_by_data_kind=summary_count_by_data_kind or {}, + summary_count_by_depth=summary_count_by_depth or {}, + summary_sample_items=summary_item_payloads, + summary_lines=_build_summary_lines(summary_items), + path_items=path_item_payloads, + tree_lines=_build_tree_lines(path_items), + source_data_ids=source_data_ids, + source_trace_ids=source_trace_ids, + ) + _validate_inspect_result(result) + return result + + +def _execute_inspect( + tx: object, + graph_namespace: str, + limit: int, +) -> tuple[ + _InspectCounts, + list[GraphVesselNeo4jInspectPathItem], + list[GraphVesselNeo4jInspectSummarySampleItem], + dict[str, int], + dict[str, int], +]: + counts = _InspectCounts( + core_count=_count_label(tx, graph_namespace, "CoreEgo"), + time_axis_count=_count_label(tx, graph_namespace, "TimeAxis"), + time_bundle_count=_count_label(tx, graph_namespace, "TimeBundle"), + raw_capsule_count=_count_label(tx, graph_namespace, "RawCapsule"), + summary_count=_count_label(tx, graph_namespace, "SummaryGraphNode"), + active_summary_count=None, + invalidated_summary_count=None, + ) + summary_records = _read_summary_records(tx, graph_namespace) + summary_items = [ + _summary_item_from_record(record) + for record in summary_records[:limit] + ] + summary_count_by_data_kind = _count_summary_items_by_data_kind(summary_records) + summary_count_by_depth = _count_summary_items_by_depth(summary_records) + active_summary_count = sum( + 1 + for record in summary_records + if _summary_payload(record).get("validity_status") == "active" + ) + invalidated_summary_count = sum( + 1 + for record in summary_records + if _text(_summary_payload(record).get("validity_status")).startswith("invalidated") + ) + counts = _InspectCounts( + core_count=counts.core_count, + time_axis_count=counts.time_axis_count, + time_bundle_count=counts.time_bundle_count, + raw_capsule_count=counts.raw_capsule_count, + summary_count=counts.summary_count, + active_summary_count=active_summary_count, + invalidated_summary_count=invalidated_summary_count, + ) + result = tx.run( + """ + MATCH (core:CoreEgo {graph_namespace: $graph_namespace}) + -[:HAS_AXIS {graph_namespace: $graph_namespace}]-> + (axis:TimeAxis {graph_namespace: $graph_namespace}) + OPTIONAL MATCH (axis)-[:HAS_BUNDLE {graph_namespace: $graph_namespace}]-> + (bundle:TimeBundle {graph_namespace: $graph_namespace}) + OPTIONAL MATCH (bundle)-[:CONTAINS_MEMORY {graph_namespace: $graph_namespace}]-> + (capsule:RawCapsule {graph_namespace: $graph_namespace}) + RETURN + core.data_id AS core_data_id, + coalesce(core.display_name, core.data_id) AS core_display_name, + axis.data_id AS axis_data_id, + coalesce(axis.display_name, axis.data_id) AS axis_display_name, + bundle.data_id AS bundle_data_id, + coalesce(bundle.display_name, bundle.data_id) AS bundle_display_name, + bundle.created_at AS bundle_created_at, + bundle.written_at AS bundle_written_at, + capsule.data_id AS raw_capsule_data_id, + coalesce(capsule.display_name, capsule.data_id) AS raw_capsule_display_name, + capsule.created_at AS raw_capsule_created_at, + capsule.written_at AS raw_capsule_written_at, + capsule.source_trace_id AS raw_capsule_source_trace_id + ORDER BY bundle.created_at, bundle.data_id, capsule.created_at, capsule.data_id + LIMIT $limit + """, + graph_namespace=graph_namespace, + limit=limit, + ) + return ( + counts, + [_path_item_from_record(record) for record in _records(result)], + summary_items, + summary_count_by_data_kind, + summary_count_by_depth, + ) + + +def _path_item_from_record(record: object) -> GraphVesselNeo4jInspectPathItem: + return GraphVesselNeo4jInspectPathItem( + core_data_id=_require_record_str(record, "core_data_id"), + core_display_name=_require_record_str(record, "core_display_name"), + axis_data_id=_require_record_str(record, "axis_data_id"), + axis_display_name=_require_record_str(record, "axis_display_name"), + bundle_data_id=_optional_record_str(record, "bundle_data_id"), + bundle_display_name=_optional_record_str(record, "bundle_display_name"), + bundle_created_at=_optional_record_str(record, "bundle_created_at"), + bundle_written_at=_optional_record_str(record, "bundle_written_at"), + raw_capsule_data_id=_optional_record_str(record, "raw_capsule_data_id"), + raw_capsule_display_name=_optional_record_str(record, "raw_capsule_display_name"), + raw_capsule_created_at=_optional_record_str(record, "raw_capsule_created_at"), + raw_capsule_written_at=_optional_record_str(record, "raw_capsule_written_at"), + raw_capsule_source_trace_id=_optional_record_str(record, "raw_capsule_source_trace_id"), + ) + + +def _build_tree_lines(path_items: list[GraphVesselNeo4jInspectPathItem]) -> list[str]: + if not path_items: + return [] + lines: list[str] = [] + seen_core: set[str] = set() + seen_axis: set[tuple[str, str]] = set() + seen_bundle: set[tuple[str, str, str]] = set() + for item in path_items: + if item.core_data_id not in seen_core: + lines.append(f"{item.core_display_name} [{item.core_data_id}]") + seen_core.add(item.core_data_id) + axis_key = (item.core_data_id, item.axis_data_id) + if axis_key not in seen_axis: + lines.append(f" HAS_AXIS -> {item.axis_display_name} [{item.axis_data_id}]") + seen_axis.add(axis_key) + if item.bundle_data_id is None: + continue + bundle_key = (item.core_data_id, item.axis_data_id, item.bundle_data_id) + if bundle_key not in seen_bundle: + lines.append( + f" HAS_BUNDLE -> {item.bundle_display_name or item.bundle_data_id} " + f"[{item.bundle_data_id}]" + ) + seen_bundle.add(bundle_key) + if item.raw_capsule_data_id is not None: + lines.append( + f" CONTAINS_MEMORY -> " + f"{item.raw_capsule_display_name or item.raw_capsule_data_id} " + f"[{item.raw_capsule_data_id}]" + ) + return lines + + +def _read_summary_records(tx: object, graph_namespace: str) -> list[object]: + result = tx.run( + """ + MATCH (summary:SummaryGraphNode {graph_namespace: $graph_namespace}) + OPTIONAL MATCH (summary)-[:SUMMARY_OF {graph_namespace: $graph_namespace}]-> + (target:VesselRecord {graph_namespace: $graph_namespace}) + RETURN + summary.data_id AS summary_data_id, + coalesce(summary.display_name, summary.data_id) AS summary_display_name, + summary.data_kind AS summary_data_kind, + summary.info_class AS summary_info_class, + summary.generated_by AS summary_generated_by, + summary.payload_json AS summary_payload_json, + target.data_id AS target_graph_node_id, + coalesce(target.display_name, target.data_id) AS target_display_name, + target.node_kind AS target_node_kind + ORDER BY summary.data_kind, summary.data_id + LIMIT 10000 + """, + graph_namespace=graph_namespace, + ) + return _records(result) + + +def _summary_item_from_record(record: object) -> GraphVesselNeo4jInspectSummarySampleItem: + payload = _summary_payload(record) + summary_text = _text(payload.get("summary_text")) + target_graph_node_id = ( + _optional_record_str(record, "target_graph_node_id") + or _text(payload.get("target_graph_node_id")) + or None + ) + return GraphVesselNeo4jInspectSummarySampleItem( + summary_data_id=_require_record_str(record, "summary_data_id"), + summary_display_name=_require_record_str(record, "summary_display_name"), + data_kind=_optional_record_str(record, "summary_data_kind") + or _text(payload.get("data_kind")) + or None, + summary_depth=_optional_int(payload.get("summary_depth")), + summary_status=_text(payload.get("summary_status")) or None, + validity_status=_text(payload.get("validity_status")) or None, + review_status=_text(payload.get("review_status")) or None, + target_graph_node_id=target_graph_node_id, + target_display_name=_optional_record_str(record, "target_display_name"), + target_node_kind=_optional_record_str(record, "target_node_kind") + or _text(payload.get("target_node_kind")) + or None, + source_leaf_count=_optional_int(payload.get("source_leaf_count")), + source_summary_count=_optional_int(payload.get("source_summary_count")), + info_class=_optional_record_str(record, "summary_info_class") + or _text(payload.get("info_class")) + or None, + generated_by=_optional_record_str(record, "summary_generated_by") + or _text(payload.get("generated_by")) + or None, + summary_text_preview=_preview(summary_text), + ) + + +def _count_summary_items_by_data_kind(records: list[object]) -> dict[str, int]: + counts: dict[str, int] = {} + for record in records: + payload = _summary_payload(record) + data_kind = ( + _optional_record_str(record, "summary_data_kind") + or _text(payload.get("data_kind")) + or "unknown" + ) + counts[data_kind] = counts.get(data_kind, 0) + 1 + return dict(sorted(counts.items())) + + +def _count_summary_items_by_depth(records: list[object]) -> dict[str, int]: + counts: dict[str, int] = {} + for record in records: + depth = _optional_int(_summary_payload(record).get("summary_depth")) + key = str(depth) if depth is not None else "unknown" + counts[key] = counts.get(key, 0) + 1 + return dict(sorted(counts.items(), key=lambda item: item[0])) + + +def _build_summary_lines( + summary_items: list[GraphVesselNeo4jInspectSummarySampleItem], +) -> list[str]: + if not summary_items: + return [] + lines = ["Summary samples"] + for item in summary_items: + lines.append( + " " + f"{item.data_kind or 'unknown'}" + f"(depth={item.summary_depth if item.summary_depth is not None else 'unknown'}, " + f"info={item.info_class or 'unknown'}) " + f"[{item.summary_data_id}]" + ) + if item.target_graph_node_id: + target_name = item.target_display_name or item.target_graph_node_id + lines.append(f" SUMMARY_OF -> {target_name} [{item.target_graph_node_id}]") + if item.summary_text_preview: + lines.append(f" preview: {item.summary_text_preview}") + return lines + + +def _count_label(tx: object, graph_namespace: str, label: str) -> int: + _validate_cypher_token(label) + result = tx.run( + f""" + MATCH (n:{label} {{graph_namespace: $graph_namespace}}) + RETURN count(n) AS count + """, + graph_namespace=graph_namespace, + ) + return _single_count(result) + + +def _config_failure(config: GraphVesselNeo4jConfig) -> tuple[str, str] | None: + if not config.uri or not config.user or not config.database: + return ( + "neo4j_config_missing", + "Neo4j uri/user/database config is incomplete.", + ) + if config.password is None and not config.allow_no_auth: + return ( + "neo4j_config_missing", + "Neo4j password is not configured. Set SONGRYEON_NEO4J_PASSWORD or pass --password.", + ) + return None + + +def _empty_counts() -> _InspectCounts: + return _InspectCounts( + core_count=None, + time_axis_count=None, + time_bundle_count=None, + raw_capsule_count=None, + summary_count=None, + active_summary_count=None, + invalidated_summary_count=None, + ) + + +def _validate_inspect_result(result: GraphVesselNeo4jInspectResultFrame) -> None: + if result.generated_by != GRAPH_VESSEL_NEO4J_INSPECT_GENERATOR: + raise ValueError("GraphVesselNeo4jInspectResultFrame.generated_by must be inspector") + if result.info_class != "absolute": + raise ValueError("GraphVesselNeo4jInspectResultFrame.info_class must be absolute") + if result.semantic_judgement_status != "not_run": + raise ValueError("GraphVesselNeo4jInspectResultFrame.semantic_judgement_status must be not_run") + if result.inspect_status not in GRAPH_VESSEL_NEO4J_INSPECT_STATUSES: + raise ValueError("GraphVesselNeo4jInspectResultFrame.inspect_status is invalid") + if result.inspect_status == "passed" and result.inspected_path_count < 1: + raise ValueError("passed inspect result must include at least one path item") + + +def _record_payload_if_missing( + *, + data_store: DataStore, + data_id: str, + data_type: str, + payload: dict[str, object], + created_at: str, + source_trace_id: str, + created_data_ids: list[str], + existing_data_ids: list[str], +) -> None: + existing = data_store.get_record(data_id) + if existing is not None: + if existing.data_type != data_type: + raise ValueError(f"Neo4j inspect result data_id collision with different type: {data_id}") + if existing.payload != payload: + raise ValueError(f"Neo4j inspect result data_id collision with different payload: {data_id}") + existing_data_ids.append(data_id) + return + data_store.create_record( + data_id=data_id, + data_type=data_type, + exists=True, + created_at=created_at, + source_trace_id=source_trace_id, + payload=payload, + ) + created_data_ids.append(data_id) + + +def _load_neo4j_driver_factory() -> Neo4jDriverFactory | None: + try: + from neo4j import GraphDatabase + except ImportError: + return None + return GraphDatabase.driver + + +def _records(result: object) -> list[object]: + return list(result) + + +def _require_record_str(record: object, field_name: str) -> str: + value = _record_value(record, field_name) + if not isinstance(value, str) or not value: + raise ValueError(f"Neo4j inspect record field is missing: {field_name}") + return value + + +def _optional_record_str(record: object, field_name: str) -> str | None: + value = _record_value(record, field_name) + return value if isinstance(value, str) and value else None + + +def _record_value(record: object, field_name: str) -> object: + try: + return record[field_name] # type: ignore[index] + except (KeyError, TypeError): + data = getattr(record, "data", None) + if callable(data): + payload = data() + if isinstance(payload, dict): + return payload.get(field_name) + return None + + +def _summary_payload(record: object) -> dict[str, object]: + payload_json = _optional_record_str(record, "summary_payload_json") + if not payload_json: + return {} + try: + payload = json.loads(payload_json) + except json.JSONDecodeError: + return {} + return payload if isinstance(payload, dict) else {} + + +def _optional_int(value: object) -> int | None: + if isinstance(value, bool): + return None + if isinstance(value, int): + return value + if isinstance(value, str): + try: + return int(value) + except ValueError: + return None + return None + + +def _text(value: object) -> str: + return value if isinstance(value, str) else "" + + +def _preview(text: str, *, max_chars: int = 180) -> str: + compact = " ".join(text.split()) + if len(compact) <= max_chars: + return compact + return compact[: max_chars - 3].rstrip() + "..." + + +def _single_count(result: object) -> int: + single = getattr(result, "single", None) + if callable(single): + record = single() + if record is None: + return 0 + try: + return int(record["count"]) + except (KeyError, TypeError, ValueError): + return 0 + try: + first = next(iter(result)) + return int(first["count"]) + except (KeyError, TypeError, ValueError, StopIteration): + return 0 + + +def _validate_cypher_token(value: str) -> None: + if not value or not value.replace("_", "").isalnum() or not value[0].isalpha(): + raise ValueError(f"unsafe Neo4j token: {value}") + + +def _unique_strings(values: list[str | None]) -> list[str]: + seen: set[str] = set() + result: list[str] = [] + for value in values: + if not value or value in seen: + continue + seen.add(value) + result.append(value) + return result + + +def _stable_suffix(value: str) -> str: + return value.replace(":", "_").replace("/", "_").replace("\\", "_") + + +def _now_iso() -> str: + return datetime.now().isoformat(timespec="seconds") + + +__all__ = [ + "GRAPH_VESSEL_NEO4J_INSPECT_GENERATOR", + "GRAPH_VESSEL_NEO4J_INSPECT_POLICY_ID", + "GRAPH_VESSEL_NEO4J_INSPECT_RESULT_DATA_TYPE", + "GRAPH_VESSEL_NEO4J_INSPECT_STATUSES", + "GraphVesselNeo4jInspectPathItem", + "GraphVesselNeo4jInspectResultFrame", + "GraphVesselNeo4jInspectSummarySampleItem", + "RecordedGraphVesselNeo4jInspectResult", + "graph_vessel_neo4j_inspect_result_id", + "inspect_graph_vessel_from_neo4j", + "record_graph_vessel_neo4j_inspect_result", +] diff --git a/songryeon_core/core/graph_vessel_neo4j.py b/songryeon_core/core/graph_vessel_neo4j.py new file mode 100644 index 0000000..6e8e3aa --- /dev/null +++ b/songryeon_core/core/graph_vessel_neo4j.py @@ -0,0 +1,846 @@ +from __future__ import annotations + +import json +import os +from dataclasses import asdict, dataclass +from datetime import datetime +from typing import Any, Callable + +from songryeon_core.core.data_store import DataRecord, DataStore +from songryeon_core.core.graph_memory_store import ( + SONGRYEON_GRAPH_NAMESPACE, + SONGRYEON_VESSEL_DATABASE_NAME, + SONGRYEON_VESSEL_SERVICE_NAME, +) +from songryeon_core.core.graph_vessel_adapter import ( + GRAPH_VESSEL_WRITE_PLAN_DATA_TYPE, +) +from songryeon_core.core.trace_store import TraceStore + + +GRAPH_VESSEL_NEO4J_WRITE_RESULT_DATA_TYPE = "graph_vessel:neo4j_write_result" +GRAPH_VESSEL_NEO4J_WRITER_GENERATOR = "CODE:GRAPH_VESSEL_NEO4J_WRITER" +GRAPH_VESSEL_NEO4J_WRITE_POLICY_ID = "LOCAL_VESSEL_NEO4J_FIRST_WRITE_V0" +GRAPH_VESSEL_NEO4J_WRITE_RESULT_SCHEMA_NAME = "GraphVesselNeo4jWriteResultFrame" +GRAPH_VESSEL_BASE_LABEL = "VesselRecord" +GRAPH_VESSEL_LEGACY_LABELS = ( + "SongRyeonRecord", + "SongRyeonGraphNode", + "SongRyeonGraphEdgeRecord", + "SongRyeonSupportRecord", + "SongRyeonSourcePayload", +) +GRAPH_VESSEL_LEGACY_RELATIONSHIP_TYPE = "SONGRYEON_GRAPH_EDGE" +GRAPH_VESSEL_NEO4J_WRITE_STATUSES = { + "written", + "adapter_unavailable", + "blocked_plan_not_ready", + "blocked_external_status_not_not_run", + "blocked_integrity_failed", + "blocked_target_adapter_mismatch", + "write_failed", +} + +Neo4jDriverFactory = Callable[..., Any] + + +@dataclass(frozen=True) +class GraphVesselNeo4jConfig: + uri: str + user: str + password: str | None + database: str + allow_no_auth: bool = False + + +@dataclass(frozen=True) +class GraphVesselNeo4jWriteResultFrame: + result_id: str + plan_id: str + created_at: str + policy_id: str + target_adapter_name: str + vessel_database_name: str + graph_namespace: str + write_status: str + failure_type: str | None + failure_reason: str | None + external_write_status: str + attempted_operation_count: int + written_operation_count: int + operation_count_by_kind: dict[str, int] + neo4j_record_node_count: int | None + neo4j_graph_edge_count: int | None + source_data_ids: list[str] + source_trace_ids: list[str] + generated_by: str = GRAPH_VESSEL_NEO4J_WRITER_GENERATOR + info_class: str = "absolute" + semantic_judgement_status: str = "not_run" + schema_name: str = GRAPH_VESSEL_NEO4J_WRITE_RESULT_SCHEMA_NAME + + +@dataclass(frozen=True) +class RecordedGraphVesselNeo4jWriteResult: + result: GraphVesselNeo4jWriteResultFrame + trace_event_id: str + created_data_ids: list[str] + existing_data_ids: list[str] + + +def graph_vessel_neo4j_write_result_id(plan_id: str) -> str: + if not plan_id: + raise ValueError("plan_id must not be empty") + return f"graph_vessel:neo4j_write_result:{_stable_suffix(plan_id)}" + + +def graph_vessel_neo4j_config_from_env( + *, + uri: str | None = None, + user: str | None = None, + password: str | None = None, + database: str | None = None, + allow_no_auth: bool = False, +) -> GraphVesselNeo4jConfig: + return GraphVesselNeo4jConfig( + uri=uri + or os.environ.get("SONGRYEON_NEO4J_URI") + or os.environ.get("NEO4J_URI") + or "bolt://localhost:7687", + user=user + or os.environ.get("SONGRYEON_NEO4J_USER") + or os.environ.get("NEO4J_USER") + or "neo4j", + password=( + password + if password is not None + else os.environ.get("SONGRYEON_NEO4J_PASSWORD") + or os.environ.get("NEO4J_PASSWORD") + ), + database=database + or os.environ.get("SONGRYEON_NEO4J_DATABASE") + or os.environ.get("NEO4J_DATABASE") + or SONGRYEON_VESSEL_DATABASE_NAME, + allow_no_auth=allow_no_auth, + ) + + +def write_graph_vessel_plan_to_neo4j( + *, + data_store: DataStore, + plan_id: str, + config: GraphVesselNeo4jConfig, + created_at: str | None = None, + driver_factory: Neo4jDriverFactory | None = None, +) -> GraphVesselNeo4jWriteResultFrame: + plan_record = data_store.require_record(plan_id) + if plan_record.data_type != GRAPH_VESSEL_WRITE_PLAN_DATA_TYPE: + raise ValueError(f"expected graph vessel write plan: {plan_id}") + if not isinstance(plan_record.payload, dict): + raise TypeError("graph vessel write plan payload must be dict") + + plan = plan_record.payload + timestamp = created_at or _now_iso() + source_trace_ids = _string_list(plan.get("source_trace_ids")) + result_id = graph_vessel_neo4j_write_result_id(plan_id) + base = _ResultBase( + result_id=result_id, + plan_id=plan_id, + created_at=timestamp, + target_adapter_name=_optional_str(plan.get("target_adapter_name")) + or SONGRYEON_VESSEL_SERVICE_NAME, + vessel_database_name=config.database, + graph_namespace=SONGRYEON_GRAPH_NAMESPACE, + source_data_ids=_unique_strings([plan_id, *_string_list(plan.get("source_data_ids"))]), + source_trace_ids=source_trace_ids, + ) + + block = _plan_block_status(plan) + if block is not None: + status, failure_type, failure_reason = block + return _make_result( + base=base, + write_status=status, + failure_type=failure_type, + failure_reason=failure_reason, + attempted_operation_count=0, + written_operation_count=0, + operation_count_by_kind={}, + neo4j_record_node_count=None, + neo4j_graph_edge_count=None, + ) + + if not config.uri or not config.user or not config.database: + return _make_result( + base=base, + write_status="adapter_unavailable", + failure_type="neo4j_config_missing", + failure_reason="Neo4j uri/user/database config is incomplete.", + attempted_operation_count=0, + written_operation_count=0, + operation_count_by_kind={}, + neo4j_record_node_count=None, + neo4j_graph_edge_count=None, + ) + if config.password is None and not config.allow_no_auth: + return _make_result( + base=base, + write_status="adapter_unavailable", + failure_type="neo4j_config_missing", + failure_reason="Neo4j password is not configured. Set SONGRYEON_NEO4J_PASSWORD or pass --password.", + attempted_operation_count=0, + written_operation_count=0, + operation_count_by_kind={}, + neo4j_record_node_count=None, + neo4j_graph_edge_count=None, + ) + + driver_factory = driver_factory or _load_neo4j_driver_factory() + if driver_factory is None: + return _make_result( + base=base, + write_status="adapter_unavailable", + failure_type="neo4j_driver_missing", + failure_reason="Python neo4j package is not installed.", + attempted_operation_count=0, + written_operation_count=0, + operation_count_by_kind={}, + neo4j_record_node_count=None, + neo4j_graph_edge_count=None, + ) + + operations = _operation_dicts(plan.get("operations")) + try: + auth = None if config.allow_no_auth and config.password is None else (config.user, config.password) + driver = driver_factory(config.uri, auth=auth) + try: + with driver.session(database=config.database) as session: + written = 0 + for operation in operations: + session.execute_write( + _execute_write_operation, + data_store, + operation, + SONGRYEON_GRAPH_NAMESPACE, + timestamp, + ) + written += 1 + record_count, edge_count = session.execute_read( + _read_vessel_counts, + SONGRYEON_GRAPH_NAMESPACE, + ) + finally: + close = getattr(driver, "close", None) + if callable(close): + close() + except Exception as exc: + return _make_result( + base=base, + write_status="write_failed", + failure_type="neo4j_write_failed", + failure_reason=f"{type(exc).__name__}: {exc}", + attempted_operation_count=len(operations), + written_operation_count=0, + operation_count_by_kind=_count_operations_by_kind(operations), + neo4j_record_node_count=None, + neo4j_graph_edge_count=None, + ) + + return _make_result( + base=base, + write_status="written", + failure_type=None, + failure_reason=None, + attempted_operation_count=len(operations), + written_operation_count=written, + operation_count_by_kind=_count_operations_by_kind(operations), + neo4j_record_node_count=record_count, + neo4j_graph_edge_count=edge_count, + ) + + +def record_graph_vessel_neo4j_write_result( + *, + trace_store: TraceStore, + data_store: DataStore, + turn_id: str, + plan_id: str, + config: GraphVesselNeo4jConfig, + created_at: str | None = None, + driver_factory: Neo4jDriverFactory | None = None, +) -> RecordedGraphVesselNeo4jWriteResult: + result = write_graph_vessel_plan_to_neo4j( + data_store=data_store, + plan_id=plan_id, + config=config, + created_at=created_at, + driver_factory=driver_factory, + ) + event = trace_store.create_event( + turn_id=turn_id, + actor="graph_vessel_neo4j_writer", + event_type="node_output", + timestamp=result.created_at, + input_ref=result.source_trace_ids, + output_ref=[result.result_id], + schema_status="passed", + ) + created_data_ids: list[str] = [] + existing_data_ids: list[str] = [] + _record_payload_if_missing( + data_store=data_store, + data_id=result.result_id, + data_type=GRAPH_VESSEL_NEO4J_WRITE_RESULT_DATA_TYPE, + payload=asdict(result), + created_at=event.timestamp, + source_trace_id=event.event_id, + created_data_ids=created_data_ids, + existing_data_ids=existing_data_ids, + ) + return RecordedGraphVesselNeo4jWriteResult( + result=result, + trace_event_id=event.event_id, + created_data_ids=created_data_ids, + existing_data_ids=existing_data_ids, + ) + + +@dataclass(frozen=True) +class _ResultBase: + result_id: str + plan_id: str + created_at: str + target_adapter_name: str + vessel_database_name: str + graph_namespace: str + source_data_ids: list[str] + source_trace_ids: list[str] + + +def _make_result( + *, + base: _ResultBase, + write_status: str, + failure_type: str | None, + failure_reason: str | None, + attempted_operation_count: int, + written_operation_count: int, + operation_count_by_kind: dict[str, int], + neo4j_record_node_count: int | None, + neo4j_graph_edge_count: int | None, +) -> GraphVesselNeo4jWriteResultFrame: + result = GraphVesselNeo4jWriteResultFrame( + result_id=base.result_id, + plan_id=base.plan_id, + created_at=base.created_at, + policy_id=GRAPH_VESSEL_NEO4J_WRITE_POLICY_ID, + target_adapter_name=base.target_adapter_name, + vessel_database_name=base.vessel_database_name, + graph_namespace=base.graph_namespace, + write_status=write_status, + failure_type=failure_type, + failure_reason=failure_reason, + external_write_status="written" if write_status == "written" else "not_run", + attempted_operation_count=attempted_operation_count, + written_operation_count=written_operation_count, + operation_count_by_kind=operation_count_by_kind, + neo4j_record_node_count=neo4j_record_node_count, + neo4j_graph_edge_count=neo4j_graph_edge_count, + source_data_ids=base.source_data_ids, + source_trace_ids=base.source_trace_ids, + ) + _validate_write_result(result) + return result + + +def _plan_block_status( + plan: dict[str, object], +) -> tuple[str, str, str] | None: + target_adapter_name = _optional_str(plan.get("target_adapter_name")) + if target_adapter_name != SONGRYEON_VESSEL_SERVICE_NAME: + return ( + "blocked_target_adapter_mismatch", + "target_adapter_mismatch", + f"Expected {SONGRYEON_VESSEL_SERVICE_NAME}, got {target_adapter_name}.", + ) + if _optional_str(plan.get("plan_status")) != "ready_to_write": + return ( + "blocked_plan_not_ready", + "plan_status_not_ready", + "GraphVesselWritePlan.plan_status is not ready_to_write.", + ) + if _optional_str(plan.get("external_write_status")) != "not_run": + return ( + "blocked_external_status_not_not_run", + "external_write_status_not_not_run", + "GraphVesselWritePlan.external_write_status is not not_run.", + ) + if _optional_str(plan.get("graph_integrity_status")) != "passed": + return ( + "blocked_integrity_failed", + "graph_integrity_failed", + "GraphVesselWritePlan.graph_integrity_status is not passed.", + ) + return None + + +def _execute_write_operation( + tx: object, + data_store: DataStore, + operation: dict[str, object], + graph_namespace: str, + written_at: str, +) -> None: + operation_kind = _require_str(operation, "operation_kind") + source_data_id = _require_str(operation, "source_data_id") + record = data_store.require_record(source_data_id) + if operation_kind == "upsert_source_payload": + _upsert_record_node( + tx, + record=record, + graph_namespace=graph_namespace, + written_at=written_at, + labels=["GraphMemorySource"], + ) + elif operation_kind == "upsert_graph_node": + _upsert_record_node( + tx, + record=record, + graph_namespace=graph_namespace, + written_at=written_at, + labels=_graph_node_labels(record), + ) + elif operation_kind == "upsert_graph_edge": + _upsert_record_node( + tx, + record=record, + graph_namespace=graph_namespace, + written_at=written_at, + labels=["GraphMemoryEdgeRecord"], + ) + _upsert_graph_relationship( + tx, + record=record, + graph_namespace=graph_namespace, + written_at=written_at, + ) + elif operation_kind == "upsert_support_record": + _upsert_record_node( + tx, + record=record, + graph_namespace=graph_namespace, + written_at=written_at, + labels=_support_record_labels(record), + ) + else: + raise ValueError(f"unknown graph vessel operation_kind: {operation_kind}") + + +def _upsert_record_node( + tx: object, + *, + record: DataRecord, + graph_namespace: str, + written_at: str, + labels: list[str], +) -> None: + label_clause = _label_clause([GRAPH_VESSEL_BASE_LABEL, *labels]) + tx.run( + f""" + MERGE (n {{data_id: $data_id, graph_namespace: $graph_namespace}}) + {_legacy_label_remove_clause("n")} + SET n += $properties + SET n{label_clause} + """, + data_id=record.data_id, + graph_namespace=graph_namespace, + properties=_record_properties(record, graph_namespace=graph_namespace, written_at=written_at), + ) + + +def _upsert_graph_relationship( + tx: object, + *, + record: DataRecord, + graph_namespace: str, + written_at: str, +) -> None: + if not isinstance(record.payload, dict): + raise TypeError(f"graph edge payload must be dict: {record.data_id}") + edge_id = _require_str(record.payload, "edge_id") + from_node_id = _require_str(record.payload, "from_node_id") + to_node_id = _require_str(record.payload, "to_node_id") + edge_kind = _require_str(record.payload, "edge_kind") + relationship_type = _relationship_type(record.payload) + tx.run( + f""" + OPTIONAL MATCH ()-[old:{GRAPH_VESSEL_LEGACY_RELATIONSHIP_TYPE} {{edge_id: $edge_id, graph_namespace: $graph_namespace}}]->() + WITH collect(old) AS old_edges + FOREACH (old_edge IN old_edges | DELETE old_edge) + MERGE (from_node {{data_id: $from_node_id, graph_namespace: $graph_namespace}}) + MERGE (to_node {{data_id: $to_node_id, graph_namespace: $graph_namespace}}) + SET from_node:{GRAPH_VESSEL_BASE_LABEL} + SET to_node:{GRAPH_VESSEL_BASE_LABEL} + MERGE (from_node)-[r:{relationship_type} {{edge_id: $edge_id, graph_namespace: $graph_namespace}}]->(to_node) + SET r += $properties + """, + from_node_id=from_node_id, + to_node_id=to_node_id, + edge_id=edge_id, + graph_namespace=graph_namespace, + properties={ + "edge_id": edge_id, + "edge_kind": edge_kind, + "display_relationship_type": relationship_type, + "source_data_id": record.data_id, + "source_data_type": record.data_type, + "payload_json": _json(record.payload), + "written_at": written_at, + "generated_by": _payload_str(record.payload, "generated_by"), + "info_class": _payload_str(record.payload, "info_class"), + "semantic_judgement_status": _payload_str( + record.payload, + "semantic_judgement_status", + ), + }, + ) + + +def _read_vessel_counts(tx: object, graph_namespace: str) -> tuple[int, int]: + node_result = tx.run( + """ + MATCH (n:VesselRecord {graph_namespace: $graph_namespace}) + RETURN count(n) AS count + """, + graph_namespace=graph_namespace, + ) + edge_result = tx.run( + """ + MATCH ()-[r {graph_namespace: $graph_namespace}]->() + WHERE r.display_relationship_type IS NOT NULL + RETURN count(r) AS count + """, + graph_namespace=graph_namespace, + ) + return _single_count(node_result), _single_count(edge_result) + + +def _record_properties( + record: DataRecord, + *, + graph_namespace: str, + written_at: str, +) -> dict[str, object]: + payload = record.payload if isinstance(record.payload, dict) else {} + return _drop_none( + { + "data_id": record.data_id, + "graph_namespace": graph_namespace, + "source_data_type": record.data_type, + "exists": record.exists, + "created_at": record.created_at, + "source_trace_id": record.source_trace_id, + "payload_json": _json(record.payload), + "payload_char_count": len(_json(record.payload)), + "written_at": written_at, + "display_name": _display_name(record), + "display_kind": _display_kind(record), + "display_label": _display_label(record), + "generated_by": _payload_str(payload, "generated_by"), + "info_class": _payload_str(payload, "info_class"), + "semantic_judgement_status": _payload_str(payload, "semantic_judgement_status"), + "node_kind": _payload_str(payload, "node_kind"), + "data_kind": _payload_str(payload, "data_kind"), + "edge_kind": _payload_str(payload, "edge_kind"), + "source_bundle_kind": _payload_str(payload, "source_bundle_kind"), + "source_data_ids_json": _json(payload.get("source_data_ids", [])), + "source_trace_ids_json": _json(payload.get("source_trace_ids", [])), + "source_graph_node_ids_json": _json(payload.get("source_graph_node_ids", [])), + } + ) + + +def _validate_write_result(result: GraphVesselNeo4jWriteResultFrame) -> None: + if result.generated_by != GRAPH_VESSEL_NEO4J_WRITER_GENERATOR: + raise ValueError("GraphVesselNeo4jWriteResultFrame.generated_by must be writer") + if result.info_class != "absolute": + raise ValueError("GraphVesselNeo4jWriteResultFrame.info_class must be absolute") + if result.semantic_judgement_status != "not_run": + raise ValueError("GraphVesselNeo4jWriteResultFrame.semantic_judgement_status must be not_run") + if result.write_status not in GRAPH_VESSEL_NEO4J_WRITE_STATUSES: + raise ValueError("GraphVesselNeo4jWriteResultFrame.write_status is invalid") + if result.external_write_status not in {"not_run", "written"}: + raise ValueError("GraphVesselNeo4jWriteResultFrame.external_write_status is invalid") + if result.write_status == "written" and result.external_write_status != "written": + raise ValueError("written result must mark external_write_status=written") + if result.write_status != "written" and result.external_write_status != "not_run": + raise ValueError("non-written result must mark external_write_status=not_run") + + +def _graph_node_labels(record: DataRecord) -> list[str]: + payload = record.payload if isinstance(record.payload, dict) else {} + node_kind = _payload_str(payload, "node_kind") + label_by_node_kind = { + "core_ego": "CoreEgo", + "time_axis": "TimeAxis", + "time_bundle": "TimeBundle", + "raw_capsule": "RawCapsule", + "summary": "SummaryGraphNode", + "raw_source": "RawSource", + "source_kind_bundle": "SourceKindBundle", + "source_ingest_time_bundle": "SourceIngestBundle", + } + return ["GraphMemoryNode", label_by_node_kind.get(node_kind, "GraphMemoryNode")] + + +def _support_record_labels(record: DataRecord) -> list[str]: + if record.data_type == "graph_memory:snapshot": + return ["GraphMemorySnapshot"] + if record.data_type == "graph_memory:rloop_guide_packet": + return ["RLoopGuide"] + return ["GraphMemoryIndex"] + + +def _relationship_type(payload: dict[str, object]) -> str: + edge_kind = _require_str(payload, "edge_kind") + from_node_id = _require_str(payload, "from_node_id") + to_node_id = _require_str(payload, "to_node_id") + if edge_kind == "NEXT": + return "NEXT" + if edge_kind == "CHILD_OF_TIME_AXIS": + return "HAS_BUNDLE" + if edge_kind == "CONTAINS": + if from_node_id == "graph:core_ego:root" and to_node_id == "graph:axis:time": + return "HAS_AXIS" + if from_node_id.startswith("graph:time_bundle:") and to_node_id.startswith("graph:raw_capsule:"): + return "CONTAINS_MEMORY" + if from_node_id.startswith("graph:source_ingest_time_bundle:") and to_node_id.startswith("graph:source_kind_bundle:"): + return "HAS_SOURCE_KIND" + if from_node_id.startswith("graph:source_kind_bundle:") and to_node_id.startswith("graph:raw_source:"): + return "CONTAINS_SOURCE" + return "CONTAINS" + return _safe_relationship_type(edge_kind) + + +def _display_name(record: DataRecord) -> str: + payload = record.payload if isinstance(record.payload, dict) else {} + node_kind = _payload_str(payload, "node_kind") + if node_kind == "core_ego": + return "CoreEgo" + if node_kind == "time_axis": + return "Time Axis" + if node_kind == "time_bundle": + return "Time Bundle" + if node_kind == "raw_capsule": + return f"Raw Capsule: {_payload_str(payload, 'source_turn_id') or record.data_id}" + if node_kind == "summary": + return f"Summary: {_payload_str(payload, 'target_graph_node_id') or record.data_id}" + if node_kind == "raw_source": + return f"Raw Source: {_payload_str(payload, 'data_kind') or record.data_id}" + if record.data_type == "graph_memory:snapshot": + return "Graph Memory Snapshot" + if record.data_type == "graph_memory:rloop_guide_packet": + return "R Loop Guide" + if record.data_type == "graph_memory:core_ego_time_axis_frame": + return "CoreEgo Time Axis Index" + if record.data_type == "graph_source:source_version_lineage_frame": + return "Source Version Lineage" + if record.data_type == "graph_source:source_observation_ledger_frame": + return "Source Observation Ledger" + if record.data_type == "graph_source:summary_invalidation_ledger_frame": + return "Summary Invalidation Ledger" + if record.data_type.startswith("graph_source:"): + return f"Graph Source: {record.data_type}" + if record.data_type.startswith("graph_memory:edge:"): + return f"Graph Edge Record: {_payload_str(payload, 'edge_kind') or record.data_id}" + return record.data_id + + +def _display_kind(record: DataRecord) -> str: + payload = record.payload if isinstance(record.payload, dict) else {} + return ( + _payload_str(payload, "node_kind") + or _payload_str(payload, "edge_kind") + or record.data_type + ) + + +def _display_label(record: DataRecord) -> str: + labels = ( + _graph_node_labels(record) + if record.data_type.startswith("graph_memory:node:") + else _support_record_labels(record) + if record.data_type in { + "graph_memory:snapshot", + "graph_memory:rloop_guide_packet", + "graph_memory:core_ego_time_axis_frame", + "graph_source:source_kind_ingest_frame", + "graph_source:songryeon_core_source_manifest_frame", + "graph_source:source_version_lineage_frame", + "graph_source:source_observation_ledger_frame", + "graph_source:summary_invalidation_ledger_frame", + } + else ["GraphMemorySource"] + if record.data_type.startswith("graph_source:") + else ["GraphMemoryEdgeRecord"] + if record.data_type.startswith("graph_memory:edge:") + else [GRAPH_VESSEL_BASE_LABEL] + ) + return labels[-1] + + +def _label_clause(labels: list[str]) -> str: + return "".join(f":{_safe_label(label)}" for label in _unique_strings(labels)) + + +def _legacy_label_remove_clause(variable_name: str) -> str: + labels = "".join(f":{label}" for label in GRAPH_VESSEL_LEGACY_LABELS) + return f"REMOVE {variable_name}{labels}" + + +def _safe_label(value: str) -> str: + if not value or not value.replace("_", "").isalnum() or not value[0].isalpha(): + raise ValueError(f"unsafe Neo4j label: {value}") + return value + + +def _safe_relationship_type(value: str) -> str: + candidate = "".join(char if char.isalnum() else "_" for char in value.upper()) + while "__" in candidate: + candidate = candidate.replace("__", "_") + candidate = candidate.strip("_") + if not candidate or not candidate[0].isalpha(): + raise ValueError(f"unsafe Neo4j relationship type: {value}") + return candidate + + +def _record_payload_if_missing( + *, + data_store: DataStore, + data_id: str, + data_type: str, + payload: dict[str, object], + created_at: str, + source_trace_id: str, + created_data_ids: list[str], + existing_data_ids: list[str], +) -> None: + existing = data_store.get_record(data_id) + if existing is not None: + if existing.data_type != data_type: + raise ValueError(f"Neo4j write result data_id collision with different type: {data_id}") + if existing.payload != payload: + raise ValueError(f"Neo4j write result data_id collision with different payload: {data_id}") + existing_data_ids.append(data_id) + return + data_store.create_record( + data_id=data_id, + data_type=data_type, + exists=True, + created_at=created_at, + source_trace_id=source_trace_id, + payload=payload, + ) + created_data_ids.append(data_id) + + +def _load_neo4j_driver_factory() -> Neo4jDriverFactory | None: + try: + from neo4j import GraphDatabase + except ImportError: + return None + return GraphDatabase.driver + + +def _operation_dicts(value: object) -> list[dict[str, object]]: + if not isinstance(value, list): + return [] + return [item for item in value if isinstance(item, dict)] + + +def _count_operations_by_kind(operations: list[dict[str, object]]) -> dict[str, int]: + counts: dict[str, int] = {} + for operation in operations: + operation_kind = _optional_str(operation.get("operation_kind")) + if operation_kind: + counts[operation_kind] = counts.get(operation_kind, 0) + 1 + return counts + + +def _single_count(result: object) -> int: + single = getattr(result, "single", None) + if callable(single): + record = single() + if record is None: + return 0 + try: + return int(record["count"]) + except (KeyError, TypeError, ValueError): + return 0 + try: + first = next(iter(result)) + return int(first["count"]) + except (KeyError, TypeError, ValueError, StopIteration): + return 0 + + +def _require_str(payload: dict[str, object], field_name: str) -> str: + value = payload.get(field_name) + if not isinstance(value, str) or not value: + raise ValueError(f"{field_name} must be a non-empty string") + return value + + +def _optional_str(value: object) -> str | None: + return value if isinstance(value, str) and value else None + + +def _payload_str(payload: dict[str, object], field_name: str) -> str | None: + return _optional_str(payload.get(field_name)) + + +def _string_list(value: object) -> list[str]: + if not isinstance(value, list): + return [] + return [item for item in value if isinstance(item, str) and item] + + +def _unique_strings(values: list[str | None]) -> list[str]: + seen: set[str] = set() + result: list[str] = [] + for value in values: + if not value or value in seen: + continue + seen.add(value) + result.append(value) + return result + + +def _json(value: object) -> str: + return json.dumps(value, ensure_ascii=False, sort_keys=True) + + +def _drop_none(payload: dict[str, object | None]) -> dict[str, object]: + return {key: value for key, value in payload.items() if value is not None} + + +def _stable_suffix(value: str) -> str: + return value.replace(":", "_").replace("/", "_").replace("\\", "_") + + +def _now_iso() -> str: + return datetime.now().isoformat(timespec="seconds") + + +__all__ = [ + "GRAPH_VESSEL_NEO4J_WRITE_RESULT_DATA_TYPE", + "GRAPH_VESSEL_NEO4J_WRITE_STATUSES", + "GRAPH_VESSEL_NEO4J_WRITER_GENERATOR", + "GRAPH_VESSEL_NEO4J_WRITE_POLICY_ID", + "GraphVesselNeo4jConfig", + "GraphVesselNeo4jWriteResultFrame", + "RecordedGraphVesselNeo4jWriteResult", + "graph_vessel_neo4j_config_from_env", + "graph_vessel_neo4j_write_result_id", + "record_graph_vessel_neo4j_write_result", + "write_graph_vessel_plan_to_neo4j", +] diff --git a/songryeon_core/core/graph_vessel_readback.py b/songryeon_core/core/graph_vessel_readback.py new file mode 100644 index 0000000..fe78541 --- /dev/null +++ b/songryeon_core/core/graph_vessel_readback.py @@ -0,0 +1,505 @@ +from __future__ import annotations + +from dataclasses import asdict, dataclass +from datetime import datetime +from typing import Any, Callable + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.graph_memory_store import ( + SONGRYEON_GRAPH_NAMESPACE, + SONGRYEON_VESSEL_SERVICE_NAME, +) +from songryeon_core.core.graph_vessel_neo4j import ( + GRAPH_VESSEL_BASE_LABEL, + GraphVesselNeo4jConfig, +) +from songryeon_core.core.trace_store import TraceStore + + +GRAPH_VESSEL_NEO4J_READBACK_RESULT_DATA_TYPE = "graph_vessel:neo4j_readback_result" +GRAPH_VESSEL_NEO4J_READBACK_GENERATOR = "CODE:GRAPH_VESSEL_NEO4J_READBACK_VERIFIER" +GRAPH_VESSEL_NEO4J_READBACK_POLICY_ID = "VESSEL_READBACK_VERIFICATION_V0" +GRAPH_VESSEL_NEO4J_READBACK_SCHEMA_NAME = "GraphVesselNeo4jReadbackResultFrame" +GRAPH_VESSEL_NEO4J_READBACK_STATUSES = { + "passed", + "failed", + "adapter_unavailable", + "read_failed", +} + +Neo4jDriverFactory = Callable[..., Any] + + +@dataclass(frozen=True) +class GraphVesselNeo4jReadbackResultFrame: + result_id: str + created_at: str + policy_id: str + target_adapter_name: str + vessel_database_name: str + graph_namespace: str + readback_status: str + failure_type: str | None + failure_reason: str | None + core_path_exists: bool + core_path_count: int | None + vessel_record_count: int | None + vessel_relationship_count: int | None + core_ego_count: int | None + time_axis_count: int | None + time_bundle_count: int | None + raw_capsule_count: int | None + has_axis_count: int | None + has_bundle_count: int | None + contains_memory_count: int | None + required_property_missing_count: int | None + required_property_missing_samples: list[str] + source_data_ids: list[str] + source_trace_ids: list[str] + generated_by: str = GRAPH_VESSEL_NEO4J_READBACK_GENERATOR + info_class: str = "absolute" + semantic_judgement_status: str = "not_run" + schema_name: str = GRAPH_VESSEL_NEO4J_READBACK_SCHEMA_NAME + + +@dataclass(frozen=True) +class RecordedGraphVesselNeo4jReadbackResult: + result: GraphVesselNeo4jReadbackResultFrame + trace_event_id: str + created_data_ids: list[str] + existing_data_ids: list[str] + + +def graph_vessel_neo4j_readback_result_id(batch_id: str) -> str: + if not batch_id: + raise ValueError("batch_id must not be empty") + return f"graph_vessel:neo4j_readback_result:{_stable_suffix(batch_id)}" + + +def readback_graph_vessel_from_neo4j( + *, + config: GraphVesselNeo4jConfig, + batch_id: str = "manual_vessel_readback", + created_at: str | None = None, + driver_factory: Neo4jDriverFactory | None = None, +) -> GraphVesselNeo4jReadbackResultFrame: + timestamp = created_at or _now_iso() + base = _ReadbackBase( + result_id=graph_vessel_neo4j_readback_result_id(batch_id), + created_at=timestamp, + target_adapter_name=SONGRYEON_VESSEL_SERVICE_NAME, + vessel_database_name=config.database, + graph_namespace=SONGRYEON_GRAPH_NAMESPACE, + ) + + config_failure = _config_failure(config) + if config_failure is not None: + failure_type, failure_reason = config_failure + return _make_result( + base=base, + readback_status="adapter_unavailable", + failure_type=failure_type, + failure_reason=failure_reason, + checks=_empty_checks(), + ) + + driver_factory = driver_factory or _load_neo4j_driver_factory() + if driver_factory is None: + return _make_result( + base=base, + readback_status="adapter_unavailable", + failure_type="neo4j_driver_missing", + failure_reason="Python neo4j package is not installed.", + checks=_empty_checks(), + ) + + try: + auth = None if config.allow_no_auth and config.password is None else (config.user, config.password) + driver = driver_factory(config.uri, auth=auth) + try: + with driver.session(database=config.database) as session: + checks = session.execute_read(_execute_readback_checks, SONGRYEON_GRAPH_NAMESPACE) + finally: + close = getattr(driver, "close", None) + if callable(close): + close() + except Exception as exc: + return _make_result( + base=base, + readback_status="read_failed", + failure_type="neo4j_read_failed", + failure_reason=f"{type(exc).__name__}: {exc}", + checks=_empty_checks(), + ) + + failure_reason = _readback_failure_reason(checks) + return _make_result( + base=base, + readback_status="passed" if failure_reason is None else "failed", + failure_type=None if failure_reason is None else "readback_check_failed", + failure_reason=failure_reason, + checks=checks, + ) + + +def record_graph_vessel_neo4j_readback_result( + *, + trace_store: TraceStore, + data_store: DataStore, + turn_id: str, + config: GraphVesselNeo4jConfig, + batch_id: str = "manual_vessel_readback", + created_at: str | None = None, + driver_factory: Neo4jDriverFactory | None = None, +) -> RecordedGraphVesselNeo4jReadbackResult: + result = readback_graph_vessel_from_neo4j( + config=config, + batch_id=batch_id, + created_at=created_at, + driver_factory=driver_factory, + ) + event = trace_store.create_event( + turn_id=turn_id, + actor="graph_vessel_neo4j_readback", + event_type="node_output", + timestamp=result.created_at, + input_ref=[], + output_ref=[result.result_id], + schema_status="passed", + ) + created_data_ids: list[str] = [] + existing_data_ids: list[str] = [] + _record_payload_if_missing( + data_store=data_store, + data_id=result.result_id, + data_type=GRAPH_VESSEL_NEO4J_READBACK_RESULT_DATA_TYPE, + payload=asdict(result), + created_at=event.timestamp, + source_trace_id=event.event_id, + created_data_ids=created_data_ids, + existing_data_ids=existing_data_ids, + ) + return RecordedGraphVesselNeo4jReadbackResult( + result=result, + trace_event_id=event.event_id, + created_data_ids=created_data_ids, + existing_data_ids=existing_data_ids, + ) + + +@dataclass(frozen=True) +class _ReadbackBase: + result_id: str + created_at: str + target_adapter_name: str + vessel_database_name: str + graph_namespace: str + + +@dataclass(frozen=True) +class _ReadbackChecks: + core_path_count: int | None + vessel_record_count: int | None + vessel_relationship_count: int | None + core_ego_count: int | None + time_axis_count: int | None + time_bundle_count: int | None + raw_capsule_count: int | None + has_axis_count: int | None + has_bundle_count: int | None + contains_memory_count: int | None + required_property_missing_count: int | None + required_property_missing_samples: list[str] + + +def _make_result( + *, + base: _ReadbackBase, + readback_status: str, + failure_type: str | None, + failure_reason: str | None, + checks: _ReadbackChecks, +) -> GraphVesselNeo4jReadbackResultFrame: + result = GraphVesselNeo4jReadbackResultFrame( + result_id=base.result_id, + created_at=base.created_at, + policy_id=GRAPH_VESSEL_NEO4J_READBACK_POLICY_ID, + target_adapter_name=base.target_adapter_name, + vessel_database_name=base.vessel_database_name, + graph_namespace=base.graph_namespace, + readback_status=readback_status, + failure_type=failure_type, + failure_reason=failure_reason, + core_path_exists=(checks.core_path_count or 0) > 0, + core_path_count=checks.core_path_count, + vessel_record_count=checks.vessel_record_count, + vessel_relationship_count=checks.vessel_relationship_count, + core_ego_count=checks.core_ego_count, + time_axis_count=checks.time_axis_count, + time_bundle_count=checks.time_bundle_count, + raw_capsule_count=checks.raw_capsule_count, + has_axis_count=checks.has_axis_count, + has_bundle_count=checks.has_bundle_count, + contains_memory_count=checks.contains_memory_count, + required_property_missing_count=checks.required_property_missing_count, + required_property_missing_samples=checks.required_property_missing_samples, + source_data_ids=[], + source_trace_ids=[], + ) + _validate_readback_result(result) + return result + + +def _execute_readback_checks(tx: object, graph_namespace: str) -> _ReadbackChecks: + return _ReadbackChecks( + core_path_count=_count_core_path(tx, graph_namespace), + vessel_record_count=_count_label(tx, graph_namespace, GRAPH_VESSEL_BASE_LABEL), + vessel_relationship_count=_count_vessel_relationships(tx, graph_namespace), + core_ego_count=_count_label(tx, graph_namespace, "CoreEgo"), + time_axis_count=_count_label(tx, graph_namespace, "TimeAxis"), + time_bundle_count=_count_label(tx, graph_namespace, "TimeBundle"), + raw_capsule_count=_count_label(tx, graph_namespace, "RawCapsule"), + has_axis_count=_count_relationship(tx, graph_namespace, "HAS_AXIS"), + has_bundle_count=_count_relationship(tx, graph_namespace, "HAS_BUNDLE"), + contains_memory_count=_count_relationship(tx, graph_namespace, "CONTAINS_MEMORY"), + required_property_missing_count=_required_property_missing_count(tx, graph_namespace), + required_property_missing_samples=_required_property_missing_samples(tx, graph_namespace), + ) + + +def _count_core_path(tx: object, graph_namespace: str) -> int: + result = tx.run( + """ + MATCH (:CoreEgo {graph_namespace: $graph_namespace}) + -[:HAS_AXIS {graph_namespace: $graph_namespace}]-> + (:TimeAxis {graph_namespace: $graph_namespace}) + -[:HAS_BUNDLE {graph_namespace: $graph_namespace}]-> + (:TimeBundle {graph_namespace: $graph_namespace}) + -[:CONTAINS_MEMORY {graph_namespace: $graph_namespace}]-> + (:RawCapsule {graph_namespace: $graph_namespace}) + RETURN count(*) AS count + """, + graph_namespace=graph_namespace, + ) + return _single_count(result) + + +def _count_label(tx: object, graph_namespace: str, label: str) -> int: + _validate_cypher_token(label) + result = tx.run( + f""" + MATCH (n:{label} {{graph_namespace: $graph_namespace}}) + RETURN count(n) AS count + """, + graph_namespace=graph_namespace, + ) + return _single_count(result) + + +def _count_relationship(tx: object, graph_namespace: str, relationship_type: str) -> int: + _validate_cypher_token(relationship_type) + result = tx.run( + f""" + MATCH ()-[r:{relationship_type} {{graph_namespace: $graph_namespace}}]->() + RETURN count(r) AS count + """, + graph_namespace=graph_namespace, + ) + return _single_count(result) + + +def _count_vessel_relationships(tx: object, graph_namespace: str) -> int: + result = tx.run( + """ + MATCH ()-[r {graph_namespace: $graph_namespace}]->() + WHERE r.display_relationship_type IS NOT NULL + RETURN count(r) AS count + """, + graph_namespace=graph_namespace, + ) + return _single_count(result) + + +def _required_property_missing_count(tx: object, graph_namespace: str) -> int: + result = tx.run( + """ + MATCH (n:VesselRecord {graph_namespace: $graph_namespace}) + WHERE n.data_id IS NULL + OR n.generated_by IS NULL + OR n.info_class IS NULL + OR n.semantic_judgement_status IS NULL + OR n.payload_json IS NULL + RETURN count(n) AS count + """, + graph_namespace=graph_namespace, + ) + return _single_count(result) + + +def _required_property_missing_samples(tx: object, graph_namespace: str) -> list[str]: + result = tx.run( + """ + MATCH (n:VesselRecord {graph_namespace: $graph_namespace}) + WHERE n.data_id IS NULL + OR n.generated_by IS NULL + OR n.info_class IS NULL + OR n.semantic_judgement_status IS NULL + OR n.payload_json IS NULL + RETURN collect(coalesce(n.data_id, ""))[0..5] AS samples + """, + graph_namespace=graph_namespace, + ) + single = getattr(result, "single", None) + if callable(single): + record = single() or {} + samples = record.get("samples", []) + if isinstance(samples, list): + return [item for item in samples if isinstance(item, str)] + return [] + + +def _readback_failure_reason(checks: _ReadbackChecks) -> str | None: + if (checks.core_path_count or 0) < 1: + return "CoreEgo -> TimeAxis -> TimeBundle -> RawCapsule path was not found." + required_positive_counts = { + "vessel_record_count": checks.vessel_record_count, + "vessel_relationship_count": checks.vessel_relationship_count, + "core_ego_count": checks.core_ego_count, + "time_axis_count": checks.time_axis_count, + "time_bundle_count": checks.time_bundle_count, + "raw_capsule_count": checks.raw_capsule_count, + "has_axis_count": checks.has_axis_count, + "has_bundle_count": checks.has_bundle_count, + "contains_memory_count": checks.contains_memory_count, + } + for field_name, count in required_positive_counts.items(): + if (count or 0) < 1: + return f"{field_name} is zero." + if (checks.required_property_missing_count or 0) > 0: + return "Some VesselRecord nodes are missing required provenance properties." + return None + + +def _config_failure(config: GraphVesselNeo4jConfig) -> tuple[str, str] | None: + if not config.uri or not config.user or not config.database: + return ( + "neo4j_config_missing", + "Neo4j uri/user/database config is incomplete.", + ) + if config.password is None and not config.allow_no_auth: + return ( + "neo4j_config_missing", + "Neo4j password is not configured. Set SONGRYEON_NEO4J_PASSWORD or pass --password.", + ) + return None + + +def _empty_checks() -> _ReadbackChecks: + return _ReadbackChecks( + core_path_count=None, + vessel_record_count=None, + vessel_relationship_count=None, + core_ego_count=None, + time_axis_count=None, + time_bundle_count=None, + raw_capsule_count=None, + has_axis_count=None, + has_bundle_count=None, + contains_memory_count=None, + required_property_missing_count=None, + required_property_missing_samples=[], + ) + + +def _validate_readback_result(result: GraphVesselNeo4jReadbackResultFrame) -> None: + if result.generated_by != GRAPH_VESSEL_NEO4J_READBACK_GENERATOR: + raise ValueError("GraphVesselNeo4jReadbackResultFrame.generated_by must be readback verifier") + if result.info_class != "absolute": + raise ValueError("GraphVesselNeo4jReadbackResultFrame.info_class must be absolute") + if result.semantic_judgement_status != "not_run": + raise ValueError("GraphVesselNeo4jReadbackResultFrame.semantic_judgement_status must be not_run") + if result.readback_status not in GRAPH_VESSEL_NEO4J_READBACK_STATUSES: + raise ValueError("GraphVesselNeo4jReadbackResultFrame.readback_status is invalid") + if result.readback_status == "passed" and not result.core_path_exists: + raise ValueError("passed readback must have core_path_exists=True") + + +def _record_payload_if_missing( + *, + data_store: DataStore, + data_id: str, + data_type: str, + payload: dict[str, object], + created_at: str, + source_trace_id: str, + created_data_ids: list[str], + existing_data_ids: list[str], +) -> None: + existing = data_store.get_record(data_id) + if existing is not None: + if existing.data_type != data_type: + raise ValueError(f"Neo4j readback result data_id collision with different type: {data_id}") + if existing.payload != payload: + raise ValueError(f"Neo4j readback result data_id collision with different payload: {data_id}") + existing_data_ids.append(data_id) + return + data_store.create_record( + data_id=data_id, + data_type=data_type, + exists=True, + created_at=created_at, + source_trace_id=source_trace_id, + payload=payload, + ) + created_data_ids.append(data_id) + + +def _load_neo4j_driver_factory() -> Neo4jDriverFactory | None: + try: + from neo4j import GraphDatabase + except ImportError: + return None + return GraphDatabase.driver + + +def _single_count(result: object) -> int: + single = getattr(result, "single", None) + if callable(single): + record = single() + if record is None: + return 0 + try: + return int(record["count"]) + except (KeyError, TypeError, ValueError): + return 0 + try: + first = next(iter(result)) + return int(first["count"]) + except (KeyError, TypeError, ValueError, StopIteration): + return 0 + + +def _validate_cypher_token(value: str) -> None: + if not value or not value.replace("_", "").isalnum() or not value[0].isalpha(): + raise ValueError(f"unsafe Neo4j token: {value}") + + +def _stable_suffix(value: str) -> str: + return value.replace(":", "_").replace("/", "_").replace("\\", "_") + + +def _now_iso() -> str: + return datetime.now().isoformat(timespec="seconds") + + +__all__ = [ + "GRAPH_VESSEL_NEO4J_READBACK_GENERATOR", + "GRAPH_VESSEL_NEO4J_READBACK_POLICY_ID", + "GRAPH_VESSEL_NEO4J_READBACK_RESULT_DATA_TYPE", + "GRAPH_VESSEL_NEO4J_READBACK_STATUSES", + "GraphVesselNeo4jReadbackResultFrame", + "RecordedGraphVesselNeo4jReadbackResult", + "graph_vessel_neo4j_readback_result_id", + "readback_graph_vessel_from_neo4j", + "record_graph_vessel_neo4j_readback_result", +] diff --git a/songryeon_core/core/r_loop_state_machine.py b/songryeon_core/core/r_loop_state_machine.py new file mode 100644 index 0000000..e2cef81 --- /dev/null +++ b/songryeon_core/core/r_loop_state_machine.py @@ -0,0 +1,165 @@ +from __future__ import annotations + +from dataclasses import asdict + +from songryeon_core.core.schemas import ( + R3GraphInspectionFrame, + RLoopBudgetFrame, + RLoopContinuationFrame, + validate_r3_graph_inspection_frame, + validate_r_loop_budget_frame, + validate_r_loop_continuation_frame, +) + + +def decide_r_loop_continuation( + *, + frame_id: str, + r3_inspection: R3GraphInspectionFrame, + budget: RLoopBudgetFrame, + source_trace_ids: list[str] | None = None, +) -> RLoopContinuationFrame: + """Decide R-loop continuation from structured R3 status and numeric budgets only.""" + + if not frame_id: + raise ValueError("frame_id must not be empty") + validate_r3_graph_inspection_frame(r3_inspection) + validate_r_loop_budget_frame(budget) + + remaining_traversal_depth = max( + budget.max_traversal_depth - budget.used_traversal_depth, + 0, + ) + remaining_branch_switches = max( + budget.max_branch_switches - budget.used_branch_switches, + 0, + ) + remaining_node_reads = max(budget.max_node_reads - budget.used_node_reads, 0) + remaining_context_tokens = max( + budget.max_context_tokens - budget.used_context_tokens, + 0, + ) + + continuation_status, reason_code, next_target_node = _decide_status( + r3_inspection=r3_inspection, + remaining_traversal_depth=remaining_traversal_depth, + remaining_branch_switches=remaining_branch_switches, + remaining_node_reads=remaining_node_reads, + remaining_context_tokens=remaining_context_tokens, + ) + + frame = RLoopContinuationFrame( + frame_id=frame_id, + source_r3_inspection_frame_id=r3_inspection.frame_id, + source_budget_frame_id=budget.frame_id, + continuation_status=continuation_status, + continuation_reason_code=reason_code, + next_target_node=next_target_node, + remaining_traversal_depth=remaining_traversal_depth, + remaining_branch_switches=remaining_branch_switches, + remaining_node_reads=remaining_node_reads, + remaining_context_tokens=remaining_context_tokens, + source_data_ids=[ + r3_inspection.frame_id, + budget.frame_id, + *r3_inspection.source_data_ids, + *budget.source_data_ids, + ], + source_trace_ids=_unique_strings( + [ + *(source_trace_ids or []), + *r3_inspection.source_trace_ids, + *budget.source_trace_ids, + ] + ), + ) + frame.source_data_ids = _unique_strings(frame.source_data_ids) + validate_r_loop_continuation_frame(frame) + return frame + + +def r_loop_continuation_payload(frame: RLoopContinuationFrame) -> dict[str, object]: + validate_r_loop_continuation_frame(frame) + return asdict(frame) + + +def _decide_status( + *, + r3_inspection: R3GraphInspectionFrame, + remaining_traversal_depth: int, + remaining_branch_switches: int, + remaining_node_reads: int, + remaining_context_tokens: int, +) -> tuple[str, str, str]: + if ( + r3_inspection.sufficiency_status == "sufficient" + or r3_inspection.recommended_next_action == "stop" + ): + return "stop_sufficient", "CODE_STATUS:r3_sufficient", "return_summary" + + if remaining_node_reads <= 0 or remaining_context_tokens <= 0: + return ( + "stop_budget_exhausted", + "CODE_STATUS:r_loop_node_or_context_budget_exhausted", + "return_summary", + ) + + wants_deeper = ( + r3_inspection.recommended_next_action == "deeper" + or r3_inspection.granularity_problem_status == "needs_lower_granularity" + ) + if wants_deeper: + if r3_inspection.child_node_count <= 0: + return ( + "stop_no_actionable_path", + "CODE_STATUS:r_loop_no_child_node_for_deeper_traversal", + "return_summary", + ) + if remaining_traversal_depth <= 0: + return ( + "stop_budget_exhausted", + "CODE_STATUS:r_loop_traversal_depth_budget_exhausted", + "return_summary", + ) + return ( + "continue_deeper", + "CODE_STATUS:r3_needs_lower_granularity", + "R2", + ) + + wants_branch_switch = ( + r3_inspection.recommended_next_action == "switch_branch" + or r3_inspection.branch_problem_status == "wrong_branch" + ) + if wants_branch_switch: + if remaining_branch_switches <= 0: + return ( + "stop_budget_exhausted", + "CODE_STATUS:r_loop_branch_switch_budget_exhausted", + "return_summary", + ) + return ( + "continue_switch_branch", + "CODE_STATUS:r3_recommends_branch_switch", + "R2", + ) + + if r3_inspection.recommended_next_action == "fail": + return "stop_failed_final", "CODE_STATUS:r3_failed_final", "return_summary" + + return ( + "stop_no_actionable_path", + "CODE_STATUS:r_loop_no_actionable_path", + "return_summary", + ) + + +def _unique_strings(values: list[str | None]) -> list[str]: + seen: set[str] = set() + result: list[str] = [] + for value in values: + if not value or value in seen: + continue + seen.add(value) + result.append(value) + return result diff --git a/songryeon_core/core/r_loop_vessel_read_packet.py b/songryeon_core/core/r_loop_vessel_read_packet.py new file mode 100644 index 0000000..289f579 --- /dev/null +++ b/songryeon_core/core/r_loop_vessel_read_packet.py @@ -0,0 +1,1143 @@ +from __future__ import annotations + +import json +from dataclasses import asdict, dataclass +from datetime import datetime +from typing import Any, Callable + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.graph_memory_store import ( + SONGRYEON_GRAPH_NAMESPACE, + SONGRYEON_VESSEL_SERVICE_NAME, +) +from songryeon_core.core.graph_vessel_neo4j import GraphVesselNeo4jConfig +from songryeon_core.core.trace_store import TraceStore + + +R_LOOP_VESSEL_READ_PACKET_DATA_TYPE = "r_loop:vessel_read_packet" +R_LOOP_VESSEL_READ_PACKET_GENERATOR = "CODE:R_LOOP_VESSEL_READ_PACKET_BUILDER" +R_LOOP_VESSEL_READ_PACKET_POLICY_ID = "R_LOOP_VESSEL_READ_PACKET_V0" +R_LOOP_VESSEL_READ_PACKET_SCHEMA_NAME = "RLoopVesselReadPacketFrame" +R_LOOP_VESSEL_EXACT_CHILD_EXPANSION_POLICY_ID = ( + "R_LOOP_VESSEL_EXACT_CHILD_EXPANSION_V0" +) +R_LOOP_VESSEL_EXACT_CHILD_EXPANSION_MAX_DEPTH = 2 +R_LOOP_VESSEL_SUMMARY_CHILD_EXPANSION_POLICY_ID = ( + "R_LOOP_VESSEL_SUMMARY_CHILD_EXPANSION_V0" +) +R_LOOP_VESSEL_SUMMARY_CHILD_EXPANSION_MAX_DEPTH = 2 +R_LOOP_VESSEL_READ_PACKET_STATUSES = { + "passed", + "empty", + "adapter_unavailable", + "read_failed", +} + +Neo4jDriverFactory = Callable[..., Any] + + +@dataclass(frozen=True) +class RLoopVesselEntryCandidateRecord: + candidate_node_id: str + candidate_kind: str + display_name: str + node_kind: str | None + data_kind: str | None + created_at: str | None + written_at: str | None + source_trace_id: str | None + summary_depth: int | None + source_leaf_count: int | None + source_summary_count: int | None + source_graph_node_ids: list[str] + parent_graph_node_ids: list[str] + + +@dataclass(frozen=True) +class RLoopVesselSummaryCandidateRecord: + summary_node_id: str + summary_display_name: str + data_kind: str | None + summary_depth: int | None + summary_status: str | None + validity_status: str | None + review_status: str | None + info_class: str | None + generated_by: str | None + target_graph_node_id: str | None + target_display_name: str | None + target_node_kind: str | None + source_leaf_count: int | None + source_summary_count: int | None + source_graph_node_ids: list[str] + source_data_ids: list[str] + source_trace_ids: list[str] + summary_text_char_count: int + summary_text: str + summary_text_preview: str + + +@dataclass(frozen=True) +class RLoopVesselReadPacketFrame: + packet_id: str + created_at: str + policy_id: str + target_adapter_name: str + vessel_database_name: str + graph_namespace: str + target_consumer: str + read_status: str + failure_type: str | None + failure_reason: str | None + entry_candidate_count: int + summary_candidate_count: int + total_summary_scanned_count: int | None + active_summary_candidate_count: int | None + skipped_summary_count: int | None + summary_child_expansion_policy_id: str + base_summary_candidate_count: int + summary_child_expanded_count: int + summary_child_expanded_node_ids: list[str] + summary_child_expansion_truncated: bool + exact_child_expansion_policy_id: str + base_entry_candidate_count: int + exact_child_expanded_entry_count: int + exact_child_expanded_node_ids: list[str] + exact_child_expansion_truncated: bool + summary_count_by_data_kind: dict[str, int] + summary_count_by_depth: dict[str, int] + entry_candidate_records: list[dict[str, object]] + summary_candidate_records: list[dict[str, object]] + packet_lines: list[str] + source_data_ids: list[str] + source_trace_ids: list[str] + generated_by: str = R_LOOP_VESSEL_READ_PACKET_GENERATOR + info_class: str = "absolute" + semantic_judgement_status: str = "not_run" + schema_name: str = R_LOOP_VESSEL_READ_PACKET_SCHEMA_NAME + + +@dataclass(frozen=True) +class RecordedRLoopVesselReadPacket: + packet: RLoopVesselReadPacketFrame + trace_event_id: str + created_data_ids: list[str] + existing_data_ids: list[str] + + +def r_loop_vessel_read_packet_id(batch_id: str) -> str: + if not batch_id: + raise ValueError("batch_id must not be empty") + return f"r_loop:vessel_read_packet:{_stable_suffix(batch_id)}" + + +def build_r_loop_vessel_read_packet_from_neo4j( + *, + config: GraphVesselNeo4jConfig, + batch_id: str = "manual_r_loop_vessel_read_packet", + limit: int = 50, + created_at: str | None = None, + driver_factory: Neo4jDriverFactory | None = None, +) -> RLoopVesselReadPacketFrame: + timestamp = created_at or _now_iso() + safe_limit = max(1, min(limit, 500)) + base = _PacketBase( + packet_id=r_loop_vessel_read_packet_id(batch_id), + created_at=timestamp, + target_adapter_name=SONGRYEON_VESSEL_SERVICE_NAME, + vessel_database_name=config.database, + graph_namespace=SONGRYEON_GRAPH_NAMESPACE, + ) + + config_failure = _config_failure(config) + if config_failure is not None: + failure_type, failure_reason = config_failure + return _make_packet( + base=base, + read_status="adapter_unavailable", + failure_type=failure_type, + failure_reason=failure_reason, + entry_candidates=[], + active_summary_candidates=[], + total_summary_scanned_count=None, + ) + + driver_factory = driver_factory or _load_neo4j_driver_factory() + if driver_factory is None: + return _make_packet( + base=base, + read_status="adapter_unavailable", + failure_type="neo4j_driver_missing", + failure_reason="Python neo4j package is not installed.", + entry_candidates=[], + active_summary_candidates=[], + total_summary_scanned_count=None, + ) + + try: + auth = None if config.allow_no_auth and config.password is None else (config.user, config.password) + driver = driver_factory(config.uri, auth=auth) + try: + with driver.session(database=config.database) as session: + ( + entry_candidates, + active_summary_candidates, + total_summary_scanned_count, + expanded_entry_records, + expanded_summary_records, + ) = session.execute_read( + _execute_read_packet, + SONGRYEON_GRAPH_NAMESPACE, + safe_limit, + ) + finally: + close = getattr(driver, "close", None) + if callable(close): + close() + except Exception as exc: + return _make_packet( + base=base, + read_status="read_failed", + failure_type="neo4j_read_failed", + failure_reason=f"{type(exc).__name__}: {exc}", + entry_candidates=[], + active_summary_candidates=[], + total_summary_scanned_count=None, + ) + + read_status = "passed" if entry_candidates or active_summary_candidates else "empty" + return _make_packet( + base=base, + read_status=read_status, + failure_type=None if read_status == "passed" else "no_r_loop_vessel_candidates", + failure_reason=None + if read_status == "passed" + else "No R loop Vessel entry or active summary candidates were found.", + entry_candidates=entry_candidates, + active_summary_candidates=active_summary_candidates, + total_summary_scanned_count=total_summary_scanned_count, + base_entry_candidate_count=expanded_entry_records.base_entry_candidate_count, + exact_child_expanded_node_ids=expanded_entry_records.exact_child_expanded_node_ids, + exact_child_expansion_truncated=( + expanded_entry_records.exact_child_expansion_truncated + ), + base_summary_candidate_count=( + expanded_summary_records.base_summary_candidate_count + ), + summary_child_expanded_node_ids=( + expanded_summary_records.summary_child_expanded_node_ids + ), + summary_child_expansion_truncated=( + expanded_summary_records.summary_child_expansion_truncated + ), + ) + + +def record_r_loop_vessel_read_packet( + *, + trace_store: TraceStore, + data_store: DataStore, + turn_id: str, + config: GraphVesselNeo4jConfig, + batch_id: str = "manual_r_loop_vessel_read_packet", + limit: int = 50, + created_at: str | None = None, + driver_factory: Neo4jDriverFactory | None = None, +) -> RecordedRLoopVesselReadPacket: + packet = build_r_loop_vessel_read_packet_from_neo4j( + config=config, + batch_id=batch_id, + limit=limit, + created_at=created_at, + driver_factory=driver_factory, + ) + event = trace_store.create_event( + turn_id=turn_id, + actor="r_loop_vessel_read_packet_builder", + event_type="node_output", + timestamp=packet.created_at, + input_ref=[], + output_ref=[packet.packet_id], + schema_status="passed", + ) + created_data_ids: list[str] = [] + existing_data_ids: list[str] = [] + _record_payload_if_missing( + data_store=data_store, + data_id=packet.packet_id, + data_type=R_LOOP_VESSEL_READ_PACKET_DATA_TYPE, + payload=asdict(packet), + created_at=event.timestamp, + source_trace_id=event.event_id, + created_data_ids=created_data_ids, + existing_data_ids=existing_data_ids, + ) + return RecordedRLoopVesselReadPacket( + packet=packet, + trace_event_id=event.event_id, + created_data_ids=created_data_ids, + existing_data_ids=existing_data_ids, + ) + + +@dataclass(frozen=True) +class _PacketBase: + packet_id: str + created_at: str + target_adapter_name: str + vessel_database_name: str + graph_namespace: str + + +@dataclass(frozen=True) +class _EntryExpansion: + records: list[object] + base_entry_candidate_count: int + exact_child_expanded_node_ids: list[str] + exact_child_expansion_truncated: bool + + +@dataclass(frozen=True) +class _SummaryExpansion: + candidates: list[RLoopVesselSummaryCandidateRecord] + base_summary_candidate_count: int + summary_child_expanded_node_ids: list[str] + summary_child_expansion_truncated: bool + + +def _make_packet( + *, + base: _PacketBase, + read_status: str, + failure_type: str | None, + failure_reason: str | None, + entry_candidates: list[RLoopVesselEntryCandidateRecord], + active_summary_candidates: list[RLoopVesselSummaryCandidateRecord], + total_summary_scanned_count: int | None, + base_entry_candidate_count: int | None = None, + exact_child_expanded_node_ids: list[str] | None = None, + exact_child_expansion_truncated: bool = False, + base_summary_candidate_count: int | None = None, + summary_child_expanded_node_ids: list[str] | None = None, + summary_child_expansion_truncated: bool = False, +) -> RLoopVesselReadPacketFrame: + entry_payloads = [asdict(candidate) for candidate in entry_candidates] + summary_payloads = [asdict(candidate) for candidate in active_summary_candidates] + active_summary_count = len(active_summary_candidates) + expanded_node_ids = exact_child_expanded_node_ids or [] + expanded_summary_node_ids = summary_child_expanded_node_ids or [] + skipped_summary_count = ( + None + if total_summary_scanned_count is None + else max(0, total_summary_scanned_count - active_summary_count) + ) + source_data_ids = _unique_strings( + [candidate.candidate_node_id for candidate in entry_candidates] + + [ + source_id + for candidate in entry_candidates + for source_id in candidate.source_graph_node_ids + ] + + [ + parent_id + for candidate in entry_candidates + for parent_id in candidate.parent_graph_node_ids + ] + + [candidate.summary_node_id for candidate in active_summary_candidates] + + [candidate.target_graph_node_id for candidate in active_summary_candidates] + + [ + source_id + for candidate in active_summary_candidates + for source_id in candidate.source_data_ids + ] + ) + source_trace_ids = _unique_strings( + [candidate.source_trace_id for candidate in entry_candidates] + + [ + source_trace_id + for candidate in active_summary_candidates + for source_trace_id in candidate.source_trace_ids + ] + ) + packet = RLoopVesselReadPacketFrame( + packet_id=base.packet_id, + created_at=base.created_at, + policy_id=R_LOOP_VESSEL_READ_PACKET_POLICY_ID, + target_adapter_name=base.target_adapter_name, + vessel_database_name=base.vessel_database_name, + graph_namespace=base.graph_namespace, + target_consumer="R_LOOP", + read_status=read_status, + failure_type=failure_type, + failure_reason=failure_reason, + entry_candidate_count=len(entry_candidates), + summary_candidate_count=active_summary_count, + total_summary_scanned_count=total_summary_scanned_count, + active_summary_candidate_count=active_summary_count + if total_summary_scanned_count is not None + else None, + skipped_summary_count=skipped_summary_count, + summary_child_expansion_policy_id=( + R_LOOP_VESSEL_SUMMARY_CHILD_EXPANSION_POLICY_ID + ), + base_summary_candidate_count=( + active_summary_count + if base_summary_candidate_count is None + else base_summary_candidate_count + ), + summary_child_expanded_count=len(expanded_summary_node_ids), + summary_child_expanded_node_ids=expanded_summary_node_ids, + summary_child_expansion_truncated=summary_child_expansion_truncated, + exact_child_expansion_policy_id=R_LOOP_VESSEL_EXACT_CHILD_EXPANSION_POLICY_ID, + base_entry_candidate_count=( + len(entry_candidates) + if base_entry_candidate_count is None + else base_entry_candidate_count + ), + exact_child_expanded_entry_count=len(expanded_node_ids), + exact_child_expanded_node_ids=expanded_node_ids, + exact_child_expansion_truncated=exact_child_expansion_truncated, + summary_count_by_data_kind=_count_summaries_by_data_kind(active_summary_candidates), + summary_count_by_depth=_count_summaries_by_depth(active_summary_candidates), + entry_candidate_records=entry_payloads, + summary_candidate_records=summary_payloads, + packet_lines=_build_packet_lines( + entry_candidates, + active_summary_candidates, + base_entry_candidate_count=( + len(entry_candidates) + if base_entry_candidate_count is None + else base_entry_candidate_count + ), + exact_child_expanded_node_ids=expanded_node_ids, + exact_child_expansion_truncated=exact_child_expansion_truncated, + base_summary_candidate_count=( + active_summary_count + if base_summary_candidate_count is None + else base_summary_candidate_count + ), + summary_child_expanded_node_ids=expanded_summary_node_ids, + summary_child_expansion_truncated=summary_child_expansion_truncated, + ), + source_data_ids=source_data_ids, + source_trace_ids=source_trace_ids, + ) + _validate_packet(packet) + return packet + + +def _execute_read_packet( + tx: object, + graph_namespace: str, + limit: int, +) -> tuple[ + list[RLoopVesselEntryCandidateRecord], + list[RLoopVesselSummaryCandidateRecord], + int, + _EntryExpansion, + _SummaryExpansion, +]: + entry_records = _read_entry_records(tx, graph_namespace) + summary_records = _read_summary_records(tx, graph_namespace) + expanded_entry_records = _expand_entry_records_by_exact_children( + entry_records=entry_records, + limit=limit, + ) + entry_candidates = [ + _entry_candidate_from_record(record) + for record in expanded_entry_records.records + ] + active_summary_candidates_all = [ + _summary_candidate_from_record(record) + for record in summary_records + if _is_active_r_loop_summary(record) + ] + base_active_summary_candidates = _balanced_summary_candidate_limit( + active_summary_candidates_all, + limit, + ) + summary_expansion = _expand_summary_candidates_by_source_summary_children( + base_candidates=base_active_summary_candidates, + all_candidates=active_summary_candidates_all, + limit=limit, + ) + return ( + entry_candidates, + summary_expansion.candidates, + len(summary_records), + expanded_entry_records, + summary_expansion, + ) + + +def _expand_entry_records_by_exact_children( + *, + entry_records: list[object], + limit: int, +) -> _EntryExpansion: + base_records = list(entry_records[:limit]) + if not base_records: + return _EntryExpansion( + records=[], + base_entry_candidate_count=0, + exact_child_expanded_node_ids=[], + exact_child_expansion_truncated=False, + ) + + records_by_id: dict[str, object] = {} + record_index_by_id: dict[str, int] = {} + parent_index: dict[str, list[str]] = {} + for index, record in enumerate(entry_records): + candidate_id = _optional_record_str(record, "candidate_node_id") + if not candidate_id: + continue + records_by_id[candidate_id] = record + record_index_by_id[candidate_id] = index + for parent_id in _record_list_strings(record, "parent_graph_node_ids"): + parent_index.setdefault(parent_id, []).append(candidate_id) + + selected_ids = { + candidate_id + for record in base_records + for candidate_id in [_optional_record_str(record, "candidate_node_id")] + if candidate_id + } + expanded_records = list(base_records) + expanded_node_ids: list[str] = [] + frontier = list(base_records) + expansion_budget = limit + truncated = False + + for _depth in range(R_LOOP_VESSEL_EXACT_CHILD_EXPANSION_MAX_DEPTH): + child_ids = _ordered_exact_child_ids( + frontier=frontier, + parent_index=parent_index, + record_index_by_id=record_index_by_id, + ) + next_frontier: list[object] = [] + for child_id in child_ids: + if child_id in selected_ids: + continue + child_record = records_by_id.get(child_id) + if child_record is None: + continue + if expansion_budget <= 0: + truncated = True + break + expanded_records.append(child_record) + expanded_node_ids.append(child_id) + selected_ids.add(child_id) + next_frontier.append(child_record) + expansion_budget -= 1 + if truncated or not next_frontier: + break + frontier = next_frontier + + return _EntryExpansion( + records=expanded_records, + base_entry_candidate_count=len(base_records), + exact_child_expanded_node_ids=expanded_node_ids, + exact_child_expansion_truncated=truncated, + ) + + +def _ordered_exact_child_ids( + *, + frontier: list[object], + parent_index: dict[str, list[str]], + record_index_by_id: dict[str, int], +) -> list[str]: + child_ids: list[str] = [] + for record in frontier: + payload = _payload(record) + child_ids.extend(_list_strings(payload.get("source_graph_node_ids"))) + candidate_id = _optional_record_str(record, "candidate_node_id") + if candidate_id: + child_ids.extend(parent_index.get(candidate_id, [])) + return sorted( + _unique_strings(child_ids), + key=lambda item: (record_index_by_id.get(item, 1_000_000), item), + ) + + +def _read_entry_records(tx: object, graph_namespace: str) -> list[object]: + result = tx.run( + """ + MATCH (core:CoreEgo {graph_namespace: $graph_namespace}) + -[:HAS_AXIS {graph_namespace: $graph_namespace}]-> + (axis:TimeAxis {graph_namespace: $graph_namespace}) + RETURN + axis.data_id AS candidate_node_id, + coalesce(axis.display_name, axis.data_id) AS display_name, + axis.node_kind AS node_kind, + axis.data_kind AS data_kind, + axis.created_at AS created_at, + axis.written_at AS written_at, + axis.source_trace_id AS source_trace_id, + axis.payload_json AS payload_json, + labels(axis) AS labels, + [] AS parent_graph_node_ids, + 0 AS traversal_level + UNION + MATCH (core:CoreEgo {graph_namespace: $graph_namespace}) + -[:HAS_AXIS {graph_namespace: $graph_namespace}]-> + (axis:TimeAxis {graph_namespace: $graph_namespace}) + MATCH (axis)-[:HAS_BUNDLE {graph_namespace: $graph_namespace}]-> + (entry:VesselRecord {graph_namespace: $graph_namespace}) + WHERE entry:TimeBundle OR entry:SourceIngestBundle + RETURN + entry.data_id AS candidate_node_id, + coalesce(entry.display_name, entry.data_id) AS display_name, + entry.node_kind AS node_kind, + entry.data_kind AS data_kind, + entry.created_at AS created_at, + entry.written_at AS written_at, + entry.source_trace_id AS source_trace_id, + entry.payload_json AS payload_json, + labels(entry) AS labels, + [axis.data_id] AS parent_graph_node_ids, + 1 AS traversal_level + UNION + MATCH (entry:VesselRecord {graph_namespace: $graph_namespace}) + WHERE entry:SourceKindBundle + OR entry:RawSource + OR entry.node_kind = "source_kind_bundle" + OR entry.node_kind = "raw_source" + OR entry.node_kind = "token_budget_summary_bundle" + OPTIONAL MATCH (parent:VesselRecord {graph_namespace: $graph_namespace})-[rel]->(entry) + WHERE type(rel) IN ["HAS_SOURCE_KIND", "CONTAINS_SOURCE", "CONTAINS"] + WITH entry, collect(parent.data_id) AS parent_graph_node_ids + RETURN + entry.data_id AS candidate_node_id, + coalesce(entry.display_name, entry.data_id) AS display_name, + entry.node_kind AS node_kind, + entry.data_kind AS data_kind, + entry.created_at AS created_at, + entry.written_at AS written_at, + entry.source_trace_id AS source_trace_id, + entry.payload_json AS payload_json, + labels(entry) AS labels, + [id IN parent_graph_node_ids WHERE id IS NOT NULL] AS parent_graph_node_ids, + 2 AS traversal_level + ORDER BY traversal_level, created_at, candidate_node_id + LIMIT 10000 + """, + graph_namespace=graph_namespace, + ) + return _records(result) + + +def _read_summary_records(tx: object, graph_namespace: str) -> list[object]: + result = tx.run( + """ + MATCH (summary:SummaryGraphNode {graph_namespace: $graph_namespace}) + OPTIONAL MATCH (summary)-[:SUMMARY_OF {graph_namespace: $graph_namespace}]-> + (target:VesselRecord {graph_namespace: $graph_namespace}) + RETURN + summary.data_id AS summary_node_id, + coalesce(summary.display_name, summary.data_id) AS summary_display_name, + summary.data_kind AS data_kind, + summary.info_class AS info_class, + summary.generated_by AS generated_by, + summary.payload_json AS payload_json, + target.data_id AS target_graph_node_id, + coalesce(target.display_name, target.data_id) AS target_display_name, + target.node_kind AS target_node_kind + ORDER BY summary.data_kind, summary.data_id + LIMIT 10000 + """, + graph_namespace=graph_namespace, + ) + return _records(result) + + +def _entry_candidate_from_record(record: object) -> RLoopVesselEntryCandidateRecord: + payload = _payload(record) + candidate_id = _require_record_str(record, "candidate_node_id") + return RLoopVesselEntryCandidateRecord( + candidate_node_id=candidate_id, + candidate_kind=_entry_candidate_kind(record, payload), + display_name=_require_record_str(record, "display_name"), + node_kind=_optional_record_str(record, "node_kind") + or _text(payload.get("node_kind")) + or None, + data_kind=_optional_record_str(record, "data_kind") + or _text(payload.get("data_kind")) + or None, + created_at=_optional_record_str(record, "created_at") + or _text(payload.get("created_at")) + or None, + written_at=_optional_record_str(record, "written_at") + or _text(payload.get("written_at")) + or None, + source_trace_id=_optional_record_str(record, "source_trace_id") + or _first_string(_list_strings(payload.get("source_trace_ids"))), + summary_depth=_optional_int(payload.get("summary_depth")), + source_leaf_count=_optional_int(payload.get("source_leaf_count")), + source_summary_count=_optional_int(payload.get("source_summary_count")), + source_graph_node_ids=_list_strings(payload.get("source_graph_node_ids")), + parent_graph_node_ids=_record_list_strings(record, "parent_graph_node_ids"), + ) + + +def _summary_candidate_from_record(record: object) -> RLoopVesselSummaryCandidateRecord: + payload = _payload(record) + summary_text = _text(payload.get("summary_text")) + source_data_ids = _list_strings(payload.get("source_data_ids")) + target_graph_node_id = ( + _optional_record_str(record, "target_graph_node_id") + or _text(payload.get("target_graph_node_id")) + or None + ) + return RLoopVesselSummaryCandidateRecord( + summary_node_id=_require_record_str(record, "summary_node_id"), + summary_display_name=_require_record_str(record, "summary_display_name"), + data_kind=_optional_record_str(record, "data_kind") + or _text(payload.get("data_kind")) + or None, + summary_depth=_optional_int(payload.get("summary_depth")), + summary_status=_text(payload.get("summary_status")) or None, + validity_status=_text(payload.get("validity_status")) or None, + review_status=_text(payload.get("review_status")) or None, + info_class=_optional_record_str(record, "info_class") + or _text(payload.get("info_class")) + or None, + generated_by=_optional_record_str(record, "generated_by") + or _text(payload.get("generated_by")) + or None, + target_graph_node_id=target_graph_node_id, + target_display_name=_optional_record_str(record, "target_display_name"), + target_node_kind=_optional_record_str(record, "target_node_kind") + or _text(payload.get("target_node_kind")) + or None, + source_leaf_count=_optional_int(payload.get("source_leaf_count")), + source_summary_count=_optional_int(payload.get("source_summary_count")), + source_graph_node_ids=_list_strings(payload.get("source_graph_node_ids")), + source_data_ids=_unique_strings(source_data_ids + [target_graph_node_id]), + source_trace_ids=_list_strings(payload.get("source_trace_ids")), + summary_text_char_count=len(summary_text), + summary_text=summary_text, + summary_text_preview=_preview(summary_text), + ) + + +def _entry_candidate_kind(record: object, payload: dict[str, object]) -> str: + labels = _record_value(record, "labels") + if isinstance(labels, list): + if "TimeAxis" in labels: + return "time_axis" + if "SourceIngestBundle" in labels: + return "source_ingest_bundle" + if "SourceKindBundle" in labels: + return "source_kind_bundle" + if "RawSource" in labels: + return "raw_source" + if "TimeBundle" in labels: + return "time_bundle" + node_kind = _optional_record_str(record, "node_kind") or _text(payload.get("node_kind")) + if node_kind == "time_axis": + return "time_axis" + if node_kind in { + "time_bundle", + "source_ingest_time_bundle", + "source_kind_bundle", + "raw_source", + "token_budget_summary_bundle", + }: + return node_kind + return "entry_bundle" + + +def _is_active_r_loop_summary(record: object) -> bool: + payload = _payload(record) + return ( + _text(payload.get("summary_status")) == "ran" + and _text(payload.get("validity_status")) == "active" + ) + + +def _balanced_summary_candidate_limit( + candidates: list[RLoopVesselSummaryCandidateRecord], + limit: int, +) -> list[RLoopVesselSummaryCandidateRecord]: + if len(candidates) <= limit: + return list(candidates) + + groups: dict[tuple[str, str, str], list[RLoopVesselSummaryCandidateRecord]] = {} + for candidate in candidates: + key = _summary_candidate_balance_key(candidate) + groups.setdefault(key, []).append(candidate) + + ordered_keys = sorted(groups) + selected: list[RLoopVesselSummaryCandidateRecord] = [] + while len(selected) < limit and ordered_keys: + next_keys: list[tuple[str, str, str]] = [] + for key in ordered_keys: + group = groups[key] + if group and len(selected) < limit: + selected.append(group.pop(0)) + if group: + next_keys.append(key) + ordered_keys = next_keys + return selected + + +def _expand_summary_candidates_by_source_summary_children( + *, + base_candidates: list[RLoopVesselSummaryCandidateRecord], + all_candidates: list[RLoopVesselSummaryCandidateRecord], + limit: int, +) -> _SummaryExpansion: + """Copy source summary children needed for hierarchical R traversal. + + This is not a relevance decision. The code only follows summary->source IDs + that already point at active summary records loaded from Neo4j. + """ + + selected = list(base_candidates) + selected_ids = {candidate.summary_node_id for candidate in selected} + candidates_by_id = { + candidate.summary_node_id: candidate for candidate in all_candidates + } + expanded_node_ids: list[str] = [] + expansion_budget = max(0, limit) + frontier = list(base_candidates) + truncated = False + + for _depth in range(R_LOOP_VESSEL_SUMMARY_CHILD_EXPANSION_MAX_DEPTH): + next_frontier: list[RLoopVesselSummaryCandidateRecord] = [] + for source_id in _summary_source_summary_ids(frontier): + if source_id in selected_ids: + continue + child = candidates_by_id.get(source_id) + if child is None: + continue + if expansion_budget <= 0: + truncated = True + break + selected.append(child) + selected_ids.add(child.summary_node_id) + expanded_node_ids.append(child.summary_node_id) + next_frontier.append(child) + expansion_budget -= 1 + if truncated or not next_frontier: + break + frontier = next_frontier + + return _SummaryExpansion( + candidates=selected, + base_summary_candidate_count=len(base_candidates), + summary_child_expanded_node_ids=expanded_node_ids, + summary_child_expansion_truncated=truncated, + ) + + +def _summary_source_summary_ids( + candidates: list[RLoopVesselSummaryCandidateRecord], +) -> list[str]: + values: list[str] = [] + for candidate in candidates: + values.extend( + source_id + for source_id in candidate.source_graph_node_ids + if source_id.startswith("graph:summary:") + ) + values.extend( + source_id + for source_id in candidate.source_data_ids + if source_id.startswith("graph:summary:") + ) + return _unique_strings(values) + + +def _summary_candidate_balance_key( + candidate: RLoopVesselSummaryCandidateRecord, +) -> tuple[int, int, str, str]: + depth = candidate.summary_depth if candidate.summary_depth is not None else -1 + return ( + _summary_candidate_data_kind_rank(candidate.data_kind), + -depth, + candidate.data_kind or "unknown", + candidate.info_class or "unknown", + ) + + +def _summary_candidate_data_kind_rank(data_kind: str | None) -> int: + if data_kind == "token_budget_bundle_summary": + return 0 + if data_kind == "source_kind_summary": + return 1 + if data_kind == "source_leaf_summary": + return 2 + return 3 + + +def _count_summaries_by_data_kind( + candidates: list[RLoopVesselSummaryCandidateRecord], +) -> dict[str, int]: + counts: dict[str, int] = {} + for candidate in candidates: + key = candidate.data_kind or "unknown" + counts[key] = counts.get(key, 0) + 1 + return dict(sorted(counts.items())) + + +def _count_summaries_by_depth( + candidates: list[RLoopVesselSummaryCandidateRecord], +) -> dict[str, int]: + counts: dict[str, int] = {} + for candidate in candidates: + key = str(candidate.summary_depth) if candidate.summary_depth is not None else "unknown" + counts[key] = counts.get(key, 0) + 1 + return dict(sorted(counts.items(), key=lambda item: item[0])) + + +def _build_packet_lines( + entry_candidates: list[RLoopVesselEntryCandidateRecord], + active_summary_candidates: list[RLoopVesselSummaryCandidateRecord], + *, + base_entry_candidate_count: int, + exact_child_expanded_node_ids: list[str], + exact_child_expansion_truncated: bool, + base_summary_candidate_count: int, + summary_child_expanded_node_ids: list[str], + summary_child_expansion_truncated: bool, +) -> list[str]: + lines = [ + f"R loop Vessel entry candidates: {len(entry_candidates)}", + f"R loop Vessel base entry candidates: {base_entry_candidate_count}", + "R loop Vessel exact child expansion: " + f"{len(exact_child_expanded_node_ids)}" + f" / truncated={str(exact_child_expansion_truncated).lower()}", + f"R loop active summary candidates: {len(active_summary_candidates)}", + f"R loop base active summary candidates: {base_summary_candidate_count}", + "R loop summary child expansion: " + f"{len(summary_child_expanded_node_ids)}" + f" / truncated={str(summary_child_expansion_truncated).lower()}", + ] + if entry_candidates: + lines.append("Entry candidates") + for candidate in entry_candidates: + lines.append( + " " + f"{candidate.candidate_kind} " + f"[{candidate.candidate_node_id}] " + f"{candidate.display_name}" + ) + if active_summary_candidates: + lines.append("Active summary candidates") + for candidate in active_summary_candidates: + lines.append( + " " + f"{candidate.data_kind or 'unknown'}" + f"(depth={candidate.summary_depth if candidate.summary_depth is not None else 'unknown'}, " + f"info={candidate.info_class or 'unknown'}) " + f"[{candidate.summary_node_id}]" + ) + if candidate.target_graph_node_id: + target_name = candidate.target_display_name or candidate.target_graph_node_id + lines.append(f" SUMMARY_OF -> {target_name} [{candidate.target_graph_node_id}]") + if candidate.summary_text_preview: + lines.append(f" preview: {candidate.summary_text_preview}") + return lines + + +def _config_failure(config: GraphVesselNeo4jConfig) -> tuple[str, str] | None: + if not config.uri or not config.user or not config.database: + return ( + "neo4j_config_missing", + "Neo4j uri/user/database config is incomplete.", + ) + if config.password is None and not config.allow_no_auth: + return ( + "neo4j_config_missing", + "Neo4j password is not configured. Set SONGRYEON_NEO4J_PASSWORD or pass --password.", + ) + return None + + +def _validate_packet(packet: RLoopVesselReadPacketFrame) -> None: + if packet.generated_by != R_LOOP_VESSEL_READ_PACKET_GENERATOR: + raise ValueError("RLoopVesselReadPacketFrame.generated_by must be packet builder") + if packet.info_class != "absolute": + raise ValueError("RLoopVesselReadPacketFrame.info_class must be absolute") + if packet.semantic_judgement_status != "not_run": + raise ValueError("RLoopVesselReadPacketFrame.semantic_judgement_status must be not_run") + if packet.read_status not in R_LOOP_VESSEL_READ_PACKET_STATUSES: + raise ValueError("RLoopVesselReadPacketFrame.read_status is invalid") + if packet.target_consumer != "R_LOOP": + raise ValueError("RLoopVesselReadPacketFrame.target_consumer must be R_LOOP") + if ( + packet.summary_child_expansion_policy_id + != R_LOOP_VESSEL_SUMMARY_CHILD_EXPANSION_POLICY_ID + ): + raise ValueError("RLoopVesselReadPacketFrame summary child expansion policy is invalid") + if packet.base_summary_candidate_count < 0: + raise ValueError("RLoopVesselReadPacketFrame base_summary_candidate_count is invalid") + if packet.summary_candidate_count < packet.base_summary_candidate_count: + raise ValueError("RLoopVesselReadPacketFrame summary count is below base count") + if packet.summary_child_expanded_count != len(packet.summary_child_expanded_node_ids): + raise ValueError("RLoopVesselReadPacketFrame summary child expansion count mismatch") + if packet.exact_child_expansion_policy_id != R_LOOP_VESSEL_EXACT_CHILD_EXPANSION_POLICY_ID: + raise ValueError("RLoopVesselReadPacketFrame exact child expansion policy is invalid") + if packet.base_entry_candidate_count < 0: + raise ValueError("RLoopVesselReadPacketFrame base_entry_candidate_count is invalid") + if packet.exact_child_expanded_entry_count != len(packet.exact_child_expanded_node_ids): + raise ValueError("RLoopVesselReadPacketFrame exact child expansion count mismatch") + if packet.entry_candidate_count < packet.base_entry_candidate_count: + raise ValueError("RLoopVesselReadPacketFrame entry count is below base count") + if packet.read_status == "passed": + if packet.entry_candidate_count + packet.summary_candidate_count < 1: + raise ValueError("passed RLoopVesselReadPacketFrame must include candidates") + + +def _record_payload_if_missing( + *, + data_store: DataStore, + data_id: str, + data_type: str, + payload: dict[str, object], + created_at: str, + source_trace_id: str, + created_data_ids: list[str], + existing_data_ids: list[str], +) -> None: + existing = data_store.get_record(data_id) + if existing is not None: + if existing.data_type != data_type: + raise ValueError(f"R loop Vessel read packet data_id collision with different type: {data_id}") + if existing.payload != payload: + raise ValueError(f"R loop Vessel read packet data_id collision with different payload: {data_id}") + existing_data_ids.append(data_id) + return + data_store.create_record( + data_id=data_id, + data_type=data_type, + exists=True, + created_at=created_at, + source_trace_id=source_trace_id, + payload=payload, + ) + created_data_ids.append(data_id) + + +def _load_neo4j_driver_factory() -> Neo4jDriverFactory | None: + try: + from neo4j import GraphDatabase + except ImportError: + return None + return GraphDatabase.driver + + +def _records(result: object) -> list[object]: + return list(result) + + +def _require_record_str(record: object, field_name: str) -> str: + value = _record_value(record, field_name) + if not isinstance(value, str) or not value: + raise ValueError(f"R loop Vessel record field is missing: {field_name}") + return value + + +def _optional_record_str(record: object, field_name: str) -> str | None: + value = _record_value(record, field_name) + return value if isinstance(value, str) and value else None + + +def _record_list_strings(record: object, field_name: str) -> list[str]: + return _list_strings(_record_value(record, field_name)) + + +def _record_value(record: object, field_name: str) -> object: + try: + return record[field_name] # type: ignore[index] + except (KeyError, TypeError): + data = getattr(record, "data", None) + if callable(data): + payload = data() + if isinstance(payload, dict): + return payload.get(field_name) + return None + + +def _payload(record: object) -> dict[str, object]: + payload_json = _optional_record_str(record, "payload_json") + if not payload_json: + return {} + try: + payload = json.loads(payload_json) + except json.JSONDecodeError: + return {} + return payload if isinstance(payload, dict) else {} + + +def _optional_int(value: object) -> int | None: + if isinstance(value, bool): + return None + if isinstance(value, int): + return value + if isinstance(value, str): + try: + return int(value) + except ValueError: + return None + return None + + +def _list_strings(value: object) -> list[str]: + if not isinstance(value, list): + return [] + return [item for item in value if isinstance(item, str) and item] + + +def _first_string(values: list[str]) -> str | None: + return values[0] if values else None + + +def _text(value: object) -> str: + return value if isinstance(value, str) else "" + + +def _preview(text: str, *, max_chars: int = 180) -> str: + compact = " ".join(text.split()) + if len(compact) <= max_chars: + return compact + return compact[: max_chars - 3].rstrip() + "..." + + +def _unique_strings(values: list[str | None]) -> list[str]: + seen: set[str] = set() + result: list[str] = [] + for value in values: + if not value or value in seen: + continue + seen.add(value) + result.append(value) + return result + + +def _stable_suffix(value: str) -> str: + return value.replace(":", "_").replace("/", "_").replace("\\", "_") + + +def _now_iso() -> str: + return datetime.now().isoformat(timespec="seconds") + + +__all__ = [ + "R_LOOP_VESSEL_READ_PACKET_DATA_TYPE", + "R_LOOP_VESSEL_EXACT_CHILD_EXPANSION_POLICY_ID", + "R_LOOP_VESSEL_SUMMARY_CHILD_EXPANSION_POLICY_ID", + "R_LOOP_VESSEL_READ_PACKET_GENERATOR", + "R_LOOP_VESSEL_READ_PACKET_POLICY_ID", + "R_LOOP_VESSEL_READ_PACKET_SCHEMA_NAME", + "R_LOOP_VESSEL_READ_PACKET_STATUSES", + "RLoopVesselEntryCandidateRecord", + "RLoopVesselReadPacketFrame", + "RLoopVesselSummaryCandidateRecord", + "RecordedRLoopVesselReadPacket", + "build_r_loop_vessel_read_packet_from_neo4j", + "r_loop_vessel_read_packet_id", + "record_r_loop_vessel_read_packet", +] diff --git a/songryeon_core/core/registry.py b/songryeon_core/core/registry.py index a8ca2c7..7852db7 100644 --- a/songryeon_core/core/registry.py +++ b/songryeon_core/core/registry.py @@ -117,6 +117,13 @@ def build_default_prompt_registry() -> PromptRegistry: purpose="절대정보 경계를 만든다.", system_prompt_ref="songryeon_core/prompts/node_2_metainfo_boundary_v0.md", ), + PromptRecord( + node_id="node_2_answer_basis", + prompt_name="answer_basis_selector", + version="0.1", + purpose="node_3 최종 답변의 근거 말하기 모드를 고른다.", + system_prompt_ref="songryeon_core/prompts/node_2_answer_basis_selector_v0.md", + ), PromptRecord( node_id="node_3", prompt_name="reporter", @@ -154,6 +161,28 @@ def build_default_schema_registry() -> SchemaRegistry: required=True, fields=["absolute_info"], ), + SchemaRecord( + schema_name="Node2AnswerBasisFrame", + version="0.1", + target_node="node_2_answer_basis", + required=True, + fields=[ + "frame_id", + "turn_id", + "answer_basis_mode", + "basis_reason_codes", + "mode_selection_reason", + "mode_selection_reason_info_class", + "evidence_roles", + "generated_by", + "info_class", + "semantic_judgement_status", + "source_trace_ids", + "source_data_ids", + "schema_name", + "schema_version", + ], + ), SchemaRecord( schema_name="Node2InputFrame", version="0.1", diff --git a/songryeon_core/core/schema_parts/__init__.py b/songryeon_core/core/schema_parts/__init__.py index fdf6189..880d9e2 100644 --- a/songryeon_core/core/schema_parts/__init__.py +++ b/songryeon_core/core/schema_parts/__init__.py @@ -5,6 +5,137 @@ NodeMovement, SchemaBinding, ) +from songryeon_core.core.schema_parts.graph_memory import ( + CORE_EGO_GUIDE_WORKER_HINT_FAILURE_TYPES, + CORE_EGO_GUIDE_WORKER_HINT_FRAME_SCHEMA_NAME, + CORE_EGO_GUIDE_WORKER_HINT_FRAME_SCHEMA_VERSION, + CORE_EGO_GUIDE_WORKER_HINT_STATUSES, + CORE_EGO_GUIDE_WORKER_PARSE_STATUSES, + CORE_EGO_TIME_AXIS_FRAME_SCHEMA_NAME, + CORE_EGO_TIME_AXIS_FRAME_SCHEMA_VERSION, + GRAPH_ACCESS_LEDGER_CODE_GENERATOR, + GRAPH_ACCESS_STAGES, + GRAPH_MEMORY_CODE_GENERATOR, + GRAPH_MEMORY_EDGE_FRAME_SCHEMA_NAME, + GRAPH_MEMORY_EDGE_FRAME_SCHEMA_VERSION, + GRAPH_MEMORY_EDGE_KINDS, + GRAPH_MEMORY_NODE_FRAME_SCHEMA_NAME, + GRAPH_MEMORY_NODE_FRAME_SCHEMA_VERSION, + GRAPH_MEMORY_NODE_KINDS, + GRAPH_MEMORY_SNAPSHOT_FRAME_SCHEMA_NAME, + GRAPH_MEMORY_SNAPSHOT_FRAME_SCHEMA_VERSION, + RLOOP_GRAPH_GUIDE_PACKET_FRAME_SCHEMA_NAME, + RLOOP_GRAPH_GUIDE_PACKET_FRAME_SCHEMA_VERSION, + RLOOP_GUIDE_CODE_GENERATOR, + R_LOOP_MEMORY_HANDOFF_PACKET_FRAME_SCHEMA_NAME, + R_LOOP_MEMORY_HANDOFF_PACKET_FRAME_SCHEMA_VERSION, + R_LOOP_MEMORY_HANDOFF_SEMANTIC_HINT_STATUSES, + R_LOOP_MEMORY_HANDOFF_STATUSES, + SOURCE_OBSERVATION_LEDGER_CODE_GENERATOR, + SOURCE_OBSERVATION_LEDGER_FRAME_SCHEMA_NAME, + SOURCE_OBSERVATION_LEDGER_FRAME_SCHEMA_VERSION, + SOURCE_OBSERVATION_LEDGER_STATUSES, + SOURCE_OBSERVATION_STATUSES, + SOURCE_VERSION_LINEAGE_CODE_GENERATOR, + SOURCE_VERSION_LINEAGE_FRAME_SCHEMA_NAME, + SOURCE_VERSION_LINEAGE_FRAME_SCHEMA_VERSION, + SOURCE_VERSION_LINEAGE_STATUSES, + SUMMARY_INVALIDATION_LEDGER_CODE_GENERATOR, + SUMMARY_INVALIDATION_LEDGER_FRAME_SCHEMA_NAME, + SUMMARY_INVALIDATION_LEDGER_FRAME_SCHEMA_VERSION, + SUMMARY_INVALIDATION_LEDGER_STATUSES, + NIGHT_SOURCE_LEAF_SUMMARY_FAILURE_TYPES, + NIGHT_SOURCE_LEAF_SUMMARY_FRAME_SCHEMA_NAME, + NIGHT_SOURCE_LEAF_SUMMARY_FRAME_SCHEMA_VERSION, + NIGHT_SOURCE_LEAF_SUMMARY_PARSE_STATUSES, + NIGHT_SOURCE_LEAF_SUMMARY_REVIEW_STATUSES, + NIGHT_SOURCE_LEAF_SUMMARY_STATUSES, + NIGHT_SOURCE_LEAF_SUMMARY_VALIDITY_STATUSES, + NIGHT_TIME_BUNDLE_SUMMARY_FAILURE_TYPES, + NIGHT_TIME_BUNDLE_SUMMARY_FRAME_SCHEMA_NAME, + NIGHT_TIME_BUNDLE_SUMMARY_FRAME_SCHEMA_VERSION, + NIGHT_TIME_BUNDLE_SUMMARY_PARSE_STATUSES, + NIGHT_TIME_BUNDLE_SUMMARY_REVIEW_STATUSES, + NIGHT_TIME_BUNDLE_SUMMARY_STATUSES, + NIGHT_TIME_BUNDLE_SUMMARY_VALIDITY_STATUSES, + TURN_ACTIVITY_GRAPH_LINK_ACTIVITY_KINDS, + TURN_ACTIVITY_GRAPH_LINK_CODE_GENERATOR, + TURN_ACTIVITY_GRAPH_LINK_FRAME_SCHEMA_NAME, + TURN_ACTIVITY_GRAPH_LINK_FRAME_SCHEMA_VERSION, + TURN_GRAPH_ACCESS_LEDGER_FRAME_SCHEMA_NAME, + TURN_GRAPH_ACCESS_LEDGER_FRAME_SCHEMA_VERSION, + CoreEgoGuideWorkerHintFrame, + CoreEgoTimeAxisFrame, + GraphMemoryEdgeFrame, + GraphMemoryNodeFrame, + GraphMemorySnapshotFrame, + NightSourceLeafSummaryFrame, + NightTimeBundleSummaryFrame, + RLoopGraphGuidePacketFrame, + RLoopMemoryHandoffPacketFrame, + SourceObservationLedgerFrame, + SourceVersionLineageFrame, + SummaryInvalidationLedgerFrame, + TurnActivityGraphLinkFrame, + TurnGraphAccessLedgerFrame, + validate_core_ego_guide_worker_hint_frame, + validate_core_ego_time_axis_frame, + validate_graph_memory_edge_frame, + validate_graph_memory_node_frame, + validate_graph_memory_snapshot_frame, + validate_night_source_leaf_summary_frame, + validate_night_time_bundle_summary_frame, + validate_r_loop_memory_handoff_packet_frame, + validate_rloop_graph_guide_packet_frame, + validate_source_observation_ledger_frame, + validate_source_version_lineage_frame, + validate_summary_invalidation_ledger_frame, + validate_turn_activity_graph_link_frame, + validate_turn_graph_access_ledger_frame, +) +from songryeon_core.core.schema_parts.r_loop import ( + R1GraphGoalFrame, + R2GraphNodeSelectionFrame, + R3GraphInspectionFrame, + RGraphTraversalCandidateSurfaceFrame, + RLoopBudgetFrame, + RLoopContinuationFrame, + RLoopReturnSummaryFrame, + R1_GRAPH_GOAL_FRAME_SCHEMA_NAME, + R1_GRAPH_GOAL_FRAME_SCHEMA_VERSION, + R2_GRAPH_NODE_SELECTION_FRAME_SCHEMA_NAME, + R2_GRAPH_NODE_SELECTION_FRAME_SCHEMA_VERSION, + R3_GRAPH_INSPECTION_FRAME_SCHEMA_NAME, + R3_GRAPH_INSPECTION_FRAME_SCHEMA_VERSION, + R_GRAPH_CANDIDATE_RELATIONS, + R_GRAPH_TRAVERSAL_CANDIDATE_SURFACE_FRAME_SCHEMA_NAME, + R_GRAPH_TRAVERSAL_CANDIDATE_SURFACE_FRAME_SCHEMA_VERSION, + R_GRAPH_TRAVERSAL_CANDIDATE_SURFACE_GENERATOR, + R_LOOP_BUDGET_FRAME_SCHEMA_NAME, + R_LOOP_BUDGET_FRAME_SCHEMA_VERSION, + R_LOOP_CONTINUATION_FRAME_SCHEMA_NAME, + R_LOOP_CONTINUATION_FRAME_SCHEMA_VERSION, + R_LOOP_CONTINUATION_STATUSES, + R_LOOP_RETURN_SUMMARY_FRAME_SCHEMA_NAME, + R_LOOP_RETURN_SUMMARY_FRAME_SCHEMA_VERSION, + R_LOOP_SCHEMA_ONLY_GENERATOR, + validate_r1_graph_goal_frame, + validate_r2_graph_node_selection_frame, + validate_r3_graph_inspection_frame, + validate_r_graph_traversal_candidate_surface_frame, + validate_r_loop_budget_frame, + validate_r_loop_continuation_frame, + validate_r_loop_return_summary_frame, +) +from songryeon_core.core.schema_parts.loop_activity import ( + L_LOOP_ACTIVITY_LEDGER_DATA_TYPE, + L_LOOP_ACTIVITY_LEDGER_FRAME_SCHEMA_NAME, + L_LOOP_ACTIVITY_LEDGER_FRAME_SCHEMA_VERSION, + L_LOOP_ACTIVITY_LEDGER_GENERATOR, + L_LOOP_ACTIVITY_STAGES, + LLoopActivityLedgerFrame, + validate_l_loop_activity_ledger_frame, +) from songryeon_core.core.schema_parts.task_ledger import ( TASK_FRAME_SCHEMA_NAME, TASK_FRAME_SCHEMA_VERSION, @@ -26,10 +157,105 @@ __all__ = [ "DataRef", + "CORE_EGO_TIME_AXIS_FRAME_SCHEMA_NAME", + "CORE_EGO_TIME_AXIS_FRAME_SCHEMA_VERSION", + "CORE_EGO_GUIDE_WORKER_HINT_FAILURE_TYPES", + "CORE_EGO_GUIDE_WORKER_HINT_FRAME_SCHEMA_NAME", + "CORE_EGO_GUIDE_WORKER_HINT_FRAME_SCHEMA_VERSION", + "CORE_EGO_GUIDE_WORKER_HINT_STATUSES", + "CORE_EGO_GUIDE_WORKER_PARSE_STATUSES", + "CoreEgoGuideWorkerHintFrame", + "CoreEgoTimeAxisFrame", + "GRAPH_ACCESS_LEDGER_CODE_GENERATOR", + "GRAPH_ACCESS_STAGES", + "GRAPH_MEMORY_CODE_GENERATOR", + "GRAPH_MEMORY_EDGE_FRAME_SCHEMA_NAME", + "GRAPH_MEMORY_EDGE_FRAME_SCHEMA_VERSION", + "GRAPH_MEMORY_EDGE_KINDS", + "GRAPH_MEMORY_NODE_FRAME_SCHEMA_NAME", + "GRAPH_MEMORY_NODE_FRAME_SCHEMA_VERSION", + "GRAPH_MEMORY_NODE_KINDS", + "GRAPH_MEMORY_SNAPSHOT_FRAME_SCHEMA_NAME", + "GRAPH_MEMORY_SNAPSHOT_FRAME_SCHEMA_VERSION", + "GraphMemoryEdgeFrame", + "GraphMemoryNodeFrame", + "GraphMemorySnapshotFrame", + "NightSourceLeafSummaryFrame", + "NightTimeBundleSummaryFrame", + "L_LOOP_ACTIVITY_LEDGER_DATA_TYPE", + "L_LOOP_ACTIVITY_LEDGER_FRAME_SCHEMA_NAME", + "L_LOOP_ACTIVITY_LEDGER_FRAME_SCHEMA_VERSION", + "L_LOOP_ACTIVITY_LEDGER_GENERATOR", + "L_LOOP_ACTIVITY_STAGES", + "LLoopActivityLedgerFrame", "MemoryPacketFrom0", "NodeMovement", + "RLOOP_GRAPH_GUIDE_PACKET_FRAME_SCHEMA_NAME", + "RLOOP_GRAPH_GUIDE_PACKET_FRAME_SCHEMA_VERSION", + "RLOOP_GUIDE_CODE_GENERATOR", + "R_LOOP_MEMORY_HANDOFF_PACKET_FRAME_SCHEMA_NAME", + "R_LOOP_MEMORY_HANDOFF_PACKET_FRAME_SCHEMA_VERSION", + "R_LOOP_MEMORY_HANDOFF_SEMANTIC_HINT_STATUSES", + "R_LOOP_MEMORY_HANDOFF_STATUSES", + "SOURCE_OBSERVATION_LEDGER_CODE_GENERATOR", + "SOURCE_OBSERVATION_LEDGER_FRAME_SCHEMA_NAME", + "SOURCE_OBSERVATION_LEDGER_FRAME_SCHEMA_VERSION", + "SOURCE_OBSERVATION_LEDGER_STATUSES", + "SOURCE_OBSERVATION_STATUSES", + "SOURCE_VERSION_LINEAGE_CODE_GENERATOR", + "SOURCE_VERSION_LINEAGE_FRAME_SCHEMA_NAME", + "SOURCE_VERSION_LINEAGE_FRAME_SCHEMA_VERSION", + "SOURCE_VERSION_LINEAGE_STATUSES", + "SUMMARY_INVALIDATION_LEDGER_CODE_GENERATOR", + "SUMMARY_INVALIDATION_LEDGER_FRAME_SCHEMA_NAME", + "SUMMARY_INVALIDATION_LEDGER_FRAME_SCHEMA_VERSION", + "SUMMARY_INVALIDATION_LEDGER_STATUSES", + "NIGHT_TIME_BUNDLE_SUMMARY_FAILURE_TYPES", + "NIGHT_TIME_BUNDLE_SUMMARY_FRAME_SCHEMA_NAME", + "NIGHT_TIME_BUNDLE_SUMMARY_FRAME_SCHEMA_VERSION", + "NIGHT_TIME_BUNDLE_SUMMARY_PARSE_STATUSES", + "NIGHT_TIME_BUNDLE_SUMMARY_REVIEW_STATUSES", + "NIGHT_TIME_BUNDLE_SUMMARY_STATUSES", + "NIGHT_TIME_BUNDLE_SUMMARY_VALIDITY_STATUSES", + "NIGHT_SOURCE_LEAF_SUMMARY_FAILURE_TYPES", + "NIGHT_SOURCE_LEAF_SUMMARY_FRAME_SCHEMA_NAME", + "NIGHT_SOURCE_LEAF_SUMMARY_FRAME_SCHEMA_VERSION", + "NIGHT_SOURCE_LEAF_SUMMARY_PARSE_STATUSES", + "NIGHT_SOURCE_LEAF_SUMMARY_REVIEW_STATUSES", + "NIGHT_SOURCE_LEAF_SUMMARY_STATUSES", + "NIGHT_SOURCE_LEAF_SUMMARY_VALIDITY_STATUSES", + "R1GraphGoalFrame", + "R1_GRAPH_GOAL_FRAME_SCHEMA_NAME", + "R1_GRAPH_GOAL_FRAME_SCHEMA_VERSION", + "R2GraphNodeSelectionFrame", + "R2_GRAPH_NODE_SELECTION_FRAME_SCHEMA_NAME", + "R2_GRAPH_NODE_SELECTION_FRAME_SCHEMA_VERSION", + "R3GraphInspectionFrame", + "R3_GRAPH_INSPECTION_FRAME_SCHEMA_NAME", + "R3_GRAPH_INSPECTION_FRAME_SCHEMA_VERSION", + "RGraphTraversalCandidateSurfaceFrame", + "R_GRAPH_CANDIDATE_RELATIONS", + "R_GRAPH_TRAVERSAL_CANDIDATE_SURFACE_FRAME_SCHEMA_NAME", + "R_GRAPH_TRAVERSAL_CANDIDATE_SURFACE_FRAME_SCHEMA_VERSION", + "R_GRAPH_TRAVERSAL_CANDIDATE_SURFACE_GENERATOR", + "RLoopBudgetFrame", + "RLoopContinuationFrame", + "RLoopGraphGuidePacketFrame", + "RLoopMemoryHandoffPacketFrame", + "RLoopReturnSummaryFrame", "RoutingDecision", + "R_LOOP_BUDGET_FRAME_SCHEMA_NAME", + "R_LOOP_BUDGET_FRAME_SCHEMA_VERSION", + "R_LOOP_CONTINUATION_FRAME_SCHEMA_NAME", + "R_LOOP_CONTINUATION_FRAME_SCHEMA_VERSION", + "R_LOOP_CONTINUATION_STATUSES", + "R_LOOP_RETURN_SUMMARY_FRAME_SCHEMA_NAME", + "R_LOOP_RETURN_SUMMARY_FRAME_SCHEMA_VERSION", + "R_LOOP_SCHEMA_ONLY_GENERATOR", "SchemaBinding", + "SourceObservationLedgerFrame", + "SourceVersionLineageFrame", + "SummaryInvalidationLedgerFrame", "TASK_FRAME_SCHEMA_NAME", "TASK_FRAME_SCHEMA_VERSION", "TASK_RESULT_FRAME_SCHEMA_NAME", @@ -37,9 +263,39 @@ "TaskFrame", "TaskResultFrame", "TraceEvent", + "TURN_ACTIVITY_GRAPH_LINK_ACTIVITY_KINDS", + "TURN_ACTIVITY_GRAPH_LINK_CODE_GENERATOR", + "TURN_ACTIVITY_GRAPH_LINK_FRAME_SCHEMA_NAME", + "TURN_ACTIVITY_GRAPH_LINK_FRAME_SCHEMA_VERSION", + "TURN_GRAPH_ACCESS_LEDGER_FRAME_SCHEMA_NAME", + "TURN_GRAPH_ACCESS_LEDGER_FRAME_SCHEMA_VERSION", + "TurnActivityGraphLinkFrame", "TurnStateCapsule", + "TurnGraphAccessLedgerFrame", "UnifiedState", "ZeroState", + "validate_core_ego_guide_worker_hint_frame", + "validate_core_ego_time_axis_frame", + "validate_graph_memory_edge_frame", + "validate_graph_memory_node_frame", + "validate_graph_memory_snapshot_frame", + "validate_night_source_leaf_summary_frame", + "validate_night_time_bundle_summary_frame", + "validate_l_loop_activity_ledger_frame", + "validate_r_loop_memory_handoff_packet_frame", + "validate_r1_graph_goal_frame", + "validate_r2_graph_node_selection_frame", + "validate_r3_graph_inspection_frame", + "validate_r_graph_traversal_candidate_surface_frame", + "validate_r_loop_budget_frame", + "validate_r_loop_continuation_frame", + "validate_r_loop_return_summary_frame", + "validate_rloop_graph_guide_packet_frame", + "validate_source_observation_ledger_frame", + "validate_source_version_lineage_frame", + "validate_summary_invalidation_ledger_frame", + "validate_turn_graph_access_ledger_frame", "validate_task_frame", "validate_task_result_frame", + "validate_turn_activity_graph_link_frame", ] diff --git a/songryeon_core/core/schema_parts/graph_memory.py b/songryeon_core/core/schema_parts/graph_memory.py new file mode 100644 index 0000000..bfeb5d4 --- /dev/null +++ b/songryeon_core/core/schema_parts/graph_memory.py @@ -0,0 +1,2060 @@ +from __future__ import annotations + +from dataclasses import dataclass, field + +from songryeon_core.core.schema_parts.base import ( + _validate_no_duplicates, + _validate_string_list, +) + + +GRAPH_MEMORY_NODE_FRAME_SCHEMA_NAME = "GraphMemoryNodeFrame" +GRAPH_MEMORY_NODE_FRAME_SCHEMA_VERSION = "0.1" +GRAPH_MEMORY_EDGE_FRAME_SCHEMA_NAME = "GraphMemoryEdgeFrame" +GRAPH_MEMORY_EDGE_FRAME_SCHEMA_VERSION = "0.1" +GRAPH_MEMORY_SNAPSHOT_FRAME_SCHEMA_NAME = "GraphMemorySnapshotFrame" +GRAPH_MEMORY_SNAPSHOT_FRAME_SCHEMA_VERSION = "0.1" +TURN_GRAPH_ACCESS_LEDGER_FRAME_SCHEMA_NAME = "TurnGraphAccessLedgerFrame" +TURN_GRAPH_ACCESS_LEDGER_FRAME_SCHEMA_VERSION = "0.1" +TURN_ACTIVITY_GRAPH_LINK_FRAME_SCHEMA_NAME = "TurnActivityGraphLinkFrame" +TURN_ACTIVITY_GRAPH_LINK_FRAME_SCHEMA_VERSION = "0.1" +CORE_EGO_TIME_AXIS_FRAME_SCHEMA_NAME = "CoreEgoTimeAxisFrame" +CORE_EGO_TIME_AXIS_FRAME_SCHEMA_VERSION = "0.1" +RLOOP_GRAPH_GUIDE_PACKET_FRAME_SCHEMA_NAME = "RLoopGraphGuidePacketFrame" +RLOOP_GRAPH_GUIDE_PACKET_FRAME_SCHEMA_VERSION = "0.1" +CORE_EGO_GUIDE_WORKER_HINT_FRAME_SCHEMA_NAME = "CoreEgoGuideWorkerHintFrame" +CORE_EGO_GUIDE_WORKER_HINT_FRAME_SCHEMA_VERSION = "0.1" +R_LOOP_MEMORY_HANDOFF_PACKET_FRAME_SCHEMA_NAME = "RLoopMemoryHandoffPacketFrame" +R_LOOP_MEMORY_HANDOFF_PACKET_FRAME_SCHEMA_VERSION = "0.1" +SOURCE_VERSION_LINEAGE_FRAME_SCHEMA_NAME = "SourceVersionLineageFrame" +SOURCE_VERSION_LINEAGE_FRAME_SCHEMA_VERSION = "0.1" +SOURCE_OBSERVATION_LEDGER_FRAME_SCHEMA_NAME = "SourceObservationLedgerFrame" +SOURCE_OBSERVATION_LEDGER_FRAME_SCHEMA_VERSION = "0.1" +SUMMARY_INVALIDATION_LEDGER_FRAME_SCHEMA_NAME = "SummaryInvalidationLedgerFrame" +SUMMARY_INVALIDATION_LEDGER_FRAME_SCHEMA_VERSION = "0.1" +NIGHT_TIME_BUNDLE_SUMMARY_FRAME_SCHEMA_NAME = "NightTimeBundleSummaryFrame" +NIGHT_TIME_BUNDLE_SUMMARY_FRAME_SCHEMA_VERSION = "0.1" +NIGHT_SOURCE_LEAF_SUMMARY_FRAME_SCHEMA_NAME = "NightSourceLeafSummaryFrame" +NIGHT_SOURCE_LEAF_SUMMARY_FRAME_SCHEMA_VERSION = "0.1" + +GRAPH_MEMORY_NODE_KINDS = { + "raw_capsule", + "raw_bundle", + "summary", + "core_ego", + "time_axis", + "time_bundle", + "activity_ledger", + "raw_source", + "source_ingest_time_bundle", + "source_kind_bundle", + "token_budget_summary_bundle", +} +GRAPH_MEMORY_EDGE_KINDS = { + "CONTAINS", + "CHILD_OF_TIME_AXIS", + "NEXT", + "HAS_ACTIVITY_LEDGER", + "SOURCE_OF", + "SUMMARY_OF", +} +GRAPH_MEMORY_CODE_GENERATOR = "CODE:GRAPH_MEMORY_BUILDER" +GRAPH_ACCESS_LEDGER_CODE_GENERATOR = "CODE:GRAPH_ACCESS_LEDGER" +TURN_ACTIVITY_GRAPH_LINK_CODE_GENERATOR = "CODE:TURN_ACTIVITY_GRAPH_LINK_BUILDER" +RLOOP_GUIDE_CODE_GENERATOR = "CODE:GRAPH_MEMORY_GUIDE_BUILDER" +GRAPH_ACCESS_STAGES = { + "candidate_seen", + "selected", + "inspected", + "read", + "used_as_answer_source", +} +CORE_EGO_GUIDE_WORKER_HINT_FAILURE_TYPES = { + "none", + "adapter_missing", + "adapter_failed", + "parse_failed", + "schema_failed", +} +CORE_EGO_GUIDE_WORKER_HINT_STATUSES = {"ran", "failed"} +CORE_EGO_GUIDE_WORKER_PARSE_STATUSES = {"passed", "failed", "not_checked"} +R_LOOP_MEMORY_HANDOFF_STATUSES = {"available", "missing"} +R_LOOP_MEMORY_HANDOFF_SEMANTIC_HINT_STATUSES = {"not_run", "ran", "failed"} +TURN_ACTIVITY_GRAPH_LINK_ACTIVITY_KINDS = { + "l_loop_activity_ledger", + "r_graph_access_ledger", +} +SOURCE_VERSION_LINEAGE_CODE_GENERATOR = "CODE:SOURCE_VERSION_LINEAGE_BUILDER" +SOURCE_OBSERVATION_LEDGER_CODE_GENERATOR = "CODE:SOURCE_OBSERVATION_LEDGER_BUILDER" +SUMMARY_INVALIDATION_LEDGER_CODE_GENERATOR = "CODE:SUMMARY_INVALIDATION_LEDGER_BUILDER" +SOURCE_VERSION_LINEAGE_STATUSES = { + "single_version", + "content_changed", +} +SOURCE_OBSERVATION_LEDGER_STATUSES = {"no_observations", "recorded"} +SOURCE_OBSERVATION_STATUSES = { + "new_source_version", + "unchanged", + "content_changed", +} +SUMMARY_INVALIDATION_LEDGER_STATUSES = { + "no_invalidations", + "invalidations_recorded", +} +NIGHT_TIME_BUNDLE_SUMMARY_STATUSES = {"ran", "failed"} +NIGHT_TIME_BUNDLE_SUMMARY_FAILURE_TYPES = { + "none", + "adapter_missing", + "adapter_failed", + "parse_failed", + "schema_failed", +} +NIGHT_TIME_BUNDLE_SUMMARY_PARSE_STATUSES = {"passed", "failed", "not_checked"} +NIGHT_TIME_BUNDLE_SUMMARY_VALIDITY_STATUSES = { + "active", + "invalidated_by_source_change", +} +NIGHT_TIME_BUNDLE_SUMMARY_REVIEW_STATUSES = { + "not_reviewed", + "approved", + "rejected", +} +NIGHT_SOURCE_LEAF_SUMMARY_STATUSES = { + "ran", + "failed", + "skipped_no_text_snapshot", + "skipped_empty_text", +} +NIGHT_SOURCE_LEAF_SUMMARY_FAILURE_TYPES = { + "none", + "adapter_missing", + "adapter_failed", + "parse_failed", + "schema_failed", + "no_text_snapshot", + "empty_text", +} +NIGHT_SOURCE_LEAF_SUMMARY_PARSE_STATUSES = {"passed", "failed", "not_checked"} +NIGHT_SOURCE_LEAF_SUMMARY_VALIDITY_STATUSES = { + "active", + "invalidated_by_source_change", +} +NIGHT_SOURCE_LEAF_SUMMARY_REVIEW_STATUSES = { + "not_reviewed", + "approved", + "rejected", +} + + +@dataclass +class GraphMemoryNodeFrame: + """A graph-memory node that keeps source coordinates, not semantic memory text.""" + + node_id: str + node_kind: str + data_kind: str + source_turn_id: str | None = None + trace_count: int = 0 + movement_count: int = 0 + user_input_trace_id: str | None = None + final_response_trace_id: str | None = None + summary_depth: int = 0 + source_depth_min: int = 0 + source_depth_max: int = 0 + source_leaf_count: int = 0 + source_summary_count: int = 0 + source_bundle_kind: str = "none" + bundle_policy_id: str = "not_applicable" + char_budget: int | None = None + source_char_count: int = 0 + char_budget_status: str = "not_applicable" + observed_at: str | None = None + ingested_at: str | None = None + source_last_modified_at: str | None = None + exists_at_ingest: bool | None = None + content_sha1: str | None = None + source_graph_node_ids: list[str] = field(default_factory=list) + source_trace_ids: list[str] = field(default_factory=list) + source_data_ids: list[str] = field(default_factory=list) + generated_by: str = GRAPH_MEMORY_CODE_GENERATOR + info_class: str = "absolute" + semantic_judgement_status: str = "not_run" + schema_name: str = GRAPH_MEMORY_NODE_FRAME_SCHEMA_NAME + schema_version: str = GRAPH_MEMORY_NODE_FRAME_SCHEMA_VERSION + + +@dataclass +class NightTimeBundleSummaryFrame: + """An LLM-generated summary graph node attached to one TimeBundle target.""" + + frame_id: str + summary_graph_node_id: str + target_graph_node_id: str + target_node_kind: str + summary_text: str = "" + summary_status: str = "ran" + failure_type: str = "none" + payload_parse_status: str = "passed" + node_kind: str = "summary" + data_kind: str = "time_bundle_summary" + summary_depth: int = 1 + source_depth_min: int = 0 + source_depth_max: int = 0 + source_leaf_count: int = 0 + source_summary_count: int = 0 + source_bundle_kind: str = "time_bundle" + validity_status: str = "active" + review_status: str = "not_reviewed" + llm_call_data_id: str | None = None + llm_trace_event_id: str | None = None + prompt_ref: str = "songryeon_core/prompts/night_summarize_time_bundle_v0.md" + source_mode: str = "source_bundle" + claim_alignment: str = "multi_source_bundle" + source_graph_node_ids: list[str] = field(default_factory=list) + source_trace_ids: list[str] = field(default_factory=list) + source_data_ids: list[str] = field(default_factory=list) + generated_by: str = "LLM:unknown:night_summarize_time_bundle" + info_class: str = "mixed" + semantic_judgement_status: str = "ran" + schema_name: str = NIGHT_TIME_BUNDLE_SUMMARY_FRAME_SCHEMA_NAME + schema_version: str = NIGHT_TIME_BUNDLE_SUMMARY_FRAME_SCHEMA_VERSION + + +@dataclass +class NightSourceLeafSummaryFrame: + """An LLM-generated summary attached to exactly one raw_source leaf.""" + + frame_id: str + summary_graph_node_id: str + target_graph_node_id: str + target_node_kind: str + source_kind: str + source_path: str + source_file_data_id: str + content_sha1: str + text_snapshot_data_id: str | None = None + summary_text: str = "" + summary_status: str = "ran" + failure_type: str = "none" + payload_parse_status: str = "passed" + node_kind: str = "summary" + data_kind: str = "source_leaf_summary" + summary_depth: int = 1 + source_depth_min: int = 0 + source_depth_max: int = 0 + source_leaf_count: int = 1 + source_summary_count: int = 0 + source_bundle_kind: str = "raw_source" + validity_status: str = "active" + review_status: str = "not_reviewed" + llm_call_data_id: str | None = None + llm_trace_event_id: str | None = None + prompt_ref: str = "songryeon_core/prompts/night_summarize_source_leaf_v0.md" + source_mode: str = "single_source" + claim_alignment: str = "single_absolute_record" + source_graph_node_ids: list[str] = field(default_factory=list) + source_trace_ids: list[str] = field(default_factory=list) + source_data_ids: list[str] = field(default_factory=list) + generated_by: str = "LLM:unknown:night_summarize_source_leaf" + info_class: str = "relative" + semantic_judgement_status: str = "ran" + schema_name: str = NIGHT_SOURCE_LEAF_SUMMARY_FRAME_SCHEMA_NAME + schema_version: str = NIGHT_SOURCE_LEAF_SUMMARY_FRAME_SCHEMA_VERSION + + +@dataclass +class GraphMemoryEdgeFrame: + """A graph-memory edge with deterministic source and target node coordinates.""" + + edge_id: str + edge_kind: str + from_node_id: str + to_node_id: str + source_graph_node_ids: list[str] = field(default_factory=list) + source_trace_ids: list[str] = field(default_factory=list) + source_data_ids: list[str] = field(default_factory=list) + generated_by: str = GRAPH_MEMORY_CODE_GENERATOR + info_class: str = "absolute" + semantic_judgement_status: str = "not_run" + schema_name: str = GRAPH_MEMORY_EDGE_FRAME_SCHEMA_NAME + schema_version: str = GRAPH_MEMORY_EDGE_FRAME_SCHEMA_VERSION + + +@dataclass +class GraphMemorySnapshotFrame: + """The code-checkable graph-memory node and edge set for one batch.""" + + snapshot_id: str + batch_id: str + root_node_id: str + time_axis_node_id: str + graph_node_ids: list[str] = field(default_factory=list) + graph_edge_ids: list[str] = field(default_factory=list) + node_kind_counts: dict[str, int] = field(default_factory=dict) + edge_kind_counts: dict[str, int] = field(default_factory=dict) + data_kind_counts: dict[str, int] = field(default_factory=dict) + source_graph_node_ids: list[str] = field(default_factory=list) + source_trace_ids: list[str] = field(default_factory=list) + source_data_ids: list[str] = field(default_factory=list) + generated_by: str = GRAPH_MEMORY_CODE_GENERATOR + info_class: str = "absolute" + semantic_judgement_status: str = "not_run" + schema_name: str = GRAPH_MEMORY_SNAPSHOT_FRAME_SCHEMA_NAME + schema_version: str = GRAPH_MEMORY_SNAPSHOT_FRAME_SCHEMA_VERSION + + +@dataclass +class TurnGraphAccessLedgerFrame: + """A per-turn absolute ledger of graph nodes seen by an R traversal.""" + + frame_id: str + turn_id: str + turn_capsule_graph_node_id: str + candidate_graph_node_ids: list[str] = field(default_factory=list) + selected_graph_node_ids: list[str] = field(default_factory=list) + inspected_graph_node_ids: list[str] = field(default_factory=list) + read_graph_node_ids: list[str] = field(default_factory=list) + used_as_answer_source_graph_node_ids: list[str] = field(default_factory=list) + access_records: list[dict[str, str]] = field(default_factory=list) + source_trace_ids: list[str] = field(default_factory=list) + source_data_ids: list[str] = field(default_factory=list) + generated_by: str = GRAPH_ACCESS_LEDGER_CODE_GENERATOR + info_class: str = "absolute" + semantic_judgement_status: str = "not_run" + schema_name: str = TURN_GRAPH_ACCESS_LEDGER_FRAME_SCHEMA_NAME + schema_version: str = TURN_GRAPH_ACCESS_LEDGER_FRAME_SCHEMA_VERSION + + +@dataclass +class TurnActivityGraphLinkFrame: + """A code-generated graph link index from one raw capsule to activity ledgers.""" + + frame_id: str + turn_id: str + turn_capsule_graph_node_id: str + l_loop_activity_ledger_data_ids: list[str] = field(default_factory=list) + r_graph_access_ledger_data_ids: list[str] = field(default_factory=list) + activity_ledger_graph_node_ids: list[str] = field(default_factory=list) + activity_ledger_graph_edge_ids: list[str] = field(default_factory=list) + link_records: list[dict[str, str]] = field(default_factory=list) + source_trace_ids: list[str] = field(default_factory=list) + source_data_ids: list[str] = field(default_factory=list) + generated_by: str = TURN_ACTIVITY_GRAPH_LINK_CODE_GENERATOR + info_class: str = "absolute" + semantic_judgement_status: str = "not_run" + schema_name: str = TURN_ACTIVITY_GRAPH_LINK_FRAME_SCHEMA_NAME + schema_version: str = TURN_ACTIVITY_GRAPH_LINK_FRAME_SCHEMA_VERSION + + +@dataclass +class CoreEgoTimeAxisFrame: + """The first CoreEgo graph entry surface: root to time axis only.""" + + frame_id: str + batch_id: str + core_ego_node_id: str + time_axis_node_id: str + time_bundle_node_ids: list[str] = field(default_factory=list) + raw_capsule_node_ids: list[str] = field(default_factory=list) + edge_ids: list[str] = field(default_factory=list) + source_graph_node_ids: list[str] = field(default_factory=list) + source_trace_ids: list[str] = field(default_factory=list) + source_data_ids: list[str] = field(default_factory=list) + generated_by: str = GRAPH_MEMORY_CODE_GENERATOR + info_class: str = "absolute" + semantic_judgement_status: str = "not_run" + semantic_axis_status: str = "not_created" + schema_name: str = CORE_EGO_TIME_AXIS_FRAME_SCHEMA_NAME + schema_version: str = CORE_EGO_TIME_AXIS_FRAME_SCHEMA_VERSION + + +@dataclass +class RLoopGraphGuidePacketFrame: + """A code-generated guide packet for a future R loop graph traversal.""" + + packet_id: str + graph_snapshot_id: str + target_consumer: str = "R_LOOP" + available_entry_nodes: list[str] = field(default_factory=list) + node_kind_counts: dict[str, int] = field(default_factory=dict) + data_kind_counts: dict[str, int] = field(default_factory=dict) + summary_depth_range: list[int] = field(default_factory=lambda: [0, 0]) + source_leaf_count_range: list[int] = field(default_factory=lambda: [0, 0]) + risky_or_unreviewed_node_ids: list[str] = field(default_factory=list) + recommended_traversal_hints: list[str] = field(default_factory=list) + recommended_traversal_hints_status: str = "not_run" + source_graph_node_ids: list[str] = field(default_factory=list) + source_trace_ids: list[str] = field(default_factory=list) + source_data_ids: list[str] = field(default_factory=list) + generated_by: str = RLOOP_GUIDE_CODE_GENERATOR + info_class: str = "absolute" + semantic_judgement_status: str = "not_run" + schema_name: str = RLOOP_GRAPH_GUIDE_PACKET_FRAME_SCHEMA_NAME + schema_version: str = RLOOP_GRAPH_GUIDE_PACKET_FRAME_SCHEMA_VERSION + + +@dataclass +class CoreEgoGuideWorkerHintFrame: + """An LLM-generated traversal hint over a code-generated graph guide packet.""" + + frame_id: str + source_rloop_graph_guide_packet_id: str + graph_snapshot_id: str + available_entry_node_ids: list[str] = field(default_factory=list) + available_source_graph_node_ids: list[str] = field(default_factory=list) + recommended_entry_node_ids: list[str] = field(default_factory=list) + avoid_entry_node_ids: list[str] = field(default_factory=list) + traversal_strategy_hint: str = "" + reason_summary: str = "" + risk_notes: list[str] = field(default_factory=list) + expected_depth_policy: str = "" + hint_status: str = "failed" + failure_type: str = "adapter_missing" + payload_parse_status: str = "not_checked" + llm_call_data_id: str | None = None + llm_trace_event_id: str | None = None + prompt_ref: str = "" + source_graph_node_ids: list[str] = field(default_factory=list) + source_trace_ids: list[str] = field(default_factory=list) + source_data_ids: list[str] = field(default_factory=list) + generated_by: str = "LLM:unknown:core_ego_guide_worker" + info_class: str = "mixed" + source_mode: str = "source_bundle" + claim_alignment: str = "multi_source_bundle" + semantic_judgement_status: str = "failed" + schema_name: str = CORE_EGO_GUIDE_WORKER_HINT_FRAME_SCHEMA_NAME + schema_version: str = CORE_EGO_GUIDE_WORKER_HINT_FRAME_SCHEMA_VERSION + + +@dataclass +class RLoopMemoryHandoffPacketFrame: + """A node_0 handoff packet that copies graph guide coordinates for a future R loop.""" + + packet_id: str + target: str = "R_LOOP" + mode: str = "graph_guide_handoff" + packet_status: str = "missing" + graph_snapshot_id: str = "" + r_loop_graph_guide_packet_id: str = "" + available_entry_node_ids: list[str] = field(default_factory=list) + node_kind_counts: dict[str, int] = field(default_factory=dict) + data_kind_counts: dict[str, int] = field(default_factory=dict) + summary_depth_range: list[int] = field(default_factory=lambda: [0, 0]) + source_leaf_count_range: list[int] = field(default_factory=lambda: [0, 0]) + semantic_hint_status: str = "not_run" + semantic_hint_frame_id: str | None = None + source_graph_node_ids: list[str] = field(default_factory=list) + source_trace_ids: list[str] = field(default_factory=list) + source_data_ids: list[str] = field(default_factory=list) + generated_by: str = "CODE:node_0_memory_supplier" + info_class: str = "absolute" + semantic_judgement_status: str = "not_run" + schema_name: str = R_LOOP_MEMORY_HANDOFF_PACKET_FRAME_SCHEMA_NAME + schema_version: str = R_LOOP_MEMORY_HANDOFF_PACKET_FRAME_SCHEMA_VERSION + + +@dataclass +class SourceVersionLineageFrame: + """A code-generated version lineage for one source_kind + path identity.""" + + frame_id: str + source_identity_key: str + source_kind: str + path: str + lineage_status: str + active_source_graph_node_id: str + version_source_graph_node_ids: list[str] = field(default_factory=list) + superseded_source_graph_node_ids: list[str] = field(default_factory=list) + source_file_data_ids: list[str] = field(default_factory=list) + version_records: list[dict[str, str]] = field(default_factory=list) + content_sha1_by_version: dict[str, str] = field(default_factory=dict) + observed_at_by_version: dict[str, str] = field(default_factory=dict) + source_graph_node_ids: list[str] = field(default_factory=list) + source_trace_ids: list[str] = field(default_factory=list) + source_data_ids: list[str] = field(default_factory=list) + generated_by: str = SOURCE_VERSION_LINEAGE_CODE_GENERATOR + info_class: str = "absolute" + semantic_judgement_status: str = "not_run" + schema_name: str = SOURCE_VERSION_LINEAGE_FRAME_SCHEMA_NAME + schema_version: str = SOURCE_VERSION_LINEAGE_FRAME_SCHEMA_VERSION + + +@dataclass +class SourceObservationLedgerFrame: + """A code-generated ledger of source checks that do not always create versions.""" + + frame_id: str + batch_id: str + ledger_status: str + observation_records: list[dict[str, str]] = field(default_factory=list) + observation_status_counts: dict[str, int] = field(default_factory=dict) + observed_source_file_data_ids: list[str] = field(default_factory=list) + active_source_graph_node_ids: list[str] = field(default_factory=list) + source_graph_node_ids: list[str] = field(default_factory=list) + source_trace_ids: list[str] = field(default_factory=list) + source_data_ids: list[str] = field(default_factory=list) + generated_by: str = SOURCE_OBSERVATION_LEDGER_CODE_GENERATOR + info_class: str = "absolute" + semantic_judgement_status: str = "not_run" + schema_name: str = SOURCE_OBSERVATION_LEDGER_FRAME_SCHEMA_NAME + schema_version: str = SOURCE_OBSERVATION_LEDGER_FRAME_SCHEMA_VERSION + + +@dataclass +class SummaryInvalidationLedgerFrame: + """A code-generated ledger of summaries invalidated by source changes.""" + + frame_id: str + batch_id: str + ledger_status: str + invalidated_summary_node_ids: list[str] = field(default_factory=list) + invalidation_records: list[dict[str, str]] = field(default_factory=list) + changed_source_lineage_frame_ids: list[str] = field(default_factory=list) + changed_source_graph_node_ids: list[str] = field(default_factory=list) + active_source_graph_node_ids: list[str] = field(default_factory=list) + source_graph_node_ids: list[str] = field(default_factory=list) + source_trace_ids: list[str] = field(default_factory=list) + source_data_ids: list[str] = field(default_factory=list) + generated_by: str = SUMMARY_INVALIDATION_LEDGER_CODE_GENERATOR + info_class: str = "absolute" + semantic_judgement_status: str = "not_run" + schema_name: str = SUMMARY_INVALIDATION_LEDGER_FRAME_SCHEMA_NAME + schema_version: str = SUMMARY_INVALIDATION_LEDGER_FRAME_SCHEMA_VERSION + + +def validate_night_time_bundle_summary_frame(frame: NightTimeBundleSummaryFrame) -> None: + _require_text_fields( + "NightTimeBundleSummaryFrame", + { + "frame_id": frame.frame_id, + "summary_graph_node_id": frame.summary_graph_node_id, + "target_graph_node_id": frame.target_graph_node_id, + "target_node_kind": frame.target_node_kind, + "summary_status": frame.summary_status, + "failure_type": frame.failure_type, + "payload_parse_status": frame.payload_parse_status, + "node_kind": frame.node_kind, + "data_kind": frame.data_kind, + "source_bundle_kind": frame.source_bundle_kind, + "validity_status": frame.validity_status, + "review_status": frame.review_status, + "prompt_ref": frame.prompt_ref, + "source_mode": frame.source_mode, + "claim_alignment": frame.claim_alignment, + "generated_by": frame.generated_by, + "info_class": frame.info_class, + "semantic_judgement_status": frame.semantic_judgement_status, + "schema_name": frame.schema_name, + "schema_version": frame.schema_version, + }, + ) + if frame.schema_name != NIGHT_TIME_BUNDLE_SUMMARY_FRAME_SCHEMA_NAME: + raise ValueError( + f"unknown NightTimeBundleSummaryFrame.schema_name: {frame.schema_name}" + ) + if frame.schema_version != NIGHT_TIME_BUNDLE_SUMMARY_FRAME_SCHEMA_VERSION: + raise ValueError( + f"unknown NightTimeBundleSummaryFrame.schema_version: {frame.schema_version}" + ) + if frame.node_kind != "summary": + raise ValueError("NightTimeBundleSummaryFrame.node_kind must be summary") + if frame.target_node_kind != "time_bundle": + raise ValueError("NightTimeBundleSummaryFrame target_node_kind must be time_bundle") + if not frame.summary_graph_node_id.startswith("graph:summary:"): + raise ValueError( + "NightTimeBundleSummaryFrame.summary_graph_node_id must be a graph summary id" + ) + if not frame.generated_by.startswith("LLM:"): + raise ValueError("NightTimeBundleSummaryFrame.generated_by must be LLM") + if frame.summary_status not in NIGHT_TIME_BUNDLE_SUMMARY_STATUSES: + raise ValueError( + f"unknown NightTimeBundleSummaryFrame.summary_status: {frame.summary_status}" + ) + if frame.failure_type not in NIGHT_TIME_BUNDLE_SUMMARY_FAILURE_TYPES: + raise ValueError( + f"unknown NightTimeBundleSummaryFrame.failure_type: {frame.failure_type}" + ) + if frame.payload_parse_status not in NIGHT_TIME_BUNDLE_SUMMARY_PARSE_STATUSES: + raise ValueError( + "NightTimeBundleSummaryFrame.payload_parse_status is invalid" + ) + if frame.validity_status not in NIGHT_TIME_BUNDLE_SUMMARY_VALIDITY_STATUSES: + raise ValueError("NightTimeBundleSummaryFrame.validity_status is invalid") + if frame.review_status not in NIGHT_TIME_BUNDLE_SUMMARY_REVIEW_STATUSES: + raise ValueError("NightTimeBundleSummaryFrame.review_status is invalid") + if frame.info_class not in {"relative", "mixed"}: + raise ValueError("NightTimeBundleSummaryFrame.info_class must be relative or mixed") + if frame.semantic_judgement_status not in {"ran", "failed"}: + raise ValueError( + "NightTimeBundleSummaryFrame.semantic_judgement_status must be ran or failed" + ) + + _validate_non_negative_ints( + "NightTimeBundleSummaryFrame", + { + "summary_depth": frame.summary_depth, + "source_depth_min": frame.source_depth_min, + "source_depth_max": frame.source_depth_max, + "source_leaf_count": frame.source_leaf_count, + "source_summary_count": frame.source_summary_count, + }, + ) + if frame.summary_depth < 1: + raise ValueError("NightTimeBundleSummaryFrame.summary_depth must be >= 1") + if frame.source_depth_min > frame.source_depth_max: + raise ValueError("NightTimeBundleSummaryFrame source depth range is inverted") + + for field_name, values in { + "source_graph_node_ids": frame.source_graph_node_ids, + "source_trace_ids": frame.source_trace_ids, + "source_data_ids": frame.source_data_ids, + }.items(): + _validate_string_list(f"NightTimeBundleSummaryFrame.{field_name}", values) + _validate_no_duplicates(f"NightTimeBundleSummaryFrame.{field_name}", values) + + if frame.target_graph_node_id not in frame.source_data_ids: + raise ValueError( + "NightTimeBundleSummaryFrame.source_data_ids must include target_graph_node_id" + ) + for source_graph_node_id in frame.source_graph_node_ids: + if source_graph_node_id not in frame.source_data_ids: + raise ValueError( + "NightTimeBundleSummaryFrame.source_data_ids must include source graph nodes" + ) + + expected_info_class = ( + "relative" + if frame.source_leaf_count == 1 + and frame.source_summary_count == 0 + and len(frame.source_graph_node_ids) == 1 + else "mixed" + ) + if frame.info_class != expected_info_class: + raise ValueError( + "NightTimeBundleSummaryFrame.info_class must follow source cardinality" + ) + expected_source_mode = ( + "single_source" if expected_info_class == "relative" else "source_bundle" + ) + expected_claim_alignment = ( + "single_absolute_record" + if expected_info_class == "relative" + else "multi_source_bundle" + ) + if frame.source_mode != expected_source_mode: + raise ValueError( + "NightTimeBundleSummaryFrame.source_mode must follow source cardinality" + ) + if frame.claim_alignment != expected_claim_alignment: + raise ValueError( + "NightTimeBundleSummaryFrame.claim_alignment must follow source cardinality" + ) + + if frame.summary_status == "ran": + if not frame.summary_text.strip(): + raise ValueError("ran NightTimeBundleSummaryFrame must include summary_text") + if frame.failure_type != "none": + raise ValueError("ran NightTimeBundleSummaryFrame failure_type must be none") + if frame.payload_parse_status != "passed": + raise ValueError( + "ran NightTimeBundleSummaryFrame payload_parse_status must be passed" + ) + if frame.semantic_judgement_status != "ran": + raise ValueError( + "ran NightTimeBundleSummaryFrame semantic_judgement_status must be ran" + ) + if frame.llm_call_data_id is None or frame.llm_call_data_id not in frame.source_data_ids: + raise ValueError( + "ran NightTimeBundleSummaryFrame.source_data_ids must include llm_call_data_id" + ) + else: + if frame.summary_text.strip(): + raise ValueError("failed NightTimeBundleSummaryFrame must not include summary_text") + if frame.failure_type == "none": + raise ValueError("failed NightTimeBundleSummaryFrame failure_type must not be none") + if frame.semantic_judgement_status != "failed": + raise ValueError( + "failed NightTimeBundleSummaryFrame semantic_judgement_status must be failed" + ) + + +def validate_night_source_leaf_summary_frame(frame: NightSourceLeafSummaryFrame) -> None: + _require_text_fields( + "NightSourceLeafSummaryFrame", + { + "frame_id": frame.frame_id, + "summary_graph_node_id": frame.summary_graph_node_id, + "target_graph_node_id": frame.target_graph_node_id, + "target_node_kind": frame.target_node_kind, + "source_kind": frame.source_kind, + "source_path": frame.source_path, + "source_file_data_id": frame.source_file_data_id, + "content_sha1": frame.content_sha1, + "summary_status": frame.summary_status, + "failure_type": frame.failure_type, + "payload_parse_status": frame.payload_parse_status, + "node_kind": frame.node_kind, + "data_kind": frame.data_kind, + "source_bundle_kind": frame.source_bundle_kind, + "validity_status": frame.validity_status, + "review_status": frame.review_status, + "prompt_ref": frame.prompt_ref, + "source_mode": frame.source_mode, + "claim_alignment": frame.claim_alignment, + "generated_by": frame.generated_by, + "info_class": frame.info_class, + "semantic_judgement_status": frame.semantic_judgement_status, + "schema_name": frame.schema_name, + "schema_version": frame.schema_version, + }, + ) + if frame.schema_name != NIGHT_SOURCE_LEAF_SUMMARY_FRAME_SCHEMA_NAME: + raise ValueError( + f"unknown NightSourceLeafSummaryFrame.schema_name: {frame.schema_name}" + ) + if frame.schema_version != NIGHT_SOURCE_LEAF_SUMMARY_FRAME_SCHEMA_VERSION: + raise ValueError( + f"unknown NightSourceLeafSummaryFrame.schema_version: {frame.schema_version}" + ) + if frame.node_kind != "summary": + raise ValueError("NightSourceLeafSummaryFrame.node_kind must be summary") + if frame.data_kind != "source_leaf_summary": + raise ValueError( + "NightSourceLeafSummaryFrame.data_kind must be source_leaf_summary" + ) + if frame.target_node_kind != "raw_source": + raise ValueError( + "NightSourceLeafSummaryFrame target_node_kind must be raw_source" + ) + if not frame.summary_graph_node_id.startswith("graph:summary:source_leaf:"): + raise ValueError( + "NightSourceLeafSummaryFrame.summary_graph_node_id must be a source leaf summary id" + ) + if frame.summary_status not in NIGHT_SOURCE_LEAF_SUMMARY_STATUSES: + raise ValueError( + f"unknown NightSourceLeafSummaryFrame.summary_status: {frame.summary_status}" + ) + if frame.failure_type not in NIGHT_SOURCE_LEAF_SUMMARY_FAILURE_TYPES: + raise ValueError( + f"unknown NightSourceLeafSummaryFrame.failure_type: {frame.failure_type}" + ) + if frame.payload_parse_status not in NIGHT_SOURCE_LEAF_SUMMARY_PARSE_STATUSES: + raise ValueError( + "NightSourceLeafSummaryFrame.payload_parse_status is invalid" + ) + if frame.validity_status not in NIGHT_SOURCE_LEAF_SUMMARY_VALIDITY_STATUSES: + raise ValueError("NightSourceLeafSummaryFrame.validity_status is invalid") + if frame.review_status not in NIGHT_SOURCE_LEAF_SUMMARY_REVIEW_STATUSES: + raise ValueError("NightSourceLeafSummaryFrame.review_status is invalid") + + _validate_non_negative_ints( + "NightSourceLeafSummaryFrame", + { + "summary_depth": frame.summary_depth, + "source_depth_min": frame.source_depth_min, + "source_depth_max": frame.source_depth_max, + "source_leaf_count": frame.source_leaf_count, + "source_summary_count": frame.source_summary_count, + }, + ) + if frame.summary_depth != 1: + raise ValueError("NightSourceLeafSummaryFrame.summary_depth must be 1") + if frame.source_depth_min != 0 or frame.source_depth_max != 0: + raise ValueError("NightSourceLeafSummaryFrame source depth must be 0") + if frame.source_leaf_count != 1: + raise ValueError("NightSourceLeafSummaryFrame.source_leaf_count must be 1") + if frame.source_summary_count != 0: + raise ValueError("NightSourceLeafSummaryFrame.source_summary_count must be 0") + if frame.source_bundle_kind != "raw_source": + raise ValueError( + "NightSourceLeafSummaryFrame.source_bundle_kind must be raw_source" + ) + if frame.source_mode != "single_source": + raise ValueError("NightSourceLeafSummaryFrame.source_mode must be single_source") + if frame.claim_alignment != "single_absolute_record": + raise ValueError( + "NightSourceLeafSummaryFrame.claim_alignment must be single_absolute_record" + ) + + for field_name, values in { + "source_graph_node_ids": frame.source_graph_node_ids, + "source_trace_ids": frame.source_trace_ids, + "source_data_ids": frame.source_data_ids, + }.items(): + _validate_string_list(f"NightSourceLeafSummaryFrame.{field_name}", values) + _validate_no_duplicates(f"NightSourceLeafSummaryFrame.{field_name}", values) + + if frame.source_graph_node_ids != [frame.target_graph_node_id]: + raise ValueError( + "NightSourceLeafSummaryFrame.source_graph_node_ids must contain only the target raw source" + ) + for required_id in (frame.target_graph_node_id, frame.source_file_data_id): + if required_id not in frame.source_data_ids: + raise ValueError( + "NightSourceLeafSummaryFrame.source_data_ids must include target and source file data" + ) + if frame.text_snapshot_data_id is not None and frame.text_snapshot_data_id not in ( + frame.source_data_ids + ): + raise ValueError( + "NightSourceLeafSummaryFrame.source_data_ids must include text_snapshot_data_id" + ) + + if frame.summary_status == "ran": + if not frame.generated_by.startswith("LLM:"): + raise ValueError("ran NightSourceLeafSummaryFrame.generated_by must be LLM") + if frame.info_class != "relative": + raise ValueError("ran NightSourceLeafSummaryFrame.info_class must be relative") + if frame.semantic_judgement_status != "ran": + raise ValueError( + "ran NightSourceLeafSummaryFrame.semantic_judgement_status must be ran" + ) + if not frame.summary_text.strip(): + raise ValueError("ran NightSourceLeafSummaryFrame must include summary_text") + if frame.failure_type != "none": + raise ValueError("ran NightSourceLeafSummaryFrame.failure_type must be none") + if frame.payload_parse_status != "passed": + raise ValueError( + "ran NightSourceLeafSummaryFrame.payload_parse_status must be passed" + ) + if frame.text_snapshot_data_id is None: + raise ValueError( + "ran NightSourceLeafSummaryFrame must include text_snapshot_data_id" + ) + if frame.llm_call_data_id is None or frame.llm_call_data_id not in ( + frame.source_data_ids + ): + raise ValueError( + "ran NightSourceLeafSummaryFrame.source_data_ids must include llm_call_data_id" + ) + return + + if frame.summary_text.strip(): + raise ValueError("non-ran NightSourceLeafSummaryFrame must not include summary_text") + + if frame.summary_status == "failed": + if not frame.generated_by.startswith("LLM:"): + raise ValueError("failed NightSourceLeafSummaryFrame.generated_by must be LLM") + if frame.info_class != "relative": + raise ValueError( + "failed NightSourceLeafSummaryFrame.info_class must be relative" + ) + if frame.semantic_judgement_status != "failed": + raise ValueError( + "failed NightSourceLeafSummaryFrame.semantic_judgement_status must be failed" + ) + if frame.failure_type in {"none", "no_text_snapshot", "empty_text"}: + raise ValueError( + "failed NightSourceLeafSummaryFrame.failure_type must be an LLM failure" + ) + return + + if not frame.generated_by.startswith("CODE:"): + raise ValueError("skipped NightSourceLeafSummaryFrame.generated_by must be CODE") + if frame.info_class != "absolute": + raise ValueError("skipped NightSourceLeafSummaryFrame.info_class must be absolute") + if frame.semantic_judgement_status != "not_run": + raise ValueError( + "skipped NightSourceLeafSummaryFrame.semantic_judgement_status must be not_run" + ) + if frame.llm_call_data_id is not None or frame.llm_trace_event_id is not None: + raise ValueError("skipped NightSourceLeafSummaryFrame must not include llm ids") + if frame.payload_parse_status != "not_checked": + raise ValueError( + "skipped NightSourceLeafSummaryFrame.payload_parse_status must be not_checked" + ) + expected_failure_type = ( + "no_text_snapshot" + if frame.summary_status == "skipped_no_text_snapshot" + else "empty_text" + ) + if frame.failure_type != expected_failure_type: + raise ValueError( + "skipped NightSourceLeafSummaryFrame.failure_type must match skip status" + ) + + +def validate_graph_memory_node_frame(frame: GraphMemoryNodeFrame) -> None: + _require_text_fields( + "GraphMemoryNodeFrame", + { + "node_id": frame.node_id, + "node_kind": frame.node_kind, + "data_kind": frame.data_kind, + "source_bundle_kind": frame.source_bundle_kind, + "bundle_policy_id": frame.bundle_policy_id, + "char_budget_status": frame.char_budget_status, + "generated_by": frame.generated_by, + "info_class": frame.info_class, + "semantic_judgement_status": frame.semantic_judgement_status, + "schema_name": frame.schema_name, + "schema_version": frame.schema_version, + }, + ) + if frame.schema_name != GRAPH_MEMORY_NODE_FRAME_SCHEMA_NAME: + raise ValueError(f"unknown graph memory node schema_name: {frame.schema_name}") + if frame.schema_version != GRAPH_MEMORY_NODE_FRAME_SCHEMA_VERSION: + raise ValueError(f"unknown graph memory node schema_version: {frame.schema_version}") + if frame.node_kind not in GRAPH_MEMORY_NODE_KINDS: + raise ValueError(f"unknown graph memory node_kind: {frame.node_kind}") + if frame.generated_by != GRAPH_MEMORY_CODE_GENERATOR: + raise ValueError("GraphMemoryNodeFrame.generated_by must be code builder") + if frame.info_class != "absolute": + raise ValueError("GraphMemoryNodeFrame.info_class must be absolute") + if frame.semantic_judgement_status != "not_run": + raise ValueError("GraphMemoryNodeFrame.semantic_judgement_status must be not_run") + + _validate_non_negative_ints( + "GraphMemoryNodeFrame", + { + "trace_count": frame.trace_count, + "movement_count": frame.movement_count, + "summary_depth": frame.summary_depth, + "source_depth_min": frame.source_depth_min, + "source_depth_max": frame.source_depth_max, + "source_leaf_count": frame.source_leaf_count, + "source_summary_count": frame.source_summary_count, + "source_char_count": frame.source_char_count, + }, + ) + if frame.char_budget is not None and frame.char_budget <= 0: + raise ValueError("GraphMemoryNodeFrame.char_budget must be positive when set") + if frame.source_depth_min > frame.source_depth_max: + raise ValueError("GraphMemoryNodeFrame source depth range is inverted") + + _validate_string_list("GraphMemoryNodeFrame.source_graph_node_ids", frame.source_graph_node_ids) + _validate_string_list("GraphMemoryNodeFrame.source_trace_ids", frame.source_trace_ids) + _validate_string_list("GraphMemoryNodeFrame.source_data_ids", frame.source_data_ids) + _validate_no_duplicates( + "GraphMemoryNodeFrame.source_graph_node_ids", + frame.source_graph_node_ids, + ) + _validate_no_duplicates("GraphMemoryNodeFrame.source_trace_ids", frame.source_trace_ids) + _validate_no_duplicates("GraphMemoryNodeFrame.source_data_ids", frame.source_data_ids) + + if frame.user_input_trace_id is not None and frame.user_input_trace_id not in frame.source_trace_ids: + raise ValueError("raw user trace anchor must be present in source_trace_ids") + if frame.final_response_trace_id is not None and frame.final_response_trace_id not in frame.source_trace_ids: + raise ValueError("raw final trace anchor must be present in source_trace_ids") + + if frame.node_kind == "raw_capsule": + if not frame.source_turn_id: + raise ValueError("raw capsule graph node must include source_turn_id") + if frame.summary_depth != 0: + raise ValueError("raw capsule summary_depth must be 0") + if frame.source_leaf_count != 1: + raise ValueError("raw capsule source_leaf_count must be 1") + if frame.source_summary_count != 0: + raise ValueError("raw capsule source_summary_count must be 0") + if frame.node_kind == "raw_source": + if frame.summary_depth != 0: + raise ValueError("raw source summary_depth must be 0") + if frame.source_leaf_count != 1: + raise ValueError("raw source source_leaf_count must be 1") + if frame.source_summary_count != 0: + raise ValueError("raw source source_summary_count must be 0") + if not frame.source_data_ids: + raise ValueError("raw source graph node must cite source file data") + if not frame.observed_at: + raise ValueError("raw source graph node must include observed_at") + if not frame.ingested_at: + raise ValueError("raw source graph node must include ingested_at") + if not frame.source_last_modified_at: + raise ValueError("raw source graph node must include source_last_modified_at") + if frame.exists_at_ingest is not True: + raise ValueError("raw source graph node must have exists_at_ingest=True") + if not frame.content_sha1: + raise ValueError("raw source graph node must include content_sha1") + if frame.node_kind == "source_ingest_time_bundle": + if not frame.source_graph_node_ids: + raise ValueError("source ingest time bundle must contain source kind bundles") + if frame.source_leaf_count < len(frame.source_graph_node_ids): + raise ValueError("source ingest time bundle source_leaf_count is too small") + if not frame.observed_at: + raise ValueError("source ingest time bundle must include observed_at") + if not frame.ingested_at: + raise ValueError("source ingest time bundle must include ingested_at") + if frame.node_kind == "source_kind_bundle": + if not frame.source_graph_node_ids: + raise ValueError("source kind bundle must contain raw source graph nodes") + if frame.source_leaf_count != len(frame.source_graph_node_ids): + raise ValueError("source kind bundle source_leaf_count must mirror children") + if not frame.observed_at: + raise ValueError("source kind bundle must include observed_at") + if not frame.ingested_at: + raise ValueError("source kind bundle must include ingested_at") + + +def validate_graph_memory_edge_frame(frame: GraphMemoryEdgeFrame) -> None: + _require_text_fields( + "GraphMemoryEdgeFrame", + { + "edge_id": frame.edge_id, + "edge_kind": frame.edge_kind, + "from_node_id": frame.from_node_id, + "to_node_id": frame.to_node_id, + "generated_by": frame.generated_by, + "info_class": frame.info_class, + "semantic_judgement_status": frame.semantic_judgement_status, + "schema_name": frame.schema_name, + "schema_version": frame.schema_version, + }, + ) + if frame.schema_name != GRAPH_MEMORY_EDGE_FRAME_SCHEMA_NAME: + raise ValueError(f"unknown graph memory edge schema_name: {frame.schema_name}") + if frame.schema_version != GRAPH_MEMORY_EDGE_FRAME_SCHEMA_VERSION: + raise ValueError(f"unknown graph memory edge schema_version: {frame.schema_version}") + if frame.edge_kind not in GRAPH_MEMORY_EDGE_KINDS: + raise ValueError(f"unknown graph memory edge_kind: {frame.edge_kind}") + if frame.from_node_id == frame.to_node_id: + raise ValueError("GraphMemoryEdgeFrame must not be a self edge") + if frame.generated_by != GRAPH_MEMORY_CODE_GENERATOR: + raise ValueError("GraphMemoryEdgeFrame.generated_by must be code builder") + if frame.info_class != "absolute": + raise ValueError("GraphMemoryEdgeFrame.info_class must be absolute") + if frame.semantic_judgement_status != "not_run": + raise ValueError("GraphMemoryEdgeFrame.semantic_judgement_status must be not_run") + _validate_string_list("GraphMemoryEdgeFrame.source_graph_node_ids", frame.source_graph_node_ids) + _validate_string_list("GraphMemoryEdgeFrame.source_trace_ids", frame.source_trace_ids) + _validate_string_list("GraphMemoryEdgeFrame.source_data_ids", frame.source_data_ids) + _validate_no_duplicates( + "GraphMemoryEdgeFrame.source_graph_node_ids", + frame.source_graph_node_ids, + ) + _validate_no_duplicates("GraphMemoryEdgeFrame.source_trace_ids", frame.source_trace_ids) + _validate_no_duplicates("GraphMemoryEdgeFrame.source_data_ids", frame.source_data_ids) + + +def validate_graph_memory_snapshot_frame(frame: GraphMemorySnapshotFrame) -> None: + _require_text_fields( + "GraphMemorySnapshotFrame", + { + "snapshot_id": frame.snapshot_id, + "batch_id": frame.batch_id, + "root_node_id": frame.root_node_id, + "time_axis_node_id": frame.time_axis_node_id, + "generated_by": frame.generated_by, + "info_class": frame.info_class, + "semantic_judgement_status": frame.semantic_judgement_status, + "schema_name": frame.schema_name, + "schema_version": frame.schema_version, + }, + ) + if frame.schema_name != GRAPH_MEMORY_SNAPSHOT_FRAME_SCHEMA_NAME: + raise ValueError(f"unknown graph snapshot schema_name: {frame.schema_name}") + if frame.schema_version != GRAPH_MEMORY_SNAPSHOT_FRAME_SCHEMA_VERSION: + raise ValueError(f"unknown graph snapshot schema_version: {frame.schema_version}") + if frame.generated_by != GRAPH_MEMORY_CODE_GENERATOR: + raise ValueError("GraphMemorySnapshotFrame.generated_by must be code builder") + if frame.info_class != "absolute": + raise ValueError("GraphMemorySnapshotFrame.info_class must be absolute") + if frame.semantic_judgement_status != "not_run": + raise ValueError("GraphMemorySnapshotFrame.semantic_judgement_status must be not_run") + _validate_string_list("GraphMemorySnapshotFrame.graph_node_ids", frame.graph_node_ids) + _validate_string_list("GraphMemorySnapshotFrame.graph_edge_ids", frame.graph_edge_ids) + _validate_string_list( + "GraphMemorySnapshotFrame.source_graph_node_ids", + frame.source_graph_node_ids, + ) + _validate_string_list("GraphMemorySnapshotFrame.source_trace_ids", frame.source_trace_ids) + _validate_string_list("GraphMemorySnapshotFrame.source_data_ids", frame.source_data_ids) + _validate_no_duplicates("GraphMemorySnapshotFrame.graph_node_ids", frame.graph_node_ids) + _validate_no_duplicates("GraphMemorySnapshotFrame.graph_edge_ids", frame.graph_edge_ids) + _validate_no_duplicates( + "GraphMemorySnapshotFrame.source_graph_node_ids", + frame.source_graph_node_ids, + ) + _validate_no_duplicates("GraphMemorySnapshotFrame.source_trace_ids", frame.source_trace_ids) + _validate_no_duplicates("GraphMemorySnapshotFrame.source_data_ids", frame.source_data_ids) + if frame.root_node_id not in frame.graph_node_ids: + raise ValueError("GraphMemorySnapshotFrame.graph_node_ids must include root_node_id") + if frame.time_axis_node_id not in frame.graph_node_ids: + raise ValueError("GraphMemorySnapshotFrame.graph_node_ids must include time_axis_node_id") + _validate_counts("GraphMemorySnapshotFrame.node_kind_counts", frame.node_kind_counts) + _validate_counts("GraphMemorySnapshotFrame.edge_kind_counts", frame.edge_kind_counts) + _validate_counts("GraphMemorySnapshotFrame.data_kind_counts", frame.data_kind_counts) + + +def validate_turn_graph_access_ledger_frame(frame: TurnGraphAccessLedgerFrame) -> None: + _require_text_fields( + "TurnGraphAccessLedgerFrame", + { + "frame_id": frame.frame_id, + "turn_id": frame.turn_id, + "turn_capsule_graph_node_id": frame.turn_capsule_graph_node_id, + "generated_by": frame.generated_by, + "info_class": frame.info_class, + "semantic_judgement_status": frame.semantic_judgement_status, + "schema_name": frame.schema_name, + "schema_version": frame.schema_version, + }, + ) + if frame.schema_name != TURN_GRAPH_ACCESS_LEDGER_FRAME_SCHEMA_NAME: + raise ValueError(f"unknown turn graph access ledger schema_name: {frame.schema_name}") + if frame.schema_version != TURN_GRAPH_ACCESS_LEDGER_FRAME_SCHEMA_VERSION: + raise ValueError(f"unknown turn graph access ledger schema_version: {frame.schema_version}") + if frame.generated_by != GRAPH_ACCESS_LEDGER_CODE_GENERATOR: + raise ValueError("TurnGraphAccessLedgerFrame.generated_by must be graph access ledger code") + if frame.info_class != "absolute": + raise ValueError("TurnGraphAccessLedgerFrame.info_class must be absolute") + if frame.semantic_judgement_status != "not_run": + raise ValueError("TurnGraphAccessLedgerFrame.semantic_judgement_status must be not_run") + + graph_id_fields = { + "candidate_graph_node_ids": frame.candidate_graph_node_ids, + "selected_graph_node_ids": frame.selected_graph_node_ids, + "inspected_graph_node_ids": frame.inspected_graph_node_ids, + "read_graph_node_ids": frame.read_graph_node_ids, + "used_as_answer_source_graph_node_ids": frame.used_as_answer_source_graph_node_ids, + } + for field_name, values in graph_id_fields.items(): + _validate_string_list(f"TurnGraphAccessLedgerFrame.{field_name}", values) + _validate_no_duplicates(f"TurnGraphAccessLedgerFrame.{field_name}", values) + _validate_string_list("TurnGraphAccessLedgerFrame.source_trace_ids", frame.source_trace_ids) + _validate_string_list("TurnGraphAccessLedgerFrame.source_data_ids", frame.source_data_ids) + _validate_no_duplicates("TurnGraphAccessLedgerFrame.source_trace_ids", frame.source_trace_ids) + _validate_no_duplicates("TurnGraphAccessLedgerFrame.source_data_ids", frame.source_data_ids) + + known_graph_node_ids = { + frame.turn_capsule_graph_node_id, + *frame.candidate_graph_node_ids, + *frame.selected_graph_node_ids, + *frame.inspected_graph_node_ids, + *frame.read_graph_node_ids, + *frame.used_as_answer_source_graph_node_ids, + } + if not frame.access_records: + raise ValueError("TurnGraphAccessLedgerFrame.access_records must not be empty") + for index, record in enumerate(frame.access_records, start=1): + if not isinstance(record, dict): + raise ValueError("TurnGraphAccessLedgerFrame.access_records must contain dict records") + stage = record.get("stage") + graph_node_id = record.get("graph_node_id") + source_frame_id = record.get("source_frame_id") + source_field = record.get("source_field") + if not stage or not graph_node_id or not source_frame_id or not source_field: + raise ValueError( + "TurnGraphAccessLedgerFrame.access_records entries require " + "stage, graph_node_id, source_frame_id, source_field" + ) + if stage not in GRAPH_ACCESS_STAGES: + raise ValueError(f"unknown graph access stage: {stage}") + if graph_node_id not in known_graph_node_ids: + raise ValueError( + "TurnGraphAccessLedgerFrame.access_records graph_node_id must be listed " + f"in ledger graph node fields at index {index}" + ) + if source_frame_id not in frame.source_data_ids: + raise ValueError( + "TurnGraphAccessLedgerFrame.source_data_ids must include access record source_frame_id" + ) + + +def validate_turn_activity_graph_link_frame(frame: TurnActivityGraphLinkFrame) -> None: + _require_text_fields( + "TurnActivityGraphLinkFrame", + { + "frame_id": frame.frame_id, + "turn_id": frame.turn_id, + "turn_capsule_graph_node_id": frame.turn_capsule_graph_node_id, + "generated_by": frame.generated_by, + "info_class": frame.info_class, + "semantic_judgement_status": frame.semantic_judgement_status, + "schema_name": frame.schema_name, + "schema_version": frame.schema_version, + }, + ) + if frame.schema_name != TURN_ACTIVITY_GRAPH_LINK_FRAME_SCHEMA_NAME: + raise ValueError( + f"unknown TurnActivityGraphLinkFrame.schema_name: {frame.schema_name}" + ) + if frame.schema_version != TURN_ACTIVITY_GRAPH_LINK_FRAME_SCHEMA_VERSION: + raise ValueError( + f"unknown TurnActivityGraphLinkFrame.schema_version: {frame.schema_version}" + ) + if frame.generated_by != TURN_ACTIVITY_GRAPH_LINK_CODE_GENERATOR: + raise ValueError("TurnActivityGraphLinkFrame.generated_by must reveal code builder") + if frame.info_class != "absolute": + raise ValueError("TurnActivityGraphLinkFrame.info_class must be absolute") + if frame.semantic_judgement_status != "not_run": + raise ValueError("TurnActivityGraphLinkFrame.semantic_judgement_status must be not_run") + + list_fields = { + "l_loop_activity_ledger_data_ids": frame.l_loop_activity_ledger_data_ids, + "r_graph_access_ledger_data_ids": frame.r_graph_access_ledger_data_ids, + "activity_ledger_graph_node_ids": frame.activity_ledger_graph_node_ids, + "activity_ledger_graph_edge_ids": frame.activity_ledger_graph_edge_ids, + "source_trace_ids": frame.source_trace_ids, + "source_data_ids": frame.source_data_ids, + } + for field_name, values in list_fields.items(): + _validate_string_list(f"TurnActivityGraphLinkFrame.{field_name}", values) + _validate_no_duplicates(f"TurnActivityGraphLinkFrame.{field_name}", values) + + ledger_data_ids = { + *frame.l_loop_activity_ledger_data_ids, + *frame.r_graph_access_ledger_data_ids, + } + if len(frame.activity_ledger_graph_node_ids) != len(ledger_data_ids): + raise ValueError( + "TurnActivityGraphLinkFrame.activity_ledger_graph_node_ids must mirror ledger count" + ) + if len(frame.activity_ledger_graph_edge_ids) != len(ledger_data_ids): + raise ValueError( + "TurnActivityGraphLinkFrame.activity_ledger_graph_edge_ids must mirror ledger count" + ) + if len(frame.link_records) != len(ledger_data_ids): + raise ValueError("TurnActivityGraphLinkFrame.link_records must mirror ledger count") + + source_data_ids = set(frame.source_data_ids) + required_sources = { + frame.turn_capsule_graph_node_id, + *ledger_data_ids, + *frame.activity_ledger_graph_node_ids, + *frame.activity_ledger_graph_edge_ids, + } + if not required_sources <= source_data_ids: + missing = sorted(required_sources - source_data_ids) + raise ValueError( + "TurnActivityGraphLinkFrame.source_data_ids must include graph/link sources: " + f"{missing}" + ) + + node_ids = set(frame.activity_ledger_graph_node_ids) + edge_ids = set(frame.activity_ledger_graph_edge_ids) + for index, record in enumerate(frame.link_records, start=1): + if not isinstance(record, dict): + raise ValueError("TurnActivityGraphLinkFrame.link_records must contain dict records") + activity_kind = record.get("activity_kind") + ledger_data_id = record.get("ledger_data_id") + graph_node_id = record.get("graph_node_id") + edge_id = record.get("edge_id") + source_field = record.get("source_field") + if not activity_kind or not ledger_data_id or not graph_node_id or not edge_id or not source_field: + raise ValueError( + "TurnActivityGraphLinkFrame.link_records entries require " + "activity_kind, ledger_data_id, graph_node_id, edge_id, source_field" + ) + if activity_kind not in TURN_ACTIVITY_GRAPH_LINK_ACTIVITY_KINDS: + raise ValueError(f"unknown turn activity graph link kind: {activity_kind}") + if ledger_data_id not in ledger_data_ids: + raise ValueError( + f"TurnActivityGraphLinkFrame.link_records ledger_data_id is not listed at {index}" + ) + if graph_node_id not in node_ids: + raise ValueError( + f"TurnActivityGraphLinkFrame.link_records graph_node_id is not listed at {index}" + ) + if edge_id not in edge_ids: + raise ValueError( + f"TurnActivityGraphLinkFrame.link_records edge_id is not listed at {index}" + ) + + +def validate_core_ego_time_axis_frame(frame: CoreEgoTimeAxisFrame) -> None: + _require_text_fields( + "CoreEgoTimeAxisFrame", + { + "frame_id": frame.frame_id, + "batch_id": frame.batch_id, + "core_ego_node_id": frame.core_ego_node_id, + "time_axis_node_id": frame.time_axis_node_id, + "generated_by": frame.generated_by, + "info_class": frame.info_class, + "semantic_judgement_status": frame.semantic_judgement_status, + "semantic_axis_status": frame.semantic_axis_status, + "schema_name": frame.schema_name, + "schema_version": frame.schema_version, + }, + ) + if frame.schema_name != CORE_EGO_TIME_AXIS_FRAME_SCHEMA_NAME: + raise ValueError(f"unknown CoreEgoTimeAxisFrame schema_name: {frame.schema_name}") + if frame.schema_version != CORE_EGO_TIME_AXIS_FRAME_SCHEMA_VERSION: + raise ValueError(f"unknown CoreEgoTimeAxisFrame schema_version: {frame.schema_version}") + if frame.generated_by != GRAPH_MEMORY_CODE_GENERATOR: + raise ValueError("CoreEgoTimeAxisFrame.generated_by must be code builder") + if frame.info_class != "absolute": + raise ValueError("CoreEgoTimeAxisFrame.info_class must be absolute") + if frame.semantic_judgement_status != "not_run": + raise ValueError("CoreEgoTimeAxisFrame.semantic_judgement_status must be not_run") + if frame.semantic_axis_status != "not_created": + raise ValueError("CoreEgoTimeAxisFrame must not create semantic axis") + _validate_string_list("CoreEgoTimeAxisFrame.time_bundle_node_ids", frame.time_bundle_node_ids) + _validate_string_list("CoreEgoTimeAxisFrame.raw_capsule_node_ids", frame.raw_capsule_node_ids) + _validate_string_list("CoreEgoTimeAxisFrame.edge_ids", frame.edge_ids) + _validate_string_list( + "CoreEgoTimeAxisFrame.source_graph_node_ids", + frame.source_graph_node_ids, + ) + _validate_string_list("CoreEgoTimeAxisFrame.source_trace_ids", frame.source_trace_ids) + _validate_string_list("CoreEgoTimeAxisFrame.source_data_ids", frame.source_data_ids) + _validate_no_duplicates( + "CoreEgoTimeAxisFrame.time_bundle_node_ids", + frame.time_bundle_node_ids, + ) + _validate_no_duplicates( + "CoreEgoTimeAxisFrame.raw_capsule_node_ids", + frame.raw_capsule_node_ids, + ) + _validate_no_duplicates("CoreEgoTimeAxisFrame.edge_ids", frame.edge_ids) + + +def validate_rloop_graph_guide_packet_frame(frame: RLoopGraphGuidePacketFrame) -> None: + _require_text_fields( + "RLoopGraphGuidePacketFrame", + { + "packet_id": frame.packet_id, + "graph_snapshot_id": frame.graph_snapshot_id, + "target_consumer": frame.target_consumer, + "recommended_traversal_hints_status": frame.recommended_traversal_hints_status, + "generated_by": frame.generated_by, + "info_class": frame.info_class, + "semantic_judgement_status": frame.semantic_judgement_status, + "schema_name": frame.schema_name, + "schema_version": frame.schema_version, + }, + ) + if frame.schema_name != RLOOP_GRAPH_GUIDE_PACKET_FRAME_SCHEMA_NAME: + raise ValueError(f"unknown RLoopGraphGuidePacketFrame schema_name: {frame.schema_name}") + if frame.schema_version != RLOOP_GRAPH_GUIDE_PACKET_FRAME_SCHEMA_VERSION: + raise ValueError( + f"unknown RLoopGraphGuidePacketFrame schema_version: {frame.schema_version}" + ) + if frame.target_consumer != "R_LOOP": + raise ValueError("RLoopGraphGuidePacketFrame.target_consumer must be R_LOOP") + if frame.generated_by != RLOOP_GUIDE_CODE_GENERATOR: + raise ValueError("RLoopGraphGuidePacketFrame.generated_by must be guide builder") + if frame.info_class != "absolute": + raise ValueError("RLoopGraphGuidePacketFrame.info_class must be absolute") + if frame.semantic_judgement_status != "not_run": + raise ValueError("RLoopGraphGuidePacketFrame.semantic_judgement_status must be not_run") + if frame.recommended_traversal_hints_status != "not_run": + raise ValueError("RLoopGraphGuidePacketFrame hints must remain not_run") + if frame.recommended_traversal_hints: + raise ValueError("RLoopGraphGuidePacketFrame must not include LLM traversal hints") + + _validate_string_list("RLoopGraphGuidePacketFrame.available_entry_nodes", frame.available_entry_nodes) + _validate_string_list( + "RLoopGraphGuidePacketFrame.risky_or_unreviewed_node_ids", + frame.risky_or_unreviewed_node_ids, + ) + _validate_string_list( + "RLoopGraphGuidePacketFrame.source_graph_node_ids", + frame.source_graph_node_ids, + ) + _validate_string_list("RLoopGraphGuidePacketFrame.source_trace_ids", frame.source_trace_ids) + _validate_string_list("RLoopGraphGuidePacketFrame.source_data_ids", frame.source_data_ids) + _validate_no_duplicates( + "RLoopGraphGuidePacketFrame.available_entry_nodes", + frame.available_entry_nodes, + ) + _validate_no_duplicates( + "RLoopGraphGuidePacketFrame.source_graph_node_ids", + frame.source_graph_node_ids, + ) + _validate_no_duplicates("RLoopGraphGuidePacketFrame.source_trace_ids", frame.source_trace_ids) + _validate_no_duplicates("RLoopGraphGuidePacketFrame.source_data_ids", frame.source_data_ids) + _validate_counts("RLoopGraphGuidePacketFrame.node_kind_counts", frame.node_kind_counts) + _validate_counts("RLoopGraphGuidePacketFrame.data_kind_counts", frame.data_kind_counts) + _validate_range("RLoopGraphGuidePacketFrame.summary_depth_range", frame.summary_depth_range) + _validate_range( + "RLoopGraphGuidePacketFrame.source_leaf_count_range", + frame.source_leaf_count_range, + ) + + +def validate_core_ego_guide_worker_hint_frame(frame: CoreEgoGuideWorkerHintFrame) -> None: + _require_text_fields( + "CoreEgoGuideWorkerHintFrame", + { + "frame_id": frame.frame_id, + "source_rloop_graph_guide_packet_id": frame.source_rloop_graph_guide_packet_id, + "graph_snapshot_id": frame.graph_snapshot_id, + "hint_status": frame.hint_status, + "failure_type": frame.failure_type, + "payload_parse_status": frame.payload_parse_status, + "prompt_ref": frame.prompt_ref, + "generated_by": frame.generated_by, + "info_class": frame.info_class, + "source_mode": frame.source_mode, + "claim_alignment": frame.claim_alignment, + "semantic_judgement_status": frame.semantic_judgement_status, + "schema_name": frame.schema_name, + "schema_version": frame.schema_version, + }, + ) + if frame.schema_name != CORE_EGO_GUIDE_WORKER_HINT_FRAME_SCHEMA_NAME: + raise ValueError(f"unknown CoreEgoGuideWorkerHintFrame schema_name: {frame.schema_name}") + if frame.schema_version != CORE_EGO_GUIDE_WORKER_HINT_FRAME_SCHEMA_VERSION: + raise ValueError( + f"unknown CoreEgoGuideWorkerHintFrame schema_version: {frame.schema_version}" + ) + if frame.hint_status not in CORE_EGO_GUIDE_WORKER_HINT_STATUSES: + raise ValueError(f"unknown CoreEgoGuideWorkerHintFrame.hint_status: {frame.hint_status}") + if frame.failure_type not in CORE_EGO_GUIDE_WORKER_HINT_FAILURE_TYPES: + raise ValueError(f"unknown CoreEgoGuideWorkerHintFrame.failure_type: {frame.failure_type}") + if frame.payload_parse_status not in CORE_EGO_GUIDE_WORKER_PARSE_STATUSES: + raise ValueError( + f"unknown CoreEgoGuideWorkerHintFrame.payload_parse_status: {frame.payload_parse_status}" + ) + if not frame.generated_by.startswith("LLM:"): + raise ValueError("CoreEgoGuideWorkerHintFrame.generated_by must start with LLM:") + if frame.info_class != "mixed": + raise ValueError("CoreEgoGuideWorkerHintFrame.info_class must be mixed") + if frame.source_mode != "source_bundle": + raise ValueError("CoreEgoGuideWorkerHintFrame.source_mode must be source_bundle") + if frame.claim_alignment != "multi_source_bundle": + raise ValueError("CoreEgoGuideWorkerHintFrame.claim_alignment must be multi_source_bundle") + if frame.semantic_judgement_status not in {"ran", "failed"}: + raise ValueError( + "CoreEgoGuideWorkerHintFrame.semantic_judgement_status must be ran or failed" + ) + + _validate_string_list( + "CoreEgoGuideWorkerHintFrame.available_entry_node_ids", + frame.available_entry_node_ids, + ) + _validate_string_list( + "CoreEgoGuideWorkerHintFrame.available_source_graph_node_ids", + frame.available_source_graph_node_ids, + ) + _validate_string_list( + "CoreEgoGuideWorkerHintFrame.recommended_entry_node_ids", + frame.recommended_entry_node_ids, + ) + _validate_string_list( + "CoreEgoGuideWorkerHintFrame.avoid_entry_node_ids", + frame.avoid_entry_node_ids, + ) + _validate_string_list("CoreEgoGuideWorkerHintFrame.risk_notes", frame.risk_notes) + _validate_string_list( + "CoreEgoGuideWorkerHintFrame.source_graph_node_ids", + frame.source_graph_node_ids, + ) + _validate_string_list("CoreEgoGuideWorkerHintFrame.source_trace_ids", frame.source_trace_ids) + _validate_string_list("CoreEgoGuideWorkerHintFrame.source_data_ids", frame.source_data_ids) + _validate_no_duplicates( + "CoreEgoGuideWorkerHintFrame.available_entry_node_ids", + frame.available_entry_node_ids, + ) + _validate_no_duplicates( + "CoreEgoGuideWorkerHintFrame.available_source_graph_node_ids", + frame.available_source_graph_node_ids, + ) + _validate_no_duplicates( + "CoreEgoGuideWorkerHintFrame.recommended_entry_node_ids", + frame.recommended_entry_node_ids, + ) + _validate_no_duplicates( + "CoreEgoGuideWorkerHintFrame.avoid_entry_node_ids", + frame.avoid_entry_node_ids, + ) + _validate_no_duplicates( + "CoreEgoGuideWorkerHintFrame.source_graph_node_ids", + frame.source_graph_node_ids, + ) + _validate_no_duplicates("CoreEgoGuideWorkerHintFrame.source_trace_ids", frame.source_trace_ids) + _validate_no_duplicates("CoreEgoGuideWorkerHintFrame.source_data_ids", frame.source_data_ids) + + available_entries = set(frame.available_entry_node_ids) + for node_id in [*frame.recommended_entry_node_ids, *frame.avoid_entry_node_ids]: + if node_id not in available_entries: + raise ValueError("CoreEgoGuideWorkerHintFrame entry node id must be available") + + available_source_nodes = set(frame.available_source_graph_node_ids) + for node_id in frame.source_graph_node_ids: + if node_id not in available_source_nodes: + raise ValueError("CoreEgoGuideWorkerHintFrame source_graph_node_ids must be in snapshot") + + if frame.source_rloop_graph_guide_packet_id not in frame.source_data_ids: + raise ValueError( + "CoreEgoGuideWorkerHintFrame.source_data_ids must include source guide packet" + ) + if frame.graph_snapshot_id not in frame.source_data_ids: + raise ValueError("CoreEgoGuideWorkerHintFrame.source_data_ids must include graph_snapshot_id") + if frame.llm_call_data_id is not None: + if not frame.llm_call_data_id: + raise ValueError("CoreEgoGuideWorkerHintFrame.llm_call_data_id must not be empty") + if frame.llm_call_data_id not in frame.source_data_ids: + raise ValueError("CoreEgoGuideWorkerHintFrame.source_data_ids must include llm_call_data_id") + if frame.llm_trace_event_id is not None: + if not frame.llm_trace_event_id: + raise ValueError("CoreEgoGuideWorkerHintFrame.llm_trace_event_id must not be empty") + if frame.llm_trace_event_id not in frame.source_trace_ids: + raise ValueError( + "CoreEgoGuideWorkerHintFrame.source_trace_ids must include llm_trace_event_id" + ) + + if frame.hint_status == "ran": + if frame.semantic_judgement_status != "ran": + raise ValueError("ran CoreEgoGuideWorkerHintFrame must have semantic_judgement_status=ran") + if frame.failure_type != "none": + raise ValueError("ran CoreEgoGuideWorkerHintFrame must have failure_type=none") + if frame.payload_parse_status != "passed": + raise ValueError("ran CoreEgoGuideWorkerHintFrame must have payload_parse_status=passed") + if not frame.traversal_strategy_hint.strip(): + raise ValueError("CoreEgoGuideWorkerHintFrame.traversal_strategy_hint must not be empty") + if not frame.reason_summary.strip(): + raise ValueError("CoreEgoGuideWorkerHintFrame.reason_summary must not be empty") + if not frame.expected_depth_policy.strip(): + raise ValueError("CoreEgoGuideWorkerHintFrame.expected_depth_policy must not be empty") + if not frame.source_graph_node_ids: + raise ValueError("ran CoreEgoGuideWorkerHintFrame.source_graph_node_ids must not be empty") + return + + if frame.semantic_judgement_status != "failed": + raise ValueError("failed CoreEgoGuideWorkerHintFrame must have semantic_judgement_status=failed") + if frame.failure_type == "none": + raise ValueError("failed CoreEgoGuideWorkerHintFrame must include failure_type") + if frame.recommended_entry_node_ids or frame.avoid_entry_node_ids: + raise ValueError("failed CoreEgoGuideWorkerHintFrame must not include recommendations") + + +def validate_r_loop_memory_handoff_packet_frame(frame: RLoopMemoryHandoffPacketFrame) -> None: + _require_text_fields( + "RLoopMemoryHandoffPacketFrame", + { + "packet_id": frame.packet_id, + "target": frame.target, + "mode": frame.mode, + "packet_status": frame.packet_status, + "semantic_hint_status": frame.semantic_hint_status, + "generated_by": frame.generated_by, + "info_class": frame.info_class, + "semantic_judgement_status": frame.semantic_judgement_status, + "schema_name": frame.schema_name, + "schema_version": frame.schema_version, + }, + ) + if frame.schema_name != R_LOOP_MEMORY_HANDOFF_PACKET_FRAME_SCHEMA_NAME: + raise ValueError(f"unknown RLoopMemoryHandoffPacketFrame schema_name: {frame.schema_name}") + if frame.schema_version != R_LOOP_MEMORY_HANDOFF_PACKET_FRAME_SCHEMA_VERSION: + raise ValueError( + f"unknown RLoopMemoryHandoffPacketFrame schema_version: {frame.schema_version}" + ) + if frame.target != "R_LOOP": + raise ValueError("RLoopMemoryHandoffPacketFrame.target must be R_LOOP") + if frame.mode != "graph_guide_handoff": + raise ValueError("RLoopMemoryHandoffPacketFrame.mode must be graph_guide_handoff") + if frame.packet_status not in R_LOOP_MEMORY_HANDOFF_STATUSES: + raise ValueError(f"unknown RLoopMemoryHandoffPacketFrame.packet_status: {frame.packet_status}") + if frame.semantic_hint_status not in R_LOOP_MEMORY_HANDOFF_SEMANTIC_HINT_STATUSES: + raise ValueError( + f"unknown RLoopMemoryHandoffPacketFrame.semantic_hint_status: {frame.semantic_hint_status}" + ) + if frame.generated_by != "CODE:node_0_memory_supplier": + raise ValueError("RLoopMemoryHandoffPacketFrame.generated_by must be node_0 code") + if frame.info_class != "absolute": + raise ValueError("RLoopMemoryHandoffPacketFrame.info_class must be absolute") + if frame.semantic_judgement_status != "not_run": + raise ValueError("RLoopMemoryHandoffPacketFrame.semantic_judgement_status must be not_run") + + _validate_string_list( + "RLoopMemoryHandoffPacketFrame.available_entry_node_ids", + frame.available_entry_node_ids, + ) + _validate_string_list( + "RLoopMemoryHandoffPacketFrame.source_graph_node_ids", + frame.source_graph_node_ids, + ) + _validate_string_list("RLoopMemoryHandoffPacketFrame.source_trace_ids", frame.source_trace_ids) + _validate_string_list("RLoopMemoryHandoffPacketFrame.source_data_ids", frame.source_data_ids) + _validate_no_duplicates( + "RLoopMemoryHandoffPacketFrame.available_entry_node_ids", + frame.available_entry_node_ids, + ) + _validate_no_duplicates( + "RLoopMemoryHandoffPacketFrame.source_graph_node_ids", + frame.source_graph_node_ids, + ) + _validate_no_duplicates("RLoopMemoryHandoffPacketFrame.source_trace_ids", frame.source_trace_ids) + _validate_no_duplicates("RLoopMemoryHandoffPacketFrame.source_data_ids", frame.source_data_ids) + _validate_counts("RLoopMemoryHandoffPacketFrame.node_kind_counts", frame.node_kind_counts) + _validate_counts("RLoopMemoryHandoffPacketFrame.data_kind_counts", frame.data_kind_counts) + _validate_range("RLoopMemoryHandoffPacketFrame.summary_depth_range", frame.summary_depth_range) + _validate_range( + "RLoopMemoryHandoffPacketFrame.source_leaf_count_range", + frame.source_leaf_count_range, + ) + + if frame.packet_status == "available": + if not frame.graph_snapshot_id: + raise ValueError("available RLoopMemoryHandoffPacketFrame must include graph_snapshot_id") + if not frame.r_loop_graph_guide_packet_id: + raise ValueError( + "available RLoopMemoryHandoffPacketFrame must include r_loop_graph_guide_packet_id" + ) + if not frame.available_entry_node_ids: + raise ValueError("available RLoopMemoryHandoffPacketFrame must include entry nodes") + if not frame.source_graph_node_ids: + raise ValueError("available RLoopMemoryHandoffPacketFrame must include source graph nodes") + if frame.graph_snapshot_id not in frame.source_data_ids: + raise ValueError( + "RLoopMemoryHandoffPacketFrame.source_data_ids must include graph_snapshot_id" + ) + if frame.r_loop_graph_guide_packet_id not in frame.source_data_ids: + raise ValueError( + "RLoopMemoryHandoffPacketFrame.source_data_ids must include guide packet id" + ) + else: + if frame.graph_snapshot_id or frame.r_loop_graph_guide_packet_id: + raise ValueError("missing RLoopMemoryHandoffPacketFrame must not cite graph guide IDs") + if frame.available_entry_node_ids or frame.source_graph_node_ids: + raise ValueError("missing RLoopMemoryHandoffPacketFrame must not include graph nodes") + + if frame.semantic_hint_status in {"ran", "failed"}: + if not frame.semantic_hint_frame_id: + raise ValueError("semantic hint status ran/failed requires semantic_hint_frame_id") + if frame.semantic_hint_frame_id not in frame.source_data_ids: + raise ValueError( + "RLoopMemoryHandoffPacketFrame.source_data_ids must include semantic_hint_frame_id" + ) + + +def validate_source_version_lineage_frame(frame: SourceVersionLineageFrame) -> None: + _require_text_fields( + "SourceVersionLineageFrame", + { + "frame_id": frame.frame_id, + "source_identity_key": frame.source_identity_key, + "source_kind": frame.source_kind, + "path": frame.path, + "lineage_status": frame.lineage_status, + "active_source_graph_node_id": frame.active_source_graph_node_id, + "generated_by": frame.generated_by, + "info_class": frame.info_class, + "semantic_judgement_status": frame.semantic_judgement_status, + "schema_name": frame.schema_name, + "schema_version": frame.schema_version, + }, + ) + if frame.schema_name != SOURCE_VERSION_LINEAGE_FRAME_SCHEMA_NAME: + raise ValueError(f"unknown SourceVersionLineageFrame.schema_name: {frame.schema_name}") + if frame.schema_version != SOURCE_VERSION_LINEAGE_FRAME_SCHEMA_VERSION: + raise ValueError( + f"unknown SourceVersionLineageFrame.schema_version: {frame.schema_version}" + ) + if frame.generated_by != SOURCE_VERSION_LINEAGE_CODE_GENERATOR: + raise ValueError("SourceVersionLineageFrame.generated_by must be code builder") + if frame.info_class != "absolute": + raise ValueError("SourceVersionLineageFrame.info_class must be absolute") + if frame.semantic_judgement_status != "not_run": + raise ValueError("SourceVersionLineageFrame.semantic_judgement_status must be not_run") + if frame.lineage_status not in SOURCE_VERSION_LINEAGE_STATUSES: + raise ValueError(f"unknown SourceVersionLineageFrame.lineage_status: {frame.lineage_status}") + + _validate_string_list( + "SourceVersionLineageFrame.version_source_graph_node_ids", + frame.version_source_graph_node_ids, + ) + _validate_string_list( + "SourceVersionLineageFrame.superseded_source_graph_node_ids", + frame.superseded_source_graph_node_ids, + ) + _validate_string_list("SourceVersionLineageFrame.source_file_data_ids", frame.source_file_data_ids) + _validate_string_list( + "SourceVersionLineageFrame.source_graph_node_ids", + frame.source_graph_node_ids, + ) + _validate_string_list("SourceVersionLineageFrame.source_trace_ids", frame.source_trace_ids) + _validate_string_list("SourceVersionLineageFrame.source_data_ids", frame.source_data_ids) + _validate_no_duplicates( + "SourceVersionLineageFrame.version_source_graph_node_ids", + frame.version_source_graph_node_ids, + ) + _validate_no_duplicates( + "SourceVersionLineageFrame.superseded_source_graph_node_ids", + frame.superseded_source_graph_node_ids, + ) + _validate_no_duplicates("SourceVersionLineageFrame.source_file_data_ids", frame.source_file_data_ids) + _validate_no_duplicates( + "SourceVersionLineageFrame.source_graph_node_ids", + frame.source_graph_node_ids, + ) + _validate_no_duplicates("SourceVersionLineageFrame.source_trace_ids", frame.source_trace_ids) + _validate_no_duplicates("SourceVersionLineageFrame.source_data_ids", frame.source_data_ids) + + if not frame.version_source_graph_node_ids: + raise ValueError("SourceVersionLineageFrame must include at least one source version") + if frame.active_source_graph_node_id not in frame.version_source_graph_node_ids: + raise ValueError("SourceVersionLineageFrame.active_source_graph_node_id must be a version") + version_set = set(frame.version_source_graph_node_ids) + superseded_set = set(frame.superseded_source_graph_node_ids) + if not superseded_set.issubset(version_set): + raise ValueError("SourceVersionLineageFrame superseded versions must be in version list") + if frame.active_source_graph_node_id in superseded_set: + raise ValueError("SourceVersionLineageFrame active version must not be superseded") + if set(frame.source_graph_node_ids) != version_set: + raise ValueError("SourceVersionLineageFrame.source_graph_node_ids must mirror versions") + if not set(frame.source_file_data_ids).issubset(set(frame.source_data_ids)): + raise ValueError("SourceVersionLineageFrame.source_data_ids must include source_file_data_ids") + + if len(frame.version_records) != len(frame.version_source_graph_node_ids): + raise ValueError("SourceVersionLineageFrame.version_records must mirror versions") + required_record_fields = { + "version_index", + "source_graph_node_id", + "source_file_data_id", + "content_sha1", + "observed_at", + "ingested_at", + "source_last_modified_at", + "supersedes_source_graph_node_id", + } + for record in frame.version_records: + if not isinstance(record, dict): + raise TypeError("SourceVersionLineageFrame.version_records items must be dicts") + missing = required_record_fields - set(record) + if missing: + raise ValueError( + "SourceVersionLineageFrame.version_records missing fields: " + + ", ".join(sorted(missing)) + ) + for field_name, value in record.items(): + if not isinstance(value, str): + raise TypeError( + f"SourceVersionLineageFrame.version_records.{field_name} must be string" + ) + if record["source_graph_node_id"] not in version_set: + raise ValueError("SourceVersionLineageFrame record source_graph_node_id unknown") + if record["source_file_data_id"] not in frame.source_file_data_ids: + raise ValueError("SourceVersionLineageFrame record source_file_data_id unknown") + if not record["content_sha1"]: + raise ValueError("SourceVersionLineageFrame record content_sha1 must not be empty") + if not record["observed_at"]: + raise ValueError("SourceVersionLineageFrame record observed_at must not be empty") + + if set(frame.content_sha1_by_version) != version_set: + raise ValueError("SourceVersionLineageFrame.content_sha1_by_version must mirror versions") + if set(frame.observed_at_by_version) != version_set: + raise ValueError("SourceVersionLineageFrame.observed_at_by_version must mirror versions") + for value in [*frame.content_sha1_by_version.values(), *frame.observed_at_by_version.values()]: + if not value: + raise ValueError("SourceVersionLineageFrame version maps must not contain empty values") + + +def validate_source_observation_ledger_frame(frame: SourceObservationLedgerFrame) -> None: + _require_text_fields( + "SourceObservationLedgerFrame", + { + "frame_id": frame.frame_id, + "batch_id": frame.batch_id, + "ledger_status": frame.ledger_status, + "generated_by": frame.generated_by, + "info_class": frame.info_class, + "semantic_judgement_status": frame.semantic_judgement_status, + "schema_name": frame.schema_name, + "schema_version": frame.schema_version, + }, + ) + if frame.schema_name != SOURCE_OBSERVATION_LEDGER_FRAME_SCHEMA_NAME: + raise ValueError(f"unknown SourceObservationLedgerFrame.schema_name: {frame.schema_name}") + if frame.schema_version != SOURCE_OBSERVATION_LEDGER_FRAME_SCHEMA_VERSION: + raise ValueError( + f"unknown SourceObservationLedgerFrame.schema_version: {frame.schema_version}" + ) + if frame.generated_by != SOURCE_OBSERVATION_LEDGER_CODE_GENERATOR: + raise ValueError("SourceObservationLedgerFrame.generated_by must be code builder") + if frame.info_class != "absolute": + raise ValueError("SourceObservationLedgerFrame.info_class must be absolute") + if frame.semantic_judgement_status != "not_run": + raise ValueError("SourceObservationLedgerFrame.semantic_judgement_status must be not_run") + if frame.ledger_status not in SOURCE_OBSERVATION_LEDGER_STATUSES: + raise ValueError(f"unknown SourceObservationLedgerFrame.ledger_status: {frame.ledger_status}") + + for field_name, values in { + "observed_source_file_data_ids": frame.observed_source_file_data_ids, + "active_source_graph_node_ids": frame.active_source_graph_node_ids, + "source_graph_node_ids": frame.source_graph_node_ids, + "source_trace_ids": frame.source_trace_ids, + "source_data_ids": frame.source_data_ids, + }.items(): + _validate_string_list(f"SourceObservationLedgerFrame.{field_name}", values) + _validate_no_duplicates(f"SourceObservationLedgerFrame.{field_name}", values) + + _validate_counts( + "SourceObservationLedgerFrame.observation_status_counts", + frame.observation_status_counts, + ) + for key in frame.observation_status_counts: + if key not in SOURCE_OBSERVATION_STATUSES: + raise ValueError(f"unknown SourceObservationLedgerFrame status count key: {key}") + + if frame.ledger_status == "no_observations": + if frame.observation_records or frame.observed_source_file_data_ids: + raise ValueError("no_observations ledger must not include observation records") + if frame.ledger_status == "recorded": + if not frame.observation_records: + raise ValueError("recorded SourceObservationLedgerFrame must include records") + if not frame.observed_source_file_data_ids: + raise ValueError("recorded SourceObservationLedgerFrame must include source file ids") + + required_record_fields = { + "source_file_data_id", + "source_kind", + "path", + "observed_at", + "content_sha1", + "observation_status", + "active_source_graph_node_id", + "previous_active_source_graph_node_id", + } + for record in frame.observation_records: + if not isinstance(record, dict): + raise TypeError("SourceObservationLedgerFrame.observation_records items must be dicts") + missing = required_record_fields - set(record) + if missing: + raise ValueError( + "SourceObservationLedgerFrame.observation_records missing fields: " + + ", ".join(sorted(missing)) + ) + for field_name, value in record.items(): + if not isinstance(value, str): + raise TypeError( + f"SourceObservationLedgerFrame.observation_records.{field_name} must be string" + ) + for field_name in required_record_fields - {"previous_active_source_graph_node_id"}: + if not record[field_name]: + raise ValueError( + f"SourceObservationLedgerFrame.observation_records.{field_name} must not be empty" + ) + if record["observation_status"] not in SOURCE_OBSERVATION_STATUSES: + raise ValueError( + f"unknown SourceObservationLedgerFrame observation_status: {record['observation_status']}" + ) + if record["source_file_data_id"] not in frame.observed_source_file_data_ids: + raise ValueError("SourceObservationLedgerFrame record source_file_data_id not listed") + if record["active_source_graph_node_id"] not in frame.active_source_graph_node_ids: + raise ValueError("SourceObservationLedgerFrame record active source id not listed") + if record["active_source_graph_node_id"] not in frame.source_graph_node_ids: + raise ValueError("SourceObservationLedgerFrame source_graph_node_ids missing active id") + + counted = _count_observation_statuses(frame.observation_records) + if counted != frame.observation_status_counts: + raise ValueError("SourceObservationLedgerFrame.observation_status_counts mismatch") + + +def validate_summary_invalidation_ledger_frame(frame: SummaryInvalidationLedgerFrame) -> None: + _require_text_fields( + "SummaryInvalidationLedgerFrame", + { + "frame_id": frame.frame_id, + "batch_id": frame.batch_id, + "ledger_status": frame.ledger_status, + "generated_by": frame.generated_by, + "info_class": frame.info_class, + "semantic_judgement_status": frame.semantic_judgement_status, + "schema_name": frame.schema_name, + "schema_version": frame.schema_version, + }, + ) + if frame.schema_name != SUMMARY_INVALIDATION_LEDGER_FRAME_SCHEMA_NAME: + raise ValueError( + f"unknown SummaryInvalidationLedgerFrame.schema_name: {frame.schema_name}" + ) + if frame.schema_version != SUMMARY_INVALIDATION_LEDGER_FRAME_SCHEMA_VERSION: + raise ValueError( + f"unknown SummaryInvalidationLedgerFrame.schema_version: {frame.schema_version}" + ) + if frame.generated_by != SUMMARY_INVALIDATION_LEDGER_CODE_GENERATOR: + raise ValueError("SummaryInvalidationLedgerFrame.generated_by must be code builder") + if frame.info_class != "absolute": + raise ValueError("SummaryInvalidationLedgerFrame.info_class must be absolute") + if frame.semantic_judgement_status != "not_run": + raise ValueError( + "SummaryInvalidationLedgerFrame.semantic_judgement_status must be not_run" + ) + if frame.ledger_status not in SUMMARY_INVALIDATION_LEDGER_STATUSES: + raise ValueError( + f"unknown SummaryInvalidationLedgerFrame.ledger_status: {frame.ledger_status}" + ) + + for field_name, values in { + "invalidated_summary_node_ids": frame.invalidated_summary_node_ids, + "changed_source_lineage_frame_ids": frame.changed_source_lineage_frame_ids, + "changed_source_graph_node_ids": frame.changed_source_graph_node_ids, + "active_source_graph_node_ids": frame.active_source_graph_node_ids, + "source_graph_node_ids": frame.source_graph_node_ids, + "source_trace_ids": frame.source_trace_ids, + "source_data_ids": frame.source_data_ids, + }.items(): + _validate_string_list(f"SummaryInvalidationLedgerFrame.{field_name}", values) + _validate_no_duplicates(f"SummaryInvalidationLedgerFrame.{field_name}", values) + + if frame.ledger_status == "no_invalidations": + if frame.invalidated_summary_node_ids or frame.invalidation_records: + raise ValueError("no_invalidations ledger must not include invalidation records") + if frame.ledger_status == "invalidations_recorded": + if not frame.invalidated_summary_node_ids: + raise ValueError("invalidations_recorded ledger must include summary ids") + if not frame.invalidation_records: + raise ValueError("invalidations_recorded ledger must include records") + + required_record_fields = { + "summary_graph_node_id", + "invalidated_reason_code", + "source_lineage_frame_id", + "superseded_source_graph_node_id", + "superseding_source_graph_node_id", + "invalidated_at", + "validity_status", + } + for record in frame.invalidation_records: + if not isinstance(record, dict): + raise TypeError("SummaryInvalidationLedgerFrame.invalidation_records items must be dicts") + missing = required_record_fields - set(record) + if missing: + raise ValueError( + "SummaryInvalidationLedgerFrame.invalidation_records missing fields: " + + ", ".join(sorted(missing)) + ) + for field_name, value in record.items(): + if not isinstance(value, str): + raise TypeError( + f"SummaryInvalidationLedgerFrame.invalidation_records.{field_name} must be string" + ) + if not value: + raise ValueError( + f"SummaryInvalidationLedgerFrame.invalidation_records.{field_name} must not be empty" + ) + if record["summary_graph_node_id"] not in frame.invalidated_summary_node_ids: + raise ValueError("SummaryInvalidationLedgerFrame record summary id not listed") + if record["source_lineage_frame_id"] not in frame.changed_source_lineage_frame_ids: + raise ValueError("SummaryInvalidationLedgerFrame record lineage id not listed") + if record["superseded_source_graph_node_id"] not in frame.changed_source_graph_node_ids: + raise ValueError("SummaryInvalidationLedgerFrame record superseded source id not listed") + if record["superseding_source_graph_node_id"] not in frame.active_source_graph_node_ids: + raise ValueError("SummaryInvalidationLedgerFrame record active source id not listed") + if record["invalidated_reason_code"] != "source_content_changed": + raise ValueError("SummaryInvalidationLedgerFrame reason code must be source_content_changed") + if record["validity_status"] != "invalidated_by_source_change": + raise ValueError("SummaryInvalidationLedgerFrame validity_status is invalid") + + +def _require_text_fields(frame_name: str, fields: dict[str, str | None]) -> None: + for field_name, value in fields.items(): + if not value: + raise ValueError(f"{frame_name}.{field_name} must not be empty") + + +def _validate_non_negative_ints(frame_name: str, fields: dict[str, int]) -> None: + for field_name, value in fields.items(): + if value < 0: + raise ValueError(f"{frame_name}.{field_name} must be >= 0") + + +def _validate_counts(field_name: str, counts: dict[str, int]) -> None: + for key, value in counts.items(): + if not key: + raise ValueError(f"{field_name} must not contain empty keys") + if value < 0: + raise ValueError(f"{field_name}.{key} must be >= 0") + + +def _validate_range(field_name: str, values: list[int]) -> None: + if len(values) != 2: + raise ValueError(f"{field_name} must contain [min, max]") + if values[0] < 0 or values[1] < 0: + raise ValueError(f"{field_name} values must be >= 0") + if values[0] > values[1]: + raise ValueError(f"{field_name} is inverted") + + +def _count_observation_statuses(records: list[dict[str, str]]) -> dict[str, int]: + counts: dict[str, int] = {} + for record in records: + status = record.get("observation_status", "") + if not status: + continue + counts[status] = counts.get(status, 0) + 1 + return counts + + +__all__ = [ + "CORE_EGO_TIME_AXIS_FRAME_SCHEMA_NAME", + "CORE_EGO_TIME_AXIS_FRAME_SCHEMA_VERSION", + "CORE_EGO_GUIDE_WORKER_HINT_FAILURE_TYPES", + "CORE_EGO_GUIDE_WORKER_HINT_FRAME_SCHEMA_NAME", + "CORE_EGO_GUIDE_WORKER_HINT_FRAME_SCHEMA_VERSION", + "CORE_EGO_GUIDE_WORKER_HINT_STATUSES", + "CORE_EGO_GUIDE_WORKER_PARSE_STATUSES", + "GRAPH_ACCESS_LEDGER_CODE_GENERATOR", + "GRAPH_ACCESS_STAGES", + "GRAPH_MEMORY_CODE_GENERATOR", + "GRAPH_MEMORY_EDGE_FRAME_SCHEMA_NAME", + "GRAPH_MEMORY_EDGE_FRAME_SCHEMA_VERSION", + "GRAPH_MEMORY_EDGE_KINDS", + "GRAPH_MEMORY_NODE_FRAME_SCHEMA_NAME", + "GRAPH_MEMORY_NODE_FRAME_SCHEMA_VERSION", + "GRAPH_MEMORY_NODE_KINDS", + "GRAPH_MEMORY_SNAPSHOT_FRAME_SCHEMA_NAME", + "GRAPH_MEMORY_SNAPSHOT_FRAME_SCHEMA_VERSION", + "TURN_ACTIVITY_GRAPH_LINK_ACTIVITY_KINDS", + "TURN_ACTIVITY_GRAPH_LINK_CODE_GENERATOR", + "TURN_ACTIVITY_GRAPH_LINK_FRAME_SCHEMA_NAME", + "TURN_ACTIVITY_GRAPH_LINK_FRAME_SCHEMA_VERSION", + "TURN_GRAPH_ACCESS_LEDGER_FRAME_SCHEMA_NAME", + "TURN_GRAPH_ACCESS_LEDGER_FRAME_SCHEMA_VERSION", + "RLOOP_GRAPH_GUIDE_PACKET_FRAME_SCHEMA_NAME", + "RLOOP_GRAPH_GUIDE_PACKET_FRAME_SCHEMA_VERSION", + "RLOOP_GUIDE_CODE_GENERATOR", + "R_LOOP_MEMORY_HANDOFF_PACKET_FRAME_SCHEMA_NAME", + "R_LOOP_MEMORY_HANDOFF_PACKET_FRAME_SCHEMA_VERSION", + "R_LOOP_MEMORY_HANDOFF_SEMANTIC_HINT_STATUSES", + "R_LOOP_MEMORY_HANDOFF_STATUSES", + "SOURCE_VERSION_LINEAGE_CODE_GENERATOR", + "SOURCE_VERSION_LINEAGE_FRAME_SCHEMA_NAME", + "SOURCE_VERSION_LINEAGE_FRAME_SCHEMA_VERSION", + "SOURCE_VERSION_LINEAGE_STATUSES", + "SOURCE_OBSERVATION_LEDGER_CODE_GENERATOR", + "SOURCE_OBSERVATION_LEDGER_FRAME_SCHEMA_NAME", + "SOURCE_OBSERVATION_LEDGER_FRAME_SCHEMA_VERSION", + "SOURCE_OBSERVATION_LEDGER_STATUSES", + "SOURCE_OBSERVATION_STATUSES", + "SUMMARY_INVALIDATION_LEDGER_CODE_GENERATOR", + "SUMMARY_INVALIDATION_LEDGER_FRAME_SCHEMA_NAME", + "SUMMARY_INVALIDATION_LEDGER_FRAME_SCHEMA_VERSION", + "SUMMARY_INVALIDATION_LEDGER_STATUSES", + "NIGHT_TIME_BUNDLE_SUMMARY_FRAME_SCHEMA_NAME", + "NIGHT_TIME_BUNDLE_SUMMARY_FRAME_SCHEMA_VERSION", + "NIGHT_TIME_BUNDLE_SUMMARY_STATUSES", + "NIGHT_TIME_BUNDLE_SUMMARY_FAILURE_TYPES", + "NIGHT_TIME_BUNDLE_SUMMARY_PARSE_STATUSES", + "NIGHT_TIME_BUNDLE_SUMMARY_VALIDITY_STATUSES", + "NIGHT_TIME_BUNDLE_SUMMARY_REVIEW_STATUSES", + "NIGHT_SOURCE_LEAF_SUMMARY_FAILURE_TYPES", + "NIGHT_SOURCE_LEAF_SUMMARY_FRAME_SCHEMA_NAME", + "NIGHT_SOURCE_LEAF_SUMMARY_FRAME_SCHEMA_VERSION", + "NIGHT_SOURCE_LEAF_SUMMARY_PARSE_STATUSES", + "NIGHT_SOURCE_LEAF_SUMMARY_REVIEW_STATUSES", + "NIGHT_SOURCE_LEAF_SUMMARY_STATUSES", + "NIGHT_SOURCE_LEAF_SUMMARY_VALIDITY_STATUSES", + "CoreEgoGuideWorkerHintFrame", + "CoreEgoTimeAxisFrame", + "GraphMemoryEdgeFrame", + "GraphMemoryNodeFrame", + "GraphMemorySnapshotFrame", + "NightSourceLeafSummaryFrame", + "NightTimeBundleSummaryFrame", + "RLoopMemoryHandoffPacketFrame", + "RLoopGraphGuidePacketFrame", + "SourceObservationLedgerFrame", + "SourceVersionLineageFrame", + "SummaryInvalidationLedgerFrame", + "TurnActivityGraphLinkFrame", + "TurnGraphAccessLedgerFrame", + "validate_core_ego_guide_worker_hint_frame", + "validate_core_ego_time_axis_frame", + "validate_graph_memory_edge_frame", + "validate_graph_memory_node_frame", + "validate_graph_memory_snapshot_frame", + "validate_night_source_leaf_summary_frame", + "validate_night_time_bundle_summary_frame", + "validate_r_loop_memory_handoff_packet_frame", + "validate_rloop_graph_guide_packet_frame", + "validate_source_observation_ledger_frame", + "validate_source_version_lineage_frame", + "validate_summary_invalidation_ledger_frame", + "validate_turn_activity_graph_link_frame", + "validate_turn_graph_access_ledger_frame", +] diff --git a/songryeon_core/core/schema_parts/loop_activity.py b/songryeon_core/core/schema_parts/loop_activity.py new file mode 100644 index 0000000..6add00d --- /dev/null +++ b/songryeon_core/core/schema_parts/loop_activity.py @@ -0,0 +1,287 @@ +from __future__ import annotations + +from dataclasses import dataclass, field + +from songryeon_core.core.schema_parts.base import ( + _validate_no_duplicates, + _validate_string_list, +) + + +L_LOOP_ACTIVITY_LEDGER_FRAME_SCHEMA_NAME = "LLoopActivityLedgerFrame" +L_LOOP_ACTIVITY_LEDGER_FRAME_SCHEMA_VERSION = "0.1" +L_LOOP_ACTIVITY_LEDGER_GENERATOR = "CODE:L_LOOP_ACTIVITY_LEDGER" +L_LOOP_ACTIVITY_LEDGER_DATA_TYPE = "loop_activity:l_loop_activity_ledger_frame" + +L_LOOP_ACTIVITY_STAGES = { + "run_frame", + "goal", + "budget_plan", + "tool_scope", + "tool_budget_partition", + "tool_catalog", + "tool_choice", + "query_plan", + "query", + "control", + "tool_result", + "tool_distillation", + "tool_budget", + "continuation", + "revision_input", + "revision_query_plan", + "revision_query", + "failure_signal", + "explicit_artifact_reference", + "document_context_pack", + "l3_preserved", + "l3_achievement", + "return_summary", + "document_material_packet", +} + + +@dataclass +class LLoopActivityLedgerFrame: + """An absolute per-turn index of records produced or used by one L loop run.""" + + frame_id: str + turn_id: str + run_index: int + turn_capsule_graph_node_id: str + loop_id: str = "L" + run_frame_data_ids: list[str] = field(default_factory=list) + goal_data_ids: list[str] = field(default_factory=list) + budget_plan_data_ids: list[str] = field(default_factory=list) + tool_scope_data_ids: list[str] = field(default_factory=list) + tool_budget_partition_data_ids: list[str] = field(default_factory=list) + tool_catalog_data_ids: list[str] = field(default_factory=list) + tool_choice_data_ids: list[str] = field(default_factory=list) + query_plan_data_ids: list[str] = field(default_factory=list) + query_data_ids: list[str] = field(default_factory=list) + control_data_ids: list[str] = field(default_factory=list) + tool_result_data_ids: list[str] = field(default_factory=list) + tool_distillation_data_ids: list[str] = field(default_factory=list) + tool_budget_data_ids: list[str] = field(default_factory=list) + continuation_data_ids: list[str] = field(default_factory=list) + revision_input_data_ids: list[str] = field(default_factory=list) + revision_query_plan_data_ids: list[str] = field(default_factory=list) + revision_query_data_ids: list[str] = field(default_factory=list) + failure_signal_data_ids: list[str] = field(default_factory=list) + explicit_artifact_reference_data_ids: list[str] = field(default_factory=list) + document_context_pack_data_ids: list[str] = field(default_factory=list) + preserved_data_ids: list[str] = field(default_factory=list) + achievement_data_ids: list[str] = field(default_factory=list) + return_summary_frame_id: str | None = None + document_material_packet_frame_id: str | None = None + output_data_ids: list[str] = field(default_factory=list) + search_candidate_doc_ids: list[str] = field(default_factory=list) + read_doc_ids: list[str] = field(default_factory=list) + read_code_file_paths: list[str] = field(default_factory=list) + output_data_id_count: int = 0 + tool_result_count: int = 0 + search_candidate_doc_count: int = 0 + actual_read_doc_count: int = 0 + actual_read_code_file_count: int = 0 + document_material_item_count: int = 0 + activity_records: list[dict[str, str]] = field(default_factory=list) + source_trace_ids: list[str] = field(default_factory=list) + source_data_ids: list[str] = field(default_factory=list) + generated_by: str = L_LOOP_ACTIVITY_LEDGER_GENERATOR + info_class: str = "absolute" + semantic_judgement_status: str = "not_run" + schema_name: str = L_LOOP_ACTIVITY_LEDGER_FRAME_SCHEMA_NAME + schema_version: str = L_LOOP_ACTIVITY_LEDGER_FRAME_SCHEMA_VERSION + + +def validate_l_loop_activity_ledger_frame(frame: LLoopActivityLedgerFrame) -> None: + """Validate that an L loop ledger only indexes existing absolute coordinates.""" + + for field_name, value in { + "frame_id": frame.frame_id, + "turn_id": frame.turn_id, + "turn_capsule_graph_node_id": frame.turn_capsule_graph_node_id, + "loop_id": frame.loop_id, + "generated_by": frame.generated_by, + "info_class": frame.info_class, + "semantic_judgement_status": frame.semantic_judgement_status, + "schema_name": frame.schema_name, + "schema_version": frame.schema_version, + }.items(): + if not value: + raise ValueError(f"LLoopActivityLedgerFrame.{field_name} must not be empty") + if frame.schema_name != L_LOOP_ACTIVITY_LEDGER_FRAME_SCHEMA_NAME: + raise ValueError(f"unknown LLoopActivityLedgerFrame.schema_name: {frame.schema_name}") + if frame.schema_version != L_LOOP_ACTIVITY_LEDGER_FRAME_SCHEMA_VERSION: + raise ValueError( + f"unknown LLoopActivityLedgerFrame.schema_version: {frame.schema_version}" + ) + if frame.loop_id != "L": + raise ValueError("LLoopActivityLedgerFrame.loop_id must be L") + if frame.run_index < 1: + raise ValueError("LLoopActivityLedgerFrame.run_index must be positive") + if frame.generated_by != L_LOOP_ACTIVITY_LEDGER_GENERATOR: + raise ValueError("LLoopActivityLedgerFrame.generated_by must reveal code ledger") + if frame.info_class != "absolute": + raise ValueError("LLoopActivityLedgerFrame.info_class must be absolute") + if frame.semantic_judgement_status != "not_run": + raise ValueError("LLoopActivityLedgerFrame.semantic_judgement_status must be not_run") + + list_fields = { + "run_frame_data_ids": frame.run_frame_data_ids, + "goal_data_ids": frame.goal_data_ids, + "budget_plan_data_ids": frame.budget_plan_data_ids, + "tool_scope_data_ids": frame.tool_scope_data_ids, + "tool_budget_partition_data_ids": frame.tool_budget_partition_data_ids, + "tool_catalog_data_ids": frame.tool_catalog_data_ids, + "tool_choice_data_ids": frame.tool_choice_data_ids, + "query_plan_data_ids": frame.query_plan_data_ids, + "query_data_ids": frame.query_data_ids, + "control_data_ids": frame.control_data_ids, + "tool_result_data_ids": frame.tool_result_data_ids, + "tool_distillation_data_ids": frame.tool_distillation_data_ids, + "tool_budget_data_ids": frame.tool_budget_data_ids, + "continuation_data_ids": frame.continuation_data_ids, + "revision_input_data_ids": frame.revision_input_data_ids, + "revision_query_plan_data_ids": frame.revision_query_plan_data_ids, + "revision_query_data_ids": frame.revision_query_data_ids, + "failure_signal_data_ids": frame.failure_signal_data_ids, + "explicit_artifact_reference_data_ids": frame.explicit_artifact_reference_data_ids, + "document_context_pack_data_ids": frame.document_context_pack_data_ids, + "preserved_data_ids": frame.preserved_data_ids, + "achievement_data_ids": frame.achievement_data_ids, + "output_data_ids": frame.output_data_ids, + "search_candidate_doc_ids": frame.search_candidate_doc_ids, + "read_doc_ids": frame.read_doc_ids, + "read_code_file_paths": frame.read_code_file_paths, + "source_trace_ids": frame.source_trace_ids, + "source_data_ids": frame.source_data_ids, + } + for field_name, values in list_fields.items(): + _validate_string_list(f"LLoopActivityLedgerFrame.{field_name}", values) + _validate_no_duplicates(f"LLoopActivityLedgerFrame.{field_name}", values) + + expected_counts = { + "output_data_id_count": len(frame.output_data_ids), + "tool_result_count": len(frame.tool_result_data_ids), + "search_candidate_doc_count": len(frame.search_candidate_doc_ids), + "actual_read_doc_count": len(frame.read_doc_ids), + "actual_read_code_file_count": len(frame.read_code_file_paths), + } + actual_counts = { + "output_data_id_count": frame.output_data_id_count, + "tool_result_count": frame.tool_result_count, + "search_candidate_doc_count": frame.search_candidate_doc_count, + "actual_read_doc_count": frame.actual_read_doc_count, + "actual_read_code_file_count": frame.actual_read_code_file_count, + } + for field_name, value in actual_counts.items(): + if not isinstance(value, int): + raise TypeError(f"LLoopActivityLedgerFrame.{field_name} must be an integer") + if value < 0: + raise ValueError(f"LLoopActivityLedgerFrame.{field_name} must not be negative") + if value != expected_counts[field_name]: + raise ValueError(f"LLoopActivityLedgerFrame.{field_name} must mirror source list") + if not isinstance(frame.document_material_item_count, int): + raise TypeError("LLoopActivityLedgerFrame.document_material_item_count must be an integer") + if frame.document_material_item_count < 0: + raise ValueError("LLoopActivityLedgerFrame.document_material_item_count must not be negative") + + if frame.return_summary_frame_id is not None: + if not frame.return_summary_frame_id: + raise ValueError("LLoopActivityLedgerFrame.return_summary_frame_id must not be empty") + if frame.return_summary_frame_id not in frame.source_data_ids: + raise ValueError( + "LLoopActivityLedgerFrame.source_data_ids must include return_summary_frame_id" + ) + if frame.document_material_packet_frame_id is not None: + if not frame.document_material_packet_frame_id: + raise ValueError( + "LLoopActivityLedgerFrame.document_material_packet_frame_id must not be empty" + ) + if frame.document_material_packet_frame_id not in frame.source_data_ids: + raise ValueError( + "LLoopActivityLedgerFrame.source_data_ids must include " + "document_material_packet_frame_id" + ) + + activity_data_ids = { + data_id + for values in _activity_source_lists(frame) + for data_id in values + } + missing_source_ids = sorted(activity_data_ids - set(frame.source_data_ids)) + if missing_source_ids: + raise ValueError( + "LLoopActivityLedgerFrame.source_data_ids must include activity data ids: " + f"{missing_source_ids}" + ) + if not frame.activity_records: + raise ValueError("LLoopActivityLedgerFrame.activity_records must not be empty") + for record in frame.activity_records: + if not isinstance(record, dict): + raise ValueError("LLoopActivityLedgerFrame.activity_records must contain dict records") + stage = record.get("stage") + data_id = record.get("data_id") + source_field = record.get("source_field") + if not stage or not data_id or not source_field: + raise ValueError( + "LLoopActivityLedgerFrame.activity_records entries require " + "stage, data_id, source_field" + ) + if stage not in L_LOOP_ACTIVITY_STAGES: + raise ValueError(f"unknown L loop activity stage: {stage}") + if data_id not in frame.source_data_ids: + raise ValueError( + "LLoopActivityLedgerFrame.source_data_ids must include activity record data_id" + ) + + +def _activity_source_lists(frame: LLoopActivityLedgerFrame) -> list[list[str]]: + result = [ + frame.run_frame_data_ids, + frame.goal_data_ids, + frame.budget_plan_data_ids, + frame.tool_scope_data_ids, + frame.tool_budget_partition_data_ids, + frame.tool_catalog_data_ids, + frame.tool_choice_data_ids, + frame.query_plan_data_ids, + frame.query_data_ids, + frame.control_data_ids, + frame.tool_result_data_ids, + frame.tool_distillation_data_ids, + frame.tool_budget_data_ids, + frame.continuation_data_ids, + frame.revision_input_data_ids, + frame.revision_query_plan_data_ids, + frame.revision_query_data_ids, + frame.failure_signal_data_ids, + frame.explicit_artifact_reference_data_ids, + frame.document_context_pack_data_ids, + frame.preserved_data_ids, + frame.achievement_data_ids, + frame.output_data_ids, + ] + optional_ids = [ + data_id + for data_id in [ + frame.return_summary_frame_id, + frame.document_material_packet_frame_id, + ] + if data_id is not None + ] + if optional_ids: + result.append(optional_ids) + return result + + +__all__ = [ + "L_LOOP_ACTIVITY_LEDGER_DATA_TYPE", + "L_LOOP_ACTIVITY_LEDGER_FRAME_SCHEMA_NAME", + "L_LOOP_ACTIVITY_LEDGER_FRAME_SCHEMA_VERSION", + "L_LOOP_ACTIVITY_LEDGER_GENERATOR", + "L_LOOP_ACTIVITY_STAGES", + "LLoopActivityLedgerFrame", + "validate_l_loop_activity_ledger_frame", +] diff --git a/songryeon_core/core/schema_parts/r_loop.py b/songryeon_core/core/schema_parts/r_loop.py new file mode 100644 index 0000000..75ae1a2 --- /dev/null +++ b/songryeon_core/core/schema_parts/r_loop.py @@ -0,0 +1,716 @@ +from __future__ import annotations + +from dataclasses import dataclass, field + +from songryeon_core.core.schema_parts.base import ( + _validate_no_duplicates, + _validate_string_list, +) +from songryeon_core.core.schema_parts.graph_memory import GRAPH_MEMORY_NODE_KINDS + + +R1_GRAPH_GOAL_FRAME_SCHEMA_NAME = "R1GraphGoalFrame" +R1_GRAPH_GOAL_FRAME_SCHEMA_VERSION = "0.1" +R_LOOP_BUDGET_FRAME_SCHEMA_NAME = "RLoopBudgetFrame" +R_LOOP_BUDGET_FRAME_SCHEMA_VERSION = "0.1" +R2_GRAPH_NODE_SELECTION_FRAME_SCHEMA_NAME = "R2GraphNodeSelectionFrame" +R2_GRAPH_NODE_SELECTION_FRAME_SCHEMA_VERSION = "0.1" +R3_GRAPH_INSPECTION_FRAME_SCHEMA_NAME = "R3GraphInspectionFrame" +R3_GRAPH_INSPECTION_FRAME_SCHEMA_VERSION = "0.1" +R_GRAPH_TRAVERSAL_CANDIDATE_SURFACE_FRAME_SCHEMA_NAME = ( + "RGraphTraversalCandidateSurfaceFrame" +) +R_GRAPH_TRAVERSAL_CANDIDATE_SURFACE_FRAME_SCHEMA_VERSION = "0.1" +R_LOOP_CONTINUATION_FRAME_SCHEMA_NAME = "RLoopContinuationFrame" +R_LOOP_CONTINUATION_FRAME_SCHEMA_VERSION = "0.1" +R_LOOP_RETURN_SUMMARY_FRAME_SCHEMA_NAME = "RLoopReturnSummaryFrame" +R_LOOP_RETURN_SUMMARY_FRAME_SCHEMA_VERSION = "0.1" + +R_LOOP_SCHEMA_ONLY_GENERATOR = "CODE:R_LOOP_SCHEMA_ONLY" +R_GRAPH_TRAVERSAL_CANDIDATE_SURFACE_GENERATOR = ( + "CODE:R_GRAPH_TRAVERSAL_CANDIDATE_SURFACE" +) + +R_INFORMATION_GRANULARITIES = { + "raw", + "low_summary", + "medium_summary", + "high_summary", + "unknown", +} +R_SELECTION_STATUSES = {"selected", "none_selected", "failed"} +R_SUFFICIENCY_STATUSES = {"sufficient", "insufficient", "unknown"} +R_GRANULARITY_PROBLEM_STATUSES = {"none", "needs_lower_granularity", "unknown"} +R_BRANCH_PROBLEM_STATUSES = {"none", "wrong_branch", "unknown"} +R_RECOMMENDED_NEXT_ACTIONS = {"stop", "deeper", "switch_branch", "fail"} +R_LOOP_CONTINUATION_STATUSES = { + "stop_sufficient", + "continue_deeper", + "continue_switch_branch", + "stop_budget_exhausted", + "stop_no_actionable_path", + "stop_failed_final", +} +R_LOOP_NEXT_TARGETS = {"R2", "return_summary"} +R_LOOP_TASK_STATUSES = {"not_run", "sufficient", "partial", "failed"} +R_LOOP_INFO_CLASSES = {"relative", "mixed"} +R_LOOP_SEMANTIC_STATUSES = {"not_run", "ran", "failed"} +R_GRAPH_CANDIDATE_RELATIONS = {"child", "next", "previous"} + + +@dataclass +class R1GraphGoalFrame: + frame_id: str + graph_search_goal: str + required_information_granularity: str + allowed_summary_depth: int + max_traversal_depth: int + max_branch_switches: int + max_node_reads: int + max_context_tokens: int + stop_condition: str + source_graph_guide_packet_id: str + user_question_anchor_id: str = "" + source_data_ids: list[str] = field(default_factory=list) + source_trace_ids: list[str] = field(default_factory=list) + generated_by: str = R_LOOP_SCHEMA_ONLY_GENERATOR + info_class: str = "mixed" + semantic_judgement_status: str = "not_run" + schema_name: str = R1_GRAPH_GOAL_FRAME_SCHEMA_NAME + schema_version: str = R1_GRAPH_GOAL_FRAME_SCHEMA_VERSION + + +@dataclass +class RLoopBudgetFrame: + frame_id: str + source_r1_goal_frame_id: str + max_traversal_depth: int + max_branch_switches: int + max_node_reads: int + max_context_tokens: int + used_traversal_depth: int = 0 + used_branch_switches: int = 0 + used_node_reads: int = 0 + used_context_tokens: int = 0 + budget_status: str = "within_budget" + source_data_ids: list[str] = field(default_factory=list) + source_trace_ids: list[str] = field(default_factory=list) + generated_by: str = R_LOOP_SCHEMA_ONLY_GENERATOR + info_class: str = "absolute" + semantic_judgement_status: str = "not_run" + schema_name: str = R_LOOP_BUDGET_FRAME_SCHEMA_NAME + schema_version: str = R_LOOP_BUDGET_FRAME_SCHEMA_VERSION + + +@dataclass +class R2GraphNodeSelectionFrame: + frame_id: str + selection_scope: str + available_graph_node_ids: list[str] + selection_status: str + selected_graph_node_id: str | None + selection_reason: str + expected_information_granularity: str + expected_source_kind: str + source_r1_goal_frame_id: str + source_data_ids: list[str] = field(default_factory=list) + source_trace_ids: list[str] = field(default_factory=list) + generated_by: str = R_LOOP_SCHEMA_ONLY_GENERATOR + info_class: str = "mixed" + semantic_judgement_status: str = "not_run" + schema_name: str = R2_GRAPH_NODE_SELECTION_FRAME_SCHEMA_NAME + schema_version: str = R2_GRAPH_NODE_SELECTION_FRAME_SCHEMA_VERSION + + +@dataclass +class R3GraphInspectionFrame: + frame_id: str + inspected_graph_node_id: str + node_kind: str + child_node_count: int + child_node_ids: list[str] + summary_depth: int + source_leaf_count: int + current_information_granularity: str + sufficiency_status: str + granularity_problem_status: str + branch_problem_status: str + recommended_next_action: str + inspection_reason: str + source_r2_selection_frame_id: str + source_data_ids: list[str] = field(default_factory=list) + source_trace_ids: list[str] = field(default_factory=list) + generated_by: str = R_LOOP_SCHEMA_ONLY_GENERATOR + info_class: str = "mixed" + semantic_judgement_status: str = "not_run" + schema_name: str = R3_GRAPH_INSPECTION_FRAME_SCHEMA_NAME + schema_version: str = R3_GRAPH_INSPECTION_FRAME_SCHEMA_VERSION + + +@dataclass +class RGraphTraversalCandidateSurfaceFrame: + frame_id: str + source_r3_inspection_frame_id: str + inspected_graph_node_id: str + child_candidate_node_ids: list[str] = field(default_factory=list) + next_candidate_node_ids: list[str] = field(default_factory=list) + previous_candidate_node_ids: list[str] = field(default_factory=list) + candidate_graph_node_ids: list[str] = field(default_factory=list) + candidate_records: list[dict[str, str]] = field(default_factory=list) + candidate_count: int = 0 + source_data_ids: list[str] = field(default_factory=list) + source_trace_ids: list[str] = field(default_factory=list) + generated_by: str = R_GRAPH_TRAVERSAL_CANDIDATE_SURFACE_GENERATOR + info_class: str = "absolute" + semantic_judgement_status: str = "not_run" + schema_name: str = R_GRAPH_TRAVERSAL_CANDIDATE_SURFACE_FRAME_SCHEMA_NAME + schema_version: str = R_GRAPH_TRAVERSAL_CANDIDATE_SURFACE_FRAME_SCHEMA_VERSION + + +@dataclass +class RLoopContinuationFrame: + frame_id: str + source_r3_inspection_frame_id: str + source_budget_frame_id: str + continuation_status: str + continuation_reason_code: str + next_target_node: str + remaining_traversal_depth: int + remaining_branch_switches: int + remaining_node_reads: int + remaining_context_tokens: int + source_data_ids: list[str] = field(default_factory=list) + source_trace_ids: list[str] = field(default_factory=list) + generated_by: str = R_LOOP_SCHEMA_ONLY_GENERATOR + info_class: str = "absolute" + semantic_judgement_status: str = "not_run" + schema_name: str = R_LOOP_CONTINUATION_FRAME_SCHEMA_NAME + schema_version: str = R_LOOP_CONTINUATION_FRAME_SCHEMA_VERSION + + +@dataclass +class RLoopReturnSummaryFrame: + frame_id: str + r_loop_task_status: str + selected_entry_node_ids: list[str] + inspected_graph_node_ids: list[str] + final_information_granularity: str + summary_depth_used: int + continuation_status: str + budget_status: str + source_graph_node_ids: list[str] = field(default_factory=list) + source_data_ids: list[str] = field(default_factory=list) + source_trace_ids: list[str] = field(default_factory=list) + generated_by: str = R_LOOP_SCHEMA_ONLY_GENERATOR + info_class: str = "absolute" + semantic_judgement_status: str = "not_run" + schema_name: str = R_LOOP_RETURN_SUMMARY_FRAME_SCHEMA_NAME + schema_version: str = R_LOOP_RETURN_SUMMARY_FRAME_SCHEMA_VERSION + + +def validate_r1_graph_goal_frame(frame: R1GraphGoalFrame) -> None: + _require_text_fields( + "R1GraphGoalFrame", + { + "frame_id": frame.frame_id, + "graph_search_goal": frame.graph_search_goal, + "required_information_granularity": frame.required_information_granularity, + "stop_condition": frame.stop_condition, + "source_graph_guide_packet_id": frame.source_graph_guide_packet_id, + "generated_by": frame.generated_by, + "info_class": frame.info_class, + "semantic_judgement_status": frame.semantic_judgement_status, + "schema_name": frame.schema_name, + "schema_version": frame.schema_version, + }, + ) + _validate_schema(frame.schema_name, R1_GRAPH_GOAL_FRAME_SCHEMA_NAME, "R1GraphGoalFrame") + _validate_schema_version( + frame.schema_version, + R1_GRAPH_GOAL_FRAME_SCHEMA_VERSION, + "R1GraphGoalFrame", + ) + _validate_granularity("R1GraphGoalFrame.required_information_granularity", frame.required_information_granularity) + _validate_loop_semantic_fields( + info_class=frame.info_class, + semantic_judgement_status=frame.semantic_judgement_status, + class_name="R1GraphGoalFrame", + ) + _validate_non_negative_ints( + "R1GraphGoalFrame", + { + "allowed_summary_depth": frame.allowed_summary_depth, + "max_traversal_depth": frame.max_traversal_depth, + "max_branch_switches": frame.max_branch_switches, + "max_node_reads": frame.max_node_reads, + "max_context_tokens": frame.max_context_tokens, + }, + ) + _validate_string_list("R1GraphGoalFrame.source_data_ids", frame.source_data_ids) + _validate_string_list("R1GraphGoalFrame.source_trace_ids", frame.source_trace_ids) + _validate_no_duplicates("R1GraphGoalFrame.source_data_ids", frame.source_data_ids) + _validate_no_duplicates("R1GraphGoalFrame.source_trace_ids", frame.source_trace_ids) + if frame.source_graph_guide_packet_id not in frame.source_data_ids: + raise ValueError("R1GraphGoalFrame.source_data_ids must include source_graph_guide_packet_id") + + +def validate_r_loop_budget_frame(frame: RLoopBudgetFrame) -> None: + _require_text_fields( + "RLoopBudgetFrame", + { + "frame_id": frame.frame_id, + "source_r1_goal_frame_id": frame.source_r1_goal_frame_id, + "budget_status": frame.budget_status, + "generated_by": frame.generated_by, + "info_class": frame.info_class, + "semantic_judgement_status": frame.semantic_judgement_status, + "schema_name": frame.schema_name, + "schema_version": frame.schema_version, + }, + ) + _validate_schema(frame.schema_name, R_LOOP_BUDGET_FRAME_SCHEMA_NAME, "RLoopBudgetFrame") + _validate_schema_version( + frame.schema_version, + R_LOOP_BUDGET_FRAME_SCHEMA_VERSION, + "RLoopBudgetFrame", + ) + if frame.info_class != "absolute": + raise ValueError("RLoopBudgetFrame.info_class must be absolute") + if frame.semantic_judgement_status != "not_run": + raise ValueError("RLoopBudgetFrame.semantic_judgement_status must be not_run") + _validate_non_negative_ints( + "RLoopBudgetFrame", + { + "max_traversal_depth": frame.max_traversal_depth, + "max_branch_switches": frame.max_branch_switches, + "max_node_reads": frame.max_node_reads, + "max_context_tokens": frame.max_context_tokens, + "used_traversal_depth": frame.used_traversal_depth, + "used_branch_switches": frame.used_branch_switches, + "used_node_reads": frame.used_node_reads, + "used_context_tokens": frame.used_context_tokens, + }, + ) + if frame.used_traversal_depth > frame.max_traversal_depth: + raise ValueError("RLoopBudgetFrame.used_traversal_depth must not exceed max") + if frame.used_branch_switches > frame.max_branch_switches: + raise ValueError("RLoopBudgetFrame.used_branch_switches must not exceed max") + if frame.used_node_reads > frame.max_node_reads: + raise ValueError("RLoopBudgetFrame.used_node_reads must not exceed max") + if frame.used_context_tokens > frame.max_context_tokens: + raise ValueError("RLoopBudgetFrame.used_context_tokens must not exceed max") + _validate_string_list("RLoopBudgetFrame.source_data_ids", frame.source_data_ids) + _validate_string_list("RLoopBudgetFrame.source_trace_ids", frame.source_trace_ids) + _validate_no_duplicates("RLoopBudgetFrame.source_data_ids", frame.source_data_ids) + _validate_no_duplicates("RLoopBudgetFrame.source_trace_ids", frame.source_trace_ids) + if frame.source_r1_goal_frame_id not in frame.source_data_ids: + raise ValueError("RLoopBudgetFrame.source_data_ids must include source_r1_goal_frame_id") + + +def validate_r2_graph_node_selection_frame(frame: R2GraphNodeSelectionFrame) -> None: + _require_text_fields( + "R2GraphNodeSelectionFrame", + { + "frame_id": frame.frame_id, + "selection_scope": frame.selection_scope, + "selection_status": frame.selection_status, + "expected_information_granularity": frame.expected_information_granularity, + "expected_source_kind": frame.expected_source_kind, + "source_r1_goal_frame_id": frame.source_r1_goal_frame_id, + "generated_by": frame.generated_by, + "info_class": frame.info_class, + "semantic_judgement_status": frame.semantic_judgement_status, + "schema_name": frame.schema_name, + "schema_version": frame.schema_version, + }, + ) + _validate_schema( + frame.schema_name, + R2_GRAPH_NODE_SELECTION_FRAME_SCHEMA_NAME, + "R2GraphNodeSelectionFrame", + ) + _validate_schema_version( + frame.schema_version, + R2_GRAPH_NODE_SELECTION_FRAME_SCHEMA_VERSION, + "R2GraphNodeSelectionFrame", + ) + _validate_member("R2GraphNodeSelectionFrame.selection_status", frame.selection_status, R_SELECTION_STATUSES) + _validate_granularity("R2GraphNodeSelectionFrame.expected_information_granularity", frame.expected_information_granularity) + _validate_loop_semantic_fields( + info_class=frame.info_class, + semantic_judgement_status=frame.semantic_judgement_status, + class_name="R2GraphNodeSelectionFrame", + ) + _validate_string_list( + "R2GraphNodeSelectionFrame.available_graph_node_ids", + frame.available_graph_node_ids, + ) + _validate_string_list("R2GraphNodeSelectionFrame.source_data_ids", frame.source_data_ids) + _validate_string_list("R2GraphNodeSelectionFrame.source_trace_ids", frame.source_trace_ids) + _validate_no_duplicates( + "R2GraphNodeSelectionFrame.available_graph_node_ids", + frame.available_graph_node_ids, + ) + _validate_no_duplicates("R2GraphNodeSelectionFrame.source_data_ids", frame.source_data_ids) + _validate_no_duplicates("R2GraphNodeSelectionFrame.source_trace_ids", frame.source_trace_ids) + if frame.source_r1_goal_frame_id not in frame.source_data_ids: + raise ValueError("R2GraphNodeSelectionFrame.source_data_ids must include source_r1_goal_frame_id") + if frame.selection_status == "selected": + if not frame.selected_graph_node_id: + raise ValueError("selected R2GraphNodeSelectionFrame must include selected_graph_node_id") + if frame.selected_graph_node_id not in frame.available_graph_node_ids: + raise ValueError("R2GraphNodeSelectionFrame.selected_graph_node_id must be available") + elif frame.selected_graph_node_id is not None: + raise ValueError("non-selected R2GraphNodeSelectionFrame must not include selected_graph_node_id") + + +def validate_r3_graph_inspection_frame(frame: R3GraphInspectionFrame) -> None: + _require_text_fields( + "R3GraphInspectionFrame", + { + "frame_id": frame.frame_id, + "inspected_graph_node_id": frame.inspected_graph_node_id, + "node_kind": frame.node_kind, + "current_information_granularity": frame.current_information_granularity, + "sufficiency_status": frame.sufficiency_status, + "granularity_problem_status": frame.granularity_problem_status, + "branch_problem_status": frame.branch_problem_status, + "recommended_next_action": frame.recommended_next_action, + "source_r2_selection_frame_id": frame.source_r2_selection_frame_id, + "generated_by": frame.generated_by, + "info_class": frame.info_class, + "semantic_judgement_status": frame.semantic_judgement_status, + "schema_name": frame.schema_name, + "schema_version": frame.schema_version, + }, + ) + _validate_schema(frame.schema_name, R3_GRAPH_INSPECTION_FRAME_SCHEMA_NAME, "R3GraphInspectionFrame") + _validate_schema_version( + frame.schema_version, + R3_GRAPH_INSPECTION_FRAME_SCHEMA_VERSION, + "R3GraphInspectionFrame", + ) + _validate_member("R3GraphInspectionFrame.node_kind", frame.node_kind, GRAPH_MEMORY_NODE_KINDS) + _validate_granularity("R3GraphInspectionFrame.current_information_granularity", frame.current_information_granularity) + _validate_member("R3GraphInspectionFrame.sufficiency_status", frame.sufficiency_status, R_SUFFICIENCY_STATUSES) + _validate_member( + "R3GraphInspectionFrame.granularity_problem_status", + frame.granularity_problem_status, + R_GRANULARITY_PROBLEM_STATUSES, + ) + _validate_member( + "R3GraphInspectionFrame.branch_problem_status", + frame.branch_problem_status, + R_BRANCH_PROBLEM_STATUSES, + ) + _validate_member( + "R3GraphInspectionFrame.recommended_next_action", + frame.recommended_next_action, + R_RECOMMENDED_NEXT_ACTIONS, + ) + _validate_loop_semantic_fields( + info_class=frame.info_class, + semantic_judgement_status=frame.semantic_judgement_status, + class_name="R3GraphInspectionFrame", + ) + _validate_non_negative_ints( + "R3GraphInspectionFrame", + { + "child_node_count": frame.child_node_count, + "summary_depth": frame.summary_depth, + "source_leaf_count": frame.source_leaf_count, + }, + ) + if frame.child_node_count != len(frame.child_node_ids): + raise ValueError("R3GraphInspectionFrame.child_node_count must mirror child_node_ids length") + _validate_string_list("R3GraphInspectionFrame.child_node_ids", frame.child_node_ids) + _validate_string_list("R3GraphInspectionFrame.source_data_ids", frame.source_data_ids) + _validate_string_list("R3GraphInspectionFrame.source_trace_ids", frame.source_trace_ids) + _validate_no_duplicates("R3GraphInspectionFrame.child_node_ids", frame.child_node_ids) + _validate_no_duplicates("R3GraphInspectionFrame.source_data_ids", frame.source_data_ids) + _validate_no_duplicates("R3GraphInspectionFrame.source_trace_ids", frame.source_trace_ids) + if frame.source_r2_selection_frame_id not in frame.source_data_ids: + raise ValueError("R3GraphInspectionFrame.source_data_ids must include source_r2_selection_frame_id") + + +def validate_r_graph_traversal_candidate_surface_frame( + frame: RGraphTraversalCandidateSurfaceFrame, +) -> None: + _require_text_fields( + "RGraphTraversalCandidateSurfaceFrame", + { + "frame_id": frame.frame_id, + "source_r3_inspection_frame_id": frame.source_r3_inspection_frame_id, + "inspected_graph_node_id": frame.inspected_graph_node_id, + "generated_by": frame.generated_by, + "info_class": frame.info_class, + "semantic_judgement_status": frame.semantic_judgement_status, + "schema_name": frame.schema_name, + "schema_version": frame.schema_version, + }, + ) + _validate_schema( + frame.schema_name, + R_GRAPH_TRAVERSAL_CANDIDATE_SURFACE_FRAME_SCHEMA_NAME, + "RGraphTraversalCandidateSurfaceFrame", + ) + _validate_schema_version( + frame.schema_version, + R_GRAPH_TRAVERSAL_CANDIDATE_SURFACE_FRAME_SCHEMA_VERSION, + "RGraphTraversalCandidateSurfaceFrame", + ) + if frame.generated_by != R_GRAPH_TRAVERSAL_CANDIDATE_SURFACE_GENERATOR: + raise ValueError("RGraphTraversalCandidateSurfaceFrame.generated_by must be code") + if frame.info_class != "absolute": + raise ValueError("RGraphTraversalCandidateSurfaceFrame.info_class must be absolute") + if frame.semantic_judgement_status != "not_run": + raise ValueError( + "RGraphTraversalCandidateSurfaceFrame.semantic_judgement_status must be not_run" + ) + graph_id_fields = { + "child_candidate_node_ids": frame.child_candidate_node_ids, + "next_candidate_node_ids": frame.next_candidate_node_ids, + "previous_candidate_node_ids": frame.previous_candidate_node_ids, + "candidate_graph_node_ids": frame.candidate_graph_node_ids, + } + for field_name, values in graph_id_fields.items(): + _validate_string_list(f"RGraphTraversalCandidateSurfaceFrame.{field_name}", values) + _validate_no_duplicates(f"RGraphTraversalCandidateSurfaceFrame.{field_name}", values) + _validate_string_list("RGraphTraversalCandidateSurfaceFrame.source_data_ids", frame.source_data_ids) + _validate_string_list("RGraphTraversalCandidateSurfaceFrame.source_trace_ids", frame.source_trace_ids) + _validate_no_duplicates("RGraphTraversalCandidateSurfaceFrame.source_data_ids", frame.source_data_ids) + _validate_no_duplicates("RGraphTraversalCandidateSurfaceFrame.source_trace_ids", frame.source_trace_ids) + if frame.candidate_count != len(frame.candidate_graph_node_ids): + raise ValueError( + "RGraphTraversalCandidateSurfaceFrame.candidate_count must mirror candidate_graph_node_ids" + ) + if frame.source_r3_inspection_frame_id not in frame.source_data_ids: + raise ValueError( + "RGraphTraversalCandidateSurfaceFrame.source_data_ids must include R3 frame" + ) + expected_candidates = set( + [ + *frame.child_candidate_node_ids, + *frame.next_candidate_node_ids, + *frame.previous_candidate_node_ids, + ] + ) + if set(frame.candidate_graph_node_ids) != expected_candidates: + raise ValueError( + "RGraphTraversalCandidateSurfaceFrame.candidate_graph_node_ids must match relation fields" + ) + for index, record in enumerate(frame.candidate_records, start=1): + if not isinstance(record, dict): + raise ValueError("RGraphTraversalCandidateSurfaceFrame.candidate_records must be dicts") + candidate_node_id = record.get("candidate_node_id") + relation = record.get("relation") + source_id = record.get("source_id") + source_field = record.get("source_field") + if not candidate_node_id or not relation or not source_id or not source_field: + raise ValueError( + "RGraphTraversalCandidateSurfaceFrame.candidate_records entries require " + "candidate_node_id, relation, source_id, source_field" + ) + if relation not in R_GRAPH_CANDIDATE_RELATIONS: + raise ValueError(f"unknown R graph candidate relation: {relation}") + if candidate_node_id not in expected_candidates: + raise ValueError( + "RGraphTraversalCandidateSurfaceFrame candidate record node must be listed " + f"in relation fields at index {index}" + ) + if source_id not in frame.source_data_ids: + raise ValueError( + "RGraphTraversalCandidateSurfaceFrame.source_data_ids must include candidate source_id" + ) + + +def validate_r_loop_continuation_frame(frame: RLoopContinuationFrame) -> None: + _require_text_fields( + "RLoopContinuationFrame", + { + "frame_id": frame.frame_id, + "source_r3_inspection_frame_id": frame.source_r3_inspection_frame_id, + "source_budget_frame_id": frame.source_budget_frame_id, + "continuation_status": frame.continuation_status, + "continuation_reason_code": frame.continuation_reason_code, + "next_target_node": frame.next_target_node, + "generated_by": frame.generated_by, + "info_class": frame.info_class, + "semantic_judgement_status": frame.semantic_judgement_status, + "schema_name": frame.schema_name, + "schema_version": frame.schema_version, + }, + ) + _validate_schema(frame.schema_name, R_LOOP_CONTINUATION_FRAME_SCHEMA_NAME, "RLoopContinuationFrame") + _validate_schema_version( + frame.schema_version, + R_LOOP_CONTINUATION_FRAME_SCHEMA_VERSION, + "RLoopContinuationFrame", + ) + _validate_member( + "RLoopContinuationFrame.continuation_status", + frame.continuation_status, + R_LOOP_CONTINUATION_STATUSES, + ) + _validate_member("RLoopContinuationFrame.next_target_node", frame.next_target_node, R_LOOP_NEXT_TARGETS) + if frame.info_class != "absolute": + raise ValueError("RLoopContinuationFrame.info_class must be absolute") + if frame.semantic_judgement_status != "not_run": + raise ValueError("RLoopContinuationFrame.semantic_judgement_status must be not_run") + _validate_non_negative_ints( + "RLoopContinuationFrame", + { + "remaining_traversal_depth": frame.remaining_traversal_depth, + "remaining_branch_switches": frame.remaining_branch_switches, + "remaining_node_reads": frame.remaining_node_reads, + "remaining_context_tokens": frame.remaining_context_tokens, + }, + ) + _validate_string_list("RLoopContinuationFrame.source_data_ids", frame.source_data_ids) + _validate_string_list("RLoopContinuationFrame.source_trace_ids", frame.source_trace_ids) + _validate_no_duplicates("RLoopContinuationFrame.source_data_ids", frame.source_data_ids) + _validate_no_duplicates("RLoopContinuationFrame.source_trace_ids", frame.source_trace_ids) + if frame.source_r3_inspection_frame_id not in frame.source_data_ids: + raise ValueError("RLoopContinuationFrame.source_data_ids must include R3 frame") + if frame.source_budget_frame_id not in frame.source_data_ids: + raise ValueError("RLoopContinuationFrame.source_data_ids must include budget frame") + + +def validate_r_loop_return_summary_frame(frame: RLoopReturnSummaryFrame) -> None: + _require_text_fields( + "RLoopReturnSummaryFrame", + { + "frame_id": frame.frame_id, + "r_loop_task_status": frame.r_loop_task_status, + "final_information_granularity": frame.final_information_granularity, + "continuation_status": frame.continuation_status, + "budget_status": frame.budget_status, + "generated_by": frame.generated_by, + "info_class": frame.info_class, + "semantic_judgement_status": frame.semantic_judgement_status, + "schema_name": frame.schema_name, + "schema_version": frame.schema_version, + }, + ) + _validate_schema( + frame.schema_name, + R_LOOP_RETURN_SUMMARY_FRAME_SCHEMA_NAME, + "RLoopReturnSummaryFrame", + ) + _validate_schema_version( + frame.schema_version, + R_LOOP_RETURN_SUMMARY_FRAME_SCHEMA_VERSION, + "RLoopReturnSummaryFrame", + ) + _validate_member("RLoopReturnSummaryFrame.r_loop_task_status", frame.r_loop_task_status, R_LOOP_TASK_STATUSES) + _validate_granularity("RLoopReturnSummaryFrame.final_information_granularity", frame.final_information_granularity) + _validate_member( + "RLoopReturnSummaryFrame.continuation_status", + frame.continuation_status, + R_LOOP_CONTINUATION_STATUSES, + ) + if frame.info_class != "absolute": + raise ValueError("RLoopReturnSummaryFrame.info_class must be absolute") + if frame.semantic_judgement_status != "not_run": + raise ValueError("RLoopReturnSummaryFrame.semantic_judgement_status must be not_run") + _validate_non_negative_ints( + "RLoopReturnSummaryFrame", + {"summary_depth_used": frame.summary_depth_used}, + ) + _validate_string_list("RLoopReturnSummaryFrame.selected_entry_node_ids", frame.selected_entry_node_ids) + _validate_string_list("RLoopReturnSummaryFrame.inspected_graph_node_ids", frame.inspected_graph_node_ids) + _validate_string_list("RLoopReturnSummaryFrame.source_graph_node_ids", frame.source_graph_node_ids) + _validate_string_list("RLoopReturnSummaryFrame.source_data_ids", frame.source_data_ids) + _validate_string_list("RLoopReturnSummaryFrame.source_trace_ids", frame.source_trace_ids) + _validate_no_duplicates( + "RLoopReturnSummaryFrame.selected_entry_node_ids", + frame.selected_entry_node_ids, + ) + _validate_no_duplicates( + "RLoopReturnSummaryFrame.inspected_graph_node_ids", + frame.inspected_graph_node_ids, + ) + _validate_no_duplicates("RLoopReturnSummaryFrame.source_graph_node_ids", frame.source_graph_node_ids) + _validate_no_duplicates("RLoopReturnSummaryFrame.source_data_ids", frame.source_data_ids) + _validate_no_duplicates("RLoopReturnSummaryFrame.source_trace_ids", frame.source_trace_ids) + + +def _validate_loop_semantic_fields( + *, + info_class: str, + semantic_judgement_status: str, + class_name: str, +) -> None: + if info_class not in R_LOOP_INFO_CLASSES: + raise ValueError(f"{class_name}.info_class must be relative or mixed") + if semantic_judgement_status not in R_LOOP_SEMANTIC_STATUSES: + raise ValueError(f"unknown {class_name}.semantic_judgement_status: {semantic_judgement_status}") + + +def _validate_granularity(field_name: str, value: str) -> None: + _validate_member(field_name, value, R_INFORMATION_GRANULARITIES) + + +def _validate_schema(value: str, expected: str, class_name: str) -> None: + if value != expected: + raise ValueError(f"unknown {class_name} schema_name: {value}") + + +def _validate_schema_version(value: str, expected: str, class_name: str) -> None: + if value != expected: + raise ValueError(f"unknown {class_name} schema_version: {value}") + + +def _validate_member(field_name: str, value: str, allowed_values: set[str]) -> None: + if value not in allowed_values: + raise ValueError(f"unknown {field_name}: {value}") + + +def _validate_non_negative_ints(class_name: str, values: dict[str, int]) -> None: + for field_name, value in values.items(): + if not isinstance(value, int): + raise TypeError(f"{class_name}.{field_name} must be an integer") + if value < 0: + raise ValueError(f"{class_name}.{field_name} must not be negative") + + +def _require_text_fields(class_name: str, values: dict[str, str]) -> None: + for field_name, value in values.items(): + if not isinstance(value, str) or not value: + raise ValueError(f"{class_name}.{field_name} must not be empty") + + +__all__ = [ + "R1GraphGoalFrame", + "R2GraphNodeSelectionFrame", + "R3GraphInspectionFrame", + "RGraphTraversalCandidateSurfaceFrame", + "RLoopBudgetFrame", + "RLoopContinuationFrame", + "RLoopReturnSummaryFrame", + "R1_GRAPH_GOAL_FRAME_SCHEMA_NAME", + "R1_GRAPH_GOAL_FRAME_SCHEMA_VERSION", + "R2_GRAPH_NODE_SELECTION_FRAME_SCHEMA_NAME", + "R2_GRAPH_NODE_SELECTION_FRAME_SCHEMA_VERSION", + "R3_GRAPH_INSPECTION_FRAME_SCHEMA_NAME", + "R3_GRAPH_INSPECTION_FRAME_SCHEMA_VERSION", + "R_GRAPH_CANDIDATE_RELATIONS", + "R_GRAPH_TRAVERSAL_CANDIDATE_SURFACE_FRAME_SCHEMA_NAME", + "R_GRAPH_TRAVERSAL_CANDIDATE_SURFACE_FRAME_SCHEMA_VERSION", + "R_GRAPH_TRAVERSAL_CANDIDATE_SURFACE_GENERATOR", + "R_LOOP_BUDGET_FRAME_SCHEMA_NAME", + "R_LOOP_BUDGET_FRAME_SCHEMA_VERSION", + "R_LOOP_CONTINUATION_FRAME_SCHEMA_NAME", + "R_LOOP_CONTINUATION_FRAME_SCHEMA_VERSION", + "R_LOOP_RETURN_SUMMARY_FRAME_SCHEMA_NAME", + "R_LOOP_RETURN_SUMMARY_FRAME_SCHEMA_VERSION", + "R_LOOP_CONTINUATION_STATUSES", + "R_LOOP_SCHEMA_ONLY_GENERATOR", + "validate_r1_graph_goal_frame", + "validate_r2_graph_node_selection_frame", + "validate_r3_graph_inspection_frame", + "validate_r_graph_traversal_candidate_surface_frame", + "validate_r_loop_budget_frame", + "validate_r_loop_continuation_frame", + "validate_r_loop_return_summary_frame", +] diff --git a/songryeon_core/core/schemas.py b/songryeon_core/core/schemas.py index 145f97b..a9d45b9 100644 --- a/songryeon_core/core/schemas.py +++ b/songryeon_core/core/schemas.py @@ -9,6 +9,137 @@ _validate_no_duplicates, _validate_string_list, ) +from songryeon_core.core.schema_parts.graph_memory import ( + CORE_EGO_GUIDE_WORKER_HINT_FAILURE_TYPES, + CORE_EGO_GUIDE_WORKER_HINT_FRAME_SCHEMA_NAME, + CORE_EGO_GUIDE_WORKER_HINT_FRAME_SCHEMA_VERSION, + CORE_EGO_GUIDE_WORKER_HINT_STATUSES, + CORE_EGO_GUIDE_WORKER_PARSE_STATUSES, + CORE_EGO_TIME_AXIS_FRAME_SCHEMA_NAME, + CORE_EGO_TIME_AXIS_FRAME_SCHEMA_VERSION, + GRAPH_ACCESS_LEDGER_CODE_GENERATOR, + GRAPH_ACCESS_STAGES, + GRAPH_MEMORY_CODE_GENERATOR, + GRAPH_MEMORY_EDGE_FRAME_SCHEMA_NAME, + GRAPH_MEMORY_EDGE_FRAME_SCHEMA_VERSION, + GRAPH_MEMORY_EDGE_KINDS, + GRAPH_MEMORY_NODE_FRAME_SCHEMA_NAME, + GRAPH_MEMORY_NODE_FRAME_SCHEMA_VERSION, + GRAPH_MEMORY_NODE_KINDS, + GRAPH_MEMORY_SNAPSHOT_FRAME_SCHEMA_NAME, + GRAPH_MEMORY_SNAPSHOT_FRAME_SCHEMA_VERSION, + RLOOP_GRAPH_GUIDE_PACKET_FRAME_SCHEMA_NAME, + RLOOP_GRAPH_GUIDE_PACKET_FRAME_SCHEMA_VERSION, + RLOOP_GUIDE_CODE_GENERATOR, + R_LOOP_MEMORY_HANDOFF_PACKET_FRAME_SCHEMA_NAME, + R_LOOP_MEMORY_HANDOFF_PACKET_FRAME_SCHEMA_VERSION, + R_LOOP_MEMORY_HANDOFF_SEMANTIC_HINT_STATUSES, + R_LOOP_MEMORY_HANDOFF_STATUSES, + SOURCE_OBSERVATION_LEDGER_CODE_GENERATOR, + SOURCE_OBSERVATION_LEDGER_FRAME_SCHEMA_NAME, + SOURCE_OBSERVATION_LEDGER_FRAME_SCHEMA_VERSION, + SOURCE_OBSERVATION_LEDGER_STATUSES, + SOURCE_OBSERVATION_STATUSES, + SOURCE_VERSION_LINEAGE_CODE_GENERATOR, + SOURCE_VERSION_LINEAGE_FRAME_SCHEMA_NAME, + SOURCE_VERSION_LINEAGE_FRAME_SCHEMA_VERSION, + SOURCE_VERSION_LINEAGE_STATUSES, + SUMMARY_INVALIDATION_LEDGER_CODE_GENERATOR, + SUMMARY_INVALIDATION_LEDGER_FRAME_SCHEMA_NAME, + SUMMARY_INVALIDATION_LEDGER_FRAME_SCHEMA_VERSION, + SUMMARY_INVALIDATION_LEDGER_STATUSES, + NIGHT_SOURCE_LEAF_SUMMARY_FAILURE_TYPES, + NIGHT_SOURCE_LEAF_SUMMARY_FRAME_SCHEMA_NAME, + NIGHT_SOURCE_LEAF_SUMMARY_FRAME_SCHEMA_VERSION, + NIGHT_SOURCE_LEAF_SUMMARY_PARSE_STATUSES, + NIGHT_SOURCE_LEAF_SUMMARY_REVIEW_STATUSES, + NIGHT_SOURCE_LEAF_SUMMARY_STATUSES, + NIGHT_SOURCE_LEAF_SUMMARY_VALIDITY_STATUSES, + NIGHT_TIME_BUNDLE_SUMMARY_FAILURE_TYPES, + NIGHT_TIME_BUNDLE_SUMMARY_FRAME_SCHEMA_NAME, + NIGHT_TIME_BUNDLE_SUMMARY_FRAME_SCHEMA_VERSION, + NIGHT_TIME_BUNDLE_SUMMARY_PARSE_STATUSES, + NIGHT_TIME_BUNDLE_SUMMARY_REVIEW_STATUSES, + NIGHT_TIME_BUNDLE_SUMMARY_STATUSES, + NIGHT_TIME_BUNDLE_SUMMARY_VALIDITY_STATUSES, + TURN_ACTIVITY_GRAPH_LINK_ACTIVITY_KINDS, + TURN_ACTIVITY_GRAPH_LINK_CODE_GENERATOR, + TURN_ACTIVITY_GRAPH_LINK_FRAME_SCHEMA_NAME, + TURN_ACTIVITY_GRAPH_LINK_FRAME_SCHEMA_VERSION, + TURN_GRAPH_ACCESS_LEDGER_FRAME_SCHEMA_NAME, + TURN_GRAPH_ACCESS_LEDGER_FRAME_SCHEMA_VERSION, + CoreEgoGuideWorkerHintFrame, + CoreEgoTimeAxisFrame, + GraphMemoryEdgeFrame, + GraphMemoryNodeFrame, + GraphMemorySnapshotFrame, + NightSourceLeafSummaryFrame, + NightTimeBundleSummaryFrame, + RLoopGraphGuidePacketFrame, + RLoopMemoryHandoffPacketFrame, + SourceObservationLedgerFrame, + SourceVersionLineageFrame, + SummaryInvalidationLedgerFrame, + TurnActivityGraphLinkFrame, + TurnGraphAccessLedgerFrame, + validate_core_ego_guide_worker_hint_frame, + validate_core_ego_time_axis_frame, + validate_graph_memory_edge_frame, + validate_graph_memory_node_frame, + validate_graph_memory_snapshot_frame, + validate_night_source_leaf_summary_frame, + validate_night_time_bundle_summary_frame, + validate_r_loop_memory_handoff_packet_frame, + validate_rloop_graph_guide_packet_frame, + validate_source_observation_ledger_frame, + validate_source_version_lineage_frame, + validate_summary_invalidation_ledger_frame, + validate_turn_activity_graph_link_frame, + validate_turn_graph_access_ledger_frame, +) +from songryeon_core.core.schema_parts.r_loop import ( + R1GraphGoalFrame, + R2GraphNodeSelectionFrame, + R3GraphInspectionFrame, + RGraphTraversalCandidateSurfaceFrame, + RLoopBudgetFrame, + RLoopContinuationFrame, + RLoopReturnSummaryFrame, + R1_GRAPH_GOAL_FRAME_SCHEMA_NAME, + R1_GRAPH_GOAL_FRAME_SCHEMA_VERSION, + R2_GRAPH_NODE_SELECTION_FRAME_SCHEMA_NAME, + R2_GRAPH_NODE_SELECTION_FRAME_SCHEMA_VERSION, + R3_GRAPH_INSPECTION_FRAME_SCHEMA_NAME, + R3_GRAPH_INSPECTION_FRAME_SCHEMA_VERSION, + R_GRAPH_CANDIDATE_RELATIONS, + R_GRAPH_TRAVERSAL_CANDIDATE_SURFACE_FRAME_SCHEMA_NAME, + R_GRAPH_TRAVERSAL_CANDIDATE_SURFACE_FRAME_SCHEMA_VERSION, + R_GRAPH_TRAVERSAL_CANDIDATE_SURFACE_GENERATOR, + R_LOOP_BUDGET_FRAME_SCHEMA_NAME, + R_LOOP_BUDGET_FRAME_SCHEMA_VERSION, + R_LOOP_CONTINUATION_FRAME_SCHEMA_NAME, + R_LOOP_CONTINUATION_FRAME_SCHEMA_VERSION, + R_LOOP_CONTINUATION_STATUSES, + R_LOOP_RETURN_SUMMARY_FRAME_SCHEMA_NAME, + R_LOOP_RETURN_SUMMARY_FRAME_SCHEMA_VERSION, + R_LOOP_SCHEMA_ONLY_GENERATOR, + validate_r1_graph_goal_frame, + validate_r2_graph_node_selection_frame, + validate_r3_graph_inspection_frame, + validate_r_graph_traversal_candidate_surface_frame, + validate_r_loop_budget_frame, + validate_r_loop_continuation_frame, + validate_r_loop_return_summary_frame, +) +from songryeon_core.core.schema_parts.loop_activity import ( + L_LOOP_ACTIVITY_LEDGER_DATA_TYPE, + L_LOOP_ACTIVITY_LEDGER_FRAME_SCHEMA_NAME, + L_LOOP_ACTIVITY_LEDGER_FRAME_SCHEMA_VERSION, + L_LOOP_ACTIVITY_LEDGER_GENERATOR, + L_LOOP_ACTIVITY_STAGES, + LLoopActivityLedgerFrame, + validate_l_loop_activity_ledger_frame, +) from songryeon_core.core.schema_parts.task_ledger import ( TASK_FRAME_SCHEMA_NAME, TASK_FRAME_SCHEMA_VERSION, @@ -859,6 +990,10 @@ class RoutingDecisionFrame: schema_version: str = "0.2" +R_ROUTE_EXPERIMENTAL_POLICY_FLAG = "enable_r_route_experimental" +R_ROUTE_EXPERIMENTAL_NEXT_0_MODE = "r_loop_graph_guide_handoff" + + def validate_routing_decision_frame(frame: RoutingDecisionFrame) -> None: """RoutingDecisionFrame의 최소 절대정보 규칙을 확인한다.""" @@ -873,7 +1008,18 @@ def validate_routing_decision_frame(frame: RoutingDecisionFrame) -> None: }.items(): if not value: raise ValueError(f"RoutingDecisionFrame.{field_name} must not be empty") - if frame.route not in {"L", "2"}: + if frame.route == "R": + if frame.policy_flag != R_ROUTE_EXPERIMENTAL_POLICY_FLAG: + raise ValueError("RoutingDecisionFrame route=R requires experimental policy flag") + if frame.expected_next_0_mode != R_ROUTE_EXPERIMENTAL_NEXT_0_MODE: + raise ValueError("RoutingDecisionFrame route=R requires R graph handoff mode") + if not frame.route_source.startswith("LLM:"): + raise ValueError("RoutingDecisionFrame route=R must be selected by node_1 LLM") + if frame.llm_routing_status != "ran": + raise ValueError("RoutingDecisionFrame route=R requires llm_routing_status=ran") + if frame.route_rule_id != "llm_router": + raise ValueError("RoutingDecisionFrame route=R requires llm_router rule id") + elif frame.route not in {"L", "2"}: raise ValueError(f"unknown route: {frame.route}") if frame.llm_routing_status not in {"not_run", "ran", "failed"}: raise ValueError(f"unknown llm_routing_status: {frame.llm_routing_status}") @@ -987,6 +1133,191 @@ def validate_node2_boundary_review_frame(frame: Node2BoundaryReviewFrame) -> Non raise ValueError("Node2BoundaryReviewFrame.source_data_ids must not contain empty values") +NODE2_ANSWER_BASIS_FRAME_SCHEMA_NAME = "Node2AnswerBasisFrame" +NODE2_ANSWER_BASIS_FRAME_SCHEMA_VERSION = "0.1" +ANSWER_BASIS_MODES = { + "absolute_first", + "relative_allowed", + "mixed_or_uncertain", +} +BASIS_REASON_CODES = { + "code_verified_fact_required", + "user_asked_for_interpretation", + "multi_source_bundle", + "source_mapping_unclear", + "insufficient_grounding", + "partial_evidence_only", + "recent_conversation_basis_present", + "document_basis_present", + "runtime_state_basis_present", + "llm_mode_selection_failed", +} +EVIDENCE_ROLES = { + "primary_answer_basis", + "supporting_context", + "available_but_not_used", + "candidate_not_read", + "excluded_by_budget", + "failed_or_empty", + "not_supplied", +} +ANSWER_BASIS_INFO_CLASSES = {"relative", "mixed", "absolute_status"} +ANSWER_BASIS_SEMANTIC_STATUSES = {"ran", "failed"} +ANSWER_BASIS_FAILURE_TYPES = { + "none", + "adapter_missing", + "parse_failed", + "schema_failed", + "adapter_failed", +} +ANSWER_BASIS_PAYLOAD_PARSE_STATUSES = {"passed", "failed", "not_checked"} + + +@dataclass +class Node2EvidenceRole: + """node_2가 source 하나에 부여한 답변 근거 역할 판단.""" + + source_data_id: str + evidence_role: str + role_reason: str = "" + role_reason_info_class: str = "mixed" + + +@dataclass +class Node2AnswerBasisFrame: + """node_2가 node_3에게 넘기는 최종 답변 근거 자세 프레임.""" + + frame_id: str + turn_id: str + answer_basis_mode: str + basis_reason_codes: list[str] + mode_selection_reason: str + mode_selection_reason_info_class: str + evidence_roles: list[Node2EvidenceRole] = field(default_factory=list) + generated_by: str = "LLM:NODE_2" + info_class: str = "mixed" + semantic_judgement_status: str = "ran" + answer_basis_failure_type: str = "none" + answer_basis_llm_call_data_id: str | None = None + answer_basis_trace_event_id: str | None = None + answer_basis_validation_error: str = "" + answer_basis_raw_text_present: bool = False + answer_basis_prompt_ref: str = "" + answer_basis_payload_parse_status: str = "not_checked" + source_trace_ids: list[str] = field(default_factory=list) + source_data_ids: list[str] = field(default_factory=list) + schema_name: str = NODE2_ANSWER_BASIS_FRAME_SCHEMA_NAME + schema_version: str = NODE2_ANSWER_BASIS_FRAME_SCHEMA_VERSION + + +def validate_node2_answer_basis_frame(frame: Node2AnswerBasisFrame) -> None: + """Node2AnswerBasisFrame의 enum과 metainfo 경계를 확인한다.""" + + for field_name, value in { + "frame_id": frame.frame_id, + "turn_id": frame.turn_id, + "answer_basis_mode": frame.answer_basis_mode, + "mode_selection_reason": frame.mode_selection_reason, + "mode_selection_reason_info_class": frame.mode_selection_reason_info_class, + "generated_by": frame.generated_by, + "info_class": frame.info_class, + "semantic_judgement_status": frame.semantic_judgement_status, + "schema_name": frame.schema_name, + "schema_version": frame.schema_version, + }.items(): + if not value: + raise ValueError(f"Node2AnswerBasisFrame.{field_name} must not be empty") + if frame.schema_name != NODE2_ANSWER_BASIS_FRAME_SCHEMA_NAME: + raise ValueError(f"unknown Node2AnswerBasisFrame schema_name: {frame.schema_name}") + if frame.schema_version != NODE2_ANSWER_BASIS_FRAME_SCHEMA_VERSION: + raise ValueError(f"unknown Node2AnswerBasisFrame schema_version: {frame.schema_version}") + if frame.answer_basis_mode not in ANSWER_BASIS_MODES: + raise ValueError(f"unknown answer_basis_mode: {frame.answer_basis_mode}") + if frame.mode_selection_reason_info_class not in ANSWER_BASIS_INFO_CLASSES: + raise ValueError( + "unknown Node2AnswerBasisFrame.mode_selection_reason_info_class: " + f"{frame.mode_selection_reason_info_class}" + ) + if frame.info_class not in ANSWER_BASIS_INFO_CLASSES: + raise ValueError(f"unknown Node2AnswerBasisFrame.info_class: {frame.info_class}") + if frame.semantic_judgement_status not in ANSWER_BASIS_SEMANTIC_STATUSES: + raise ValueError( + "unknown Node2AnswerBasisFrame.semantic_judgement_status: " + f"{frame.semantic_judgement_status}" + ) + if frame.answer_basis_failure_type not in ANSWER_BASIS_FAILURE_TYPES: + raise ValueError( + "unknown Node2AnswerBasisFrame.answer_basis_failure_type: " + f"{frame.answer_basis_failure_type}" + ) + if frame.answer_basis_payload_parse_status not in ANSWER_BASIS_PAYLOAD_PARSE_STATUSES: + raise ValueError( + "unknown Node2AnswerBasisFrame.answer_basis_payload_parse_status: " + f"{frame.answer_basis_payload_parse_status}" + ) + if not isinstance(frame.answer_basis_raw_text_present, bool): + raise TypeError("Node2AnswerBasisFrame.answer_basis_raw_text_present must be bool") + if frame.answer_basis_llm_call_data_id is not None and not frame.answer_basis_llm_call_data_id: + raise ValueError("Node2AnswerBasisFrame.answer_basis_llm_call_data_id must not be empty") + if frame.answer_basis_trace_event_id is not None and not frame.answer_basis_trace_event_id: + raise ValueError("Node2AnswerBasisFrame.answer_basis_trace_event_id must not be empty") + if not frame.basis_reason_codes: + raise ValueError("Node2AnswerBasisFrame.basis_reason_codes must not be empty") + for code in frame.basis_reason_codes: + if code not in BASIS_REASON_CODES: + raise ValueError(f"unknown basis_reason_code: {code}") + if frame.semantic_judgement_status == "failed": + if frame.generated_by != "CODE:FALLBACK": + raise ValueError("failed answer basis frame must use generated_by=CODE:FALLBACK") + if frame.info_class != "absolute_status": + raise ValueError("failed answer basis frame must use info_class=absolute_status") + if frame.mode_selection_reason_info_class != "absolute_status": + raise ValueError( + "failed answer basis frame must use mode_selection_reason_info_class=absolute_status" + ) + if frame.basis_reason_codes != ["llm_mode_selection_failed"]: + raise ValueError("failed answer basis frame must use only llm_mode_selection_failed") + if frame.answer_basis_failure_type == "none": + raise ValueError("failed answer basis frame must expose a failure type") + else: + if not frame.generated_by.startswith("LLM:"): + raise ValueError("ran answer basis frame must preserve generated_by=LLM:*") + if frame.info_class not in {"relative", "mixed"}: + raise ValueError("ran answer basis frame must be relative or mixed") + if frame.mode_selection_reason_info_class not in {"relative", "mixed"}: + raise ValueError("ran answer basis reason must be relative or mixed") + if frame.answer_basis_failure_type != "none": + raise ValueError("ran answer basis frame must use failure_type=none") + + _validate_no_duplicates("Node2AnswerBasisFrame.basis_reason_codes", frame.basis_reason_codes) + _validate_string_list("Node2AnswerBasisFrame.source_trace_ids", frame.source_trace_ids) + _validate_string_list("Node2AnswerBasisFrame.source_data_ids", frame.source_data_ids) + for role in frame.evidence_roles: + _validate_node2_evidence_role(role, allowed_source_data_ids=frame.source_data_ids) + + +def _validate_node2_evidence_role( + role: Node2EvidenceRole, + *, + allowed_source_data_ids: list[str], +) -> None: + for field_name, value in { + "source_data_id": role.source_data_id, + "evidence_role": role.evidence_role, + "role_reason_info_class": role.role_reason_info_class, + }.items(): + if not value: + raise ValueError(f"Node2EvidenceRole.{field_name} must not be empty") + if role.evidence_role not in EVIDENCE_ROLES: + raise ValueError(f"unknown evidence_role: {role.evidence_role}") + if role.role_reason_info_class not in ANSWER_BASIS_INFO_CLASSES: + raise ValueError( + f"unknown Node2EvidenceRole.role_reason_info_class: {role.role_reason_info_class}" + ) + if role.evidence_role != "not_supplied" and role.source_data_id not in allowed_source_data_ids: + raise ValueError("Node2EvidenceRole.source_data_id must exist in frame.source_data_ids") + + NODE4_GATEKEEPER_FRAME_SCHEMA_NAME = "Node4GatekeeperFrame" NODE4_GATEKEEPER_FRAME_SCHEMA_VERSION = "0.1" NODE4_GATE_STATUSES = {"pass", "needs_revision", "failed"} @@ -1192,6 +1523,190 @@ def validate_turn_outcome_frame(frame: TurnOutcomeFrame) -> None: raise ValueError(f"TurnOutcomeFrame.{field_name} must not be empty") +NODE0_DOCUMENT_MATERIAL_PACKET_FRAME_SCHEMA_NAME = "Node0DocumentMaterialPacketFrame" +NODE0_DOCUMENT_MATERIAL_PACKET_FRAME_SCHEMA_VERSION = "0.1" +NODE0_DOCUMENT_MATERIAL_ROLES = { + "search_candidate", + "actual_tool_read_doc", + "supplied_document_context", + "excluded_document_context", + "unread_candidate", +} + + +@dataclass +class Node0DocumentMaterialItem: + """node_0이 L 이후 문서별 역할을 정리한 절대정보 항목.""" + + doc_id: str + document_name: str + source_roles: list[str] = field(default_factory=list) + was_search_candidate: bool = False + was_actual_tool_read_doc: bool = False + was_supplied_document_context: bool = False + was_excluded_document_context: bool = False + was_unread_candidate: bool = False + search_candidate_rank: int = 0 + actual_read_rank: int = 0 + supplied_context_rank: int = 0 + excluded_context_rank: int = 0 + char_count: int = 0 + source_trace_ids: list[str] = field(default_factory=list) + source_data_ids: list[str] = field(default_factory=list) + + +@dataclass +class Node0DocumentMaterialPacketFrame: + """L 이후 node_0이 문서 검색/읽기/context 공급 상태를 한 장부로 묶은 frame.""" + + frame_id: str + turn_id: str + items: list[Node0DocumentMaterialItem] = field(default_factory=list) + item_count: int = 0 + search_candidate_count: int = 0 + actual_tool_read_doc_count: int = 0 + supplied_document_context_count: int = 0 + excluded_document_context_count: int = 0 + unread_candidate_count: int = 0 + generated_by: str = "CODE:NODE0_DOCUMENT_MATERIAL_PACKET" + info_class: str = "absolute_material_index" + semantic_judgement_status: str = "not_run" + source_trace_ids: list[str] = field(default_factory=list) + source_data_ids: list[str] = field(default_factory=list) + schema_name: str = NODE0_DOCUMENT_MATERIAL_PACKET_FRAME_SCHEMA_NAME + schema_version: str = NODE0_DOCUMENT_MATERIAL_PACKET_FRAME_SCHEMA_VERSION + + +def validate_node0_document_material_packet_frame( + frame: Node0DocumentMaterialPacketFrame, +) -> None: + """Node0DocumentMaterialPacketFrame이 의미 판단 없이 문서 장부만 담는지 확인한다.""" + + for field_name, value in { + "frame_id": frame.frame_id, + "turn_id": frame.turn_id, + "generated_by": frame.generated_by, + "info_class": frame.info_class, + "semantic_judgement_status": frame.semantic_judgement_status, + "schema_name": frame.schema_name, + "schema_version": frame.schema_version, + }.items(): + if not value: + raise ValueError( + f"Node0DocumentMaterialPacketFrame.{field_name} must not be empty" + ) + if frame.schema_name != NODE0_DOCUMENT_MATERIAL_PACKET_FRAME_SCHEMA_NAME: + raise ValueError( + "unknown Node0DocumentMaterialPacketFrame schema_name: " + f"{frame.schema_name}" + ) + if frame.schema_version != NODE0_DOCUMENT_MATERIAL_PACKET_FRAME_SCHEMA_VERSION: + raise ValueError( + "unknown Node0DocumentMaterialPacketFrame schema_version: " + f"{frame.schema_version}" + ) + if frame.generated_by != "CODE:NODE0_DOCUMENT_MATERIAL_PACKET": + raise ValueError("Node0DocumentMaterialPacketFrame.generated_by must reveal node_0 code") + if frame.info_class != "absolute_material_index": + raise ValueError( + "Node0DocumentMaterialPacketFrame.info_class must be absolute_material_index" + ) + if frame.semantic_judgement_status != "not_run": + raise ValueError( + "Node0DocumentMaterialPacketFrame.semantic_judgement_status must be not_run" + ) + + expected_counts = { + "item_count": len(frame.items), + "search_candidate_count": sum(1 for item in frame.items if item.was_search_candidate), + "actual_tool_read_doc_count": sum( + 1 for item in frame.items if item.was_actual_tool_read_doc + ), + "supplied_document_context_count": sum( + 1 for item in frame.items if item.was_supplied_document_context + ), + "excluded_document_context_count": sum( + 1 for item in frame.items if item.was_excluded_document_context + ), + "unread_candidate_count": sum(1 for item in frame.items if item.was_unread_candidate), + } + actual_counts = { + "item_count": frame.item_count, + "search_candidate_count": frame.search_candidate_count, + "actual_tool_read_doc_count": frame.actual_tool_read_doc_count, + "supplied_document_context_count": frame.supplied_document_context_count, + "excluded_document_context_count": frame.excluded_document_context_count, + "unread_candidate_count": frame.unread_candidate_count, + } + for field_name, value in actual_counts.items(): + if not isinstance(value, int): + raise TypeError(f"Node0DocumentMaterialPacketFrame.{field_name} must be an integer") + if value < 0: + raise ValueError(f"Node0DocumentMaterialPacketFrame.{field_name} must not be negative") + if value != expected_counts[field_name]: + raise ValueError( + f"Node0DocumentMaterialPacketFrame.{field_name} must mirror items" + ) + + seen_doc_ids: set[str] = set() + for item in frame.items: + _validate_node0_document_material_item(item) + if item.doc_id in seen_doc_ids: + raise ValueError("Node0DocumentMaterialPacketFrame.items must have unique doc_id") + seen_doc_ids.add(item.doc_id) + for trace_id in frame.source_trace_ids: + if not trace_id: + raise ValueError( + "Node0DocumentMaterialPacketFrame.source_trace_ids must not contain empty values" + ) + for data_id in frame.source_data_ids: + if not data_id: + raise ValueError( + "Node0DocumentMaterialPacketFrame.source_data_ids must not contain empty values" + ) + + +def _validate_node0_document_material_item(item: Node0DocumentMaterialItem) -> None: + if not item.doc_id: + raise ValueError("Node0DocumentMaterialItem.doc_id must not be empty") + if not item.document_name: + raise ValueError("Node0DocumentMaterialItem.document_name must not be empty") + for role in item.source_roles: + if role not in NODE0_DOCUMENT_MATERIAL_ROLES: + raise ValueError(f"unknown Node0DocumentMaterialItem.source_role: {role}") + if item.was_search_candidate and "search_candidate" not in item.source_roles: + raise ValueError("search candidate item must include search_candidate role") + if item.was_actual_tool_read_doc and "actual_tool_read_doc" not in item.source_roles: + raise ValueError("actual read item must include actual_tool_read_doc role") + if item.was_supplied_document_context and "supplied_document_context" not in item.source_roles: + raise ValueError("supplied context item must include supplied_document_context role") + if item.was_excluded_document_context and "excluded_document_context" not in item.source_roles: + raise ValueError("excluded context item must include excluded_document_context role") + if item.was_unread_candidate and "unread_candidate" not in item.source_roles: + raise ValueError("unread candidate item must include unread_candidate role") + if item.was_unread_candidate and not item.was_search_candidate: + raise ValueError("unread candidate item must also be a search candidate") + if item.was_unread_candidate and item.was_actual_tool_read_doc: + raise ValueError("unread candidate item must not be an actual read document") + for field_name, value in { + "search_candidate_rank": item.search_candidate_rank, + "actual_read_rank": item.actual_read_rank, + "supplied_context_rank": item.supplied_context_rank, + "excluded_context_rank": item.excluded_context_rank, + "char_count": item.char_count, + }.items(): + if not isinstance(value, int): + raise TypeError(f"Node0DocumentMaterialItem.{field_name} must be an integer") + if value < 0: + raise ValueError(f"Node0DocumentMaterialItem.{field_name} must not be negative") + for trace_id in item.source_trace_ids: + if not trace_id: + raise ValueError("Node0DocumentMaterialItem.source_trace_ids must not contain empty values") + for data_id in item.source_data_ids: + if not data_id: + raise ValueError("Node0DocumentMaterialItem.source_data_ids must not contain empty values") + + @dataclass class Node2InputFrame: """0이 2 메타정보 경계관에게 넘길 입력 범위를 정리한 프레임.""" @@ -1277,8 +1792,18 @@ class Node2HandoffFrame: reportable_document_count: int = 0 raw_document_extract_record_count: int = 0 empty_document_extract_record_count: int = 0 + reportable_code_file_count: int = 0 + raw_code_extract_record_count: int = 0 + empty_code_extract_record_count: int = 0 # 호환 필드. ORDER_098부터 의미는 reportable_document_count와 같다. read_doc_count: int = 0 + document_context_pack_frame_id: str | None = None + document_context_included_count: int = 0 + document_context_excluded_count: int = 0 + document_context_cutoff_reason: str = "" + document_material_packet_frame_id: str | None = None + document_material_item_count: int = 0 + document_material_unread_candidate_count: int = 0 actual_l_run_count: int = 0 blocked_same_turn_l_reroute_request_count: int = 0 same_turn_l_reroute_controller_decisions: list[str] = field(default_factory=list) @@ -1330,7 +1855,14 @@ def validate_node2_handoff_frame(frame: Node2HandoffFrame) -> None: "reportable_document_count": frame.reportable_document_count, "raw_document_extract_record_count": frame.raw_document_extract_record_count, "empty_document_extract_record_count": frame.empty_document_extract_record_count, + "reportable_code_file_count": frame.reportable_code_file_count, + "raw_code_extract_record_count": frame.raw_code_extract_record_count, + "empty_code_extract_record_count": frame.empty_code_extract_record_count, "read_doc_count": frame.read_doc_count, + "document_context_included_count": frame.document_context_included_count, + "document_context_excluded_count": frame.document_context_excluded_count, + "document_material_item_count": frame.document_material_item_count, + "document_material_unread_candidate_count": frame.document_material_unread_candidate_count, "actual_l_run_count": frame.actual_l_run_count, "blocked_same_turn_l_reroute_request_count": frame.blocked_same_turn_l_reroute_request_count, "l_internal_revision_count": frame.l_internal_revision_count, @@ -1345,7 +1877,10 @@ def validate_node2_handoff_frame(frame: Node2HandoffFrame) -> None: raise ValueError(f"Node2HandoffFrame.{field_name} must not be negative") if frame.read_doc_count != frame.reportable_document_count: raise ValueError("Node2HandoffFrame.read_doc_count must mirror reportable_document_count") - if frame.raw_document_extract_record_count < frame.reportable_document_count: + if ( + frame.document_context_pack_frame_id is None + and frame.raw_document_extract_record_count < frame.reportable_document_count + ): raise ValueError( "Node2HandoffFrame.raw_document_extract_record_count must not be smaller than reportable_document_count" ) @@ -1353,6 +1888,32 @@ def validate_node2_handoff_frame(frame: Node2HandoffFrame) -> None: raise ValueError( "Node2HandoffFrame.empty_document_extract_record_count must not exceed raw_document_extract_record_count" ) + if frame.empty_code_extract_record_count > frame.raw_code_extract_record_count: + raise ValueError( + "Node2HandoffFrame.empty_code_extract_record_count must not exceed raw_code_extract_record_count" + ) + if frame.document_context_pack_frame_id is not None: + if not frame.document_context_pack_frame_id: + raise ValueError("Node2HandoffFrame.document_context_pack_frame_id must not be empty") + if frame.document_context_pack_frame_id not in frame.source_data_ids: + raise ValueError( + "Node2HandoffFrame.source_data_ids must include document_context_pack_frame_id" + ) + if frame.reportable_document_count != frame.document_context_included_count: + raise ValueError( + "document context pack handoff reportable count must mirror included count" + ) + elif frame.document_context_included_count != 0 or frame.document_context_excluded_count != 0: + raise ValueError("missing document context pack frame requires zero pack counts") + if frame.document_material_packet_frame_id is not None: + if not frame.document_material_packet_frame_id: + raise ValueError("Node2HandoffFrame.document_material_packet_frame_id must not be empty") + if frame.document_material_packet_frame_id not in frame.source_data_ids: + raise ValueError( + "Node2HandoffFrame.source_data_ids must include document_material_packet_frame_id" + ) + elif frame.document_material_item_count != 0 or frame.document_material_unread_candidate_count != 0: + raise ValueError("missing document material packet frame requires zero material counts") if frame.memory_relevance_selection_status not in ( MEMORY_RELEVANCE_SELECTION_STATUSES | {"not_recorded"} ): @@ -1438,6 +1999,42 @@ def validate_node2_handoff_frame(frame: Node2HandoffFrame) -> None: NODE3_INPUT_BRIEF_FRAME_SCHEMA_NAME = "Node3InputBriefFrame" NODE3_INPUT_BRIEF_FRAME_SCHEMA_VERSION = "0.1" NODE3_INPUT_BRIEF_STATUSES = {"ready", "insufficient"} +NODE3_DOCUMENT_CONTEXT_PACK_STATUSES = { + "not_recorded", + "packed", + "no_candidates", +} +NODE3_MATERIAL_DELIVERY_POLICY_FRAME_SCHEMA_NAME = "Node3MaterialDeliveryPolicyFrame" +NODE3_MATERIAL_DELIVERY_POLICY_FRAME_SCHEMA_VERSION = "0.1" +NODE3_MATERIAL_DELIVERY_MODES = { + "raw_document_primary", + "l3_summary_replaces_raw_context", + "l3_summary_replaces_raw_context_with_uncertainty", + "raw_document_fallback_no_l3_summary", +} +NODE3_RAW_DOCUMENT_POLICIES = { + "preserve_supplied_raw_context", + "omit_raw_text_from_llm_payload", + "preserve_raw_context_because_l3_summary_missing", +} +NODE3_L3_SUMMARY_POLICIES = { + "auxiliary_only", + "replace_raw_context_with_labeled_l3_summary", + "replace_raw_context_with_labeled_l3_summary_and_limits", + "unavailable", +} +NODE3_UNCERTAINTY_POLICIES = { + "do_not_replace_raw_with_summary", + "keep_summary_boundary_visible", + "surface_partial_or_bundle_based_grounding", + "expose_summary_absence", +} +NODE3_MATERIAL_POLICY_REASON_CODES = { + "absolute_first_requires_checkable_material", + "relative_allowed_uses_l3_summary_to_reduce_context_volume", + "mixed_or_uncertain_uses_l3_summary_with_limit_visibility", + "l3_summary_unavailable_cannot_replace_raw_context", +} @dataclass @@ -1450,6 +2047,18 @@ class Node3BriefDocument: source_data_id: str = "" +@dataclass +class Node3ExcludedDocumentContext: + """node_3에게 읽은 문서가 아니라는 경계와 함께 전달되는 제외 후보.""" + + document_name: str + char_count: int + selection_basis: str + exclusion_reason: str + would_exceed_budget: bool + source_data_id: str = "" + + @dataclass class Node3BriefClaim: """node_3가 답변 재료로 사용할 수 있는 근거 달린 의미 주장.""" @@ -1496,6 +2105,54 @@ class Node3SelectedRecentMemoryContext: copied_from: str = "" +@dataclass +class Node3L3DocumentSummaryMaterial: + """node_3가 L3 문서별 요약을 정보 등급 경계와 함께 볼 수 있게 하는 재료.""" + + document_name: str + source_char_count: int + summary_status: str + plain_document_summary: str = "" + plain_summary_info_class: str = "relative" + plain_summary_source_mode: str = "direct_record" + plain_summary_claim_alignment: str = "one_document_to_one_summary" + task_relevant_summary: str = "" + task_relevant_summary_info_class: str = "mixed" + task_relevant_summary_source_mode: str = "source_bundle" + task_relevant_summary_claim_alignment: str = "one_document_plus_task_context" + summary_limit_note: str = "" + generated_by: str = "" + semantic_judgement_status: str = "not_run" + source_data_id: str = "" + + +@dataclass +class Node3MaterialDeliveryPolicyFrame: + """node_2 answer_basis_mode를 node_3 재료 전달 정책으로 고정 변환한 프레임.""" + + frame_id: str + turn_id: str + answer_basis_mode: str + material_delivery_mode: str + raw_document_policy: str + l3_summary_policy: str + uncertainty_policy: str + policy_reason_code: str + supplied_document_context_count: int = 0 + l3_document_summary_count: int = 0 + llm_raw_document_text_count: int = 0 + llm_l3_summary_context_count: int = 0 + raw_context_replaced_by_summary_count: int = 0 + answer_basis_frame_id: str | None = None + generated_by: str = "CODE:ANSWER_BASIS_MATERIAL_POLICY" + info_class: str = "absolute_policy_decision" + semantic_judgement_status: str = "not_run" + source_trace_ids: list[str] = field(default_factory=list) + source_data_ids: list[str] = field(default_factory=list) + schema_name: str = NODE3_MATERIAL_DELIVERY_POLICY_FRAME_SCHEMA_NAME + schema_version: str = NODE3_MATERIAL_DELIVERY_POLICY_FRAME_SCHEMA_VERSION + + @dataclass class Node3BriefRuntimeTask: """node_3가 현재 턴 실행 순서를 설명할 때 사용할 수 있는 task 요약.""" @@ -1516,6 +2173,65 @@ class Node3BriefRuntimeTask: evidence_data_count: int = 0 +@dataclass +class Node3RLoopResultMaterial: + """node_3가 R route experimental 결과를 과장 없이 볼 수 있게 하는 장부.""" + + source_data_id: str + r_loop_task_status: str + continuation_status: str + budget_status: str + final_information_granularity: str + summary_depth_used: int + selected_entry_node_count: int + inspected_graph_node_count: int + source_graph_node_count: int + generated_by: str + info_class: str + semantic_judgement_status: str + attitude_hint: str = "r_loop_partial_or_skeleton_only" + + +@dataclass +class Node3SourceCodeSymbol: + """read_code_file 원문에서 code가 문법적으로 확인한 top-level symbol.""" + + # 절대 정보: Python AST에서 확인한 이름. 의미 요약이 아니다. + name: str + # 절대 정보: function, async_function, class, constant 중 하나. + symbol_kind: str + # 절대 정보: 원본 파일 안의 1-based line number. + line_number: int + # 절대 정보: 이름이 underscore로 시작하지 않는지에 따른 공개 표지. + is_public: bool + # 절대 정보: docstring 존재 여부. docstring 내용 자체는 여기서 요약하지 않는다. + docstring_present: bool = False + + +@dataclass +class Node3SourceCodeOutline: + """node_3가 source-code 답변 coverage를 놓치지 않게 보는 문법 장부.""" + + # 절대 정보: read_code_file payload의 file_path. + file_path: str + # 절대 정보: 현재 파서는 python만 구조화한다. + language: str + # 절대 정보: parsed, unsupported_language, parse_failed 중 하나. + parse_status: str + # 절대 정보: 이 outline이 대응하는 read_code_file DataStore record. + source_data_id: str + # 절대 정보: top-level symbol 수. + top_level_symbol_count: int = 0 + # 절대 정보: public symbol 수. + public_symbol_count: int = 0 + # 절대 정보: public top-level function 이름만 모은 coverage checklist. + public_function_names: list[str] = field(default_factory=list) + # 절대 정보: top-level symbol 목록. + top_level_symbols: list[Node3SourceCodeSymbol] = field(default_factory=list) + # 절대 정보: 파싱 실패 시 예외 종류만 기록한다. 의미 판단이 아니다. + parse_error_type: str = "" + + @dataclass class Node3InputBriefFrame: """node_3에게 내부 ID 장부 대신 의미 단위 입력을 제공하기 위한 브리프.""" @@ -1526,16 +2242,70 @@ class Node3InputBriefFrame: brief_status: str handoff_frame_id: str read_documents: list[Node3BriefDocument] = field(default_factory=list) - # 절대 정보: L3가 보존한 search_docs 후보 문서의 고유 개수. - # 후보 문서는 "검색 결과에 등장한 문서"일 뿐이고, read_documents처럼 원문을 읽은 문서가 아니다. + # 절대 정보: L루프에서 실제 read_doc 계열 도구가 기록한 원문 읽기 수. + # document_context_pack included count와 섞으면 안 된다. + actual_tool_read_doc_count: int = 0 + # 절대 정보: 실제 read_doc/read_artifact 도구가 기록한 문서명 목록. + actual_tool_read_doc_documents: list[str] = field(default_factory=list) + # 절대 정보: 실제 read_code_file 도구가 기록한 source/config 파일 수. + actual_tool_read_code_file_count: int = 0 + # 절대 정보: 실제 read_code_file 도구가 기록한 source/config 파일 경로 목록. + actual_tool_read_code_file_paths: list[str] = field(default_factory=list) + # 절대 정보: node_3에게 본문 context로 공급된 문서 수. + supplied_document_context_count: int = 0 + # 절대 정보: node_3에게 본문 context로 공급된 source-code context 수. + supplied_source_code_context_count: int = 0 + # 절대 정보: read_code_file 원문에서 code가 문법적으로 뽑은 source-code 구조 목록. + source_code_outlines: list[Node3SourceCodeOutline] = field(default_factory=list) + document_context_pack_frame_id: str | None = None + document_context_pack_status: str = "not_recorded" + excluded_document_contexts: list[Node3ExcludedDocumentContext] = field(default_factory=list) + document_material_packet_frame_id: str | None = None + document_material_items: list[Node0DocumentMaterialItem] = field(default_factory=list) + # 절대 정보: 최종/최신 L3 return summary 기준 search candidate 문서 수. + final_search_candidate_count: int = 0 + final_search_candidate_documents: list[str] = field(default_factory=list) + # 절대 정보: L3 initial/revision preserved frame 전체를 훑은 누적 search candidate 문서 수. + accumulated_search_candidate_count: int = 0 + accumulated_search_candidate_documents: list[str] = field(default_factory=list) + # 호환용 alias: 현재 의미는 final_search_candidate_*와 같다. search_candidate_count: int = 0 - # 절대 정보: node_3가 후보 규모를 사실대로 말할 수 있게 만든 문서명 목록. - # 내부 추적 ID 대신 사람이 읽을 수 있는 문서명만 넣는다. search_candidate_documents: list[str] = field(default_factory=list) allowed_claims: list[Node3BriefClaim] = field(default_factory=list) memory_selection_material: Node3MemorySelectionMaterial | None = None selected_recent_memory_contexts: list[Node3SelectedRecentMemoryContext] = field(default_factory=list) + l3_document_summaries: list[Node3L3DocumentSummaryMaterial] = field(default_factory=list) + material_delivery_policy_frame_id: str | None = None + material_delivery_mode: str = "raw_document_fallback_no_l3_summary" + raw_document_policy: str = "preserve_raw_context_because_l3_summary_missing" + l3_summary_policy: str = "unavailable" + uncertainty_policy: str = "expose_summary_absence" + material_policy_reason_code: str = "l3_summary_unavailable_cannot_replace_raw_context" + llm_raw_document_text_count: int = 0 + llm_l3_summary_context_count: int = 0 + raw_context_replaced_by_summary_count: int = 0 + material_policy_generated_by: str = "CODE:ANSWER_BASIS_MATERIAL_POLICY" + material_policy_info_class: str = "absolute_policy_decision" + material_policy_semantic_judgement_status: str = "not_run" runtime_tasks: list[Node3BriefRuntimeTask] = field(default_factory=list) + r_loop_result_material: Node3RLoopResultMaterial | None = None + l_loop_return_summary_frame_id: str | None = None + l_loop_task_status: str = "not_recorded" + l_loop_failure_level: str = "none" + l3_goal_match_status: str = "not_run" + l3_semantic_goal_match_status: str = "not_run" + remaining_query_attempts: int = 0 + remaining_read_doc_calls: int = 0 + l_loop_result_attitude_hint: str = "not_recorded" + answer_basis_frame_id: str | None = None + answer_basis_mode: str = "mixed_or_uncertain" + basis_reason_codes: list[str] = field(default_factory=lambda: ["llm_mode_selection_failed"]) + mode_selection_reason: str = "CODE_STATUS:node2_answer_basis_mode_selection_failed" + mode_selection_reason_info_class: str = "absolute_status" + evidence_roles: list[Node2EvidenceRole] = field(default_factory=list) + answer_basis_generated_by: str = "CODE:FALLBACK" + answer_basis_info_class: str = "absolute_status" + answer_basis_semantic_judgement_status: str = "failed" reporting_rules: list[str] = field(default_factory=list) insufficiency_reasons: list[str] = field(default_factory=list) source_trace_ids: list[str] = field(default_factory=list) @@ -1566,23 +2336,129 @@ def validate_node3_input_brief_frame(frame: Node3InputBriefFrame) -> None: raise ValueError(f"unknown Node3InputBriefFrame.brief_status: {frame.brief_status}") if frame.handoff_frame_id not in frame.source_data_ids: raise ValueError("Node3InputBriefFrame.source_data_ids must include handoff_frame_id") - if not isinstance(frame.search_candidate_count, int): - raise TypeError("Node3InputBriefFrame.search_candidate_count must be an integer") - if frame.search_candidate_count < 0: - raise ValueError("Node3InputBriefFrame.search_candidate_count must not be negative") - for document_name in frame.search_candidate_documents: - if not document_name: - raise ValueError("Node3InputBriefFrame.search_candidate_documents must not contain empty values") + if frame.document_context_pack_status not in NODE3_DOCUMENT_CONTEXT_PACK_STATUSES: + raise ValueError( + "unknown Node3InputBriefFrame.document_context_pack_status: " + f"{frame.document_context_pack_status}" + ) + if frame.document_context_pack_frame_id is not None: + if not frame.document_context_pack_frame_id: + raise ValueError("Node3InputBriefFrame.document_context_pack_frame_id must not be empty") + if frame.document_context_pack_frame_id not in frame.source_data_ids: + raise ValueError( + "Node3InputBriefFrame.source_data_ids must include document_context_pack_frame_id" + ) + elif frame.document_context_pack_status != "not_recorded": + raise ValueError("document context pack status requires document_context_pack_frame_id") + if frame.document_material_packet_frame_id is not None: + if not frame.document_material_packet_frame_id: + raise ValueError("Node3InputBriefFrame.document_material_packet_frame_id must not be empty") + if frame.document_material_packet_frame_id not in frame.source_data_ids: + raise ValueError( + "Node3InputBriefFrame.source_data_ids must include document_material_packet_frame_id" + ) + for field_name, (count, documents) in { + "final_search_candidate": ( + frame.final_search_candidate_count, + frame.final_search_candidate_documents, + ), + "accumulated_search_candidate": ( + frame.accumulated_search_candidate_count, + frame.accumulated_search_candidate_documents, + ), + "search_candidate": ( + frame.search_candidate_count, + frame.search_candidate_documents, + ), + }.items(): + if not isinstance(count, int): + raise TypeError(f"Node3InputBriefFrame.{field_name}_count must be an integer") + if count < 0: + raise ValueError(f"Node3InputBriefFrame.{field_name}_count must not be negative") + if count != len(documents): + raise ValueError( + f"Node3InputBriefFrame.{field_name}_count must mirror documents length" + ) + for document_name in documents: + if not document_name: + raise ValueError( + f"Node3InputBriefFrame.{field_name}_documents must not contain empty values" + ) + if frame.search_candidate_count != frame.final_search_candidate_count: + raise ValueError( + "Node3InputBriefFrame.search_candidate_count must mirror final_search_candidate_count" + ) + if frame.search_candidate_documents != frame.final_search_candidate_documents: + raise ValueError( + "Node3InputBriefFrame.search_candidate_documents must mirror final_search_candidate_documents" + ) for document in frame.read_documents: _validate_node3_brief_document(document) + for outline in frame.source_code_outlines: + _validate_node3_source_code_outline(outline) + if outline.source_data_id not in frame.source_data_ids: + raise ValueError( + "Node3InputBriefFrame.source_data_ids must include source_code_outline source_data_id" + ) + for field_name, value in { + "actual_tool_read_doc_count": frame.actual_tool_read_doc_count, + "actual_tool_read_code_file_count": frame.actual_tool_read_code_file_count, + "supplied_document_context_count": frame.supplied_document_context_count, + "supplied_source_code_context_count": frame.supplied_source_code_context_count, + }.items(): + if not isinstance(value, int): + raise TypeError(f"Node3InputBriefFrame.{field_name} must be an integer") + if value < 0: + raise ValueError(f"Node3InputBriefFrame.{field_name} must not be negative") + if frame.supplied_document_context_count != len(frame.read_documents): + raise ValueError( + "Node3InputBriefFrame.supplied_document_context_count must mirror read_documents length" + ) + for document_name in frame.actual_tool_read_doc_documents: + if not document_name: + raise ValueError( + "Node3InputBriefFrame.actual_tool_read_doc_documents must not contain empty values" + ) + if frame.actual_tool_read_code_file_count != len(frame.actual_tool_read_code_file_paths): + raise ValueError( + "Node3InputBriefFrame.actual_tool_read_code_file_count must mirror actual_tool_read_code_file_paths length" + ) + if frame.supplied_source_code_context_count > frame.supplied_document_context_count: + raise ValueError( + "Node3InputBriefFrame.supplied_source_code_context_count must not exceed supplied_document_context_count" + ) + for file_path in frame.actual_tool_read_code_file_paths: + if not file_path: + raise ValueError( + "Node3InputBriefFrame.actual_tool_read_code_file_paths must not contain empty values" + ) + for document in frame.excluded_document_contexts: + _validate_node3_excluded_document_context(document) + for item in frame.document_material_items: + _validate_node0_document_material_item(item) for claim in frame.allowed_claims: _validate_node3_brief_claim(claim) if frame.memory_selection_material is not None: _validate_node3_memory_selection_material(frame.memory_selection_material) for context in frame.selected_recent_memory_contexts: _validate_node3_selected_recent_memory_context(context) + for summary in frame.l3_document_summaries: + _validate_node3_l3_document_summary_material(summary) + if summary.source_data_id not in frame.source_data_ids: + raise ValueError( + "Node3InputBriefFrame.source_data_ids must include L3 document summary source_data_id" + ) + _validate_node3_material_delivery_fields(frame) for runtime_task in frame.runtime_tasks: _validate_node3_brief_runtime_task(runtime_task) + if frame.r_loop_result_material is not None: + _validate_node3_r_loop_result_material(frame.r_loop_result_material) + if frame.r_loop_result_material.source_data_id not in frame.source_data_ids: + raise ValueError( + "Node3InputBriefFrame.source_data_ids must include R loop result source_data_id" + ) + _validate_node3_l_loop_result_fields(frame) + _validate_node3_answer_basis_fields(frame) for rule in frame.reporting_rules: if not rule: raise ValueError("Node3InputBriefFrame.reporting_rules must not contain empty values") @@ -1607,6 +2483,486 @@ def _validate_node3_brief_document(document: Node3BriefDocument) -> None: raise ValueError("Node3BriefDocument.char_count must not be negative") +def _validate_node3_source_code_outline(outline: Node3SourceCodeOutline) -> None: + for field_name, value in { + "file_path": outline.file_path, + "language": outline.language, + "parse_status": outline.parse_status, + "source_data_id": outline.source_data_id, + }.items(): + if not value: + raise ValueError(f"Node3SourceCodeOutline.{field_name} must not be empty") + if outline.parse_status not in {"parsed", "unsupported_language", "parse_failed"}: + raise ValueError(f"unknown Node3SourceCodeOutline.parse_status: {outline.parse_status}") + for field_name, value in { + "top_level_symbol_count": outline.top_level_symbol_count, + "public_symbol_count": outline.public_symbol_count, + }.items(): + if not isinstance(value, int): + raise TypeError(f"Node3SourceCodeOutline.{field_name} must be an integer") + if value < 0: + raise ValueError(f"Node3SourceCodeOutline.{field_name} must not be negative") + if outline.top_level_symbol_count != len(outline.top_level_symbols): + raise ValueError( + "Node3SourceCodeOutline.top_level_symbol_count must mirror top_level_symbols length" + ) + if outline.public_symbol_count != sum(1 for symbol in outline.top_level_symbols if symbol.is_public): + raise ValueError( + "Node3SourceCodeOutline.public_symbol_count must mirror public top_level_symbols length" + ) + for name in outline.public_function_names: + if not name: + raise ValueError("Node3SourceCodeOutline.public_function_names must not contain empty values") + symbol_public_functions = [ + symbol.name + for symbol in outline.top_level_symbols + if symbol.is_public and symbol.symbol_kind in {"function", "async_function"} + ] + if outline.public_function_names != symbol_public_functions: + raise ValueError( + "Node3SourceCodeOutline.public_function_names must mirror public function symbols" + ) + for symbol in outline.top_level_symbols: + _validate_node3_source_code_symbol(symbol) + + +def _validate_node3_source_code_symbol(symbol: Node3SourceCodeSymbol) -> None: + if not symbol.name: + raise ValueError("Node3SourceCodeSymbol.name must not be empty") + if symbol.symbol_kind not in {"function", "async_function", "class", "constant"}: + raise ValueError(f"unknown Node3SourceCodeSymbol.symbol_kind: {symbol.symbol_kind}") + if not isinstance(symbol.line_number, int): + raise TypeError("Node3SourceCodeSymbol.line_number must be an integer") + if symbol.line_number < 1: + raise ValueError("Node3SourceCodeSymbol.line_number must be positive") + if not isinstance(symbol.is_public, bool): + raise TypeError("Node3SourceCodeSymbol.is_public must be a boolean") + if not isinstance(symbol.docstring_present, bool): + raise TypeError("Node3SourceCodeSymbol.docstring_present must be a boolean") + + +def _validate_node3_l_loop_result_fields(frame: Node3InputBriefFrame) -> None: + if frame.l_loop_return_summary_frame_id is not None: + if not frame.l_loop_return_summary_frame_id: + raise ValueError("Node3InputBriefFrame.l_loop_return_summary_frame_id must not be empty") + if frame.l_loop_return_summary_frame_id not in frame.source_data_ids: + raise ValueError( + "Node3InputBriefFrame.source_data_ids must include l_loop_return_summary_frame_id" + ) + if frame.l_loop_task_status not in {"not_recorded", "achieved", "partial", "failed", "unknown"}: + raise ValueError(f"unknown Node3InputBriefFrame.l_loop_task_status: {frame.l_loop_task_status}") + if frame.l_loop_failure_level not in { + "not_recorded", + "none", + "l2_retryable", + "l1_replan_needed", + "budget_exhausted", + "give_up_recommended", + "unknown", + }: + raise ValueError( + f"unknown Node3InputBriefFrame.l_loop_failure_level: {frame.l_loop_failure_level}" + ) + if frame.l3_goal_match_status not in { + "matched", + "partial", + "missing", + "not_applicable", + "not_run", + }: + raise ValueError( + f"unknown Node3InputBriefFrame.l3_goal_match_status: {frame.l3_goal_match_status}" + ) + if frame.l3_semantic_goal_match_status not in { + "matched", + "partial", + "missing", + "not_run", + "not_applicable", + }: + raise ValueError( + "unknown Node3InputBriefFrame.l3_semantic_goal_match_status: " + f"{frame.l3_semantic_goal_match_status}" + ) + for field_name, value in { + "remaining_query_attempts": frame.remaining_query_attempts, + "remaining_read_doc_calls": frame.remaining_read_doc_calls, + }.items(): + if not isinstance(value, int): + raise TypeError(f"Node3InputBriefFrame.{field_name} must be an integer") + if value < 0: + raise ValueError(f"Node3InputBriefFrame.{field_name} must not be negative") + if frame.l_loop_result_attitude_hint not in { + "not_recorded", + "l_loop_achieved", + "l_loop_partial_or_failed", + "l_loop_budget_exhausted", + "l_loop_missing_or_uncertain", + }: + raise ValueError( + "unknown Node3InputBriefFrame.l_loop_result_attitude_hint: " + f"{frame.l_loop_result_attitude_hint}" + ) + + +def _validate_node3_r_loop_result_material(material: Node3RLoopResultMaterial) -> None: + for field_name, value in { + "source_data_id": material.source_data_id, + "r_loop_task_status": material.r_loop_task_status, + "continuation_status": material.continuation_status, + "budget_status": material.budget_status, + "final_information_granularity": material.final_information_granularity, + "generated_by": material.generated_by, + "info_class": material.info_class, + "semantic_judgement_status": material.semantic_judgement_status, + "attitude_hint": material.attitude_hint, + }.items(): + if not value: + raise ValueError(f"Node3RLoopResultMaterial.{field_name} must not be empty") + if material.r_loop_task_status not in {"not_run", "sufficient", "partial", "failed"}: + raise ValueError( + f"unknown Node3RLoopResultMaterial.r_loop_task_status: {material.r_loop_task_status}" + ) + if material.continuation_status not in { + "stop_sufficient", + "continue_deeper", + "continue_switch_branch", + "stop_budget_exhausted", + "stop_no_actionable_path", + "stop_failed_final", + "not_run", + }: + raise ValueError( + f"unknown Node3RLoopResultMaterial.continuation_status: {material.continuation_status}" + ) + if material.budget_status not in {"within_budget", "exhausted", "not_run"}: + raise ValueError( + f"unknown Node3RLoopResultMaterial.budget_status: {material.budget_status}" + ) + if material.info_class != "absolute": + raise ValueError("Node3RLoopResultMaterial.info_class must be absolute") + if material.semantic_judgement_status != "not_run": + raise ValueError("Node3RLoopResultMaterial.semantic_judgement_status must be not_run") + if material.attitude_hint not in { + "not_recorded", + "r_loop_sufficient", + "r_loop_partial_or_skeleton_only", + "r_loop_budget_exhausted", + "r_loop_failed", + }: + raise ValueError( + f"unknown Node3RLoopResultMaterial.attitude_hint: {material.attitude_hint}" + ) + for field_name, value in { + "summary_depth_used": material.summary_depth_used, + "selected_entry_node_count": material.selected_entry_node_count, + "inspected_graph_node_count": material.inspected_graph_node_count, + "source_graph_node_count": material.source_graph_node_count, + }.items(): + if not isinstance(value, int): + raise TypeError(f"Node3RLoopResultMaterial.{field_name} must be an integer") + if value < 0: + raise ValueError(f"Node3RLoopResultMaterial.{field_name} must not be negative") + + +def _validate_node3_l3_document_summary_material( + material: Node3L3DocumentSummaryMaterial, +) -> None: + for field_name, value in { + "document_name": material.document_name, + "summary_status": material.summary_status, + "plain_summary_info_class": material.plain_summary_info_class, + "plain_summary_source_mode": material.plain_summary_source_mode, + "plain_summary_claim_alignment": material.plain_summary_claim_alignment, + "task_relevant_summary_info_class": material.task_relevant_summary_info_class, + "task_relevant_summary_source_mode": material.task_relevant_summary_source_mode, + "task_relevant_summary_claim_alignment": material.task_relevant_summary_claim_alignment, + "generated_by": material.generated_by, + "semantic_judgement_status": material.semantic_judgement_status, + "source_data_id": material.source_data_id, + }.items(): + if not value: + raise ValueError(f"Node3L3DocumentSummaryMaterial.{field_name} must not be empty") + if not isinstance(material.source_char_count, int): + raise TypeError("Node3L3DocumentSummaryMaterial.source_char_count must be an integer") + if material.source_char_count < 0: + raise ValueError("Node3L3DocumentSummaryMaterial.source_char_count must not be negative") + if material.summary_status not in L3_DOCUMENT_SUMMARY_STATUSES: + raise ValueError( + f"unknown Node3L3DocumentSummaryMaterial.summary_status: {material.summary_status}" + ) + if material.semantic_judgement_status not in {"ran", "failed"}: + raise ValueError( + "unknown Node3L3DocumentSummaryMaterial.semantic_judgement_status: " + f"{material.semantic_judgement_status}" + ) + if material.plain_summary_info_class != "relative": + raise ValueError("plain document summary material must be relative") + if material.plain_summary_source_mode != "direct_record": + raise ValueError("plain document summary material must use direct_record") + if material.plain_summary_claim_alignment != "one_document_to_one_summary": + raise ValueError( + "plain document summary material must use one_document_to_one_summary" + ) + if material.task_relevant_summary_info_class != "mixed": + raise ValueError("task relevant summary material must be mixed") + if material.task_relevant_summary_source_mode != "source_bundle": + raise ValueError("task relevant summary material must use source_bundle") + if material.task_relevant_summary_claim_alignment != "one_document_plus_task_context": + raise ValueError( + "task relevant summary material must use one_document_plus_task_context" + ) + if material.summary_status == "ran": + if not material.plain_document_summary.strip(): + raise ValueError("plain_document_summary must not be empty when summary ran") + if not material.task_relevant_summary.strip(): + raise ValueError("task_relevant_summary must not be empty when summary ran") + + +def validate_node3_material_delivery_policy_frame( + frame: Node3MaterialDeliveryPolicyFrame, +) -> None: + """Node3 material delivery policy가 고정 mapping 결과인지 확인한다.""" + + for field_name, value in { + "frame_id": frame.frame_id, + "turn_id": frame.turn_id, + "answer_basis_mode": frame.answer_basis_mode, + "material_delivery_mode": frame.material_delivery_mode, + "raw_document_policy": frame.raw_document_policy, + "l3_summary_policy": frame.l3_summary_policy, + "uncertainty_policy": frame.uncertainty_policy, + "policy_reason_code": frame.policy_reason_code, + "generated_by": frame.generated_by, + "info_class": frame.info_class, + "semantic_judgement_status": frame.semantic_judgement_status, + "schema_name": frame.schema_name, + "schema_version": frame.schema_version, + }.items(): + if not value: + raise ValueError(f"Node3MaterialDeliveryPolicyFrame.{field_name} must not be empty") + if frame.schema_name != NODE3_MATERIAL_DELIVERY_POLICY_FRAME_SCHEMA_NAME: + raise ValueError( + "unknown Node3MaterialDeliveryPolicyFrame.schema_name: " + f"{frame.schema_name}" + ) + if frame.schema_version != NODE3_MATERIAL_DELIVERY_POLICY_FRAME_SCHEMA_VERSION: + raise ValueError( + "unknown Node3MaterialDeliveryPolicyFrame.schema_version: " + f"{frame.schema_version}" + ) + if frame.answer_basis_mode not in ANSWER_BASIS_MODES: + raise ValueError( + f"unknown Node3MaterialDeliveryPolicyFrame.answer_basis_mode: {frame.answer_basis_mode}" + ) + _validate_node3_material_policy_values( + material_delivery_mode=frame.material_delivery_mode, + raw_document_policy=frame.raw_document_policy, + l3_summary_policy=frame.l3_summary_policy, + uncertainty_policy=frame.uncertainty_policy, + policy_reason_code=frame.policy_reason_code, + supplied_document_context_count=frame.supplied_document_context_count, + l3_document_summary_count=frame.l3_document_summary_count, + llm_raw_document_text_count=frame.llm_raw_document_text_count, + llm_l3_summary_context_count=frame.llm_l3_summary_context_count, + raw_context_replaced_by_summary_count=frame.raw_context_replaced_by_summary_count, + ) + if frame.generated_by != "CODE:ANSWER_BASIS_MATERIAL_POLICY": + raise ValueError("Node3MaterialDeliveryPolicyFrame.generated_by must be CODE policy") + if frame.info_class != "absolute_policy_decision": + raise ValueError("Node3MaterialDeliveryPolicyFrame.info_class must be absolute_policy_decision") + if frame.semantic_judgement_status != "not_run": + raise ValueError( + "Node3MaterialDeliveryPolicyFrame.semantic_judgement_status must be not_run" + ) + if frame.answer_basis_frame_id is not None: + if not frame.answer_basis_frame_id: + raise ValueError("Node3MaterialDeliveryPolicyFrame.answer_basis_frame_id must not be empty") + if frame.answer_basis_frame_id not in frame.source_data_ids: + raise ValueError( + "Node3MaterialDeliveryPolicyFrame.source_data_ids must include answer_basis_frame_id" + ) + for data_id in frame.source_data_ids: + if not data_id: + raise ValueError("Node3MaterialDeliveryPolicyFrame.source_data_ids must not contain empty values") + for trace_id in frame.source_trace_ids: + if not trace_id: + raise ValueError("Node3MaterialDeliveryPolicyFrame.source_trace_ids must not contain empty values") + + +def _validate_node3_material_delivery_fields(frame: Node3InputBriefFrame) -> None: + if frame.material_delivery_policy_frame_id is not None: + if not frame.material_delivery_policy_frame_id: + raise ValueError("Node3InputBriefFrame.material_delivery_policy_frame_id must not be empty") + if frame.material_delivery_policy_frame_id not in frame.source_data_ids: + raise ValueError( + "Node3InputBriefFrame.source_data_ids must include material_delivery_policy_frame_id" + ) + else: + for field_name, value in { + "llm_raw_document_text_count": frame.llm_raw_document_text_count, + "llm_l3_summary_context_count": frame.llm_l3_summary_context_count, + "raw_context_replaced_by_summary_count": frame.raw_context_replaced_by_summary_count, + }.items(): + if not isinstance(value, int): + raise TypeError(f"Node3InputBriefFrame.{field_name} must be an integer") + if value < 0: + raise ValueError(f"Node3InputBriefFrame.{field_name} must not be negative") + return + _validate_node3_material_policy_values( + material_delivery_mode=frame.material_delivery_mode, + raw_document_policy=frame.raw_document_policy, + l3_summary_policy=frame.l3_summary_policy, + uncertainty_policy=frame.uncertainty_policy, + policy_reason_code=frame.material_policy_reason_code, + supplied_document_context_count=frame.supplied_document_context_count, + l3_document_summary_count=len(frame.l3_document_summaries), + llm_raw_document_text_count=frame.llm_raw_document_text_count, + llm_l3_summary_context_count=frame.llm_l3_summary_context_count, + raw_context_replaced_by_summary_count=frame.raw_context_replaced_by_summary_count, + ) + if frame.material_policy_generated_by != "CODE:ANSWER_BASIS_MATERIAL_POLICY": + raise ValueError("Node3InputBriefFrame.material_policy_generated_by must be CODE policy") + if frame.material_policy_info_class != "absolute_policy_decision": + raise ValueError( + "Node3InputBriefFrame.material_policy_info_class must be absolute_policy_decision" + ) + if frame.material_policy_semantic_judgement_status != "not_run": + raise ValueError( + "Node3InputBriefFrame.material_policy_semantic_judgement_status must be not_run" + ) + + +def _validate_node3_material_policy_values( + *, + material_delivery_mode: str, + raw_document_policy: str, + l3_summary_policy: str, + uncertainty_policy: str, + policy_reason_code: str, + supplied_document_context_count: int, + l3_document_summary_count: int, + llm_raw_document_text_count: int, + llm_l3_summary_context_count: int, + raw_context_replaced_by_summary_count: int, +) -> None: + if material_delivery_mode not in NODE3_MATERIAL_DELIVERY_MODES: + raise ValueError(f"unknown material_delivery_mode: {material_delivery_mode}") + if raw_document_policy not in NODE3_RAW_DOCUMENT_POLICIES: + raise ValueError(f"unknown raw_document_policy: {raw_document_policy}") + if l3_summary_policy not in NODE3_L3_SUMMARY_POLICIES: + raise ValueError(f"unknown l3_summary_policy: {l3_summary_policy}") + if uncertainty_policy not in NODE3_UNCERTAINTY_POLICIES: + raise ValueError(f"unknown uncertainty_policy: {uncertainty_policy}") + if policy_reason_code not in NODE3_MATERIAL_POLICY_REASON_CODES: + raise ValueError(f"unknown material policy reason code: {policy_reason_code}") + for field_name, value in { + "supplied_document_context_count": supplied_document_context_count, + "l3_document_summary_count": l3_document_summary_count, + "llm_raw_document_text_count": llm_raw_document_text_count, + "llm_l3_summary_context_count": llm_l3_summary_context_count, + "raw_context_replaced_by_summary_count": raw_context_replaced_by_summary_count, + }.items(): + if not isinstance(value, int): + raise TypeError(f"{field_name} must be an integer") + if value < 0: + raise ValueError(f"{field_name} must not be negative") + if llm_raw_document_text_count > supplied_document_context_count: + raise ValueError("llm_raw_document_text_count must not exceed supplied context count") + if raw_context_replaced_by_summary_count > supplied_document_context_count: + raise ValueError("raw_context_replaced_by_summary_count must not exceed supplied context count") + if llm_l3_summary_context_count > l3_document_summary_count: + raise ValueError("llm_l3_summary_context_count must not exceed L3 summary count") + if material_delivery_mode == "raw_document_primary": + if llm_raw_document_text_count != supplied_document_context_count: + raise ValueError("raw_document_primary must preserve all supplied raw document text") + if raw_context_replaced_by_summary_count != 0: + raise ValueError("raw_document_primary must not replace raw context") + if material_delivery_mode in { + "l3_summary_replaces_raw_context", + "l3_summary_replaces_raw_context_with_uncertainty", + }: + if l3_document_summary_count < 1: + raise ValueError("summary replacement requires at least one L3 summary") + if llm_raw_document_text_count != 0: + raise ValueError("summary replacement must omit raw document text from LLM payload") + if raw_context_replaced_by_summary_count != supplied_document_context_count: + raise ValueError("summary replacement must account for every supplied raw context") + if llm_l3_summary_context_count != l3_document_summary_count: + raise ValueError("summary replacement must expose all L3 summary materials") + if material_delivery_mode == "raw_document_fallback_no_l3_summary": + if l3_document_summary_count != 0: + raise ValueError("raw fallback is only valid when L3 summaries are absent") + if llm_raw_document_text_count != supplied_document_context_count: + raise ValueError("raw fallback must preserve supplied raw document text") + if raw_context_replaced_by_summary_count != 0: + raise ValueError("raw fallback must not replace raw context") + + +def _validate_node3_answer_basis_fields(frame: Node3InputBriefFrame) -> None: + if frame.answer_basis_frame_id is not None: + if not frame.answer_basis_frame_id: + raise ValueError("Node3InputBriefFrame.answer_basis_frame_id must not be empty") + if frame.answer_basis_frame_id not in frame.source_data_ids: + raise ValueError( + "Node3InputBriefFrame.source_data_ids must include answer_basis_frame_id" + ) + if frame.answer_basis_mode not in ANSWER_BASIS_MODES: + raise ValueError(f"unknown Node3InputBriefFrame.answer_basis_mode: {frame.answer_basis_mode}") + if not frame.basis_reason_codes: + raise ValueError("Node3InputBriefFrame.basis_reason_codes must not be empty") + for code in frame.basis_reason_codes: + if code not in BASIS_REASON_CODES: + raise ValueError(f"unknown Node3InputBriefFrame.basis_reason_code: {code}") + if not frame.mode_selection_reason: + raise ValueError("Node3InputBriefFrame.mode_selection_reason must not be empty") + if frame.mode_selection_reason_info_class not in ANSWER_BASIS_INFO_CLASSES: + raise ValueError( + "unknown Node3InputBriefFrame.mode_selection_reason_info_class: " + f"{frame.mode_selection_reason_info_class}" + ) + if not frame.answer_basis_generated_by: + raise ValueError("Node3InputBriefFrame.answer_basis_generated_by must not be empty") + if frame.answer_basis_info_class not in ANSWER_BASIS_INFO_CLASSES: + raise ValueError( + f"unknown Node3InputBriefFrame.answer_basis_info_class: {frame.answer_basis_info_class}" + ) + if frame.answer_basis_semantic_judgement_status not in ANSWER_BASIS_SEMANTIC_STATUSES: + raise ValueError( + "unknown Node3InputBriefFrame.answer_basis_semantic_judgement_status: " + f"{frame.answer_basis_semantic_judgement_status}" + ) + _validate_no_duplicates( + "Node3InputBriefFrame.basis_reason_codes", + frame.basis_reason_codes, + ) + for role in frame.evidence_roles: + _validate_node2_evidence_role(role, allowed_source_data_ids=frame.source_data_ids) + + +def _validate_node3_excluded_document_context(document: Node3ExcludedDocumentContext) -> None: + for field_name, value in { + "document_name": document.document_name, + "selection_basis": document.selection_basis, + "exclusion_reason": document.exclusion_reason, + }.items(): + if not value: + raise ValueError(f"Node3ExcludedDocumentContext.{field_name} must not be empty") + if document.exclusion_reason not in { + "excluded_due_to_context_budget", + "excluded_after_strict_rank_cutoff", + "excluded_not_readable_markdown_document", + }: + raise ValueError( + "unknown Node3ExcludedDocumentContext.exclusion_reason: " + f"{document.exclusion_reason}" + ) + if not isinstance(document.char_count, int): + raise TypeError("Node3ExcludedDocumentContext.char_count must be an integer") + if document.char_count < 0: + raise ValueError("Node3ExcludedDocumentContext.char_count must not be negative") + + def _validate_node3_brief_claim(claim: Node3BriefClaim) -> None: for field_name, value in { "kind": claim.kind, @@ -1872,6 +3228,32 @@ def validate_l1_goal_frame(frame: L1GoalFrame) -> None: L_LOOP_BUDGET_PLAN_FRAME_SCHEMA_NAME = "LLoopBudgetPlanFrame" L_LOOP_BUDGET_PLAN_FRAME_SCHEMA_VERSION = "0.1" +L_TOOL_SCOPE_FRAME_SCHEMA_NAME = "LToolScopeFrame" +L_TOOL_SCOPE_FRAME_SCHEMA_VERSION = "0.1" +L_TOOL_BUDGET_PARTITION_FRAME_SCHEMA_NAME = "LToolBudgetPartitionFrame" +L_TOOL_BUDGET_PARTITION_FRAME_SCHEMA_VERSION = "0.1" +L_TOOL_SCOPE_MODES = { + "document_only", + "code_only", + "document_and_code", + "runtime_trace_only", + "mixed_evidence", +} +L_TOOL_GROUPS = { + "document_tools", + "code_inspection_tools", + "runtime_record_tools", +} +L_REQUIRED_MATERIALS = { + "order_document", + "source_code_file", + "code_search_result", + "runtime_trace", + "execution_record", + "project_document", +} +L_TOOL_SCOPE_INFO_CLASSES = {"mixed", "absolute_status"} +L_TOOL_SCOPE_SEMANTIC_STATUSES = {"ran", "failed"} @dataclass @@ -1996,6 +3378,217 @@ def validate_l_loop_budget_plan_frame(frame: LLoopBudgetPlanFrame) -> None: raise ValueError("LLoopBudgetPlanFrame.source_data_ids must not contain empty values") +@dataclass +class LToolScopeFrame: + """L루프가 L2에 넘기기 전에 이번 턴의 허용 도구 범위를 정한 frame.""" + + frame_id: str + turn_id: str + tool_scope_mode: str + allowed_tool_groups: list[str] + required_materials: list[str] + scope_reason: str + scope_reason_info_class: str + generated_by: str = "LLM:L_TOOL_SCOPE" + info_class: str = "mixed" + semantic_judgement_status: str = "ran" + scope_failure_type: str = "none" + llm_call_data_id: str | None = None + prompt_ref: str = "" + source_trace_ids: list[str] = field(default_factory=list) + source_data_ids: list[str] = field(default_factory=list) + schema_name: str = L_TOOL_SCOPE_FRAME_SCHEMA_NAME + schema_version: str = L_TOOL_SCOPE_FRAME_SCHEMA_VERSION + + +def validate_l_tool_scope_frame(frame: LToolScopeFrame) -> None: + """LToolScopeFrame의 enum, fallback 정직성, source 경계를 검증한다.""" + + for field_name, value in { + "frame_id": frame.frame_id, + "turn_id": frame.turn_id, + "tool_scope_mode": frame.tool_scope_mode, + "scope_reason": frame.scope_reason, + "scope_reason_info_class": frame.scope_reason_info_class, + "generated_by": frame.generated_by, + "info_class": frame.info_class, + "semantic_judgement_status": frame.semantic_judgement_status, + "schema_name": frame.schema_name, + "schema_version": frame.schema_version, + }.items(): + if not value: + raise ValueError(f"LToolScopeFrame.{field_name} must not be empty") + if frame.schema_name != L_TOOL_SCOPE_FRAME_SCHEMA_NAME: + raise ValueError(f"unknown LToolScopeFrame.schema_name: {frame.schema_name}") + if frame.schema_version != L_TOOL_SCOPE_FRAME_SCHEMA_VERSION: + raise ValueError(f"unknown LToolScopeFrame.schema_version: {frame.schema_version}") + if frame.tool_scope_mode not in L_TOOL_SCOPE_MODES: + raise ValueError(f"unknown L tool_scope_mode: {frame.tool_scope_mode}") + if frame.scope_reason_info_class not in L_TOOL_SCOPE_INFO_CLASSES: + raise ValueError( + "unknown LToolScopeFrame.scope_reason_info_class: " + f"{frame.scope_reason_info_class}" + ) + if frame.info_class not in L_TOOL_SCOPE_INFO_CLASSES: + raise ValueError(f"unknown LToolScopeFrame.info_class: {frame.info_class}") + if frame.semantic_judgement_status not in L_TOOL_SCOPE_SEMANTIC_STATUSES: + raise ValueError( + "unknown LToolScopeFrame.semantic_judgement_status: " + f"{frame.semantic_judgement_status}" + ) + if not frame.allowed_tool_groups: + raise ValueError("LToolScopeFrame.allowed_tool_groups must not be empty") + if not frame.required_materials: + raise ValueError("LToolScopeFrame.required_materials must not be empty") + _validate_no_duplicates("LToolScopeFrame.allowed_tool_groups", frame.allowed_tool_groups) + _validate_no_duplicates("LToolScopeFrame.required_materials", frame.required_materials) + for group in frame.allowed_tool_groups: + if group not in L_TOOL_GROUPS: + raise ValueError(f"unknown L tool group: {group}") + for material in frame.required_materials: + if material not in L_REQUIRED_MATERIALS: + raise ValueError(f"unknown L required material: {material}") + if frame.llm_call_data_id is not None and not frame.llm_call_data_id: + raise ValueError("LToolScopeFrame.llm_call_data_id must not be empty") + + if frame.semantic_judgement_status == "failed": + if frame.generated_by != "CODE:FALLBACK": + raise ValueError("failed LToolScopeFrame must use generated_by=CODE:FALLBACK") + if frame.info_class != "absolute_status": + raise ValueError("failed LToolScopeFrame must use info_class=absolute_status") + if frame.scope_reason_info_class != "absolute_status": + raise ValueError( + "failed LToolScopeFrame must use scope_reason_info_class=absolute_status" + ) + else: + if not frame.generated_by.startswith("LLM:"): + raise ValueError("ran LToolScopeFrame must preserve generated_by=LLM:*") + if frame.info_class != "mixed": + raise ValueError("ran LToolScopeFrame must use info_class=mixed") + if frame.scope_reason_info_class != "mixed": + raise ValueError("ran LToolScopeFrame reason must use info_class=mixed") + + if frame.tool_scope_mode == "document_only" and frame.allowed_tool_groups != ["document_tools"]: + raise ValueError("document_only scope must allow only document_tools") + if frame.tool_scope_mode == "code_only" and frame.allowed_tool_groups != ["code_inspection_tools"]: + raise ValueError("code_only scope must allow only code_inspection_tools") + if frame.tool_scope_mode == "document_and_code": + for group in ("document_tools", "code_inspection_tools"): + if group not in frame.allowed_tool_groups: + raise ValueError("document_and_code scope must allow document and code tools") + if frame.tool_scope_mode == "runtime_trace_only" and frame.allowed_tool_groups != ["runtime_record_tools"]: + raise ValueError("runtime_trace_only scope must allow only runtime_record_tools") + if frame.tool_scope_mode == "mixed_evidence" and len(frame.allowed_tool_groups) < 2: + raise ValueError("mixed_evidence scope must allow at least two tool groups") + + for trace_id in frame.source_trace_ids: + if not trace_id: + raise ValueError("LToolScopeFrame.source_trace_ids must not contain empty values") + for data_id in frame.source_data_ids: + if not data_id: + raise ValueError("LToolScopeFrame.source_data_ids must not contain empty values") + + +@dataclass +class LToolBudgetPartitionFrame: + """승인된 L 예산을 tool scope에 따라 도구군별로 나눈 code policy frame.""" + + frame_id: str + turn_id: str + tool_scope_frame_id: str + budget_plan_frame_id: str + tool_scope_mode: str + allowed_tool_groups: list[str] + document_tool_call_budget: int = 0 + document_query_budget: int = 0 + document_read_budget: int = 0 + code_tool_call_budget: int = 0 + code_query_budget: int = 0 + code_read_budget: int = 0 + runtime_record_budget: int = 0 + partition_policy_id: str = "l_tool_scope_budget_partition_v0" + partition_reason: str = "" + generated_by: str = "CODE:L_TOOL_BUDGET_PARTITION_POLICY" + info_class: str = "absolute_policy_decision" + semantic_judgement_status: str = "not_run" + source_trace_ids: list[str] = field(default_factory=list) + source_data_ids: list[str] = field(default_factory=list) + schema_name: str = L_TOOL_BUDGET_PARTITION_FRAME_SCHEMA_NAME + schema_version: str = L_TOOL_BUDGET_PARTITION_FRAME_SCHEMA_VERSION + + +def validate_l_tool_budget_partition_frame(frame: LToolBudgetPartitionFrame) -> None: + """Tool scope에서 파생한 예산 분배 frame의 절대 정책 경계를 검증한다.""" + + for field_name, value in { + "frame_id": frame.frame_id, + "turn_id": frame.turn_id, + "tool_scope_frame_id": frame.tool_scope_frame_id, + "budget_plan_frame_id": frame.budget_plan_frame_id, + "tool_scope_mode": frame.tool_scope_mode, + "partition_policy_id": frame.partition_policy_id, + "partition_reason": frame.partition_reason, + "generated_by": frame.generated_by, + "info_class": frame.info_class, + "semantic_judgement_status": frame.semantic_judgement_status, + "schema_name": frame.schema_name, + "schema_version": frame.schema_version, + }.items(): + if not value: + raise ValueError(f"LToolBudgetPartitionFrame.{field_name} must not be empty") + if frame.schema_name != L_TOOL_BUDGET_PARTITION_FRAME_SCHEMA_NAME: + raise ValueError( + f"unknown LToolBudgetPartitionFrame.schema_name: {frame.schema_name}" + ) + if frame.schema_version != L_TOOL_BUDGET_PARTITION_FRAME_SCHEMA_VERSION: + raise ValueError( + f"unknown LToolBudgetPartitionFrame.schema_version: {frame.schema_version}" + ) + if frame.tool_scope_mode not in L_TOOL_SCOPE_MODES: + raise ValueError(f"unknown budget partition tool_scope_mode: {frame.tool_scope_mode}") + if frame.generated_by != "CODE:L_TOOL_BUDGET_PARTITION_POLICY": + raise ValueError("LToolBudgetPartitionFrame.generated_by must reveal code policy") + if frame.info_class != "absolute_policy_decision": + raise ValueError("LToolBudgetPartitionFrame.info_class must be absolute_policy_decision") + if frame.semantic_judgement_status != "not_run": + raise ValueError("LToolBudgetPartitionFrame.semantic_judgement_status must be not_run") + if frame.tool_scope_frame_id not in frame.source_data_ids: + raise ValueError( + "LToolBudgetPartitionFrame.source_data_ids must include tool_scope_frame_id" + ) + if frame.budget_plan_frame_id not in frame.source_data_ids: + raise ValueError( + "LToolBudgetPartitionFrame.source_data_ids must include budget_plan_frame_id" + ) + _validate_no_duplicates("LToolBudgetPartitionFrame.allowed_tool_groups", frame.allowed_tool_groups) + for group in frame.allowed_tool_groups: + if group not in L_TOOL_GROUPS: + raise ValueError(f"unknown budget partition tool group: {group}") + for field_name, value in { + "document_tool_call_budget": frame.document_tool_call_budget, + "document_query_budget": frame.document_query_budget, + "document_read_budget": frame.document_read_budget, + "code_tool_call_budget": frame.code_tool_call_budget, + "code_query_budget": frame.code_query_budget, + "code_read_budget": frame.code_read_budget, + "runtime_record_budget": frame.runtime_record_budget, + }.items(): + if not isinstance(value, int): + raise TypeError(f"LToolBudgetPartitionFrame.{field_name} must be an integer") + if value < 0: + raise ValueError(f"LToolBudgetPartitionFrame.{field_name} must not be negative") + for trace_id in frame.source_trace_ids: + if not trace_id: + raise ValueError( + "LToolBudgetPartitionFrame.source_trace_ids must not contain empty values" + ) + for data_id in frame.source_data_ids: + if not data_id: + raise ValueError( + "LToolBudgetPartitionFrame.source_data_ids must not contain empty values" + ) + + L2_QUERY_FRAME_SCHEMA_NAME = "L2QueryFrame" L2_QUERY_FRAME_SCHEMA_VERSION = "0.1" L2_QUERY_PLAN_FRAME_SCHEMA_NAME = "L2QueryPlanFrame" @@ -2008,8 +3601,29 @@ def validate_l_loop_budget_plan_frame(frame: LLoopBudgetPlanFrame) -> None: "revision_llm_query_plan", "revision_fallback_query_plan", } -L2_QUERY_MODES = {"embedding_search", "exact_artifact_ref"} -L2_TARGET_TOOL_NAMES = {"search_docs", "read_artifact"} +L2_QUERY_MODES = { + "embedding_search", + "exact_artifact_ref", + "direct_doc_read", + "code_file_list", + "code_search", + "code_file_read", +} +L2_TARGET_TOOL_NAMES = { + "search_docs", + "read_artifact", + "list_code_files", + "search_code", + "read_code_file", +} +L2_REVISION_TARGET_TOOL_NAMES = { + "search_docs", + "read_artifact", + "read_doc", + "list_code_files", + "search_code", + "read_code_file", +} L2_QUERY_PLANNER_MODES = {"llm", "fallback", "revision_llm", "revision_fallback"} L2_REVISION_PREVIOUS_TOOL_NAMES = {"search_docs", "read_doc", "read_artifact"} L2_REVISION_L3_GOAL_STATUSES = {"achieved", "partial", "failed", "missing", "not_run"} @@ -2069,8 +3683,29 @@ def validate_l2_query_frame(frame: L2QueryFrame) -> None: raise ValueError(f"unknown L2 query_source: {frame.query_source}") if frame.query_mode not in L2_QUERY_MODES: raise ValueError(f"unknown L2 query_mode: {frame.query_mode}") - if frame.target_tool_name not in L2_TARGET_TOOL_NAMES: + allowed_target_tools = ( + L2_REVISION_TARGET_TOOL_NAMES + if frame.query_source in {"revision_llm_query_plan", "revision_fallback_query_plan"} + else L2_TARGET_TOOL_NAMES + ) + if frame.target_tool_name not in allowed_target_tools: raise ValueError(f"unknown L2 target_tool_name: {frame.target_tool_name}") + if frame.target_tool_name == "read_doc" and frame.query_mode != "direct_doc_read": + raise ValueError("read_doc L2 query must use direct_doc_read mode") + if frame.query_mode == "direct_doc_read" and frame.target_tool_name != "read_doc": + raise ValueError("direct_doc_read mode must target read_doc") + if frame.target_tool_name == "list_code_files" and frame.query_mode != "code_file_list": + raise ValueError("list_code_files L2 query must use code_file_list mode") + if frame.query_mode == "code_file_list" and frame.target_tool_name != "list_code_files": + raise ValueError("code_file_list mode must target list_code_files") + if frame.target_tool_name == "search_code" and frame.query_mode != "code_search": + raise ValueError("search_code L2 query must use code_search mode") + if frame.query_mode == "code_search" and frame.target_tool_name != "search_code": + raise ValueError("code_search mode must target search_code") + if frame.target_tool_name == "read_code_file" and frame.query_mode != "code_file_read": + raise ValueError("read_code_file L2 query must use code_file_read mode") + if frame.query_mode == "code_file_read" and frame.target_tool_name != "read_code_file": + raise ValueError("code_file_read mode must target read_code_file") for trace_id in frame.source_trace_ids: if not trace_id: @@ -2161,10 +3796,14 @@ def validate_l2_query_plan_frame(frame: L2QueryPlanFrame) -> None: raise ValueError("L2QueryPlanFrame.source_data_ids must not contain empty values") for candidate in frame.candidates: - _validate_l2_query_plan_candidate(candidate) + _validate_l2_query_plan_candidate(candidate, planner_mode=frame.planner_mode) -def _validate_l2_query_plan_candidate(candidate: L2QueryPlanCandidate) -> None: +def _validate_l2_query_plan_candidate( + candidate: L2QueryPlanCandidate, + *, + planner_mode: str, +) -> None: """L2 query 후보 하나가 최소 규칙을 지키는지 확인한다.""" required_text_fields = { @@ -2181,7 +3820,12 @@ def _validate_l2_query_plan_candidate(candidate: L2QueryPlanCandidate) -> None: raise TypeError("L2QueryPlanCandidate.priority must be an integer") if candidate.priority < 1: raise ValueError("L2QueryPlanCandidate.priority must be positive") - if candidate.target_tool_name not in L2_TARGET_TOOL_NAMES: + allowed_target_tools = ( + L2_REVISION_TARGET_TOOL_NAMES + if planner_mode in {"revision_llm", "revision_fallback"} + else L2_TARGET_TOOL_NAMES + ) + if candidate.target_tool_name not in allowed_target_tools: raise ValueError(f"unknown L2 target_tool_name: {candidate.target_tool_name}") if not candidate.source_data_ids: raise ValueError("L2QueryPlanCandidate.source_data_ids must not be empty") @@ -2216,6 +3860,8 @@ class L2RevisionInputFrame: previous_tool_name: str # 절대 정보/문서 근거: 이미 읽은 문서 이름 목록. raw data_id 대신 사람이 읽는 이름만 둔다. read_document_names: list[str] = field(default_factory=list) + # 절대 정보: 아직 읽지 않은 검색 후보 문서 ID 목록. revision read_doc은 이 목록 안에서만 가능하다. + unread_candidate_doc_ids: list[str] = field(default_factory=list) # 분류 경계: L3가 남긴 후보 설명을 복사한 값이다. 원 생성자의 정보 등급을 따라간다. # 코드가 이 문장의 의미를 해석해 재시도 여부를 결정하면 안 된다. unread_candidate_summaries: list[str] = field(default_factory=list) @@ -2298,6 +3944,9 @@ def validate_l2_revision_input_frame(frame: L2RevisionInputFrame) -> None: for document_name in frame.read_document_names: if not document_name: raise ValueError("L2RevisionInputFrame.read_document_names must not contain empty values") + for doc_id in frame.unread_candidate_doc_ids: + if not doc_id: + raise ValueError("L2RevisionInputFrame.unread_candidate_doc_ids must not contain empty values") for summary in frame.unread_candidate_summaries: if not summary: raise ValueError("L2RevisionInputFrame.unread_candidate_summaries must not contain empty values") @@ -2418,6 +4067,320 @@ def _validate_document_memory_index_item(item: DocumentMemoryIndexItem) -> None: raise ValueError("DocumentMemoryIndexItem.size_bytes must be non-negative") +EXPLICIT_ARTIFACT_REFERENCE_FRAME_SCHEMA_NAME = "ExplicitArtifactReferenceFrame" +EXPLICIT_ARTIFACT_REFERENCE_FRAME_SCHEMA_VERSION = "0.1" +EXPLICIT_ARTIFACT_RESOLVE_STATUSES = { + "unique", + "ambiguous", + "not_found", + "invalid_ref", +} + + +@dataclass +class ExplicitArtifactResolvedReference: + """사용자 입력 표면에서 추출한 artifact reference와 resolver 결과.""" + + raw_ref: str + normalized_ref: str + occurrence_index: int + resolve_status: str + candidate_count: int + selected_doc_id: str | None = None + match_type: str | None = None + char_count: int = 0 + candidates: list[dict[str, object]] = field(default_factory=list) + + +@dataclass +class ExplicitArtifactReferenceFrame: + """명시 artifact reference 추출과 direct resolve 결과를 담는 절대정보 frame.""" + + frame_id: str + turn_id: str + source_user_text: str + extracted_reference_count: int + resolved_references: list[ExplicitArtifactResolvedReference] = field(default_factory=list) + unique_count: int = 0 + ambiguous_count: int = 0 + not_found_count: int = 0 + invalid_count: int = 0 + generated_by: str = "CODE:EXPLICIT_ARTIFACT_RESOLVER" + info_class: str = "absolute_resolve_result" + semantic_judgement_status: str = "not_run" + source_trace_ids: list[str] = field(default_factory=list) + source_data_ids: list[str] = field(default_factory=list) + schema_name: str = EXPLICIT_ARTIFACT_REFERENCE_FRAME_SCHEMA_NAME + schema_version: str = EXPLICIT_ARTIFACT_REFERENCE_FRAME_SCHEMA_VERSION + + +def validate_explicit_artifact_reference_frame( + frame: ExplicitArtifactReferenceFrame, +) -> None: + """ExplicitArtifactReferenceFrame이 표면 문자열 resolver 결과만 담는지 확인한다.""" + + for field_name, value in { + "frame_id": frame.frame_id, + "turn_id": frame.turn_id, + "generated_by": frame.generated_by, + "info_class": frame.info_class, + "semantic_judgement_status": frame.semantic_judgement_status, + "schema_name": frame.schema_name, + "schema_version": frame.schema_version, + }.items(): + if not value: + raise ValueError(f"ExplicitArtifactReferenceFrame.{field_name} must not be empty") + if frame.schema_name != EXPLICIT_ARTIFACT_REFERENCE_FRAME_SCHEMA_NAME: + raise ValueError( + f"unknown ExplicitArtifactReferenceFrame.schema_name: {frame.schema_name}" + ) + if frame.schema_version != EXPLICIT_ARTIFACT_REFERENCE_FRAME_SCHEMA_VERSION: + raise ValueError( + f"unknown ExplicitArtifactReferenceFrame.schema_version: {frame.schema_version}" + ) + if frame.generated_by != "CODE:EXPLICIT_ARTIFACT_RESOLVER": + raise ValueError("ExplicitArtifactReferenceFrame.generated_by must reveal code resolver") + if frame.info_class != "absolute_resolve_result": + raise ValueError("ExplicitArtifactReferenceFrame.info_class must be absolute_resolve_result") + if frame.semantic_judgement_status != "not_run": + raise ValueError("ExplicitArtifactReferenceFrame semantic judgement must not run") + if frame.extracted_reference_count != len(frame.resolved_references): + raise ValueError("explicit reference count must match resolved references") + status_counts = { + "unique": frame.unique_count, + "ambiguous": frame.ambiguous_count, + "not_found": frame.not_found_count, + "invalid_ref": frame.invalid_count, + } + for status, count in status_counts.items(): + if not isinstance(count, int): + raise TypeError(f"ExplicitArtifactReferenceFrame.{status}_count must be integer") + if count < 0: + raise ValueError(f"ExplicitArtifactReferenceFrame.{status}_count must be non-negative") + actual_counts = {status: 0 for status in EXPLICIT_ARTIFACT_RESOLVE_STATUSES} + for item in frame.resolved_references: + _validate_explicit_artifact_resolved_reference(item) + actual_counts[item.resolve_status] += 1 + if status_counts != actual_counts: + raise ValueError("explicit reference status counts do not match resolved references") + for trace_id in frame.source_trace_ids: + if not trace_id: + raise ValueError("ExplicitArtifactReferenceFrame.source_trace_ids must not contain empty values") + for data_id in frame.source_data_ids: + if not data_id: + raise ValueError("ExplicitArtifactReferenceFrame.source_data_ids must not contain empty values") + + +def _validate_explicit_artifact_resolved_reference( + item: ExplicitArtifactResolvedReference, +) -> None: + for field_name, value in { + "raw_ref": item.raw_ref, + "normalized_ref": item.normalized_ref, + "resolve_status": item.resolve_status, + }.items(): + if not value: + raise ValueError(f"ExplicitArtifactResolvedReference.{field_name} must not be empty") + if item.resolve_status not in EXPLICIT_ARTIFACT_RESOLVE_STATUSES: + raise ValueError(f"unknown explicit artifact resolve status: {item.resolve_status}") + if item.occurrence_index < 1: + raise ValueError("ExplicitArtifactResolvedReference.occurrence_index must be positive") + if item.candidate_count < 0: + raise ValueError("ExplicitArtifactResolvedReference.candidate_count must be non-negative") + if item.char_count < 0: + raise ValueError("ExplicitArtifactResolvedReference.char_count must be non-negative") + if item.resolve_status == "unique": + if not item.selected_doc_id: + raise ValueError("unique explicit reference must include selected_doc_id") + if not item.match_type: + raise ValueError("unique explicit reference must include match_type") + if item.candidate_count != 1: + raise ValueError("unique explicit reference candidate_count must be 1") + else: + if item.selected_doc_id is not None: + raise ValueError("non-unique explicit reference must not include selected_doc_id") + + +DOCUMENT_CONTEXT_PACK_FRAME_SCHEMA_NAME = "DocumentContextPackFrame" +DOCUMENT_CONTEXT_PACK_FRAME_SCHEMA_VERSION = "0.1" +DOCUMENT_CONTEXT_EXCLUSION_REASONS = { + "excluded_due_to_context_budget", + "excluded_after_strict_rank_cutoff", + "excluded_not_readable_markdown_document", +} +DOCUMENT_CONTEXT_BUDGET_UNITS = {"chars"} + + +@dataclass +class DocumentContextPackIncludedDocument: + """node_3에 전체 원문으로 공급된 문서 하나.""" + + doc_id: str + document_name: str + char_count: int + rank_index: int + selection_basis: str + text: str + source_data_id: str + + +@dataclass +class DocumentContextPackExcludedDocument: + """후보였지만 whole-document packing 정책 때문에 공급되지 않은 문서.""" + + doc_id: str + document_name: str + char_count: int + rank_index: int + selection_basis: str + exclusion_reason: str + would_exceed_budget: bool + source_data_id: str + + +@dataclass +class DocumentContextPackFrame: + """node_3 read_documents에 넣을 whole-document context packing 결과.""" + + frame_id: str + turn_id: str + max_document_context_chars: int + budget_unit: str + whole_document_only: bool + strict_rank_order: bool + included_documents: list[DocumentContextPackIncludedDocument] = field(default_factory=list) + excluded_documents: list[DocumentContextPackExcludedDocument] = field(default_factory=list) + included_document_count: int = 0 + excluded_document_count: int = 0 + included_total_chars: int = 0 + cutoff_reason: str = "none" + source_query_frame_ids: list[str] = field(default_factory=list) + source_search_result_data_ids: list[str] = field(default_factory=list) + source_explicit_reference_data_id: str | None = None + generated_by: str = "CODE:DOCUMENT_CONTEXT_PACKER" + info_class: str = "absolute_context_packing_result" + semantic_judgement_status: str = "not_run" + source_trace_ids: list[str] = field(default_factory=list) + source_data_ids: list[str] = field(default_factory=list) + schema_name: str = DOCUMENT_CONTEXT_PACK_FRAME_SCHEMA_NAME + schema_version: str = DOCUMENT_CONTEXT_PACK_FRAME_SCHEMA_VERSION + + +def validate_document_context_pack_frame(frame: DocumentContextPackFrame) -> None: + """DocumentContextPackFrame이 whole-document/strict rank 정책을 지키는지 확인한다.""" + + for field_name, value in { + "frame_id": frame.frame_id, + "turn_id": frame.turn_id, + "budget_unit": frame.budget_unit, + "cutoff_reason": frame.cutoff_reason, + "generated_by": frame.generated_by, + "info_class": frame.info_class, + "semantic_judgement_status": frame.semantic_judgement_status, + "schema_name": frame.schema_name, + "schema_version": frame.schema_version, + }.items(): + if not value: + raise ValueError(f"DocumentContextPackFrame.{field_name} must not be empty") + if frame.schema_name != DOCUMENT_CONTEXT_PACK_FRAME_SCHEMA_NAME: + raise ValueError(f"unknown DocumentContextPackFrame.schema_name: {frame.schema_name}") + if frame.schema_version != DOCUMENT_CONTEXT_PACK_FRAME_SCHEMA_VERSION: + raise ValueError( + f"unknown DocumentContextPackFrame.schema_version: {frame.schema_version}" + ) + if frame.budget_unit not in DOCUMENT_CONTEXT_BUDGET_UNITS: + raise ValueError(f"unknown DocumentContextPackFrame.budget_unit: {frame.budget_unit}") + if frame.max_document_context_chars < 1: + raise ValueError("DocumentContextPackFrame.max_document_context_chars must be positive") + if frame.whole_document_only is not True: + raise ValueError("DocumentContextPackFrame.whole_document_only must be true") + if frame.strict_rank_order is not True: + raise ValueError("DocumentContextPackFrame.strict_rank_order must be true") + if frame.generated_by != "CODE:DOCUMENT_CONTEXT_PACKER": + raise ValueError("DocumentContextPackFrame.generated_by must reveal code packer") + if frame.info_class != "absolute_context_packing_result": + raise ValueError("DocumentContextPackFrame.info_class must be absolute_context_packing_result") + if frame.semantic_judgement_status != "not_run": + raise ValueError("DocumentContextPackFrame semantic judgement must not run") + if frame.included_document_count != len(frame.included_documents): + raise ValueError("included_document_count must match included_documents") + if frame.excluded_document_count != len(frame.excluded_documents): + raise ValueError("excluded_document_count must match excluded_documents") + if frame.included_total_chars != sum(item.char_count for item in frame.included_documents): + raise ValueError("included_total_chars must match included document char counts") + if frame.included_total_chars > frame.max_document_context_chars: + raise ValueError("included_total_chars must not exceed max_document_context_chars") + if frame.source_explicit_reference_data_id and ( + frame.source_explicit_reference_data_id not in frame.source_data_ids + ): + raise ValueError("source_data_ids must include source_explicit_reference_data_id") + for data_id in frame.source_query_frame_ids: + if not data_id: + raise ValueError("DocumentContextPackFrame.source_query_frame_ids must not contain empty values") + for data_id in frame.source_search_result_data_ids: + if not data_id: + raise ValueError( + "DocumentContextPackFrame.source_search_result_data_ids must not contain empty values" + ) + for item in frame.included_documents: + _validate_document_context_pack_included_document(item) + cutoff_seen = False + for item in frame.excluded_documents: + _validate_document_context_pack_excluded_document(item) + if item.exclusion_reason == "excluded_due_to_context_budget": + cutoff_seen = True + elif item.exclusion_reason == "excluded_after_strict_rank_cutoff" and not cutoff_seen: + raise ValueError("strict rank cutoff exclusion must follow a budget exclusion") + for trace_id in frame.source_trace_ids: + if not trace_id: + raise ValueError("DocumentContextPackFrame.source_trace_ids must not contain empty values") + for data_id in frame.source_data_ids: + if not data_id: + raise ValueError("DocumentContextPackFrame.source_data_ids must not contain empty values") + + +def _validate_document_context_pack_included_document( + item: DocumentContextPackIncludedDocument, +) -> None: + for field_name, value in { + "doc_id": item.doc_id, + "document_name": item.document_name, + "selection_basis": item.selection_basis, + "text": item.text, + "source_data_id": item.source_data_id, + }.items(): + if not value: + raise ValueError(f"DocumentContextPackIncludedDocument.{field_name} must not be empty") + if item.rank_index < 1: + raise ValueError("DocumentContextPackIncludedDocument.rank_index must be positive") + if item.char_count < 0: + raise ValueError("DocumentContextPackIncludedDocument.char_count must be non-negative") + if len(item.text) != item.char_count: + raise ValueError("included document text length must match char_count") + + +def _validate_document_context_pack_excluded_document( + item: DocumentContextPackExcludedDocument, +) -> None: + for field_name, value in { + "doc_id": item.doc_id, + "document_name": item.document_name, + "selection_basis": item.selection_basis, + "exclusion_reason": item.exclusion_reason, + "source_data_id": item.source_data_id, + }.items(): + if not value: + raise ValueError(f"DocumentContextPackExcludedDocument.{field_name} must not be empty") + if item.rank_index < 1: + raise ValueError("DocumentContextPackExcludedDocument.rank_index must be positive") + if item.char_count < 0: + raise ValueError("DocumentContextPackExcludedDocument.char_count must be non-negative") + if item.exclusion_reason not in DOCUMENT_CONTEXT_EXCLUSION_REASONS: + raise ValueError(f"unknown document context exclusion reason: {item.exclusion_reason}") + if item.exclusion_reason == "excluded_due_to_context_budget" and not item.would_exceed_budget: + raise ValueError("budget exclusion must mark would_exceed_budget=true") + + TOOL_CATALOG_FRAME_SCHEMA_NAME = "ToolCatalogFrame" TOOL_CATALOG_FRAME_SCHEMA_VERSION = "0.1" TOOL_CHOICE_FRAME_SCHEMA_NAME = "ToolChoiceFrame" @@ -2577,7 +4540,14 @@ def validate_tool_choice_frame(frame: ToolChoiceFrame) -> None: TOOL_RESULT_DISTILLATION_FRAME_SCHEMA_NAME = "ToolResultDistillationFrame" TOOL_RESULT_DISTILLATION_FRAME_SCHEMA_VERSION = "0.1" -TOOL_RESULT_DISTILLATION_TOOL_NAMES = {"search_docs", "read_doc", "read_artifact"} +TOOL_RESULT_DISTILLATION_TOOL_NAMES = { + "search_docs", + "read_doc", + "read_artifact", + "list_code_files", + "search_code", + "read_code_file", +} TOOL_RESULT_DISTILLED_ITEM_KINDS = {"search_result", "read_doc_excerpt"} @@ -2904,6 +4874,9 @@ def _validate_tool_cache_status_record(record: ToolCacheStatusRecord) -> None: L_LOOP_CONTROL_DECISIONS = { "continue_search", "continue_read_artifact", + "continue_code_search", + "list_code_files", + "read_code_file", "read_document", "stop_success", "stop_failed", @@ -3027,6 +5000,16 @@ def validate_l_loop_control_frame(frame: LLoopControlFrame) -> None: raise ValueError("continue_read_artifact must select read_artifact") if frame.decision == "continue_read_artifact" and frame.query_text is None: raise ValueError("continue_read_artifact must include query_text") + if frame.decision == "continue_code_search" and frame.selected_tool_name != "search_code": + raise ValueError("continue_code_search must select search_code") + if frame.decision == "continue_code_search" and frame.query_text is None: + raise ValueError("continue_code_search must include query_text") + if frame.decision == "list_code_files" and frame.selected_tool_name != "list_code_files": + raise ValueError("list_code_files must select list_code_files") + if frame.decision == "read_code_file" and frame.selected_tool_name != "read_code_file": + raise ValueError("read_code_file must select read_code_file") + if frame.decision == "read_code_file" and frame.query_text is None: + raise ValueError("read_code_file must include query_text") if frame.decision == "read_document" and frame.selected_tool_name != "read_doc": raise ValueError("read_document must select read_doc") if frame.decision == "read_document" and frame.doc_id is None: @@ -3191,6 +5174,10 @@ class LLoopReturnSummaryFrame: route_hint_reason: str # 절대 정보: 실제 읽은 문서 ID 목록. read_doc_ids: list[str] = field(default_factory=list) + # 절대 정보: 실제 read_code_file로 읽은 source/config 파일 경로 목록. + read_code_file_paths: list[str] = field(default_factory=list) + # 절대 정보: read_code_file 성공 기록 수. read_doc 수와 섞지 않는다. + actual_read_code_file_count: int = 0 # 절대 정보: 검색 후보 문서 ID 목록. search_result_doc_ids: list[str] = field(default_factory=list) # 절대 정보: 입력 근거 trace ID 목록. @@ -3258,6 +5245,7 @@ def validate_l_loop_return_summary_frame(frame: LLoopReturnSummaryFrame) -> None "remaining_tool_calls": frame.remaining_tool_calls, "remaining_read_doc_calls": frame.remaining_read_doc_calls, "remaining_query_attempts": frame.remaining_query_attempts, + "actual_read_code_file_count": frame.actual_read_code_file_count, }.items(): if not isinstance(value, int): raise TypeError(f"LLoopReturnSummaryFrame.{field_name} must be an integer") @@ -3267,6 +5255,13 @@ def validate_l_loop_return_summary_frame(frame: LLoopReturnSummaryFrame) -> None for doc_id in frame.read_doc_ids: if not doc_id: raise ValueError("LLoopReturnSummaryFrame.read_doc_ids must not contain empty values") + if frame.actual_read_code_file_count != len(frame.read_code_file_paths): + raise ValueError( + "LLoopReturnSummaryFrame.actual_read_code_file_count must mirror read_code_file_paths length" + ) + for file_path in frame.read_code_file_paths: + if not file_path: + raise ValueError("LLoopReturnSummaryFrame.read_code_file_paths must not contain empty values") for doc_id in frame.search_result_doc_ids: if not doc_id: raise ValueError("LLoopReturnSummaryFrame.search_result_doc_ids must not contain empty values") @@ -3373,6 +5368,17 @@ def validate_l_loop_run_frame(frame: LLoopRunFrame) -> None: L3_ACHIEVEMENT_STATUSES = {"achieved", "partial", "failed"} L3_GOAL_MATCH_STATUSES = {"matched", "partial", "missing", "not_applicable"} L3_SEMANTIC_GOAL_MATCH_STATUSES = {"matched", "partial", "missing", "not_run"} +L3_PER_DOCUMENT_SUMMARY_FRAME_SCHEMA_NAME = "L3PerDocumentSummaryFrame" +L3_PER_DOCUMENT_SUMMARY_FRAME_SCHEMA_VERSION = "0.1" +L3_DOCUMENT_SUMMARY_STATUSES = {"ran", "failed"} +L3_DOCUMENT_SUMMARY_FAILURE_TYPES = { + "none", + "parse_failed", + "schema_failed", + "adapter_failed", + "timeout", + "unknown", +} @dataclass @@ -3534,6 +5540,10 @@ class L3AchievementFrame: requested_doc_hint: str = "" # 절대 정보: 이번 L루프에서 실제 read_doc으로 읽은 문서 ID 목록. read_doc_ids: list[str] = field(default_factory=list) + # 절대 정보: 이번 L루프에서 실제 read_code_file로 읽은 source/config 파일 경로 목록. + read_code_file_paths: list[str] = field(default_factory=list) + # 절대 정보: read_code_file 성공 기록 수. read_doc 수와 섞지 않는다. + actual_read_code_file_count: int = 0 # 절대 정보: search_docs 결과에서 L3가 보존한 문서 ID 목록. search_result_doc_ids: list[str] = field(default_factory=list) # 분류 경계: 특정 문서 요청과 실제 검색/읽기 결과의 대응 상태. @@ -3590,10 +5600,21 @@ def validate_l3_achievement_frame(frame: L3AchievementFrame) -> None: raise TypeError("L3AchievementFrame.candidate_count must be an integer") if frame.candidate_count < 0: raise ValueError("L3AchievementFrame.candidate_count must not be negative") + if not isinstance(frame.actual_read_code_file_count, int): + raise TypeError("L3AchievementFrame.actual_read_code_file_count must be an integer") + if frame.actual_read_code_file_count < 0: + raise ValueError("L3AchievementFrame.actual_read_code_file_count must not be negative") + if frame.actual_read_code_file_count != len(frame.read_code_file_paths): + raise ValueError( + "L3AchievementFrame.actual_read_code_file_count must mirror read_code_file_paths length" + ) for doc_id in frame.read_doc_ids: if not doc_id: raise ValueError("L3AchievementFrame.read_doc_ids must not contain empty values") + for file_path in frame.read_code_file_paths: + if not file_path: + raise ValueError("L3AchievementFrame.read_code_file_paths must not contain empty values") for doc_id in frame.search_result_doc_ids: if not doc_id: raise ValueError("L3AchievementFrame.search_result_doc_ids must not contain empty values") @@ -3611,6 +5632,166 @@ def validate_l3_achievement_frame(frame: L3AchievementFrame) -> None: raise ValueError("L3AchievementFrame.source_data_ids must not contain empty values") +@dataclass +class L3PerDocumentSummaryFrame: + """L3가 실제 읽은 문서 하나에 대해 남기는 문서별 의미 요약 프레임.""" + + # 절대 정보: DataStore에 저장될 summary frame data_id와 같은 값. + frame_id: str + # 절대 정보: 이 요약 프레임이 만들어진 사용자 턴 ID. + turn_id: str + # 절대 정보: 요약 대상이 된 원본 document extract record ID. + source_document_data_id: str + # 절대 정보: 원본 document extract payload의 doc_id. + source_doc_id: str + # 절대 정보/표시명: doc_id에서 만든 문서명. + source_document_name: str + # 절대 정보: 원본 문서 text 길이. + source_char_count: int + # 절대 정보: LLM 요약 실행 상태. + summary_status: str + # 상대 정보: 원본 문서 하나에만 대응하는 담백 요약. + plain_document_summary: str = "" + # 절대 정보: 담백 요약의 정보 분류 라벨. + plain_summary_info_class: str = "relative" + # 절대 정보: 담백 요약이 직접 record 하나에 붙어 있다는 source mode. + plain_summary_source_mode: str = "direct_record" + # 절대 정보: 담백 요약과 원본 문서의 대응 방식. + plain_summary_claim_alignment: str = "one_document_to_one_summary" + # 절대 정보: 담백 요약이 직접 대응하는 원본 record ID. + plain_summary_source_data_id: str = "" + # 혼합 정보: 현재 질문/L1 목표와 문서 원문을 함께 본 상황 맞춤 요약. + task_relevant_summary: str = "" + # 절대 정보: 상황 요약의 정보 분류 라벨. + task_relevant_summary_info_class: str = "mixed" + # 절대 정보: 상황 요약이 source bundle에 기대고 있다는 source mode. + task_relevant_summary_source_mode: str = "source_bundle" + # 절대 정보: 상황 요약과 source bundle의 대응 방식. + task_relevant_summary_claim_alignment: str = "one_document_plus_task_context" + # 절대 정보: 상황 요약이 참조한 source bundle record ID 목록. + task_relevant_summary_source_data_ids: list[str] = field(default_factory=list) + # 상대/혼합 정보의 한계 메모. LLM이 요약 범위를 드러낼 때만 채운다. + summary_limit_note: str = "" + # 절대 정보: 요약을 생성한 실행자. + generated_by: str = "LLM:unknown" + # 절대 정보: LLM 의미 생성 실행 상태. + semantic_judgement_status: str = "failed" + # 절대 정보: 실패 유형. 성공이면 none. + summary_failure_type: str = "unknown" + # 절대 정보: 이 요약을 만든 LLM call record ID. + llm_call_data_id: str | None = None + # 절대 정보: 사용한 prompt 파일. + prompt_ref: str = "" + # 절대 정보: 근거 trace ID 목록. + source_trace_ids: list[str] = field(default_factory=list) + # 절대 정보: 근거 DataStore record ID 목록. + source_data_ids: list[str] = field(default_factory=list) + # 절대 정보: 적용된 스키마 이름. + schema_name: str = L3_PER_DOCUMENT_SUMMARY_FRAME_SCHEMA_NAME + # 절대 정보: 적용된 스키마 버전. + schema_version: str = L3_PER_DOCUMENT_SUMMARY_FRAME_SCHEMA_VERSION + + +def validate_l3_per_document_summary_frame(frame: L3PerDocumentSummaryFrame) -> None: + """L3 문서별 요약이 정보 등급/출처 경계를 지키는지 확인한다.""" + + for field_name, value in { + "frame_id": frame.frame_id, + "turn_id": frame.turn_id, + "source_document_data_id": frame.source_document_data_id, + "source_doc_id": frame.source_doc_id, + "source_document_name": frame.source_document_name, + "summary_status": frame.summary_status, + "plain_summary_info_class": frame.plain_summary_info_class, + "plain_summary_source_mode": frame.plain_summary_source_mode, + "plain_summary_claim_alignment": frame.plain_summary_claim_alignment, + "plain_summary_source_data_id": frame.plain_summary_source_data_id, + "task_relevant_summary_info_class": frame.task_relevant_summary_info_class, + "task_relevant_summary_source_mode": frame.task_relevant_summary_source_mode, + "task_relevant_summary_claim_alignment": frame.task_relevant_summary_claim_alignment, + "generated_by": frame.generated_by, + "semantic_judgement_status": frame.semantic_judgement_status, + "summary_failure_type": frame.summary_failure_type, + "prompt_ref": frame.prompt_ref, + "schema_name": frame.schema_name, + "schema_version": frame.schema_version, + }.items(): + if not value: + raise ValueError(f"L3PerDocumentSummaryFrame.{field_name} must not be empty") + + if frame.schema_name != L3_PER_DOCUMENT_SUMMARY_FRAME_SCHEMA_NAME: + raise ValueError( + f"unknown L3 per-document summary schema_name: {frame.schema_name}" + ) + if frame.schema_version != L3_PER_DOCUMENT_SUMMARY_FRAME_SCHEMA_VERSION: + raise ValueError( + f"unknown L3 per-document summary schema_version: {frame.schema_version}" + ) + if not isinstance(frame.source_char_count, int): + raise TypeError("L3PerDocumentSummaryFrame.source_char_count must be an integer") + if frame.source_char_count < 0: + raise ValueError("L3PerDocumentSummaryFrame.source_char_count must not be negative") + if frame.summary_status not in L3_DOCUMENT_SUMMARY_STATUSES: + raise ValueError(f"unknown L3 summary_status: {frame.summary_status}") + if frame.summary_failure_type not in L3_DOCUMENT_SUMMARY_FAILURE_TYPES: + raise ValueError(f"unknown L3 summary_failure_type: {frame.summary_failure_type}") + if frame.summary_status == "ran": + if frame.semantic_judgement_status != "ran": + raise ValueError("successful L3 summary must have semantic_judgement_status=ran") + if frame.summary_failure_type != "none": + raise ValueError("successful L3 summary must have summary_failure_type=none") + if not frame.plain_document_summary.strip(): + raise ValueError("plain_document_summary must not be empty when summary ran") + if not frame.task_relevant_summary.strip(): + raise ValueError("task_relevant_summary must not be empty when summary ran") + else: + if frame.semantic_judgement_status != "failed": + raise ValueError("failed L3 summary must have semantic_judgement_status=failed") + if frame.summary_failure_type == "none": + raise ValueError("failed L3 summary must include failure type") + + if frame.plain_summary_info_class != "relative": + raise ValueError("plain_document_summary must be marked relative") + if frame.plain_summary_source_mode != "direct_record": + raise ValueError("plain_document_summary must use direct_record") + if frame.plain_summary_claim_alignment != "one_document_to_one_summary": + raise ValueError("plain_document_summary must use one_document_to_one_summary") + if frame.plain_summary_source_data_id != frame.source_document_data_id: + raise ValueError("plain summary source must be the source document data_id") + + if frame.task_relevant_summary_info_class != "mixed": + raise ValueError("task_relevant_summary must be marked mixed") + if frame.task_relevant_summary_source_mode != "source_bundle": + raise ValueError("task_relevant_summary must use source_bundle") + if frame.task_relevant_summary_claim_alignment != "one_document_plus_task_context": + raise ValueError("task_relevant_summary must use one_document_plus_task_context") + if len(frame.task_relevant_summary_source_data_ids) < 2: + raise ValueError("task_relevant_summary_source_data_ids must contain a source bundle") + + if frame.source_document_data_id not in frame.source_data_ids: + raise ValueError("L3 summary source_data_ids must include source_document_data_id") + for data_id in frame.task_relevant_summary_source_data_ids: + if not data_id: + raise ValueError( + "L3 summary task_relevant_summary_source_data_ids must not contain empty values" + ) + if data_id not in frame.source_data_ids: + raise ValueError( + "L3 summary source_data_ids must include every task_relevant source data_id" + ) + if frame.llm_call_data_id is not None: + if not frame.llm_call_data_id: + raise ValueError("L3 summary llm_call_data_id must not be empty") + if frame.llm_call_data_id not in frame.source_data_ids: + raise ValueError("L3 summary source_data_ids must include llm_call_data_id") + for trace_id in frame.source_trace_ids: + if not trace_id: + raise ValueError("L3 summary source_trace_ids must not contain empty values") + for data_id in frame.source_data_ids: + if not data_id: + raise ValueError("L3 summary source_data_ids must not contain empty values") + + @dataclass class FailureSignal: """기억 부족이나 스키마 실패 같은 문제 발생 사실을 알리는 신호.""" diff --git a/songryeon_core/core/songryeon_source_manifest.py b/songryeon_core/core/songryeon_source_manifest.py new file mode 100644 index 0000000..2b90cab --- /dev/null +++ b/songryeon_core/core/songryeon_source_manifest.py @@ -0,0 +1,230 @@ +from __future__ import annotations + +from dataclasses import dataclass +from pathlib import Path + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.graph_source_ingest import ( + GraphSourceKindIngestResult, + record_graph_source_kind_ingest, +) +from songryeon_core.core.trace_store import TraceStore + + +SONGRYEON_CORE_SOURCE_MANIFEST_POLICY_ID = "SONGRYEON_CORE_SOURCE_MANIFEST_V0" +SONGRYEON_CORE_SOURCE_MANIFEST_GENERATOR = "CODE:SONGRYEON_CORE_SOURCE_MANIFEST" +SONGRYEON_CORE_SOURCE_MANIFEST_DATA_TYPE = "graph_source:songryeon_core_source_manifest_frame" +SONGRYEON_CORE_SOURCE_MANIFEST_EXPLICIT_PATHS = { + "internal_document": [ + "AGENTS.md", + "README.md", + ], + "source_code_file": [ + "main.py", + ], +} +SONGRYEON_CORE_SOURCE_MANIFEST_GLOBS = { + "internal_document": [ + "Administrative_Reform_1/**/*.md", + ], + "source_code_file": [ + "songryeon_core/**/*.py", + "tests/**/*.py", + ], +} + + +@dataclass(frozen=True) +class SongRyeonCoreSourceManifest: + frame_id: str + root_path: str + policy_id: str + explicit_paths_by_kind: dict[str, list[str]] + glob_patterns_by_kind: dict[str, list[str]] + source_paths_by_kind: dict[str, list[str]] + source_kind_counts: dict[str, int] + generated_by: str = SONGRYEON_CORE_SOURCE_MANIFEST_GENERATOR + info_class: str = "absolute" + semantic_judgement_status: str = "not_run" + + +@dataclass(frozen=True) +class SongRyeonCoreSourceManifestIngestResult: + manifest: SongRyeonCoreSourceManifest + manifest_trace_event_id: str + manifest_data_id: str + ingest_result: GraphSourceKindIngestResult + + +def songryeon_core_source_manifest_frame_id(batch_id: str) -> str: + return f"graph_source:songryeon_core_source_manifest:{batch_id}" + + +def resolve_songryeon_core_source_manifest( + *, + root_path: str | Path, + batch_id: str, +) -> SongRyeonCoreSourceManifest: + root = Path(root_path).resolve() + if not root.exists(): + raise FileNotFoundError(root.as_posix()) + if not root.is_dir(): + raise ValueError(f"SongRyeon Core root must be a directory: {root.as_posix()}") + + source_paths_by_kind: dict[str, list[str]] = {} + for source_kind in sorted( + { + *SONGRYEON_CORE_SOURCE_MANIFEST_EXPLICIT_PATHS.keys(), + *SONGRYEON_CORE_SOURCE_MANIFEST_GLOBS.keys(), + } + ): + resolved_paths: list[Path] = [] + for relative_path in SONGRYEON_CORE_SOURCE_MANIFEST_EXPLICIT_PATHS.get( + source_kind, + [], + ): + path = root / relative_path + if not path.exists(): + raise FileNotFoundError(path.as_posix()) + if not path.is_file(): + raise ValueError(f"manifest explicit path must be a file: {path.as_posix()}") + resolved_paths.append(path) + + for pattern in SONGRYEON_CORE_SOURCE_MANIFEST_GLOBS.get(source_kind, []): + resolved_paths.extend( + path + for path in sorted(root.glob(pattern)) + if path.is_file() + ) + source_paths_by_kind[source_kind] = _dedupe_paths(resolved_paths) + + return SongRyeonCoreSourceManifest( + frame_id=songryeon_core_source_manifest_frame_id(batch_id), + root_path=root.as_posix(), + policy_id=SONGRYEON_CORE_SOURCE_MANIFEST_POLICY_ID, + explicit_paths_by_kind={ + key: list(values) + for key, values in SONGRYEON_CORE_SOURCE_MANIFEST_EXPLICIT_PATHS.items() + }, + glob_patterns_by_kind={ + key: list(values) + for key, values in SONGRYEON_CORE_SOURCE_MANIFEST_GLOBS.items() + }, + source_paths_by_kind=source_paths_by_kind, + source_kind_counts={ + source_kind: len(paths) + for source_kind, paths in source_paths_by_kind.items() + }, + ) + + +def record_songryeon_core_source_manifest_ingest( + *, + trace_store: TraceStore, + data_store: DataStore, + root_path: str | Path, + turn_id: str, + batch_id: str, + observed_at: str | None = None, + ingested_at: str | None = None, +) -> SongRyeonCoreSourceManifestIngestResult: + """Resolve the approved SongRyeon Core manifest and ingest raw source snapshots.""" + + manifest = resolve_songryeon_core_source_manifest( + root_path=root_path, + batch_id=batch_id, + ) + manifest_event = trace_store.create_event( + turn_id=turn_id, + actor="songryeon_core_source_manifest", + event_type="node_output", + output_ref=[manifest.frame_id], + schema_status="passed", + ) + _record_manifest_payload_if_missing( + data_store=data_store, + manifest=manifest, + created_at=manifest_event.timestamp, + source_trace_id=manifest_event.event_id, + ) + ingest_result = record_graph_source_kind_ingest( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + batch_id=batch_id, + source_paths_by_kind=manifest.source_paths_by_kind, + input_ref=[manifest_event.event_id], + observed_at=observed_at, + ingested_at=ingested_at, + store_text_snapshots=True, + ) + return SongRyeonCoreSourceManifestIngestResult( + manifest=manifest, + manifest_trace_event_id=manifest_event.event_id, + manifest_data_id=manifest.frame_id, + ingest_result=ingest_result, + ) + + +def _record_manifest_payload_if_missing( + *, + data_store: DataStore, + manifest: SongRyeonCoreSourceManifest, + created_at: str, + source_trace_id: str, +) -> None: + payload = { + "frame_id": manifest.frame_id, + "root_path": manifest.root_path, + "policy_id": manifest.policy_id, + "explicit_paths_by_kind": manifest.explicit_paths_by_kind, + "glob_patterns_by_kind": manifest.glob_patterns_by_kind, + "source_paths_by_kind": manifest.source_paths_by_kind, + "source_kind_counts": manifest.source_kind_counts, + "generated_by": manifest.generated_by, + "info_class": manifest.info_class, + "semantic_judgement_status": manifest.semantic_judgement_status, + } + existing = data_store.get_record(manifest.frame_id) + if existing is not None: + if existing.data_type != SONGRYEON_CORE_SOURCE_MANIFEST_DATA_TYPE: + raise ValueError(f"manifest data_id collision with different type: {manifest.frame_id}") + if existing.payload != payload: + raise ValueError( + f"manifest data_id collision with different payload: {manifest.frame_id}" + ) + return + data_store.create_record( + data_id=manifest.frame_id, + data_type=SONGRYEON_CORE_SOURCE_MANIFEST_DATA_TYPE, + exists=True, + created_at=created_at, + source_trace_id=source_trace_id, + payload=payload, + ) + + +def _dedupe_paths(paths: list[Path]) -> list[str]: + seen: set[str] = set() + result: list[str] = [] + for path in paths: + normalized = path.resolve().as_posix() + if normalized in seen: + continue + seen.add(normalized) + result.append(normalized) + return result + + +__all__ = [ + "SONGRYEON_CORE_SOURCE_MANIFEST_DATA_TYPE", + "SONGRYEON_CORE_SOURCE_MANIFEST_EXPLICIT_PATHS", + "SONGRYEON_CORE_SOURCE_MANIFEST_GENERATOR", + "SONGRYEON_CORE_SOURCE_MANIFEST_GLOBS", + "SONGRYEON_CORE_SOURCE_MANIFEST_POLICY_ID", + "SongRyeonCoreSourceManifest", + "SongRyeonCoreSourceManifestIngestResult", + "record_songryeon_core_source_manifest_ingest", + "resolve_songryeon_core_source_manifest", + "songryeon_core_source_manifest_frame_id", +] diff --git a/songryeon_core/core/source_version_lineage.py b/songryeon_core/core/source_version_lineage.py new file mode 100644 index 0000000..08b0286 --- /dev/null +++ b/songryeon_core/core/source_version_lineage.py @@ -0,0 +1,705 @@ +from __future__ import annotations + +import hashlib +from dataclasses import asdict, dataclass +from datetime import datetime + +from songryeon_core.core.data_store import DataRecord, DataStore +from songryeon_core.core.schemas import ( + SourceObservationLedgerFrame, + SourceVersionLineageFrame, + SummaryInvalidationLedgerFrame, + validate_source_observation_ledger_frame, + validate_source_version_lineage_frame, + validate_summary_invalidation_ledger_frame, +) +from songryeon_core.core.trace_store import TraceStore + + +SOURCE_VERSION_LINEAGE_FRAME_DATA_TYPE = "graph_source:source_version_lineage_frame" +SOURCE_OBSERVATION_LEDGER_FRAME_DATA_TYPE = "graph_source:source_observation_ledger_frame" +SUMMARY_INVALIDATION_LEDGER_FRAME_DATA_TYPE = "graph_source:summary_invalidation_ledger_frame" +SOURCE_VERSION_LINEAGE_POLICY_ID = "SOURCE_VERSION_LINEAGE_V0" +SOURCE_OBSERVATION_LEDGER_POLICY_ID = "SOURCE_OBSERVATION_LEDGER_V0" +SUMMARY_INVALIDATION_LEDGER_POLICY_ID = "SUMMARY_INVALIDATION_LEDGER_V0" + + +@dataclass(frozen=True) +class RecordedSourceVersionLineageResult: + lineage_frames: list[SourceVersionLineageFrame] + observation_ledger: SourceObservationLedgerFrame + invalidation_ledger: SummaryInvalidationLedgerFrame + trace_event_id: str + created_data_ids: list[str] + existing_data_ids: list[str] + + +@dataclass(frozen=True) +class _SourceVersion: + source_kind: str + path: str + source_graph_node_id: str + source_file_data_id: str + content_sha1: str + observed_at: str + ingested_at: str + source_last_modified_at: str + source_trace_id: str | None + + +def source_version_lineage_frame_id( + *, + source_kind: str, + path: str, + active_source_graph_node_id: str | None = None, +) -> str: + identity_digest = _source_identity_digest(source_kind, path) + if active_source_graph_node_id is None: + return f"graph_source:source_version_lineage:{identity_digest}" + active_digest = hashlib.sha1( + active_source_graph_node_id.encode("utf-8") + ).hexdigest()[:16] + return f"graph_source:source_version_lineage:{identity_digest}:{active_digest}" + + +def source_identity_key(*, source_kind: str, path: str) -> str: + return f"source_identity:{_source_identity_digest(source_kind, path)}" + + +def summary_invalidation_ledger_frame_id(batch_id: str) -> str: + if not batch_id: + raise ValueError("batch_id must not be empty") + return f"graph_source:summary_invalidation_ledger:{batch_id}" + + +def source_observation_ledger_frame_id(batch_id: str) -> str: + if not batch_id: + raise ValueError("batch_id must not be empty") + return f"graph_source:source_observation_ledger:{batch_id}" + + +def build_source_version_lineage_frames( + *, + data_store: DataStore, +) -> list[SourceVersionLineageFrame]: + """Build absolute source version lineages from existing raw_source records.""" + + versions_by_identity: dict[tuple[str, str], list[_SourceVersion]] = {} + for version in _iter_source_versions(data_store): + versions_by_identity.setdefault((version.source_kind, version.path), []).append(version) + + frames: list[SourceVersionLineageFrame] = [] + for (source_kind, path), versions in sorted(versions_by_identity.items()): + ordered_versions = sorted( + versions, + key=lambda item: (item.observed_at, item.source_graph_node_id), + ) + frame = _build_lineage_frame( + source_kind=source_kind, + path=path, + versions=ordered_versions, + ) + validate_source_version_lineage_frame(frame) + frames.append(frame) + return frames + + +def build_summary_invalidation_ledger_frame( + *, + data_store: DataStore, + batch_id: str, + lineage_frames: list[SourceVersionLineageFrame], + invalidated_at: str | None = None, +) -> SummaryInvalidationLedgerFrame: + """Build an absolute ledger for summaries derived from superseded source versions.""" + + timestamp = invalidated_at or _now_iso() + changed_lineages = [ + frame for frame in lineage_frames if frame.lineage_status == "content_changed" + ] + superseded_to_active: dict[str, tuple[SourceVersionLineageFrame, str]] = {} + for frame in changed_lineages: + for superseded_source_id in frame.superseded_source_graph_node_ids: + superseded_to_active[superseded_source_id] = ( + frame, + frame.active_source_graph_node_id, + ) + + invalidated_summary_node_ids: list[str] = [] + invalidation_records: list[dict[str, str]] = [] + for summary_record in _iter_summary_records(data_store): + payload = summary_record.payload + if not isinstance(payload, dict): + continue + summary_node_id = _payload_text(payload, "node_id") or summary_record.data_id + source_graph_node_ids = _string_list(payload.get("source_graph_node_ids")) + for source_graph_node_id in source_graph_node_ids: + if source_graph_node_id not in superseded_to_active: + continue + lineage_frame, active_source_graph_node_id = superseded_to_active[ + source_graph_node_id + ] + invalidated_summary_node_ids.append(summary_node_id) + invalidation_records.append( + { + "summary_graph_node_id": summary_node_id, + "invalidated_reason_code": "source_content_changed", + "source_lineage_frame_id": lineage_frame.frame_id, + "superseded_source_graph_node_id": source_graph_node_id, + "superseding_source_graph_node_id": active_source_graph_node_id, + "invalidated_at": timestamp, + "validity_status": "invalidated_by_source_change", + } + ) + + frame = SummaryInvalidationLedgerFrame( + frame_id=summary_invalidation_ledger_frame_id(batch_id), + batch_id=batch_id, + ledger_status="invalidations_recorded" + if invalidation_records + else "no_invalidations", + invalidated_summary_node_ids=_unique_strings(invalidated_summary_node_ids), + invalidation_records=_dedupe_invalidation_records(invalidation_records), + changed_source_lineage_frame_ids=_unique_strings( + [frame.frame_id for frame in changed_lineages] + ), + changed_source_graph_node_ids=_unique_strings( + [ + source_graph_node_id + for frame in changed_lineages + for source_graph_node_id in frame.superseded_source_graph_node_ids + ] + ), + active_source_graph_node_ids=_unique_strings( + [frame.active_source_graph_node_id for frame in changed_lineages] + ), + source_graph_node_ids=_unique_strings( + [ + source_graph_node_id + for frame in changed_lineages + for source_graph_node_id in frame.version_source_graph_node_ids + ] + ), + source_trace_ids=_unique_strings( + [ + trace_id + for frame in changed_lineages + for trace_id in frame.source_trace_ids + ] + ), + source_data_ids=_unique_strings( + [ + *[frame.frame_id for frame in changed_lineages], + *[ + data_id + for frame in changed_lineages + for data_id in frame.source_data_ids + ], + *[record["summary_graph_node_id"] for record in invalidation_records], + ] + ), + ) + validate_summary_invalidation_ledger_frame(frame) + return frame + + +def build_source_observation_ledger_frame( + *, + data_store: DataStore, + batch_id: str, + observed_source_file_data_ids: list[str] | None = None, +) -> SourceObservationLedgerFrame: + """Build a per-batch absolute ledger of source observations.""" + + metadata_records = _metadata_records_by_data_id(data_store) + raw_source_by_identity_content = _raw_source_id_by_identity_content(data_store) + metadata_by_identity = _metadata_records_by_identity(data_store) + target_ids = ( + _unique_strings(observed_source_file_data_ids or []) + if observed_source_file_data_ids is not None + else sorted(metadata_records) + ) + + observation_records: list[dict[str, str]] = [] + for source_file_data_id in target_ids: + metadata_record = metadata_records.get(source_file_data_id) + if metadata_record is None or not isinstance(metadata_record.payload, dict): + continue + payload = metadata_record.payload + source_kind = _payload_text(payload, "source_kind") + path = _payload_text(payload, "path") + content_sha1 = _payload_text(payload, "content_sha1") + observed_at = _payload_text(payload, "observed_at") + if not all([source_kind, path, content_sha1, observed_at]): + continue + + active_source_graph_node_id = raw_source_by_identity_content.get( + (source_kind, path, content_sha1), + "", + ) + previous_active_source_graph_node_id = "" + observation_status = "new_source_version" + previous_metadata = _previous_metadata_record( + metadata_by_identity.get((source_kind, path), []), + source_file_data_id=source_file_data_id, + observed_at=observed_at, + ) + if previous_metadata is not None and isinstance(previous_metadata.payload, dict): + previous_content_sha1 = _payload_text(previous_metadata.payload, "content_sha1") + previous_active_source_graph_node_id = raw_source_by_identity_content.get( + ( + source_kind, + path, + previous_content_sha1, + ), + "", + ) + observation_status = ( + "unchanged" + if previous_content_sha1 == content_sha1 + else "content_changed" + ) + + if not active_source_graph_node_id: + continue + observation_records.append( + { + "source_file_data_id": source_file_data_id, + "source_kind": source_kind, + "path": path, + "observed_at": observed_at, + "content_sha1": content_sha1, + "observation_status": observation_status, + "active_source_graph_node_id": active_source_graph_node_id, + "previous_active_source_graph_node_id": previous_active_source_graph_node_id, + } + ) + + frame = SourceObservationLedgerFrame( + frame_id=source_observation_ledger_frame_id(batch_id), + batch_id=batch_id, + ledger_status="recorded" if observation_records else "no_observations", + observation_records=observation_records, + observation_status_counts=_count_observation_statuses(observation_records), + observed_source_file_data_ids=_unique_strings( + [record["source_file_data_id"] for record in observation_records] + ), + active_source_graph_node_ids=_unique_strings( + [record["active_source_graph_node_id"] for record in observation_records] + ), + source_graph_node_ids=_unique_strings( + [record["active_source_graph_node_id"] for record in observation_records] + ), + source_trace_ids=_unique_strings( + [ + metadata_records[record["source_file_data_id"]].source_trace_id + for record in observation_records + if record["source_file_data_id"] in metadata_records + ] + ), + source_data_ids=_unique_strings( + [ + *[record["source_file_data_id"] for record in observation_records], + *[record["active_source_graph_node_id"] for record in observation_records], + ] + ), + ) + validate_source_observation_ledger_frame(frame) + return frame + + +def record_source_version_lineage_and_summary_invalidation( + *, + trace_store: TraceStore, + data_store: DataStore, + turn_id: str, + batch_id: str, + input_ref: list[str] | None = None, + observed_source_file_data_ids: list[str] | None = None, + created_at: str | None = None, +) -> RecordedSourceVersionLineageResult: + """Record lineage and invalidation frames without mutating source or summary records.""" + + if not turn_id: + raise ValueError("turn_id must not be empty") + if not batch_id: + raise ValueError("batch_id must not be empty") + + timestamp = created_at or _now_iso() + lineage_frames = build_source_version_lineage_frames(data_store=data_store) + observation_ledger = build_source_observation_ledger_frame( + data_store=data_store, + batch_id=batch_id, + observed_source_file_data_ids=observed_source_file_data_ids, + ) + invalidation_ledger = build_summary_invalidation_ledger_frame( + data_store=data_store, + batch_id=batch_id, + lineage_frames=lineage_frames, + invalidated_at=timestamp, + ) + output_ref = [ + *[frame.frame_id for frame in lineage_frames], + observation_ledger.frame_id, + invalidation_ledger.frame_id, + ] + event = trace_store.create_event( + turn_id=turn_id, + actor="source_version_lineage_builder", + event_type="node_output", + timestamp=timestamp, + input_ref=input_ref or [], + output_ref=output_ref, + schema_status="passed", + ) + + created_data_ids: list[str] = [] + existing_data_ids: list[str] = [] + for frame in lineage_frames: + _record_payload_if_missing( + data_store=data_store, + data_id=frame.frame_id, + data_type=SOURCE_VERSION_LINEAGE_FRAME_DATA_TYPE, + payload=asdict(frame), + created_at=timestamp, + source_trace_id=event.event_id, + created_data_ids=created_data_ids, + existing_data_ids=existing_data_ids, + ) + _record_payload_if_missing( + data_store=data_store, + data_id=observation_ledger.frame_id, + data_type=SOURCE_OBSERVATION_LEDGER_FRAME_DATA_TYPE, + payload=asdict(observation_ledger), + created_at=timestamp, + source_trace_id=event.event_id, + created_data_ids=created_data_ids, + existing_data_ids=existing_data_ids, + ) + _record_payload_if_missing( + data_store=data_store, + data_id=invalidation_ledger.frame_id, + data_type=SUMMARY_INVALIDATION_LEDGER_FRAME_DATA_TYPE, + payload=asdict(invalidation_ledger), + created_at=timestamp, + source_trace_id=event.event_id, + created_data_ids=created_data_ids, + existing_data_ids=existing_data_ids, + ) + + return RecordedSourceVersionLineageResult( + lineage_frames=lineage_frames, + observation_ledger=observation_ledger, + invalidation_ledger=invalidation_ledger, + trace_event_id=event.event_id, + created_data_ids=created_data_ids, + existing_data_ids=existing_data_ids, + ) + + +def _build_lineage_frame( + *, + source_kind: str, + path: str, + versions: list[_SourceVersion], +) -> SourceVersionLineageFrame: + source_graph_node_ids = [version.source_graph_node_id for version in versions] + source_file_data_ids = [version.source_file_data_id for version in versions] + active_source_graph_node_id = source_graph_node_ids[-1] + superseded_source_graph_node_ids = source_graph_node_ids[:-1] + content_sha1_values = {version.content_sha1 for version in versions} + if len(versions) == 1: + lineage_status = "single_version" + else: + lineage_status = "content_changed" + + version_records: list[dict[str, str]] = [] + previous_source_graph_node_id = "" + for index, version in enumerate(versions, start=1): + version_records.append( + { + "version_index": str(index), + "source_graph_node_id": version.source_graph_node_id, + "source_file_data_id": version.source_file_data_id, + "content_sha1": version.content_sha1, + "observed_at": version.observed_at, + "ingested_at": version.ingested_at, + "source_last_modified_at": version.source_last_modified_at, + "supersedes_source_graph_node_id": previous_source_graph_node_id, + } + ) + previous_source_graph_node_id = version.source_graph_node_id + + return SourceVersionLineageFrame( + frame_id=source_version_lineage_frame_id( + source_kind=source_kind, + path=path, + active_source_graph_node_id=active_source_graph_node_id, + ), + source_identity_key=source_identity_key(source_kind=source_kind, path=path), + source_kind=source_kind, + path=path, + lineage_status=lineage_status, + active_source_graph_node_id=active_source_graph_node_id, + version_source_graph_node_ids=source_graph_node_ids, + superseded_source_graph_node_ids=superseded_source_graph_node_ids, + source_file_data_ids=source_file_data_ids, + version_records=version_records, + content_sha1_by_version={ + version.source_graph_node_id: version.content_sha1 for version in versions + }, + observed_at_by_version={ + version.source_graph_node_id: version.observed_at for version in versions + }, + source_graph_node_ids=source_graph_node_ids, + source_trace_ids=_unique_strings([version.source_trace_id for version in versions]), + source_data_ids=_unique_strings( + [ + *source_file_data_ids, + *source_graph_node_ids, + ] + ), + ) + + +def _iter_source_versions(data_store: DataStore) -> list[_SourceVersion]: + metadata_by_data_id = _metadata_records_by_data_id(data_store) + versions: list[_SourceVersion] = [] + for record in data_store.list_records(): + if record.data_type != "graph_memory:node:raw_source": + continue + if not isinstance(record.payload, dict): + continue + for source_file_data_id in _string_list(record.payload.get("source_data_ids")): + metadata_record = metadata_by_data_id.get(source_file_data_id) + if metadata_record is None or not isinstance(metadata_record.payload, dict): + continue + source_kind = _payload_text(metadata_record.payload, "source_kind") + path = _payload_text(metadata_record.payload, "path") + content_sha1 = _payload_text(metadata_record.payload, "content_sha1") + observed_at = _payload_text(metadata_record.payload, "observed_at") + ingested_at = _payload_text(metadata_record.payload, "ingested_at") + source_last_modified_at = _payload_text( + metadata_record.payload, + "source_last_modified_at", + ) + if not all( + [ + source_kind, + path, + content_sha1, + observed_at, + ingested_at, + source_last_modified_at, + ] + ): + continue + versions.append( + _SourceVersion( + source_kind=source_kind, + path=path, + source_graph_node_id=record.data_id, + source_file_data_id=source_file_data_id, + content_sha1=content_sha1, + observed_at=observed_at, + ingested_at=ingested_at, + source_last_modified_at=source_last_modified_at, + source_trace_id=record.source_trace_id or metadata_record.source_trace_id, + ) + ) + return versions + + +def _metadata_records_by_data_id(data_store: DataStore) -> dict[str, DataRecord]: + return { + record.data_id: record + for record in data_store.list_records() + if record.data_type == "graph_source:file_metadata" + } + + +def _metadata_records_by_identity(data_store: DataStore) -> dict[tuple[str, str], list[DataRecord]]: + records_by_identity: dict[tuple[str, str], list[DataRecord]] = {} + for record in data_store.list_records(): + if record.data_type != "graph_source:file_metadata": + continue + if not isinstance(record.payload, dict): + continue + source_kind = _payload_text(record.payload, "source_kind") + path = _payload_text(record.payload, "path") + if not source_kind or not path: + continue + records_by_identity.setdefault((source_kind, path), []).append(record) + for records in records_by_identity.values(): + records.sort( + key=lambda item: ( + _payload_text(item.payload, "observed_at") + if isinstance(item.payload, dict) + else "", + item.data_id, + ) + ) + return records_by_identity + + +def _raw_source_id_by_identity_content(data_store: DataStore) -> dict[tuple[str, str, str], str]: + metadata_by_data_id = _metadata_records_by_data_id(data_store) + raw_source_ids: dict[tuple[str, str, str], str] = {} + for record in data_store.list_records(): + if record.data_type != "graph_memory:node:raw_source": + continue + if not isinstance(record.payload, dict): + continue + for source_file_data_id in _string_list(record.payload.get("source_data_ids")): + metadata_record = metadata_by_data_id.get(source_file_data_id) + if metadata_record is None or not isinstance(metadata_record.payload, dict): + continue + source_kind = _payload_text(metadata_record.payload, "source_kind") + path = _payload_text(metadata_record.payload, "path") + content_sha1 = _payload_text(metadata_record.payload, "content_sha1") + if not all([source_kind, path, content_sha1]): + continue + raw_source_ids.setdefault((source_kind, path, content_sha1), record.data_id) + return raw_source_ids + + +def _previous_metadata_record( + records: list[DataRecord], + *, + source_file_data_id: str, + observed_at: str, +) -> DataRecord | None: + previous: DataRecord | None = None + for record in records: + if record.data_id == source_file_data_id: + return previous + if not isinstance(record.payload, dict): + continue + record_observed_at = _payload_text(record.payload, "observed_at") + if record_observed_at > observed_at: + return previous + previous = record + return previous + + +def _iter_summary_records(data_store: DataStore) -> list[DataRecord]: + records: list[DataRecord] = [] + for record in data_store.list_records(): + if record.data_type == "graph_memory:node:summary": + records.append(record) + continue + if not isinstance(record.payload, dict): + continue + if record.payload.get("node_kind") == "summary": + records.append(record) + return records + + +def _dedupe_invalidation_records(records: list[dict[str, str]]) -> list[dict[str, str]]: + seen: set[tuple[str, str, str]] = set() + result: list[dict[str, str]] = [] + for record in records: + key = ( + record["summary_graph_node_id"], + record["source_lineage_frame_id"], + record["superseded_source_graph_node_id"], + ) + if key in seen: + continue + seen.add(key) + result.append(record) + return result + + +def _record_payload_if_missing( + *, + data_store: DataStore, + data_id: str, + data_type: str, + payload: dict[str, object], + created_at: str, + source_trace_id: str, + created_data_ids: list[str], + existing_data_ids: list[str], +) -> None: + existing = data_store.get_record(data_id) + if existing is not None: + if existing.data_type != data_type: + raise ValueError(f"source lineage data_id collision with different type: {data_id}") + if existing.payload != payload: + raise ValueError(f"source lineage data_id collision with different payload: {data_id}") + existing_data_ids.append(data_id) + return + data_store.create_record( + data_id=data_id, + data_type=data_type, + exists=True, + created_at=created_at, + source_trace_id=source_trace_id, + payload=payload, + ) + created_data_ids.append(data_id) + + +def _payload_text(payload: dict[str, object], field_name: str) -> str: + value = payload.get(field_name) + return value if isinstance(value, str) else "" + + +def _string_list(value: object) -> list[str]: + if not isinstance(value, list): + return [] + return [item for item in value if isinstance(item, str) and item] + + +def _unique_strings(values: list[str | None]) -> list[str]: + seen: set[str] = set() + result: list[str] = [] + for value in values: + if not value or value in seen: + continue + seen.add(value) + result.append(value) + return result + + +def _count_observation_statuses(records: list[dict[str, str]]) -> dict[str, int]: + counts: dict[str, int] = {} + for record in records: + status = record.get("observation_status", "") + if not status: + continue + counts[status] = counts.get(status, 0) + 1 + return counts + + +def _source_identity_digest(source_kind: str, path: str) -> str: + if not source_kind: + raise ValueError("source_kind must not be empty") + if not path: + raise ValueError("path must not be empty") + digest_source = f"{source_kind}:{path}" + return hashlib.sha1(digest_source.encode("utf-8")).hexdigest()[:16] + + +def _now_iso() -> str: + return datetime.now().isoformat(timespec="seconds") + + +__all__ = [ + "SOURCE_VERSION_LINEAGE_FRAME_DATA_TYPE", + "SOURCE_OBSERVATION_LEDGER_FRAME_DATA_TYPE", + "SOURCE_VERSION_LINEAGE_POLICY_ID", + "SOURCE_OBSERVATION_LEDGER_POLICY_ID", + "SUMMARY_INVALIDATION_LEDGER_FRAME_DATA_TYPE", + "SUMMARY_INVALIDATION_LEDGER_POLICY_ID", + "RecordedSourceVersionLineageResult", + "build_source_version_lineage_frames", + "build_source_observation_ledger_frame", + "build_summary_invalidation_ledger_frame", + "record_source_version_lineage_and_summary_invalidation", + "source_identity_key", + "source_observation_ledger_frame_id", + "source_version_lineage_frame_id", + "summary_invalidation_ledger_frame_id", +] diff --git a/songryeon_core/core/turn_activity_graph_links.py b/songryeon_core/core/turn_activity_graph_links.py new file mode 100644 index 0000000..1d1ed55 --- /dev/null +++ b/songryeon_core/core/turn_activity_graph_links.py @@ -0,0 +1,320 @@ +from __future__ import annotations + +import json +from dataclasses import asdict + +from songryeon_core.core.data_store import DataRecord, DataStore +from songryeon_core.core.graph_memory import raw_capsule_graph_node_id +from songryeon_core.core.schemas import ( + GraphMemoryEdgeFrame, + GraphMemoryNodeFrame, + TurnActivityGraphLinkFrame, + validate_graph_memory_edge_frame, + validate_graph_memory_node_frame, + validate_turn_activity_graph_link_frame, +) +from songryeon_core.core.trace_store import TraceStore + + +L_LOOP_ACTIVITY_LEDGER_DATA_TYPE = "loop_activity:l_loop_activity_ledger_frame" +R_GRAPH_ACCESS_LEDGER_DATA_TYPE = "graph_memory:turn_access_ledger_frame" +TURN_ACTIVITY_GRAPH_LINK_DATA_TYPE = "graph_memory:turn_activity_graph_link_frame" + + +def turn_activity_graph_link_frame_id(turn_id: str) -> str: + if not turn_id: + raise ValueError("turn_id must not be empty") + return f"graph:turn_activity_graph_link:{turn_id}" + + +def activity_ledger_graph_node_id(ledger_data_id: str) -> str: + if not ledger_data_id: + raise ValueError("ledger_data_id must not be empty") + return f"graph:activity_ledger:{ledger_data_id}" + + +def activity_ledger_graph_edge_id(*, raw_capsule_node_id: str, activity_node_id: str) -> str: + if not raw_capsule_node_id: + raise ValueError("raw_capsule_node_id must not be empty") + if not activity_node_id: + raise ValueError("activity_node_id must not be empty") + return f"graph:edge:has_activity_ledger:{raw_capsule_node_id}:{activity_node_id}" + + +def record_turn_activity_graph_links( + *, + trace_store: TraceStore, + data_store: DataStore, + turn_id: str, + l_loop_activity_ledger_data_ids: list[str] | None = None, + r_graph_access_ledger_data_ids: list[str] | None = None, + frame_id: str | None = None, +) -> tuple[str, str, TurnActivityGraphLinkFrame]: + """Create graph nodes/edges connecting a raw capsule to L/R activity ledgers.""" + + raw_node_id = raw_capsule_graph_node_id(turn_id) + l_ledger_records = _ledger_records( + data_store=data_store, + turn_id=turn_id, + data_type=L_LOOP_ACTIVITY_LEDGER_DATA_TYPE, + explicit_data_ids=l_loop_activity_ledger_data_ids, + ) + r_ledger_records = _ledger_records( + data_store=data_store, + turn_id=turn_id, + data_type=R_GRAPH_ACCESS_LEDGER_DATA_TYPE, + explicit_data_ids=r_graph_access_ledger_data_ids, + ) + activity_nodes: list[GraphMemoryNodeFrame] = [] + activity_edges: list[GraphMemoryEdgeFrame] = [] + link_records: list[dict[str, str]] = [] + + for activity_kind, records, source_field in [ + ("l_loop_activity_ledger", l_ledger_records, "l_loop_activity_ledger_data_ids"), + ("r_graph_access_ledger", r_ledger_records, "r_graph_access_ledger_data_ids"), + ]: + for record in records: + node = _build_activity_ledger_node( + turn_id=turn_id, + raw_node_id=raw_node_id, + ledger_record=record, + activity_kind=activity_kind, + ) + edge = _build_activity_ledger_edge( + raw_node_id=raw_node_id, + activity_node=node, + ledger_data_id=record.data_id, + ) + activity_nodes.append(node) + activity_edges.append(edge) + link_records.append( + { + "activity_kind": activity_kind, + "ledger_data_id": record.data_id, + "graph_node_id": node.node_id, + "edge_id": edge.edge_id, + "source_field": source_field, + } + ) + + source_trace_ids = _unique_strings( + [ + trace_id + for node in activity_nodes + for trace_id in node.source_trace_ids + ] + ) + source_data_ids = _unique_strings( + [ + raw_node_id, + *[record.data_id for record in l_ledger_records], + *[record.data_id for record in r_ledger_records], + *[node.node_id for node in activity_nodes], + *[edge.edge_id for edge in activity_edges], + ] + ) + frame = TurnActivityGraphLinkFrame( + frame_id=frame_id or turn_activity_graph_link_frame_id(turn_id), + turn_id=turn_id, + turn_capsule_graph_node_id=raw_node_id, + l_loop_activity_ledger_data_ids=[record.data_id for record in l_ledger_records], + r_graph_access_ledger_data_ids=[record.data_id for record in r_ledger_records], + activity_ledger_graph_node_ids=[node.node_id for node in activity_nodes], + activity_ledger_graph_edge_ids=[edge.edge_id for edge in activity_edges], + link_records=link_records, + source_trace_ids=source_trace_ids, + source_data_ids=source_data_ids, + ) + validate_turn_activity_graph_link_frame(frame) + + event = trace_store.create_event( + turn_id=turn_id, + actor="graph_activity_link_builder", + event_type="node_output", + input_ref=source_trace_ids, + output_ref=[ + frame.frame_id, + *[node.node_id for node in activity_nodes], + *[edge.edge_id for edge in activity_edges], + ], + schema_status="passed", + ) + created_data_ids: list[str] = [] + existing_data_ids: list[str] = [] + for node in activity_nodes: + _record_payload_if_missing( + data_store=data_store, + data_id=node.node_id, + data_type=f"graph_memory:node:{node.node_kind}", + payload=asdict(node), + created_at=event.timestamp, + source_trace_id=event.event_id, + created_data_ids=created_data_ids, + existing_data_ids=existing_data_ids, + ) + for edge in activity_edges: + _record_payload_if_missing( + data_store=data_store, + data_id=edge.edge_id, + data_type=f"graph_memory:edge:{edge.edge_kind}", + payload=asdict(edge), + created_at=event.timestamp, + source_trace_id=event.event_id, + created_data_ids=created_data_ids, + existing_data_ids=existing_data_ids, + ) + _record_payload_if_missing( + data_store=data_store, + data_id=frame.frame_id, + data_type=TURN_ACTIVITY_GRAPH_LINK_DATA_TYPE, + payload=asdict(frame), + created_at=event.timestamp, + source_trace_id=event.event_id, + created_data_ids=created_data_ids, + existing_data_ids=existing_data_ids, + ) + return event.event_id, frame.frame_id, frame + + +def _build_activity_ledger_node( + *, + turn_id: str, + raw_node_id: str, + ledger_record: DataRecord, + activity_kind: str, +) -> GraphMemoryNodeFrame: + payload = ledger_record.payload if isinstance(ledger_record.payload, dict) else {} + source_trace_ids = _unique_strings(_string_list(payload.get("source_trace_ids"))) + node = GraphMemoryNodeFrame( + node_id=activity_ledger_graph_node_id(ledger_record.data_id), + node_kind="activity_ledger", + data_kind=activity_kind, + source_turn_id=turn_id, + trace_count=len(source_trace_ids), + movement_count=0, + summary_depth=0, + source_depth_min=0, + source_depth_max=0, + source_leaf_count=1, + source_summary_count=0, + source_bundle_kind="activity_ledger", + bundle_policy_id="turn_activity_graph_link_v0", + source_char_count=_payload_char_count(ledger_record.payload), + source_graph_node_ids=[raw_node_id], + source_trace_ids=source_trace_ids, + source_data_ids=[ledger_record.data_id], + ) + validate_graph_memory_node_frame(node) + return node + + +def _build_activity_ledger_edge( + *, + raw_node_id: str, + activity_node: GraphMemoryNodeFrame, + ledger_data_id: str, +) -> GraphMemoryEdgeFrame: + edge = GraphMemoryEdgeFrame( + edge_id=activity_ledger_graph_edge_id( + raw_capsule_node_id=raw_node_id, + activity_node_id=activity_node.node_id, + ), + edge_kind="HAS_ACTIVITY_LEDGER", + from_node_id=raw_node_id, + to_node_id=activity_node.node_id, + source_graph_node_ids=[raw_node_id, activity_node.node_id], + source_trace_ids=list(activity_node.source_trace_ids), + source_data_ids=[raw_node_id, activity_node.node_id, ledger_data_id], + ) + validate_graph_memory_edge_frame(edge) + return edge + + +def _ledger_records( + *, + data_store: DataStore, + turn_id: str, + data_type: str, + explicit_data_ids: list[str] | None, +) -> list[DataRecord]: + if explicit_data_ids is not None: + records = [data_store.require_record(data_id) for data_id in explicit_data_ids] + else: + records = [ + record + for record in data_store.list_records() + if record.data_type == data_type + ] + filtered: list[DataRecord] = [] + seen: set[str] = set() + for record in records: + if record.data_type != data_type: + raise ValueError(f"unexpected ledger data_type for {record.data_id}: {record.data_type}") + payload = record.payload if isinstance(record.payload, dict) else {} + if payload.get("turn_id") != turn_id: + continue + if record.data_id in seen: + continue + seen.add(record.data_id) + filtered.append(record) + return filtered + + +def _record_payload_if_missing( + *, + data_store: DataStore, + data_id: str, + data_type: str, + payload: object, + created_at: str, + source_trace_id: str, + created_data_ids: list[str], + existing_data_ids: list[str], +) -> None: + existing = data_store.get_record(data_id) + if existing is not None: + if existing.data_type != data_type or existing.payload != payload: + raise ValueError(f"data_id collision with different payload: {data_id}") + existing_data_ids.append(data_id) + return + data_store.create_record( + data_id=data_id, + data_type=data_type, + exists=True, + created_at=created_at, + source_trace_id=source_trace_id, + payload=payload, + ) + created_data_ids.append(data_id) + + +def _payload_char_count(payload: object) -> int: + return len(json.dumps(payload, ensure_ascii=False, sort_keys=True)) + + +def _string_list(value: object) -> list[str]: + if not isinstance(value, list): + return [] + return [item for item in value if isinstance(item, str) and item] + + +def _unique_strings(values: list[str | None]) -> list[str]: + seen: set[str] = set() + result: list[str] = [] + for value in values: + if value is None or value in seen: + continue + seen.add(value) + result.append(value) + return result + + +__all__ = [ + "L_LOOP_ACTIVITY_LEDGER_DATA_TYPE", + "R_GRAPH_ACCESS_LEDGER_DATA_TYPE", + "TURN_ACTIVITY_GRAPH_LINK_DATA_TYPE", + "activity_ledger_graph_edge_id", + "activity_ledger_graph_node_id", + "record_turn_activity_graph_links", + "turn_activity_graph_link_frame_id", +] diff --git a/songryeon_core/llm/fake.py b/songryeon_core/llm/fake.py index 13511a3..76ae4bd 100644 --- a/songryeon_core/llm/fake.py +++ b/songryeon_core/llm/fake.py @@ -222,6 +222,42 @@ def complete(self, request: LLMRequest) -> LLMResponse: macro_goal = str(revision_input.get("macro_goal") or "internal document evidence").strip() l3_status = str(revision_input.get("l3_goal_status") or "partial").strip() previous_query = str(revision_input.get("previous_query_text") or "").strip() + unread_candidate_doc_ids = revision_input.get("unread_candidate_doc_ids") + remaining_query_attempts = revision_input.get("remaining_query_attempts") + remaining_read_doc_calls = revision_input.get("remaining_read_doc_calls") + if ( + isinstance(unread_candidate_doc_ids, list) + and unread_candidate_doc_ids + and remaining_query_attempts == 0 + and isinstance(remaining_read_doc_calls, int) + and remaining_read_doc_calls > 0 + ): + first_doc_id = next( + (item for item in unread_candidate_doc_ids if isinstance(item, str) and item), + None, + ) + if first_doc_id: + payload = { + "planner_mode": "revision_llm", + "selected_candidate_id": "L2:revision_query_candidate_0001", + "candidates": [ + { + "candidate_id": "L2:revision_query_candidate_0001", + "query_text": first_doc_id, + "purpose": "query 예산이 소진되어 이미 보존된 unread candidate 원문을 읽는다.", + "expected_signal": "해당 후보 문서의 read_doc 원문", + "priority": 1, + "target_tool_name": "read_doc", + "source_data_ids": source_data_ids, + } + ], + } + return LLMResponse( + text=json.dumps(payload, ensure_ascii=False), + model_id=self.model_id, + raw=payload, + ) + revised_query = f"{macro_goal} revised evidence after {l3_status}" if previous_query and revised_query == previous_query: revised_query = f"{previous_query} alternate evidence" @@ -268,14 +304,21 @@ def complete(self, request: LLMRequest) -> LLMResponse: payload = self._node_1_payload(request) elif "L1 Goal Setter" in prompt: payload = self._l1_payload() + elif "L Tool Scope Planner" in prompt: + payload = self._l_tool_scope_payload(request) elif "L3 Result Keeper" in prompt: payload = self._l3_payload(request) + elif "node_2 Answer Basis Selector" in prompt: + payload = self._node_2_answer_basis_payload(request) elif "node_2 Metainfo Boundary" in prompt: payload = self._node_2_payload(request) elif "node_3 Reporter" in prompt or "Final Reporter" in prompt: payload = self._node_3_payload(request) elif "node_4 Gatekeeper" in prompt: - payload = self._node_4_payload() + try: + payload = self._node_4_payload(request) + except TypeError: + payload = self._node_4_payload() elif "Memory Relevance Selector" in prompt: payload = MemoryRelevanceNoneSelectedFakeLLMAdapter().complete(request).raw elif "L2" in prompt or "query" in prompt.lower(): @@ -336,6 +379,15 @@ def _l1_payload(self) -> dict[str, object]: "budget_request_reason": "fake smoke에서는 여러 후보와 최소 2개 문서 열람 요청을 재현하기 위해 보수적 추가 예산을 요청한다.", } + def _l_tool_scope_payload(self, request: LLMRequest) -> dict[str, object]: + return { + "tool_scope_mode": "document_only", + "allowed_tool_groups": ["document_tools"], + "required_materials": ["project_document"], + "scope_reason": "fake adapter selected document tools for smoke testing.", + "scope_reason_info_class": "mixed", + } + def _l3_payload(self, request: LLMRequest) -> dict[str, object]: candidate_count = int(request.input_payload.get("candidate_count") or 0) controller_decision = request.input_payload.get("controller_decision") @@ -367,11 +419,72 @@ def _node_2_payload(self, request: LLMRequest) -> dict[str, object]: "excluded_claims": [], } + def _node_2_answer_basis_payload(self, request: LLMRequest) -> dict[str, object]: + user_question = str(request.input_payload.get("user_question") or "") + source_data_ids = request.input_payload.get("source_data_ids") + if not isinstance(source_data_ids, list): + source_data_ids = [] + source_ids = [item for item in source_data_ids if isinstance(item, str) and item] + primary_source = source_ids[0] if source_ids else "not_supplied" + document_source = next( + ( + source_id + for source_id in source_ids + if "boundary" in source_id or "handoff" in source_id or "L3" in source_id + ), + primary_source, + ) + if any( + keyword in user_question + for keyword in ("몇 개", "count", "route", "smoke", "통과", "문서에 뭐", "원문", "trace") + ): + mode = "absolute_first" + reason_codes = ["code_verified_fact_required", "runtime_state_basis_present"] + reason = "사용자 요청이 count, route, trace, 문서 원문처럼 확인 가능한 값 중심 답변을 요구한다." + elif any( + keyword in user_question + for keyword in ("어때", "좋을까", "아이디어", "브레인스토밍", "설명해", "개선") + ): + mode = "relative_allowed" + reason_codes = ["user_asked_for_interpretation"] + reason = "사용자 요청이 구조 의견이나 설명처럼 해석을 허용하는 답변을 요구한다." + else: + mode = "mixed_or_uncertain" + reason_codes = ["multi_source_bundle", "partial_evidence_only"] + reason = "사용자 요청은 제공된 여러 source bundle과 부분 근거를 함께 보며 한계를 표시해야 한다." + return { + "answer_basis_mode": mode, + "basis_reason_codes": reason_codes, + "mode_selection_reason": reason, + "mode_selection_reason_info_class": "mixed", + "evidence_roles": [ + { + "source_data_id": document_source, + "evidence_role": "primary_answer_basis", + "role_reason": "fake adapter가 supplied source bundle 안에서 대표 근거 역할을 부여했다.", + "role_reason_info_class": "mixed", + } + ] + if source_ids + else [], + } + def _node_3_payload(self, request: LLMRequest) -> dict[str, object]: - extracts = request.input_payload.get("read_documents") + extracts = request.input_payload.get("supplied_document_contexts") + if not isinstance(extracts, list): + extracts = request.input_payload.get("read_documents") if not isinstance(extracts, list): extracts = request.input_payload.get("document_extracts") selected_contexts = request.input_payload.get("selected_recent_memory_contexts") + l_loop_result = request.input_payload.get("l_loop_result") + if not isinstance(l_loop_result, dict): + l_loop_result = {} + l_loop_attitude_hint = str(l_loop_result.get("attitude_hint") or "not_recorded") + l_loop_limit_note = "" + if l_loop_attitude_hint in {"l_loop_budget_exhausted", "l_loop_partial_or_failed"}: + l_loop_limit_note = ( + "\n\nL 검색 목표는 완전 성공으로 기록되지 않았으므로, 아래 내용은 공급된 자료 범위의 제한적 답변이야." + ) read_count = int(request.input_payload.get("available_document_extract_count") or 0) search_candidate_count = int(request.input_payload.get("available_search_candidate_document_count") or 0) runtime_task_count = int(request.input_payload.get("available_runtime_task_count") or 0) @@ -395,9 +508,10 @@ def _node_3_payload(self, request: LLMRequest) -> dict[str, object]: text = str(first.get("text") or "").strip() preview = " ".join(text.split())[:600] body_markdown = ( - f"이번 턴에서는 `{doc_id}` 문서를 읽은 문서 근거로 사용했어.\n\n" - f"읽은 원문 미리보기: {preview}\n\n" + f"이번 턴에서는 `{doc_id}` 문서 context를 답변 근거로 사용했어.\n\n" + f"공급 원문 미리보기: {preview}\n\n" "주의: 이것은 문서와 도구 결과에 근거한 보고이며, 문서 내용의 최종 진실성 판정은 아니다." + f"{l_loop_limit_note}" ) else: body_markdown = ( @@ -405,7 +519,25 @@ def _node_3_payload(self, request: LLMRequest) -> dict[str, object]: ) return {"body_markdown": body_markdown} - def _node_4_payload(self) -> dict[str, object]: + def _node_4_payload(self, request: LLMRequest) -> dict[str, object]: + rendered_markdown = str(request.input_payload.get("rendered_markdown") or "") + brief = request.input_payload.get("node3_input_brief") + l_loop_result = brief.get("l_loop_result") if isinstance(brief, dict) else {} + if not isinstance(l_loop_result, dict): + l_loop_result = {} + if ( + str(l_loop_result.get("attitude_hint") or "") + in {"l_loop_budget_exhausted", "l_loop_partial_or_failed"} + and "L 검색 목표가 성공" in rendered_markdown + ): + return { + "gate_status": "needs_revision", + "reason": "L loop 실패/예산소진 신호와 검색 성공 표현이 충돌한다.", + "checked_claims": ["l_loop_result_attitude"], + "unsupported_claims": ["L 검색 목표가 성공"], + "contradictions": ["l_loop_failure_hidden_as_success"], + "revision_targets": ["L 검색 목표 실패/예산소진 신호와 공급 자료 사용 범위를 분리해 말한다."], + } return { "gate_status": "pass", "reason": "보고문이 제공된 node3_input_brief 범위 안에서 작성되었다.", diff --git a/songryeon_core/loops/l_loop.py b/songryeon_core/loops/l_loop.py index 7d8b4b1..fe36b9e 100644 --- a/songryeon_core/loops/l_loop.py +++ b/songryeon_core/loops/l_loop.py @@ -48,6 +48,11 @@ run_l3_result_keeper, run_l3_revision_result_keeper, ) +from songryeon_core.nodes.l_tool_scope import ( + filter_available_tools_for_scope, + record_l_tool_budget_partition, + run_l_tool_scope_planner, +) from songryeon_core.nodes.node_0_memory_supplier import ( record_l3_continuation_summary_for_l2, ) @@ -60,6 +65,12 @@ tool_catalog_data_id, tool_choice_data_id, ) +from songryeon_core.tools.document_context_pack import ( + document_context_pack_frame_id, + explicit_artifact_reference_frame_id, + record_document_context_pack_frame, + record_explicit_artifact_reference_frame, +) from songryeon_core.tools.tool_efficiency_policy import ( cache_status_from_search_payload, distilled_input_size, @@ -89,8 +100,11 @@ class LLoopResult: continuation_trace_ids: list[str] = field(default_factory=list) revision_trace_ids: list[str] = field(default_factory=list) failure_trace_ids: list[str] = field(default_factory=list) + document_context_trace_ids: list[str] = field(default_factory=list) goal_data_ids: list[str] = field(default_factory=list) budget_plan_data_ids: list[str] = field(default_factory=list) + tool_scope_data_ids: list[str] = field(default_factory=list) + tool_budget_partition_data_ids: list[str] = field(default_factory=list) tool_catalog_data_ids: list[str] = field(default_factory=list) tool_choice_data_ids: list[str] = field(default_factory=list) query_plan_data_ids: list[str] = field(default_factory=list) @@ -104,6 +118,8 @@ class LLoopResult: revision_query_plan_data_ids: list[str] = field(default_factory=list) revision_query_data_ids: list[str] = field(default_factory=list) failure_signal_data_ids: list[str] = field(default_factory=list) + explicit_artifact_reference_data_ids: list[str] = field(default_factory=list) + document_context_pack_data_ids: list[str] = field(default_factory=list) preserved_data_ids: list[str] = field(default_factory=list) achievement_data_ids: list[str] = field(default_factory=list) output_data_ids: list[str] = field(default_factory=list) @@ -125,6 +141,7 @@ def run_l_loop( zero_state: ZeroState | None = None, document_root: str | Path = "Administrative_Reform_1", l1_goal_adapter: LLMAdapter | None = None, + l_tool_scope_adapter: LLMAdapter | None = None, l2_query_planner_adapter: LLMAdapter | None = None, l3_result_adapter: LLMAdapter | None = None, max_iterations: int = 3, @@ -134,6 +151,7 @@ def run_l_loop( max_query_candidates: int | None = None, max_read_doc_calls: int = 1, max_input_chars: int = 6000, + max_document_context_chars: int = 30000, run_index: int = 1, same_turn_rerun_allowed: bool = False, rerun_block_reason: str = L_REROUTE_REMAINING_BLOCK_REASON, @@ -157,6 +175,8 @@ def run_l_loop( raise ValueError("max_read_doc_calls must be at least 1") if max_input_chars < 1: raise ValueError("max_input_chars must be at least 1") + if max_document_context_chars < 1: + raise ValueError("max_document_context_chars must be at least 1") # L루프 전체 실행 단위의 ID 묶음이다. # run_index=1은 기존 고정 ID를 유지한다. @@ -234,14 +254,50 @@ def run_l_loop( if isinstance(catalog_payload, dict) and isinstance(catalog_payload.get("tools"), list) else [] ) + tool_scope_trace_id, tool_scope_data_id, tool_scope_frame = run_l_tool_scope_planner( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + l1_event=l1, + user_query=search_query, + goal_data_id=l1_goal_data_id, + budget_plan_data_id=budget_plan_data_id, + tool_catalog_data_id=catalog_id, + available_tools=available_tools, + adapter=l_tool_scope_adapter, + id_namespace=run_ids, + ) + ( + tool_budget_partition_trace_id, + tool_budget_partition_data_id, + tool_budget_partition_frame, + ) = record_l_tool_budget_partition( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + tool_scope_frame=tool_scope_frame, + tool_scope_trace_id=tool_scope_trace_id, + budget_plan_data_id=budget_plan_data_id, + budget_plan_trace_id=budget_plan_trace_id, + id_namespace=run_ids, + ) + scoped_available_tools = filter_available_tools_for_scope( + available_tools, + tool_scope_frame, + ) query_text = search_query query_source = "user_input_fallback" - query_source_data_ids = [l1_goal_data_id, budget_plan_data_id] + query_source_data_ids = [ + l1_goal_data_id, + budget_plan_data_id, + tool_scope_data_id, + tool_budget_partition_data_id, + ] query_extra_trace_ids: list[str] = [] query_plan_data_ids: list[str] = [] l2_plan_trace_ids: list[str] = [] - selected_tool_name = "search_docs" + selected_tool_name = _fallback_l2_tool_for_available_tools(scoped_available_tools) if l2_query_planner_adapter is not None: try: plan_event = run_l2_query_planner( @@ -251,8 +307,16 @@ def run_l_loop( l1_event=l1, user_input=search_query, adapter=l2_query_planner_adapter, - source_data_ids=[l1_goal_data_id, budget_plan_data_id, catalog_id], - available_tools=available_tools, + source_data_ids=[ + l1_goal_data_id, + budget_plan_data_id, + catalog_id, + tool_scope_data_id, + tool_budget_partition_data_id, + ], + available_tools=scoped_available_tools, + l_tool_scope=asdict(tool_scope_frame), + budget_partition=asdict(tool_budget_partition_frame), query_plan_frame_data_id=l2_query_plan_data_id, ) plan_record = data_store.require_record(l2_query_plan_data_id) @@ -263,6 +327,8 @@ def run_l_loop( l1_goal_data_id, budget_plan_data_id, catalog_id, + tool_scope_data_id, + tool_budget_partition_data_id, l2_query_plan_data_id, ] query_extra_trace_ids = [plan_event.event_id] @@ -306,6 +372,19 @@ def run_l_loop( ] tool_choice_trace_ids = [tool_choice_trace_id] + explicit_trace_id, explicit_data_id, _ = record_explicit_artifact_reference_frame( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + user_text=search_query, + document_root=document_root, + source_trace_ids=_unique_strings([run_trace_id, l1.event_id, budget_plan_trace_id]), + source_data_ids=_unique_strings([run_data_id, l1_goal_data_id, budget_plan_data_id]), + frame_id=explicit_artifact_reference_frame_id(id_namespace=run_ids), + ) + document_context_trace_ids = [explicit_trace_id] + explicit_artifact_reference_data_ids = [explicit_data_id] + l2_query_frame = data_store.require_record(l2_query_data_id) query_text = _read_query_text_from_l2_frame(l2_query_frame.payload) tool_runner = ToolRunner(tool_registry) @@ -600,12 +679,8 @@ def record_budget( data_store=data_store, turn_id=turn_id, iteration_index=next_iteration_index, - decision="continue_read_artifact" if selected_tool_name == "read_artifact" else "continue_search", - reason=( - "CODE_STATUS:continue_read_artifact" - if selected_tool_name == "read_artifact" - else "CODE_STATUS:continue_search_docs" - ), + decision=_initial_control_decision_for_tool(selected_tool_name), + reason=_initial_control_reason_for_tool(selected_tool_name), max_iterations=max_iterations, max_tool_calls=max_tool_calls, tool_call_count=tool_call_count, @@ -640,6 +715,85 @@ def record_budget( control_data_ids.append(control_data_id) next_iteration_index += 1 + if _is_code_inspection_tool(selected_tool_name): + code_result = _run_code_inspection_tool( + tool_runner=tool_runner, + selected_tool_name=selected_tool_name, + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + input_ref=[l2.event_id, tool_choice_trace_id, control_trace_id], + id_namespace=run_ids, + query_text=current_query, + ) + tool_call_count += 1 + tool_call_trace_ids.append(code_result.trace_event_id) + tool_result_data_ids.append(code_result.data_ref.data_id) + code_distillation = record_tool_result_distillation( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + tool_result=code_result, + id_namespace=run_ids, + ) + distillation_trace_ids.append(code_distillation.trace_event_id) + tool_distillation_data_ids.append(code_distillation.data_id) + input_chars_used += len(current_query) + distilled_input_size(code_distillation.frame) + code_stop_reason = _code_inspection_stop_reason( + tool_name=selected_tool_name, + payload=code_result.payload, + ) + code_budget_trace_id, code_budget_data_id = record_budget( + stop_reason=code_stop_reason, + reason=f"CODE_STATUS:{selected_tool_name}_{code_stop_reason}", + condition_flags=[selected_tool_name, code_stop_reason], + source_trace_ids=[ + code_result.trace_event_id, + code_distillation.trace_event_id, + ], + source_data_ids=[ + code_result.data_ref.data_id, + code_distillation.data_id, + ], + ) + stop_control_trace_id, stop_control_data_id = _record_l_loop_control( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + iteration_index=next_iteration_index, + decision="stop_success" if code_stop_reason == "completed" else "stop_failed", + reason=( + f"CODE_STATUS:stop_success_{selected_tool_name}" + if code_stop_reason == "completed" + else f"CODE_STATUS:stop_failed_{selected_tool_name}_{code_stop_reason}" + ), + max_iterations=max_iterations, + max_tool_calls=max_tool_calls, + tool_call_count=tool_call_count, + source_trace_ids=_unique_strings( + [ + control_trace_id, + code_result.trace_event_id, + code_distillation.trace_event_id, + code_budget_trace_id, + ] + ), + source_data_ids=_unique_strings( + [ + control_data_id, + code_result.data_ref.data_id, + code_distillation.data_id, + code_budget_data_id, + ] + ), + id_namespace=run_ids, + ) + control_trace_ids.append(stop_control_trace_id) + control_data_ids.append(stop_control_data_id) + final_control_data_id = stop_control_data_id + final_control_decision = "stop_success" if code_stop_reason == "completed" else "stop_failed" + break + if selected_tool_name == "read_artifact": artifact_result = tool_runner.run( tool_name="read_artifact", @@ -1122,6 +1276,8 @@ def record_budget( run_data_id, l1_goal_data_id, *budget_plan_data_ids, + tool_scope_data_id, + tool_budget_partition_data_id, catalog_id, *query_plan_data_ids, l2_query_data_id, @@ -1205,7 +1361,9 @@ def record_budget( turn_id=turn_id, revision_input_data_id=revision_input_id, adapter=l2_query_planner_adapter, - available_tools=available_tools, + available_tools=scoped_available_tools, + l_tool_scope=asdict(tool_scope_frame), + budget_partition=asdict(tool_budget_partition_frame), id_namespace=run_ids, ) except Exception: @@ -1277,6 +1435,7 @@ def record_budget( revision_tool_source_data_ids=revision_tool_result.source_data_ids, user_query=search_query, l1_goal_data_id=l1_goal_data_id, + adapter=l3_result_adapter, id_namespace=run_ids, ) revision_trace_ids.append(revision_l3_event.event_id) @@ -1296,12 +1455,56 @@ def record_budget( l3 = revision_l3_event continuation_attempt_index += 1 + pack_trace_id, pack_data_id, _ = record_document_context_pack_frame( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + document_root=document_root, + max_document_context_chars=max_document_context_chars, + frame_id=document_context_pack_frame_id(id_namespace=run_ids), + explicit_reference_data_id=explicit_data_id, + source_trace_ids=_unique_strings( + [ + explicit_trace_id, + l3.event_id, + *tool_call_trace_ids, + *distillation_trace_ids, + *revision_trace_ids, + ] + ), + source_data_ids=_unique_strings( + [ + explicit_data_id, + l2_query_data_id, + *revision_query_data_ids, + *tool_result_data_ids, + *tool_distillation_data_ids, + l3_preserved_data_id, + *revision_preserved_data_ids, + ] + ), + id_namespace=run_ids, + ) + document_context_trace_ids.append(pack_trace_id) + document_context_pack_data_ids = [pack_data_id] + l3_per_document_summary_data_ids = _l3_per_document_summary_data_ids( + data_store, + id_namespace=run_ids, + ) + l3_per_document_summary_trace_ids = _source_trace_ids_for_data_ids( + data_store=data_store, + data_ids=l3_per_document_summary_data_ids, + ) + source_trace_ids = _unique_strings( [ run_trace_id, l1.event_id, *budget_plan_trace_ids, + tool_scope_trace_id, + tool_budget_partition_trace_id, tool_catalog_trace_id, + *document_context_trace_ids, *l2_plan_trace_ids, l2.event_id, *tool_choice_trace_ids, @@ -1312,6 +1515,7 @@ def record_budget( *continuation_trace_ids, *revision_trace_ids, *failure_trace_ids, + *l3_per_document_summary_trace_ids, l3.event_id, ] ) @@ -1320,7 +1524,10 @@ def record_budget( run_data_id, l1_goal_data_id, *budget_plan_data_ids, + tool_scope_data_id, + tool_budget_partition_data_id, catalog_id, + *explicit_artifact_reference_data_ids, *query_plan_data_ids, l2_query_data_id, *tool_choice_ids, @@ -1337,6 +1544,8 @@ def record_budget( l3_achievement_data_id, *revision_preserved_data_ids, *revision_achievement_data_ids, + *l3_per_document_summary_data_ids, + *document_context_pack_data_ids, ] ) return LLoopResult( @@ -1356,9 +1565,12 @@ def record_budget( continuation_trace_ids=continuation_trace_ids, revision_trace_ids=revision_trace_ids, failure_trace_ids=failure_trace_ids, + document_context_trace_ids=document_context_trace_ids, run_data_ids=[run_data_id], goal_data_ids=[l1_goal_data_id], budget_plan_data_ids=budget_plan_data_ids, + tool_scope_data_ids=[tool_scope_data_id], + tool_budget_partition_data_ids=[tool_budget_partition_data_id], tool_catalog_data_ids=[catalog_id], tool_choice_data_ids=tool_choice_ids, query_plan_data_ids=query_plan_data_ids, @@ -1372,6 +1584,8 @@ def record_budget( revision_query_plan_data_ids=revision_query_plan_data_ids, revision_query_data_ids=revision_query_data_ids, failure_signal_data_ids=failure_signal_data_ids, + explicit_artifact_reference_data_ids=explicit_artifact_reference_data_ids, + document_context_pack_data_ids=document_context_pack_data_ids, preserved_data_ids=[l3_preserved_data_id, *revision_preserved_data_ids], achievement_data_ids=[l3_achievement_data_id, *revision_achievement_data_ids], output_data_ids=output_data_ids, @@ -1556,6 +1770,109 @@ def _control_condition_flags( return flags +def _initial_control_decision_for_tool(tool_name: str) -> str: + if tool_name == "read_artifact": + return "continue_read_artifact" + if tool_name == "list_code_files": + return "list_code_files" + if tool_name == "search_code": + return "continue_code_search" + if tool_name == "read_code_file": + return "read_code_file" + return "continue_search" + + +def _initial_control_reason_for_tool(tool_name: str) -> str: + if tool_name == "read_artifact": + return "CODE_STATUS:continue_read_artifact" + if tool_name == "list_code_files": + return "CODE_STATUS:list_code_files" + if tool_name == "search_code": + return "CODE_STATUS:continue_search_code" + if tool_name == "read_code_file": + return "CODE_STATUS:read_code_file" + return "CODE_STATUS:continue_search_docs" + + +def _is_code_inspection_tool(tool_name: str) -> bool: + return tool_name in {"list_code_files", "search_code", "read_code_file"} + + +def _fallback_l2_tool_for_available_tools(available_tools: list[dict[str, object]]) -> str: + available_names = { + str(tool.get("tool_name") or tool.get("name")) + for tool in available_tools + if isinstance(tool, dict) and isinstance(tool.get("tool_name") or tool.get("name"), str) + } + for tool_name in ( + "search_docs", + "read_artifact", + "search_code", + "list_code_files", + "read_code_file", + ): + if tool_name in available_names: + return tool_name + return "search_docs" + + +def _run_code_inspection_tool( + *, + tool_runner: ToolRunner, + selected_tool_name: str, + trace_store: TraceStore, + data_store: DataStore, + turn_id: str, + input_ref: list[str], + id_namespace: LRunIds, + query_text: str, +) -> ToolRunResult: + if selected_tool_name == "list_code_files": + return tool_runner.run( + tool_name="list_code_files", + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + input_ref=input_ref, + id_namespace=id_namespace, + ) + if selected_tool_name == "search_code": + return tool_runner.run( + tool_name="search_code", + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + input_ref=input_ref, + id_namespace=id_namespace, + query=query_text, + ) + if selected_tool_name == "read_code_file": + return tool_runner.run( + tool_name="read_code_file", + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + input_ref=input_ref, + id_namespace=id_namespace, + file_path=query_text, + ) + raise ValueError(f"unsupported code inspection tool: {selected_tool_name}") + + +def _code_inspection_stop_reason(*, tool_name: str, payload: object) -> str: + if not isinstance(payload, dict): + return "low_yield_stop" + if tool_name == "list_code_files": + count = payload.get("returned_file_count") + return "completed" if isinstance(count, int) and count > 0 else "low_yield_stop" + if tool_name == "search_code": + count = payload.get("returned_match_count") + return "completed" if isinstance(count, int) and count > 0 else "low_yield_stop" + if tool_name == "read_code_file": + return "completed" if payload.get("read_status") == "ok" else "low_yield_stop" + return "low_yield_stop" + + def _read_query_text_from_l2_frame(payload: object) -> str: """DataStore에 저장된 L2 query frame payload에서 query_text를 꺼낸다.""" @@ -1616,6 +1933,34 @@ def _refine_search_query(query: str, *, attempt_count: int) -> str: return f"{query}{suffix}" +def _l3_per_document_summary_data_ids( + data_store: DataStore, + *, + id_namespace: LRunIds, +) -> list[str]: + return _unique_strings( + [ + record.data_id + for record in data_store.list_records() + if record.data_type == "node_output:L3_per_document_summary_frame" + and id_namespace.owns_data_id(record.data_id) + ] + ) + + +def _source_trace_ids_for_data_ids( + *, + data_store: DataStore, + data_ids: list[str], +) -> list[str]: + trace_ids: list[str] = [] + for data_id in data_ids: + record = data_store.get_record(data_id) + if record is not None and record.source_trace_id: + trace_ids.append(record.source_trace_id) + return _unique_strings(trace_ids) + + def _unique_strings(values: list[str | None]) -> list[str]: """None과 중복을 제거하되 처음 등장한 순서를 보존한다.""" diff --git a/songryeon_core/loops/l_loop_activity_ledger.py b/songryeon_core/loops/l_loop_activity_ledger.py new file mode 100644 index 0000000..c06a246 --- /dev/null +++ b/songryeon_core/loops/l_loop_activity_ledger.py @@ -0,0 +1,270 @@ +from __future__ import annotations + +from dataclasses import asdict + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.graph_memory import raw_capsule_graph_node_id +from songryeon_core.core.schemas import ( + L_LOOP_ACTIVITY_LEDGER_DATA_TYPE, + LLoopActivityLedgerFrame, + validate_l_loop_activity_ledger_frame, +) +from songryeon_core.core.trace_store import TraceStore +from songryeon_core.loops.l_loop import LLoopResult +from songryeon_core.loops.l_loop_namespace import LRunIds + + +def record_l_loop_activity_ledger( + *, + trace_store: TraceStore, + data_store: DataStore, + turn_id: str, + l_result: LLoopResult, + return_summary_frame_id: str | None = None, + document_material_packet_frame_id: str | None = None, + id_namespace: LRunIds | None = None, +) -> tuple[str, str, LLoopActivityLedgerFrame]: + """Record one absolute ledger for the DataStore records touched by an L run.""" + + return_summary_payload = _payload_for(data_store, return_summary_frame_id) + document_material_payload = _payload_for(data_store, document_material_packet_frame_id) + run_index = id_namespace.run_index if id_namespace is not None else _run_index_from_result( + data_store, + l_result, + ) + frame_id = ( + id_namespace.activity_ledger_frame_id() + if id_namespace is not None + else "L:activity_ledger_frame" + ) + output_data_ids = _unique_strings( + [ + *l_result.output_data_ids, + return_summary_frame_id, + document_material_packet_frame_id, + ] + ) + source_trace_ids = _unique_strings( + [ + *l_result.source_trace_ids, + *_string_list(return_summary_payload.get("source_trace_ids")), + *_string_list(document_material_payload.get("source_trace_ids")), + ] + ) + source_data_ids = _unique_strings( + [ + *output_data_ids, + *_string_list(return_summary_payload.get("source_data_ids")), + *_string_list(document_material_payload.get("source_data_ids")), + ] + ) + read_doc_ids = _read_doc_ids(return_summary_payload, document_material_payload) + search_candidate_doc_ids = _search_candidate_doc_ids( + return_summary_payload, + document_material_payload, + ) + read_code_file_paths = _unique_strings( + _string_list(return_summary_payload.get("read_code_file_paths")) + ) + material_items = document_material_payload.get("items") + document_material_item_count = len(material_items) if isinstance(material_items, list) else 0 + + frame = LLoopActivityLedgerFrame( + frame_id=frame_id, + turn_id=turn_id, + run_index=run_index, + turn_capsule_graph_node_id=raw_capsule_graph_node_id(turn_id), + run_frame_data_ids=list(l_result.run_data_ids), + goal_data_ids=list(l_result.goal_data_ids), + budget_plan_data_ids=list(l_result.budget_plan_data_ids), + tool_scope_data_ids=list(l_result.tool_scope_data_ids), + tool_budget_partition_data_ids=list(l_result.tool_budget_partition_data_ids), + tool_catalog_data_ids=list(l_result.tool_catalog_data_ids), + tool_choice_data_ids=list(l_result.tool_choice_data_ids), + query_plan_data_ids=list(l_result.query_plan_data_ids), + query_data_ids=list(l_result.query_data_ids), + control_data_ids=list(l_result.control_data_ids), + tool_result_data_ids=list(l_result.tool_result_data_ids), + tool_distillation_data_ids=list(l_result.tool_distillation_data_ids), + tool_budget_data_ids=list(l_result.tool_budget_data_ids), + continuation_data_ids=list(l_result.continuation_data_ids), + revision_input_data_ids=list(l_result.revision_input_data_ids), + revision_query_plan_data_ids=list(l_result.revision_query_plan_data_ids), + revision_query_data_ids=list(l_result.revision_query_data_ids), + failure_signal_data_ids=list(l_result.failure_signal_data_ids), + explicit_artifact_reference_data_ids=list(l_result.explicit_artifact_reference_data_ids), + document_context_pack_data_ids=list(l_result.document_context_pack_data_ids), + preserved_data_ids=list(l_result.preserved_data_ids), + achievement_data_ids=list(l_result.achievement_data_ids), + return_summary_frame_id=return_summary_frame_id, + document_material_packet_frame_id=document_material_packet_frame_id, + output_data_ids=output_data_ids, + search_candidate_doc_ids=search_candidate_doc_ids, + read_doc_ids=read_doc_ids, + read_code_file_paths=read_code_file_paths, + output_data_id_count=len(output_data_ids), + tool_result_count=len(l_result.tool_result_data_ids), + search_candidate_doc_count=len(search_candidate_doc_ids), + actual_read_doc_count=len(read_doc_ids), + actual_read_code_file_count=len(read_code_file_paths), + document_material_item_count=document_material_item_count, + activity_records=_build_activity_records( + l_result=l_result, + return_summary_frame_id=return_summary_frame_id, + document_material_packet_frame_id=document_material_packet_frame_id, + ), + source_trace_ids=source_trace_ids, + source_data_ids=source_data_ids, + ) + validate_l_loop_activity_ledger_frame(frame) + + event = trace_store.create_event( + turn_id=turn_id, + actor="node_0", + event_type="node_output", + input_ref=source_trace_ids, + output_ref=[frame_id], + schema_status="passed", + ) + data_store.create_record( + data_id=frame_id, + data_type=L_LOOP_ACTIVITY_LEDGER_DATA_TYPE, + exists=True, + created_at=event.timestamp, + source_trace_id=event.event_id, + payload=asdict(frame), + ) + return event.event_id, frame_id, frame + + +def _build_activity_records( + *, + l_result: LLoopResult, + return_summary_frame_id: str | None, + document_material_packet_frame_id: str | None, +) -> list[dict[str, str]]: + records: list[dict[str, str]] = [] + field_map = { + "run_frame": l_result.run_data_ids, + "goal": l_result.goal_data_ids, + "budget_plan": l_result.budget_plan_data_ids, + "tool_scope": l_result.tool_scope_data_ids, + "tool_budget_partition": l_result.tool_budget_partition_data_ids, + "tool_catalog": l_result.tool_catalog_data_ids, + "tool_choice": l_result.tool_choice_data_ids, + "query_plan": l_result.query_plan_data_ids, + "query": l_result.query_data_ids, + "control": l_result.control_data_ids, + "tool_result": l_result.tool_result_data_ids, + "tool_distillation": l_result.tool_distillation_data_ids, + "tool_budget": l_result.tool_budget_data_ids, + "continuation": l_result.continuation_data_ids, + "revision_input": l_result.revision_input_data_ids, + "revision_query_plan": l_result.revision_query_plan_data_ids, + "revision_query": l_result.revision_query_data_ids, + "failure_signal": l_result.failure_signal_data_ids, + "explicit_artifact_reference": l_result.explicit_artifact_reference_data_ids, + "document_context_pack": l_result.document_context_pack_data_ids, + "l3_preserved": l_result.preserved_data_ids, + "l3_achievement": l_result.achievement_data_ids, + } + for stage, data_ids in field_map.items(): + for data_id in data_ids: + records.append( + { + "stage": stage, + "data_id": data_id, + "source_field": f"{stage}_data_ids", + } + ) + if return_summary_frame_id: + records.append( + { + "stage": "return_summary", + "data_id": return_summary_frame_id, + "source_field": "return_summary_frame_id", + } + ) + if document_material_packet_frame_id: + records.append( + { + "stage": "document_material_packet", + "data_id": document_material_packet_frame_id, + "source_field": "document_material_packet_frame_id", + } + ) + return records + + +def _payload_for(data_store: DataStore, data_id: str | None) -> dict[str, object]: + if not data_id: + return {} + record = data_store.get_record(data_id) + if record is None or not isinstance(record.payload, dict): + return {} + return record.payload + + +def _run_index_from_result(data_store: DataStore, l_result: LLoopResult) -> int: + for data_id in l_result.run_data_ids: + payload = _payload_for(data_store, data_id) + run_index = payload.get("run_index") + if isinstance(run_index, int) and run_index > 0: + return run_index + return 1 + + +def _read_doc_ids( + return_summary_payload: dict[str, object], + document_material_payload: dict[str, object], +) -> list[str]: + read_doc_ids = _unique_strings(_string_list(return_summary_payload.get("read_doc_ids"))) + if read_doc_ids: + return read_doc_ids + return _doc_ids_from_material_items(document_material_payload, role="actual_tool_read_doc") + + +def _search_candidate_doc_ids( + return_summary_payload: dict[str, object], + document_material_payload: dict[str, object], +) -> list[str]: + search_doc_ids = _unique_strings( + _string_list(return_summary_payload.get("search_result_doc_ids")) + ) + if search_doc_ids: + return search_doc_ids + return _doc_ids_from_material_items(document_material_payload, role="search_candidate") + + +def _doc_ids_from_material_items(payload: dict[str, object], *, role: str) -> list[str]: + items = payload.get("items") + if not isinstance(items, list): + return [] + doc_ids: list[str] = [] + role_flag = f"was_{role}" + for item in items: + if not isinstance(item, dict): + continue + if item.get(role_flag) is True: + doc_ids.append(str(item.get("doc_id") or "")) + return _unique_strings(doc_ids) + + +def _string_list(value: object) -> list[str]: + if not isinstance(value, list): + return [] + return [item for item in value if isinstance(item, str) and item] + + +def _unique_strings(values: list[str | None]) -> list[str]: + seen: set[str] = set() + result: list[str] = [] + for value in values: + if value is None or value in seen: + continue + seen.add(value) + result.append(value) + return result + + +__all__ = ["record_l_loop_activity_ledger"] diff --git a/songryeon_core/loops/l_loop_continuation.py b/songryeon_core/loops/l_loop_continuation.py index 639a97f..8072a25 100644 --- a/songryeon_core/loops/l_loop_continuation.py +++ b/songryeon_core/loops/l_loop_continuation.py @@ -164,9 +164,25 @@ def _decide_continuation( "loop_return_summary", ) - has_query_budget = remaining_query_attempts > 0 has_read_budget_for_unread_candidates = remaining_read_doc_calls > 0 and bool(unread_candidate_doc_ids) - if not has_query_budget and not has_read_budget_for_unread_candidates: + + # ORDER_122 이후 revision L2는 새 search_docs query 없이도 기존 unread candidate를 + # read_doc으로 고를 수 있다. 따라서 query budget이 0이어도 읽을 후보와 read budget이 + # 있으면 continuation을 닫지 않는다. + if remaining_query_attempts <= 0: + if has_read_budget_for_unread_candidates: + return ( + "continue", + "CODE_STATUS:l3_not_achieved_read_unread_candidate_after_query_budget", + "L2", + ) + return ( + "stop_budget_exhausted", + "CODE_STATUS:continuation_query_budget_exhausted", + "loop_return_summary", + ) + + if not has_read_budget_for_unread_candidates and remaining_tool_calls <= 0: return ( "stop_no_actionable_gap", "CODE_STATUS:no_unread_candidate_or_revision_plan", diff --git a/songryeon_core/loops/l_loop_namespace.py b/songryeon_core/loops/l_loop_namespace.py index 2ebcbed..12c3b6e 100644 --- a/songryeon_core/loops/l_loop_namespace.py +++ b/songryeon_core/loops/l_loop_namespace.py @@ -124,6 +124,9 @@ def tool_distillation_data_id(self, *, tool_name: str, original_trace_id: str) - def return_summary_frame_id(self) -> str: return self.scoped_data_id("L:return_summary_frame") + def activity_ledger_frame_id(self) -> str: + return self.scoped_data_id("L:activity_ledger_frame") + def loop_return_memory_packet_id(self) -> str: return self.scoped_data_id("memory_packet:node_1:loop_return_summary") @@ -154,6 +157,9 @@ def route2_handoff_frame_id(self) -> str: def node3_input_brief_frame_id(self) -> str: return self.scoped_data_id("node_3:input_brief_frame") + def node2_answer_basis_frame_id(self) -> str: + return self.scoped_data_id("node_2:answer_basis_frame") + def node3_report_id(self) -> str: return self.scoped_data_id("report_dry_001") diff --git a/songryeon_core/loops/l_loop_revision_tool_attempt.py b/songryeon_core/loops/l_loop_revision_tool_attempt.py index f8b7fb8..222ede1 100644 --- a/songryeon_core/loops/l_loop_revision_tool_attempt.py +++ b/songryeon_core/loops/l_loop_revision_tool_attempt.py @@ -69,7 +69,14 @@ def run_l_loop_revision_tool_attempt( attempt_index = _attempt_index_from_revision_query_frame_id(revision_query_frame_data_id) query_text = _required_text(query_payload, "query_text") tool_name = _required_text(query_payload, "target_tool_name") - if tool_name not in {"search_docs", "read_artifact"}: + if tool_name not in { + "search_docs", + "read_artifact", + "read_doc", + "list_code_files", + "search_code", + "read_code_file", + }: raise ValueError(f"unsupported revision tool: {tool_name}") registry = build_document_tool_registry(document_root) @@ -117,6 +124,45 @@ def run_l_loop_revision_tool_attempt( query=query_text, top_k=search_top_k, ) + elif tool_name == "read_doc": + tool_result = runner.run( + tool_name="read_doc", + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + input_ref=_unique_strings([query_record.source_trace_id, choice_trace_id]), + id_namespace=id_namespace, + doc_id=query_text, + ) + elif tool_name == "list_code_files": + tool_result = runner.run( + tool_name="list_code_files", + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + input_ref=_unique_strings([query_record.source_trace_id, choice_trace_id]), + id_namespace=id_namespace, + ) + elif tool_name == "search_code": + tool_result = runner.run( + tool_name="search_code", + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + input_ref=_unique_strings([query_record.source_trace_id, choice_trace_id]), + id_namespace=id_namespace, + query=query_text, + ) + elif tool_name == "read_code_file": + tool_result = runner.run( + tool_name="read_code_file", + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + input_ref=_unique_strings([query_record.source_trace_id, choice_trace_id]), + id_namespace=id_namespace, + file_path=query_text, + ) else: tool_result = runner.run( tool_name="read_artifact", @@ -142,7 +188,7 @@ def run_l_loop_revision_tool_attempt( ) latest_budget_payload = _require_dict_payload(latest_budget) if latest_budget is not None else {} executed_queries = _string_list(latest_budget_payload.get("executed_queries")) - if query_text not in executed_queries: + if tool_name in {"search_docs", "search_code"} and query_text not in executed_queries: executed_queries.append(query_text) read_doc_ids = _string_list(latest_budget_payload.get("read_doc_ids")) read_doc_id = _read_doc_id_from_tool_payload(tool_name=tool_name, payload=tool_result.payload) @@ -336,6 +382,9 @@ def _next_budget_sequence_index( def _read_doc_id_from_tool_payload(*, tool_name: str, payload: object) -> str | None: if not isinstance(payload, dict): return None + if tool_name == "read_doc": + doc_id = payload.get("doc_id") + return doc_id if isinstance(doc_id, str) and doc_id else None if tool_name == "read_artifact" and payload.get("match_status") == "unique": doc_id = payload.get("doc_id") or payload.get("selected_doc_id") return doc_id if isinstance(doc_id, str) and doc_id else None @@ -350,6 +399,17 @@ def _tool_attempt_stop_reason(*, tool_name: str, payload: object) -> str: return "completed" if isinstance(result_count, int) and result_count > 0 else "low_yield_stop" if tool_name == "read_artifact": return "completed" if payload.get("match_status") == "unique" else "low_yield_stop" + if tool_name == "read_doc": + text = payload.get("text") + return "completed" if isinstance(text, str) and text else "low_yield_stop" + if tool_name == "list_code_files": + count = payload.get("returned_file_count") + return "completed" if isinstance(count, int) and count > 0 else "low_yield_stop" + if tool_name == "search_code": + count = payload.get("returned_match_count") + return "completed" if isinstance(count, int) and count > 0 else "low_yield_stop" + if tool_name == "read_code_file": + return "completed" if payload.get("read_status") == "ok" else "low_yield_stop" return "low_yield_stop" diff --git a/songryeon_core/loops/r_loop_dry_run.py b/songryeon_core/loops/r_loop_dry_run.py new file mode 100644 index 0000000..1f7b0f8 --- /dev/null +++ b/songryeon_core/loops/r_loop_dry_run.py @@ -0,0 +1,843 @@ +from __future__ import annotations + +from dataclasses import asdict, dataclass + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.graph_memory import raw_capsule_graph_node_id +from songryeon_core.core.r_loop_state_machine import decide_r_loop_continuation +from songryeon_core.core.schemas import ( + R1GraphGoalFrame, + R2GraphNodeSelectionFrame, + R3GraphInspectionFrame, + RGraphTraversalCandidateSurfaceFrame, + RLoopBudgetFrame, + RLoopContinuationFrame, + RLoopMemoryHandoffPacketFrame, + RLoopReturnSummaryFrame, + TurnGraphAccessLedgerFrame, + validate_r1_graph_goal_frame, + validate_r2_graph_node_selection_frame, + validate_r3_graph_inspection_frame, + validate_r_graph_traversal_candidate_surface_frame, + validate_turn_graph_access_ledger_frame, + validate_r_loop_budget_frame, + validate_r_loop_continuation_frame, + validate_r_loop_memory_handoff_packet_frame, + validate_r_loop_return_summary_frame, +) +from songryeon_core.core.trace_store import TraceStore + + +R_DRY_RUN_GENERATOR = "CODE:R_LOOP_DRY_RUN_ONLY" +R_EXPERIMENTAL_ROUTE_GENERATOR = "CODE:R_ROUTE_EXPERIMENTAL_GATE" + + +@dataclass +class RLoopDryRunResult: + r1_goal: R1GraphGoalFrame + budget: RLoopBudgetFrame + r2_selection: R2GraphNodeSelectionFrame + r3_inspection: R3GraphInspectionFrame + candidate_surface: RGraphTraversalCandidateSurfaceFrame + continuation: RLoopContinuationFrame + return_summary: RLoopReturnSummaryFrame + access_ledger: TurnGraphAccessLedgerFrame + budgets: list[RLoopBudgetFrame] + r2_selections: list[R2GraphNodeSelectionFrame] + r3_inspections: list[R3GraphInspectionFrame] + candidate_surfaces: list[RGraphTraversalCandidateSurfaceFrame] + continuations: list[RLoopContinuationFrame] + trace_event_ids: list[str] + output_data_ids: list[str] + + +def run_r_loop_dry_run_skeleton( + *, + trace_store: TraceStore, + data_store: DataStore, + turn_id: str, + handoff_packet: RLoopMemoryHandoffPacketFrame, + input_ref: list[str] | None = None, + force_budget_exhausted: bool = False, + frame_label: str = "dry_run", + generated_by: str = R_DRY_RUN_GENERATOR, + graph_node_payloads: dict[str, dict[str, object]] | None = None, + graph_edge_payloads: list[dict[str, object]] | None = None, +) -> RLoopDryRunResult: + """Run a deterministic R-loop skeleton for dry-run tests only.""" + + validate_r_loop_memory_handoff_packet_frame(handoff_packet) + if handoff_packet.packet_status != "available": + raise ValueError("R dry-run skeleton requires an available graph guide handoff") + + frame_label = _safe_frame_label(frame_label) + r1 = _build_r1_goal( + handoff_packet=handoff_packet, + frame_label=frame_label, + generated_by=generated_by, + ) + budgets: list[RLoopBudgetFrame] = [] + r2_selections: list[R2GraphNodeSelectionFrame] = [] + r3_inspections: list[R3GraphInspectionFrame] = [] + candidate_surfaces: list[RGraphTraversalCandidateSurfaceFrame] = [] + continuations: list[RLoopContinuationFrame] = [] + + available_graph_node_ids = list(handoff_packet.available_entry_node_ids) + previous_candidate_surface: RGraphTraversalCandidateSurfaceFrame | None = None + for step_index in range(1, r1.max_node_reads + 1): + if not available_graph_node_ids: + break + budget = _build_budget( + r1=r1, + force_budget_exhausted=force_budget_exhausted and step_index == 1, + frame_label=frame_label, + generated_by=generated_by, + step_index=step_index, + ) + r2 = _build_r2_selection( + handoff_packet=handoff_packet, + r1=r1, + available_graph_node_ids=available_graph_node_ids, + previous_candidate_surface=previous_candidate_surface, + frame_label=frame_label, + generated_by=generated_by, + step_index=step_index, + ) + r3 = _build_r3_inspection( + data_store=data_store, + handoff_packet=handoff_packet, + r2=r2, + frame_label=frame_label, + generated_by=generated_by, + step_index=step_index, + graph_node_payloads=graph_node_payloads, + ) + candidate_surface = _build_candidate_surface( + data_store=data_store, + r3=r3, + frame_label=frame_label, + step_index=step_index, + graph_edge_payloads=graph_edge_payloads, + ) + continuation = decide_r_loop_continuation( + frame_id=f"R:{frame_label}:continuation_frame:{step_index:04d}", + r3_inspection=r3, + budget=budget, + source_trace_ids=input_ref or [], + ) + continuation.generated_by = generated_by + validate_r_loop_continuation_frame(continuation) + + budgets.append(budget) + r2_selections.append(r2) + r3_inspections.append(r3) + candidate_surfaces.append(candidate_surface) + continuations.append(continuation) + + if continuation.continuation_status not in { + "continue_deeper", + "continue_switch_branch", + }: + break + if continuation.next_target_node != "R2": + break + available_graph_node_ids = list(candidate_surface.candidate_graph_node_ids) + previous_candidate_surface = candidate_surface + + if not budgets or not r2_selections or not r3_inspections or not candidate_surfaces or not continuations: + raise ValueError("R dry-run skeleton did not produce traversal frames") + + budget = budgets[-1] + r2 = r2_selections[-1] + r3 = r3_inspections[-1] + candidate_surface = candidate_surfaces[-1] + continuation = continuations[-1] + summary = _build_return_summary( + handoff_packet=handoff_packet, + r1=r1, + budgets=budgets, + r2_selections=r2_selections, + r3_inspections=r3_inspections, + candidate_surfaces=candidate_surfaces, + continuation=continuation, + frame_label=frame_label, + generated_by=generated_by, + ) + access_ledger = _build_access_ledger( + turn_id=turn_id, + handoff_packet=handoff_packet, + r1=r1, + budgets=budgets, + r2_selections=r2_selections, + r3_inspections=r3_inspections, + candidate_surfaces=candidate_surfaces, + continuation=continuation, + summary=summary, + frame_label=frame_label, + ) + + frames: list[tuple[str, str, object]] = [("R1", "node_output:R1_graph_goal_frame", r1)] + for step_budget, step_r2, step_r3, step_surface, step_continuation in zip( + budgets, + r2_selections, + r3_inspections, + candidate_surfaces, + continuations, + ): + frames.extend( + [ + ("R:budget", "node_output:R_loop_budget_frame", step_budget), + ("R2", "node_output:R2_graph_node_selection_frame", step_r2), + ("R3", "node_output:R3_graph_inspection_frame", step_r3), + ( + "R:candidate_surface", + "node_output:R_graph_traversal_candidate_surface_frame", + step_surface, + ), + ("R:continuation", "node_output:R_loop_continuation_frame", step_continuation), + ] + ) + frames.extend( + [ + ("R:return_summary", "node_output:R_loop_return_summary_frame", summary), + ("R:access_ledger", "graph_memory:turn_access_ledger_frame", access_ledger), + ] + ) + trace_event_ids: list[str] = [] + output_data_ids: list[str] = [] + previous_trace_ids = list(input_ref or []) + for actor, data_type, frame in frames: + frame_payload = asdict(frame) + frame_id = str(frame_payload["frame_id"]) + event = trace_store.create_event( + turn_id=turn_id, + actor=actor, + event_type="node_output", + input_ref=previous_trace_ids, + output_ref=[frame_id], + schema_status="passed", + ) + data_store.create_record( + data_id=frame_id, + data_type=data_type, + exists=True, + created_at=event.timestamp, + source_trace_id=event.event_id, + payload=frame_payload, + ) + trace_event_ids.append(event.event_id) + output_data_ids.append(frame_id) + previous_trace_ids = [event.event_id] + + return RLoopDryRunResult( + r1_goal=r1, + budget=budget, + r2_selection=r2, + r3_inspection=r3, + candidate_surface=candidate_surface, + continuation=continuation, + return_summary=summary, + access_ledger=access_ledger, + budgets=budgets, + r2_selections=r2_selections, + r3_inspections=r3_inspections, + candidate_surfaces=candidate_surfaces, + continuations=continuations, + trace_event_ids=trace_event_ids, + output_data_ids=output_data_ids, + ) + + +def _build_r1_goal( + *, + handoff_packet: RLoopMemoryHandoffPacketFrame, + frame_label: str, + generated_by: str, +) -> R1GraphGoalFrame: + frame = R1GraphGoalFrame( + frame_id=f"R1:{frame_label}:graph_goal_frame", + graph_search_goal=f"CODE_STATUS:r_route_{frame_label}_inspect_graph_guide_handoff", + required_information_granularity="unknown", + allowed_summary_depth=max(handoff_packet.summary_depth_range), + max_traversal_depth=2, + max_branch_switches=1, + max_node_reads=3, + max_context_tokens=1200, + stop_condition=f"CODE_STATUS:r_route_{frame_label}_stop_after_first_continuation_check", + source_graph_guide_packet_id=handoff_packet.r_loop_graph_guide_packet_id, + source_data_ids=_unique_strings( + [ + handoff_packet.packet_id, + handoff_packet.r_loop_graph_guide_packet_id, + handoff_packet.graph_snapshot_id, + ] + ), + source_trace_ids=list(handoff_packet.source_trace_ids), + generated_by=generated_by, + info_class="mixed", + semantic_judgement_status="not_run", + ) + validate_r1_graph_goal_frame(frame) + return frame + + +def _build_budget( + *, + r1: R1GraphGoalFrame, + force_budget_exhausted: bool, + frame_label: str, + generated_by: str, + step_index: int, +) -> RLoopBudgetFrame: + used_node_reads = r1.max_node_reads if force_budget_exhausted else step_index + used_traversal_depth = ( + r1.max_traversal_depth + if force_budget_exhausted + else max(step_index - 1, 0) + ) + used_context_tokens = ( + r1.max_context_tokens + if force_budget_exhausted + else min(240 * step_index, r1.max_context_tokens) + ) + frame = RLoopBudgetFrame( + frame_id=f"R:{frame_label}:budget_frame:{step_index:04d}", + source_r1_goal_frame_id=r1.frame_id, + max_traversal_depth=r1.max_traversal_depth, + max_branch_switches=r1.max_branch_switches, + max_node_reads=r1.max_node_reads, + max_context_tokens=r1.max_context_tokens, + used_traversal_depth=used_traversal_depth, + used_branch_switches=0, + used_node_reads=used_node_reads, + used_context_tokens=used_context_tokens, + budget_status="exhausted" if force_budget_exhausted else "within_budget", + source_data_ids=[r1.frame_id], + source_trace_ids=list(r1.source_trace_ids), + generated_by=generated_by, + info_class="absolute", + semantic_judgement_status="not_run", + ) + validate_r_loop_budget_frame(frame) + return frame + + +def _build_r2_selection( + *, + handoff_packet: RLoopMemoryHandoffPacketFrame, + r1: R1GraphGoalFrame, + available_graph_node_ids: list[str], + previous_candidate_surface: RGraphTraversalCandidateSurfaceFrame | None, + frame_label: str, + generated_by: str, + step_index: int, +) -> R2GraphNodeSelectionFrame: + if not available_graph_node_ids: + raise ValueError("R2 dry-run selection requires available graph node ids") + selected_id = available_graph_node_ids[0] + if previous_candidate_surface is None: + selection_scope = "core_ego_graph_guide_handoff" + expected_source_kind = "graph_entry_node" + source_data_ids = [r1.frame_id, handoff_packet.packet_id] + source_trace_ids = list(handoff_packet.source_trace_ids) + reason = f"CODE_STATUS:{frame_label}_select_first_available_entry_node" + else: + selection_scope = "r_graph_traversal_candidate_surface" + expected_source_kind = "graph_candidate_node" + source_data_ids = [ + r1.frame_id, + previous_candidate_surface.frame_id, + *available_graph_node_ids, + ] + source_trace_ids = list(previous_candidate_surface.source_trace_ids) + reason = f"CODE_STATUS:{frame_label}_select_first_candidate_surface_coordinate" + frame = R2GraphNodeSelectionFrame( + frame_id=f"R2:{frame_label}:graph_node_selection_frame:{step_index:04d}", + selection_scope=selection_scope, + available_graph_node_ids=list(available_graph_node_ids), + selection_status="selected", + selected_graph_node_id=selected_id, + selection_reason=reason, + expected_information_granularity="unknown", + expected_source_kind=expected_source_kind, + source_r1_goal_frame_id=r1.frame_id, + source_data_ids=_unique_strings(source_data_ids), + source_trace_ids=_unique_strings(source_trace_ids), + generated_by=generated_by, + info_class="mixed", + semantic_judgement_status="not_run", + ) + validate_r2_graph_node_selection_frame(frame) + return frame + + +def _build_r3_inspection( + *, + data_store: DataStore, + handoff_packet: RLoopMemoryHandoffPacketFrame, + r2: R2GraphNodeSelectionFrame, + frame_label: str, + generated_by: str, + step_index: int, + graph_node_payloads: dict[str, dict[str, object]] | None = None, +) -> R3GraphInspectionFrame: + selected_id = r2.selected_graph_node_id + if selected_id is None: + raise ValueError("R dry-run R2 selection did not select a graph node") + node_payload = _graph_node_payload( + data_store=data_store, + node_id=selected_id, + graph_node_payloads=graph_node_payloads, + ) + if node_payload: + child_node_ids = _string_list(node_payload.get("source_graph_node_ids")) + node_kind = _text(node_payload.get("node_kind"), fallback="time_axis") + summary_depth = _int(node_payload.get("summary_depth")) + source_leaf_count = _int(node_payload.get("source_leaf_count")) + else: + child_node_ids = list(handoff_packet.source_graph_node_ids) + node_kind = "time_axis" + summary_depth = min(handoff_packet.summary_depth_range) + source_leaf_count = max(handoff_packet.source_leaf_count_range) + recommended_next_action = "deeper" if child_node_ids else "stop" + granularity_status = "needs_lower_granularity" if child_node_ids else "none" + sufficiency_status = "insufficient" if child_node_ids else "sufficient" + frame = R3GraphInspectionFrame( + frame_id=f"R3:{frame_label}:graph_inspection_frame:{step_index:04d}", + inspected_graph_node_id=selected_id, + node_kind=node_kind, + child_node_count=len(child_node_ids), + child_node_ids=child_node_ids, + summary_depth=summary_depth, + source_leaf_count=source_leaf_count, + current_information_granularity="unknown", + sufficiency_status=sufficiency_status, + granularity_problem_status=granularity_status, + branch_problem_status="none", + recommended_next_action=recommended_next_action, + inspection_reason=f"CODE_STATUS:{frame_label}_copied_graph_node_child_coordinates", + source_r2_selection_frame_id=r2.frame_id, + source_data_ids=[r2.frame_id, handoff_packet.packet_id, selected_id], + source_trace_ids=list(handoff_packet.source_trace_ids), + generated_by=generated_by, + info_class="mixed", + semantic_judgement_status="not_run", + ) + validate_r3_graph_inspection_frame(frame) + return frame + + +def _build_candidate_surface( + *, + data_store: DataStore, + r3: R3GraphInspectionFrame, + frame_label: str, + step_index: int, + graph_edge_payloads: list[dict[str, object]] | None = None, +) -> RGraphTraversalCandidateSurfaceFrame: + next_edge_payloads = _graph_next_edge_payloads( + data_store=data_store, + from_node_id=r3.inspected_graph_node_id, + graph_edge_payloads=graph_edge_payloads, + ) + previous_edge_payloads = _graph_next_edge_payloads( + data_store=data_store, + to_node_id=r3.inspected_graph_node_id, + graph_edge_payloads=graph_edge_payloads, + ) + next_candidate_node_ids = _unique_strings( + [_text(edge.get("to_node_id"), fallback="") for edge in next_edge_payloads] + ) + previous_candidate_node_ids = _unique_strings( + [_text(edge.get("from_node_id"), fallback="") for edge in previous_edge_payloads] + ) + child_candidate_node_ids = _unique_strings(list(r3.child_node_ids)) + candidate_graph_node_ids = _unique_strings( + [ + *child_candidate_node_ids, + *next_candidate_node_ids, + *previous_candidate_node_ids, + ] + ) + candidate_records: list[dict[str, str]] = [] + for graph_node_id in child_candidate_node_ids: + candidate_records.append( + { + "candidate_node_id": graph_node_id, + "relation": "child", + "source_id": r3.frame_id, + "source_field": "child_node_ids", + } + ) + for edge in next_edge_payloads: + edge_id = _text(edge.get("edge_id"), fallback="") + graph_node_id = _text(edge.get("to_node_id"), fallback="") + if edge_id and graph_node_id: + candidate_records.append( + { + "candidate_node_id": graph_node_id, + "relation": "next", + "source_id": edge_id, + "source_field": "to_node_id", + } + ) + for edge in previous_edge_payloads: + edge_id = _text(edge.get("edge_id"), fallback="") + graph_node_id = _text(edge.get("from_node_id"), fallback="") + if edge_id and graph_node_id: + candidate_records.append( + { + "candidate_node_id": graph_node_id, + "relation": "previous", + "source_id": edge_id, + "source_field": "from_node_id", + } + ) + source_data_ids = _unique_strings( + [ + r3.frame_id, + r3.inspected_graph_node_id, + *child_candidate_node_ids, + *next_candidate_node_ids, + *previous_candidate_node_ids, + *[_text(edge.get("edge_id"), fallback="") for edge in next_edge_payloads], + *[_text(edge.get("edge_id"), fallback="") for edge in previous_edge_payloads], + ] + ) + source_trace_ids = _unique_strings( + [ + *r3.source_trace_ids, + *[ + trace_id + for edge in [*next_edge_payloads, *previous_edge_payloads] + for trace_id in _string_list(edge.get("source_trace_ids")) + ], + ] + ) + frame = RGraphTraversalCandidateSurfaceFrame( + frame_id=f"R:{frame_label}:graph_traversal_candidate_surface_frame:{step_index:04d}", + source_r3_inspection_frame_id=r3.frame_id, + inspected_graph_node_id=r3.inspected_graph_node_id, + child_candidate_node_ids=child_candidate_node_ids, + next_candidate_node_ids=next_candidate_node_ids, + previous_candidate_node_ids=previous_candidate_node_ids, + candidate_graph_node_ids=candidate_graph_node_ids, + candidate_records=candidate_records, + candidate_count=len(candidate_graph_node_ids), + source_data_ids=source_data_ids, + source_trace_ids=source_trace_ids, + ) + validate_r_graph_traversal_candidate_surface_frame(frame) + return frame + + +def _build_return_summary( + *, + handoff_packet: RLoopMemoryHandoffPacketFrame, + r1: R1GraphGoalFrame, + budgets: list[RLoopBudgetFrame], + r2_selections: list[R2GraphNodeSelectionFrame], + r3_inspections: list[R3GraphInspectionFrame], + candidate_surfaces: list[RGraphTraversalCandidateSurfaceFrame], + continuation: RLoopContinuationFrame, + frame_label: str, + generated_by: str, +) -> RLoopReturnSummaryFrame: + if not budgets or not r2_selections or not r3_inspections or not candidate_surfaces: + raise ValueError("R return summary requires traversal step frames") + final_budget = budgets[-1] + final_r3 = r3_inspections[-1] + if continuation.continuation_status == "stop_sufficient": + task_status = "sufficient" + elif continuation.continuation_status == "stop_failed_final": + task_status = "failed" + else: + task_status = "partial" + selected_entry_node_ids = _unique_strings( + [ + r2.selected_graph_node_id + for r2 in r2_selections + if r2.selected_graph_node_id is not None + ] + ) + inspected_graph_node_ids = _unique_strings( + [r3.inspected_graph_node_id for r3 in r3_inspections] + ) + frame = RLoopReturnSummaryFrame( + frame_id=f"R:{frame_label}:return_summary_frame", + r_loop_task_status=task_status, + selected_entry_node_ids=selected_entry_node_ids, + inspected_graph_node_ids=inspected_graph_node_ids, + final_information_granularity=final_r3.current_information_granularity, + summary_depth_used=final_r3.summary_depth, + continuation_status=continuation.continuation_status, + budget_status=final_budget.budget_status, + source_graph_node_ids=_unique_strings( + [ + *selected_entry_node_ids, + *inspected_graph_node_ids, + *[ + child_id + for r3 in r3_inspections + for child_id in r3.child_node_ids + ], + *[ + candidate_id + for surface in candidate_surfaces + for candidate_id in surface.candidate_graph_node_ids + ], + *handoff_packet.source_graph_node_ids, + ] + ), + source_data_ids=_unique_strings( + [ + handoff_packet.packet_id, + r1.frame_id, + *[budget.frame_id for budget in budgets], + *[r2.frame_id for r2 in r2_selections], + *[r3.frame_id for r3 in r3_inspections], + *[surface.frame_id for surface in candidate_surfaces], + continuation.frame_id, + ] + ), + source_trace_ids=_unique_strings( + [ + *handoff_packet.source_trace_ids, + *continuation.source_trace_ids, + *[ + trace_id + for surface in candidate_surfaces + for trace_id in surface.source_trace_ids + ], + ] + ), + generated_by=generated_by, + info_class="absolute", + semantic_judgement_status="not_run", + ) + validate_r_loop_return_summary_frame(frame) + return frame + + +def _build_access_ledger( + *, + turn_id: str, + handoff_packet: RLoopMemoryHandoffPacketFrame, + r1: R1GraphGoalFrame, + budgets: list[RLoopBudgetFrame], + r2_selections: list[R2GraphNodeSelectionFrame], + r3_inspections: list[R3GraphInspectionFrame], + candidate_surfaces: list[RGraphTraversalCandidateSurfaceFrame], + continuation: RLoopContinuationFrame, + summary: RLoopReturnSummaryFrame, + frame_label: str, +) -> TurnGraphAccessLedgerFrame: + candidate_graph_node_ids = _unique_strings( + [ + *[ + graph_node_id + for r2 in r2_selections + for graph_node_id in r2.available_graph_node_ids + ], + *[ + child_id + for r3 in r3_inspections + for child_id in r3.child_node_ids + ], + *[ + candidate_id + for surface in candidate_surfaces + for candidate_id in surface.candidate_graph_node_ids + ], + ] + ) + selected_graph_node_ids = _unique_strings( + [ + r2.selected_graph_node_id + for r2 in r2_selections + if r2.selected_graph_node_id is not None + ] + ) + inspected_graph_node_ids = _unique_strings( + [r3.inspected_graph_node_id for r3 in r3_inspections] + ) + access_records: list[dict[str, str]] = [] + for r2 in r2_selections: + for graph_node_id in r2.available_graph_node_ids: + access_records.append( + { + "stage": "candidate_seen", + "graph_node_id": graph_node_id, + "source_frame_id": r2.frame_id, + "source_field": "available_graph_node_ids", + } + ) + if r2.selected_graph_node_id is not None: + access_records.append( + { + "stage": "selected", + "graph_node_id": r2.selected_graph_node_id, + "source_frame_id": r2.frame_id, + "source_field": "selected_graph_node_id", + } + ) + for r3 in r3_inspections: + access_records.append( + { + "stage": "inspected", + "graph_node_id": r3.inspected_graph_node_id, + "source_frame_id": r3.frame_id, + "source_field": "inspected_graph_node_id", + } + ) + for graph_node_id in r3.child_node_ids: + access_records.append( + { + "stage": "candidate_seen", + "graph_node_id": graph_node_id, + "source_frame_id": r3.frame_id, + "source_field": "child_node_ids", + } + ) + for surface in candidate_surfaces: + for record in surface.candidate_records: + graph_node_id = record.get("candidate_node_id") + if graph_node_id: + access_records.append( + { + "stage": "candidate_seen", + "graph_node_id": graph_node_id, + "source_frame_id": surface.frame_id, + "source_field": f"candidate_records.{record.get('relation', 'unknown')}", + } + ) + + source_data_ids = _unique_strings( + [ + handoff_packet.packet_id, + handoff_packet.r_loop_graph_guide_packet_id, + handoff_packet.graph_snapshot_id, + r1.frame_id, + *[budget.frame_id for budget in budgets], + *[r2.frame_id for r2 in r2_selections], + *[r3.frame_id for r3 in r3_inspections], + *[surface.frame_id for surface in candidate_surfaces], + continuation.frame_id, + summary.frame_id, + *candidate_graph_node_ids, + *selected_graph_node_ids, + *inspected_graph_node_ids, + ] + ) + frame = TurnGraphAccessLedgerFrame( + frame_id=f"R:{frame_label}:turn_graph_access_ledger_frame", + turn_id=turn_id, + turn_capsule_graph_node_id=raw_capsule_graph_node_id(turn_id), + candidate_graph_node_ids=candidate_graph_node_ids, + selected_graph_node_ids=selected_graph_node_ids, + inspected_graph_node_ids=inspected_graph_node_ids, + read_graph_node_ids=[], + used_as_answer_source_graph_node_ids=[], + access_records=access_records, + source_trace_ids=_unique_strings( + [ + *handoff_packet.source_trace_ids, + *continuation.source_trace_ids, + *summary.source_trace_ids, + *[ + trace_id + for surface in candidate_surfaces + for trace_id in surface.source_trace_ids + ], + ] + ), + source_data_ids=source_data_ids, + ) + validate_turn_graph_access_ledger_frame(frame) + return frame + + +def _safe_frame_label(value: str) -> str: + normalized = value.strip().replace("-", "_").replace(" ", "_") + if not normalized: + return "dry_run" + if not all(ch.isalnum() or ch == "_" for ch in normalized): + raise ValueError("R loop frame_label must contain only letters, numbers, or underscore") + return normalized + + +def _graph_node_payload( + *, + data_store: DataStore, + node_id: str, + graph_node_payloads: dict[str, dict[str, object]] | None = None, +) -> dict[str, object]: + if graph_node_payloads is not None: + payload = graph_node_payloads.get(node_id) + if payload is not None: + return payload + record = data_store.get_record(node_id) + if record is None: + return {} + if not isinstance(record.payload, dict): + raise TypeError(f"graph node payload must be a dict: {node_id}") + return record.payload + + +def _graph_next_edge_payloads( + *, + data_store: DataStore, + from_node_id: str | None = None, + to_node_id: str | None = None, + graph_edge_payloads: list[dict[str, object]] | None = None, +) -> list[dict[str, object]]: + edge_payloads: list[dict[str, object]] = [] + for payload in graph_edge_payloads or []: + if payload.get("edge_kind") != "NEXT": + continue + if from_node_id is not None and payload.get("from_node_id") != from_node_id: + continue + if to_node_id is not None and payload.get("to_node_id") != to_node_id: + continue + edge_payloads.append(payload) + for record in data_store.list_records(): + if record.data_type != "graph_memory:edge:NEXT": + continue + if not isinstance(record.payload, dict): + continue + payload = record.payload + if from_node_id is not None and payload.get("from_node_id") != from_node_id: + continue + if to_node_id is not None and payload.get("to_node_id") != to_node_id: + continue + edge_payloads.append(payload) + return edge_payloads + + +def _text(value: object, *, fallback: str) -> str: + return value if isinstance(value, str) and value else fallback + + +def _int(value: object) -> int: + return value if isinstance(value, int) else 0 + + +def _string_list(value: object) -> list[str]: + if not isinstance(value, list): + return [] + result: list[str] = [] + for item in value: + if isinstance(item, str) and item: + result.append(item) + return result + + +def _unique_strings(values: list[str | None]) -> list[str]: + seen: set[str] = set() + result: list[str] = [] + for value in values: + if value is None or value in seen: + continue + seen.add(value) + result.append(value) + return result diff --git a/songryeon_core/loops/r_loop_vessel_one_step.py b/songryeon_core/loops/r_loop_vessel_one_step.py new file mode 100644 index 0000000..41baafe --- /dev/null +++ b/songryeon_core/loops/r_loop_vessel_one_step.py @@ -0,0 +1,3826 @@ +from __future__ import annotations + +import hashlib +import re +from dataclasses import asdict, dataclass +from pathlib import Path + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.r_loop_state_machine import decide_r_loop_continuation +from songryeon_core.core.r_loop_vessel_read_packet import RLoopVesselReadPacketFrame +from songryeon_core.core.schemas import ( + R1GraphGoalFrame, + R2GraphNodeSelectionFrame, + R3GraphInspectionFrame, + RGraphTraversalCandidateSurfaceFrame, + RLoopBudgetFrame, + RLoopContinuationFrame, + RLoopReturnSummaryFrame, + validate_r1_graph_goal_frame, + validate_r2_graph_node_selection_frame, + validate_r3_graph_inspection_frame, + validate_r_graph_traversal_candidate_surface_frame, + validate_r_loop_budget_frame, + validate_r_loop_continuation_frame, + validate_r_loop_return_summary_frame, +) +from songryeon_core.core.trace_store import TraceStore +from songryeon_core.llm.base import LLMAdapter, LLMRequest, LLMResponse +from songryeon_core.llm.node_executor import LLMNodeExecutor + + +R_LOOP_VESSEL_ONE_STEP_GENERATOR = "CODE:R_LOOP_VESSEL_ONE_STEP_ASSEMBLER" +R_LOOP_VESSEL_ONE_STEP_RESULT_DATA_TYPE = "r_loop:vessel_one_step_result" +R_LOOP_VESSEL_ONE_STEP_SCHEMA_NAME = "RLoopVesselOneStepResultFrame" +R_LOOP_VESSEL_ONE_STEP_POLICY_ID = "R_LOOP_VESSEL_ONE_STEP_TRAVERSAL_V0" +R_LOOP_VESSEL_ONE_STEP_STATUSES = {"completed", "failed", "not_run"} +R_LOOP_VESSEL_TRAVERSE_GENERATOR = "CODE:R_LOOP_VESSEL_MULTI_STEP_ASSEMBLER" +R_LOOP_VESSEL_TRAVERSE_RESULT_DATA_TYPE = "r_loop:vessel_traverse_result" +R_LOOP_VESSEL_TRAVERSE_SCHEMA_NAME = "RLoopVesselTraverseResultFrame" +R_LOOP_VESSEL_TRAVERSE_POLICY_ID = "R_LOOP_VESSEL_MULTI_STEP_TRAVERSAL_MVP_V0" +R_LOOP_VESSEL_TRAVERSE_STATUSES = {"completed", "failed", "not_run"} +R_LOOP_VESSEL_CANDIDATE_LAYER_SURFACE_GENERATOR = ( + "CODE:R_LOOP_VESSEL_CANDIDATE_LAYER_SURFACE_BUILDER" +) +R_LOOP_VESSEL_CANDIDATE_LAYER_SURFACE_DATA_TYPE = ( + "r_loop:vessel_candidate_layer_surface" +) +R_LOOP_VESSEL_CANDIDATE_LAYER_SURFACE_SCHEMA_NAME = ( + "RLoopVesselCandidateLayerSurfaceFrame" +) +R_LOOP_VESSEL_SURFACE_SELECTION_DATA_TYPE = ( + "node_output:R_loop_vessel_surface_selection_frame" +) +R_LOOP_VESSEL_SURFACE_SELECTION_SCHEMA_NAME = "RLoopVesselSurfaceSelectionFrame" + +R1_VESSEL_GOAL_PROMPT_REF = "songryeon_core/prompts/r1_vessel_goal_setter_v0.md" +R2_VESSEL_SELECTOR_PROMPT_REF = "songryeon_core/prompts/r2_vessel_node_selector_v0.md" +R3_VESSEL_INSPECTOR_PROMPT_REF = "songryeon_core/prompts/r3_vessel_inspector_v0.md" + +R1_VESSEL_NODE_ID = "R1_vessel_goal_setter" +R2_VESSEL_NODE_ID = "R2_vessel_node_selector" +R3_VESSEL_NODE_ID = "R3_vessel_inspector" + +R_ONE_STEP_MAX_TRAVERSAL_DEPTH = 1 +R_ONE_STEP_MAX_BRANCH_SWITCHES = 0 +R_ONE_STEP_MAX_NODE_READS = 1 +R_ONE_STEP_MAX_CONTEXT_TOKENS = 4000 +R_TRAVERSE_MAX_TRAVERSAL_DEPTH = 6 +R_TRAVERSE_MAX_BRANCH_SWITCHES = 0 +R_TRAVERSE_MAX_NODE_READS = 6 +R_TRAVERSE_MAX_CONTEXT_TOKENS = 8000 +R_TRAVERSE_MIN_TERMINAL_MATERIAL_READS = 1 +R_TRAVERSE_MAX_RAW_ORIGINAL_MATERIAL_READS = 5 + + +@dataclass(frozen=True) +class RLoopVesselOneStepResultFrame: + frame_id: str + created_at: str + policy_id: str + one_step_status: str + failure_stage: str | None + failure_type: str | None + failure_reason: str | None + source_packet_id: str + r1_goal_frame_id: str | None + r2_selection_frame_id: str | None + r3_inspection_frame_id: str | None + budget_frame_id: str | None + continuation_frame_id: str | None + return_summary_frame_id: str | None + selected_graph_node_id: str | None + inspected_graph_node_id: str | None + sufficiency_status: str | None + continuation_status: str | None + llm_call_data_ids: list[str] + output_data_ids: list[str] + source_data_ids: list[str] + source_trace_ids: list[str] + failure_payload_summary: dict[str, object] | None = None + generated_by: str = R_LOOP_VESSEL_ONE_STEP_GENERATOR + info_class: str = "absolute" + semantic_judgement_status: str = "not_run" + schema_name: str = R_LOOP_VESSEL_ONE_STEP_SCHEMA_NAME + + +@dataclass(frozen=True) +class RLoopVesselTraverseResultFrame: + frame_id: str + created_at: str + policy_id: str + traverse_status: str + failure_stage: str | None + failure_type: str | None + failure_reason: str | None + source_packet_id: str + r1_goal_frame_id: str | None + final_budget_frame_id: str | None + return_summary_frame_id: str | None + step_count: int + selected_graph_node_ids: list[str] + inspected_graph_node_ids: list[str] + candidate_surface_frame_ids: list[str] + graph_traversal_candidate_surface_frame_ids: list[str] + r2_selection_frame_ids: list[str] + r3_inspection_frame_ids: list[str] + continuation_frame_ids: list[str] + final_graph_node_id: str | None + final_sufficiency_status: str | None + final_continuation_status: str | None + r_loop_task_status: str | None + final_information_granularity: str | None + summary_depth_used: int | None + terminal_material_seen_count: int + min_terminal_material_count: int + raw_original_material_seen_count: int + max_raw_original_material_count: int + raw_original_read_cap_reached: bool + early_stop_guard_trigger_count: int + llm_call_data_ids: list[str] + output_data_ids: list[str] + source_data_ids: list[str] + source_trace_ids: list[str] + failure_payload_summary: dict[str, object] | None = None + generated_by: str = R_LOOP_VESSEL_TRAVERSE_GENERATOR + info_class: str = "absolute" + semantic_judgement_status: str = "not_run" + schema_name: str = R_LOOP_VESSEL_TRAVERSE_SCHEMA_NAME + + +@dataclass(frozen=True) +class RLoopVesselCandidateLayerSurfaceFrame: + frame_id: str + created_at: str + source_packet_id: str + surface_count: int + total_candidate_count: int + available_surface_ids: list[str] + surface_records: list[dict[str, object]] + source_data_ids: list[str] + source_trace_ids: list[str] + generated_by: str = R_LOOP_VESSEL_CANDIDATE_LAYER_SURFACE_GENERATOR + info_class: str = "absolute" + semantic_judgement_status: str = "not_run" + schema_name: str = R_LOOP_VESSEL_CANDIDATE_LAYER_SURFACE_SCHEMA_NAME + + +@dataclass(frozen=True) +class RLoopVesselSurfaceSelectionFrame: + frame_id: str + selection_scope: str + available_surface_ids: list[str] + selection_status: str + selected_surface_id: str | None + selected_surface_candidate_count: int + surface_selection_reason: str + source_r1_goal_frame_id: str + source_surface_frame_id: str + source_data_ids: list[str] + source_trace_ids: list[str] + generated_by: str + info_class: str = "mixed" + semantic_judgement_status: str = "ran" + schema_name: str = R_LOOP_VESSEL_SURFACE_SELECTION_SCHEMA_NAME + + +@dataclass(frozen=True) +class RLoopVesselOneStepRun: + result_frame: RLoopVesselOneStepResultFrame + r1_goal: R1GraphGoalFrame | None + budget: RLoopBudgetFrame | None + r2_selection: R2GraphNodeSelectionFrame | None + r3_inspection: R3GraphInspectionFrame | None + continuation: RLoopContinuationFrame | None + return_summary: RLoopReturnSummaryFrame | None + trace_event_ids: list[str] + output_data_ids: list[str] + candidate_layer_surface: RLoopVesselCandidateLayerSurfaceFrame | None = None + surface_selection: RLoopVesselSurfaceSelectionFrame | None = None + graph_traversal_candidate_surface: RGraphTraversalCandidateSurfaceFrame | None = None + + +@dataclass(frozen=True) +class RLoopVesselTraverseRun: + result_frame: RLoopVesselTraverseResultFrame + r1_goal: R1GraphGoalFrame | None + final_budget: RLoopBudgetFrame | None + r2_selections: list[R2GraphNodeSelectionFrame] + r3_inspections: list[R3GraphInspectionFrame] + continuations: list[RLoopContinuationFrame] + return_summary: RLoopReturnSummaryFrame | None + candidate_layer_surfaces: list[RLoopVesselCandidateLayerSurfaceFrame] + surface_selections: list[RLoopVesselSurfaceSelectionFrame] + graph_traversal_candidate_surfaces: list[RGraphTraversalCandidateSurfaceFrame] + trace_event_ids: list[str] + output_data_ids: list[str] + + +class RLoopVesselOneStepFakeLLMAdapter: + """Deterministic fake adapter for the R Vessel one-step CLI/tests.""" + + model_id = "r-loop-vessel-one-step-fake-llm-adapter" + + def complete(self, request: LLMRequest) -> LLMResponse: + if "R1 Vessel Goal Setter" in request.prompt: + user_question = str(request.input_payload.get("user_question") or "").strip() + payload = { + "graph_search_goal": ( + "Inspect one active Vessel summary candidate for the current question" + f": {user_question}" + ), + "user_question_anchor_id": _r1_anchor_id_from_input_payload( + request.input_payload + ), + "required_information_granularity": "low_summary", + "allowed_summary_depth": 1, + "stop_condition": "Stop after one selected candidate inspection.", + } + elif "R2 Vessel Node Selector" in request.prompt: + available_surfaces = request.input_payload.get("available_surface_refs") + selected_surface_id = None + records_by_surface = request.input_payload.get("candidate_records_by_surface_ref") + selected_id = None + if isinstance(available_surfaces, list) and isinstance(records_by_surface, dict): + for surface_id in available_surfaces: + if not isinstance(surface_id, str) or not surface_id: + continue + value = records_by_surface.get(surface_id) + if not isinstance(value, list): + continue + for item in value: + if ( + isinstance(item, dict) + and isinstance(item.get("node_ref"), str) + and item.get("node_ref") + and item.get("summary_text") + ): + selected_surface_id = surface_id + selected_id = str(item["node_ref"]) + break + if selected_id: + break + if not selected_id: + selected_surface_id = next( + (item for item in available_surfaces if isinstance(item, str) and item), + None, + ) + if selected_surface_id: + value = records_by_surface.get(selected_surface_id) + if isinstance(value, list): + selected_id = next( + ( + str(item.get("node_ref")) + for item in value + if isinstance(item, dict) + and isinstance(item.get("node_ref"), str) + and item.get("node_ref") + ), + None, + ) + payload = { + "selection_status": "selected" if selected_id else "none_selected", + "selected_surface_ref": selected_surface_id if selected_id else None, + "selected_node_ref": selected_id, + "selection_reason": ( + "Fake adapter selected the first supplied candidate inside the first supplied Vessel surface." + if selected_id + else "Fake adapter received no available Vessel candidates." + ), + "expected_information_granularity": "low_summary", + "expected_source_kind": "summary_or_entry_candidate", + } + elif "R3 Vessel Inspector" in request.prompt: + selected = request.input_payload.get("selected_candidate_record") + summary_text = "" + selected_node_id = "" + if isinstance(selected, dict): + summary_text = str(selected.get("summary_text") or "") + selected_node_id = str( + selected.get("summary_node_id") + or selected.get("candidate_node_id") + or "" + ) + has_candidate_material = bool(summary_text or selected_node_id) + payload = { + "current_information_granularity": ( + "low_summary" if summary_text else "raw" if selected_node_id else "unknown" + ), + "sufficiency_status": "sufficient" if has_candidate_material else "insufficient", + "granularity_problem_status": "none" if has_candidate_material else "needs_lower_granularity", + "branch_problem_status": "none", + "recommended_next_action": "stop" if has_candidate_material else "deeper", + "inspection_reason": ( + "Fake adapter treats supplied summary text as sufficient for one-step smoke." + if summary_text + else "Fake adapter treats the supplied entry candidate as sufficient for one-step smoke." + if selected_node_id + else "Fake adapter found no selected candidate material." + ), + } + else: + payload = {"error": "unknown R Vessel one-step prompt"} + import json + + return LLMResponse( + text=json.dumps(payload, ensure_ascii=False), + model_id=self.model_id, + raw=payload, + ) + + +class RLoopVesselTraverseFakeLLMAdapter: + """Deterministic fake adapter for multi-step R Vessel traversal tests/CLI.""" + + model_id = "r-loop-vessel-traverse-fake-llm-adapter" + + def complete(self, request: LLMRequest) -> LLMResponse: + if "R1 Vessel Goal Setter" in request.prompt: + user_question = str(request.input_payload.get("user_question") or "").strip() + payload = { + "graph_search_goal": f"Traverse Vessel hierarchy for: {user_question}", + "user_question_anchor_id": _r1_anchor_id_from_input_payload( + request.input_payload + ), + "required_information_granularity": "low_summary", + "allowed_summary_depth": 1, + "stop_condition": "Stop when a selected summary is sufficient or traversal budget ends.", + } + elif "R2 Vessel Node Selector" in request.prompt: + surface_ref, node_ref, expected_source_kind = _first_visible_ref( + request.input_payload, + ) + payload = { + "selection_status": "selected" if node_ref else "none_selected", + "selected_surface_ref": surface_ref if node_ref else None, + "selected_node_ref": node_ref, + "selection_reason": ( + "Fake traversal adapter selected the first visible hierarchy candidate." + if node_ref + else "Fake traversal adapter received no hierarchy candidates." + ), + "expected_information_granularity": "low_summary", + "expected_source_kind": expected_source_kind or "hierarchy_candidate", + } + elif "R3 Vessel Inspector" in request.prompt: + selected = request.input_payload.get("selected_candidate_record") + summary_text = "" + if isinstance(selected, dict): + summary_text = str(selected.get("summary_text") or "") + child_count = request.input_payload.get("hierarchy_child_candidate_count") + has_children = isinstance(child_count, int) and child_count > 0 + sufficient = bool(summary_text) or not has_children + payload = { + "current_information_granularity": "low_summary" if summary_text else "raw", + "sufficiency_status": "sufficient" if sufficient else "insufficient", + "granularity_problem_status": "none" if sufficient else "needs_lower_granularity", + "branch_problem_status": "none", + "recommended_next_action": "stop" if sufficient else "deeper", + "inspection_reason": ( + "Fake traversal adapter treats selected summary text as sufficient." + if summary_text + else "Fake traversal adapter recommends deeper traversal because child candidates exist." + if has_children + else "Fake traversal adapter stops because no child candidates remain." + ), + } + else: + payload = {"error": "unknown R Vessel traversal prompt"} + import json + + return LLMResponse( + text=json.dumps(payload, ensure_ascii=False), + model_id=self.model_id, + raw=payload, + ) + + +def run_r_loop_vessel_one_step( + *, + trace_store: TraceStore, + data_store: DataStore, + turn_id: str, + user_question: str, + read_packet: RLoopVesselReadPacketFrame, + adapter: LLMAdapter | None, + frame_label: str = "manual_vessel_r_one_step", + input_ref: list[str] | None = None, +) -> RLoopVesselOneStepRun: + frame_label = _safe_frame_label(frame_label) + source_trace_ids = _unique_strings([*(input_ref or []), *read_packet.source_trace_ids]) + result_frame_id = _result_frame_id(frame_label) + if read_packet.read_status != "passed": + return _record_failure_result( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + frame_id=result_frame_id, + created_at=_now_from_trace(trace_store), + source_packet_id=read_packet.packet_id, + failure_stage="read_packet", + failure_type=read_packet.failure_type or "read_packet_not_passed", + failure_reason=read_packet.failure_reason or "R Vessel read packet is not passed.", + source_data_ids=[read_packet.packet_id], + source_trace_ids=source_trace_ids, + ) + if adapter is None: + return _record_failure_result( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + frame_id=result_frame_id, + created_at=_now_from_trace(trace_store), + source_packet_id=read_packet.packet_id, + failure_stage="adapter", + failure_type="adapter_missing", + failure_reason="R Vessel one-step traversal requires an LLM adapter.", + source_data_ids=[read_packet.packet_id], + source_trace_ids=source_trace_ids, + ) + + executor = LLMNodeExecutor(adapter) + llm_call_data_ids: list[str] = [] + trace_event_ids: list[str] = [] + output_data_ids: list[str] = [] + + r1_result = executor.run( + node_id=R1_VESSEL_NODE_ID, + prompt=_prompt(R1_VESSEL_GOAL_PROMPT_REF), + input_payload=_r1_input_payload( + user_question=user_question, + read_packet=read_packet, + policy_name="one_step", + max_traversal_depth=R_ONE_STEP_MAX_TRAVERSAL_DEPTH, + max_branch_switches=R_ONE_STEP_MAX_BRANCH_SWITCHES, + max_node_reads=R_ONE_STEP_MAX_NODE_READS, + max_context_tokens=R_ONE_STEP_MAX_CONTEXT_TOKENS, + ), + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + prompt_ref=R1_VESSEL_GOAL_PROMPT_REF, + input_ref=source_trace_ids, + source_data_ids=[read_packet.packet_id], + payload_validator=lambda payload: _validate_r1_payload( + payload, + read_packet=read_packet, + user_question=user_question, + ), + ) + _append_llm_refs(r1_result, llm_call_data_ids, trace_event_ids) + if r1_result.failure_type != "none" or r1_result.validation.payload is None: + return _record_failure_result( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + frame_id=result_frame_id, + created_at=_now_from_trace(trace_store), + source_packet_id=read_packet.packet_id, + failure_stage="R1", + failure_type=r1_result.failure_type, + failure_reason=r1_result.validation.error or "R1 payload validation failed.", + source_data_ids=_unique_strings([read_packet.packet_id, r1_result.call_data_id]), + source_trace_ids=_unique_strings([*source_trace_ids, r1_result.trace_event_id]), + llm_call_data_ids=llm_call_data_ids, + ) + + r1 = _r1_frame_from_payload( + payload=r1_result.validation.payload, + frame_label=frame_label, + read_packet=read_packet, + model_id=r1_result.model_id, + llm_call_data_id=r1_result.call_data_id, + source_trace_ids=_unique_strings([*source_trace_ids, r1_result.trace_event_id]), + ) + _record_frame( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + actor="R1", + data_type="node_output:R1_graph_goal_frame", + frame_id=r1.frame_id, + payload=asdict(r1), + input_ref=_unique_strings([*source_trace_ids, r1_result.trace_event_id]), + trace_event_ids=trace_event_ids, + output_data_ids=output_data_ids, + ) + + budget = _budget_frame(r1=r1, frame_label=frame_label) + _record_frame( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + actor="R:budget", + data_type="node_output:R_loop_budget_frame", + frame_id=budget.frame_id, + payload=asdict(budget), + input_ref=[r1.frame_id], + trace_event_ids=trace_event_ids, + output_data_ids=output_data_ids, + ) + + candidate_layer_surface = _candidate_layer_surface_frame( + frame_label=frame_label, + read_packet=read_packet, + created_at=_now_from_trace(trace_store), + ) + _record_frame( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + actor="R:candidate_layer_surface", + data_type=R_LOOP_VESSEL_CANDIDATE_LAYER_SURFACE_DATA_TYPE, + frame_id=candidate_layer_surface.frame_id, + payload=asdict(candidate_layer_surface), + input_ref=[read_packet.packet_id], + trace_event_ids=trace_event_ids, + output_data_ids=output_data_ids, + ) + + available_graph_node_ids = _available_graph_node_ids(read_packet) + r2_result = executor.run( + node_id=R2_VESSEL_NODE_ID, + prompt=_prompt(R2_VESSEL_SELECTOR_PROMPT_REF), + input_payload=_r2_input_payload( + user_question=user_question, + read_packet=read_packet, + r1=r1, + candidate_layer_surface=candidate_layer_surface, + available_graph_node_ids=available_graph_node_ids, + ), + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + prompt_ref=R2_VESSEL_SELECTOR_PROMPT_REF, + input_ref=_unique_strings([*source_trace_ids, *trace_event_ids]), + source_data_ids=[read_packet.packet_id, r1.frame_id, candidate_layer_surface.frame_id], + payload_validator=lambda payload: _validate_r2_payload( + payload, + available_graph_node_ids=available_graph_node_ids, + candidate_layer_surface=candidate_layer_surface, + ), + ) + _append_llm_refs(r2_result, llm_call_data_ids, trace_event_ids) + if r2_result.failure_type != "none" or r2_result.validation.payload is None: + return _record_failure_result( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + frame_id=result_frame_id, + created_at=_now_from_trace(trace_store), + source_packet_id=read_packet.packet_id, + failure_stage="R2", + failure_type=r2_result.failure_type, + failure_reason=r2_result.validation.error or "R2 payload validation failed.", + source_data_ids=_unique_strings( + [ + read_packet.packet_id, + r1.frame_id, + candidate_layer_surface.frame_id, + r2_result.call_data_id, + ] + ), + source_trace_ids=_unique_strings([*source_trace_ids, *trace_event_ids, r2_result.trace_event_id]), + llm_call_data_ids=llm_call_data_ids, + r1_goal_frame_id=r1.frame_id, + budget_frame_id=budget.frame_id, + output_data_ids=output_data_ids, + failure_payload_summary=_r2_failure_payload_summary( + r2_result.validation.payload, + available_graph_node_ids=available_graph_node_ids, + candidate_layer_surface=candidate_layer_surface, + ), + ) + + surface_selection = _surface_selection_frame_from_payload( + payload=r2_result.validation.payload, + frame_label=frame_label, + r1=r1, + candidate_layer_surface=candidate_layer_surface, + model_id=r2_result.model_id, + llm_call_data_id=r2_result.call_data_id, + source_trace_ids=_unique_strings([*source_trace_ids, r2_result.trace_event_id]), + ) + _record_frame( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + actor="R2:surface", + data_type=R_LOOP_VESSEL_SURFACE_SELECTION_DATA_TYPE, + frame_id=surface_selection.frame_id, + payload=asdict(surface_selection), + input_ref=_unique_strings([*source_trace_ids, r2_result.trace_event_id]), + trace_event_ids=trace_event_ids, + output_data_ids=output_data_ids, + ) + + r2 = _r2_frame_from_payload( + payload=r2_result.validation.payload, + frame_label=frame_label, + read_packet=read_packet, + r1=r1, + candidate_layer_surface=candidate_layer_surface, + available_graph_node_ids=_surface_candidate_ids_for_selection( + candidate_layer_surface, + surface_selection.selected_surface_id, + fallback_graph_node_ids=available_graph_node_ids, + ), + model_id=r2_result.model_id, + llm_call_data_id=r2_result.call_data_id, + source_trace_ids=_unique_strings([*source_trace_ids, r2_result.trace_event_id]), + ) + _record_frame( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + actor="R2", + data_type="node_output:R2_graph_node_selection_frame", + frame_id=r2.frame_id, + payload=asdict(r2), + input_ref=_unique_strings([*source_trace_ids, r2_result.trace_event_id]), + trace_event_ids=trace_event_ids, + output_data_ids=output_data_ids, + ) + + if r2.selection_status != "selected" or r2.selected_graph_node_id is None: + return_summary = _return_summary_for_no_selection( + frame_label=frame_label, + read_packet=read_packet, + r1=r1, + budget=budget, + r2=r2, + ) + _record_frame( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + actor="R:return_summary", + data_type="node_output:R_loop_return_summary_frame", + frame_id=return_summary.frame_id, + payload=asdict(return_summary), + input_ref=[r2.frame_id], + trace_event_ids=trace_event_ids, + output_data_ids=output_data_ids, + ) + result_frame = _result_frame( + frame_id=result_frame_id, + created_at=_now_from_trace(trace_store), + one_step_status="completed", + source_packet_id=read_packet.packet_id, + r1_goal_frame_id=r1.frame_id, + r2_selection_frame_id=r2.frame_id, + budget_frame_id=budget.frame_id, + return_summary_frame_id=return_summary.frame_id, + llm_call_data_ids=llm_call_data_ids, + output_data_ids=output_data_ids, + source_data_ids=[read_packet.packet_id, r1.frame_id, budget.frame_id, r2.frame_id, return_summary.frame_id], + source_trace_ids=_unique_strings([*source_trace_ids, *trace_event_ids]), + ) + _record_result_frame(trace_store, data_store, turn_id, result_frame, trace_event_ids, output_data_ids) + return RLoopVesselOneStepRun( + result_frame=result_frame, + r1_goal=r1, + budget=budget, + r2_selection=r2, + r3_inspection=None, + continuation=None, + return_summary=return_summary, + trace_event_ids=trace_event_ids, + output_data_ids=output_data_ids, + candidate_layer_surface=candidate_layer_surface, + surface_selection=surface_selection, + ) + + selected_record = _selected_candidate_record(read_packet, r2.selected_graph_node_id) + r3_result = executor.run( + node_id=R3_VESSEL_NODE_ID, + prompt=_prompt(R3_VESSEL_INSPECTOR_PROMPT_REF), + input_payload=_r3_input_payload( + user_question=user_question, + r1=r1, + r2=r2, + selected_record=selected_record, + ), + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + prompt_ref=R3_VESSEL_INSPECTOR_PROMPT_REF, + input_ref=_unique_strings([*source_trace_ids, *trace_event_ids]), + source_data_ids=[read_packet.packet_id, r1.frame_id, r2.frame_id, r2.selected_graph_node_id], + payload_validator=_validate_r3_payload, + ) + _append_llm_refs(r3_result, llm_call_data_ids, trace_event_ids) + if r3_result.failure_type != "none" or r3_result.validation.payload is None: + return _record_failure_result( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + frame_id=result_frame_id, + created_at=_now_from_trace(trace_store), + source_packet_id=read_packet.packet_id, + failure_stage="R3", + failure_type=r3_result.failure_type, + failure_reason=r3_result.validation.error or "R3 payload validation failed.", + source_data_ids=_unique_strings( + [ + read_packet.packet_id, + r1.frame_id, + budget.frame_id, + candidate_layer_surface.frame_id, + surface_selection.frame_id, + r2.frame_id, + r3_result.call_data_id, + ] + ), + source_trace_ids=_unique_strings([*source_trace_ids, *trace_event_ids, r3_result.trace_event_id]), + llm_call_data_ids=llm_call_data_ids, + r1_goal_frame_id=r1.frame_id, + r2_selection_frame_id=r2.frame_id, + budget_frame_id=budget.frame_id, + selected_graph_node_id=r2.selected_graph_node_id, + output_data_ids=output_data_ids, + ) + + r3 = _r3_frame_from_payload( + payload=r3_result.validation.payload, + frame_label=frame_label, + selected_record=selected_record, + read_packet=read_packet, + r2=r2, + model_id=r3_result.model_id, + llm_call_data_id=r3_result.call_data_id, + source_trace_ids=_unique_strings([*source_trace_ids, r3_result.trace_event_id]), + ) + _record_frame( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + actor="R3", + data_type="node_output:R3_graph_inspection_frame", + frame_id=r3.frame_id, + payload=asdict(r3), + input_ref=_unique_strings([*source_trace_ids, r3_result.trace_event_id]), + trace_event_ids=trace_event_ids, + output_data_ids=output_data_ids, + ) + + graph_traversal_candidate_surface = _graph_traversal_candidate_surface_frame( + frame_label=frame_label, + r3=r3, + selected_record=selected_record, + ) + _record_frame( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + actor="R:graph_traversal_candidate_surface", + data_type="node_output:R_graph_traversal_candidate_surface_frame", + frame_id=graph_traversal_candidate_surface.frame_id, + payload=asdict(graph_traversal_candidate_surface), + input_ref=[r3.frame_id], + trace_event_ids=trace_event_ids, + output_data_ids=output_data_ids, + ) + + continuation = decide_r_loop_continuation( + frame_id=f"R:{frame_label}:vessel_one_step_continuation_frame", + r3_inspection=r3, + budget=budget, + source_trace_ids=_unique_strings([*source_trace_ids, *trace_event_ids]), + ) + validate_r_loop_continuation_frame(continuation) + _record_frame( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + actor="R:continuation", + data_type="node_output:R_loop_continuation_frame", + frame_id=continuation.frame_id, + payload=asdict(continuation), + input_ref=[r3.frame_id, budget.frame_id], + trace_event_ids=trace_event_ids, + output_data_ids=output_data_ids, + ) + + return_summary = _return_summary_for_completed_step( + frame_label=frame_label, + read_packet=read_packet, + r1=r1, + budget=budget, + r2=r2, + r3=r3, + continuation=continuation, + ) + _record_frame( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + actor="R:return_summary", + data_type="node_output:R_loop_return_summary_frame", + frame_id=return_summary.frame_id, + payload=asdict(return_summary), + input_ref=[continuation.frame_id], + trace_event_ids=trace_event_ids, + output_data_ids=output_data_ids, + ) + + result_frame = _result_frame( + frame_id=result_frame_id, + created_at=_now_from_trace(trace_store), + one_step_status="completed", + source_packet_id=read_packet.packet_id, + r1_goal_frame_id=r1.frame_id, + r2_selection_frame_id=r2.frame_id, + r3_inspection_frame_id=r3.frame_id, + budget_frame_id=budget.frame_id, + continuation_frame_id=continuation.frame_id, + return_summary_frame_id=return_summary.frame_id, + selected_graph_node_id=r2.selected_graph_node_id, + inspected_graph_node_id=r3.inspected_graph_node_id, + sufficiency_status=r3.sufficiency_status, + continuation_status=continuation.continuation_status, + llm_call_data_ids=llm_call_data_ids, + output_data_ids=output_data_ids, + source_data_ids=[ + read_packet.packet_id, + r1.frame_id, + budget.frame_id, + candidate_layer_surface.frame_id, + surface_selection.frame_id, + r2.frame_id, + r3.frame_id, + graph_traversal_candidate_surface.frame_id, + continuation.frame_id, + return_summary.frame_id, + r2.selected_graph_node_id, + ], + source_trace_ids=_unique_strings([*source_trace_ids, *trace_event_ids]), + ) + _record_result_frame(trace_store, data_store, turn_id, result_frame, trace_event_ids, output_data_ids) + return RLoopVesselOneStepRun( + result_frame=result_frame, + r1_goal=r1, + budget=budget, + r2_selection=r2, + r3_inspection=r3, + continuation=continuation, + return_summary=return_summary, + trace_event_ids=trace_event_ids, + output_data_ids=output_data_ids, + candidate_layer_surface=candidate_layer_surface, + surface_selection=surface_selection, + graph_traversal_candidate_surface=graph_traversal_candidate_surface, + ) + + +def run_r_loop_vessel_traverse( + *, + trace_store: TraceStore, + data_store: DataStore, + turn_id: str, + user_question: str, + read_packet: RLoopVesselReadPacketFrame, + adapter: LLMAdapter | None, + frame_label: str = "manual_vessel_r_traverse", + input_ref: list[str] | None = None, + max_traversal_depth: int = R_TRAVERSE_MAX_TRAVERSAL_DEPTH, + max_node_reads: int = R_TRAVERSE_MAX_NODE_READS, + max_branch_switches: int = R_TRAVERSE_MAX_BRANCH_SWITCHES, + max_context_tokens: int = R_TRAVERSE_MAX_CONTEXT_TOKENS, + max_raw_original_material_reads: int = R_TRAVERSE_MAX_RAW_ORIGINAL_MATERIAL_READS, +) -> RLoopVesselTraverseRun: + frame_label = _safe_frame_label(frame_label) + source_trace_ids = _unique_strings([*(input_ref or []), *read_packet.source_trace_ids]) + result_frame_id = _traverse_result_frame_id(frame_label) + if read_packet.read_status != "passed": + return _record_traverse_failure_result( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + frame_id=result_frame_id, + created_at=_now_from_trace(trace_store), + source_packet_id=read_packet.packet_id, + failure_stage="read_packet", + failure_type=read_packet.failure_type or "read_packet_not_passed", + failure_reason=read_packet.failure_reason or "R Vessel read packet is not passed.", + source_data_ids=[read_packet.packet_id], + source_trace_ids=source_trace_ids, + ) + if adapter is None: + return _record_traverse_failure_result( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + frame_id=result_frame_id, + created_at=_now_from_trace(trace_store), + source_packet_id=read_packet.packet_id, + failure_stage="adapter", + failure_type="adapter_missing", + failure_reason="R Vessel traversal requires an LLM adapter.", + source_data_ids=[read_packet.packet_id], + source_trace_ids=source_trace_ids, + ) + + executor = LLMNodeExecutor(adapter) + llm_call_data_ids: list[str] = [] + trace_event_ids: list[str] = [] + output_data_ids: list[str] = [] + candidate_layer_surfaces: list[RLoopVesselCandidateLayerSurfaceFrame] = [] + surface_selections: list[RLoopVesselSurfaceSelectionFrame] = [] + r2_selections: list[R2GraphNodeSelectionFrame] = [] + r3_inspections: list[R3GraphInspectionFrame] = [] + graph_surfaces: list[RGraphTraversalCandidateSurfaceFrame] = [] + continuations: list[RLoopContinuationFrame] = [] + terminal_material_seen_count = 0 + raw_original_material_seen_count = 0 + raw_original_read_cap_reached = False + early_stop_guard_trigger_count = 0 + + r1_result = executor.run( + node_id=R1_VESSEL_NODE_ID, + prompt=_prompt(R1_VESSEL_GOAL_PROMPT_REF), + input_payload=_r1_input_payload( + user_question=user_question, + read_packet=read_packet, + policy_name="multi_step_traversal", + max_traversal_depth=max_traversal_depth, + max_branch_switches=max_branch_switches, + max_node_reads=max_node_reads, + max_context_tokens=max_context_tokens, + ), + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + prompt_ref=R1_VESSEL_GOAL_PROMPT_REF, + input_ref=source_trace_ids, + source_data_ids=[read_packet.packet_id], + payload_validator=lambda payload: _validate_r1_payload( + payload, + read_packet=read_packet, + user_question=user_question, + ), + ) + _append_llm_refs(r1_result, llm_call_data_ids, trace_event_ids) + if r1_result.failure_type != "none" or r1_result.validation.payload is None: + return _record_traverse_failure_result( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + frame_id=result_frame_id, + created_at=_now_from_trace(trace_store), + source_packet_id=read_packet.packet_id, + failure_stage="R1", + failure_type=r1_result.failure_type, + failure_reason=r1_result.validation.error or "R1 payload validation failed.", + source_data_ids=_unique_strings([read_packet.packet_id, r1_result.call_data_id]), + source_trace_ids=_unique_strings([*source_trace_ids, r1_result.trace_event_id]), + llm_call_data_ids=llm_call_data_ids, + ) + + r1 = _r1_frame_from_payload( + payload=r1_result.validation.payload, + frame_label=frame_label, + read_packet=read_packet, + model_id=r1_result.model_id, + llm_call_data_id=r1_result.call_data_id, + source_trace_ids=_unique_strings([*source_trace_ids, r1_result.trace_event_id]), + max_traversal_depth=max_traversal_depth, + max_branch_switches=max_branch_switches, + max_node_reads=max_node_reads, + max_context_tokens=max_context_tokens, + ) + _record_frame( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + actor="R1", + data_type="node_output:R1_graph_goal_frame", + frame_id=r1.frame_id, + payload=asdict(r1), + input_ref=_unique_strings([*source_trace_ids, r1_result.trace_event_id]), + trace_event_ids=trace_event_ids, + output_data_ids=output_data_ids, + ) + + next_candidate_surface = _candidate_layer_surface_frame( + frame_label=f"{frame_label}_step_0001", + read_packet=read_packet, + created_at=_now_from_trace(trace_store), + ) + final_budget: RLoopBudgetFrame | None = None + final_continuation: RLoopContinuationFrame | None = None + return_summary: RLoopReturnSummaryFrame | None = None + + for step_index in range(1, max_node_reads + 1): + step_label = f"{frame_label}_step_{step_index:04d}" + candidate_layer_surface = next_candidate_surface + candidate_layer_surfaces.append(candidate_layer_surface) + _record_frame( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + actor="R:candidate_layer_surface", + data_type=R_LOOP_VESSEL_CANDIDATE_LAYER_SURFACE_DATA_TYPE, + frame_id=candidate_layer_surface.frame_id, + payload=asdict(candidate_layer_surface), + input_ref=[read_packet.packet_id], + trace_event_ids=trace_event_ids, + output_data_ids=output_data_ids, + ) + + available_graph_node_ids = _surface_all_candidate_ids(candidate_layer_surface) + r2_result = executor.run( + node_id=R2_VESSEL_NODE_ID, + prompt=_prompt(R2_VESSEL_SELECTOR_PROMPT_REF), + input_payload=_r2_input_payload( + user_question=user_question, + read_packet=read_packet, + r1=r1, + candidate_layer_surface=candidate_layer_surface, + available_graph_node_ids=available_graph_node_ids, + current_graph_node_id=_surface_current_graph_node_id(candidate_layer_surface), + traversal_policy="hierarchical_child_candidates" + if step_index > 1 + else "core_ego_direct_entry_candidates_only", + ), + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + prompt_ref=R2_VESSEL_SELECTOR_PROMPT_REF, + input_ref=_unique_strings([*source_trace_ids, *trace_event_ids]), + source_data_ids=[read_packet.packet_id, r1.frame_id, candidate_layer_surface.frame_id], + payload_validator=lambda payload, surface=candidate_layer_surface, ids=available_graph_node_ids: _validate_r2_payload( + payload, + available_graph_node_ids=ids, + candidate_layer_surface=surface, + ), + ) + _append_llm_refs(r2_result, llm_call_data_ids, trace_event_ids) + if r2_result.failure_type != "none" or r2_result.validation.payload is None: + return _record_traverse_failure_result( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + frame_id=result_frame_id, + created_at=_now_from_trace(trace_store), + source_packet_id=read_packet.packet_id, + failure_stage=f"R2:step_{step_index:04d}", + failure_type=r2_result.failure_type, + failure_reason=r2_result.validation.error or "R2 payload validation failed.", + source_data_ids=_unique_strings( + [ + read_packet.packet_id, + r1.frame_id, + candidate_layer_surface.frame_id, + r2_result.call_data_id, + ] + ), + source_trace_ids=_unique_strings([*source_trace_ids, *trace_event_ids, r2_result.trace_event_id]), + llm_call_data_ids=llm_call_data_ids, + r1_goal_frame_id=r1.frame_id, + output_data_ids=output_data_ids, + failure_payload_summary=_r2_failure_payload_summary( + r2_result.validation.payload, + available_graph_node_ids=available_graph_node_ids, + candidate_layer_surface=candidate_layer_surface, + ), + candidate_layer_surfaces=candidate_layer_surfaces, + surface_selections=surface_selections, + r2_selections=r2_selections, + r3_inspections=r3_inspections, + graph_surfaces=graph_surfaces, + continuations=continuations, + ) + + surface_selection = _surface_selection_frame_from_payload( + payload=r2_result.validation.payload, + frame_label=step_label, + r1=r1, + candidate_layer_surface=candidate_layer_surface, + model_id=r2_result.model_id, + llm_call_data_id=r2_result.call_data_id, + source_trace_ids=_unique_strings([*source_trace_ids, r2_result.trace_event_id]), + ) + surface_selections.append(surface_selection) + _record_frame( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + actor="R2:surface", + data_type=R_LOOP_VESSEL_SURFACE_SELECTION_DATA_TYPE, + frame_id=surface_selection.frame_id, + payload=asdict(surface_selection), + input_ref=_unique_strings([*source_trace_ids, r2_result.trace_event_id]), + trace_event_ids=trace_event_ids, + output_data_ids=output_data_ids, + ) + + r2 = _r2_frame_from_payload( + payload=r2_result.validation.payload, + frame_label=step_label, + read_packet=read_packet, + r1=r1, + candidate_layer_surface=candidate_layer_surface, + available_graph_node_ids=_surface_candidate_ids_for_selection( + candidate_layer_surface, + surface_selection.selected_surface_id, + fallback_graph_node_ids=available_graph_node_ids, + ), + model_id=r2_result.model_id, + llm_call_data_id=r2_result.call_data_id, + source_trace_ids=_unique_strings([*source_trace_ids, r2_result.trace_event_id]), + ) + r2_selections.append(r2) + _record_frame( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + actor="R2", + data_type="node_output:R2_graph_node_selection_frame", + frame_id=r2.frame_id, + payload=asdict(r2), + input_ref=_unique_strings([*source_trace_ids, r2_result.trace_event_id]), + trace_event_ids=trace_event_ids, + output_data_ids=output_data_ids, + ) + + if r2.selection_status != "selected" or r2.selected_graph_node_id is None: + final_budget = _traverse_budget_frame(r1=r1, frame_label=frame_label, step_index=step_index) + return_summary = _return_summary_for_traverse( + frame_label=frame_label, + read_packet=read_packet, + r1=r1, + final_budget=final_budget, + r2_selections=r2_selections, + r3_inspections=r3_inspections, + final_continuation=None, + forced_status="partial", + forced_continuation_status="stop_no_actionable_path", + ) + break + + selected_record = _selected_candidate_record(read_packet, r2.selected_graph_node_id) + selected_is_terminal_material = _is_terminal_material_record(selected_record) + selected_is_raw_original_material = _is_raw_original_material_record(selected_record) + r3_result = executor.run( + node_id=R3_VESSEL_NODE_ID, + prompt=_prompt(R3_VESSEL_INSPECTOR_PROMPT_REF), + input_payload=_r3_input_payload( + user_question=user_question, + r1=r1, + r2=r2, + selected_record=selected_record, + ), + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + prompt_ref=R3_VESSEL_INSPECTOR_PROMPT_REF, + input_ref=_unique_strings([*source_trace_ids, *trace_event_ids]), + source_data_ids=[read_packet.packet_id, r1.frame_id, r2.frame_id, r2.selected_graph_node_id], + payload_validator=_validate_r3_payload, + ) + _append_llm_refs(r3_result, llm_call_data_ids, trace_event_ids) + if r3_result.failure_type != "none" or r3_result.validation.payload is None: + return _record_traverse_failure_result( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + frame_id=result_frame_id, + created_at=_now_from_trace(trace_store), + source_packet_id=read_packet.packet_id, + failure_stage=f"R3:step_{step_index:04d}", + failure_type=r3_result.failure_type, + failure_reason=r3_result.validation.error or "R3 payload validation failed.", + source_data_ids=_unique_strings( + [ + read_packet.packet_id, + r1.frame_id, + candidate_layer_surface.frame_id, + r2.frame_id, + r3_result.call_data_id, + ] + ), + source_trace_ids=_unique_strings([*source_trace_ids, *trace_event_ids, r3_result.trace_event_id]), + llm_call_data_ids=llm_call_data_ids, + r1_goal_frame_id=r1.frame_id, + output_data_ids=output_data_ids, + candidate_layer_surfaces=candidate_layer_surfaces, + surface_selections=surface_selections, + r2_selections=r2_selections, + r3_inspections=r3_inspections, + graph_surfaces=graph_surfaces, + continuations=continuations, + ) + + r3 = _r3_frame_from_payload( + payload=r3_result.validation.payload, + frame_label=step_label, + selected_record=selected_record, + read_packet=read_packet, + r2=r2, + model_id=r3_result.model_id, + llm_call_data_id=r3_result.call_data_id, + source_trace_ids=_unique_strings([*source_trace_ids, r3_result.trace_event_id]), + ) + r3_inspections.append(r3) + if selected_is_terminal_material: + terminal_material_seen_count += 1 + if selected_is_raw_original_material: + raw_original_material_seen_count += 1 + raw_original_read_cap_reached = ( + raw_original_material_seen_count >= max_raw_original_material_reads + ) + _record_frame( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + actor="R3", + data_type="node_output:R3_graph_inspection_frame", + frame_id=r3.frame_id, + payload=asdict(r3), + input_ref=_unique_strings([*source_trace_ids, r3_result.trace_event_id]), + trace_event_ids=trace_event_ids, + output_data_ids=output_data_ids, + ) + + graph_surface = _graph_traversal_candidate_surface_frame( + frame_label=step_label, + r3=r3, + selected_record=selected_record, + ) + graph_surfaces.append(graph_surface) + _record_frame( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + actor="R:graph_traversal_candidate_surface", + data_type="node_output:R_graph_traversal_candidate_surface_frame", + frame_id=graph_surface.frame_id, + payload=asdict(graph_surface), + input_ref=[r3.frame_id], + trace_event_ids=trace_event_ids, + output_data_ids=output_data_ids, + ) + + final_budget = _traverse_budget_frame( + r1=r1, + frame_label=frame_label, + step_index=step_index, + ) + _record_frame( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + actor="R:budget", + data_type="node_output:R_loop_budget_frame", + frame_id=final_budget.frame_id, + payload=asdict(final_budget), + input_ref=[r1.frame_id, r3.frame_id], + trace_event_ids=trace_event_ids, + output_data_ids=output_data_ids, + ) + + final_continuation = decide_r_loop_continuation( + frame_id=f"R:{step_label}:vessel_traverse_continuation_frame", + r3_inspection=r3, + budget=final_budget, + source_trace_ids=_unique_strings([*source_trace_ids, *trace_event_ids]), + ) + if _should_force_deeper_for_terminal_material( + continuation=final_continuation, + graph_surface=graph_surface, + terminal_material_seen_count=terminal_material_seen_count, + min_terminal_material_count=R_TRAVERSE_MIN_TERMINAL_MATERIAL_READS, + ): + final_continuation = _terminal_material_guard_continuation_frame( + frame_id=final_continuation.frame_id, + r3=r3, + budget=final_budget, + source_trace_ids=_unique_strings([*source_trace_ids, *trace_event_ids]), + ) + early_stop_guard_trigger_count += 1 + if _should_stop_for_raw_original_cap( + continuation=final_continuation, + raw_original_material_seen_count=raw_original_material_seen_count, + max_raw_original_material_count=max_raw_original_material_reads, + ): + final_continuation = _raw_original_cap_continuation_frame( + frame_id=final_continuation.frame_id, + r3=r3, + budget=final_budget, + source_trace_ids=_unique_strings([*source_trace_ids, *trace_event_ids]), + ) + raw_original_read_cap_reached = True + continuations.append(final_continuation) + _record_frame( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + actor="R:continuation", + data_type="node_output:R_loop_continuation_frame", + frame_id=final_continuation.frame_id, + payload=asdict(final_continuation), + input_ref=[r3.frame_id, final_budget.frame_id], + trace_event_ids=trace_event_ids, + output_data_ids=output_data_ids, + ) + + if final_continuation.continuation_status == "continue_deeper": + next_candidate_surface = _candidate_layer_surface_frame_from_graph_surface( + frame_label=f"{frame_label}_step_{step_index + 1:04d}", + read_packet=read_packet, + graph_surface=graph_surface, + created_at=_now_from_trace(trace_store), + ) + if next_candidate_surface.total_candidate_count <= 0: + return_summary = _return_summary_for_traverse( + frame_label=frame_label, + read_packet=read_packet, + r1=r1, + final_budget=final_budget, + r2_selections=r2_selections, + r3_inspections=r3_inspections, + final_continuation=final_continuation, + forced_status="partial", + forced_continuation_status="stop_no_actionable_path", + ) + break + continue + + return_summary = _return_summary_for_traverse( + frame_label=frame_label, + read_packet=read_packet, + r1=r1, + final_budget=final_budget, + r2_selections=r2_selections, + r3_inspections=r3_inspections, + final_continuation=final_continuation, + ) + break + + if return_summary is None: + final_budget = final_budget or _traverse_budget_frame( + r1=r1, + frame_label=frame_label, + step_index=max_node_reads, + ) + return_summary = _return_summary_for_traverse( + frame_label=frame_label, + read_packet=read_packet, + r1=r1, + final_budget=final_budget, + r2_selections=r2_selections, + r3_inspections=r3_inspections, + final_continuation=final_continuation, + forced_status="partial", + forced_continuation_status="stop_budget_exhausted", + ) + + _record_frame( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + actor="R:return_summary", + data_type="node_output:R_loop_return_summary_frame", + frame_id=return_summary.frame_id, + payload=asdict(return_summary), + input_ref=[continuations[-1].frame_id] if continuations else [r1.frame_id], + trace_event_ids=trace_event_ids, + output_data_ids=output_data_ids, + ) + + result_frame = _traverse_result_frame( + frame_id=result_frame_id, + created_at=_now_from_trace(trace_store), + traverse_status="completed", + source_packet_id=read_packet.packet_id, + r1=r1, + final_budget=final_budget, + return_summary=return_summary, + candidate_layer_surfaces=candidate_layer_surfaces, + graph_surfaces=graph_surfaces, + r2_selections=r2_selections, + r3_inspections=r3_inspections, + continuations=continuations, + llm_call_data_ids=llm_call_data_ids, + output_data_ids=output_data_ids, + source_data_ids=[read_packet.packet_id], + source_trace_ids=_unique_strings([*source_trace_ids, *trace_event_ids]), + terminal_material_seen_count=terminal_material_seen_count, + min_terminal_material_count=R_TRAVERSE_MIN_TERMINAL_MATERIAL_READS, + raw_original_material_seen_count=raw_original_material_seen_count, + max_raw_original_material_count=max_raw_original_material_reads, + raw_original_read_cap_reached=raw_original_read_cap_reached, + early_stop_guard_trigger_count=early_stop_guard_trigger_count, + ) + _record_traverse_result_frame( + trace_store, + data_store, + turn_id, + result_frame, + trace_event_ids, + output_data_ids, + ) + return RLoopVesselTraverseRun( + result_frame=result_frame, + r1_goal=r1, + final_budget=final_budget, + r2_selections=r2_selections, + r3_inspections=r3_inspections, + continuations=continuations, + return_summary=return_summary, + candidate_layer_surfaces=candidate_layer_surfaces, + surface_selections=surface_selections, + graph_traversal_candidate_surfaces=graph_surfaces, + trace_event_ids=trace_event_ids, + output_data_ids=output_data_ids, + ) + + +def _r1_input_payload( + *, + user_question: str, + read_packet: RLoopVesselReadPacketFrame, + policy_name: str = "one_step", + max_traversal_depth: int = R_ONE_STEP_MAX_TRAVERSAL_DEPTH, + max_branch_switches: int = R_ONE_STEP_MAX_BRANCH_SWITCHES, + max_node_reads: int = R_ONE_STEP_MAX_NODE_READS, + max_context_tokens: int = R_ONE_STEP_MAX_CONTEXT_TOKENS, +) -> dict[str, object]: + # R1 is a goal setter. Candidate text belongs to R2/R3, not to R1. + anchor_id = _user_question_anchor_id(user_question) + return { + "user_question": user_question, + "user_question_anchor": { + "anchor_id": anchor_id, + "source_field": "user_question", + "copy_required": True, + }, + "read_packet_id": read_packet.packet_id, + "entry_candidate_count": read_packet.entry_candidate_count, + "summary_candidate_count": read_packet.summary_candidate_count, + "summary_count_by_data_kind": read_packet.summary_count_by_data_kind, + "summary_count_by_depth": read_packet.summary_count_by_depth, + "traversal_policy": { + "policy_name": policy_name, + "max_traversal_depth": max_traversal_depth, + "max_branch_switches": max_branch_switches, + "max_node_reads": max_node_reads, + "max_context_tokens": max_context_tokens, + }, + "candidate_text_visibility_policy": ( + "R1 does not receive candidate IDs, summary text, or summary previews. " + "R1 must copy user_question_anchor.anchor_id exactly into user_question_anchor_id " + "and set graph_search_goal from the user question plus packet-level counts only." + ), + "source_data_ids": [read_packet.packet_id], + } + + +def _r2_input_payload( + *, + user_question: str, + read_packet: RLoopVesselReadPacketFrame, + r1: R1GraphGoalFrame, + candidate_layer_surface: RLoopVesselCandidateLayerSurfaceFrame, + available_graph_node_ids: list[str], + current_graph_node_id: str = "graph:core_ego:root", + traversal_policy: str = "core_ego_direct_entry_candidates_only", +) -> dict[str, object]: + selection_ref_map = _r2_selection_ref_map(read_packet, candidate_layer_surface) + return { + "user_question": user_question, + "r1_goal": asdict(r1), + "read_packet_id": read_packet.packet_id, + "candidate_layer_surface_ref_records": selection_ref_map["surface_records"], + "available_surface_refs": selection_ref_map["available_surface_refs"], + "candidate_records_by_surface_ref": selection_ref_map["candidate_records_by_surface_ref"], + "selection_ref_contract": { + "surface_output_field": "selected_surface_ref", + "node_output_field": "selected_node_ref", + "surface_ref_source": "available_surface_refs", + "node_ref_source": "candidate_records_by_surface_ref[*][*].node_ref", + "actual_graph_ids_hidden_from_r2": True, + "current_graph_node_id": current_graph_node_id, + "first_step_policy": traversal_policy, + }, + "source_data_ids": [read_packet.packet_id, r1.frame_id, candidate_layer_surface.frame_id], + } + + +def _r3_input_payload( + *, + user_question: str, + r1: R1GraphGoalFrame, + r2: R2GraphNodeSelectionFrame, + selected_record: dict[str, object], +) -> dict[str, object]: + return { + "user_question": user_question, + "r1_goal": asdict(r1), + "r2_selection": asdict(r2), + "selected_candidate_record": selected_record, + "known_child_node_ids": _candidate_child_node_ids(selected_record), + "hierarchy_child_candidate_records": _hierarchy_child_candidate_record_views(selected_record), + "hierarchy_child_candidate_count": len(_candidate_child_node_ids(selected_record)), + "hierarchy_read_policy": { + "child_candidates_generated_by": "CODE:R_VESSEL_HIERARCHY_CHILD_CANDIDATE_BUILDER", + "child_candidates_info_class": "absolute", + "llm_may_judge_sufficiency": True, + "llm_must_not_invent_child_ids": True, + }, + "source_data_ids": [r1.frame_id, r2.frame_id, _record_node_id(selected_record)], + } + + +def _summary_candidate_view(records: list[dict[str, object]]) -> list[dict[str, object]]: + view: list[dict[str, object]] = [] + for record in records: + view.append(_summary_candidate_record_view(record)) + return view + + +def _summary_candidate_record_view(record: dict[str, object]) -> dict[str, object]: + return { + "summary_display_name": record.get("summary_display_name"), + "data_kind": record.get("data_kind"), + "branch_role": _branch_role_for_record(record), + "summary_depth": record.get("summary_depth"), + "info_class": record.get("info_class"), + "target_display_name": record.get("target_display_name"), + "target_node_kind": record.get("target_node_kind"), + "source_leaf_count": record.get("source_leaf_count"), + "source_summary_count": record.get("source_summary_count"), + "summary_text": record.get("summary_text"), + "summary_text_char_count": record.get("summary_text_char_count"), + } + + +def _validate_r1_payload( + payload: dict[str, object], + *, + read_packet: RLoopVesselReadPacketFrame, + user_question: str, +) -> None: + granularity = _payload_text(payload, "required_information_granularity") + if granularity not in {"raw", "low_summary", "medium_summary", "high_summary", "unknown"}: + raise ValueError("R1 required_information_granularity is invalid") + allowed_depth = _payload_int(payload, "allowed_summary_depth") + if allowed_depth < 0: + raise ValueError("R1 allowed_summary_depth must not be negative") + max_seen_depth = _max_summary_depth(read_packet) + if max_seen_depth is not None and allowed_depth > max_seen_depth: + raise ValueError("R1 allowed_summary_depth must not exceed supplied summary depth") + if not _payload_text(payload, "graph_search_goal"): + raise ValueError("R1 graph_search_goal must not be empty") + if not _payload_text(payload, "stop_condition"): + raise ValueError("R1 stop_condition must not be empty") + expected_anchor_id = _user_question_anchor_id(user_question) + if _payload_text(payload, "user_question_anchor_id") != expected_anchor_id: + raise ValueError("R1 user_question_anchor_id must copy the supplied user question anchor") + + +def _validate_r2_payload( + payload: dict[str, object], + *, + available_graph_node_ids: list[str], + candidate_layer_surface: RLoopVesselCandidateLayerSurfaceFrame, +) -> None: + selection_ref_map = _r2_selection_ref_map_from_surface(candidate_layer_surface) + status = _payload_text(payload, "selection_status") + if status not in {"selected", "none_selected"}: + raise ValueError("R2 selection_status must be selected or none_selected") + selected_surface_ref = _selected_surface_ref_from_r2_payload(payload) + selected_node_ref = _selected_node_ref_from_r2_payload(payload) + selected_surface_id = _selected_surface_id_from_r2_payload( + payload, + selection_ref_map=selection_ref_map, + ) + selected_id = _selected_graph_node_id_from_r2_payload( + payload, + selection_ref_map=selection_ref_map, + ) + if status == "selected": + if selected_surface_ref is not None and selected_surface_id is None: + raise ValueError("R2 selected_surface_ref must be in available_surface_refs") + if selected_node_ref is not None and selected_id is None: + raise ValueError("R2 selected_node_ref must be in available_node_refs") + if not isinstance(selected_surface_id, str) or not selected_surface_id: + raise ValueError("selected R2 payload must include selected_surface_ref") + if selected_surface_id not in candidate_layer_surface.available_surface_ids: + raise ValueError("R2 selected_surface_id must be in available_surface_ids") + if not isinstance(selected_id, str) or not selected_id: + raise ValueError("selected R2 payload must include selected_node_ref") + if selected_id not in available_graph_node_ids: + raise ValueError("R2 selected_graph_node_id must be in available_graph_node_ids") + surface_candidate_ids = _surface_candidate_ids(candidate_layer_surface, selected_surface_id) + if selected_id not in surface_candidate_ids: + raise ValueError("R2 selected_graph_node_id must belong to selected_surface_id") + elif selected_id is not None or selected_node_ref is not None: + raise ValueError("none_selected R2 payload must set selected_node_ref to null") + elif selected_surface_id is not None or selected_surface_ref is not None: + raise ValueError("none_selected R2 payload must set selected_surface_ref to null") + granularity = _payload_text(payload, "expected_information_granularity") + if granularity not in {"raw", "low_summary", "medium_summary", "high_summary", "unknown"}: + raise ValueError("R2 expected_information_granularity is invalid") + if not _payload_text(payload, "selection_reason"): + raise ValueError("R2 selection_reason must not be empty") + if not _payload_text(payload, "expected_source_kind"): + raise ValueError("R2 expected_source_kind must not be empty") + + +def _validate_r3_payload(payload: dict[str, object]) -> None: + if _payload_text(payload, "current_information_granularity") not in { + "raw", + "low_summary", + "medium_summary", + "high_summary", + "unknown", + }: + raise ValueError("R3 current_information_granularity is invalid") + if _payload_text(payload, "sufficiency_status") not in { + "sufficient", + "insufficient", + "unknown", + }: + raise ValueError("R3 sufficiency_status is invalid") + if _payload_text(payload, "granularity_problem_status") not in { + "none", + "needs_lower_granularity", + "unknown", + }: + raise ValueError("R3 granularity_problem_status is invalid") + if _payload_text(payload, "branch_problem_status") not in { + "none", + "wrong_branch", + "unknown", + }: + raise ValueError("R3 branch_problem_status is invalid") + if _payload_text(payload, "recommended_next_action") not in { + "stop", + "deeper", + "switch_branch", + "fail", + }: + raise ValueError("R3 recommended_next_action is invalid") + if not _payload_text(payload, "inspection_reason"): + raise ValueError("R3 inspection_reason must not be empty") + + +def _r2_failure_payload_summary( + payload: dict[str, object] | None, + *, + available_graph_node_ids: list[str], + candidate_layer_surface: RLoopVesselCandidateLayerSurfaceFrame, +) -> dict[str, object] | None: + if not isinstance(payload, dict): + return None + selection_ref_map = _r2_selection_ref_map_from_surface(candidate_layer_surface) + selected_surface_ref = _selected_surface_ref_from_r2_payload(payload) + selected_node_ref = _selected_node_ref_from_r2_payload(payload) + selected_surface_id = _selected_surface_id_from_r2_payload( + payload, + selection_ref_map=selection_ref_map, + ) + selected_graph_node_id = _selected_graph_node_id_from_r2_payload( + payload, + selection_ref_map=selection_ref_map, + ) + surface_candidate_ids = _surface_candidate_ids( + candidate_layer_surface, + selected_surface_id, + ) + available_surface_refs = selection_ref_map.get("available_surface_refs") + node_ref_to_graph_node_id = selection_ref_map.get("node_ref_to_graph_node_id") + return { + "selection_status": payload.get("selection_status"), + "selected_surface_ref": selected_surface_ref, + "selected_node_ref": selected_node_ref, + "selected_surface_id": selected_surface_id, + "selected_graph_node_id": selected_graph_node_id, + "selected_surface_ref_in_available": selected_surface_ref in available_surface_refs + if isinstance(available_surface_refs, list) and selected_surface_ref is not None + else False, + "selected_node_ref_in_available": selected_node_ref in node_ref_to_graph_node_id + if isinstance(node_ref_to_graph_node_id, dict) and selected_node_ref is not None + else False, + "selected_surface_id_in_available": selected_surface_id + in candidate_layer_surface.available_surface_ids + if selected_surface_id is not None + else False, + "selected_graph_node_id_in_available": selected_graph_node_id + in available_graph_node_ids + if selected_graph_node_id is not None + else False, + "selected_graph_node_id_in_selected_surface": selected_graph_node_id + in surface_candidate_ids + if selected_graph_node_id is not None + else False, + "selected_surface_candidate_count": len(surface_candidate_ids), + } + + +def _r1_frame_from_payload( + *, + payload: dict[str, object], + frame_label: str, + read_packet: RLoopVesselReadPacketFrame, + model_id: str, + llm_call_data_id: str | None, + source_trace_ids: list[str], + max_traversal_depth: int = R_ONE_STEP_MAX_TRAVERSAL_DEPTH, + max_branch_switches: int = R_ONE_STEP_MAX_BRANCH_SWITCHES, + max_node_reads: int = R_ONE_STEP_MAX_NODE_READS, + max_context_tokens: int = R_ONE_STEP_MAX_CONTEXT_TOKENS, +) -> R1GraphGoalFrame: + frame = R1GraphGoalFrame( + frame_id=f"R1:{frame_label}:vessel_goal_frame", + graph_search_goal=_payload_text(payload, "graph_search_goal"), + required_information_granularity=_payload_text(payload, "required_information_granularity"), + allowed_summary_depth=_payload_int(payload, "allowed_summary_depth"), + max_traversal_depth=max_traversal_depth, + max_branch_switches=max_branch_switches, + max_node_reads=max_node_reads, + max_context_tokens=max_context_tokens, + stop_condition=_payload_text(payload, "stop_condition"), + source_graph_guide_packet_id=read_packet.packet_id, + user_question_anchor_id=_payload_text(payload, "user_question_anchor_id"), + source_data_ids=_unique_strings([read_packet.packet_id, llm_call_data_id]), + source_trace_ids=source_trace_ids, + generated_by=f"LLM:{model_id}:R1_vessel_goal_setter", + info_class="mixed", + semantic_judgement_status="ran", + ) + validate_r1_graph_goal_frame(frame) + return frame + + +def _budget_frame(*, r1: R1GraphGoalFrame, frame_label: str) -> RLoopBudgetFrame: + frame = RLoopBudgetFrame( + frame_id=f"R:{frame_label}:vessel_one_step_budget_frame", + source_r1_goal_frame_id=r1.frame_id, + max_traversal_depth=r1.max_traversal_depth, + max_branch_switches=r1.max_branch_switches, + max_node_reads=r1.max_node_reads, + max_context_tokens=r1.max_context_tokens, + used_traversal_depth=0, + used_branch_switches=0, + used_node_reads=1, + used_context_tokens=0, + budget_status="within_budget", + source_data_ids=[r1.frame_id], + source_trace_ids=list(r1.source_trace_ids), + generated_by=R_LOOP_VESSEL_ONE_STEP_GENERATOR, + info_class="absolute", + semantic_judgement_status="not_run", + ) + validate_r_loop_budget_frame(frame) + return frame + + +def _traverse_budget_frame( + *, + r1: R1GraphGoalFrame, + frame_label: str, + step_index: int, +) -> RLoopBudgetFrame: + safe_step = max(0, step_index) + frame = RLoopBudgetFrame( + frame_id=f"R:{frame_label}:vessel_traverse_budget_frame:{safe_step:04d}", + source_r1_goal_frame_id=r1.frame_id, + max_traversal_depth=r1.max_traversal_depth, + max_branch_switches=r1.max_branch_switches, + max_node_reads=r1.max_node_reads, + max_context_tokens=r1.max_context_tokens, + used_traversal_depth=safe_step, + used_branch_switches=0, + used_node_reads=safe_step, + used_context_tokens=0, + budget_status="within_budget", + source_data_ids=[r1.frame_id], + source_trace_ids=list(r1.source_trace_ids), + generated_by=R_LOOP_VESSEL_TRAVERSE_GENERATOR, + info_class="absolute", + semantic_judgement_status="not_run", + ) + validate_r_loop_budget_frame(frame) + return frame + + +def _should_force_deeper_for_terminal_material( + *, + continuation: RLoopContinuationFrame, + graph_surface: RGraphTraversalCandidateSurfaceFrame, + terminal_material_seen_count: int, + min_terminal_material_count: int, +) -> bool: + if continuation.continuation_status != "stop_sufficient": + return False + if terminal_material_seen_count >= min_terminal_material_count: + return False + if graph_surface.candidate_count <= 0: + return False + if continuation.remaining_node_reads <= 0: + return False + if continuation.remaining_traversal_depth <= 0: + return False + if continuation.remaining_context_tokens <= 0: + return False + return True + + +def _terminal_material_guard_continuation_frame( + *, + frame_id: str, + r3: R3GraphInspectionFrame, + budget: RLoopBudgetFrame, + source_trace_ids: list[str], +) -> RLoopContinuationFrame: + remaining_traversal_depth = max( + budget.max_traversal_depth - budget.used_traversal_depth, + 0, + ) + remaining_branch_switches = max( + budget.max_branch_switches - budget.used_branch_switches, + 0, + ) + remaining_node_reads = max(budget.max_node_reads - budget.used_node_reads, 0) + remaining_context_tokens = max( + budget.max_context_tokens - budget.used_context_tokens, + 0, + ) + frame = RLoopContinuationFrame( + frame_id=frame_id, + source_r3_inspection_frame_id=r3.frame_id, + source_budget_frame_id=budget.frame_id, + continuation_status="continue_deeper", + continuation_reason_code="CODE_STATUS:r_loop_terminal_material_not_seen", + next_target_node="R2", + remaining_traversal_depth=remaining_traversal_depth, + remaining_branch_switches=remaining_branch_switches, + remaining_node_reads=remaining_node_reads, + remaining_context_tokens=remaining_context_tokens, + source_data_ids=_unique_strings( + [ + r3.frame_id, + budget.frame_id, + *r3.source_data_ids, + *budget.source_data_ids, + ] + ), + source_trace_ids=_unique_strings( + [ + *source_trace_ids, + *r3.source_trace_ids, + *budget.source_trace_ids, + ] + ), + ) + validate_r_loop_continuation_frame(frame) + return frame + + +def _should_stop_for_raw_original_cap( + *, + continuation: RLoopContinuationFrame, + raw_original_material_seen_count: int, + max_raw_original_material_count: int, +) -> bool: + if raw_original_material_seen_count < max_raw_original_material_count: + return False + return continuation.continuation_status in { + "continue_deeper", + "continue_switch_branch", + } + + +def _raw_original_cap_continuation_frame( + *, + frame_id: str, + r3: R3GraphInspectionFrame, + budget: RLoopBudgetFrame, + source_trace_ids: list[str], +) -> RLoopContinuationFrame: + remaining_traversal_depth = max( + budget.max_traversal_depth - budget.used_traversal_depth, + 0, + ) + remaining_branch_switches = max( + budget.max_branch_switches - budget.used_branch_switches, + 0, + ) + remaining_node_reads = max(budget.max_node_reads - budget.used_node_reads, 0) + remaining_context_tokens = max( + budget.max_context_tokens - budget.used_context_tokens, + 0, + ) + frame = RLoopContinuationFrame( + frame_id=frame_id, + source_r3_inspection_frame_id=r3.frame_id, + source_budget_frame_id=budget.frame_id, + continuation_status="stop_budget_exhausted", + continuation_reason_code="CODE_STATUS:r_loop_raw_original_read_cap_reached", + next_target_node="return_summary", + remaining_traversal_depth=remaining_traversal_depth, + remaining_branch_switches=remaining_branch_switches, + remaining_node_reads=remaining_node_reads, + remaining_context_tokens=remaining_context_tokens, + source_data_ids=_unique_strings( + [ + r3.frame_id, + budget.frame_id, + *r3.source_data_ids, + *budget.source_data_ids, + ] + ), + source_trace_ids=_unique_strings( + [ + *source_trace_ids, + *r3.source_trace_ids, + *budget.source_trace_ids, + ] + ), + ) + validate_r_loop_continuation_frame(frame) + return frame + + +def _surface_selection_frame_from_payload( + *, + payload: dict[str, object], + frame_label: str, + r1: R1GraphGoalFrame, + candidate_layer_surface: RLoopVesselCandidateLayerSurfaceFrame, + model_id: str, + llm_call_data_id: str | None, + source_trace_ids: list[str], +) -> RLoopVesselSurfaceSelectionFrame: + selected_surface_id = _selected_surface_id_from_r2_payload( + payload, + selection_ref_map=_r2_selection_ref_map_from_surface(candidate_layer_surface), + ) + frame = RLoopVesselSurfaceSelectionFrame( + frame_id=f"R2:{frame_label}:vessel_surface_selection_frame", + selection_scope="r_loop_vessel_candidate_layer_surface", + available_surface_ids=list(candidate_layer_surface.available_surface_ids), + selection_status=_payload_text(payload, "selection_status"), + selected_surface_id=selected_surface_id, + selected_surface_candidate_count=len( + _surface_candidate_ids(candidate_layer_surface, selected_surface_id) + ) + if selected_surface_id + else 0, + surface_selection_reason=_payload_text(payload, "selection_reason"), + source_r1_goal_frame_id=r1.frame_id, + source_surface_frame_id=candidate_layer_surface.frame_id, + source_data_ids=_unique_strings( + [candidate_layer_surface.frame_id, r1.frame_id, llm_call_data_id] + ), + source_trace_ids=source_trace_ids, + generated_by=f"LLM:{model_id}:R2_vessel_node_selector", + ) + _validate_surface_selection_frame(frame) + return frame + + +def _r2_frame_from_payload( + *, + payload: dict[str, object], + frame_label: str, + read_packet: RLoopVesselReadPacketFrame, + r1: R1GraphGoalFrame, + candidate_layer_surface: RLoopVesselCandidateLayerSurfaceFrame, + available_graph_node_ids: list[str], + model_id: str, + llm_call_data_id: str | None, + source_trace_ids: list[str], +) -> R2GraphNodeSelectionFrame: + selected_id = _selected_graph_node_id_from_r2_payload( + payload, + selection_ref_map=_r2_selection_ref_map_from_surface(candidate_layer_surface), + ) + frame = R2GraphNodeSelectionFrame( + frame_id=f"R2:{frame_label}:vessel_node_selection_frame", + selection_scope="r_loop_vessel_read_packet", + available_graph_node_ids=available_graph_node_ids, + selection_status=_payload_text(payload, "selection_status"), + selected_graph_node_id=selected_id if isinstance(selected_id, str) else None, + selection_reason=_payload_text(payload, "selection_reason"), + expected_information_granularity=_payload_text(payload, "expected_information_granularity"), + expected_source_kind=_payload_text(payload, "expected_source_kind"), + source_r1_goal_frame_id=r1.frame_id, + source_data_ids=_unique_strings([read_packet.packet_id, r1.frame_id, llm_call_data_id]), + source_trace_ids=source_trace_ids, + generated_by=f"LLM:{model_id}:R2_vessel_node_selector", + info_class="mixed", + semantic_judgement_status="ran", + ) + validate_r2_graph_node_selection_frame(frame) + return frame + + +def _r3_frame_from_payload( + *, + payload: dict[str, object], + frame_label: str, + selected_record: dict[str, object], + read_packet: RLoopVesselReadPacketFrame, + r2: R2GraphNodeSelectionFrame, + model_id: str, + llm_call_data_id: str | None, + source_trace_ids: list[str], +) -> R3GraphInspectionFrame: + selected_node_id = _record_node_id(selected_record) + child_node_ids = _candidate_child_node_ids(selected_record) + frame = R3GraphInspectionFrame( + frame_id=f"R3:{frame_label}:vessel_inspection_frame", + inspected_graph_node_id=selected_node_id, + node_kind=_candidate_node_kind(selected_record), + child_node_count=len(child_node_ids), + child_node_ids=child_node_ids, + summary_depth=_candidate_summary_depth(selected_record), + source_leaf_count=_candidate_source_leaf_count(selected_record), + current_information_granularity=_payload_text(payload, "current_information_granularity"), + sufficiency_status=_payload_text(payload, "sufficiency_status"), + granularity_problem_status=_payload_text(payload, "granularity_problem_status"), + branch_problem_status=_payload_text(payload, "branch_problem_status"), + recommended_next_action=_payload_text(payload, "recommended_next_action"), + inspection_reason=_payload_text(payload, "inspection_reason"), + source_r2_selection_frame_id=r2.frame_id, + source_data_ids=_unique_strings([read_packet.packet_id, r2.frame_id, selected_node_id, llm_call_data_id]), + source_trace_ids=source_trace_ids, + generated_by=f"LLM:{model_id}:R3_vessel_inspector", + info_class="mixed", + semantic_judgement_status="ran", + ) + validate_r3_graph_inspection_frame(frame) + return frame + + +def _graph_traversal_candidate_surface_frame( + *, + frame_label: str, + r3: R3GraphInspectionFrame, + selected_record: dict[str, object], +) -> RGraphTraversalCandidateSurfaceFrame: + child_candidate_node_ids = _unique_strings(list(r3.child_node_ids)) + candidate_records = [ + { + "candidate_node_id": child_id, + "relation": "child", + "source_id": r3.frame_id, + "source_field": "hierarchy_child_node_ids", + } + for child_id in child_candidate_node_ids + ] + source_data_ids = _unique_strings( + [ + r3.frame_id, + r3.inspected_graph_node_id, + *child_candidate_node_ids, + *_hierarchy_child_record_source_ids(selected_record), + ] + ) + frame = RGraphTraversalCandidateSurfaceFrame( + frame_id=f"R:{frame_label}:vessel_graph_traversal_candidate_surface_frame", + source_r3_inspection_frame_id=r3.frame_id, + inspected_graph_node_id=r3.inspected_graph_node_id, + child_candidate_node_ids=child_candidate_node_ids, + next_candidate_node_ids=[], + previous_candidate_node_ids=[], + candidate_graph_node_ids=child_candidate_node_ids, + candidate_records=candidate_records, + candidate_count=len(child_candidate_node_ids), + source_data_ids=source_data_ids, + source_trace_ids=list(r3.source_trace_ids), + ) + validate_r_graph_traversal_candidate_surface_frame(frame) + return frame + + +def _hierarchy_child_record_source_ids(selected_record: dict[str, object]) -> list[str]: + records = selected_record.get("hierarchy_child_candidate_records") + if not isinstance(records, list): + return [] + source_ids: list[str] = [] + for record in records: + if not isinstance(record, dict): + continue + candidate_id = record.get("candidate_node_id") + if isinstance(candidate_id, str): + source_ids.append(candidate_id) + return _unique_strings(source_ids) + + +def _return_summary_for_no_selection( + *, + frame_label: str, + read_packet: RLoopVesselReadPacketFrame, + r1: R1GraphGoalFrame, + budget: RLoopBudgetFrame, + r2: R2GraphNodeSelectionFrame, +) -> RLoopReturnSummaryFrame: + frame = RLoopReturnSummaryFrame( + frame_id=f"R:{frame_label}:vessel_one_step_return_summary_frame", + r_loop_task_status="partial", + selected_entry_node_ids=[], + inspected_graph_node_ids=[], + final_information_granularity="unknown", + summary_depth_used=0, + continuation_status="stop_no_actionable_path", + budget_status=budget.budget_status, + source_graph_node_ids=[], + source_data_ids=[read_packet.packet_id, r1.frame_id, budget.frame_id, r2.frame_id], + source_trace_ids=_unique_strings([*read_packet.source_trace_ids, *r2.source_trace_ids]), + generated_by=R_LOOP_VESSEL_ONE_STEP_GENERATOR, + info_class="absolute", + semantic_judgement_status="not_run", + ) + validate_r_loop_return_summary_frame(frame) + return frame + + +def _return_summary_for_completed_step( + *, + frame_label: str, + read_packet: RLoopVesselReadPacketFrame, + r1: R1GraphGoalFrame, + budget: RLoopBudgetFrame, + r2: R2GraphNodeSelectionFrame, + r3: R3GraphInspectionFrame, + continuation: RLoopContinuationFrame, +) -> RLoopReturnSummaryFrame: + task_status = "sufficient" if continuation.continuation_status == "stop_sufficient" else "partial" + if continuation.continuation_status == "stop_failed_final": + task_status = "failed" + frame = RLoopReturnSummaryFrame( + frame_id=f"R:{frame_label}:vessel_one_step_return_summary_frame", + r_loop_task_status=task_status, + selected_entry_node_ids=[r2.selected_graph_node_id] if r2.selected_graph_node_id else [], + inspected_graph_node_ids=[r3.inspected_graph_node_id], + final_information_granularity=r3.current_information_granularity, + summary_depth_used=r3.summary_depth, + continuation_status=continuation.continuation_status, + budget_status=budget.budget_status, + source_graph_node_ids=_unique_strings( + [r2.selected_graph_node_id, r3.inspected_graph_node_id, *r3.child_node_ids] + ), + source_data_ids=_unique_strings( + [ + read_packet.packet_id, + r1.frame_id, + budget.frame_id, + r2.frame_id, + r3.frame_id, + continuation.frame_id, + r2.selected_graph_node_id, + ] + ), + source_trace_ids=_unique_strings( + [ + *read_packet.source_trace_ids, + *r1.source_trace_ids, + *r2.source_trace_ids, + *r3.source_trace_ids, + *continuation.source_trace_ids, + ] + ), + generated_by=R_LOOP_VESSEL_ONE_STEP_GENERATOR, + info_class="absolute", + semantic_judgement_status="not_run", + ) + validate_r_loop_return_summary_frame(frame) + return frame + + +def _return_summary_for_traverse( + *, + frame_label: str, + read_packet: RLoopVesselReadPacketFrame, + r1: R1GraphGoalFrame, + final_budget: RLoopBudgetFrame, + r2_selections: list[R2GraphNodeSelectionFrame], + r3_inspections: list[R3GraphInspectionFrame], + final_continuation: RLoopContinuationFrame | None, + forced_status: str | None = None, + forced_continuation_status: str | None = None, +) -> RLoopReturnSummaryFrame: + selected_ids = _unique_strings( + [r2.selected_graph_node_id for r2 in r2_selections if r2.selected_graph_node_id] + ) + inspected_ids = _unique_strings([r3.inspected_graph_node_id for r3 in r3_inspections]) + child_ids = _unique_strings( + [child_id for r3 in r3_inspections for child_id in r3.child_node_ids] + ) + continuation_status = forced_continuation_status or ( + final_continuation.continuation_status + if final_continuation is not None + else "stop_no_actionable_path" + ) + task_status = forced_status or ( + "sufficient" if continuation_status == "stop_sufficient" else "partial" + ) + if continuation_status == "stop_failed_final": + task_status = "failed" + final_r3 = r3_inspections[-1] if r3_inspections else None + frame = RLoopReturnSummaryFrame( + frame_id=f"R:{frame_label}:vessel_traverse_return_summary_frame", + r_loop_task_status=task_status, + selected_entry_node_ids=selected_ids, + inspected_graph_node_ids=inspected_ids, + final_information_granularity=( + final_r3.current_information_granularity if final_r3 is not None else "unknown" + ), + summary_depth_used=final_r3.summary_depth if final_r3 is not None else 0, + continuation_status=continuation_status, + budget_status=final_budget.budget_status, + source_graph_node_ids=_unique_strings([*selected_ids, *inspected_ids, *child_ids]), + source_data_ids=_unique_strings( + [ + read_packet.packet_id, + r1.frame_id, + final_budget.frame_id, + *[r2.frame_id for r2 in r2_selections], + *[r3.frame_id for r3 in r3_inspections], + final_continuation.frame_id if final_continuation is not None else None, + *selected_ids, + *inspected_ids, + ] + ), + source_trace_ids=_unique_strings( + [ + *read_packet.source_trace_ids, + *r1.source_trace_ids, + *final_budget.source_trace_ids, + *[ + trace_id + for r2 in r2_selections + for trace_id in r2.source_trace_ids + ], + *[ + trace_id + for r3 in r3_inspections + for trace_id in r3.source_trace_ids + ], + *( + final_continuation.source_trace_ids + if final_continuation is not None + else [] + ), + ] + ), + generated_by=R_LOOP_VESSEL_TRAVERSE_GENERATOR, + info_class="absolute", + semantic_judgement_status="not_run", + ) + validate_r_loop_return_summary_frame(frame) + return frame + + +def _record_failure_result( + *, + trace_store: TraceStore, + data_store: DataStore, + turn_id: str, + frame_id: str, + created_at: str, + source_packet_id: str, + failure_stage: str, + failure_type: str, + failure_reason: str, + source_data_ids: list[str], + source_trace_ids: list[str], + llm_call_data_ids: list[str] | None = None, + r1_goal_frame_id: str | None = None, + r2_selection_frame_id: str | None = None, + budget_frame_id: str | None = None, + selected_graph_node_id: str | None = None, + output_data_ids: list[str] | None = None, + failure_payload_summary: dict[str, object] | None = None, +) -> RLoopVesselOneStepRun: + result_frame = _result_frame( + frame_id=frame_id, + created_at=created_at, + one_step_status="failed", + failure_stage=failure_stage, + failure_type=failure_type, + failure_reason=failure_reason, + source_packet_id=source_packet_id, + r1_goal_frame_id=r1_goal_frame_id, + r2_selection_frame_id=r2_selection_frame_id, + budget_frame_id=budget_frame_id, + selected_graph_node_id=selected_graph_node_id, + llm_call_data_ids=llm_call_data_ids or [], + output_data_ids=output_data_ids or [], + source_data_ids=source_data_ids, + source_trace_ids=source_trace_ids, + failure_payload_summary=failure_payload_summary, + ) + trace_event_ids: list[str] = [] + final_output_data_ids = list(output_data_ids or []) + _record_result_frame(trace_store, data_store, turn_id, result_frame, trace_event_ids, final_output_data_ids) + return RLoopVesselOneStepRun( + result_frame=result_frame, + r1_goal=None, + budget=None, + r2_selection=None, + r3_inspection=None, + continuation=None, + return_summary=None, + trace_event_ids=trace_event_ids, + output_data_ids=final_output_data_ids, + ) + + +def _record_traverse_failure_result( + *, + trace_store: TraceStore, + data_store: DataStore, + turn_id: str, + frame_id: str, + created_at: str, + source_packet_id: str, + failure_stage: str, + failure_type: str, + failure_reason: str, + source_data_ids: list[str], + source_trace_ids: list[str], + llm_call_data_ids: list[str] | None = None, + r1_goal_frame_id: str | None = None, + output_data_ids: list[str] | None = None, + failure_payload_summary: dict[str, object] | None = None, + candidate_layer_surfaces: list[RLoopVesselCandidateLayerSurfaceFrame] | None = None, + surface_selections: list[RLoopVesselSurfaceSelectionFrame] | None = None, + r2_selections: list[R2GraphNodeSelectionFrame] | None = None, + r3_inspections: list[R3GraphInspectionFrame] | None = None, + graph_surfaces: list[RGraphTraversalCandidateSurfaceFrame] | None = None, + continuations: list[RLoopContinuationFrame] | None = None, +) -> RLoopVesselTraverseRun: + result_frame = _traverse_result_frame( + frame_id=frame_id, + created_at=created_at, + traverse_status="failed", + source_packet_id=source_packet_id, + failure_stage=failure_stage, + failure_type=failure_type, + failure_reason=failure_reason, + r1_goal_frame_id=r1_goal_frame_id, + final_budget=None, + return_summary=None, + candidate_layer_surfaces=list(candidate_layer_surfaces or []), + graph_surfaces=list(graph_surfaces or []), + r2_selections=list(r2_selections or []), + r3_inspections=list(r3_inspections or []), + continuations=list(continuations or []), + llm_call_data_ids=llm_call_data_ids or [], + output_data_ids=output_data_ids or [], + source_data_ids=source_data_ids, + source_trace_ids=source_trace_ids, + failure_payload_summary=failure_payload_summary, + ) + trace_event_ids: list[str] = [] + final_output_data_ids = list(output_data_ids or []) + _record_traverse_result_frame( + trace_store, + data_store, + turn_id, + result_frame, + trace_event_ids, + final_output_data_ids, + ) + return RLoopVesselTraverseRun( + result_frame=result_frame, + r1_goal=None, + final_budget=None, + r2_selections=list(r2_selections or []), + r3_inspections=list(r3_inspections or []), + continuations=list(continuations or []), + return_summary=None, + candidate_layer_surfaces=list(candidate_layer_surfaces or []), + surface_selections=list(surface_selections or []), + graph_traversal_candidate_surfaces=list(graph_surfaces or []), + trace_event_ids=trace_event_ids, + output_data_ids=final_output_data_ids, + ) + + +def _result_frame( + *, + frame_id: str, + created_at: str, + one_step_status: str, + source_packet_id: str, + failure_stage: str | None = None, + failure_type: str | None = None, + failure_reason: str | None = None, + r1_goal_frame_id: str | None = None, + r2_selection_frame_id: str | None = None, + r3_inspection_frame_id: str | None = None, + budget_frame_id: str | None = None, + continuation_frame_id: str | None = None, + return_summary_frame_id: str | None = None, + selected_graph_node_id: str | None = None, + inspected_graph_node_id: str | None = None, + sufficiency_status: str | None = None, + continuation_status: str | None = None, + llm_call_data_ids: list[str] | None = None, + output_data_ids: list[str] | None = None, + source_data_ids: list[str] | None = None, + source_trace_ids: list[str] | None = None, + failure_payload_summary: dict[str, object] | None = None, +) -> RLoopVesselOneStepResultFrame: + frame = RLoopVesselOneStepResultFrame( + frame_id=frame_id, + created_at=created_at, + policy_id=R_LOOP_VESSEL_ONE_STEP_POLICY_ID, + one_step_status=one_step_status, + failure_stage=failure_stage, + failure_type=failure_type, + failure_reason=failure_reason, + source_packet_id=source_packet_id, + r1_goal_frame_id=r1_goal_frame_id, + r2_selection_frame_id=r2_selection_frame_id, + r3_inspection_frame_id=r3_inspection_frame_id, + budget_frame_id=budget_frame_id, + continuation_frame_id=continuation_frame_id, + return_summary_frame_id=return_summary_frame_id, + selected_graph_node_id=selected_graph_node_id, + inspected_graph_node_id=inspected_graph_node_id, + sufficiency_status=sufficiency_status, + continuation_status=continuation_status, + llm_call_data_ids=_unique_strings(llm_call_data_ids or []), + output_data_ids=_unique_strings(output_data_ids or []), + source_data_ids=_unique_strings([source_packet_id, *(source_data_ids or [])]), + source_trace_ids=_unique_strings(source_trace_ids or []), + failure_payload_summary=failure_payload_summary, + ) + _validate_result_frame(frame) + return frame + + +def _record_result_frame( + trace_store: TraceStore, + data_store: DataStore, + turn_id: str, + result_frame: RLoopVesselOneStepResultFrame, + trace_event_ids: list[str], + output_data_ids: list[str], +) -> None: + event = trace_store.create_event( + turn_id=turn_id, + actor="R:vessel_one_step", + event_type="node_output", + input_ref=result_frame.source_trace_ids, + output_ref=[result_frame.frame_id], + schema_status="passed" if result_frame.one_step_status == "completed" else "failed", + ) + data_store.create_record( + data_id=result_frame.frame_id, + data_type=R_LOOP_VESSEL_ONE_STEP_RESULT_DATA_TYPE, + exists=True, + created_at=event.timestamp, + source_trace_id=event.event_id, + payload=asdict(result_frame), + ) + trace_event_ids.append(event.event_id) + output_data_ids.append(result_frame.frame_id) + + +def _traverse_result_frame( + *, + frame_id: str, + created_at: str, + traverse_status: str, + source_packet_id: str, + failure_stage: str | None = None, + failure_type: str | None = None, + failure_reason: str | None = None, + r1: R1GraphGoalFrame | None = None, + r1_goal_frame_id: str | None = None, + final_budget: RLoopBudgetFrame | None = None, + return_summary: RLoopReturnSummaryFrame | None = None, + candidate_layer_surfaces: list[RLoopVesselCandidateLayerSurfaceFrame] | None = None, + graph_surfaces: list[RGraphTraversalCandidateSurfaceFrame] | None = None, + r2_selections: list[R2GraphNodeSelectionFrame] | None = None, + r3_inspections: list[R3GraphInspectionFrame] | None = None, + continuations: list[RLoopContinuationFrame] | None = None, + llm_call_data_ids: list[str] | None = None, + output_data_ids: list[str] | None = None, + source_data_ids: list[str] | None = None, + source_trace_ids: list[str] | None = None, + failure_payload_summary: dict[str, object] | None = None, + terminal_material_seen_count: int = 0, + min_terminal_material_count: int = R_TRAVERSE_MIN_TERMINAL_MATERIAL_READS, + raw_original_material_seen_count: int = 0, + max_raw_original_material_count: int = R_TRAVERSE_MAX_RAW_ORIGINAL_MATERIAL_READS, + raw_original_read_cap_reached: bool = False, + early_stop_guard_trigger_count: int = 0, +) -> RLoopVesselTraverseResultFrame: + candidate_layer_surfaces = list(candidate_layer_surfaces or []) + graph_surfaces = list(graph_surfaces or []) + r2_selections = list(r2_selections or []) + r3_inspections = list(r3_inspections or []) + continuations = list(continuations or []) + selected_ids = _unique_strings( + [r2.selected_graph_node_id for r2 in r2_selections if r2.selected_graph_node_id] + ) + inspected_ids = _unique_strings([r3.inspected_graph_node_id for r3 in r3_inspections]) + final_r3 = r3_inspections[-1] if r3_inspections else None + frame = RLoopVesselTraverseResultFrame( + frame_id=frame_id, + created_at=created_at, + policy_id=R_LOOP_VESSEL_TRAVERSE_POLICY_ID, + traverse_status=traverse_status, + failure_stage=failure_stage, + failure_type=failure_type, + failure_reason=failure_reason, + source_packet_id=source_packet_id, + r1_goal_frame_id=r1.frame_id if r1 is not None else r1_goal_frame_id, + final_budget_frame_id=final_budget.frame_id if final_budget is not None else None, + return_summary_frame_id=return_summary.frame_id if return_summary is not None else None, + step_count=len(r3_inspections), + selected_graph_node_ids=selected_ids, + inspected_graph_node_ids=inspected_ids, + candidate_surface_frame_ids=[surface.frame_id for surface in candidate_layer_surfaces], + graph_traversal_candidate_surface_frame_ids=[surface.frame_id for surface in graph_surfaces], + r2_selection_frame_ids=[r2.frame_id for r2 in r2_selections], + r3_inspection_frame_ids=[r3.frame_id for r3 in r3_inspections], + continuation_frame_ids=[continuation.frame_id for continuation in continuations], + final_graph_node_id=final_r3.inspected_graph_node_id if final_r3 is not None else None, + final_sufficiency_status=final_r3.sufficiency_status if final_r3 is not None else None, + final_continuation_status=( + return_summary.continuation_status if return_summary is not None else None + ), + r_loop_task_status=return_summary.r_loop_task_status if return_summary is not None else None, + final_information_granularity=( + return_summary.final_information_granularity if return_summary is not None else None + ), + summary_depth_used=return_summary.summary_depth_used if return_summary is not None else None, + terminal_material_seen_count=terminal_material_seen_count, + min_terminal_material_count=min_terminal_material_count, + raw_original_material_seen_count=raw_original_material_seen_count, + max_raw_original_material_count=max_raw_original_material_count, + raw_original_read_cap_reached=raw_original_read_cap_reached, + early_stop_guard_trigger_count=early_stop_guard_trigger_count, + llm_call_data_ids=_unique_strings(llm_call_data_ids or []), + output_data_ids=_unique_strings(output_data_ids or []), + source_data_ids=_unique_strings( + [ + source_packet_id, + *(source_data_ids or []), + r1.frame_id if r1 is not None else r1_goal_frame_id, + final_budget.frame_id if final_budget is not None else None, + return_summary.frame_id if return_summary is not None else None, + *[surface.frame_id for surface in candidate_layer_surfaces], + *[surface.frame_id for surface in graph_surfaces], + *[r2.frame_id for r2 in r2_selections], + *[r3.frame_id for r3 in r3_inspections], + *[continuation.frame_id for continuation in continuations], + *selected_ids, + *inspected_ids, + ] + ), + source_trace_ids=_unique_strings(source_trace_ids or []), + failure_payload_summary=failure_payload_summary, + ) + _validate_traverse_result_frame(frame) + return frame + + +def _record_traverse_result_frame( + trace_store: TraceStore, + data_store: DataStore, + turn_id: str, + result_frame: RLoopVesselTraverseResultFrame, + trace_event_ids: list[str], + output_data_ids: list[str], +) -> None: + event = trace_store.create_event( + turn_id=turn_id, + actor="R:vessel_traverse", + event_type="node_output", + input_ref=result_frame.source_trace_ids, + output_ref=[result_frame.frame_id], + schema_status="passed" if result_frame.traverse_status == "completed" else "failed", + ) + data_store.create_record( + data_id=result_frame.frame_id, + data_type=R_LOOP_VESSEL_TRAVERSE_RESULT_DATA_TYPE, + exists=True, + created_at=event.timestamp, + source_trace_id=event.event_id, + payload=asdict(result_frame), + ) + trace_event_ids.append(event.event_id) + output_data_ids.append(result_frame.frame_id) + + +def _record_frame( + *, + trace_store: TraceStore, + data_store: DataStore, + turn_id: str, + actor: str, + data_type: str, + frame_id: str, + payload: dict[str, object], + input_ref: list[str], + trace_event_ids: list[str], + output_data_ids: list[str], +) -> None: + event = trace_store.create_event( + turn_id=turn_id, + actor=actor, + event_type="node_output", + input_ref=input_ref, + output_ref=[frame_id], + schema_status="passed", + ) + data_store.create_record( + data_id=frame_id, + data_type=data_type, + exists=True, + created_at=event.timestamp, + source_trace_id=event.event_id, + payload=payload, + ) + trace_event_ids.append(event.event_id) + output_data_ids.append(frame_id) + + +def _validate_result_frame(frame: RLoopVesselOneStepResultFrame) -> None: + if frame.generated_by != R_LOOP_VESSEL_ONE_STEP_GENERATOR: + raise ValueError("RLoopVesselOneStepResultFrame.generated_by must be assembler") + if frame.info_class != "absolute": + raise ValueError("RLoopVesselOneStepResultFrame.info_class must be absolute") + if frame.semantic_judgement_status != "not_run": + raise ValueError("RLoopVesselOneStepResultFrame.semantic_judgement_status must be not_run") + if frame.one_step_status not in R_LOOP_VESSEL_ONE_STEP_STATUSES: + raise ValueError("RLoopVesselOneStepResultFrame.one_step_status is invalid") + if frame.one_step_status == "failed": + if not frame.failure_stage or not frame.failure_type or not frame.failure_reason: + raise ValueError("failed RLoopVesselOneStepResultFrame requires failure details") + if frame.one_step_status == "completed" and frame.failure_stage is not None: + raise ValueError("completed RLoopVesselOneStepResultFrame must not include failure_stage") + + +def _validate_traverse_result_frame(frame: RLoopVesselTraverseResultFrame) -> None: + if frame.generated_by != R_LOOP_VESSEL_TRAVERSE_GENERATOR: + raise ValueError("RLoopVesselTraverseResultFrame.generated_by must be assembler") + if frame.info_class != "absolute": + raise ValueError("RLoopVesselTraverseResultFrame.info_class must be absolute") + if frame.semantic_judgement_status != "not_run": + raise ValueError("RLoopVesselTraverseResultFrame.semantic status must be not_run") + if frame.traverse_status not in R_LOOP_VESSEL_TRAVERSE_STATUSES: + raise ValueError("RLoopVesselTraverseResultFrame.traverse_status is invalid") + if frame.traverse_status == "failed": + if not frame.failure_stage or not frame.failure_type or not frame.failure_reason: + raise ValueError("failed RLoopVesselTraverseResultFrame requires failure details") + if frame.traverse_status == "completed" and frame.failure_stage is not None: + raise ValueError("completed RLoopVesselTraverseResultFrame must not include failure_stage") + if frame.step_count != len(frame.r3_inspection_frame_ids): + raise ValueError("RLoopVesselTraverseResultFrame.step_count mismatch") + if len(frame.selected_graph_node_ids) != len(frame.r2_selection_frame_ids): + raise ValueError("RLoopVesselTraverseResultFrame selected path mismatch") + for field_name, value in { + "terminal_material_seen_count": frame.terminal_material_seen_count, + "min_terminal_material_count": frame.min_terminal_material_count, + "raw_original_material_seen_count": frame.raw_original_material_seen_count, + "max_raw_original_material_count": frame.max_raw_original_material_count, + "early_stop_guard_trigger_count": frame.early_stop_guard_trigger_count, + }.items(): + if not isinstance(value, int): + raise TypeError(f"RLoopVesselTraverseResultFrame.{field_name} must be an integer") + if value < 0: + raise ValueError(f"RLoopVesselTraverseResultFrame.{field_name} must not be negative") + if frame.max_raw_original_material_count < 1: + raise ValueError("RLoopVesselTraverseResultFrame.max_raw_original_material_count must be positive") + if frame.raw_original_material_seen_count > frame.max_raw_original_material_count: + raise ValueError("RLoopVesselTraverseResultFrame raw original count exceeds cap") + if not isinstance(frame.raw_original_read_cap_reached, bool): + raise TypeError("RLoopVesselTraverseResultFrame.raw_original_read_cap_reached must be bool") + + +def _candidate_layer_surface_frame( + *, + frame_label: str, + read_packet: RLoopVesselReadPacketFrame, + created_at: str, +) -> RLoopVesselCandidateLayerSurfaceFrame: + surface_records = _candidate_layer_surface_records(read_packet) + frame = RLoopVesselCandidateLayerSurfaceFrame( + frame_id=f"R:{frame_label}:vessel_candidate_layer_surface_frame", + created_at=created_at, + source_packet_id=read_packet.packet_id, + surface_count=len(surface_records), + total_candidate_count=sum( + int(record.get("candidate_count") or 0) + for record in surface_records + ), + available_surface_ids=[ + str(record["surface_id"]) + for record in surface_records + if isinstance(record.get("surface_id"), str) + ], + surface_records=surface_records, + source_data_ids=_unique_strings( + [read_packet.packet_id, *read_packet.source_data_ids] + ), + source_trace_ids=list(read_packet.source_trace_ids), + ) + _validate_candidate_layer_surface_frame(frame) + return frame + + +def _candidate_layer_surface_frame_from_graph_surface( + *, + frame_label: str, + read_packet: RLoopVesselReadPacketFrame, + graph_surface: RGraphTraversalCandidateSurfaceFrame, + created_at: str, +) -> RLoopVesselCandidateLayerSurfaceFrame: + surface_records = _candidate_layer_surface_records_from_graph_surface( + read_packet=read_packet, + graph_surface=graph_surface, + ) + frame = RLoopVesselCandidateLayerSurfaceFrame( + frame_id=f"R:{frame_label}:vessel_candidate_layer_surface_frame", + created_at=created_at, + source_packet_id=read_packet.packet_id, + surface_count=len(surface_records), + total_candidate_count=sum( + int(record.get("candidate_count") or 0) + for record in surface_records + ), + available_surface_ids=[ + str(record["surface_id"]) + for record in surface_records + if isinstance(record.get("surface_id"), str) + ], + surface_records=surface_records, + source_data_ids=_unique_strings( + [read_packet.packet_id, graph_surface.frame_id, *graph_surface.source_data_ids] + ), + source_trace_ids=_unique_strings( + [*read_packet.source_trace_ids, *graph_surface.source_trace_ids] + ), + ) + _validate_candidate_layer_surface_frame(frame) + return frame + + +def _candidate_layer_surface_records( + read_packet: RLoopVesselReadPacketFrame, +) -> list[dict[str, object]]: + grouped_ids: dict[str, list[str]] = {} + grouped_meta: dict[str, dict[str, object]] = {} + first_step_entry_records = _core_ego_first_step_entry_records(read_packet) + + for record in first_step_entry_records: + graph_node_id = record.get("candidate_node_id") + if not isinstance(graph_node_id, str) or not graph_node_id: + continue + candidate_kind = _surface_text(record.get("candidate_kind"), "entry_bundle") + data_kind = _surface_text(record.get("data_kind"), candidate_kind) + surface_id = f"surface:entry:{candidate_kind}:data:{data_kind}" + grouped_ids.setdefault(surface_id, []).append(graph_node_id) + grouped_meta.setdefault( + surface_id, + { + "surface_id": surface_id, + "surface_kind": "entry_candidate_kind", + "candidate_kind": candidate_kind, + "data_kind": data_kind, + "branch_role": _branch_role_for_candidate_kind(candidate_kind), + "summary_depth": None, + "info_class": "absolute", + }, + ) + + surface_records: list[dict[str, object]] = [] + for surface_id in _ordered_surface_ids(grouped_meta): + candidate_ids = _unique_strings(grouped_ids.get(surface_id, [])) + record = dict(grouped_meta[surface_id]) + record["candidate_count"] = len(candidate_ids) + record["candidate_graph_node_ids"] = candidate_ids + surface_records.append(record) + return surface_records + + +def _candidate_layer_surface_records_from_graph_surface( + *, + read_packet: RLoopVesselReadPacketFrame, + graph_surface: RGraphTraversalCandidateSurfaceFrame, +) -> list[dict[str, object]]: + records_by_id = _candidate_records_by_id(read_packet) + grouped_ids: dict[str, list[str]] = {} + grouped_meta: dict[str, dict[str, object]] = {} + for graph_node_id in graph_surface.candidate_graph_node_ids: + if graph_node_id not in records_by_id: + continue + record = records_by_id[graph_node_id] + candidate_kind = _surface_text(record.get("candidate_kind"), "hierarchy_candidate") + data_kind = _surface_text(record.get("data_kind"), candidate_kind) + summary_depth = record.get("summary_depth") + depth_part = str(summary_depth) if isinstance(summary_depth, int) else "unknown" + surface_id = ( + "surface:hierarchy:" + f"{_safe_surface_part(candidate_kind)}:" + f"data:{_safe_surface_part(data_kind)}:" + f"depth:{depth_part}" + ) + grouped_ids.setdefault(surface_id, []).append(graph_node_id) + grouped_meta.setdefault( + surface_id, + { + "surface_id": surface_id, + "surface_kind": "hierarchy_child_candidate_kind", + "candidate_kind": candidate_kind, + "data_kind": data_kind, + "branch_role": _branch_role_for_record(record), + "summary_depth": summary_depth if isinstance(summary_depth, int) else None, + "info_class": _surface_text(record.get("info_class"), "absolute"), + "current_graph_node_id": graph_surface.inspected_graph_node_id, + }, + ) + + surface_records: list[dict[str, object]] = [] + for surface_id in _ordered_surface_ids(grouped_meta): + candidate_ids = _unique_strings(grouped_ids.get(surface_id, [])) + record = dict(grouped_meta[surface_id]) + record["candidate_count"] = len(candidate_ids) + record["candidate_graph_node_ids"] = candidate_ids + surface_records.append(record) + return surface_records + + +def _ordered_surface_ids( + grouped_meta: dict[str, dict[str, object]], +) -> list[str]: + return sorted( + grouped_meta, + key=lambda surface_id: _surface_sort_key(grouped_meta[surface_id]), + ) + + +def _surface_sort_key(record: dict[str, object]) -> tuple[int, int, str]: + surface_id = str(record.get("surface_id") or "") + candidate_kind = record.get("candidate_kind") + branch_role = record.get("branch_role") + depth = record.get("summary_depth") + if branch_role == "source_material_ingest": + return (0, 0, surface_id) + if candidate_kind == "summary" and isinstance(depth, int): + return (1, -depth, surface_id) + if branch_role == "conversation_time_memory": + return (3, 0, surface_id) + return (2, 0, surface_id) + + +def _core_ego_first_step_entry_records( + read_packet: RLoopVesselReadPacketFrame, +) -> list[dict[str, object]]: + time_axis_records = [ + record + for record in read_packet.entry_candidate_records + if record.get("candidate_kind") == "time_axis" + or record.get("node_kind") == "time_axis" + ] + if time_axis_records: + return time_axis_records + return list(read_packet.entry_candidate_records) + + +def _surface_all_candidate_ids( + candidate_layer_surface: RLoopVesselCandidateLayerSurfaceFrame, +) -> list[str]: + result: list[str] = [] + for record in candidate_layer_surface.surface_records: + values = record.get("candidate_graph_node_ids") + if isinstance(values, list): + result.extend(item for item in values if isinstance(item, str)) + return _unique_strings(result) + + +def _surface_current_graph_node_id( + candidate_layer_surface: RLoopVesselCandidateLayerSurfaceFrame, +) -> str: + for record in candidate_layer_surface.surface_records: + value = record.get("current_graph_node_id") + if isinstance(value, str) and value: + return value + return "graph:core_ego:root" + + +def _candidate_records_by_surface( + read_packet: RLoopVesselReadPacketFrame, + candidate_layer_surface: RLoopVesselCandidateLayerSurfaceFrame, +) -> dict[str, list[dict[str, object]]]: + records_by_id = _candidate_records_by_id(read_packet) + grouped: dict[str, list[dict[str, object]]] = {} + for surface_record in candidate_layer_surface.surface_records: + surface_id = surface_record.get("surface_id") + if not isinstance(surface_id, str): + continue + grouped[surface_id] = [] + for graph_node_id in _surface_candidate_ids(candidate_layer_surface, surface_id): + candidate_record = records_by_id.get(graph_node_id) + if candidate_record is not None: + grouped[surface_id].append(candidate_record) + return grouped + + +def _r2_selection_ref_map( + read_packet: RLoopVesselReadPacketFrame, + candidate_layer_surface: RLoopVesselCandidateLayerSurfaceFrame, +) -> dict[str, object]: + candidate_records_by_actual_surface = _candidate_records_by_surface( + read_packet, + candidate_layer_surface, + ) + surface_ref_to_surface_id: dict[str, str] = {} + node_ref_to_graph_node_id: dict[str, str] = {} + node_refs_by_surface_ref: dict[str, list[str]] = {} + surface_records: list[dict[str, object]] = [] + candidate_records_by_surface_ref: dict[str, list[dict[str, object]]] = {} + + node_index = 1 + for surface_index, surface_record in enumerate(candidate_layer_surface.surface_records, start=1): + surface_id = surface_record.get("surface_id") + if not isinstance(surface_id, str): + continue + surface_ref = f"surface_{surface_index:03d}" + surface_ref_to_surface_id[surface_ref] = surface_id + candidate_records_by_surface_ref[surface_ref] = [] + node_refs_by_surface_ref[surface_ref] = [] + + candidate_node_refs: list[str] = [] + for candidate_record in candidate_records_by_actual_surface.get(surface_id, []): + graph_node_id = candidate_record.get("graph_node_id") + if not isinstance(graph_node_id, str): + continue + node_ref = f"node_{node_index:03d}" + node_index += 1 + node_ref_to_graph_node_id[node_ref] = graph_node_id + node_refs_by_surface_ref[surface_ref].append(node_ref) + candidate_node_refs.append(node_ref) + visible_record = dict(candidate_record) + visible_record.pop("graph_node_id", None) + visible_record["node_ref"] = node_ref + candidate_records_by_surface_ref[surface_ref].append(visible_record) + + surface_records.append( + { + "surface_ref": surface_ref, + "surface_kind": surface_record.get("surface_kind"), + "candidate_kind": surface_record.get("candidate_kind"), + "data_kind": surface_record.get("data_kind"), + "branch_role": surface_record.get("branch_role"), + "summary_depth": surface_record.get("summary_depth"), + "info_class": surface_record.get("info_class"), + "candidate_count": surface_record.get("candidate_count"), + "candidate_node_refs": candidate_node_refs, + } + ) + + return { + "available_surface_refs": list(surface_ref_to_surface_id), + "surface_records": surface_records, + "candidate_records_by_surface_ref": candidate_records_by_surface_ref, + "surface_ref_to_surface_id": surface_ref_to_surface_id, + "node_ref_to_graph_node_id": node_ref_to_graph_node_id, + "node_refs_by_surface_ref": node_refs_by_surface_ref, + } + + +def _first_visible_ref(payload: dict[str, object]) -> tuple[str | None, str | None, str | None]: + available_surface_refs = payload.get("available_surface_refs") + records_by_surface = payload.get("candidate_records_by_surface_ref") + if not isinstance(available_surface_refs, list) or not isinstance(records_by_surface, dict): + return None, None, None + for surface_ref in available_surface_refs: + if not isinstance(surface_ref, str): + continue + records = records_by_surface.get(surface_ref) + if not isinstance(records, list): + continue + for record in records: + if not isinstance(record, dict): + continue + node_ref = record.get("node_ref") + if isinstance(node_ref, str) and node_ref: + candidate_kind = record.get("candidate_kind") + return ( + surface_ref, + node_ref, + candidate_kind if isinstance(candidate_kind, str) else None, + ) + return None, None, None + + +def _r2_selection_ref_map_from_surface( + candidate_layer_surface: RLoopVesselCandidateLayerSurfaceFrame, +) -> dict[str, object]: + surface_ref_to_surface_id: dict[str, str] = {} + node_ref_to_graph_node_id: dict[str, str] = {} + node_refs_by_surface_ref: dict[str, list[str]] = {} + node_index = 1 + for surface_index, surface_record in enumerate(candidate_layer_surface.surface_records, start=1): + surface_id = surface_record.get("surface_id") + if not isinstance(surface_id, str): + continue + surface_ref = f"surface_{surface_index:03d}" + surface_ref_to_surface_id[surface_ref] = surface_id + node_refs_by_surface_ref[surface_ref] = [] + values = surface_record.get("candidate_graph_node_ids") + if isinstance(values, list): + for graph_node_id in values: + if not isinstance(graph_node_id, str): + continue + node_ref = f"node_{node_index:03d}" + node_index += 1 + node_ref_to_graph_node_id[node_ref] = graph_node_id + node_refs_by_surface_ref[surface_ref].append(node_ref) + return { + "available_surface_refs": list(surface_ref_to_surface_id), + "surface_ref_to_surface_id": surface_ref_to_surface_id, + "node_ref_to_graph_node_id": node_ref_to_graph_node_id, + "node_refs_by_surface_ref": node_refs_by_surface_ref, + } + + +def _selected_surface_id_from_r2_payload( + payload: dict[str, object], + *, + selection_ref_map: dict[str, object], +) -> str | None: + selected_surface_ref = payload.get("selected_surface_ref") + surface_ref_to_surface_id = selection_ref_map.get("surface_ref_to_surface_id") + if isinstance(selected_surface_ref, str) and isinstance(surface_ref_to_surface_id, dict): + surface_id = surface_ref_to_surface_id.get(selected_surface_ref) + return surface_id if isinstance(surface_id, str) else None + selected_surface_id = payload.get("selected_surface_id") + return selected_surface_id if isinstance(selected_surface_id, str) else None + + +def _selected_graph_node_id_from_r2_payload( + payload: dict[str, object], + *, + selection_ref_map: dict[str, object], +) -> str | None: + selected_node_ref = payload.get("selected_node_ref") + node_ref_to_graph_node_id = selection_ref_map.get("node_ref_to_graph_node_id") + if isinstance(selected_node_ref, str) and isinstance(node_ref_to_graph_node_id, dict): + graph_node_id = node_ref_to_graph_node_id.get(selected_node_ref) + return graph_node_id if isinstance(graph_node_id, str) else None + selected_graph_node_id = payload.get("selected_graph_node_id") + return selected_graph_node_id if isinstance(selected_graph_node_id, str) else None + + +def _selected_surface_ref_from_r2_payload(payload: dict[str, object]) -> str | None: + value = payload.get("selected_surface_ref") + return value if isinstance(value, str) else None + + +def _selected_node_ref_from_r2_payload(payload: dict[str, object]) -> str | None: + value = payload.get("selected_node_ref") + return value if isinstance(value, str) else None + + +def _candidate_records_by_id( + read_packet: RLoopVesselReadPacketFrame, +) -> dict[str, dict[str, object]]: + records: dict[str, dict[str, object]] = {} + for record in read_packet.summary_candidate_records: + graph_node_id = record.get("summary_node_id") + if isinstance(graph_node_id, str) and graph_node_id: + item = _summary_candidate_record_view(record) + item["graph_node_id"] = graph_node_id + item["candidate_kind"] = "summary" + records[graph_node_id] = item + for record in read_packet.entry_candidate_records: + graph_node_id = record.get("candidate_node_id") + if isinstance(graph_node_id, str) and graph_node_id: + item = _entry_candidate_record_view(record) + item["graph_node_id"] = graph_node_id + records[graph_node_id] = item + return records + + +def _entry_candidate_record_view(record: dict[str, object]) -> dict[str, object]: + return { + "candidate_kind": record.get("candidate_kind"), + "branch_role": _branch_role_for_record(record), + "display_name": record.get("display_name"), + "node_kind": record.get("node_kind"), + "data_kind": record.get("data_kind"), + "created_at": record.get("created_at"), + "written_at": record.get("written_at"), + "summary_depth": record.get("summary_depth"), + "source_leaf_count": record.get("source_leaf_count"), + "source_summary_count": record.get("source_summary_count"), + } + + +def _surface_candidate_ids( + candidate_layer_surface: RLoopVesselCandidateLayerSurfaceFrame, + surface_id: str | None, +) -> list[str]: + if not surface_id: + return [] + for record in candidate_layer_surface.surface_records: + if record.get("surface_id") == surface_id: + values = record.get("candidate_graph_node_ids") + if isinstance(values, list): + return [item for item in values if isinstance(item, str)] + return [] + + +def _surface_candidate_ids_for_selection( + candidate_layer_surface: RLoopVesselCandidateLayerSurfaceFrame, + selected_surface_id: str | None, + *, + fallback_graph_node_ids: list[str], +) -> list[str]: + selected_surface_ids = _surface_candidate_ids(candidate_layer_surface, selected_surface_id) + return selected_surface_ids or list(fallback_graph_node_ids) + + +def _validate_candidate_layer_surface_frame( + frame: RLoopVesselCandidateLayerSurfaceFrame, +) -> None: + if frame.generated_by != R_LOOP_VESSEL_CANDIDATE_LAYER_SURFACE_GENERATOR: + raise ValueError("RLoopVesselCandidateLayerSurfaceFrame.generated_by is invalid") + if frame.info_class != "absolute": + raise ValueError("RLoopVesselCandidateLayerSurfaceFrame.info_class must be absolute") + if frame.semantic_judgement_status != "not_run": + raise ValueError("RLoopVesselCandidateLayerSurfaceFrame semantic status must be not_run") + if frame.surface_count != len(frame.surface_records): + raise ValueError("RLoopVesselCandidateLayerSurfaceFrame.surface_count mismatch") + surface_ids = [ + record.get("surface_id") + for record in frame.surface_records + if isinstance(record.get("surface_id"), str) + ] + if frame.available_surface_ids != surface_ids: + raise ValueError("RLoopVesselCandidateLayerSurfaceFrame.available_surface_ids mismatch") + for record in frame.surface_records: + if not isinstance(record.get("candidate_graph_node_ids"), list): + raise ValueError("candidate layer surface record requires candidate_graph_node_ids") + if not isinstance(record.get("branch_role"), str) or not record.get("branch_role"): + raise ValueError("candidate layer surface record requires branch_role") + if "summary_text" in record: + raise ValueError("candidate layer surface record must not include summary_text") + + +def _validate_surface_selection_frame(frame: RLoopVesselSurfaceSelectionFrame) -> None: + if frame.info_class != "mixed": + raise ValueError("RLoopVesselSurfaceSelectionFrame.info_class must be mixed") + if frame.semantic_judgement_status != "ran": + raise ValueError("RLoopVesselSurfaceSelectionFrame semantic status must be ran") + if frame.selection_status not in {"selected", "none_selected"}: + raise ValueError("RLoopVesselSurfaceSelectionFrame.selection_status is invalid") + if frame.selection_status == "selected": + if not frame.selected_surface_id: + raise ValueError("selected RLoopVesselSurfaceSelectionFrame requires selected_surface_id") + if frame.selected_surface_id not in frame.available_surface_ids: + raise ValueError("selected_surface_id must be in available_surface_ids") + if frame.selected_surface_candidate_count < 1: + raise ValueError("selected surface must contain candidates") + elif frame.selected_surface_id is not None: + raise ValueError("none_selected surface selection must not include selected_surface_id") + + +def _available_graph_node_ids(read_packet: RLoopVesselReadPacketFrame) -> list[str]: + summary_ids = [ + str(record["summary_node_id"]) + for record in read_packet.summary_candidate_records + if isinstance(record.get("summary_node_id"), str) + ] + entry_ids = [ + str(record["candidate_node_id"]) + for record in read_packet.entry_candidate_records + if isinstance(record.get("candidate_node_id"), str) + ] + return _unique_strings([*summary_ids, *entry_ids]) + + +def _selected_candidate_record( + read_packet: RLoopVesselReadPacketFrame, + selected_graph_node_id: str, +) -> dict[str, object]: + for record in read_packet.summary_candidate_records: + if record.get("summary_node_id") == selected_graph_node_id: + return _with_hierarchy_child_candidates( + read_packet=read_packet, + selected_record=dict(record), + ) + for record in read_packet.entry_candidate_records: + if record.get("candidate_node_id") == selected_graph_node_id: + return _with_hierarchy_child_candidates( + read_packet=read_packet, + selected_record=dict(record), + ) + raise ValueError("selected graph node id is not present in R Vessel read packet") + + +def _record_node_id(record: dict[str, object]) -> str: + value = record.get("summary_node_id") or record.get("candidate_node_id") + return value if isinstance(value, str) else "" + + +def _candidate_node_kind(record: dict[str, object]) -> str: + if record.get("summary_node_id"): + return "summary" + value = record.get("node_kind") + return value if isinstance(value, str) and value else "time_bundle" + + +def _candidate_summary_depth(record: dict[str, object]) -> int: + value = record.get("summary_depth") + return value if isinstance(value, int) and value >= 0 else 0 + + +def _candidate_source_leaf_count(record: dict[str, object]) -> int: + value = record.get("source_leaf_count") + return value if isinstance(value, int) and value >= 0 else 0 + + +def _is_terminal_material_record(record: dict[str, object]) -> bool: + if isinstance(record.get("summary_node_id"), str): + summary_text = record.get("summary_text") + if isinstance(summary_text, str) and summary_text.strip(): + return True + summary_text_char_count = record.get("summary_text_char_count") + return isinstance(summary_text_char_count, int) and summary_text_char_count > 0 + + node_kind = _candidate_node_kind(record) + candidate_kind = record.get("candidate_kind") + terminal_kinds = {"raw_source", "raw_capsule"} + return node_kind in terminal_kinds or candidate_kind in terminal_kinds + + +def _is_raw_original_material_record(record: dict[str, object]) -> bool: + node_kind = _candidate_node_kind(record) + candidate_kind = record.get("candidate_kind") + return node_kind in {"raw_source", "raw_capsule"} or candidate_kind in { + "raw_source", + "raw_capsule", + } + + +def _candidate_child_node_ids(record: dict[str, object]) -> list[str]: + values: list[str | None] = [] + hierarchy_child_node_ids = record.get("hierarchy_child_node_ids") + if isinstance(hierarchy_child_node_ids, list): + return _unique_strings( + [item for item in hierarchy_child_node_ids if isinstance(item, str)] + ) + source_graph_node_ids = record.get("source_graph_node_ids") + if isinstance(source_graph_node_ids, list): + values.extend(item for item in source_graph_node_ids if isinstance(item, str)) + target_graph_node_id = record.get("target_graph_node_id") + if isinstance(target_graph_node_id, str): + values.append(target_graph_node_id) + return _unique_strings(values) + + +def _with_hierarchy_child_candidates( + *, + read_packet: RLoopVesselReadPacketFrame, + selected_record: dict[str, object], +) -> dict[str, object]: + selected_node_id = _record_node_id(selected_record) + child_records = _hierarchy_child_candidate_records( + read_packet=read_packet, + selected_node_id=selected_node_id, + selected_record=selected_record, + ) + child_node_ids = _unique_strings( + [ + value + for record in child_records + for value in [record.get("candidate_node_id")] + if isinstance(value, str) + ] + ) + result = dict(selected_record) + result["hierarchy_child_node_ids"] = child_node_ids + result["hierarchy_child_candidate_records"] = child_records + return result + + +def _hierarchy_child_candidate_records( + *, + read_packet: RLoopVesselReadPacketFrame, + selected_node_id: str, + selected_record: dict[str, object], +) -> list[dict[str, object]]: + records_by_id = _full_candidate_records_by_id(read_packet) + summary_child_records = _summary_node_layer_child_records( + selected_record=selected_record, + records_by_id=records_by_id, + ) + if summary_child_records or _summary_node_owns_child_policy(selected_record): + return summary_child_records + + summary_layer_records = _source_kind_summary_layer_child_records( + read_packet=read_packet, + selected_node_id=selected_node_id, + selected_record=selected_record, + records_by_id=records_by_id, + ) + if summary_layer_records: + return summary_layer_records + + child_ids: list[str] = [] + + if _candidate_node_kind(selected_record) == "time_axis": + child_ids.extend( + str(record["candidate_node_id"]) + for record in read_packet.entry_candidate_records + if isinstance(record.get("candidate_node_id"), str) + and record.get("candidate_node_id") != selected_node_id + and record.get("candidate_kind") + in {"time_bundle", "source_ingest_bundle", "source_ingest_time_bundle"} + ) + + source_graph_node_ids = selected_record.get("source_graph_node_ids") + if isinstance(source_graph_node_ids, list): + child_ids.extend(item for item in source_graph_node_ids if isinstance(item, str)) + + for record in read_packet.entry_candidate_records: + parent_ids = record.get("parent_graph_node_ids") + if isinstance(parent_ids, list) and selected_node_id in parent_ids: + child_id = record.get("candidate_node_id") + if isinstance(child_id, str): + child_ids.append(child_id) + + for record in read_packet.summary_candidate_records: + target_id = record.get("target_graph_node_id") + if target_id == selected_node_id: + summary_id = record.get("summary_node_id") + if isinstance(summary_id, str): + child_ids.append(summary_id) + + child_records: list[dict[str, object]] = [] + for child_id in _unique_strings(child_ids): + child_record = records_by_id.get(child_id) + if child_record is None: + child_record = { + "candidate_node_id": child_id, + "candidate_kind": "unresolved_graph_node", + "display_name": child_id, + } + child_records.append(child_record) + return child_records + + +def _summary_node_layer_child_records( + *, + selected_record: dict[str, object], + records_by_id: dict[str, dict[str, object]], +) -> list[dict[str, object]]: + if _candidate_node_kind(selected_record) != "summary": + return [] + + data_kind = selected_record.get("data_kind") + if data_kind == "token_budget_bundle_summary": + return _candidate_records_for_ids( + records_by_id=records_by_id, + graph_node_ids=_summary_child_summary_ids(selected_record), + ) + + if data_kind == "source_leaf_summary": + return _candidate_records_for_ids( + records_by_id=records_by_id, + graph_node_ids=_summary_child_raw_ids(selected_record), + ) + + return [] + + +def _summary_node_owns_child_policy(record: dict[str, object]) -> bool: + return _candidate_node_kind(record) == "summary" and record.get("data_kind") in { + "token_budget_bundle_summary", + "source_leaf_summary", + } + + +def _summary_child_summary_ids(record: dict[str, object]) -> list[str]: + values: list[str] = [] + for field_name in ("source_graph_node_ids", "source_data_ids"): + field_value = record.get(field_name) + if isinstance(field_value, list): + values.extend( + item + for item in field_value + if isinstance(item, str) and item.startswith("graph:summary:") + ) + return _unique_strings(values) + + +def _summary_child_raw_ids(record: dict[str, object]) -> list[str]: + values: list[str] = [] + for field_name in ("source_graph_node_ids", "source_data_ids"): + field_value = record.get(field_name) + if isinstance(field_value, list): + values.extend(item for item in field_value if _is_raw_source_id(item)) + target_graph_node_id = record.get("target_graph_node_id") + if _is_raw_source_id(target_graph_node_id): + values.append(target_graph_node_id) # type: ignore[arg-type] + return _unique_strings(values) + + +def _candidate_records_for_ids( + *, + records_by_id: dict[str, dict[str, object]], + graph_node_ids: list[str], +) -> list[dict[str, object]]: + return [ + records_by_id[graph_node_id] + for graph_node_id in graph_node_ids + if graph_node_id in records_by_id + ] + + +def _source_kind_summary_layer_child_records( + *, + read_packet: RLoopVesselReadPacketFrame, + selected_node_id: str, + selected_record: dict[str, object], + records_by_id: dict[str, dict[str, object]], +) -> list[dict[str, object]]: + if _candidate_node_kind(selected_record) != "source_kind_bundle": + return [] + + raw_child_ids = _source_kind_raw_child_ids( + read_packet=read_packet, + selected_node_id=selected_node_id, + selected_record=selected_record, + ) + if not raw_child_ids: + return [] + + token_summary_ids = _summary_ids_covering_any_source( + read_packet=read_packet, + source_ids=raw_child_ids, + data_kind="token_budget_bundle_summary", + ) + if token_summary_ids: + return [ + records_by_id[summary_id] + for summary_id in token_summary_ids + if summary_id in records_by_id + ] + + leaf_summary_ids = _summary_ids_covering_any_source( + read_packet=read_packet, + source_ids=raw_child_ids, + data_kind="source_leaf_summary", + ) + return [ + records_by_id[summary_id] + for summary_id in leaf_summary_ids + if summary_id in records_by_id + ] + + +def _source_kind_raw_child_ids( + *, + read_packet: RLoopVesselReadPacketFrame, + selected_node_id: str, + selected_record: dict[str, object], +) -> list[str]: + child_ids: list[str] = [] + source_graph_node_ids = selected_record.get("source_graph_node_ids") + if isinstance(source_graph_node_ids, list): + child_ids.extend(item for item in source_graph_node_ids if _is_raw_source_id(item)) + + for record in read_packet.entry_candidate_records: + parent_ids = record.get("parent_graph_node_ids") + child_id = record.get("candidate_node_id") + if ( + isinstance(parent_ids, list) + and selected_node_id in parent_ids + and isinstance(child_id, str) + and _entry_record_is_raw_source(record) + ): + child_ids.append(child_id) + return _unique_strings(child_ids) + + +def _summary_ids_covering_any_source( + *, + read_packet: RLoopVesselReadPacketFrame, + source_ids: list[str], + data_kind: str, +) -> list[str]: + source_id_set = set(source_ids) + summary_ids: list[str] = [] + for record in read_packet.summary_candidate_records: + if record.get("data_kind") != data_kind: + continue + if not source_id_set.intersection(_summary_record_source_ids(record)): + continue + summary_id = record.get("summary_node_id") + if isinstance(summary_id, str) and summary_id: + summary_ids.append(summary_id) + return _unique_strings(summary_ids) + + +def _summary_record_source_ids(record: dict[str, object]) -> list[str]: + values: list[str | None] = [] + source_graph_node_ids = record.get("source_graph_node_ids") + if isinstance(source_graph_node_ids, list): + values.extend(item for item in source_graph_node_ids if isinstance(item, str)) + source_data_ids = record.get("source_data_ids") + if isinstance(source_data_ids, list): + values.extend(item for item in source_data_ids if isinstance(item, str)) + target_graph_node_id = record.get("target_graph_node_id") + if isinstance(target_graph_node_id, str): + values.append(target_graph_node_id) + return _unique_strings(values) + + +def _entry_record_is_raw_source(record: dict[str, object]) -> bool: + return ( + record.get("candidate_kind") == "raw_source" + or record.get("node_kind") == "raw_source" + or _is_raw_source_id(record.get("candidate_node_id")) + ) + + +def _branch_role_for_record(record: dict[str, object]) -> str: + candidate_kind = record.get("candidate_kind") + if isinstance(candidate_kind, str) and candidate_kind: + return _branch_role_for_candidate_kind(candidate_kind) + + node_kind = record.get("node_kind") + if isinstance(node_kind, str) and node_kind: + return _branch_role_for_candidate_kind(node_kind) + + if isinstance(record.get("summary_node_id"), str): + return "summary_memory" + + return "unknown_graph_branch" + + +def _branch_role_for_candidate_kind(candidate_kind: str) -> str: + if candidate_kind == "time_axis": + return "graph_axis" + if candidate_kind in {"source_ingest_bundle", "source_ingest_time_bundle"}: + return "source_material_ingest" + if candidate_kind == "source_kind_bundle": + return "source_material_ingest" + if candidate_kind in {"raw_source", "token_budget_summary_bundle"}: + return "source_material_leaf" + if candidate_kind == "time_bundle": + return "conversation_time_memory" + if candidate_kind == "raw_capsule": + return "conversation_time_memory" + if candidate_kind == "summary": + return "summary_memory" + return "unknown_graph_branch" + + +def _is_raw_source_id(value: object) -> bool: + return isinstance(value, str) and value.startswith("graph:raw_source:") + + +def _full_candidate_records_by_id( + read_packet: RLoopVesselReadPacketFrame, +) -> dict[str, dict[str, object]]: + records: dict[str, dict[str, object]] = {} + for record in read_packet.entry_candidate_records: + graph_node_id = record.get("candidate_node_id") + if isinstance(graph_node_id, str) and graph_node_id: + records[graph_node_id] = { + "candidate_node_id": graph_node_id, + "candidate_kind": record.get("candidate_kind"), + "display_name": record.get("display_name"), + "node_kind": record.get("node_kind"), + "data_kind": record.get("data_kind"), + "branch_role": _branch_role_for_record(record), + "summary_depth": record.get("summary_depth"), + "source_leaf_count": record.get("source_leaf_count"), + "source_summary_count": record.get("source_summary_count"), + } + for record in read_packet.summary_candidate_records: + graph_node_id = record.get("summary_node_id") + if isinstance(graph_node_id, str) and graph_node_id: + records[graph_node_id] = { + "candidate_node_id": graph_node_id, + "candidate_kind": "summary", + "display_name": record.get("summary_display_name"), + "node_kind": "summary", + "data_kind": record.get("data_kind"), + "branch_role": _branch_role_for_record(record), + "summary_depth": record.get("summary_depth"), + "source_leaf_count": record.get("source_leaf_count"), + "source_summary_count": record.get("source_summary_count"), + "target_graph_node_id": record.get("target_graph_node_id"), + "source_graph_node_ids": record.get("source_graph_node_ids"), + "source_data_ids": record.get("source_data_ids"), + "summary_text_preview": record.get("summary_text_preview"), + } + return records + + +def _hierarchy_child_candidate_record_views( + selected_record: dict[str, object], +) -> list[dict[str, object]]: + records = selected_record.get("hierarchy_child_candidate_records") + if not isinstance(records, list): + return [] + result: list[dict[str, object]] = [] + for record in records: + if not isinstance(record, dict): + continue + result.append( + { + "candidate_kind": record.get("candidate_kind"), + "display_name": record.get("display_name"), + "node_kind": record.get("node_kind"), + "data_kind": record.get("data_kind"), + "summary_depth": record.get("summary_depth"), + "source_leaf_count": record.get("source_leaf_count"), + "source_summary_count": record.get("source_summary_count"), + "summary_text_preview": record.get("summary_text_preview"), + } + ) + return result + + +def _max_summary_depth(read_packet: RLoopVesselReadPacketFrame) -> int | None: + depths = [ + record.get("summary_depth") + for record in read_packet.summary_candidate_records + if isinstance(record.get("summary_depth"), int) + ] + return max(depths) if depths else None + + +def _payload_text(payload: dict[str, object], field_name: str) -> str: + value = payload.get(field_name) + return value.strip() if isinstance(value, str) else "" + + +def _user_question_anchor_id(user_question: str) -> str: + digest = hashlib.sha256(user_question.encode("utf-8")).hexdigest()[:16] + return f"r1_user_question_anchor:{digest}" + + +def _r1_anchor_id_from_input_payload(input_payload: dict[str, object]) -> str: + anchor = input_payload.get("user_question_anchor") + if isinstance(anchor, dict): + anchor_id = anchor.get("anchor_id") + if isinstance(anchor_id, str): + return anchor_id + return "" + + +def _surface_text(value: object, fallback: str) -> str: + if isinstance(value, str) and value.strip(): + return _safe_surface_part(value) + return fallback + + +def _safe_surface_part(value: str) -> str: + return re.sub(r"[^a-zA-Z0-9_:-]+", "_", value.strip()) or "unknown" + + +def _payload_int(payload: dict[str, object], field_name: str) -> int: + value = payload.get(field_name) + if isinstance(value, bool): + return 0 + if isinstance(value, int): + return value + if isinstance(value, str): + try: + return int(value) + except ValueError: + return 0 + return 0 + + +def _shares_user_question_anchor(*, user_question: str, generated_goal: str) -> bool: + anchors = _question_anchor_tokens(user_question) + if not anchors: + return True + normalized_goal = _normalize_anchor_text(generated_goal) + return any(anchor in normalized_goal for anchor in anchors) + + +def _question_anchor_tokens(user_question: str) -> list[str]: + normalized = _normalize_anchor_text(user_question) + raw_tokens = re.findall(r"[a-z0-9_]{2,}", normalized) + blocked = { + "the", + "and", + "for", + "with", + "this", + "that", + "how", + "what", + "why", + "one", + "step", + "한", + "단계", + "골라봐", + "어떻게", + "구조", + } + anchors: list[str] = [] + for token in raw_tokens: + if token in blocked: + continue + if token in anchors: + continue + anchors.append(token) + return anchors[:8] + + +def _normalize_anchor_text(text: str) -> str: + return "".join( + character.lower() if character.isalnum() or character == "_" else " " + for character in text + ) + + +def _prompt(prompt_ref: str) -> str: + return Path(prompt_ref).read_text(encoding="utf-8") + + +def _append_llm_refs(result, call_data_ids: list[str], trace_event_ids: list[str]) -> None: + if result.call_data_id: + call_data_ids.append(result.call_data_id) + if result.trace_event_id: + trace_event_ids.append(result.trace_event_id) + + +def _safe_frame_label(value: str) -> str: + normalized = value.strip().replace("-", "_").replace(" ", "_") + if not normalized: + return "manual_vessel_r_one_step" + if not all(ch.isalnum() or ch == "_" for ch in normalized): + raise ValueError("R Vessel one-step frame_label must contain only letters, numbers, or underscore") + return normalized + + +def _result_frame_id(frame_label: str) -> str: + return f"R:{frame_label}:vessel_one_step_result_frame" + + +def _traverse_result_frame_id(frame_label: str) -> str: + return f"R:{frame_label}:vessel_traverse_result_frame" + + +def _now_from_trace(trace_store: TraceStore) -> str: + events = trace_store.list_events() + if events: + return events[-1].timestamp + from datetime import datetime + + return datetime.now().isoformat(timespec="seconds") + + +def _unique_strings(values: list[str | None]) -> list[str]: + seen: set[str] = set() + result: list[str] = [] + for value in values: + if not value or value in seen: + continue + seen.add(value) + result.append(value) + return result + + +__all__ = [ + "R1_VESSEL_GOAL_PROMPT_REF", + "R2_VESSEL_SELECTOR_PROMPT_REF", + "R3_VESSEL_INSPECTOR_PROMPT_REF", + "RLoopVesselCandidateLayerSurfaceFrame", + "RLoopVesselOneStepFakeLLMAdapter", + "RLoopVesselOneStepResultFrame", + "RLoopVesselOneStepRun", + "RLoopVesselSurfaceSelectionFrame", + "RLoopVesselTraverseFakeLLMAdapter", + "RLoopVesselTraverseResultFrame", + "RLoopVesselTraverseRun", + "R_LOOP_VESSEL_CANDIDATE_LAYER_SURFACE_DATA_TYPE", + "R_LOOP_VESSEL_CANDIDATE_LAYER_SURFACE_GENERATOR", + "R_LOOP_VESSEL_SURFACE_SELECTION_DATA_TYPE", + "R_LOOP_VESSEL_ONE_STEP_GENERATOR", + "R_LOOP_VESSEL_ONE_STEP_POLICY_ID", + "R_LOOP_VESSEL_ONE_STEP_RESULT_DATA_TYPE", + "R_LOOP_VESSEL_ONE_STEP_SCHEMA_NAME", + "R_LOOP_VESSEL_TRAVERSE_GENERATOR", + "R_LOOP_VESSEL_TRAVERSE_POLICY_ID", + "R_LOOP_VESSEL_TRAVERSE_RESULT_DATA_TYPE", + "R_LOOP_VESSEL_TRAVERSE_SCHEMA_NAME", + "run_r_loop_vessel_one_step", + "run_r_loop_vessel_traverse", +] diff --git a/songryeon_core/nodes/core_ego_guide_worker.py b/songryeon_core/nodes/core_ego_guide_worker.py new file mode 100644 index 0000000..f25df4e --- /dev/null +++ b/songryeon_core/nodes/core_ego_guide_worker.py @@ -0,0 +1,395 @@ +from __future__ import annotations + +from dataclasses import asdict +from pathlib import Path + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.schemas import ( + CoreEgoGuideWorkerHintFrame, + RLoopGraphGuidePacketFrame, + validate_core_ego_guide_worker_hint_frame, +) +from songryeon_core.core.trace_store import TraceStore +from songryeon_core.llm.base import LLMAdapter +from songryeon_core.llm.node_executor import LLMNodeExecutor + + +CORE_EGO_GUIDE_WORKER_NODE_ID = "core_ego_guide_worker" +CORE_EGO_GUIDE_WORKER_PROMPT_REF = ( + "songryeon_core/prompts/core_ego_guide_worker_v0.md" +) + + +def core_ego_guide_worker_hint_frame_data_id(guide_packet_id: str) -> str: + return f"{guide_packet_id}:core_ego_guide_worker_hint" + + +def core_ego_guide_worker_input_data_id(guide_packet_id: str) -> str: + return f"{guide_packet_id}:core_ego_guide_worker_input" + + +def run_core_ego_guide_worker_hint( + *, + trace_store: TraceStore, + data_store: DataStore, + turn_id: str, + guide_packet: RLoopGraphGuidePacketFrame, + adapter: LLMAdapter | None, + max_retries: int = 0, +) -> tuple[str, str, CoreEgoGuideWorkerHintFrame]: + """Ask an LLM for graph traversal hints without changing the code guide packet.""" + + frame_id = core_ego_guide_worker_hint_frame_data_id(guide_packet.packet_id) + input_data_id = core_ego_guide_worker_input_data_id(guide_packet.packet_id) + base_source_data_ids = _unique_strings( + [ + guide_packet.packet_id, + guide_packet.graph_snapshot_id, + *guide_packet.source_data_ids, + ] + ) + input_payload = _guide_worker_input_payload( + guide_packet=guide_packet, + input_data_id=input_data_id, + source_data_ids=base_source_data_ids, + ) + input_trace_id = _record_guide_worker_input( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + input_data_id=input_data_id, + input_payload=input_payload, + source_trace_ids=guide_packet.source_trace_ids, + source_data_ids=base_source_data_ids, + ) + base_source_trace_ids = _unique_strings( + [*guide_packet.source_trace_ids, input_trace_id] + ) + + if adapter is None: + frame = _failed_hint_frame( + frame_id=frame_id, + guide_packet=guide_packet, + model_id="adapter_missing", + failure_type="adapter_missing", + payload_parse_status="not_checked", + source_trace_ids=base_source_trace_ids, + source_data_ids=_unique_strings([*base_source_data_ids, input_data_id]), + llm_call_data_id=None, + llm_trace_event_id=None, + ) + return _record_hint_frame( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + frame=frame, + ) + + prompt = Path(CORE_EGO_GUIDE_WORKER_PROMPT_REF).read_text(encoding="utf-8") + llm_result = LLMNodeExecutor(adapter).run( + node_id=CORE_EGO_GUIDE_WORKER_NODE_ID, + prompt=prompt, + input_payload=input_payload, + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + prompt_ref=CORE_EGO_GUIDE_WORKER_PROMPT_REF, + input_ref=base_source_trace_ids, + source_data_ids=_unique_strings([*base_source_data_ids, input_data_id]), + max_retries=max_retries, + payload_validator=lambda payload: _validate_guide_worker_payload( + payload=payload, + available_entry_node_ids=guide_packet.available_entry_nodes, + available_source_graph_node_ids=guide_packet.source_graph_node_ids, + available_source_data_ids=base_source_data_ids, + ), + ) + source_trace_ids = _unique_strings( + [*base_source_trace_ids, llm_result.trace_event_id] + ) + source_data_ids = _unique_strings( + [*base_source_data_ids, input_data_id, llm_result.call_data_id] + ) + if llm_result.failure_type != "none" or llm_result.validation.payload is None: + frame = _failed_hint_frame( + frame_id=frame_id, + guide_packet=guide_packet, + model_id=llm_result.model_id, + failure_type=llm_result.failure_type, + payload_parse_status=_payload_parse_status(llm_result.failure_type), + source_trace_ids=source_trace_ids, + source_data_ids=source_data_ids, + llm_call_data_id=llm_result.call_data_id, + llm_trace_event_id=llm_result.trace_event_id, + ) + else: + frame = _hint_frame_from_payload( + payload=llm_result.validation.payload, + frame_id=frame_id, + guide_packet=guide_packet, + model_id=llm_result.model_id, + source_trace_ids=source_trace_ids, + source_data_ids=source_data_ids, + llm_call_data_id=llm_result.call_data_id, + llm_trace_event_id=llm_result.trace_event_id, + ) + + return _record_hint_frame( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + frame=frame, + ) + + +def _guide_worker_input_payload( + *, + guide_packet: RLoopGraphGuidePacketFrame, + input_data_id: str, + source_data_ids: list[str], +) -> dict[str, object]: + return { + "input_data_id": input_data_id, + "source_rloop_graph_guide_packet_id": guide_packet.packet_id, + "graph_snapshot_id": guide_packet.graph_snapshot_id, + "available_entry_node_ids": list(guide_packet.available_entry_nodes), + "available_source_graph_node_ids": list(guide_packet.source_graph_node_ids), + "node_kind_counts": dict(guide_packet.node_kind_counts), + "data_kind_counts": dict(guide_packet.data_kind_counts), + "summary_depth_range": list(guide_packet.summary_depth_range), + "source_leaf_count_range": list(guide_packet.source_leaf_count_range), + "risky_or_unreviewed_node_ids": list(guide_packet.risky_or_unreviewed_node_ids), + "source_trace_ids": list(guide_packet.source_trace_ids), + "source_data_ids": source_data_ids, + "generated_by": "CODE:CORE_EGO_GUIDE_WORKER_INPUT_BUILDER", + "semantic_judgement_status": "not_run", + } + + +def _record_guide_worker_input( + *, + trace_store: TraceStore, + data_store: DataStore, + turn_id: str, + input_data_id: str, + input_payload: dict[str, object], + source_trace_ids: list[str], + source_data_ids: list[str], +) -> str: + event = trace_store.create_event( + turn_id=turn_id, + actor=CORE_EGO_GUIDE_WORKER_NODE_ID, + event_type="node_input", + input_ref=source_trace_ids, + output_ref=[input_data_id], + schema_status="not_checked", + ) + data_store.create_record( + data_id=input_data_id, + data_type="node_input:core_ego_guide_worker_input", + exists=True, + created_at=event.timestamp, + source_trace_id=event.event_id, + payload={ + **input_payload, + "source_trace_ids": source_trace_ids, + "source_data_ids": source_data_ids, + }, + ) + return event.event_id + + +def _record_hint_frame( + *, + trace_store: TraceStore, + data_store: DataStore, + turn_id: str, + frame: CoreEgoGuideWorkerHintFrame, +) -> tuple[str, str, CoreEgoGuideWorkerHintFrame]: + validate_core_ego_guide_worker_hint_frame(frame) + event = trace_store.create_event( + turn_id=turn_id, + actor=CORE_EGO_GUIDE_WORKER_NODE_ID, + event_type="node_output", + input_ref=frame.source_trace_ids, + output_ref=[frame.frame_id], + schema_status="passed", + ) + data_store.create_record( + data_id=frame.frame_id, + data_type="node_output:core_ego_guide_worker_hint_frame", + exists=True, + created_at=event.timestamp, + source_trace_id=event.event_id, + payload=asdict(frame), + ) + return event.event_id, frame.frame_id, frame + + +def _hint_frame_from_payload( + *, + payload: dict[str, object], + frame_id: str, + guide_packet: RLoopGraphGuidePacketFrame, + model_id: str, + source_trace_ids: list[str], + source_data_ids: list[str], + llm_call_data_id: str | None, + llm_trace_event_id: str | None, +) -> CoreEgoGuideWorkerHintFrame: + payload_source_data_ids = _string_list(payload.get("source_data_ids")) + frame = CoreEgoGuideWorkerHintFrame( + frame_id=frame_id, + source_rloop_graph_guide_packet_id=guide_packet.packet_id, + graph_snapshot_id=guide_packet.graph_snapshot_id, + available_entry_node_ids=list(guide_packet.available_entry_nodes), + available_source_graph_node_ids=list(guide_packet.source_graph_node_ids), + recommended_entry_node_ids=_string_list(payload.get("recommended_entry_node_ids")), + avoid_entry_node_ids=_string_list(payload.get("avoid_entry_node_ids")), + traversal_strategy_hint=str(payload.get("traversal_strategy_hint") or "").strip(), + reason_summary=str(payload.get("reason_summary") or "").strip(), + risk_notes=_string_list(payload.get("risk_notes")), + expected_depth_policy=str(payload.get("expected_depth_policy") or "").strip(), + hint_status="ran", + failure_type="none", + payload_parse_status="passed", + llm_call_data_id=llm_call_data_id, + llm_trace_event_id=llm_trace_event_id, + prompt_ref=CORE_EGO_GUIDE_WORKER_PROMPT_REF, + source_graph_node_ids=_string_list(payload.get("source_graph_node_ids")), + source_trace_ids=source_trace_ids, + source_data_ids=_unique_strings([*source_data_ids, *payload_source_data_ids]), + generated_by=f"LLM:{model_id}:core_ego_guide_worker", + info_class="mixed", + source_mode="source_bundle", + claim_alignment="multi_source_bundle", + semantic_judgement_status="ran", + ) + validate_core_ego_guide_worker_hint_frame(frame) + return frame + + +def _failed_hint_frame( + *, + frame_id: str, + guide_packet: RLoopGraphGuidePacketFrame, + model_id: str, + failure_type: str, + payload_parse_status: str, + source_trace_ids: list[str], + source_data_ids: list[str], + llm_call_data_id: str | None, + llm_trace_event_id: str | None, +) -> CoreEgoGuideWorkerHintFrame: + frame = CoreEgoGuideWorkerHintFrame( + frame_id=frame_id, + source_rloop_graph_guide_packet_id=guide_packet.packet_id, + graph_snapshot_id=guide_packet.graph_snapshot_id, + available_entry_node_ids=list(guide_packet.available_entry_nodes), + available_source_graph_node_ids=list(guide_packet.source_graph_node_ids), + recommended_entry_node_ids=[], + avoid_entry_node_ids=[], + traversal_strategy_hint="", + reason_summary="", + risk_notes=[], + expected_depth_policy="", + hint_status="failed", + failure_type=failure_type, + payload_parse_status=payload_parse_status, + llm_call_data_id=llm_call_data_id, + llm_trace_event_id=llm_trace_event_id, + prompt_ref=CORE_EGO_GUIDE_WORKER_PROMPT_REF, + source_graph_node_ids=[], + source_trace_ids=source_trace_ids, + source_data_ids=source_data_ids, + generated_by=f"LLM:{model_id}:core_ego_guide_worker", + info_class="mixed", + source_mode="source_bundle", + claim_alignment="multi_source_bundle", + semantic_judgement_status="failed", + ) + validate_core_ego_guide_worker_hint_frame(frame) + return frame + + +def _validate_guide_worker_payload( + *, + payload: dict[str, object], + available_entry_node_ids: list[str], + available_source_graph_node_ids: list[str], + available_source_data_ids: list[str], +) -> None: + recommended = _string_list(payload.get("recommended_entry_node_ids")) + avoided = _string_list(payload.get("avoid_entry_node_ids")) + source_graph_node_ids = _string_list(payload.get("source_graph_node_ids")) + payload_source_data_ids = _string_list(payload.get("source_data_ids")) + + if len(recommended) != len(set(recommended)): + raise ValueError("recommended_entry_node_ids must not contain duplicates") + if len(avoided) != len(set(avoided)): + raise ValueError("avoid_entry_node_ids must not contain duplicates") + if len(source_graph_node_ids) != len(set(source_graph_node_ids)): + raise ValueError("source_graph_node_ids must not contain duplicates") + + available_entries = set(available_entry_node_ids) + for node_id in [*recommended, *avoided]: + if node_id not in available_entries: + raise ValueError("recommended or avoided entry node id is not available") + + available_source_nodes = set(available_source_graph_node_ids) + for node_id in source_graph_node_ids: + if node_id not in available_source_nodes: + raise ValueError("source graph node id is not in graph snapshot") + + available_source_ids = set(available_source_data_ids) + for data_id in payload_source_data_ids: + if data_id not in available_source_ids: + raise ValueError("source_data_id is not in allowed graph guide source bundle") + + traversal_strategy_hint = str(payload.get("traversal_strategy_hint") or "").strip() + reason_summary = str(payload.get("reason_summary") or "").strip() + expected_depth_policy = str(payload.get("expected_depth_policy") or "").strip() + if not traversal_strategy_hint: + raise ValueError("traversal_strategy_hint must not be empty") + if not reason_summary: + raise ValueError("reason_summary must not be empty") + if not expected_depth_policy: + raise ValueError("expected_depth_policy must not be empty") + if not source_graph_node_ids: + raise ValueError("source_graph_node_ids must not be empty") + + info_class = payload.get("info_class") + if info_class is not None and info_class != "mixed": + raise ValueError("CoreEgo guide worker info_class must be mixed when supplied") + semantic_status = payload.get("semantic_judgement_status") + if semantic_status is not None and semantic_status != "ran": + raise ValueError("CoreEgo guide worker semantic_judgement_status must be ran when supplied") + + +def _payload_parse_status(failure_type: str) -> str: + if failure_type == "parse_failed": + return "failed" + if failure_type == "adapter_failed": + return "not_checked" + return "passed" + + +def _string_list(value: object) -> list[str]: + if not isinstance(value, list): + return [] + result: list[str] = [] + for item in value: + if isinstance(item, str) and item: + result.append(item) + return result + + +def _unique_strings(values: list[str | None]) -> list[str]: + seen: set[str] = set() + result: list[str] = [] + for value in values: + if value is None or value in seen: + continue + seen.add(value) + result.append(value) + return result diff --git a/songryeon_core/nodes/l2_query_setter.py b/songryeon_core/nodes/l2_query_setter.py index 6c6360d..7b54a46 100644 --- a/songryeon_core/nodes/l2_query_setter.py +++ b/songryeon_core/nodes/l2_query_setter.py @@ -5,6 +5,7 @@ from songryeon_core.core.data_store import DataStore from songryeon_core.core.schemas import ( + L2_REVISION_TARGET_TOOL_NAMES, L2_TARGET_TOOL_NAMES, L2QueryFrame, L2QueryPlanCandidate, @@ -77,7 +78,7 @@ def run_l2_query_setter( turn_id=turn_id, query_text=query_text, query_source=query_source, - query_mode="exact_artifact_ref" if target_tool_name == "read_artifact" else "embedding_search", + query_mode=_query_mode_for_tool(target_tool_name), target_tool_name=target_tool_name, source_trace_ids=input_ref, source_data_ids=source_data_ids or [], @@ -114,6 +115,8 @@ def run_l2_query_planner( adapter: LLMAdapter, source_data_ids: list[str], available_tools: list[dict[str, object]] | None = None, + l_tool_scope: dict[str, object] | None = None, + budget_partition: dict[str, object] | None = None, max_retries: int = 0, query_plan_frame_data_id: str = L2_QUERY_PLAN_FRAME_DATA_ID, ) -> TraceEvent: @@ -128,15 +131,19 @@ def run_l2_query_planner( prompt = Path(prompt_ref).read_text(encoding="utf-8") l1_goal = _read_l1_goal_payload(data_store=data_store, source_data_ids=source_data_ids) attribution_source_data_ids = _l2_attribution_source_data_ids(source_data_ids) + supplied_available_tools = available_tools or _default_l2_available_tools() + allowed_target_tools = _allowed_l2_target_tools_from_available(supplied_available_tools) input_payload = { "user_input": user_input, "l1_goal": l1_goal, "l2_planning_contract": _l2_planning_contract(l1_goal), + "l_tool_scope": l_tool_scope or {}, + "budget_partition": budget_partition or {}, # LLM에게 모든 내부 record ID를 의미 입력으로 보여주면 # L:budget_plan_frame 같은 추적용 ID를 "예산 문서"로 오해할 수 있다. # 그래서 L2에게는 의미 판단용 목표와 별도로, 후보가 복사할 출처 ID만 공급한다. "attribution_source_data_ids": attribution_source_data_ids, - "available_tools": available_tools or [{"name": "search_docs", "read_only": True}], + "available_tools": supplied_available_tools, } llm_result = LLMNodeExecutor(adapter).run( node_id="L2", @@ -149,7 +156,10 @@ def run_l2_query_planner( input_ref=source_trace_ids, source_data_ids=source_data_ids, max_retries=max_retries, - payload_validator=_validate_l2_query_plan_payload, + payload_validator=lambda payload: _validate_l2_query_plan_payload( + payload, + allowed_target_tools=allowed_target_tools, + ), ) if llm_result.failure_type != "none" or llm_result.validation.payload is None: raise ValueError(f"L2 query planner failed: {llm_result.failure_type}") @@ -167,6 +177,7 @@ def run_l2_query_planner( source_trace_ids=frame_source_trace_ids, source_data_ids=frame_source_data_ids, frame_id=query_plan_frame_data_id, + allowed_target_tools=allowed_target_tools, ) validate_l2_query_plan_frame(frame) event = trace_store.create_event( @@ -196,6 +207,8 @@ def run_l2_revision_query_planner( revision_input_data_id: str, adapter: LLMAdapter, available_tools: list[dict[str, object]] | None = None, + l_tool_scope: dict[str, object] | None = None, + budget_partition: dict[str, object] | None = None, max_retries: int = 0, id_namespace: LRunIds | None = None, ) -> TraceEvent: @@ -228,12 +241,19 @@ def run_l2_revision_query_planner( ) prompt_ref = "songryeon_core/prompts/l2_revision_query_setter_v0.md" prompt = Path(prompt_ref).read_text(encoding="utf-8") + supplied_available_tools = available_tools or _default_l2_available_tools(revision=True) + allowed_target_tools = _allowed_l2_target_tools_from_available( + supplied_available_tools, + revision=True, + ) input_payload = { "planner_mode": "revision_query_plan", "revision_input_data_id": revision_input_data_id, "revision_input": revision_input, "source_data_ids": source_data_ids, - "available_tools": available_tools or [{"name": "search_docs", "read_only": True}], + "l_tool_scope": l_tool_scope or {}, + "budget_partition": budget_partition or {}, + "available_tools": supplied_available_tools, } llm_result = LLMNodeExecutor(adapter).run( node_id="L2", @@ -246,7 +266,11 @@ def run_l2_revision_query_planner( input_ref=source_trace_ids, source_data_ids=source_data_ids, max_retries=max_retries, - payload_validator=_validate_l2_revision_query_plan_payload, + payload_validator=lambda payload: _validate_l2_revision_query_plan_payload( + payload, + revision_input=revision_input, + allowed_target_tools=allowed_target_tools, + ), ) if llm_result.failure_type != "none" or llm_result.validation.payload is None: raise ValueError(f"L2 revision query planner failed: {llm_result.failure_type}") @@ -269,6 +293,11 @@ def run_l2_revision_query_planner( source_data_ids=_unique_strings(frame_source_data_ids), frame_id=frame_id, default_planner_mode="revision_llm", + allowed_target_tools=allowed_target_tools, + ) + _validate_l2_revision_query_plan_against_input( + frame, + revision_input=revision_input, ) validate_l2_query_plan_frame(frame) event = trace_store.create_event( @@ -339,7 +368,7 @@ def run_l2_revision_query_setter( turn_id=turn_id, query_text=selected_query, query_source=query_source, - query_mode="exact_artifact_ref" if selected_tool == "read_artifact" else "embedding_search", + query_mode=_query_mode_for_tool(selected_tool), target_tool_name=selected_tool, source_trace_ids=source_trace_ids, source_data_ids=source_data_ids, @@ -403,7 +432,11 @@ def selected_target_tool_from_plan(payload: object) -> str: raise ValueError("selected L2 query candidate target tool was not found") -def _validate_l2_query_plan_payload(payload: dict[str, object]) -> None: +def _validate_l2_query_plan_payload( + payload: dict[str, object], + *, + allowed_target_tools: set[str] | None = None, +) -> None: """LLM raw payload가 L2QueryPlanFrame으로 바뀔 수 있는지 확인한다.""" frame = _build_query_plan_frame_from_payload( @@ -411,11 +444,17 @@ def _validate_l2_query_plan_payload(payload: dict[str, object]) -> None: turn_id="validation_turn", source_trace_ids=["validation_trace"], source_data_ids=["validation_data"], + allowed_target_tools=allowed_target_tools, ) validate_l2_query_plan_frame(frame) -def _validate_l2_revision_query_plan_payload(payload: dict[str, object]) -> None: +def _validate_l2_revision_query_plan_payload( + payload: dict[str, object], + *, + revision_input: dict[str, object], + allowed_target_tools: set[str] | None = None, +) -> None: """LLM raw payload가 revision query plan frame으로 바뀔 수 있는지 확인한다.""" frame = _build_query_plan_frame_from_payload( @@ -425,6 +464,11 @@ def _validate_l2_revision_query_plan_payload(payload: dict[str, object]) -> None source_data_ids=["validation_data"], frame_id="L2:revision_query_plan:0001", default_planner_mode="revision_llm", + allowed_target_tools=allowed_target_tools, + ) + _validate_l2_revision_query_plan_against_input( + frame, + revision_input=revision_input, ) validate_l2_query_plan_frame(frame) @@ -437,19 +481,30 @@ def _build_query_plan_frame_from_payload( source_data_ids: list[str], frame_id: str = L2_QUERY_PLAN_FRAME_DATA_ID, default_planner_mode: str = "llm", + allowed_target_tools: set[str] | None = None, ) -> L2QueryPlanFrame: candidates_payload = payload.get("candidates") if not isinstance(candidates_payload, list): raise ValueError("L2 query plan candidates must be a list") + planner_mode = str(payload.get("planner_mode") or default_planner_mode) + schema_allowed_target_tools = ( + L2_REVISION_TARGET_TOOL_NAMES + if planner_mode in {"revision_llm", "revision_fallback"} + else L2_TARGET_TOOL_NAMES + ) + if allowed_target_tools is not None: + effective_allowed_target_tools = set(allowed_target_tools) & set(schema_allowed_target_tools) + else: + effective_allowed_target_tools = set(schema_allowed_target_tools) candidates: list[L2QueryPlanCandidate] = [] for raw_candidate in candidates_payload: if not isinstance(raw_candidate, dict): continue target_tool_name = str(raw_candidate.get("target_tool_name") or "search_docs") - # L2는 검색어 계획 노드다. 도구 목록에 list_docs/read_doc이 있어도 - # 현재 L2QueryPlanFrame에는 실제 검색으로 이어질 search_docs 후보만 남긴다. - if target_tool_name not in L2_TARGET_TOOL_NAMES: + # 일반 L2는 검색/명시 문서 참조만 허용한다. + # revision L2만 이미 확보된 unread candidate doc_id를 read_doc으로 고를 수 있다. + if target_tool_name not in effective_allowed_target_tools: continue raw_source_data_ids = raw_candidate.get("source_data_ids") candidate_source_data_ids = ( @@ -476,7 +531,7 @@ def _build_query_plan_frame_from_payload( return L2QueryPlanFrame( frame_id=frame_id, turn_id=turn_id, - planner_mode=str(payload.get("planner_mode") or default_planner_mode), + planner_mode=planner_mode, selected_candidate_id=selected_candidate_id, candidates=candidates, source_trace_ids=source_trace_ids, @@ -484,6 +539,91 @@ def _build_query_plan_frame_from_payload( ) +def _query_mode_for_tool(target_tool_name: str) -> str: + if target_tool_name == "read_doc": + return "direct_doc_read" + if target_tool_name == "read_artifact": + return "exact_artifact_ref" + if target_tool_name == "list_code_files": + return "code_file_list" + if target_tool_name == "search_code": + return "code_search" + if target_tool_name == "read_code_file": + return "code_file_read" + return "embedding_search" + + +def _allowed_l2_target_tools_from_available( + available_tools: list[dict[str, object]], + *, + revision: bool = False, +) -> set[str]: + schema_allowed = L2_REVISION_TARGET_TOOL_NAMES if revision else L2_TARGET_TOOL_NAMES + result: set[str] = set() + for item in available_tools: + if not isinstance(item, dict): + continue + name = item.get("tool_name") or item.get("name") + if isinstance(name, str) and name in schema_allowed: + result.add(name) + return result + + +def _default_l2_available_tools(*, revision: bool = False) -> list[dict[str, object]]: + tools: list[dict[str, object]] = [{"tool_name": "search_docs", "read_only": True}] + if revision: + tools.append({"tool_name": "read_doc", "read_only": True}) + return tools + + +def _validate_l2_revision_query_plan_against_input( + frame: L2QueryPlanFrame, + *, + revision_input: dict[str, object], +) -> None: + """Revision plan이 L2RevisionInputFrame의 후보/예산 경계를 넘지 않는지 확인한다.""" + + unread_doc_ids = _allowed_revision_read_doc_ids(revision_input) + remaining_query_attempts = _int(revision_input.get("remaining_query_attempts")) + remaining_read_doc_calls = _int(revision_input.get("remaining_read_doc_calls")) + + if remaining_query_attempts <= 0: + for candidate in frame.candidates: + if candidate.target_tool_name != "read_doc": + raise ValueError( + "L2 revision plan must target read_doc when remaining_query_attempts is 0" + ) + + for candidate in frame.candidates: + if candidate.target_tool_name == "read_doc": + if remaining_read_doc_calls <= 0: + raise ValueError("L2 revision read_doc candidate requires remaining read_doc budget") + if candidate.query_text not in unread_doc_ids: + raise ValueError("L2 revision read_doc candidate must use an unread candidate doc_id") + elif candidate.target_tool_name == "search_docs" and remaining_query_attempts <= 0: + raise ValueError("L2 revision search_docs candidate requires remaining query budget") + elif candidate.target_tool_name == "read_artifact" and remaining_query_attempts <= 0: + raise ValueError("L2 revision read_artifact candidate is not allowed after query budget exhaustion") + + +def _allowed_revision_read_doc_ids(revision_input: dict[str, object]) -> set[str]: + doc_ids = set(_string_list(revision_input.get("unread_candidate_doc_ids"))) + if doc_ids: + return doc_ids + for summary in _string_list(revision_input.get("unread_candidate_summaries")): + marker = "doc_id=" + if marker not in summary: + continue + doc_id = summary.split(marker, 1)[1].split(";", 1)[0].strip() + if doc_id: + doc_ids.add(doc_id) + return doc_ids + + +def _int(value: object) -> int: + return value if isinstance(value, int) else 0 + + def _string_list(value: object) -> list[str]: if not isinstance(value, list): return [] diff --git a/songryeon_core/nodes/l2_revision_input.py b/songryeon_core/nodes/l2_revision_input.py index 47b20e2..8f89be4 100644 --- a/songryeon_core/nodes/l2_revision_input.py +++ b/songryeon_core/nodes/l2_revision_input.py @@ -145,6 +145,7 @@ def record_l2_revision_input_frame( previous_query_text=_text(l2_payload, "query_text", fallback="unknown_query"), previous_tool_name=_previous_tool_name(data_store=data_store, l2_payload=l2_payload), read_document_names=read_doc_ids, + unread_candidate_doc_ids=unread_candidate_doc_ids, unread_candidate_summaries=unread_candidate_summaries, l3_goal_status=_text(l3_payload, "achievement_status", fallback="not_run"), l3_goal_match_status=_text(l3_payload, "goal_match_status", fallback="not_run"), diff --git a/songryeon_core/nodes/l3_result_keeper.py b/songryeon_core/nodes/l3_result_keeper.py index c2defb1..6b734c2 100644 --- a/songryeon_core/nodes/l3_result_keeper.py +++ b/songryeon_core/nodes/l3_result_keeper.py @@ -7,10 +7,12 @@ from songryeon_core.core.data_store import DataStore from songryeon_core.core.schemas import ( L3AchievementFrame, + L3PerDocumentSummaryFrame, L3PreservedInfoFrame, L3PreservedSearchCandidate, TraceEvent, validate_l3_achievement_frame, + validate_l3_per_document_summary_frame, validate_l3_preserved_info_frame, ) from songryeon_core.core.trace_store import TraceStore @@ -23,6 +25,9 @@ L3_ACHIEVEMENT_FRAME_DATA_ID = "L3:achievement_frame" L3_REVISION_PRESERVED_FRAME_DATA_ID_PREFIX = "L3:revision_preserved_info" L3_REVISION_ACHIEVEMENT_FRAME_DATA_ID_PREFIX = "L3:revision_achievement" +L3_PER_DOCUMENT_SUMMARY_FRAME_DATA_ID_PREFIX = "L3:per_document_summary" +L3_PER_DOCUMENT_SUMMARY_PROMPT_REF = "songryeon_core/prompts/l3_per_document_summary_v0.md" +L3_PER_DOCUMENT_SUMMARY_MAX_SOURCE_CHARS = 12000 def l3_revision_preserved_frame_data_id( @@ -157,6 +162,19 @@ def run_l3_result_keeper( source_trace_id=event.event_id, payload=asdict(achievement_frame), ) + if adapter is not None: + record_l3_per_document_summary_frames( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + input_trace_ids=input_ref, + input_data_ids=source_data_ids, + user_query=user_query, + adapter=adapter, + preserved_frame_data_id=preserved_frame_data_id, + target_goal_data_id=target_goal_data_id, + frame_id_prefix=f"{preserved_frame_data_id}:per_document_summary", + ) return event @@ -172,6 +190,7 @@ def run_l3_revision_result_keeper( user_query: str = "", final_control_data_id: str | None = None, l1_goal_data_id: str = "L1:goal_frame", + adapter: LLMAdapter | None = None, id_namespace: LRunIds | None = None, ) -> TraceEvent: """Re-run the L3 preservation/achievement check after one revision tool attempt. @@ -243,9 +262,252 @@ def run_l3_revision_result_keeper( source_trace_id=event.event_id, payload=asdict(achievement_frame), ) + if adapter is not None: + record_l3_per_document_summary_frames( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + input_trace_ids=input_trace_ids, + input_data_ids=input_data_ids, + user_query=user_query, + adapter=adapter, + preserved_frame_data_id=preserved_frame_id, + target_goal_data_id=l1_goal_data_id, + frame_id_prefix=f"{preserved_frame_id}:per_document_summary", + ) return event +def record_l3_per_document_summary_frames( + *, + trace_store: TraceStore, + data_store: DataStore, + turn_id: str, + input_trace_ids: list[str], + input_data_ids: list[str], + user_query: str, + adapter: LLMAdapter, + preserved_frame_data_id: str, + target_goal_data_id: str, + frame_id_prefix: str = L3_PER_DOCUMENT_SUMMARY_FRAME_DATA_ID_PREFIX, +) -> list[str]: + """실제 document extract record마다 L3 문서별 요약 frame을 기록한다.""" + + source_documents = _document_extract_records_for_ids( + data_store=data_store, + input_data_ids=input_data_ids, + ) + created_frame_ids: list[str] = [] + for index, source_document in enumerate(source_documents, start=1): + frame_id = f"{frame_id_prefix}:{index:04d}" + frame = _build_l3_per_document_summary_frame( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + frame_id=frame_id, + source_document=source_document, + input_trace_ids=input_trace_ids, + user_query=user_query, + adapter=adapter, + preserved_frame_data_id=preserved_frame_data_id, + target_goal_data_id=target_goal_data_id, + ) + validate_l3_per_document_summary_frame(frame) + event = trace_store.create_event( + turn_id=turn_id, + actor="L3", + event_type="node_output", + input_ref=frame.source_trace_ids, + output_ref=[frame_id], + schema_status="passed" if frame.summary_status == "ran" else "failed", + ) + data_store.create_record( + data_id=frame_id, + data_type="node_output:L3_per_document_summary_frame", + exists=True, + created_at=event.timestamp, + source_trace_id=event.event_id, + payload=asdict(frame), + ) + created_frame_ids.append(frame_id) + return created_frame_ids + + +def _build_l3_per_document_summary_frame( + *, + trace_store: TraceStore, + data_store: DataStore, + turn_id: str, + frame_id: str, + source_document: dict[str, object], + input_trace_ids: list[str], + user_query: str, + adapter: LLMAdapter, + preserved_frame_data_id: str, + target_goal_data_id: str, +) -> L3PerDocumentSummaryFrame: + source_document_data_id = str(source_document["source_data_id"]) + source_doc_id = str(source_document["doc_id"]) + source_text = str(source_document.get("text") or "") + source_trace_id = source_document.get("source_trace_id") + source_document_name = _document_name_from_doc_id(source_doc_id) + source_char_count = int(source_document.get("char_count") or len(source_text)) + task_source_data_ids = _unique_strings( + [source_document_data_id, target_goal_data_id, preserved_frame_data_id] + ) + source_text_for_llm = source_text[:L3_PER_DOCUMENT_SUMMARY_MAX_SOURCE_CHARS] + source_text_truncated = len(source_text) > len(source_text_for_llm) + l1_goal = _read_l1_goal_frame( + data_store=data_store, + input_data_ids=[target_goal_data_id], + ) + prompt = Path(L3_PER_DOCUMENT_SUMMARY_PROMPT_REF).read_text(encoding="utf-8") + llm_result = LLMNodeExecutor(adapter).run( + node_id="L3_document_summary", + prompt=prompt, + input_payload={ + "user_query": user_query, + "l1_goal": l1_goal, + "source_document": { + "document_name": source_document_name, + "char_count": source_char_count, + "text": source_text_for_llm, + "text_truncated": source_text_truncated, + }, + "summary_contract": { + "plain_document_summary_info_class": "relative", + "plain_document_summary_source_mode": "direct_record", + "task_relevant_summary_info_class": "mixed", + "task_relevant_summary_source_mode": "source_bundle", + }, + }, + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + prompt_ref=L3_PER_DOCUMENT_SUMMARY_PROMPT_REF, + input_ref=_unique_strings( + [ + *input_trace_ids, + source_trace_id if isinstance(source_trace_id, str) else None, + ] + ), + source_data_ids=task_source_data_ids, + payload_validator=_validate_l3_per_document_summary_payload, + ) + + frame_source_trace_ids = _unique_strings( + [ + *input_trace_ids, + source_trace_id if isinstance(source_trace_id, str) else None, + llm_result.trace_event_id, + ] + ) + frame_source_data_ids = _unique_strings( + [ + *task_source_data_ids, + llm_result.call_data_id, + ] + ) + generated_by = f"LLM:{llm_result.model_id}" + if llm_result.failure_type != "none" or llm_result.validation.payload is None: + return L3PerDocumentSummaryFrame( + frame_id=frame_id, + turn_id=turn_id, + source_document_data_id=source_document_data_id, + source_doc_id=source_doc_id, + source_document_name=source_document_name, + source_char_count=source_char_count, + summary_status="failed", + plain_summary_source_data_id=source_document_data_id, + task_relevant_summary_source_data_ids=task_source_data_ids, + generated_by=generated_by, + semantic_judgement_status="failed", + summary_failure_type=llm_result.failure_type or "unknown", + llm_call_data_id=llm_result.call_data_id, + prompt_ref=L3_PER_DOCUMENT_SUMMARY_PROMPT_REF, + source_trace_ids=frame_source_trace_ids, + source_data_ids=frame_source_data_ids, + ) + + payload = llm_result.validation.payload + summary_limit_note = str(payload.get("summary_limit_note") or "").strip() + if source_text_truncated and not summary_limit_note: + summary_limit_note = "원문이 L3 요약 입력 한도에서 잘려 요약 범위에 한계가 있다." + + return L3PerDocumentSummaryFrame( + frame_id=frame_id, + turn_id=turn_id, + source_document_data_id=source_document_data_id, + source_doc_id=source_doc_id, + source_document_name=source_document_name, + source_char_count=source_char_count, + summary_status="ran", + plain_document_summary=str(payload.get("plain_document_summary") or "").strip(), + plain_summary_source_data_id=source_document_data_id, + task_relevant_summary=str(payload.get("task_relevant_summary") or "").strip(), + task_relevant_summary_source_data_ids=task_source_data_ids, + summary_limit_note=summary_limit_note, + generated_by=generated_by, + semantic_judgement_status="ran", + summary_failure_type="none", + llm_call_data_id=llm_result.call_data_id, + prompt_ref=L3_PER_DOCUMENT_SUMMARY_PROMPT_REF, + source_trace_ids=frame_source_trace_ids, + source_data_ids=frame_source_data_ids, + ) + + +def _validate_l3_per_document_summary_payload(payload: dict[str, object]) -> None: + for field_name in ("plain_document_summary", "task_relevant_summary"): + value = payload.get(field_name) + if not isinstance(value, str) or not value.strip(): + raise ValueError(f"L3 document summary payload {field_name} must not be empty") + limit_note = payload.get("summary_limit_note") + if limit_note is not None and not isinstance(limit_note, str): + raise TypeError("L3 document summary payload summary_limit_note must be a string") + + +def _document_extract_records_for_ids( + *, + data_store: DataStore, + input_data_ids: list[str], +) -> list[dict[str, object]]: + documents: list[dict[str, object]] = [] + seen_data_ids: set[str] = set() + for data_id in input_data_ids: + if data_id in seen_data_ids: + continue + seen_data_ids.add(data_id) + record = data_store.get_record(data_id) + if record is None or not _is_document_extract_record(record.data_type): + continue + payload = record.payload + if not isinstance(payload, dict): + continue + text = payload.get("text") + doc_id = payload.get("doc_id") + if not isinstance(text, str) or not text.strip(): + continue + if not isinstance(doc_id, str) or not doc_id.strip(): + continue + char_count = payload.get("char_count") + documents.append( + { + "source_data_id": record.data_id, + "source_trace_id": record.source_trace_id, + "doc_id": doc_id.strip(), + "text": text, + "char_count": char_count if isinstance(char_count, int) else len(text), + } + ) + return documents + + +def _document_name_from_doc_id(doc_id: str) -> str: + normalized = doc_id.replace("\\", "/").strip() + return normalized.rsplit("/", 1)[-1] if normalized else doc_id + + def _build_preserved_frame( *, frame_id: str, @@ -365,6 +627,33 @@ def _build_achievement_frame( if achievement_status != original_status: generation_source = f"{generation_source}+CODE:GOAL_MATCH_GUARD" + status_before_l1_requirement_guard = achievement_status + ( + achievement_status, + reason, + macro_status, + macro_reason, + micro_status, + micro_reason, + ) = _apply_l1_requirement_count_guard( + achievement_status=achievement_status, + reason=reason, + macro_status=macro_status, + macro_reason=macro_reason, + micro_status=micro_status, + micro_reason=micro_reason, + l1_goal=l1_goal, + read_document_count=len(goal_match["read_doc_ids"]), + read_code_file_count=len(goal_match["read_code_file_paths"]), + source_code_evidence_expected=_source_code_evidence_expected( + l1_goal=l1_goal, + data_store=data_store, + input_data_ids=input_data_ids, + ), + ) + if achievement_status != status_before_l1_requirement_guard: + generation_source = f"{generation_source}+CODE:L1_REQUIREMENT_COUNT_GUARD" + evidence_data_ids = _unique_strings( [ *input_data_ids, @@ -396,6 +685,8 @@ def _build_achievement_frame( micro_achievement_reason=micro_reason, requested_doc_hint=str(goal_match["requested_doc_hint"]), read_doc_ids=list(goal_match["read_doc_ids"]), + read_code_file_paths=list(goal_match["read_code_file_paths"]), + actual_read_code_file_count=len(goal_match["read_code_file_paths"]), search_result_doc_ids=list(goal_match["search_result_doc_ids"]), goal_match_status=str(goal_match["goal_match_status"]), goal_match_reason=str(goal_match["goal_match_reason"]), @@ -433,6 +724,7 @@ def _build_llm_achievement_frame( data_store=data_store, ) read_doc_ids = list(goal_match["read_doc_ids"]) + read_code_file_paths = list(goal_match["read_code_file_paths"]) search_result_doc_ids = list(goal_match["search_result_doc_ids"]) input_payload = { "user_query": user_query, @@ -446,11 +738,14 @@ def _build_llm_achievement_frame( "preserved_candidate_count": len(preserved_frame.candidates), "unique_search_result_document_count": len(search_result_doc_ids), "read_document_count": len(read_doc_ids), + "read_code_file_count": len(read_code_file_paths), }, "read_doc_ids": read_doc_ids, + "read_code_file_paths": read_code_file_paths, "search_result_doc_ids": search_result_doc_ids, "specific_document_request": goal_match, "read_document_previews": _read_doc_previews_from_data_store(data_store), + "read_code_file_previews": _read_code_file_previews_from_data_store(data_store), "candidate_previews": [ { "candidate_id": candidate.candidate_id, @@ -561,6 +856,12 @@ def _build_llm_achievement_frame( micro_reason=micro_reason, l1_goal=l1_goal, read_document_count=len(read_doc_ids), + read_code_file_count=len(read_code_file_paths), + source_code_evidence_expected=_source_code_evidence_expected( + l1_goal=l1_goal, + data_store=data_store, + input_data_ids=input_data_ids, + ), ) if achievement_status != status_before_l1_requirement_guard: generation_source = f"{generation_source}+CODE:L1_REQUIREMENT_COUNT_GUARD" @@ -589,6 +890,8 @@ def _build_llm_achievement_frame( micro_achievement_reason=micro_reason, requested_doc_hint=str(goal_match["requested_doc_hint"]), read_doc_ids=list(goal_match["read_doc_ids"]), + read_code_file_paths=list(goal_match["read_code_file_paths"]), + actual_read_code_file_count=len(goal_match["read_code_file_paths"]), search_result_doc_ids=list(goal_match["search_result_doc_ids"]), goal_match_status=str(goal_match["goal_match_status"]), goal_match_reason=str(goal_match["goal_match_reason"]), @@ -623,7 +926,9 @@ def _l3_judgement_contract(l1_goal: dict[str, object]) -> list[str]: evidence_kind = str(l1_goal.get("evidence_requirement_kind") or "unspecified") contract = [ "Judge the current user_query and L1 success condition, not whether a read document describes some implementation success.", - "Use read_document_count and read_doc_ids as the only proof that original document text was read.", + "Use read_document_count and read_doc_ids as proof that original Markdown/document text was read.", + "Use read_code_file_count and read_code_file_paths as proof that original source/config code text was read.", + "Do not rename source-code evidence into read_doc evidence; keep document evidence and source-code evidence separate.", "If the evidence kind is exploratory or multi-document, judge whether the read documents can support cross-document relationship analysis.", ] if evidence_kind in {"exploratory_multi_doc", "multi_doc_relationship"}: @@ -780,6 +1085,7 @@ def _build_goal_match_context( requested_doc_hint = _extract_requested_doc_hint(user_query) read_doc_ids = _read_doc_ids_from_data_store(data_store) + read_code_file_paths = _read_code_file_paths_from_data_store(data_store) search_result_doc_ids = _unique_strings( [candidate.doc_id for candidate in preserved_frame.candidates if candidate.doc_id] ) @@ -788,6 +1094,7 @@ def _build_goal_match_context( return { "requested_doc_hint": "", "read_doc_ids": read_doc_ids, + "read_code_file_paths": read_code_file_paths, "search_result_doc_ids": search_result_doc_ids, "goal_match_status": "not_applicable", "goal_match_reason": "CODE_STATUS:no_specific_doc_hint_detected", @@ -797,24 +1104,37 @@ def _build_goal_match_context( return { "requested_doc_hint": requested_doc_hint, "read_doc_ids": read_doc_ids, + "read_code_file_paths": read_code_file_paths, "search_result_doc_ids": search_result_doc_ids, "goal_match_status": "matched", "goal_match_reason": "CODE_STATUS:requested_doc_read_doc_matched", } + if any(_doc_matches_hint(file_path, requested_doc_hint) for file_path in read_code_file_paths): + return { + "requested_doc_hint": requested_doc_hint, + "read_doc_ids": read_doc_ids, + "read_code_file_paths": read_code_file_paths, + "search_result_doc_ids": search_result_doc_ids, + "goal_match_status": "matched", + "goal_match_reason": "CODE_STATUS:requested_source_code_file_read_code_file_matched", + } + if any(_doc_matches_hint(doc_id, requested_doc_hint) for doc_id in search_result_doc_ids): return { "requested_doc_hint": requested_doc_hint, "read_doc_ids": read_doc_ids, + "read_code_file_paths": read_code_file_paths, "search_result_doc_ids": search_result_doc_ids, "goal_match_status": "partial", "goal_match_reason": "CODE_STATUS:requested_doc_found_in_search_results_but_not_read", } - if read_doc_ids or search_result_doc_ids: + if read_doc_ids or read_code_file_paths or search_result_doc_ids: return { "requested_doc_hint": requested_doc_hint, "read_doc_ids": read_doc_ids, + "read_code_file_paths": read_code_file_paths, "search_result_doc_ids": search_result_doc_ids, "goal_match_status": "partial", "goal_match_reason": "CODE_STATUS:requested_doc_not_matched_but_l_loop_has_other_evidence", @@ -823,6 +1143,7 @@ def _build_goal_match_context( return { "requested_doc_hint": requested_doc_hint, "read_doc_ids": read_doc_ids, + "read_code_file_paths": read_code_file_paths, "search_result_doc_ids": search_result_doc_ids, "goal_match_status": "missing", "goal_match_reason": "CODE_STATUS:requested_doc_not_matched_and_no_l_loop_evidence", @@ -916,8 +1237,10 @@ def _apply_l1_requirement_count_guard( micro_reason: str, l1_goal: dict[str, object], read_document_count: int, + read_code_file_count: int, + source_code_evidence_expected: bool, ) -> tuple[str, str, str, str, str, str]: - """L1의 최소 원문 열람 요구보다 실제 read_doc 수가 적으면 achieved를 낮춘다.""" + """L1의 최소 원문 열람 요구보다 실제 source evidence 수가 적으면 achieved를 낮춘다.""" minimum_read_documents = l1_goal.get("minimum_read_documents") if not isinstance(minimum_read_documents, int) or minimum_read_documents <= 0: @@ -929,7 +1252,10 @@ def _apply_l1_requirement_count_guard( micro_status, micro_reason, ) - if read_document_count >= minimum_read_documents: + actual_evidence_count = read_document_count + if source_code_evidence_expected: + actual_evidence_count += read_code_file_count + if actual_evidence_count >= minimum_read_documents: return ( achievement_status, reason, @@ -941,7 +1267,7 @@ def _apply_l1_requirement_count_guard( guard_reason = ( "CODE_STATUS:l1_minimum_read_documents_not_met" - f":required_{minimum_read_documents}_actual_{read_document_count}" + f":required_{minimum_read_documents}_actual_{actual_evidence_count}" ) reason = _append_guard_reason(reason, guard_reason) macro_reason = _append_guard_reason(macro_reason, guard_reason) @@ -1029,6 +1355,28 @@ def _read_doc_ids_from_data_store(data_store: DataStore | None) -> list[str]: return _unique_strings(doc_ids) +def _read_code_file_paths_from_data_store(data_store: DataStore | None) -> list[str]: + if data_store is None: + return [] + + paths: list[str] = [] + for record in data_store.list_records(): + if not _is_code_extract_record(record.data_type): + continue + payload = record.payload + if not isinstance(payload, dict): + continue + if payload.get("read_status") != "ok": + continue + text = payload.get("text") + if not isinstance(text, str) or not text.strip(): + continue + file_path = payload.get("file_path") + if isinstance(file_path, str) and file_path: + paths.append(file_path) + return _unique_strings(paths) + + def _read_doc_previews_from_data_store( data_store: DataStore | None, *, @@ -1066,10 +1414,82 @@ def _read_doc_previews_from_data_store( return previews +def _read_code_file_previews_from_data_store( + data_store: DataStore | None, + *, + max_files: int = 3, + max_text_chars: int = 1200, +) -> list[dict[str, object]]: + """Package read_code_file outputs for L3 LLM judgement without interpretation.""" + + if data_store is None: + return [] + + previews: list[dict[str, object]] = [] + for record in data_store.list_records(): + if not _is_code_extract_record(record.data_type): + continue + payload = record.payload + if not isinstance(payload, dict): + continue + if payload.get("read_status") != "ok": + continue + file_path = payload.get("file_path") + text = payload.get("text") + if not isinstance(file_path, str) or not file_path: + continue + if not isinstance(text, str): + text = "" + previews.append( + { + "source_data_id": record.data_id, + "file_path": file_path, + "char_count": payload.get("char_count"), + "line_count": payload.get("line_count"), + "text_preview": text[:max_text_chars], + } + ) + if len(previews) >= max_files: + break + return previews + + def _is_document_extract_record(data_type: str) -> bool: return data_type.startswith("tool_result:read_doc") or data_type.startswith("tool_result:read_artifact") +def _is_code_extract_record(data_type: str) -> bool: + return data_type.startswith("tool_result:read_code_file") + + +def _source_code_evidence_expected( + *, + l1_goal: dict[str, object], + data_store: DataStore | None, + input_data_ids: list[str], +) -> bool: + if _string_in_list(l1_goal.get("required_materials"), "source_code_file"): + return True + if data_store is None: + return False + for data_id in input_data_ids: + record = data_store.get_record(data_id) + if record is None or record.data_type != "node_output:L_tool_scope_frame": + continue + payload = record.payload + if not isinstance(payload, dict): + continue + if payload.get("tool_scope_mode") in {"code_only", "document_and_code", "mixed_evidence"}: + return True + if _string_in_list(payload.get("required_materials"), "source_code_file"): + return True + return False + + +def _string_in_list(value: object, expected: str) -> bool: + return isinstance(value, list) and any(item == expected for item in value) + + def _doc_matches_hint(doc_id: str, hint: str) -> bool: normalized_doc_id = _normalize_doc_token(doc_id) normalized_hint = _normalize_doc_token(hint) diff --git a/songryeon_core/nodes/l_tool_scope.py b/songryeon_core/nodes/l_tool_scope.py new file mode 100644 index 0000000..c2aab46 --- /dev/null +++ b/songryeon_core/nodes/l_tool_scope.py @@ -0,0 +1,401 @@ +from __future__ import annotations + +from dataclasses import asdict +from pathlib import Path + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.schemas import ( + L_REQUIRED_MATERIALS, + L_TOOL_GROUPS, + L_TOOL_SCOPE_MODES, + LToolBudgetPartitionFrame, + LToolScopeFrame, + TraceEvent, + validate_l_tool_budget_partition_frame, + validate_l_tool_scope_frame, +) +from songryeon_core.core.trace_store import TraceStore +from songryeon_core.llm.base import LLMAdapter +from songryeon_core.llm.node_executor import LLMNodeExecutor +from songryeon_core.loops.l_loop_namespace import LRunIds + + +L_TOOL_SCOPE_FRAME_DATA_ID = "L:tool_scope_frame" +L_TOOL_BUDGET_PARTITION_FRAME_DATA_ID = "L:tool_budget_partition_frame" +L_TOOL_SCOPE_PROMPT_REF = "songryeon_core/prompts/l_tool_scope_planner_v0.md" + +DOCUMENT_TOOL_NAMES = {"list_docs", "search_docs", "read_doc", "read_artifact"} +CODE_INSPECTION_TOOL_NAMES = {"list_code_files", "search_code", "read_code_file"} +RUNTIME_RECORD_TOOL_NAMES: set[str] = set() + + +def run_l_tool_scope_planner( + *, + trace_store: TraceStore, + data_store: DataStore, + turn_id: str, + l1_event: TraceEvent, + user_query: str, + goal_data_id: str, + budget_plan_data_id: str, + tool_catalog_data_id: str, + available_tools: list[dict[str, object]], + adapter: LLMAdapter | None, + id_namespace: LRunIds | None = None, +) -> tuple[str, str, LToolScopeFrame]: + """L2 전에 이번 L루프의 허용 도구군을 명시 frame으로 기록한다.""" + + frame_id = _scope_frame_data_id(id_namespace=id_namespace) + source_trace_ids = [l1_event.event_id] + source_data_ids = [goal_data_id, budget_plan_data_id, tool_catalog_data_id] + llm_call_trace_id: str | None = None + llm_call_data_id: str | None = None + failure_type = "adapter_missing" + + if adapter is not None: + prompt = Path(L_TOOL_SCOPE_PROMPT_REF).read_text(encoding="utf-8") + llm_result = LLMNodeExecutor(adapter).run( + node_id="L_tool_scope", + prompt=prompt, + input_payload={ + "user_query": user_query, + "l1_goal": _payload(data_store, goal_data_id), + "budget_plan": _payload(data_store, budget_plan_data_id), + "available_tools": available_tools, + "allowed_tool_scope_modes": sorted(L_TOOL_SCOPE_MODES), + "allowed_tool_groups": sorted(L_TOOL_GROUPS), + "allowed_required_materials": sorted(L_REQUIRED_MATERIALS), + }, + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + prompt_ref=L_TOOL_SCOPE_PROMPT_REF, + input_ref=source_trace_ids, + source_data_ids=source_data_ids, + payload_validator=lambda payload: _validate_scope_payload( + payload=payload, + turn_id=turn_id, + frame_id=frame_id, + source_trace_ids=["validation_trace"], + source_data_ids=["validation_data"], + ), + ) + llm_call_trace_id = llm_result.trace_event_id + llm_call_data_id = llm_result.call_data_id + failure_type = llm_result.failure_type + if llm_result.failure_type == "none" and llm_result.validation.payload is not None: + frame = _frame_from_payload( + payload=llm_result.validation.payload, + turn_id=turn_id, + frame_id=frame_id, + source_trace_ids=_unique_strings([*source_trace_ids, llm_call_trace_id]), + source_data_ids=_unique_strings([*source_data_ids, llm_call_data_id]), + generated_by=f"LLM:{llm_result.model_id}", + llm_call_data_id=llm_call_data_id, + ) + return _record_scope_frame( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + frame=frame, + ) + + frame = _fallback_scope_frame( + turn_id=turn_id, + frame_id=frame_id, + source_trace_ids=_unique_strings([*source_trace_ids, llm_call_trace_id]), + source_data_ids=_unique_strings([*source_data_ids, llm_call_data_id]), + failure_type=failure_type, + llm_call_data_id=llm_call_data_id, + ) + return _record_scope_frame( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + frame=frame, + ) + + +def record_l_tool_budget_partition( + *, + trace_store: TraceStore, + data_store: DataStore, + turn_id: str, + tool_scope_frame: LToolScopeFrame, + tool_scope_trace_id: str, + budget_plan_data_id: str, + budget_plan_trace_id: str, + id_namespace: LRunIds | None = None, +) -> tuple[str, str, LToolBudgetPartitionFrame]: + """승인된 L 예산을 tool scope에 따라 도구군별로 나눠 기록한다.""" + + budget_plan = _payload(data_store, budget_plan_data_id) + frame_id = _budget_partition_frame_data_id(id_namespace=id_namespace) + frame = _partition_frame( + turn_id=turn_id, + frame_id=frame_id, + tool_scope_frame=tool_scope_frame, + budget_plan_data_id=budget_plan_data_id, + budget_plan=budget_plan, + source_trace_ids=_unique_strings([tool_scope_trace_id, budget_plan_trace_id]), + source_data_ids=_unique_strings([tool_scope_frame.frame_id, budget_plan_data_id]), + ) + validate_l_tool_budget_partition_frame(frame) + event = trace_store.create_event( + turn_id=turn_id, + actor="CODE:L_TOOL_BUDGET_PARTITION_POLICY", + event_type="node_output", + input_ref=frame.source_trace_ids, + output_ref=[frame_id], + schema_status="passed", + ) + data_store.create_record( + data_id=frame_id, + data_type="node_output:L_tool_budget_partition_frame", + exists=True, + created_at=event.timestamp, + source_trace_id=event.event_id, + payload=asdict(frame), + ) + return event.event_id, frame_id, frame + + +def filter_available_tools_for_scope( + available_tools: list[dict[str, object]], + tool_scope_frame: LToolScopeFrame, +) -> list[dict[str, object]]: + """LToolScopeFrame이 허용한 tool group에 속한 도구만 L2에 노출한다.""" + + allowed_names = allowed_tool_names_for_groups(tool_scope_frame.allowed_tool_groups) + return [ + tool + for tool in available_tools + if isinstance(tool.get("tool_name") or tool.get("name"), str) + and str(tool.get("tool_name") or tool.get("name")) in allowed_names + ] + + +def allowed_tool_names_for_groups(groups: list[str]) -> set[str]: + allowed: set[str] = set() + if "document_tools" in groups: + allowed.update(DOCUMENT_TOOL_NAMES) + if "code_inspection_tools" in groups: + allowed.update(CODE_INSPECTION_TOOL_NAMES) + if "runtime_record_tools" in groups: + allowed.update(RUNTIME_RECORD_TOOL_NAMES) + return allowed + + +def _scope_frame_data_id(*, id_namespace: LRunIds | None) -> str: + if id_namespace is None: + return L_TOOL_SCOPE_FRAME_DATA_ID + return id_namespace.scoped_data_id(L_TOOL_SCOPE_FRAME_DATA_ID) + + +def _budget_partition_frame_data_id(*, id_namespace: LRunIds | None) -> str: + if id_namespace is None: + return L_TOOL_BUDGET_PARTITION_FRAME_DATA_ID + return id_namespace.scoped_data_id(L_TOOL_BUDGET_PARTITION_FRAME_DATA_ID) + + +def _validate_scope_payload( + *, + payload: dict[str, object], + turn_id: str, + frame_id: str, + source_trace_ids: list[str], + source_data_ids: list[str], +) -> None: + frame = _frame_from_payload( + payload=payload, + turn_id=turn_id, + frame_id=frame_id, + source_trace_ids=source_trace_ids, + source_data_ids=source_data_ids, + generated_by="LLM:validation-model", + llm_call_data_id=None, + ) + validate_l_tool_scope_frame(frame) + + +def _frame_from_payload( + *, + payload: dict[str, object], + turn_id: str, + frame_id: str, + source_trace_ids: list[str], + source_data_ids: list[str], + generated_by: str, + llm_call_data_id: str | None, +) -> LToolScopeFrame: + return LToolScopeFrame( + frame_id=frame_id, + turn_id=turn_id, + tool_scope_mode=str(payload.get("tool_scope_mode") or "").strip(), + allowed_tool_groups=_string_list(payload.get("allowed_tool_groups")), + required_materials=_string_list(payload.get("required_materials")), + scope_reason=str(payload.get("scope_reason") or "").strip(), + scope_reason_info_class=str(payload.get("scope_reason_info_class") or "mixed").strip(), + generated_by=generated_by, + info_class="mixed", + semantic_judgement_status="ran", + scope_failure_type="none", + llm_call_data_id=llm_call_data_id, + prompt_ref=L_TOOL_SCOPE_PROMPT_REF, + source_trace_ids=source_trace_ids, + source_data_ids=source_data_ids, + ) + + +def _fallback_scope_frame( + *, + turn_id: str, + frame_id: str, + source_trace_ids: list[str], + source_data_ids: list[str], + failure_type: str, + llm_call_data_id: str | None, +) -> LToolScopeFrame: + return LToolScopeFrame( + frame_id=frame_id, + turn_id=turn_id, + tool_scope_mode="document_only", + allowed_tool_groups=["document_tools"], + required_materials=["project_document"], + scope_reason="CODE_STATUS:l_tool_scope_selection_failed_compat_document_only", + scope_reason_info_class="absolute_status", + generated_by="CODE:FALLBACK", + info_class="absolute_status", + semantic_judgement_status="failed", + scope_failure_type=failure_type, + llm_call_data_id=llm_call_data_id, + prompt_ref=L_TOOL_SCOPE_PROMPT_REF, + source_trace_ids=source_trace_ids, + source_data_ids=source_data_ids, + ) + + +def _record_scope_frame( + *, + trace_store: TraceStore, + data_store: DataStore, + turn_id: str, + frame: LToolScopeFrame, +) -> tuple[str, str, LToolScopeFrame]: + validate_l_tool_scope_frame(frame) + event = trace_store.create_event( + turn_id=turn_id, + actor="L_tool_scope", + event_type="node_output", + input_ref=frame.source_trace_ids, + output_ref=[frame.frame_id], + schema_status="passed" if frame.semantic_judgement_status == "ran" else "failed", + ) + data_store.create_record( + data_id=frame.frame_id, + data_type="node_output:L_tool_scope_frame", + exists=True, + created_at=event.timestamp, + source_trace_id=event.event_id, + payload=asdict(frame), + ) + return event.event_id, frame.frame_id, frame + + +def _partition_frame( + *, + turn_id: str, + frame_id: str, + tool_scope_frame: LToolScopeFrame, + budget_plan_data_id: str, + budget_plan: dict[str, object], + source_trace_ids: list[str], + source_data_ids: list[str], +) -> LToolBudgetPartitionFrame: + tool_calls = _int(budget_plan.get("approved_max_tool_calls")) + query_attempts = _int(budget_plan.get("approved_max_query_attempts")) + read_calls = _int(budget_plan.get("approved_max_read_doc_calls")) + mode = tool_scope_frame.tool_scope_mode + + document_tool = document_query = document_read = 0 + code_tool = code_query = code_read = 0 + runtime_budget = 0 + + if mode == "document_only": + document_tool, document_query, document_read = tool_calls, query_attempts, read_calls + elif mode == "code_only": + code_tool, code_query, code_read = tool_calls, query_attempts, read_calls + elif mode in {"document_and_code", "mixed_evidence"}: + if "document_tools" in tool_scope_frame.allowed_tool_groups and "code_inspection_tools" in tool_scope_frame.allowed_tool_groups: + document_tool, code_tool = _split_budget(tool_calls) + document_query, code_query = _split_budget(query_attempts) + document_read, code_read = _split_budget(read_calls) + elif "document_tools" in tool_scope_frame.allowed_tool_groups: + document_tool, document_query, document_read = tool_calls, query_attempts, read_calls + elif "code_inspection_tools" in tool_scope_frame.allowed_tool_groups: + code_tool, code_query, code_read = tool_calls, query_attempts, read_calls + if "runtime_record_tools" in tool_scope_frame.allowed_tool_groups: + runtime_budget = 1 + elif mode == "runtime_trace_only": + runtime_budget = 1 + + return LToolBudgetPartitionFrame( + frame_id=frame_id, + turn_id=turn_id, + tool_scope_frame_id=tool_scope_frame.frame_id, + budget_plan_frame_id=budget_plan_data_id, + tool_scope_mode=mode, + allowed_tool_groups=list(tool_scope_frame.allowed_tool_groups), + document_tool_call_budget=document_tool, + document_query_budget=document_query, + document_read_budget=document_read, + code_tool_call_budget=code_tool, + code_query_budget=code_query, + code_read_budget=code_read, + runtime_record_budget=runtime_budget, + partition_reason=f"CODE_STATUS:partition_from_l_tool_scope_mode:{mode}", + source_trace_ids=source_trace_ids, + source_data_ids=source_data_ids, + ) + + +def _split_budget(total: int) -> tuple[int, int]: + if total <= 0: + return 0, 0 + if total == 1: + return 1, 0 + document_budget = max(1, total // 2) + code_budget = max(1, total - document_budget) + return document_budget, code_budget + + +def _payload(data_store: DataStore, data_id: str) -> dict[str, object]: + record = data_store.get_record(data_id) + if record is None or not isinstance(record.payload, dict): + return {} + return record.payload + + +def _int(value: object) -> int: + return value if isinstance(value, int) and value >= 0 else 0 + + +def _string_list(value: object) -> list[str]: + if not isinstance(value, list): + return [] + result: list[str] = [] + for item in value: + if isinstance(item, str) and item and item not in result: + result.append(item) + return result + + +def _unique_strings(values: list[str | None]) -> list[str]: + seen: set[str] = set() + result: list[str] = [] + for value in values: + if value is None or value in seen: + continue + seen.add(value) + result.append(value) + return result diff --git a/songryeon_core/nodes/night_summarize_source_leaf.py b/songryeon_core/nodes/night_summarize_source_leaf.py new file mode 100644 index 0000000..a151258 --- /dev/null +++ b/songryeon_core/nodes/night_summarize_source_leaf.py @@ -0,0 +1,961 @@ +from __future__ import annotations + +"""Night-government worker that summarizes changed raw source leaves one-to-one.""" + +import hashlib +from dataclasses import asdict, dataclass +from pathlib import Path + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.graph_source_ingest import ( + GRAPH_SOURCE_FILE_DATA_TYPE, + GRAPH_SOURCE_TEXT_SNAPSHOT_DATA_TYPE, +) +from songryeon_core.core.schemas import ( + GraphMemoryEdgeFrame, + NightSourceLeafSummaryFrame, + SourceObservationLedgerFrame, + validate_graph_memory_edge_frame, + validate_night_source_leaf_summary_frame, + validate_source_observation_ledger_frame, +) +from songryeon_core.core.source_version_lineage import ( + SOURCE_OBSERVATION_LEDGER_FRAME_DATA_TYPE, +) +from songryeon_core.core.trace_store import TraceStore +from songryeon_core.llm.base import LLMAdapter +from songryeon_core.llm.node_executor import LLMNodeExecutor + + +NIGHT_SUMMARIZE_SOURCE_LEAF_NODE_ID = "night_summarize_source_leaf" +NIGHT_SUMMARIZE_SOURCE_LEAF_PROMPT_REF = ( + "songryeon_core/prompts/night_summarize_source_leaf_v0.md" +) +NIGHT_SUMMARIZE_SOURCE_LEAF_FRAME_DATA_TYPE = ( + "node_output:night_summarize_source_leaf_frame" +) +NIGHT_SUMMARIZE_SOURCE_LEAF_INPUT_DATA_TYPE = ( + "node_input:night_summarize_source_leaf_input" +) +NIGHT_SOURCE_LEAF_SUMMARY_BATCH_DATA_TYPE = ( + "node_output:night_summarize_changed_source_leaves_batch" +) +SUMMARIZABLE_SOURCE_OBSERVATION_STATUSES = { + "new_source_version", + "content_changed", +} + + +@dataclass(frozen=True) +class RecordedNightSourceLeafSummaryResult: + frame: NightSourceLeafSummaryFrame + trace_event_id: str + frame_data_id: str + summary_graph_node_data_id: str | None + summary_edge_data_id: str | None + created_data_ids: list[str] + existing_data_ids: list[str] + + +@dataclass(frozen=True) +class RecordedNightChangedSourceLeafSummaryBatch: + batch_data_id: str + trace_event_id: str + source_observation_ledger_frame_id: str + summary_run_id: str + selected_source_graph_node_ids: list[str] + skipped_unchanged_source_graph_node_ids: list[str] + results: list[RecordedNightSourceLeafSummaryResult] + created_data_ids: list[str] + existing_data_ids: list[str] + + +@dataclass(frozen=True) +class NightChangedSourceLeafSelection: + source_observation_ledger_frame_id: str + selected_source_graph_node_ids: list[str] + skipped_unchanged_source_graph_node_ids: list[str] + + +def night_summarize_source_leaf_frame_id( + raw_source_graph_node_id: str, + *, + summary_run_id: str, +) -> str: + _require_text("raw_source_graph_node_id", raw_source_graph_node_id) + _require_text("summary_run_id", summary_run_id) + return ( + "night_summary:source_leaf:" + f"{_stable_suffix(raw_source_graph_node_id)}:{_safe_id_part(summary_run_id)}" + ) + + +def night_summarize_source_leaf_graph_node_id( + raw_source_graph_node_id: str, + *, + summary_run_id: str, +) -> str: + _require_text("raw_source_graph_node_id", raw_source_graph_node_id) + _require_text("summary_run_id", summary_run_id) + return ( + "graph:summary:source_leaf:" + f"{_stable_suffix(raw_source_graph_node_id)}:{_safe_id_part(summary_run_id)}" + ) + + +def night_summarize_changed_source_leaves_batch_id( + source_observation_ledger_frame_id: str, + *, + summary_run_id: str, +) -> str: + _require_text("source_observation_ledger_frame_id", source_observation_ledger_frame_id) + _require_text("summary_run_id", summary_run_id) + return ( + "night_summary:changed_source_leaves:" + f"{_stable_suffix(source_observation_ledger_frame_id)}:" + f"{_safe_id_part(summary_run_id)}" + ) + + +def run_night_summarize_changed_source_leaves( + *, + trace_store: TraceStore, + data_store: DataStore, + turn_id: str, + source_observation_ledger_frame_id: str, + summary_run_id: str, + adapter: LLMAdapter | None, + max_retries: int = 0, +) -> RecordedNightChangedSourceLeafSummaryBatch: + """Summarize only new or changed raw source leaves from one observation ledger.""" + + selection = select_changed_source_leaf_summary_candidates( + data_store=data_store, + source_observation_ledger_frame_id=source_observation_ledger_frame_id, + ) + ledger = _require_source_observation_ledger( + data_store=data_store, + source_observation_ledger_frame_id=source_observation_ledger_frame_id, + ) + selected_source_graph_node_ids = selection.selected_source_graph_node_ids + skipped_unchanged_source_graph_node_ids = ( + selection.skipped_unchanged_source_graph_node_ids + ) + + results = [ + run_night_summarize_source_leaf( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + raw_source_graph_node_id=source_graph_node_id, + summary_run_id=summary_run_id, + adapter=adapter, + max_retries=max_retries, + ) + for source_graph_node_id in selected_source_graph_node_ids + ] + + batch_data_id = night_summarize_changed_source_leaves_batch_id( + source_observation_ledger_frame_id, + summary_run_id=summary_run_id, + ) + event = trace_store.create_event( + turn_id=turn_id, + actor=NIGHT_SUMMARIZE_SOURCE_LEAF_NODE_ID, + event_type="node_output", + input_ref=ledger.source_trace_ids, + output_ref=[batch_data_id], + schema_status="passed", + ) + payload = { + "batch_data_id": batch_data_id, + "source_observation_ledger_frame_id": source_observation_ledger_frame_id, + "summary_run_id": summary_run_id, + "selected_source_graph_node_ids": selected_source_graph_node_ids, + "skipped_unchanged_source_graph_node_ids": skipped_unchanged_source_graph_node_ids, + "result_frame_ids": [result.frame.frame_id for result in results], + "summary_graph_node_ids": [ + result.frame.summary_graph_node_id + for result in results + if result.summary_graph_node_data_id is not None + ], + "summary_status_counts": _count_result_statuses(results), + "generated_by": "CODE:NIGHT_SUMMARIZE_CHANGED_SOURCE_LEAVES_BATCH", + "info_class": "absolute", + "semantic_judgement_status": "not_run", + } + created_data_ids: list[str] = [] + existing_data_ids: list[str] = [] + _record_payload_if_missing( + data_store=data_store, + data_id=batch_data_id, + data_type=NIGHT_SOURCE_LEAF_SUMMARY_BATCH_DATA_TYPE, + payload=payload, + created_at=event.timestamp, + source_trace_id=event.event_id, + created_data_ids=created_data_ids, + existing_data_ids=existing_data_ids, + ) + return RecordedNightChangedSourceLeafSummaryBatch( + batch_data_id=batch_data_id, + trace_event_id=event.event_id, + source_observation_ledger_frame_id=source_observation_ledger_frame_id, + summary_run_id=summary_run_id, + selected_source_graph_node_ids=selected_source_graph_node_ids, + skipped_unchanged_source_graph_node_ids=skipped_unchanged_source_graph_node_ids, + results=results, + created_data_ids=created_data_ids, + existing_data_ids=existing_data_ids, + ) + + +def select_changed_source_leaf_summary_candidates( + *, + data_store: DataStore, + source_observation_ledger_frame_id: str, +) -> NightChangedSourceLeafSelection: + """Select summarizable raw_source leaf ids using the observation ledger only.""" + + ledger = _require_source_observation_ledger( + data_store=data_store, + source_observation_ledger_frame_id=source_observation_ledger_frame_id, + ) + selected_source_graph_node_ids: list[str] = [] + skipped_unchanged_source_graph_node_ids: list[str] = [] + for record in ledger.observation_records: + source_graph_node_id = _text(record.get("active_source_graph_node_id")) + observation_status = _text(record.get("observation_status")) + if not source_graph_node_id: + continue + if observation_status in SUMMARIZABLE_SOURCE_OBSERVATION_STATUSES: + if source_graph_node_id not in selected_source_graph_node_ids: + selected_source_graph_node_ids.append(source_graph_node_id) + elif observation_status == "unchanged": + if source_graph_node_id not in skipped_unchanged_source_graph_node_ids: + skipped_unchanged_source_graph_node_ids.append(source_graph_node_id) + return NightChangedSourceLeafSelection( + source_observation_ledger_frame_id=source_observation_ledger_frame_id, + selected_source_graph_node_ids=selected_source_graph_node_ids, + skipped_unchanged_source_graph_node_ids=skipped_unchanged_source_graph_node_ids, + ) + + +def run_night_summarize_source_leaf( + *, + trace_store: TraceStore, + data_store: DataStore, + turn_id: str, + raw_source_graph_node_id: str, + summary_run_id: str, + adapter: LLMAdapter | None, + max_retries: int = 0, +) -> RecordedNightSourceLeafSummaryResult: + """Create an LLM summary graph node for exactly one raw_source graph leaf.""" + + target_payload = _require_graph_node_payload( + data_store=data_store, + graph_node_id=raw_source_graph_node_id, + expected_node_kind="raw_source", + ) + source_file_record = _require_first_source_record( + data_store=data_store, + target_payload=target_payload, + data_type=GRAPH_SOURCE_FILE_DATA_TYPE, + record_name="source file metadata", + ) + source_file_payload = _require_dict_payload( + source_file_record.payload, + record_name=source_file_record.data_id, + ) + text_snapshot_record = _first_source_record( + data_store=data_store, + target_payload=target_payload, + data_type=GRAPH_SOURCE_TEXT_SNAPSHOT_DATA_TYPE, + ) + text_snapshot_payload = ( + _require_dict_payload(text_snapshot_record.payload, record_name=text_snapshot_record.data_id) + if text_snapshot_record is not None + else None + ) + frame_id = night_summarize_source_leaf_frame_id( + raw_source_graph_node_id, + summary_run_id=summary_run_id, + ) + summary_graph_node_id = night_summarize_source_leaf_graph_node_id( + raw_source_graph_node_id, + summary_run_id=summary_run_id, + ) + base_source_data_ids = _unique_strings( + [ + raw_source_graph_node_id, + source_file_record.data_id, + text_snapshot_record.data_id if text_snapshot_record is not None else None, + ] + ) + base_source_trace_ids = _unique_strings( + [ + source_file_record.source_trace_id, + text_snapshot_record.source_trace_id if text_snapshot_record is not None else None, + ] + ) + + if text_snapshot_payload is None: + frame = _skipped_frame( + frame_id=frame_id, + summary_graph_node_id=summary_graph_node_id, + target_payload=target_payload, + source_file_data_id=source_file_record.data_id, + source_file_payload=source_file_payload, + text_snapshot_data_id=None, + summary_status="skipped_no_text_snapshot", + failure_type="no_text_snapshot", + source_trace_ids=base_source_trace_ids, + source_data_ids=base_source_data_ids, + ) + return _record_failed_or_skipped_frame( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + frame=frame, + ) + + source_text = str(text_snapshot_payload.get("text") or "") + if not source_text.strip(): + frame = _skipped_frame( + frame_id=frame_id, + summary_graph_node_id=summary_graph_node_id, + target_payload=target_payload, + source_file_data_id=source_file_record.data_id, + source_file_payload=source_file_payload, + text_snapshot_data_id=text_snapshot_record.data_id, + summary_status="skipped_empty_text", + failure_type="empty_text", + source_trace_ids=base_source_trace_ids, + source_data_ids=base_source_data_ids, + ) + return _record_failed_or_skipped_frame( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + frame=frame, + ) + + input_data_id = f"{frame_id}:input" + input_payload = _input_payload( + input_data_id=input_data_id, + summary_run_id=summary_run_id, + target_payload=target_payload, + source_file_data_id=source_file_record.data_id, + source_file_payload=source_file_payload, + text_snapshot_data_id=text_snapshot_record.data_id, + text_snapshot_payload=text_snapshot_payload, + source_text=source_text, + source_trace_ids=base_source_trace_ids, + source_data_ids=base_source_data_ids, + ) + input_trace_id = _record_input( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + input_data_id=input_data_id, + input_payload=input_payload, + source_trace_ids=base_source_trace_ids, + source_data_ids=base_source_data_ids, + ) + frame_source_trace_ids = _unique_strings([*base_source_trace_ids, input_trace_id]) + frame_base_source_data_ids = _unique_strings([*base_source_data_ids, input_data_id]) + + if adapter is None: + frame = _failed_frame( + frame_id=frame_id, + summary_graph_node_id=summary_graph_node_id, + target_payload=target_payload, + source_file_data_id=source_file_record.data_id, + source_file_payload=source_file_payload, + text_snapshot_data_id=text_snapshot_record.data_id, + model_id="adapter_missing", + failure_type="adapter_missing", + payload_parse_status="not_checked", + source_trace_ids=frame_source_trace_ids, + source_data_ids=frame_base_source_data_ids, + llm_call_data_id=None, + llm_trace_event_id=None, + ) + return _record_failed_or_skipped_frame( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + frame=frame, + ) + + prompt = Path(NIGHT_SUMMARIZE_SOURCE_LEAF_PROMPT_REF).read_text(encoding="utf-8") + llm_result = LLMNodeExecutor(adapter).run( + node_id=NIGHT_SUMMARIZE_SOURCE_LEAF_NODE_ID, + prompt=prompt, + input_payload=input_payload, + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + prompt_ref=NIGHT_SUMMARIZE_SOURCE_LEAF_PROMPT_REF, + input_ref=frame_source_trace_ids, + source_data_ids=frame_base_source_data_ids, + max_retries=max_retries, + payload_validator=_validate_summary_payload, + ) + source_trace_ids_after_llm = _unique_strings( + [*frame_source_trace_ids, llm_result.trace_event_id] + ) + source_data_ids_after_llm = _unique_strings( + [*frame_base_source_data_ids, llm_result.call_data_id] + ) + + if llm_result.failure_type != "none" or llm_result.validation.payload is None: + frame = _failed_frame( + frame_id=frame_id, + summary_graph_node_id=summary_graph_node_id, + target_payload=target_payload, + source_file_data_id=source_file_record.data_id, + source_file_payload=source_file_payload, + text_snapshot_data_id=text_snapshot_record.data_id, + model_id=llm_result.model_id, + failure_type=llm_result.failure_type, + payload_parse_status=_payload_parse_status(llm_result.failure_type), + source_trace_ids=source_trace_ids_after_llm, + source_data_ids=source_data_ids_after_llm, + llm_call_data_id=llm_result.call_data_id, + llm_trace_event_id=llm_result.trace_event_id, + ) + return _record_failed_or_skipped_frame( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + frame=frame, + ) + + frame = _frame_from_payload( + payload=llm_result.validation.payload, + frame_id=frame_id, + summary_graph_node_id=summary_graph_node_id, + target_payload=target_payload, + source_file_data_id=source_file_record.data_id, + source_file_payload=source_file_payload, + text_snapshot_data_id=text_snapshot_record.data_id, + model_id=llm_result.model_id, + source_trace_ids=source_trace_ids_after_llm, + source_data_ids=source_data_ids_after_llm, + llm_call_data_id=llm_result.call_data_id, + llm_trace_event_id=llm_result.trace_event_id, + ) + return _record_success_graph_summary( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + frame=frame, + ) + + +def _frame_from_payload( + *, + payload: dict[str, object], + frame_id: str, + summary_graph_node_id: str, + target_payload: dict[str, object], + source_file_data_id: str, + source_file_payload: dict[str, object], + text_snapshot_data_id: str, + model_id: str, + source_trace_ids: list[str], + source_data_ids: list[str], + llm_call_data_id: str | None, + llm_trace_event_id: str | None, +) -> NightSourceLeafSummaryFrame: + frame = NightSourceLeafSummaryFrame( + frame_id=frame_id, + summary_graph_node_id=summary_graph_node_id, + target_graph_node_id=_text(target_payload.get("node_id")), + target_node_kind=_text(target_payload.get("node_kind")), + source_kind=_text(source_file_payload.get("source_kind")), + source_path=_text(source_file_payload.get("path")), + source_file_data_id=source_file_data_id, + content_sha1=_text(source_file_payload.get("content_sha1")), + text_snapshot_data_id=text_snapshot_data_id, + summary_text=str(payload.get("summary_text") or "").strip(), + summary_status="ran", + failure_type="none", + payload_parse_status="passed", + llm_call_data_id=llm_call_data_id, + llm_trace_event_id=llm_trace_event_id, + prompt_ref=NIGHT_SUMMARIZE_SOURCE_LEAF_PROMPT_REF, + source_graph_node_ids=[_text(target_payload.get("node_id"))], + source_trace_ids=source_trace_ids, + source_data_ids=source_data_ids, + generated_by=f"LLM:{model_id}:night_summarize_source_leaf", + info_class="relative", + semantic_judgement_status="ran", + ) + validate_night_source_leaf_summary_frame(frame) + return frame + + +def _failed_frame( + *, + frame_id: str, + summary_graph_node_id: str, + target_payload: dict[str, object], + source_file_data_id: str, + source_file_payload: dict[str, object], + text_snapshot_data_id: str, + model_id: str, + failure_type: str, + payload_parse_status: str, + source_trace_ids: list[str], + source_data_ids: list[str], + llm_call_data_id: str | None, + llm_trace_event_id: str | None, +) -> NightSourceLeafSummaryFrame: + frame = NightSourceLeafSummaryFrame( + frame_id=frame_id, + summary_graph_node_id=summary_graph_node_id, + target_graph_node_id=_text(target_payload.get("node_id")), + target_node_kind=_text(target_payload.get("node_kind")), + source_kind=_text(source_file_payload.get("source_kind")), + source_path=_text(source_file_payload.get("path")), + source_file_data_id=source_file_data_id, + content_sha1=_text(source_file_payload.get("content_sha1")), + text_snapshot_data_id=text_snapshot_data_id, + summary_text="", + summary_status="failed", + failure_type=failure_type, + payload_parse_status=payload_parse_status, + llm_call_data_id=llm_call_data_id, + llm_trace_event_id=llm_trace_event_id, + prompt_ref=NIGHT_SUMMARIZE_SOURCE_LEAF_PROMPT_REF, + source_graph_node_ids=[_text(target_payload.get("node_id"))], + source_trace_ids=source_trace_ids, + source_data_ids=source_data_ids, + generated_by=f"LLM:{model_id}:night_summarize_source_leaf", + info_class="relative", + semantic_judgement_status="failed", + ) + validate_night_source_leaf_summary_frame(frame) + return frame + + +def _skipped_frame( + *, + frame_id: str, + summary_graph_node_id: str, + target_payload: dict[str, object], + source_file_data_id: str, + source_file_payload: dict[str, object], + text_snapshot_data_id: str | None, + summary_status: str, + failure_type: str, + source_trace_ids: list[str], + source_data_ids: list[str], +) -> NightSourceLeafSummaryFrame: + frame = NightSourceLeafSummaryFrame( + frame_id=frame_id, + summary_graph_node_id=summary_graph_node_id, + target_graph_node_id=_text(target_payload.get("node_id")), + target_node_kind=_text(target_payload.get("node_kind")), + source_kind=_text(source_file_payload.get("source_kind")), + source_path=_text(source_file_payload.get("path")), + source_file_data_id=source_file_data_id, + content_sha1=_text(source_file_payload.get("content_sha1")), + text_snapshot_data_id=text_snapshot_data_id, + summary_text="", + summary_status=summary_status, + failure_type=failure_type, + payload_parse_status="not_checked", + llm_call_data_id=None, + llm_trace_event_id=None, + prompt_ref=NIGHT_SUMMARIZE_SOURCE_LEAF_PROMPT_REF, + source_graph_node_ids=[_text(target_payload.get("node_id"))], + source_trace_ids=source_trace_ids, + source_data_ids=source_data_ids, + generated_by="CODE:NIGHT_SUMMARIZE_SOURCE_LEAF_SKIPPED", + info_class="absolute", + semantic_judgement_status="not_run", + ) + validate_night_source_leaf_summary_frame(frame) + return frame + + +def _record_success_graph_summary( + *, + trace_store: TraceStore, + data_store: DataStore, + turn_id: str, + frame: NightSourceLeafSummaryFrame, +) -> RecordedNightSourceLeafSummaryResult: + validate_night_source_leaf_summary_frame(frame) + edge = _summary_of_edge(frame) + validate_graph_memory_edge_frame(edge) + event = trace_store.create_event( + turn_id=turn_id, + actor=NIGHT_SUMMARIZE_SOURCE_LEAF_NODE_ID, + event_type="node_output", + input_ref=frame.source_trace_ids, + output_ref=[frame.summary_graph_node_id, edge.edge_id], + schema_status="passed", + ) + created_data_ids: list[str] = [] + existing_data_ids: list[str] = [] + _record_payload_if_missing( + data_store=data_store, + data_id=frame.summary_graph_node_id, + data_type="graph_memory:node:summary", + payload=asdict(frame), + created_at=event.timestamp, + source_trace_id=event.event_id, + created_data_ids=created_data_ids, + existing_data_ids=existing_data_ids, + ) + _record_payload_if_missing( + data_store=data_store, + data_id=edge.edge_id, + data_type="graph_memory:edge:SUMMARY_OF", + payload=asdict(edge), + created_at=event.timestamp, + source_trace_id=event.event_id, + created_data_ids=created_data_ids, + existing_data_ids=existing_data_ids, + ) + return RecordedNightSourceLeafSummaryResult( + frame=frame, + trace_event_id=event.event_id, + frame_data_id=frame.summary_graph_node_id, + summary_graph_node_data_id=frame.summary_graph_node_id, + summary_edge_data_id=edge.edge_id, + created_data_ids=created_data_ids, + existing_data_ids=existing_data_ids, + ) + + +def _record_failed_or_skipped_frame( + *, + trace_store: TraceStore, + data_store: DataStore, + turn_id: str, + frame: NightSourceLeafSummaryFrame, +) -> RecordedNightSourceLeafSummaryResult: + validate_night_source_leaf_summary_frame(frame) + event = trace_store.create_event( + turn_id=turn_id, + actor=NIGHT_SUMMARIZE_SOURCE_LEAF_NODE_ID, + event_type="node_output", + input_ref=frame.source_trace_ids, + output_ref=[frame.frame_id], + schema_status="failed" if frame.summary_status == "failed" else "passed", + ) + created_data_ids: list[str] = [] + existing_data_ids: list[str] = [] + _record_payload_if_missing( + data_store=data_store, + data_id=frame.frame_id, + data_type=NIGHT_SUMMARIZE_SOURCE_LEAF_FRAME_DATA_TYPE, + payload=asdict(frame), + created_at=event.timestamp, + source_trace_id=event.event_id, + created_data_ids=created_data_ids, + existing_data_ids=existing_data_ids, + ) + return RecordedNightSourceLeafSummaryResult( + frame=frame, + trace_event_id=event.event_id, + frame_data_id=frame.frame_id, + summary_graph_node_data_id=None, + summary_edge_data_id=None, + created_data_ids=created_data_ids, + existing_data_ids=existing_data_ids, + ) + + +def _summary_of_edge(frame: NightSourceLeafSummaryFrame) -> GraphMemoryEdgeFrame: + return GraphMemoryEdgeFrame( + edge_id=( + "graph:edge:summary_of:" + f"{frame.summary_graph_node_id}:{frame.target_graph_node_id}" + ), + edge_kind="SUMMARY_OF", + from_node_id=frame.summary_graph_node_id, + to_node_id=frame.target_graph_node_id, + source_graph_node_ids=[ + frame.summary_graph_node_id, + frame.target_graph_node_id, + ], + source_trace_ids=list(frame.source_trace_ids), + source_data_ids=[ + frame.summary_graph_node_id, + frame.target_graph_node_id, + ], + ) + + +def _input_payload( + *, + input_data_id: str, + summary_run_id: str, + target_payload: dict[str, object], + source_file_data_id: str, + source_file_payload: dict[str, object], + text_snapshot_data_id: str, + text_snapshot_payload: dict[str, object], + source_text: str, + source_trace_ids: list[str], + source_data_ids: list[str], +) -> dict[str, object]: + return { + "input_data_id": input_data_id, + "summary_run_id": summary_run_id, + "target_raw_source_graph_node_id": _text(target_payload.get("node_id")), + "target_raw_source_payload": dict(target_payload), + "source_file_data_id": source_file_data_id, + "source_file_payload": dict(source_file_payload), + "text_snapshot_data_id": text_snapshot_data_id, + "text_snapshot_metadata": { + key: value + for key, value in text_snapshot_payload.items() + if key != "text" + }, + "source_text": source_text, + "expected_info_class": "relative", + "classification_rule": ( + "a successful summary is relative because it is grounded in exactly " + "one raw_source graph leaf" + ), + "source_trace_ids": source_trace_ids, + "source_data_ids": source_data_ids, + "generated_by": "CODE:NIGHT_SUMMARIZE_SOURCE_LEAF_INPUT_BUILDER", + "semantic_judgement_status": "not_run", + } + + +def _record_input( + *, + trace_store: TraceStore, + data_store: DataStore, + turn_id: str, + input_data_id: str, + input_payload: dict[str, object], + source_trace_ids: list[str], + source_data_ids: list[str], +) -> str: + event = trace_store.create_event( + turn_id=turn_id, + actor=NIGHT_SUMMARIZE_SOURCE_LEAF_NODE_ID, + event_type="node_input", + input_ref=source_trace_ids, + output_ref=[input_data_id], + schema_status="not_checked", + ) + data_store.create_record( + data_id=input_data_id, + data_type=NIGHT_SUMMARIZE_SOURCE_LEAF_INPUT_DATA_TYPE, + exists=True, + created_at=event.timestamp, + source_trace_id=event.event_id, + payload={ + **input_payload, + "source_trace_ids": source_trace_ids, + "source_data_ids": source_data_ids, + }, + ) + return event.event_id + + +def _validate_summary_payload(payload: dict[str, object]) -> None: + summary_text = str(payload.get("summary_text") or "").strip() + if not summary_text: + raise ValueError("summary_text must not be empty") + supplied_info_class = payload.get("info_class") + if supplied_info_class is not None and supplied_info_class != "relative": + raise ValueError("source leaf summary info_class must be relative") + supplied_source_ids = payload.get("source_graph_node_ids") + if supplied_source_ids: + raise ValueError("LLM must not supply source_graph_node_ids") + + +def _require_source_observation_ledger( + *, + data_store: DataStore, + source_observation_ledger_frame_id: str, +) -> SourceObservationLedgerFrame: + record = data_store.require_record(source_observation_ledger_frame_id) + if record.data_type != SOURCE_OBSERVATION_LEDGER_FRAME_DATA_TYPE: + raise ValueError( + "source_observation_ledger_frame_id must point to source observation ledger" + ) + if not isinstance(record.payload, dict): + raise TypeError("source observation ledger payload must be dict") + frame = SourceObservationLedgerFrame(**record.payload) + validate_source_observation_ledger_frame(frame) + return frame + + +def _require_graph_node_payload( + *, + data_store: DataStore, + graph_node_id: str, + expected_node_kind: str, +) -> dict[str, object]: + record = data_store.require_record(graph_node_id) + if not record.data_type.startswith("graph_memory:node:"): + raise ValueError(f"expected graph memory node record: {graph_node_id}") + if not isinstance(record.payload, dict): + raise TypeError(f"graph node payload must be dict: {graph_node_id}") + node_kind = _text(record.payload.get("node_kind")) + if node_kind != expected_node_kind: + raise ValueError( + f"expected {expected_node_kind} graph node, got {node_kind}: {graph_node_id}" + ) + return dict(record.payload) + + +def _require_first_source_record( + *, + data_store: DataStore, + target_payload: dict[str, object], + data_type: str, + record_name: str, +): + record = _first_source_record( + data_store=data_store, + target_payload=target_payload, + data_type=data_type, + ) + if record is None: + raise ValueError(f"raw_source node is missing {record_name}") + return record + + +def _first_source_record( + *, + data_store: DataStore, + target_payload: dict[str, object], + data_type: str, +): + for data_id in _string_list(target_payload.get("source_data_ids")): + record = data_store.get_record(data_id) + if record is not None and record.data_type == data_type: + return record + return None + + +def _require_dict_payload(payload: object, *, record_name: str) -> dict[str, object]: + if not isinstance(payload, dict): + raise TypeError(f"{record_name} payload must be dict") + return dict(payload) + + +def _record_payload_if_missing( + *, + data_store: DataStore, + data_id: str, + data_type: str, + payload: dict[str, object], + created_at: str, + source_trace_id: str, + created_data_ids: list[str], + existing_data_ids: list[str], +) -> None: + existing = data_store.get_record(data_id) + if existing is not None: + if existing.data_type != data_type: + raise ValueError( + f"night source leaf summary data_id collision with different type: {data_id}" + ) + if existing.payload != payload: + raise ValueError( + f"night source leaf summary data_id collision with different payload: {data_id}" + ) + existing_data_ids.append(data_id) + return + data_store.create_record( + data_id=data_id, + data_type=data_type, + exists=True, + created_at=created_at, + source_trace_id=source_trace_id, + payload=payload, + ) + created_data_ids.append(data_id) + + +def _count_result_statuses( + results: list[RecordedNightSourceLeafSummaryResult], +) -> dict[str, int]: + counts: dict[str, int] = {} + for result in results: + counts[result.frame.summary_status] = counts.get(result.frame.summary_status, 0) + 1 + return counts + + +def _payload_parse_status(failure_type: str) -> str: + if failure_type == "parse_failed": + return "failed" + if failure_type in {"adapter_failed", "adapter_missing"}: + return "not_checked" + return "passed" + + +def _require_text(field_name: str, value: str) -> None: + if not value: + raise ValueError(f"{field_name} must not be empty") + + +def _text(value: object) -> str: + return value if isinstance(value, str) else "" + + +def _string_list(value: object) -> list[str]: + if not isinstance(value, list): + return [] + result: list[str] = [] + for item in value: + if isinstance(item, str) and item: + result.append(item) + return result + + +def _unique_strings(values: list[str | None]) -> list[str]: + seen: set[str] = set() + result: list[str] = [] + for value in values: + if not value or value in seen: + continue + seen.add(value) + result.append(value) + return result + + +def _stable_suffix(value: str) -> str: + return hashlib.sha1(value.encode("utf-8")).hexdigest()[:16] + + +def _safe_id_part(value: str) -> str: + cleaned = "".join(char if char.isalnum() else "_" for char in value.strip()) + cleaned = cleaned.strip("_") + if not cleaned: + raise ValueError("safe id part must not be empty") + return cleaned + + +__all__ = [ + "NIGHT_SOURCE_LEAF_SUMMARY_BATCH_DATA_TYPE", + "NIGHT_SUMMARIZE_SOURCE_LEAF_FRAME_DATA_TYPE", + "NIGHT_SUMMARIZE_SOURCE_LEAF_INPUT_DATA_TYPE", + "NIGHT_SUMMARIZE_SOURCE_LEAF_NODE_ID", + "NIGHT_SUMMARIZE_SOURCE_LEAF_PROMPT_REF", + "SUMMARIZABLE_SOURCE_OBSERVATION_STATUSES", + "RecordedNightChangedSourceLeafSummaryBatch", + "RecordedNightSourceLeafSummaryResult", + "NightChangedSourceLeafSelection", + "night_summarize_changed_source_leaves_batch_id", + "night_summarize_source_leaf_frame_id", + "night_summarize_source_leaf_graph_node_id", + "run_night_summarize_changed_source_leaves", + "run_night_summarize_source_leaf", + "select_changed_source_leaf_summary_candidates", +] diff --git a/songryeon_core/nodes/night_summarize_time_bundle.py b/songryeon_core/nodes/night_summarize_time_bundle.py new file mode 100644 index 0000000..5e4879a --- /dev/null +++ b/songryeon_core/nodes/night_summarize_time_bundle.py @@ -0,0 +1,768 @@ +from __future__ import annotations + +"""Night-government worker that summarizes one TimeBundle graph node.""" + +import hashlib +from dataclasses import asdict, dataclass +from pathlib import Path + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.schemas import ( + GraphMemoryEdgeFrame, + NightTimeBundleSummaryFrame, + validate_graph_memory_edge_frame, + validate_night_time_bundle_summary_frame, +) +from songryeon_core.core.trace_store import TraceStore +from songryeon_core.llm.base import LLMAdapter +from songryeon_core.llm.node_executor import LLMNodeExecutor + + +NIGHT_SUMMARIZE_TIME_BUNDLE_NODE_ID = "night_summarize_time_bundle" +NIGHT_SUMMARIZE_TIME_BUNDLE_PROMPT_REF = ( + "songryeon_core/prompts/night_summarize_time_bundle_v0.md" +) +NIGHT_SUMMARIZE_TIME_BUNDLE_FRAME_DATA_TYPE = ( + "node_output:night_summarize_time_bundle_frame" +) +NIGHT_TIME_BUNDLE_SUMMARY_WORKER_NODE_ID = NIGHT_SUMMARIZE_TIME_BUNDLE_NODE_ID +NIGHT_TIME_BUNDLE_SUMMARY_PROMPT_REF = NIGHT_SUMMARIZE_TIME_BUNDLE_PROMPT_REF +NIGHT_TIME_BUNDLE_SUMMARY_FRAME_DATA_TYPE = NIGHT_SUMMARIZE_TIME_BUNDLE_FRAME_DATA_TYPE + + +@dataclass(frozen=True) +class RecordedNightTimeBundleSummaryResult: + frame: NightTimeBundleSummaryFrame + trace_event_id: str + frame_data_id: str + summary_graph_node_data_id: str | None + summary_edge_data_id: str | None + created_data_ids: list[str] + existing_data_ids: list[str] + + +def night_summarize_time_bundle_frame_id( + target_time_bundle_node_id: str, + *, + summary_run_id: str, +) -> str: + _require_text("target_time_bundle_node_id", target_time_bundle_node_id) + _require_text("summary_run_id", summary_run_id) + return ( + "night_summary:time_bundle:" + f"{_stable_suffix(target_time_bundle_node_id)}:{_safe_id_part(summary_run_id)}" + ) + + +def night_summarize_time_bundle_graph_node_id( + target_time_bundle_node_id: str, + *, + summary_run_id: str, +) -> str: + _require_text("target_time_bundle_node_id", target_time_bundle_node_id) + _require_text("summary_run_id", summary_run_id) + return ( + "graph:summary:time_bundle:" + f"{_stable_suffix(target_time_bundle_node_id)}:{_safe_id_part(summary_run_id)}" + ) + + +def night_time_bundle_summary_frame_id( + target_time_bundle_node_id: str, + *, + summary_run_id: str, +) -> str: + """Compatibility wrapper for night_summarize_time_bundle_frame_id.""" + + return night_summarize_time_bundle_frame_id( + target_time_bundle_node_id, + summary_run_id=summary_run_id, + ) + + +def night_time_bundle_summary_graph_node_id( + target_time_bundle_node_id: str, + *, + summary_run_id: str, +) -> str: + """Compatibility wrapper for night_summarize_time_bundle_graph_node_id.""" + + return night_summarize_time_bundle_graph_node_id( + target_time_bundle_node_id, + summary_run_id=summary_run_id, + ) + + +def run_night_summarize_time_bundle( + *, + trace_store: TraceStore, + data_store: DataStore, + turn_id: str, + target_time_bundle_node_id: str, + summary_run_id: str, + adapter: LLMAdapter | None, + max_retries: int = 0, +) -> RecordedNightTimeBundleSummaryResult: + """Create an LLM summary graph node for one code-generated TimeBundle.""" + + target_payload = _require_graph_node_payload( + data_store=data_store, + graph_node_id=target_time_bundle_node_id, + expected_node_kind="time_bundle", + ) + frame_id = night_summarize_time_bundle_frame_id( + target_time_bundle_node_id, + summary_run_id=summary_run_id, + ) + summary_graph_node_id = night_summarize_time_bundle_graph_node_id( + target_time_bundle_node_id, + summary_run_id=summary_run_id, + ) + source_graph_node_ids = _string_list(target_payload.get("source_graph_node_ids")) + source_node_payloads = _source_node_payloads( + data_store=data_store, + source_graph_node_ids=source_graph_node_ids, + ) + source_trace_ids = _unique_strings( + [ + *_string_list(target_payload.get("source_trace_ids")), + *[ + trace_id + for payload in source_node_payloads + for trace_id in _string_list(payload.get("source_trace_ids")) + ], + ] + ) + base_source_data_ids = _unique_strings( + [target_time_bundle_node_id, *source_graph_node_ids] + ) + input_data_id = f"{frame_id}:input" + input_payload = _input_payload( + input_data_id=input_data_id, + target_time_bundle_node_id=target_time_bundle_node_id, + target_payload=target_payload, + source_node_payloads=source_node_payloads, + summary_run_id=summary_run_id, + source_data_ids=base_source_data_ids, + source_trace_ids=source_trace_ids, + ) + input_trace_id = _record_input( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + input_data_id=input_data_id, + input_payload=input_payload, + source_trace_ids=source_trace_ids, + source_data_ids=base_source_data_ids, + ) + frame_source_trace_ids = _unique_strings([*source_trace_ids, input_trace_id]) + frame_base_source_data_ids = _unique_strings([*base_source_data_ids, input_data_id]) + + if adapter is None: + frame = _failed_frame( + frame_id=frame_id, + summary_graph_node_id=summary_graph_node_id, + target_payload=target_payload, + source_graph_node_ids=source_graph_node_ids, + source_node_payloads=source_node_payloads, + model_id="adapter_missing", + failure_type="adapter_missing", + payload_parse_status="not_checked", + source_trace_ids=frame_source_trace_ids, + source_data_ids=frame_base_source_data_ids, + llm_call_data_id=None, + llm_trace_event_id=None, + ) + return _record_failed_frame( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + frame=frame, + ) + + expected_info_class = _expected_info_class( + source_graph_node_ids=source_graph_node_ids, + source_leaf_count=_int(target_payload.get("source_leaf_count")), + source_summary_count=_int(target_payload.get("source_summary_count")), + ) + prompt = Path(NIGHT_SUMMARIZE_TIME_BUNDLE_PROMPT_REF).read_text(encoding="utf-8") + llm_result = LLMNodeExecutor(adapter).run( + node_id=NIGHT_SUMMARIZE_TIME_BUNDLE_NODE_ID, + prompt=prompt, + input_payload=input_payload, + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + prompt_ref=NIGHT_SUMMARIZE_TIME_BUNDLE_PROMPT_REF, + input_ref=frame_source_trace_ids, + source_data_ids=frame_base_source_data_ids, + max_retries=max_retries, + payload_validator=lambda payload: _validate_summary_payload( + payload=payload, + expected_info_class=expected_info_class, + allowed_source_graph_node_ids=source_graph_node_ids, + ), + ) + source_trace_ids_after_llm = _unique_strings( + [*frame_source_trace_ids, llm_result.trace_event_id] + ) + source_data_ids_after_llm = _unique_strings( + [*frame_base_source_data_ids, llm_result.call_data_id] + ) + + if llm_result.failure_type != "none" or llm_result.validation.payload is None: + frame = _failed_frame( + frame_id=frame_id, + summary_graph_node_id=summary_graph_node_id, + target_payload=target_payload, + source_graph_node_ids=source_graph_node_ids, + source_node_payloads=source_node_payloads, + model_id=llm_result.model_id, + failure_type=llm_result.failure_type, + payload_parse_status=_payload_parse_status(llm_result.failure_type), + source_trace_ids=source_trace_ids_after_llm, + source_data_ids=source_data_ids_after_llm, + llm_call_data_id=llm_result.call_data_id, + llm_trace_event_id=llm_result.trace_event_id, + ) + return _record_failed_frame( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + frame=frame, + ) + + frame = _frame_from_payload( + payload=llm_result.validation.payload, + frame_id=frame_id, + summary_graph_node_id=summary_graph_node_id, + target_payload=target_payload, + source_graph_node_ids=source_graph_node_ids, + source_node_payloads=source_node_payloads, + model_id=llm_result.model_id, + source_trace_ids=source_trace_ids_after_llm, + source_data_ids=source_data_ids_after_llm, + llm_call_data_id=llm_result.call_data_id, + llm_trace_event_id=llm_result.trace_event_id, + ) + return _record_success_graph_summary( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + frame=frame, + ) + + +def run_night_time_bundle_summary_worker( + *, + trace_store: TraceStore, + data_store: DataStore, + turn_id: str, + target_time_bundle_node_id: str, + summary_run_id: str, + adapter: LLMAdapter | None, + max_retries: int = 0, +) -> RecordedNightTimeBundleSummaryResult: + """Compatibility wrapper for the clearer run_night_summarize_time_bundle name.""" + + return run_night_summarize_time_bundle( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + target_time_bundle_node_id=target_time_bundle_node_id, + summary_run_id=summary_run_id, + adapter=adapter, + max_retries=max_retries, + ) + + +def _frame_from_payload( + *, + payload: dict[str, object], + frame_id: str, + summary_graph_node_id: str, + target_payload: dict[str, object], + source_graph_node_ids: list[str], + source_node_payloads: list[dict[str, object]], + model_id: str, + source_trace_ids: list[str], + source_data_ids: list[str], + llm_call_data_id: str | None, + llm_trace_event_id: str | None, +) -> NightTimeBundleSummaryFrame: + source_leaf_count = _int(target_payload.get("source_leaf_count")) + source_summary_count = _int(target_payload.get("source_summary_count")) + info_class = _expected_info_class( + source_graph_node_ids=source_graph_node_ids, + source_leaf_count=source_leaf_count, + source_summary_count=source_summary_count, + ) + frame = NightTimeBundleSummaryFrame( + frame_id=frame_id, + summary_graph_node_id=summary_graph_node_id, + target_graph_node_id=_text(target_payload.get("node_id")), + target_node_kind=_text(target_payload.get("node_kind")), + summary_text=str(payload.get("summary_text") or "").strip(), + summary_status="ran", + failure_type="none", + payload_parse_status="passed", + summary_depth=_summary_depth(source_node_payloads, target_payload), + source_depth_min=_source_depth_min(source_node_payloads, target_payload), + source_depth_max=_source_depth_max(source_node_payloads, target_payload), + source_leaf_count=source_leaf_count, + source_summary_count=source_summary_count, + source_bundle_kind=_text(target_payload.get("source_bundle_kind")) or "time_bundle", + validity_status="active", + review_status="not_reviewed", + llm_call_data_id=llm_call_data_id, + llm_trace_event_id=llm_trace_event_id, + prompt_ref=NIGHT_SUMMARIZE_TIME_BUNDLE_PROMPT_REF, + source_mode=_source_mode(info_class), + claim_alignment=_claim_alignment(info_class), + source_graph_node_ids=source_graph_node_ids, + source_trace_ids=source_trace_ids, + source_data_ids=source_data_ids, + generated_by=f"LLM:{model_id}:night_summarize_time_bundle", + info_class=info_class, + semantic_judgement_status="ran", + ) + validate_night_time_bundle_summary_frame(frame) + return frame + + +def _failed_frame( + *, + frame_id: str, + summary_graph_node_id: str, + target_payload: dict[str, object], + source_graph_node_ids: list[str], + source_node_payloads: list[dict[str, object]], + model_id: str, + failure_type: str, + payload_parse_status: str, + source_trace_ids: list[str], + source_data_ids: list[str], + llm_call_data_id: str | None, + llm_trace_event_id: str | None, +) -> NightTimeBundleSummaryFrame: + source_leaf_count = _int(target_payload.get("source_leaf_count")) + source_summary_count = _int(target_payload.get("source_summary_count")) + info_class = _expected_info_class( + source_graph_node_ids=source_graph_node_ids, + source_leaf_count=source_leaf_count, + source_summary_count=source_summary_count, + ) + frame = NightTimeBundleSummaryFrame( + frame_id=frame_id, + summary_graph_node_id=summary_graph_node_id, + target_graph_node_id=_text(target_payload.get("node_id")), + target_node_kind=_text(target_payload.get("node_kind")), + summary_text="", + summary_status="failed", + failure_type=failure_type, + payload_parse_status=payload_parse_status, + summary_depth=_summary_depth(source_node_payloads, target_payload), + source_depth_min=_source_depth_min(source_node_payloads, target_payload), + source_depth_max=_source_depth_max(source_node_payloads, target_payload), + source_leaf_count=source_leaf_count, + source_summary_count=source_summary_count, + source_bundle_kind=_text(target_payload.get("source_bundle_kind")) or "time_bundle", + validity_status="active", + review_status="not_reviewed", + llm_call_data_id=llm_call_data_id, + llm_trace_event_id=llm_trace_event_id, + prompt_ref=NIGHT_SUMMARIZE_TIME_BUNDLE_PROMPT_REF, + source_mode=_source_mode(info_class), + claim_alignment=_claim_alignment(info_class), + source_graph_node_ids=source_graph_node_ids, + source_trace_ids=source_trace_ids, + source_data_ids=source_data_ids, + generated_by=f"LLM:{model_id}:night_summarize_time_bundle", + info_class=info_class, + semantic_judgement_status="failed", + ) + validate_night_time_bundle_summary_frame(frame) + return frame + + +def _record_success_graph_summary( + *, + trace_store: TraceStore, + data_store: DataStore, + turn_id: str, + frame: NightTimeBundleSummaryFrame, +) -> RecordedNightTimeBundleSummaryResult: + validate_night_time_bundle_summary_frame(frame) + edge = _summary_of_edge(frame) + validate_graph_memory_edge_frame(edge) + event = trace_store.create_event( + turn_id=turn_id, + actor=NIGHT_SUMMARIZE_TIME_BUNDLE_NODE_ID, + event_type="node_output", + input_ref=frame.source_trace_ids, + output_ref=[frame.summary_graph_node_id, edge.edge_id], + schema_status="passed", + ) + created_data_ids: list[str] = [] + existing_data_ids: list[str] = [] + _record_payload_if_missing( + data_store=data_store, + data_id=frame.summary_graph_node_id, + data_type="graph_memory:node:summary", + payload=asdict(frame), + created_at=event.timestamp, + source_trace_id=event.event_id, + created_data_ids=created_data_ids, + existing_data_ids=existing_data_ids, + ) + _record_payload_if_missing( + data_store=data_store, + data_id=edge.edge_id, + data_type="graph_memory:edge:SUMMARY_OF", + payload=asdict(edge), + created_at=event.timestamp, + source_trace_id=event.event_id, + created_data_ids=created_data_ids, + existing_data_ids=existing_data_ids, + ) + return RecordedNightTimeBundleSummaryResult( + frame=frame, + trace_event_id=event.event_id, + frame_data_id=frame.summary_graph_node_id, + summary_graph_node_data_id=frame.summary_graph_node_id, + summary_edge_data_id=edge.edge_id, + created_data_ids=created_data_ids, + existing_data_ids=existing_data_ids, + ) + + +def _record_failed_frame( + *, + trace_store: TraceStore, + data_store: DataStore, + turn_id: str, + frame: NightTimeBundleSummaryFrame, +) -> RecordedNightTimeBundleSummaryResult: + validate_night_time_bundle_summary_frame(frame) + event = trace_store.create_event( + turn_id=turn_id, + actor=NIGHT_SUMMARIZE_TIME_BUNDLE_NODE_ID, + event_type="node_output", + input_ref=frame.source_trace_ids, + output_ref=[frame.frame_id], + schema_status="failed", + ) + created_data_ids: list[str] = [] + existing_data_ids: list[str] = [] + _record_payload_if_missing( + data_store=data_store, + data_id=frame.frame_id, + data_type=NIGHT_SUMMARIZE_TIME_BUNDLE_FRAME_DATA_TYPE, + payload=asdict(frame), + created_at=event.timestamp, + source_trace_id=event.event_id, + created_data_ids=created_data_ids, + existing_data_ids=existing_data_ids, + ) + return RecordedNightTimeBundleSummaryResult( + frame=frame, + trace_event_id=event.event_id, + frame_data_id=frame.frame_id, + summary_graph_node_data_id=None, + summary_edge_data_id=None, + created_data_ids=created_data_ids, + existing_data_ids=existing_data_ids, + ) + + +def _summary_of_edge(frame: NightTimeBundleSummaryFrame) -> GraphMemoryEdgeFrame: + return GraphMemoryEdgeFrame( + edge_id=( + "graph:edge:summary_of:" + f"{frame.summary_graph_node_id}:{frame.target_graph_node_id}" + ), + edge_kind="SUMMARY_OF", + from_node_id=frame.summary_graph_node_id, + to_node_id=frame.target_graph_node_id, + source_graph_node_ids=[ + frame.summary_graph_node_id, + frame.target_graph_node_id, + *frame.source_graph_node_ids, + ], + source_trace_ids=list(frame.source_trace_ids), + source_data_ids=[ + frame.summary_graph_node_id, + frame.target_graph_node_id, + *frame.source_graph_node_ids, + ], + ) + + +def _input_payload( + *, + input_data_id: str, + target_time_bundle_node_id: str, + target_payload: dict[str, object], + source_node_payloads: list[dict[str, object]], + summary_run_id: str, + source_data_ids: list[str], + source_trace_ids: list[str], +) -> dict[str, object]: + source_leaf_count = _int(target_payload.get("source_leaf_count")) + source_summary_count = _int(target_payload.get("source_summary_count")) + source_graph_node_ids = _string_list(target_payload.get("source_graph_node_ids")) + return { + "input_data_id": input_data_id, + "summary_run_id": summary_run_id, + "target_time_bundle_node_id": target_time_bundle_node_id, + "target_time_bundle_payload": dict(target_payload), + "source_graph_node_payloads": [dict(payload) for payload in source_node_payloads], + "expected_info_class": _expected_info_class( + source_graph_node_ids=source_graph_node_ids, + source_leaf_count=source_leaf_count, + source_summary_count=source_summary_count, + ), + "classification_rule": ( + "relative only when exactly one raw leaf source is summarized; " + "otherwise mixed" + ), + "source_trace_ids": source_trace_ids, + "source_data_ids": source_data_ids, + "generated_by": "CODE:NIGHT_SUMMARIZE_TIME_BUNDLE_INPUT_BUILDER", + "semantic_judgement_status": "not_run", + } + + +def _record_input( + *, + trace_store: TraceStore, + data_store: DataStore, + turn_id: str, + input_data_id: str, + input_payload: dict[str, object], + source_trace_ids: list[str], + source_data_ids: list[str], +) -> str: + event = trace_store.create_event( + turn_id=turn_id, + actor=NIGHT_SUMMARIZE_TIME_BUNDLE_NODE_ID, + event_type="node_input", + input_ref=source_trace_ids, + output_ref=[input_data_id], + schema_status="not_checked", + ) + data_store.create_record( + data_id=input_data_id, + data_type="node_input:night_summarize_time_bundle_input", + exists=True, + created_at=event.timestamp, + source_trace_id=event.event_id, + payload={ + **input_payload, + "source_trace_ids": source_trace_ids, + "source_data_ids": source_data_ids, + }, + ) + return event.event_id + + +def _validate_summary_payload( + *, + payload: dict[str, object], + expected_info_class: str, + allowed_source_graph_node_ids: list[str], +) -> None: + summary_text = str(payload.get("summary_text") or "").strip() + if not summary_text: + raise ValueError("summary_text must not be empty") + supplied_info_class = payload.get("info_class") + if supplied_info_class is not None and supplied_info_class != expected_info_class: + raise ValueError("summary info_class does not match source cardinality") + supplied_source_ids = _string_list(payload.get("source_graph_node_ids")) + if supplied_source_ids and set(supplied_source_ids) != set(allowed_source_graph_node_ids): + raise ValueError("summary source_graph_node_ids must match the target bundle") + + +def _require_graph_node_payload( + *, + data_store: DataStore, + graph_node_id: str, + expected_node_kind: str, +) -> dict[str, object]: + record = data_store.require_record(graph_node_id) + if not record.data_type.startswith("graph_memory:node:"): + raise ValueError(f"expected graph memory node record: {graph_node_id}") + if not isinstance(record.payload, dict): + raise TypeError(f"graph node payload must be dict: {graph_node_id}") + node_kind = _text(record.payload.get("node_kind")) + if node_kind != expected_node_kind: + raise ValueError( + f"expected {expected_node_kind} graph node, got {node_kind}: {graph_node_id}" + ) + return dict(record.payload) + + +def _source_node_payloads( + *, + data_store: DataStore, + source_graph_node_ids: list[str], +) -> list[dict[str, object]]: + payloads: list[dict[str, object]] = [] + for graph_node_id in source_graph_node_ids: + record = data_store.require_record(graph_node_id) + if not record.data_type.startswith("graph_memory:node:"): + raise ValueError(f"source graph id is not a graph node: {graph_node_id}") + if not isinstance(record.payload, dict): + raise TypeError(f"source graph node payload must be dict: {graph_node_id}") + payloads.append(dict(record.payload)) + return payloads + + +def _record_payload_if_missing( + *, + data_store: DataStore, + data_id: str, + data_type: str, + payload: dict[str, object], + created_at: str, + source_trace_id: str, + created_data_ids: list[str], + existing_data_ids: list[str], +) -> None: + existing = data_store.get_record(data_id) + if existing is not None: + if existing.data_type != data_type: + raise ValueError(f"night summary data_id collision with different type: {data_id}") + if existing.payload != payload: + raise ValueError(f"night summary data_id collision with different payload: {data_id}") + existing_data_ids.append(data_id) + return + data_store.create_record( + data_id=data_id, + data_type=data_type, + exists=True, + created_at=created_at, + source_trace_id=source_trace_id, + payload=payload, + ) + created_data_ids.append(data_id) + + +def _summary_depth( + source_node_payloads: list[dict[str, object]], + target_payload: dict[str, object], +) -> int: + source_depths = [_int(payload.get("summary_depth")) for payload in source_node_payloads] + if source_depths: + return max(source_depths) + 1 + return _int(target_payload.get("summary_depth")) + 1 + + +def _source_depth_min( + source_node_payloads: list[dict[str, object]], + target_payload: dict[str, object], +) -> int: + source_depths = [_int(payload.get("summary_depth")) for payload in source_node_payloads] + if source_depths: + return min(source_depths) + return _int(target_payload.get("summary_depth")) + + +def _source_depth_max( + source_node_payloads: list[dict[str, object]], + target_payload: dict[str, object], +) -> int: + source_depths = [_int(payload.get("summary_depth")) for payload in source_node_payloads] + if source_depths: + return max(source_depths) + return _int(target_payload.get("summary_depth")) + + +def _expected_info_class( + *, + source_graph_node_ids: list[str], + source_leaf_count: int, + source_summary_count: int, +) -> str: + if source_leaf_count == 1 and source_summary_count == 0 and len(source_graph_node_ids) == 1: + return "relative" + return "mixed" + + +def _source_mode(info_class: str) -> str: + return "single_source" if info_class == "relative" else "source_bundle" + + +def _claim_alignment(info_class: str) -> str: + return "single_absolute_record" if info_class == "relative" else "multi_source_bundle" + + +def _payload_parse_status(failure_type: str) -> str: + if failure_type == "parse_failed": + return "failed" + if failure_type in {"adapter_failed", "adapter_missing"}: + return "not_checked" + return "passed" + + +def _require_text(field_name: str, value: str) -> None: + if not value: + raise ValueError(f"{field_name} must not be empty") + + +def _text(value: object) -> str: + return value if isinstance(value, str) else "" + + +def _int(value: object) -> int: + return value if isinstance(value, int) and value >= 0 else 0 + + +def _string_list(value: object) -> list[str]: + if not isinstance(value, list): + return [] + result: list[str] = [] + for item in value: + if isinstance(item, str) and item: + result.append(item) + return result + + +def _unique_strings(values: list[str | None]) -> list[str]: + seen: set[str] = set() + result: list[str] = [] + for value in values: + if not value or value in seen: + continue + seen.add(value) + result.append(value) + return result + + +def _stable_suffix(value: str) -> str: + return hashlib.sha1(value.encode("utf-8")).hexdigest()[:16] + + +def _safe_id_part(value: str) -> str: + cleaned = "".join(char if char.isalnum() else "_" for char in value.strip()) + cleaned = cleaned.strip("_") + if not cleaned: + raise ValueError("safe id part must not be empty") + return cleaned + + +__all__ = [ + "NIGHT_SUMMARIZE_TIME_BUNDLE_FRAME_DATA_TYPE", + "NIGHT_SUMMARIZE_TIME_BUNDLE_NODE_ID", + "NIGHT_SUMMARIZE_TIME_BUNDLE_PROMPT_REF", + "NIGHT_TIME_BUNDLE_SUMMARY_FRAME_DATA_TYPE", + "NIGHT_TIME_BUNDLE_SUMMARY_PROMPT_REF", + "NIGHT_TIME_BUNDLE_SUMMARY_WORKER_NODE_ID", + "RecordedNightTimeBundleSummaryResult", + "night_summarize_time_bundle_frame_id", + "night_summarize_time_bundle_graph_node_id", + "night_time_bundle_summary_frame_id", + "night_time_bundle_summary_graph_node_id", + "run_night_summarize_time_bundle", + "run_night_time_bundle_summary_worker", +] diff --git a/songryeon_core/nodes/night_summarize_token_budget_bundle.py b/songryeon_core/nodes/night_summarize_token_budget_bundle.py new file mode 100644 index 0000000..24d1207 --- /dev/null +++ b/songryeon_core/nodes/night_summarize_token_budget_bundle.py @@ -0,0 +1,940 @@ +from __future__ import annotations + +"""Night-government worker for one token-budget summary layer bundle.""" + +import hashlib +from dataclasses import asdict, dataclass +from pathlib import Path + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.schemas import ( + GRAPH_MEMORY_CODE_GENERATOR, + GraphMemoryEdgeFrame, + GraphMemoryNodeFrame, + validate_graph_memory_edge_frame, + validate_graph_memory_node_frame, +) +from songryeon_core.core.trace_store import TraceStore +from songryeon_core.llm.base import LLMAdapter +from songryeon_core.llm.node_executor import LLMNodeExecutor + + +NIGHT_SUMMARIZE_TOKEN_BUDGET_BUNDLE_NODE_ID = "night_summarize_token_budget_bundle" +NIGHT_SUMMARIZE_TOKEN_BUDGET_BUNDLE_PROMPT_REF = ( + "songryeon_core/prompts/night_summarize_token_budget_bundle_v0.md" +) +NIGHT_SUMMARIZE_TOKEN_BUDGET_BUNDLE_FRAME_DATA_TYPE = ( + "node_output:night_summarize_token_budget_bundle_frame" +) +TOKEN_BUDGET_SUMMARY_BUNDLE_NODE_DATA_TYPE = ( + "graph_memory:node:token_budget_summary_bundle" +) +TOKEN_BUDGET_SUMMARY_BUNDLE_POLICY_ID = "TOKEN_BUDGET_SUMMARY_LAYER_V0" +TOKEN_BUDGET_SUMMARY_BUNDLE_NODE_KIND = "token_budget_summary_bundle" +TOKEN_BUDGET_SUMMARY_NODE_DATA_KIND = "token_budget_bundle_summary" +TOKEN_BUDGET_SUMMARY_BUNDLE_DATA_KIND = "token_budget_summary_bundle" + + +@dataclass(frozen=True) +class TokenBudgetSummaryBundleSpec: + bundle_index: int + bundle_graph_node_id: str + source_summary_graph_node_ids: list[str] + source_summary_char_count: int + char_budget: int + char_budget_status: str + + +@dataclass(frozen=True) +class NightTokenBudgetBundleSummaryFrame: + frame_id: str + summary_graph_node_id: str + target_graph_node_id: str + target_node_kind: str + summary_text: str = "" + summary_status: str = "ran" + failure_type: str = "none" + payload_parse_status: str = "passed" + node_kind: str = "summary" + data_kind: str = TOKEN_BUDGET_SUMMARY_NODE_DATA_KIND + summary_depth: int = 2 + source_depth_min: int = 1 + source_depth_max: int = 1 + source_leaf_count: int = 0 + source_summary_count: int = 0 + source_bundle_kind: str = TOKEN_BUDGET_SUMMARY_BUNDLE_NODE_KIND + char_budget: int = 0 + input_summary_char_count: int = 0 + validity_status: str = "active" + review_status: str = "not_reviewed" + llm_call_data_id: str | None = None + llm_trace_event_id: str | None = None + prompt_ref: str = NIGHT_SUMMARIZE_TOKEN_BUDGET_BUNDLE_PROMPT_REF + source_mode: str = "source_bundle" + claim_alignment: str = "multi_source_bundle" + source_graph_node_ids: list[str] | None = None + source_trace_ids: list[str] | None = None + source_data_ids: list[str] | None = None + generated_by: str = "LLM:unknown:night_summarize_token_budget_bundle" + info_class: str = "mixed" + semantic_judgement_status: str = "ran" + schema_name: str = "NightTokenBudgetBundleSummaryFrame" + schema_version: str = "0.1" + + def __post_init__(self) -> None: + object.__setattr__(self, "source_graph_node_ids", self.source_graph_node_ids or []) + object.__setattr__(self, "source_trace_ids", self.source_trace_ids or []) + object.__setattr__(self, "source_data_ids", self.source_data_ids or []) + + +@dataclass(frozen=True) +class RecordedNightTokenBudgetBundleSummaryResult: + frame: NightTokenBudgetBundleSummaryFrame + trace_event_id: str + frame_data_id: str + target_bundle_graph_node_id: str + summary_graph_node_data_id: str | None + summary_edge_data_id: str | None + bundle_edge_data_ids: list[str] + created_data_ids: list[str] + existing_data_ids: list[str] + + +def collect_active_source_leaf_summary_graph_node_ids( + data_store: DataStore, +) -> list[str]: + """Return active source leaf summary graph nodes in deterministic order.""" + + items: list[tuple[str, str, str]] = [] + for record in data_store.list_records(): + if record.data_type != "graph_memory:node:summary": + continue + if not isinstance(record.payload, dict): + continue + payload = record.payload + if payload.get("data_kind") != "source_leaf_summary": + continue + if payload.get("summary_status") != "ran": + continue + if payload.get("validity_status") != "active": + continue + path = _text(payload.get("source_path")) + source_kind = _text(payload.get("source_kind")) + items.append((source_kind, path, record.data_id)) + return [data_id for _, _, data_id in sorted(items)] + + +def build_token_budget_summary_bundle_specs( + *, + data_store: DataStore, + summary_run_id: str, + source_summary_graph_node_ids: list[str], + max_bundle_chars: int, +) -> list[TokenBudgetSummaryBundleSpec]: + """Group summary nodes without splitting any individual summary text.""" + + if max_bundle_chars <= 0: + raise ValueError("max_bundle_chars must be positive") + specs: list[TokenBudgetSummaryBundleSpec] = [] + current_ids: list[str] = [] + current_chars = 0 + + def flush(*, forced_status: str | None = None) -> None: + nonlocal current_ids, current_chars + if not current_ids: + return + bundle_index = len(specs) + 1 + status = forced_status or ( + "within_budget" + if current_chars <= max_bundle_chars + else "single_item_exceeds_budget" + ) + specs.append( + TokenBudgetSummaryBundleSpec( + bundle_index=bundle_index, + bundle_graph_node_id=token_budget_summary_bundle_graph_node_id( + summary_run_id, + bundle_index=bundle_index, + ), + source_summary_graph_node_ids=list(current_ids), + source_summary_char_count=current_chars, + char_budget=max_bundle_chars, + char_budget_status=status, + ) + ) + current_ids = [] + current_chars = 0 + + for source_summary_graph_node_id in source_summary_graph_node_ids: + payload = _require_summary_payload( + data_store=data_store, + summary_graph_node_id=source_summary_graph_node_id, + ) + summary_chars = len(_text(payload.get("summary_text"))) + if summary_chars > max_bundle_chars: + flush() + current_ids = [source_summary_graph_node_id] + current_chars = summary_chars + flush(forced_status="single_item_exceeds_budget") + continue + if current_ids and current_chars + summary_chars > max_bundle_chars: + flush() + current_ids.append(source_summary_graph_node_id) + current_chars += summary_chars + flush() + return specs + + +def token_budget_summary_bundle_graph_node_id( + summary_run_id: str, + *, + bundle_index: int, +) -> str: + _require_text("summary_run_id", summary_run_id) + if bundle_index <= 0: + raise ValueError("bundle_index must be positive") + return ( + "graph:token_budget_summary_bundle:" + f"{_safe_id_part(summary_run_id)}:{bundle_index:04d}" + ) + + +def night_summarize_token_budget_bundle_frame_id( + target_bundle_graph_node_id: str, + *, + summary_run_id: str, +) -> str: + _require_text("target_bundle_graph_node_id", target_bundle_graph_node_id) + _require_text("summary_run_id", summary_run_id) + return ( + "night_summary:token_budget_bundle:" + f"{_stable_suffix(target_bundle_graph_node_id)}:{_safe_id_part(summary_run_id)}" + ) + + +def night_summarize_token_budget_bundle_graph_node_id( + target_bundle_graph_node_id: str, + *, + summary_run_id: str, +) -> str: + _require_text("target_bundle_graph_node_id", target_bundle_graph_node_id) + _require_text("summary_run_id", summary_run_id) + return ( + "graph:summary:token_budget_bundle:" + f"{_stable_suffix(target_bundle_graph_node_id)}:{_safe_id_part(summary_run_id)}" + ) + + +def record_token_budget_summary_bundle_node( + *, + trace_store: TraceStore, + data_store: DataStore, + turn_id: str, + spec: TokenBudgetSummaryBundleSpec, +) -> tuple[GraphMemoryNodeFrame, str, list[str], list[str], list[str]]: + """Record one code-generated bundle node and its CONTAINS edges.""" + + source_payloads = [ + _require_summary_payload(data_store=data_store, summary_graph_node_id=data_id) + for data_id in spec.source_summary_graph_node_ids + ] + source_trace_ids = _unique_strings( + [ + trace_id + for payload in source_payloads + for trace_id in _string_list(payload.get("source_trace_ids")) + ] + ) + source_data_ids = _unique_strings( + [ + *spec.source_summary_graph_node_ids, + *[ + data_id + for payload in source_payloads + for data_id in _string_list(payload.get("source_data_ids")) + ], + ] + ) + node = GraphMemoryNodeFrame( + node_id=spec.bundle_graph_node_id, + node_kind=TOKEN_BUDGET_SUMMARY_BUNDLE_NODE_KIND, + data_kind=TOKEN_BUDGET_SUMMARY_BUNDLE_DATA_KIND, + summary_depth=max([_int(payload.get("summary_depth")) for payload in source_payloads] or [1]), + source_depth_min=min( + [_int(payload.get("source_depth_min")) for payload in source_payloads] or [0] + ), + source_depth_max=max( + [_int(payload.get("summary_depth")) for payload in source_payloads] or [1] + ), + source_leaf_count=sum(_int(payload.get("source_leaf_count")) for payload in source_payloads), + source_summary_count=sum( + _int(payload.get("source_summary_count")) + 1 for payload in source_payloads + ), + source_bundle_kind=TOKEN_BUDGET_SUMMARY_BUNDLE_NODE_KIND, + bundle_policy_id=TOKEN_BUDGET_SUMMARY_BUNDLE_POLICY_ID, + char_budget=spec.char_budget, + source_char_count=spec.source_summary_char_count, + char_budget_status=spec.char_budget_status, + source_graph_node_ids=list(spec.source_summary_graph_node_ids), + source_trace_ids=source_trace_ids, + source_data_ids=source_data_ids, + generated_by=GRAPH_MEMORY_CODE_GENERATOR, + info_class="absolute", + semantic_judgement_status="not_run", + ) + validate_graph_memory_node_frame(node) + edges = [_contains_edge(node, child_id) for child_id in spec.source_summary_graph_node_ids] + for edge in edges: + validate_graph_memory_edge_frame(edge) + + event = trace_store.create_event( + turn_id=turn_id, + actor="night_token_budget_summary_bundle_builder", + event_type="node_output", + input_ref=source_trace_ids, + output_ref=[node.node_id, *[edge.edge_id for edge in edges]], + schema_status="passed", + ) + created_data_ids: list[str] = [] + existing_data_ids: list[str] = [] + _record_payload_if_missing( + data_store=data_store, + data_id=node.node_id, + data_type=TOKEN_BUDGET_SUMMARY_BUNDLE_NODE_DATA_TYPE, + payload=asdict(node), + created_at=event.timestamp, + source_trace_id=event.event_id, + created_data_ids=created_data_ids, + existing_data_ids=existing_data_ids, + ) + for edge in edges: + _record_payload_if_missing( + data_store=data_store, + data_id=edge.edge_id, + data_type=f"graph_memory:edge:{edge.edge_kind}", + payload=asdict(edge), + created_at=event.timestamp, + source_trace_id=event.event_id, + created_data_ids=created_data_ids, + existing_data_ids=existing_data_ids, + ) + return ( + node, + event.event_id, + [edge.edge_id for edge in edges], + created_data_ids, + existing_data_ids, + ) + + +def run_night_summarize_token_budget_bundle( + *, + trace_store: TraceStore, + data_store: DataStore, + turn_id: str, + spec: TokenBudgetSummaryBundleSpec, + summary_run_id: str, + adapter: LLMAdapter | None, + max_retries: int = 0, +) -> RecordedNightTokenBudgetBundleSummaryResult: + bundle_node, bundle_trace_id, bundle_edge_ids, created_ids, existing_ids = ( + record_token_budget_summary_bundle_node( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + spec=spec, + ) + ) + frame_id = night_summarize_token_budget_bundle_frame_id( + bundle_node.node_id, + summary_run_id=summary_run_id, + ) + summary_graph_node_id = night_summarize_token_budget_bundle_graph_node_id( + bundle_node.node_id, + summary_run_id=summary_run_id, + ) + source_payloads = [ + _require_summary_payload(data_store=data_store, summary_graph_node_id=data_id) + for data_id in spec.source_summary_graph_node_ids + ] + input_data_id = f"{frame_id}:input" + input_payload = _input_payload( + input_data_id=input_data_id, + summary_run_id=summary_run_id, + bundle_node_payload=asdict(bundle_node), + source_summary_payloads=source_payloads, + ) + input_trace_id = _record_input( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + input_data_id=input_data_id, + input_payload=input_payload, + source_trace_ids=_unique_strings([bundle_trace_id, *bundle_node.source_trace_ids]), + source_data_ids=_unique_strings( + [bundle_node.node_id, *bundle_node.source_graph_node_ids, *bundle_node.source_data_ids] + ), + ) + frame_source_trace_ids = _unique_strings( + [bundle_trace_id, *bundle_node.source_trace_ids, input_trace_id] + ) + frame_source_data_ids = _unique_strings( + [ + bundle_node.node_id, + *bundle_node.source_graph_node_ids, + *bundle_node.source_data_ids, + input_data_id, + ] + ) + if adapter is None: + frame = _failed_frame( + frame_id=frame_id, + summary_graph_node_id=summary_graph_node_id, + bundle_node=bundle_node, + model_id="adapter_missing", + failure_type="adapter_missing", + payload_parse_status="not_checked", + source_trace_ids=frame_source_trace_ids, + source_data_ids=frame_source_data_ids, + llm_call_data_id=None, + llm_trace_event_id=None, + ) + result = _record_failed_frame( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + frame=frame, + ) + return _with_bundle_ids(result, bundle_node.node_id, bundle_edge_ids, created_ids, existing_ids) + + prompt = Path(NIGHT_SUMMARIZE_TOKEN_BUDGET_BUNDLE_PROMPT_REF).read_text(encoding="utf-8") + llm_result = LLMNodeExecutor(adapter).run( + node_id=NIGHT_SUMMARIZE_TOKEN_BUDGET_BUNDLE_NODE_ID, + prompt=prompt, + input_payload=input_payload, + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + prompt_ref=NIGHT_SUMMARIZE_TOKEN_BUDGET_BUNDLE_PROMPT_REF, + input_ref=frame_source_trace_ids, + source_data_ids=frame_source_data_ids, + max_retries=max_retries, + payload_validator=_validate_summary_payload, + ) + source_trace_ids_after_llm = _unique_strings( + [*frame_source_trace_ids, llm_result.trace_event_id] + ) + source_data_ids_after_llm = _unique_strings( + [*frame_source_data_ids, llm_result.call_data_id] + ) + if llm_result.failure_type != "none" or llm_result.validation.payload is None: + frame = _failed_frame( + frame_id=frame_id, + summary_graph_node_id=summary_graph_node_id, + bundle_node=bundle_node, + model_id=llm_result.model_id, + failure_type=llm_result.failure_type, + payload_parse_status=_payload_parse_status(llm_result.failure_type), + source_trace_ids=source_trace_ids_after_llm, + source_data_ids=source_data_ids_after_llm, + llm_call_data_id=llm_result.call_data_id, + llm_trace_event_id=llm_result.trace_event_id, + ) + result = _record_failed_frame( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + frame=frame, + ) + return _with_bundle_ids(result, bundle_node.node_id, bundle_edge_ids, created_ids, existing_ids) + + frame = _frame_from_payload( + payload=llm_result.validation.payload, + frame_id=frame_id, + summary_graph_node_id=summary_graph_node_id, + bundle_node=bundle_node, + model_id=llm_result.model_id, + source_trace_ids=source_trace_ids_after_llm, + source_data_ids=source_data_ids_after_llm, + llm_call_data_id=llm_result.call_data_id, + llm_trace_event_id=llm_result.trace_event_id, + ) + result = _record_success_graph_summary( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + frame=frame, + ) + return _with_bundle_ids(result, bundle_node.node_id, bundle_edge_ids, created_ids, existing_ids) + + +def _with_bundle_ids( + result: RecordedNightTokenBudgetBundleSummaryResult, + bundle_graph_node_id: str, + bundle_edge_ids: list[str], + created_ids: list[str], + existing_ids: list[str], +) -> RecordedNightTokenBudgetBundleSummaryResult: + return RecordedNightTokenBudgetBundleSummaryResult( + frame=result.frame, + trace_event_id=result.trace_event_id, + frame_data_id=result.frame_data_id, + target_bundle_graph_node_id=bundle_graph_node_id, + summary_graph_node_data_id=result.summary_graph_node_data_id, + summary_edge_data_id=result.summary_edge_data_id, + bundle_edge_data_ids=bundle_edge_ids, + created_data_ids=[*created_ids, *result.created_data_ids], + existing_data_ids=[*existing_ids, *result.existing_data_ids], + ) + + +def _frame_from_payload( + *, + payload: dict[str, object], + frame_id: str, + summary_graph_node_id: str, + bundle_node: GraphMemoryNodeFrame, + model_id: str, + source_trace_ids: list[str], + source_data_ids: list[str], + llm_call_data_id: str | None, + llm_trace_event_id: str | None, +) -> NightTokenBudgetBundleSummaryFrame: + frame = NightTokenBudgetBundleSummaryFrame( + frame_id=frame_id, + summary_graph_node_id=summary_graph_node_id, + target_graph_node_id=bundle_node.node_id, + target_node_kind=bundle_node.node_kind, + summary_text=str(payload.get("summary_text") or "").strip(), + summary_status="ran", + failure_type="none", + payload_parse_status="passed", + summary_depth=bundle_node.summary_depth + 1, + source_depth_min=bundle_node.source_depth_min, + source_depth_max=bundle_node.source_depth_max, + source_leaf_count=bundle_node.source_leaf_count, + source_summary_count=bundle_node.source_summary_count, + char_budget=bundle_node.char_budget or 0, + input_summary_char_count=bundle_node.source_char_count, + llm_call_data_id=llm_call_data_id, + llm_trace_event_id=llm_trace_event_id, + source_graph_node_ids=[bundle_node.node_id, *bundle_node.source_graph_node_ids], + source_trace_ids=source_trace_ids, + source_data_ids=source_data_ids, + generated_by=f"LLM:{model_id}:night_summarize_token_budget_bundle", + info_class="mixed", + semantic_judgement_status="ran", + ) + _validate_token_budget_summary_frame(frame) + return frame + + +def _failed_frame( + *, + frame_id: str, + summary_graph_node_id: str, + bundle_node: GraphMemoryNodeFrame, + model_id: str, + failure_type: str, + payload_parse_status: str, + source_trace_ids: list[str], + source_data_ids: list[str], + llm_call_data_id: str | None, + llm_trace_event_id: str | None, +) -> NightTokenBudgetBundleSummaryFrame: + frame = NightTokenBudgetBundleSummaryFrame( + frame_id=frame_id, + summary_graph_node_id=summary_graph_node_id, + target_graph_node_id=bundle_node.node_id, + target_node_kind=bundle_node.node_kind, + summary_text="", + summary_status="failed", + failure_type=failure_type, + payload_parse_status=payload_parse_status, + summary_depth=bundle_node.summary_depth + 1, + source_depth_min=bundle_node.source_depth_min, + source_depth_max=bundle_node.source_depth_max, + source_leaf_count=bundle_node.source_leaf_count, + source_summary_count=bundle_node.source_summary_count, + char_budget=bundle_node.char_budget or 0, + input_summary_char_count=bundle_node.source_char_count, + llm_call_data_id=llm_call_data_id, + llm_trace_event_id=llm_trace_event_id, + source_graph_node_ids=[bundle_node.node_id, *bundle_node.source_graph_node_ids], + source_trace_ids=source_trace_ids, + source_data_ids=source_data_ids, + generated_by=f"LLM:{model_id}:night_summarize_token_budget_bundle", + info_class="mixed", + semantic_judgement_status="failed", + ) + _validate_token_budget_summary_frame(frame) + return frame + + +def _record_success_graph_summary( + *, + trace_store: TraceStore, + data_store: DataStore, + turn_id: str, + frame: NightTokenBudgetBundleSummaryFrame, +) -> RecordedNightTokenBudgetBundleSummaryResult: + edge = _summary_of_edge(frame) + validate_graph_memory_edge_frame(edge) + event = trace_store.create_event( + turn_id=turn_id, + actor=NIGHT_SUMMARIZE_TOKEN_BUDGET_BUNDLE_NODE_ID, + event_type="node_output", + input_ref=frame.source_trace_ids or [], + output_ref=[frame.summary_graph_node_id, edge.edge_id], + schema_status="passed", + ) + created_data_ids: list[str] = [] + existing_data_ids: list[str] = [] + _record_payload_if_missing( + data_store=data_store, + data_id=frame.summary_graph_node_id, + data_type="graph_memory:node:summary", + payload=asdict(frame), + created_at=event.timestamp, + source_trace_id=event.event_id, + created_data_ids=created_data_ids, + existing_data_ids=existing_data_ids, + ) + _record_payload_if_missing( + data_store=data_store, + data_id=edge.edge_id, + data_type="graph_memory:edge:SUMMARY_OF", + payload=asdict(edge), + created_at=event.timestamp, + source_trace_id=event.event_id, + created_data_ids=created_data_ids, + existing_data_ids=existing_data_ids, + ) + return RecordedNightTokenBudgetBundleSummaryResult( + frame=frame, + trace_event_id=event.event_id, + frame_data_id=frame.summary_graph_node_id, + target_bundle_graph_node_id=frame.target_graph_node_id, + summary_graph_node_data_id=frame.summary_graph_node_id, + summary_edge_data_id=edge.edge_id, + bundle_edge_data_ids=[], + created_data_ids=created_data_ids, + existing_data_ids=existing_data_ids, + ) + + +def _record_failed_frame( + *, + trace_store: TraceStore, + data_store: DataStore, + turn_id: str, + frame: NightTokenBudgetBundleSummaryFrame, +) -> RecordedNightTokenBudgetBundleSummaryResult: + event = trace_store.create_event( + turn_id=turn_id, + actor=NIGHT_SUMMARIZE_TOKEN_BUDGET_BUNDLE_NODE_ID, + event_type="node_output", + input_ref=frame.source_trace_ids or [], + output_ref=[frame.frame_id], + schema_status="failed", + ) + created_data_ids: list[str] = [] + existing_data_ids: list[str] = [] + _record_payload_if_missing( + data_store=data_store, + data_id=frame.frame_id, + data_type=NIGHT_SUMMARIZE_TOKEN_BUDGET_BUNDLE_FRAME_DATA_TYPE, + payload=asdict(frame), + created_at=event.timestamp, + source_trace_id=event.event_id, + created_data_ids=created_data_ids, + existing_data_ids=existing_data_ids, + ) + return RecordedNightTokenBudgetBundleSummaryResult( + frame=frame, + trace_event_id=event.event_id, + frame_data_id=frame.frame_id, + target_bundle_graph_node_id=frame.target_graph_node_id, + summary_graph_node_data_id=None, + summary_edge_data_id=None, + bundle_edge_data_ids=[], + created_data_ids=created_data_ids, + existing_data_ids=existing_data_ids, + ) + + +def _contains_edge(bundle_node: GraphMemoryNodeFrame, child_node_id: str) -> GraphMemoryEdgeFrame: + return GraphMemoryEdgeFrame( + edge_id=f"graph:edge:contains:{bundle_node.node_id}:{child_node_id}", + edge_kind="CONTAINS", + from_node_id=bundle_node.node_id, + to_node_id=child_node_id, + source_graph_node_ids=[bundle_node.node_id, child_node_id], + source_trace_ids=list(bundle_node.source_trace_ids), + source_data_ids=[bundle_node.node_id, child_node_id], + ) + + +def _summary_of_edge(frame: NightTokenBudgetBundleSummaryFrame) -> GraphMemoryEdgeFrame: + return GraphMemoryEdgeFrame( + edge_id=( + "graph:edge:summary_of:" + f"{frame.summary_graph_node_id}:{frame.target_graph_node_id}" + ), + edge_kind="SUMMARY_OF", + from_node_id=frame.summary_graph_node_id, + to_node_id=frame.target_graph_node_id, + source_graph_node_ids=_unique_strings( + [ + frame.summary_graph_node_id, + frame.target_graph_node_id, + *(frame.source_graph_node_ids or []), + ] + ), + source_trace_ids=list(frame.source_trace_ids or []), + source_data_ids=_unique_strings( + [ + frame.summary_graph_node_id, + frame.target_graph_node_id, + *(frame.source_graph_node_ids or []), + ] + ), + ) + + +def _input_payload( + *, + input_data_id: str, + summary_run_id: str, + bundle_node_payload: dict[str, object], + source_summary_payloads: list[dict[str, object]], +) -> dict[str, object]: + return { + "input_data_id": input_data_id, + "summary_run_id": summary_run_id, + "target_token_budget_bundle_payload": dict(bundle_node_payload), + "source_summary_payloads": [ + { + "summary_graph_node_id": _text(payload.get("summary_graph_node_id")) + or _text(payload.get("frame_id")), + "source_kind": _text(payload.get("source_kind")), + "source_path": _text(payload.get("source_path")), + "summary_text": _text(payload.get("summary_text")), + "summary_status": _text(payload.get("summary_status")), + "info_class": _text(payload.get("info_class")), + } + for payload in source_summary_payloads + ], + "expected_info_class": "mixed", + "budget_unit": "characters", + "classification_rule": ( + "token-budget layer v0 uses code-counted summary_text characters; " + "successful bundle summaries are mixed because they depend on multiple summaries" + ), + "generated_by": "CODE:NIGHT_TOKEN_BUDGET_BUNDLE_INPUT_BUILDER", + "semantic_judgement_status": "not_run", + } + + +def _record_input( + *, + trace_store: TraceStore, + data_store: DataStore, + turn_id: str, + input_data_id: str, + input_payload: dict[str, object], + source_trace_ids: list[str], + source_data_ids: list[str], +) -> str: + event = trace_store.create_event( + turn_id=turn_id, + actor=NIGHT_SUMMARIZE_TOKEN_BUDGET_BUNDLE_NODE_ID, + event_type="node_input", + input_ref=source_trace_ids, + output_ref=[input_data_id], + schema_status="not_checked", + ) + data_store.create_record( + data_id=input_data_id, + data_type="node_input:night_summarize_token_budget_bundle_input", + exists=True, + created_at=event.timestamp, + source_trace_id=event.event_id, + payload={ + **input_payload, + "source_trace_ids": source_trace_ids, + "source_data_ids": source_data_ids, + }, + ) + return event.event_id + + +def _validate_summary_payload(payload: dict[str, object]) -> None: + summary_text = str(payload.get("summary_text") or "").strip() + if not summary_text: + raise ValueError("summary_text must not be empty") + supplied_info_class = payload.get("info_class") + if supplied_info_class is not None and supplied_info_class != "mixed": + raise ValueError("token budget bundle summary info_class must be mixed") + supplied_source_ids = payload.get("source_graph_node_ids") + if supplied_source_ids: + raise ValueError("LLM must not supply source_graph_node_ids") + + +def _validate_token_budget_summary_frame(frame: NightTokenBudgetBundleSummaryFrame) -> None: + for field_name, value in { + "frame_id": frame.frame_id, + "summary_graph_node_id": frame.summary_graph_node_id, + "target_graph_node_id": frame.target_graph_node_id, + "target_node_kind": frame.target_node_kind, + "summary_status": frame.summary_status, + "failure_type": frame.failure_type, + "payload_parse_status": frame.payload_parse_status, + "generated_by": frame.generated_by, + "info_class": frame.info_class, + "semantic_judgement_status": frame.semantic_judgement_status, + }.items(): + if not value: + raise ValueError(f"NightTokenBudgetBundleSummaryFrame.{field_name} must not be empty") + if frame.target_node_kind != TOKEN_BUDGET_SUMMARY_BUNDLE_NODE_KIND: + raise ValueError("token budget bundle summary target must be token budget bundle") + if frame.info_class != "mixed": + raise ValueError("token budget bundle summary info_class must be mixed") + if frame.semantic_judgement_status not in {"ran", "failed"}: + raise ValueError("token budget bundle semantic status is invalid") + if frame.summary_status not in {"ran", "failed"}: + raise ValueError("token budget bundle summary_status is invalid") + if frame.failure_type not in { + "none", + "adapter_missing", + "adapter_failed", + "parse_failed", + "schema_failed", + }: + raise ValueError("token budget bundle failure_type is invalid") + if frame.summary_status == "ran" and not frame.summary_text: + raise ValueError("successful token budget bundle summary_text must not be empty") + if frame.char_budget <= 0: + raise ValueError("token budget bundle char_budget must be positive") + if frame.input_summary_char_count < 0: + raise ValueError("token budget bundle input_summary_char_count must be non-negative") + if frame.target_graph_node_id not in (frame.source_data_ids or []): + raise ValueError("token budget bundle summary source_data_ids must include target") + for graph_node_id in frame.source_graph_node_ids or []: + if graph_node_id not in (frame.source_data_ids or []): + raise ValueError("token budget bundle summary source_data_ids must include graph ids") + + +def _require_summary_payload( + *, + data_store: DataStore, + summary_graph_node_id: str, +) -> dict[str, object]: + record = data_store.require_record(summary_graph_node_id) + if record.data_type != "graph_memory:node:summary": + raise ValueError(f"expected graph summary node: {summary_graph_node_id}") + if not isinstance(record.payload, dict): + raise TypeError(f"summary node payload must be dict: {summary_graph_node_id}") + return dict(record.payload) + + +def _record_payload_if_missing( + *, + data_store: DataStore, + data_id: str, + data_type: str, + payload: dict[str, object], + created_at: str, + source_trace_id: str, + created_data_ids: list[str], + existing_data_ids: list[str], +) -> None: + existing = data_store.get_record(data_id) + if existing is not None: + if existing.data_type != data_type: + raise ValueError(f"token budget summary data_id collision: {data_id}") + if existing.payload != payload: + raise ValueError( + f"token budget summary data_id collision with different payload: {data_id}" + ) + existing_data_ids.append(data_id) + return + data_store.create_record( + data_id=data_id, + data_type=data_type, + exists=True, + created_at=created_at, + source_trace_id=source_trace_id, + payload=payload, + ) + created_data_ids.append(data_id) + + +def _payload_parse_status(failure_type: str) -> str: + if failure_type == "parse_failed": + return "failed" + if failure_type in {"adapter_failed", "adapter_missing"}: + return "not_checked" + return "passed" + + +def _require_text(field_name: str, value: str) -> None: + if not value: + raise ValueError(f"{field_name} must not be empty") + + +def _text(value: object) -> str: + return value if isinstance(value, str) else "" + + +def _int(value: object) -> int: + return value if isinstance(value, int) and value >= 0 else 0 + + +def _string_list(value: object) -> list[str]: + if not isinstance(value, list): + return [] + return [item for item in value if isinstance(item, str) and item] + + +def _unique_strings(values: list[str | None]) -> list[str]: + seen: set[str] = set() + result: list[str] = [] + for value in values: + if not value or value in seen: + continue + seen.add(value) + result.append(value) + return result + + +def _stable_suffix(value: str) -> str: + return hashlib.sha1(value.encode("utf-8")).hexdigest()[:16] + + +def _safe_id_part(value: str) -> str: + cleaned = "".join(char if char.isalnum() else "_" for char in value.strip()) + cleaned = cleaned.strip("_") + if not cleaned: + raise ValueError("safe id part must not be empty") + return cleaned + + +__all__ = [ + "NIGHT_SUMMARIZE_TOKEN_BUDGET_BUNDLE_FRAME_DATA_TYPE", + "NIGHT_SUMMARIZE_TOKEN_BUDGET_BUNDLE_NODE_ID", + "NIGHT_SUMMARIZE_TOKEN_BUDGET_BUNDLE_PROMPT_REF", + "TOKEN_BUDGET_SUMMARY_BUNDLE_NODE_DATA_TYPE", + "TOKEN_BUDGET_SUMMARY_BUNDLE_NODE_KIND", + "TOKEN_BUDGET_SUMMARY_BUNDLE_POLICY_ID", + "NightTokenBudgetBundleSummaryFrame", + "RecordedNightTokenBudgetBundleSummaryResult", + "TokenBudgetSummaryBundleSpec", + "build_token_budget_summary_bundle_specs", + "collect_active_source_leaf_summary_graph_node_ids", + "night_summarize_token_budget_bundle_frame_id", + "night_summarize_token_budget_bundle_graph_node_id", + "record_token_budget_summary_bundle_node", + "run_night_summarize_token_budget_bundle", + "token_budget_summary_bundle_graph_node_id", +] diff --git a/songryeon_core/nodes/night_time_bundle_summary_worker.py b/songryeon_core/nodes/night_time_bundle_summary_worker.py new file mode 100644 index 0000000..c227bf5 --- /dev/null +++ b/songryeon_core/nodes/night_time_bundle_summary_worker.py @@ -0,0 +1,37 @@ +"""Compatibility imports for the old ORDER 166 worker module name. + +New code should import from songryeon_core.nodes.night_summarize_time_bundle. +""" + +from songryeon_core.nodes.night_summarize_time_bundle import ( + NIGHT_SUMMARIZE_TIME_BUNDLE_FRAME_DATA_TYPE, + NIGHT_SUMMARIZE_TIME_BUNDLE_NODE_ID, + NIGHT_SUMMARIZE_TIME_BUNDLE_PROMPT_REF, + NIGHT_TIME_BUNDLE_SUMMARY_FRAME_DATA_TYPE, + NIGHT_TIME_BUNDLE_SUMMARY_PROMPT_REF, + NIGHT_TIME_BUNDLE_SUMMARY_WORKER_NODE_ID, + RecordedNightTimeBundleSummaryResult, + night_summarize_time_bundle_frame_id, + night_summarize_time_bundle_graph_node_id, + night_time_bundle_summary_frame_id, + night_time_bundle_summary_graph_node_id, + run_night_summarize_time_bundle, + run_night_time_bundle_summary_worker, +) + + +__all__ = [ + "NIGHT_SUMMARIZE_TIME_BUNDLE_FRAME_DATA_TYPE", + "NIGHT_SUMMARIZE_TIME_BUNDLE_NODE_ID", + "NIGHT_SUMMARIZE_TIME_BUNDLE_PROMPT_REF", + "NIGHT_TIME_BUNDLE_SUMMARY_FRAME_DATA_TYPE", + "NIGHT_TIME_BUNDLE_SUMMARY_PROMPT_REF", + "NIGHT_TIME_BUNDLE_SUMMARY_WORKER_NODE_ID", + "RecordedNightTimeBundleSummaryResult", + "night_summarize_time_bundle_frame_id", + "night_summarize_time_bundle_graph_node_id", + "night_time_bundle_summary_frame_id", + "night_time_bundle_summary_graph_node_id", + "run_night_summarize_time_bundle", + "run_night_time_bundle_summary_worker", +] diff --git a/songryeon_core/nodes/node_0_memory_supplier.py b/songryeon_core/nodes/node_0_memory_supplier.py index b90e679..9902241 100644 --- a/songryeon_core/nodes/node_0_memory_supplier.py +++ b/songryeon_core/nodes/node_0_memory_supplier.py @@ -9,10 +9,16 @@ MemoryPacketFrom0, MemoryPacketPayload, MemoryRelevanceCandidateFrame, + Node0DocumentMaterialItem, + Node0DocumentMaterialPacketFrame, RawMemoryCompressionCandidateFrame, + RLoopGraphGuidePacketFrame, + RLoopMemoryHandoffPacketFrame, ZeroState, validate_l_loop_return_summary_frame, validate_memory_packet_payload, + validate_node0_document_material_packet_frame, + validate_r_loop_memory_handoff_packet_frame, ) from songryeon_core.core.trace_store import TraceStore from songryeon_core.loops.l_loop_namespace import LRunIds @@ -35,6 +41,8 @@ RAW_MEMORY_COMPRESSION_BATCH_SIZE = 4 L3_CONTINUATION_SUMMARY_MODE = "l3_continuation_summary_for_L2" L_LOOP_RETURN_SUMMARY_FRAME_DATA_ID = "L:return_summary_frame" +NODE0_DOCUMENT_MATERIAL_PACKET_FRAME_DATA_ID = "node_0:document_material_packet_frame" +NODE0_R_LOOP_MEMORY_HANDOFF_PACKET_FRAME_DATA_ID = "node_0:r_loop_memory_handoff_packet_frame" NODE_0_MODES = { "pre_route_report", @@ -54,6 +62,20 @@ def memory_packet_data_id(target: str, mode: str, packet_id_suffix: str | None = return f"{base_id}:{packet_id_suffix}" +def document_material_packet_frame_data_id(*, id_namespace: LRunIds | None) -> str: + """L 이후 node_0 문서 장부 frame ID를 만든다.""" + + if id_namespace is None: + return NODE0_DOCUMENT_MATERIAL_PACKET_FRAME_DATA_ID + return id_namespace.scoped_data_id(NODE0_DOCUMENT_MATERIAL_PACKET_FRAME_DATA_ID) + + +def r_loop_memory_handoff_packet_frame_data_id() -> str: + """R loop graph guide handoff frame ID를 만든다.""" + + return NODE0_R_LOOP_MEMORY_HANDOFF_PACKET_FRAME_DATA_ID + + def supply_memory( *, target: str, @@ -436,6 +458,106 @@ def record_memory_packet( return event.event_id +def build_r_loop_memory_handoff_packet_frame( + *, + guide_packet: RLoopGraphGuidePacketFrame | None, + packet_id: str | None = None, + source_trace_ids: list[str] | None = None, + source_data_ids: list[str] | None = None, + semantic_hint_status: str = "not_run", + semantic_hint_frame_id: str | None = None, +) -> RLoopMemoryHandoffPacketFrame: + """R loop이 나중에 읽을 graph guide 좌표를 node_0 absolute packet으로 복사한다.""" + + packet_id = packet_id or r_loop_memory_handoff_packet_frame_data_id() + if guide_packet is None: + frame = RLoopMemoryHandoffPacketFrame( + packet_id=packet_id, + packet_status="missing", + semantic_hint_status="not_run", + source_trace_ids=_unique_strings(source_trace_ids or []), + source_data_ids=_unique_strings(source_data_ids or []), + ) + validate_r_loop_memory_handoff_packet_frame(frame) + return frame + + copied_source_data_ids = _unique_strings( + [ + guide_packet.packet_id, + guide_packet.graph_snapshot_id, + *(source_data_ids or []), + *guide_packet.source_data_ids, + semantic_hint_frame_id, + ] + ) + frame = RLoopMemoryHandoffPacketFrame( + packet_id=packet_id, + target="R_LOOP", + mode="graph_guide_handoff", + packet_status="available", + graph_snapshot_id=guide_packet.graph_snapshot_id, + r_loop_graph_guide_packet_id=guide_packet.packet_id, + available_entry_node_ids=list(guide_packet.available_entry_nodes), + node_kind_counts=dict(guide_packet.node_kind_counts), + data_kind_counts=dict(guide_packet.data_kind_counts), + summary_depth_range=list(guide_packet.summary_depth_range), + source_leaf_count_range=list(guide_packet.source_leaf_count_range), + semantic_hint_status=semantic_hint_status, + semantic_hint_frame_id=semantic_hint_frame_id, + source_graph_node_ids=list(guide_packet.source_graph_node_ids), + source_trace_ids=_unique_strings( + [*(source_trace_ids or []), *guide_packet.source_trace_ids] + ), + source_data_ids=copied_source_data_ids, + generated_by="CODE:node_0_memory_supplier", + info_class="absolute", + semantic_judgement_status="not_run", + ) + validate_r_loop_memory_handoff_packet_frame(frame) + return frame + + +def record_r_loop_memory_handoff_packet( + *, + trace_store: TraceStore, + data_store: DataStore, + turn_id: str, + guide_packet: RLoopGraphGuidePacketFrame | None, + packet_id: str | None = None, + input_ref: list[str] | None = None, + source_data_ids: list[str] | None = None, + semantic_hint_status: str = "not_run", + semantic_hint_frame_id: str | None = None, +) -> tuple[str, str, RLoopMemoryHandoffPacketFrame]: + """0이 R_LOOP에게 graph guide 좌표를 공급했다는 사실을 trace/data에 남긴다.""" + + frame = build_r_loop_memory_handoff_packet_frame( + guide_packet=guide_packet, + packet_id=packet_id, + source_trace_ids=input_ref or [], + source_data_ids=source_data_ids or [], + semantic_hint_status=semantic_hint_status, + semantic_hint_frame_id=semantic_hint_frame_id, + ) + event = trace_store.create_event( + turn_id=turn_id, + actor="node_0", + event_type="memory_packet", + input_ref=input_ref or frame.source_trace_ids, + output_ref=[frame.packet_id], + schema_status="passed", + ) + data_store.create_record( + data_id=frame.packet_id, + data_type="node_output:r_loop_memory_handoff_packet_frame", + exists=True, + created_at=event.timestamp, + source_trace_id=event.event_id, + payload=asdict(frame), + ) + return event.event_id, frame.packet_id, frame + + def record_l3_continuation_summary_for_l2( *, trace_store: TraceStore, @@ -532,6 +654,14 @@ def record_l_loop_return_summary_for_node1( source_trace_id=summary_event.event_id, payload=asdict(frame), ) + material_trace_id, material_frame_id, material_frame = record_node0_document_material_packet( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + frame_id=document_material_packet_frame_data_id(id_namespace=id_namespace), + source_trace_ids=_unique_strings([*input_ref, summary_event.event_id]), + source_data_ids=_unique_strings([return_summary_frame_id, *source_data_ids]), + ) packet = supply_memory( target="node_1", @@ -546,13 +676,16 @@ def record_l_loop_return_summary_for_node1( turn_id=turn_id, packet=packet, mode="loop_return_summary", - input_ref=_unique_strings([*input_ref, summary_event.event_id]), + input_ref=_unique_strings([*input_ref, summary_event.event_id, material_trace_id]), source_data_ids=_unique_strings( - [return_summary_frame_id, *source_data_ids] + [return_summary_frame_id, material_frame_id, *source_data_ids] ), compression_summary="CODE_STATUS:l_loop_return_summary_supplied", operation_label="CODE_STATUS:l_loop_return_summary_supplied", - memory_items=build_l_loop_return_summary_items(frame), + memory_items=[ + *build_l_loop_return_summary_items(frame), + build_document_material_packet_status_item(material_frame), + ], id_namespace=id_namespace, ) packet_data_id = ( @@ -563,6 +696,218 @@ def record_l_loop_return_summary_for_node1( return packet_trace_id, packet_data_id, return_summary_frame_id, packet +def record_node0_document_material_packet( + *, + trace_store: TraceStore, + data_store: DataStore, + turn_id: str, + frame_id: str, + source_trace_ids: list[str], + source_data_ids: list[str], +) -> tuple[str, str, Node0DocumentMaterialPacketFrame]: + """L 이후 문서 검색/읽기/context 공급 상태를 node_0 절대정보 장부로 기록한다.""" + + frame = build_node0_document_material_packet_frame( + data_store=data_store, + turn_id=turn_id, + frame_id=frame_id, + source_trace_ids=source_trace_ids, + source_data_ids=source_data_ids, + ) + validate_node0_document_material_packet_frame(frame) + event = trace_store.create_event( + turn_id=turn_id, + actor="node_0", + event_type="node_output", + input_ref=frame.source_trace_ids, + output_ref=[frame_id], + schema_status="passed", + ) + data_store.create_record( + data_id=frame_id, + data_type="node_output:node0_document_material_packet_frame", + exists=True, + created_at=event.timestamp, + source_trace_id=event.event_id, + payload=asdict(frame), + ) + return event.event_id, frame_id, frame + + +def build_node0_document_material_packet_frame( + *, + data_store: DataStore, + turn_id: str, + frame_id: str, + source_trace_ids: list[str], + source_data_ids: list[str], +) -> Node0DocumentMaterialPacketFrame: + """흩어진 L 산출 문서 좌표를 문서별 절대정보 장부로 합친다.""" + + return_summary_payload = _latest_payload_by_exact_type( + data_store=data_store, + source_data_ids=source_data_ids, + data_type="node_output:l_loop_return_summary_frame", + ) + l3_payload = _latest_payload_by_type_fragment( + data_store=data_store, + source_data_ids=source_data_ids, + type_fragment="L3_achievement_frame", + ) + context_pack_payload = _latest_payload_by_exact_type( + data_store=data_store, + source_data_ids=source_data_ids, + data_type="node_output:document_context_pack_frame", + ) + read_doc_ids = _unique_strings( + [ + *_string_list(return_summary_payload.get("read_doc_ids")), + *_document_extract_doc_ids( + data_store=data_store, + source_data_ids=source_data_ids, + require_text=True, + ), + ] + ) + if not read_doc_ids: + read_doc_ids = _string_list(l3_payload.get("read_doc_ids")) + search_doc_ids = _string_list(return_summary_payload.get("search_result_doc_ids")) + if not search_doc_ids: + search_doc_ids = _string_list(l3_payload.get("search_result_doc_ids")) + + item_map: dict[str, dict[str, object]] = {} + source_trace_ids_by_data_id = _source_trace_ids_by_data_id( + data_store=data_store, + source_data_ids=source_data_ids, + ) + return_summary_data_id = _text(return_summary_payload, "frame_id", fallback="") + l3_data_id = _text(l3_payload, "frame_id", fallback="") + context_pack_data_id = _text(context_pack_payload, "frame_id", fallback="") + + for rank, doc_id in enumerate(search_doc_ids, start=1): + entry = _material_entry(item_map, doc_id) + _add_material_role(entry, "search_candidate") + entry["was_search_candidate"] = True + entry["search_candidate_rank"] = rank + _extend_material_sources( + entry, + data_ids=[return_summary_data_id, l3_data_id], + trace_ids=source_trace_ids_by_data_id, + ) + + tool_read_sources = _tool_read_sources_by_doc_id( + data_store=data_store, + source_data_ids=source_data_ids, + ) + for rank, doc_id in enumerate(read_doc_ids, start=1): + entry = _material_entry(item_map, doc_id) + _add_material_role(entry, "actual_tool_read_doc") + entry["was_actual_tool_read_doc"] = True + entry["actual_read_rank"] = rank + tool_source = tool_read_sources.get(doc_id, {}) + char_count = tool_source.get("char_count") + if isinstance(char_count, int) and char_count > int(entry["char_count"]): + entry["char_count"] = char_count + _extend_material_sources( + entry, + data_ids=[return_summary_data_id, l3_data_id, str(tool_source.get("data_id") or "")], + trace_ids=source_trace_ids_by_data_id, + ) + + included_documents = context_pack_payload.get("included_documents") + if isinstance(included_documents, list): + for rank, item in enumerate(included_documents, start=1): + if not isinstance(item, dict): + continue + doc_id = _text_from_value(item.get("doc_id")) + if not doc_id: + continue + entry = _material_entry(item_map, doc_id) + _add_material_role(entry, "supplied_document_context") + entry["was_supplied_document_context"] = True + entry["supplied_context_rank"] = rank + entry["document_name"] = _text_from_value(item.get("document_name")) or _document_name(doc_id) + char_count = item.get("char_count") + if isinstance(char_count, int) and char_count > int(entry["char_count"]): + entry["char_count"] = char_count + _extend_material_sources( + entry, + data_ids=[context_pack_data_id, _text_from_value(item.get("source_data_id"))], + trace_ids=source_trace_ids_by_data_id, + ) + + excluded_documents = context_pack_payload.get("excluded_documents") + if isinstance(excluded_documents, list): + for rank, item in enumerate(excluded_documents, start=1): + if not isinstance(item, dict): + continue + doc_id = _text_from_value(item.get("doc_id")) + if not doc_id: + continue + entry = _material_entry(item_map, doc_id) + _add_material_role(entry, "excluded_document_context") + entry["was_excluded_document_context"] = True + entry["excluded_context_rank"] = rank + entry["document_name"] = _text_from_value(item.get("document_name")) or _document_name(doc_id) + char_count = item.get("char_count") + if isinstance(char_count, int) and char_count > int(entry["char_count"]): + entry["char_count"] = char_count + _extend_material_sources( + entry, + data_ids=[context_pack_data_id, _text_from_value(item.get("source_data_id"))], + trace_ids=source_trace_ids_by_data_id, + ) + + for entry in item_map.values(): + if bool(entry["was_search_candidate"]) and not bool(entry["was_actual_tool_read_doc"]): + _add_material_role(entry, "unread_candidate") + entry["was_unread_candidate"] = True + + items = [_entry_to_document_material_item(entry) for entry in item_map.values()] + items.sort( + key=lambda item: ( + item.search_candidate_rank or 999999, + item.actual_read_rank or 999999, + item.supplied_context_rank or 999999, + item.excluded_context_rank or 999999, + item.document_name, + ) + ) + return Node0DocumentMaterialPacketFrame( + frame_id=frame_id, + turn_id=turn_id, + items=items, + item_count=len(items), + search_candidate_count=sum(1 for item in items if item.was_search_candidate), + actual_tool_read_doc_count=sum(1 for item in items if item.was_actual_tool_read_doc), + supplied_document_context_count=sum(1 for item in items if item.was_supplied_document_context), + excluded_document_context_count=sum(1 for item in items if item.was_excluded_document_context), + unread_candidate_count=sum(1 for item in items if item.was_unread_candidate), + source_trace_ids=_unique_strings(source_trace_ids), + source_data_ids=_unique_strings(source_data_ids), + ) + + +def build_document_material_packet_status_item( + frame: Node0DocumentMaterialPacketFrame, +) -> MemoryItem: + """node_1도 문서 장부 존재와 count를 알 수 있게 memory item으로 복사한다.""" + + return _memory_item( + packet_source_id=frame.frame_id, + item_type="document_material_packet_status", + text=( + "COPIED_FIELDS:" + f"item_count={frame.item_count};" + f"search_candidate_count={frame.search_candidate_count};" + f"actual_tool_read_doc_count={frame.actual_tool_read_doc_count};" + f"supplied_document_context_count={frame.supplied_document_context_count};" + f"unread_candidate_count={frame.unread_candidate_count}" + ), + source_data_ids=[frame.frame_id, *frame.source_data_ids], + ) + + def build_l_loop_return_summary_frame( *, data_store: DataStore, @@ -600,11 +945,31 @@ def build_l_loop_return_summary_frame( fallback="unspecified", ) required_min_read_documents = _int(l1_payload, "minimum_read_documents") - read_doc_ids = _string_list(l3_payload.get("read_doc_ids")) + read_doc_ids = _unique_strings( + [ + *_string_list(l3_payload.get("read_doc_ids")), + *_document_extract_doc_ids( + data_store=data_store, + source_data_ids=source_data_ids, + require_text=True, + ), + ] + ) if not read_doc_ids: read_doc_ids = _string_list(budget_payload.get("read_doc_ids")) + read_code_file_paths = _unique_strings( + [ + *_string_list(l3_payload.get("read_code_file_paths")), + *_code_extract_file_paths( + data_store=data_store, + source_data_ids=source_data_ids, + require_text=True, + ), + ] + ) search_result_doc_ids = _string_list(l3_payload.get("search_result_doc_ids")) actual_read_doc_count = len(read_doc_ids) + actual_read_code_file_count = len(read_code_file_paths) search_candidate_count = len(search_result_doc_ids) if search_candidate_count == 0: search_candidate_count = _int(l3_payload, "candidate_count") @@ -643,6 +1008,7 @@ def build_l_loop_return_summary_frame( l3_semantic_goal_match_status=l3_semantic_goal_match_status, required_min_read_documents=required_min_read_documents, actual_read_doc_count=actual_read_doc_count, + actual_source_evidence_count=actual_read_doc_count + actual_read_code_file_count, budget_stop_reason=budget_stop_reason, final_continuation_status=final_continuation_status, remaining_tool_calls=remaining_tool_calls, @@ -659,6 +1025,7 @@ def build_l_loop_return_summary_frame( evidence_requirement_kind=evidence_requirement_kind, required_min_read_documents=required_min_read_documents, actual_read_doc_count=actual_read_doc_count, + actual_read_code_file_count=actual_read_code_file_count, search_candidate_count=search_candidate_count, final_continuation_status=final_continuation_status, budget_stop_reason=budget_stop_reason, @@ -670,6 +1037,7 @@ def build_l_loop_return_summary_frame( recommended_next_route_for_node1=route_hint, route_hint_reason=route_hint_reason, read_doc_ids=read_doc_ids, + read_code_file_paths=read_code_file_paths, search_result_doc_ids=search_result_doc_ids, source_trace_ids=_unique_strings(source_trace_ids), source_data_ids=_unique_strings(source_data_ids), @@ -700,6 +1068,7 @@ def build_l_loop_return_summary_items(frame: LLoopReturnSummaryFrame) -> list[Me f"evidence_requirement_kind={frame.evidence_requirement_kind};" f"required_min_read_documents={frame.required_min_read_documents};" f"actual_read_doc_count={frame.actual_read_doc_count};" + f"actual_read_code_file_count={frame.actual_read_code_file_count};" f"search_candidate_count={frame.search_candidate_count}" ), source_data_ids=source_data_ids, @@ -732,6 +1101,7 @@ def build_l_loop_return_summary_items(frame: LLoopReturnSummaryFrame) -> list[Me text=( "COPIED_FIELDS:" f"read_doc_ids={_format_list(frame.read_doc_ids)};" + f"read_code_file_paths={_format_list(frame.read_code_file_paths)};" f"search_result_doc_ids={_format_list(frame.search_result_doc_ids)}" ), source_data_ids=source_data_ids, @@ -784,6 +1154,7 @@ def _return_failure_level_and_route_hint( l3_semantic_goal_match_status: str, required_min_read_documents: int, actual_read_doc_count: int, + actual_source_evidence_count: int, budget_stop_reason: str, final_continuation_status: str, remaining_tool_calls: int, @@ -794,7 +1165,7 @@ def _return_failure_level_and_route_hint( minimum_evidence_missing = ( required_min_read_documents > 0 - and actual_read_doc_count < required_min_read_documents + and actual_source_evidence_count < required_min_read_documents ) l3_unsatisfied = ( l_loop_task_status in {"partial", "failed", "unknown"} @@ -938,6 +1309,209 @@ def build_l3_continuation_summary_items( return items +def _material_entry(item_map: dict[str, dict[str, object]], doc_id: str) -> dict[str, object]: + if doc_id not in item_map: + item_map[doc_id] = { + "doc_id": doc_id, + "document_name": _document_name(doc_id), + "source_roles": [], + "was_search_candidate": False, + "was_actual_tool_read_doc": False, + "was_supplied_document_context": False, + "was_excluded_document_context": False, + "was_unread_candidate": False, + "search_candidate_rank": 0, + "actual_read_rank": 0, + "supplied_context_rank": 0, + "excluded_context_rank": 0, + "char_count": 0, + "source_trace_ids": [], + "source_data_ids": [], + } + return item_map[doc_id] + + +def _add_material_role(entry: dict[str, object], role: str) -> None: + roles = entry.get("source_roles") + if not isinstance(roles, list): + roles = [] + entry["source_roles"] = roles + if role not in roles: + roles.append(role) + + +def _extend_material_sources( + entry: dict[str, object], + *, + data_ids: list[str], + trace_ids: dict[str, str], +) -> None: + source_data_ids = entry.get("source_data_ids") + if not isinstance(source_data_ids, list): + source_data_ids = [] + source_trace_ids = entry.get("source_trace_ids") + if not isinstance(source_trace_ids, list): + source_trace_ids = [] + + for data_id in data_ids: + if not data_id: + continue + if data_id not in source_data_ids: + source_data_ids.append(data_id) + trace_id = trace_ids.get(data_id) + if trace_id and trace_id not in source_trace_ids: + source_trace_ids.append(trace_id) + + entry["source_data_ids"] = source_data_ids + entry["source_trace_ids"] = source_trace_ids + + +def _entry_to_document_material_item(entry: dict[str, object]) -> Node0DocumentMaterialItem: + return Node0DocumentMaterialItem( + doc_id=str(entry.get("doc_id") or ""), + document_name=str(entry.get("document_name") or ""), + source_roles=_string_list(entry.get("source_roles")), + was_search_candidate=bool(entry.get("was_search_candidate")), + was_actual_tool_read_doc=bool(entry.get("was_actual_tool_read_doc")), + was_supplied_document_context=bool(entry.get("was_supplied_document_context")), + was_excluded_document_context=bool(entry.get("was_excluded_document_context")), + was_unread_candidate=bool(entry.get("was_unread_candidate")), + search_candidate_rank=_int(entry, "search_candidate_rank"), + actual_read_rank=_int(entry, "actual_read_rank"), + supplied_context_rank=_int(entry, "supplied_context_rank"), + excluded_context_rank=_int(entry, "excluded_context_rank"), + char_count=_int(entry, "char_count"), + source_trace_ids=_string_list(entry.get("source_trace_ids")), + source_data_ids=_string_list(entry.get("source_data_ids")), + ) + + +def _source_trace_ids_by_data_id( + *, + data_store: DataStore, + source_data_ids: list[str], +) -> dict[str, str]: + trace_ids: dict[str, str] = {} + for data_id in source_data_ids: + record = data_store.get_record(data_id) + if record is None or not record.source_trace_id: + continue + trace_ids[data_id] = record.source_trace_id + return trace_ids + + +def _tool_read_sources_by_doc_id( + *, + data_store: DataStore, + source_data_ids: list[str], +) -> dict[str, dict[str, object]]: + sources: dict[str, dict[str, object]] = {} + for source in _document_extract_sources( + data_store=data_store, + source_data_ids=source_data_ids, + ): + if source.get("has_text") is not True: + continue + doc_id = str(source.get("doc_id") or "") + if doc_id and doc_id not in sources: + sources[doc_id] = source + return sources + + +def _document_extract_doc_ids( + *, + data_store: DataStore, + source_data_ids: list[str], + require_text: bool, +) -> list[str]: + doc_ids: list[str] = [] + for source in _document_extract_sources( + data_store=data_store, + source_data_ids=source_data_ids, + ): + if require_text and source.get("has_text") is not True: + continue + doc_id = source.get("doc_id") + if isinstance(doc_id, str) and doc_id and doc_id not in doc_ids: + doc_ids.append(doc_id) + return doc_ids + + +def _code_extract_file_paths( + *, + data_store: DataStore, + source_data_ids: list[str], + require_text: bool, +) -> list[str]: + file_paths: list[str] = [] + for data_id in source_data_ids: + record = data_store.get_record(data_id) + if record is None or not _is_code_extract_record(record.data_type): + continue + payload = record.payload if isinstance(record.payload, dict) else {} + if payload.get("read_status") != "ok": + continue + if require_text: + text = payload.get("text") + if not isinstance(text, str) or not text.strip(): + continue + file_path = _text_from_value(payload.get("file_path")) + if file_path and file_path not in file_paths: + file_paths.append(file_path) + return file_paths + + +def _document_extract_sources( + *, + data_store: DataStore, + source_data_ids: list[str], +) -> list[dict[str, object]]: + sources: list[dict[str, object]] = [] + seen_data_ids: set[str] = set() + for data_id in source_data_ids: + if data_id in seen_data_ids: + continue + seen_data_ids.add(data_id) + record = data_store.get_record(data_id) + if record is None: + continue + if not _is_document_extract_record(record.data_type): + continue + payload = record.payload if isinstance(record.payload, dict) else {} + doc_id = _text_from_value(payload.get("doc_id")) + if not doc_id: + continue + text = payload.get("text") + has_text = isinstance(text, str) and bool(text.strip()) + sources.append( + { + "doc_id": doc_id, + "data_id": record.data_id, + "source_trace_id": record.source_trace_id or "", + "char_count": payload.get("char_count"), + "has_text": has_text, + } + ) + return sources + + +def _document_name(doc_id: str) -> str: + normalized = doc_id.replace("\\", "/").strip("/") + return normalized.rsplit("/", 1)[-1] or doc_id + + +def _text_from_value(value: object) -> str: + return value.strip() if isinstance(value, str) and value.strip() else "" + + +def _is_document_extract_record(data_type: str) -> bool: + return data_type.startswith("tool_result:read_doc") or data_type.startswith("tool_result:read_artifact") + + +def _is_code_extract_record(data_type: str) -> bool: + return data_type.startswith("tool_result:read_code_file") + + def _memory_item( *, packet_source_id: str, diff --git a/songryeon_core/nodes/node_1_router.py b/songryeon_core/nodes/node_1_router.py index 1a4cd96..ee2320e 100644 --- a/songryeon_core/nodes/node_1_router.py +++ b/songryeon_core/nodes/node_1_router.py @@ -6,6 +6,8 @@ from songryeon_core.core.data_store import DataStore from songryeon_core.core.schemas import ( MemoryPacketFrom0, + R_ROUTE_EXPERIMENTAL_NEXT_0_MODE, + R_ROUTE_EXPERIMENTAL_POLICY_FLAG, RoutingDecision, RoutingDecisionFrame, validate_routing_decision_frame, @@ -110,6 +112,7 @@ def route_next_with_llm( turn_id: str, input_ref: list[str], source_data_ids: list[str], + allow_r_route_experimental: bool = False, max_retries: int = 0, ) -> RoutingDecision: """LLM으로 1번 라우팅을 수행한다. @@ -120,6 +123,14 @@ def route_next_with_llm( prompt_ref = "songryeon_core/prompts/node_1_router_v0.md" prompt = Path(prompt_ref).read_text(encoding="utf-8") + allowed_routes = ["L", "2"] + route_meanings = { + "L": "내부 문서/장기기억 검색 루프", + "2": "최종 메타정보 경계 및 보고 단계", + } + if allow_r_route_experimental: + allowed_routes.append("R") + route_meanings["R"] = "EXPERIMENTAL: graph memory R-loop skeleton; only available when the runtime flag is explicitly enabled." input_payload = { "user_input": user_input, "memory_packet": { @@ -135,10 +146,14 @@ def route_next_with_llm( data_store=data_store, source_data_ids=source_data_ids, ), - "allowed_routes": ["L", "2"], - "route_meanings": { - "L": "내부 문서/장기기억 검색 루프", - "2": "최종 메타정보 경계 및 보고 단계", + "allowed_routes": allowed_routes, + "route_meanings": route_meanings, + "experimental_route_policy": { + "R": { + "enabled": allow_r_route_experimental, + "policy_flag": R_ROUTE_EXPERIMENTAL_POLICY_FLAG, + "expected_next_0_mode": R_ROUTE_EXPERIMENTAL_NEXT_0_MODE, + } }, "source_data_ids": source_data_ids, } @@ -153,7 +168,10 @@ def route_next_with_llm( input_ref=input_ref, source_data_ids=source_data_ids, max_retries=max_retries, - payload_validator=_validate_llm_routing_payload, + payload_validator=lambda payload: _validate_llm_routing_payload( + payload, + allow_r_route_experimental=allow_r_route_experimental, + ), ) if llm_result.failure_type != "none" or llm_result.validation.payload is None: raise Node1RouterLLMFailure( @@ -168,6 +186,7 @@ def route_next_with_llm( model_id=llm_result.model_id, llm_trace_event_id=llm_result.trace_event_id, llm_call_data_id=llm_result.call_data_id, + allow_r_route_experimental=allow_r_route_experimental, ) @@ -186,6 +205,7 @@ def route_next_with_llm_or_policy_fallback( force_l_route: bool = False, fallback_policy: str = ROUTER_FALLBACK_POLICY_DEV_SMOKE, fallback_allowed_by_runtime_policy: bool = True, + allow_r_route_experimental: bool = False, max_retries: int = 0, ) -> RoutingDecision: """LLM router 실패와 code fallback 결정을 분리해서 보존한다.""" @@ -201,6 +221,7 @@ def route_next_with_llm_or_policy_fallback( turn_id=turn_id, input_ref=input_ref, source_data_ids=source_data_ids, + allow_r_route_experimental=allow_r_route_experimental, max_retries=max_retries, ) except Node1RouterLLMFailure as exc: @@ -377,13 +398,18 @@ def _mark_llm_failure_fallback( return decision -def _validate_llm_routing_payload(payload: dict[str, object]) -> None: +def _validate_llm_routing_payload( + payload: dict[str, object], + *, + allow_r_route_experimental: bool = False, +) -> None: _routing_decision_from_llm_payload( payload=payload, schema_registry=None, model_id="validation-model", llm_trace_event_id="validation_trace", llm_call_data_id="validation_call", + allow_r_route_experimental=allow_r_route_experimental, ) @@ -394,17 +420,30 @@ def _routing_decision_from_llm_payload( model_id: str, llm_trace_event_id: str | None, llm_call_data_id: str | None, + allow_r_route_experimental: bool = False, ) -> RoutingDecision: route = str(payload.get("route") or "").strip() - if route not in {"L", "2"}: - raise ValueError("node_1 route must be L or 2") + allowed_routes = {"L", "2"} + if allow_r_route_experimental: + allowed_routes.add("R") + if route not in allowed_routes: + raise ValueError( + "node_1 route must be L or 2" + if not allow_r_route_experimental + else "node_1 route must be L, 2, or experimental R" + ) route_reason = str(payload.get("route_reason") or payload.get("reason") or "").strip() if not route_reason: raise ValueError("node_1 route_reason must not be empty") expected_next_0_mode = str(payload.get("expected_next_0_mode") or "").strip() - expected_for_route = "targeted_memory_supply" if route == "L" else "final_trace_for_2" + if route == "L": + expected_for_route = "targeted_memory_supply" + elif route == "R": + expected_for_route = R_ROUTE_EXPERIMENTAL_NEXT_0_MODE + else: + expected_for_route = "final_trace_for_2" if not expected_next_0_mode or expected_next_0_mode != expected_for_route: expected_next_0_mode = expected_for_route @@ -412,7 +451,7 @@ def _routing_decision_from_llm_payload( if route_confidence is not None and (route_confidence < 0.0 or route_confidence > 1.0): raise ValueError("node_1 route_confidence must be between 0 and 1") - target_node = "node_2" if route == "2" else "L" + target_node = "node_2" if route == "2" else "L" if route == "L" else "R_LOOP" return RoutingDecision( route=route, route_reason=route_reason, @@ -421,7 +460,11 @@ def _routing_decision_from_llm_payload( expected_next_0_mode=expected_next_0_mode, route_rule_id="llm_router", matched_keywords=[], - policy_flag=_optional_text(payload.get("policy_flag")), + policy_flag=( + R_ROUTE_EXPERIMENTAL_POLICY_FLAG + if route == "R" + else _optional_text(payload.get("policy_flag")) + ), route_confidence=route_confidence, needs_more_memory=bool(payload.get("needs_more_memory") or False), llm_routing_status="ran", diff --git a/songryeon_core/nodes/node_2_handoff.py b/songryeon_core/nodes/node_2_handoff.py index 8c6c09a..aac5bfc 100644 --- a/songryeon_core/nodes/node_2_handoff.py +++ b/songryeon_core/nodes/node_2_handoff.py @@ -1,15 +1,25 @@ from __future__ import annotations +import ast from dataclasses import asdict from songryeon_core.core.data_store import DataStore from songryeon_core.core.schemas import ( MetainfoBoundary, NodeMovement, + Node2AnswerBasisFrame, + Node2EvidenceRole, Node2HandoffFrame, + Node0DocumentMaterialItem, Node3BriefClaim, Node3BriefDocument, + Node3ExcludedDocumentContext, + Node3L3DocumentSummaryMaterial, + Node3MaterialDeliveryPolicyFrame, Node3MemorySelectionMaterial, + Node3RLoopResultMaterial, + Node3SourceCodeOutline, + Node3SourceCodeSymbol, Node3SelectedRecentMemoryContext, Node3InputBriefFrame, Node3BriefRuntimeTask, @@ -17,15 +27,18 @@ SelectedRecentMemoryContextItem, ZeroState, validate_node2_handoff_frame, + validate_node3_material_delivery_policy_frame, validate_node3_input_brief_frame, validate_selected_recent_memory_context_frame, ) from songryeon_core.core.trace_store import TraceStore from songryeon_core.loops.l_loop_namespace import LRunIds +from songryeon_core.tools.document_context_pack import latest_document_context_pack_payload NODE2_HANDOFF_FRAME_DATA_ID = "node_2:handoff_frame" NODE3_INPUT_BRIEF_FRAME_DATA_ID = "node_3:input_brief_frame" +NODE3_MATERIAL_DELIVERY_POLICY_FRAME_DATA_ID = "node_3:material_delivery_policy_frame" SELECTED_MEMORY_RAW_USER_TEXT_MAX_CHARS = 800 SELECTED_MEMORY_RAW_ASSISTANT_TEXT_MAX_CHARS = 1200 SELECTED_MEMORY_MAX_ITEMS = 3 @@ -254,10 +267,30 @@ def record_route2_handoff( data_store, id_namespace=id_namespace, ) - reportable_document_count = document_counts["reportable"] - raw_document_extract_record_count = document_counts["raw"] - empty_document_extract_record_count = document_counts["empty"] + reportable_document_count = int(document_counts["reportable"]) + raw_document_extract_record_count = int(document_counts["raw"]) + empty_document_extract_record_count = int(document_counts["empty"]) + code_counts = _code_extract_counts( + data_store, + id_namespace=id_namespace, + ) + reportable_code_file_count = int(code_counts["reportable"]) + raw_code_extract_record_count = int(code_counts["raw"]) + empty_code_extract_record_count = int(code_counts["empty"]) read_doc_count = reportable_document_count + document_context_pack_frame_id = document_counts["pack_frame_id"] + document_context_included_count = int(document_counts["pack_included"]) + document_context_excluded_count = int(document_counts["pack_excluded"]) + document_context_cutoff_reason = str(document_counts["pack_cutoff_reason"]) + document_material_frame_id, document_material_payload = _latest_document_material_packet( + data_store, + id_namespace=id_namespace, + ) + document_material_item_count = _int(document_material_payload, "item_count") + document_material_unread_candidate_count = _int( + document_material_payload, + "unread_candidate_count", + ) actual_l_run_count = _count_records_by_type(data_store, "node_output:L_loop_run_frame") blocked_reroute_count = _blocked_same_turn_l_reroute_request_count(data_store) controller_decisions = _same_turn_l_reroute_controller_decisions(data_store) @@ -292,7 +325,7 @@ def record_route2_handoff( insufficiency_reasons.append("missing_l3_preserved_frame") if not l3_achievement_present: insufficiency_reasons.append("missing_l3_achievement_frame") - if search_result_count < 1 and reportable_document_count < 1: + if search_result_count < 1 and reportable_document_count < 1 and reportable_code_file_count < 1: insufficiency_reasons.append("missing_l_evidence_result") has_blocker = any(reason.startswith("missing_") for reason in insufficiency_reasons[:3]) @@ -304,6 +337,8 @@ def record_route2_handoff( turn_outcome_id, memory_relevance_summary["frame_id"], selected_memory_context_summary["frame_id"], + document_context_pack_frame_id if isinstance(document_context_pack_frame_id, str) else None, + document_material_frame_id, *route_ids, *l_loop_output_ids, *_same_turn_l_reroute_controller_ids(data_store), @@ -328,7 +363,21 @@ def record_route2_handoff( reportable_document_count=reportable_document_count, raw_document_extract_record_count=raw_document_extract_record_count, empty_document_extract_record_count=empty_document_extract_record_count, + reportable_code_file_count=reportable_code_file_count, + raw_code_extract_record_count=raw_code_extract_record_count, + empty_code_extract_record_count=empty_code_extract_record_count, read_doc_count=read_doc_count, + document_context_pack_frame_id=( + document_context_pack_frame_id + if isinstance(document_context_pack_frame_id, str) and document_context_pack_frame_id + else None + ), + document_context_included_count=document_context_included_count, + document_context_excluded_count=document_context_excluded_count, + document_context_cutoff_reason=document_context_cutoff_reason, + document_material_packet_frame_id=document_material_frame_id, + document_material_item_count=document_material_item_count, + document_material_unread_candidate_count=document_material_unread_candidate_count, actual_l_run_count=actual_l_run_count, blocked_same_turn_l_reroute_request_count=blocked_reroute_count, same_turn_l_reroute_controller_decisions=controller_decisions, @@ -349,7 +398,7 @@ def record_route2_handoff( selected_recent_memory_context_info_class=str( selected_memory_context_summary["info_class"] ), - brief_available=reportable_document_count > 0, + brief_available=reportable_document_count > 0 or reportable_code_file_count > 0, insufficiency_reasons=insufficiency_reasons, source_trace_ids=[node2_input_trace_id], source_data_ids=source_data_ids, @@ -384,6 +433,7 @@ def record_node3_input_brief( boundary: MetainfoBoundary, input_trace_ids: list[str], source_data_ids: list[str], + answer_basis_frame: Node2AnswerBasisFrame | None = None, runtime_movements: list[NodeMovement] | None = None, assigned_model_by_node: dict[str, str] | None = None, id_namespace: LRunIds | None = None, @@ -396,10 +446,55 @@ def record_node3_input_brief( else NODE3_INPUT_BRIEF_FRAME_DATA_ID ) read_documents = _read_documents(data_store, id_namespace=id_namespace) - search_candidate_documents = _search_candidate_documents( + document_context_pack = latest_document_context_pack_payload( + data_store, + id_namespace=id_namespace, + ) + l_loop_return_summary_frame_id, l_loop_return_summary = _latest_l_loop_return_summary( data_store, id_namespace=id_namespace, ) + actual_tool_read_doc_documents = _actual_tool_read_doc_documents( + data_store=data_store, + l_loop_return_summary=l_loop_return_summary, + id_namespace=id_namespace, + ) + actual_tool_read_doc_count = _actual_tool_read_doc_count( + data_store=data_store, + l_loop_return_summary=l_loop_return_summary, + id_namespace=id_namespace, + document_names=actual_tool_read_doc_documents, + ) + actual_tool_read_code_file_paths = _actual_tool_read_code_file_paths( + data_store=data_store, + l_loop_return_summary=l_loop_return_summary, + id_namespace=id_namespace, + ) + actual_tool_read_code_file_count = len(actual_tool_read_code_file_paths) + supplied_source_code_context_count = _supplied_source_code_context_count( + data_store=data_store, + read_documents=read_documents, + ) + source_code_outlines = _source_code_outlines( + data_store=data_store, + read_documents=read_documents, + ) + excluded_document_contexts = _excluded_document_contexts(document_context_pack) + document_context_pack_frame_id = _text(document_context_pack, "frame_id", fallback="") + document_context_pack_status = _document_context_pack_status(document_context_pack) + accumulated_search_candidate_documents = _accumulated_search_candidate_documents( + data_store, + id_namespace=id_namespace, + ) + document_material_frame_id, document_material_payload = _latest_document_material_packet( + data_store, + id_namespace=id_namespace, + ) + document_material_items = _node3_document_material_items(document_material_payload) + final_search_candidate_documents = _final_search_candidate_documents( + document_material_items=document_material_items, + l_loop_return_summary=l_loop_return_summary, + ) memory_selection_material = _node3_memory_selection_material( data_store=data_store, handoff_frame_id=handoff_frame_id, @@ -408,10 +503,15 @@ def record_node3_input_brief( data_store=data_store, handoff_frame_id=handoff_frame_id, ) + l3_document_summaries = _node3_l3_document_summary_materials( + data_store=data_store, + id_namespace=id_namespace, + ) runtime_tasks = _runtime_tasks( runtime_movements or [], assigned_model_by_node or {}, ) + r_loop_result_material = _node3_r_loop_result_material(data_store) # 학습 메모: node_3 brief는 claim의 의미 등급을 다시 판단하지 않는다. # node_2 boundary가 붙인 relative/mixed 라벨을 사용자 답변 재료까지 그대로 운반한다. allowed_claims = [ @@ -442,12 +542,14 @@ def record_node3_input_brief( and memory_selection_material.selected_memory_count > 0 ) has_selected_memory_context = bool(selected_recent_memory_contexts) + has_l3_document_summary_material = bool(l3_document_summaries) if ( not read_documents and not allowed_claims and not runtime_tasks and not has_selected_memory_material and not has_selected_memory_context + and not has_l3_document_summary_material ): insufficiency_reasons.append("no_document_or_claim_material") @@ -462,6 +564,57 @@ def record_node3_input_brief( data_store=data_store, handoff_frame_id=handoff_frame_id, ) + answer_basis_frame_id = answer_basis_frame.frame_id if answer_basis_frame is not None else None + answer_basis_mode = ( + answer_basis_frame.answer_basis_mode + if answer_basis_frame is not None + else "mixed_or_uncertain" + ) + material_policy_frame_id = ( + id_namespace.scoped_data_id(NODE3_MATERIAL_DELIVERY_POLICY_FRAME_DATA_ID) + if id_namespace is not None + else NODE3_MATERIAL_DELIVERY_POLICY_FRAME_DATA_ID + ) + material_policy_source_data_ids = _unique_strings( + [ + handoff_frame_id, + answer_basis_frame_id, + document_context_pack_frame_id or None, + document_material_frame_id, + *[ + summary.source_data_id + for summary in l3_document_summaries + if summary.source_data_id + ], + ] + ) + material_policy_frame = build_node3_material_delivery_policy_frame( + frame_id=material_policy_frame_id, + turn_id=turn_id, + answer_basis_frame_id=answer_basis_frame_id, + answer_basis_mode=answer_basis_mode, + supplied_document_context_count=len(read_documents), + l3_document_summary_count=len(l3_document_summaries), + source_trace_ids=_unique_strings([*input_trace_ids, *runtime_trace_ids]), + source_data_ids=material_policy_source_data_ids, + ) + validate_node3_material_delivery_policy_frame(material_policy_frame) + material_policy_event = trace_store.create_event( + turn_id=turn_id, + actor="node_2", + event_type="node_output", + input_ref=material_policy_frame.source_trace_ids, + output_ref=[material_policy_frame_id], + schema_status="passed", + ) + data_store.create_record( + data_id=material_policy_frame_id, + data_type="node_output:node3_material_delivery_policy_frame", + exists=True, + created_at=material_policy_event.timestamp, + source_trace_id=material_policy_event.event_id, + payload=asdict(material_policy_frame), + ) frame = Node3InputBriefFrame( frame_id=brief_frame_id, turn_id=turn_id, @@ -469,21 +622,118 @@ def record_node3_input_brief( brief_status="ready" if not insufficiency_reasons else "insufficient", handoff_frame_id=handoff_frame_id, read_documents=read_documents, - search_candidate_count=len(search_candidate_documents), - search_candidate_documents=search_candidate_documents, + actual_tool_read_doc_count=actual_tool_read_doc_count, + actual_tool_read_doc_documents=actual_tool_read_doc_documents, + actual_tool_read_code_file_count=actual_tool_read_code_file_count, + actual_tool_read_code_file_paths=actual_tool_read_code_file_paths, + supplied_document_context_count=len(read_documents), + supplied_source_code_context_count=supplied_source_code_context_count, + source_code_outlines=source_code_outlines, + document_context_pack_frame_id=document_context_pack_frame_id or None, + document_context_pack_status=document_context_pack_status, + excluded_document_contexts=excluded_document_contexts, + document_material_packet_frame_id=document_material_frame_id, + document_material_items=document_material_items, + final_search_candidate_count=len(final_search_candidate_documents), + final_search_candidate_documents=final_search_candidate_documents, + accumulated_search_candidate_count=len(accumulated_search_candidate_documents), + accumulated_search_candidate_documents=accumulated_search_candidate_documents, + search_candidate_count=len(final_search_candidate_documents), + search_candidate_documents=final_search_candidate_documents, allowed_claims=allowed_claims, memory_selection_material=memory_selection_material, selected_recent_memory_contexts=selected_recent_memory_contexts, + l3_document_summaries=l3_document_summaries, + material_delivery_policy_frame_id=material_policy_frame_id, + material_delivery_mode=material_policy_frame.material_delivery_mode, + raw_document_policy=material_policy_frame.raw_document_policy, + l3_summary_policy=material_policy_frame.l3_summary_policy, + uncertainty_policy=material_policy_frame.uncertainty_policy, + material_policy_reason_code=material_policy_frame.policy_reason_code, + llm_raw_document_text_count=material_policy_frame.llm_raw_document_text_count, + llm_l3_summary_context_count=material_policy_frame.llm_l3_summary_context_count, + raw_context_replaced_by_summary_count=( + material_policy_frame.raw_context_replaced_by_summary_count + ), + material_policy_generated_by=material_policy_frame.generated_by, + material_policy_info_class=material_policy_frame.info_class, + material_policy_semantic_judgement_status=( + material_policy_frame.semantic_judgement_status + ), runtime_tasks=runtime_tasks, + r_loop_result_material=r_loop_result_material, + l_loop_return_summary_frame_id=l_loop_return_summary_frame_id, + l_loop_task_status=_text(l_loop_return_summary, "l_loop_task_status", fallback="not_recorded"), + l_loop_failure_level=_text(l_loop_return_summary, "failure_level", fallback="none"), + l3_goal_match_status=_text(l_loop_return_summary, "l3_goal_match_status", fallback="not_run"), + l3_semantic_goal_match_status=_text( + l_loop_return_summary, + "l3_semantic_goal_match_status", + fallback="not_run", + ), + remaining_query_attempts=_int(l_loop_return_summary, "remaining_query_attempts"), + remaining_read_doc_calls=_int(l_loop_return_summary, "remaining_read_doc_calls"), + l_loop_result_attitude_hint=_l_loop_result_attitude_hint(l_loop_return_summary), + answer_basis_frame_id=answer_basis_frame_id, + answer_basis_mode=answer_basis_mode, + basis_reason_codes=( + list(answer_basis_frame.basis_reason_codes) + if answer_basis_frame is not None + else ["llm_mode_selection_failed"] + ), + mode_selection_reason=( + answer_basis_frame.mode_selection_reason + if answer_basis_frame is not None + else "CODE_STATUS:node2_answer_basis_mode_selection_failed" + ), + mode_selection_reason_info_class=( + answer_basis_frame.mode_selection_reason_info_class + if answer_basis_frame is not None + else "absolute_status" + ), + evidence_roles=( + list(answer_basis_frame.evidence_roles) + if answer_basis_frame is not None + else [] + ), + answer_basis_generated_by=( + answer_basis_frame.generated_by if answer_basis_frame is not None else "CODE:FALLBACK" + ), + answer_basis_info_class=( + answer_basis_frame.info_class if answer_basis_frame is not None else "absolute_status" + ), + answer_basis_semantic_judgement_status=( + answer_basis_frame.semantic_judgement_status + if answer_basis_frame is not None + else "failed" + ), reporting_rules=[ "사용자 질문에 직접 답한다.", "문서 재료가 있으면 문서나 데이터가 없다고 말하지 않는다.", "답변 첫머리의 '근거 기준:' 블록은 code가 Node3InputBriefFrame의 절대 count로 고정 생성한다.", "node_3 LLM은 읽은 문서 수, 검색 후보 문서 수, 현재 턴 실행 순서 자료 수를 직접 쓰지 않고 본문만 작성한다.", + "실제 read_doc 도구 원문 읽기 수와 node_3 공급 문서 context 수는 다른 count다.", + "실제 read_code_file 도구 원문 읽기 수와 read_doc 도구 원문 읽기 수는 다른 count다.", + "사용자가 read_doc 수를 물으면 actual_tool_read_doc_count만 기준으로 답하고, supplied_document_context_count를 read_doc 수로 말하지 않는다.", + "사용자가 코드 파일 원문 읽기 여부를 물으면 actual_tool_read_code_file_count와 actual_tool_read_code_file_paths를 기준으로 답한다.", + "source_code_outlines는 read_code_file 원문에서 code가 뽑은 문법 장부이며, source 파일 기능 설명의 coverage checklist로 사용한다.", + "source_code_outlines의 함수명만 보고 의미를 단정하지 않고, supplied source text를 함께 근거로 설명한다.", + "최종 검색 후보와 L3 누적 검색 후보는 다른 count다.", "검색 후보 문서는 원문을 읽은 문서가 아니다. 검색 후보만 보고 읽은 문서처럼 말하지 않는다.", + "document_context_pack excluded 문서는 읽은 문서가 아니다. 예산 때문에 공급되지 않은 후보로만 언급한다.", + "document_material_packet의 문서별 role flag가 true인 역할만 해당 문서에 주장한다.", + "문서가 node_3 context로 공급됐더라도 was_actual_tool_read_doc=false이면 read_doc으로 읽었다고 말하지 않는다.", + "README, 요약, 실행기록은 사용자가 명시한 ORDER 원문을 대체한다고 말하지 않는다.", + "L3 문서별 요약은 L3 LLM이 만든 의미 요약 재료이며 원문이나 code fact를 대체하지 않는다.", + "L3 담백 문서 요약은 문서 하나에 직접 대응하는 relative 정보로만 취급한다.", + "L3 상황 맞춤 요약은 현재 질문/목표와 문서 원문 source bundle에 근거한 mixed 정보로만 취급한다.", + "material_delivery_policy가 l3_summary_replaces_raw_context 계열이면 node_3 LLM payload의 원문 text는 생략되고 L3 요약 재료를 기준으로 답한다.", + "원문 text가 payload에서 생략되어도 DataStore의 원문 document extract record는 삭제된 것이 아니다.", "현재 턴 실행 순서 자료가 있으면 실행 순서와 작업 장부 설명의 근거로 사용할 수 있다.", "기억 선택 결과가 있으면 LLM selector가 고른 mixed 판단으로만 취급하고, 선택된 과거 턴을 code 사실처럼 단정하지 않는다.", "선택된 최근 기억 context가 있으면 raw_user_text/raw_assistant_text에 복사된 범위에서만 이전 대화를 언급한다.", + "선택된 최근 기억 context는 과거 대화 원문을 code가 복사한 context이며, 실행기록 문서나 새로 읽은 문서가 아니다.", + "과거 대화 안에서 문서/실행기록이 언급되었더라도, 그것을 이번 턴에 직접 읽은 문서처럼 말하지 않는다.", "선택된 최근 기억 context의 관련성은 selector의 mixed 판단이며 code fact가 아니다.", "truncated=true인 최근 기억 context는 전체 이전 대화라고 단정하지 않는다.", "해석, 정의, 평가, 요약을 말할 때는 답변 안에 근거 기준을 짧게 밝히고, 문서/허용 주장/현재 턴 실행 순서 자료 중 무엇에 기대는지와 왜 그 근거로 말할 수 있는지 설명한다.", @@ -491,15 +741,38 @@ def record_node3_input_brief( "node_0/node_1/node_2/node_3 같은 내부 역할명은 자기정체성으로 쓰지 않고, 필요할 때 실행 경로 설명에만 제한적으로 쓴다.", "내부 추적용 raw ID를 답변 본문에 쓰지 않는다.", "문서 내용의 최종 진실성은 단정하지 않는다.", + "answer_basis_mode=absolute_first이면 코드/문서/trace/data로 확인 가능한 사실을 우선하고 추측을 줄인다.", + "answer_basis_mode=relative_allowed이면 해석/조언/비판/구상이 가능하되 절대정보처럼 단정하지 않는다.", + "answer_basis_mode=mixed_or_uncertain이면 출처 묶음, 부분 근거, 불확실성을 드러내고 부족한 근거를 지어내지 않는다.", + "R route 실험 결과가 있으면 graph memory skeleton 실행 상태로만 말하고, 완전한 장기기억 탐색 성공처럼 말하지 않는다.", + "R loop result material은 R return summary 장부에서 복사된 절대 상태이며, R1/R2/R3 의미 판단이 실행됐다는 뜻이 아니다.", ], insufficiency_reasons=insufficiency_reasons, source_trace_ids=_unique_strings([*input_trace_ids, *runtime_trace_ids]), source_data_ids=_unique_strings( [ handoff_frame_id, + material_policy_frame_id, *source_data_ids, + answer_basis_frame_id, memory_selection_source_data_id, selected_memory_context_source_data_id, + document_context_pack_frame_id or None, + document_material_frame_id, + l_loop_return_summary_frame_id, + *[ + outline.source_data_id + for outline in source_code_outlines + if outline.source_data_id + ], + *[ + summary.source_data_id + for summary in l3_document_summaries + if summary.source_data_id + ], + r_loop_result_material.source_data_id + if r_loop_result_material is not None + else None, *runtime_data_ids, ] ), @@ -509,7 +782,7 @@ def record_node3_input_brief( turn_id=turn_id, actor="node_2", event_type="node_output", - input_ref=frame.source_trace_ids, + input_ref=_unique_strings([*frame.source_trace_ids, material_policy_event.event_id]), output_ref=[brief_frame_id], schema_status="passed" if frame.brief_status == "ready" else "failed", ) @@ -524,23 +797,348 @@ def record_node3_input_brief( return event.event_id, brief_frame_id, frame +def build_node3_material_delivery_policy_frame( + *, + frame_id: str, + turn_id: str, + answer_basis_frame_id: str | None, + answer_basis_mode: str, + supplied_document_context_count: int, + l3_document_summary_count: int, + source_trace_ids: list[str], + source_data_ids: list[str], +) -> Node3MaterialDeliveryPolicyFrame: + """answer_basis_mode와 summary availability를 고정 정책표로 변환한다.""" + + ( + material_delivery_mode, + raw_document_policy, + l3_summary_policy, + uncertainty_policy, + policy_reason_code, + llm_raw_document_text_count, + llm_l3_summary_context_count, + raw_context_replaced_by_summary_count, + ) = _node3_material_delivery_policy_values( + answer_basis_mode=answer_basis_mode, + supplied_document_context_count=supplied_document_context_count, + l3_document_summary_count=l3_document_summary_count, + ) + return Node3MaterialDeliveryPolicyFrame( + frame_id=frame_id, + turn_id=turn_id, + answer_basis_frame_id=answer_basis_frame_id, + answer_basis_mode=answer_basis_mode, + material_delivery_mode=material_delivery_mode, + raw_document_policy=raw_document_policy, + l3_summary_policy=l3_summary_policy, + uncertainty_policy=uncertainty_policy, + policy_reason_code=policy_reason_code, + supplied_document_context_count=supplied_document_context_count, + l3_document_summary_count=l3_document_summary_count, + llm_raw_document_text_count=llm_raw_document_text_count, + llm_l3_summary_context_count=llm_l3_summary_context_count, + raw_context_replaced_by_summary_count=raw_context_replaced_by_summary_count, + source_trace_ids=_unique_strings(source_trace_ids), + source_data_ids=_unique_strings(source_data_ids), + ) + + +def _node3_material_delivery_policy_values( + *, + answer_basis_mode: str, + supplied_document_context_count: int, + l3_document_summary_count: int, +) -> tuple[str, str, str, str, str, int, int, int]: + """ORDER_132의 명시 정책표. 의미 판단 없이 모드와 summary 존재 여부만 본다.""" + + if answer_basis_mode == "absolute_first": + return ( + "raw_document_primary", + "preserve_supplied_raw_context", + "auxiliary_only", + "do_not_replace_raw_with_summary", + "absolute_first_requires_checkable_material", + supplied_document_context_count, + l3_document_summary_count, + 0, + ) + if l3_document_summary_count < 1: + return ( + "raw_document_fallback_no_l3_summary", + "preserve_raw_context_because_l3_summary_missing", + "unavailable", + "expose_summary_absence", + "l3_summary_unavailable_cannot_replace_raw_context", + supplied_document_context_count, + 0, + 0, + ) + if answer_basis_mode == "relative_allowed": + return ( + "l3_summary_replaces_raw_context", + "omit_raw_text_from_llm_payload", + "replace_raw_context_with_labeled_l3_summary", + "keep_summary_boundary_visible", + "relative_allowed_uses_l3_summary_to_reduce_context_volume", + 0, + l3_document_summary_count, + supplied_document_context_count, + ) + return ( + "l3_summary_replaces_raw_context_with_uncertainty", + "omit_raw_text_from_llm_payload", + "replace_raw_context_with_labeled_l3_summary_and_limits", + "surface_partial_or_bundle_based_grounding", + "mixed_or_uncertain_uses_l3_summary_with_limit_visibility", + 0, + l3_document_summary_count, + supplied_document_context_count, + ) + + +def _node3_r_loop_result_material( + data_store: DataStore, +) -> Node3RLoopResultMaterial | None: + data_id, payload = _latest_r_loop_return_summary(data_store) + if not data_id: + return None + return Node3RLoopResultMaterial( + source_data_id=data_id, + r_loop_task_status=_text(payload, "r_loop_task_status", fallback="not_run"), + continuation_status=_text(payload, "continuation_status", fallback="not_run"), + budget_status=_text(payload, "budget_status", fallback="not_run"), + final_information_granularity=_text( + payload, + "final_information_granularity", + fallback="unknown", + ), + summary_depth_used=_int(payload, "summary_depth_used"), + selected_entry_node_count=len( + _string_list(payload.get("selected_entry_node_ids")) + ), + inspected_graph_node_count=len( + _string_list(payload.get("inspected_graph_node_ids")) + ), + source_graph_node_count=len(_string_list(payload.get("source_graph_node_ids"))), + generated_by=_text(payload, "generated_by", fallback="unknown"), + info_class=_text(payload, "info_class", fallback="absolute"), + semantic_judgement_status=_text( + payload, + "semantic_judgement_status", + fallback="not_run", + ), + attitude_hint=_r_loop_result_attitude_hint(payload), + ) + + +def _latest_r_loop_return_summary(data_store: DataStore) -> tuple[str | None, dict[str, object]]: + for record in reversed(data_store.list_records()): + if record.data_type != "node_output:R_loop_return_summary_frame": + continue + if not isinstance(record.payload, dict): + continue + return record.data_id, record.payload + return None, {} + + +def _r_loop_result_attitude_hint(payload: dict[str, object]) -> str: + if not payload: + return "not_recorded" + task_status = _text(payload, "r_loop_task_status", fallback="not_run") + continuation_status = _text(payload, "continuation_status", fallback="not_run") + budget_status = _text(payload, "budget_status", fallback="not_run") + if task_status == "sufficient": + return "r_loop_sufficient" + if task_status == "failed": + return "r_loop_failed" + if budget_status == "exhausted" or continuation_status == "stop_budget_exhausted": + return "r_loop_budget_exhausted" + if task_status == "partial": + return "r_loop_partial_or_skeleton_only" + return "not_recorded" + + def node3_brief_llm_payload(frame: Node3InputBriefFrame) -> dict[str, object]: """내부 ID를 제거한 node_3 LLM용 payload를 만든다.""" + raw_document_payloads = _node3_raw_document_payloads(frame) + omitted_raw_document_payloads = _node3_omitted_raw_document_payloads(frame) return { "user_question": frame.user_question, "brief_status": frame.brief_status, - "available_document_extract_count": len(frame.read_documents), - "available_search_candidate_document_count": frame.search_candidate_count, - "search_candidate_documents": list(frame.search_candidate_documents), - "read_documents": [ + "actual_tool_read_doc": { + "count": frame.actual_tool_read_doc_count, + "document_names": list(frame.actual_tool_read_doc_documents), + }, + "actual_tool_read_code_file": { + "count": frame.actual_tool_read_code_file_count, + "file_paths": list(frame.actual_tool_read_code_file_paths), + "boundary": "This counts successful read_code_file tool outputs only. It is separate from read_doc.", + }, + "supplied_document_context": { + "count": frame.supplied_document_context_count, + "source_code_context_count": frame.supplied_source_code_context_count, + "raw_text_payload_count": frame.llm_raw_document_text_count, + "raw_text_replaced_by_l3_summary_count": frame.raw_context_replaced_by_summary_count, + "source_kind": ( + "document_context_pack" + if frame.document_context_pack_frame_id + else "tool_document_extract_records" + ), + "boundary": ( + "count is the preserved Node3InputBriefFrame document context count. " + "source_code_context_count is the subset copied from read_code_file outputs. " + "raw_text_payload_count is the number of full raw document texts actually included " + "in this LLM payload. These are not the same count as actual read_doc tool calls." + ), + }, + "source_code_outlines": { + "count": len(frame.source_code_outlines), + "boundary": ( + "Code-built syntax inventory from successful read_code_file outputs. " + "It lists top-level names and line numbers only; it is not a semantic summary. " + "Use public_function_names as a coverage checklist when explaining a source file." + ), + "items": _node3_source_code_outline_payloads(frame), + }, + "material_delivery_policy": _node3_material_delivery_policy_llm_payload(frame), + "document_material_packet": { + "status": "present" if frame.document_material_packet_frame_id else "not_recorded", + "item_count": len(frame.document_material_items), + "unread_candidate_count": sum( + 1 for item in frame.document_material_items if item.was_unread_candidate + ), + "boundary": ( + "This is a code-built document material ledger. " + "It records roles such as search candidate, actual tool read, " + "supplied context, excluded context, and unread candidate. " + "It is not a semantic summary." + ), + "items": [ + { + "document_name": item.document_name, + "source_roles": list(item.source_roles), + "was_search_candidate": item.was_search_candidate, + "was_actual_tool_read_doc": item.was_actual_tool_read_doc, + "was_supplied_document_context": item.was_supplied_document_context, + "was_excluded_document_context": item.was_excluded_document_context, + "was_unread_candidate": item.was_unread_candidate, + "search_candidate_rank": item.search_candidate_rank, + "actual_read_rank": item.actual_read_rank, + "supplied_context_rank": item.supplied_context_rank, + "excluded_context_rank": item.excluded_context_rank, + "char_count": item.char_count, + } + for item in frame.document_material_items + ], + }, + "document_evidence_role_boundaries": { + "boundary": ( + "For each document, claim only roles whose corresponding role flag is true. " + "A supplied context document is usable as context, but it is not an actual read_doc " + "tool read unless it appears in actual_tool_read_doc_document_names." + ), + "actual_tool_read_doc_document_names": list(frame.actual_tool_read_doc_documents), + "actual_tool_read_code_file_paths": list(frame.actual_tool_read_code_file_paths), + "supplied_context_document_names": [ + item.document_name + for item in frame.document_material_items + if item.was_supplied_document_context + ], + "search_candidate_document_names": [ + item.document_name + for item in frame.document_material_items + if item.was_search_candidate + ], + "excluded_context_document_names": [ + item.document_name + for item in frame.document_material_items + if item.was_excluded_document_context + ], + "unread_candidate_document_names": [ + item.document_name + for item in frame.document_material_items + if item.was_unread_candidate + ], + "supplied_but_not_actual_read_doc_document_names": [ + item.document_name + for item in frame.document_material_items + if item.was_supplied_document_context and not item.was_actual_tool_read_doc + ], + }, + "available_document_extract_count": frame.supplied_document_context_count, + "available_raw_document_text_count": frame.llm_raw_document_text_count, + "available_search_candidate_document_count": frame.final_search_candidate_count, + "search_candidate_scope": { + "legacy_search_candidate_count_is": "final_search_candidate_count", + "final_search_candidate": { + "count": frame.final_search_candidate_count, + "document_names": list(frame.final_search_candidate_documents), + "source_scope": "latest_l3_return_summary_or_node0_material_packet", + }, + "accumulated_search_candidate": { + "count": frame.accumulated_search_candidate_count, + "document_names": list(frame.accumulated_search_candidate_documents), + "source_scope": "all_l3_preserved_info_frames_in_namespace", + }, + "boundary": ( + "Final search candidates and accumulated preserved candidates are different scopes. " + "Use final_search_candidate for current grounding counts, and use accumulated_search_candidate " + "only when explaining L3 revision/search churn." + ), + }, + "search_candidate_documents": list(frame.final_search_candidate_documents), + "document_context_pack_status": frame.document_context_pack_status, + "excluded_document_contexts": [ { "document_name": document.document_name, "char_count": document.char_count, - "text": document.text, + "selection_basis": document.selection_basis, + "exclusion_reason": document.exclusion_reason, + "would_exceed_budget": document.would_exceed_budget, } - for document in frame.read_documents + for document in frame.excluded_document_contexts + ], + "supplied_document_contexts": [ + dict(document) + for document in raw_document_payloads ], + "omitted_supplied_document_contexts": omitted_raw_document_payloads, + "read_documents": [ + {**document, "legacy_alias": "supplied_document_context"} + for document in raw_document_payloads + ], + "l3_document_summaries": { + "count": len(frame.l3_document_summaries), + "boundary": ( + "These are L3-generated semantic summaries of individual document extracts. " + "plain_document_summary is relative/direct_record/one_document_to_one_summary. " + "task_relevant_summary is mixed/source_bundle/one_document_plus_task_context. " + "They do not replace the original supplied document text." + ), + "items": [ + { + "document_name": summary.document_name, + "source_char_count": summary.source_char_count, + "summary_status": summary.summary_status, + "plain_document_summary": summary.plain_document_summary, + "plain_summary_info_class": summary.plain_summary_info_class, + "plain_summary_source_mode": summary.plain_summary_source_mode, + "plain_summary_claim_alignment": summary.plain_summary_claim_alignment, + "task_relevant_summary": summary.task_relevant_summary, + "task_relevant_summary_info_class": summary.task_relevant_summary_info_class, + "task_relevant_summary_source_mode": summary.task_relevant_summary_source_mode, + "task_relevant_summary_claim_alignment": ( + summary.task_relevant_summary_claim_alignment + ), + "summary_limit_note": summary.summary_limit_note, + "generated_by": summary.generated_by, + "semantic_judgement_status": summary.semantic_judgement_status, + } + for summary in frame.l3_document_summaries + ], + }, "allowed_claims": [ { "kind": claim.kind, @@ -572,10 +1170,40 @@ def node3_brief_llm_payload(frame: Node3InputBriefFrame) -> dict[str, object]: for context in frame.selected_recent_memory_contexts ], "available_runtime_task_count": len(frame.runtime_tasks), + "answer_basis": { + "answer_basis_mode": frame.answer_basis_mode, + "basis_reason_codes": list(frame.basis_reason_codes), + "mode_selection_reason": frame.mode_selection_reason, + "mode_selection_reason_info_class": frame.mode_selection_reason_info_class, + "generated_by": frame.answer_basis_generated_by, + "info_class": frame.answer_basis_info_class, + "semantic_judgement_status": frame.answer_basis_semantic_judgement_status, + "evidence_roles": [ + _node3_evidence_role_llm_payload(role) + for role in frame.evidence_roles + ], + }, + "l_loop_result": { + "task_status": frame.l_loop_task_status, + "failure_level": frame.l_loop_failure_level, + "l3_goal_match_status": frame.l3_goal_match_status, + "l3_semantic_goal_match_status": frame.l3_semantic_goal_match_status, + "remaining_query_attempts": frame.remaining_query_attempts, + "remaining_read_doc_calls": frame.remaining_read_doc_calls, + "attitude_hint": frame.l_loop_result_attitude_hint, + }, + "r_loop_result": _node3_r_loop_result_llm_payload( + frame.r_loop_result_material + ), "runtime_task_sequence_note": ( "This sequence is captured before node_3 report generation. " "Later node_3 reporting and node_4 gatekeeping tasks may appear in the final runtime ledger." ), + "selected_recent_memory_source_boundary": ( + "Selected recent memory context is copied previous conversation text. " + "It is not a read document, not an execution-record document, and not newly read evidence. " + "Even if that conversation text mentions a document or execution record, do not describe it as a document read in this turn." + ), "reporter_identity_boundary": ( "Speak from SongRyeon's final-report perspective, not as node_0, node_1, node_2, node_3, " "or any other internal implementation role." @@ -597,6 +1225,154 @@ def node3_brief_llm_payload(frame: Node3InputBriefFrame) -> dict[str, object]: } +def _node3_r_loop_result_llm_payload( + material: Node3RLoopResultMaterial | None, +) -> dict[str, object]: + if material is None: + return { + "status": "not_recorded", + "boundary": "No R route result material was supplied to node_3.", + } + return { + "status": "present", + "task_status": material.r_loop_task_status, + "continuation_status": material.continuation_status, + "budget_status": material.budget_status, + "final_information_granularity": material.final_information_granularity, + "summary_depth_used": material.summary_depth_used, + "selected_entry_node_count": material.selected_entry_node_count, + "inspected_graph_node_count": material.inspected_graph_node_count, + "source_graph_node_count": material.source_graph_node_count, + "generated_by": material.generated_by, + "info_class": material.info_class, + "semantic_judgement_status": material.semantic_judgement_status, + "attitude_hint": material.attitude_hint, + "boundary": ( + "This is a code-copied R return summary ledger. " + "It reports that an experimental R skeleton ran; it does not prove that " + "R graph traversal fully answered the user goal." + ), + } + + +def _node3_material_delivery_policy_llm_payload( + frame: Node3InputBriefFrame, +) -> dict[str, object]: + return { + "material_delivery_mode": frame.material_delivery_mode, + "raw_document_policy": frame.raw_document_policy, + "l3_summary_policy": frame.l3_summary_policy, + "uncertainty_policy": frame.uncertainty_policy, + "policy_reason_code": frame.material_policy_reason_code, + "generated_by": frame.material_policy_generated_by, + "info_class": frame.material_policy_info_class, + "semantic_judgement_status": frame.material_policy_semantic_judgement_status, + "supplied_document_context_count": frame.supplied_document_context_count, + "l3_document_summary_count": len(frame.l3_document_summaries), + "llm_raw_document_text_count": frame.llm_raw_document_text_count, + "llm_l3_summary_context_count": frame.llm_l3_summary_context_count, + "raw_context_replaced_by_summary_count": frame.raw_context_replaced_by_summary_count, + "boundary": ( + "This is a CODE policy mapping from answer_basis_mode and L3 summary availability. " + "It is not a semantic judgement about document importance. " + "When raw_document_policy=omit_raw_text_from_llm_payload, original document records remain in DataStore " + "but full raw text is omitted from this node_3 LLM payload." + ), + } + + +def _node3_raw_document_payloads( + frame: Node3InputBriefFrame, +) -> list[dict[str, object]]: + if frame.raw_document_policy == "omit_raw_text_from_llm_payload": + return [] + return [ + { + "document_name": document.document_name, + "char_count": document.char_count, + "text": document.text, + "text_payload_status": "included", + } + for document in frame.read_documents + ] + + +def _node3_omitted_raw_document_payloads( + frame: Node3InputBriefFrame, +) -> list[dict[str, object]]: + if frame.raw_document_policy != "omit_raw_text_from_llm_payload": + return [] + return [ + { + "document_name": document.document_name, + "char_count": document.char_count, + "text_payload_status": "omitted_by_material_delivery_policy", + "replacement_material": "l3_document_summaries", + } + for document in frame.read_documents + ] + + +def _node3_source_code_outline_payloads( + frame: Node3InputBriefFrame, +) -> list[dict[str, object]]: + payloads: list[dict[str, object]] = [] + for outline in frame.source_code_outlines: + payloads.append( + { + "file_path": outline.file_path, + "language": outline.language, + "parse_status": outline.parse_status, + "top_level_symbol_count": outline.top_level_symbol_count, + "public_symbol_count": outline.public_symbol_count, + "public_function_names": list(outline.public_function_names), + "top_level_symbols": [ + { + "name": symbol.name, + "symbol_kind": symbol.symbol_kind, + "line_number": symbol.line_number, + "is_public": symbol.is_public, + "docstring_present": symbol.docstring_present, + } + for symbol in outline.top_level_symbols + ], + "parse_error_type": outline.parse_error_type, + } + ) + return payloads + + +def _node3_evidence_role_llm_payload(role: Node2EvidenceRole) -> dict[str, object]: + return { + "source_label": _safe_evidence_source_label(role.source_data_id), + "evidence_role": role.evidence_role, + "role_reason": role.role_reason, + "role_reason_info_class": role.role_reason_info_class, + } + + +def _safe_evidence_source_label(source_data_id: str) -> str: + if "selected_recent_memory_context" in source_data_id: + return "선택된 최근 기억" + if "memory_relevance" in source_data_id: + return "최근 기억 선택 판단" + if "read_doc" in source_data_id or "read_artifact" in source_data_id: + return "읽은 문서" + if "document_context_pack" in source_data_id: + return "문서 context pack" + if "L3" in source_data_id or "preserved" in source_data_id or "achievement" in source_data_id: + return "L loop 결과" + if "route" in source_data_id: + return "라우팅 결과" + if "boundary" in source_data_id: + return "메타정보 경계" + if "handoff" in source_data_id: + return "node_2 handoff" + if "node2_input" in source_data_id: + return "node_2 입력 프레임" + return "공급된 근거 자료" + + def _memory_relevance_handoff_summary( *, data_store: DataStore, @@ -835,11 +1611,101 @@ def _node3_memory_selection_llm_payload( } +def _node3_l3_document_summary_materials( + *, + data_store: DataStore, + id_namespace: LRunIds | None, +) -> list[Node3L3DocumentSummaryMaterial]: + materials: list[Node3L3DocumentSummaryMaterial] = [] + for record in data_store.list_records(): + if not _record_in_namespace(record.data_id, id_namespace=id_namespace): + continue + if record.data_type != "node_output:L3_per_document_summary_frame": + continue + if not isinstance(record.payload, dict): + continue + payload = record.payload + document_name = _text(payload, "source_document_name", fallback="") + source_data_id = _text(payload, "frame_id", fallback=record.data_id) + summary_status = _text(payload, "summary_status", fallback="") + generated_by = _text(payload, "generated_by", fallback="") + semantic_status = _text(payload, "semantic_judgement_status", fallback="") + if not document_name or not summary_status or not generated_by or not semantic_status: + continue + materials.append( + Node3L3DocumentSummaryMaterial( + document_name=document_name, + source_char_count=_int(payload, "source_char_count"), + summary_status=summary_status, + plain_document_summary=_text( + payload, + "plain_document_summary", + fallback="", + ), + plain_summary_info_class=_text( + payload, + "plain_summary_info_class", + fallback="relative", + ), + plain_summary_source_mode=_text( + payload, + "plain_summary_source_mode", + fallback="direct_record", + ), + plain_summary_claim_alignment=_text( + payload, + "plain_summary_claim_alignment", + fallback="one_document_to_one_summary", + ), + task_relevant_summary=_text( + payload, + "task_relevant_summary", + fallback="", + ), + task_relevant_summary_info_class=_text( + payload, + "task_relevant_summary_info_class", + fallback="mixed", + ), + task_relevant_summary_source_mode=_text( + payload, + "task_relevant_summary_source_mode", + fallback="source_bundle", + ), + task_relevant_summary_claim_alignment=_text( + payload, + "task_relevant_summary_claim_alignment", + fallback="one_document_plus_task_context", + ), + summary_limit_note=_text(payload, "summary_limit_note", fallback=""), + generated_by=generated_by, + semantic_judgement_status=semantic_status, + source_data_id=source_data_id, + ) + ) + return materials + + def _read_documents( data_store: DataStore, *, id_namespace: LRunIds | None, ) -> list[Node3BriefDocument]: + pack_payload = latest_document_context_pack_payload( + data_store, + id_namespace=id_namespace, + ) + if pack_payload: + documents = _packed_read_documents(pack_payload) + documents.extend( + _read_code_documents( + data_store, + id_namespace=id_namespace, + existing_source_data_ids={document.source_data_id for document in documents}, + ) + ) + return documents + documents: list[Node3BriefDocument] = [] for record in data_store.list_records(): if not _record_in_namespace(record.data_id, id_namespace=id_namespace): @@ -861,17 +1727,176 @@ def _read_documents( source_data_id=record.data_id, ) ) + documents.extend( + _read_code_documents( + data_store, + id_namespace=id_namespace, + existing_source_data_ids={document.source_data_id for document in documents}, + ) + ) + return documents + + +def _read_code_documents( + data_store: DataStore, + *, + id_namespace: LRunIds | None, + existing_source_data_ids: set[str], +) -> list[Node3BriefDocument]: + """read_code_file 결과를 node_3가 읽을 수 있는 source-code context로 복사한다.""" + + documents: list[Node3BriefDocument] = [] + for record in data_store.list_records(): + if record.data_id in existing_source_data_ids: + continue + if not _record_in_namespace(record.data_id, id_namespace=id_namespace): + continue + if not _is_code_extract_record(record.data_type): + continue + if not isinstance(record.payload, dict): + continue + if record.payload.get("read_status") != "ok": + continue + text = record.payload.get("text") + if not isinstance(text, str) or not text.strip(): + continue + file_path = record.payload.get("file_path") + char_count = record.payload.get("char_count") + documents.append( + Node3BriefDocument( + document_name=file_path if isinstance(file_path, str) and file_path else "읽은 코드 파일", + char_count=char_count if isinstance(char_count, int) else len(text), + text=text, + source_data_id=record.data_id, + ) + ) return documents -def _search_candidate_documents( +def _packed_read_documents(pack_payload: dict[str, object]) -> list[Node3BriefDocument]: + included_documents = pack_payload.get("included_documents") + if not isinstance(included_documents, list): + return [] + documents: list[Node3BriefDocument] = [] + pack_frame_id = _text(pack_payload, "frame_id", fallback="") + for item in included_documents: + if not isinstance(item, dict): + continue + text = item.get("text") + if not isinstance(text, str) or not text: + continue + char_count = item.get("char_count") + documents.append( + Node3BriefDocument( + document_name=_document_name(item.get("doc_id")), + char_count=char_count if isinstance(char_count, int) else len(text), + text=text, + source_data_id=pack_frame_id, + ) + ) + return documents + + +def _excluded_document_contexts( + pack_payload: dict[str, object], +) -> list[Node3ExcludedDocumentContext]: + excluded_documents = pack_payload.get("excluded_documents") + if not isinstance(excluded_documents, list): + return [] + pack_frame_id = _text(pack_payload, "frame_id", fallback="") + contexts: list[Node3ExcludedDocumentContext] = [] + for item in excluded_documents: + if not isinstance(item, dict): + continue + char_count = item.get("char_count") + contexts.append( + Node3ExcludedDocumentContext( + document_name=_document_name(item.get("doc_id")), + char_count=char_count if isinstance(char_count, int) else 0, + selection_basis=_text(item, "selection_basis", fallback="unknown"), + exclusion_reason=_text(item, "exclusion_reason", fallback="excluded_due_to_context_budget"), + would_exceed_budget=bool(item.get("would_exceed_budget")), + source_data_id=pack_frame_id, + ) + ) + return contexts + + +def _node3_document_material_items( + material_payload: dict[str, object], +) -> list[Node0DocumentMaterialItem]: + raw_items = material_payload.get("items") + if not isinstance(raw_items, list): + return [] + items: list[Node0DocumentMaterialItem] = [] + for raw_item in raw_items: + if not isinstance(raw_item, dict): + continue + doc_id = _text(raw_item, "doc_id", fallback="") + document_name = _text(raw_item, "document_name", fallback="") + if not doc_id or not document_name: + continue + items.append( + Node0DocumentMaterialItem( + doc_id=doc_id, + document_name=document_name, + source_roles=_string_list(raw_item.get("source_roles")), + was_search_candidate=bool(raw_item.get("was_search_candidate")), + was_actual_tool_read_doc=bool(raw_item.get("was_actual_tool_read_doc")), + was_supplied_document_context=bool( + raw_item.get("was_supplied_document_context") + ), + was_excluded_document_context=bool( + raw_item.get("was_excluded_document_context") + ), + was_unread_candidate=bool(raw_item.get("was_unread_candidate")), + search_candidate_rank=_int(raw_item, "search_candidate_rank"), + actual_read_rank=_int(raw_item, "actual_read_rank"), + supplied_context_rank=_int(raw_item, "supplied_context_rank"), + excluded_context_rank=_int(raw_item, "excluded_context_rank"), + char_count=_int(raw_item, "char_count"), + source_trace_ids=_string_list(raw_item.get("source_trace_ids")), + source_data_ids=_string_list(raw_item.get("source_data_ids")), + ) + ) + return items + + +def _document_context_pack_status(pack_payload: dict[str, object]) -> str: + if not pack_payload: + return "not_recorded" + included = pack_payload.get("included_document_count") + excluded = pack_payload.get("excluded_document_count") + if included == 0 and excluded == 0: + return "no_candidates" + return "packed" + + +def _final_search_candidate_documents( + *, + document_material_items: list[Node0DocumentMaterialItem], + l_loop_return_summary: dict[str, object], +) -> list[str]: + """최신 L return/material packet 기준 search candidate 문서를 doc_id identity로 표시한다.""" + + doc_ids = [ + item.doc_id + for item in document_material_items + if item.was_search_candidate and item.doc_id + ] + if not doc_ids: + doc_ids = _string_list(l_loop_return_summary.get("search_result_doc_ids")) + return _read_doc_display_labels(_unique_strings(doc_ids)) + + +def _accumulated_search_candidate_documents( data_store: DataStore, *, id_namespace: LRunIds | None, ) -> list[str]: - """L3가 보존한 검색 후보 문서명을 node_3용 안전한 형태로 모은다.""" + """L3 preserved frame 전체의 검색 후보를 doc_id identity 기준으로 모은다.""" - names: list[str] = [] + doc_ids: list[str] = [] for record in data_store.list_records(): if not _record_in_namespace(record.data_id, id_namespace=id_namespace): continue @@ -887,8 +1912,8 @@ def _search_candidate_documents( continue doc_id = candidate.get("doc_id") if isinstance(doc_id, str) and doc_id.strip(): - names.append(_document_name(doc_id)) - return _unique_strings(names) + doc_ids.append(doc_id.strip()) + return _read_doc_display_labels(_unique_strings(doc_ids)) def _runtime_tasks( @@ -957,7 +1982,7 @@ def _document_extract_counts( data_store: DataStore, *, id_namespace: LRunIds | None, -) -> dict[str, int]: +) -> dict[str, object]: raw_count = 0 reportable_count = 0 empty_count = 0 @@ -973,6 +1998,47 @@ def _document_extract_counts( reportable_count += 1 else: empty_count += 1 + pack_payload = latest_document_context_pack_payload( + data_store, + id_namespace=id_namespace, + ) + pack_frame_id = _text(pack_payload, "frame_id", fallback="") + pack_included = _int(pack_payload, "included_document_count") + pack_excluded = _int(pack_payload, "excluded_document_count") + pack_cutoff_reason = _text(pack_payload, "cutoff_reason", fallback="") + if pack_frame_id: + reportable_count = pack_included + return { + "raw": raw_count, + "reportable": reportable_count, + "empty": empty_count, + "pack_frame_id": pack_frame_id, + "pack_included": pack_included, + "pack_excluded": pack_excluded, + "pack_cutoff_reason": pack_cutoff_reason, + } + + +def _code_extract_counts( + data_store: DataStore, + *, + id_namespace: LRunIds | None, +) -> dict[str, int]: + raw_count = 0 + reportable_count = 0 + empty_count = 0 + for record in data_store.list_records(): + if not _record_in_namespace(record.data_id, id_namespace=id_namespace): + continue + if not _is_code_extract_record(record.data_type): + continue + raw_count += 1 + payload = record.payload if isinstance(record.payload, dict) else {} + text = payload.get("text") if isinstance(payload, dict) else None + if payload.get("read_status") == "ok" and isinstance(text, str) and text.strip(): + reportable_count += 1 + else: + empty_count += 1 return { "raw": raw_count, "reportable": reportable_count, @@ -1014,6 +2080,299 @@ def _same_turn_l_reroute_controller_payloads(data_store: DataStore) -> list[dict return frames +def _latest_l_loop_return_summary( + data_store: DataStore, + *, + id_namespace: LRunIds | None, +) -> tuple[str | None, dict[str, object]]: + latest_id: str | None = None + latest_payload: dict[str, object] = {} + for record in data_store.list_records(): + if record.data_type != "node_output:l_loop_return_summary_frame": + continue + if not _record_in_namespace(record.data_id, id_namespace=id_namespace): + continue + if isinstance(record.payload, dict): + latest_id = record.data_id + latest_payload = record.payload + return latest_id, latest_payload + + +def _latest_document_material_packet( + data_store: DataStore, + *, + id_namespace: LRunIds | None, +) -> tuple[str | None, dict[str, object]]: + latest_id: str | None = None + latest_payload: dict[str, object] = {} + for record in data_store.list_records(): + if record.data_type != "node_output:node0_document_material_packet_frame": + continue + if not _record_in_namespace(record.data_id, id_namespace=id_namespace): + continue + if isinstance(record.payload, dict): + latest_id = record.data_id + latest_payload = record.payload + return latest_id, latest_payload + + +def _actual_tool_read_doc_documents( + *, + data_store: DataStore, + l_loop_return_summary: dict[str, object], + id_namespace: LRunIds | None, +) -> list[str]: + doc_ids: list[str] = [] + for doc_id in _string_list(l_loop_return_summary.get("read_doc_ids")): + doc_ids.append(doc_id) + + for record in data_store.list_records(): + if not _record_in_namespace(record.data_id, id_namespace=id_namespace): + continue + if not _is_document_extract_record(record.data_type): + continue + if not isinstance(record.payload, dict): + continue + text = record.payload.get("text") + if not isinstance(text, str) or not text.strip(): + continue + doc_id = record.payload.get("doc_id") + if isinstance(doc_id, str) and doc_id.strip(): + doc_ids.append(doc_id.strip()) + else: + doc_ids.append(record.data_id) + return _read_doc_display_labels(_unique_strings(doc_ids)) + + +def _actual_tool_read_doc_count( + *, + data_store: DataStore, + l_loop_return_summary: dict[str, object], + id_namespace: LRunIds | None, + document_names: list[str], +) -> int: + if document_names: + return len(document_names) + summary_count = l_loop_return_summary.get("actual_read_doc_count") + if isinstance(summary_count, int) and summary_count >= 0: + return summary_count + return sum( + 1 + for record in data_store.list_records() + if _record_in_namespace(record.data_id, id_namespace=id_namespace) + and _is_document_extract_record(record.data_type) + and isinstance(record.payload, dict) + and isinstance(record.payload.get("text"), str) + and bool(str(record.payload.get("text") or "").strip()) + ) + + +def _actual_tool_read_code_file_paths( + *, + data_store: DataStore, + l_loop_return_summary: dict[str, object], + id_namespace: LRunIds | None, +) -> list[str]: + file_paths: list[str] = [] + for file_path in _string_list(l_loop_return_summary.get("read_code_file_paths")): + file_paths.append(file_path) + + for record in data_store.list_records(): + if not _record_in_namespace(record.data_id, id_namespace=id_namespace): + continue + if not _is_code_extract_record(record.data_type): + continue + if not isinstance(record.payload, dict): + continue + if record.payload.get("read_status") != "ok": + continue + text = record.payload.get("text") + if not isinstance(text, str) or not text.strip(): + continue + file_path = record.payload.get("file_path") + if isinstance(file_path, str) and file_path.strip(): + file_paths.append(file_path.strip()) + return _unique_strings(file_paths) + + +def _supplied_source_code_context_count( + *, + data_store: DataStore, + read_documents: list[Node3BriefDocument], +) -> int: + count = 0 + for document in read_documents: + record = data_store.get_record(document.source_data_id) + if record is not None and _is_code_extract_record(record.data_type): + count += 1 + return count + + +def _source_code_outlines( + *, + data_store: DataStore, + read_documents: list[Node3BriefDocument], +) -> list[Node3SourceCodeOutline]: + outlines: list[Node3SourceCodeOutline] = [] + for document in read_documents: + record = data_store.get_record(document.source_data_id) + if record is None or not _is_code_extract_record(record.data_type): + continue + payload = record.payload if isinstance(record.payload, dict) else {} + if payload.get("read_status") != "ok": + continue + text = payload.get("text") + if not isinstance(text, str) or not text.strip(): + continue + file_path = payload.get("file_path") + if not isinstance(file_path, str) or not file_path.strip(): + file_path = document.document_name + outlines.append( + _build_source_code_outline( + file_path=file_path, + text=text, + source_data_id=record.data_id, + ) + ) + return outlines + + +def _build_source_code_outline( + *, + file_path: str, + text: str, + source_data_id: str, +) -> Node3SourceCodeOutline: + language = _source_code_language(file_path) + if language != "python": + return Node3SourceCodeOutline( + file_path=file_path, + language=language, + parse_status="unsupported_language", + source_data_id=source_data_id, + top_level_symbol_count=0, + public_symbol_count=0, + public_function_names=[], + top_level_symbols=[], + ) + + try: + tree = ast.parse(text, filename=file_path) + except SyntaxError as exc: + return Node3SourceCodeOutline( + file_path=file_path, + language=language, + parse_status="parse_failed", + source_data_id=source_data_id, + top_level_symbol_count=0, + public_symbol_count=0, + public_function_names=[], + top_level_symbols=[], + parse_error_type=type(exc).__name__, + ) + + symbols: list[Node3SourceCodeSymbol] = [] + for node in tree.body: + if isinstance(node, ast.FunctionDef): + symbols.append(_source_code_symbol(node.name, "function", node.lineno, ast.get_docstring(node) is not None)) + elif isinstance(node, ast.AsyncFunctionDef): + symbols.append( + _source_code_symbol( + node.name, + "async_function", + node.lineno, + ast.get_docstring(node) is not None, + ) + ) + elif isinstance(node, ast.ClassDef): + symbols.append(_source_code_symbol(node.name, "class", node.lineno, ast.get_docstring(node) is not None)) + elif isinstance(node, (ast.Assign, ast.AnnAssign)): + for name in _assigned_uppercase_names(node): + symbols.append(_source_code_symbol(name, "constant", node.lineno, False)) + + public_function_names = [ + symbol.name + for symbol in symbols + if symbol.is_public and symbol.symbol_kind in {"function", "async_function"} + ] + return Node3SourceCodeOutline( + file_path=file_path, + language=language, + parse_status="parsed", + source_data_id=source_data_id, + top_level_symbol_count=len(symbols), + public_symbol_count=sum(1 for symbol in symbols if symbol.is_public), + public_function_names=public_function_names, + top_level_symbols=symbols, + ) + + +def _source_code_symbol( + name: str, + symbol_kind: str, + line_number: int, + docstring_present: bool, +) -> Node3SourceCodeSymbol: + return Node3SourceCodeSymbol( + name=name, + symbol_kind=symbol_kind, + line_number=line_number, + is_public=not name.startswith("_"), + docstring_present=docstring_present, + ) + + +def _assigned_uppercase_names(node: ast.Assign | ast.AnnAssign) -> list[str]: + targets: list[ast.expr] = [] + if isinstance(node, ast.Assign): + targets = list(node.targets) + elif isinstance(node, ast.AnnAssign): + targets = [node.target] + + names: list[str] = [] + for target in targets: + names.extend(_uppercase_names_from_target(target)) + return _unique_strings(names) + + +def _uppercase_names_from_target(target: ast.expr) -> list[str]: + if isinstance(target, ast.Name) and target.id.isupper(): + return [target.id] + if isinstance(target, (ast.Tuple, ast.List)): + names: list[str] = [] + for element in target.elts: + names.extend(_uppercase_names_from_target(element)) + return names + return [] + + +def _source_code_language(file_path: str) -> str: + normalized = file_path.lower().strip() + if normalized.endswith(".py"): + return "python" + return "unsupported" + + +def _l_loop_result_attitude_hint(payload: dict[str, object]) -> str: + if not payload: + return "not_recorded" + task_status = _text(payload, "l_loop_task_status", fallback="unknown") + failure_level = _text(payload, "failure_level", fallback="unknown") + goal_match = _text(payload, "l3_goal_match_status", fallback="not_run") + semantic_match = _text(payload, "l3_semantic_goal_match_status", fallback="not_run") + if task_status == "achieved" and failure_level == "none": + return "l_loop_achieved" + if failure_level == "budget_exhausted": + return "l_loop_budget_exhausted" + if ( + task_status in {"partial", "failed", "unknown"} + or goal_match in {"partial", "missing"} + or semantic_match in {"partial", "missing"} + ): + return "l_loop_partial_or_failed" + return "l_loop_missing_or_uncertain" + + def _same_turn_l_reroute_controller_ids(data_store: DataStore) -> list[str]: return [ record.data_id @@ -1059,6 +2418,10 @@ def _is_document_extract_record(data_type: str) -> bool: return data_type.startswith("tool_result:read_doc") or data_type.startswith("tool_result:read_artifact") +def _is_code_extract_record(data_type: str) -> bool: + return data_type.startswith("tool_result:read_code_file") + + def _route_path(route_ids: list[str], *, actual_l_run_count: int) -> list[str]: path: list[str] = [] l_route_seen = 0 @@ -1077,6 +2440,12 @@ def _route_path(route_ids: list[str], *, actual_l_run_count: int) -> list[str]: elif route == "2": path.append("1:route=2") path.append("0:final_trace_for_2") + elif route == "R": + path.append("1:route=R_experimental") + path.append("0:r_loop_graph_guide_handoff") + path.append("R:R1_R2_R3_experimental_skeleton") + path.append("1:route=2_after_R_experimental") + path.append("0:final_trace_for_2") return path @@ -1085,6 +2454,8 @@ def _route_value(route_id: str) -> str | None: return "L" if route_id == "route:2" or route_id.endswith(":route:2"): return "2" + if route_id == "route:R" or route_id.endswith(":route:R"): + return "R" return None @@ -1095,6 +2466,22 @@ def _document_name(value: object) -> str: return normalized.rsplit("/", 1)[-1] or "읽은 문서" +def _read_doc_display_labels(doc_ids: list[str]) -> list[str]: + """같은 파일명이 여러 경로에 있으면 전체 doc_id로 구분해 표시한다.""" + + name_counts: dict[str, int] = {} + for doc_id in doc_ids: + name = _document_name(doc_id) + name_counts[name] = name_counts.get(name, 0) + 1 + + labels: list[str] = [] + for doc_id in doc_ids: + name = _document_name(doc_id) + label = doc_id.replace("\\", "/").strip("/") if name_counts.get(name, 0) > 1 else name + labels.append(label or name) + return labels + + def _payload(data_store: DataStore, data_id: str) -> dict[str, object]: if not data_id: return {} diff --git a/songryeon_core/nodes/node_2_metainfo_boundary.py b/songryeon_core/nodes/node_2_metainfo_boundary.py index f0d45c8..c7214a6 100644 --- a/songryeon_core/nodes/node_2_metainfo_boundary.py +++ b/songryeon_core/nodes/node_2_metainfo_boundary.py @@ -8,8 +8,11 @@ DataRef, MetainfoBoundary, MixedInfoRef, + Node2AnswerBasisFrame, RelativeInfoRef, Node2BoundaryReviewFrame, + Node2EvidenceRole, + validate_node2_answer_basis_frame, validate_mixed_info_ref, validate_relative_info_ref, validate_node2_boundary_review_frame, @@ -17,6 +20,10 @@ from songryeon_core.core.trace_store import TraceStore from songryeon_core.llm.base import LLMAdapter from songryeon_core.llm.node_executor import LLMNodeExecutor +from songryeon_core.loops.l_loop_namespace import LRunIds + + +NODE2_ANSWER_BASIS_FRAME_DATA_ID = "node_2:answer_basis_frame" def build_metainfo_boundary( @@ -296,6 +303,392 @@ def run_node2_boundary_review( return event.event_id +def run_node2_answer_basis_selection( + *, + trace_store: TraceStore, + data_store: DataStore, + turn_id: str, + user_question: str, + boundary_id: str, + boundary: MetainfoBoundary, + handoff_frame_id: str, + adapter: LLMAdapter | None, + input_ref: list[str], + source_data_ids: list[str], + id_namespace: LRunIds | None = None, +) -> tuple[str, str, Node2AnswerBasisFrame]: + """node_2 LLM이 node_3 답변 근거 모드를 고르고 실패 시 안전 fallback을 기록한다.""" + + frame_id = ( + id_namespace.scoped_data_id(NODE2_ANSWER_BASIS_FRAME_DATA_ID) + if id_namespace is not None + else NODE2_ANSWER_BASIS_FRAME_DATA_ID + ) + base_source_data_ids = _unique_strings( + [*source_data_ids, boundary_id, handoff_frame_id] + ) + available_evidence_sources = _answer_basis_available_evidence_sources( + boundary=boundary, + base_source_data_ids=base_source_data_ids, + ) + allowed_answer_basis_source_data_ids = _unique_strings( + [ + str(source["source_data_id"]) + for source in available_evidence_sources + if isinstance(source.get("source_data_id"), str) + ] + ) + prompt_ref = "songryeon_core/prompts/node_2_answer_basis_selector_v0.md" + if adapter is None: + frame = _fallback_answer_basis_frame( + frame_id=frame_id, + turn_id=turn_id, + source_trace_ids=input_ref, + source_data_ids=allowed_answer_basis_source_data_ids, + failure_type="adapter_missing", + llm_call_data_id=None, + trace_event_id=None, + validation_error="adapter_missing", + raw_text_present=False, + prompt_ref=prompt_ref, + payload_parse_status="not_checked", + ) + return _record_answer_basis_frame( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + frame=frame, + schema_status="failed", + ) + + prompt = Path(prompt_ref).read_text(encoding="utf-8") + llm_result = LLMNodeExecutor(adapter).run( + node_id="node_2", + prompt=prompt, + input_payload={ + "user_question": user_question, + "boundary_id": boundary_id, + "handoff_frame_id": handoff_frame_id, + "absolute_info_count": len(boundary.absolute_info), + "relative_info_count": len(boundary.relative_info), + "mixed_info_count": len(boundary.mixed_info), + "absolute_info_samples": [ + asdict(data_ref) for data_ref in boundary.absolute_info[:16] + ], + "relative_info_samples": [ + asdict(info_ref) for info_ref in boundary.relative_info[:12] + ], + "mixed_info_samples": [ + asdict(info_ref) for info_ref in boundary.mixed_info[:12] + ], + "source_data_ids": allowed_answer_basis_source_data_ids, + "available_evidence_sources": available_evidence_sources, + "answer_basis_modes": [ + "absolute_first", + "relative_allowed", + "mixed_or_uncertain", + ], + "basis_reason_codes": [ + "code_verified_fact_required", + "user_asked_for_interpretation", + "multi_source_bundle", + "source_mapping_unclear", + "insufficient_grounding", + "partial_evidence_only", + "recent_conversation_basis_present", + "document_basis_present", + "runtime_state_basis_present", + "llm_mode_selection_failed", + ], + "evidence_role_values": [ + "primary_answer_basis", + "supporting_context", + "available_but_not_used", + "candidate_not_read", + "excluded_by_budget", + "failed_or_empty", + "not_supplied", + ], + }, + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + prompt_ref=prompt_ref, + input_ref=input_ref, + source_data_ids=base_source_data_ids, + payload_validator=lambda payload: _validate_answer_basis_payload( + payload, + allowed_source_data_ids=allowed_answer_basis_source_data_ids, + ), + ) + frame_source_trace_ids = list(input_ref) + if llm_result.trace_event_id: + frame_source_trace_ids.append(llm_result.trace_event_id) + frame_source_data_ids = _unique_strings( + [*allowed_answer_basis_source_data_ids, llm_result.call_data_id] + ) + if llm_result.failure_type == "none" and llm_result.validation.payload is not None: + payload = llm_result.validation.payload + frame = Node2AnswerBasisFrame( + frame_id=frame_id, + turn_id=turn_id, + answer_basis_mode=str(payload.get("answer_basis_mode") or "").strip(), + basis_reason_codes=_string_list(payload.get("basis_reason_codes")), + mode_selection_reason=str(payload.get("mode_selection_reason") or "").strip(), + mode_selection_reason_info_class=str( + payload.get("mode_selection_reason_info_class") or "mixed" + ).strip(), + evidence_roles=_evidence_roles_from_payload(payload.get("evidence_roles")), + generated_by=f"LLM:{llm_result.model_id}", + info_class=str(payload.get("mode_selection_reason_info_class") or "mixed").strip(), + semantic_judgement_status="ran", + answer_basis_failure_type="none", + answer_basis_llm_call_data_id=llm_result.call_data_id, + answer_basis_trace_event_id=llm_result.trace_event_id, + answer_basis_validation_error="", + answer_basis_raw_text_present=bool(llm_result.raw_text), + answer_basis_prompt_ref=prompt_ref, + answer_basis_payload_parse_status=_answer_basis_payload_parse_status(llm_result.failure_type), + source_trace_ids=_unique_strings(frame_source_trace_ids), + source_data_ids=frame_source_data_ids, + ) + validate_node2_answer_basis_frame(frame) + return _record_answer_basis_frame( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + frame=frame, + schema_status="passed", + ) + + frame = _fallback_answer_basis_frame( + frame_id=frame_id, + turn_id=turn_id, + source_trace_ids=frame_source_trace_ids, + source_data_ids=frame_source_data_ids, + failure_type=llm_result.failure_type, + llm_call_data_id=llm_result.call_data_id, + trace_event_id=llm_result.trace_event_id, + validation_error=_short_diagnostic_text(llm_result.validation.error or ""), + raw_text_present=bool(llm_result.raw_text), + prompt_ref=prompt_ref, + payload_parse_status=_answer_basis_payload_parse_status(llm_result.failure_type), + ) + return _record_answer_basis_frame( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + frame=frame, + schema_status="failed", + ) + + +def _answer_basis_available_evidence_sources( + *, + boundary: MetainfoBoundary, + base_source_data_ids: list[str], +) -> list[dict[str, str]]: + """answer-basis LLM이 evidence role로 고를 수 있는 source ID 표를 만든다.""" + + candidate_ids = _unique_strings( + [ + *base_source_data_ids, + *(ref.data_id for ref in boundary.absolute_info[:16]), + *(ref.source_data_id for ref in boundary.relative_info[:12]), + *(ref.source_data_id for ref in boundary.mixed_info[:12]), + ] + ) + return [ + { + "source_data_id": source_data_id, + "source_label": _answer_basis_source_label(source_data_id), + "source_kind": _answer_basis_source_kind(source_data_id), + } + for source_data_id in candidate_ids + ] + + +def _answer_basis_source_label(source_data_id: str) -> str: + if "selected_recent_memory_context" in source_data_id: + return "선택된 최근 기억" + if "memory_relevance" in source_data_id: + return "최근 기억 선택 판단" + if "l_loop_return_summary" in source_data_id or "L:return_summary_frame" in source_data_id: + return "L loop 반환 요약" + if "read_doc" in source_data_id or "read_artifact" in source_data_id: + return "읽은 문서" + if "document_material_packet" in source_data_id: + return "0 문서 장부" + if "document_context_pack" in source_data_id: + return "문서 context pack" + if "L3" in source_data_id or "achievement" in source_data_id or "preserved" in source_data_id: + return "L3 검색 성패 판단" + if "route" in source_data_id: + return "라우팅 결과" + if "boundary" in source_data_id: + return "메타정보 경계" + if "handoff" in source_data_id: + return "node_2 handoff" + if "node2_input" in source_data_id: + return "node_2 입력 프레임" + return "공급된 근거 자료" + + +def _answer_basis_source_kind(source_data_id: str) -> str: + if "selected_recent_memory_context" in source_data_id: + return "selected_recent_memory_context" + if "memory_relevance" in source_data_id: + return "memory_relevance_selection" + if "l_loop_return_summary" in source_data_id or "L:return_summary_frame" in source_data_id: + return "l_loop_return_summary" + if "read_doc" in source_data_id or "read_artifact" in source_data_id: + return "read_document" + if "document_material_packet" in source_data_id: + return "document_material_packet" + if "document_context_pack" in source_data_id: + return "document_context_pack" + if "L3" in source_data_id or "achievement" in source_data_id or "preserved" in source_data_id: + return "l3_result" + if "route" in source_data_id: + return "route" + if "boundary" in source_data_id: + return "metainfo_boundary" + if "handoff" in source_data_id: + return "node2_handoff" + if "node2_input" in source_data_id: + return "node2_input" + return "supplied_source" + + +def _fallback_answer_basis_frame( + *, + frame_id: str, + turn_id: str, + source_trace_ids: list[str], + source_data_ids: list[str], + failure_type: str, + llm_call_data_id: str | None, + trace_event_id: str | None, + validation_error: str, + raw_text_present: bool, + prompt_ref: str, + payload_parse_status: str, +) -> Node2AnswerBasisFrame: + frame = Node2AnswerBasisFrame( + frame_id=frame_id, + turn_id=turn_id, + answer_basis_mode="mixed_or_uncertain", + basis_reason_codes=["llm_mode_selection_failed"], + mode_selection_reason="CODE_STATUS:node2_answer_basis_mode_selection_failed", + mode_selection_reason_info_class="absolute_status", + evidence_roles=[], + generated_by="CODE:FALLBACK", + info_class="absolute_status", + semantic_judgement_status="failed", + answer_basis_failure_type=failure_type, + answer_basis_llm_call_data_id=llm_call_data_id, + answer_basis_trace_event_id=trace_event_id, + answer_basis_validation_error=validation_error, + answer_basis_raw_text_present=raw_text_present, + answer_basis_prompt_ref=prompt_ref, + answer_basis_payload_parse_status=payload_parse_status, + source_trace_ids=_unique_strings(source_trace_ids), + source_data_ids=_unique_strings(source_data_ids), + ) + validate_node2_answer_basis_frame(frame) + return frame + + +def _answer_basis_payload_parse_status(failure_type: str) -> str: + if failure_type in {"none", "schema_failed"}: + return "passed" + if failure_type == "parse_failed": + return "failed" + return "not_checked" + + +def _short_diagnostic_text(text: str, *, limit: int = 240) -> str: + compact = " ".join(text.split()) + if len(compact) <= limit: + return compact + return f"{compact[: limit - 3]}..." + + +def _record_answer_basis_frame( + *, + trace_store: TraceStore, + data_store: DataStore, + turn_id: str, + frame: Node2AnswerBasisFrame, + schema_status: str, +) -> tuple[str, str, Node2AnswerBasisFrame]: + event = trace_store.create_event( + turn_id=turn_id, + actor="node_2", + event_type="node_output", + input_ref=frame.source_trace_ids, + output_ref=[frame.frame_id], + schema_status=schema_status, + ) + data_store.create_record( + data_id=frame.frame_id, + data_type="node_output:node2_answer_basis_frame", + exists=True, + created_at=event.timestamp, + source_trace_id=event.event_id, + payload=asdict(frame), + ) + return event.event_id, frame.frame_id, frame + + +def _validate_answer_basis_payload( + payload: dict[str, object], + *, + allowed_source_data_ids: list[str], +) -> None: + frame = Node2AnswerBasisFrame( + frame_id="validation_answer_basis", + turn_id="validation_turn", + answer_basis_mode=str(payload.get("answer_basis_mode") or "").strip(), + basis_reason_codes=_string_list(payload.get("basis_reason_codes")), + mode_selection_reason=str(payload.get("mode_selection_reason") or "").strip(), + mode_selection_reason_info_class=str( + payload.get("mode_selection_reason_info_class") or "mixed" + ).strip(), + evidence_roles=_evidence_roles_from_payload(payload.get("evidence_roles")), + generated_by="LLM:validation-model", + info_class=str(payload.get("mode_selection_reason_info_class") or "mixed").strip(), + semantic_judgement_status="ran", + source_trace_ids=["validation_trace"], + source_data_ids=_unique_strings(["validation_data", *allowed_source_data_ids]), + ) + validate_node2_answer_basis_frame(frame) + + +def _evidence_roles_from_payload(value: object) -> list[Node2EvidenceRole]: + if not isinstance(value, list): + return [] + roles: list[Node2EvidenceRole] = [] + for item in value: + if not isinstance(item, dict): + continue + source_data_id = str(item.get("source_data_id") or "").strip() + evidence_role = str(item.get("evidence_role") or "").strip() + if not source_data_id or not evidence_role: + continue + roles.append( + Node2EvidenceRole( + source_data_id=source_data_id, + evidence_role=evidence_role, + role_reason=str(item.get("role_reason") or "").strip(), + role_reason_info_class=str( + item.get("role_reason_info_class") or "mixed" + ).strip(), + ) + ) + return roles + + def _data_record_metadata_refs( *, data_id: str, diff --git a/songryeon_core/nodes/node_3_reporter.py b/songryeon_core/nodes/node_3_reporter.py index 6ebaf25..377c307 100644 --- a/songryeon_core/nodes/node_3_reporter.py +++ b/songryeon_core/nodes/node_3_reporter.py @@ -147,9 +147,20 @@ def build_node3_grounding_block(brief_frame: Node3InputBriefFrame) -> str: return "\n".join( [ "근거 기준:", - f"- 읽은 문서: {len(brief_frame.read_documents)}개", - f"- 검색 후보 문서: {brief_frame.search_candidate_count}개", + f"- 실제 read_doc 도구 원문 읽기: {brief_frame.actual_tool_read_doc_count}개", + f"- 실제 read_code_file 도구 원문 읽기: {brief_frame.actual_tool_read_code_file_count}개", + f"- node_3 공급 문서 context: {brief_frame.supplied_document_context_count}개", + f"- node_3 공급 source-code context: {brief_frame.supplied_source_code_context_count}개", + f"- source-code 구조 목록: {len(brief_frame.source_code_outlines)}개", + f"- node_3 LLM 원문 text: {brief_frame.llm_raw_document_text_count}개", + f"- L3 문서별 요약 재료: {brief_frame.llm_l3_summary_context_count}개", + f"- 검색 후보 문서(최종): {brief_frame.final_search_candidate_count}개", + f"- 검색 후보 문서(누적): {brief_frame.accumulated_search_candidate_count}개", f"- 현재 턴 실행 순서 자료: {len(brief_frame.runtime_tasks)}개", + *_l_loop_grounding_lines(brief_frame), + *_r_loop_grounding_lines(brief_frame), + f"- 답변 근거 자세: {_answer_basis_mode_label(brief_frame.answer_basis_mode)}", + f"- 재료 전달 정책: {_material_delivery_mode_label(brief_frame.material_delivery_mode)}", f"- 답변 한계: {_grounding_limit_text(brief_frame)}", ] ) @@ -162,7 +173,10 @@ def assemble_node3_report_markdown( ) -> str: """CODE count block과 LLM 본문을 합쳐 최종 node_3 보고문을 만든다.""" - return f"{build_node3_grounding_block(brief_frame)}\n\n{body_markdown.strip()}" + body = _strip_accidental_grounding_block(body_markdown) + if not body: + return build_node3_grounding_block(brief_frame) + return f"{build_node3_grounding_block(brief_frame)}\n\n{body}" def record_report( @@ -235,10 +249,13 @@ def _report_body_from_payload(payload: dict[str, object]) -> str: def _strip_accidental_grounding_block(markdown: str) -> str: stripped = markdown.strip() - if not stripped.startswith("근거 기준:"): + lines = stripped.splitlines() + if not lines: + return "" + first_line = lines[0].strip() + if first_line not in {"근거 기준:", "**근거 기준:**"}: return stripped - lines = stripped.splitlines() index = 1 while index < len(lines): line = lines[index] @@ -253,11 +270,70 @@ def _strip_accidental_grounding_block(markdown: str) -> str: def _grounding_limit_text(brief_frame: Node3InputBriefFrame) -> str: + if brief_frame.raw_document_policy == "omit_raw_text_from_llm_payload": + return "원문 record는 보존되어 있지만 node_3 LLM 입력에서는 원문 text를 생략하고 L3 요약 재료를 사용한다." + if brief_frame.l_loop_result_attitude_hint == "l_loop_budget_exhausted": + return "L 검색 목표가 예산 소진으로 닫힌 상태라, 공급된 문서가 있어도 검색 성공으로 단정하지 않는다." + if brief_frame.l_loop_result_attitude_hint == "l_loop_partial_or_failed": + return "L 검색 목표가 부분/실패 신호를 남겼으므로, 공급된 문서 범위와 한계를 분리해 말한다." + if ( + brief_frame.r_loop_result_material is not None + and brief_frame.r_loop_result_material.attitude_hint != "r_loop_sufficient" + ): + return "R route 실험 결과는 skeleton/부분 장부이므로 graph memory 탐색 성공으로 단정하지 않는다." if brief_frame.insufficiency_reasons: return "자료 부족 신호가 있어 제공된 문서/허용 주장/현재 턴 실행 순서 자료 범위 안에서만 답한다." + if brief_frame.answer_basis_mode == "absolute_first": + return "확인 가능한 코드/문서/trace/data 근거를 우선하고 확인되지 않은 내용은 단정하지 않는다." + if brief_frame.answer_basis_mode == "mixed_or_uncertain": + return "제공된 출처 묶음과 부분 근거의 한계를 드러내며 단정하지 않는다." return "제공된 문서/허용 주장/현재 턴 실행 순서 자료 범위 안에서만 답한다. 검색 후보는 읽은 문서가 아니다." +def _l_loop_grounding_lines(brief_frame: Node3InputBriefFrame) -> list[str]: + if brief_frame.l_loop_result_attitude_hint == "not_recorded": + return [] + return [ + ( + "- L 검색 목표 상태: " + f"{brief_frame.l_loop_task_status} / {brief_frame.l_loop_failure_level} " + f"/ semantic={brief_frame.l3_semantic_goal_match_status}" + ) + ] + + +def _r_loop_grounding_lines(brief_frame: Node3InputBriefFrame) -> list[str]: + material = brief_frame.r_loop_result_material + if material is None: + return [] + return [ + ( + "- R 탐색 실험 상태: " + f"{material.r_loop_task_status} / {material.continuation_status} " + f"/ budget={material.budget_status} / hint={material.attitude_hint}" + ) + ] + + +def _answer_basis_mode_label(mode: str) -> str: + labels = { + "absolute_first": "절대정보 우선", + "relative_allowed": "해석 허용", + "mixed_or_uncertain": "혼합/불확실성 표시", + } + return labels.get(mode, "혼합/불확실성 표시") + + +def _material_delivery_mode_label(mode: str) -> str: + labels = { + "raw_document_primary": "원문 우선", + "l3_summary_replaces_raw_context": "L3 요약이 원문 text 대체", + "l3_summary_replaces_raw_context_with_uncertainty": "L3 요약 대체/불확실성 표시", + "raw_document_fallback_no_l3_summary": "L3 요약 없음/raw fallback", + } + return labels.get(mode, "재료 전달 정책 불명") + + def _unique_strings(values: list[str | None]) -> list[str]: seen: set[str] = set() unique_values: list[str] = [] diff --git a/songryeon_core/nodes/node_4_gatekeeper.py b/songryeon_core/nodes/node_4_gatekeeper.py index 5fb05eb..2beb445 100644 --- a/songryeon_core/nodes/node_4_gatekeeper.py +++ b/songryeon_core/nodes/node_4_gatekeeper.py @@ -19,6 +19,9 @@ NODE4_GATEKEEPER_FRAME_DATA_ID = "node_4:gatekeeper_frame" RECENT_MEMORY_INTERNAL_ID_LEAK = "CODE_STATUS:recent_memory_internal_id_leak" +DOCUMENT_EVIDENCE_ROLE_CLAIM_MISMATCH = ( + "CODE_STATUS:document_evidence_role_claim_mismatch" +) def run_node4_gatekeeper( @@ -67,6 +70,8 @@ def run_node4_gatekeeper( "문서 추출이 있는데 없다고 말하는 모순이 있는지 확인한다.", "내부 추적용 식별자나 장부용 필드명이 노출됐는지 확인한다.", "최근 기억 발화가 selected_recent_memory_contexts 범위를 벗어나는지 확인한다.", + "answer_basis_mode에 맞는 말하기 자세를 유지했는지 확인한다.", + "L loop 실패/예산소진 신호를 검색 성공처럼 말하는지 확인한다.", ], }, trace_store=trace_store, @@ -142,6 +147,27 @@ def run_node4_gatekeeper( elif recent_memory_guard["status"] == "pass": checked_claims = _unique_strings([*checked_claims, "recent_memory_internal_id_guard"]) + document_role_guard = _document_evidence_role_code_guard( + rendered_markdown=rendered_markdown, + brief_frame=brief_frame, + ) + if document_role_guard["status"] == "needs_revision": + if gate_status == "pass": + gate_status = "needs_revision" + if "CODE:DOCUMENT_EVIDENCE_ROLE_GUARD" not in gate_generation_source: + gate_generation_source = f"{gate_generation_source}+CODE:DOCUMENT_EVIDENCE_ROLE_GUARD" + for reason_code in document_role_guard["reason_codes"]: + reason = _append_reason(reason, reason_code) + checked_claims = _unique_strings([*checked_claims, "document_evidence_role_guard"]) + contradictions = _unique_strings( + [*contradictions, *document_role_guard["contradictions"]] + ) + revision_targets = _unique_strings( + [*revision_targets, *document_role_guard["revision_targets"]] + ) + elif document_role_guard["status"] == "pass": + checked_claims = _unique_strings([*checked_claims, "document_evidence_role_guard"]) + frame_source_trace_ids = list(input_ref) if llm_result.trace_event_id: frame_source_trace_ids.append(llm_result.trace_event_id) @@ -205,8 +231,10 @@ def _grounding_count_violations( # 장기적으로는 node_3가 ReportGroundingFrame 같은 구조화 출력을 함께 만들고, # node_4는 렌더링된 문장 대신 그 frame을 검사하는 쪽이 더 건강하다. expected_counts = { - "읽은 문서": len(brief_frame.read_documents), - "검색 후보 문서": brief_frame.search_candidate_count, + "실제 read_doc 도구 원문 읽기": brief_frame.actual_tool_read_doc_count, + "node_3 공급 문서 context": brief_frame.supplied_document_context_count, + "검색 후보 문서(최종)": brief_frame.final_search_candidate_count, + "검색 후보 문서(누적)": brief_frame.accumulated_search_candidate_count, "현재 턴 실행 순서 자료": len(brief_frame.runtime_tasks), } if not rendered_markdown.startswith("근거 기준:"): @@ -258,6 +286,131 @@ def _recent_memory_code_guard( } +def _document_evidence_role_code_guard( + *, + rendered_markdown: str, + brief_frame: Node3InputBriefFrame, +) -> dict[str, object]: + """문서 장부의 명시 role과 보고문 속 명시 role 주장이 충돌하는지 검사한다.""" + + contradictions: list[str] = [] + revision_targets: list[str] = [] + for item in brief_frame.document_material_items: + if not item.was_actual_tool_read_doc and _claims_explicit_read_doc_role( + rendered_markdown=rendered_markdown, + document_name=item.document_name, + ): + contradictions.append( + "read_doc_claim_without_actual_tool_read_doc:" + f"{_safe_report_token(item.document_name)}" + ) + revision_targets.append( + "read_doc으로 읽은 문서와 node_3 context로 공급된 문서를 분리해 말한다." + ) + if not item.was_supplied_document_context and _claims_explicit_supplied_context_role( + rendered_markdown=rendered_markdown, + document_name=item.document_name, + ): + contradictions.append( + "supplied_context_claim_without_supplied_context:" + f"{_safe_report_token(item.document_name)}" + ) + revision_targets.append( + "node_3 context로 공급된 문서와 후보/제외 문서를 분리해 말한다." + ) + + unique_contradictions = _unique_strings(contradictions) + reason_codes = ( + [DOCUMENT_EVIDENCE_ROLE_CLAIM_MISMATCH] if unique_contradictions else [] + ) + return { + "status": "needs_revision" if unique_contradictions else "pass", + "reason_codes": reason_codes, + "contradictions": unique_contradictions, + "revision_targets": _unique_strings(revision_targets), + } + + +def _claims_explicit_read_doc_role( + *, + rendered_markdown: str, + document_name: str, +) -> bool: + for line in _lines_mentioning_document(rendered_markdown, document_name): + if _has_negated_role_claim(line): + continue + if re.search( + r"(?:`?read_doc`?|actual_tool_read_doc)\s*(?:으로|로)?\s*" + r"(?:읽혔|읽힌|읽었|읽어|실행|수행)", + line, + flags=re.IGNORECASE, + ): + return True + if "actual_tool_read_doc" in line: + return True + return False + + +def _claims_explicit_supplied_context_role( + *, + rendered_markdown: str, + document_name: str, +) -> bool: + for line in _lines_mentioning_document(rendered_markdown, document_name): + if _has_negated_role_claim(line): + continue + if re.search( + r"(?:node_3\s*)?(?:공급\s*)?(?:문서\s*)?context(?:로|로서|로는)?\s*" + r"(?:공급|포함|전달)", + line, + flags=re.IGNORECASE, + ): + return True + if "supplied_document_context" in line: + return True + return False + + +def _lines_mentioning_document(rendered_markdown: str, document_name: str) -> list[str]: + if not document_name: + return [] + tokens = _document_name_tokens(document_name) + return [ + line + for line in rendered_markdown.splitlines() + if any(token and token in line for token in tokens) + ] + + +def _document_name_tokens(document_name: str) -> list[str]: + tokens = [document_name.strip()] + basename = document_name.replace("\\", "/").rsplit("/", 1)[-1].strip() + if basename and basename not in tokens: + tokens.append(basename) + return tokens + + +def _has_negated_role_claim(line: str) -> bool: + lowered = line.lower() + negation_tokens = [ + "아니다", + "아니며", + "않", + "못", + "읽히지", + "읽지", + "not ", + "not_", + "never", + "without", + ] + return any(token in lowered for token in negation_tokens) + + +def _safe_report_token(value: str) -> str: + return value.replace(" ", "_")[:120] + + def _internal_id_leak_count(rendered_markdown: str) -> int: patterns = [ r"memory_packet:[A-Za-z0-9_:\-]+", diff --git a/songryeon_core/prompts/core_ego_guide_worker_v0.md b/songryeon_core/prompts/core_ego_guide_worker_v0.md new file mode 100644 index 0000000..be069ef --- /dev/null +++ b/songryeon_core/prompts/core_ego_guide_worker_v0.md @@ -0,0 +1,36 @@ +# CoreEgo Guide Worker v0 + +You are not the final user-facing reporter. + +You read a code-generated `RLoopGraphGuidePacket` snapshot and write one traversal hint for a future R loop. + +Return JSON only. + +Allowed output fields: + +```json +{ + "recommended_entry_node_ids": ["graph:axis:time"], + "avoid_entry_node_ids": [], + "traversal_strategy_hint": "Start from the time axis and inspect the newest time bundle first.", + "reason_summary": "The current graph exposes only the time axis as an entry point.", + "risk_notes": ["No semantic axis exists yet."], + "expected_depth_policy": "Use shallow traversal first; descend only when raw capsules are needed.", + "source_graph_node_ids": ["graph:axis:time"], + "source_data_ids": ["rloop:graph_guide:..."], + "info_class": "mixed", + "semantic_judgement_status": "ran" +} +``` + +Rules: + +- Choose `recommended_entry_node_ids` and `avoid_entry_node_ids` only from `available_entry_node_ids`. +- Choose `source_graph_node_ids` only from `available_source_graph_node_ids`. +- Choose `source_data_ids` only from the supplied `source_data_ids`. +- Do not invent graph node IDs. +- Do not claim that a semantic axis exists. +- Do not create R1/R2/R3 execution results. +- Do not answer the user. +- Keep the hint short and operational. +- This hint is mixed information because it interprets a source bundle. diff --git a/songryeon_core/prompts/l2_query_setter_v0.md b/songryeon_core/prompts/l2_query_setter_v0.md index 8111491..31c6daf 100644 --- a/songryeon_core/prompts/l2_query_setter_v0.md +++ b/songryeon_core/prompts/l2_query_setter_v0.md @@ -33,10 +33,16 @@ Create 1-3 candidates. The selected candidate ID must match one candidate. ## Rules -- Choose `read_artifact` only when the user gives an explicit document name, file name, path, or order ID to read. -- Choose `search_docs` when the user describes a topic, concept, question, or evidence need. +- Choose `read_artifact` only when the user gives an explicit document/order/Markdown artifact name, file name, path, or order ID to read. +- Choose `search_docs` when the user describes a project document topic, concept, question, or evidence need. +- Choose `list_code_files` when the user asks about the repository/code file layout, module map, or source tree structure. +- Choose `search_code` when the user asks where a function, class, schema, field, constant, prompt reference, or runtime hook appears in source code. +- Choose `read_code_file` only when the user gives an explicit source/config file path to inspect. - `search_docs` is semantic search: write a concise description of the wanted content. - `read_artifact` is exact reference reading: write only the explicit artifact reference, such as `CODE_STRUCTURE_MAP_v1` or `ORDER_084_NODE4_REMAND_BLOCKING`. +- `search_code` is literal source-code substring search: write the exact identifier, path fragment, field name, or short code phrase to find. +- `read_code_file` is exact file reading: write only the workspace-relative source/config file path. +- `list_code_files` ignores `query_text` semantically; use a concise label such as `codebase file layout`. - Read the supplied `l1_goal` and `l2_planning_contract`. They are the main source for what evidence the L loop must gather. - Treat `attribution_source_data_ids` only as source IDs to copy into candidate `source_data_ids`. Do not interpret those IDs as search topics. - Do not turn internal record names such as `budget_plan_frame`, `tool_catalog`, `query_frame`, or `trace` into document-search topics. @@ -45,6 +51,7 @@ Create 1-3 candidates. The selected candidate ID must match one candidate. - If `l1_goal.minimum_read_documents` is 2 or more, the query should help produce at least that many plausible document candidates. Do not write a query that naturally points to only one artifact. - For exploratory/random requests, `search_docs` is not true randomness. Use a semantic exploration query and make the limitation visible in `purpose`. - Do not choose `read_doc` or `list_docs`; they are controller/internal tools, not L2 planning targets. +- Code tools are read-only inspection tools. Do not propose edits, patches, file writes, tests, or command execution through L2. - Keep query source explicit. - Do not hide whether the query is a fallback. - Query candidates must not claim document contents are true. diff --git a/songryeon_core/prompts/l2_revision_query_setter_v0.md b/songryeon_core/prompts/l2_revision_query_setter_v0.md index caf8cb1..cb017ff 100644 --- a/songryeon_core/prompts/l2_revision_query_setter_v0.md +++ b/songryeon_core/prompts/l2_revision_query_setter_v0.md @@ -2,7 +2,7 @@ ## Role -Plan a revised internal document search after L3 reported that the previous L loop attempt was not fully successful. +Plan a revised internal document action after L3 reported that the previous L loop attempt was not fully successful. You receive an `L2RevisionInputFrame`. Use it to create a new `L2QueryPlanFrame`. @@ -19,8 +19,8 @@ Use this exact shape: "candidates": [ { "candidate_id": "L2:revision_query_candidate_0001", - "query_text": "revised internal document search query", - "purpose": "why this revised query is better than the previous attempt", + "query_text": "revised internal document search query or exact unread candidate doc_id", + "purpose": "why this revised action is better than the previous attempt", "expected_signal": "what kind of evidence should appear if this retry works", "priority": 1, "target_tool_name": "search_docs", @@ -40,10 +40,16 @@ Create 1-3 candidates. The selected candidate ID must match one candidate. - Choose `search_docs` when the next attempt should broaden, narrow, or redirect semantic search. - `search_docs` is semantic search: describe the wanted content clearly. - `read_artifact` is exact reference reading: write only the explicit artifact reference. -- Do not choose `read_doc` or `list_docs`; they are controller/internal tools, not L2 planning targets. +- Choose `read_doc` only for revision reading of an existing unread candidate. +- Choose `list_code_files`, `search_code`, or `read_code_file` only when the original user request is about source code structure or a specific source/config file, not project documents. +- `search_code` is literal source-code substring search; use an exact identifier, field name, path fragment, or short code phrase. +- `read_code_file` is exact file reading; write only the workspace-relative source/config file path. +- For `read_doc`, `query_text` must be exactly one doc_id from `revision_input.unread_candidate_doc_ids`. +- If `revision_input.remaining_query_attempts` is 0 and `revision_input.remaining_read_doc_calls` is greater than 0, do not create `search_docs` candidates. Create `read_doc` candidates from unread candidate doc IDs. +- Do not choose `list_docs`; it is not an L2 planning target. - Do not claim that document contents are true. - Do not pretend that the retry has already succeeded. - Every candidate must include at least one source data ID from `source_data_ids`. - If L3 feedback is present, use it as guidance, but do not treat it as absolute truth. -- If unread candidate summaries exist, you may use them to choose a more precise query or exact artifact reference. +- If unread candidate summaries exist and read budget remains, prefer reading one of those candidates over creating another broad search. - Keep `query_text` concise. diff --git a/songryeon_core/prompts/l3_per_document_summary_v0.md b/songryeon_core/prompts/l3_per_document_summary_v0.md new file mode 100644 index 0000000..a981614 --- /dev/null +++ b/songryeon_core/prompts/l3_per_document_summary_v0.md @@ -0,0 +1,26 @@ +# L3 Per-Document Summary v0 + +You write one summary frame for one document that L3 actually received as a document extract. + +Return only one JSON object: + +```json +{ + "plain_document_summary": "Korean summary based only on this source document", + "task_relevant_summary": "Korean summary of the parts relevant to the current user task", + "summary_limit_note": "short Korean note about limits, or empty string" +} +``` + +Rules: + +- Write in Korean. +- Do not expose raw internal IDs. +- `plain_document_summary` must use only `source_document.text`. +- `plain_document_summary` must not use the user question, L1 goal, search history, or outside knowledge. +- `task_relevant_summary` may use `user_query`, `l1_goal`, and the same `source_document.text`. +- `task_relevant_summary` is still only a per-document summary. Do not combine this document with other documents. +- Do not claim the document is ultimately true. You are summarizing supplied text. +- If the source text is truncated, mention that limit in `summary_limit_note`. +- Do not invent facts that are not present in the supplied source document. +- Keep both summaries concise but useful. diff --git a/songryeon_core/prompts/l3_result_keeper_v0.md b/songryeon_core/prompts/l3_result_keeper_v0.md index 274f155..4a1a68d 100644 --- a/songryeon_core/prompts/l3_result_keeper_v0.md +++ b/songryeon_core/prompts/l3_result_keeper_v0.md @@ -30,16 +30,18 @@ Rules: - Judge only operational success, not final truth of document contents. - First judge the supplied `user_query`, `l1_goal`, `l1_success_requirements`, and `l3_judgement_contract`. - Read document content is evidence material, not the goal itself. If a read document says that some feature was implemented, that does not automatically mean this turn's L1 goal was achieved. -- Your achievement reasons must be about this turn's L1 success condition, `evidence_counts`, `read_doc_ids`, and whether the evidence material is ready for node 3. +- Your achievement reasons must be about this turn's L1 success condition, `evidence_counts`, `read_doc_ids`, `read_code_file_paths`, and whether the evidence material is ready for node 3. - Do not use implementation claims found inside a read document as the reason why this L loop achieved its current goal. - If a read document is about an old order, implementation, or execution record, that content may be evidence for node 3 later, but it is not by itself proof that this turn's L1 goal succeeded. - Use only the supplied user query, controller decision, L1 goals, preserved candidate previews, - and read document previews. + read document previews, and read code file previews. - `l1_success_requirements.l_loop_success_condition` states what evidence condition L1 expected before the L loop returns. Use it when judging macro achievement. - Treat `controller_decision` as a loop-stop signal, not proof that the macro/micro goals were achieved. - Treat `candidate_count` and `evidence_counts.preserved_candidate_count` as preserved search/tool candidates, not proof that documents were read. - Treat `evidence_counts.unique_search_result_document_count` as the number of unique documents that appeared in search results. -- Treat `evidence_counts.read_document_count` and `read_doc_ids` as the only proof that document text was actually read. +- Treat `evidence_counts.read_document_count` and `read_doc_ids` as proof that Markdown/document text was actually read. +- Treat `evidence_counts.read_code_file_count` and `read_code_file_paths` as proof that source/config file text was actually read. +- Do not rename source-code evidence into `read_doc` evidence; keep both evidence channels separate. - Do not say "read", "viewed", "analyzed", or "relationship analysis completed" for search candidates whose document text was not in `read_document_previews`. - The L loop may have a wide search/read budget. Do not treat an old minimum such as two read documents as automatically sufficient when the user explicitly asks for broad coverage, "as many as possible", or several named ORDER/document identifiers. - If the user query names several explicit ORDER/document identifiers, judge coverage against those named targets using `read_doc_ids` and `read_document_previews`. @@ -57,7 +59,7 @@ Rules: - If L1's micro goal asks to retrieve 4-6 random documents, do not count duplicate candidates as separate read documents, and do not treat 3 search-result documents as 4-6 read documents. - Do not use keyword presets, identity presets, hidden project knowledge, or hardcoded routing assumptions. - If `specific_document_request.requested_doc_hint` is present, do not call the turn `achieved` - unless the requested document was directly read or the supplied code context says it matched. + unless the requested document/source file was directly read through the matching evidence channel. - If the requested document appeared only in search results but was not read, use `partial`. - If the requested document did not appear in either read documents or search results, use `partial` when other evidence exists and `failed` when there is no evidence. diff --git a/songryeon_core/prompts/l_tool_scope_planner_v0.md b/songryeon_core/prompts/l_tool_scope_planner_v0.md new file mode 100644 index 0000000..15b6c12 --- /dev/null +++ b/songryeon_core/prompts/l_tool_scope_planner_v0.md @@ -0,0 +1,63 @@ +# L Tool Scope Planner v0 + +## Role + +Choose the evidence/tool scope for this L loop before L2 plans individual tool calls. + +This is not a query planner. Do not choose exact queries here. Your job is to decide which families of evidence tools should be open for this turn. + +## Output + +Return JSON only. Do not wrap it in Markdown. + +Use this exact shape: + +```json +{ + "tool_scope_mode": "document_and_code", + "allowed_tool_groups": ["document_tools", "code_inspection_tools"], + "required_materials": ["order_document", "source_code_file", "code_search_result"], + "scope_reason": "why these tool groups and materials are required for this L loop", + "scope_reason_info_class": "mixed" +} +``` + +## Allowed Values + +`tool_scope_mode` must be one of: + +- `document_only` +- `code_only` +- `document_and_code` +- `runtime_trace_only` +- `mixed_evidence` + +`allowed_tool_groups` may contain: + +- `document_tools` +- `code_inspection_tools` +- `runtime_record_tools` + +`required_materials` may contain: + +- `order_document` +- `source_code_file` +- `code_search_result` +- `runtime_trace` +- `execution_record` +- `project_document` + +## Rules + +- Choose scope from the supplied user query, L1 goal, budget plan, and available tools. +- Do not use hidden project knowledge. +- Do not choose a tool group that is not needed for the L1 goal. +- Do not choose exact search strings or file paths here. +- Do not claim any tool has already run. +- If the request requires comparing an order/design document with actual source code, choose `document_and_code`. +- If the request is only about internal design/order/execution documents, choose `document_only`. +- If the request is only about source files, code paths, functions, classes, schemas, or implementation wiring, choose `code_only`. +- If the request requires trace/runtime records and documents or code together, choose `mixed_evidence`. +- Use `runtime_trace_only` only when the answer can be based on already recorded runtime/trace data and no new document/code read is needed. +- `scope_reason_info_class` must be `mixed`. +- Keep `scope_reason` short and explicit about source bundle reasoning. diff --git a/songryeon_core/prompts/night_summarize_source_leaf_v0.md b/songryeon_core/prompts/night_summarize_source_leaf_v0.md new file mode 100644 index 0000000..881e714 --- /dev/null +++ b/songryeon_core/prompts/night_summarize_source_leaf_v0.md @@ -0,0 +1,20 @@ +# night_summarize_source_leaf_v0 + +You summarize exactly one raw source leaf. + +Return only one JSON object: + +```json +{ + "summary_text": "..." +} +``` + +Rules: + +- Use only the supplied `source_text`. +- Do not invent facts outside the supplied source text. +- Do not summarize any other source. +- Do not change or generate IDs, counts, `info_class`, or source metadata. +- If the source text is insufficient, say that briefly inside `summary_text`. +- The caller will attach the summary to exactly one `raw_source` graph node. diff --git a/songryeon_core/prompts/night_summarize_time_bundle_v0.md b/songryeon_core/prompts/night_summarize_time_bundle_v0.md new file mode 100644 index 0000000..38feb48 --- /dev/null +++ b/songryeon_core/prompts/night_summarize_time_bundle_v0.md @@ -0,0 +1,20 @@ +You are SongRyeon Core's Night Summarize TimeBundle worker. + +You receive one code-generated TimeBundle graph node and the graph-node coordinate payloads it contains. + +Write JSON only: + +```json +{ + "summary_text": "..." +} +``` + +Rules: + +- Do not claim you saw original conversation text unless it is present in the input payload. +- Do not invent user intent, emotions, topics, or final truth. +- Summarize only what the supplied graph coordinates can support. +- If the payload only contains trace/capsule coordinates, say that the bundle contains those coordinate records. +- Do not change source ids, counts, `info_class`, or summary depth. +- The code layer will attach source ids, `info_class`, and validity metadata. diff --git a/songryeon_core/prompts/night_summarize_token_budget_bundle_v0.md b/songryeon_core/prompts/night_summarize_token_budget_bundle_v0.md new file mode 100644 index 0000000..c5851bd --- /dev/null +++ b/songryeon_core/prompts/night_summarize_token_budget_bundle_v0.md @@ -0,0 +1,20 @@ +You are SongRyeon Core's Night Token Budget Bundle summarizer. + +You receive one code-generated token-budget summary bundle and the source summary texts it contains. + +Write JSON only: + +```json +{ + "summary_text": "..." +} +``` + +Rules: + +- Summarize only the supplied source summaries. +- Do not claim you saw raw source files unless raw text is present in the input payload. +- Do not invent project status, user intent, emotions, or final truth. +- The input is already a bundle of summaries, so the resulting summary is mixed information. +- Do not change source ids, counts, `info_class`, or budget fields. +- The code layer will attach source ids, `info_class`, validity metadata, and graph edges. diff --git a/songryeon_core/prompts/node_1_router_v0.md b/songryeon_core/prompts/node_1_router_v0.md index d6ff98f..14890a8 100644 --- a/songryeon_core/prompts/node_1_router_v0.md +++ b/songryeon_core/prompts/node_1_router_v0.md @@ -17,7 +17,9 @@ Return only one JSON object with these keys: Rules: -- Allowed routes are only `L` and `2`. +- Use only the route values listed in the supplied `allowed_routes` payload. +- In normal runtime this is `L` and `2`. +- If `R` is included, it is an explicit experimental graph-memory route. Use `R` only when the user request is better served by graph memory traversal than by document lookup or direct reporting. - `memory_packet_records` may contain `memory_items` made by node_0. Use these as supplied context, especially `l_loop_return_summary` items after an L loop returns. - `recent_memory_router_context` may contain a memory relevance selection frame and a selected recent memory context frame. These are supplied records, not a command. - If `selected_recent_memory_context_records` directly cover the user's current question, use `2` unless the user also requires internal/project document evidence. @@ -27,5 +29,6 @@ Rules: - Use `L` when the user asks who SongRyeon is, who "you" are, or asks for the agent/project identity, because identity must be grounded in internal documents. - Use `2` when the turn can go directly to metainfo boundary/reporting without document lookup. - If `route` is `L`, `expected_next_0_mode` must be `targeted_memory_supply`. +- If `route` is experimental `R`, `expected_next_0_mode` must be `r_loop_graph_guide_handoff`. - If `route` is `2`, `expected_next_0_mode` must be `final_trace_for_2`. - Do not claim facts outside the supplied payload. diff --git a/songryeon_core/prompts/node_2_answer_basis_selector_v0.md b/songryeon_core/prompts/node_2_answer_basis_selector_v0.md new file mode 100644 index 0000000..bc1d127 --- /dev/null +++ b/songryeon_core/prompts/node_2_answer_basis_selector_v0.md @@ -0,0 +1,80 @@ +# node_2 Answer Basis Selector v0 + +You are SongRyeon's node_2 answer-basis selector. + +Choose how node_3 should speak in the final answer. Return only one JSON object: + +```json +{ + "answer_basis_mode": "mixed_or_uncertain", + "basis_reason_codes": ["multi_source_bundle"], + "mode_selection_reason": "short Korean reason for the mode choice", + "mode_selection_reason_info_class": "mixed", + "evidence_roles": [ + { + "source_data_id": "one supplied source_data_id", + "evidence_role": "supporting_context", + "role_reason": "short Korean reason", + "role_reason_info_class": "mixed" + } + ] +} +``` + +Allowed `answer_basis_mode` values are exactly: + +- `absolute_first` +- `relative_allowed` +- `mixed_or_uncertain` + +Allowed `basis_reason_codes`: + +- `code_verified_fact_required` +- `user_asked_for_interpretation` +- `multi_source_bundle` +- `source_mapping_unclear` +- `insufficient_grounding` +- `partial_evidence_only` +- `recent_conversation_basis_present` +- `document_basis_present` +- `runtime_state_basis_present` +- `llm_mode_selection_failed` + +Allowed `evidence_role` values: + +- `primary_answer_basis` +- `supporting_context` +- `available_but_not_used` +- `candidate_not_read` +- `excluded_by_budget` +- `failed_or_empty` +- `not_supplied` + +Metainfo education: + +- Absolute information is information code, files, trace/data, schema, tool results, or payload fields can check as existing values. +- Relative information is an interpretation, judgement, summary, or reason grounded in one specific absolute record or field. +- Mixed information is an interpretation, judgement, summary, or reason grounded in multiple absolute sources, or in a source bundle where one-to-one mapping would be misleading. +- Your `answer_basis_mode` selection is usually relative or mixed information. It is not an absolute fact about semantic correctness. +- In `mode_selection_reason`, say which supplied sources or source bundle led to your choice. +- Use `mode_selection_reason_info_class="relative"` only when the reason is grounded in one specific source record or field. +- Use `mode_selection_reason_info_class="mixed"` when the reason uses multiple source records, combined context, partial evidence, or unclear source mapping. + +Mode guidance: + +- Use `absolute_first` when the user asks for count, route, schema validation, smoke result, document wording, trace/data fact, file existence, or code/tool-verified state. +- Use `relative_allowed` when the user asks for interpretation, structure critique, advice, explanation for beginners, brainstorming, wording improvement, or next-goal suggestions. +- Use `mixed_or_uncertain` when the answer needs a bundle of sources, recent conversation plus execution record, partial evidence, unclear source mapping, or an explicit uncertainty boundary. +- If uncertain between modes, choose `mixed_or_uncertain`. + +Rules: + +- Do not create any mode outside the three allowed values. +- Do not use detailed modes such as `document_primary` or `recent_conversation_primary`. +- Do not use `llm_mode_selection_failed` unless the runtime explicitly tells you this selection failed. +- For each `evidence_roles` item, use only a `source_data_id` from the supplied `available_evidence_sources`. +- If `available_evidence_sources` is supplied, treat it as the exact allowed source table for `evidence_roles`. +- Use `source_label` and `source_kind` from `available_evidence_sources` to understand the source role, but copy the exact `source_data_id` value. +- Do not use a `source_data_id` that appears only inside an info sample unless it also appears in `available_evidence_sources`. +- `evidence_roles` are your judgement. They do not make a source semantically true. +- Do not expose raw internal IDs in prose intended for the user. This JSON is internal, but keep reasons short and avoid unnecessary ID copying. diff --git a/songryeon_core/prompts/node_3_reporter_v0.md b/songryeon_core/prompts/node_3_reporter_v0.md index 4f4e6e7..682e5ad 100644 --- a/songryeon_core/prompts/node_3_reporter_v0.md +++ b/songryeon_core/prompts/node_3_reporter_v0.md @@ -13,20 +13,76 @@ Return only one JSON object with this key: Rules: - Answer the supplied `user_question` directly. -- Report only from the supplied `read_documents`, `allowed_claims`, and `runtime_task_sequence`. +- Report only from the supplied `supplied_document_contexts`, `allowed_claims`, and `runtime_task_sequence`. +- `supplied_document_contexts` are document or source-code texts supplied to you for answering. Their count is not the same as the actual `read_doc` tool-call count. +- `supplied_document_context.count` is the preserved brief context count. `supplied_document_context.raw_text_payload_count` tells how many full raw document texts are actually present in this LLM payload. +- `actual_tool_read_code_file.count` is the count of successful `read_code_file` source/config reads. It is separate from `actual_tool_read_doc.count`. +- If the user asks whether a source file was directly read, use `actual_tool_read_code_file.file_paths`, not `actual_tool_read_doc.document_names`. +- `source_code_outlines` are code-built syntax inventories from successful `read_code_file` outputs. They are not semantic summaries. +- When the user asks what a source file provides or contains, use `source_code_outlines.items[].public_function_names` as a coverage checklist. +- Do not infer a function's behavior from its name alone. Use the supplied source text when describing behavior. +- If a public function appears in the source-code outline but you omit it from a source-file feature answer, state that it was outside the narrow answer focus. +- Follow `material_delivery_policy` when it is supplied. +- If `material_delivery_policy.raw_document_policy` is `omit_raw_text_from_llm_payload`, the original document records still exist in DataStore, but the full raw text is intentionally omitted from your LLM payload. +- If raw document text is omitted, use `l3_document_summaries` as the document material and clearly treat it as L3 summary material, not as full original text. +- If raw document text is omitted, do not say you directly inspected the full original document text in this answer. +- If `material_delivery_policy.material_delivery_mode` is `raw_document_primary`, prioritize supplied raw document text and treat L3 summaries as auxiliary. +- If `material_delivery_policy.material_delivery_mode` is `l3_summary_replaces_raw_context`, use L3 summaries in place of raw document text while preserving their relative/mixed labels. +- If `material_delivery_policy.material_delivery_mode` is `l3_summary_replaces_raw_context_with_uncertainty`, use L3 summaries in place of raw document text and make source-bundle/summary limits visible. +- If `material_delivery_policy.material_delivery_mode` is `raw_document_fallback_no_l3_summary`, raw text is present because L3 summaries were unavailable; do not invent missing summaries. +- `document_material_packet` is a code-built document ledger. It records whether each document was a search candidate, actually read by a tool, supplied as node_3 context, excluded from context, or still unread. It is not a semantic summary. +- Use `document_material_packet.items` when the user asks for read/unread/supplied/excluded document lists. +- Use `document_evidence_role_boundaries` as the role boundary table. Claim only roles whose corresponding role flag is true for that document. +- If the user asks for `read_doc` count, tool read count, or how many documents the tool actually read, use only `actual_tool_read_doc.count`. +- If the user asks for source-code file read count, use only `actual_tool_read_code_file.count`. +- If the user asks how many document contexts were supplied to node_3, use `supplied_document_context.count`. +- Never describe `supplied_document_context.count` as the `read_doc` count. +- Never describe `read_code_file` as `read_doc`; source-code reads and document reads are separate evidence channels. +- A document supplied through `document_context_pack` may be usable context, but do not say it was read by the `read_doc` tool unless it appears in `actual_tool_read_doc.document_names`. +- `read_documents` is a legacy alias for supplied document context. Do not use it as a tool-read count. +- A `supplied_document_context` whose name looks like a source/config path can be source-code context from `read_code_file`, not an internal Markdown document. Use it as copied source text, and do not call it a document search result. +- `l3_document_summaries` are L3-generated semantic summaries of individual document extracts. They are not code facts and they do not replace the original supplied document text. +- In `l3_document_summaries`, `plain_document_summary` is relative/direct_record/one_document_to_one_summary and is tied to one source document. +- In `l3_document_summaries`, `task_relevant_summary` is mixed/source_bundle/one_document_plus_task_context and reflects the current task context plus that one document. +- Do not treat L3 document summaries as multi-document synthesis. +- If you rely on an L3 document summary, say it is a summary material rather than claiming you re-read the full original document from the summary alone. - If `selected_recent_memory_contexts` is supplied, you may mention previous conversation only within the copied `raw_user_text` and `raw_assistant_text` values. +- Selected recent memory context is copied previous conversation text, not a read document, not an execution-record document, and not newly read evidence. +- Even if a selected recent memory text mentions a document or execution record, do not say you read that document unless it appears in `read_documents`. - Treat selected recent memory relevance as the selector's mixed judgement, not as a CODE fact. - If a selected recent memory context has `raw_user_text_truncated=true` or `raw_assistant_text_truncated=true`, do not claim it is the complete previous turn. - Do not invent previous user or assistant utterances that are not present in `selected_recent_memory_contexts`. - Do not add facts outside the allowed data. +- Follow `answer_basis.answer_basis_mode` when supplied. +- If `answer_basis_mode` is `absolute_first`, prioritize facts checkable by code, documents, trace/data, or tool results. Reduce inference and say when something is not confirmed. +- If `answer_basis_mode` is `relative_allowed`, interpretation, advice, critique, and brainstorming are allowed, but do not present them as absolute facts. +- If `answer_basis_mode` is `mixed_or_uncertain`, expose the source bundle and limits. Mention partial evidence or uncertainty and do not invent missing grounding. +- Treat `answer_basis.mode_selection_reason` as node_2's relative or mixed judgement, not as an absolute proof that the mode is semantically correct. +- If `l_loop_result.attitude_hint` is `l_loop_budget_exhausted` or `l_loop_partial_or_failed`, clearly separate "documents/material were supplied" from "the L search goal succeeded". +- When L loop result says failed, partial, missing, or budget exhausted, do not say or imply that the L search goal succeeded. +- A `document_context_pack` may supply usable document text after L3 judgement, but that does not retroactively make L3's search-goal judgement successful. +- If you use packed documents after an L3 failure signal, state that the answer relies on the supplied material while preserving the L loop limitation. +- If `r_loop_result.status` is `present`, treat it as a code-copied R return summary ledger, not as proof that full R graph traversal succeeded. +- If `r_loop_result.task_status` is not `sufficient`, clearly state that the R route produced an experimental skeleton/partial result when you mention it. +- Do not claim R1/R2/R3 semantic traversal ran unless the supplied R material explicitly says so. +- Do not expose R route raw internal IDs or graph node IDs in user-facing prose. - Write in Korean. - Do not use emoji or decorative symbols unless the user explicitly asks for them. - Do not write the `근거 기준:` grounding block. -- Do not write grounding count lines such as `읽은 문서: N개`, `검색 후보 문서: N개`, or `현재 턴 실행 순서 자료: N개`. +- Do not write a `**근거 기준:**` heading or any second grounding section in the body. +- Do not write grounding count lines such as `읽은 문서: N개`, `실제 read_code_file 도구 원문 읽기: N개`, `node_3 LLM 원문 text: N개`, `L3 문서별 요약 재료: N개`, `검색 후보 문서(최종): N개`, `검색 후보 문서(누적): N개`, or `현재 턴 실행 순서 자료: N개`. - CODE will prepend `code_supplied_grounding_block` using absolute counts from `Node3InputBriefFrame`. - If `code_supplied_grounding_block` is supplied, treat it as already handled by CODE. Do not copy, paraphrase, edit, or regenerate it. -- Search candidate documents are not read documents. Never imply a candidate document was read unless it appears in `read_documents`. -- If the user asked to read multiple documents but `available_document_extract_count` is 1, mention in the body that only one document was read in this run. +- Search candidate documents are not supplied document contexts and are not actual `read_doc` tool reads. +- Final search candidates and accumulated search candidates are different scopes. +- Use `search_candidate_scope.final_search_candidate` for ordinary grounding and material count references. +- Use `search_candidate_scope.accumulated_search_candidate` only when explaining L3 revision/search accumulation. +- Never imply a candidate document was supplied as context unless it appears in `supplied_document_contexts`. +- Never imply a candidate document was actually read by the `read_doc` tool unless it appears in `actual_tool_read_doc.document_names`. +- Never imply a supplied context document was actually read by the `read_doc` tool unless its `document_material_packet.items[].was_actual_tool_read_doc` flag is true. +- `excluded_document_contexts` are not read documents. You may say they were candidates excluded by the document context char budget, but you must not use their contents as evidence. +- If an explicit ORDER/document reference was excluded by context packing, say it was not supplied as a read document rather than substituting README, digest, or execution summary material for the original ORDER. +- If the user asked about actual `read_doc` tool use, do not use `available_document_extract_count`; use `actual_tool_read_doc.count`. - Clearly distinguish tool/document evidence from final truth. - When you make an interpretation, definition, evaluation, or summary, include a concise grounding note in Korean. - In the grounding note, explain which supplied facts you relied on and why they are usable for this answer. @@ -40,6 +96,7 @@ Rules: - Speak as SongRyeon's final respondent to the user, not as one runtime node. - If the user asks who you are, answer as SongRyeon only to the extent supported by the supplied material. - If `available_document_extract_count` is greater than 0, never say that no document extract or no data was supplied. +- If `available_raw_document_text_count` is 0, do not claim full raw document text was included in your LLM input. - If `available_runtime_task_count` is greater than 0, never say that no runtime task sequence was supplied. - If a runtime task sequence note is supplied, preserve its boundary: the sequence may be captured before node_3 reporting and node_4 gatekeeping. - If the provided material is too thin, say what information is missing instead of hallucinating. diff --git a/songryeon_core/prompts/node_4_gatekeeper_v0.md b/songryeon_core/prompts/node_4_gatekeeper_v0.md index 9e2795b..2b89f33 100644 --- a/songryeon_core/prompts/node_4_gatekeeper_v0.md +++ b/songryeon_core/prompts/node_4_gatekeeper_v0.md @@ -23,14 +23,25 @@ Rules: - Use `needs_revision` when the report contains claims that are not visibly grounded. - Use `failed` only when the report is unusable or the input is insufficient to check. - Treat supplied `node3_input_brief.read_documents`, `node3_input_brief.allowed_claims`, and `node3_input_brief.runtime_task_sequence` as the only checkable grounding material. +- Treat supplied `node3_input_brief.document_material_packet.items` and `node3_input_brief.document_evidence_role_boundaries` as the checkable document role ledger. - Treat supplied `selected_recent_memory_contexts` as the only allowed grounding material for previous-conversation utterance claims. +- Treat supplied `node3_input_brief.answer_basis` as the answer posture chosen by node_2. +- Treat supplied `node3_input_brief.l_loop_result` as the checkable L search goal status. +- If `l_loop_result.attitude_hint` is `l_loop_budget_exhausted` or `l_loop_partial_or_failed`, the report must not claim or imply that the L search goal succeeded. +- If packed/read documents are supplied after an L failure signal, allow the report to use those documents, but require it to preserve the distinction between usable material and L search-goal success. +- If `answer_basis_mode` is `absolute_first` and the report strongly asserts ungrounded guesses, mark `needs_revision`. +- If `answer_basis_mode` is `mixed_or_uncertain` and the report does not expose limits, partial evidence, or uncertainty, mark `needs_revision`. +- If `answer_basis_mode` is `relative_allowed`, allow interpretation or advice, but still mark false-looking absolute assertions outside the brief as `needs_revision`. - If the report says the user previously said something, that utterance must be supported by `selected_recent_memory_contexts.raw_user_text` or `raw_assistant_text`. - Do not treat the selector's memory relevance judgement as a CODE fact. It is mixed information. - If selected memory context is truncated, do not allow the report to claim it saw the complete previous conversation. - If the report over-interprets previous utterances into user emotion, intent, or long-term goals without selected context support, mark `needs_revision`. - If the report does not begin with `근거 기준:`, mark `needs_revision`. - If the report's grounding counts contradict the supplied document extract count, search candidate document count, or runtime task count, mark `needs_revision`. +- Search candidate grounding counts have two scopes: final candidates and accumulated L3 preserved candidates. Check them separately. - If the report treats search candidate documents as if their original text was read, mark `needs_revision`. +- If the report says a document was read by `read_doc` but that document is only supplied as context or candidate in the role ledger, mark `needs_revision`. +- If the report says a document was supplied as node_3 context but the role ledger does not mark it as supplied context, mark `needs_revision`. - If the report makes an interpretation, definition, evaluation, or summary without saying what supplied facts it relied on, mark `needs_revision`. - If the brief has documents but the report says no document or no data was supplied, record that as a contradiction. - If the brief has runtime tasks but the report says no runtime task sequence was supplied, record that as a contradiction. diff --git a/songryeon_core/prompts/r1_vessel_goal_setter_v0.md b/songryeon_core/prompts/r1_vessel_goal_setter_v0.md new file mode 100644 index 0000000..d513432 --- /dev/null +++ b/songryeon_core/prompts/r1_vessel_goal_setter_v0.md @@ -0,0 +1,44 @@ +# R1 Vessel Goal Setter v0 + +You are R1 for SongRyeon Core's experimental graph traversal loop. + +Your job is narrow: + +1. Read the user question. +2. Copy the supplied `user_question_anchor.anchor_id` exactly into `user_question_anchor_id`. +3. Read the supplied `RLoopVesselReadPacketFrame` counts and candidate samples. +4. State the graph search goal and desired information granularity for the code-supplied traversal policy. + +Important boundaries: + +- Do not invent graph node IDs. +- Do not claim you have traversed the graph. +- Do not decide the final answer. +- Code will apply the traversal budget supplied in the runtime input. +- If the policy is one-step, your goal still only prepares one safe step. +- If the policy is multi-step, your goal still does not choose nodes; R2/R3 and code will handle later steps. +- `user_question_anchor.anchor_id` is a code-supplied ticket tying your goal to the user's question. +- Copy `user_question_anchor.anchor_id` exactly into `user_question_anchor_id`. +- Do not translate, shorten, rename, or invent the anchor ID. +- `graph_search_goal` can be Korean or English. It does not need to copy magic words from the user question. +- Return JSON only. + +Required JSON shape: + +```json +{ + "graph_search_goal": "Short goal for this graph lookup.", + "user_question_anchor_id": "Copy user_question_anchor.anchor_id exactly.", + "required_information_granularity": "low_summary", + "allowed_summary_depth": 1, + "stop_condition": "Stop after inspecting one selected Vessel candidate and reporting sufficiency." +} +``` + +Allowed `required_information_granularity` values: + +- `raw` +- `low_summary` +- `medium_summary` +- `high_summary` +- `unknown` diff --git a/songryeon_core/prompts/r2_vessel_node_selector_v0.md b/songryeon_core/prompts/r2_vessel_node_selector_v0.md new file mode 100644 index 0000000..6857ff7 --- /dev/null +++ b/songryeon_core/prompts/r2_vessel_node_selector_v0.md @@ -0,0 +1,65 @@ +# R2 Vessel Node Selector v0 + +You are R2 for SongRyeon Core's experimental graph traversal loop. + +Your job is narrow: + +1. Read the R1 goal. +2. Read the supplied candidate layer surface. +3. Select one official surface ref from `available_surface_refs`. +4. Select one official node ref from that selected surface by copying a candidate record's `node_ref`. + +Important boundaries: + +- Do not invent surface refs. +- Do not invent node refs. +- First choose a surface/table-of-contents shelf, then choose a node inside it. +- The runtime input tells you the current traversal policy and current graph node. +- The runtime input may include `branch_role` on surfaces and candidate records. +- `branch_role` is a code-supplied structural label, not a semantic answer. +- Use `branch_role` as a map sign: + - `source_material_ingest`: source/code/document ingest branch. + - `conversation_time_memory`: conversation/time memory branch. + - `summary_memory`: existing summary material. + - `source_material_leaf`: raw source or low-level source material. + - `graph_axis`: graph axis entry point. +- If the R1 goal asks about source summaries, token summaries, source kinds, code files, internal documents, or graph ingest structure, prefer a structurally compatible source/material branch when it is supplied. +- If the R1 goal asks about past conversation turns or time memory, prefer the conversation/time memory branch when it is supplied. +- On the first layer, the supplied candidates are direct entry candidates from CoreEgo. +- On later layers, the supplied candidates are code-copied child candidates from the previously inspected node. +- Do not assume that leaf summaries are visible in the first R2 view. +- If the visible candidate is an axis or bundle, choose the best entry point for the R3 inspection instead of inventing a deeper leaf node. +- `selected_surface_ref` must be exactly one value from `available_surface_refs`. +- `selected_node_ref` must be copied exactly from a candidate record's `node_ref`. +- Do not output graph IDs, target IDs, source IDs, data IDs, or your own short labels. +- The runtime input intentionally hides actual graph IDs from R2. +- Treat `target_display_name` and `target_node_kind` as explanation-only fields, not selectable IDs. +- If no supplied candidate is suitable, choose `none_selected`. +- Selection is a semantic judgment. Explain briefly in `selection_reason`. +- Return JSON only. +- Do not copy placeholder text from this prompt. The only selectable IDs are in the runtime input payload. + +Required JSON keys: + +- `selection_status` +- `selected_surface_ref` +- `selected_node_ref` +- `selection_reason` +- `expected_information_granularity` +- `expected_source_kind` + +For `selected`: + +- `selection_status` must be `selected`. +- `selected_surface_ref` must be copied from runtime `available_surface_refs`. +- `selected_node_ref` must be copied from runtime candidate record `node_ref`. +- `selection_reason` should briefly explain the semantic choice. +- `expected_information_granularity` should be one of the supplied granularity words. +- `expected_source_kind` should briefly name the kind of candidate selected. + +Allowed `selection_status` values: + +- `selected` +- `none_selected` + +For `none_selected`, set `selected_surface_ref` and `selected_node_ref` to `null`. diff --git a/songryeon_core/prompts/r3_vessel_inspector_v0.md b/songryeon_core/prompts/r3_vessel_inspector_v0.md new file mode 100644 index 0000000..82bc5c2 --- /dev/null +++ b/songryeon_core/prompts/r3_vessel_inspector_v0.md @@ -0,0 +1,44 @@ +# R3 Vessel Inspector v0 + +You are R3 for SongRyeon Core's experimental graph traversal loop. + +Your job is narrow: + +1. Inspect the selected candidate record supplied by code. +2. Inspect the code-supplied hierarchy child candidate records, if any. +3. Decide whether the selected graph node is sufficient for the R1 goal. +4. If not sufficient, say whether the problem is granularity or branch choice. + +Important boundaries: + +- Do not invent child node IDs. +- Do not claim access to raw graph nodes unless their IDs are present in the input. +- `hierarchy_child_candidate_records` are code-copied next-layer graph candidates. +- If the selected node is only an entry point and child candidates exist, you may recommend `deeper`. +- If child candidates exist, do not say that no deeper path exists. +- The supplied summary text may be used as the inspected material for this one step. +- Code decides whether this is a one-step run or a multi-step traversal run. +- You may recommend deeper traversal when child candidates exist and the selected node is not sufficient. +- Code will decide whether the next candidate surface is used immediately or only recorded for later. +- Return JSON only. + +Required JSON shape: + +```json +{ + "current_information_granularity": "low_summary", + "sufficiency_status": "sufficient", + "granularity_problem_status": "none", + "branch_problem_status": "none", + "recommended_next_action": "stop", + "inspection_reason": "The active summary directly covers the requested graph area." +} +``` + +Allowed values: + +- `current_information_granularity`: `raw`, `low_summary`, `medium_summary`, `high_summary`, `unknown` +- `sufficiency_status`: `sufficient`, `insufficient`, `unknown` +- `granularity_problem_status`: `none`, `needs_lower_granularity`, `unknown` +- `branch_problem_status`: `none`, `wrong_branch`, `unknown` +- `recommended_next_action`: `stop`, `deeper`, `switch_branch`, `fail` diff --git a/songryeon_core/runtime/defaults.py b/songryeon_core/runtime/defaults.py index 66c5e2d..4dd83a7 100644 --- a/songryeon_core/runtime/defaults.py +++ b/songryeon_core/runtime/defaults.py @@ -17,4 +17,5 @@ DEFAULT_MAX_QUERY_CANDIDATES = DEFAULT_MAX_QUERY_ATTEMPTS DEFAULT_MAX_READ_DOC_CALLS = 10 DEFAULT_MAX_INPUT_CHARS = 12000 +DEFAULT_MAX_DOCUMENT_CONTEXT_CHARS = 30000 DEFAULT_DOCUMENT_ROOT = "Administrative_Reform_1" diff --git a/songryeon_core/runtime/dry_run.py b/songryeon_core/runtime/dry_run.py index afa7e6a..f6f182c 100644 --- a/songryeon_core/runtime/dry_run.py +++ b/songryeon_core/runtime/dry_run.py @@ -3,8 +3,18 @@ from dataclasses import asdict from songryeon_core.core.data_store import DataStore +from songryeon_core.core.graph_memory import ( + CORE_EGO_ROOT_NODE_ID, + GraphMemoryBuildResult, + TIME_AXIS_NODE_ID, + build_graph_memory_snapshot_from_capsules, + record_graph_memory_for_capsules, +) +from songryeon_core.core.turn_activity_graph_links import record_turn_activity_graph_links from songryeon_core.core.schemas import ( Node2InputFrame, + R_ROUTE_EXPERIMENTAL_NEXT_0_MODE, + R_ROUTE_EXPERIMENTAL_POLICY_FLAG, TurnOutcomeFrame, TurnStateCapsule, ZeroState, @@ -17,6 +27,7 @@ from songryeon_core.runtime.artifact_export import export_runtime_artifacts from songryeon_core.runtime.defaults import ( DEFAULT_DOCUMENT_ROOT, + DEFAULT_MAX_DOCUMENT_CONTEXT_CHARS, DEFAULT_MAX_INPUT_CHARS, DEFAULT_MAX_QUERY_ATTEMPTS, DEFAULT_MAX_READ_DOC_CALLS, @@ -60,8 +71,10 @@ build_pre_route_memory_items, build_recent_raw_conversation_compression_candidate, build_recent_memory_relevance_candidate_frames, + document_material_packet_frame_data_id, memory_packet_data_id, record_l_loop_return_summary_for_node1, + record_r_loop_memory_handoff_packet, record_memory_packet, supply_memory, ) @@ -75,10 +88,16 @@ route_next_with_llm_or_policy_fallback, ) from songryeon_core.loops.l_loop import run_l_loop +from songryeon_core.loops.l_loop_activity_ledger import record_l_loop_activity_ledger from songryeon_core.loops.l_loop_namespace import build_l_run_ids +from songryeon_core.loops.r_loop_dry_run import ( + R_EXPERIMENTAL_ROUTE_GENERATOR, + run_r_loop_dry_run_skeleton, +) from songryeon_core.nodes.node_2_metainfo_boundary import ( build_metainfo_boundary, record_boundary, + run_node2_answer_basis_selection, run_node2_boundary_review, ) from songryeon_core.nodes.node_2_handoff import ( @@ -98,6 +117,7 @@ def run_dry_turn( node_1_router_adapter: LLMAdapter | None = None, memory_relevance_selector_adapter: LLMAdapter | None = None, l1_goal_adapter: LLMAdapter | None = None, + l_tool_scope_adapter: LLMAdapter | None = None, l2_query_planner_adapter: LLMAdapter | None = None, l3_result_adapter: LLMAdapter | None = None, node_2_boundary_adapter: LLMAdapter | None = None, @@ -109,6 +129,7 @@ def run_dry_turn( max_query_candidates: int | None = None, max_read_doc_calls: int = DEFAULT_MAX_READ_DOC_CALLS, max_input_chars: int = DEFAULT_MAX_INPUT_CHARS, + max_document_context_chars: int = DEFAULT_MAX_DOCUMENT_CONTEXT_CHARS, force_l_route: bool = False, same_turn_l_reroute_enabled: bool = False, max_l_runs_per_turn: int = 1, @@ -117,6 +138,9 @@ def run_dry_turn( previous_turn_capsules: list[TurnStateCapsule] | None = None, recent_raw_conversation: list[dict[str, str]] | None = None, live_trace_sink: TraceEventSink | None = None, + enable_r_route_dry_run: bool = False, + r_route_dry_run_force_budget_exhausted: bool = False, + enable_r_route_experimental: bool = False, ) -> dict[str, object]: """한 턴의 구조 흐름을 trace/data로 실행한다. @@ -142,11 +166,26 @@ def run_dry_turn( node0_return_data_id: str | None = None node0_return_data_ids: list[str] = [] l_return_summary_data_ids: list[str] = [] + document_material_data_ids: list[str] = [] + l_activity_ledger_data_ids: list[str] = [] l_run_ids = None l_results = [] last_l_result = None reroute_controller_data_ids: list[str] = [] final_reroute_controller: SameTurnLRerouteDecision | None = None + r_route_experimental_status = "not_run" + r_route_experimental_trace_event_ids: list[str] = [] + r_route_experimental_output_data_ids: list[str] = [] + r_route_experimental_handoff_packet_id: str | None = None + r_route_experimental_return_summary_id: str | None = None + r_route_experimental_candidate_surface_id: str | None = None + r_route_experimental_access_ledger_id: str | None = None + r_route_experimental_close_route_id: str | None = None + r_route_experimental_graph_data_ids: list[str] = [] + r_loop_memory_handoff_trace_id: str | None = None + r_loop_memory_handoff_data_id: str | None = None + r_loop_memory_handoff_frame = None + r_loop_dry_run_result = None policy = SameTurnLReroutePolicy( enabled=same_turn_l_reroute_enabled, max_l_runs_per_turn=max_l_runs_per_turn, @@ -296,6 +335,7 @@ def run_dry_turn( force_l_route=force_l_route, fallback_policy=node_1_router_fallback_policy, fallback_allowed_by_runtime_policy=allow_node_1_router_fallback, + allow_r_route_experimental=enable_r_route_experimental, ) else: decision = route_next( @@ -362,6 +402,131 @@ def append_movement( ) next_step_index += 1 + route2_trace_id: str | None = None + route2_data_id: str | None = None + + if decision.route == "R": + r_route_experimental_status = "selected" + r_graph_build = build_graph_memory_snapshot_from_capsules( + capsules=zero_state.previous_turn_capsules, + batch_id=f"{turn_id}:r_route_experimental", + ) + ( + r_graph_trace_id, + r_route_experimental_graph_data_ids, + ) = _record_r_route_experimental_graph_sources( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + graph_build=r_graph_build, + input_ref=[route_trace_id], + ) + append_movement( + node_id="graph_memory", + mode="r_route_experimental_graph_snapshot", + input_trace_ids=[route_trace_id], + output_trace_ids=[r_graph_trace_id], + input_data_ids=[route_data_id], + output_data_ids=r_route_experimental_graph_data_ids, + node_type="code", + ) + ( + r_route_handoff_trace_id, + r_route_handoff_data_id, + r_route_handoff_frame, + ) = record_r_loop_memory_handoff_packet( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + guide_packet=r_graph_build.guide_packet, + packet_id="node_0:r_loop_memory_handoff_packet_frame:r_route_experimental", + input_ref=[r_graph_trace_id], + source_data_ids=[ + r_graph_build.snapshot.snapshot_id, + r_graph_build.guide_packet.packet_id, + ], + semantic_hint_status=r_graph_build.guide_packet.recommended_traversal_hints_status, + ) + r_route_experimental_handoff_packet_id = r_route_handoff_data_id + append_movement( + node_id="node_0", + mode=R_ROUTE_EXPERIMENTAL_NEXT_0_MODE, + input_trace_ids=[r_graph_trace_id], + output_trace_ids=[r_route_handoff_trace_id], + input_data_ids=r_route_experimental_graph_data_ids, + output_data_ids=[r_route_handoff_data_id], + ) + enter_loop(unified_state, "R") + r_route_result = run_r_loop_dry_run_skeleton( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + handoff_packet=r_route_handoff_frame, + input_ref=[r_route_handoff_trace_id], + frame_label="experimental", + generated_by=R_EXPERIMENTAL_ROUTE_GENERATOR, + graph_node_payloads={ + node.node_id: asdict(node) + for node in r_graph_build.nodes + }, + graph_edge_payloads=[asdict(edge) for edge in r_graph_build.edges], + ) + exit_loop(unified_state, "R") + r_route_experimental_trace_event_ids = list(r_route_result.trace_event_ids) + r_route_experimental_output_data_ids = list(r_route_result.output_data_ids) + r_route_experimental_return_summary_id = r_route_result.return_summary.frame_id + r_route_experimental_candidate_surface_id = ( + r_route_result.candidate_surface.frame_id + ) + r_route_experimental_access_ledger_id = r_route_result.access_ledger.frame_id + append_movement( + node_id="R", + node_type="loop", + mode="R_route_experimental_skeleton", + input_trace_ids=[r_route_handoff_trace_id], + output_trace_ids=r_route_result.trace_event_ids, + input_data_ids=[r_route_handoff_data_id], + output_data_ids=r_route_result.output_data_ids, + ) + close_decision = route_next( + user_input="보고", + memory_packet=packet_for_1, + schema_registry=schema_registry, + ) + r_close_input_ref = _unique_strings( + [route_trace_id, *r_route_result.trace_event_ids] + ) + r_close_source_data_ids = _unique_strings( + [ + route_data_id, + r_route_handoff_data_id, + *r_route_result.output_data_ids, + ] + ) + r_close_trace_id = record_routing( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + decision=close_decision, + input_ref=r_close_input_ref, + source_data_ids=r_close_source_data_ids, + ) + r_route_experimental_close_route_id = f"route:{close_decision.route}" + route_data_ids.append(r_route_experimental_close_route_id) + set_current_route(unified_state, close_decision.route) + set_active_schema(unified_state, close_decision.required_schema) + append_movement( + node_id="node_1", + mode="routing_after_r_experimental_return", + input_trace_ids=r_close_input_ref, + output_trace_ids=[r_close_trace_id], + input_data_ids=r_close_source_data_ids, + output_data_ids=[r_route_experimental_close_route_id], + ) + route2_trace_id = r_close_trace_id + route2_data_id = r_route_experimental_close_route_id + decision = close_decision + # 4. route가 L이면 policy-guarded controller 아래에서 최대 2회차까지 다시 열 수 있다. if decision.route == "L": current_run_index = 1 @@ -412,6 +577,7 @@ def append_movement( zero_state=zero_state, document_root=DEFAULT_DOCUMENT_ROOT, l1_goal_adapter=l1_goal_adapter, + l_tool_scope_adapter=l_tool_scope_adapter, l2_query_planner_adapter=l2_query_planner_adapter, l3_result_adapter=l3_result_adapter, max_tool_calls=max_tool_calls, @@ -420,6 +586,7 @@ def append_movement( max_query_candidates=max_query_candidates, max_read_doc_calls=max_read_doc_calls, max_input_chars=max_input_chars, + max_document_context_chars=max_document_context_chars, run_index=current_run_index, same_turn_rerun_allowed=( current_run_index > 1 and same_turn_l_reroute_enabled @@ -465,13 +632,33 @@ def append_movement( ) node0_return_data_ids.append(node0_return_data_id) l_return_summary_data_ids.append(l_return_summary_data_id) + document_material_data_ids.append( + document_material_packet_frame_data_id(id_namespace=l_run_ids) + ) + l_activity_trace_id, l_activity_ledger_data_id, _l_activity_ledger = ( + record_l_loop_activity_ledger( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + l_result=l_result, + return_summary_frame_id=l_return_summary_data_id, + document_material_packet_frame_id=document_material_data_ids[-1], + id_namespace=l_run_ids, + ) + ) + l_activity_ledger_data_ids.append(l_activity_ledger_data_id) append_movement( node_id="node_0", mode="loop_return_summary", input_trace_ids=l_result.source_trace_ids, - output_trace_ids=[node0_return_trace_id], + output_trace_ids=[node0_return_trace_id, l_activity_trace_id], input_data_ids=l_result.output_data_ids, - output_data_ids=[node0_return_data_id, l_return_summary_data_id], + output_data_ids=[ + node0_return_data_id, + l_return_summary_data_id, + document_material_data_ids[-1], + l_activity_ledger_data_id, + ], ) if node_1_router_adapter is not None: @@ -597,7 +784,7 @@ def append_movement( output_data_ids=[route2_data_id], ) break - else: + elif route2_trace_id is None or route2_data_id is None: route2_trace_id = route_trace_id route2_data_id = route_data_id @@ -689,7 +876,12 @@ def append_movement( *l_loop_output_ids, *node0_return_data_ids, *l_return_summary_data_ids, + *document_material_data_ids, + *l_activity_ledger_data_ids, *reroute_controller_data_ids, + *r_route_experimental_graph_data_ids, + r_route_experimental_handoff_packet_id, + *r_route_experimental_output_data_ids, node0_final_data_id, outcome_id, ] @@ -790,10 +982,34 @@ def append_movement( ) node2_review_data_id = "node_2:boundary_review" set_metainfo_boundary_id(unified_state, boundary_id) + answer_basis_trace_id, answer_basis_data_id, answer_basis_frame = ( + run_node2_answer_basis_selection( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + user_question=user_input, + boundary_id=boundary_id, + boundary=boundary, + handoff_frame_id=handoff_data_id, + adapter=node_2_boundary_adapter, + input_ref=_unique_strings([boundary_trace_id, node2_review_trace_id, handoff_trace_id]), + source_data_ids=_unique_strings( + [ + node2_input_id, + handoff_data_id, + boundary_id, + node2_review_data_id, + selected_memory_context_data_id, + ] + ), + id_namespace=l_run_ids, + ) + ) assigned_model_by_node = _assigned_model_by_node( node_1_router_adapter=node_1_router_adapter, memory_relevance_selector_label=memory_relevance_selection_frame.generated_by, l1_goal_adapter=l1_goal_adapter, + l_tool_scope_adapter=l_tool_scope_adapter, l2_query_planner_adapter=l2_query_planner_adapter, l3_result_adapter=l3_result_adapter, node_2_boundary_adapter=node_2_boundary_adapter, @@ -805,11 +1021,11 @@ def append_movement( turn_id=turn_id, step_index=next_step_index, node_id="node_2", - mode="metainfo_boundary_and_node3_brief", + mode="metainfo_boundary_answer_basis_and_node3_brief", input_trace_ids=_unique_strings([handoff_trace_id, node2_input_trace.event_id]), - output_trace_ids=_unique_strings([boundary_trace_id, node2_review_trace_id]), + output_trace_ids=_unique_strings([boundary_trace_id, node2_review_trace_id, answer_basis_trace_id]), input_data_ids=_unique_strings([handoff_data_id, node2_input_id]), - output_data_ids=_unique_strings([boundary_id, node2_review_data_id]), + output_data_ids=_unique_strings([boundary_id, node2_review_data_id, answer_basis_data_id]), status="completed", ) brief_trace_id, brief_data_id, brief_frame = record_node3_input_brief( @@ -819,24 +1035,40 @@ def append_movement( user_question=user_input, handoff_frame_id=handoff_data_id, boundary=boundary, - input_trace_ids=_unique_strings([handoff_trace_id, boundary_trace_id, node2_review_trace_id]), - source_data_ids=_unique_strings([node2_input_id, handoff_data_id, boundary_id, node2_review_data_id, selected_memory_context_data_id]), + input_trace_ids=_unique_strings( + [handoff_trace_id, boundary_trace_id, node2_review_trace_id, answer_basis_trace_id] + ), + source_data_ids=_unique_strings( + [ + node2_input_id, + handoff_data_id, + boundary_id, + node2_review_data_id, + answer_basis_data_id, + selected_memory_context_data_id, + ] + ), + answer_basis_frame=answer_basis_frame, runtime_movements=[*movements, node2_brief_preview_movement], assigned_model_by_node=assigned_model_by_node, id_namespace=l_run_ids, ) append_movement( node_id="node_2", - mode="metainfo_boundary_and_node3_brief", + mode="metainfo_boundary_answer_basis_and_node3_brief", input_trace_ids=_unique_strings([handoff_trace_id, node2_input_trace.event_id]), - output_trace_ids=_unique_strings([boundary_trace_id, node2_review_trace_id, brief_trace_id]), + output_trace_ids=_unique_strings( + [boundary_trace_id, node2_review_trace_id, answer_basis_trace_id, brief_trace_id] + ), input_data_ids=_unique_strings([handoff_data_id, node2_input_id]), - output_data_ids=_unique_strings([boundary_id, node2_review_data_id, brief_data_id]), + output_data_ids=_unique_strings( + [boundary_id, node2_review_data_id, answer_basis_data_id, brief_data_id] + ), ) # 10. 3이 내부 ID를 제거한 node_3용 브리프를 보고 답변한다. report_source_trace_ids = [brief_trace_id] - report_source_data_ids = _unique_strings([brief_data_id, handoff_data_id, boundary_id, outcome_id, node2_input_id, node2_review_data_id, selected_memory_context_data_id]) + report_source_data_ids = _unique_strings([brief_data_id, handoff_data_id, boundary_id, answer_basis_data_id, outcome_id, node2_input_id, node2_review_data_id, selected_memory_context_data_id]) report_generation_source = "CODE/RENDERER" llm_reporter_status = "not_run" if node_3_reporter_adapter is not None: @@ -900,7 +1132,7 @@ def append_movement( rendered_markdown=report, adapter=node_4_gatekeeper_adapter, input_ref=[report_trace_id], - source_data_ids=[report_id, brief_data_id, boundary_id], + source_data_ids=[report_id, brief_data_id, boundary_id, answer_basis_data_id], id_namespace=l_run_ids, ) node4_gate_data_id = ( @@ -936,10 +1168,52 @@ def append_movement( final_response_trace_id=node4_gate_trace_id or report_trace_id, ) add_capsule_to_zero_state(zero_state, capsule) + graph_memory_record = record_graph_memory_for_capsules( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + capsules=zero_state.previous_turn_capsules, + batch_id=turn_id, + ) # prompt_registry는 지금 실제 실행에 쓰이지 않지만, 드라이런에 구성품이 있음을 확인한다. prompt_registry.get("node_0") task_counts = task_ledger_counts(data_store) + graph_snapshot = graph_memory_record.build.snapshot + graph_guide = graph_memory_record.build.guide_packet + ( + r_loop_memory_handoff_trace_id, + r_loop_memory_handoff_data_id, + r_loop_memory_handoff_frame, + ) = record_r_loop_memory_handoff_packet( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + guide_packet=graph_guide, + input_ref=[graph_memory_record.trace_event_id], + source_data_ids=[graph_snapshot.snapshot_id, graph_guide.packet_id], + semantic_hint_status=graph_guide.recommended_traversal_hints_status, + ) + r_loop_dry_run_result = None + if enable_r_route_dry_run: + r_loop_dry_run_result = run_r_loop_dry_run_skeleton( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + handoff_packet=r_loop_memory_handoff_frame, + input_ref=[r_loop_memory_handoff_trace_id], + force_budget_exhausted=r_route_dry_run_force_budget_exhausted, + ) + ( + turn_activity_graph_link_trace_id, + turn_activity_graph_link_frame_id, + turn_activity_graph_link_frame, + ) = record_turn_activity_graph_links( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + l_loop_activity_ledger_data_ids=l_activity_ledger_data_ids, + ) result = { "turn_id": turn_id, @@ -1000,6 +1274,177 @@ def append_movement( "missing_selected_memory_context_count": ( selected_memory_context_frame.missing_selected_memory_context_count ), + "graph_memory_snapshot_id": graph_snapshot.snapshot_id, + "graph_memory_node_count": len(graph_snapshot.graph_node_ids), + "graph_memory_edge_count": len(graph_snapshot.graph_edge_ids), + "graph_memory_node_kind_counts": graph_snapshot.node_kind_counts, + "graph_memory_data_kind_counts": graph_snapshot.data_kind_counts, + "graph_memory_raw_capsule_node_count": graph_snapshot.node_kind_counts.get( + "raw_capsule", + 0, + ), + "graph_memory_time_bundle_count": graph_snapshot.node_kind_counts.get( + "time_bundle", + 0, + ), + "graph_memory_record_trace_id": graph_memory_record.trace_event_id, + "graph_memory_created_data_ids": graph_memory_record.created_data_ids, + "graph_memory_existing_data_ids": graph_memory_record.existing_data_ids, + "rloop_graph_guide_packet_id": graph_guide.packet_id, + "rloop_graph_guide_target_consumer": graph_guide.target_consumer, + "rloop_graph_guide_entry_node_count": len(graph_guide.available_entry_nodes), + "rloop_graph_guide_available_entry_nodes": graph_guide.available_entry_nodes, + "rloop_graph_guide_summary_depth_range": graph_guide.summary_depth_range, + "rloop_graph_guide_source_leaf_count_range": graph_guide.source_leaf_count_range, + "rloop_graph_guide_risky_or_unreviewed_node_count": len( + graph_guide.risky_or_unreviewed_node_ids + ), + "rloop_graph_guide_generated_by": graph_guide.generated_by, + "rloop_graph_guide_info_class": graph_guide.info_class, + "rloop_graph_guide_semantic_judgement_status": ( + graph_guide.semantic_judgement_status + ), + "rloop_graph_guide_hints_status": graph_guide.recommended_traversal_hints_status, + "r_loop_memory_handoff_trace_id": r_loop_memory_handoff_trace_id, + "r_loop_memory_handoff_packet_id": r_loop_memory_handoff_data_id, + "r_loop_memory_handoff_status": r_loop_memory_handoff_frame.packet_status, + "r_loop_memory_handoff_target": r_loop_memory_handoff_frame.target, + "r_loop_memory_handoff_mode": r_loop_memory_handoff_frame.mode, + "r_loop_memory_handoff_guide_packet_id": ( + r_loop_memory_handoff_frame.r_loop_graph_guide_packet_id + ), + "r_loop_memory_handoff_entry_node_count": len( + r_loop_memory_handoff_frame.available_entry_node_ids + ), + "r_loop_memory_handoff_source_graph_node_count": len( + r_loop_memory_handoff_frame.source_graph_node_ids + ), + "r_loop_memory_handoff_summary_depth_range": ( + r_loop_memory_handoff_frame.summary_depth_range + ), + "r_loop_memory_handoff_semantic_hint_status": ( + r_loop_memory_handoff_frame.semantic_hint_status + ), + "r_loop_memory_handoff_generated_by": r_loop_memory_handoff_frame.generated_by, + "r_loop_memory_handoff_info_class": r_loop_memory_handoff_frame.info_class, + "r_loop_memory_handoff_semantic_judgement_status": ( + r_loop_memory_handoff_frame.semantic_judgement_status + ), + "r_route_experimental_enabled": enable_r_route_experimental, + "r_route_experimental_status": r_route_experimental_status, + "r_route_experimental_policy_flag": ( + R_ROUTE_EXPERIMENTAL_POLICY_FLAG if enable_r_route_experimental else None + ), + "r_route_experimental_handoff_packet_id": r_route_experimental_handoff_packet_id, + "r_route_experimental_return_summary_id": r_route_experimental_return_summary_id, + "r_route_experimental_candidate_surface_id": ( + r_route_experimental_candidate_surface_id + ), + "r_route_experimental_access_ledger_id": r_route_experimental_access_ledger_id, + "r_route_experimental_close_route_id": r_route_experimental_close_route_id, + "r_route_experimental_output_data_ids": r_route_experimental_output_data_ids, + "r_route_experimental_trace_event_ids": r_route_experimental_trace_event_ids, + "r_route_experimental_graph_data_ids": r_route_experimental_graph_data_ids, + "r_route_dry_run_enabled": enable_r_route_dry_run, + "r_route_dry_run_status": ( + r_loop_dry_run_result.return_summary.r_loop_task_status + if r_loop_dry_run_result is not None + else "not_run" + ), + "r_route_dry_run_continuation_status": ( + r_loop_dry_run_result.continuation.continuation_status + if r_loop_dry_run_result is not None + else "not_run" + ), + "r_route_dry_run_next_target_node": ( + r_loop_dry_run_result.continuation.next_target_node + if r_loop_dry_run_result is not None + else "not_run" + ), + "r_route_dry_run_output_data_ids": ( + r_loop_dry_run_result.output_data_ids + if r_loop_dry_run_result is not None + else [] + ), + "r_route_dry_run_trace_event_ids": ( + r_loop_dry_run_result.trace_event_ids + if r_loop_dry_run_result is not None + else [] + ), + "r_route_dry_run_traversal_step_count": ( + len(r_loop_dry_run_result.r3_inspections) + if r_loop_dry_run_result is not None + else 0 + ), + "r_route_dry_run_access_ledger_id": ( + r_loop_dry_run_result.access_ledger.frame_id + if r_loop_dry_run_result is not None + else None + ), + "turn_activity_graph_link_trace_id": turn_activity_graph_link_trace_id, + "turn_activity_graph_link_frame_id": turn_activity_graph_link_frame_id, + "turn_activity_graph_link_node_count": len( + turn_activity_graph_link_frame.activity_ledger_graph_node_ids + ), + "turn_activity_graph_link_edge_count": len( + turn_activity_graph_link_frame.activity_ledger_graph_edge_ids + ), + "turn_activity_graph_link_l_ledger_count": len( + turn_activity_graph_link_frame.l_loop_activity_ledger_data_ids + ), + "turn_activity_graph_link_r_ledger_count": len( + turn_activity_graph_link_frame.r_graph_access_ledger_data_ids + ), + "r_route_dry_run_candidate_surface_id": ( + r_loop_dry_run_result.candidate_surface.frame_id + if r_loop_dry_run_result is not None + else None + ), + "r_route_dry_run_candidate_surface_count": ( + r_loop_dry_run_result.candidate_surface.candidate_count + if r_loop_dry_run_result is not None + else 0 + ), + "r_route_dry_run_candidate_surface_next_count": ( + len(r_loop_dry_run_result.candidate_surface.next_candidate_node_ids) + if r_loop_dry_run_result is not None + else 0 + ), + "r_route_dry_run_candidate_surface_previous_count": ( + len(r_loop_dry_run_result.candidate_surface.previous_candidate_node_ids) + if r_loop_dry_run_result is not None + else 0 + ), + "r_route_dry_run_access_candidate_count": ( + len(r_loop_dry_run_result.access_ledger.candidate_graph_node_ids) + if r_loop_dry_run_result is not None + else 0 + ), + "r_route_dry_run_access_selected_count": ( + len(r_loop_dry_run_result.access_ledger.selected_graph_node_ids) + if r_loop_dry_run_result is not None + else 0 + ), + "r_route_dry_run_access_inspected_count": ( + len(r_loop_dry_run_result.access_ledger.inspected_graph_node_ids) + if r_loop_dry_run_result is not None + else 0 + ), + "r_route_dry_run_selected_entry_node_ids": ( + r_loop_dry_run_result.return_summary.selected_entry_node_ids + if r_loop_dry_run_result is not None + else [] + ), + "r_route_dry_run_inspected_graph_node_ids": ( + r_loop_dry_run_result.return_summary.inspected_graph_node_ids + if r_loop_dry_run_result is not None + else [] + ), + "r_route_dry_run_budget_status": ( + r_loop_dry_run_result.return_summary.budget_status + if r_loop_dry_run_result is not None + else "not_run" + ), "llm_call_count": _count_records_by_type(data_store, "llm_call"), "tool_choice_count": _count_records_by_type(data_store, "tool_choice"), "tool_result_count": _count_records_with_type_prefix(data_store, "tool_result:"), @@ -1016,6 +1461,7 @@ def append_movement( "max_query_attempts": max_query_candidates if max_query_candidates is not None else max_query_attempts, + "max_document_context_chars": max_document_context_chars, "l_loop_run_count": len(l_results), "same_turn_l_reroute_enabled": same_turn_l_reroute_enabled, "max_l_runs_per_turn": max_l_runs_per_turn, @@ -1050,6 +1496,8 @@ def append_movement( "l_loop_final_continuation_status": last_l_result.final_continuation_status if last_l_result is not None else None, "l_loop_continuation_count": len(last_l_result.continuation_data_ids) if last_l_result is not None else 0, "l_loop_revision_query_count": len(last_l_result.revision_query_data_ids) if last_l_result is not None else 0, + "l_loop_activity_ledger_data_ids": l_activity_ledger_data_ids, + "l_loop_activity_ledger_count": len(l_activity_ledger_data_ids), "l2_query_source": _read_l2_query_source(data_store), "node1_llm_routing_count": _count_node1_llm_routes(data_store), "node1_llm_routing_failed_count": _count_node1_llm_failed_routes(data_store), @@ -1073,6 +1521,72 @@ def append_movement( "brief_status", data_type="node_output:node3_input_brief_frame", ), + "node2_answer_basis_mode": _read_payload_text( + data_store, + "node_2:answer_basis_frame", + "answer_basis_mode", + data_type="node_output:node2_answer_basis_frame", + ), + "node2_answer_basis_generated_by": _read_payload_text( + data_store, + "node_2:answer_basis_frame", + "generated_by", + data_type="node_output:node2_answer_basis_frame", + ), + "node2_answer_basis_reason_codes": _read_payload_string_list( + data_store, + "node_2:answer_basis_frame", + "basis_reason_codes", + data_type="node_output:node2_answer_basis_frame", + ), + "node2_answer_basis_semantic_judgement_status": _read_payload_text( + data_store, + "node_2:answer_basis_frame", + "semantic_judgement_status", + data_type="node_output:node2_answer_basis_frame", + ), + "node2_answer_basis_failure_type": _read_payload_text( + data_store, + "node_2:answer_basis_frame", + "answer_basis_failure_type", + data_type="node_output:node2_answer_basis_frame", + ), + "node2_answer_basis_llm_call_data_id": _read_payload_text( + data_store, + "node_2:answer_basis_frame", + "answer_basis_llm_call_data_id", + data_type="node_output:node2_answer_basis_frame", + ), + "node2_answer_basis_trace_event_id": _read_payload_text( + data_store, + "node_2:answer_basis_frame", + "answer_basis_trace_event_id", + data_type="node_output:node2_answer_basis_frame", + ), + "node2_answer_basis_validation_error": _read_payload_text( + data_store, + "node_2:answer_basis_frame", + "answer_basis_validation_error", + data_type="node_output:node2_answer_basis_frame", + ), + "node2_answer_basis_raw_text_present": _read_payload_bool( + data_store, + "node_2:answer_basis_frame", + "answer_basis_raw_text_present", + data_type="node_output:node2_answer_basis_frame", + ), + "node2_answer_basis_prompt_ref": _read_payload_text( + data_store, + "node_2:answer_basis_frame", + "answer_basis_prompt_ref", + data_type="node_output:node2_answer_basis_frame", + ), + "node2_answer_basis_payload_parse_status": _read_payload_text( + data_store, + "node_2:answer_basis_frame", + "answer_basis_payload_parse_status", + data_type="node_output:node2_answer_basis_frame", + ), "node3_reporter_status": _read_payload_text( data_store, "report_dry_001", @@ -1128,6 +1642,118 @@ def _unique_strings(values: list[str | None]) -> list[str]: unique_values.append(value) return unique_values + +def _record_r_route_experimental_graph_sources( + *, + trace_store: TraceStore, + data_store: DataStore, + turn_id: str, + graph_build: GraphMemoryBuildResult, + input_ref: list[str], +) -> tuple[str, list[str]]: + event = trace_store.create_event( + turn_id=turn_id, + actor="graph_memory", + event_type="r_route_experimental_graph_snapshot", + input_ref=input_ref, + output_ref=[ + graph_build.snapshot.snapshot_id, + graph_build.guide_packet.packet_id, + ], + schema_status="passed", + ) + recorded_data_ids: list[str] = [] + skipped_shared_graph_ids = _r_route_experimental_shared_graph_ids() + + for node in graph_build.nodes: + if node.node_id in skipped_shared_graph_ids: + continue + _record_payload_if_same_or_missing( + data_store=data_store, + data_id=node.node_id, + data_type=f"graph_memory:node:{node.node_kind}", + payload=asdict(node), + created_at=event.timestamp, + source_trace_id=event.event_id, + recorded_data_ids=recorded_data_ids, + ) + for edge in graph_build.edges: + if edge.edge_id in skipped_shared_graph_ids: + continue + _record_payload_if_same_or_missing( + data_store=data_store, + data_id=edge.edge_id, + data_type=f"graph_memory:edge:{edge.edge_kind}", + payload=asdict(edge), + created_at=event.timestamp, + source_trace_id=event.event_id, + recorded_data_ids=recorded_data_ids, + ) + for data_id, data_type, payload in [ + ( + graph_build.core_ego_time_axis.frame_id, + "graph_memory:core_ego_time_axis_frame", + asdict(graph_build.core_ego_time_axis), + ), + ( + graph_build.snapshot.snapshot_id, + "graph_memory:snapshot", + asdict(graph_build.snapshot), + ), + ( + graph_build.guide_packet.packet_id, + "graph_memory:rloop_guide_packet", + asdict(graph_build.guide_packet), + ), + ]: + _record_payload_if_same_or_missing( + data_store=data_store, + data_id=data_id, + data_type=data_type, + payload=payload, + created_at=event.timestamp, + source_trace_id=event.event_id, + recorded_data_ids=recorded_data_ids, + ) + + return event.event_id, _unique_strings(recorded_data_ids) + + +def _r_route_experimental_shared_graph_ids() -> set[str]: + return { + CORE_EGO_ROOT_NODE_ID, + TIME_AXIS_NODE_ID, + f"graph:edge:contains:{CORE_EGO_ROOT_NODE_ID}:{TIME_AXIS_NODE_ID}", + } + + +def _record_payload_if_same_or_missing( + *, + data_store: DataStore, + data_id: str, + data_type: str, + payload: dict[str, object], + created_at: str, + source_trace_id: str, + recorded_data_ids: list[str], +) -> None: + existing = data_store.get_record(data_id) + if existing is not None: + if existing.data_type != data_type or existing.payload != payload: + raise ValueError(f"r experimental graph source collision: {data_id}") + recorded_data_ids.append(data_id) + return + data_store.create_record( + data_id=data_id, + data_type=data_type, + exists=True, + created_at=created_at, + source_trace_id=source_trace_id, + payload=payload, + ) + recorded_data_ids.append(data_id) + + def _count_records_by_type(data_store: DataStore, data_type: str) -> int: return sum(1 for record in data_store.list_records() if record.data_type == data_type) @@ -1219,6 +1845,40 @@ def _read_payload_int( return value if isinstance(value, int) else None +def _read_payload_bool( + data_store: DataStore, + data_id: str, + field_name: str, + *, + data_type: str | None = None, +) -> bool | None: + record = data_store.get_record(data_id) + if record is None and data_type is not None: + record = _latest_record_by_type(data_store, data_type) + if record is None or not isinstance(record.payload, dict): + return None + value = record.payload.get(field_name) + return value if isinstance(value, bool) else None + + +def _read_payload_string_list( + data_store: DataStore, + data_id: str, + field_name: str, + *, + data_type: str | None = None, +) -> list[str] | None: + record = data_store.get_record(data_id) + if record is None and data_type is not None: + record = _latest_record_by_type(data_store, data_type) + if record is None or not isinstance(record.payload, dict): + return None + value = record.payload.get(field_name) + if not isinstance(value, list): + return None + return [item for item in value if isinstance(item, str)] + + def _latest_record_by_type(data_store: DataStore, data_type: str): for record in reversed(data_store.list_records()): if record.data_type == data_type: @@ -1236,11 +1896,13 @@ def _adapter_label(adapter: LLMAdapter | None, *, fallback: str) -> str: def _loop_adapter_label( *, l1_goal_adapter: LLMAdapter | None, + l_tool_scope_adapter: LLMAdapter | None, l2_query_planner_adapter: LLMAdapter | None, l3_result_adapter: LLMAdapter | None, ) -> str: labels = [ _adapter_label(l1_goal_adapter, fallback="L1:CODE/RULE_STUB"), + _adapter_label(l_tool_scope_adapter, fallback="L_scope:CODE/FALLBACK"), _adapter_label(l2_query_planner_adapter, fallback="L2:CODE/FALLBACK"), _adapter_label(l3_result_adapter, fallback="L3:CODE/OPERATION_CHECK"), ] @@ -1253,6 +1915,7 @@ def _assigned_model_by_node( node_1_router_adapter: LLMAdapter | None, memory_relevance_selector_label: str, l1_goal_adapter: LLMAdapter | None, + l_tool_scope_adapter: LLMAdapter | None, l2_query_planner_adapter: LLMAdapter | None, l3_result_adapter: LLMAdapter | None, node_2_boundary_adapter: LLMAdapter | None, @@ -1265,6 +1928,7 @@ def _assigned_model_by_node( "memory_relevance_selector": memory_relevance_selector_label, "L": _loop_adapter_label( l1_goal_adapter=l1_goal_adapter, + l_tool_scope_adapter=l_tool_scope_adapter, l2_query_planner_adapter=l2_query_planner_adapter, l3_result_adapter=l3_result_adapter, ), diff --git a/songryeon_core/runtime/fast_test.py b/songryeon_core/runtime/fast_test.py new file mode 100644 index 0000000..2093012 --- /dev/null +++ b/songryeon_core/runtime/fast_test.py @@ -0,0 +1,180 @@ +from __future__ import annotations + +import subprocess +import sys +import time +from dataclasses import dataclass +from pathlib import Path + + +FAST_TEST_PROFILES = {"core", "graph"} + + +@dataclass(frozen=True) +class FastTestStep: + name: str + command: list[str] + + +def build_fast_test_steps( + *, + profile: str = "graph", + skip_compileall: bool = False, +) -> list[FastTestStep]: + if profile not in FAST_TEST_PROFILES: + raise ValueError(f"unknown fast-test profile: {profile}") + + steps: list[FastTestStep] = [] + if not skip_compileall: + steps.append( + FastTestStep( + name="compileall", + command=[ + sys.executable, + "-m", + "compileall", + "songryeon_core", + "main.py", + ], + ) + ) + + pytest_files = [ + "tests/test_import_baseline.py", + "tests/test_schema_split_compat.py", + ] + if profile == "graph": + pytest_files.extend( + [ + "tests/test_order_146_r_route_experimental_gate.py", + "tests/test_order_147_r_result_to_node3_brief.py", + "tests/test_order_152_raw_capsule_activity_graph_link.py", + "tests/test_order_153_graph_memory_integrity.py", + "tests/test_order_155_graph_source_kind_ingest.py", + "tests/test_order_156_graph_source_observation_time_and_core_link.py", + "tests/test_order_157_songryeon_core_source_manifest.py", + "tests/test_order_158_graph_memory_export_packet.py", + "tests/test_order_159_vessel_adapter_boundary.py", + "tests/test_order_160_local_vessel_neo4j_writer.py", + "tests/test_order_162_vessel_readback_verification.py", + "tests/test_order_163_vessel_inspect_manual_walk.py", + "tests/test_order_164_source_version_lineage_and_invalidation.py", + "tests/test_order_165_same_content_observation_ledger.py", + "tests/test_order_166_night_time_bundle_summary_node.py", + "tests/test_order_168_night_summarize_changed_source_leaves.py", + "tests/test_order_169_night_changed_source_summary_cli.py", + "tests/test_order_170_night_changed_source_one_at_a_time.py", + "tests/test_order_171_night_token_budget_layer_summary.py", + "tests/test_order_172_night_token_layer_auto_reduce.py", + "tests/test_order_173_night_checkpointed_long_runner.py", + "tests/test_order_174_vessel_inspect_summary_layer_view.py", + "tests/test_order_175_vessel_backed_r_read_packet.py", + "tests/test_order_176_vessel_r_one_step_traversal.py", + "tests/test_order_177_r1_candidate_text_blindness.py", + "tests/test_order_178_r_vessel_candidate_layer_surface.py", + "tests/test_order_179_r2_vessel_selection_id_disambiguation.py", + "tests/test_order_180_r2_prompt_example_id_removal.py", + "tests/test_order_181_r2_official_selection_ref_map.py", + "tests/test_order_182_r_core_ego_start_surface.py", + "tests/test_order_183_r_vessel_hierarchical_child_surface.py", + "tests/test_order_184_r_vessel_multi_step_traversal.py", + "tests/test_order_185_r_terminal_material_guard.py", + "tests/test_order_186_r_vessel_exact_child_expansion.py", + "tests/test_order_187_r_vessel_summary_layer_before_raw.py", + "tests/test_order_188_r_vessel_raw_original_cap.py", + "tests/test_order_190_r_vessel_token_summary_deeper_child_expansion.py", + "tests/test_order_191_r2_branch_role_surface_stability.py", + "tests/test_order_192_r1_user_question_anchor_copy.py", + ] + ) + steps.append( + FastTestStep( + name=f"pytest:{profile}", + command=[ + sys.executable, + "-m", + "pytest", + "-q", + *pytest_files, + ], + ) + ) + return steps + + +def run_fast_tests( + *, + profile: str = "graph", + skip_compileall: bool = False, + dry_run: bool = False, + cwd: str | Path | None = None, +) -> dict[str, object]: + root = Path(cwd) if cwd is not None else Path.cwd() + steps = build_fast_test_steps( + profile=profile, + skip_compileall=skip_compileall, + ) + if dry_run: + return { + "status": "FAST_TEST_PLAN", + "profile": profile, + "step_count": len(steps), + "steps": [_step_plan(step) for step in steps], + } + + started = time.perf_counter() + results: list[dict[str, object]] = [] + failed = False + for step in steps: + step_started = time.perf_counter() + completed = subprocess.run( + step.command, + cwd=root, + text=True, + capture_output=True, + ) + step_duration = time.perf_counter() - step_started + if completed.returncode != 0: + failed = True + results.append( + { + "name": step.name, + "command": step.command, + "returncode": completed.returncode, + "duration_seconds": round(step_duration, 3), + "stdout_tail": _tail(completed.stdout), + "stderr_tail": _tail(completed.stderr), + } + ) + if failed: + break + + duration = time.perf_counter() - started + return { + "status": "FAST_TEST_FAILED" if failed else "FAST_TEST_OK", + "profile": profile, + "duration_seconds": round(duration, 3), + "step_count": len(steps), + "steps": results, + } + + +def _step_plan(step: FastTestStep) -> dict[str, object]: + return { + "name": step.name, + "command": step.command, + } + + +def _tail(text: str, *, max_chars: int = 4000) -> str: + if len(text) <= max_chars: + return text + return text[-max_chars:] + + +__all__ = [ + "FAST_TEST_PROFILES", + "FastTestStep", + "build_fast_test_steps", + "run_fast_tests", +] diff --git a/songryeon_core/runtime/graph_vessel_first_write.py b/songryeon_core/runtime/graph_vessel_first_write.py new file mode 100644 index 0000000..c469b65 --- /dev/null +++ b/songryeon_core/runtime/graph_vessel_first_write.py @@ -0,0 +1,162 @@ +from __future__ import annotations + +from dataclasses import asdict +from datetime import datetime +from pathlib import Path + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.graph_memory import record_graph_memory_for_capsules +from songryeon_core.core.graph_memory_export import record_graph_memory_export_packet +from songryeon_core.core.graph_vessel_adapter import record_graph_vessel_write_plan +from songryeon_core.core.graph_vessel_neo4j import ( + GraphVesselNeo4jConfig, + graph_vessel_neo4j_config_from_env, + record_graph_vessel_neo4j_write_result, +) +from songryeon_core.core.schemas import NodeMovement, TurnStateCapsule +from songryeon_core.core.songryeon_source_manifest import ( + record_songryeon_core_source_manifest_ingest, +) +from songryeon_core.core.trace_store import TraceStore + + +def run_local_vessel_first_write( + *, + root_path: str | Path = ".", + batch_id: str = "manual_vessel_first_write", + turn_id: str = "turn_vessel_first_write_0001", + uri: str | None = None, + user: str | None = None, + password: str | None = None, + database: str | None = None, + allow_no_auth: bool = False, + include_source_manifest: bool = False, + store_text_snapshots: bool = False, +) -> dict[str, object]: + """Manual opt-in local Vessel first-write runner. + + This builds a tiny graph fixture, creates the export packet and write plan, + and then attempts the Neo4j write only through the ORDER_160 writer. + """ + + now = _now_iso() + root = Path(root_path).resolve() + trace_store = TraceStore() + data_store = DataStore() + record_graph_memory_for_capsules( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + batch_id=f"{batch_id}:core", + capsules=[_sample_capsule(turn_id=f"{turn_id}:previous")], + ) + source_manifest_frame_id = None + source_ingest_frame_id = None + if include_source_manifest: + source_result = record_songryeon_core_source_manifest_ingest( + trace_store=trace_store, + data_store=data_store, + root_path=root, + turn_id=turn_id, + batch_id=f"{batch_id}:source_manifest", + observed_at=now, + ingested_at=now, + store_text_snapshots=store_text_snapshots, + ) + source_manifest_frame_id = source_result.manifest_data_id + source_ingest_frame_id = source_result.ingest_result.frame_id + + export_result = record_graph_memory_export_packet( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + batch_id=f"{batch_id}:export", + created_at=now, + ) + plan_result = record_graph_vessel_write_plan( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + export_packet_id=export_result.packet.packet_id, + created_at=now, + ) + config: GraphVesselNeo4jConfig = graph_vessel_neo4j_config_from_env( + uri=uri, + user=user, + password=password, + database=database, + allow_no_auth=allow_no_auth, + ) + write_result = record_graph_vessel_neo4j_write_result( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + plan_id=plan_result.plan.plan_id, + config=config, + created_at=now, + ) + + result_payload = asdict(write_result.result) + return { + "status": "VESSEL_FIRST_WRITE_OK" + if write_result.result.write_status == "written" + else "VESSEL_FIRST_WRITE_NOT_WRITTEN", + "write_status": write_result.result.write_status, + "failure_type": write_result.result.failure_type, + "failure_reason": write_result.result.failure_reason, + "target_adapter_name": write_result.result.target_adapter_name, + "vessel_database_name": write_result.result.vessel_database_name, + "graph_namespace": write_result.result.graph_namespace, + "neo4j_uri": config.uri, + "neo4j_user": config.user, + "neo4j_password_configured": config.password is not None, + "neo4j_allow_no_auth": config.allow_no_auth, + "include_source_manifest": include_source_manifest, + "store_text_snapshots": store_text_snapshots, + "trace_count": len(trace_store.list_events()), + "data_record_count": len(data_store.list_records()), + "export_packet_id": export_result.packet.packet_id, + "write_plan_id": plan_result.plan.plan_id, + "write_result_id": write_result.result.result_id, + "source_manifest_frame_id": source_manifest_frame_id, + "source_ingest_frame_id": source_ingest_frame_id, + "operation_count": plan_result.plan.operation_count, + "attempted_operation_count": write_result.result.attempted_operation_count, + "written_operation_count": write_result.result.written_operation_count, + "operation_count_by_kind": write_result.result.operation_count_by_kind, + "neo4j_record_node_count": write_result.result.neo4j_record_node_count, + "neo4j_graph_edge_count": write_result.result.neo4j_graph_edge_count, + "write_result_frame": result_payload, + } + + +def _sample_capsule(turn_id: str) -> TurnStateCapsule: + return TurnStateCapsule( + turn_id=turn_id, + node_movements=[ + NodeMovement( + movement_id=f"move:{turn_id}:001", + turn_id=turn_id, + step_index=1, + node_id="node_0", + mode="pre_route_report", + input_trace_ids=[f"trace:{turn_id}:user"], + output_trace_ids=[f"trace:{turn_id}:node0"], + status="completed", + ) + ], + trace_event_ids=[ + f"trace:{turn_id}:user", + f"trace:{turn_id}:node0", + f"trace:{turn_id}:final", + ], + user_input_trace_id=f"trace:{turn_id}:user", + final_response_trace_id=f"trace:{turn_id}:final", + ) + + +def _now_iso() -> str: + return datetime.now().isoformat(timespec="seconds") + + +__all__ = ["run_local_vessel_first_write"] diff --git a/songryeon_core/runtime/graph_vessel_inspect.py b/songryeon_core/runtime/graph_vessel_inspect.py new file mode 100644 index 0000000..819197c --- /dev/null +++ b/songryeon_core/runtime/graph_vessel_inspect.py @@ -0,0 +1,123 @@ +from __future__ import annotations + +from dataclasses import asdict +from datetime import datetime + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.graph_vessel_inspect import ( + record_graph_vessel_neo4j_inspect_result, +) +from songryeon_core.core.graph_vessel_neo4j import ( + graph_vessel_neo4j_config_from_env, +) +from songryeon_core.core.trace_store import TraceStore + + +def run_local_vessel_inspect( + *, + batch_id: str = "manual_vessel_inspect", + turn_id: str = "turn_vessel_inspect_0001", + uri: str | None = None, + user: str | None = None, + password: str | None = None, + database: str | None = None, + allow_no_auth: bool = False, + limit: int = 50, +) -> dict[str, object]: + """Read-only manual walk over the local Neo4j Vessel tree.""" + + now = _now_iso() + trace_store = TraceStore() + data_store = DataStore() + config = graph_vessel_neo4j_config_from_env( + uri=uri, + user=user, + password=password, + database=database, + allow_no_auth=allow_no_auth, + ) + inspect = record_graph_vessel_neo4j_inspect_result( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + batch_id=batch_id, + config=config, + created_at=now, + limit=limit, + ) + + result_payload = asdict(inspect.result) + tree_text = "\n".join(inspect.result.tree_lines) + return { + "status": "VESSEL_INSPECT_OK" + if inspect.result.inspect_status == "passed" + else "VESSEL_INSPECT_NOT_PASSED", + "inspect_status": inspect.result.inspect_status, + "failure_type": inspect.result.failure_type, + "failure_reason": inspect.result.failure_reason, + "target_adapter_name": inspect.result.target_adapter_name, + "vessel_database_name": inspect.result.vessel_database_name, + "graph_namespace": inspect.result.graph_namespace, + "neo4j_uri": config.uri, + "neo4j_user": config.user, + "neo4j_password_configured": config.password is not None, + "neo4j_allow_no_auth": config.allow_no_auth, + "trace_count": len(trace_store.list_events()), + "data_record_count": len(data_store.list_records()), + "inspect_result_id": inspect.result.result_id, + "inspected_path_count": inspect.result.inspected_path_count, + "core_count": inspect.result.core_count, + "time_axis_count": inspect.result.time_axis_count, + "time_bundle_count": inspect.result.time_bundle_count, + "raw_capsule_count": inspect.result.raw_capsule_count, + "summary_count": inspect.result.summary_count, + "active_summary_count": inspect.result.active_summary_count, + "invalidated_summary_count": inspect.result.invalidated_summary_count, + "summary_count_by_data_kind": inspect.result.summary_count_by_data_kind, + "summary_count_by_depth": inspect.result.summary_count_by_depth, + "summary_sample_items": inspect.result.summary_sample_items, + "summary_lines": inspect.result.summary_lines, + "tree_lines": inspect.result.tree_lines, + "tree_text": tree_text, + "path_items": inspect.result.path_items, + "inspect_result_frame": result_payload, + } + + +def render_vessel_inspect_text(result: dict[str, object]) -> str: + lines = [ + f"status: {result.get('status')}", + f"inspect_status: {result.get('inspect_status')}", + ] + failure_type = result.get("failure_type") + failure_reason = result.get("failure_reason") + if failure_type or failure_reason: + lines.append(f"failure_type: {failure_type}") + lines.append(f"failure_reason: {failure_reason}") + lines.append(f"graph_namespace: {result.get('graph_namespace')}") + lines.append(f"inspected_path_count: {result.get('inspected_path_count')}") + summary_count = result.get("summary_count") + if summary_count is not None: + lines.append(f"summary_count: {summary_count}") + lines.append(f"active_summary_count: {result.get('active_summary_count')}") + lines.append( + f"invalidated_summary_count: {result.get('invalidated_summary_count')}" + ) + lines.append(f"summary_count_by_data_kind: {result.get('summary_count_by_data_kind')}") + lines.append(f"summary_count_by_depth: {result.get('summary_count_by_depth')}") + tree_text = result.get("tree_text") + if isinstance(tree_text, str) and tree_text: + lines.append("") + lines.append(tree_text) + summary_lines = result.get("summary_lines") + if isinstance(summary_lines, list) and summary_lines: + lines.append("") + lines.extend(str(line) for line in summary_lines) + return "\n".join(lines) + + +def _now_iso() -> str: + return datetime.now().isoformat(timespec="seconds") + + +__all__ = ["render_vessel_inspect_text", "run_local_vessel_inspect"] diff --git a/songryeon_core/runtime/graph_vessel_readback.py b/songryeon_core/runtime/graph_vessel_readback.py new file mode 100644 index 0000000..d2fdf49 --- /dev/null +++ b/songryeon_core/runtime/graph_vessel_readback.py @@ -0,0 +1,86 @@ +from __future__ import annotations + +from dataclasses import asdict +from datetime import datetime + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.graph_vessel_neo4j import ( + graph_vessel_neo4j_config_from_env, +) +from songryeon_core.core.graph_vessel_readback import ( + record_graph_vessel_neo4j_readback_result, +) +from songryeon_core.core.trace_store import TraceStore + + +def run_local_vessel_readback( + *, + batch_id: str = "manual_vessel_readback", + turn_id: str = "turn_vessel_readback_0001", + uri: str | None = None, + user: str | None = None, + password: str | None = None, + database: str | None = None, + allow_no_auth: bool = False, +) -> dict[str, object]: + """Read-only Neo4j Vessel check for the current graph vocabulary.""" + + now = _now_iso() + trace_store = TraceStore() + data_store = DataStore() + config = graph_vessel_neo4j_config_from_env( + uri=uri, + user=user, + password=password, + database=database, + allow_no_auth=allow_no_auth, + ) + readback = record_graph_vessel_neo4j_readback_result( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + batch_id=batch_id, + config=config, + created_at=now, + ) + + result_payload = asdict(readback.result) + return { + "status": "VESSEL_READBACK_OK" + if readback.result.readback_status == "passed" + else "VESSEL_READBACK_NOT_PASSED", + "readback_status": readback.result.readback_status, + "failure_type": readback.result.failure_type, + "failure_reason": readback.result.failure_reason, + "target_adapter_name": readback.result.target_adapter_name, + "vessel_database_name": readback.result.vessel_database_name, + "graph_namespace": readback.result.graph_namespace, + "neo4j_uri": config.uri, + "neo4j_user": config.user, + "neo4j_password_configured": config.password is not None, + "neo4j_allow_no_auth": config.allow_no_auth, + "trace_count": len(trace_store.list_events()), + "data_record_count": len(data_store.list_records()), + "readback_result_id": readback.result.result_id, + "core_path_exists": readback.result.core_path_exists, + "core_path_count": readback.result.core_path_count, + "vessel_record_count": readback.result.vessel_record_count, + "vessel_relationship_count": readback.result.vessel_relationship_count, + "core_ego_count": readback.result.core_ego_count, + "time_axis_count": readback.result.time_axis_count, + "time_bundle_count": readback.result.time_bundle_count, + "raw_capsule_count": readback.result.raw_capsule_count, + "has_axis_count": readback.result.has_axis_count, + "has_bundle_count": readback.result.has_bundle_count, + "contains_memory_count": readback.result.contains_memory_count, + "required_property_missing_count": readback.result.required_property_missing_count, + "required_property_missing_samples": readback.result.required_property_missing_samples, + "readback_result_frame": result_payload, + } + + +def _now_iso() -> str: + return datetime.now().isoformat(timespec="seconds") + + +__all__ = ["run_local_vessel_readback"] diff --git a/songryeon_core/runtime/night_changed_source_summary.py b/songryeon_core/runtime/night_changed_source_summary.py new file mode 100644 index 0000000..4f87c8a --- /dev/null +++ b/songryeon_core/runtime/night_changed_source_summary.py @@ -0,0 +1,671 @@ +from __future__ import annotations + +import json +from dataclasses import asdict +from datetime import datetime +from pathlib import Path + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.graph_memory import CORE_EGO_ROOT_NODE_ID, record_graph_memory_for_capsules +from songryeon_core.core.graph_memory_export import record_graph_memory_export_packet +from songryeon_core.core.graph_vessel_adapter import record_graph_vessel_write_plan +from songryeon_core.core.graph_vessel_neo4j import ( + graph_vessel_neo4j_config_from_env, + record_graph_vessel_neo4j_write_result, +) +from songryeon_core.core.schemas import NodeMovement, TurnStateCapsule +from songryeon_core.core.songryeon_source_manifest import ( + record_songryeon_core_source_manifest_ingest, +) +from songryeon_core.core.trace_store import TraceStore +from songryeon_core.llm.base import LLMRequest, LLMResponse +from songryeon_core.llm.runtime import ( + build_llm_adapter, + build_llm_runtime_config, + llm_runtime_status, +) +from songryeon_core.nodes.night_summarize_source_leaf import ( + RecordedNightSourceLeafSummaryResult, + night_summarize_source_leaf_frame_id, + night_summarize_source_leaf_graph_node_id, + run_night_summarize_changed_source_leaves, + run_night_summarize_source_leaf, + select_changed_source_leaf_summary_candidates, +) + + +DEFAULT_NIGHT_CHANGED_SOURCE_STORE_DIR = ( + ".songryeon_core_cache/night_changed_sources" +) +DEFAULT_NIGHT_CHANGED_SOURCE_ONE_AT_A_TIME_BATCH_ID = ( + "night_changed_sources_one_at_a_time_active" +) +NIGHT_CHANGED_SOURCE_ONE_AT_A_TIME_QUEUE_DATA_TYPE = ( + "runtime:night_changed_source_one_at_a_time_queue" +) +NIGHT_CHANGED_SOURCE_ONE_AT_A_TIME_QUEUE_GENERATOR = ( + "CODE:NIGHT_CHANGED_SOURCE_ONE_AT_A_TIME_QUEUE" +) + + +class NightSourceSummaryFakeAdapter: + """Deterministic adapter for command smoke tests.""" + + model_id = "night-source-summary-fake-adapter" + + def complete(self, request: LLMRequest) -> LLMResponse: + source_text = str(request.input_payload.get("source_text") or "").strip() + source_file = request.input_payload.get("source_file_payload") + path = "" + if isinstance(source_file, dict): + path = str(source_file.get("path") or "") + payload = { + "summary_text": ( + f"Fake source leaf summary for {Path(path).name or 'source'}: " + f"{' '.join(source_text.split())[:160]}" + ) + } + return LLMResponse( + text=json.dumps(payload, ensure_ascii=False), + model_id=self.model_id, + raw=payload, + ) + + +def run_night_changed_source_summary( + *, + root_path: str | Path = ".", + store_dir: str | Path = DEFAULT_NIGHT_CHANGED_SOURCE_STORE_DIR, + batch_id: str | None = None, + turn_id: str | None = None, + llm_mode: str = "off", + endpoint: str | None = None, + model_id: str | None = None, + timeout_seconds: int | None = None, + write_vessel: bool = False, + uri: str | None = None, + user: str | None = None, + password: str | None = None, + database: str | None = None, + allow_no_auth: bool = False, + one_at_a_time: bool = False, +) -> dict[str, object]: + """Manual opt-in source ingest and changed-leaf summary runner.""" + + now = _now_iso() + safe_batch_id = ( + batch_id + or ( + DEFAULT_NIGHT_CHANGED_SOURCE_ONE_AT_A_TIME_BATCH_ID + if one_at_a_time + else f"night_changed_sources_{_safe_timestamp(now)}" + ) + ) + safe_turn_id = turn_id or f"turn_{safe_batch_id}" + root = Path(root_path).resolve() + cache_dir = Path(store_dir).resolve() + cache_dir.mkdir(parents=True, exist_ok=True) + trace_path = cache_dir / "trace_store.json" + data_path = cache_dir / "data_store.json" + trace_store = TraceStore.load_json(trace_path) if trace_path.exists() else TraceStore() + data_store = DataStore.load_json(data_path) if data_path.exists() else DataStore() + + adapter = _build_summary_adapter( + llm_mode=llm_mode, + endpoint=endpoint, + model_id=model_id, + timeout_seconds=timeout_seconds, + ) + summary_results: list[RecordedNightSourceLeafSummaryResult] + one_at_a_time_state: dict[str, object] | None = None + if one_at_a_time: + queue_result = _run_one_at_a_time_source_summary_step( + trace_store=trace_store, + data_store=data_store, + root=root, + turn_id=safe_turn_id, + batch_id=safe_batch_id, + observed_at=now, + adapter=adapter, + trace_path=trace_path, + data_path=data_path, + ) + summary_results = list(queue_result["summary_results"]) + one_at_a_time_state = dict(queue_result["state"]) + source_manifest_frame_id = str(one_at_a_time_state.get("source_manifest_frame_id") or "") + source_ingest_frame_id = str(one_at_a_time_state.get("source_ingest_frame_id") or "") + source_observation_ledger_frame_id = str( + one_at_a_time_state.get("source_observation_ledger_frame_id") or "" + ) + source_kind_counts = _dict_value(one_at_a_time_state.get("source_kind_counts")) + source_observation_status_counts = _dict_value( + one_at_a_time_state.get("source_observation_status_counts") + ) + selected_source_graph_node_ids = _string_list( + one_at_a_time_state.get("selected_source_graph_node_ids") + ) + skipped_unchanged_source_graph_node_ids = _string_list( + one_at_a_time_state.get("skipped_unchanged_source_graph_node_ids") + ) + else: + _ensure_core_time_axis( + trace_store=trace_store, + data_store=data_store, + turn_id=safe_turn_id, + batch_id=f"{safe_batch_id}:core", + ) + manifest_result = record_songryeon_core_source_manifest_ingest( + trace_store=trace_store, + data_store=data_store, + root_path=root, + turn_id=safe_turn_id, + batch_id=f"{safe_batch_id}:source_manifest", + observed_at=now, + ingested_at=now, + ) + summary_batch = run_night_summarize_changed_source_leaves( + trace_store=trace_store, + data_store=data_store, + turn_id=safe_turn_id, + source_observation_ledger_frame_id=( + manifest_result.ingest_result.source_observation_ledger_frame_id + ), + summary_run_id=safe_batch_id, + adapter=adapter, + ) + summary_results = list(summary_batch.results) + source_manifest_frame_id = manifest_result.manifest_data_id + source_ingest_frame_id = manifest_result.ingest_result.frame_id + source_observation_ledger_frame_id = ( + manifest_result.ingest_result.source_observation_ledger_frame_id + ) + source_kind_counts = manifest_result.manifest.source_kind_counts + source_observation_status_counts = ( + manifest_result.ingest_result.source_observation_status_counts + ) + selected_source_graph_node_ids = summary_batch.selected_source_graph_node_ids + skipped_unchanged_source_graph_node_ids = ( + summary_batch.skipped_unchanged_source_graph_node_ids + ) + export_batch_id = f"{safe_batch_id}:export" + if one_at_a_time: + export_batch_id = f"{export_batch_id}:{_safe_timestamp(now)}" + export_result = record_graph_memory_export_packet( + trace_store=trace_store, + data_store=data_store, + turn_id=safe_turn_id, + batch_id=export_batch_id, + created_at=now, + ) + plan_result = record_graph_vessel_write_plan( + trace_store=trace_store, + data_store=data_store, + turn_id=safe_turn_id, + export_packet_id=export_result.packet.packet_id, + created_at=now, + ) + write_result_payload: dict[str, object] | None = None + if write_vessel: + config = graph_vessel_neo4j_config_from_env( + uri=uri, + user=user, + password=password, + database=database, + allow_no_auth=allow_no_auth, + ) + write_result = record_graph_vessel_neo4j_write_result( + trace_store=trace_store, + data_store=data_store, + turn_id=safe_turn_id, + plan_id=plan_result.plan.plan_id, + config=config, + created_at=now, + ) + write_result_payload = asdict(write_result.result) + + trace_store.save_json(trace_path) + data_store.save_json(data_path) + + summary_status_counts = _summary_status_counts(summary_results) + result_payload = { + "status": "NIGHT_CHANGED_SOURCE_SUMMARY_OK", + "execution_mode": "one_at_a_time" if one_at_a_time else "all_at_once", + "root_path": root.as_posix(), + "store_dir": cache_dir.as_posix(), + "batch_id": safe_batch_id, + "turn_id": safe_turn_id, + "llm_runtime": _llm_status_for_mode( + llm_mode=llm_mode, + endpoint=endpoint, + model_id=model_id, + timeout_seconds=timeout_seconds, + ), + "source_manifest_frame_id": source_manifest_frame_id, + "source_ingest_frame_id": source_ingest_frame_id, + "source_observation_ledger_frame_id": source_observation_ledger_frame_id, + "source_kind_counts": source_kind_counts, + "source_observation_status_counts": source_observation_status_counts, + "selected_source_leaf_count": len(selected_source_graph_node_ids), + "skipped_unchanged_source_leaf_count": len( + skipped_unchanged_source_graph_node_ids + ), + "summary_status_counts": summary_status_counts, + "summary_success_count": summary_status_counts.get("ran", 0), + "summary_failed_count": summary_status_counts.get("failed", 0), + "summary_skipped_no_text_snapshot_count": summary_status_counts.get( + "skipped_no_text_snapshot", + 0, + ), + "summary_skipped_empty_text_count": summary_status_counts.get( + "skipped_empty_text", + 0, + ), + "summary_graph_node_ids": [ + result.summary_graph_node_data_id + for result in summary_results + if result.summary_graph_node_data_id is not None + ], + "summary_edge_ids": [ + result.summary_edge_data_id + for result in summary_results + if result.summary_edge_data_id is not None + ], + "export_packet_id": export_result.packet.packet_id, + "write_plan_id": plan_result.plan.plan_id, + "write_plan_status": plan_result.plan.plan_status, + "write_vessel_requested": write_vessel, + "write_result": write_result_payload, + "trace_store_path": trace_path.as_posix(), + "data_store_path": data_path.as_posix(), + "trace_count": len(trace_store.list_events()), + "data_record_count": len(data_store.list_records()), + } + if one_at_a_time_state is not None: + result_payload.update(one_at_a_time_state) + return result_payload + + +def _run_one_at_a_time_source_summary_step( + *, + trace_store: TraceStore, + data_store: DataStore, + root: Path, + turn_id: str, + batch_id: str, + observed_at: str, + adapter, + trace_path: Path, + data_path: Path, +) -> dict[str, object]: + queue_id = _one_at_a_time_queue_id(batch_id) + queue_payload = _load_one_at_a_time_queue(data_store=data_store, queue_id=queue_id) + queue_status = "existing" + if queue_payload is None: + queue_status = "created" + _ensure_core_time_axis( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + batch_id=f"{batch_id}:core", + ) + manifest_result = record_songryeon_core_source_manifest_ingest( + trace_store=trace_store, + data_store=data_store, + root_path=root, + turn_id=turn_id, + batch_id=f"{batch_id}:source_manifest", + observed_at=observed_at, + ingested_at=observed_at, + ) + selection = select_changed_source_leaf_summary_candidates( + data_store=data_store, + source_observation_ledger_frame_id=( + manifest_result.ingest_result.source_observation_ledger_frame_id + ), + ) + queue_payload = _record_one_at_a_time_queue( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + queue_id=queue_id, + summary_run_id=batch_id, + source_manifest_frame_id=manifest_result.manifest_data_id, + source_ingest_frame_id=manifest_result.ingest_result.frame_id, + source_observation_ledger_frame_id=( + manifest_result.ingest_result.source_observation_ledger_frame_id + ), + source_kind_counts=manifest_result.manifest.source_kind_counts, + source_observation_status_counts=( + manifest_result.ingest_result.source_observation_status_counts + ), + selected_source_graph_node_ids=selection.selected_source_graph_node_ids, + skipped_unchanged_source_graph_node_ids=( + selection.skipped_unchanged_source_graph_node_ids + ), + input_trace_ids=[ + manifest_result.manifest_trace_event_id, + manifest_result.ingest_result.trace_event_id, + ], + ) + trace_store.save_json(trace_path) + data_store.save_json(data_path) + + selected_source_graph_node_ids = _string_list( + queue_payload.get("selected_source_graph_node_ids") + ) + processed_before = _processed_source_leaf_ids( + data_store=data_store, + source_graph_node_ids=selected_source_graph_node_ids, + summary_run_id=batch_id, + ) + pending_source_graph_node_ids = [ + source_graph_node_id + for source_graph_node_id in selected_source_graph_node_ids + if source_graph_node_id not in processed_before + ] + summary_results: list[RecordedNightSourceLeafSummaryResult] = [] + processed_source_graph_node_id: str | None = None + if pending_source_graph_node_ids: + processed_source_graph_node_id = pending_source_graph_node_ids[0] + summary_results.append( + run_night_summarize_source_leaf( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + raw_source_graph_node_id=processed_source_graph_node_id, + summary_run_id=batch_id, + adapter=adapter, + ) + ) + trace_store.save_json(trace_path) + data_store.save_json(data_path) + + processed_after = _processed_source_leaf_ids( + data_store=data_store, + source_graph_node_ids=selected_source_graph_node_ids, + summary_run_id=batch_id, + ) + pending_after = [ + source_graph_node_id + for source_graph_node_id in selected_source_graph_node_ids + if source_graph_node_id not in processed_after + ] + if not selected_source_graph_node_ids: + completion_status = "no_candidates" + elif pending_after: + completion_status = "in_progress" + else: + completion_status = "complete" + + state = { + **queue_payload, + "one_at_a_time_queue_id": queue_id, + "one_at_a_time_queue_status": queue_status, + "one_at_a_time_completion_status": completion_status, + "one_at_a_time_processed_this_run": len(summary_results), + "one_at_a_time_processed_source_graph_node_id": processed_source_graph_node_id, + "one_at_a_time_queue_total_count": len(selected_source_graph_node_ids), + "one_at_a_time_queue_processed_count_before": len(processed_before), + "one_at_a_time_queue_processed_count_after": len(processed_after), + "one_at_a_time_queue_pending_count_after": len(pending_after), + "one_at_a_time_next_source_graph_node_id": ( + pending_after[0] if pending_after else None + ), + } + return { + "summary_results": summary_results, + "state": state, + } + + +def _one_at_a_time_queue_id(summary_run_id: str) -> str: + return f"night_summary:changed_source_one_at_a_time_queue:{_safe_id_part(summary_run_id)}" + + +def _load_one_at_a_time_queue( + *, + data_store: DataStore, + queue_id: str, +) -> dict[str, object] | None: + record = data_store.get_record(queue_id) + if record is None: + return None + if record.data_type != NIGHT_CHANGED_SOURCE_ONE_AT_A_TIME_QUEUE_DATA_TYPE: + raise ValueError(f"one-at-a-time queue data_id collision: {queue_id}") + if not isinstance(record.payload, dict): + raise TypeError("one-at-a-time queue payload must be dict") + return dict(record.payload) + + +def _record_one_at_a_time_queue( + *, + trace_store: TraceStore, + data_store: DataStore, + turn_id: str, + queue_id: str, + summary_run_id: str, + source_manifest_frame_id: str, + source_ingest_frame_id: str, + source_observation_ledger_frame_id: str, + source_kind_counts: dict[str, object], + source_observation_status_counts: dict[str, object], + selected_source_graph_node_ids: list[str], + skipped_unchanged_source_graph_node_ids: list[str], + input_trace_ids: list[str], +) -> dict[str, object]: + payload = { + "queue_id": queue_id, + "summary_run_id": summary_run_id, + "source_manifest_frame_id": source_manifest_frame_id, + "source_ingest_frame_id": source_ingest_frame_id, + "source_observation_ledger_frame_id": source_observation_ledger_frame_id, + "source_kind_counts": dict(source_kind_counts), + "source_observation_status_counts": dict(source_observation_status_counts), + "selected_source_graph_node_ids": list(selected_source_graph_node_ids), + "skipped_unchanged_source_graph_node_ids": list( + skipped_unchanged_source_graph_node_ids + ), + "generated_by": NIGHT_CHANGED_SOURCE_ONE_AT_A_TIME_QUEUE_GENERATOR, + "info_class": "absolute", + "semantic_judgement_status": "not_run", + } + event = trace_store.create_event( + turn_id=turn_id, + actor="night_changed_source_one_at_a_time_queue", + event_type="node_output", + input_ref=input_trace_ids, + output_ref=[queue_id], + schema_status="passed", + ) + _record_payload_if_missing( + data_store=data_store, + data_id=queue_id, + data_type=NIGHT_CHANGED_SOURCE_ONE_AT_A_TIME_QUEUE_DATA_TYPE, + payload=payload, + created_at=event.timestamp, + source_trace_id=event.event_id, + ) + return payload + + +def _processed_source_leaf_ids( + *, + data_store: DataStore, + source_graph_node_ids: list[str], + summary_run_id: str, +) -> list[str]: + processed: list[str] = [] + for source_graph_node_id in source_graph_node_ids: + summary_node_id = night_summarize_source_leaf_graph_node_id( + source_graph_node_id, + summary_run_id=summary_run_id, + ) + frame_id = night_summarize_source_leaf_frame_id( + source_graph_node_id, + summary_run_id=summary_run_id, + ) + if data_store.get_record(summary_node_id) is not None: + processed.append(source_graph_node_id) + continue + if data_store.get_record(frame_id) is not None: + processed.append(source_graph_node_id) + return processed + + +def _record_payload_if_missing( + *, + data_store: DataStore, + data_id: str, + data_type: str, + payload: dict[str, object], + created_at: str, + source_trace_id: str, +) -> None: + existing = data_store.get_record(data_id) + if existing is not None: + if existing.data_type != data_type: + raise ValueError(f"night changed source data_id collision: {data_id}") + if existing.payload != payload: + raise ValueError( + f"night changed source data_id collision with different payload: {data_id}" + ) + return + data_store.create_record( + data_id=data_id, + data_type=data_type, + exists=True, + created_at=created_at, + source_trace_id=source_trace_id, + payload=payload, + ) + + +def _build_summary_adapter( + *, + llm_mode: str, + endpoint: str | None, + model_id: str | None, + timeout_seconds: int | None, +): + selected_mode = llm_mode.strip().lower() + if selected_mode == "fake": + return NightSourceSummaryFakeAdapter() + config = build_llm_runtime_config( + mode=selected_mode, + endpoint=endpoint, + model_id=model_id, + timeout_seconds=timeout_seconds, + ) + selected_endpoint = endpoint + return build_llm_adapter(config, endpoint=selected_endpoint) + + +def _llm_status_for_mode( + *, + llm_mode: str, + endpoint: str | None, + model_id: str | None, + timeout_seconds: int | None, +) -> dict[str, object]: + selected_mode = llm_mode.strip().lower() + if selected_mode == "fake": + return { + "mode": "fake", + "enabled": True, + "adapter_kind": "fake", + "model_id": NightSourceSummaryFakeAdapter.model_id, + "source": "arguments", + } + config = build_llm_runtime_config( + mode=selected_mode, + endpoint=endpoint, + model_id=model_id, + timeout_seconds=timeout_seconds, + ) + return llm_runtime_status(config) + + +def _ensure_core_time_axis( + *, + trace_store: TraceStore, + data_store: DataStore, + turn_id: str, + batch_id: str, +) -> None: + if data_store.get_record(CORE_EGO_ROOT_NODE_ID) is not None: + return + record_graph_memory_for_capsules( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + batch_id=batch_id, + capsules=[_sample_capsule(turn_id=f"{turn_id}:bootstrap")], + ) + + +def _sample_capsule(turn_id: str) -> TurnStateCapsule: + return TurnStateCapsule( + turn_id=turn_id, + node_movements=[ + NodeMovement( + movement_id=f"move:{turn_id}:001", + turn_id=turn_id, + step_index=1, + node_id="node_0", + mode="night_changed_source_summary_bootstrap", + input_trace_ids=[f"trace:{turn_id}:input"], + output_trace_ids=[f"trace:{turn_id}:output"], + status="completed", + ) + ], + trace_event_ids=[ + f"trace:{turn_id}:input", + f"trace:{turn_id}:output", + f"trace:{turn_id}:final", + ], + user_input_trace_id=f"trace:{turn_id}:input", + final_response_trace_id=f"trace:{turn_id}:final", + ) + + +def _summary_status_counts(results: object) -> dict[str, int]: + counts: dict[str, int] = {} + for result in results: + status = result.frame.summary_status + counts[status] = counts.get(status, 0) + 1 + return counts + + +def _now_iso() -> str: + return datetime.now().isoformat(timespec="microseconds") + + +def _safe_timestamp(timestamp: str) -> str: + return "".join(char if char.isalnum() else "_" for char in timestamp).strip("_") + + +def _safe_id_part(value: str) -> str: + cleaned = "".join(char if char.isalnum() else "_" for char in value.strip()) + cleaned = cleaned.strip("_") + if not cleaned: + raise ValueError("safe id part must not be empty") + return cleaned + + +def _string_list(value: object) -> list[str]: + if not isinstance(value, list): + return [] + return [item for item in value if isinstance(item, str) and item] + + +def _dict_value(value: object) -> dict[str, object]: + return dict(value) if isinstance(value, dict) else {} + + +__all__ = [ + "DEFAULT_NIGHT_CHANGED_SOURCE_STORE_DIR", + "DEFAULT_NIGHT_CHANGED_SOURCE_ONE_AT_A_TIME_BATCH_ID", + "NIGHT_CHANGED_SOURCE_ONE_AT_A_TIME_QUEUE_DATA_TYPE", + "NightSourceSummaryFakeAdapter", + "run_night_changed_source_summary", +] diff --git a/songryeon_core/runtime/night_token_budget_layer_summary.py b/songryeon_core/runtime/night_token_budget_layer_summary.py new file mode 100644 index 0000000..3d5c4c8 --- /dev/null +++ b/songryeon_core/runtime/night_token_budget_layer_summary.py @@ -0,0 +1,962 @@ +from __future__ import annotations + +import json +import time +from dataclasses import asdict +from datetime import datetime +from pathlib import Path + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.graph_memory_export import record_graph_memory_export_packet +from songryeon_core.core.graph_vessel_adapter import record_graph_vessel_write_plan +from songryeon_core.core.graph_vessel_neo4j import ( + graph_vessel_neo4j_config_from_env, + record_graph_vessel_neo4j_write_result, +) +from songryeon_core.core.trace_store import TraceStore +from songryeon_core.llm.base import LLMRequest, LLMResponse +from songryeon_core.llm.runtime import ( + build_llm_adapter, + build_llm_runtime_config, + llm_runtime_status, +) +from songryeon_core.nodes.night_summarize_token_budget_bundle import ( + TokenBudgetSummaryBundleSpec, + build_token_budget_summary_bundle_specs, + collect_active_source_leaf_summary_graph_node_ids, + night_summarize_token_budget_bundle_frame_id, + night_summarize_token_budget_bundle_graph_node_id, + run_night_summarize_token_budget_bundle, +) +from songryeon_core.runtime.night_changed_source_summary import ( + DEFAULT_NIGHT_CHANGED_SOURCE_STORE_DIR, +) + + +DEFAULT_NIGHT_TOKEN_BUDGET_LAYER_BATCH_ID = "night_token_budget_layer_active" +DEFAULT_NIGHT_TOKEN_BUDGET_LAYER_MAX_BUNDLE_CHARS = 8000 +DEFAULT_NIGHT_TOKEN_BUDGET_TARGET_CONTEXT_CHARS = 12000 +DEFAULT_NIGHT_TOKEN_BUDGET_MAX_LAYER_DEPTH = 5 +DEFAULT_NIGHT_TOKEN_BUDGET_MAX_STEPS = 5 +DEFAULT_NIGHT_TOKEN_BUDGET_PROGRESS_JSONL = "night_token_layer_progress.jsonl" +NIGHT_TOKEN_BUDGET_LAYER_QUEUE_DATA_TYPE = "runtime:night_token_budget_layer_queue" +NIGHT_TOKEN_BUDGET_LAYER_QUEUE_GENERATOR = "CODE:NIGHT_TOKEN_BUDGET_LAYER_QUEUE" +NIGHT_TOKEN_BUDGET_VESSEL_WRITE_MODES = {"none", "every-step", "at-end"} + + +class NightTokenBudgetLayerFakeAdapter: + """Deterministic adapter for token-layer command tests.""" + + model_id = "night-token-budget-layer-fake-adapter" + + def complete(self, request: LLMRequest) -> LLMResponse: + source_summaries = request.input_payload.get("source_summary_payloads") + count = len(source_summaries) if isinstance(source_summaries, list) else 0 + payload = { + "summary_text": ( + "Fake token-budget bundle summary over " + f"{count} supplied source summaries." + ) + } + return LLMResponse( + text=json.dumps(payload, ensure_ascii=False), + model_id=self.model_id, + raw=payload, + ) + + +def run_night_token_budget_layer_summary( + *, + store_dir: str | Path = DEFAULT_NIGHT_CHANGED_SOURCE_STORE_DIR, + batch_id: str | None = None, + turn_id: str | None = None, + max_bundle_chars: int = DEFAULT_NIGHT_TOKEN_BUDGET_LAYER_MAX_BUNDLE_CHARS, + llm_mode: str = "off", + endpoint: str | None = None, + model_id: str | None = None, + timeout_seconds: int | None = None, + write_vessel: bool = False, + uri: str | None = None, + user: str | None = None, + password: str | None = None, + database: str | None = None, + allow_no_auth: bool = False, + until_context_budget: bool = False, + target_context_chars: int = DEFAULT_NIGHT_TOKEN_BUDGET_TARGET_CONTEXT_CHARS, + max_layer_depth: int = DEFAULT_NIGHT_TOKEN_BUDGET_MAX_LAYER_DEPTH, + max_steps: int = 1, + source_summary_graph_node_ids: list[str] | None = None, + max_runtime_minutes: float | None = None, + progress_jsonl: str | Path | None = None, + vessel_write_mode: str | None = None, + stop_on_failure: bool = True, +) -> dict[str, object]: + """Run one step of the source-summary token-budget layer summarizer.""" + + if max_bundle_chars <= 0: + raise ValueError("max_bundle_chars must be positive") + if target_context_chars <= 0: + raise ValueError("target_context_chars must be positive") + if max_layer_depth < 2: + raise ValueError("max_layer_depth must be at least 2") + if max_steps <= 0: + raise ValueError("max_steps must be positive") + if max_runtime_minutes is not None and max_runtime_minutes <= 0: + raise ValueError("max_runtime_minutes must be positive when set") + effective_vessel_write_mode = _effective_vessel_write_mode( + write_vessel=write_vessel, + vessel_write_mode=vessel_write_mode, + ) + write_vessel = effective_vessel_write_mode == "every-step" + if until_context_budget: + return _run_auto_reduce_until_context_budget( + store_dir=store_dir, + batch_id=batch_id, + turn_id=turn_id, + max_bundle_chars=max_bundle_chars, + target_context_chars=target_context_chars, + max_layer_depth=max_layer_depth, + max_steps=max_steps, + max_runtime_minutes=max_runtime_minutes, + progress_jsonl=progress_jsonl, + vessel_write_mode=effective_vessel_write_mode, + stop_on_failure=stop_on_failure, + llm_mode=llm_mode, + endpoint=endpoint, + model_id=model_id, + timeout_seconds=timeout_seconds, + uri=uri, + user=user, + password=password, + database=database, + allow_no_auth=allow_no_auth, + ) + + now = _now_iso() + safe_batch_id = batch_id or DEFAULT_NIGHT_TOKEN_BUDGET_LAYER_BATCH_ID + safe_turn_id = turn_id or f"turn_{safe_batch_id}" + cache_dir = Path(store_dir).resolve() + cache_dir.mkdir(parents=True, exist_ok=True) + trace_path = cache_dir / "trace_store.json" + data_path = cache_dir / "data_store.json" + trace_store = TraceStore.load_json(trace_path) if trace_path.exists() else TraceStore() + data_store = DataStore.load_json(data_path) if data_path.exists() else DataStore() + adapter = _build_summary_adapter( + llm_mode=llm_mode, + endpoint=endpoint, + model_id=model_id, + timeout_seconds=timeout_seconds, + ) + + queue_payload = _load_or_create_queue( + trace_store=trace_store, + data_store=data_store, + turn_id=safe_turn_id, + batch_id=safe_batch_id, + max_bundle_chars=max_bundle_chars, + source_summary_graph_node_ids=source_summary_graph_node_ids, + ) + specs = [ + TokenBudgetSummaryBundleSpec( + bundle_index=_int(item.get("bundle_index")), + bundle_graph_node_id=_text(item.get("bundle_graph_node_id")), + source_summary_graph_node_ids=_string_list( + item.get("source_summary_graph_node_ids") + ), + source_summary_char_count=_int(item.get("source_summary_char_count")), + char_budget=_int(item.get("char_budget")), + char_budget_status=_text(item.get("char_budget_status")), + ) + for item in _dict_list(queue_payload.get("bundle_specs")) + ] + processed_before = _processed_bundle_ids( + data_store=data_store, + specs=specs, + summary_run_id=safe_batch_id, + ) + pending_specs = [ + spec for spec in specs if spec.bundle_graph_node_id not in processed_before + ] + processed_result = None + if pending_specs: + processed_result = run_night_summarize_token_budget_bundle( + trace_store=trace_store, + data_store=data_store, + turn_id=safe_turn_id, + spec=pending_specs[0], + summary_run_id=safe_batch_id, + adapter=adapter, + ) + trace_store.save_json(trace_path) + data_store.save_json(data_path) + + processed_after = _processed_bundle_ids( + data_store=data_store, + specs=specs, + summary_run_id=safe_batch_id, + ) + pending_after = [ + spec for spec in specs if spec.bundle_graph_node_id not in processed_after + ] + if not specs: + completion_status = "no_candidates" + elif pending_after: + completion_status = "in_progress" + else: + completion_status = "complete" + + export_packet_id: str | None = None + write_plan_id: str | None = None + write_plan_status = "not_run" + write_result_payload: dict[str, object] | None = None + if _has_exportable_graph_records(data_store): + export_batch_id = f"{safe_batch_id}:token_layer_export:{_safe_timestamp(now)}" + export_result = record_graph_memory_export_packet( + trace_store=trace_store, + data_store=data_store, + turn_id=safe_turn_id, + batch_id=export_batch_id, + created_at=now, + ) + plan_result = record_graph_vessel_write_plan( + trace_store=trace_store, + data_store=data_store, + turn_id=safe_turn_id, + export_packet_id=export_result.packet.packet_id, + created_at=now, + ) + export_packet_id = export_result.packet.packet_id + write_plan_id = plan_result.plan.plan_id + write_plan_status = plan_result.plan.plan_status + if write_vessel: + config = graph_vessel_neo4j_config_from_env( + uri=uri, + user=user, + password=password, + database=database, + allow_no_auth=allow_no_auth, + ) + write_result = record_graph_vessel_neo4j_write_result( + trace_store=trace_store, + data_store=data_store, + turn_id=safe_turn_id, + plan_id=plan_result.plan.plan_id, + config=config, + created_at=now, + ) + write_result_payload = asdict(write_result.result) + + trace_store.save_json(trace_path) + data_store.save_json(data_path) + + return { + "status": "NIGHT_TOKEN_BUDGET_LAYER_SUMMARY_OK", + "execution_mode": "one_bundle_at_a_time", + "store_dir": cache_dir.as_posix(), + "batch_id": safe_batch_id, + "turn_id": safe_turn_id, + "llm_runtime": _llm_status_for_mode( + llm_mode=llm_mode, + endpoint=endpoint, + model_id=model_id, + timeout_seconds=timeout_seconds, + ), + "queue_id": queue_payload["queue_id"], + "queue_status": queue_payload["queue_status"], + "budget_unit": queue_payload["budget_unit"], + "max_bundle_chars": queue_payload["max_bundle_chars"], + "source_summary_count": queue_payload["source_summary_count"], + "bundle_count": len(specs), + "processed_this_run": 1 if processed_result is not None else 0, + "processed_bundle_graph_node_id": ( + processed_result.target_bundle_graph_node_id + if processed_result is not None + else None + ), + "processed_count_before": len(processed_before), + "processed_count_after": len(processed_after), + "pending_count_after": len(pending_after), + "next_bundle_graph_node_id": ( + pending_after[0].bundle_graph_node_id if pending_after else None + ), + "completion_status": completion_status, + "summary_status": ( + processed_result.frame.summary_status if processed_result is not None else None + ), + "summary_graph_node_id": ( + processed_result.summary_graph_node_data_id + if processed_result is not None + else None + ), + "target_bundle_graph_node_id": ( + processed_result.target_bundle_graph_node_id + if processed_result is not None + else None + ), + "export_packet_id": export_packet_id, + "write_plan_id": write_plan_id, + "write_plan_status": write_plan_status, + "write_vessel_requested": write_vessel, + "write_result": write_result_payload, + "trace_store_path": trace_path.as_posix(), + "data_store_path": data_path.as_posix(), + "trace_count": len(trace_store.list_events()), + "data_record_count": len(data_store.list_records()), + } + + +def _run_auto_reduce_until_context_budget( + *, + store_dir: str | Path, + batch_id: str | None, + turn_id: str | None, + max_bundle_chars: int, + target_context_chars: int, + max_layer_depth: int, + max_steps: int, + max_runtime_minutes: float | None, + progress_jsonl: str | Path | None, + vessel_write_mode: str, + stop_on_failure: bool, + llm_mode: str, + endpoint: str | None, + model_id: str | None, + timeout_seconds: int | None, + uri: str | None, + user: str | None, + password: str | None, + database: str | None, + allow_no_auth: bool, +) -> dict[str, object]: + safe_batch_id = batch_id or DEFAULT_NIGHT_TOKEN_BUDGET_LAYER_BATCH_ID + safe_turn_id = turn_id or f"turn_{safe_batch_id}:auto_reduce" + cache_dir = Path(store_dir).resolve() + progress_path = ( + Path(progress_jsonl).resolve() + if progress_jsonl is not None + else cache_dir / DEFAULT_NIGHT_TOKEN_BUDGET_PROGRESS_JSONL + ) + started = time.monotonic() + steps: list[dict[str, object]] = [] + auto_status = "max_steps_reached" + final_layer_depth = 1 + final_summary_count = 0 + final_summary_char_count = 0 + last_result: dict[str, object] | None = None + + for step_index in range(1, max_steps + 1): + elapsed_seconds = time.monotonic() - started + if ( + max_runtime_minutes is not None + and elapsed_seconds >= max_runtime_minutes * 60 + ): + auto_status = "max_runtime_reached" + event = { + "event_type": "auto_reduce_stop", + "step_index": step_index, + "auto_reduce_status": auto_status, + "elapsed_seconds": round(elapsed_seconds, 3), + } + steps.append({**event, "processed_this_step": 0}) + _append_progress_jsonl(progress_path, event) + break + data_store = _load_data_store(cache_dir) + decision = _next_auto_reduce_decision( + data_store=data_store, + base_batch_id=safe_batch_id, + target_context_chars=target_context_chars, + max_layer_depth=max_layer_depth, + ) + final_layer_depth = _int(decision.get("current_layer_depth")) + final_summary_count = _int(decision.get("current_summary_count")) + final_summary_char_count = _int(decision.get("current_summary_char_count")) + if decision["action"] != "process_step": + auto_status = _text(decision.get("action")) + event = { + "event_type": "auto_reduce_stop", + "step_index": step_index, + **decision, + "processed_this_step": 0, + "auto_reduce_status": auto_status, + "elapsed_seconds": round(time.monotonic() - started, 3), + } + steps.append(event) + _append_progress_jsonl(progress_path, event) + break + + layer_depth = _int(decision.get("target_layer_depth")) + layer_batch_id = _text(decision.get("target_batch_id")) + source_ids = _string_list(decision.get("source_summary_graph_node_ids")) + last_result = run_night_token_budget_layer_summary( + store_dir=cache_dir, + batch_id=layer_batch_id, + turn_id=f"{safe_turn_id}:step_{step_index:04d}:layer_{layer_depth:02d}", + max_bundle_chars=max_bundle_chars, + llm_mode=llm_mode, + endpoint=endpoint, + model_id=model_id, + timeout_seconds=timeout_seconds, + write_vessel=vessel_write_mode == "every-step", + uri=uri, + user=user, + password=password, + database=database, + allow_no_auth=allow_no_auth, + until_context_budget=False, + target_context_chars=target_context_chars, + max_layer_depth=max_layer_depth, + max_steps=1, + source_summary_graph_node_ids=source_ids, + ) + step_event = { + "event_type": "auto_reduce_step", + "step_index": step_index, + **decision, + "processed_this_step": last_result.get("processed_this_run"), + "step_completion_status": last_result.get("completion_status"), + "step_summary_status": last_result.get("summary_status"), + "step_summary_graph_node_id": last_result.get("summary_graph_node_id"), + "elapsed_seconds": round(time.monotonic() - started, 3), + } + steps.append(step_event) + _append_progress_jsonl(progress_path, step_event) + if last_result.get("processed_this_run") != 1: + auto_status = "blocked_no_progress" + break + if stop_on_failure and last_result.get("summary_status") != "ran": + auto_status = "stopped_on_failure" + break + else: + data_store = _load_data_store(cache_dir) + decision = _next_auto_reduce_decision( + data_store=data_store, + base_batch_id=safe_batch_id, + target_context_chars=target_context_chars, + max_layer_depth=max_layer_depth, + ) + final_layer_depth = _int(decision.get("current_layer_depth")) + final_summary_count = _int(decision.get("current_summary_count")) + final_summary_char_count = _int(decision.get("current_summary_char_count")) + if decision["action"] != "process_step": + auto_status = _text(decision.get("action")) + + final_write_result: dict[str, object] | None = None + if vessel_write_mode == "at-end": + final_write_result = _record_final_vessel_write( + cache_dir=cache_dir, + turn_id=f"{safe_turn_id}:final_vessel_write", + batch_id=f"{safe_batch_id}:checkpointed_final", + uri=uri, + user=user, + password=password, + database=database, + allow_no_auth=allow_no_auth, + ) + _append_progress_jsonl( + progress_path, + { + "event_type": "auto_reduce_final_vessel_write", + "auto_reduce_status": auto_status, + "write_status": final_write_result.get("write_status"), + "elapsed_seconds": round(time.monotonic() - started, 3), + }, + ) + + return { + "status": "NIGHT_TOKEN_BUDGET_AUTO_REDUCE_OK", + "execution_mode": "auto_reduce_until_context_budget", + "store_dir": cache_dir.as_posix(), + "batch_id": safe_batch_id, + "turn_id": safe_turn_id, + "budget_unit": "characters", + "max_bundle_chars": max_bundle_chars, + "target_context_chars": target_context_chars, + "max_layer_depth": max_layer_depth, + "max_steps": max_steps, + "max_runtime_minutes": max_runtime_minutes, + "progress_jsonl_path": progress_path.as_posix(), + "vessel_write_mode": vessel_write_mode, + "stop_on_failure": stop_on_failure, + "step_count": len(steps), + "auto_reduce_status": auto_status, + "final_layer_depth": final_layer_depth, + "final_summary_count": final_summary_count, + "final_summary_char_count": final_summary_char_count, + "last_step_result": last_result, + "final_write_result": final_write_result, + "steps": steps, + } + + +def _next_auto_reduce_decision( + *, + data_store: DataStore, + base_batch_id: str, + target_context_chars: int, + max_layer_depth: int, +) -> dict[str, object]: + leaf_ids = collect_active_source_leaf_summary_graph_node_ids(data_store) + leaf_chars = _summary_text_char_count( + data_store=data_store, + summary_graph_node_ids=leaf_ids, + ) + if not leaf_ids: + return { + "action": "no_candidates", + "current_layer_depth": 1, + "current_summary_count": 0, + "current_summary_char_count": 0, + } + if leaf_chars <= target_context_chars: + return { + "action": "target_context_reached", + "current_layer_depth": 1, + "current_summary_count": len(leaf_ids), + "current_summary_char_count": leaf_chars, + } + + previous_layer_summary_ids = leaf_ids + previous_layer_chars = leaf_chars + previous_layer_depth = 1 + for target_layer_depth in range(2, max_layer_depth + 1): + layer_batch_id = _layer_batch_id(base_batch_id, target_layer_depth) + queue = _load_queue_payload(data_store=data_store, batch_id=layer_batch_id) + if queue is None: + return { + "action": "process_step", + "current_layer_depth": previous_layer_depth, + "current_summary_count": len(previous_layer_summary_ids), + "current_summary_char_count": previous_layer_chars, + "target_layer_depth": target_layer_depth, + "target_batch_id": layer_batch_id, + "source_summary_graph_node_ids": previous_layer_summary_ids, + "queue_status": "missing_create_next", + } + + specs = _specs_from_queue(queue) + processed_ids = _processed_bundle_ids( + data_store=data_store, + specs=specs, + summary_run_id=layer_batch_id, + ) + if len(processed_ids) < len(specs): + return { + "action": "process_step", + "current_layer_depth": previous_layer_depth, + "current_summary_count": len(previous_layer_summary_ids), + "current_summary_char_count": previous_layer_chars, + "target_layer_depth": target_layer_depth, + "target_batch_id": layer_batch_id, + "source_summary_graph_node_ids": _string_list( + queue.get("source_summary_graph_node_ids") + ), + "queue_status": "existing_incomplete", + "queue_bundle_count": len(specs), + "queue_processed_count": len(processed_ids), + "queue_pending_count": len(specs) - len(processed_ids), + } + + current_layer_summary_ids = _successful_summary_ids_for_specs( + data_store=data_store, + specs=specs, + summary_run_id=layer_batch_id, + ) + current_layer_chars = _summary_text_char_count( + data_store=data_store, + summary_graph_node_ids=current_layer_summary_ids, + ) + if not current_layer_summary_ids: + return { + "action": "blocked_no_successful_summaries", + "current_layer_depth": target_layer_depth, + "current_summary_count": 0, + "current_summary_char_count": 0, + "target_batch_id": layer_batch_id, + } + if current_layer_chars <= target_context_chars: + return { + "action": "target_context_reached", + "current_layer_depth": target_layer_depth, + "current_summary_count": len(current_layer_summary_ids), + "current_summary_char_count": current_layer_chars, + "target_batch_id": layer_batch_id, + } + if target_layer_depth == max_layer_depth: + return { + "action": "max_layer_depth_reached", + "current_layer_depth": target_layer_depth, + "current_summary_count": len(current_layer_summary_ids), + "current_summary_char_count": current_layer_chars, + "target_batch_id": layer_batch_id, + } + previous_layer_summary_ids = current_layer_summary_ids + previous_layer_chars = current_layer_chars + previous_layer_depth = target_layer_depth + + return { + "action": "max_layer_depth_reached", + "current_layer_depth": previous_layer_depth, + "current_summary_count": len(previous_layer_summary_ids), + "current_summary_char_count": previous_layer_chars, + } + + +def _load_or_create_queue( + *, + trace_store: TraceStore, + data_store: DataStore, + turn_id: str, + batch_id: str, + max_bundle_chars: int, + source_summary_graph_node_ids: list[str] | None = None, +) -> dict[str, object]: + queue_id = _queue_id(batch_id) + existing = data_store.get_record(queue_id) + if existing is not None: + if existing.data_type != NIGHT_TOKEN_BUDGET_LAYER_QUEUE_DATA_TYPE: + raise ValueError(f"night token layer queue data_id collision: {queue_id}") + if not isinstance(existing.payload, dict): + raise TypeError("night token layer queue payload must be dict") + return {**existing.payload, "queue_status": "existing"} + + selected_source_summary_graph_node_ids = ( + list(source_summary_graph_node_ids) + if source_summary_graph_node_ids is not None + else collect_active_source_leaf_summary_graph_node_ids(data_store) + ) + specs = build_token_budget_summary_bundle_specs( + data_store=data_store, + summary_run_id=batch_id, + source_summary_graph_node_ids=selected_source_summary_graph_node_ids, + max_bundle_chars=max_bundle_chars, + ) + payload = { + "queue_id": queue_id, + "summary_run_id": batch_id, + "queue_status": "created", + "budget_unit": "characters", + "max_bundle_chars": max_bundle_chars, + "source_summary_count": len(selected_source_summary_graph_node_ids), + "source_summary_graph_node_ids": selected_source_summary_graph_node_ids, + "source_summary_char_count": _summary_text_char_count( + data_store=data_store, + summary_graph_node_ids=selected_source_summary_graph_node_ids, + ), + "bundle_specs": [asdict(spec) for spec in specs], + "generated_by": NIGHT_TOKEN_BUDGET_LAYER_QUEUE_GENERATOR, + "info_class": "absolute", + "semantic_judgement_status": "not_run", + } + event = trace_store.create_event( + turn_id=turn_id, + actor="night_token_budget_layer_queue", + event_type="node_output", + output_ref=[queue_id], + schema_status="passed", + ) + data_store.create_record( + data_id=queue_id, + data_type=NIGHT_TOKEN_BUDGET_LAYER_QUEUE_DATA_TYPE, + exists=True, + created_at=event.timestamp, + source_trace_id=event.event_id, + payload=payload, + ) + return payload + + +def _processed_bundle_ids( + *, + data_store: DataStore, + specs: list[TokenBudgetSummaryBundleSpec], + summary_run_id: str, +) -> list[str]: + processed: list[str] = [] + for spec in specs: + summary_node_id = night_summarize_token_budget_bundle_graph_node_id( + spec.bundle_graph_node_id, + summary_run_id=summary_run_id, + ) + frame_id = night_summarize_token_budget_bundle_frame_id( + spec.bundle_graph_node_id, + summary_run_id=summary_run_id, + ) + if data_store.get_record(summary_node_id) is not None: + processed.append(spec.bundle_graph_node_id) + continue + if data_store.get_record(frame_id) is not None: + processed.append(spec.bundle_graph_node_id) + return processed + + +def _load_data_store(cache_dir: Path) -> DataStore: + data_path = cache_dir / "data_store.json" + return DataStore.load_json(data_path) if data_path.exists() else DataStore() + + +def _load_queue_payload( + *, + data_store: DataStore, + batch_id: str, +) -> dict[str, object] | None: + record = data_store.get_record(_queue_id(batch_id)) + if record is None: + return None + if record.data_type != NIGHT_TOKEN_BUDGET_LAYER_QUEUE_DATA_TYPE: + raise ValueError(f"night token layer queue data_id collision: {record.data_id}") + if not isinstance(record.payload, dict): + raise TypeError("night token layer queue payload must be dict") + return dict(record.payload) + + +def _specs_from_queue(queue_payload: dict[str, object]) -> list[TokenBudgetSummaryBundleSpec]: + return [ + TokenBudgetSummaryBundleSpec( + bundle_index=_int(item.get("bundle_index")), + bundle_graph_node_id=_text(item.get("bundle_graph_node_id")), + source_summary_graph_node_ids=_string_list( + item.get("source_summary_graph_node_ids") + ), + source_summary_char_count=_int(item.get("source_summary_char_count")), + char_budget=_int(item.get("char_budget")), + char_budget_status=_text(item.get("char_budget_status")), + ) + for item in _dict_list(queue_payload.get("bundle_specs")) + ] + + +def _successful_summary_ids_for_specs( + *, + data_store: DataStore, + specs: list[TokenBudgetSummaryBundleSpec], + summary_run_id: str, +) -> list[str]: + result: list[str] = [] + for spec in specs: + summary_node_id = night_summarize_token_budget_bundle_graph_node_id( + spec.bundle_graph_node_id, + summary_run_id=summary_run_id, + ) + record = data_store.get_record(summary_node_id) + if record is None or record.data_type != "graph_memory:node:summary": + continue + if not isinstance(record.payload, dict): + continue + if record.payload.get("summary_status") != "ran": + continue + if record.payload.get("validity_status") != "active": + continue + result.append(summary_node_id) + return result + + +def _summary_text_char_count( + *, + data_store: DataStore, + summary_graph_node_ids: list[str], +) -> int: + total = 0 + for summary_graph_node_id in summary_graph_node_ids: + record = data_store.get_record(summary_graph_node_id) + if record is None or not isinstance(record.payload, dict): + continue + total += len(_text(record.payload.get("summary_text"))) + return total + + +def _layer_batch_id(base_batch_id: str, layer_depth: int) -> str: + if layer_depth < 2: + raise ValueError("layer_depth must be at least 2") + if layer_depth == 2: + return base_batch_id + return f"{base_batch_id}:layer_{layer_depth:02d}" + + +def _has_exportable_graph_records(data_store: DataStore) -> bool: + for record in data_store.list_records(): + if record.data_type.startswith("graph_memory:"): + return True + if record.data_type.startswith("graph_source:"): + return True + return False + + +def _effective_vessel_write_mode( + *, + write_vessel: bool, + vessel_write_mode: str | None, +) -> str: + if vessel_write_mode is None: + return "every-step" if write_vessel else "none" + normalized = vessel_write_mode.strip().lower().replace("_", "-") + if normalized not in NIGHT_TOKEN_BUDGET_VESSEL_WRITE_MODES: + allowed = ", ".join(sorted(NIGHT_TOKEN_BUDGET_VESSEL_WRITE_MODES)) + raise ValueError(f"unknown vessel_write_mode: {vessel_write_mode}; allowed: {allowed}") + return normalized + + +def _append_progress_jsonl(path: Path, payload: dict[str, object]) -> None: + path.parent.mkdir(parents=True, exist_ok=True) + line = { + "created_at": _now_iso(), + **payload, + } + with path.open("a", encoding="utf-8", newline="\n") as handle: + handle.write(json.dumps(line, ensure_ascii=False, sort_keys=True)) + handle.write("\n") + + +def _record_final_vessel_write( + *, + cache_dir: Path, + turn_id: str, + batch_id: str, + uri: str | None, + user: str | None, + password: str | None, + database: str | None, + allow_no_auth: bool, +) -> dict[str, object]: + trace_path = cache_dir / "trace_store.json" + data_path = cache_dir / "data_store.json" + trace_store = TraceStore.load_json(trace_path) if trace_path.exists() else TraceStore() + data_store = DataStore.load_json(data_path) if data_path.exists() else DataStore() + if not _has_exportable_graph_records(data_store): + return { + "write_status": "not_run", + "failure_type": "no_exportable_graph_records", + "failure_reason": "No graph_memory or graph_source records were available.", + } + + now = _now_iso() + export_result = record_graph_memory_export_packet( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + batch_id=f"{batch_id}:export:{_safe_timestamp(now)}", + created_at=now, + ) + plan_result = record_graph_vessel_write_plan( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + export_packet_id=export_result.packet.packet_id, + created_at=now, + ) + config = graph_vessel_neo4j_config_from_env( + uri=uri, + user=user, + password=password, + database=database, + allow_no_auth=allow_no_auth, + ) + write_result = record_graph_vessel_neo4j_write_result( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + plan_id=plan_result.plan.plan_id, + config=config, + created_at=now, + ) + trace_store.save_json(trace_path) + data_store.save_json(data_path) + payload = asdict(write_result.result) + payload["export_packet_id"] = export_result.packet.packet_id + payload["write_plan_id"] = plan_result.plan.plan_id + return payload + + +def _build_summary_adapter( + *, + llm_mode: str, + endpoint: str | None, + model_id: str | None, + timeout_seconds: int | None, +): + selected_mode = llm_mode.strip().lower() + if selected_mode == "fake": + return NightTokenBudgetLayerFakeAdapter() + config = build_llm_runtime_config( + mode=selected_mode, + endpoint=endpoint, + model_id=model_id, + timeout_seconds=timeout_seconds, + ) + return build_llm_adapter(config, endpoint=endpoint) + + +def _llm_status_for_mode( + *, + llm_mode: str, + endpoint: str | None, + model_id: str | None, + timeout_seconds: int | None, +) -> dict[str, object]: + selected_mode = llm_mode.strip().lower() + if selected_mode == "fake": + return { + "mode": "fake", + "enabled": True, + "adapter_kind": "fake", + "model_id": NightTokenBudgetLayerFakeAdapter.model_id, + "source": "arguments", + } + config = build_llm_runtime_config( + mode=selected_mode, + endpoint=endpoint, + model_id=model_id, + timeout_seconds=timeout_seconds, + ) + return llm_runtime_status(config) + + +def _queue_id(batch_id: str) -> str: + return f"night_summary:token_budget_layer_queue:{_safe_id_part(batch_id)}" + + +def _now_iso() -> str: + return datetime.now().isoformat(timespec="microseconds") + + +def _safe_timestamp(timestamp: str) -> str: + return "".join(char if char.isalnum() else "_" for char in timestamp).strip("_") + + +def _safe_id_part(value: str) -> str: + cleaned = "".join(char if char.isalnum() else "_" for char in value.strip()) + cleaned = cleaned.strip("_") + if not cleaned: + raise ValueError("safe id part must not be empty") + return cleaned + + +def _text(value: object) -> str: + return value if isinstance(value, str) else "" + + +def _int(value: object) -> int: + return value if isinstance(value, int) and value >= 0 else 0 + + +def _string_list(value: object) -> list[str]: + if not isinstance(value, list): + return [] + return [item for item in value if isinstance(item, str) and item] + + +def _dict_list(value: object) -> list[dict[str, object]]: + if not isinstance(value, list): + return [] + return [dict(item) for item in value if isinstance(item, dict)] + + +__all__ = [ + "DEFAULT_NIGHT_TOKEN_BUDGET_LAYER_BATCH_ID", + "DEFAULT_NIGHT_TOKEN_BUDGET_LAYER_MAX_BUNDLE_CHARS", + "DEFAULT_NIGHT_TOKEN_BUDGET_MAX_LAYER_DEPTH", + "DEFAULT_NIGHT_TOKEN_BUDGET_MAX_STEPS", + "DEFAULT_NIGHT_TOKEN_BUDGET_TARGET_CONTEXT_CHARS", + "NIGHT_TOKEN_BUDGET_LAYER_QUEUE_DATA_TYPE", + "NightTokenBudgetLayerFakeAdapter", + "run_night_token_budget_layer_summary", +] diff --git a/songryeon_core/runtime/r_loop_vessel_one_step.py b/songryeon_core/runtime/r_loop_vessel_one_step.py new file mode 100644 index 0000000..574376e --- /dev/null +++ b/songryeon_core/runtime/r_loop_vessel_one_step.py @@ -0,0 +1,365 @@ +from __future__ import annotations + +from dataclasses import asdict +from datetime import datetime + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.graph_vessel_neo4j import graph_vessel_neo4j_config_from_env +from songryeon_core.core.r_loop_vessel_read_packet import ( + record_r_loop_vessel_read_packet, +) +from songryeon_core.core.trace_store import TraceStore +from songryeon_core.llm.runtime import ( + build_llm_adapter, + build_llm_runtime_config, + llm_runtime_status, +) +from songryeon_core.loops.r_loop_vessel_one_step import ( + RLoopVesselOneStepFakeLLMAdapter, + RLoopVesselTraverseFakeLLMAdapter, + run_r_loop_vessel_one_step, + run_r_loop_vessel_traverse, +) + + +def run_local_r_loop_vessel_one_step( + *, + user_question: str, + batch_id: str = "manual_r_loop_vessel_one_step", + turn_id: str = "turn_r_loop_vessel_one_step_0001", + uri: str | None = None, + user: str | None = None, + password: str | None = None, + database: str | None = None, + allow_no_auth: bool = False, + limit: int = 50, + llm_mode: str = "fake", + endpoint: str | None = None, + model_id: str | None = None, + timeout_seconds: int | None = None, +) -> dict[str, object]: + """Read a Vessel packet and run one guarded R traversal step.""" + + now = _now_iso() + trace_store = TraceStore() + data_store = DataStore() + config = graph_vessel_neo4j_config_from_env( + uri=uri, + user=user, + password=password, + database=database, + allow_no_auth=allow_no_auth, + ) + read_packet = record_r_loop_vessel_read_packet( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + batch_id=f"{batch_id}:read_packet", + config=config, + created_at=now, + limit=limit, + ) + runtime_config = build_llm_runtime_config( + mode=llm_mode, + endpoint=endpoint, + model_id=model_id, + timeout_seconds=timeout_seconds, + ) + if runtime_config.mode == "fake": + adapter = RLoopVesselOneStepFakeLLMAdapter() + else: + adapter = build_llm_adapter(runtime_config, endpoint=endpoint) + + run = run_r_loop_vessel_one_step( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + user_question=user_question, + read_packet=read_packet.packet, + adapter=adapter, + frame_label=batch_id, + input_ref=[read_packet.trace_event_id], + ) + + return { + "status": "R_LOOP_VESSEL_ONE_STEP_OK" + if run.result_frame.one_step_status == "completed" + else "R_LOOP_VESSEL_ONE_STEP_NOT_PASSED", + "one_step_status": run.result_frame.one_step_status, + "failure_stage": run.result_frame.failure_stage, + "failure_type": run.result_frame.failure_type, + "failure_reason": run.result_frame.failure_reason, + "failure_payload_summary": run.result_frame.failure_payload_summary, + "read_packet_status": read_packet.packet.read_status, + "packet_id": read_packet.packet.packet_id, + "entry_candidate_count": read_packet.packet.entry_candidate_count, + "summary_candidate_count": read_packet.packet.summary_candidate_count, + "selected_graph_node_id": run.result_frame.selected_graph_node_id, + "inspected_graph_node_id": run.result_frame.inspected_graph_node_id, + "sufficiency_status": run.result_frame.sufficiency_status, + "continuation_status": run.result_frame.continuation_status, + "r_loop_task_status": run.return_summary.r_loop_task_status if run.return_summary else None, + "hierarchy_child_candidate_count": ( + run.graph_traversal_candidate_surface.candidate_count + if run.graph_traversal_candidate_surface is not None + else None + ), + "hierarchy_child_candidate_node_ids": ( + run.graph_traversal_candidate_surface.candidate_graph_node_ids + if run.graph_traversal_candidate_surface is not None + else [] + ), + "llm_runtime": llm_runtime_status(runtime_config), + "neo4j_uri": config.uri, + "neo4j_user": config.user, + "neo4j_password_configured": config.password is not None, + "neo4j_allow_no_auth": config.allow_no_auth, + "trace_count": len(trace_store.list_events()), + "data_record_count": len(data_store.list_records()), + "result_frame": asdict(run.result_frame), + "r1_goal_frame": asdict(run.r1_goal) if run.r1_goal is not None else None, + "r2_selection_frame": asdict(run.r2_selection) + if run.r2_selection is not None + else None, + "r3_inspection_frame": asdict(run.r3_inspection) + if run.r3_inspection is not None + else None, + "continuation_frame": asdict(run.continuation) + if run.continuation is not None + else None, + "return_summary_frame": asdict(run.return_summary) + if run.return_summary is not None + else None, + "graph_traversal_candidate_surface_frame": ( + asdict(run.graph_traversal_candidate_surface) + if run.graph_traversal_candidate_surface is not None + else None + ), + } + + +def run_local_r_loop_vessel_traverse( + *, + user_question: str, + batch_id: str = "manual_r_loop_vessel_traverse", + turn_id: str = "turn_r_loop_vessel_traverse_0001", + uri: str | None = None, + user: str | None = None, + password: str | None = None, + database: str | None = None, + allow_no_auth: bool = False, + limit: int = 50, + llm_mode: str = "fake", + endpoint: str | None = None, + model_id: str | None = None, + timeout_seconds: int | None = None, +) -> dict[str, object]: + """Read a Vessel packet and run a guarded multi-step R traversal.""" + + now = _now_iso() + trace_store = TraceStore() + data_store = DataStore() + config = graph_vessel_neo4j_config_from_env( + uri=uri, + user=user, + password=password, + database=database, + allow_no_auth=allow_no_auth, + ) + read_packet = record_r_loop_vessel_read_packet( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + batch_id=f"{batch_id}:read_packet", + config=config, + created_at=now, + limit=limit, + ) + runtime_config = build_llm_runtime_config( + mode=llm_mode, + endpoint=endpoint, + model_id=model_id, + timeout_seconds=timeout_seconds, + ) + if runtime_config.mode == "fake": + adapter = RLoopVesselTraverseFakeLLMAdapter() + else: + adapter = build_llm_adapter(runtime_config, endpoint=endpoint) + + run = run_r_loop_vessel_traverse( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + user_question=user_question, + read_packet=read_packet.packet, + adapter=adapter, + frame_label=batch_id, + input_ref=[read_packet.trace_event_id], + ) + + return { + "status": "R_LOOP_VESSEL_TRAVERSE_OK" + if run.result_frame.traverse_status == "completed" + else "R_LOOP_VESSEL_TRAVERSE_NOT_PASSED", + "traverse_status": run.result_frame.traverse_status, + "failure_stage": run.result_frame.failure_stage, + "failure_type": run.result_frame.failure_type, + "failure_reason": run.result_frame.failure_reason, + "failure_payload_summary": run.result_frame.failure_payload_summary, + "read_packet_status": read_packet.packet.read_status, + "packet_id": read_packet.packet.packet_id, + "entry_candidate_count": read_packet.packet.entry_candidate_count, + "summary_candidate_count": read_packet.packet.summary_candidate_count, + "step_count": run.result_frame.step_count, + "selected_graph_node_ids": run.result_frame.selected_graph_node_ids, + "inspected_graph_node_ids": run.result_frame.inspected_graph_node_ids, + "final_graph_node_id": run.result_frame.final_graph_node_id, + "final_sufficiency_status": run.result_frame.final_sufficiency_status, + "final_continuation_status": run.result_frame.final_continuation_status, + "r_loop_task_status": run.result_frame.r_loop_task_status, + "terminal_material_seen_count": run.result_frame.terminal_material_seen_count, + "min_terminal_material_count": run.result_frame.min_terminal_material_count, + "raw_original_material_seen_count": ( + run.result_frame.raw_original_material_seen_count + ), + "max_raw_original_material_count": ( + run.result_frame.max_raw_original_material_count + ), + "raw_original_read_cap_reached": run.result_frame.raw_original_read_cap_reached, + "early_stop_guard_trigger_count": run.result_frame.early_stop_guard_trigger_count, + "candidate_surface_frame_ids": run.result_frame.candidate_surface_frame_ids, + "graph_traversal_candidate_surface_frame_ids": ( + run.result_frame.graph_traversal_candidate_surface_frame_ids + ), + "llm_runtime": llm_runtime_status(runtime_config), + "neo4j_uri": config.uri, + "neo4j_user": config.user, + "neo4j_password_configured": config.password is not None, + "neo4j_allow_no_auth": config.allow_no_auth, + "trace_count": len(trace_store.list_events()), + "data_record_count": len(data_store.list_records()), + "result_frame": asdict(run.result_frame), + "r1_goal_frame": asdict(run.r1_goal) if run.r1_goal is not None else None, + "r2_selection_frames": [asdict(frame) for frame in run.r2_selections], + "r3_inspection_frames": [asdict(frame) for frame in run.r3_inspections], + "continuation_frames": [asdict(frame) for frame in run.continuations], + "return_summary_frame": asdict(run.return_summary) + if run.return_summary is not None + else None, + } + + +def render_r_loop_vessel_one_step_text(result: dict[str, object]) -> str: + lines = [ + f"status: {result.get('status')}", + f"one_step_status: {result.get('one_step_status')}", + f"read_packet_status: {result.get('read_packet_status')}", + f"entry_candidate_count: {result.get('entry_candidate_count')}", + f"summary_candidate_count: {result.get('summary_candidate_count')}", + f"selected_graph_node_id: {result.get('selected_graph_node_id')}", + f"inspected_graph_node_id: {result.get('inspected_graph_node_id')}", + f"sufficiency_status: {result.get('sufficiency_status')}", + f"continuation_status: {result.get('continuation_status')}", + f"r_loop_task_status: {result.get('r_loop_task_status')}", + f"hierarchy_child_candidate_count: {result.get('hierarchy_child_candidate_count')}", + ] + child_ids = result.get("hierarchy_child_candidate_node_ids") + if isinstance(child_ids, list) and child_ids: + lines.append("hierarchy_child_candidate_node_ids:") + for child_id in child_ids[:20]: + lines.append(f" - {child_id}") + if len(child_ids) > 20: + lines.append(f" ... +{len(child_ids) - 20} more") + failure_stage = result.get("failure_stage") + failure_type = result.get("failure_type") + failure_reason = result.get("failure_reason") + if failure_stage or failure_type or failure_reason: + lines.append(f"failure_stage: {failure_stage}") + lines.append(f"failure_type: {failure_type}") + lines.append(f"failure_reason: {failure_reason}") + failure_payload_summary = result.get("failure_payload_summary") + if isinstance(failure_payload_summary, dict) and failure_payload_summary: + lines.append(f"failure_payload_summary: {failure_payload_summary}") + r1 = result.get("r1_goal_frame") + if isinstance(r1, dict): + lines.append("") + lines.append(f"R1 goal: {r1.get('graph_search_goal')}") + lines.append(f"R1 granularity: {r1.get('required_information_granularity')}") + r2 = result.get("r2_selection_frame") + if isinstance(r2, dict): + lines.append("") + lines.append(f"R2 selection_reason: {r2.get('selection_reason')}") + r3 = result.get("r3_inspection_frame") + if isinstance(r3, dict): + lines.append("") + lines.append(f"R3 inspection_reason: {r3.get('inspection_reason')}") + return "\n".join(lines) + + +def render_r_loop_vessel_traverse_text(result: dict[str, object]) -> str: + lines = [ + f"status: {result.get('status')}", + f"traverse_status: {result.get('traverse_status')}", + f"read_packet_status: {result.get('read_packet_status')}", + f"entry_candidate_count: {result.get('entry_candidate_count')}", + f"summary_candidate_count: {result.get('summary_candidate_count')}", + f"step_count: {result.get('step_count')}", + f"final_graph_node_id: {result.get('final_graph_node_id')}", + f"final_sufficiency_status: {result.get('final_sufficiency_status')}", + f"final_continuation_status: {result.get('final_continuation_status')}", + f"r_loop_task_status: {result.get('r_loop_task_status')}", + f"terminal_material_seen_count: {result.get('terminal_material_seen_count')}", + f"min_terminal_material_count: {result.get('min_terminal_material_count')}", + f"raw_original_material_seen_count: {result.get('raw_original_material_seen_count')}", + f"max_raw_original_material_count: {result.get('max_raw_original_material_count')}", + f"raw_original_read_cap_reached: {result.get('raw_original_read_cap_reached')}", + f"early_stop_guard_trigger_count: {result.get('early_stop_guard_trigger_count')}", + ] + failure_stage = result.get("failure_stage") + failure_type = result.get("failure_type") + failure_reason = result.get("failure_reason") + if failure_stage or failure_type or failure_reason: + lines.append(f"failure_stage: {failure_stage}") + lines.append(f"failure_type: {failure_type}") + lines.append(f"failure_reason: {failure_reason}") + failure_payload_summary = result.get("failure_payload_summary") + if isinstance(failure_payload_summary, dict) and failure_payload_summary: + lines.append(f"failure_payload_summary: {failure_payload_summary}") + r1 = result.get("r1_goal_frame") + if isinstance(r1, dict): + lines.append("") + lines.append(f"R1 goal: {r1.get('graph_search_goal')}") + lines.append(f"R1 granularity: {r1.get('required_information_granularity')}") + r2_frames = result.get("r2_selection_frames") + r3_frames = result.get("r3_inspection_frames") + if isinstance(r2_frames, list) or isinstance(r3_frames, list): + lines.append("") + lines.append("Traversal path:") + max_len = max( + len(r2_frames) if isinstance(r2_frames, list) else 0, + len(r3_frames) if isinstance(r3_frames, list) else 0, + ) + for index in range(max_len): + r2 = r2_frames[index] if isinstance(r2_frames, list) and index < len(r2_frames) else None + r3 = r3_frames[index] if isinstance(r3_frames, list) and index < len(r3_frames) else None + selected = r2.get("selected_graph_node_id") if isinstance(r2, dict) else None + inspected = r3.get("inspected_graph_node_id") if isinstance(r3, dict) else None + sufficiency = r3.get("sufficiency_status") if isinstance(r3, dict) else None + action = r3.get("recommended_next_action") if isinstance(r3, dict) else None + lines.append( + f" step {index + 1}: selected={selected} inspected={inspected} " + f"sufficiency={sufficiency} action={action}" + ) + return "\n".join(lines) + + +def _now_iso() -> str: + return datetime.now().isoformat(timespec="seconds") + + +__all__ = [ + "render_r_loop_vessel_one_step_text", + "render_r_loop_vessel_traverse_text", + "run_local_r_loop_vessel_one_step", + "run_local_r_loop_vessel_traverse", +] diff --git a/songryeon_core/runtime/r_loop_vessel_read_packet.py b/songryeon_core/runtime/r_loop_vessel_read_packet.py new file mode 100644 index 0000000..64e6b67 --- /dev/null +++ b/songryeon_core/runtime/r_loop_vessel_read_packet.py @@ -0,0 +1,147 @@ +from __future__ import annotations + +from dataclasses import asdict +from datetime import datetime + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.graph_vessel_neo4j import graph_vessel_neo4j_config_from_env +from songryeon_core.core.r_loop_vessel_read_packet import ( + record_r_loop_vessel_read_packet, +) +from songryeon_core.core.trace_store import TraceStore + + +def run_local_r_loop_vessel_read_packet( + *, + batch_id: str = "manual_r_loop_vessel_read_packet", + turn_id: str = "turn_r_loop_vessel_read_packet_0001", + uri: str | None = None, + user: str | None = None, + password: str | None = None, + database: str | None = None, + allow_no_auth: bool = False, + limit: int = 50, +) -> dict[str, object]: + """Build a read-only Neo4j Vessel candidate packet for future R loop traversal.""" + + now = _now_iso() + trace_store = TraceStore() + data_store = DataStore() + config = graph_vessel_neo4j_config_from_env( + uri=uri, + user=user, + password=password, + database=database, + allow_no_auth=allow_no_auth, + ) + recorded = record_r_loop_vessel_read_packet( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + batch_id=batch_id, + config=config, + created_at=now, + limit=limit, + ) + + packet_payload = asdict(recorded.packet) + packet_text = "\n".join(recorded.packet.packet_lines) + return { + "status": "R_LOOP_VESSEL_READ_PACKET_OK" + if recorded.packet.read_status == "passed" + else "R_LOOP_VESSEL_READ_PACKET_NOT_PASSED", + "read_status": recorded.packet.read_status, + "failure_type": recorded.packet.failure_type, + "failure_reason": recorded.packet.failure_reason, + "target_adapter_name": recorded.packet.target_adapter_name, + "vessel_database_name": recorded.packet.vessel_database_name, + "graph_namespace": recorded.packet.graph_namespace, + "target_consumer": recorded.packet.target_consumer, + "neo4j_uri": config.uri, + "neo4j_user": config.user, + "neo4j_password_configured": config.password is not None, + "neo4j_allow_no_auth": config.allow_no_auth, + "trace_count": len(trace_store.list_events()), + "data_record_count": len(data_store.list_records()), + "packet_id": recorded.packet.packet_id, + "entry_candidate_count": recorded.packet.entry_candidate_count, + "base_entry_candidate_count": recorded.packet.base_entry_candidate_count, + "exact_child_expanded_entry_count": ( + recorded.packet.exact_child_expanded_entry_count + ), + "exact_child_expansion_truncated": ( + recorded.packet.exact_child_expansion_truncated + ), + "summary_candidate_count": recorded.packet.summary_candidate_count, + "base_summary_candidate_count": recorded.packet.base_summary_candidate_count, + "summary_child_expanded_count": recorded.packet.summary_child_expanded_count, + "summary_child_expansion_truncated": ( + recorded.packet.summary_child_expansion_truncated + ), + "total_summary_scanned_count": recorded.packet.total_summary_scanned_count, + "active_summary_candidate_count": recorded.packet.active_summary_candidate_count, + "skipped_summary_count": recorded.packet.skipped_summary_count, + "summary_count_by_data_kind": recorded.packet.summary_count_by_data_kind, + "summary_count_by_depth": recorded.packet.summary_count_by_depth, + "entry_candidate_records": recorded.packet.entry_candidate_records, + "summary_candidate_records": recorded.packet.summary_candidate_records, + "packet_lines": recorded.packet.packet_lines, + "packet_text": packet_text, + "read_packet_frame": packet_payload, + } + + +def render_r_loop_vessel_read_packet_text(result: dict[str, object]) -> str: + lines = [ + f"status: {result.get('status')}", + f"read_status: {result.get('read_status')}", + ] + failure_type = result.get("failure_type") + failure_reason = result.get("failure_reason") + if failure_type or failure_reason: + lines.append(f"failure_type: {failure_type}") + lines.append(f"failure_reason: {failure_reason}") + lines.append(f"graph_namespace: {result.get('graph_namespace')}") + lines.append(f"target_consumer: {result.get('target_consumer')}") + lines.append(f"entry_candidate_count: {result.get('entry_candidate_count')}") + lines.append(f"base_entry_candidate_count: {result.get('base_entry_candidate_count')}") + lines.append( + "exact_child_expanded_entry_count: " + f"{result.get('exact_child_expanded_entry_count')}" + ) + lines.append( + "exact_child_expansion_truncated: " + f"{result.get('exact_child_expansion_truncated')}" + ) + lines.append(f"summary_candidate_count: {result.get('summary_candidate_count')}") + lines.append( + "base_summary_candidate_count: " + f"{result.get('base_summary_candidate_count')}" + ) + lines.append( + "summary_child_expanded_count: " + f"{result.get('summary_child_expanded_count')}" + ) + lines.append( + "summary_child_expansion_truncated: " + f"{result.get('summary_child_expansion_truncated')}" + ) + lines.append(f"total_summary_scanned_count: {result.get('total_summary_scanned_count')}") + lines.append(f"skipped_summary_count: {result.get('skipped_summary_count')}") + lines.append(f"summary_count_by_data_kind: {result.get('summary_count_by_data_kind')}") + lines.append(f"summary_count_by_depth: {result.get('summary_count_by_depth')}") + packet_text = result.get("packet_text") + if isinstance(packet_text, str) and packet_text: + lines.append("") + lines.append(packet_text) + return "\n".join(lines) + + +def _now_iso() -> str: + return datetime.now().isoformat(timespec="seconds") + + +__all__ = [ + "render_r_loop_vessel_read_packet_text", + "run_local_r_loop_vessel_read_packet", +] diff --git a/songryeon_core/runtime/smoke_test.py b/songryeon_core/runtime/smoke_test.py index ff83269..e6d0435 100644 --- a/songryeon_core/runtime/smoke_test.py +++ b/songryeon_core/runtime/smoke_test.py @@ -64,7 +64,11 @@ from songryeon_core.nodes.node_1_router import record_routing, route_next from songryeon_core.nodes.node_2_handoff import record_node3_input_brief, record_route2_handoff from songryeon_core.nodes.node_2_handoff import record_selected_recent_memory_context -from songryeon_core.nodes.node_2_metainfo_boundary import build_metainfo_boundary, record_boundary +from songryeon_core.nodes.node_2_metainfo_boundary import ( + build_metainfo_boundary, + record_boundary, + run_node2_answer_basis_selection, +) from songryeon_core.nodes.node_3_reporter import record_report, render_report from songryeon_core.nodes.node_4_gatekeeper import run_node4_gatekeeper from songryeon_core.runtime.dry_run import run_dry_turn @@ -117,6 +121,7 @@ def run_smoke_tests() -> dict[str, object]: "L3:achievement_frame", "L3:preserved_info_frame", "node_2:handoff_frame", + "node_2:answer_basis_frame", "node_3:input_brief_frame", "report_dry_001", "L:run_frame:0001", @@ -125,6 +130,9 @@ def run_smoke_tests() -> dict[str, object]: "tool_choice:L2:search_docs", "tool_choice:L_controller_0002:read_doc", "L:return_summary_frame", + "L:activity_ledger_frame", + "graph:turn_activity_graph_link:turn_dry_001", + "graph:activity_ledger:L:activity_ledger_frame", "L:control:0001", "L:control:0002", "L:control:0003", @@ -155,6 +163,8 @@ def run_smoke_tests() -> dict[str, object]: _check_route2_handoff_and_brief(records) runtime_count_smoke = _run_runtime_count_consistency_smoke() return_summary_smoke = _check_l_loop_return_summary(records) + l_activity_ledger_smoke = _check_l_loop_activity_ledger(records) + turn_activity_graph_link_smoke = _check_turn_activity_graph_link(records) _check_runtime_explanation_fields(records) runtime_label_smoke = _check_runtime_metainfo_labels(result) live_trace_smoke = _run_live_trace_progress_stream_smoke() @@ -207,6 +217,8 @@ def run_smoke_tests() -> dict[str, object]: if search_result["result_count"] < 1: raise AssertionError("search_docs returned no results") document_memory_smoke = _check_document_memory_index(records, search_result) + graph_memory_guide_smoke = _check_graph_memory_guide(result, records) + r_route_dry_run_smoke = _run_r_route_dry_run_only_smoke() return { "status": "SMOKE_TEST_OK", @@ -239,6 +251,12 @@ def run_smoke_tests() -> dict[str, object]: "runtime_count_reportable_documents": runtime_count_smoke["reportable_document_count"], "runtime_count_raw_extract_records": runtime_count_smoke["raw_document_extract_record_count"], "runtime_count_empty_extract_records": runtime_count_smoke["empty_document_extract_record_count"], + "node2_answer_basis_mode": result.get("node2_answer_basis_mode"), + "node2_answer_basis_reason_codes": result.get("node2_answer_basis_reason_codes"), + "node2_answer_basis_generated_by": result.get("node2_answer_basis_generated_by"), + "node2_answer_basis_semantic": result.get( + "node2_answer_basis_semantic_judgement_status" + ), "llm_call_records": llm_smoke["llm_call_records"], "llm_retry_failure_type": llm_smoke["llm_retry_failure_type"], "node1_router_fallback_policy": router_fallback_smoke["fallback_policy"], @@ -251,6 +269,13 @@ def run_smoke_tests() -> dict[str, object]: "l_loop_final_decision": l_loop_control_smoke["final_decision"], "l_loop_return_summary_status": return_summary_smoke["task_status"], "l_loop_return_summary_route_hint": return_summary_smoke["route_hint"], + "l_loop_activity_ledger_outputs": l_activity_ledger_smoke["output_count"], + "l_loop_activity_ledger_tool_results": l_activity_ledger_smoke["tool_result_count"], + "l_loop_activity_ledger_read_doc": l_activity_ledger_smoke["actual_read_doc_count"], + "turn_activity_graph_link_l_ledgers": turn_activity_graph_link_smoke["l_ledger_count"], + "turn_activity_graph_link_r_ledgers": turn_activity_graph_link_smoke["r_ledger_count"], + "turn_activity_graph_link_nodes": turn_activity_graph_link_smoke["node_count"], + "turn_activity_graph_link_edges": turn_activity_graph_link_smoke["edge_count"], "l_loop_read_doc_used": l_loop_control_smoke["read_doc_used"], "tool_distillation_count": distillation_smoke["distillation_count"], "tool_distillation_sources_l3": distillation_smoke["l3_uses_distillation"], @@ -339,6 +364,24 @@ def run_smoke_tests() -> dict[str, object]: "raw_memory_window_count_13_older_unmanaged": raw_memory_window_smoke["count_13_older_unmanaged"], "raw_memory_window_no_semantic_compression": raw_memory_window_smoke["no_semantic_compression"], "raw_memory_window_original_kept": raw_memory_window_smoke["original_kept"], + "graph_memory_snapshot_id": graph_memory_guide_smoke["snapshot_id"], + "graph_memory_raw_capsule_nodes": graph_memory_guide_smoke["raw_capsule_nodes"], + "graph_memory_time_bundle_nodes": graph_memory_guide_smoke["time_bundle_nodes"], + "rloop_graph_guide_packet_id": graph_memory_guide_smoke["guide_packet_id"], + "rloop_graph_guide_entry_count": graph_memory_guide_smoke["entry_count"], + "rloop_graph_guide_hints_status": graph_memory_guide_smoke["hints_status"], + "rloop_graph_guide_semantic_status": graph_memory_guide_smoke["semantic_status"], + "rloop_graph_guide_not_in_node1_or_node3": graph_memory_guide_smoke["not_in_node1_or_node3"], + "r_loop_memory_handoff_packet_id": graph_memory_guide_smoke["handoff_packet_id"], + "r_loop_memory_handoff_status": graph_memory_guide_smoke["handoff_status"], + "r_loop_memory_handoff_entry_count": graph_memory_guide_smoke["handoff_entry_count"], + "r_loop_memory_handoff_semantic_hint_status": graph_memory_guide_smoke[ + "handoff_semantic_hint_status" + ], + "r_route_dry_run_enabled": r_route_dry_run_smoke["enabled"], + "r_route_dry_run_status": r_route_dry_run_smoke["task_status"], + "r_route_dry_run_continuation": r_route_dry_run_smoke["continuation_status"], + "r_route_dry_run_not_default": r_route_dry_run_smoke["not_default"], "qwen_chat_continuity_external_turn_id": qwen_chat_continuity_smoke["external_turn_id"], "qwen_chat_continuity_two_turn_alignment": qwen_chat_continuity_smoke["two_turn_alignment"], "qwen_chat_continuity_stateless_default": qwen_chat_continuity_smoke["stateless_default"], @@ -374,6 +417,171 @@ def run_smoke_tests() -> dict[str, object]: } +def _check_graph_memory_guide( + result: dict[str, object], + records: dict[str, object], +) -> dict[str, object]: + snapshot_id = result.get("graph_memory_snapshot_id") + guide_id = result.get("rloop_graph_guide_packet_id") + handoff_id = result.get("r_loop_memory_handoff_packet_id") + if not isinstance(snapshot_id, str) or not snapshot_id: + raise AssertionError("graph memory snapshot id is missing") + if not isinstance(guide_id, str) or not guide_id: + raise AssertionError("RLoop graph guide packet id is missing") + if not isinstance(handoff_id, str) or not handoff_id: + raise AssertionError("R loop memory handoff packet id is missing") + + snapshot = records.get(snapshot_id) + guide = records.get(guide_id) + handoff = records.get(handoff_id) + if not isinstance(snapshot, dict): + raise AssertionError("graph memory snapshot payload is missing") + if not isinstance(guide, dict): + raise AssertionError("RLoop graph guide payload is missing") + if not isinstance(handoff, dict): + raise AssertionError("R loop memory handoff payload is missing") + + node_kind_counts = guide.get("node_kind_counts") + if not isinstance(node_kind_counts, dict): + raise AssertionError("RLoop graph guide node_kind_counts is missing") + raw_capsule_nodes = node_kind_counts.get("raw_capsule") + time_bundle_nodes = node_kind_counts.get("time_bundle") + if not isinstance(raw_capsule_nodes, int) or raw_capsule_nodes < 1: + raise AssertionError("RLoop graph guide has no raw capsule node") + if not isinstance(time_bundle_nodes, int) or time_bundle_nodes < 1: + raise AssertionError("RLoop graph guide has no time bundle node") + if guide.get("generated_by") != "CODE:GRAPH_MEMORY_GUIDE_BUILDER": + raise AssertionError("RLoop graph guide generated_by is not code builder") + if guide.get("info_class") != "absolute": + raise AssertionError("RLoop graph guide info_class is not absolute") + if guide.get("semantic_judgement_status") != "not_run": + raise AssertionError("RLoop graph guide semantic judgement must remain not_run") + if guide.get("recommended_traversal_hints_status") != "not_run": + raise AssertionError("RLoop graph guide LLM hints must remain not_run") + if guide.get("recommended_traversal_hints"): + raise AssertionError("RLoop graph guide must not include LLM traversal hints") + + entry_nodes = guide.get("available_entry_nodes") + if not isinstance(entry_nodes, list) or "graph:axis:time" not in entry_nodes: + raise AssertionError("RLoop graph guide must expose graph:axis:time as entry") + + if handoff.get("packet_status") != "available": + raise AssertionError("R loop handoff must be available when guide exists") + if handoff.get("target") != "R_LOOP": + raise AssertionError("R loop handoff target must be R_LOOP") + if handoff.get("mode") != "graph_guide_handoff": + raise AssertionError("R loop handoff mode must be graph_guide_handoff") + if handoff.get("r_loop_graph_guide_packet_id") != guide_id: + raise AssertionError("R loop handoff must preserve guide packet id") + if handoff.get("graph_snapshot_id") != snapshot_id: + raise AssertionError("R loop handoff must preserve graph snapshot id") + if handoff.get("available_entry_node_ids") != entry_nodes: + raise AssertionError("R loop handoff must copy available entry nodes") + if handoff.get("generated_by") != "CODE:node_0_memory_supplier": + raise AssertionError("R loop handoff generated_by must be node_0 code") + if handoff.get("info_class") != "absolute": + raise AssertionError("R loop handoff info_class must be absolute") + if handoff.get("semantic_judgement_status") != "not_run": + raise AssertionError("R loop handoff semantic judgement must remain not_run") + handoff_source_data_ids = handoff.get("source_data_ids") + if not isinstance(handoff_source_data_ids, list): + raise AssertionError("R loop handoff source_data_ids missing") + if guide_id not in handoff_source_data_ids or snapshot_id not in handoff_source_data_ids: + raise AssertionError("R loop handoff must cite guide and snapshot data ids") + + for record in result.get("data_records", []): + if not isinstance(record, dict): + continue + if record.get("data_type") not in { + "node_output:routing_decision", + "node_output:report", + }: + continue + payload = record.get("payload") + if not isinstance(payload, dict): + continue + source_data_ids = payload.get("source_data_ids") + if isinstance(source_data_ids, list) and guide_id in source_data_ids: + raise AssertionError("RLoop graph guide was injected into node_1 or node_3") + if isinstance(source_data_ids, list) and handoff_id in source_data_ids: + raise AssertionError("R loop handoff was injected into node_1 or node_3") + + return { + "snapshot_id": snapshot_id, + "guide_packet_id": guide_id, + "handoff_packet_id": handoff_id, + "raw_capsule_nodes": raw_capsule_nodes, + "time_bundle_nodes": time_bundle_nodes, + "entry_count": len(entry_nodes), + "hints_status": guide.get("recommended_traversal_hints_status"), + "semantic_status": guide.get("semantic_judgement_status"), + "handoff_status": handoff.get("packet_status"), + "handoff_entry_count": len(entry_nodes), + "handoff_semantic_hint_status": handoff.get("semantic_hint_status"), + "not_in_node1_or_node3": True, + } + + +def _run_r_route_dry_run_only_smoke() -> dict[str, object]: + default_result = run_dry_turn() + if default_result.get("r_route_dry_run_enabled") is not False: + raise AssertionError("R dry-run must be disabled by default") + if default_result.get("r_route_dry_run_status") != "not_run": + raise AssertionError("default dry-run must not run R skeleton") + + result = run_dry_turn(enable_r_route_dry_run=True) + if result.get("r_route_dry_run_enabled") is not True: + raise AssertionError("R dry-run fixture did not enable") + if result.get("r_route_dry_run_status") != "sufficient": + raise AssertionError("R dry-run return summary should be sufficient") + if result.get("r_route_dry_run_continuation_status") != "stop_sufficient": + raise AssertionError("R dry-run should stop sufficient after multi-step traversal") + if result.get("r_route_dry_run_next_target_node") != "return_summary": + raise AssertionError("R dry-run final continuation should target return_summary") + if result.get("r_route_dry_run_traversal_step_count") != 3: + raise AssertionError("R dry-run should traverse three graph nodes") + output_ids = result.get("r_route_dry_run_output_data_ids") + if not isinstance(output_ids, list) or len(output_ids) != 18: + raise AssertionError("R dry-run should record eighteen output frames") + + records = result.get("data_records") + if not isinstance(records, list): + raise AssertionError("R dry-run result has no data_records") + data_types = { + record.get("data_type") + for record in records + if isinstance(record, dict) and record.get("data_id") in output_ids + } + required_types = { + "node_output:R1_graph_goal_frame", + "node_output:R_loop_budget_frame", + "node_output:R2_graph_node_selection_frame", + "node_output:R3_graph_inspection_frame", + "node_output:R_graph_traversal_candidate_surface_frame", + "node_output:R_loop_continuation_frame", + "node_output:R_loop_return_summary_frame", + "graph_memory:turn_access_ledger_frame", + } + if not required_types.issubset(data_types): + raise AssertionError("R dry-run output frame types are incomplete") + ledger_id = result.get("r_route_dry_run_access_ledger_id") + if not isinstance(ledger_id, str) or ledger_id not in output_ids: + raise AssertionError("R dry-run access ledger id must be recorded") + candidate_surface_id = result.get("r_route_dry_run_candidate_surface_id") + if not isinstance(candidate_surface_id, str) or candidate_surface_id not in output_ids: + raise AssertionError("R dry-run candidate surface id must be recorded") + + return { + "enabled": True, + "task_status": result.get("r_route_dry_run_status"), + "continuation_status": result.get("r_route_dry_run_continuation_status"), + "traversal_step_count": result.get("r_route_dry_run_traversal_step_count"), + "candidate_surface_id": candidate_surface_id, + "access_ledger_id": ledger_id, + "not_default": True, + } + + class MemoryRelevanceRawTextProbeFakeLLMAdapter: """Smoke-only adapter: selects the candidate whose copied raw text contains 청성.""" @@ -1897,9 +2105,10 @@ def _check_route2_handoff_and_brief(records: dict[str, object]) -> None: """route=2 handoff와 node_3용 브리프가 새 경계를 지키는지 확인한다.""" handoff = records["node_2:handoff_frame"] + answer_basis = records["node_2:answer_basis_frame"] brief = records["node_3:input_brief_frame"] - if not isinstance(handoff, dict) or not isinstance(brief, dict): - raise AssertionError("route2 handoff and node3 brief payloads must be dicts") + if not isinstance(handoff, dict) or not isinstance(answer_basis, dict) or not isinstance(brief, dict): + raise AssertionError("route2 handoff, answer basis, and node3 brief payloads must be dicts") if handoff.get("handoff_status") not in {"ready", "insufficient", "blocked"}: raise AssertionError("route2 handoff status is invalid") if "1:route=2" not in (handoff.get("route_path") or []): @@ -1918,7 +2127,10 @@ def _check_route2_handoff_and_brief(records: dict[str, object]) -> None: raise AssertionError(f"route2 handoff {field_name} must be a non-negative integer") if handoff.get("read_doc_count") != handoff.get("reportable_document_count"): raise AssertionError("route2 handoff read_doc_count must mirror reportable_document_count") - if handoff.get("raw_document_extract_record_count") < handoff.get("reportable_document_count"): + if ( + not handoff.get("document_context_pack_frame_id") + and handoff.get("raw_document_extract_record_count") < handoff.get("reportable_document_count") + ): raise AssertionError("route2 handoff raw extract count must cover reportable documents") controller_decisions = handoff.get("same_turn_l_reroute_controller_decisions") if not isinstance(controller_decisions, list): @@ -1945,9 +2157,33 @@ def _check_route2_handoff_and_brief(records: dict[str, object]) -> None: raise AssertionError("node3 brief status is invalid") if brief.get("handoff_frame_id") != "node_2:handoff_frame": raise AssertionError("node3 brief must point back to route2 handoff internally") + if answer_basis.get("answer_basis_mode") not in { + "absolute_first", + "relative_allowed", + "mixed_or_uncertain", + }: + raise AssertionError("answer basis mode must be one of the three approved values") + if answer_basis.get("generated_by") != "CODE:FALLBACK": + raise AssertionError("default dry-run answer basis should reveal CODE fallback") + if answer_basis.get("basis_reason_codes") != ["llm_mode_selection_failed"]: + raise AssertionError("default dry-run answer basis fallback reason code is wrong") + if answer_basis.get("semantic_judgement_status") != "failed": + raise AssertionError("default dry-run answer basis fallback status must be failed") + if brief.get("answer_basis_frame_id") != "node_2:answer_basis_frame": + raise AssertionError("node3 brief must cite answer basis frame") + if brief.get("answer_basis_mode") != answer_basis.get("answer_basis_mode"): + raise AssertionError("node3 brief must preserve answer_basis_mode") + if brief.get("basis_reason_codes") != answer_basis.get("basis_reason_codes"): + raise AssertionError("node3 brief must preserve basis_reason_codes") + if brief.get("answer_basis_generated_by") != answer_basis.get("generated_by"): + raise AssertionError("node3 brief must preserve answer basis generated_by") read_documents = brief.get("read_documents") search_candidate_count = brief.get("search_candidate_count") search_candidate_documents = brief.get("search_candidate_documents") + final_search_candidate_count = brief.get("final_search_candidate_count") + final_search_candidate_documents = brief.get("final_search_candidate_documents") + accumulated_search_candidate_count = brief.get("accumulated_search_candidate_count") + accumulated_search_candidate_documents = brief.get("accumulated_search_candidate_documents") allowed_claims = brief.get("allowed_claims") memory_selection_material = brief.get("memory_selection_material") runtime_tasks = brief.get("runtime_tasks") @@ -1955,6 +2191,8 @@ def _check_route2_handoff_and_brief(records: dict[str, object]) -> None: if ( not isinstance(read_documents, list) or not isinstance(search_candidate_documents, list) + or not isinstance(final_search_candidate_documents, list) + or not isinstance(accumulated_search_candidate_documents, list) or not isinstance(allowed_claims, list) or not isinstance(runtime_tasks, list) or not isinstance(reporting_rules, list) @@ -1966,6 +2204,25 @@ def _check_route2_handoff_and_brief(records: dict[str, object]) -> None: raise AssertionError("node3 brief search_candidate_count must be a non-negative integer") if search_candidate_count != len(search_candidate_documents): raise AssertionError("node3 brief search_candidate_count must match search_candidate_documents") + if final_search_candidate_count != search_candidate_count: + raise AssertionError("node3 brief final_search_candidate_count must mirror legacy search_candidate_count") + if final_search_candidate_documents != search_candidate_documents: + raise AssertionError("node3 brief final_search_candidate_documents must mirror legacy search_candidate_documents") + if not isinstance(accumulated_search_candidate_count, int) or accumulated_search_candidate_count < 0: + raise AssertionError("node3 brief accumulated_search_candidate_count must be non-negative") + if accumulated_search_candidate_count != len(accumulated_search_candidate_documents): + raise AssertionError("node3 brief accumulated_search_candidate_count must match accumulated documents") + pack_frame_id = brief.get("document_context_pack_frame_id") + if isinstance(pack_frame_id, str) and pack_frame_id: + if handoff.get("document_context_pack_frame_id") != pack_frame_id: + raise AssertionError("node3 brief must cite the handoff document context pack frame") + if len(read_documents) != handoff.get("document_context_included_count"): + raise AssertionError("node3 read_documents must mirror context pack included count") + excluded_contexts = brief.get("excluded_document_contexts") + if not isinstance(excluded_contexts, list): + raise AssertionError("node3 brief excluded document contexts must be a list") + if len(excluded_contexts) != handoff.get("document_context_excluded_count"): + raise AssertionError("node3 excluded contexts must mirror pack excluded count") if not read_documents and not allowed_claims and not runtime_tasks: raise AssertionError("default dry run should provide node3 with at least one report material") for document in read_documents: @@ -2061,6 +2318,140 @@ def _check_l_loop_return_summary(records: dict[str, object]) -> dict[str, object } +def _check_l_loop_activity_ledger(records: dict[str, object]) -> dict[str, object]: + """L루프 활동 장부가 의미 판단 없이 L 산출물 좌표를 묶는지 확인한다.""" + + frame = records.get("L:activity_ledger_frame") + if not isinstance(frame, dict): + raise AssertionError("L loop activity ledger frame is missing") + if frame.get("loop_id") != "L": + raise AssertionError("L activity ledger must target loop L") + if frame.get("run_index") != 1: + raise AssertionError("first L activity ledger run index must be 1") + if frame.get("turn_capsule_graph_node_id") != "graph:raw_capsule:turn_dry_001": + raise AssertionError("L activity ledger must expose turn raw capsule graph anchor") + if frame.get("generated_by") != "CODE:L_LOOP_ACTIVITY_LEDGER": + raise AssertionError("L activity ledger must reveal code generator") + if frame.get("info_class") != "absolute": + raise AssertionError("L activity ledger must be absolute") + if frame.get("semantic_judgement_status") != "not_run": + raise AssertionError("L activity ledger must not run semantic judgement") + source_data_ids = frame.get("source_data_ids") + if not isinstance(source_data_ids, list): + raise AssertionError("L activity ledger source_data_ids must be a list") + for data_id in [ + "L:run_frame:0001", + "L1:goal_frame", + "L2:query_frame", + "L3:achievement_frame", + "L:return_summary_frame", + "node_0:document_material_packet_frame", + ]: + if data_id not in source_data_ids: + raise AssertionError(f"L activity ledger missing source data id: {data_id}") + activity_records = frame.get("activity_records") + if not isinstance(activity_records, list) or not activity_records: + raise AssertionError("L activity ledger activity_records must be non-empty") + stages = { + item.get("stage") + for item in activity_records + if isinstance(item, dict) + } + for stage in {"run_frame", "goal", "query", "tool_result", "l3_achievement", "return_summary"}: + if stage not in stages: + raise AssertionError(f"L activity ledger missing activity stage: {stage}") + output_data_ids = frame.get("output_data_ids") + if not isinstance(output_data_ids, list): + raise AssertionError("L activity ledger output_data_ids must be a list") + if frame.get("output_data_id_count") != len(output_data_ids): + raise AssertionError("L activity ledger output count must mirror output_data_ids") + tool_result_data_ids = frame.get("tool_result_data_ids") + if not isinstance(tool_result_data_ids, list): + raise AssertionError("L activity ledger tool_result_data_ids must be a list") + if frame.get("tool_result_count") != len(tool_result_data_ids): + raise AssertionError("L activity ledger tool result count must mirror tool_result_data_ids") + read_doc_ids = frame.get("read_doc_ids") + if not isinstance(read_doc_ids, list): + raise AssertionError("L activity ledger read_doc_ids must be a list") + if frame.get("actual_read_doc_count") != len(read_doc_ids): + raise AssertionError("L activity ledger read_doc count must mirror read_doc_ids") + return { + "output_count": frame.get("output_data_id_count"), + "tool_result_count": frame.get("tool_result_count"), + "actual_read_doc_count": frame.get("actual_read_doc_count"), + } + + +def _check_turn_activity_graph_link(records: dict[str, object]) -> dict[str, object]: + """raw capsule graph node와 L/R activity ledger graph node 연결을 확인한다.""" + + frame = records.get("graph:turn_activity_graph_link:turn_dry_001") + if not isinstance(frame, dict): + raise AssertionError("turn activity graph link frame is missing") + if frame.get("turn_capsule_graph_node_id") != "graph:raw_capsule:turn_dry_001": + raise AssertionError("turn activity graph link must point to raw capsule graph node") + if frame.get("generated_by") != "CODE:TURN_ACTIVITY_GRAPH_LINK_BUILDER": + raise AssertionError("turn activity graph link must reveal code builder") + if frame.get("info_class") != "absolute": + raise AssertionError("turn activity graph link must be absolute") + if frame.get("semantic_judgement_status") != "not_run": + raise AssertionError("turn activity graph link semantic judgement must be not_run") + l_ledger_ids = frame.get("l_loop_activity_ledger_data_ids") + r_ledger_ids = frame.get("r_graph_access_ledger_data_ids") + graph_node_ids = frame.get("activity_ledger_graph_node_ids") + graph_edge_ids = frame.get("activity_ledger_graph_edge_ids") + if l_ledger_ids != ["L:activity_ledger_frame"]: + raise AssertionError("turn activity graph link must include L activity ledger") + if r_ledger_ids != []: + raise AssertionError("default smoke should not include R access ledger in activity link") + if not isinstance(graph_node_ids, list) or len(graph_node_ids) != 1: + raise AssertionError("turn activity graph link must include one activity ledger node") + if not isinstance(graph_edge_ids, list) or len(graph_edge_ids) != 1: + raise AssertionError("turn activity graph link must include one activity ledger edge") + + l_activity_node_id = "graph:activity_ledger:L:activity_ledger_frame" + l_activity_edge_id = ( + "graph:edge:has_activity_ledger:" + "graph:raw_capsule:turn_dry_001:" + "graph:activity_ledger:L:activity_ledger_frame" + ) + if graph_node_ids != [l_activity_node_id]: + raise AssertionError("turn activity graph link node id mismatch") + if graph_edge_ids != [l_activity_edge_id]: + raise AssertionError("turn activity graph link edge id mismatch") + + node = records.get(l_activity_node_id) + if not isinstance(node, dict): + raise AssertionError("activity ledger graph node is missing") + if node.get("node_kind") != "activity_ledger": + raise AssertionError("activity ledger graph node kind mismatch") + if node.get("data_kind") != "l_loop_activity_ledger": + raise AssertionError("activity ledger graph node data kind mismatch") + if node.get("source_graph_node_ids") != ["graph:raw_capsule:turn_dry_001"]: + raise AssertionError("activity ledger graph node must point back to raw capsule") + if node.get("source_data_ids") != ["L:activity_ledger_frame"]: + raise AssertionError("activity ledger graph node must source L ledger") + + edge = records.get(l_activity_edge_id) + if not isinstance(edge, dict): + raise AssertionError("activity ledger graph edge is missing") + if edge.get("edge_kind") != "HAS_ACTIVITY_LEDGER": + raise AssertionError("activity ledger graph edge kind mismatch") + if edge.get("from_node_id") != "graph:raw_capsule:turn_dry_001": + raise AssertionError("activity ledger graph edge source mismatch") + if edge.get("to_node_id") != l_activity_node_id: + raise AssertionError("activity ledger graph edge target mismatch") + if "L:activity_ledger_frame" not in edge.get("source_data_ids", []): + raise AssertionError("activity ledger graph edge must source L ledger") + + return { + "l_ledger_count": len(l_ledger_ids), + "r_ledger_count": len(r_ledger_ids), + "node_count": len(graph_node_ids), + "edge_count": len(graph_edge_ids), + } + + def _check_l_loop_run_frame(records: dict[str, object]) -> dict[str, object]: """L루프 실행 단위와 현재 재실행 차단 사유가 명시됐는지 확인한다.""" @@ -2507,6 +2898,7 @@ def _run_l_loop_downstream_reroute_scope_smoke() -> dict[str, object]: first.node2_input_frame_id(turn_id), first.route2_handoff_frame_id(), first.metainfo_boundary_id(), + first.node2_answer_basis_frame_id(), first.node3_input_brief_frame_id(), first.node3_report_id(), first.node4_gatekeeper_frame_id(), @@ -2516,6 +2908,7 @@ def _run_l_loop_downstream_reroute_scope_smoke() -> dict[str, object]: second.node2_input_frame_id(turn_id), second.route2_handoff_frame_id(), second.metainfo_boundary_id(), + second.node2_answer_basis_frame_id(), second.node3_input_brief_frame_id(), second.node3_report_id(), second.node4_gatekeeper_frame_id(), @@ -2570,9 +2963,15 @@ def _run_l_loop_downstream_reroute_scope_smoke() -> dict[str, object]: raise AssertionError("second Node3InputBriefFrame.frame_id is not scoped") _assert_payload_sources_include( second_brief, - [second.route2_handoff_frame_id(), second.metainfo_boundary_id()], + [ + second.route2_handoff_frame_id(), + second.metainfo_boundary_id(), + second.node2_answer_basis_frame_id(), + ], label="second Node3InputBriefFrame.source_data_ids", ) + if second_brief.get("answer_basis_frame_id") != second.node2_answer_basis_frame_id(): + raise AssertionError("second Node3InputBriefFrame answer_basis_frame_id is not scoped") second_report = _require_payload(records, second.node3_report_id()) if second_report.get("report_id") != second.node3_report_id(): @@ -2586,6 +2985,7 @@ def _run_l_loop_downstream_reroute_scope_smoke() -> dict[str, object]: second.node3_report_id(), second.node3_input_brief_frame_id(), second.metainfo_boundary_id(), + second.node2_answer_basis_frame_id(), ], label="second Node4GatekeeperFrame.source_data_ids", ) @@ -2725,6 +3125,7 @@ def _run_policy_guarded_same_turn_l_reroute_smoke() -> dict[str, object]: second.node2_input_frame_id(str(policy_result["turn_id"])), second.route2_handoff_frame_id(), second.metainfo_boundary_id(), + second.node2_answer_basis_frame_id(), second.node3_input_brief_frame_id(), second.node3_report_id(), second.node4_gatekeeper_frame_id(), @@ -2929,6 +3330,21 @@ def _record_downstream_after_l_run( boundary=boundary, input_ref=[node2_input_trace.event_id], ) + answer_basis_trace_id, answer_basis_id, answer_basis_frame = ( + run_node2_answer_basis_selection( + trace_store=trace_store, + data_store=data_store, + turn_id=turn_id, + user_question=user_question, + boundary_id=boundary_id, + boundary=boundary, + handoff_frame_id=handoff_id, + adapter=SongRyeonAllNodesFakeLLMAdapter(), + input_ref=[handoff_trace_id, boundary_trace_id], + source_data_ids=[node2_input_id, handoff_id, boundary_id], + id_namespace=run_ids, + ) + ) brief_trace_id, brief_id, brief_frame = record_node3_input_brief( trace_store=trace_store, data_store=data_store, @@ -2936,8 +3352,9 @@ def _record_downstream_after_l_run( user_question=user_question, handoff_frame_id=handoff_id, boundary=boundary, - input_trace_ids=[handoff_trace_id, boundary_trace_id], - source_data_ids=[node2_input_id, handoff_id, boundary_id], + input_trace_ids=[handoff_trace_id, boundary_trace_id, answer_basis_trace_id], + source_data_ids=[node2_input_id, handoff_id, boundary_id, answer_basis_id], + answer_basis_frame=answer_basis_frame, id_namespace=run_ids, ) report_id = run_ids.node3_report_id() @@ -2952,7 +3369,7 @@ def _record_downstream_after_l_run( allowed_relative_info_ids=[info_ref.info_id for info_ref in boundary.relative_info], allowed_mixed_info_ids=[info_ref.info_id for info_ref in boundary.mixed_info], input_ref=[brief_trace_id], - source_data_ids=[brief_id, handoff_id, boundary_id, outcome_id, node2_input_id], + source_data_ids=[brief_id, handoff_id, boundary_id, answer_basis_id, outcome_id, node2_input_id], ) gatekeeper_trace_id = run_node4_gatekeeper( trace_store=trace_store, @@ -2964,7 +3381,7 @@ def _record_downstream_after_l_run( rendered_markdown=report, adapter=SongRyeonAllNodesFakeLLMAdapter(), input_ref=[report_trace_id], - source_data_ids=[report_id, brief_id, boundary_id], + source_data_ids=[report_id, brief_id, boundary_id, answer_basis_id], id_namespace=run_ids, ) if not gatekeeper_trace_id: @@ -2979,6 +3396,7 @@ def _record_downstream_after_l_run( "node2_input_id": node2_input_id, "handoff_id": handoff_id, "boundary_id": boundary_id, + "answer_basis_id": answer_basis_id, "brief_id": brief_id, "report_id": report_id, "gatekeeper_id": run_ids.node4_gatekeeper_frame_id(), @@ -5170,19 +5588,26 @@ def _run_node4_grounding_count_guard_smoke() -> dict[str, object]: brief = records.get("node_3:input_brief_frame") if not isinstance(brief, dict): raise AssertionError("node3 brief missing") - expected_doc_count = len(brief.get("read_documents") or []) - expected_search_count = brief.get("search_candidate_count") + expected_tool_read_count = brief.get("actual_tool_read_doc_count") + expected_context_count = brief.get("supplied_document_context_count") + expected_final_search_count = brief.get("final_search_candidate_count") + expected_accumulated_search_count = brief.get("accumulated_search_candidate_count") expected_runtime_count = len(brief.get("runtime_tasks") or []) expected_lines = [ "근거 기준:", - f"- 읽은 문서: {expected_doc_count}개", - f"- 검색 후보 문서: {expected_search_count}개", + f"- 실제 read_doc 도구 원문 읽기: {expected_tool_read_count}개", + f"- node_3 공급 문서 context: {expected_context_count}개", + f"- 검색 후보 문서(최종): {expected_final_search_count}개", + f"- 검색 후보 문서(누적): {expected_accumulated_search_count}개", f"- 현재 턴 실행 순서 자료: {expected_runtime_count}개", ] for line in expected_lines: if line not in rendered_markdown: raise AssertionError(f"code grounding block missing expected line: {line}") - if "- 읽은 문서: 0개" in rendered_markdown and expected_doc_count != 0: + if ( + "- 실제 read_doc 도구 원문 읽기: 0개" in rendered_markdown + and expected_tool_read_count != 0 + ): raise AssertionError("legacy LLM grounding count leaked into final report") answer = render_pretty_turn(result, user_input="count guard smoke") if "FINAL_BLOCKED_BY_GATEKEEPER" in answer: @@ -5413,8 +5838,10 @@ def _report_with_grounding(body: str) -> str: return "\n".join( [ "근거 기준:", - "- 읽은 문서: 0개", - "- 검색 후보 문서: 0개", + "- 실제 read_doc 도구 원문 읽기: 0개", + "- node_3 공급 문서 context: 0개", + "- 검색 후보 문서(최종): 0개", + "- 검색 후보 문서(누적): 0개", "- 현재 턴 실행 순서 자료: 0개", "- 답변 한계: 제공된 자료 범위 안에서만 답한다.", "", @@ -5561,8 +5988,10 @@ def _node_3_payload(self, request) -> dict[str, object]: return { "rendered_markdown": ( "근거 기준:\n" - "- 읽은 문서: 0개\n" - "- 검색 후보 문서: 0개\n" + "- 실제 read_doc 도구 원문 읽기: 0개\n" + "- node_3 공급 문서 context: 0개\n" + "- 검색 후보 문서(최종): 0개\n" + "- 검색 후보 문서(누적): 0개\n" "- 현재 턴 실행 순서 자료: 0개\n" "- 답변 한계: 일부러 brief count와 맞지 않는 smoke 보고문이다.\n\n" "이 본문은 LLM이 틀린 count 블록을 포함해도 code assembly가 제거해야 하는 smoke 본문이다." diff --git a/songryeon_core/runtime/terminal_view.py b/songryeon_core/runtime/terminal_view.py index 020024c..533f4a1 100644 --- a/songryeon_core/runtime/terminal_view.py +++ b/songryeon_core/runtime/terminal_view.py @@ -17,6 +17,8 @@ def render_runtime_view(result: dict[str, object], *, user_input: str) -> str: f"- 상태: {result.get('status', 'unknown')}", f"- trace/data: {result.get('trace_count', 0)} / {result.get('data_record_count', 0)}", ] + if result.get("status") == "structure_failed": + lines.extend(_structure_failure_runtime_lines(result)) session_memory = result.get("session_memory") if isinstance(session_memory, dict): lines.append( @@ -186,6 +188,223 @@ def render_runtime_view(result: dict[str, object], *, user_input: str) -> str: ) ) + graph_guides = _payloads_with_type(result, "graph_memory:rloop_guide_packet") + if graph_guides: + lines.append("- graph memory guide:") + for guide in graph_guides: + guide_id = str(guide.get("packet_id") or "unknown") + lines.append( + " - " + f"source={guide_id} / " + f"snapshot={guide.get('graph_snapshot_id', 'unknown')} / " + f"target={guide.get('target_consumer', 'unknown')} / " + f"entries={_list_count(guide.get('available_entry_nodes'))} / " + f"hints={guide.get('recommended_traversal_hints_status', 'unknown')}" + ) + lines.append( + " " + f"node_kinds={guide.get('node_kind_counts', {})} / " + f"summary_depth_range={guide.get('summary_depth_range', [])} / " + f"source_leaf_count_range={guide.get('source_leaf_count_range', [])}" + ) + lines.extend( + _metainfo_lines( + indent=4, + generated_by=str( + guide.get("generated_by") or "CODE:GRAPH_MEMORY_GUIDE_BUILDER" + ), + info_class=str(guide.get("info_class") or "absolute"), + source_data_ids=_source_data_ids(guide, fallback=[guide_id]), + semantic_judgement_status=str( + guide.get("semantic_judgement_status") or "not_run" + ), + ) + ) + + r_loop_handoffs = _payloads_with_type( + result, + "node_output:r_loop_memory_handoff_packet_frame", + ) + if r_loop_handoffs: + lines.append("- R loop memory handoff:") + for packet in r_loop_handoffs: + packet_id = str(packet.get("packet_id") or "unknown") + depth_range = packet.get("summary_depth_range") + if isinstance(depth_range, list) and len(depth_range) == 2: + depth_display = f"{depth_range[0]}..{depth_range[1]}" + else: + depth_display = "unknown" + lines.append( + " - " + f"status={packet.get('packet_status', 'unknown')} / " + f"target={packet.get('target', 'unknown')} / " + f"mode={packet.get('mode', 'unknown')} / " + f"entry_nodes={_list_count(packet.get('available_entry_node_ids'))} / " + f"summary_depth_range={depth_display} / " + f"semantic_hint_status={packet.get('semantic_hint_status', 'unknown')}" + ) + lines.extend( + _metainfo_lines( + indent=4, + generated_by=str( + packet.get("generated_by") or "CODE:node_0_memory_supplier" + ), + info_class=str(packet.get("info_class") or "absolute"), + source_data_ids=_source_data_ids(packet, fallback=[packet_id]), + semantic_judgement_status=str( + packet.get("semantic_judgement_status") or "not_run" + ), + ) + ) + + r_return_summaries = _payloads_with_type(result, "node_output:R_loop_return_summary_frame") + if r_return_summaries: + r_continuations = _payloads_with_type(result, "node_output:R_loop_continuation_frame") + header_generator = str( + r_return_summaries[-1].get("generated_by") or "CODE:R_LOOP_DRY_RUN_ONLY" + ) + header_label = ( + "R experimental route skeleton" + if header_generator == "CODE:R_ROUTE_EXPERIMENTAL_GATE" + else "R dry-run skeleton" + ) + lines.append(f"- {header_label} [{header_generator}]:") + for summary in r_return_summaries: + continuation_status = summary.get("continuation_status", "unknown") + next_target = "unknown" + if r_continuations: + next_target = str(r_continuations[-1].get("next_target_node") or "unknown") + lines.append( + " - " + f"task_status={summary.get('r_loop_task_status', 'unknown')} / " + f"selected_entries={_list_count(summary.get('selected_entry_node_ids'))} / " + f"inspected_nodes={_list_count(summary.get('inspected_graph_node_ids'))} / " + f"continuation={continuation_status} / " + f"next={next_target} / " + f"budget={summary.get('budget_status', 'unknown')}" + ) + lines.extend( + _metainfo_lines( + indent=4, + generated_by=str(summary.get("generated_by") or "CODE:R_LOOP_DRY_RUN_ONLY"), + info_class=str(summary.get("info_class") or "absolute"), + source_data_ids=_source_data_ids( + summary, + fallback=[str(summary.get("frame_id") or "R:return_summary")], + ), + semantic_judgement_status=str( + summary.get("semantic_judgement_status") or "not_run" + ), + ) + ) + + r_candidate_surfaces = _payloads_with_type( + result, + "node_output:R_graph_traversal_candidate_surface_frame", + ) + if r_candidate_surfaces: + lines.append("- R graph traversal candidates:") + for surface in r_candidate_surfaces: + lines.append( + " - " + f"inspected={surface.get('inspected_graph_node_id', 'unknown')} / " + f"candidates={surface.get('candidate_count', 'unknown')} / " + f"children={_list_count(surface.get('child_candidate_node_ids'))} / " + f"next={_list_count(surface.get('next_candidate_node_ids'))} / " + f"previous={_list_count(surface.get('previous_candidate_node_ids'))}" + ) + lines.extend( + _metainfo_lines( + indent=4, + generated_by=str( + surface.get("generated_by") + or "CODE:R_GRAPH_TRAVERSAL_CANDIDATE_SURFACE" + ), + info_class=str(surface.get("info_class") or "absolute"), + source_data_ids=_source_data_ids( + surface, + fallback=[ + str( + surface.get("frame_id") + or "R:graph_traversal_candidate_surface" + ) + ], + ), + semantic_judgement_status=str( + surface.get("semantic_judgement_status") or "not_run" + ), + ) + ) + + graph_access_ledgers = _payloads_with_type( + result, + "graph_memory:turn_access_ledger_frame", + ) + if graph_access_ledgers: + lines.append("- R graph access ledger:") + for ledger in graph_access_ledgers: + lines.append( + " - " + f"turn={ledger.get('turn_id', 'unknown')} / " + f"candidates={_list_count(ledger.get('candidate_graph_node_ids'))} / " + f"selected={_list_count(ledger.get('selected_graph_node_ids'))} / " + f"inspected={_list_count(ledger.get('inspected_graph_node_ids'))} / " + f"read={_list_count(ledger.get('read_graph_node_ids'))} / " + f"answer_sources={_list_count(ledger.get('used_as_answer_source_graph_node_ids'))}" + ) + lines.extend( + _metainfo_lines( + indent=4, + generated_by=str(ledger.get("generated_by") or "CODE:GRAPH_ACCESS_LEDGER"), + info_class=str(ledger.get("info_class") or "absolute"), + source_data_ids=_source_data_ids( + ledger, + fallback=[str(ledger.get("frame_id") or "R:turn_graph_access_ledger")], + ), + semantic_judgement_status=str( + ledger.get("semantic_judgement_status") or "not_run" + ), + ) + ) + + turn_activity_links = _payloads_with_type( + result, + "graph_memory:turn_activity_graph_link_frame", + ) + if turn_activity_links: + lines.append("- Turn activity graph links:") + for frame in turn_activity_links: + lines.append( + " - " + f"raw={frame.get('turn_capsule_graph_node_id', 'unknown')} / " + f"L_ledgers={_list_count(frame.get('l_loop_activity_ledger_data_ids'))} / " + f"R_ledgers={_list_count(frame.get('r_graph_access_ledger_data_ids'))} / " + f"nodes={_list_count(frame.get('activity_ledger_graph_node_ids'))} / " + f"edges={_list_count(frame.get('activity_ledger_graph_edge_ids'))}" + ) + lines.extend( + _metainfo_lines( + indent=4, + generated_by=str( + frame.get("generated_by") + or "CODE:TURN_ACTIVITY_GRAPH_LINK_BUILDER" + ), + info_class=str(frame.get("info_class") or "absolute"), + source_data_ids=_source_data_ids( + frame, + fallback=[ + str( + frame.get("frame_id") + or "graph:turn_activity_graph_link" + ) + ], + ), + semantic_judgement_status=str( + frame.get("semantic_judgement_status") or "not_run" + ), + ) + ) + relevance_selection_frames = _payloads_with_type( result, "node_output:memory_relevance_selection_frame", @@ -452,11 +671,52 @@ def render_runtime_view(result: dict[str, object], *, user_input: str) -> str: f" [{item.get('document_kind', 'unknown')}/{item.get('source_role', 'unknown')}{score_text}]" ) - read_record = _latest_document_extract_record(result) - read_payload = _payload_from_record(read_record) - if read_payload: + explicit_frames = _payloads_with_type(result, "node_output:explicit_artifact_reference_frame") + if explicit_frames: + frame = explicit_frames[-1] + extracted_count = frame.get("extracted_reference_count", 0) + if isinstance(extracted_count, int) and extracted_count > 0: + lines.append("- explicit_artifact_refs:") + resolved = frame.get("resolved_references") + if isinstance(resolved, list): + for item in resolved[:10]: + if not isinstance(item, dict): + continue + status = item.get("resolve_status", "unknown") + raw_ref = item.get("raw_ref", "") + selected_doc_id = item.get("selected_doc_id") + if status == "unique" and isinstance(selected_doc_id, str): + lines.append(f" - {raw_ref} -> unique: {selected_doc_id}") + elif status == "ambiguous": + lines.append( + f" - {raw_ref} -> ambiguous: " + f"{item.get('candidate_count', 0)} candidates" + ) + else: + lines.append(f" - {raw_ref} -> {status}") + if len(resolved) > 10: + lines.append(f" - ... +{len(resolved) - 10} refs") + lines.extend( + _metainfo_lines( + indent=2, + generated_by=str(frame.get("generated_by") or "CODE:EXPLICIT_ARTIFACT_RESOLVER"), + info_class=str(frame.get("info_class") or "absolute_resolve_result"), + source_data_ids=_source_data_ids( + frame, + fallback=[str(frame.get("frame_id") or "L:explicit_artifact_reference_frame")], + ), + semantic_judgement_status=str( + frame.get("semantic_judgement_status") or "not_run" + ), + ) + ) + + for read_record in _document_extract_records_for_display(result): + read_payload = _payload_from_record(read_record) + if not read_payload: + continue read_data_id = str(read_record.get("data_id") or "tool_result:document_extract") - read_tool_name = "read_artifact" if read_data_id.startswith("tool_result:read_artifact") else "read_doc" + read_tool_name = _document_extract_tool_name(read_record) lines.append( f"- {read_tool_name} [TOOL_RESULT:DOCUMENT_EXTRACT | source={read_data_id}]: " f"{read_payload.get('doc_id', '')} " @@ -475,6 +735,116 @@ def render_runtime_view(result: dict[str, object], *, user_input: str) -> str: ) ) + document_context_pack_frames = _payloads_with_type( + result, + "node_output:document_context_pack_frame", + ) + if document_context_pack_frames: + frame = document_context_pack_frames[-1] + included_count = frame.get("included_document_count", 0) + excluded_count = frame.get("excluded_document_count", 0) + included_chars = frame.get("included_total_chars", 0) + max_chars = frame.get("max_document_context_chars", "?") + lines.append("- document_context_pack:") + lines.append( + " - " + f"included={included_count} / excluded={excluded_count} / " + f"budget={included_chars}/{max_chars} {frame.get('budget_unit', 'chars')}" + ) + lines.append( + " - " + f"whole_document_only={str(frame.get('whole_document_only', False)).lower()} / " + f"strict_rank_order={str(frame.get('strict_rank_order', False)).lower()} / " + f"cutoff={frame.get('cutoff_reason', 'none')}" + ) + included_documents = frame.get("included_documents") + if isinstance(included_documents, list) and included_documents: + lines.append(" - included:") + for item in included_documents[:5]: + if not isinstance(item, dict): + continue + lines.append( + " - " + f"{item.get('rank_index', '?')}. {item.get('doc_id', '')} " + f"({item.get('char_count', 0)} chars, {item.get('selection_basis', '')})" + ) + if len(included_documents) > 5: + lines.append(f" - ... +{len(included_documents) - 5} included") + excluded_documents = frame.get("excluded_documents") + if isinstance(excluded_documents, list) and excluded_documents: + lines.append(" - excluded:") + for item in excluded_documents[:5]: + if not isinstance(item, dict): + continue + lines.append( + " - " + f"{item.get('rank_index', '?')}. {item.get('doc_id', '')} " + f"({item.get('char_count', 0)} chars, {item.get('exclusion_reason', '')})" + ) + if len(excluded_documents) > 5: + lines.append(f" - ... +{len(excluded_documents) - 5} excluded") + lines.extend( + _metainfo_lines( + indent=2, + generated_by=str(frame.get("generated_by") or "CODE:DOCUMENT_CONTEXT_PACKER"), + info_class=str(frame.get("info_class") or "absolute_context_packing_result"), + source_data_ids=_source_data_ids( + frame, + fallback=[str(frame.get("frame_id") or "L:document_context_pack_frame")], + ), + semantic_judgement_status=str( + frame.get("semantic_judgement_status") or "not_run" + ), + ) + ) + + document_material_frames = _payloads_with_type( + result, + "node_output:node0_document_material_packet_frame", + ) + if document_material_frames: + frame = document_material_frames[-1] + lines.append("- node_0 document material packet:") + lines.append( + " - " + f"items={frame.get('item_count', 0)} / " + f"search_candidates={frame.get('search_candidate_count', 0)} / " + f"actual_read={frame.get('actual_tool_read_doc_count', 0)} / " + f"supplied_contexts={frame.get('supplied_document_context_count', 0)} / " + f"unread_candidates={frame.get('unread_candidate_count', 0)}" + ) + items = frame.get("items") + if isinstance(items, list) and items: + lines.append(" - items:") + for item in items[:5]: + if not isinstance(item, dict): + continue + roles = item.get("source_roles") + role_text = ",".join(roles) if isinstance(roles, list) else "" + lines.append( + " - " + f"{item.get('document_name', item.get('doc_id', ''))} " + f"[{role_text}]" + ) + if len(items) > 5: + lines.append(f" - ... +{len(items) - 5} items") + lines.extend( + _metainfo_lines( + indent=2, + generated_by=str( + frame.get("generated_by") or "CODE:NODE0_DOCUMENT_MATERIAL_PACKET" + ), + info_class=str(frame.get("info_class") or "absolute_material_index"), + source_data_ids=_source_data_ids( + frame, + fallback=[str(frame.get("frame_id") or "node_0:document_material_packet_frame")], + ), + semantic_judgement_status=str( + frame.get("semantic_judgement_status") or "not_run" + ), + ) + ) + budget_plan_payloads = _payloads_with_type(result, "node_output:l_loop_budget_plan_frame") if budget_plan_payloads: budget_plan = budget_plan_payloads[-1] @@ -515,6 +885,65 @@ def render_runtime_view(result: dict[str, object], *, user_input: str) -> str: ) ) + tool_scope_payloads = _payloads_with_type(result, "node_output:L_tool_scope_frame") + if tool_scope_payloads: + scope = tool_scope_payloads[-1] + lines.append("- L tool scope:") + lines.append( + " -" + f" mode={scope.get('tool_scope_mode', 'unknown')}" + f" / groups={scope.get('allowed_tool_groups', [])}" + f" / materials={scope.get('required_materials', [])}" + ) + reason = scope.get("scope_reason") + if reason: + lines.append(f" - reason: {reason}") + lines.extend( + _metainfo_lines( + indent=2, + generated_by=str(scope.get("generated_by") or "unknown"), + info_class=str(scope.get("info_class") or "unknown"), + source_data_ids=_source_data_ids( + scope, + fallback=[str(scope.get("frame_id") or "L:tool_scope_frame")], + ), + semantic_judgement_status=str(scope.get("semantic_judgement_status") or "unknown"), + ) + ) + + partition_payloads = _payloads_with_type(result, "node_output:L_tool_budget_partition_frame") + if partition_payloads: + partition = partition_payloads[-1] + lines.append("- L tool budget partition [CODE]:") + lines.append( + " - document:" + f" tool={partition.get('document_tool_call_budget', 0)}" + f" / query={partition.get('document_query_budget', 0)}" + f" / read={partition.get('document_read_budget', 0)}" + ) + lines.append( + " - code:" + f" tool={partition.get('code_tool_call_budget', 0)}" + f" / query={partition.get('code_query_budget', 0)}" + f" / read={partition.get('code_read_budget', 0)}" + ) + runtime_budget = partition.get("runtime_record_budget", 0) + if runtime_budget: + lines.append(f" - runtime_record_budget={runtime_budget}") + lines.append(f" - reason: {partition.get('partition_reason', 'unknown')}") + lines.extend( + _metainfo_lines( + indent=2, + generated_by=str(partition.get("generated_by") or "CODE:L_TOOL_BUDGET_PARTITION_POLICY"), + info_class=str(partition.get("info_class") or "absolute_policy_decision"), + source_data_ids=_source_data_ids( + partition, + fallback=[str(partition.get("frame_id") or "L:tool_budget_partition_frame")], + ), + semantic_judgement_status=str(partition.get("semantic_judgement_status") or "not_run"), + ) + ) + budget_payloads = _payloads_with_type(result, "tool_use_budget") if budget_payloads: latest_budget = budget_payloads[-1] @@ -562,7 +991,8 @@ def render_runtime_view(result: dict[str, object], *, user_input: str) -> str: " - evidence: " f"kind={frame.get('evidence_requirement_kind', 'unspecified')} " f"/ required_read={frame.get('required_min_read_documents', 0)} " - f"/ actual_read={frame.get('actual_read_doc_count', 0)} " + f"/ actual_read_doc={frame.get('actual_read_doc_count', 0)} " + f"/ actual_read_code_file={frame.get('actual_read_code_file_count', 0)} " f"/ candidates={frame.get('search_candidate_count', 0)}" ) lines.append( @@ -580,8 +1010,42 @@ def render_runtime_view(result: dict[str, object], *, user_input: str) -> str: info_class="absolute_summary_from_structured_records", source_data_ids=_source_data_ids(frame, fallback=["L:return_summary_frame"]), semantic_judgement_status="not_run", + ) + ) + + l_activity_ledgers = _payloads_with_type(result, "loop_activity:l_loop_activity_ledger_frame") + if l_activity_ledgers: + lines.append("- L activity ledger:") + for frame in l_activity_ledgers: + lines.append( + " - " + f"{frame.get('frame_id', 'unknown')} " + f"run={frame.get('run_index', '?')} " + f"outputs={frame.get('output_data_id_count', 0)} " + f"tools={frame.get('tool_result_count', 0)} " + f"search_docs={frame.get('search_candidate_doc_count', 0)} " + f"read_doc={frame.get('actual_read_doc_count', 0)} " + f"read_code={frame.get('actual_read_code_file_count', 0)}" + ) + lines.append( + " graph_anchor: " + f"{frame.get('turn_capsule_graph_node_id', 'unknown')} / " + f"material={frame.get('document_material_packet_frame_id', '')}" + ) + lines.extend( + _metainfo_lines( + indent=4, + generated_by=str(frame.get("generated_by") or "CODE:L_LOOP_ACTIVITY_LEDGER"), + info_class=str(frame.get("info_class") or "absolute"), + source_data_ids=_source_data_ids( + frame, + fallback=[str(frame.get("frame_id") or "L:activity_ledger_frame")], + ), + semantic_judgement_status=str( + frame.get("semantic_judgement_status") or "not_run" + ), + ) ) - ) revision_queries = _payloads_with_type(result, "node_output:L2_revision_query_frame") if revision_queries: @@ -657,6 +1121,11 @@ def render_runtime_view(result: dict[str, object], *, user_input: str) -> str: read_doc_ids = achievement.get("read_doc_ids") if isinstance(read_doc_ids, list) and read_doc_ids: lines.append(f" - read_doc_ids: {_format_source_ids(read_doc_ids)}") + read_code_file_paths = achievement.get("read_code_file_paths") + if isinstance(read_code_file_paths, list) and read_code_file_paths: + lines.append( + f" - read_code_file_paths: {_format_source_ids(read_code_file_paths)}" + ) search_result_doc_ids = achievement.get("search_result_doc_ids") if isinstance(search_result_doc_ids, list) and search_result_doc_ids: lines.append( @@ -700,7 +1169,8 @@ def render_runtime_view(result: dict[str, object], *, user_input: str) -> str: f"L2={route2_handoff.get('l2_query_present', False)} / " f"L3={route2_handoff.get('l3_preserved_present', False)} / " f"search={route2_handoff.get('search_result_count', 0)} / " - f"reportable_documents={route2_handoff.get('reportable_document_count', route2_handoff.get('read_doc_count', 0))}" + f"reportable_documents={route2_handoff.get('reportable_document_count', route2_handoff.get('read_doc_count', 0))} / " + f"reportable_code_files={route2_handoff.get('reportable_code_file_count', 0)}" ) raw_extract_count = route2_handoff.get("raw_document_extract_record_count") reportable_count = route2_handoff.get("reportable_document_count") @@ -712,6 +1182,24 @@ def render_runtime_view(result: dict[str, object], *, user_input: str) -> str: f"raw_extract_records={raw_extract_count if isinstance(raw_extract_count, int) else route2_handoff.get('read_doc_count', 0)} / " f"empty_extract_records={empty_extract_count if isinstance(empty_extract_count, int) else 0}" ) + raw_code_count = route2_handoff.get("raw_code_extract_record_count") + reportable_code_count = route2_handoff.get("reportable_code_file_count") + empty_code_count = route2_handoff.get("empty_code_extract_record_count") + if isinstance(raw_code_count, int) or isinstance(reportable_code_count, int): + lines.append( + " - code counts: " + f"reportable={reportable_code_count if isinstance(reportable_code_count, int) else 0} / " + f"raw_extract_records={raw_code_count if isinstance(raw_code_count, int) else 0} / " + f"empty_extract_records={empty_code_count if isinstance(empty_code_count, int) else 0}" + ) + pack_frame_id = route2_handoff.get("document_context_pack_frame_id") + if isinstance(pack_frame_id, str) and pack_frame_id: + lines.append( + " - document_context_pack: " + f"included={route2_handoff.get('document_context_included_count', 0)} / " + f"excluded={route2_handoff.get('document_context_excluded_count', 0)} / " + f"cutoff={route2_handoff.get('document_context_cutoff_reason', 'none')}" + ) memory_selection_status = route2_handoff.get("memory_relevance_selection_status") if isinstance(memory_selection_status, str) and memory_selection_status: lines.append( @@ -773,6 +1261,77 @@ def render_runtime_view(result: dict[str, object], *, user_input: str) -> str: ) ) + answer_basis_record = _latest_run_scoped_record( + result, + records, + "node_2:answer_basis_frame", + data_type="node_output:node2_answer_basis_frame", + ) + answer_basis = _payload_from_record(answer_basis_record) + if answer_basis: + lines.append( + "- node_2 answer basis: " + f"mode={answer_basis.get('answer_basis_mode', 'unknown')} / " + f"generated_by={answer_basis.get('generated_by', 'unknown')} / " + f"semantic={answer_basis.get('semantic_judgement_status', 'unknown')}" + ) + reason_codes = answer_basis.get("basis_reason_codes") + if isinstance(reason_codes, list) and reason_codes: + lines.append(f" - reason_codes: {reason_codes}") + reason = answer_basis.get("mode_selection_reason") + if isinstance(reason, str) and reason: + lines.append(f" - mode_selection_reason: {reason}") + failure_type = answer_basis.get("answer_basis_failure_type") + if isinstance(failure_type, str) and failure_type and failure_type != "none": + lines.append(f" - failure_type: {failure_type}") + parse_status = answer_basis.get("answer_basis_payload_parse_status") + if isinstance(parse_status, str) and parse_status: + lines.append(f" - payload_parse_status: {parse_status}") + raw_text_present = answer_basis.get("answer_basis_raw_text_present") + if isinstance(raw_text_present, bool): + lines.append(f" - raw_text_present: {str(raw_text_present).lower()}") + validation_error = answer_basis.get("answer_basis_validation_error") + if isinstance(validation_error, str) and validation_error: + lines.append(f" - validation_error: {_short_display_text(validation_error)}") + llm_call_id = answer_basis.get("answer_basis_llm_call_data_id") + if isinstance(llm_call_id, str) and llm_call_id: + lines.append(f" - llm_call: {llm_call_id}") + trace_event_id = answer_basis.get("answer_basis_trace_event_id") + if isinstance(trace_event_id, str) and trace_event_id: + lines.append(f" - trace_event: {trace_event_id}") + prompt_ref = answer_basis.get("answer_basis_prompt_ref") + if isinstance(prompt_ref, str) and prompt_ref: + lines.append(f" - prompt: {_display_document_name(prompt_ref)}") + evidence_roles = answer_basis.get("evidence_roles") + if isinstance(evidence_roles, list) and evidence_roles: + role_summary: list[str] = [] + for role in evidence_roles[:3]: + if not isinstance(role, dict): + continue + role_summary.append( + f"{role.get('evidence_role', 'unknown')}:{role.get('source_data_id', '')}" + ) + if role_summary: + lines.append(f" - evidence_roles: {role_summary}") + if len(evidence_roles) > 3: + lines.append(f" - evidence_roles_more: {len(evidence_roles) - 3}") + lines.extend( + _metainfo_lines( + indent=2, + generated_by=str(answer_basis.get("generated_by") or "unknown"), + info_class=str(answer_basis.get("info_class") or "mixed"), + source_data_ids=_source_data_ids( + answer_basis, + fallback=[ + str(answer_basis_record.get("data_id") or "node_2:answer_basis_frame") + ], + ), + semantic_judgement_status=str( + answer_basis.get("semantic_judgement_status") or "unknown" + ), + ) + ) + node3_brief_record = _latest_run_scoped_record( result, records, @@ -786,10 +1345,40 @@ def render_runtime_view(result: dict[str, object], *, user_input: str) -> str: allowed_claims = node3_brief.get("allowed_claims") memory_selection_material = node3_brief.get("memory_selection_material") selected_recent_memory_contexts = node3_brief.get("selected_recent_memory_contexts") + l3_document_summaries = node3_brief.get("l3_document_summaries") + source_code_outlines = node3_brief.get("source_code_outlines") + excluded_document_contexts = node3_brief.get("excluded_document_contexts") + document_material_items = node3_brief.get("document_material_items") runtime_tasks = node3_brief.get("runtime_tasks") doc_count = len(read_documents) if isinstance(read_documents, list) else 0 - search_candidate_count = ( - len(search_candidate_documents) if isinstance(search_candidate_documents, list) else 0 + supplied_context_count = _payload_int( + node3_brief.get("supplied_document_context_count"), + fallback=doc_count, + ) + actual_tool_read_doc_count = _payload_int( + node3_brief.get("actual_tool_read_doc_count"), + fallback=0, + ) + actual_tool_read_code_file_count = _payload_int( + node3_brief.get("actual_tool_read_code_file_count"), + fallback=0, + ) + supplied_source_code_context_count = _payload_int( + node3_brief.get("supplied_source_code_context_count"), + fallback=0, + ) + final_search_candidate_count = _payload_int( + node3_brief.get("final_search_candidate_count"), + fallback=_payload_int( + node3_brief.get("search_candidate_count"), + fallback=len(search_candidate_documents) + if isinstance(search_candidate_documents, list) + else 0, + ), + ) + accumulated_search_candidate_count = _payload_int( + node3_brief.get("accumulated_search_candidate_count"), + fallback=final_search_candidate_count, ) claim_count = len(allowed_claims) if isinstance(allowed_claims, list) else 0 selected_context_count = ( @@ -797,15 +1386,75 @@ def render_runtime_view(result: dict[str, object], *, user_input: str) -> str: if isinstance(selected_recent_memory_contexts, list) else 0 ) + excluded_context_count = ( + len(excluded_document_contexts) + if isinstance(excluded_document_contexts, list) + else 0 + ) + material_item_count = ( + len(document_material_items) + if isinstance(document_material_items, list) + else 0 + ) + l3_document_summary_count = ( + len(l3_document_summaries) + if isinstance(l3_document_summaries, list) + else 0 + ) + source_code_outline_count = ( + len(source_code_outlines) + if isinstance(source_code_outlines, list) + else 0 + ) + llm_raw_document_text_count = _payload_int( + node3_brief.get("llm_raw_document_text_count"), + fallback=supplied_context_count, + ) + llm_l3_summary_context_count = _payload_int( + node3_brief.get("llm_l3_summary_context_count"), + fallback=l3_document_summary_count, + ) runtime_task_count = len(runtime_tasks) if isinstance(runtime_tasks, list) else 0 lines.append( "- node_3 input brief: " f"{node3_brief.get('brief_status', 'unknown')} / " - f"reportable_documents={doc_count} / search_candidates={search_candidate_count} / " + f"actual_read_doc={actual_tool_read_doc_count} / " + f"actual_read_code_file={actual_tool_read_code_file_count} / " + f"supplied_contexts={supplied_context_count} / " + f"source_code_contexts={supplied_source_code_context_count} / " + f"source_code_outlines={source_code_outline_count} / " + f"llm_raw_text={llm_raw_document_text_count} / " + f"llm_l3_summaries={llm_l3_summary_context_count} / " + f"search_candidates_final={final_search_candidate_count} / " + f"search_candidates_accumulated={accumulated_search_candidate_count} / " + f"excluded_contexts={excluded_context_count} / " + f"document_materials={material_item_count} / " + f"l3_document_summaries={l3_document_summary_count} / " f"claims={claim_count} / " f"selected_memory_contexts={selected_context_count} / " f"runtime_tasks={runtime_task_count}" ) + pack_status = node3_brief.get("document_context_pack_status") + if isinstance(pack_status, str) and pack_status != "not_recorded": + lines.append( + " - document context pack: " + f"status={pack_status} / " + f"frame={node3_brief.get('document_context_pack_frame_id', '')}" + ) + material_frame_id = node3_brief.get("document_material_packet_frame_id") + if isinstance(material_frame_id, str) and material_frame_id: + unread_count = 0 + if isinstance(document_material_items, list): + unread_count = sum( + 1 + for item in document_material_items + if isinstance(item, dict) and item.get("was_unread_candidate") is True + ) + lines.append( + " - document material packet: " + f"items={material_item_count} / unread_candidates={unread_count} / " + f"frame={material_frame_id}" + ) reasons = node3_brief.get("insufficiency_reasons") if isinstance(reasons, list) and reasons: lines.append(f" - insufficiency: {reasons}") @@ -819,6 +1468,42 @@ def render_runtime_view(result: dict[str, object], *, user_input: str) -> str: ) if selected_context_count: lines.append(f" - selected recent memory contexts: {selected_context_count}") + material_delivery_mode = node3_brief.get("material_delivery_mode") + if isinstance(material_delivery_mode, str) and material_delivery_mode: + lines.append( + " - material delivery policy: " + f"mode={material_delivery_mode} / " + f"raw_policy={node3_brief.get('raw_document_policy', '')} / " + f"summary_policy={node3_brief.get('l3_summary_policy', '')} / " + f"reason={node3_brief.get('material_policy_reason_code', '')}" + ) + l_loop_attitude_hint = node3_brief.get("l_loop_result_attitude_hint") + if isinstance(l_loop_attitude_hint, str) and l_loop_attitude_hint != "not_recorded": + lines.append( + " - L loop result in brief: " + f"task={node3_brief.get('l_loop_task_status', 'unknown')} / " + f"failure={node3_brief.get('l_loop_failure_level', 'unknown')} / " + f"semantic={node3_brief.get('l3_semantic_goal_match_status', 'unknown')} / " + f"remaining_query={node3_brief.get('remaining_query_attempts', 0)} / " + f"hint={l_loop_attitude_hint}" + ) + r_loop_material = node3_brief.get("r_loop_result_material") + if isinstance(r_loop_material, dict): + lines.append( + " - R loop result in brief: " + f"task={r_loop_material.get('r_loop_task_status', 'unknown')} / " + f"continuation={r_loop_material.get('continuation_status', 'unknown')} / " + f"budget={r_loop_material.get('budget_status', 'unknown')} / " + f"hint={r_loop_material.get('attitude_hint', 'unknown')}" + ) + answer_basis_mode = node3_brief.get("answer_basis_mode") + if isinstance(answer_basis_mode, str) and answer_basis_mode: + lines.append( + " - answer basis in brief: " + f"mode={answer_basis_mode} / " + f"reason_codes={node3_brief.get('basis_reason_codes', [])} / " + f"generated_by={node3_brief.get('answer_basis_generated_by', '')}" + ) lines.extend( _metainfo_lines( indent=2, @@ -1001,6 +1686,13 @@ def render_chat_answer(result: dict[str, object], *, user_input: str) -> str: ) return "\n".join(lines) + if result.get("status") == "structure_failed": + return _render_structure_failed_answer( + result, + search_payload=search_payload, + read_payload=read_payload, + ) + lines = [ "[answer]", "[CODE/RENDERER | LLM_REPORTER=not_run]", @@ -1058,6 +1750,80 @@ def render_chat_answer(result: dict[str, object], *, user_input: str) -> str: return "\n".join(lines) +def _render_structure_failed_answer( + result: dict[str, object], + *, + search_payload: dict[str, object], + read_payload: dict[str, object], +) -> str: + lines = [ + "[answer]", + "[CODE/RENDERER | LLM_REPORTER=not_run]", + "generated_by: CODE/RENDERER | info_class: rendered_view | semantic_judgement_status: LLM_REPORTER=not_run", + "이번 턴은 structure_failed 상태라서 송련의 정상 노드 흐름이 끝까지 기록되지 않았어.", + "node_3 최종 보고도 실행되지 않았고, 확인 가능한 trace/data 기록이 없거나 불완전해.", + ] + trace_count = result.get("trace_count") + data_record_count = result.get("data_record_count") + lines.append(f"확인된 trace/data 수: {trace_count or 0} / {data_record_count or 0}") + + stage = result.get("structure_failure_stage") + exception_type = result.get("structure_failure_exception_type") + reason = result.get("structure_failure_reason") or result.get("reason") + diagnostics: list[str] = [] + if isinstance(stage, str) and stage: + diagnostics.append(f"stage={stage}") + if isinstance(exception_type, str) and exception_type: + diagnostics.append(f"exception={exception_type}") + if isinstance(reason, str) and reason: + diagnostics.append(f"reason={_short_display_text(reason)}") + if diagnostics: + lines.append(f"짧은 진단: {', '.join(diagnostics)}") + budget_type = result.get("budget_failure_type") + if isinstance(budget_type, str) and budget_type: + lines.append( + "예산 진단: " + f"type={budget_type}, " + f"query={result.get('budget_failure_query_count', '?')}/" + f"{result.get('budget_failure_max_query_attempts', '?')}, " + f"tool={result.get('budget_failure_tool_calls', '?')}/" + f"{result.get('budget_failure_max_tool_calls', '?')}, " + f"read_doc={result.get('budget_failure_read_doc_count', '?')}/" + f"{result.get('budget_failure_max_read_doc', '?')}" + ) + budget_stage = result.get("budget_failure_stage") + if isinstance(budget_stage, str) and budget_stage: + lines.append(f"예산 실패 단계: {budget_stage}") + + if search_payload or read_payload: + lines.extend(["", "다만 DataStore에 검색/문서 payload가 남아 있어 그 범위만 재렌더링할 수 있어."]) + if search_payload: + result_count = search_payload.get("result_count", 0) + lines.append(f"- search_docs payload: 후보 {result_count}개") + top_docs = _top_search_docs(search_payload, limit=3) + if top_docs: + for index, item in enumerate(top_docs, start=1): + lines.append(f" {index}. `{_display_document_name(item.get('doc_id'))}`") + else: + lines.append("- search_docs payload: 없음") + if read_payload: + doc_name = _display_document_name(read_payload.get("doc_id")) + char_count = read_payload.get("char_count", "?") + lines.append(f"- read_doc payload: `{doc_name}` ({char_count}자)") + else: + lines.append("- read_doc payload: 없음") + else: + lines.append("확인 가능한 search_docs/read_doc payload도 없어.") + + lines.extend( + [ + "", + "따라서 사용자 질문에 대한 의미 답변은 만들지 않고, 실패 상태만 보고할게.", + ] + ) + return "\n".join(lines) + + def _node4_blocks_final_answer(node4_gate: dict[str, object]) -> bool: """node_4가 pass하지 않은 보고문은 사용자-facing 최종 답변으로 확정하지 않는다.""" @@ -1245,6 +2011,34 @@ def _first_record_with_type_prefix(result: dict[str, object], prefix: str) -> di return {} +def _document_extract_records_for_display(result: dict[str, object]) -> list[dict[str, object]]: + records = result.get("data_records") + if not isinstance(records, list): + return [] + + extract_records: list[dict[str, object]] = [] + for record in records: + if not isinstance(record, dict): + continue + data_type = record.get("data_type") + if not isinstance(data_type, str) or not _is_document_extract_data_type(data_type): + continue + extract_records.append(record) + + latest_run_index = _latest_l_run_index(result) + if latest_run_index > 1: + latest_run_prefix = f"L:run:{latest_run_index:04d}:" + latest_run_records = [ + record + for record in extract_records + if str(record.get("data_id") or "").startswith(latest_run_prefix) + ] + if latest_run_records: + return latest_run_records + + return extract_records + + def _latest_document_extract_record(result: dict[str, object]) -> dict[str, object]: records = result.get("data_records") if not isinstance(records, list): @@ -1277,6 +2071,14 @@ def _is_document_extract_data_type(data_type: str) -> bool: return data_type.startswith("tool_result:read_doc") or data_type.startswith("tool_result:read_artifact") +def _document_extract_tool_name(record: dict[str, object]) -> str: + data_type = str(record.get("data_type") or "") + data_id = str(record.get("data_id") or "") + if data_type.startswith("tool_result:read_artifact") or "tool_result:read_artifact" in data_id: + return "read_artifact" + return "read_doc" + + def _payload_from_record(record: dict[str, object]) -> dict[str, object]: payload = record.get("payload") if isinstance(record, dict) else None return payload if isinstance(payload, dict) else {} @@ -1325,6 +2127,73 @@ def _metainfo_lines( return lines +def _structure_failure_runtime_lines(result: dict[str, object]) -> list[str]: + lines: list[str] = [] + field_labels = [ + ("structure_failure_stage", "stage"), + ("structure_failure_node", "node"), + ("structure_failure_prompt_ref", "prompt"), + ("structure_failure_exception_type", "exception"), + ("structure_failure_reason", "reason"), + ("structure_failure_llm_call_data_id", "llm_call"), + ("structure_failure_trace_event_id", "trace_event"), + ] + for field_name, label in field_labels: + value = result.get(field_name) + if not isinstance(value, str) or not value: + continue + display_value = _display_document_name(value) if label == "prompt" else _short_display_text(value) + lines.append(f"- structure_failure_{label}: {display_value}") + budget_lines = _budget_failure_runtime_lines(result) + if budget_lines: + lines.extend(budget_lines) + return lines + + +def _budget_failure_runtime_lines(result: dict[str, object]) -> list[str]: + budget_type = result.get("budget_failure_type") + if not isinstance(budget_type, str) or not budget_type: + return [] + lines = [f"- budget_failure_type: {budget_type}"] + text_fields = [ + ("budget_failure_stage", "stage"), + ("budget_failure_reason", "reason"), + ("budget_failure_frame_id", "frame"), + ("budget_failure_route", "route"), + ("budget_failure_l_run_id", "l_run"), + ] + for field_name, label in text_fields: + value = result.get(field_name) + if isinstance(value, str) and value: + lines.append(f"- budget_failure_{label}: {_short_display_text(value)}") + source_ids = result.get("budget_failure_source_data_ids") + if isinstance(source_ids, list): + lines.append( + "- budget_failure_source_data_ids: " + f"{_format_source_ids([item for item in source_ids if isinstance(item, str)])}" + ) + count_fields = [ + ("budget_failure_query_count", "query_count"), + ("budget_failure_max_query_attempts", "max_query_attempts"), + ("budget_failure_tool_calls", "tool_calls"), + ("budget_failure_max_tool_calls", "max_tool_calls"), + ("budget_failure_read_doc_count", "read_doc_count"), + ("budget_failure_max_read_doc", "max_read_doc"), + ] + for field_name, label in count_fields: + value = result.get(field_name) + if isinstance(value, int): + lines.append(f"- budget_failure_{label}: {value}") + return lines + + +def _short_display_text(text: str, *, limit: int = 180) -> str: + compact = " ".join(text.split()) + if len(compact) <= limit: + return compact + return f"{compact[: limit - 3]}..." + + def _source_data_ids(payload: dict[str, object], *, fallback: list[str]) -> list[str]: values = payload.get("source_data_ids") if isinstance(values, list): @@ -1396,6 +2265,10 @@ def _list_count(value: object) -> int: return len(value) if isinstance(value, list) else 0 +def _payload_int(value: object, *, fallback: int = 0) -> int: + return value if isinstance(value, int) and value >= 0 else fallback + + def _memory_item_type_count(value: object, item_type: str) -> int: if not isinstance(value, list): return 0 diff --git a/songryeon_core/runtime/user_turn.py b/songryeon_core/runtime/user_turn.py index 78eeb0e..8b65d2f 100644 --- a/songryeon_core/runtime/user_turn.py +++ b/songryeon_core/runtime/user_turn.py @@ -11,6 +11,7 @@ llm_runtime_status, ) from songryeon_core.runtime.defaults import ( + DEFAULT_MAX_DOCUMENT_CONTEXT_CHARS, DEFAULT_MAX_INPUT_CHARS, DEFAULT_MAX_QUERY_ATTEMPTS, DEFAULT_MAX_READ_DOC_CALLS, @@ -33,10 +34,12 @@ def run_fake_user_turn( max_query_candidates: int | None = None, max_read_doc_calls: int = DEFAULT_MAX_READ_DOC_CALLS, max_input_chars: int = DEFAULT_MAX_INPUT_CHARS, + max_document_context_chars: int = DEFAULT_MAX_DOCUMENT_CONTEXT_CHARS, include_data_records: bool = False, force_l_route: bool = False, same_turn_l_reroute_enabled: bool = False, max_l_runs_per_turn: int = 1, + enable_r_route_experimental: bool = False, live_trace: bool = False, turn_id: str | None = None, previous_turn_capsules: list[TurnStateCapsule] | None = None, @@ -55,6 +58,7 @@ def run_fake_user_turn( node_1_router_adapter=adapter, memory_relevance_selector_adapter=adapter, l1_goal_adapter=adapter, + l_tool_scope_adapter=adapter, l2_query_planner_adapter=adapter, l3_result_adapter=adapter, node_2_boundary_adapter=adapter, @@ -67,9 +71,11 @@ def run_fake_user_turn( max_query_candidates=max_query_candidates, max_read_doc_calls=max_read_doc_calls, max_input_chars=max_input_chars, + max_document_context_chars=max_document_context_chars, force_l_route=force_l_route, same_turn_l_reroute_enabled=same_turn_l_reroute_enabled, max_l_runs_per_turn=max_l_runs_per_turn, + enable_r_route_experimental=enable_r_route_experimental, previous_turn_capsules=previous_turn_capsules, recent_raw_conversation=recent_raw_conversation, live_trace_sink=make_live_trace_sink(enabled=live_trace), @@ -100,10 +106,12 @@ def run_qwen_user_turn( max_query_candidates: int | None = None, max_read_doc_calls: int = DEFAULT_MAX_READ_DOC_CALLS, max_input_chars: int = DEFAULT_MAX_INPUT_CHARS, + max_document_context_chars: int = DEFAULT_MAX_DOCUMENT_CONTEXT_CHARS, include_data_records: bool = False, force_l_route: bool = False, same_turn_l_reroute_enabled: bool = False, max_l_runs_per_turn: int = 1, + enable_r_route_experimental: bool = False, live_trace: bool = False, turn_id: str | None = None, previous_turn_capsules: list[TurnStateCapsule] | None = None, @@ -139,6 +147,7 @@ def run_qwen_user_turn( node_1_router_adapter=adapter, memory_relevance_selector_adapter=adapter, l1_goal_adapter=adapter, + l_tool_scope_adapter=adapter, l2_query_planner_adapter=adapter, l3_result_adapter=adapter, node_2_boundary_adapter=adapter, @@ -151,9 +160,11 @@ def run_qwen_user_turn( max_query_candidates=max_query_candidates, max_read_doc_calls=max_read_doc_calls, max_input_chars=max_input_chars, + max_document_context_chars=max_document_context_chars, force_l_route=force_l_route, same_turn_l_reroute_enabled=same_turn_l_reroute_enabled, max_l_runs_per_turn=max_l_runs_per_turn, + enable_r_route_experimental=enable_r_route_experimental, allow_node_1_router_fallback=False, node_1_router_fallback_policy=ROUTER_FALLBACK_POLICY_QWEN_STRICT_BLOCKED, previous_turn_capsules=previous_turn_capsules, @@ -161,10 +172,12 @@ def run_qwen_user_turn( live_trace_sink=make_live_trace_sink(enabled=live_trace), ) except Exception as exc: + diagnostics = _structure_failure_diagnostics(exc) return { "status": "structure_failed", "reason": exc.__class__.__name__, "error": str(exc), + **diagnostics, "runtime": runtime, "user_input": user_input, } @@ -203,10 +216,19 @@ def _turn_response( "tool_distillation_count": result.get("tool_distillation_count"), "tool_budget_frame_count": result.get("tool_budget_frame_count"), "l_loop_budget_plan_count": result.get("l_loop_budget_plan_count"), + "r_route_experimental_enabled": result.get("r_route_experimental_enabled"), + "r_route_experimental_status": result.get("r_route_experimental_status"), + "r_route_experimental_return_summary_id": result.get( + "r_route_experimental_return_summary_id" + ), + "r_route_experimental_close_route_id": result.get( + "r_route_experimental_close_route_id" + ), "task_frame_count": result.get("task_frame_count"), "task_result_count": result.get("task_result_count"), "search_top_k": result.get("search_top_k"), "max_query_attempts": result.get("max_query_attempts"), + "max_document_context_chars": result.get("max_document_context_chars"), "l2_query_source": result.get("l2_query_source"), "l2_query_plan_present": ( "L2:query_plan_frame" in data_ids @@ -261,6 +283,29 @@ def _turn_response( "older_unmanaged_raw_turn_count": result.get("older_unmanaged_raw_turn_count"), "l1_goal_generation_source": result.get("l1_goal_generation_source"), "l3_achievement_generation_source": result.get("l3_achievement_generation_source"), + "node2_answer_basis_mode": result.get("node2_answer_basis_mode"), + "node2_answer_basis_generated_by": result.get("node2_answer_basis_generated_by"), + "node2_answer_basis_reason_codes": result.get("node2_answer_basis_reason_codes"), + "node2_answer_basis_semantic_judgement_status": result.get( + "node2_answer_basis_semantic_judgement_status" + ), + "node2_answer_basis_failure_type": result.get("node2_answer_basis_failure_type"), + "node2_answer_basis_llm_call_data_id": result.get( + "node2_answer_basis_llm_call_data_id" + ), + "node2_answer_basis_trace_event_id": result.get( + "node2_answer_basis_trace_event_id" + ), + "node2_answer_basis_validation_error": result.get( + "node2_answer_basis_validation_error" + ), + "node2_answer_basis_raw_text_present": result.get( + "node2_answer_basis_raw_text_present" + ), + "node2_answer_basis_prompt_ref": result.get("node2_answer_basis_prompt_ref"), + "node2_answer_basis_payload_parse_status": result.get( + "node2_answer_basis_payload_parse_status" + ), "node3_reporter_status": result.get("node3_reporter_status"), "node4_gate_status": result.get("node4_gate_status"), "node4_recent_memory_guard_status": result.get( @@ -284,6 +329,78 @@ def _turn_response( return response +def _structure_failure_diagnostics(exc: Exception) -> dict[str, object]: + message = _short_diagnostic_text(str(exc) or exc.__class__.__name__) + node = _infer_failure_node(message) + diagnostics = { + "structure_failure_stage": _infer_failure_stage(message, node=node), + "structure_failure_reason": message or exc.__class__.__name__, + "structure_failure_exception_type": exc.__class__.__name__, + "structure_failure_llm_call_data_id": None, + "structure_failure_trace_event_id": None, + "structure_failure_node": node, + "structure_failure_prompt_ref": _prompt_ref_for_node(node), + } + budget_diagnostics = getattr(exc, "budget_diagnostics", None) + if isinstance(budget_diagnostics, dict): + diagnostics.update(budget_diagnostics) + diagnostics["structure_failure_stage"] = "run_dry_turn" + diagnostics["structure_failure_node"] = "L" + return diagnostics + + +def _infer_failure_node(message: str) -> str: + lowered = message.lower() + if "node_4" in lowered or "gatekeeper" in lowered: + return "node_4" + if "node_3" in lowered or "reporter" in lowered: + return "node_3" + if "answer_basis" in lowered or "node2 answer" in lowered or "node_2" in lowered: + return "node_2" + if "node_1" in lowered or "router" in lowered: + return "node_1" + if "toolusebudgetframe" in lowered or "tool use budget" in lowered: + return "L" + if "l3" in lowered: + return "L3" + if "l2" in lowered: + return "L2" + if "l1" in lowered: + return "L1" + return "unknown" + + +def _infer_failure_stage(message: str, *, node: str) -> str: + lowered = message.lower() + if "schema" in lowered: + return f"{node}:schema_validation" if node != "unknown" else "schema_validation" + if "parse" in lowered or "json" in lowered: + return f"{node}:payload_parse" if node != "unknown" else "payload_parse" + if node != "unknown": + return f"{node}:run" + return "run_dry_turn" + + +def _prompt_ref_for_node(node: str) -> str | None: + prompt_refs = { + "node_1": "songryeon_core/prompts/node_1_router_v0.md", + "L1": "songryeon_core/prompts/l1_goal_setter_v0.md", + "L2": "songryeon_core/prompts/l2_query_setter_v0.md", + "L3": "songryeon_core/prompts/l3_result_keeper_v0.md", + "node_2": "songryeon_core/prompts/node_2_answer_basis_selector_v0.md", + "node_3": "songryeon_core/prompts/node_3_reporter_v0.md", + "node_4": "songryeon_core/prompts/node_4_gatekeeper_v0.md", + } + return prompt_refs.get(node) + + +def _short_diagnostic_text(text: str, *, limit: int = 240) -> str: + compact = " ".join(text.split()) + if len(compact) <= limit: + return compact + return f"{compact[: limit - 3]}..." + + def _status_from_result(result: dict[str, object]) -> str: if result.get("l2_query_source") == "llm_query_plan": return "ok" diff --git a/songryeon_core/tools/code_tools.py b/songryeon_core/tools/code_tools.py new file mode 100644 index 0000000..05923bf --- /dev/null +++ b/songryeon_core/tools/code_tools.py @@ -0,0 +1,242 @@ +from __future__ import annotations + +from pathlib import Path + + +DEFAULT_CODE_FILE_EXTENSIONS = { + ".py", + ".json", + ".toml", + ".yml", + ".yaml", +} +DEFAULT_IGNORED_DIR_NAMES = { + ".git", + ".mypy_cache", + ".pytest_cache", + ".ruff_cache", + ".venv", + "__pycache__", + "build", + "dist", + "node_modules", + "venv", +} + + +def list_code_files( + *, + root: str | Path, + max_files: int = 500, + include_extensions: list[str] | None = None, +) -> dict[str, object]: + """Workspace 안의 읽기 가능한 코드/설정 파일 목록을 절대정보로 반환한다.""" + + root_path = Path(root).resolve() + allowed_extensions = _normalized_extensions(include_extensions) + files: list[dict[str, object]] = [] + total_count = 0 + for path in _iter_code_files(root_path=root_path, allowed_extensions=allowed_extensions): + total_count += 1 + if len(files) >= max_files: + continue + files.append(_file_listing_item(root_path=root_path, path=path)) + + return { + "root": str(root_path), + "allowed_extensions": sorted(allowed_extensions), + "file_count": total_count, + "returned_file_count": len(files), + "truncated": total_count > len(files), + "files": files, + } + + +def search_code( + *, + root: str | Path, + query: str, + max_results: int = 50, + max_line_chars: int = 240, + include_extensions: list[str] | None = None, +) -> dict[str, object]: + """Workspace 코드 파일에서 단순 부분문자열 검색 결과를 절대정보로 반환한다.""" + + root_path = Path(root).resolve() + allowed_extensions = _normalized_extensions(include_extensions) + normalized_query = query.strip() + if not normalized_query: + return { + "root": str(root_path), + "query": query, + "match_count": 0, + "returned_match_count": 0, + "file_match_count": 0, + "truncated": False, + "results": [], + } + + query_key = normalized_query.casefold() + results: list[dict[str, object]] = [] + matched_files: set[str] = set() + match_count = 0 + for path in _iter_code_files(root_path=root_path, allowed_extensions=allowed_extensions): + try: + lines = path.read_text(encoding="utf-8", errors="replace").splitlines() + except OSError: + continue + relative_path = _relative_posix_path(root_path=root_path, path=path) + for line_number, line in enumerate(lines, start=1): + if query_key not in line.casefold(): + continue + match_count += 1 + matched_files.add(relative_path) + if len(results) >= max_results: + continue + line_text = line.strip() + truncated = len(line_text) > max_line_chars + results.append( + { + "result_id": f"code_match_{len(results) + 1:04d}", + "file_path": relative_path, + "line_number": line_number, + "line_text": _truncate(line_text, max_line_chars), + "line_text_truncated": truncated, + "line_char_count": len(line_text), + } + ) + + return { + "root": str(root_path), + "query": normalized_query, + "match_count": match_count, + "returned_match_count": len(results), + "file_match_count": len(matched_files), + "truncated": match_count > len(results), + "results": results, + } + + +def read_code_file( + *, + root: str | Path, + file_path: str, + max_chars: int = 12000, +) -> dict[str, object]: + """Workspace 안의 코드 파일 하나를 읽기 전용으로 반환한다.""" + + root_path = Path(root).resolve() + resolved = _resolve_code_file(root_path=root_path, file_path=file_path) + if resolved["status"] != "ok": + return { + "root": str(root_path), + "file_path": file_path, + "exists": False, + "read_status": resolved["status"], + "text": "", + "char_count": 0, + "line_count": 0, + "size_bytes": 0, + "truncated": False, + "max_chars": max_chars, + } + + path = resolved["path"] + assert isinstance(path, Path) + text = path.read_text(encoding="utf-8", errors="replace") + returned_text = text[:max_chars] + return { + "root": str(root_path), + "file_path": _relative_posix_path(root_path=root_path, path=path), + "exists": True, + "read_status": "ok", + "text": returned_text, + "char_count": len(text), + "line_count": _line_count(text), + "size_bytes": path.stat().st_size, + "truncated": len(text) > len(returned_text), + "max_chars": max_chars, + } + + +def _iter_code_files( + *, + root_path: Path, + allowed_extensions: set[str], +) -> list[Path]: + if not root_path.exists(): + return [] + files: list[Path] = [] + for path in root_path.rglob("*"): + if not path.is_file(): + continue + if _has_ignored_part(path): + continue + if path.suffix.lower() not in allowed_extensions: + continue + files.append(path) + return sorted(files, key=lambda item: _relative_posix_path(root_path=root_path, path=item)) + + +def _file_listing_item(*, root_path: Path, path: Path) -> dict[str, object]: + text = path.read_text(encoding="utf-8", errors="replace") + return { + "file_path": _relative_posix_path(root_path=root_path, path=path), + "extension": path.suffix.lower(), + "size_bytes": path.stat().st_size, + "line_count": _line_count(text), + } + + +def _resolve_code_file(*, root_path: Path, file_path: str) -> dict[str, object]: + raw_path = Path(file_path) + if raw_path.is_absolute(): + return {"status": "absolute_path_rejected"} + candidate = (root_path / raw_path).resolve() + try: + candidate.relative_to(root_path) + except ValueError: + return {"status": "path_outside_workspace_rejected"} + if _has_ignored_part(candidate): + return {"status": "ignored_path_rejected"} + if not candidate.exists(): + return {"status": "not_found"} + if not candidate.is_file(): + return {"status": "not_file"} + if candidate.suffix.lower() not in DEFAULT_CODE_FILE_EXTENSIONS: + return {"status": "unsupported_extension"} + return {"status": "ok", "path": candidate} + + +def _normalized_extensions(include_extensions: list[str] | None) -> set[str]: + if not include_extensions: + return set(DEFAULT_CODE_FILE_EXTENSIONS) + result: set[str] = set() + for extension in include_extensions: + value = extension.strip().lower() + if not value: + continue + if not value.startswith("."): + value = f".{value}" + result.add(value) + return result or set(DEFAULT_CODE_FILE_EXTENSIONS) + + +def _has_ignored_part(path: Path) -> bool: + return any(part in DEFAULT_IGNORED_DIR_NAMES for part in path.parts) + + +def _relative_posix_path(*, root_path: Path, path: Path) -> str: + return path.relative_to(root_path).as_posix() + + +def _line_count(text: str) -> int: + if not text: + return 0 + return len(text.splitlines()) + + +def _truncate(text: str, max_chars: int) -> str: + if max_chars < 4 or len(text) <= max_chars: + return text[:max_chars] + return f"{text[: max_chars - 3].rstrip()}..." diff --git a/songryeon_core/tools/document_context_pack.py b/songryeon_core/tools/document_context_pack.py new file mode 100644 index 0000000..1c82cbb --- /dev/null +++ b/songryeon_core/tools/document_context_pack.py @@ -0,0 +1,483 @@ +from __future__ import annotations + +import re +from dataclasses import asdict, dataclass +from pathlib import Path + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.schemas import ( + DocumentContextPackExcludedDocument, + DocumentContextPackFrame, + DocumentContextPackIncludedDocument, + ExplicitArtifactReferenceFrame, + ExplicitArtifactResolvedReference, + validate_document_context_pack_frame, + validate_explicit_artifact_reference_frame, +) +from songryeon_core.core.trace_store import TraceStore +from songryeon_core.loops.l_loop_namespace import LRunIds +from songryeon_core.tools.document_tools import read_artifact, read_doc + + +EXPLICIT_ARTIFACT_REFERENCE_DATA_TYPE = "node_output:explicit_artifact_reference_frame" +DOCUMENT_CONTEXT_PACK_DATA_TYPE = "node_output:document_context_pack_frame" + +_ARTIFACT_REF_RE = re.compile( + r"(?'\"`" + + +@dataclass +class _DocumentCandidate: + doc_id: str + selection_basis: str + source_data_id: str + rank_index: int = 0 + + +def explicit_artifact_reference_frame_id( + *, + id_namespace: LRunIds | None, +) -> str: + legacy_id = "L:explicit_artifact_reference_frame" + if id_namespace is None: + return legacy_id + return id_namespace.scoped_data_id(legacy_id) + + +def document_context_pack_frame_id( + *, + id_namespace: LRunIds | None, +) -> str: + legacy_id = "L:document_context_pack_frame" + if id_namespace is None: + return legacy_id + return id_namespace.scoped_data_id(legacy_id) + + +def extract_explicit_artifact_references(user_text: str) -> list[str]: + """Return visible ORDER/artifact references in user-text order.""" + + refs: list[str] = [] + seen_spans: set[tuple[int, int]] = set() + for match in _ARTIFACT_REF_RE.finditer(user_text or ""): + raw_ref = match.group(0).strip().strip(_TRAILING_PUNCTUATION) + if not raw_ref: + continue + span = match.span() + if span in seen_spans: + continue + seen_spans.add(span) + refs.append(raw_ref) + return refs + + +def record_explicit_artifact_reference_frame( + *, + trace_store: TraceStore, + data_store: DataStore, + turn_id: str, + user_text: str, + document_root: str | Path, + source_trace_ids: list[str], + source_data_ids: list[str], + frame_id: str, +) -> tuple[str, str, ExplicitArtifactReferenceFrame]: + """Extract explicit artifact refs and resolve them before embedding candidates are packed.""" + + frame = build_explicit_artifact_reference_frame( + turn_id=turn_id, + user_text=user_text, + document_root=document_root, + frame_id=frame_id, + source_trace_ids=source_trace_ids, + source_data_ids=source_data_ids, + ) + validate_explicit_artifact_reference_frame(frame) + event = trace_store.create_event( + turn_id=turn_id, + actor="document_context_builder", + event_type="node_output", + input_ref=frame.source_trace_ids, + output_ref=[frame.frame_id], + schema_status="passed", + ) + data_store.create_record( + data_id=frame.frame_id, + data_type=EXPLICIT_ARTIFACT_REFERENCE_DATA_TYPE, + exists=True, + created_at=event.timestamp, + source_trace_id=event.event_id, + payload=asdict(frame), + ) + return event.event_id, frame.frame_id, frame + + +def build_explicit_artifact_reference_frame( + *, + turn_id: str, + user_text: str, + document_root: str | Path, + frame_id: str, + source_trace_ids: list[str], + source_data_ids: list[str], +) -> ExplicitArtifactReferenceFrame: + refs = extract_explicit_artifact_references(user_text) + resolved: list[ExplicitArtifactResolvedReference] = [] + for index, raw_ref in enumerate(refs, start=1): + payload = read_artifact(root=document_root, artifact_ref=raw_ref) + status = str(payload.get("match_status") or "not_found") + if status == "invalid_ref": + resolve_status = "invalid_ref" + elif status == "ambiguous": + resolve_status = "ambiguous" + elif status == "unique": + resolve_status = "unique" + else: + resolve_status = "not_found" + candidates = payload.get("candidates") + resolved.append( + ExplicitArtifactResolvedReference( + raw_ref=raw_ref, + normalized_ref=_normalize_ref(raw_ref), + occurrence_index=index, + resolve_status=resolve_status, + candidate_count=_int(payload.get("candidate_count")), + selected_doc_id=_optional_text(payload.get("selected_doc_id")) + if resolve_status == "unique" + else None, + match_type=_optional_text(payload.get("match_type")) + if resolve_status == "unique" + else None, + char_count=_int(payload.get("char_count")) if resolve_status == "unique" else 0, + candidates=[item for item in candidates if isinstance(item, dict)] + if isinstance(candidates, list) + else [], + ) + ) + + return ExplicitArtifactReferenceFrame( + frame_id=frame_id, + turn_id=turn_id, + source_user_text=user_text, + extracted_reference_count=len(resolved), + resolved_references=resolved, + unique_count=sum(1 for item in resolved if item.resolve_status == "unique"), + ambiguous_count=sum(1 for item in resolved if item.resolve_status == "ambiguous"), + not_found_count=sum(1 for item in resolved if item.resolve_status == "not_found"), + invalid_count=sum(1 for item in resolved if item.resolve_status == "invalid_ref"), + source_trace_ids=_unique_strings(source_trace_ids), + source_data_ids=_unique_strings(source_data_ids), + ) + + +def record_document_context_pack_frame( + *, + trace_store: TraceStore, + data_store: DataStore, + turn_id: str, + document_root: str | Path, + max_document_context_chars: int, + frame_id: str, + explicit_reference_data_id: str, + source_trace_ids: list[str], + source_data_ids: list[str], + id_namespace: LRunIds | None, +) -> tuple[str, str, DocumentContextPackFrame]: + """Build and store the whole-document context pack for node_3.""" + + frame = build_document_context_pack_frame( + data_store=data_store, + turn_id=turn_id, + document_root=document_root, + max_document_context_chars=max_document_context_chars, + frame_id=frame_id, + explicit_reference_data_id=explicit_reference_data_id, + source_trace_ids=source_trace_ids, + source_data_ids=source_data_ids, + id_namespace=id_namespace, + ) + validate_document_context_pack_frame(frame) + event = trace_store.create_event( + turn_id=turn_id, + actor="document_context_builder", + event_type="node_output", + input_ref=frame.source_trace_ids, + output_ref=[frame.frame_id], + schema_status="passed", + ) + data_store.create_record( + data_id=frame.frame_id, + data_type=DOCUMENT_CONTEXT_PACK_DATA_TYPE, + exists=True, + created_at=event.timestamp, + source_trace_id=event.event_id, + payload=asdict(frame), + ) + return event.event_id, frame.frame_id, frame + + +def build_document_context_pack_frame( + *, + data_store: DataStore, + turn_id: str, + document_root: str | Path, + max_document_context_chars: int, + frame_id: str, + explicit_reference_data_id: str, + source_trace_ids: list[str], + source_data_ids: list[str], + id_namespace: LRunIds | None, +) -> DocumentContextPackFrame: + candidates = _ranked_document_candidates( + data_store=data_store, + explicit_reference_data_id=explicit_reference_data_id, + id_namespace=id_namespace, + ) + source_query_frame_ids = _source_query_frame_ids(data_store, id_namespace=id_namespace) + source_search_result_data_ids = _source_search_result_data_ids(candidates) + included: list[DocumentContextPackIncludedDocument] = [] + excluded: list[DocumentContextPackExcludedDocument] = [] + included_total_chars = 0 + strict_cutoff_seen = False + cutoff_reason = "none" + + for rank_index, candidate in enumerate(candidates, start=1): + try: + payload = read_doc(root=document_root, doc_id=candidate.doc_id) + except (FileNotFoundError, ValueError): + excluded.append( + DocumentContextPackExcludedDocument( + doc_id=candidate.doc_id, + document_name=_document_name(candidate.doc_id), + char_count=0, + rank_index=rank_index, + selection_basis=candidate.selection_basis, + exclusion_reason="excluded_not_readable_markdown_document", + would_exceed_budget=False, + source_data_id=candidate.source_data_id, + ) + ) + continue + text = str(payload.get("text") or "") + char_count = payload.get("char_count") + if not isinstance(char_count, int): + char_count = len(text) + candidate.rank_index = rank_index + if strict_cutoff_seen: + excluded.append( + DocumentContextPackExcludedDocument( + doc_id=candidate.doc_id, + document_name=_document_name(candidate.doc_id), + char_count=char_count, + rank_index=rank_index, + selection_basis=candidate.selection_basis, + exclusion_reason="excluded_after_strict_rank_cutoff", + would_exceed_budget=False, + source_data_id=candidate.source_data_id, + ) + ) + continue + if included_total_chars + char_count <= max_document_context_chars: + included.append( + DocumentContextPackIncludedDocument( + doc_id=candidate.doc_id, + document_name=_document_name(candidate.doc_id), + char_count=char_count, + rank_index=rank_index, + selection_basis=candidate.selection_basis, + text=text, + source_data_id=candidate.source_data_id, + ) + ) + included_total_chars += char_count + continue + excluded.append( + DocumentContextPackExcludedDocument( + doc_id=candidate.doc_id, + document_name=_document_name(candidate.doc_id), + char_count=char_count, + rank_index=rank_index, + selection_basis=candidate.selection_basis, + exclusion_reason="excluded_due_to_context_budget", + would_exceed_budget=True, + source_data_id=candidate.source_data_id, + ) + ) + strict_cutoff_seen = True + cutoff_reason = f"excluded_due_to_context_budget at {candidate.doc_id}" + + pack_source_data_ids = _unique_strings( + [ + explicit_reference_data_id, + *source_query_frame_ids, + *source_search_result_data_ids, + *source_data_ids, + ] + ) + return DocumentContextPackFrame( + frame_id=frame_id, + turn_id=turn_id, + max_document_context_chars=max_document_context_chars, + budget_unit="chars", + whole_document_only=True, + strict_rank_order=True, + included_documents=included, + excluded_documents=excluded, + included_document_count=len(included), + excluded_document_count=len(excluded), + included_total_chars=included_total_chars, + cutoff_reason=cutoff_reason, + source_query_frame_ids=source_query_frame_ids, + source_search_result_data_ids=source_search_result_data_ids, + source_explicit_reference_data_id=explicit_reference_data_id, + source_trace_ids=_unique_strings(source_trace_ids), + source_data_ids=pack_source_data_ids, + ) + + +def latest_document_context_pack_payload( + data_store: DataStore, + *, + id_namespace: LRunIds | None, +) -> dict[str, object]: + for record in reversed(data_store.list_records()): + if not _record_in_namespace(record.data_id, id_namespace=id_namespace): + continue + if record.data_type != DOCUMENT_CONTEXT_PACK_DATA_TYPE: + continue + return record.payload if isinstance(record.payload, dict) else {} + return {} + + +def _ranked_document_candidates( + *, + data_store: DataStore, + explicit_reference_data_id: str, + id_namespace: LRunIds | None, +) -> list[_DocumentCandidate]: + candidates: list[_DocumentCandidate] = [] + seen_doc_ids: set[str] = set() + + explicit_payload = _payload(data_store, explicit_reference_data_id) + resolved = explicit_payload.get("resolved_references") + if isinstance(resolved, list): + for item in resolved: + if not isinstance(item, dict) or item.get("resolve_status") != "unique": + continue + doc_id = item.get("selected_doc_id") + raw_ref = item.get("raw_ref") + if not isinstance(doc_id, str) or not doc_id or doc_id in seen_doc_ids: + continue + seen_doc_ids.add(doc_id) + candidates.append( + _DocumentCandidate( + doc_id=doc_id, + selection_basis=f"explicit_artifact_reference_unique:{raw_ref or doc_id}", + source_data_id=explicit_reference_data_id, + ) + ) + + for record in data_store.list_records(): + if not _record_in_namespace(record.data_id, id_namespace=id_namespace): + continue + if not record.data_type.startswith("node_output:L3") or "preserved_info_frame" not in record.data_type: + continue + if not isinstance(record.payload, dict): + continue + raw_candidates = record.payload.get("candidates") + if not isinstance(raw_candidates, list): + continue + for item in raw_candidates: + if not isinstance(item, dict): + continue + doc_id = item.get("doc_id") + if not isinstance(doc_id, str) or not doc_id or doc_id in seen_doc_ids: + continue + seen_doc_ids.add(doc_id) + source_data_id = item.get("source_data_id") + candidates.append( + _DocumentCandidate( + doc_id=doc_id, + selection_basis="embedding_search_candidate", + source_data_id=source_data_id if isinstance(source_data_id, str) and source_data_id else record.data_id, + ) + ) + return candidates + + +def _source_query_frame_ids( + data_store: DataStore, + *, + id_namespace: LRunIds | None, +) -> list[str]: + ids: list[str] = [] + for record in data_store.list_records(): + if not _record_in_namespace(record.data_id, id_namespace=id_namespace): + continue + if record.data_type in { + "node_output:L2_query_frame", + "node_output:L2_revision_query_frame", + }: + ids.append(record.data_id) + return _unique_strings(ids) + + +def _source_search_result_data_ids(candidates: list[_DocumentCandidate]) -> list[str]: + return _unique_strings( + [ + candidate.source_data_id + for candidate in candidates + if candidate.selection_basis == "embedding_search_candidate" + ] + ) + + +def _payload(data_store: DataStore, data_id: str) -> dict[str, object]: + record = data_store.get_record(data_id) + if record is None or not isinstance(record.payload, dict): + return {} + return record.payload + + +def _record_in_namespace(data_id: str, *, id_namespace: LRunIds | None) -> bool: + if id_namespace is None: + return True + return id_namespace.owns_data_id(data_id) + + +def _normalize_ref(raw_ref: str) -> str: + normalized = str(raw_ref or "").strip().strip("`'\"").replace("\\", "/") + normalized = normalized.strip().strip("/") + normalized = normalized.removeprefix("Administrative_Reform_1/") + return normalized.lower() + + +def _document_name(doc_id: str) -> str: + normalized = doc_id.replace("\\", "/").strip("/") + return normalized.rsplit("/", 1)[-1] or doc_id + + +def _optional_text(value: object) -> str | None: + return value if isinstance(value, str) and value else None + + +def _int(value: object) -> int: + return value if isinstance(value, int) else 0 + + +def _unique_strings(values: list[str | None]) -> list[str]: + seen: set[str] = set() + unique_values: list[str] = [] + for value in values: + if not value or value in seen: + continue + seen.add(value) + unique_values.append(value) + return unique_values diff --git a/songryeon_core/tools/document_tools.py b/songryeon_core/tools/document_tools.py index 7e973f2..f095e4c 100644 --- a/songryeon_core/tools/document_tools.py +++ b/songryeon_core/tools/document_tools.py @@ -1,5 +1,6 @@ from __future__ import annotations +import re from dataclasses import asdict from pathlib import Path @@ -209,7 +210,9 @@ def _count_field(items: list[object], field_name: str) -> dict[str, int]: "path_suffix_exact": 3, "filename_exact": 4, "filename_stem_exact": 5, + "order_number_prefix": 6, } +_ORDER_NUMBER_REF_RE = re.compile(r"^order_\d{3}$") def _artifact_candidates( @@ -268,6 +271,11 @@ def _artifact_match_type(normalized_ref: str, doc_id: str) -> str | None: return "filename_exact" if normalized_ref_no_suffix == filename_no_suffix: return "filename_stem_exact" + if normalized_doc.startswith("04_orders/") and _ORDER_NUMBER_REF_RE.match(normalized_ref_no_suffix) and ( + filename_no_suffix == normalized_ref_no_suffix + or filename_no_suffix.startswith(f"{normalized_ref_no_suffix}_") + ): + return "order_number_prefix" return None diff --git a/songryeon_core/tools/tool_efficiency_policy.py b/songryeon_core/tools/tool_efficiency_policy.py index b5851a3..bd5f949 100644 --- a/songryeon_core/tools/tool_efficiency_policy.py +++ b/songryeon_core/tools/tool_efficiency_policy.py @@ -13,6 +13,14 @@ from songryeon_core.loops.l_loop_namespace import LRunIds +class BudgetConsistencyError(ValueError): + """ToolUseBudgetFrame 검증 실패를 구조화된 진단과 함께 올린다.""" + + def __init__(self, message: str, *, diagnostics: dict[str, object]) -> None: + super().__init__(message) + self.budget_diagnostics = diagnostics + + def tool_budget_data_id( turn_id: str, sequence_index: int, @@ -85,7 +93,15 @@ def record_tool_use_budget_frame( source_trace_ids=source_trace_ids or [], source_data_ids=source_data_ids or [], ) - validate_tool_use_budget_frame(frame) + try: + validate_tool_use_budget_frame(frame) + except (TypeError, ValueError) as exc: + diagnostics = _budget_failure_diagnostics( + frame=frame, + reason=str(exc), + id_namespace=id_namespace, + ) + raise BudgetConsistencyError(str(exc), diagnostics=diagnostics) from exc event = trace_store.create_event( turn_id=turn_id, actor="tool_efficiency_policy", @@ -105,6 +121,51 @@ def record_tool_use_budget_frame( return event.event_id, budget_id +def _budget_failure_diagnostics( + *, + frame: ToolUseBudgetFrame, + reason: str, + id_namespace: LRunIds | None, +) -> dict[str, object]: + return { + "budget_failure_type": _budget_failure_type(frame, reason=reason), + "budget_failure_reason": reason, + "budget_failure_frame_id": frame.budget_id, + "budget_failure_source_data_ids": list(frame.source_data_ids), + "budget_failure_route": "L", + "budget_failure_l_run_id": _budget_failure_l_run_id(id_namespace), + "budget_failure_query_count": frame.query_count, + "budget_failure_max_query_attempts": frame.max_query_attempts, + "budget_failure_tool_calls": frame.tool_call_count, + "budget_failure_max_tool_calls": frame.max_tool_calls, + "budget_failure_read_doc_count": frame.read_doc_count, + "budget_failure_max_read_doc": frame.max_read_doc_calls, + "budget_failure_stage": "record_tool_use_budget_frame:validate", + } + + +def _budget_failure_type(frame: ToolUseBudgetFrame, *, reason: str) -> str: + if frame.query_count > frame.max_query_attempts: + return "query_count_exceeded_max_query_attempts" + if frame.tool_call_count > frame.max_tool_calls: + return "tool_call_count_exceeded_max_tool_calls" + if frame.read_doc_count > frame.max_read_doc_calls: + return "read_doc_count_exceeded_max_read_doc" + if frame.max_query_candidates != frame.max_query_attempts: + return "max_query_candidates_mismatch" + if "must be positive" in reason: + return "budget_limit_not_positive" + if "must not be negative" in reason: + return "budget_count_negative" + return "budget_consistency_violation" + + +def _budget_failure_l_run_id(id_namespace: LRunIds | None) -> str: + if id_namespace is None: + return "unknown" + return f"L:run:{id_namespace.run_index:04d}" + + def record_duplicate_tool_use_signal( *, trace_store: TraceStore, diff --git a/songryeon_core/tools/tool_result_distiller.py b/songryeon_core/tools/tool_result_distiller.py index 56feca2..4d2f3c1 100644 --- a/songryeon_core/tools/tool_result_distiller.py +++ b/songryeon_core/tools/tool_result_distiller.py @@ -87,12 +87,30 @@ def _build_distillation_frame( max_preview_chars=max_search_preview_chars, limits=limits, ) + elif tool_result.tool_name == "list_code_files": + items = _distill_list_code_files( + payload=tool_result.payload, + max_preview_chars=max_search_preview_chars, + limits=limits, + ) + elif tool_result.tool_name == "search_code": + items = _distill_search_code( + payload=tool_result.payload, + max_preview_chars=max_search_preview_chars, + limits=limits, + ) elif tool_result.tool_name in {"read_doc", "read_artifact"}: items = _distill_document_extract( payload=tool_result.payload, max_preview_chars=max_read_preview_chars, limits=limits, ) + elif tool_result.tool_name == "read_code_file": + items = _distill_read_code_file( + payload=tool_result.payload, + max_preview_chars=max_read_preview_chars, + limits=limits, + ) else: raise ValueError(f"unsupported distillation tool: {tool_result.tool_name}") @@ -202,6 +220,148 @@ def _distill_document_extract( ] +def _distill_list_code_files( + *, + payload: object, + max_preview_chars: int, + limits: list[str], +) -> list[ToolResultDistilledItem]: + if not isinstance(payload, dict): + limits.append("list_code_files payload가 dict가 아니어서 결과를 추출하지 못했다.") + return [] + + files = payload.get("files") + if not isinstance(files, list): + limits.append("list_code_files payload에 files 목록이 없어서 결과를 추출하지 못했다.") + return [] + + items: list[ToolResultDistilledItem] = [] + for index, raw_item in enumerate(files, start=1): + if not isinstance(raw_item, dict): + continue + file_path = str(raw_item.get("file_path") or "") + if not file_path: + continue + preview = _shorten( + ( + f"{file_path} " + f"extension={raw_item.get('extension')}; " + f"line_count={raw_item.get('line_count')}; " + f"size_bytes={raw_item.get('size_bytes')}" + ), + max_preview_chars, + ) + items.append( + ToolResultDistilledItem( + item_id=f"distilled_code_file_{index:04d}", + item_kind="search_result", + source_field_path=f"files[{index - 1}]", + doc_id=file_path, + chunk_id=f"{file_path}#file_listing", + result_id=f"code_file_{index:04d}", + score=1.0, + embedding_model_id="code-file-list", + text_preview=preview, + document_kind="source_code", + source_role="codebase_file_listing", + ) + ) + + if payload.get("truncated") is True: + limits.append("list_code_files 결과가 max_files 제한으로 잘렸다.") + if not items: + limits.append("list_code_files 결과 파일이 없어서 distillation item이 비어 있다.") + return items + + +def _distill_search_code( + *, + payload: object, + max_preview_chars: int, + limits: list[str], +) -> list[ToolResultDistilledItem]: + if not isinstance(payload, dict): + limits.append("search_code payload가 dict가 아니어서 결과를 추출하지 못했다.") + return [] + + results = payload.get("results") + if not isinstance(results, list): + limits.append("search_code payload에 results 목록이 없어서 결과를 추출하지 못했다.") + return [] + + items: list[ToolResultDistilledItem] = [] + for index, raw_item in enumerate(results, start=1): + if not isinstance(raw_item, dict): + continue + file_path = str(raw_item.get("file_path") or "") + if not file_path: + continue + line_number = raw_item.get("line_number") + line_text = str(raw_item.get("line_text") or "") + preview = _shorten( + f"{file_path}:{line_number}: {line_text}", + max_preview_chars, + ) + items.append( + ToolResultDistilledItem( + item_id=f"distilled_code_match_{index:04d}", + item_kind="search_result", + source_field_path=f"results[{index - 1}]", + doc_id=file_path, + chunk_id=f"{file_path}#L{line_number}", + result_id=str(raw_item.get("result_id") or f"code_match_{index:04d}"), + score=1.0, + embedding_model_id="code-substring-search", + text_preview=preview, + document_kind="source_code", + source_role="code_search_match", + ) + ) + + if payload.get("truncated") is True: + limits.append("search_code 결과가 max_results 제한으로 잘렸다.") + if not items: + limits.append("search_code 결과 후보가 없어서 distillation item이 비어 있다.") + return items + + +def _distill_read_code_file( + *, + payload: object, + max_preview_chars: int, + limits: list[str], +) -> list[ToolResultDistilledItem]: + if not isinstance(payload, dict): + limits.append("read_code_file payload가 dict가 아니어서 원문 발췌를 만들지 못했다.") + return [] + + file_path = str(payload.get("file_path") or "") + read_status = str(payload.get("read_status") or "") + text = str(payload.get("text") or "") + if read_status != "ok" or not file_path or not text: + limits.append(f"read_code_file has no readable source text: {read_status or 'unknown'}") + return [] + char_count = payload.get("char_count") + if not isinstance(char_count, int): + char_count = len(text) + preview = _shorten(" ".join(text.split()), max_preview_chars) + if payload.get("truncated") is True: + limits.append("read_code_file 원문이 max_chars 제한으로 잘렸다.") + + return [ + ToolResultDistilledItem( + item_id="distilled_read_code_file_0001", + item_kind="read_doc_excerpt", + source_field_path="text", + doc_id=file_path, + char_count=char_count, + text_preview=preview, + document_kind="source_code", + source_role="code_file_extract", + ) + ] + + def _distillation_data_id( *, tool_name: str, diff --git a/songryeon_core/tools/tool_runner.py b/songryeon_core/tools/tool_runner.py index 44cb01c..34b8bd6 100644 --- a/songryeon_core/tools/tool_runner.py +++ b/songryeon_core/tools/tool_runner.py @@ -15,6 +15,7 @@ ) from songryeon_core.core.trace_store import TraceStore from songryeon_core.loops.l_loop_namespace import LRunIds +from songryeon_core.tools.code_tools import list_code_files, read_code_file, search_code from songryeon_core.tools.document_tools import list_docs, read_artifact, read_doc, search_docs @@ -133,10 +134,15 @@ def run( ) -def build_document_tool_registry(document_root: str | Path) -> ToolRegistry: +def build_document_tool_registry( + document_root: str | Path, + *, + code_root: str | Path | None = None, +) -> ToolRegistry: """Administrative_Reform_1 같은 문서 루트에 묶인 읽기 전용 도구 레지스트리.""" root = Path(document_root) + codebase_root = Path.cwd() if code_root is None else Path(code_root) return ToolRegistry( [ ToolSpec( @@ -171,6 +177,38 @@ def build_document_tool_registry(document_root: str | Path) -> ToolRegistry: function=lambda query, top_k=5: search_docs(root=root, query=query, top_k=top_k), input_fields=["query", "top_k"], ), + ToolSpec( + name="list_code_files", + description="Workspace의 코드/설정 파일 목록을 읽기 전용으로 가져온다.", + read_only=True, + output_data_type="tool_result:list_code_files", + function=lambda max_files=500: list_code_files(root=codebase_root, max_files=max_files), + input_fields=["max_files"], + ), + ToolSpec( + name="search_code", + description="Workspace 코드/설정 파일에서 문자열을 읽기 전용으로 검색한다.", + read_only=True, + output_data_type="tool_result:search_code", + function=lambda query, max_results=50: search_code( + root=codebase_root, + query=query, + max_results=max_results, + ), + input_fields=["query", "max_results"], + ), + ToolSpec( + name="read_code_file", + description="Workspace 안의 코드/설정 파일 하나를 읽기 전용으로 가져온다.", + read_only=True, + output_data_type="tool_result:read_code_file", + function=lambda file_path, max_chars=12000: read_code_file( + root=codebase_root, + file_path=file_path, + max_chars=max_chars, + ), + input_fields=["file_path", "max_chars"], + ), ] ) diff --git a/tests/test_import_baseline.py b/tests/test_import_baseline.py index 4b23dfa..a7519ee 100644 --- a/tests/test_import_baseline.py +++ b/tests/test_import_baseline.py @@ -8,10 +8,24 @@ def test_core_import_baseline() -> None: "songryeon_core", "songryeon_core.core.schema_parts", "songryeon_core.core.schema_parts.base", + "songryeon_core.core.schema_parts.loop_activity", "songryeon_core.core.schema_parts.task_ledger", "songryeon_core.core.schema_parts.trace_data", "songryeon_core.core.schemas", + "songryeon_core.core.graph_memory_export", + "songryeon_core.core.graph_memory_integrity", + "songryeon_core.core.graph_source_ingest", + "songryeon_core.core.graph_vessel_neo4j", + "songryeon_core.core.graph_vessel_inspect", + "songryeon_core.core.graph_vessel_readback", + "songryeon_core.core.graph_vessel_adapter", + "songryeon_core.core.songryeon_source_manifest", "songryeon_core.core.trace_store", + "songryeon_core.core.turn_activity_graph_links", + "songryeon_core.runtime.fast_test", + "songryeon_core.runtime.graph_vessel_first_write", + "songryeon_core.runtime.graph_vessel_inspect", + "songryeon_core.runtime.graph_vessel_readback", "songryeon_core.runtime.dry_run", "songryeon_core.runtime.smoke_cases.document_memory", "songryeon_core.runtime.smoke_cases.router_fallback", diff --git a/tests/test_order_112_document_context_pack.py b/tests/test_order_112_document_context_pack.py new file mode 100644 index 0000000..ddbcb63 --- /dev/null +++ b/tests/test_order_112_document_context_pack.py @@ -0,0 +1,294 @@ +from __future__ import annotations + +from dataclasses import asdict + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.schemas import MetainfoBoundary, validate_document_context_pack_frame +from songryeon_core.core.trace_store import TraceStore +from songryeon_core.nodes.node_2_handoff import record_node3_input_brief +from songryeon_core.runtime.terminal_view import render_runtime_view +from songryeon_core.tools.document_context_pack import ( + DOCUMENT_CONTEXT_PACK_DATA_TYPE, + EXPLICIT_ARTIFACT_REFERENCE_DATA_TYPE, + build_document_context_pack_frame, + build_explicit_artifact_reference_frame, + extract_explicit_artifact_references, +) + + +def test_explicit_order_references_are_extracted_from_user_input() -> None: + text = ( + "ORDER_100, ORDER_105_MEMORY_SELECTION_TO_NODE2_HANDOFF_V0, " + "04_Orders/ORDER_105_MEMORY_SELECTION_TO_NODE2_HANDOFF_V0.md" + ) + + assert extract_explicit_artifact_references(text) == [ + "ORDER_100", + "ORDER_105_MEMORY_SELECTION_TO_NODE2_HANDOFF_V0", + "04_Orders/ORDER_105_MEMORY_SELECTION_TO_NODE2_HANDOFF_V0.md", + ] + + +def test_unique_explicit_order_is_ranked_before_embedding_candidate() -> None: + frame = build_explicit_artifact_reference_frame( + turn_id="turn_test", + user_text="ORDER_100을 직접 읽어줘", + document_root="Administrative_Reform_1", + frame_id="explicit:test", + source_trace_ids=["trace_user"], + source_data_ids=["data_user"], + ) + assert frame.resolved_references[0].resolve_status == "unique" + assert frame.resolved_references[0].selected_doc_id == ( + "04_Orders/ORDER_100_RECENT_TURN_CAPSULE_READ_WINDOW_PACKET_V0.md" + ) + + data_store = DataStore() + data_store.create_record( + data_id=frame.frame_id, + data_type=EXPLICIT_ARTIFACT_REFERENCE_DATA_TYPE, + payload=asdict(frame), + ) + data_store.create_record( + data_id="L3:preserved_info_frame", + data_type="node_output:L3_preserved_info_frame", + payload={ + "frame_id": "L3:preserved_info_frame", + "candidates": [ + { + "doc_id": "README.md", + "source_data_id": "tool_result:search_docs:test", + } + ], + }, + ) + + pack = build_document_context_pack_frame( + data_store=data_store, + turn_id="turn_test", + document_root="Administrative_Reform_1", + max_document_context_chars=100000, + frame_id="pack:test", + explicit_reference_data_id=frame.frame_id, + source_trace_ids=["trace_pack"], + source_data_ids=[frame.frame_id], + id_namespace=None, + ) + + assert pack.included_documents[0].doc_id == ( + "04_Orders/ORDER_100_RECENT_TURN_CAPSULE_READ_WINDOW_PACKET_V0.md" + ) + assert pack.included_documents[0].selection_basis.startswith( + "explicit_artifact_reference_unique" + ) + assert pack.included_documents[1].doc_id == "README.md" + + +def test_ambiguous_explicit_order_reference_is_not_selected(tmp_path) -> None: + root = tmp_path / "docs" + orders = root / "04_Orders" + orders.mkdir(parents=True) + (orders / "ORDER_100_ALPHA.md").write_text("alpha", encoding="utf-8") + (orders / "ORDER_100_BETA.md").write_text("beta", encoding="utf-8") + + frame = build_explicit_artifact_reference_frame( + turn_id="turn_test", + user_text="ORDER_100을 찾아줘", + document_root=root, + frame_id="explicit:test", + source_trace_ids=["trace_user"], + source_data_ids=["data_user"], + ) + + assert frame.resolved_references[0].resolve_status == "ambiguous" + assert frame.resolved_references[0].selected_doc_id is None + assert frame.resolved_references[0].candidate_count == 2 + + +def test_whole_document_pack_excludes_without_mid_document_cut(tmp_path) -> None: + root = tmp_path / "docs" + orders = root / "04_Orders" + orders.mkdir(parents=True) + (orders / "ORDER_100_ALPHA.md").write_text("11111", encoding="utf-8") + (orders / "ORDER_101_BETA.md").write_text("2222222", encoding="utf-8") + (orders / "ORDER_104_GAMMA.md").write_text("3333", encoding="utf-8") + + data_store = DataStore() + data_store.create_record( + data_id="explicit:test", + data_type=EXPLICIT_ARTIFACT_REFERENCE_DATA_TYPE, + payload={ + "resolved_references": [], + }, + ) + data_store.create_record( + data_id="L3:preserved_info_frame", + data_type="node_output:L3_preserved_info_frame", + payload={ + "candidates": [ + {"doc_id": "04_Orders/ORDER_100_ALPHA.md", "source_data_id": "search:1"}, + {"doc_id": "04_Orders/ORDER_101_BETA.md", "source_data_id": "search:1"}, + {"doc_id": "04_Orders/ORDER_104_GAMMA.md", "source_data_id": "search:1"}, + ], + }, + ) + + pack = build_document_context_pack_frame( + data_store=data_store, + turn_id="turn_test", + document_root=root, + max_document_context_chars=10, + frame_id="pack:test", + explicit_reference_data_id="explicit:test", + source_trace_ids=["trace_pack"], + source_data_ids=["explicit:test"], + id_namespace=None, + ) + validate_document_context_pack_frame(pack) + + assert [item.doc_id for item in pack.included_documents] == [ + "04_Orders/ORDER_100_ALPHA.md" + ] + assert pack.included_documents[0].text == "11111" + assert pack.excluded_documents[0].doc_id == "04_Orders/ORDER_101_BETA.md" + assert pack.excluded_documents[0].exclusion_reason == "excluded_due_to_context_budget" + assert pack.excluded_documents[1].doc_id == "04_Orders/ORDER_104_GAMMA.md" + assert pack.excluded_documents[1].exclusion_reason == "excluded_after_strict_rank_cutoff" + + +def test_node3_brief_reads_only_context_pack_included_documents(tmp_path) -> None: + root = tmp_path / "docs" + orders = root / "04_Orders" + orders.mkdir(parents=True) + (orders / "ORDER_100_ALPHA.md").write_text("11111", encoding="utf-8") + (orders / "ORDER_101_BETA.md").write_text("2222222", encoding="utf-8") + + data_store = DataStore() + data_store.create_record( + data_id="explicit:test", + data_type=EXPLICIT_ARTIFACT_REFERENCE_DATA_TYPE, + payload={"resolved_references": []}, + ) + data_store.create_record( + data_id="L3:preserved_info_frame", + data_type="node_output:L3_preserved_info_frame", + payload={ + "candidates": [ + {"doc_id": "04_Orders/ORDER_100_ALPHA.md", "source_data_id": "search:1"}, + {"doc_id": "04_Orders/ORDER_101_BETA.md", "source_data_id": "search:1"}, + ], + }, + ) + pack = build_document_context_pack_frame( + data_store=data_store, + turn_id="turn_test", + document_root=root, + max_document_context_chars=5, + frame_id="pack:test", + explicit_reference_data_id="explicit:test", + source_trace_ids=["trace_pack"], + source_data_ids=["explicit:test"], + id_namespace=None, + ) + data_store.create_record( + data_id=pack.frame_id, + data_type=DOCUMENT_CONTEXT_PACK_DATA_TYPE, + payload=asdict(pack), + ) + trace_store = TraceStore() + seed = trace_store.create_event( + turn_id="turn_test", + actor="test", + event_type="node_output", + output_ref=["pack:test"], + schema_status="passed", + ) + + _, _, brief = record_node3_input_brief( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_test", + user_question="ORDER_100과 ORDER_101을 읽어줘", + handoff_frame_id="node_2:handoff_frame", + boundary=MetainfoBoundary(), + input_trace_ids=[seed.event_id], + source_data_ids=["node_2:handoff_frame", "pack:test"], + ) + + assert len(brief.read_documents) == pack.included_document_count == 1 + assert brief.read_documents[0].document_name == "ORDER_100_ALPHA.md" + assert len(brief.excluded_document_contexts) == pack.excluded_document_count == 1 + assert brief.excluded_document_contexts[0].document_name == "ORDER_101_BETA.md" + + +def test_terminal_view_displays_explicit_resolve_and_context_pack() -> None: + result = { + "status": "ok", + "runtime": {"model_id": "test", "transport": "fake"}, + "trace_count": 0, + "data_record_count": 2, + "data_records": [ + { + "data_id": "explicit:test", + "data_type": EXPLICIT_ARTIFACT_REFERENCE_DATA_TYPE, + "payload": { + "frame_id": "explicit:test", + "extracted_reference_count": 1, + "resolved_references": [ + { + "raw_ref": "ORDER_100", + "resolve_status": "unique", + "selected_doc_id": "04_Orders/ORDER_100_ALPHA.md", + } + ], + "generated_by": "CODE:EXPLICIT_ARTIFACT_RESOLVER", + "info_class": "absolute_resolve_result", + "semantic_judgement_status": "not_run", + "source_data_ids": ["data_user"], + }, + }, + { + "data_id": "pack:test", + "data_type": DOCUMENT_CONTEXT_PACK_DATA_TYPE, + "payload": { + "frame_id": "pack:test", + "included_document_count": 1, + "excluded_document_count": 1, + "included_total_chars": 5, + "max_document_context_chars": 10, + "budget_unit": "chars", + "whole_document_only": True, + "strict_rank_order": True, + "cutoff_reason": "excluded_due_to_context_budget at ORDER_101", + "included_documents": [ + { + "rank_index": 1, + "doc_id": "04_Orders/ORDER_100_ALPHA.md", + "char_count": 5, + "selection_basis": "explicit_artifact_reference_unique:ORDER_100", + } + ], + "excluded_documents": [ + { + "rank_index": 2, + "doc_id": "04_Orders/ORDER_101_BETA.md", + "char_count": 7, + "exclusion_reason": "excluded_due_to_context_budget", + } + ], + "generated_by": "CODE:DOCUMENT_CONTEXT_PACKER", + "info_class": "absolute_context_packing_result", + "semantic_judgement_status": "not_run", + "source_data_ids": ["explicit:test"], + }, + }, + ], + } + + rendered = render_runtime_view(result, user_input="ORDER_100") + + assert "explicit_artifact_refs" in rendered + assert "ORDER_100 -> unique" in rendered + assert "document_context_pack" in rendered + assert "included=1 / excluded=1 / budget=5/10 chars" in rendered + assert "whole_document_only=true / strict_rank_order=true" in rendered diff --git a/tests/test_order_118_answer_basis.py b/tests/test_order_118_answer_basis.py new file mode 100644 index 0000000..2e78b49 --- /dev/null +++ b/tests/test_order_118_answer_basis.py @@ -0,0 +1,222 @@ +from __future__ import annotations + +import json +from dataclasses import asdict + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.schemas import MetainfoBoundary +from songryeon_core.core.trace_store import TraceStore +from songryeon_core.llm.base import LLMRequest, LLMResponse +from songryeon_core.llm.fake import BrokenJSONFakeLLMAdapter +from songryeon_core.nodes.node_2_handoff import ( + node3_brief_llm_payload, + record_node3_input_brief, +) +from songryeon_core.nodes.node_2_metainfo_boundary import ( + run_node2_answer_basis_selection, +) +from songryeon_core.runtime.dry_run import run_dry_turn +from songryeon_core.runtime.terminal_view import render_pretty_turn + + +class AnswerBasisPayloadFakeAdapter: + model_id = "answer-basis-payload-fake-adapter" + + def __init__(self, payload: dict[str, object]) -> None: + self.payload = payload + + def complete(self, request: LLMRequest) -> LLMResponse: + _ = request + return LLMResponse( + text=json.dumps(self.payload, ensure_ascii=False), + model_id=self.model_id, + raw=self.payload, + ) + + +def test_node2_answer_basis_absolute_first_fixture() -> None: + frame = _run_answer_basis_fixture( + { + "answer_basis_mode": "absolute_first", + "basis_reason_codes": [ + "code_verified_fact_required", + "runtime_state_basis_present", + ], + "mode_selection_reason": "runtime count 확인 요청이므로 코드/trace 값 중심으로 답해야 한다.", + "mode_selection_reason_info_class": "mixed", + "evidence_roles": [_role("source:runtime", "primary_answer_basis")], + }, + user_question="몇 개 읽었어?", + ) + + assert frame.answer_basis_mode == "absolute_first" + assert frame.generated_by == "LLM:answer-basis-payload-fake-adapter" + assert frame.info_class == "mixed" + assert frame.mode_selection_reason + + +def test_node2_answer_basis_relative_allowed_fixture() -> None: + frame = _run_answer_basis_fixture( + { + "answer_basis_mode": "relative_allowed", + "basis_reason_codes": ["user_asked_for_interpretation"], + "mode_selection_reason": "사용자가 구조 의견을 요청했으므로 해석과 조언이 허용된다.", + "mode_selection_reason_info_class": "mixed", + "evidence_roles": [_role("source:runtime", "supporting_context")], + }, + user_question="이 구조 어때?", + ) + + assert frame.answer_basis_mode == "relative_allowed" + assert frame.basis_reason_codes == ["user_asked_for_interpretation"] + assert frame.semantic_judgement_status == "ran" + + +def test_node2_answer_basis_mixed_or_uncertain_fixture() -> None: + frame = _run_answer_basis_fixture( + { + "answer_basis_mode": "mixed_or_uncertain", + "basis_reason_codes": ["multi_source_bundle", "partial_evidence_only"], + "mode_selection_reason": "최근 대화와 실행 기록 source bundle을 함께 봐야 하므로 한계를 표시해야 한다.", + "mode_selection_reason_info_class": "mixed", + "evidence_roles": [ + _role("source:runtime", "supporting_context"), + _role("source:memory", "primary_answer_basis"), + ], + }, + user_question="오늘 전체 흐름 정리해줘.", + ) + + assert frame.answer_basis_mode == "mixed_or_uncertain" + assert "multi_source_bundle" in frame.basis_reason_codes + assert "partial_evidence_only" in frame.basis_reason_codes + assert len(frame.evidence_roles) == 2 + + +def test_node2_answer_basis_llm_failure_uses_code_fallback() -> None: + trace_store, data_store, input_trace_id = _stores() + + _, frame_id, frame = run_node2_answer_basis_selection( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_118", + user_question="몇 개 읽었어?", + boundary_id="source:boundary", + boundary=MetainfoBoundary(), + handoff_frame_id="source:handoff", + adapter=BrokenJSONFakeLLMAdapter(), + input_ref=[input_trace_id], + source_data_ids=["source:runtime", "source:memory"], + ) + + assert frame_id == "node_2:answer_basis_frame" + assert frame.answer_basis_mode == "mixed_or_uncertain" + assert frame.basis_reason_codes == ["llm_mode_selection_failed"] + assert frame.generated_by == "CODE:FALLBACK" + assert frame.info_class == "absolute_status" + assert frame.semantic_judgement_status == "failed" + + +def test_node3_brief_receives_answer_basis_without_raw_id_leak_in_llm_payload() -> None: + answer_basis_frame = _run_answer_basis_fixture( + { + "answer_basis_mode": "absolute_first", + "basis_reason_codes": ["code_verified_fact_required"], + "mode_selection_reason": "확인 가능한 실행 값 중심으로 답해야 한다.", + "mode_selection_reason_info_class": "mixed", + "evidence_roles": [_role("source:runtime", "primary_answer_basis")], + }, + user_question="route가 뭐야?", + ) + trace_store, data_store, input_trace_id = _stores() + data_store.create_record( + data_id=answer_basis_frame.frame_id, + data_type="node_output:node2_answer_basis_frame", + payload=asdict(answer_basis_frame), + ) + + _, _, brief = record_node3_input_brief( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_118", + user_question="route가 뭐야?", + handoff_frame_id="source:handoff", + boundary=MetainfoBoundary(), + input_trace_ids=[input_trace_id], + source_data_ids=[ + "source:handoff", + "source:runtime", + answer_basis_frame.frame_id, + ], + answer_basis_frame=answer_basis_frame, + ) + payload = node3_brief_llm_payload(brief) + answer_basis_payload = payload["answer_basis"] + + assert brief.answer_basis_mode == "absolute_first" + assert brief.answer_basis_frame_id == "node_2:answer_basis_frame" + assert isinstance(answer_basis_payload, dict) + assert answer_basis_payload["answer_basis_mode"] == "absolute_first" + assert "source:runtime" not in json.dumps(answer_basis_payload, ensure_ascii=False) + assert answer_basis_payload["evidence_roles"][0]["source_label"] == "공급된 근거 자료" + + +def test_runtime_view_displays_answer_basis_fallback() -> None: + result = run_dry_turn(user_input="몇 개 읽었어?") + rendered = render_pretty_turn(result, user_input="몇 개 읽었어?") + + assert result["node2_answer_basis_mode"] == "mixed_or_uncertain" + assert result["node2_answer_basis_reason_codes"] == ["llm_mode_selection_failed"] + assert "node_2 answer basis" in rendered + assert "reason_codes" in rendered + assert "CODE:FALLBACK" in rendered + + +def _run_answer_basis_fixture( + payload: dict[str, object], + *, + user_question: str, +): + trace_store, data_store, input_trace_id = _stores() + _, _, frame = run_node2_answer_basis_selection( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_118", + user_question=user_question, + boundary_id="source:boundary", + boundary=MetainfoBoundary(), + handoff_frame_id="source:handoff", + adapter=AnswerBasisPayloadFakeAdapter(payload), + input_ref=[input_trace_id], + source_data_ids=["source:runtime", "source:memory"], + ) + return frame + + +def _stores() -> tuple[TraceStore, DataStore, str]: + trace_store = TraceStore() + data_store = DataStore() + seed_event = trace_store.create_event( + turn_id="turn_order_118", + actor="test", + event_type="node_output", + output_ref=["source:runtime", "source:memory", "source:boundary", "source:handoff"], + schema_status="passed", + ) + for data_id in ("source:runtime", "source:memory", "source:boundary", "source:handoff"): + data_store.create_record( + data_id=data_id, + data_type="test:source", + source_trace_id=seed_event.event_id, + payload={"data_id": data_id}, + ) + return trace_store, data_store, seed_event.event_id + + +def _role(source_data_id: str, evidence_role: str) -> dict[str, object]: + return { + "source_data_id": source_data_id, + "evidence_role": evidence_role, + "role_reason": "테스트 fixture가 지정한 역할이다.", + "role_reason_info_class": "mixed", + } diff --git a/tests/test_order_119_structure_failed_honesty.py b/tests/test_order_119_structure_failed_honesty.py new file mode 100644 index 0000000..459f01f --- /dev/null +++ b/tests/test_order_119_structure_failed_honesty.py @@ -0,0 +1,198 @@ +from __future__ import annotations + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.schemas import ( + MetainfoBoundary, + Node3InputBriefFrame, + Node3SelectedRecentMemoryContext, +) +from songryeon_core.core.trace_store import TraceStore +from songryeon_core.llm.fake import BrokenJSONFakeLLMAdapter +from songryeon_core.nodes.node_2_handoff import node3_brief_llm_payload +from songryeon_core.nodes.node_2_metainfo_boundary import run_node2_answer_basis_selection +from songryeon_core.nodes.node_3_reporter import assemble_node3_report_markdown +from songryeon_core.runtime.terminal_view import render_chat_answer, render_pretty_turn + + +def test_structure_failed_fallback_no_fake_search() -> None: + result = { + "status": "structure_failed", + "trace_count": 0, + "data_record_count": 0, + "structure_failure_stage": "node_3:run", + "structure_failure_exception_type": "ValueError", + "structure_failure_reason": "node_3 LLM reporter failed: parse_failed", + } + + rendered = render_chat_answer(result, user_input="이 구조 어때?") + + assert "node_3 최종 보고도 실행되지 않았고" in rendered + assert "의미 답변은 만들지 않고" in rendered + assert "확인 가능한 search_docs/read_doc payload도 없어" in rendered + assert "내부 문서를 찾았어" not in rendered + assert "검색 결과 payload를 찾지 못해서" not in rendered + + +def test_structure_failed_mentions_actual_search_payload_only_when_present() -> None: + result = { + "status": "structure_failed", + "trace_count": 3, + "data_record_count": 2, + "data_records": [ + _record( + data_id="tool:search:001", + data_type="tool_result:search_docs", + payload={ + "result_count": 1, + "results": [{"doc_id": "Administrative_Reform_1/04_Orders/ORDER_119.md"}], + }, + ), + _record( + data_id="tool:read:001", + data_type="tool_result:read_doc", + payload={ + "doc_id": "Administrative_Reform_1/04_Orders/ORDER_119.md", + "char_count": 119, + "text": "# ORDER 119", + }, + ), + ], + } + + rendered = render_chat_answer(result, user_input="방금 읽은 거야?") + + assert "DataStore에 검색/문서 payload가 남아 있어" in rendered + assert "search_docs payload: 후보 1개" in rendered + assert "read_doc payload: `ORDER_119.md`" in rendered + assert "내부 문서를 찾았어" not in rendered + assert "검색 결과 payload를 찾지 못해서" not in rendered + + +def test_answer_basis_failure_diagnostics_are_recorded_and_rendered() -> None: + trace_store, data_store, input_trace_id = _stores() + + _, _, frame = run_node2_answer_basis_selection( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_119", + user_question="몇 개 읽었어?", + boundary_id="source:boundary", + boundary=MetainfoBoundary(), + handoff_frame_id="source:handoff", + adapter=BrokenJSONFakeLLMAdapter(), + input_ref=[input_trace_id], + source_data_ids=["source:runtime", "source:memory"], + ) + result = { + "status": "ok", + "trace_count": len(trace_store.list_events()), + "data_record_count": len(data_store.list_records()), + "data_records": data_store.to_records(), + } + rendered = render_pretty_turn(result, user_input="몇 개 읽었어?") + + assert frame.answer_basis_mode == "mixed_or_uncertain" + assert frame.basis_reason_codes == ["llm_mode_selection_failed"] + assert frame.generated_by == "CODE:FALLBACK" + assert frame.answer_basis_failure_type == "parse_failed" + assert frame.answer_basis_llm_call_data_id + assert frame.answer_basis_trace_event_id + assert frame.answer_basis_payload_parse_status == "failed" + assert frame.answer_basis_raw_text_present is True + assert frame.answer_basis_prompt_ref.endswith("node_2_answer_basis_selector_v0.md") + assert "failure_type: parse_failed" in rendered + assert "payload_parse_status: failed" in rendered + assert "llm_call: llm_call:node_2:" in rendered + + +def test_selected_recent_memory_payload_boundary_is_not_document() -> None: + brief = Node3InputBriefFrame( + frame_id="node_3:input_brief_frame", + turn_id="turn_order_119", + user_question="그건 실행기록 문서를 읽은 거야?", + brief_status="ready", + handoff_frame_id="source:handoff", + selected_recent_memory_contexts=[ + Node3SelectedRecentMemoryContext( + source_turn_id="turn_previous", + raw_user_text="아까 실행기록 문서를 봤냐고 물었어", + raw_assistant_text="그때는 최근 대화 원문만 봤다고 답했어", + raw_user_text_chars=len("아까 실행기록 문서를 봤냐고 물었어"), + raw_assistant_text_chars=len("그때는 최근 대화 원문만 봤다고 답했어"), + raw_user_text_truncated=False, + raw_assistant_text_truncated=False, + selection_status="selected", + selection_info_class="mixed", + selection_reason="테스트 fixture가 선택한 최근 대화다.", + selection_reason_generated_by="LLM:test", + copied_from="recent_raw_conversation", + ) + ], + reporting_rules=[ + "선택된 최근 기억 context는 과거 대화 원문을 code가 복사한 context이며, 실행기록 문서나 새로 읽은 문서가 아니다.", + ], + source_data_ids=["source:handoff"], + ) + + payload = node3_brief_llm_payload(brief) + + assert payload["available_document_extract_count"] == 0 + assert "not a read document" in payload["selected_recent_memory_source_boundary"] + assert "not an execution-record document" in payload["selected_recent_memory_source_boundary"] + assert any("실행기록 문서나 새로 읽은 문서가 아니다" in rule for rule in brief.reporting_rules) + + +def test_bold_duplicate_grounding_block_is_stripped() -> None: + brief = Node3InputBriefFrame( + frame_id="node_3:input_brief_frame", + turn_id="turn_order_119", + user_question="요약해줘", + brief_status="ready", + handoff_frame_id="source:handoff", + source_data_ids=["source:handoff"], + ) + + rendered = assemble_node3_report_markdown( + brief_frame=brief, + body_markdown=( + "**근거 기준:**\n" + "- 실제 read_doc 도구 원문 읽기: 99개\n\n" + "본문만 남아야 한다." + ), + ) + + assert rendered.startswith("근거 기준:") + assert rendered.count("근거 기준:") == 1 + assert "99개" not in rendered + assert "본문만 남아야 한다." in rendered + + +def _record(*, data_id: str, data_type: str, payload: dict[str, object]) -> dict[str, object]: + return { + "data_id": data_id, + "data_type": data_type, + "exists": True, + "created_at": None, + "source_trace_id": None, + "payload": payload, + } + + +def _stores() -> tuple[TraceStore, DataStore, str]: + trace_store = TraceStore() + data_store = DataStore() + seed_event = trace_store.create_event( + turn_id="turn_order_119", + actor="test", + event_type="node_output", + output_ref=["source:runtime", "source:memory", "source:boundary", "source:handoff"], + schema_status="passed", + ) + for data_id in ("source:runtime", "source:memory", "source:boundary", "source:handoff"): + data_store.create_record( + data_id=data_id, + data_type="test:source", + source_trace_id=seed_event.event_id, + payload={"data_id": data_id}, + ) + return trace_store, data_store, seed_event.event_id diff --git a/tests/test_order_120_tool_budget_diagnostics.py b/tests/test_order_120_tool_budget_diagnostics.py new file mode 100644 index 0000000..84a6ac6 --- /dev/null +++ b/tests/test_order_120_tool_budget_diagnostics.py @@ -0,0 +1,286 @@ +from __future__ import annotations + +from dataclasses import asdict + +import pytest + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.schemas import ToolUseBudgetFrame, validate_tool_use_budget_frame +from songryeon_core.core.trace_store import TraceStore +from songryeon_core.loops.l_loop_continuation import record_l_loop_continuation_decision +from songryeon_core.runtime.dry_run import run_dry_turn +from songryeon_core.runtime.terminal_view import render_pretty_turn +from songryeon_core.runtime.user_turn import _structure_failure_diagnostics +from songryeon_core.tools.tool_efficiency_policy import ( + BudgetConsistencyError, + record_tool_use_budget_frame, +) + + +LIVE_LIKE_INPUT = ( + "지금 송련의 말하기 모드 설계가 너무 빡빡한지, 아니면 적절한지 의견을 말해줘. " + "단, 이건 네 해석이라는 점을 밝혀줘." +) + + +def test_tool_use_budget_validator_keeps_query_count_limit() -> None: + frame = _budget_frame( + budget_id="tool_budget:test:invalid", + max_query_attempts=1, + query_count=2, + executed_queries=["first", "second"], + ) + + with pytest.raises(ValueError, match="query_count must not exceed max_query_attempts"): + validate_tool_use_budget_frame(frame) + + +def test_valid_tool_use_budget_frame_records_successfully() -> None: + trace_store = TraceStore() + data_store = DataStore() + + event_id, budget_id = record_tool_use_budget_frame( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_120", + sequence_index=1, + max_tool_calls=3, + search_top_k=2, + max_query_attempts=2, + max_read_doc_calls=1, + max_input_chars=1000, + tool_call_count=1, + executed_queries=["alpha"], + read_doc_ids=[], + cache_statuses=[], + input_chars_used=10, + stop_reason="within_budget", + reason="CODE_STATUS:test_valid_budget", + source_trace_ids=["trace:test"], + source_data_ids=["source:test"], + ) + + record = data_store.require_record(budget_id) + payload = record.payload + assert event_id + assert isinstance(payload, dict) + assert payload["query_count"] == 1 + assert payload["max_query_attempts"] == 2 + assert payload["tool_call_count"] == 1 + assert payload["max_tool_calls"] == 3 + assert payload["read_doc_count"] == 0 + assert payload["max_read_doc_calls"] == 1 + + +def test_budget_failure_diagnostics_expose_query_count_mismatch() -> None: + trace_store = TraceStore() + data_store = DataStore() + + with pytest.raises(BudgetConsistencyError) as exc_info: + record_tool_use_budget_frame( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_120", + sequence_index=2, + max_tool_calls=3, + search_top_k=2, + max_query_attempts=1, + max_read_doc_calls=1, + max_input_chars=1000, + tool_call_count=1, + executed_queries=["alpha", "beta"], + read_doc_ids=[], + cache_statuses=[], + input_chars_used=20, + stop_reason="within_budget", + reason="CODE_STATUS:test_invalid_budget", + source_trace_ids=["trace:test"], + source_data_ids=["route:L", "L:budget_plan_frame"], + ) + + diagnostics = exc_info.value.budget_diagnostics + assert diagnostics["budget_failure_type"] == "query_count_exceeded_max_query_attempts" + assert diagnostics["budget_failure_frame_id"] == "tool_budget:turn_order_120:0002" + assert diagnostics["budget_failure_query_count"] == 2 + assert diagnostics["budget_failure_max_query_attempts"] == 1 + assert diagnostics["budget_failure_tool_calls"] == 1 + assert diagnostics["budget_failure_max_tool_calls"] == 3 + assert diagnostics["budget_failure_read_doc_count"] == 0 + assert diagnostics["budget_failure_max_read_doc"] == 1 + assert diagnostics["budget_failure_stage"] == "record_tool_use_budget_frame:validate" + + +def test_structure_failed_renderer_includes_budget_failure_diagnostics() -> None: + diagnostics = _structure_failure_diagnostics( + BudgetConsistencyError( + "ToolUseBudgetFrame.query_count must not exceed max_query_attempts", + diagnostics={ + "budget_failure_type": "query_count_exceeded_max_query_attempts", + "budget_failure_reason": ( + "ToolUseBudgetFrame.query_count must not exceed max_query_attempts" + ), + "budget_failure_frame_id": "tool_budget:turn_order_120:0002", + "budget_failure_source_data_ids": ["route:L", "L:budget_plan_frame"], + "budget_failure_route": "L", + "budget_failure_l_run_id": "L:run:0001", + "budget_failure_query_count": 2, + "budget_failure_max_query_attempts": 1, + "budget_failure_tool_calls": 1, + "budget_failure_max_tool_calls": 3, + "budget_failure_read_doc_count": 0, + "budget_failure_max_read_doc": 1, + "budget_failure_stage": "record_tool_use_budget_frame:validate", + }, + ) + ) + result = { + "status": "structure_failed", + "trace_count": 0, + "data_record_count": 0, + **diagnostics, + } + + rendered = render_pretty_turn(result, user_input=LIVE_LIKE_INPUT) + + assert "budget_failure_type: query_count_exceeded_max_query_attempts" in rendered + assert "budget_failure_query_count: 2" in rendered + assert "budget_failure_max_query_attempts: 1" in rendered + assert "예산 진단: type=query_count_exceeded_max_query_attempts" in rendered + assert "query=2/1" in rendered + + +def test_continuation_continues_when_query_budget_exhausted_but_unread_read_path_exists() -> None: + trace_store = TraceStore() + data_store = DataStore() + data_store.create_record( + data_id="L3:achievement_frame", + data_type="node_output:L3_achievement_frame", + payload={ + "achievement_status": "partial", + "goal_match_status": "partial", + "semantic_goal_match_status": "partial", + "read_doc_ids": [], + "search_result_doc_ids": ["doc:unread"], + }, + ) + data_store.create_record( + data_id="L2:query_frame", + data_type="node_output:L2_query_frame", + payload={"query_text": "previous query"}, + ) + exhausted_budget = _budget_frame( + budget_id="tool_budget:turn_order_120:0008", + max_query_attempts=8, + query_count=8, + executed_queries=[f"query {index}" for index in range(8)], + ) + data_store.create_record( + data_id=exhausted_budget.budget_id, + data_type="tool_use_budget", + payload=asdict(exhausted_budget), + ) + + _, _, frame = record_l_loop_continuation_decision( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_120", + attempt_index=8, + max_attempts=12, + l3_achievement_data_id="L3:achievement_frame", + l2_query_frame_data_id="L2:query_frame", + ) + + assert frame.continuation_status == "continue" + assert ( + frame.continuation_reason_code + == "CODE_STATUS:l3_not_achieved_read_unread_candidate_after_query_budget" + ) + assert frame.next_target_node == "L2" + + +def test_continuation_stops_when_query_budget_exhausted_and_no_unread_read_path() -> None: + trace_store = TraceStore() + data_store = DataStore() + data_store.create_record( + data_id="L3:achievement_frame", + data_type="node_output:L3_achievement_frame", + payload={ + "achievement_status": "partial", + "goal_match_status": "partial", + "semantic_goal_match_status": "partial", + "read_doc_ids": [], + "search_result_doc_ids": [], + }, + ) + data_store.create_record( + data_id="L2:query_frame", + data_type="node_output:L2_query_frame", + payload={"query_text": "previous query"}, + ) + exhausted_budget = _budget_frame( + budget_id="tool_budget:turn_order_120:0008", + max_query_attempts=8, + query_count=8, + executed_queries=[f"query {index}" for index in range(8)], + ) + data_store.create_record( + data_id=exhausted_budget.budget_id, + data_type="tool_use_budget", + payload=asdict(exhausted_budget), + ) + + _, _, frame = record_l_loop_continuation_decision( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_120", + attempt_index=8, + max_attempts=12, + l3_achievement_data_id="L3:achievement_frame", + l2_query_frame_data_id="L2:query_frame", + ) + + assert frame.continuation_status == "stop_budget_exhausted" + assert frame.continuation_reason_code == "CODE_STATUS:continuation_query_budget_exhausted" + assert frame.next_target_node == "loop_return_summary" + + +def test_live_like_opinion_input_no_budget_query_count_structure_failure() -> None: + result = run_dry_turn(user_input=LIVE_LIKE_INPUT) + + assert result["current_route"] == "2" + for record in result["data_records"]: + if record.get("data_type") != "tool_use_budget": + continue + payload = record.get("payload") + assert isinstance(payload, dict) + assert payload["query_count"] <= payload["max_query_attempts"] + + +def _budget_frame( + *, + budget_id: str, + max_query_attempts: int, + query_count: int, + executed_queries: list[str], +) -> ToolUseBudgetFrame: + return ToolUseBudgetFrame( + budget_id=budget_id, + turn_id="turn_order_120", + loop_id="L", + sequence_index=1, + max_tool_calls=3, + search_top_k=2, + max_query_attempts=max_query_attempts, + max_query_candidates=max_query_attempts, + max_read_doc_calls=1, + max_input_chars=1000, + tool_call_count=1, + query_count=query_count, + read_doc_count=0, + input_chars_used=10, + executed_queries=executed_queries, + read_doc_ids=[], + cache_statuses=[], + stop_reason="within_budget", + reason="CODE_STATUS:test_budget_frame", + ) diff --git a/tests/test_order_121_answer_basis_and_l3_attitude.py b/tests/test_order_121_answer_basis_and_l3_attitude.py new file mode 100644 index 0000000..e0fd1f7 --- /dev/null +++ b/tests/test_order_121_answer_basis_and_l3_attitude.py @@ -0,0 +1,219 @@ +from __future__ import annotations + +import json + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.schemas import DataRef, MetainfoBoundary, Node3InputBriefFrame +from songryeon_core.core.trace_store import TraceStore +from songryeon_core.llm.base import LLMRequest, LLMResponse +from songryeon_core.llm.fake import SongRyeonAllNodesFakeLLMAdapter +from songryeon_core.nodes.node_2_handoff import ( + node3_brief_llm_payload, + record_node3_input_brief, +) +from songryeon_core.nodes.node_2_metainfo_boundary import run_node2_answer_basis_selection +from songryeon_core.nodes.node_3_reporter import build_node3_grounding_block +from songryeon_core.nodes.node_4_gatekeeper import run_node4_gatekeeper + + +class AnswerBasisPayloadFakeAdapter: + model_id = "order-121-answer-basis-fake" + + def __init__(self, payload: dict[str, object]) -> None: + self.payload = payload + + def complete(self, request: LLMRequest) -> LLMResponse: + return LLMResponse( + text=json.dumps(self.payload, ensure_ascii=False), + model_id=self.model_id, + raw=self.payload, + ) + + +def test_node2_answer_basis_accepts_available_evidence_source_from_boundary_sample() -> None: + trace_store, data_store, trace_id = _stores() + boundary = MetainfoBoundary( + absolute_info=[ + DataRef( + data_id="source:sample_boundary_record", + data_type="test:boundary_sample", + source_trace_id=trace_id, + ) + ] + ) + + _, _, frame = run_node2_answer_basis_selection( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_121", + user_question="이 구조 어때?", + boundary_id="source:boundary", + boundary=boundary, + handoff_frame_id="source:handoff", + adapter=AnswerBasisPayloadFakeAdapter( + { + "answer_basis_mode": "relative_allowed", + "basis_reason_codes": ["user_asked_for_interpretation"], + "mode_selection_reason": "boundary sample을 근거로 해석 답변이 가능하다.", + "mode_selection_reason_info_class": "mixed", + "evidence_roles": [ + { + "source_data_id": "source:sample_boundary_record", + "evidence_role": "supporting_context", + "role_reason": "available_evidence_sources 안에 있는 sample 근거다.", + "role_reason_info_class": "mixed", + } + ], + } + ), + input_ref=[trace_id], + source_data_ids=["source:runtime"], + ) + + assert frame.semantic_judgement_status == "ran" + assert frame.generated_by == "LLM:order-121-answer-basis-fake" + assert frame.evidence_roles[0].source_data_id == "source:sample_boundary_record" + assert "source:sample_boundary_record" in frame.source_data_ids + + +def test_node2_answer_basis_rejects_source_outside_available_evidence_sources() -> None: + trace_store, data_store, trace_id = _stores() + + _, _, frame = run_node2_answer_basis_selection( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_121", + user_question="이 구조 어때?", + boundary_id="source:boundary", + boundary=MetainfoBoundary(), + handoff_frame_id="source:handoff", + adapter=AnswerBasisPayloadFakeAdapter( + { + "answer_basis_mode": "relative_allowed", + "basis_reason_codes": ["user_asked_for_interpretation"], + "mode_selection_reason": "허용되지 않은 근거 ID를 일부러 쓴다.", + "mode_selection_reason_info_class": "mixed", + "evidence_roles": [ + { + "source_data_id": "source:not_in_available_table", + "evidence_role": "supporting_context", + "role_reason": "validator가 막아야 한다.", + "role_reason_info_class": "mixed", + } + ], + } + ), + input_ref=[trace_id], + source_data_ids=["source:runtime"], + ) + + assert frame.generated_by == "CODE:FALLBACK" + assert frame.answer_basis_mode == "mixed_or_uncertain" + assert frame.answer_basis_failure_type == "schema_failed" + assert "source_data_id must exist" in frame.answer_basis_validation_error + + +def test_node3_brief_preserves_l_loop_failure_attitude() -> None: + trace_store, data_store, trace_id = _stores() + _record_l_loop_return_summary(data_store, trace_id) + + _, _, brief = record_node3_input_brief( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_121", + user_question="말하기 모드 설계 어때?", + handoff_frame_id="source:handoff", + boundary=MetainfoBoundary(), + input_trace_ids=[trace_id], + source_data_ids=["source:handoff"], + ) + payload = node3_brief_llm_payload(brief) + grounding = build_node3_grounding_block(brief) + + assert brief.l_loop_task_status == "failed" + assert brief.l_loop_failure_level == "budget_exhausted" + assert brief.l_loop_result_attitude_hint == "l_loop_budget_exhausted" + assert payload["l_loop_result"]["attitude_hint"] == "l_loop_budget_exhausted" + assert "L 검색 목표 상태: failed / budget_exhausted" in grounding + assert "검색 성공으로 단정하지 않는다" in grounding + + +def test_node4_flags_l_loop_failure_hidden_as_success() -> None: + trace_store, data_store, trace_id = _stores() + brief = Node3InputBriefFrame( + frame_id="node_3:input_brief_frame", + turn_id="turn_order_121", + user_question="말하기 모드 설계 어때?", + brief_status="ready", + handoff_frame_id="source:handoff", + l_loop_return_summary_frame_id="L:return_summary_frame", + l_loop_task_status="failed", + l_loop_failure_level="budget_exhausted", + l3_goal_match_status="missing", + l3_semantic_goal_match_status="missing", + remaining_query_attempts=0, + remaining_read_doc_calls=8, + l_loop_result_attitude_hint="l_loop_budget_exhausted", + source_trace_ids=[trace_id], + source_data_ids=["source:handoff", "L:return_summary_frame"], + ) + rendered_markdown = ( + build_node3_grounding_block(brief) + + "\n\nL 검색 목표가 성공했으므로 설계 평가는 충분히 확정됐다." + ) + + run_node4_gatekeeper( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_121", + report_id="report:test", + boundary_id="boundary:test", + brief_frame=brief, + rendered_markdown=rendered_markdown, + adapter=SongRyeonAllNodesFakeLLMAdapter(), + input_ref=[trace_id], + source_data_ids=["report:test", brief.frame_id, "boundary:test"], + ) + gate = data_store.require_record("node_4:gatekeeper_frame").payload + + assert gate["gate_status"] == "needs_revision" + assert "l_loop_failure_hidden_as_success" in gate["contradictions"] + + +def _stores() -> tuple[TraceStore, DataStore, str]: + trace_store = TraceStore() + data_store = DataStore() + event = trace_store.create_event( + turn_id="turn_order_121", + actor="test", + event_type="node_output", + output_ref=["source:runtime", "source:boundary", "source:handoff"], + schema_status="passed", + ) + for data_id in ("source:runtime", "source:boundary", "source:handoff"): + data_store.create_record( + data_id=data_id, + data_type="test:source", + source_trace_id=event.event_id, + payload={"data_id": data_id}, + ) + return trace_store, data_store, event.event_id + + +def _record_l_loop_return_summary(data_store: DataStore, trace_id: str) -> None: + data_store.create_record( + data_id="L:return_summary_frame", + data_type="node_output:l_loop_return_summary_frame", + source_trace_id=trace_id, + payload={ + "frame_id": "L:return_summary_frame", + "turn_id": "turn_order_121", + "loop_id": "L", + "l_loop_task_status": "failed", + "failure_level": "budget_exhausted", + "l3_goal_match_status": "missing", + "l3_semantic_goal_match_status": "missing", + "remaining_query_attempts": 0, + "remaining_read_doc_calls": 8, + }, + ) diff --git a/tests/test_order_122_l_revision_read_doc_path.py b/tests/test_order_122_l_revision_read_doc_path.py new file mode 100644 index 0000000..fab13f9 --- /dev/null +++ b/tests/test_order_122_l_revision_read_doc_path.py @@ -0,0 +1,339 @@ +from __future__ import annotations + +from dataclasses import asdict +import json + +import pytest + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.schemas import ToolUseBudgetFrame +from songryeon_core.core.trace_store import TraceStore +from songryeon_core.llm.base import LLMRequest, LLMResponse +from songryeon_core.loops.l_loop_continuation import record_l_loop_continuation_decision +from songryeon_core.loops.l_loop_revision_tool_attempt import run_l_loop_revision_tool_attempt +from songryeon_core.nodes.l2_query_setter import ( + l2_revision_query_plan_data_id, + run_l2_revision_query_planner, + selected_query_from_plan, + selected_target_tool_from_plan, +) +from songryeon_core.nodes.l3_result_keeper import ( + l3_revision_achievement_frame_data_id, + run_l3_revision_result_keeper, +) + + +class RevisionPayloadAdapter: + model_id = "order-122-revision-payload-adapter" + + def __init__(self, payload: dict[str, object]) -> None: + self.payload = payload + + def complete(self, request: LLMRequest) -> LLMResponse: + return LLMResponse( + text=json.dumps(self.payload, ensure_ascii=False), + model_id=self.model_id, + raw=self.payload, + ) + + +def test_revision_l2_accepts_read_doc_for_exact_unread_candidate() -> None: + trace_store, data_store, trace_id = _stores() + _record_revision_input( + data_store=data_store, + trace_id=trace_id, + unread_doc_ids=["ORDER_122_DOC.md"], + remaining_query_attempts=0, + remaining_read_doc_calls=1, + ) + + run_l2_revision_query_planner( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_122", + revision_input_data_id="L2:revision_input:0001", + adapter=RevisionPayloadAdapter(_read_doc_plan("ORDER_122_DOC.md")), + ) + + plan_payload = data_store.require_record(l2_revision_query_plan_data_id(1)).payload + assert isinstance(plan_payload, dict) + assert selected_target_tool_from_plan(plan_payload) == "read_doc" + assert selected_query_from_plan(plan_payload) == "ORDER_122_DOC.md" + + +def test_revision_l2_rejects_read_doc_outside_unread_candidates() -> None: + trace_store, data_store, trace_id = _stores() + _record_revision_input( + data_store=data_store, + trace_id=trace_id, + unread_doc_ids=["ORDER_122_DOC.md"], + remaining_query_attempts=0, + remaining_read_doc_calls=1, + ) + + with pytest.raises(ValueError, match="L2 revision query planner failed: schema_failed"): + run_l2_revision_query_planner( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_122", + revision_input_data_id="L2:revision_input:0001", + adapter=RevisionPayloadAdapter(_read_doc_plan("OTHER_DOC.md")), + ) + + +def test_continuation_allows_read_path_after_query_budget_exhaustion() -> None: + trace_store = TraceStore() + data_store = DataStore() + data_store.create_record( + data_id="L3:achievement_frame", + data_type="node_output:L3_achievement_frame", + payload={ + "achievement_status": "partial", + "goal_match_status": "partial", + "semantic_goal_match_status": "partial", + "read_doc_ids": [], + "search_result_doc_ids": ["ORDER_122_DOC.md"], + }, + ) + data_store.create_record( + data_id="L2:query_frame", + data_type="node_output:L2_query_frame", + payload={"query_text": "previous query"}, + ) + data_store.create_record( + data_id="tool_budget:turn_order_122:0008", + data_type="tool_use_budget", + payload=asdict( + _budget_frame( + budget_id="tool_budget:turn_order_122:0008", + max_query_attempts=8, + query_count=8, + max_read_doc_calls=1, + read_doc_count=0, + tool_call_count=8, + max_tool_calls=10, + ) + ), + ) + + _, _, frame = record_l_loop_continuation_decision( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_122", + attempt_index=8, + max_attempts=12, + l3_achievement_data_id="L3:achievement_frame", + l2_query_frame_data_id="L2:query_frame", + ) + + assert frame.continuation_status == "continue" + assert frame.next_target_node == "L2" + assert frame.unread_candidate_doc_ids == ["ORDER_122_DOC.md"] + + +def test_revision_read_doc_increments_read_count_without_query_count(tmp_path) -> None: + doc_id = "ORDER_122_DOC.md" + (tmp_path / doc_id).write_text("# ORDER 122\n\n읽어야 하는 후보 원문.", encoding="utf-8") + trace_store, data_store, trace_id = _stores() + _record_l1_goal(data_store=data_store, trace_id=trace_id) + _record_revision_query_frame( + data_store=data_store, + trace_id=trace_id, + doc_id=doc_id, + ) + data_store.create_record( + data_id="tool_budget:turn_order_122:0008", + data_type="tool_use_budget", + payload=asdict( + _budget_frame( + budget_id="tool_budget:turn_order_122:0008", + max_query_attempts=8, + query_count=8, + max_read_doc_calls=1, + read_doc_count=0, + tool_call_count=8, + max_tool_calls=10, + ) + ), + ) + + result = run_l_loop_revision_tool_attempt( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_122", + revision_query_frame_data_id="L2:revision_query_frame:0001", + document_root=tmp_path, + max_tool_calls=10, + max_query_attempts=8, + max_read_doc_calls=1, + max_input_chars=6000, + ) + + budget_payload = data_store.require_record(result.tool_budget_data_id).payload + assert result.tool_name == "read_doc" + assert isinstance(budget_payload, dict) + assert budget_payload["query_count"] == 8 + assert budget_payload["read_doc_count"] == 1 + assert budget_payload["read_doc_ids"] == [doc_id] + assert budget_payload["stop_reason"] == "completed" + + run_l3_revision_result_keeper( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_122", + attempt_index=1, + revision_query_frame_data_id="L2:revision_query_frame:0001", + revision_tool_source_trace_ids=result.source_trace_ids, + revision_tool_source_data_ids=result.source_data_ids, + user_query="ORDER_122 문서를 읽어줘", + ) + achievement_payload = data_store.require_record(l3_revision_achievement_frame_data_id(1)).payload + assert isinstance(achievement_payload, dict) + assert doc_id in achievement_payload["read_doc_ids"] + + +def _stores() -> tuple[TraceStore, DataStore, str]: + trace_store = TraceStore() + data_store = DataStore() + event = trace_store.create_event( + turn_id="turn_order_122", + actor="test", + event_type="node_output", + output_ref=["source:test"], + schema_status="passed", + ) + return trace_store, data_store, event.event_id + + +def _record_revision_input( + *, + data_store: DataStore, + trace_id: str, + unread_doc_ids: list[str], + remaining_query_attempts: int, + remaining_read_doc_calls: int, +) -> None: + data_store.create_record( + data_id="L2:revision_input:0001", + data_type="node_input:L2_revision_input_frame", + source_trace_id=trace_id, + payload={ + "frame_id": "L2:revision_input:0001", + "turn_id": "turn_order_122", + "attempt_index": 1, + "max_attempts": 3, + "macro_goal": "read unread candidate", + "micro_goal": "use exact unread candidate doc id", + "previous_query_text": "previous query", + "previous_tool_name": "search_docs", + "read_document_names": [], + "unread_candidate_doc_ids": unread_doc_ids, + "unread_candidate_summaries": [ + f"doc_id={doc_id}; preview=후보 문서" for doc_id in unread_doc_ids + ], + "l3_goal_status": "partial", + "l3_goal_match_status": "partial", + "l3_semantic_goal_match_status": "partial", + "l3_feedback_text": "need more original documents", + "remaining_tool_calls": 2, + "remaining_query_attempts": remaining_query_attempts, + "remaining_read_doc_calls": remaining_read_doc_calls, + "source_trace_ids": [trace_id], + "source_data_ids": ["L:continuation:0001"], + "schema_name": "L2RevisionInputFrame", + "schema_version": "0.1", + }, + ) + + +def _read_doc_plan(doc_id: str) -> dict[str, object]: + return { + "planner_mode": "revision_llm", + "selected_candidate_id": "L2:revision_query_candidate_0001", + "candidates": [ + { + "candidate_id": "L2:revision_query_candidate_0001", + "query_text": doc_id, + "purpose": "query 예산 없이 보존된 unread candidate를 원문 읽는다.", + "expected_signal": "read_doc 원문", + "priority": 1, + "target_tool_name": "read_doc", + "source_data_ids": ["L2:revision_input:0001"], + } + ], + } + + +def _record_l1_goal(*, data_store: DataStore, trace_id: str) -> None: + data_store.create_record( + data_id="L1:goal_frame", + data_type="node_output:L1_goal_frame", + source_trace_id=trace_id, + payload={ + "frame_id": "L1:goal_frame", + "turn_id": "turn_order_122", + "macro_goal": "read requested document", + "micro_goal": "read one candidate", + "minimum_read_documents": 1, + "source_trace_ids": [trace_id], + "source_data_ids": [], + }, + ) + + +def _record_revision_query_frame( + *, + data_store: DataStore, + trace_id: str, + doc_id: str, +) -> None: + data_store.create_record( + data_id="L2:revision_query_frame:0001", + data_type="node_output:L2_revision_query_frame", + source_trace_id=trace_id, + payload={ + "frame_id": "L2:revision_query_frame:0001", + "turn_id": "turn_order_122", + "query_text": doc_id, + "query_source": "revision_llm_query_plan", + "query_mode": "direct_doc_read", + "target_tool_name": "read_doc", + "source_trace_ids": [trace_id], + "source_data_ids": ["L2:revision_query_plan:0001"], + }, + ) + + +def _budget_frame( + *, + budget_id: str, + max_query_attempts: int, + query_count: int, + max_read_doc_calls: int, + read_doc_count: int, + tool_call_count: int, + max_tool_calls: int, +) -> ToolUseBudgetFrame: + return ToolUseBudgetFrame( + budget_id=budget_id, + turn_id="turn_order_122", + loop_id="L", + sequence_index=8, + max_tool_calls=max_tool_calls, + search_top_k=2, + max_query_attempts=max_query_attempts, + max_query_candidates=max_query_attempts, + max_read_doc_calls=max_read_doc_calls, + max_input_chars=6000, + tool_call_count=tool_call_count, + query_count=query_count, + read_doc_count=read_doc_count, + input_chars_used=100, + executed_queries=[f"query {index}" for index in range(query_count)], + read_doc_ids=[], + cache_statuses=[], + stop_reason="within_budget", + reason="CODE_STATUS:test_budget", + source_trace_ids=["trace:test"], + source_data_ids=["L2:query_frame"], + ) diff --git a/tests/test_order_123_actual_read_doc_vs_context_pack_count.py b/tests/test_order_123_actual_read_doc_vs_context_pack_count.py new file mode 100644 index 0000000..10953db --- /dev/null +++ b/tests/test_order_123_actual_read_doc_vs_context_pack_count.py @@ -0,0 +1,215 @@ +from __future__ import annotations + +from dataclasses import asdict + +import pytest + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.schemas import ( + DocumentContextPackFrame, + DocumentContextPackIncludedDocument, + MetainfoBoundary, + Node3BriefDocument, + Node3InputBriefFrame, + validate_node3_input_brief_frame, +) +from songryeon_core.core.trace_store import TraceStore +from songryeon_core.llm.fake import SongRyeonAllNodesFakeLLMAdapter +from songryeon_core.nodes.node_2_handoff import ( + node3_brief_llm_payload, + record_node3_input_brief, +) +from songryeon_core.nodes.node_3_reporter import build_node3_grounding_block +from songryeon_core.nodes.node_4_gatekeeper import run_node4_gatekeeper +from songryeon_core.tools.document_context_pack import DOCUMENT_CONTEXT_PACK_DATA_TYPE + + +def test_node3_grounding_separates_actual_read_doc_from_supplied_context() -> None: + brief = _brief(actual_read_doc_count=2, supplied_context_count=10) + + grounding = build_node3_grounding_block(brief) + + assert "- 실제 read_doc 도구 원문 읽기: 2개" in grounding + assert "- node_3 공급 문서 context: 10개" in grounding + assert "- 읽은 문서: 10개" not in grounding + + +def test_node3_llm_payload_exposes_structured_count_boundary() -> None: + brief = _brief(actual_read_doc_count=2, supplied_context_count=10) + + payload = node3_brief_llm_payload(brief) + + assert payload["actual_tool_read_doc"]["count"] == 2 + assert payload["actual_tool_read_doc"]["document_names"] == ["ACTUAL_A.md", "ACTUAL_B.md"] + assert payload["supplied_document_context"]["count"] == 10 + assert len(payload["supplied_document_contexts"]) == 10 + assert len(payload["read_documents"]) == 10 + assert payload["read_documents"][0]["legacy_alias"] == "supplied_document_context" + + +def test_node3_input_brief_rejects_context_count_mismatch() -> None: + brief = _brief(actual_read_doc_count=2, supplied_context_count=10) + brief.supplied_document_context_count = 9 + + with pytest.raises(ValueError, match="supplied_document_context_count"): + validate_node3_input_brief_frame(brief) + + +def test_record_node3_input_brief_keeps_l_tool_count_and_context_pack_count_separate() -> None: + trace_store, data_store, trace_id = _stores() + _record_l_loop_return_summary(data_store=data_store, trace_id=trace_id) + _record_document_context_pack(data_store=data_store, trace_id=trace_id) + + _, _, brief = record_node3_input_brief( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_123", + user_question="read_doc 수와 공급 context 수를 구분해줘", + handoff_frame_id="node_2:handoff_frame", + boundary=MetainfoBoundary(), + input_trace_ids=[trace_id], + source_data_ids=["node_2:handoff_frame", "L:document_context_pack_frame"], + ) + + assert brief.actual_tool_read_doc_count == 2 + assert brief.actual_tool_read_doc_documents == ["ACTUAL_A.md", "ACTUAL_B.md"] + assert brief.supplied_document_context_count == 3 + assert len(brief.read_documents) == 3 + assert [document.document_name for document in brief.read_documents] == [ + "PACKED_1.md", + "PACKED_2.md", + "PACKED_3.md", + ] + + +def test_node4_count_guard_uses_actual_read_doc_count_not_context_count() -> None: + trace_store, data_store, trace_id = _stores() + brief = _brief(actual_read_doc_count=2, supplied_context_count=10) + rendered_markdown = "\n".join( + [ + "근거 기준:", + "- 실제 read_doc 도구 원문 읽기: 10개", + "- node_3 공급 문서 context: 10개", + "- 검색 후보 문서(최종): 0개", + "- 검색 후보 문서(누적): 0개", + "- 현재 턴 실행 순서 자료: 0개", + "- 답변 한계: 일부러 actual read_doc count를 context count와 섞은 보고문이다.", + "", + "본문", + ] + ) + + run_node4_gatekeeper( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_123", + report_id="report:order_123", + boundary_id="boundary:order_123", + brief_frame=brief, + rendered_markdown=rendered_markdown, + adapter=SongRyeonAllNodesFakeLLMAdapter(), + input_ref=[trace_id], + source_data_ids=["report:order_123", brief.frame_id, "boundary:order_123"], + ) + + gate = data_store.require_record("node_4:gatekeeper_frame").payload + assert gate["gate_status"] == "needs_revision" + assert "grounding_count_mismatch" in gate["reason"] + assert "실제 read_doc 도구 원문 읽기:actual_10_expected_2" in gate["contradictions"] + + +def _brief(*, actual_read_doc_count: int, supplied_context_count: int) -> Node3InputBriefFrame: + read_documents = [ + Node3BriefDocument( + document_name=f"PACKED_{index}.md", + char_count=len("context"), + text="context", + source_data_id="L:document_context_pack_frame", + ) + for index in range(1, supplied_context_count + 1) + ] + return Node3InputBriefFrame( + frame_id="node_3:input_brief_frame", + turn_id="turn_order_123", + user_question="read_doc 수와 공급 context 수를 구분해줘", + brief_status="ready", + handoff_frame_id="node_2:handoff_frame", + read_documents=read_documents, + actual_tool_read_doc_count=actual_read_doc_count, + actual_tool_read_doc_documents=["ACTUAL_A.md", "ACTUAL_B.md"][:actual_read_doc_count], + supplied_document_context_count=supplied_context_count, + source_data_ids=["node_2:handoff_frame"], + ) + + +def _stores() -> tuple[TraceStore, DataStore, str]: + trace_store = TraceStore() + data_store = DataStore() + event = trace_store.create_event( + turn_id="turn_order_123", + actor="test", + event_type="node_output", + output_ref=["node_2:handoff_frame"], + schema_status="passed", + ) + data_store.create_record( + data_id="node_2:handoff_frame", + data_type="test:handoff", + source_trace_id=event.event_id, + payload={"frame_id": "node_2:handoff_frame"}, + ) + return trace_store, data_store, event.event_id + + +def _record_l_loop_return_summary(*, data_store: DataStore, trace_id: str) -> None: + data_store.create_record( + data_id="L:return_summary_frame", + data_type="node_output:l_loop_return_summary_frame", + source_trace_id=trace_id, + payload={ + "frame_id": "L:return_summary_frame", + "turn_id": "turn_order_123", + "l_loop_task_status": "achieved", + "failure_level": "none", + "l3_goal_match_status": "matched", + "l3_semantic_goal_match_status": "matched", + "remaining_query_attempts": 0, + "remaining_read_doc_calls": 8, + "actual_read_doc_count": 2, + "read_doc_ids": ["docs/ACTUAL_A.md", "docs/ACTUAL_B.md"], + }, + ) + + +def _record_document_context_pack(*, data_store: DataStore, trace_id: str) -> None: + documents = [ + DocumentContextPackIncludedDocument( + doc_id=f"docs/PACKED_{index}.md", + document_name=f"PACKED_{index}.md", + char_count=len(f"packed context {index}"), + rank_index=index, + selection_basis="test_pack", + text=f"packed context {index}", + source_data_id=f"source:{index}", + ) + for index in range(1, 4) + ] + frame = DocumentContextPackFrame( + frame_id="L:document_context_pack_frame", + turn_id="turn_order_123", + max_document_context_chars=100000, + budget_unit="chars", + whole_document_only=True, + strict_rank_order=True, + included_documents=documents, + included_document_count=len(documents), + included_total_chars=sum(document.char_count for document in documents), + source_trace_ids=[trace_id], + source_data_ids=["L:return_summary_frame"], + ) + data_store.create_record( + data_id=frame.frame_id, + data_type=DOCUMENT_CONTEXT_PACK_DATA_TYPE, + source_trace_id=trace_id, + payload=asdict(frame), + ) diff --git a/tests/test_order_124_node0_document_material_packet.py b/tests/test_order_124_node0_document_material_packet.py new file mode 100644 index 0000000..8df1881 --- /dev/null +++ b/tests/test_order_124_node0_document_material_packet.py @@ -0,0 +1,209 @@ +from __future__ import annotations + +from dataclasses import asdict + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.schemas import ( + DocumentContextPackFrame, + DocumentContextPackIncludedDocument, + MetainfoBoundary, +) +from songryeon_core.core.trace_store import TraceStore +from songryeon_core.nodes.node_0_memory_supplier import ( + NODE0_DOCUMENT_MATERIAL_PACKET_FRAME_DATA_ID, + build_node0_document_material_packet_frame, + record_node0_document_material_packet, +) +from songryeon_core.nodes.node_2_handoff import ( + node3_brief_llm_payload, + record_node3_input_brief, + record_route2_handoff, +) +from songryeon_core.tools.document_context_pack import DOCUMENT_CONTEXT_PACK_DATA_TYPE + + +def test_node0_document_material_packet_marks_document_roles_without_summary() -> None: + trace_store, data_store, trace_id = _stores() + _record_l_return_summary(data_store, trace_id) + _record_context_pack(data_store, trace_id) + _record_read_doc(data_store, trace_id, "docs/A.md", "A 원문") + _record_read_doc(data_store, trace_id, "docs/B.md", "B 원문") + + frame = build_node0_document_material_packet_frame( + data_store=data_store, + turn_id="turn_order_124", + frame_id=NODE0_DOCUMENT_MATERIAL_PACKET_FRAME_DATA_ID, + source_trace_ids=[trace_id], + source_data_ids=[ + "L:return_summary_frame", + "L:document_context_pack_frame", + "tool_result:read_doc:A", + "tool_result:read_doc:B", + ], + ) + + assert frame.semantic_judgement_status == "not_run" + assert frame.item_count == 3 + assert frame.search_candidate_count == 3 + assert frame.actual_tool_read_doc_count == 2 + assert frame.supplied_document_context_count == 3 + assert frame.unread_candidate_count == 1 + + by_name = {item.document_name: item for item in frame.items} + assert by_name["A.md"].was_search_candidate is True + assert by_name["A.md"].was_actual_tool_read_doc is True + assert by_name["A.md"].was_supplied_document_context is True + assert by_name["C.md"].was_unread_candidate is True + assert by_name["C.md"].was_actual_tool_read_doc is False + + +def test_route2_handoff_preserves_document_material_packet_counts() -> None: + trace_store, data_store, trace_id = _stores() + _record_route2_minimum_records(data_store, trace_id) + _record_l_return_summary(data_store, trace_id) + _, material_id, _ = record_node0_document_material_packet( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_124", + frame_id=NODE0_DOCUMENT_MATERIAL_PACKET_FRAME_DATA_ID, + source_trace_ids=[trace_id], + source_data_ids=["L:return_summary_frame"], + ) + + _, handoff_id = record_route2_handoff( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_124", + user_question="문서 장부 확인", + node2_input_frame_id="node2_input:turn_order_124", + node2_input_trace_id=trace_id, + final_memory_packet_id="memory_packet:node_2:final_trace_for_2", + turn_outcome_id="turn_outcome:turn_order_124", + route_ids=["route:2"], + l_loop_output_ids=[], + ) + + payload = data_store.require_record(handoff_id).payload + assert payload["document_material_packet_frame_id"] == material_id + assert payload["document_material_item_count"] == 3 + assert payload["document_material_unread_candidate_count"] == 1 + assert material_id in payload["source_data_ids"] + + +def test_node3_brief_and_payload_receive_document_material_ledger() -> None: + trace_store, data_store, trace_id = _stores() + _record_l_return_summary(data_store, trace_id) + _record_context_pack(data_store, trace_id) + _, material_id, _ = record_node0_document_material_packet( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_124", + frame_id=NODE0_DOCUMENT_MATERIAL_PACKET_FRAME_DATA_ID, + source_trace_ids=[trace_id], + source_data_ids=["L:return_summary_frame", "L:document_context_pack_frame"], + ) + + _, _, brief = record_node3_input_brief( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_124", + user_question="읽은 문서와 안 읽은 후보를 구분해줘", + handoff_frame_id="node_2:handoff_frame", + boundary=MetainfoBoundary(), + input_trace_ids=[trace_id], + source_data_ids=["node_2:handoff_frame", "L:return_summary_frame", "L:document_context_pack_frame", material_id], + ) + payload = node3_brief_llm_payload(brief) + + assert brief.document_material_packet_frame_id == material_id + assert len(brief.document_material_items) == 3 + assert any(item.was_unread_candidate for item in brief.document_material_items) + assert payload["document_material_packet"]["status"] == "present" + assert payload["document_material_packet"]["item_count"] == 3 + assert payload["document_material_packet"]["unread_candidate_count"] == 1 + assert "source_data_ids" not in payload["document_material_packet"]["items"][0] + + +def _stores() -> tuple[TraceStore, DataStore, str]: + trace_store = TraceStore() + data_store = DataStore() + event = trace_store.create_event( + turn_id="turn_order_124", + actor="test", + event_type="node_output", + output_ref=["seed"], + schema_status="passed", + ) + return trace_store, data_store, event.event_id + + +def _record_l_return_summary(data_store: DataStore, trace_id: str) -> None: + data_store.create_record( + data_id="L:return_summary_frame", + data_type="node_output:l_loop_return_summary_frame", + source_trace_id=trace_id, + payload={ + "frame_id": "L:return_summary_frame", + "turn_id": "turn_order_124", + "read_doc_ids": ["docs/A.md", "docs/B.md"], + "search_result_doc_ids": ["docs/A.md", "docs/B.md", "docs/C.md"], + }, + ) + + +def _record_context_pack(data_store: DataStore, trace_id: str) -> None: + documents = [ + DocumentContextPackIncludedDocument( + doc_id=f"docs/{name}.md", + document_name=f"{name}.md", + char_count=len(name), + rank_index=index, + selection_basis="test", + text=f"{name} context", + source_data_id=f"source:{name}", + ) + for index, name in enumerate(["A", "B", "C"], start=1) + ] + frame = DocumentContextPackFrame( + frame_id="L:document_context_pack_frame", + turn_id="turn_order_124", + max_document_context_chars=1000, + budget_unit="chars", + whole_document_only=True, + strict_rank_order=True, + included_documents=documents, + included_document_count=len(documents), + included_total_chars=sum(item.char_count for item in documents), + source_trace_ids=[trace_id], + source_data_ids=["L:return_summary_frame"], + ) + data_store.create_record( + data_id=frame.frame_id, + data_type=DOCUMENT_CONTEXT_PACK_DATA_TYPE, + source_trace_id=trace_id, + payload=asdict(frame), + ) + + +def _record_read_doc(data_store: DataStore, trace_id: str, doc_id: str, text: str) -> None: + data_store.create_record( + data_id=f"tool_result:read_doc:{doc_id.rsplit('/', 1)[-1].removesuffix('.md')}", + data_type="tool_result:read_doc", + source_trace_id=trace_id, + payload={"doc_id": doc_id, "text": text, "char_count": len(text)}, + ) + + +def _record_route2_minimum_records(data_store: DataStore, trace_id: str) -> None: + for data_id, data_type in { + "node2_input:turn_order_124": "node_output:node2_input_frame", + "memory_packet:node_2:final_trace_for_2": "memory_packet", + "turn_outcome:turn_order_124": "node_output:turn_outcome", + "route:2": "node_output:route_decision", + }.items(): + data_store.create_record( + data_id=data_id, + data_type=data_type, + source_trace_id=trace_id, + payload={"data_id": data_id}, + ) diff --git a/tests/test_order_125_l3_per_document_summary_frame.py b/tests/test_order_125_l3_per_document_summary_frame.py new file mode 100644 index 0000000..8265c21 --- /dev/null +++ b/tests/test_order_125_l3_per_document_summary_frame.py @@ -0,0 +1,233 @@ +from __future__ import annotations + +from dataclasses import asdict +import json + +import pytest + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.schemas import ( + L3PerDocumentSummaryFrame, + MetainfoBoundary, + validate_l3_per_document_summary_frame, +) +from songryeon_core.core.trace_store import TraceStore +from songryeon_core.llm.base import LLMRequest, LLMResponse +from songryeon_core.nodes.l3_result_keeper import ( + run_l3_result_keeper, +) +from songryeon_core.nodes.node_2_handoff import ( + node3_brief_llm_payload, + record_node3_input_brief, +) + + +class SequencePayloadAdapter: + model_id = "order-125-sequence-fake" + + def __init__(self, payloads: list[dict[str, object]]) -> None: + self.payloads = list(payloads) + + def complete(self, request: LLMRequest) -> LLMResponse: + if not self.payloads: + raise RuntimeError("no fake payload left") + payload = self.payloads.pop(0) + return LLMResponse( + text=json.dumps(payload, ensure_ascii=False), + model_id=self.model_id, + raw=payload, + ) + + +def test_l3_per_document_summary_schema_separates_relative_and_mixed() -> None: + frame = _valid_summary_frame() + validate_l3_per_document_summary_frame(frame) + + frame.plain_summary_info_class = "mixed" + with pytest.raises(ValueError, match="plain_document_summary must be marked relative"): + validate_l3_per_document_summary_frame(frame) + + +def test_l3_result_keeper_records_per_document_summary_frame() -> None: + trace_store, data_store, l1_event, l2_event, source_trace_id = _stores_with_l_inputs() + data_store.create_record( + data_id="tool_result:read_doc:001", + data_type="tool_result:read_doc", + source_trace_id=source_trace_id, + payload={ + "doc_id": "Administrative_Reform_1/04_Orders/ORDER_125_TEST.md", + "text": "ORDER_125는 L3가 읽은 문서마다 두 종류의 요약을 남기는 구조다.", + "char_count": 37, + }, + ) + + run_l3_result_keeper( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_125", + l1_event=l1_event, + l2_event=l2_event, + extra_input_trace_ids=[source_trace_id], + extra_input_data_ids=["L1:goal_frame", "tool_result:read_doc:001"], + user_query="ORDER_125 요약 흐름을 알려줘", + adapter=SequencePayloadAdapter( + [ + _achievement_payload(), + { + "plain_document_summary": "ORDER_125가 문서별 두 종류 요약을 다룬다고 설명한다.", + "task_relevant_summary": "현재 질문 기준으로 L3 문서별 요약 frame의 용도와 경계가 핵심이다.", + "summary_limit_note": "", + }, + ] + ), + ) + + summary_records = [ + record + for record in data_store.list_records() + if record.data_type == "node_output:L3_per_document_summary_frame" + ] + assert len(summary_records) == 1 + payload = summary_records[0].payload + assert isinstance(payload, dict) + assert payload["summary_status"] == "ran" + assert payload["plain_summary_info_class"] == "relative" + assert payload["plain_summary_source_mode"] == "direct_record" + assert payload["plain_summary_claim_alignment"] == "one_document_to_one_summary" + assert payload["task_relevant_summary_info_class"] == "mixed" + assert payload["task_relevant_summary_source_mode"] == "source_bundle" + assert payload["task_relevant_summary_claim_alignment"] == "one_document_plus_task_context" + assert payload["source_document_data_id"] in payload["source_data_ids"] + assert payload["llm_call_data_id"] in payload["source_data_ids"] + + +def test_node3_brief_receives_l3_summary_material_without_raw_payload_ids() -> None: + trace_store, data_store, _, _, source_trace_id = _stores_with_l_inputs() + summary_frame = _valid_summary_frame() + data_store.create_record( + data_id=summary_frame.frame_id, + data_type="node_output:L3_per_document_summary_frame", + source_trace_id=source_trace_id, + payload=asdict(summary_frame), + ) + data_store.create_record( + data_id="source:handoff", + data_type="test:handoff", + source_trace_id=source_trace_id, + payload={"frame_id": "source:handoff"}, + ) + + _, _, brief = record_node3_input_brief( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_125", + user_question="L3 요약은 어떻게 전달돼?", + handoff_frame_id="source:handoff", + boundary=MetainfoBoundary(), + input_trace_ids=[source_trace_id], + source_data_ids=["source:handoff"], + ) + payload = node3_brief_llm_payload(brief) + + assert brief.brief_status == "ready" + assert len(brief.l3_document_summaries) == 1 + assert summary_frame.frame_id in brief.source_data_ids + l3_summary_payload = payload["l3_document_summaries"] + assert isinstance(l3_summary_payload, dict) + assert l3_summary_payload["count"] == 1 + item = l3_summary_payload["items"][0] + assert "source_data_id" not in item + assert item["plain_summary_info_class"] == "relative" + assert item["task_relevant_summary_info_class"] == "mixed" + assert item["plain_document_summary"] == summary_frame.plain_document_summary + assert item["task_relevant_summary"] == summary_frame.task_relevant_summary + + +def _stores_with_l_inputs() -> tuple[TraceStore, DataStore, object, object, str]: + trace_store = TraceStore() + data_store = DataStore() + l1_event = trace_store.create_event( + turn_id="turn_order_125", + actor="L1", + event_type="node_output", + output_ref=["L1:goal_frame"], + schema_status="passed", + ) + data_store.create_record( + data_id="L1:goal_frame", + data_type="node_output:L1_goal_frame", + source_trace_id=l1_event.event_id, + payload={ + "macro_goal": "문서별 요약 경계를 확인한다.", + "micro_goal": "L3 요약 frame이 node_3로 전달되는지 본다.", + "minimum_read_documents": 1, + }, + ) + l2_event = trace_store.create_event( + turn_id="turn_order_125", + actor="L2", + event_type="node_output", + output_ref=["L2:query_frame"], + schema_status="passed", + ) + data_store.create_record( + data_id="L2:query_frame", + data_type="node_output:L2_query_frame", + source_trace_id=l2_event.event_id, + payload={"query_text": "ORDER_125"}, + ) + source_event = trace_store.create_event( + turn_id="turn_order_125", + actor="tool:read_doc", + event_type="tool_result", + output_ref=["tool_result:read_doc:001"], + schema_status="passed", + ) + return trace_store, data_store, l1_event, l2_event, source_event.event_id + + +def _achievement_payload() -> dict[str, object]: + return { + "achievement_status": "partial", + "reason": "읽은 문서 하나를 기준으로 부분 판단한다.", + "macro_achievement_status": "partial", + "macro_achievement_reason": "문서 요약 경계 확인에는 일부 근거가 있다.", + "micro_achievement_status": "partial", + "micro_achievement_reason": "read_doc 문서가 하나 있다.", + "goal_match_status": "not_applicable", + "goal_match_reason": "CODE_STATUS:no_specific_doc_hint_detected", + "semantic_goal_match_status": "partial", + "semantic_goal_match_reason": "읽은 문서 하나만 있으므로 부분 근거다.", + } + + +def _valid_summary_frame() -> L3PerDocumentSummaryFrame: + return L3PerDocumentSummaryFrame( + frame_id="L3:per_document_summary:0001", + turn_id="turn_order_125", + source_document_data_id="tool_result:read_doc:001", + source_doc_id="Administrative_Reform_1/04_Orders/ORDER_125_TEST.md", + source_document_name="ORDER_125_TEST.md", + source_char_count=37, + summary_status="ran", + plain_document_summary="ORDER_125가 L3 문서별 요약 frame을 설명한다.", + plain_summary_source_data_id="tool_result:read_doc:001", + task_relevant_summary="현재 질문 기준으로 L3 요약 전달 경계가 중요하다.", + task_relevant_summary_source_data_ids=[ + "tool_result:read_doc:001", + "L1:goal_frame", + "L3:preserved_info_frame", + ], + generated_by="LLM:order-125-sequence-fake", + semantic_judgement_status="ran", + summary_failure_type="none", + llm_call_data_id="llm_call:L3_document_summary:trace_000010", + prompt_ref="songryeon_core/prompts/l3_per_document_summary_v0.md", + source_trace_ids=["trace_000001"], + source_data_ids=[ + "tool_result:read_doc:001", + "L1:goal_frame", + "L3:preserved_info_frame", + "llm_call:L3_document_summary:trace_000010", + ], + ) diff --git a/tests/test_order_126_runtime_document_extract_display.py b/tests/test_order_126_runtime_document_extract_display.py new file mode 100644 index 0000000..2223a98 --- /dev/null +++ b/tests/test_order_126_runtime_document_extract_display.py @@ -0,0 +1,95 @@ +from __future__ import annotations + +from songryeon_core.runtime.terminal_view import render_runtime_view + + +def test_runtime_view_lists_all_document_extract_records() -> None: + output = render_runtime_view( + { + "status": "ok", + "trace_count": 3, + "data_record_count": 3, + "data_records": [ + _document_extract_record( + data_id="tool_result:read_doc:A", + data_type="tool_result:read_doc", + doc_id="docs/A.md", + text="A 원문", + ), + _document_extract_record( + data_id="tool_result:read_doc:B", + data_type="tool_result:read_doc", + doc_id="docs/B.md", + text="B 원문", + ), + _document_extract_record( + data_id="tool_result:read_artifact:C", + data_type="tool_result:read_artifact", + doc_id="docs/C.md", + text="C 원문", + ), + ], + }, + user_input="문서 읽기 표시 확인", + ) + + assert "source=tool_result:read_doc:A" in output + assert "source=tool_result:read_doc:B" in output + assert "source=tool_result:read_artifact:C" in output + assert output.count("[TOOL_RESULT:DOCUMENT_EXTRACT") == 3 + + +def test_runtime_view_prefers_latest_run_scoped_document_extract_records() -> None: + output = render_runtime_view( + { + "status": "ok", + "trace_count": 4, + "data_record_count": 4, + "data_records": [ + { + "data_id": "L:run:0001:frame", + "data_type": "node_output:L_loop_run_frame", + "payload": {"run_index": 1}, + }, + { + "data_id": "L:run:0002:frame", + "data_type": "node_output:L_loop_run_frame", + "payload": {"run_index": 2}, + }, + _document_extract_record( + data_id="L:run:0001:tool_result:read_doc:OLD", + data_type="tool_result:read_doc", + doc_id="docs/OLD.md", + text="old", + ), + _document_extract_record( + data_id="L:run:0002:tool_result:read_doc:NEW", + data_type="tool_result:read_doc", + doc_id="docs/NEW.md", + text="new", + ), + ], + }, + user_input="최신 L run 문서 읽기 표시 확인", + ) + + assert "source=L:run:0002:tool_result:read_doc:NEW" in output + assert "source=L:run:0001:tool_result:read_doc:OLD" not in output + + +def _document_extract_record( + *, + data_id: str, + data_type: str, + doc_id: str, + text: str, +) -> dict[str, object]: + return { + "data_id": data_id, + "data_type": data_type, + "payload": { + "doc_id": doc_id, + "text": text, + "char_count": len(text), + }, + } diff --git a/tests/test_order_127_revision_document_extract_count_alignment.py b/tests/test_order_127_revision_document_extract_count_alignment.py new file mode 100644 index 0000000..b8acc0c --- /dev/null +++ b/tests/test_order_127_revision_document_extract_count_alignment.py @@ -0,0 +1,208 @@ +from __future__ import annotations + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.schemas import MetainfoBoundary +from songryeon_core.core.trace_store import TraceStore +from songryeon_core.nodes.node_0_memory_supplier import ( + NODE0_DOCUMENT_MATERIAL_PACKET_FRAME_DATA_ID, + build_l_loop_return_summary_frame, + build_node0_document_material_packet_frame, +) +from songryeon_core.nodes.node_2_handoff import record_node3_input_brief + + +def test_l_loop_return_summary_merges_revision_document_extract_records() -> None: + _, data_store, trace_id = _stores() + _record_l1_goal(data_store, trace_id) + _record_l3_achievement(data_store, trace_id, read_doc_ids=["docs/A.md", "docs/B.md"]) + _record_budget(data_store, trace_id) + for name in ["A", "B", "C", "D"]: + _record_read_doc(data_store, trace_id, f"docs/{name}.md", f"{name} 원문") + + frame = build_l_loop_return_summary_frame( + data_store=data_store, + turn_id="turn_order_127", + source_trace_ids=[trace_id], + source_data_ids=[ + "L1:goal_frame", + "L3:achievement_frame", + "L:budget:latest", + "tool_result:read_doc:A", + "tool_result:read_doc:B", + "tool_result:read_doc:C", + "tool_result:read_doc:D", + ], + ) + + assert frame.actual_read_doc_count == 4 + assert frame.read_doc_ids == ["docs/A.md", "docs/B.md", "docs/C.md", "docs/D.md"] + + +def test_node0_material_packet_marks_revision_read_docs_as_actual_reads() -> None: + _, data_store, trace_id = _stores() + _record_stale_return_summary(data_store, trace_id, read_doc_ids=["docs/A.md", "docs/B.md"]) + for name in ["A", "B", "C", "D"]: + _record_read_doc(data_store, trace_id, f"docs/{name}.md", f"{name} 원문") + + frame = build_node0_document_material_packet_frame( + data_store=data_store, + turn_id="turn_order_127", + frame_id=NODE0_DOCUMENT_MATERIAL_PACKET_FRAME_DATA_ID, + source_trace_ids=[trace_id], + source_data_ids=[ + "L:return_summary_frame", + "tool_result:read_doc:A", + "tool_result:read_doc:B", + "tool_result:read_doc:C", + "tool_result:read_doc:D", + ], + ) + + assert frame.actual_tool_read_doc_count == 4 + by_name = {item.document_name: item for item in frame.items} + assert by_name["C.md"].was_actual_tool_read_doc is True + assert by_name["D.md"].was_actual_tool_read_doc is True + + +def test_node3_brief_uses_revision_document_extract_records_for_actual_read_count() -> None: + trace_store, data_store, trace_id = _stores() + _record_handoff(data_store, trace_id) + _record_stale_return_summary(data_store, trace_id, read_doc_ids=["docs/A.md", "docs/B.md"]) + for name in ["A", "B", "C", "D"]: + _record_read_doc(data_store, trace_id, f"docs/{name}.md", f"{name} 원문") + + _, _, brief = record_node3_input_brief( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_127", + user_question="revision read_doc count를 확인해줘", + handoff_frame_id="node_2:handoff_frame", + boundary=MetainfoBoundary(), + input_trace_ids=[trace_id], + source_data_ids=[ + "node_2:handoff_frame", + "L:return_summary_frame", + "tool_result:read_doc:A", + "tool_result:read_doc:B", + "tool_result:read_doc:C", + "tool_result:read_doc:D", + ], + ) + + assert brief.actual_tool_read_doc_count == 4 + assert brief.actual_tool_read_doc_documents == ["A.md", "B.md", "C.md", "D.md"] + + +def _stores() -> tuple[TraceStore, DataStore, str]: + trace_store = TraceStore() + data_store = DataStore() + event = trace_store.create_event( + turn_id="turn_order_127", + actor="test", + event_type="node_output", + output_ref=["seed"], + schema_status="passed", + ) + return trace_store, data_store, event.event_id + + +def _record_l1_goal(data_store: DataStore, trace_id: str) -> None: + data_store.create_record( + data_id="L1:goal_frame", + data_type="node_output:L1_goal_frame", + source_trace_id=trace_id, + payload={ + "frame_id": "L1:goal_frame", + "minimum_read_documents": 1, + "evidence_requirement_kind": "multi_doc_relationship", + }, + ) + + +def _record_l3_achievement( + data_store: DataStore, + trace_id: str, + *, + read_doc_ids: list[str], +) -> None: + data_store.create_record( + data_id="L3:achievement_frame", + data_type="node_output:L3_achievement_frame", + source_trace_id=trace_id, + payload={ + "frame_id": "L3:achievement_frame", + "achievement_status": "partial", + "goal_match_status": "partial", + "semantic_goal_match_status": "partial", + "read_doc_ids": read_doc_ids, + "search_result_doc_ids": [ + "docs/A.md", + "docs/B.md", + "docs/C.md", + "docs/D.md", + ], + }, + ) + + +def _record_budget(data_store: DataStore, trace_id: str) -> None: + data_store.create_record( + data_id="L:budget:latest", + data_type="tool_use_budget", + source_trace_id=trace_id, + payload={ + "max_tool_calls": 10, + "tool_call_count": 4, + "max_read_doc_calls": 10, + "read_doc_count": 4, + "max_query_attempts": 8, + "query_count": 1, + "stop_reason": "completed", + }, + ) + + +def _record_stale_return_summary( + data_store: DataStore, + trace_id: str, + *, + read_doc_ids: list[str], +) -> None: + data_store.create_record( + data_id="L:return_summary_frame", + data_type="node_output:l_loop_return_summary_frame", + source_trace_id=trace_id, + payload={ + "frame_id": "L:return_summary_frame", + "turn_id": "turn_order_127", + "actual_read_doc_count": len(read_doc_ids), + "read_doc_ids": read_doc_ids, + "search_result_doc_ids": [ + "docs/A.md", + "docs/B.md", + "docs/C.md", + "docs/D.md", + ], + }, + ) + + +def _record_read_doc(data_store: DataStore, trace_id: str, doc_id: str, text: str) -> None: + data_store.create_record( + data_id=f"tool_result:read_doc:{doc_id.rsplit('/', 1)[-1].removesuffix('.md')}", + data_type="tool_result:read_doc", + source_trace_id=trace_id, + payload={"doc_id": doc_id, "text": text, "char_count": len(text)}, + ) + + +def _record_handoff(data_store: DataStore, trace_id: str) -> None: + data_store.create_record( + data_id="node_2:handoff_frame", + data_type="node_output:node2_handoff_frame", + source_trace_id=trace_id, + payload={ + "frame_id": "node_2:handoff_frame", + "source_data_ids": ["L:return_summary_frame"], + }, + ) diff --git a/tests/test_order_128_node3_actual_read_doc_identity_key.py b/tests/test_order_128_node3_actual_read_doc_identity_key.py new file mode 100644 index 0000000..32b9e92 --- /dev/null +++ b/tests/test_order_128_node3_actual_read_doc_identity_key.py @@ -0,0 +1,115 @@ +from __future__ import annotations + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.schemas import MetainfoBoundary +from songryeon_core.core.trace_store import TraceStore +from songryeon_core.nodes.node_2_handoff import record_node3_input_brief + + +def test_node3_actual_read_doc_count_uses_doc_identity_not_file_name() -> None: + trace_store, data_store, trace_id = _stores() + _record_handoff(data_store, trace_id) + _record_return_summary( + data_store, + trace_id, + read_doc_ids=[ + "04_Orders/README.md", + "05_Execution_Records/README.md", + ], + ) + _record_read_doc( + data_store, + trace_id, + "tool_result:read_doc:orders_readme", + "04_Orders/README.md", + "orders readme", + ) + _record_read_doc( + data_store, + trace_id, + "tool_result:read_doc:records_readme", + "05_Execution_Records/README.md", + "records readme", + ) + + _, _, brief = record_node3_input_brief( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_128", + user_question="README 두 개를 따로 세어줘", + handoff_frame_id="node_2:handoff_frame", + boundary=MetainfoBoundary(), + input_trace_ids=[trace_id], + source_data_ids=[ + "node_2:handoff_frame", + "L:return_summary_frame", + "tool_result:read_doc:orders_readme", + "tool_result:read_doc:records_readme", + ], + ) + + assert brief.actual_tool_read_doc_count == 2 + assert brief.actual_tool_read_doc_documents == [ + "04_Orders/README.md", + "05_Execution_Records/README.md", + ] + + +def _stores() -> tuple[TraceStore, DataStore, str]: + trace_store = TraceStore() + data_store = DataStore() + event = trace_store.create_event( + turn_id="turn_order_128", + actor="test", + event_type="node_output", + output_ref=["seed"], + schema_status="passed", + ) + return trace_store, data_store, event.event_id + + +def _record_handoff(data_store: DataStore, trace_id: str) -> None: + data_store.create_record( + data_id="node_2:handoff_frame", + data_type="node_output:node2_handoff_frame", + source_trace_id=trace_id, + payload={ + "frame_id": "node_2:handoff_frame", + "source_data_ids": ["L:return_summary_frame"], + }, + ) + + +def _record_return_summary( + data_store: DataStore, + trace_id: str, + *, + read_doc_ids: list[str], +) -> None: + data_store.create_record( + data_id="L:return_summary_frame", + data_type="node_output:l_loop_return_summary_frame", + source_trace_id=trace_id, + payload={ + "frame_id": "L:return_summary_frame", + "turn_id": "turn_order_128", + "actual_read_doc_count": len(read_doc_ids), + "read_doc_ids": read_doc_ids, + "search_result_doc_ids": read_doc_ids, + }, + ) + + +def _record_read_doc( + data_store: DataStore, + trace_id: str, + data_id: str, + doc_id: str, + text: str, +) -> None: + data_store.create_record( + data_id=data_id, + data_type="tool_result:read_doc", + source_trace_id=trace_id, + payload={"doc_id": doc_id, "text": text, "char_count": len(text)}, + ) diff --git a/tests/test_order_130_document_evidence_role_claim_guard.py b/tests/test_order_130_document_evidence_role_claim_guard.py new file mode 100644 index 0000000..eb0e965 --- /dev/null +++ b/tests/test_order_130_document_evidence_role_claim_guard.py @@ -0,0 +1,188 @@ +from __future__ import annotations + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.schemas import Node0DocumentMaterialItem, Node3InputBriefFrame +from songryeon_core.core.trace_store import TraceStore +from songryeon_core.llm.fake import SongRyeonAllNodesFakeLLMAdapter +from songryeon_core.nodes.node_2_handoff import node3_brief_llm_payload +from songryeon_core.nodes.node_3_reporter import build_node3_grounding_block +from songryeon_core.nodes.node_4_gatekeeper import run_node4_gatekeeper + + +def test_node3_payload_exposes_document_role_boundaries() -> None: + brief = _brief( + [ + Node0DocumentMaterialItem( + doc_id="04_Orders/ORDER_122.md", + document_name="ORDER_122.md", + source_roles=["supplied_document_context"], + was_supplied_document_context=True, + supplied_context_rank=1, + ) + ] + ) + + payload = node3_brief_llm_payload(brief) + boundaries = payload["document_evidence_role_boundaries"] + + assert boundaries["supplied_context_document_names"] == ["ORDER_122.md"] + assert boundaries["supplied_but_not_actual_read_doc_document_names"] == [ + "ORDER_122.md" + ] + assert boundaries["actual_tool_read_doc_document_names"] == [] + + +def test_node4_blocks_read_doc_claim_for_supplied_only_document() -> None: + trace_store, data_store, trace_id = _stores() + brief = _brief( + [ + Node0DocumentMaterialItem( + doc_id="04_Orders/ORDER_122.md", + document_name="ORDER_122.md", + source_roles=["supplied_document_context"], + was_supplied_document_context=True, + supplied_context_rank=1, + ) + ] + ) + rendered_markdown = ( + build_node3_grounding_block(brief) + + "\n\nORDER_122.md는 `read_doc`으로 읽혔고 node_3에게 전달됐다." + ) + + run_node4_gatekeeper( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_130", + report_id="report:order_130", + boundary_id="boundary:order_130", + brief_frame=brief, + rendered_markdown=rendered_markdown, + adapter=SongRyeonAllNodesFakeLLMAdapter(), + input_ref=[trace_id], + source_data_ids=["report:order_130", brief.frame_id, "boundary:order_130"], + ) + + gate = data_store.require_record("node_4:gatekeeper_frame").payload + assert gate["gate_status"] == "needs_revision" + assert "CODE_STATUS:document_evidence_role_claim_mismatch" in gate["reason"] + assert any( + item.startswith("read_doc_claim_without_actual_tool_read_doc:ORDER_122.md") + for item in gate["contradictions"] + ) + + +def test_node4_allows_context_claim_for_supplied_only_document() -> None: + trace_store, data_store, trace_id = _stores() + brief = _brief( + [ + Node0DocumentMaterialItem( + doc_id="04_Orders/ORDER_122.md", + document_name="ORDER_122.md", + source_roles=["supplied_document_context"], + was_supplied_document_context=True, + supplied_context_rank=1, + ) + ] + ) + rendered_markdown = ( + build_node3_grounding_block(brief) + + "\n\nORDER_122.md는 node_3 문서 context로 공급됐고, 실제 read_doc 문서라고는 단정하지 않는다." + ) + + run_node4_gatekeeper( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_130", + report_id="report:order_130", + boundary_id="boundary:order_130", + brief_frame=brief, + rendered_markdown=rendered_markdown, + adapter=SongRyeonAllNodesFakeLLMAdapter(), + input_ref=[trace_id], + source_data_ids=["report:order_130", brief.frame_id, "boundary:order_130"], + ) + + gate = data_store.require_record("node_4:gatekeeper_frame").payload + assert gate["gate_status"] == "pass" + assert "document_evidence_role_guard" in gate["checked_claims"] + + +def test_node4_allows_read_doc_claim_for_actual_read_document() -> None: + trace_store, data_store, trace_id = _stores() + brief = _brief( + [ + Node0DocumentMaterialItem( + doc_id="04_Orders/ORDER_122.md", + document_name="ORDER_122.md", + source_roles=["actual_tool_read_doc", "supplied_document_context"], + was_actual_tool_read_doc=True, + was_supplied_document_context=True, + actual_read_rank=1, + supplied_context_rank=1, + ) + ], + actual_tool_read_doc_count=1, + actual_tool_read_doc_documents=["ORDER_122.md"], + ) + rendered_markdown = ( + build_node3_grounding_block(brief) + + "\n\nORDER_122.md는 `read_doc`으로 읽혔고 node_3 문서 context로도 공급됐다." + ) + + run_node4_gatekeeper( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_130", + report_id="report:order_130", + boundary_id="boundary:order_130", + brief_frame=brief, + rendered_markdown=rendered_markdown, + adapter=SongRyeonAllNodesFakeLLMAdapter(), + input_ref=[trace_id], + source_data_ids=["report:order_130", brief.frame_id, "boundary:order_130"], + ) + + gate = data_store.require_record("node_4:gatekeeper_frame").payload + assert gate["gate_status"] == "pass" + + +def _brief( + document_material_items: list[Node0DocumentMaterialItem], + *, + actual_tool_read_doc_count: int = 0, + actual_tool_read_doc_documents: list[str] | None = None, +) -> Node3InputBriefFrame: + supplied_count = sum( + 1 for item in document_material_items if item.was_supplied_document_context + ) + return Node3InputBriefFrame( + frame_id="node_3:input_brief_frame", + turn_id="turn_order_130", + user_question="문서 역할을 구분해줘", + brief_status="ready", + handoff_frame_id="node_2:handoff_frame", + actual_tool_read_doc_count=actual_tool_read_doc_count, + actual_tool_read_doc_documents=actual_tool_read_doc_documents or [], + supplied_document_context_count=supplied_count, + document_material_packet_frame_id="node_0:document_material_packet_frame", + document_material_items=document_material_items, + source_trace_ids=["trace_order_130"], + source_data_ids=[ + "node_2:handoff_frame", + "node_0:document_material_packet_frame", + ], + ) + + +def _stores() -> tuple[TraceStore, DataStore, str]: + trace_store = TraceStore() + data_store = DataStore() + event = trace_store.create_event( + turn_id="turn_order_130", + actor="test", + event_type="node_output", + output_ref=["seed"], + schema_status="passed", + ) + return trace_store, data_store, event.event_id diff --git a/tests/test_order_131_search_candidate_scope_split.py b/tests/test_order_131_search_candidate_scope_split.py new file mode 100644 index 0000000..15cb83c --- /dev/null +++ b/tests/test_order_131_search_candidate_scope_split.py @@ -0,0 +1,165 @@ +from __future__ import annotations + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.schemas import MetainfoBoundary +from songryeon_core.core.trace_store import TraceStore +from songryeon_core.nodes.node_0_memory_supplier import ( + NODE0_DOCUMENT_MATERIAL_PACKET_FRAME_DATA_ID, + record_node0_document_material_packet, +) +from songryeon_core.nodes.node_2_handoff import ( + node3_brief_llm_payload, + record_node3_input_brief, +) +from songryeon_core.nodes.node_3_reporter import build_node3_grounding_block + + +def test_node3_brief_splits_final_and_accumulated_search_candidate_scopes() -> None: + trace_store, data_store, trace_id = _stores() + _record_l_return_summary( + data_store, + trace_id, + search_result_doc_ids=["docs/final/A.md", "docs/final/B.md"], + ) + _record_l3_preserved_frame( + data_store, + trace_id, + data_id="L3:preserved_info_frame", + doc_ids=["docs/a/README.md", "docs/b/README.md"], + ) + _record_l3_preserved_frame( + data_store, + trace_id, + data_id="L3:revision_preserved_info_frame:0001", + doc_ids=["docs/final/A.md", "docs/c/C.md"], + ) + _, material_id, _ = record_node0_document_material_packet( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_131", + frame_id=NODE0_DOCUMENT_MATERIAL_PACKET_FRAME_DATA_ID, + source_trace_ids=[trace_id], + source_data_ids=["L:return_summary_frame"], + ) + + _, _, brief = record_node3_input_brief( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_131", + user_question="검색 후보 count를 구분해줘", + handoff_frame_id="node_2:handoff_frame", + boundary=MetainfoBoundary(), + input_trace_ids=[trace_id], + source_data_ids=[ + "node_2:handoff_frame", + "L:return_summary_frame", + "L3:preserved_info_frame", + "L3:revision_preserved_info_frame:0001", + material_id, + ], + ) + payload = node3_brief_llm_payload(brief) + grounding = build_node3_grounding_block(brief) + + assert brief.final_search_candidate_count == 2 + assert brief.final_search_candidate_documents == ["A.md", "B.md"] + assert brief.search_candidate_count == brief.final_search_candidate_count + assert brief.search_candidate_documents == brief.final_search_candidate_documents + assert brief.accumulated_search_candidate_count == 4 + assert brief.accumulated_search_candidate_documents == [ + "docs/a/README.md", + "docs/b/README.md", + "A.md", + "C.md", + ] + assert payload["search_candidate_scope"]["final_search_candidate"]["count"] == 2 + assert payload["search_candidate_scope"]["accumulated_search_candidate"]["count"] == 4 + assert "- 검색 후보 문서(최종): 2개" in grounding + assert "- 검색 후보 문서(누적): 4개" in grounding + + +def test_node3_brief_final_search_candidates_fallback_to_return_summary_without_material_packet() -> None: + trace_store, data_store, trace_id = _stores() + _record_l_return_summary( + data_store, + trace_id, + search_result_doc_ids=["docs/final/A.md", "docs/final/B.md"], + ) + + _, _, brief = record_node3_input_brief( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_131", + user_question="material packet이 없어도 최종 후보를 세어줘", + handoff_frame_id="node_2:handoff_frame", + boundary=MetainfoBoundary(), + input_trace_ids=[trace_id], + source_data_ids=["node_2:handoff_frame", "L:return_summary_frame"], + ) + + assert brief.final_search_candidate_count == 2 + assert brief.final_search_candidate_documents == ["A.md", "B.md"] + assert brief.accumulated_search_candidate_count == 0 + + +def _stores() -> tuple[TraceStore, DataStore, str]: + trace_store = TraceStore() + data_store = DataStore() + event = trace_store.create_event( + turn_id="turn_order_131", + actor="test", + event_type="node_output", + output_ref=["node_2:handoff_frame"], + schema_status="passed", + ) + data_store.create_record( + data_id="node_2:handoff_frame", + data_type="test:handoff", + source_trace_id=event.event_id, + payload={"frame_id": "node_2:handoff_frame"}, + ) + return trace_store, data_store, event.event_id + + +def _record_l_return_summary( + data_store: DataStore, + trace_id: str, + *, + search_result_doc_ids: list[str], +) -> None: + data_store.create_record( + data_id="L:return_summary_frame", + data_type="node_output:l_loop_return_summary_frame", + source_trace_id=trace_id, + payload={ + "frame_id": "L:return_summary_frame", + "turn_id": "turn_order_131", + "search_result_doc_ids": search_result_doc_ids, + "search_candidate_count": len(search_result_doc_ids), + }, + ) + + +def _record_l3_preserved_frame( + data_store: DataStore, + trace_id: str, + *, + data_id: str, + doc_ids: list[str], +) -> None: + data_store.create_record( + data_id=data_id, + data_type="node_output:L3_preserved_info_frame", + source_trace_id=trace_id, + payload={ + "frame_id": data_id, + "turn_id": "turn_order_131", + "candidates": [ + { + "candidate_id": f"candidate:{index}", + "doc_id": doc_id, + } + for index, doc_id in enumerate(doc_ids, start=1) + ], + }, + ) diff --git a/tests/test_order_132_material_delivery_policy.py b/tests/test_order_132_material_delivery_policy.py new file mode 100644 index 0000000..1c48de7 --- /dev/null +++ b/tests/test_order_132_material_delivery_policy.py @@ -0,0 +1,204 @@ +from __future__ import annotations + +from dataclasses import asdict +import json + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.schemas import ( + L3PerDocumentSummaryFrame, + MetainfoBoundary, + Node2AnswerBasisFrame, +) +from songryeon_core.core.trace_store import TraceStore +from songryeon_core.nodes.node_2_handoff import ( + node3_brief_llm_payload, + record_node3_input_brief, +) +from songryeon_core.nodes.node_3_reporter import build_node3_grounding_block + + +def test_relative_allowed_replaces_raw_text_with_l3_summary_payload() -> None: + trace_store, data_store, trace_id = _stores_with_raw_document() + _record_l3_summary(data_store, trace_id) + answer_basis = _answer_basis_frame("relative_allowed") + + _, _, brief = record_node3_input_brief( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_132", + user_question="요약 기준으로 설명해줘", + handoff_frame_id="source:handoff", + boundary=MetainfoBoundary(), + input_trace_ids=[trace_id], + source_data_ids=["source:handoff"], + answer_basis_frame=answer_basis, + ) + payload = node3_brief_llm_payload(brief) + payload_text = json.dumps(payload, ensure_ascii=False) + + assert brief.material_delivery_mode == "l3_summary_replaces_raw_context" + assert brief.raw_document_policy == "omit_raw_text_from_llm_payload" + assert brief.llm_raw_document_text_count == 0 + assert brief.llm_l3_summary_context_count == 1 + assert brief.raw_context_replaced_by_summary_count == 1 + assert payload["supplied_document_contexts"] == [] + assert payload["read_documents"] == [] + assert len(payload["omitted_supplied_document_contexts"]) == 1 + assert "원문 전체 텍스트 payload" not in payload_text + assert "L3 task 요약" in payload_text + + raw_record = data_store.require_record("tool_result:read_doc:001") + assert isinstance(raw_record.payload, dict) + assert raw_record.payload["text"] == "원문 전체 텍스트 payload" + + +def test_absolute_first_preserves_raw_text_even_when_l3_summary_exists() -> None: + trace_store, data_store, trace_id = _stores_with_raw_document() + _record_l3_summary(data_store, trace_id) + + _, _, brief = record_node3_input_brief( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_132", + user_question="원문 기준으로 확인해줘", + handoff_frame_id="source:handoff", + boundary=MetainfoBoundary(), + input_trace_ids=[trace_id], + source_data_ids=["source:handoff"], + answer_basis_frame=_answer_basis_frame("absolute_first"), + ) + payload = node3_brief_llm_payload(brief) + + assert brief.material_delivery_mode == "raw_document_primary" + assert brief.llm_raw_document_text_count == 1 + assert brief.raw_context_replaced_by_summary_count == 0 + assert payload["supplied_document_contexts"][0]["text"] == "원문 전체 텍스트 payload" + assert payload["material_delivery_policy"]["l3_summary_policy"] == "auxiliary_only" + + +def test_mixed_mode_without_l3_summary_falls_back_to_raw_text() -> None: + trace_store, data_store, trace_id = _stores_with_raw_document() + + _, _, brief = record_node3_input_brief( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_132", + user_question="불확실성을 드러내줘", + handoff_frame_id="source:handoff", + boundary=MetainfoBoundary(), + input_trace_ids=[trace_id], + source_data_ids=["source:handoff"], + answer_basis_frame=_answer_basis_frame("mixed_or_uncertain"), + ) + payload = node3_brief_llm_payload(brief) + + assert brief.material_delivery_mode == "raw_document_fallback_no_l3_summary" + assert brief.raw_document_policy == "preserve_raw_context_because_l3_summary_missing" + assert brief.llm_raw_document_text_count == 1 + assert payload["supplied_document_contexts"][0]["text"] == "원문 전체 텍스트 payload" + + +def test_grounding_reports_llm_raw_text_and_l3_summary_counts() -> None: + trace_store, data_store, trace_id = _stores_with_raw_document() + _record_l3_summary(data_store, trace_id) + + _, _, brief = record_node3_input_brief( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_132", + user_question="요약 기준으로 설명해줘", + handoff_frame_id="source:handoff", + boundary=MetainfoBoundary(), + input_trace_ids=[trace_id], + source_data_ids=["source:handoff"], + answer_basis_frame=_answer_basis_frame("relative_allowed"), + ) + grounding = build_node3_grounding_block(brief) + + assert "- node_3 공급 문서 context: 1개" in grounding + assert "- node_3 LLM 원문 text: 0개" in grounding + assert "- L3 문서별 요약 재료: 1개" in grounding + assert "L3 요약이 원문 text 대체" in grounding + + +def _stores_with_raw_document() -> tuple[TraceStore, DataStore, str]: + trace_store = TraceStore() + data_store = DataStore() + event = trace_store.create_event( + turn_id="turn_order_132", + actor="test", + event_type="node_output", + output_ref=["source:handoff", "tool_result:read_doc:001"], + schema_status="passed", + ) + data_store.create_record( + data_id="source:handoff", + data_type="test:handoff", + source_trace_id=event.event_id, + payload={"frame_id": "source:handoff"}, + ) + data_store.create_record( + data_id="tool_result:read_doc:001", + data_type="tool_result:read_doc", + source_trace_id=event.event_id, + payload={ + "doc_id": "Administrative_Reform_1/04_Orders/ORDER_132_TEST.md", + "text": "원문 전체 텍스트 payload", + "char_count": 14, + }, + ) + return trace_store, data_store, event.event_id + + +def _record_l3_summary(data_store: DataStore, trace_id: str) -> None: + frame = L3PerDocumentSummaryFrame( + frame_id="L3:per_document_summary:0001", + turn_id="turn_order_132", + source_document_data_id="tool_result:read_doc:001", + source_doc_id="Administrative_Reform_1/04_Orders/ORDER_132_TEST.md", + source_document_name="ORDER_132_TEST.md", + source_char_count=14, + summary_status="ran", + plain_document_summary="L3 plain 요약", + plain_summary_source_data_id="tool_result:read_doc:001", + task_relevant_summary="L3 task 요약", + task_relevant_summary_source_data_ids=[ + "tool_result:read_doc:001", + "L1:goal_frame", + "L3:preserved_info_frame", + ], + generated_by="LLM:test", + semantic_judgement_status="ran", + summary_failure_type="none", + llm_call_data_id="llm_call:L3_document_summary:trace_000010", + prompt_ref="songryeon_core/prompts/l3_per_document_summary_v0.md", + source_trace_ids=[trace_id], + source_data_ids=[ + "tool_result:read_doc:001", + "L1:goal_frame", + "L3:preserved_info_frame", + "llm_call:L3_document_summary:trace_000010", + ], + ) + data_store.create_record( + data_id=frame.frame_id, + data_type="node_output:L3_per_document_summary_frame", + source_trace_id=trace_id, + payload=asdict(frame), + ) + + +def _answer_basis_frame(mode: str) -> Node2AnswerBasisFrame: + return Node2AnswerBasisFrame( + frame_id="node_2:answer_basis_frame", + turn_id="turn_order_132", + answer_basis_mode=mode, + basis_reason_codes=["document_basis_present"], + mode_selection_reason="테스트용 answer_basis_mode", + mode_selection_reason_info_class="mixed", + generated_by="LLM:test", + info_class="mixed", + semantic_judgement_status="ran", + source_trace_ids=["trace_answer_basis"], + source_data_ids=["node_2:answer_basis_frame"], + ) diff --git a/tests/test_order_133_codebase_readonly_inspection.py b/tests/test_order_133_codebase_readonly_inspection.py new file mode 100644 index 0000000..7a4dd8c --- /dev/null +++ b/tests/test_order_133_codebase_readonly_inspection.py @@ -0,0 +1,183 @@ +from __future__ import annotations + +import json + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.schemas import MemoryPacketFrom0 +from songryeon_core.core.trace_store import TraceStore +from songryeon_core.llm.base import LLMRequest, LLMResponse +from songryeon_core.loops.l_loop import run_l_loop +from songryeon_core.nodes.l2_query_setter import run_l2_query_setter +from songryeon_core.nodes.node_2_handoff import record_node3_input_brief +from songryeon_core.core.schemas import MetainfoBoundary +from songryeon_core.tools.code_tools import list_code_files, read_code_file, search_code +from songryeon_core.tools.tool_runner import build_document_tool_registry + + +class CodeReadQueryPlannerAdapter: + model_id = "order-133-code-read-query-planner" + + def complete(self, request: LLMRequest) -> LLMResponse: + source_data_ids = request.input_payload.get("attribution_source_data_ids") + if not isinstance(source_data_ids, list) or not source_data_ids: + source_data_ids = ["L1:goal_frame"] + payload = { + "planner_mode": "llm", + "selected_candidate_id": "L2:query_candidate_0001", + "candidates": [ + { + "candidate_id": "L2:query_candidate_0001", + "query_text": "songryeon_core/tools/code_tools.py", + "purpose": "사용자가 코드 구조 읽기 MVP를 요청했으므로 실제 source file을 읽는다.", + "expected_signal": "read_code_file로 source text가 복사된다.", + "priority": 1, + "target_tool_name": "read_code_file", + "source_data_ids": source_data_ids, + } + ], + } + return LLMResponse( + text=json.dumps(payload, ensure_ascii=False), + model_id=self.model_id, + raw=payload, + ) + + +class CodeInspectionScopeAdapter: + model_id = "order-133-code-inspection-scope-adapter" + + def complete(self, request: LLMRequest) -> LLMResponse: + payload = { + "tool_scope_mode": "code_only", + "allowed_tool_groups": ["code_inspection_tools"], + "required_materials": ["source_code_file"], + "scope_reason": "test adapter explicitly opens code inspection tools.", + "scope_reason_info_class": "mixed", + } + return LLMResponse( + text=json.dumps(payload, ensure_ascii=False), + model_id=self.model_id, + raw=payload, + ) + + +def test_code_tools_list_search_read_and_block_path_escape(tmp_path) -> None: + source_dir = tmp_path / "pkg" + source_dir.mkdir() + source_file = source_dir / "sample.py" + source_file.write_text("class Sample:\n pass\n", encoding="utf-8") + ignored_dir = tmp_path / "__pycache__" + ignored_dir.mkdir() + (ignored_dir / "hidden.py").write_text("SHOULD_NOT_APPEAR = True\n", encoding="utf-8") + + listed = list_code_files(root=tmp_path) + listed_paths = [item["file_path"] for item in listed["files"]] + assert "pkg/sample.py" in listed_paths + assert "__pycache__/hidden.py" not in listed_paths + + searched = search_code(root=tmp_path, query="class Sample") + assert searched["match_count"] == 1 + assert searched["results"][0]["file_path"] == "pkg/sample.py" + assert searched["results"][0]["line_number"] == 1 + + read = read_code_file(root=tmp_path, file_path="pkg/sample.py") + assert read["read_status"] == "ok" + assert read["line_count"] == 2 + assert "class Sample" in read["text"] + + escaped = read_code_file(root=tmp_path, file_path="../outside.py") + assert escaped["read_status"] == "path_outside_workspace_rejected" + assert escaped["text"] == "" + + +def test_tool_registry_exposes_code_tools_as_read_only(tmp_path) -> None: + registry = build_document_tool_registry(tmp_path, code_root=tmp_path) + + for tool_name in ("list_code_files", "search_code", "read_code_file"): + spec = registry.get(tool_name) + assert spec.read_only is True + assert spec.output_data_type == f"tool_result:{tool_name}" + + +def test_l2_query_setter_accepts_code_tool_modes() -> None: + trace_store = TraceStore() + data_store = DataStore() + l1_event = trace_store.create_event( + turn_id="turn_order_133", + actor="L1", + event_type="node_output", + output_ref=["L1:goal_frame"], + schema_status="passed", + ) + + run_l2_query_setter( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_133", + l1_event=l1_event, + query_text="songryeon_core/tools/code_tools.py", + query_source="llm_query_plan", + target_tool_name="read_code_file", + source_data_ids=["L1:goal_frame"], + ) + + payload = data_store.require_record("L2:query_frame").payload + assert isinstance(payload, dict) + assert payload["target_tool_name"] == "read_code_file" + assert payload["query_mode"] == "code_file_read" + + +def test_l_loop_runs_read_code_file_and_node3_brief_receives_source_context() -> None: + trace_store = TraceStore() + data_store = DataStore() + memory_packet = MemoryPacketFrom0(target="L", trace_evidence_ids=[]) + + result = run_l_loop( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_133", + memory_packet=memory_packet, + search_query="코드 구조 읽기 MVP source file을 읽어줘", + l_tool_scope_adapter=CodeInspectionScopeAdapter(), + l2_query_planner_adapter=CodeReadQueryPlannerAdapter(), + max_tool_calls=3, + max_query_attempts=2, + max_read_doc_calls=1, + ) + + code_records = [ + record + for record in data_store.list_records() + if record.data_type == "tool_result:read_code_file" + ] + assert len(code_records) == 1 + code_payload = code_records[0].payload + assert isinstance(code_payload, dict) + assert code_payload["read_status"] == "ok" + assert code_payload["file_path"] == "songryeon_core/tools/code_tools.py" + + seed = trace_store.create_event( + turn_id="turn_order_133", + actor="test", + event_type="node_output", + output_ref=["node_2:handoff_frame"], + schema_status="passed", + ) + _, _, brief = record_node3_input_brief( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_133", + user_question="코드 구조 읽기 MVP source file을 읽어줘", + handoff_frame_id="node_2:handoff_frame", + boundary=MetainfoBoundary(), + input_trace_ids=[seed.event_id, *result.source_trace_ids], + source_data_ids=["node_2:handoff_frame", *result.output_data_ids], + ) + + source_contexts = [ + document + for document in brief.read_documents + if document.document_name == "songryeon_core/tools/code_tools.py" + ] + assert len(source_contexts) == 1 + assert "def read_code_file" in source_contexts[0].text diff --git a/tests/test_order_134_l_tool_scope_budget_partition.py b/tests/test_order_134_l_tool_scope_budget_partition.py new file mode 100644 index 0000000..d5895eb --- /dev/null +++ b/tests/test_order_134_l_tool_scope_budget_partition.py @@ -0,0 +1,308 @@ +from __future__ import annotations + +import json + +import pytest + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.trace_store import TraceStore +from songryeon_core.llm.base import LLMRequest, LLMResponse +from songryeon_core.nodes.l2_query_setter import run_l2_query_planner +from songryeon_core.nodes.l_tool_scope import ( + filter_available_tools_for_scope, + record_l_tool_budget_partition, + run_l_tool_scope_planner, +) + + +class PayloadAdapter: + model_id = "order-134-payload-adapter" + + def __init__(self, payload: dict[str, object]) -> None: + self.payload = payload + self.last_input_payload: dict[str, object] | None = None + + def complete(self, request: LLMRequest) -> LLMResponse: + self.last_input_payload = request.input_payload + return LLMResponse( + text=json.dumps(self.payload, ensure_ascii=False), + model_id=self.model_id, + raw=self.payload, + ) + + +def test_l_tool_scope_llm_selects_document_and_code_and_budget_splits() -> None: + trace_store, data_store, l1_event = _seed_l_scope_stores() + scope_adapter = PayloadAdapter(_scope_payload("document_and_code")) + + scope_trace_id, scope_data_id, scope_frame = run_l_tool_scope_planner( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_134", + l1_event=l1_event, + user_query="ORDER_133과 실제 source code를 둘 다 확인해줘", + goal_data_id="L1:goal_frame", + budget_plan_data_id="L:budget_plan_frame", + tool_catalog_data_id="tool_catalog:turn_order_134", + available_tools=_all_tool_catalog_items(), + adapter=scope_adapter, + ) + + assert scope_data_id == "L:tool_scope_frame" + assert scope_frame.generated_by == "LLM:order-134-payload-adapter" + assert scope_frame.info_class == "mixed" + assert scope_frame.tool_scope_mode == "document_and_code" + assert scope_frame.allowed_tool_groups == ["document_tools", "code_inspection_tools"] + + _, _, partition = record_l_tool_budget_partition( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_134", + tool_scope_frame=scope_frame, + tool_scope_trace_id=scope_trace_id, + budget_plan_data_id="L:budget_plan_frame", + budget_plan_trace_id=l1_event.event_id, + ) + + assert partition.document_tool_call_budget > 0 + assert partition.code_tool_call_budget > 0 + assert partition.document_query_budget > 0 + assert partition.code_query_budget > 0 + assert partition.document_read_budget > 0 + assert partition.code_read_budget > 0 + assert partition.generated_by == "CODE:L_TOOL_BUDGET_PARTITION_POLICY" + assert partition.semantic_judgement_status == "not_run" + + +def test_l_tool_scope_fallback_is_explicit_failed_document_only() -> None: + trace_store, data_store, l1_event = _seed_l_scope_stores() + + _, _, frame = run_l_tool_scope_planner( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_134", + l1_event=l1_event, + user_query="anything", + goal_data_id="L1:goal_frame", + budget_plan_data_id="L:budget_plan_frame", + tool_catalog_data_id="tool_catalog:turn_order_134", + available_tools=_all_tool_catalog_items(), + adapter=None, + ) + + assert frame.tool_scope_mode == "document_only" + assert frame.allowed_tool_groups == ["document_tools"] + assert frame.generated_by == "CODE:FALLBACK" + assert frame.info_class == "absolute_status" + assert frame.semantic_judgement_status == "failed" + assert frame.scope_failure_type == "adapter_missing" + + +def test_tool_catalog_is_filtered_by_scope_groups() -> None: + scope = _scope_frame_dict( + mode="code_only", + groups=["code_inspection_tools"], + materials=["source_code_file"], + ) + trace_store, data_store, l1_event = _seed_l_scope_stores() + adapter = PayloadAdapter(scope) + _, _, frame = run_l_tool_scope_planner( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_134", + l1_event=l1_event, + user_query="actual code only", + goal_data_id="L1:goal_frame", + budget_plan_data_id="L:budget_plan_frame", + tool_catalog_data_id="tool_catalog:turn_order_134", + available_tools=_all_tool_catalog_items(), + adapter=adapter, + ) + + filtered = filter_available_tools_for_scope(_all_tool_catalog_items(), frame) + filtered_names = {str(item["tool_name"]) for item in filtered} + + assert filtered_names == {"list_code_files", "search_code", "read_code_file"} + + +def test_l2_receives_filtered_tools_and_scope_payload() -> None: + trace_store, data_store, l1_event = _seed_l_scope_stores() + planner_adapter = PayloadAdapter( + { + "planner_mode": "llm", + "selected_candidate_id": "L2:query_candidate_0001", + "candidates": [ + { + "candidate_id": "L2:query_candidate_0001", + "query_text": "songryeon_core/tools/code_tools.py", + "purpose": "scope가 허용한 code inspection tool로 source file을 읽는다.", + "expected_signal": "read_code_file source text", + "priority": 1, + "target_tool_name": "read_code_file", + "source_data_ids": ["L1:goal_frame"], + } + ], + } + ) + available_tools = [ + item for item in _all_tool_catalog_items() if item["tool_name"] in {"read_code_file"} + ] + + run_l2_query_planner( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_134", + l1_event=l1_event, + user_input="source file 읽기", + adapter=planner_adapter, + source_data_ids=["L1:goal_frame", "L:tool_scope_frame", "L:tool_budget_partition_frame"], + available_tools=available_tools, + l_tool_scope=_scope_frame_dict( + mode="code_only", + groups=["code_inspection_tools"], + materials=["source_code_file"], + ), + budget_partition={"code_read_budget": 1}, + ) + + assert planner_adapter.last_input_payload is not None + assert planner_adapter.last_input_payload["l_tool_scope"]["tool_scope_mode"] == "code_only" + available_names = { + str(item["tool_name"]) + for item in planner_adapter.last_input_payload["available_tools"] + if isinstance(item, dict) + } + assert available_names == {"read_code_file"} + plan_payload = data_store.require_record("L2:query_plan_frame").payload + assert isinstance(plan_payload, dict) + assert plan_payload["candidates"][0]["target_tool_name"] == "read_code_file" + + +def test_l2_rejects_candidate_outside_filtered_scope() -> None: + trace_store, data_store, l1_event = _seed_l_scope_stores() + planner_adapter = PayloadAdapter( + { + "planner_mode": "llm", + "selected_candidate_id": "L2:query_candidate_0001", + "candidates": [ + { + "candidate_id": "L2:query_candidate_0001", + "query_text": "문서 검색", + "purpose": "허용되지 않은 document tool을 고른 잘못된 후보.", + "expected_signal": "search docs result", + "priority": 1, + "target_tool_name": "search_docs", + "source_data_ids": ["L1:goal_frame"], + } + ], + } + ) + available_tools = [ + item for item in _all_tool_catalog_items() if item["tool_name"] == "read_code_file" + ] + + with pytest.raises(ValueError, match="schema_failed"): + run_l2_query_planner( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_134", + l1_event=l1_event, + user_input="source file 읽기", + adapter=planner_adapter, + source_data_ids=["L1:goal_frame", "L:tool_scope_frame"], + available_tools=available_tools, + l_tool_scope=_scope_frame_dict( + mode="code_only", + groups=["code_inspection_tools"], + materials=["source_code_file"], + ), + budget_partition={"code_read_budget": 1}, + ) + + +def _seed_l_scope_stores() -> tuple[TraceStore, DataStore, object]: + trace_store = TraceStore() + data_store = DataStore() + l1_event = trace_store.create_event( + turn_id="turn_order_134", + actor="L1", + event_type="node_output", + output_ref=["L1:goal_frame"], + schema_status="passed", + ) + data_store.create_record( + data_id="L1:goal_frame", + data_type="node_output:L1_goal_frame", + source_trace_id=l1_event.event_id, + payload={ + "frame_id": "L1:goal_frame", + "turn_id": "turn_order_134", + "macro_goal": "read order and source code", + "macro_goal_reason": "LLM selected mixed source bundle.", + "micro_goal": "prepare tool scope", + "micro_goal_reason": "tool groups must be explicit before L2", + "goal_source": "llm_l_route", + "target_loop": "L", + "evidence_requirement_kind": "multi_doc_relationship", + "minimum_read_documents": 2, + "requires_cross_document_analysis": True, + "randomness_mode": "not_random", + "l_loop_success_condition": "order and source materials are available", + }, + ) + data_store.create_record( + data_id="L:budget_plan_frame", + data_type="node_output:l_loop_budget_plan_frame", + source_trace_id=l1_event.event_id, + payload={ + "frame_id": "L:budget_plan_frame", + "turn_id": "turn_order_134", + "approved_max_tool_calls": 6, + "approved_max_query_attempts": 4, + "approved_max_read_doc_calls": 2, + }, + ) + data_store.create_record( + data_id="tool_catalog:turn_order_134", + data_type="tool_catalog", + source_trace_id=l1_event.event_id, + payload={"tools": _all_tool_catalog_items()}, + ) + return trace_store, data_store, l1_event + + +def _scope_payload(mode: str) -> dict[str, object]: + if mode == "document_and_code": + return _scope_frame_dict( + mode="document_and_code", + groups=["document_tools", "code_inspection_tools"], + materials=["order_document", "source_code_file", "code_search_result"], + ) + raise ValueError(f"unsupported test mode: {mode}") + + +def _scope_frame_dict( + *, + mode: str, + groups: list[str], + materials: list[str], +) -> dict[str, object]: + return { + "tool_scope_mode": mode, + "allowed_tool_groups": groups, + "required_materials": materials, + "scope_reason": "test adapter supplied explicit scope payload.", + "scope_reason_info_class": "mixed", + } + + +def _all_tool_catalog_items() -> list[dict[str, object]]: + return [ + {"tool_name": "list_docs", "read_only": True}, + {"tool_name": "read_doc", "read_only": True}, + {"tool_name": "read_artifact", "read_only": True}, + {"tool_name": "search_docs", "read_only": True}, + {"tool_name": "list_code_files", "read_only": True}, + {"tool_name": "search_code", "read_only": True}, + {"tool_name": "read_code_file", "read_only": True}, + ] diff --git a/tests/test_order_135_code_evidence_accounting.py b/tests/test_order_135_code_evidence_accounting.py new file mode 100644 index 0000000..fa3dbb5 --- /dev/null +++ b/tests/test_order_135_code_evidence_accounting.py @@ -0,0 +1,190 @@ +from __future__ import annotations + +import json + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.schemas import MemoryPacketFrom0, MetainfoBoundary, ZeroState +from songryeon_core.core.trace_store import TraceStore +from songryeon_core.llm.base import LLMRequest, LLMResponse +from songryeon_core.loops.l_loop import run_l_loop +from songryeon_core.nodes.node_0_memory_supplier import record_l_loop_return_summary_for_node1 +from songryeon_core.nodes.node_2_handoff import ( + node3_brief_llm_payload, + record_node3_input_brief, +) +from songryeon_core.nodes.node_3_reporter import build_node3_grounding_block + + +class CodeExactL1Adapter: + model_id = "order-135-code-l1" + + def complete(self, request: LLMRequest) -> LLMResponse: + payload = { + "macro_goal": "songryeon_core/tools/code_tools.py source file을 읽어 node_3가 답할 수 있게 한다.", + "macro_goal_reason": "사용자가 실제 코드 원문 기준 답변을 요구했다.", + "micro_goal": "songryeon_core/tools/code_tools.py를 read_code_file로 읽는다.", + "micro_goal_reason": "정확한 source path가 주어졌다.", + "evidence_requirement_kind": "exact_artifact_lookup", + "minimum_read_documents": 1, + "requires_cross_document_analysis": False, + "randomness_mode": "not_random", + "l_loop_success_condition": "requested source file text is read", + "requested_search_top_k": 1, + "requested_max_tool_calls": 1, + "requested_max_read_doc_calls": 1, + "requested_max_query_attempts": 1, + "budget_request_reason": "one exact source read is enough", + } + return LLMResponse( + text=json.dumps(payload, ensure_ascii=False), + model_id=self.model_id, + raw=payload, + ) + + +class CodeOnlyScopeAdapter: + model_id = "order-135-code-scope" + + def complete(self, request: LLMRequest) -> LLMResponse: + payload = { + "tool_scope_mode": "code_only", + "allowed_tool_groups": ["code_inspection_tools"], + "required_materials": ["source_code_file"], + "scope_reason": "source-code file evidence is required.", + "scope_reason_info_class": "mixed", + } + return LLMResponse( + text=json.dumps(payload, ensure_ascii=False), + model_id=self.model_id, + raw=payload, + ) + + +class ReadCodeFileL2Adapter: + model_id = "order-135-code-l2" + + def complete(self, request: LLMRequest) -> LLMResponse: + source_data_ids = request.input_payload.get("attribution_source_data_ids") + if not isinstance(source_data_ids, list) or not source_data_ids: + source_data_ids = ["L1:goal_frame"] + payload = { + "planner_mode": "llm", + "selected_candidate_id": "L2:query_candidate_0001", + "candidates": [ + { + "candidate_id": "L2:query_candidate_0001", + "query_text": "songryeon_core/tools/code_tools.py", + "purpose": "read the exact source file requested by the user.", + "expected_signal": "read_code_file returns source text.", + "priority": 1, + "target_tool_name": "read_code_file", + "source_data_ids": source_data_ids, + } + ], + } + return LLMResponse( + text=json.dumps(payload, ensure_ascii=False), + model_id=self.model_id, + raw=payload, + ) + + +def test_read_code_file_counts_as_source_code_evidence_without_becoming_read_doc() -> None: + trace_store = TraceStore() + data_store = DataStore() + memory_packet = MemoryPacketFrom0(target="L", trace_evidence_ids=[]) + user_query = ( + "songryeon_core/tools/code_tools.py 파일을 직접 읽고 read-only code inspection 기능을 정리해줘." + ) + + result = run_l_loop( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_135", + memory_packet=memory_packet, + search_query=user_query, + l1_goal_adapter=CodeExactL1Adapter(), + l_tool_scope_adapter=CodeOnlyScopeAdapter(), + l2_query_planner_adapter=ReadCodeFileL2Adapter(), + max_tool_calls=3, + max_query_attempts=2, + max_read_doc_calls=1, + ) + + achievement_payload = data_store.require_record("L3:achievement_frame").payload + assert isinstance(achievement_payload, dict) + assert achievement_payload["achievement_status"] == "achieved" + assert achievement_payload["goal_match_status"] == "matched" + assert achievement_payload["goal_match_reason"] == ( + "CODE_STATUS:requested_source_code_file_read_code_file_matched" + ) + assert achievement_payload["read_doc_ids"] == [] + assert achievement_payload["read_code_file_paths"] == ["songryeon_core/tools/code_tools.py"] + assert achievement_payload["actual_read_code_file_count"] == 1 + + _, _, return_summary_id, _ = record_l_loop_return_summary_for_node1( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_135", + zero_state=ZeroState(), + input_ref=result.source_trace_ids, + source_data_ids=result.output_data_ids, + ) + return_summary = data_store.require_record(return_summary_id).payload + assert isinstance(return_summary, dict) + assert return_summary["actual_read_doc_count"] == 0 + assert return_summary["actual_read_code_file_count"] == 1 + assert return_summary["read_code_file_paths"] == ["songryeon_core/tools/code_tools.py"] + assert return_summary["failure_level"] == "none" + + seed = trace_store.create_event( + turn_id="turn_order_135", + actor="test", + event_type="node_output", + output_ref=["node_2:handoff_frame"], + schema_status="passed", + ) + _, _, brief = record_node3_input_brief( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_135", + user_question=user_query, + handoff_frame_id="node_2:handoff_frame", + boundary=MetainfoBoundary(), + input_trace_ids=[seed.event_id, *result.source_trace_ids], + source_data_ids=["node_2:handoff_frame", return_summary_id, *result.output_data_ids], + ) + + assert brief.actual_tool_read_doc_count == 0 + assert brief.actual_tool_read_code_file_count == 1 + assert brief.actual_tool_read_code_file_paths == ["songryeon_core/tools/code_tools.py"] + assert brief.supplied_source_code_context_count == 1 + assert len(brief.source_code_outlines) == 1 + outline = brief.source_code_outlines[0] + assert outline.file_path == "songryeon_core/tools/code_tools.py" + assert outline.parse_status == "parsed" + assert { + "list_code_files", + "search_code", + "read_code_file", + }.issubset(set(outline.public_function_names)) + top_level_symbols = {symbol.name for symbol in outline.top_level_symbols} + assert "DEFAULT_CODE_FILE_EXTENSIONS" in top_level_symbols + assert "DEFAULT_IGNORED_DIR_NAMES" in top_level_symbols + assert any( + document.document_name == "songryeon_core/tools/code_tools.py" + and "def read_code_file" in document.text + for document in brief.read_documents + ) + + payload = node3_brief_llm_payload(brief) + outline_payload = payload["source_code_outlines"] + assert outline_payload["count"] == 1 + assert outline_payload["items"][0]["public_function_names"] == outline.public_function_names + assert "source_data_id" not in outline_payload["items"][0] + + grounding = build_node3_grounding_block(brief) + assert "실제 read_doc 도구 원문 읽기: 0개" in grounding + assert "실제 read_code_file 도구 원문 읽기: 1개" in grounding + assert "node_3 공급 source-code context: 1개" in grounding + assert "source-code 구조 목록: 1개" in grounding diff --git a/tests/test_order_139_graph_memory_foundation.py b/tests/test_order_139_graph_memory_foundation.py new file mode 100644 index 0000000..20a860c --- /dev/null +++ b/tests/test_order_139_graph_memory_foundation.py @@ -0,0 +1,200 @@ +from __future__ import annotations + +from dataclasses import asdict + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.graph_memory import ( + CORE_EGO_ROOT_NODE_ID, + TIME_AXIS_NODE_ID, + build_graph_memory_snapshot_from_capsules, + raw_capsule_graph_node_id, + record_graph_memory_for_capsules, +) +from songryeon_core.core.schemas import NodeMovement, TurnStateCapsule +from songryeon_core.core.trace_store import TraceStore +from songryeon_core.runtime.dry_run import run_dry_turn + + +def test_duplicate_capsule_ingest_creates_one_raw_graph_node() -> None: + capsule = _sample_capsule() + + build = build_graph_memory_snapshot_from_capsules( + capsules=[capsule, capsule], + batch_id="batch_order_139", + ) + + raw_nodes = [node for node in build.nodes if node.node_kind == "raw_capsule"] + assert len(raw_nodes) == 1 + assert raw_nodes[0].node_id == raw_capsule_graph_node_id(capsule.turn_id) + + trace_store = TraceStore() + data_store = DataStore() + first = record_graph_memory_for_capsules( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_139", + capsules=[capsule, capsule], + batch_id="batch_order_139", + ) + second = record_graph_memory_for_capsules( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_139", + capsules=[capsule, capsule], + batch_id="batch_order_139", + ) + + raw_records = [ + record + for record in data_store.list_records() + if record.data_type == "graph_memory:node:raw_capsule" + ] + assert len(raw_records) == 1 + assert first.created_data_ids + assert not second.created_data_ids + assert raw_records[0].data_id == raw_capsule_graph_node_id(capsule.turn_id) + + +def test_raw_capsule_node_contains_absolute_coordinates_only() -> None: + capsule = _sample_capsule() + + build = build_graph_memory_snapshot_from_capsules( + capsules=[capsule], + batch_id="batch_order_139", + ) + raw_node = next(node for node in build.nodes if node.node_kind == "raw_capsule") + payload = asdict(raw_node) + + assert payload["source_turn_id"] == capsule.turn_id + assert payload["trace_count"] == len(capsule.trace_event_ids) + assert payload["movement_count"] == len(capsule.node_movements) + assert payload["user_input_trace_id"] == capsule.user_input_trace_id + assert payload["final_response_trace_id"] == capsule.final_response_trace_id + assert payload["generated_by"] == "CODE:GRAPH_MEMORY_BUILDER" + assert payload["info_class"] == "absolute" + assert payload["semantic_judgement_status"] == "not_run" + for forbidden_field in ( + "memory_text", + "summary_text", + "raw_user_text", + "raw_assistant_text", + "semantic_topic", + "topic_label", + "importance_reason", + "relevance_reason", + ): + assert forbidden_field not in payload + + +def test_core_ego_time_axis_edges_are_created_without_semantic_axis() -> None: + capsule = _sample_capsule() + + build = build_graph_memory_snapshot_from_capsules( + capsules=[capsule], + batch_id="batch_order_139", + ) + + edge_tuples = { + (edge.edge_kind, edge.from_node_id, edge.to_node_id) + for edge in build.edges + } + bundle_node = next(node for node in build.nodes if node.node_kind == "time_bundle") + raw_node_id = raw_capsule_graph_node_id(capsule.turn_id) + + assert ("CONTAINS", CORE_EGO_ROOT_NODE_ID, TIME_AXIS_NODE_ID) in edge_tuples + assert ("CHILD_OF_TIME_AXIS", TIME_AXIS_NODE_ID, bundle_node.node_id) in edge_tuples + assert ("CONTAINS", bundle_node.node_id, raw_node_id) in edge_tuples + assert all(node.node_kind != "semantic_topic" for node in build.nodes) + assert build.core_ego_time_axis.semantic_axis_status == "not_created" + + +def test_raw_capsule_summary_depth_and_source_counts_are_zero_depth_leaf() -> None: + build = build_graph_memory_snapshot_from_capsules( + capsules=[_sample_capsule()], + batch_id="batch_order_139", + ) + raw_node = next(node for node in build.nodes if node.node_kind == "raw_capsule") + + assert raw_node.summary_depth == 0 + assert raw_node.source_depth_min == 0 + assert raw_node.source_depth_max == 0 + assert raw_node.source_leaf_count == 1 + assert raw_node.source_summary_count == 0 + assert raw_node.source_bundle_kind == "raw_leaf" + + +def test_rloop_graph_guide_packet_records_counts_ranges_and_sources() -> None: + capsule = _sample_capsule() + + build = build_graph_memory_snapshot_from_capsules( + capsules=[capsule], + batch_id="batch_order_139", + ) + guide = build.guide_packet + raw_node_id = raw_capsule_graph_node_id(capsule.turn_id) + + assert guide.graph_snapshot_id == build.snapshot.snapshot_id + assert guide.target_consumer == "R_LOOP" + assert guide.available_entry_nodes == [TIME_AXIS_NODE_ID] + assert guide.node_kind_counts["raw_capsule"] == 1 + assert guide.node_kind_counts["time_bundle"] == 1 + assert guide.data_kind_counts["turn_state_capsule"] == 1 + assert guide.summary_depth_range == [0, 0] + assert guide.source_leaf_count_range == [1, 1] + assert raw_node_id in guide.source_graph_node_ids + assert build.snapshot.snapshot_id in guide.source_data_ids + assert set(capsule.trace_event_ids).issubset(set(guide.source_trace_ids)) + assert guide.risky_or_unreviewed_node_ids == [] + assert guide.generated_by == "CODE:GRAPH_MEMORY_GUIDE_BUILDER" + assert guide.info_class == "absolute" + assert guide.semantic_judgement_status == "not_run" + assert guide.recommended_traversal_hints_status == "not_run" + assert guide.recommended_traversal_hints == [] + + +def test_runtime_graph_guide_is_not_injected_into_node1_or_node3() -> None: + result = run_dry_turn() + guide_id = result["rloop_graph_guide_packet_id"] + + assert result["rloop_graph_guide_hints_status"] == "not_run" + assert result["rloop_graph_guide_semantic_judgement_status"] == "not_run" + assert result["graph_memory_raw_capsule_node_count"] >= 1 + + guide_records = [ + record + for record in result["data_records"] + if record["data_type"] == "graph_memory:rloop_guide_packet" + ] + assert len(guide_records) == 1 + assert guide_records[0]["data_id"] == guide_id + + for record in result["data_records"]: + if record["data_type"] not in {"node_output:routing_decision", "node_output:report"}: + continue + source_data_ids = record["payload"].get("source_data_ids", []) + assert guide_id not in source_data_ids + + +def _sample_capsule(turn_id: str = "turn_order_139_previous") -> TurnStateCapsule: + return TurnStateCapsule( + turn_id=turn_id, + node_movements=[ + NodeMovement( + movement_id=f"move:{turn_id}:001", + turn_id=turn_id, + step_index=1, + node_id="node_0", + mode="pre_route_report", + input_trace_ids=[f"trace:{turn_id}:user"], + output_trace_ids=[f"trace:{turn_id}:node0"], + status="completed", + ) + ], + trace_event_ids=[ + f"trace:{turn_id}:user", + f"trace:{turn_id}:node0", + f"trace:{turn_id}:final", + ], + user_input_trace_id=f"trace:{turn_id}:user", + final_response_trace_id=f"trace:{turn_id}:final", + ) diff --git a/tests/test_order_140_r_loop_frame_state_machine.py b/tests/test_order_140_r_loop_frame_state_machine.py new file mode 100644 index 0000000..2cfb39f --- /dev/null +++ b/tests/test_order_140_r_loop_frame_state_machine.py @@ -0,0 +1,217 @@ +from __future__ import annotations + +import pytest + +from songryeon_core.core.r_loop_state_machine import decide_r_loop_continuation +from songryeon_core.core.schemas import ( + R1GraphGoalFrame, + R2GraphNodeSelectionFrame, + R3GraphInspectionFrame, + RLoopBudgetFrame, + RLoopReturnSummaryFrame, + validate_r1_graph_goal_frame, + validate_r2_graph_node_selection_frame, + validate_r3_graph_inspection_frame, + validate_r_loop_return_summary_frame, +) + + +def test_r2_selection_accepts_only_available_graph_node_id() -> None: + frame = _r2_selection(selected_graph_node_id="graph:axis:time") + + validate_r2_graph_node_selection_frame(frame) + + +def test_r2_selection_rejects_node_id_outside_available_list() -> None: + frame = _r2_selection(selected_graph_node_id="graph:raw_capsule:missing") + + with pytest.raises(ValueError, match="selected_graph_node_id must be available"): + validate_r2_graph_node_selection_frame(frame) + + +def test_r3_inspection_preserves_raw_bundle_and_summary_node_kinds() -> None: + raw = _r3_inspection( + frame_id="R3:inspect:raw", + inspected_graph_node_id="graph:raw_capsule:turn_001", + node_kind="raw_capsule", + summary_depth=0, + source_leaf_count=1, + ) + bundle = _r3_inspection( + frame_id="R3:inspect:bundle", + inspected_graph_node_id="graph:time_bundle:batch_001", + node_kind="time_bundle", + summary_depth=0, + source_leaf_count=3, + ) + summary = _r3_inspection( + frame_id="R3:inspect:summary", + inspected_graph_node_id="graph:summary:001", + node_kind="summary", + summary_depth=1, + source_leaf_count=3, + ) + + for frame in (raw, bundle, summary): + validate_r3_graph_inspection_frame(frame) + + assert raw.node_kind == "raw_capsule" + assert bundle.node_kind == "time_bundle" + assert summary.node_kind == "summary" + assert summary.summary_depth == 1 + + +def test_continuation_distinguishes_deeper_from_branch_switch() -> None: + budget = _budget() + deeper = decide_r_loop_continuation( + frame_id="R:continuation:deeper", + r3_inspection=_r3_inspection( + recommended_next_action="deeper", + granularity_problem_status="needs_lower_granularity", + child_node_ids=["graph:raw_capsule:turn_001"], + ), + budget=budget, + ) + switch = decide_r_loop_continuation( + frame_id="R:continuation:switch", + r3_inspection=_r3_inspection( + recommended_next_action="switch_branch", + granularity_problem_status="none", + branch_problem_status="wrong_branch", + child_node_ids=[], + ), + budget=budget, + ) + + assert deeper.continuation_status == "continue_deeper" + assert deeper.next_target_node == "R2" + assert deeper.continuation_reason_code == "CODE_STATUS:r3_needs_lower_granularity" + assert switch.continuation_status == "continue_switch_branch" + assert switch.next_target_node == "R2" + assert switch.continuation_reason_code == "CODE_STATUS:r3_recommends_branch_switch" + + +def test_continuation_closes_when_budget_is_exhausted() -> None: + continuation = decide_r_loop_continuation( + frame_id="R:continuation:budget", + r3_inspection=_r3_inspection( + recommended_next_action="deeper", + granularity_problem_status="needs_lower_granularity", + child_node_ids=["graph:raw_capsule:turn_001"], + ), + budget=_budget(used_node_reads=3), + ) + + assert continuation.continuation_status == "stop_budget_exhausted" + assert continuation.next_target_node == "return_summary" + assert continuation.remaining_node_reads == 0 + + +def test_schema_only_frames_keep_llm_semantic_judgement_not_run() -> None: + r1 = R1GraphGoalFrame( + frame_id="R1:goal", + graph_search_goal="schema-only graph goal placeholder", + required_information_granularity="unknown", + allowed_summary_depth=0, + max_traversal_depth=2, + max_branch_switches=1, + max_node_reads=3, + max_context_tokens=1200, + stop_condition="schema_only_stop_condition", + source_graph_guide_packet_id="rloop:graph_guide:graph:snapshot:turn_001", + source_data_ids=["rloop:graph_guide:graph:snapshot:turn_001"], + ) + r2 = _r2_selection(selected_graph_node_id="graph:axis:time") + r3 = _r3_inspection() + budget = _budget() + continuation = decide_r_loop_continuation( + frame_id="R:continuation:not_run", + r3_inspection=r3, + budget=budget, + ) + summary = RLoopReturnSummaryFrame( + frame_id="R:return_summary", + r_loop_task_status="not_run", + selected_entry_node_ids=["graph:axis:time"], + inspected_graph_node_ids=[r3.inspected_graph_node_id], + final_information_granularity="unknown", + summary_depth_used=0, + continuation_status=continuation.continuation_status, + budget_status=budget.budget_status, + source_graph_node_ids=["graph:axis:time"], + source_data_ids=[r1.frame_id, r2.frame_id, r3.frame_id, continuation.frame_id], + ) + + validate_r1_graph_goal_frame(r1) + validate_r2_graph_node_selection_frame(r2) + validate_r3_graph_inspection_frame(r3) + validate_r_loop_return_summary_frame(summary) + + assert r1.semantic_judgement_status == "not_run" + assert r2.semantic_judgement_status == "not_run" + assert r3.semantic_judgement_status == "not_run" + assert continuation.semantic_judgement_status == "not_run" + assert summary.semantic_judgement_status == "not_run" + + +def _r2_selection(*, selected_graph_node_id: str | None) -> R2GraphNodeSelectionFrame: + return R2GraphNodeSelectionFrame( + frame_id="R2:selection", + selection_scope="core_ego_time_axis", + available_graph_node_ids=["graph:axis:time", "graph:time_bundle:turn_001"], + selection_status="selected", + selected_graph_node_id=selected_graph_node_id, + selection_reason="", + expected_information_granularity="unknown", + expected_source_kind="time_axis", + source_r1_goal_frame_id="R1:goal", + source_data_ids=["R1:goal"], + ) + + +def _r3_inspection( + *, + frame_id: str = "R3:inspection", + inspected_graph_node_id: str = "graph:time_bundle:turn_001", + node_kind: str = "time_bundle", + summary_depth: int = 0, + source_leaf_count: int = 1, + recommended_next_action: str = "deeper", + granularity_problem_status: str = "needs_lower_granularity", + branch_problem_status: str = "none", + child_node_ids: list[str] | None = None, +) -> R3GraphInspectionFrame: + children = ["graph:raw_capsule:turn_001"] if child_node_ids is None else child_node_ids + return R3GraphInspectionFrame( + frame_id=frame_id, + inspected_graph_node_id=inspected_graph_node_id, + node_kind=node_kind, + child_node_count=len(children), + child_node_ids=children, + summary_depth=summary_depth, + source_leaf_count=source_leaf_count, + current_information_granularity="medium_summary", + sufficiency_status="insufficient", + granularity_problem_status=granularity_problem_status, + branch_problem_status=branch_problem_status, + recommended_next_action=recommended_next_action, + inspection_reason="", + source_r2_selection_frame_id="R2:selection", + source_data_ids=["R2:selection"], + ) + + +def _budget(*, used_node_reads: int = 1) -> RLoopBudgetFrame: + return RLoopBudgetFrame( + frame_id="R:budget", + source_r1_goal_frame_id="R1:goal", + max_traversal_depth=2, + max_branch_switches=1, + max_node_reads=3, + max_context_tokens=1200, + used_traversal_depth=1, + used_branch_switches=0, + used_node_reads=used_node_reads, + used_context_tokens=200, + source_data_ids=["R1:goal"], + ) diff --git a/tests/test_order_141_core_ego_guide_worker_hints.py b/tests/test_order_141_core_ego_guide_worker_hints.py new file mode 100644 index 0000000..d950d58 --- /dev/null +++ b/tests/test_order_141_core_ego_guide_worker_hints.py @@ -0,0 +1,232 @@ +from __future__ import annotations + +import json + +import pytest + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.graph_memory import ( + TIME_AXIS_NODE_ID, + build_graph_memory_snapshot_from_capsules, +) +from songryeon_core.core.schemas import ( + CoreEgoGuideWorkerHintFrame, + NodeMovement, + TurnStateCapsule, + validate_core_ego_guide_worker_hint_frame, +) +from songryeon_core.core.trace_store import TraceStore +from songryeon_core.llm.base import LLMRequest, LLMResponse +from songryeon_core.llm.fake import BrokenJSONFakeLLMAdapter +from songryeon_core.nodes.core_ego_guide_worker import ( + CORE_EGO_GUIDE_WORKER_PROMPT_REF, + run_core_ego_guide_worker_hint, +) + + +class CoreEgoGuideWorkerPayloadFakeAdapter: + model_id = "core-ego-guide-worker-payload-fake-adapter" + + def __init__(self, payload: dict[str, object]) -> None: + self.payload = payload + self.last_input_payload: dict[str, object] | None = None + + def complete(self, request: LLMRequest) -> LLMResponse: + self.last_input_payload = request.input_payload + return LLMResponse( + text=json.dumps(self.payload, ensure_ascii=False), + model_id=self.model_id, + raw=self.payload, + ) + + +def test_core_ego_guide_worker_accepts_available_entry_id() -> None: + trace_store, data_store, guide = _stores_and_guide() + adapter = CoreEgoGuideWorkerPayloadFakeAdapter( + { + "recommended_entry_node_ids": [TIME_AXIS_NODE_ID], + "avoid_entry_node_ids": [], + "traversal_strategy_hint": "Start from the time axis and inspect recent time bundles first.", + "reason_summary": "The code guide exposes the time axis as the only current entry.", + "risk_notes": ["semantic axis is not created"], + "expected_depth_policy": "Use shallow traversal, then descend only if raw capsules are needed.", + "source_graph_node_ids": [TIME_AXIS_NODE_ID], + "source_data_ids": [guide.packet_id, guide.graph_snapshot_id], + "info_class": "mixed", + "semantic_judgement_status": "ran", + } + ) + + _, frame_id, frame = run_core_ego_guide_worker_hint( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_141", + guide_packet=guide, + adapter=adapter, + ) + + assert frame_id == f"{guide.packet_id}:core_ego_guide_worker_hint" + assert frame.recommended_entry_node_ids == [TIME_AXIS_NODE_ID] + assert frame.hint_status == "ran" + assert frame.failure_type == "none" + assert frame.generated_by == f"LLM:{adapter.model_id}:core_ego_guide_worker" + assert frame.info_class == "mixed" + assert frame.source_mode == "source_bundle" + assert frame.claim_alignment == "multi_source_bundle" + assert frame.semantic_judgement_status == "ran" + assert guide.packet_id in frame.source_data_ids + assert guide.graph_snapshot_id in frame.source_data_ids + assert frame.llm_call_data_id in frame.source_data_ids + assert data_store.require_record(frame.frame_id).data_type == ( + "node_output:core_ego_guide_worker_hint_frame" + ) + assert adapter.last_input_payload is not None + assert adapter.last_input_payload["available_entry_node_ids"] == [TIME_AXIS_NODE_ID] + + +def test_core_ego_guide_worker_rejects_recommendation_outside_available_entries() -> None: + trace_store, data_store, guide = _stores_and_guide() + adapter = CoreEgoGuideWorkerPayloadFakeAdapter( + { + "recommended_entry_node_ids": ["graph:axis:semantic"], + "avoid_entry_node_ids": [], + "traversal_strategy_hint": "Use the semantic axis.", + "reason_summary": "This payload intentionally invents an unavailable entry.", + "risk_notes": [], + "expected_depth_policy": "shallow", + "source_graph_node_ids": [TIME_AXIS_NODE_ID], + "source_data_ids": [guide.packet_id], + } + ) + + _, _, frame = run_core_ego_guide_worker_hint( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_141", + guide_packet=guide, + adapter=adapter, + ) + + assert frame.hint_status == "failed" + assert frame.failure_type == "schema_failed" + assert frame.payload_parse_status == "passed" + assert frame.recommended_entry_node_ids == [] + assert frame.semantic_judgement_status == "failed" + assert frame.llm_call_data_id is not None + llm_call = data_store.require_record(frame.llm_call_data_id) + assert llm_call.payload["failure_type"] == "schema_failed" + + +def test_core_ego_guide_worker_parse_failure_records_empty_failed_hint() -> None: + trace_store, data_store, guide = _stores_and_guide() + + _, _, frame = run_core_ego_guide_worker_hint( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_141", + guide_packet=guide, + adapter=BrokenJSONFakeLLMAdapter(), + ) + + assert frame.hint_status == "failed" + assert frame.failure_type == "parse_failed" + assert frame.payload_parse_status == "failed" + assert frame.recommended_entry_node_ids == [] + assert frame.avoid_entry_node_ids == [] + assert frame.traversal_strategy_hint == "" + assert frame.llm_call_data_id in frame.source_data_ids + + +def test_core_ego_guide_worker_hint_schema_rejects_direct_invalid_frame() -> None: + _, _, guide = _stores_and_guide() + frame = CoreEgoGuideWorkerHintFrame( + frame_id=f"{guide.packet_id}:core_ego_guide_worker_hint", + source_rloop_graph_guide_packet_id=guide.packet_id, + graph_snapshot_id=guide.graph_snapshot_id, + available_entry_node_ids=[TIME_AXIS_NODE_ID], + available_source_graph_node_ids=list(guide.source_graph_node_ids), + recommended_entry_node_ids=["graph:axis:semantic"], + traversal_strategy_hint="invalid semantic entry", + reason_summary="test fixture", + expected_depth_policy="shallow", + hint_status="ran", + failure_type="none", + payload_parse_status="passed", + prompt_ref=CORE_EGO_GUIDE_WORKER_PROMPT_REF, + source_graph_node_ids=[TIME_AXIS_NODE_ID], + source_data_ids=[guide.packet_id, guide.graph_snapshot_id], + generated_by="LLM:test:core_ego_guide_worker", + info_class="mixed", + source_mode="source_bundle", + claim_alignment="multi_source_bundle", + semantic_judgement_status="ran", + ) + + with pytest.raises(ValueError, match="entry node id must be available"): + validate_core_ego_guide_worker_hint_frame(frame) + + +def test_code_guide_packet_and_llm_hint_stay_separate() -> None: + trace_store, data_store, guide = _stores_and_guide() + adapter = CoreEgoGuideWorkerPayloadFakeAdapter( + { + "recommended_entry_node_ids": [TIME_AXIS_NODE_ID], + "avoid_entry_node_ids": [], + "traversal_strategy_hint": "The first future R loop should start at the time axis.", + "reason_summary": "Only time-axis entry data is code-confirmed in this snapshot.", + "risk_notes": [], + "expected_depth_policy": "Inspect one level before descending.", + "source_graph_node_ids": [TIME_AXIS_NODE_ID], + "source_data_ids": [guide.packet_id, guide.graph_snapshot_id], + } + ) + + _, _, frame = run_core_ego_guide_worker_hint( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_141", + guide_packet=guide, + adapter=adapter, + ) + + assert guide.generated_by == "CODE:GRAPH_MEMORY_GUIDE_BUILDER" + assert guide.info_class == "absolute" + assert guide.semantic_judgement_status == "not_run" + assert guide.recommended_traversal_hints == [] + assert guide.recommended_traversal_hints_status == "not_run" + assert frame.generated_by.startswith("LLM:") + assert frame.info_class == "mixed" + assert frame.source_rloop_graph_guide_packet_id == guide.packet_id + + +def _stores_and_guide(): + build = build_graph_memory_snapshot_from_capsules( + capsules=[_sample_capsule()], + batch_id="batch_order_141", + ) + return TraceStore(), DataStore(), build.guide_packet + + +def _sample_capsule(turn_id: str = "turn_order_141_previous") -> TurnStateCapsule: + return TurnStateCapsule( + turn_id=turn_id, + node_movements=[ + NodeMovement( + movement_id=f"move:{turn_id}:001", + turn_id=turn_id, + step_index=1, + node_id="node_0", + mode="pre_route_report", + input_trace_ids=[f"trace:{turn_id}:user"], + output_trace_ids=[f"trace:{turn_id}:node0"], + status="completed", + ) + ], + trace_event_ids=[ + f"trace:{turn_id}:user", + f"trace:{turn_id}:node0", + f"trace:{turn_id}:final", + ], + user_input_trace_id=f"trace:{turn_id}:user", + final_response_trace_id=f"trace:{turn_id}:final", + ) diff --git a/tests/test_order_142_graph_memory_store_boundary.py b/tests/test_order_142_graph_memory_store_boundary.py new file mode 100644 index 0000000..7658106 --- /dev/null +++ b/tests/test_order_142_graph_memory_store_boundary.py @@ -0,0 +1,175 @@ +from __future__ import annotations + +from dataclasses import asdict, replace + +import pytest + +from songryeon_core.core.graph_memory import ( + CORE_EGO_ROOT_NODE_ID, + TIME_AXIS_NODE_ID, + build_graph_memory_snapshot_from_capsules, +) +from songryeon_core.core.graph_memory_store import ( + SONGRYEON_GRAPH_NAMESPACE, + SONGRYEON_VESSEL_DATABASE_NAME, + SONGRYEON_VESSEL_SERVICE_NAME, + InMemoryGraphMemoryStore, +) +from songryeon_core.core.schemas import NodeMovement, TurnStateCapsule + + +def test_in_memory_store_upserts_and_reads_graph_nodes_and_edges() -> None: + build = build_graph_memory_snapshot_from_capsules( + capsules=[_sample_capsule()], + batch_id="batch_order_142", + ) + store = InMemoryGraphMemoryStore() + + for node in build.nodes: + store.upsert_node(node) + for edge in build.edges: + store.upsert_edge(edge) + + assert store.node_count() == len(build.nodes) + assert store.edge_count() == len(build.edges) + assert store.get_node(CORE_EGO_ROOT_NODE_ID).node_id == CORE_EGO_ROOT_NODE_ID + + core_children = store.list_children(CORE_EGO_ROOT_NODE_ID) + assert [node.node_id for node in core_children] == [TIME_AXIS_NODE_ID] + + time_entries = store.list_core_ego_entries(axis="time") + assert [node.node_id for node in time_entries] == [TIME_AXIS_NODE_ID] + assert store.list_core_ego_entries(axis="semantic") == [] + + +def test_in_memory_store_upsert_is_idempotent_for_same_payload() -> None: + build = build_graph_memory_snapshot_from_capsules( + capsules=[_sample_capsule()], + batch_id="batch_order_142", + ) + store = InMemoryGraphMemoryStore() + node = build.nodes[0] + edge = build.edges[0] + + store.upsert_node(node) + store.upsert_node(node) + for required_node_id in {edge.from_node_id, edge.to_node_id}: + required_node = next(item for item in build.nodes if item.node_id == required_node_id) + store.upsert_node(required_node) + store.upsert_edge(edge) + store.upsert_edge(edge) + + assert store.node_count() == len({node.node_id, edge.from_node_id, edge.to_node_id}) + assert store.edge_count() == 1 + + +def test_in_memory_store_rejects_same_id_with_different_payload() -> None: + build = build_graph_memory_snapshot_from_capsules( + capsules=[_sample_capsule()], + batch_id="batch_order_142", + ) + store = InMemoryGraphMemoryStore() + node = next(item for item in build.nodes if item.node_kind == "raw_capsule") + + store.upsert_node(node) + changed = replace(node, trace_count=node.trace_count + 1) + + with pytest.raises(ValueError, match="different payload"): + store.upsert_node(changed) + + +def test_graph_memory_store_preserves_source_provenance_round_trip() -> None: + capsule = _sample_capsule() + build = build_graph_memory_snapshot_from_capsules( + capsules=[capsule], + batch_id="batch_order_142", + ) + store = InMemoryGraphMemoryStore() + raw_node = next(node for node in build.nodes if node.node_kind == "raw_capsule") + + store.upsert_node(raw_node) + retrieved = store.get_node(raw_node.node_id) + + assert retrieved is not None + assert retrieved.source_trace_ids == raw_node.source_trace_ids + assert retrieved.source_data_ids == raw_node.source_data_ids + assert retrieved.source_graph_node_ids == raw_node.source_graph_node_ids + assert set(capsule.trace_event_ids).issubset(set(retrieved.source_trace_ids)) + + +def test_graph_memory_store_snapshot_round_trips_counts_without_external_db() -> None: + build = build_graph_memory_snapshot_from_capsules( + capsules=[_sample_capsule()], + batch_id="batch_order_142", + ) + store = InMemoryGraphMemoryStore() + for node in build.nodes: + store.upsert_node(node) + for edge in build.edges: + store.upsert_edge(edge) + + snapshot = store.snapshot( + snapshot_id="graph:snapshot:store_boundary", + batch_id="batch_order_142", + ) + + assert snapshot.node_kind_counts["raw_capsule"] == 1 + assert snapshot.node_kind_counts["core_ego"] == 1 + assert snapshot.edge_kind_counts["CONTAINS"] >= 1 + assert snapshot.root_node_id == CORE_EGO_ROOT_NODE_ID + assert snapshot.time_axis_node_id == TIME_AXIS_NODE_ID + + +def test_graph_memory_store_does_not_generate_semantic_topic_fields() -> None: + build = build_graph_memory_snapshot_from_capsules( + capsules=[_sample_capsule()], + batch_id="batch_order_142", + ) + store = InMemoryGraphMemoryStore() + raw_node = next(node for node in build.nodes if node.node_kind == "raw_capsule") + + store.upsert_node(raw_node) + payload = asdict(store.get_node(raw_node.node_id)) + + assert payload["generated_by"] == "CODE:GRAPH_MEMORY_BUILDER" + assert payload["info_class"] == "absolute" + assert payload["semantic_judgement_status"] == "not_run" + for forbidden_field in ( + "semantic_topic", + "topic_label", + "topic_assignment_generated_by", + "meaning_cluster", + "embedding_id", + ): + assert forbidden_field not in payload + + +def test_songryeon_vessel_names_are_reserved_without_connecting_external_db() -> None: + assert SONGRYEON_VESSEL_SERVICE_NAME == "songryeon-neo4j-vessel" + assert SONGRYEON_VESSEL_DATABASE_NAME == "songryeon_vessel" + assert SONGRYEON_GRAPH_NAMESPACE == "songryeon_core_graph_v0" + + +def _sample_capsule(turn_id: str = "turn_order_142") -> TurnStateCapsule: + return TurnStateCapsule( + turn_id=turn_id, + node_movements=[ + NodeMovement( + movement_id=f"move:{turn_id}:001", + turn_id=turn_id, + step_index=1, + node_id="node_0", + mode="pre_route_report", + input_trace_ids=[f"trace:{turn_id}:user"], + output_trace_ids=[f"trace:{turn_id}:node0"], + status="completed", + ) + ], + trace_event_ids=[ + f"trace:{turn_id}:user", + f"trace:{turn_id}:node0", + f"trace:{turn_id}:final", + ], + user_input_trace_id=f"trace:{turn_id}:user", + final_response_trace_id=f"trace:{turn_id}:final", + ) diff --git a/tests/test_order_143_r_loop_node0_memory_handoff.py b/tests/test_order_143_r_loop_node0_memory_handoff.py new file mode 100644 index 0000000..1299bdd --- /dev/null +++ b/tests/test_order_143_r_loop_node0_memory_handoff.py @@ -0,0 +1,158 @@ +from __future__ import annotations + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.graph_memory import ( + TIME_AXIS_NODE_ID, + build_graph_memory_snapshot_from_capsules, +) +from songryeon_core.core.schemas import NodeMovement, TurnStateCapsule +from songryeon_core.core.trace_store import TraceStore +from songryeon_core.nodes.node_0_memory_supplier import ( + build_r_loop_memory_handoff_packet_frame, + record_r_loop_memory_handoff_packet, +) +from songryeon_core.runtime.dry_run import run_dry_turn +from songryeon_core.runtime.terminal_view import render_runtime_view + + +def test_node0_r_loop_handoff_preserves_graph_guide_coordinates() -> None: + guide = _guide() + frame = build_r_loop_memory_handoff_packet_frame( + guide_packet=guide, + source_trace_ids=["trace:graph_memory_builder"], + source_data_ids=[guide.packet_id, guide.graph_snapshot_id], + ) + + assert frame.packet_status == "available" + assert frame.target == "R_LOOP" + assert frame.mode == "graph_guide_handoff" + assert frame.r_loop_graph_guide_packet_id == guide.packet_id + assert frame.graph_snapshot_id == guide.graph_snapshot_id + assert frame.available_entry_node_ids == [TIME_AXIS_NODE_ID] + assert frame.node_kind_counts == guide.node_kind_counts + assert frame.summary_depth_range == guide.summary_depth_range + assert frame.source_graph_node_ids == guide.source_graph_node_ids + assert guide.packet_id in frame.source_data_ids + assert guide.graph_snapshot_id in frame.source_data_ids + assert frame.generated_by == "CODE:node_0_memory_supplier" + assert frame.info_class == "absolute" + assert frame.semantic_judgement_status == "not_run" + + +def test_node0_r_loop_handoff_records_trace_and_data_store_frame() -> None: + guide = _guide() + trace_store = TraceStore() + data_store = DataStore() + + trace_id, data_id, frame = record_r_loop_memory_handoff_packet( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_143", + guide_packet=guide, + input_ref=["trace:graph_memory_builder"], + source_data_ids=[guide.packet_id, guide.graph_snapshot_id], + ) + + record = data_store.require_record(data_id) + event = trace_store.get_event(trace_id) + assert event is not None + assert event.actor == "node_0" + assert event.event_type == "memory_packet" + assert record.data_type == "node_output:r_loop_memory_handoff_packet_frame" + assert record.payload["packet_id"] == frame.packet_id + assert record.payload["r_loop_graph_guide_packet_id"] == guide.packet_id + + +def test_node0_r_loop_handoff_missing_guide_closes_without_semantic_fallback() -> None: + frame = build_r_loop_memory_handoff_packet_frame( + guide_packet=None, + source_trace_ids=["trace:no_guide"], + source_data_ids=[], + ) + + assert frame.packet_status == "missing" + assert frame.r_loop_graph_guide_packet_id == "" + assert frame.graph_snapshot_id == "" + assert frame.available_entry_node_ids == [] + assert frame.source_graph_node_ids == [] + assert frame.semantic_hint_status == "not_run" + assert frame.semantic_judgement_status == "not_run" + assert frame.info_class == "absolute" + + +def test_dry_run_records_r_loop_handoff_without_node1_or_node3_injection() -> None: + result = run_dry_turn() + handoff_id = result["r_loop_memory_handoff_packet_id"] + guide_id = result["rloop_graph_guide_packet_id"] + snapshot_id = result["graph_memory_snapshot_id"] + + assert result["r_loop_memory_handoff_status"] == "available" + assert result["r_loop_memory_handoff_target"] == "R_LOOP" + assert result["r_loop_memory_handoff_mode"] == "graph_guide_handoff" + assert result["r_loop_memory_handoff_guide_packet_id"] == guide_id + assert result["r_loop_memory_handoff_entry_node_count"] == 1 + assert result["r_loop_memory_handoff_semantic_hint_status"] == "not_run" + assert result["r_loop_memory_handoff_info_class"] == "absolute" + assert result["r_loop_memory_handoff_semantic_judgement_status"] == "not_run" + + records = result["data_records"] + handoff_records = [ + record + for record in records + if record["data_type"] == "node_output:r_loop_memory_handoff_packet_frame" + ] + assert len(handoff_records) == 1 + assert handoff_records[0]["data_id"] == handoff_id + assert guide_id in handoff_records[0]["payload"]["source_data_ids"] + assert snapshot_id in handoff_records[0]["payload"]["source_data_ids"] + + for record in records: + if record["data_type"] not in {"node_output:routing_decision", "node_output:report"}: + continue + source_data_ids = record["payload"].get("source_data_ids", []) + assert handoff_id not in source_data_ids + assert guide_id not in source_data_ids + + +def test_runtime_view_displays_r_loop_handoff_status_only() -> None: + result = run_dry_turn() + rendered = render_runtime_view(result, user_input="R handoff smoke") + + assert "- R loop memory handoff:" in rendered + assert "status=available" in rendered + assert "target=R_LOOP" in rendered + assert "entry_nodes=1" in rendered + assert "semantic_hint_status=not_run" in rendered + + +def _guide(): + build = build_graph_memory_snapshot_from_capsules( + capsules=[_sample_capsule()], + batch_id="batch_order_143", + ) + return build.guide_packet + + +def _sample_capsule(turn_id: str = "turn_order_143_previous") -> TurnStateCapsule: + return TurnStateCapsule( + turn_id=turn_id, + node_movements=[ + NodeMovement( + movement_id=f"move:{turn_id}:001", + turn_id=turn_id, + step_index=1, + node_id="node_0", + mode="pre_route_report", + input_trace_ids=[f"trace:{turn_id}:user"], + output_trace_ids=[f"trace:{turn_id}:node0"], + status="completed", + ) + ], + trace_event_ids=[ + f"trace:{turn_id}:user", + f"trace:{turn_id}:node0", + f"trace:{turn_id}:final", + ], + user_input_trace_id=f"trace:{turn_id}:user", + final_response_trace_id=f"trace:{turn_id}:final", + ) diff --git a/tests/test_order_144_r_route_dry_run_only.py b/tests/test_order_144_r_route_dry_run_only.py new file mode 100644 index 0000000..1833cb1 --- /dev/null +++ b/tests/test_order_144_r_route_dry_run_only.py @@ -0,0 +1,138 @@ +from __future__ import annotations + +import pytest + +from songryeon_core.core.schemas import ( + R2GraphNodeSelectionFrame, + validate_r2_graph_node_selection_frame, +) +from songryeon_core.runtime.dry_run import run_dry_turn +from songryeon_core.runtime.terminal_view import render_runtime_view + + +def test_default_dry_run_does_not_open_r_route_skeleton() -> None: + result = run_dry_turn() + + assert result["current_route"] in {"2", "L"} + assert result["r_route_dry_run_enabled"] is False + assert result["r_route_dry_run_status"] == "not_run" + assert result["r_route_dry_run_output_data_ids"] == [] + assert not _payloads_with_type(result, "node_output:R_loop_return_summary_frame") + + +def test_r_route_dry_run_fixture_records_r1_r2_r3_continuation_and_summary() -> None: + result = run_dry_turn(enable_r_route_dry_run=True) + + assert result["r_route_dry_run_enabled"] is True + assert result["r_route_dry_run_status"] == "sufficient" + assert result["r_route_dry_run_continuation_status"] == "stop_sufficient" + assert result["r_route_dry_run_next_target_node"] == "return_summary" + assert result["r_route_dry_run_budget_status"] == "within_budget" + assert result["r_route_dry_run_traversal_step_count"] == 3 + assert result["r_route_dry_run_selected_entry_node_ids"] == [ + "graph:axis:time", + "graph:time_bundle:turn_dry_001", + "graph:raw_capsule:turn_dry_001", + ] + assert result["r_route_dry_run_inspected_graph_node_ids"] == [ + "graph:axis:time", + "graph:time_bundle:turn_dry_001", + "graph:raw_capsule:turn_dry_001", + ] + + assert len(_payloads_with_type(result, "node_output:R1_graph_goal_frame")) == 1 + assert len(_payloads_with_type(result, "node_output:R_loop_budget_frame")) == 3 + assert len(_payloads_with_type(result, "node_output:R2_graph_node_selection_frame")) == 3 + assert len(_payloads_with_type(result, "node_output:R3_graph_inspection_frame")) == 3 + assert len(_payloads_with_type(result, "node_output:R_graph_traversal_candidate_surface_frame")) == 3 + assert len(_payloads_with_type(result, "node_output:R_loop_continuation_frame")) == 3 + assert len(_payloads_with_type(result, "node_output:R_loop_return_summary_frame")) == 1 + + summary = _payloads_with_type(result, "node_output:R_loop_return_summary_frame")[0] + handoff_id = result["r_loop_memory_handoff_packet_id"] + assert handoff_id in summary["source_data_ids"] + assert summary["generated_by"] == "CODE:R_LOOP_DRY_RUN_ONLY" + assert summary["info_class"] == "absolute" + assert summary["semantic_judgement_status"] == "not_run" + + +def test_r_route_dry_run_r2_selection_accepts_only_available_graph_node_id() -> None: + selected = _r2_selection(selected_graph_node_id="graph:axis:time") + validate_r2_graph_node_selection_frame(selected) + + invalid = _r2_selection(selected_graph_node_id="graph:axis:semantic") + with pytest.raises(ValueError, match="selected_graph_node_id must be available"): + validate_r2_graph_node_selection_frame(invalid) + + +def test_r_route_dry_run_budget_exhaustion_closes_with_stop_budget_exhausted() -> None: + result = run_dry_turn( + enable_r_route_dry_run=True, + r_route_dry_run_force_budget_exhausted=True, + ) + + assert result["r_route_dry_run_status"] == "partial" + assert result["r_route_dry_run_continuation_status"] == "stop_budget_exhausted" + assert result["r_route_dry_run_next_target_node"] == "return_summary" + assert result["r_route_dry_run_budget_status"] == "exhausted" + + continuation = _payloads_with_type(result, "node_output:R_loop_continuation_frame")[0] + assert continuation["remaining_node_reads"] == 0 + assert continuation["continuation_reason_code"] == ( + "CODE_STATUS:r_loop_node_or_context_budget_exhausted" + ) + + +def test_terminal_runtime_displays_r_dry_run_as_code_only_when_enabled() -> None: + disabled = render_runtime_view(run_dry_turn(), user_input="R dry run disabled") + assert "- R dry-run skeleton" not in disabled + + enabled_result = run_dry_turn(enable_r_route_dry_run=True) + enabled = render_runtime_view(enabled_result, user_input="R dry run enabled") + assert "- R dry-run skeleton [CODE:R_LOOP_DRY_RUN_ONLY]:" in enabled + assert "task_status=sufficient" in enabled + assert "continuation=stop_sufficient" in enabled + assert "semantic_judgement_status: not_run" in enabled + + +def test_r_route_dry_run_does_not_inject_r_frames_into_node1_or_node3() -> None: + result = run_dry_turn(enable_r_route_dry_run=True) + r_data_ids = set(result["r_route_dry_run_output_data_ids"]) + r_data_ids.add(result["r_loop_memory_handoff_packet_id"]) + + for record in result["data_records"]: + if record["data_type"] not in {"node_output:routing_decision", "node_output:report"}: + continue + source_data_ids = set(record["payload"].get("source_data_ids", [])) + assert not (source_data_ids & r_data_ids) + + +def _r2_selection(*, selected_graph_node_id: str) -> R2GraphNodeSelectionFrame: + return R2GraphNodeSelectionFrame( + frame_id="R2:dry_run:test_selection", + selection_scope="test_scope", + available_graph_node_ids=["graph:axis:time"], + selection_status="selected", + selected_graph_node_id=selected_graph_node_id, + selection_reason="CODE_STATUS:test_selection", + expected_information_granularity="unknown", + expected_source_kind="graph_entry_node", + source_r1_goal_frame_id="R1:dry_run:graph_goal_frame", + source_data_ids=["R1:dry_run:graph_goal_frame"], + ) + + +def _payloads_with_type(result: dict[str, object], data_type: str) -> list[dict[str, object]]: + records = result.get("data_records") + if not isinstance(records, list): + return [] + payloads: list[dict[str, object]] = [] + for record in records: + if not isinstance(record, dict): + continue + if record.get("data_type") != data_type: + continue + payload = record.get("payload") + if isinstance(payload, dict): + payloads.append(payload) + return payloads diff --git a/tests/test_order_145_r_loop_pre_live_route_baseline.py b/tests/test_order_145_r_loop_pre_live_route_baseline.py new file mode 100644 index 0000000..845a4ca --- /dev/null +++ b/tests/test_order_145_r_loop_pre_live_route_baseline.py @@ -0,0 +1,163 @@ +from __future__ import annotations + +import pytest + +from songryeon_core.core.schemas import ( + MemoryPacketFrom0, + RoutingDecisionFrame, + validate_routing_decision_frame, +) +from songryeon_core.nodes.node_1_router import _validate_llm_routing_payload, route_next +from songryeon_core.core.registry import build_default_schema_registry +from songryeon_core.runtime.dry_run import run_dry_turn + + +def test_pre_live_route_r_is_not_allowed_by_routing_frame_validator() -> None: + frame = RoutingDecisionFrame( + frame_id="route:R", + turn_id="turn_order_145", + route="R", + route_reason="CODE_STATUS:test_r_route_not_live", + expected_next_0_mode="r_loop_memory_supply", + ) + + with pytest.raises(ValueError, match="experimental policy flag"): + validate_routing_decision_frame(frame) + + +def test_pre_live_node1_llm_payload_rejects_route_r() -> None: + payload = { + "route": "R", + "route_reason": "LLM wanted graph traversal", + "expected_next_0_mode": "r_loop_memory_supply", + } + + with pytest.raises(ValueError, match="node_1 route must be L or 2"): + _validate_llm_routing_payload(payload) + + +def test_pre_live_policy_router_still_outputs_only_l_or_2() -> None: + registry = build_default_schema_registry() + memory_packet = MemoryPacketFrom0(target="node_1") + + default_decision = route_next( + user_input="안녕", + memory_packet=memory_packet, + schema_registry=registry, + ) + l_decision = route_next( + user_input="송련 내부 문서를 검색해줘", + memory_packet=memory_packet, + schema_registry=registry, + ) + + assert default_decision.route == "2" + assert l_decision.route == "L" + + +def test_pre_live_default_dry_run_records_no_r_route_skeleton() -> None: + result = run_dry_turn() + + assert result["r_route_dry_run_enabled"] is False + assert result["r_route_dry_run_status"] == "not_run" + assert result["r_route_dry_run_output_data_ids"] == [] + assert not _payloads_with_type(result, "node_output:R_loop_return_summary_frame") + + +def test_pre_live_opt_in_r_dry_run_frames_remain_code_generated_not_run() -> None: + result = run_dry_turn(enable_r_route_dry_run=True) + + assert result["r_route_dry_run_enabled"] is True + assert result["r_route_dry_run_status"] == "sufficient" + + r_payloads = [ + record["payload"] + for record in result["data_records"] + if str(record["payload"].get("frame_id", "")).startswith("R") + and record["data_type"] != "graph_memory:turn_access_ledger_frame" + and record["data_type"] != "node_output:R_graph_traversal_candidate_surface_frame" + ] + assert r_payloads + for payload in r_payloads: + assert payload.get("generated_by") == "CODE:R_LOOP_DRY_RUN_ONLY" + assert payload.get("semantic_judgement_status") == "not_run" + + access_ledgers = _payloads_with_type(result, "graph_memory:turn_access_ledger_frame") + assert len(access_ledgers) == 1 + assert access_ledgers[0]["generated_by"] == "CODE:GRAPH_ACCESS_LEDGER" + assert access_ledgers[0]["semantic_judgement_status"] == "not_run" + + candidate_surfaces = _payloads_with_type( + result, + "node_output:R_graph_traversal_candidate_surface_frame", + ) + assert len(candidate_surfaces) == 3 + for surface in candidate_surfaces: + assert surface["generated_by"] == "CODE:R_GRAPH_TRAVERSAL_CANDIDATE_SURFACE" + assert surface["semantic_judgement_status"] == "not_run" + + +def test_pre_live_r_dry_run_control_frames_are_absolute() -> None: + result = run_dry_turn(enable_r_route_dry_run=True) + + control_types = { + "node_output:R_loop_budget_frame", + "node_output:R_loop_continuation_frame", + "node_output:R_loop_return_summary_frame", + } + control_payloads = [ + record["payload"] + for record in result["data_records"] + if record["data_type"] in control_types + ] + + assert len(control_payloads) == 7 + for payload in control_payloads: + assert payload["generated_by"] == "CODE:R_LOOP_DRY_RUN_ONLY" + assert payload["info_class"] == "absolute" + assert payload["semantic_judgement_status"] == "not_run" + + +def test_pre_live_r_dry_run_output_is_not_injected_into_node1_or_node3() -> None: + result = run_dry_turn(enable_r_route_dry_run=True) + r_data_ids = set(result["r_route_dry_run_output_data_ids"]) + r_data_ids.add(result["r_loop_memory_handoff_packet_id"]) + + for record in result["data_records"]: + if record["data_type"] not in { + "node_output:routing_decision", + "node_output:report", + }: + continue + source_data_ids = set(record["payload"].get("source_data_ids", [])) + assert not (source_data_ids & r_data_ids) + + +def _all_payloads(result: dict[str, object]) -> list[dict[str, object]]: + records = result.get("data_records") + if not isinstance(records, list): + return [] + payloads: list[dict[str, object]] = [] + for record in records: + if not isinstance(record, dict): + continue + payload = record.get("payload") + if isinstance(payload, dict): + payloads.append(payload) + return payloads + + +def _payloads_with_type(result: dict[str, object], data_type: str) -> list[dict[str, object]]: + records = result.get("data_records") + if not isinstance(records, list): + return [] + payloads: list[dict[str, object]] = [] + for record in records: + if not isinstance(record, dict): + continue + if record.get("data_type") != data_type: + continue + payload = record.get("payload") + if isinstance(payload, dict): + payloads.append(payload) + return payloads diff --git a/tests/test_order_146_r_route_experimental_gate.py b/tests/test_order_146_r_route_experimental_gate.py new file mode 100644 index 0000000..6aafca7 --- /dev/null +++ b/tests/test_order_146_r_route_experimental_gate.py @@ -0,0 +1,171 @@ +from __future__ import annotations + +import json + +import pytest + +from songryeon_core.core.schemas import ( + R_ROUTE_EXPERIMENTAL_NEXT_0_MODE, + R_ROUTE_EXPERIMENTAL_POLICY_FLAG, + RoutingDecisionFrame, + validate_routing_decision_frame, +) +from songryeon_core.llm.base import LLMRequest, LLMResponse +from songryeon_core.nodes.node_1_router import _validate_llm_routing_payload +from songryeon_core.runtime.dry_run import run_dry_turn +from songryeon_core.runtime.terminal_view import render_runtime_view + + +def test_route_r_llm_payload_is_rejected_without_explicit_experimental_gate() -> None: + payload = { + "route": "R", + "route_reason": "graph memory traversal looks useful", + "expected_next_0_mode": R_ROUTE_EXPERIMENTAL_NEXT_0_MODE, + } + + with pytest.raises(ValueError, match="node_1 route must be L or 2"): + _validate_llm_routing_payload(payload) + + +def test_route_r_llm_payload_is_accepted_with_explicit_experimental_gate() -> None: + payload = { + "route": "R", + "route_reason": "graph memory traversal looks useful", + "expected_next_0_mode": R_ROUTE_EXPERIMENTAL_NEXT_0_MODE, + } + + _validate_llm_routing_payload(payload, allow_r_route_experimental=True) + + +def test_route_r_frame_requires_policy_flag_llm_source_and_r_handoff_mode() -> None: + frame = RoutingDecisionFrame( + frame_id="route:R", + turn_id="turn_order_146", + route="R", + route_reason="graph memory traversal looks useful", + expected_next_0_mode=R_ROUTE_EXPERIMENTAL_NEXT_0_MODE, + route_source="LLM:r-route-fake", + llm_routing_status="ran", + route_rule_id="llm_router", + policy_flag=R_ROUTE_EXPERIMENTAL_POLICY_FLAG, + ) + validate_routing_decision_frame(frame) + + missing_policy = RoutingDecisionFrame( + frame_id="route:R", + turn_id="turn_order_146", + route="R", + route_reason="graph memory traversal looks useful", + expected_next_0_mode=R_ROUTE_EXPERIMENTAL_NEXT_0_MODE, + route_source="LLM:r-route-fake", + llm_routing_status="ran", + route_rule_id="llm_router", + ) + with pytest.raises(ValueError, match="experimental policy flag"): + validate_routing_decision_frame(missing_policy) + + +def test_experimental_r_route_runs_skeleton_then_closes_to_route_2() -> None: + result = run_dry_turn( + user_input="그래프 기억을 R로 한 번만 실험해줘", + node_1_router_adapter=RRouteFakeAdapter(), + enable_r_route_experimental=True, + ) + + assert result["current_route"] == "2" + assert result["r_route_experimental_enabled"] is True + assert result["r_route_experimental_status"] == "selected" + assert result["r_route_experimental_handoff_packet_id"] == ( + "node_0:r_loop_memory_handoff_packet_frame:r_route_experimental" + ) + assert result["r_route_experimental_return_summary_id"] == ( + "R:experimental:return_summary_frame" + ) + assert result["r_route_experimental_close_route_id"] == "route:2" + assert len(result["r_route_experimental_output_data_ids"]) == 13 + assert result["r_route_experimental_candidate_surface_id"] in result[ + "r_route_experimental_output_data_ids" + ] + assert result["r_route_experimental_access_ledger_id"] in result[ + "r_route_experimental_output_data_ids" + ] + assert "route:R" in result["data_ids"] + assert "route:2" in result["data_ids"] + + route_r = _payload(result, "route:R") + assert route_r["policy_flag"] == R_ROUTE_EXPERIMENTAL_POLICY_FLAG + assert route_r["route_source"] == "LLM:r-route-fake" + assert route_r["expected_next_0_mode"] == R_ROUTE_EXPERIMENTAL_NEXT_0_MODE + + summary = _payload(result, "R:experimental:return_summary_frame") + assert summary["r_loop_task_status"] == "sufficient" + assert summary["continuation_status"] == "stop_sufficient" + assert summary["generated_by"] == "CODE:R_ROUTE_EXPERIMENTAL_GATE" + assert summary["info_class"] == "absolute" + assert summary["semantic_judgement_status"] == "not_run" + + ledger = _payload(result, result["r_route_experimental_access_ledger_id"]) + assert ledger["generated_by"] == "CODE:GRAPH_ACCESS_LEDGER" + assert ledger["info_class"] == "absolute" + assert ledger["semantic_judgement_status"] == "not_run" + + surface = _payload(result, result["r_route_experimental_candidate_surface_id"]) + assert surface["generated_by"] == "CODE:R_GRAPH_TRAVERSAL_CANDIDATE_SURFACE" + assert surface["info_class"] == "absolute" + assert surface["semantic_judgement_status"] == "not_run" + + +def test_experimental_r_route_is_not_available_without_gate() -> None: + result = run_dry_turn( + user_input="그래프 기억을 R로 한 번만 실험해줘", + node_1_router_adapter=RRouteFakeAdapter(), + enable_r_route_experimental=False, + ) + + assert result["current_route"] in {"2", "L"} + assert result["r_route_experimental_enabled"] is False + assert result["r_route_experimental_status"] == "not_run" + assert "route:R" not in result["data_ids"] + + +def test_terminal_marks_experimental_r_separately_from_dry_run() -> None: + result = run_dry_turn( + user_input="그래프 기억을 R로 한 번만 실험해줘", + node_1_router_adapter=RRouteFakeAdapter(), + enable_r_route_experimental=True, + ) + + rendered = render_runtime_view(result, user_input="R experimental") + + assert "- R experimental route skeleton [CODE:R_ROUTE_EXPERIMENTAL_GATE]:" in rendered + assert "- R dry-run skeleton [CODE:R_LOOP_DRY_RUN_ONLY]:" not in rendered + + +class RRouteFakeAdapter: + model_id = "r-route-fake" + + def complete(self, request: LLMRequest) -> LLMResponse: + if "node_1 Router" not in request.prompt: + return LLMResponse(text="{}", model_id=self.model_id, raw={}) + + payload = { + "route": "R", + "route_reason": "graph memory traversal looks useful", + "expected_next_0_mode": R_ROUTE_EXPERIMENTAL_NEXT_0_MODE, + "route_confidence": 0.71, + "needs_more_memory": False, + } + return LLMResponse( + text=json.dumps(payload, ensure_ascii=False), + model_id=self.model_id, + raw=payload, + ) + + +def _payload(result: dict[str, object], data_id: str) -> dict[str, object]: + for record in result["data_records"]: + if record["data_id"] == data_id: + payload = record["payload"] + if isinstance(payload, dict): + return payload + raise AssertionError(f"missing payload: {data_id}") diff --git a/tests/test_order_147_r_result_to_node3_brief.py b/tests/test_order_147_r_result_to_node3_brief.py new file mode 100644 index 0000000..ea6e70a --- /dev/null +++ b/tests/test_order_147_r_result_to_node3_brief.py @@ -0,0 +1,128 @@ +from __future__ import annotations + +import json + +from songryeon_core.core.schemas import ( + Node3InputBriefFrame, + Node3RLoopResultMaterial, + R_ROUTE_EXPERIMENTAL_NEXT_0_MODE, +) +from songryeon_core.llm.base import LLMRequest, LLMResponse +from songryeon_core.nodes.node_2_handoff import node3_brief_llm_payload +from songryeon_core.nodes.node_3_reporter import build_node3_grounding_block +from songryeon_core.runtime.dry_run import run_dry_turn +from songryeon_core.runtime.terminal_view import render_runtime_view + + +def test_experimental_r_summary_is_preserved_in_node3_brief() -> None: + result = run_dry_turn( + user_input="그래프 기억을 R로 한 번만 실험해줘", + node_1_router_adapter=RRouteFakeAdapter(), + enable_r_route_experimental=True, + ) + + brief = _payload(result, "node_3:input_brief_frame") + material = brief["r_loop_result_material"] + + assert material["source_data_id"] == "R:experimental:return_summary_frame" + assert material["r_loop_task_status"] == "sufficient" + assert material["continuation_status"] == "stop_sufficient" + assert material["budget_status"] == "within_budget" + assert material["generated_by"] == "CODE:R_ROUTE_EXPERIMENTAL_GATE" + assert material["info_class"] == "absolute" + assert material["semantic_judgement_status"] == "not_run" + assert material["attitude_hint"] == "r_loop_sufficient" + assert "R:experimental:return_summary_frame" in brief["source_data_ids"] + + +def test_node3_grounding_block_marks_r_skeleton_as_limited() -> None: + frame = _minimal_node3_brief_with_r_material() + + grounding_block = build_node3_grounding_block(frame) + + assert "R 탐색 실험 상태: partial / continue_deeper" in grounding_block + assert "graph memory 탐색 성공으로 단정하지 않는다" in grounding_block + + +def test_node3_llm_payload_contains_safe_r_result_boundary() -> None: + frame = _minimal_node3_brief_with_r_material() + + payload = node3_brief_llm_payload(frame) + + r_loop_result = payload["r_loop_result"] + assert r_loop_result["status"] == "present" + assert r_loop_result["task_status"] == "partial" + assert r_loop_result["attitude_hint"] == "r_loop_partial_or_skeleton_only" + assert "does not prove" in r_loop_result["boundary"] + + +def test_terminal_marks_r_result_material_in_node3_brief() -> None: + result = run_dry_turn( + user_input="그래프 기억을 R로 한 번만 실험해줘", + node_1_router_adapter=RRouteFakeAdapter(), + enable_r_route_experimental=True, + ) + + rendered = render_runtime_view(result, user_input="R experimental") + + assert "R loop result in brief: task=sufficient" in rendered + assert "hint=r_loop_sufficient" in rendered + + +def _minimal_node3_brief_with_r_material() -> Node3InputBriefFrame: + return Node3InputBriefFrame( + frame_id="node_3:input_brief_frame:test", + turn_id="turn_order_147", + user_question="R 결과를 설명해줘", + brief_status="ready", + handoff_frame_id="node_2:handoff_frame:test", + r_loop_result_material=Node3RLoopResultMaterial( + source_data_id="R:experimental:return_summary_frame", + r_loop_task_status="partial", + continuation_status="continue_deeper", + budget_status="within_budget", + final_information_granularity="summary_depth_0_graph_skeleton", + summary_depth_used=0, + selected_entry_node_count=1, + inspected_graph_node_count=1, + source_graph_node_count=1, + generated_by="CODE:R_ROUTE_EXPERIMENTAL_GATE", + info_class="absolute", + semantic_judgement_status="not_run", + attitude_hint="r_loop_partial_or_skeleton_only", + ), + source_data_ids=[ + "node_2:handoff_frame:test", + "R:experimental:return_summary_frame", + ], + ) + + +class RRouteFakeAdapter: + model_id = "r-route-fake" + + def complete(self, request: LLMRequest) -> LLMResponse: + if "node_1 Router" not in request.prompt: + return LLMResponse(text="{}", model_id=self.model_id, raw={}) + + payload = { + "route": "R", + "route_reason": "graph memory traversal looks useful", + "expected_next_0_mode": R_ROUTE_EXPERIMENTAL_NEXT_0_MODE, + "route_confidence": 0.71, + "needs_more_memory": False, + } + return LLMResponse( + text=json.dumps(payload, ensure_ascii=False), + model_id=self.model_id, + raw=payload, + ) + + +def _payload(result: dict[str, object], data_id: str) -> dict[str, object]: + for record in result["data_records"]: + if record["data_id"] == data_id: + payload = record["payload"] + if isinstance(payload, dict): + return payload + raise AssertionError(f"missing payload: {data_id}") diff --git a/tests/test_order_148_graph_next_and_access_ledger.py b/tests/test_order_148_graph_next_and_access_ledger.py new file mode 100644 index 0000000..c066cc2 --- /dev/null +++ b/tests/test_order_148_graph_next_and_access_ledger.py @@ -0,0 +1,131 @@ +from __future__ import annotations + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.graph_memory import ( + TIME_AXIS_NODE_ID, + build_graph_memory_snapshot_from_capsules, + raw_capsule_graph_node_id, + record_graph_memory_for_capsules, +) +from songryeon_core.core.schemas import NodeMovement, TurnStateCapsule +from songryeon_core.core.trace_store import TraceStore +from songryeon_core.loops.r_loop_dry_run import run_r_loop_dry_run_skeleton +from songryeon_core.nodes.node_0_memory_supplier import record_r_loop_memory_handoff_packet + + +def test_raw_capsule_next_edges_follow_deduped_capsule_order() -> None: + capsules = [ + _sample_capsule("turn_order_148_001"), + _sample_capsule("turn_order_148_002"), + _sample_capsule("turn_order_148_002"), + _sample_capsule("turn_order_148_003"), + ] + + build = build_graph_memory_snapshot_from_capsules( + capsules=capsules, + batch_id="batch_order_148", + ) + + next_edges = [edge for edge in build.edges if edge.edge_kind == "NEXT"] + next_pairs = [(edge.from_node_id, edge.to_node_id) for edge in next_edges] + + assert next_pairs == [ + ( + raw_capsule_graph_node_id("turn_order_148_001"), + raw_capsule_graph_node_id("turn_order_148_002"), + ), + ( + raw_capsule_graph_node_id("turn_order_148_002"), + raw_capsule_graph_node_id("turn_order_148_003"), + ), + ] + assert build.snapshot.edge_kind_counts["NEXT"] == 2 + assert all(edge.info_class == "absolute" for edge in next_edges) + assert all(edge.semantic_judgement_status == "not_run" for edge in next_edges) + + +def test_r_dry_run_records_turn_graph_access_ledger_without_semantic_judgement() -> None: + trace_store = TraceStore() + data_store = DataStore() + graph_record = record_graph_memory_for_capsules( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_148_graph_build", + capsules=[ + _sample_capsule("turn_order_148_previous_001"), + _sample_capsule("turn_order_148_previous_002"), + ], + batch_id="batch_order_148_access", + ) + handoff_trace_id, handoff_data_id, handoff = record_r_loop_memory_handoff_packet( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_148_runtime", + guide_packet=graph_record.build.guide_packet, + input_ref=[graph_record.trace_event_id], + source_data_ids=[ + graph_record.build.snapshot.snapshot_id, + graph_record.build.guide_packet.packet_id, + ], + ) + + result = run_r_loop_dry_run_skeleton( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_148_runtime", + handoff_packet=handoff, + input_ref=[handoff_trace_id], + frame_label="order_148", + ) + + ledger = result.access_ledger + ledger_record = data_store.require_record(ledger.frame_id) + + assert ledger_record.data_type == "graph_memory:turn_access_ledger_frame" + assert ledger.frame_id in result.output_data_ids + assert ledger.turn_id == "turn_order_148_runtime" + assert ledger.turn_capsule_graph_node_id == raw_capsule_graph_node_id( + "turn_order_148_runtime" + ) + assert ledger.candidate_graph_node_ids[0] == TIME_AXIS_NODE_ID + assert ledger.selected_graph_node_ids == [ + TIME_AXIS_NODE_ID, + "graph:time_bundle:batch_order_148_access", + raw_capsule_graph_node_id("turn_order_148_previous_001"), + ] + assert ledger.inspected_graph_node_ids == ledger.selected_graph_node_ids + assert ledger.read_graph_node_ids == [] + assert ledger.used_as_answer_source_graph_node_ids == [] + assert ledger.generated_by == "CODE:GRAPH_ACCESS_LEDGER" + assert ledger.info_class == "absolute" + assert ledger.semantic_judgement_status == "not_run" + assert handoff_data_id in ledger.source_data_ids + assert any(record["stage"] == "selected" for record in ledger.access_records) + assert any(record["stage"] == "inspected" for record in ledger.access_records) + assert "relevance_reason" not in ledger_record.payload + assert "importance_reason" not in ledger_record.payload + + +def _sample_capsule(turn_id: str) -> TurnStateCapsule: + return TurnStateCapsule( + turn_id=turn_id, + node_movements=[ + NodeMovement( + movement_id=f"move:{turn_id}:001", + turn_id=turn_id, + step_index=1, + node_id="node_0", + mode="pre_route_report", + input_trace_ids=[f"trace:{turn_id}:user"], + output_trace_ids=[f"trace:{turn_id}:node0"], + status="completed", + ) + ], + trace_event_ids=[ + f"trace:{turn_id}:user", + f"trace:{turn_id}:node0", + f"trace:{turn_id}:final", + ], + user_input_trace_id=f"trace:{turn_id}:user", + final_response_trace_id=f"trace:{turn_id}:final", + ) diff --git a/tests/test_order_149_r_graph_traversal_candidate_surface.py b/tests/test_order_149_r_graph_traversal_candidate_surface.py new file mode 100644 index 0000000..d28ab51 --- /dev/null +++ b/tests/test_order_149_r_graph_traversal_candidate_surface.py @@ -0,0 +1,147 @@ +from __future__ import annotations + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.graph_memory import ( + build_graph_memory_snapshot_from_capsules, + raw_capsule_graph_node_id, + record_graph_memory_for_capsules, +) +from songryeon_core.core.schemas import ( + NodeMovement, + RLoopMemoryHandoffPacketFrame, + TurnStateCapsule, +) +from songryeon_core.core.trace_store import TraceStore +from songryeon_core.loops.r_loop_dry_run import run_r_loop_dry_run_skeleton +from songryeon_core.nodes.node_0_memory_supplier import record_r_loop_memory_handoff_packet + + +def test_candidate_surface_copies_child_candidates_from_r3_inspection() -> None: + trace_store = TraceStore() + data_store = DataStore() + graph_record = record_graph_memory_for_capsules( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_149_graph_build", + capsules=[ + _sample_capsule("turn_order_149_child_001"), + _sample_capsule("turn_order_149_child_002"), + ], + batch_id="batch_order_149_child", + ) + handoff_trace_id, _handoff_data_id, handoff = record_r_loop_memory_handoff_packet( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_149_runtime", + guide_packet=graph_record.build.guide_packet, + input_ref=[graph_record.trace_event_id], + source_data_ids=[ + graph_record.build.snapshot.snapshot_id, + graph_record.build.guide_packet.packet_id, + ], + ) + + result = run_r_loop_dry_run_skeleton( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_149_runtime", + handoff_packet=handoff, + input_ref=[handoff_trace_id], + frame_label="order_149_child", + ) + + surface = result.candidate_surfaces[0] + record = data_store.require_record(surface.frame_id) + + assert record.data_type == "node_output:R_graph_traversal_candidate_surface_frame" + assert surface.child_candidate_node_ids + assert surface.candidate_graph_node_ids == surface.child_candidate_node_ids + assert surface.candidate_count == len(surface.candidate_graph_node_ids) + assert surface.generated_by == "CODE:R_GRAPH_TRAVERSAL_CANDIDATE_SURFACE" + assert surface.info_class == "absolute" + assert surface.semantic_judgement_status == "not_run" + assert any(item["relation"] == "child" for item in surface.candidate_records) + + +def test_candidate_surface_copies_next_and_previous_candidates_from_next_edges() -> None: + trace_store = TraceStore() + data_store = DataStore() + graph_record = record_graph_memory_for_capsules( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_149_next_graph_build", + capsules=[ + _sample_capsule("turn_order_149_next_001"), + _sample_capsule("turn_order_149_next_002"), + _sample_capsule("turn_order_149_next_003"), + ], + batch_id="batch_order_149_next", + ) + inspected_node_id = raw_capsule_graph_node_id("turn_order_149_next_002") + handoff = RLoopMemoryHandoffPacketFrame( + packet_id="node_0:r_loop_memory_handoff_packet_frame:order_149_next", + packet_status="available", + graph_snapshot_id=graph_record.build.snapshot.snapshot_id, + r_loop_graph_guide_packet_id=graph_record.build.guide_packet.packet_id, + available_entry_node_ids=[inspected_node_id], + node_kind_counts=dict(graph_record.build.snapshot.node_kind_counts), + data_kind_counts=dict(graph_record.build.snapshot.data_kind_counts), + summary_depth_range=list(graph_record.build.guide_packet.summary_depth_range), + source_leaf_count_range=list(graph_record.build.guide_packet.source_leaf_count_range), + source_graph_node_ids=list(graph_record.build.snapshot.source_graph_node_ids), + source_trace_ids=list(graph_record.build.snapshot.source_trace_ids), + source_data_ids=[ + graph_record.build.snapshot.snapshot_id, + graph_record.build.guide_packet.packet_id, + ], + ) + + result = run_r_loop_dry_run_skeleton( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_149_next_runtime", + handoff_packet=handoff, + input_ref=[graph_record.trace_event_id], + frame_label="order_149_next", + ) + + surface = result.candidate_surface + previous_node_id = raw_capsule_graph_node_id("turn_order_149_next_001") + next_node_id = raw_capsule_graph_node_id("turn_order_149_next_003") + + assert surface.child_candidate_node_ids == [] + assert surface.previous_candidate_node_ids == [previous_node_id] + assert surface.next_candidate_node_ids == [next_node_id] + assert set(surface.candidate_graph_node_ids) == {previous_node_id, next_node_id} + assert any(item["relation"] == "previous" for item in surface.candidate_records) + assert any(item["relation"] == "next" for item in surface.candidate_records) + assert result.access_ledger.candidate_graph_node_ids[-2:] == [ + next_node_id, + previous_node_id, + ] + assert result.access_ledger.source_data_ids.count(surface.frame_id) == 1 + + +def _sample_capsule(turn_id: str) -> TurnStateCapsule: + return TurnStateCapsule( + turn_id=turn_id, + node_movements=[ + NodeMovement( + movement_id=f"move:{turn_id}:001", + turn_id=turn_id, + step_index=1, + node_id="node_0", + mode="pre_route_report", + input_trace_ids=[f"trace:{turn_id}:user"], + output_trace_ids=[f"trace:{turn_id}:node0"], + status="completed", + ) + ], + trace_event_ids=[ + f"trace:{turn_id}:user", + f"trace:{turn_id}:node0", + f"trace:{turn_id}:final", + ], + user_input_trace_id=f"trace:{turn_id}:user", + final_response_trace_id=f"trace:{turn_id}:final", + ) diff --git a/tests/test_order_150_r_loop_multi_step_traversal_dry_run.py b/tests/test_order_150_r_loop_multi_step_traversal_dry_run.py new file mode 100644 index 0000000..f2865b1 --- /dev/null +++ b/tests/test_order_150_r_loop_multi_step_traversal_dry_run.py @@ -0,0 +1,100 @@ +from __future__ import annotations + +from songryeon_core.runtime.dry_run import run_dry_turn + + +def test_r_dry_run_consumes_candidate_surface_until_raw_capsule() -> None: + result = run_dry_turn(enable_r_route_dry_run=True) + + assert result["r_route_dry_run_status"] == "sufficient" + assert result["r_route_dry_run_continuation_status"] == "stop_sufficient" + assert result["r_route_dry_run_traversal_step_count"] == 3 + assert result["r_route_dry_run_selected_entry_node_ids"] == [ + "graph:axis:time", + "graph:time_bundle:turn_dry_001", + "graph:raw_capsule:turn_dry_001", + ] + + r2_frames = _payloads_with_type(result, "node_output:R2_graph_node_selection_frame") + candidate_surfaces = _payloads_with_type( + result, + "node_output:R_graph_traversal_candidate_surface_frame", + ) + continuations = _payloads_with_type(result, "node_output:R_loop_continuation_frame") + + assert [frame["selection_scope"] for frame in r2_frames] == [ + "core_ego_graph_guide_handoff", + "r_graph_traversal_candidate_surface", + "r_graph_traversal_candidate_surface", + ] + assert r2_frames[1]["selected_graph_node_id"] in candidate_surfaces[0][ + "candidate_graph_node_ids" + ] + assert r2_frames[2]["selected_graph_node_id"] in candidate_surfaces[1][ + "candidate_graph_node_ids" + ] + assert [frame["continuation_status"] for frame in continuations] == [ + "continue_deeper", + "continue_deeper", + "stop_sufficient", + ] + + +def test_r_dry_run_budget_counts_entry_depth_as_zero_then_edges_as_depth() -> None: + result = run_dry_turn(enable_r_route_dry_run=True) + + budgets = _payloads_with_type(result, "node_output:R_loop_budget_frame") + + assert [frame["used_node_reads"] for frame in budgets] == [1, 2, 3] + assert [frame["used_traversal_depth"] for frame in budgets] == [0, 1, 2] + assert [frame["budget_status"] for frame in budgets] == [ + "within_budget", + "within_budget", + "within_budget", + ] + + +def test_r_dry_run_access_ledger_accumulates_multi_step_graph_nodes() -> None: + result = run_dry_turn(enable_r_route_dry_run=True) + + ledger = _payloads_with_type(result, "graph_memory:turn_access_ledger_frame")[0] + + assert ledger["selected_graph_node_ids"] == [ + "graph:axis:time", + "graph:time_bundle:turn_dry_001", + "graph:raw_capsule:turn_dry_001", + ] + assert ledger["inspected_graph_node_ids"] == ledger["selected_graph_node_ids"] + assert ledger["read_graph_node_ids"] == [] + assert ledger["used_as_answer_source_graph_node_ids"] == [] + assert ledger["generated_by"] == "CODE:GRAPH_ACCESS_LEDGER" + assert ledger["semantic_judgement_status"] == "not_run" + + +def test_r_dry_run_forced_budget_exhaustion_still_stops_after_first_step() -> None: + result = run_dry_turn( + enable_r_route_dry_run=True, + r_route_dry_run_force_budget_exhausted=True, + ) + + assert result["r_route_dry_run_status"] == "partial" + assert result["r_route_dry_run_continuation_status"] == "stop_budget_exhausted" + assert result["r_route_dry_run_traversal_step_count"] == 1 + assert len(_payloads_with_type(result, "node_output:R2_graph_node_selection_frame")) == 1 + assert len(_payloads_with_type(result, "node_output:R3_graph_inspection_frame")) == 1 + + +def _payloads_with_type(result: dict[str, object], data_type: str) -> list[dict[str, object]]: + records = result.get("data_records") + if not isinstance(records, list): + return [] + payloads: list[dict[str, object]] = [] + for record in records: + if not isinstance(record, dict): + continue + if record.get("data_type") != data_type: + continue + payload = record.get("payload") + if isinstance(payload, dict): + payloads.append(payload) + return payloads diff --git a/tests/test_order_151_l_loop_activity_ledger.py b/tests/test_order_151_l_loop_activity_ledger.py new file mode 100644 index 0000000..ca0e167 --- /dev/null +++ b/tests/test_order_151_l_loop_activity_ledger.py @@ -0,0 +1,82 @@ +from __future__ import annotations + +import pytest + +from songryeon_core.core.schemas import ( + L_LOOP_ACTIVITY_LEDGER_DATA_TYPE, + LLoopActivityLedgerFrame, + validate_l_loop_activity_ledger_frame, +) +from songryeon_core.runtime.dry_run import run_dry_turn + + +def test_l_loop_activity_ledger_is_recorded_from_dry_turn() -> None: + result = run_dry_turn() + records = { + item["data_id"]: item + for item in result["data_records"] + if isinstance(item, dict) + } + ledger_record = records.get("L:activity_ledger_frame") + assert ledger_record is not None + assert ledger_record["data_type"] == L_LOOP_ACTIVITY_LEDGER_DATA_TYPE + ledger = ledger_record["payload"] + + assert ledger["generated_by"] == "CODE:L_LOOP_ACTIVITY_LEDGER" + assert ledger["info_class"] == "absolute" + assert ledger["semantic_judgement_status"] == "not_run" + assert ledger["turn_capsule_graph_node_id"] == "graph:raw_capsule:turn_dry_001" + assert ledger["return_summary_frame_id"] == "L:return_summary_frame" + assert ledger["document_material_packet_frame_id"] == "node_0:document_material_packet_frame" + assert "L1:goal_frame" in ledger["goal_data_ids"] + assert "L2:query_frame" in ledger["query_data_ids"] + assert "L3:achievement_frame" in ledger["achievement_data_ids"] + assert ledger["output_data_id_count"] == len(ledger["output_data_ids"]) + assert ledger["tool_result_count"] == len(ledger["tool_result_data_ids"]) + assert ledger["actual_read_doc_count"] == len(ledger["read_doc_ids"]) + assert ledger["search_candidate_doc_count"] == len(ledger["search_candidate_doc_ids"]) + + +def test_l_loop_activity_ledger_sources_include_indexed_records() -> None: + result = run_dry_turn() + records = { + item["data_id"]: item["payload"] + for item in result["data_records"] + if isinstance(item, dict) + } + ledger = records["L:activity_ledger_frame"] + source_data_ids = set(ledger["source_data_ids"]) + required = { + "L:run_frame:0001", + "L:return_summary_frame", + "node_0:document_material_packet_frame", + *ledger["output_data_ids"], + } + + assert required <= source_data_ids + for record in ledger["activity_records"]: + assert record["data_id"] in source_data_ids + + +def test_l_loop_activity_ledger_validator_rejects_semantic_status() -> None: + frame = LLoopActivityLedgerFrame( + frame_id="L:activity_ledger_frame", + turn_id="turn_test", + run_index=1, + turn_capsule_graph_node_id="graph:raw_capsule:turn_test", + run_frame_data_ids=["L:run_frame:0001"], + output_data_ids=["L:run_frame:0001"], + output_data_id_count=1, + activity_records=[ + { + "stage": "run_frame", + "data_id": "L:run_frame:0001", + "source_field": "run_frame_data_ids", + } + ], + source_data_ids=["L:run_frame:0001"], + semantic_judgement_status="ran", + ) + + with pytest.raises(ValueError, match="semantic_judgement_status"): + validate_l_loop_activity_ledger_frame(frame) diff --git a/tests/test_order_152_raw_capsule_activity_graph_link.py b/tests/test_order_152_raw_capsule_activity_graph_link.py new file mode 100644 index 0000000..8493374 --- /dev/null +++ b/tests/test_order_152_raw_capsule_activity_graph_link.py @@ -0,0 +1,106 @@ +from __future__ import annotations + +import pytest + +from songryeon_core.core.schemas import ( + TurnActivityGraphLinkFrame, + validate_turn_activity_graph_link_frame, +) +from songryeon_core.core.turn_activity_graph_links import ( + TURN_ACTIVITY_GRAPH_LINK_DATA_TYPE, + activity_ledger_graph_edge_id, + activity_ledger_graph_node_id, + turn_activity_graph_link_frame_id, +) +from songryeon_core.runtime.dry_run import run_dry_turn + + +def test_raw_capsule_links_to_l_activity_ledger() -> None: + result = run_dry_turn() + records = _records_by_id(result) + link_record = records[turn_activity_graph_link_frame_id("turn_dry_001")] + link = link_record["payload"] + l_node_id = activity_ledger_graph_node_id("L:activity_ledger_frame") + edge_id = activity_ledger_graph_edge_id( + raw_capsule_node_id="graph:raw_capsule:turn_dry_001", + activity_node_id=l_node_id, + ) + + assert link_record["data_type"] == TURN_ACTIVITY_GRAPH_LINK_DATA_TYPE + assert link["generated_by"] == "CODE:TURN_ACTIVITY_GRAPH_LINK_BUILDER" + assert link["info_class"] == "absolute" + assert link["semantic_judgement_status"] == "not_run" + assert link["turn_capsule_graph_node_id"] == "graph:raw_capsule:turn_dry_001" + assert link["l_loop_activity_ledger_data_ids"] == ["L:activity_ledger_frame"] + assert link["r_graph_access_ledger_data_ids"] == [] + assert link["activity_ledger_graph_node_ids"] == [l_node_id] + assert link["activity_ledger_graph_edge_ids"] == [edge_id] + assert link["link_records"] == [ + { + "activity_kind": "l_loop_activity_ledger", + "ledger_data_id": "L:activity_ledger_frame", + "graph_node_id": l_node_id, + "edge_id": edge_id, + "source_field": "l_loop_activity_ledger_data_ids", + } + ] + + node = records[l_node_id]["payload"] + assert node["node_kind"] == "activity_ledger" + assert node["data_kind"] == "l_loop_activity_ledger" + assert node["source_graph_node_ids"] == ["graph:raw_capsule:turn_dry_001"] + assert node["source_data_ids"] == ["L:activity_ledger_frame"] + + edge = records[edge_id]["payload"] + assert edge["edge_kind"] == "HAS_ACTIVITY_LEDGER" + assert edge["from_node_id"] == "graph:raw_capsule:turn_dry_001" + assert edge["to_node_id"] == l_node_id + assert "L:activity_ledger_frame" in edge["source_data_ids"] + + +def test_raw_capsule_links_to_r_access_ledger_when_r_dry_run_runs() -> None: + result = run_dry_turn(enable_r_route_dry_run=True) + records = _records_by_id(result) + link = records[turn_activity_graph_link_frame_id("turn_dry_001")]["payload"] + r_ledger_id = "R:dry_run:turn_graph_access_ledger_frame" + r_node_id = activity_ledger_graph_node_id(r_ledger_id) + r_edge_id = activity_ledger_graph_edge_id( + raw_capsule_node_id="graph:raw_capsule:turn_dry_001", + activity_node_id=r_node_id, + ) + + assert link["l_loop_activity_ledger_data_ids"] == ["L:activity_ledger_frame"] + assert link["r_graph_access_ledger_data_ids"] == [r_ledger_id] + assert r_node_id in link["activity_ledger_graph_node_ids"] + assert r_edge_id in link["activity_ledger_graph_edge_ids"] + + node = records[r_node_id]["payload"] + assert node["node_kind"] == "activity_ledger" + assert node["data_kind"] == "r_graph_access_ledger" + assert node["source_data_ids"] == [r_ledger_id] + + edge = records[r_edge_id]["payload"] + assert edge["edge_kind"] == "HAS_ACTIVITY_LEDGER" + assert edge["from_node_id"] == "graph:raw_capsule:turn_dry_001" + assert edge["to_node_id"] == r_node_id + + +def test_turn_activity_graph_link_validator_rejects_semantic_status() -> None: + frame = TurnActivityGraphLinkFrame( + frame_id="graph:turn_activity_graph_link:turn_test", + turn_id="turn_test", + turn_capsule_graph_node_id="graph:raw_capsule:turn_test", + semantic_judgement_status="ran", + source_data_ids=["graph:raw_capsule:turn_test"], + ) + + with pytest.raises(ValueError, match="semantic_judgement_status"): + validate_turn_activity_graph_link_frame(frame) + + +def _records_by_id(result: dict[str, object]) -> dict[str, dict[str, object]]: + return { + item["data_id"]: item + for item in result["data_records"] + if isinstance(item, dict) + } diff --git a/tests/test_order_153_graph_memory_integrity.py b/tests/test_order_153_graph_memory_integrity.py new file mode 100644 index 0000000..94f4290 --- /dev/null +++ b/tests/test_order_153_graph_memory_integrity.py @@ -0,0 +1,99 @@ +from __future__ import annotations + +import json + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.graph_memory_integrity import audit_graph_memory_integrity +from songryeon_core.core.schemas import R_ROUTE_EXPERIMENTAL_NEXT_0_MODE +from songryeon_core.llm.base import LLMRequest, LLMResponse +from songryeon_core.runtime.dry_run import run_dry_turn + + +def test_default_dry_turn_graph_memory_integrity_passes() -> None: + result = run_dry_turn() + report = audit_graph_memory_integrity(_data_store_from_result(result)) + + assert report.passed, report.to_summary() + + +def test_r_experimental_route_records_graph_sources_before_handoff() -> None: + result = run_dry_turn( + user_input="그래프 기억을 R로 한 번만 실험해줘", + node_1_router_adapter=RRouteFakeAdapter(), + enable_r_route_experimental=True, + ) + graph_data_ids = result["r_route_experimental_graph_data_ids"] + report = audit_graph_memory_integrity(_data_store_from_result(result)) + + assert result["r_route_experimental_status"] == "selected" + assert "graph:snapshot:turn_dry_001:r_route_experimental" in graph_data_ids + assert ( + "rloop:graph_guide:graph:snapshot:turn_dry_001:r_route_experimental" + in graph_data_ids + ) + assert "graph:time_bundle:turn_dry_001:r_route_experimental" in graph_data_ids + assert report.passed, report.to_summary() + + +def test_graph_integrity_audit_reports_missing_source_ref() -> None: + data_store = DataStore() + data_store.create_record( + data_id="node_output:bad_graph_ref", + data_type="node_output:test", + payload={"source_data_ids": ["graph:missing:node"]}, + ) + + report = audit_graph_memory_integrity(data_store) + + assert not report.passed + assert report.missing_source_refs[0].missing_ref == "graph:missing:node" + assert report.missing_edge_endpoints == [] + + +def test_graph_integrity_audit_reports_missing_edge_endpoint() -> None: + data_store = DataStore() + data_store.create_record( + data_id="graph:edge:bad", + data_type="graph_memory:edge:CONTAINS", + payload={ + "edge_id": "graph:edge:bad", + "edge_kind": "CONTAINS", + "from_node_id": "graph:missing:from", + "to_node_id": "graph:missing:to", + "source_data_ids": [], + }, + ) + + report = audit_graph_memory_integrity(data_store) + + assert not report.passed + assert {issue.missing_ref for issue in report.missing_edge_endpoints} == { + "graph:missing:from", + "graph:missing:to", + } + + +class RRouteFakeAdapter: + model_id = "r-route-fake" + + def complete(self, request: LLMRequest) -> LLMResponse: + if "node_1 Router" not in request.prompt: + return LLMResponse(text="{}", model_id=self.model_id, raw={}) + payload = { + "route": "R", + "route_reason": "graph memory traversal looks useful", + "expected_next_0_mode": R_ROUTE_EXPERIMENTAL_NEXT_0_MODE, + "route_confidence": 0.71, + "needs_more_memory": False, + } + return LLMResponse( + text=json.dumps(payload, ensure_ascii=False), + model_id=self.model_id, + raw=payload, + ) + + +def _data_store_from_result(result: dict[str, object]) -> DataStore: + records = result["data_records"] + assert isinstance(records, list) + return DataStore.from_records(records) diff --git a/tests/test_order_154_fast_test_gate.py b/tests/test_order_154_fast_test_gate.py new file mode 100644 index 0000000..cccc453 --- /dev/null +++ b/tests/test_order_154_fast_test_gate.py @@ -0,0 +1,48 @@ +from __future__ import annotations + +import pytest + +from songryeon_core.runtime.fast_test import build_fast_test_steps, run_fast_tests + + +def test_fast_test_core_profile_is_small() -> None: + steps = build_fast_test_steps(profile="core") + + assert [step.name for step in steps] == ["compileall", "pytest:core"] + pytest_command = steps[-1].command + assert "tests/test_import_baseline.py" in pytest_command + assert "tests/test_schema_split_compat.py" in pytest_command + assert "tests/test_order_153_graph_memory_integrity.py" not in pytest_command + + +def test_fast_test_graph_profile_includes_current_graph_boundary_tests() -> None: + steps = build_fast_test_steps(profile="graph", skip_compileall=True) + + assert [step.name for step in steps] == ["pytest:graph"] + pytest_command = steps[0].command + assert "tests/test_order_146_r_route_experimental_gate.py" in pytest_command + assert "tests/test_order_147_r_result_to_node3_brief.py" in pytest_command + assert "tests/test_order_152_raw_capsule_activity_graph_link.py" in pytest_command + assert "tests/test_order_153_graph_memory_integrity.py" in pytest_command + assert "tests/test_order_155_graph_source_kind_ingest.py" in pytest_command + assert "tests/test_order_156_graph_source_observation_time_and_core_link.py" in pytest_command + assert "tests/test_order_157_songryeon_core_source_manifest.py" in pytest_command + assert "tests/test_order_158_graph_memory_export_packet.py" in pytest_command + assert "tests/test_order_159_vessel_adapter_boundary.py" in pytest_command + assert "tests/test_order_160_local_vessel_neo4j_writer.py" in pytest_command + assert "tests/test_order_164_source_version_lineage_and_invalidation.py" in pytest_command + assert "tests/test_order_165_same_content_observation_ledger.py" in pytest_command + + +def test_fast_test_rejects_unknown_profile() -> None: + with pytest.raises(ValueError, match="unknown fast-test profile"): + build_fast_test_steps(profile="unknown") + + +def test_fast_test_dry_run_does_not_execute_steps() -> None: + result = run_fast_tests(profile="core", dry_run=True) + + assert result["status"] == "FAST_TEST_PLAN" + assert result["profile"] == "core" + assert result["step_count"] == 2 + assert result["steps"][0]["name"] == "compileall" diff --git a/tests/test_order_155_graph_source_kind_ingest.py b/tests/test_order_155_graph_source_kind_ingest.py new file mode 100644 index 0000000..6e17515 --- /dev/null +++ b/tests/test_order_155_graph_source_kind_ingest.py @@ -0,0 +1,273 @@ +from __future__ import annotations + +import pytest + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.graph_memory import record_graph_memory_for_capsules +from songryeon_core.core.graph_memory_integrity import audit_graph_memory_integrity +from songryeon_core.core.graph_source_ingest import ( + GRAPH_SOURCE_FILE_DATA_TYPE, + GRAPH_SOURCE_INGEST_FRAME_DATA_TYPE, + GRAPH_SOURCE_KIND_INGEST_GENERATOR, + record_graph_source_kind_ingest, +) +from songryeon_core.core.schemas import NodeMovement, TurnStateCapsule +from songryeon_core.core.trace_store import TraceStore + + +FIXED_OBSERVED_AT = "2026-07-01T12:00:00" +FIXED_INGESTED_AT = "2026-07-01T12:00:01" + + +def test_source_kind_ingest_creates_separate_bundles(tmp_path) -> None: + internal_doc = tmp_path / "ORDER_155_NOTE.md" + source_code = tmp_path / "module.py" + external_file = tmp_path / "external.md" + internal_doc.write_text("# internal\n문서 좌표\n", encoding="utf-8") + source_code.write_text("def run():\n return 1\n", encoding="utf-8") + external_file.write_text("external project note\n", encoding="utf-8") + + trace_store = TraceStore() + data_store = _data_store_with_core_time_axis(trace_store) + + result = record_graph_source_kind_ingest( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_155", + batch_id="batch_order_155", + source_paths_by_kind={ + "internal_document": [internal_doc], + "source_code_file": [source_code], + "external_project_file": [external_file], + }, + observed_at=FIXED_OBSERVED_AT, + ingested_at=FIXED_INGESTED_AT, + ) + + assert result.source_kind_counts == { + "external_project_file": 1, + "internal_document": 1, + "source_code_file": 1, + } + assert len(result.source_kind_bundle_node_ids) == 3 + assert len(result.raw_source_node_ids) == 3 + assert len(result.graph_edge_ids) == 7 + + bundle_records = [ + record + for record in data_store.list_records() + if record.data_type == "graph_memory:node:source_kind_bundle" + ] + assert len(bundle_records) == 3 + for record in bundle_records: + payload = record.payload + assert isinstance(payload, dict) + source_kind = payload["source_bundle_kind"] + assert payload["node_kind"] == "source_kind_bundle" + assert payload["data_kind"] == f"{source_kind}_bundle" + assert payload["source_leaf_count"] == 1 + assert payload["generated_by"] == "CODE:GRAPH_MEMORY_BUILDER" + assert payload["info_class"] == "absolute" + assert payload["semantic_judgement_status"] == "not_run" + + child_id = payload["source_graph_node_ids"][0] + child_record = data_store.require_record(child_id) + child_payload = child_record.payload + assert isinstance(child_payload, dict) + assert child_payload["node_kind"] == "raw_source" + assert child_payload["data_kind"] == source_kind + + frame = data_store.require_record(result.frame_id) + assert frame.data_type == GRAPH_SOURCE_INGEST_FRAME_DATA_TYPE + assert frame.payload["generated_by"] == GRAPH_SOURCE_KIND_INGEST_GENERATOR + assert frame.payload["info_class"] == "absolute" + assert frame.payload["semantic_judgement_status"] == "not_run" + + report = audit_graph_memory_integrity(data_store) + assert report.passed, report.to_summary() + + +def test_raw_source_records_absolute_file_coordinates_only(tmp_path) -> None: + source = tmp_path / "policy.md" + source.write_text("절대 좌표만 저장한다.\n", encoding="utf-8") + + data_store = _data_store_with_core_time_axis() + result = record_graph_source_kind_ingest( + trace_store=TraceStore(), + data_store=data_store, + turn_id="turn_order_155", + batch_id="batch_order_155_single", + source_paths_by_kind={"internal_document": [source]}, + observed_at=FIXED_OBSERVED_AT, + ingested_at=FIXED_INGESTED_AT, + ) + + raw_record = data_store.require_record(result.raw_source_node_ids[0]) + raw_payload = raw_record.payload + assert isinstance(raw_payload, dict) + assert raw_payload["node_kind"] == "raw_source" + assert raw_payload["data_kind"] == "internal_document" + assert raw_payload["source_leaf_count"] == 1 + assert raw_payload["source_summary_count"] == 0 + assert raw_payload["semantic_judgement_status"] == "not_run" + + file_record = data_store.require_record(result.source_file_data_ids[0]) + file_payload = file_record.payload + assert file_record.data_type == GRAPH_SOURCE_FILE_DATA_TYPE + assert isinstance(file_payload, dict) + assert file_payload["source_kind"] == "internal_document" + assert file_payload["path"].endswith("policy.md") + assert file_payload["path_name"] == "policy.md" + assert file_payload["suffix"] == ".md" + assert file_payload["exists"] is True + assert file_payload["exists_at_ingest"] is True + assert file_payload["char_count"] == len("절대 좌표만 저장한다.\n") + assert file_payload["observed_at"] == FIXED_OBSERVED_AT + assert file_payload["ingested_at"] == FIXED_INGESTED_AT + assert file_payload["source_last_modified_at"] + assert file_payload["content_sha1"] + assert file_payload["generated_by"] == GRAPH_SOURCE_KIND_INGEST_GENERATOR + + for forbidden_field in ( + "summary", + "semantic_topic", + "importance_score", + "importance_reason", + "relevance_reason", + "meaning_cluster", + ): + assert forbidden_field not in raw_payload + assert forbidden_field not in file_payload + + +def test_same_file_in_two_source_kinds_is_not_merged(tmp_path) -> None: + source = tmp_path / "shared.txt" + source.write_text("same bytes, different source kind\n", encoding="utf-8") + + data_store = _data_store_with_core_time_axis() + result = record_graph_source_kind_ingest( + trace_store=TraceStore(), + data_store=data_store, + turn_id="turn_order_155", + batch_id="batch_order_155_shared", + source_paths_by_kind={ + "internal_document": [source], + "external_project_file": [source], + }, + observed_at=FIXED_OBSERVED_AT, + ingested_at=FIXED_INGESTED_AT, + ) + + assert len(result.source_file_data_ids) == 2 + assert len(set(result.source_file_data_ids)) == 2 + assert len(result.raw_source_node_ids) == 2 + assert len(set(result.raw_source_node_ids)) == 2 + + raw_kinds = { + data_store.require_record(node_id).payload["data_kind"] + for node_id in result.raw_source_node_ids + } + assert raw_kinds == {"internal_document", "external_project_file"} + + +def test_duplicate_paths_within_one_kind_are_deduped(tmp_path) -> None: + source = tmp_path / "duplicate.md" + source.write_text("dedupe me\n", encoding="utf-8") + + result = record_graph_source_kind_ingest( + trace_store=TraceStore(), + data_store=_data_store_with_core_time_axis(), + turn_id="turn_order_155", + batch_id="batch_order_155_dedupe", + source_paths_by_kind={"internal_document": [source, source]}, + observed_at=FIXED_OBSERVED_AT, + ingested_at=FIXED_INGESTED_AT, + ) + + assert result.source_kind_counts == {"internal_document": 1} + assert len(result.source_file_data_ids) == 1 + assert len(result.raw_source_node_ids) == 1 + assert len(result.source_kind_bundle_node_ids) == 1 + assert len(result.graph_edge_ids) == 3 + + +def test_unknown_source_kind_is_rejected(tmp_path) -> None: + source = tmp_path / "bad.txt" + source.write_text("bad kind\n", encoding="utf-8") + + with pytest.raises(ValueError, match="unknown graph source kind"): + record_graph_source_kind_ingest( + trace_store=TraceStore(), + data_store=DataStore(), + turn_id="turn_order_155", + batch_id="batch_order_155_bad_kind", + source_paths_by_kind={"semantic_topic": [source]}, + ) + + +def test_recording_same_batch_twice_is_idempotent(tmp_path) -> None: + source = tmp_path / "stable.md" + source.write_text("stable content\n", encoding="utf-8") + trace_store = TraceStore() + data_store = _data_store_with_core_time_axis(trace_store) + + first = record_graph_source_kind_ingest( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_155", + batch_id="batch_order_155_idempotent", + source_paths_by_kind={"internal_document": [source]}, + observed_at=FIXED_OBSERVED_AT, + ingested_at=FIXED_INGESTED_AT, + ) + second = record_graph_source_kind_ingest( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_155", + batch_id="batch_order_155_idempotent", + source_paths_by_kind={"internal_document": [source]}, + observed_at=FIXED_OBSERVED_AT, + ingested_at=FIXED_INGESTED_AT, + ) + + assert first.created_data_ids + assert not second.created_data_ids + assert set(second.existing_data_ids) == set(first.created_data_ids) + + +def _data_store_with_core_time_axis(trace_store: TraceStore | None = None) -> DataStore: + trace_store = trace_store or TraceStore() + data_store = DataStore() + record_graph_memory_for_capsules( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_155_core", + batch_id="batch_order_155_core", + capsules=[_sample_capsule()], + ) + return data_store + + +def _sample_capsule(turn_id: str = "turn_order_155_previous") -> TurnStateCapsule: + return TurnStateCapsule( + turn_id=turn_id, + node_movements=[ + NodeMovement( + movement_id=f"move:{turn_id}:001", + turn_id=turn_id, + step_index=1, + node_id="node_0", + mode="pre_route_report", + input_trace_ids=[f"trace:{turn_id}:user"], + output_trace_ids=[f"trace:{turn_id}:node0"], + status="completed", + ) + ], + trace_event_ids=[ + f"trace:{turn_id}:user", + f"trace:{turn_id}:node0", + f"trace:{turn_id}:final", + ], + user_input_trace_id=f"trace:{turn_id}:user", + final_response_trace_id=f"trace:{turn_id}:final", + ) diff --git a/tests/test_order_156_graph_source_observation_time_and_core_link.py b/tests/test_order_156_graph_source_observation_time_and_core_link.py new file mode 100644 index 0000000..c98f439 --- /dev/null +++ b/tests/test_order_156_graph_source_observation_time_and_core_link.py @@ -0,0 +1,218 @@ +from __future__ import annotations + +import pytest + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.graph_memory import ( + TIME_AXIS_NODE_ID, + record_graph_memory_for_capsules, +) +from songryeon_core.core.graph_memory_integrity import audit_graph_memory_integrity +from songryeon_core.core.graph_source_ingest import ( + graph_source_ingest_snapshot_id, + record_graph_source_kind_ingest, + source_ingest_time_bundle_graph_node_id, +) +from songryeon_core.core.schemas import NodeMovement, TurnStateCapsule +from songryeon_core.core.trace_store import TraceStore + + +OBSERVED_AT = "2026-07-01T13:00:00" +INGESTED_AT = "2026-07-01T13:00:01" + + +def test_source_ingest_records_observation_time_on_file_and_graph_node(tmp_path) -> None: + source = tmp_path / "changing_policy.md" + source.write_text("version one\n", encoding="utf-8") + trace_store, data_store = _core_time_axis_store() + + result = record_graph_source_kind_ingest( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_156", + batch_id="batch_order_156", + source_paths_by_kind={"internal_document": [source]}, + observed_at=OBSERVED_AT, + ingested_at=INGESTED_AT, + ) + + file_record = data_store.require_record(result.source_file_data_ids[0]) + raw_record = data_store.require_record(result.raw_source_node_ids[0]) + assert isinstance(file_record.payload, dict) + assert isinstance(raw_record.payload, dict) + + for payload in (file_record.payload, raw_record.payload): + assert payload["observed_at"] == OBSERVED_AT + assert payload["ingested_at"] == INGESTED_AT + assert payload["source_last_modified_at"] + assert payload["exists_at_ingest"] is True + assert payload["content_sha1"] + + assert raw_record.payload["node_kind"] == "raw_source" + assert raw_record.payload["semantic_judgement_status"] == "not_run" + assert raw_record.payload["info_class"] == "absolute" + + +def test_source_kind_bundles_are_linked_under_core_ego_time_axis(tmp_path) -> None: + internal_doc = tmp_path / "doc.md" + code_file = tmp_path / "tool.py" + internal_doc.write_text("doc material\n", encoding="utf-8") + code_file.write_text("print('tool')\n", encoding="utf-8") + trace_store, data_store = _core_time_axis_store() + + result = record_graph_source_kind_ingest( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_156", + batch_id="batch_order_156_link", + source_paths_by_kind={ + "internal_document": [internal_doc], + "source_code_file": [code_file], + }, + observed_at=OBSERVED_AT, + ingested_at=INGESTED_AT, + ) + + source_ingest_node = data_store.require_record( + result.source_ingest_time_bundle_node_id + ) + assert source_ingest_node.payload["node_kind"] == "source_ingest_time_bundle" + assert source_ingest_node.payload["source_graph_node_ids"] == result.source_kind_bundle_node_ids + + edge_records = [ + record + for record in data_store.list_records() + if record.data_type.startswith("graph_memory:edge:") + ] + edge_pairs = { + (record.payload["edge_kind"], record.payload["from_node_id"], record.payload["to_node_id"]) + for record in edge_records + } + assert ( + "CHILD_OF_TIME_AXIS", + TIME_AXIS_NODE_ID, + result.source_ingest_time_bundle_node_id, + ) in edge_pairs + for bundle_id in result.source_kind_bundle_node_ids: + assert ("CONTAINS", result.source_ingest_time_bundle_node_id, bundle_id) in edge_pairs + + report = audit_graph_memory_integrity(data_store) + assert report.passed, report.to_summary() + + +def test_source_ingest_creates_source_snapshot_and_rloop_guide(tmp_path) -> None: + source = tmp_path / "guide.md" + source.write_text("guide me\n", encoding="utf-8") + trace_store, data_store = _core_time_axis_store() + + result = record_graph_source_kind_ingest( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_156", + batch_id="batch_order_156_guide", + source_paths_by_kind={"internal_document": [source]}, + observed_at=OBSERVED_AT, + ingested_at=INGESTED_AT, + ) + + snapshot = data_store.require_record(result.graph_snapshot_id) + guide = data_store.require_record(result.rloop_graph_guide_packet_id) + assert result.graph_snapshot_id == graph_source_ingest_snapshot_id("batch_order_156_guide") + assert snapshot.payload["root_node_id"] == "graph:core_ego:root" + assert snapshot.payload["time_axis_node_id"] == TIME_AXIS_NODE_ID + assert snapshot.payload["node_kind_counts"]["source_ingest_time_bundle"] == 1 + assert snapshot.payload["node_kind_counts"]["source_kind_bundle"] == 1 + assert snapshot.payload["node_kind_counts"]["raw_source"] == 1 + assert result.source_ingest_time_bundle_node_id in snapshot.payload["graph_node_ids"] + + assert guide.payload["graph_snapshot_id"] == result.graph_snapshot_id + assert guide.payload["available_entry_nodes"] == [TIME_AXIS_NODE_ID] + assert guide.payload["recommended_traversal_hints_status"] == "not_run" + assert guide.payload["semantic_judgement_status"] == "not_run" + + +def test_source_ingest_requires_existing_core_ego_time_axis(tmp_path) -> None: + source = tmp_path / "orphan.md" + source.write_text("orphan source\n", encoding="utf-8") + + with pytest.raises(ValueError, match="requires existing CoreEgo time axis"): + record_graph_source_kind_ingest( + trace_store=TraceStore(), + data_store=DataStore(), + turn_id="turn_order_156", + batch_id="batch_order_156_orphan", + source_paths_by_kind={"internal_document": [source]}, + observed_at=OBSERVED_AT, + ingested_at=INGESTED_AT, + ) + + +def test_same_file_same_content_check_reuses_raw_source_snapshot(tmp_path) -> None: + source = tmp_path / "same_content.md" + source.write_text("same content\n", encoding="utf-8") + trace_store, data_store = _core_time_axis_store() + + first = record_graph_source_kind_ingest( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_156", + batch_id="batch_order_156_obs_001", + source_paths_by_kind={"internal_document": [source]}, + observed_at="2026-07-01T14:00:00", + ingested_at="2026-07-01T14:00:01", + ) + second = record_graph_source_kind_ingest( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_156", + batch_id="batch_order_156_obs_002", + source_paths_by_kind={"internal_document": [source]}, + observed_at="2026-07-01T15:00:00", + ingested_at="2026-07-01T15:00:01", + ) + + assert first.raw_source_node_ids == second.raw_source_node_ids + assert first.source_file_data_ids != second.source_file_data_ids + assert ( + data_store.require_record(first.raw_source_node_ids[0]).payload["content_sha1"] + == data_store.require_record(second.raw_source_node_ids[0]).payload["content_sha1"] + ) + assert second.source_observation_status_counts == {"unchanged": 1} + + +def _core_time_axis_store() -> tuple[TraceStore, DataStore]: + trace_store = TraceStore() + data_store = DataStore() + record_graph_memory_for_capsules( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_156_core", + batch_id="batch_order_156_core", + capsules=[_sample_capsule()], + ) + return trace_store, data_store + + +def _sample_capsule(turn_id: str = "turn_order_156_previous") -> TurnStateCapsule: + return TurnStateCapsule( + turn_id=turn_id, + node_movements=[ + NodeMovement( + movement_id=f"move:{turn_id}:001", + turn_id=turn_id, + step_index=1, + node_id="node_0", + mode="pre_route_report", + input_trace_ids=[f"trace:{turn_id}:user"], + output_trace_ids=[f"trace:{turn_id}:node0"], + status="completed", + ) + ], + trace_event_ids=[ + f"trace:{turn_id}:user", + f"trace:{turn_id}:node0", + f"trace:{turn_id}:final", + ], + user_input_trace_id=f"trace:{turn_id}:user", + final_response_trace_id=f"trace:{turn_id}:final", + ) diff --git a/tests/test_order_157_songryeon_core_source_manifest.py b/tests/test_order_157_songryeon_core_source_manifest.py new file mode 100644 index 0000000..94cdbfa --- /dev/null +++ b/tests/test_order_157_songryeon_core_source_manifest.py @@ -0,0 +1,229 @@ +from __future__ import annotations + +import pytest + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.graph_memory import record_graph_memory_for_capsules +from songryeon_core.core.graph_memory_integrity import audit_graph_memory_integrity +from songryeon_core.core.graph_source_ingest import GRAPH_SOURCE_TEXT_SNAPSHOT_DATA_TYPE +from songryeon_core.core.schemas import NodeMovement, TurnStateCapsule +from songryeon_core.core.songryeon_source_manifest import ( + SONGRYEON_CORE_SOURCE_MANIFEST_DATA_TYPE, + SONGRYEON_CORE_SOURCE_MANIFEST_GENERATOR, + record_songryeon_core_source_manifest_ingest, + resolve_songryeon_core_source_manifest, +) +from songryeon_core.core.trace_store import TraceStore + + +OBSERVED_AT = "2026-07-01T16:00:00" +INGESTED_AT = "2026-07-01T16:00:01" + + +def test_manifest_resolves_internal_docs_source_code_and_tests(tmp_path) -> None: + root = _sample_repo(tmp_path) + + manifest = resolve_songryeon_core_source_manifest( + root_path=root, + batch_id="batch_order_157", + ) + + assert manifest.generated_by == SONGRYEON_CORE_SOURCE_MANIFEST_GENERATOR + assert manifest.info_class == "absolute" + assert manifest.semantic_judgement_status == "not_run" + assert manifest.source_kind_counts == { + "internal_document": 4, + "source_code_file": 3, + } + internal_names = {path.split("/")[-1] for path in manifest.source_paths_by_kind["internal_document"]} + code_names = {path.split("/")[-1] for path in manifest.source_paths_by_kind["source_code_file"]} + assert {"AGENTS.md", "README.md", "ORDER_001.md", "note.md"}.issubset(internal_names) + assert {"main.py", "module.py", "test_module.py"}.issubset(code_names) + assert "external_project_file" not in manifest.source_paths_by_kind + + +def test_manifest_ingest_stores_text_snapshots_and_core_link(tmp_path) -> None: + root = _sample_repo(tmp_path) + trace_store, data_store = _core_time_axis_store() + + result = record_songryeon_core_source_manifest_ingest( + trace_store=trace_store, + data_store=data_store, + root_path=root, + turn_id="turn_order_157", + batch_id="batch_order_157", + observed_at=OBSERVED_AT, + ingested_at=INGESTED_AT, + ) + + manifest_record = data_store.require_record(result.manifest_data_id) + assert manifest_record.data_type == SONGRYEON_CORE_SOURCE_MANIFEST_DATA_TYPE + assert manifest_record.payload["source_kind_counts"] == { + "internal_document": 4, + "source_code_file": 3, + } + + ingest = result.ingest_result + assert ingest.source_kind_counts == { + "internal_document": 4, + "source_code_file": 3, + } + assert len(ingest.source_text_snapshot_data_ids) == 7 + assert ingest.source_ingest_time_bundle_node_id + assert ingest.graph_snapshot_id + assert ingest.rloop_graph_guide_packet_id + + text_records = [ + data_store.require_record(data_id) + for data_id in ingest.source_text_snapshot_data_ids + ] + assert {record.data_type for record in text_records} == {GRAPH_SOURCE_TEXT_SNAPSHOT_DATA_TYPE} + payloads = [record.payload for record in text_records] + assert all(payload["info_class"] == "absolute_copied_source" for payload in payloads) + assert all(payload["semantic_judgement_status"] == "not_run" for payload in payloads) + assert all(payload["observed_at"] == OBSERVED_AT for payload in payloads) + assert any(payload["text"] == "# agent rules\n" for payload in payloads) + assert any(payload["text"] == "def run():\n return 1\n" for payload in payloads) + + frame = data_store.require_record(ingest.frame_id) + assert frame.payload["text_snapshot_status"] == "stored" + assert set(frame.payload["source_text_snapshot_data_ids"]) == set( + ingest.source_text_snapshot_data_ids + ) + + report = audit_graph_memory_integrity(data_store) + assert report.passed, report.to_summary() + + +def test_manifest_missing_explicit_path_fails(tmp_path) -> None: + root = tmp_path / "repo" + root.mkdir() + (root / "README.md").write_text("# readme\n", encoding="utf-8") + (root / "main.py").write_text("print('main')\n", encoding="utf-8") + + with pytest.raises(FileNotFoundError, match="AGENTS.md"): + resolve_songryeon_core_source_manifest( + root_path=root, + batch_id="batch_order_157_missing", + ) + + +def test_manifest_glob_absence_is_not_missing_explicit_failure(tmp_path) -> None: + root = tmp_path / "repo" + root.mkdir() + (root / "AGENTS.md").write_text("# agent rules\n", encoding="utf-8") + (root / "README.md").write_text("# readme\n", encoding="utf-8") + (root / "main.py").write_text("print('main')\n", encoding="utf-8") + + manifest = resolve_songryeon_core_source_manifest( + root_path=root, + batch_id="batch_order_157_no_globs", + ) + + assert manifest.source_kind_counts == { + "internal_document": 2, + "source_code_file": 1, + } + + +def test_manifest_ingest_does_not_create_semantic_fields(tmp_path) -> None: + root = _sample_repo(tmp_path) + trace_store, data_store = _core_time_axis_store() + + result = record_songryeon_core_source_manifest_ingest( + trace_store=trace_store, + data_store=data_store, + root_path=root, + turn_id="turn_order_157", + batch_id="batch_order_157_boundary", + observed_at=OBSERVED_AT, + ingested_at=INGESTED_AT, + ) + + checked_payloads = [ + data_store.require_record(data_id).payload + for data_id in [ + result.manifest_data_id, + result.ingest_result.frame_id, + *result.ingest_result.raw_source_node_ids, + *result.ingest_result.source_text_snapshot_data_ids, + ] + ] + for payload in checked_payloads: + for forbidden_field in ( + "summary", + "semantic_topic", + "importance_score", + "importance_reason", + "relevance_reason", + "meaning_cluster", + ): + assert forbidden_field not in payload + + +def _sample_repo(tmp_path): + root = tmp_path / "repo" + (root / "Administrative_Reform_1" / "04_Orders").mkdir(parents=True) + (root / "Administrative_Reform_1" / "05_Execution_Records").mkdir(parents=True) + (root / "songryeon_core").mkdir() + (root / "tests").mkdir() + + (root / "AGENTS.md").write_text("# agent rules\n", encoding="utf-8") + (root / "README.md").write_text("# readme\n", encoding="utf-8") + (root / "Administrative_Reform_1" / "04_Orders" / "ORDER_001.md").write_text( + "# order\n", + encoding="utf-8", + ) + (root / "Administrative_Reform_1" / "05_Execution_Records" / "note.md").write_text( + "# note\n", + encoding="utf-8", + ) + (root / "main.py").write_text("print('main')\n", encoding="utf-8") + (root / "songryeon_core" / "module.py").write_text( + "def run():\n return 1\n", + encoding="utf-8", + ) + (root / "tests" / "test_module.py").write_text( + "def test_run():\n assert True\n", + encoding="utf-8", + ) + (root / "external.md").write_text("# external\n", encoding="utf-8") + return root + + +def _core_time_axis_store() -> tuple[TraceStore, DataStore]: + trace_store = TraceStore() + data_store = DataStore() + record_graph_memory_for_capsules( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_157_core", + batch_id="batch_order_157_core", + capsules=[_sample_capsule()], + ) + return trace_store, data_store + + +def _sample_capsule(turn_id: str = "turn_order_157_previous") -> TurnStateCapsule: + return TurnStateCapsule( + turn_id=turn_id, + node_movements=[ + NodeMovement( + movement_id=f"move:{turn_id}:001", + turn_id=turn_id, + step_index=1, + node_id="node_0", + mode="pre_route_report", + input_trace_ids=[f"trace:{turn_id}:user"], + output_trace_ids=[f"trace:{turn_id}:node0"], + status="completed", + ) + ], + trace_event_ids=[ + f"trace:{turn_id}:user", + f"trace:{turn_id}:node0", + f"trace:{turn_id}:final", + ], + user_input_trace_id=f"trace:{turn_id}:user", + final_response_trace_id=f"trace:{turn_id}:final", + ) diff --git a/tests/test_order_158_graph_memory_export_packet.py b/tests/test_order_158_graph_memory_export_packet.py new file mode 100644 index 0000000..9d833ea --- /dev/null +++ b/tests/test_order_158_graph_memory_export_packet.py @@ -0,0 +1,222 @@ +from __future__ import annotations + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.graph_memory import record_graph_memory_for_capsules +from songryeon_core.core.graph_memory_export import ( + GRAPH_MEMORY_EXPORT_PACKET_DATA_TYPE, + GRAPH_MEMORY_EXPORT_PACKET_GENERATOR, + GRAPH_MEMORY_EXPORT_TARGET_ADAPTER, + graph_memory_export_packet_id, + record_graph_memory_export_packet, +) +from songryeon_core.core.graph_source_ingest import ( + GRAPH_SOURCE_TEXT_SNAPSHOT_DATA_TYPE, +) +from songryeon_core.core.schemas import NodeMovement, TurnStateCapsule +from songryeon_core.core.songryeon_source_manifest import ( + SONGRYEON_CORE_SOURCE_MANIFEST_DATA_TYPE, + record_songryeon_core_source_manifest_ingest, +) +from songryeon_core.core.trace_store import TraceStore + + +OBSERVED_AT = "2026-07-01T18:00:00" +INGESTED_AT = "2026-07-01T18:00:01" +EXPORTED_AT = "2026-07-01T18:00:02" + + +def test_export_packet_collects_graph_and_source_records(tmp_path) -> None: + root = _sample_repo(tmp_path) + trace_store, data_store = _graph_and_source_store(root) + + result = record_graph_memory_export_packet( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_158", + batch_id="batch_order_158_export", + created_at=EXPORTED_AT, + ) + + packet = result.packet + assert packet.packet_id == graph_memory_export_packet_id("batch_order_158_export") + assert packet.generated_by == GRAPH_MEMORY_EXPORT_PACKET_GENERATOR + assert packet.info_class == "absolute" + assert packet.semantic_judgement_status == "not_run" + assert packet.external_write_status == "not_run" + assert packet.target_adapter_name == GRAPH_MEMORY_EXPORT_TARGET_ADAPTER + assert packet.graph_integrity_status == "passed" + assert packet.graph_integrity_summary["passed"] is True + + assert packet.graph_node_data_ids + assert packet.graph_edge_data_ids + assert packet.graph_snapshot_data_ids + assert packet.core_ego_time_axis_frame_ids + assert packet.rloop_guide_packet_data_ids + assert packet.source_file_metadata_data_ids + assert packet.source_text_snapshot_data_ids + assert packet.source_ingest_frame_data_ids + assert packet.source_manifest_frame_data_ids + assert set(packet.source_text_snapshot_data_ids).issubset(packet.included_data_ids) + + record = data_store.require_record(packet.packet_id) + assert record.data_type == GRAPH_MEMORY_EXPORT_PACKET_DATA_TYPE + assert record.payload["included_data_ids"] == packet.included_data_ids + assert result.created_data_ids == [packet.packet_id] + assert data_store.require_record(packet.source_text_snapshot_data_ids[0]).data_type == ( + GRAPH_SOURCE_TEXT_SNAPSHOT_DATA_TYPE + ) + assert data_store.require_record(packet.source_manifest_frame_data_ids[0]).data_type == ( + SONGRYEON_CORE_SOURCE_MANIFEST_DATA_TYPE + ) + + +def test_export_packet_excludes_unrelated_datastore_records(tmp_path) -> None: + root = _sample_repo(tmp_path) + trace_store, data_store = _graph_and_source_store(root) + data_store.create_record( + data_id="node_output:unrelated", + data_type="node_output:test", + payload={"text": "not graph memory"}, + ) + + result = record_graph_memory_export_packet( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_158", + batch_id="batch_order_158_exclude", + created_at=EXPORTED_AT, + ) + + assert "node_output:unrelated" not in result.packet.included_data_ids + assert "node_output:test" not in result.packet.data_type_counts + + +def test_export_packet_records_failed_integrity_without_writing_external_db() -> None: + trace_store = TraceStore() + data_store = DataStore() + data_store.create_record( + data_id="graph:edge:contains:graph:missing_parent:graph:missing_child", + data_type="graph_memory:edge:CONTAINS", + payload={ + "edge_id": "graph:edge:contains:graph:missing_parent:graph:missing_child", + "edge_kind": "CONTAINS", + "from_node_id": "graph:missing_parent", + "to_node_id": "graph:missing_child", + "source_data_ids": ["graph:missing_parent", "graph:missing_child"], + }, + ) + + result = record_graph_memory_export_packet( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_158", + batch_id="batch_order_158_failed_integrity", + created_at=EXPORTED_AT, + ) + + packet = result.packet + assert packet.graph_integrity_status == "failed" + assert packet.graph_integrity_summary["missing_source_ref_count"] == 2 + assert packet.graph_integrity_summary["missing_edge_endpoint_count"] == 2 + assert packet.external_write_status == "not_run" + + +def test_export_packet_is_idempotent_for_same_batch_and_timestamp(tmp_path) -> None: + root = _sample_repo(tmp_path) + trace_store, data_store = _graph_and_source_store(root) + + first = record_graph_memory_export_packet( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_158", + batch_id="batch_order_158_idempotent", + created_at=EXPORTED_AT, + ) + second = record_graph_memory_export_packet( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_158", + batch_id="batch_order_158_idempotent", + created_at=EXPORTED_AT, + ) + + assert first.created_data_ids == [first.packet.packet_id] + assert second.created_data_ids == [] + assert second.existing_data_ids == [first.packet.packet_id] + assert first.packet == second.packet + + +def _graph_and_source_store(root) -> tuple[TraceStore, DataStore]: + trace_store = TraceStore() + data_store = DataStore() + record_graph_memory_for_capsules( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_158_core", + batch_id="batch_order_158_core", + capsules=[_sample_capsule("turn_order_158_previous")], + ) + record_songryeon_core_source_manifest_ingest( + trace_store=trace_store, + data_store=data_store, + root_path=root, + turn_id="turn_order_158_source", + batch_id="batch_order_158_source", + observed_at=OBSERVED_AT, + ingested_at=INGESTED_AT, + ) + return trace_store, data_store + + +def _sample_repo(tmp_path): + root = tmp_path / "repo" + (root / "Administrative_Reform_1" / "04_Orders").mkdir(parents=True) + (root / "Administrative_Reform_1" / "05_Execution_Records").mkdir(parents=True) + (root / "songryeon_core").mkdir() + (root / "tests").mkdir() + + (root / "AGENTS.md").write_text("# agent rules\n", encoding="utf-8") + (root / "README.md").write_text("# readme\n", encoding="utf-8") + (root / "Administrative_Reform_1" / "04_Orders" / "ORDER_001.md").write_text( + "# order\n", + encoding="utf-8", + ) + (root / "Administrative_Reform_1" / "05_Execution_Records" / "note.md").write_text( + "# note\n", + encoding="utf-8", + ) + (root / "main.py").write_text("print('main')\n", encoding="utf-8") + (root / "songryeon_core" / "module.py").write_text( + "def run():\n return 1\n", + encoding="utf-8", + ) + (root / "tests" / "test_module.py").write_text( + "def test_run():\n assert True\n", + encoding="utf-8", + ) + return root + + +def _sample_capsule(turn_id: str) -> TurnStateCapsule: + return TurnStateCapsule( + turn_id=turn_id, + node_movements=[ + NodeMovement( + movement_id=f"move:{turn_id}:001", + turn_id=turn_id, + step_index=1, + node_id="node_0", + mode="pre_route_report", + input_trace_ids=[f"trace:{turn_id}:user"], + output_trace_ids=[f"trace:{turn_id}:node0"], + status="completed", + ) + ], + trace_event_ids=[ + f"trace:{turn_id}:user", + f"trace:{turn_id}:node0", + f"trace:{turn_id}:final", + ], + user_input_trace_id=f"trace:{turn_id}:user", + final_response_trace_id=f"trace:{turn_id}:final", + ) diff --git a/tests/test_order_159_vessel_adapter_boundary.py b/tests/test_order_159_vessel_adapter_boundary.py new file mode 100644 index 0000000..8e0a69a --- /dev/null +++ b/tests/test_order_159_vessel_adapter_boundary.py @@ -0,0 +1,279 @@ +from __future__ import annotations + +from dataclasses import asdict + +import pytest + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.graph_memory import record_graph_memory_for_capsules +from songryeon_core.core.graph_memory_export import ( + GRAPH_MEMORY_EXPORT_PACKET_DATA_TYPE, + record_graph_memory_export_packet, +) +from songryeon_core.core.graph_memory_store import ( + SONGRYEON_GRAPH_NAMESPACE, + SONGRYEON_VESSEL_DATABASE_NAME, + SONGRYEON_VESSEL_SERVICE_NAME, +) +from songryeon_core.core.graph_vessel_adapter import ( + GRAPH_VESSEL_ADAPTER_BOUNDARY_GENERATOR, + GRAPH_VESSEL_WRITE_PLAN_DATA_TYPE, + graph_vessel_write_plan_id, + record_graph_vessel_write_plan, +) +from songryeon_core.core.schemas import NodeMovement, TurnStateCapsule +from songryeon_core.core.songryeon_source_manifest import ( + record_songryeon_core_source_manifest_ingest, +) +from songryeon_core.core.trace_store import TraceStore + + +OBSERVED_AT = "2026-07-01T19:00:00" +INGESTED_AT = "2026-07-01T19:00:01" +EXPORTED_AT = "2026-07-01T19:00:02" +PLANNED_AT = "2026-07-01T19:00:03" + + +def test_vessel_write_plan_builds_ordered_no_write_operations(tmp_path) -> None: + root = _sample_repo(tmp_path) + trace_store, data_store, export_packet_id = _store_with_export_packet(root) + + result = record_graph_vessel_write_plan( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_159", + export_packet_id=export_packet_id, + created_at=PLANNED_AT, + ) + + plan = result.plan + assert plan.plan_id == graph_vessel_write_plan_id("batch_order_159_export") + assert plan.export_packet_id == export_packet_id + assert plan.generated_by == GRAPH_VESSEL_ADAPTER_BOUNDARY_GENERATOR + assert plan.info_class == "absolute" + assert plan.semantic_judgement_status == "not_run" + assert plan.external_write_status == "not_run" + assert plan.plan_status == "ready_to_write" + assert plan.block_reason is None + assert plan.target_adapter_name == SONGRYEON_VESSEL_SERVICE_NAME + assert plan.vessel_database_name == SONGRYEON_VESSEL_DATABASE_NAME + assert plan.graph_namespace == SONGRYEON_GRAPH_NAMESPACE + assert plan.operation_count == len(plan.operations) + assert plan.operation_count_by_kind["upsert_graph_node"] > 0 + assert plan.operation_count_by_kind["upsert_graph_edge"] > 0 + assert plan.operation_count_by_kind["upsert_source_payload"] > 0 + assert plan.operation_count_by_kind["upsert_support_record"] > 0 + assert {operation.external_write_status for operation in plan.operations} == {"not_run"} + + operation_kinds = [operation.operation_kind for operation in plan.operations] + assert operation_kinds == sorted( + operation_kinds, + key={ + "upsert_source_payload": 0, + "upsert_graph_node": 1, + "upsert_graph_edge": 2, + "upsert_support_record": 3, + }.__getitem__, + ) + + record = data_store.require_record(plan.plan_id) + assert record.data_type == GRAPH_VESSEL_WRITE_PLAN_DATA_TYPE + assert record.payload["operation_count"] == plan.operation_count + assert result.created_data_ids == [plan.plan_id] + + +def test_vessel_write_plan_blocks_failed_integrity_export_packet() -> None: + trace_store = TraceStore() + data_store = DataStore() + data_store.create_record( + data_id="graph:edge:contains:graph:missing_parent:graph:missing_child", + data_type="graph_memory:edge:CONTAINS", + payload={ + "edge_id": "graph:edge:contains:graph:missing_parent:graph:missing_child", + "edge_kind": "CONTAINS", + "from_node_id": "graph:missing_parent", + "to_node_id": "graph:missing_child", + "source_data_ids": ["graph:missing_parent", "graph:missing_child"], + }, + ) + export_result = record_graph_memory_export_packet( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_159", + batch_id="batch_order_159_failed_integrity", + created_at=EXPORTED_AT, + ) + + result = record_graph_vessel_write_plan( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_159", + export_packet_id=export_result.packet.packet_id, + created_at=PLANNED_AT, + ) + + plan = result.plan + assert plan.plan_status == "blocked_integrity_failed" + assert plan.block_reason == "CODE_STATUS:graph_integrity_failed" + assert plan.operations == [] + assert plan.operation_count == 0 + assert plan.external_write_status == "not_run" + assert plan.graph_integrity_status == "failed" + + +def test_vessel_write_plan_rejects_wrong_target_adapter(tmp_path) -> None: + root = _sample_repo(tmp_path) + trace_store, data_store, export_packet_id = _store_with_export_packet(root) + packet_record = data_store.require_record(export_packet_id) + payload = dict(packet_record.payload) + payload["target_adapter_name"] = "wrong-adapter" + data_store.create_record( + data_id="graph_memory:export_packet:wrong_target", + data_type=GRAPH_MEMORY_EXPORT_PACKET_DATA_TYPE, + payload=payload, + ) + + with pytest.raises(ValueError, match="unexpected target adapter name"): + record_graph_vessel_write_plan( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_159", + export_packet_id="graph_memory:export_packet:wrong_target", + created_at=PLANNED_AT, + ) + + +def test_vessel_write_plan_is_idempotent_for_same_packet_and_timestamp(tmp_path) -> None: + root = _sample_repo(tmp_path) + trace_store, data_store, export_packet_id = _store_with_export_packet(root) + + first = record_graph_vessel_write_plan( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_159", + export_packet_id=export_packet_id, + created_at=PLANNED_AT, + ) + second = record_graph_vessel_write_plan( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_159", + export_packet_id=export_packet_id, + created_at=PLANNED_AT, + ) + + assert first.created_data_ids == [first.plan.plan_id] + assert second.created_data_ids == [] + assert second.existing_data_ids == [first.plan.plan_id] + assert asdict(first.plan) == asdict(second.plan) + + +def test_vessel_write_plan_does_not_create_semantic_fields(tmp_path) -> None: + root = _sample_repo(tmp_path) + trace_store, data_store, export_packet_id = _store_with_export_packet(root) + + result = record_graph_vessel_write_plan( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_159", + export_packet_id=export_packet_id, + created_at=PLANNED_AT, + ) + + payload = data_store.require_record(result.plan.plan_id).payload + for forbidden_field in ( + "summary", + "semantic_topic", + "topic_label", + "importance_score", + "relevance_reason", + "meaning_cluster", + "embedding_id", + ): + assert forbidden_field not in payload + assert payload["generated_by"] == GRAPH_VESSEL_ADAPTER_BOUNDARY_GENERATOR + assert payload["info_class"] == "absolute" + assert payload["semantic_judgement_status"] == "not_run" + + +def _store_with_export_packet(root) -> tuple[TraceStore, DataStore, str]: + trace_store = TraceStore() + data_store = DataStore() + record_graph_memory_for_capsules( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_159_core", + batch_id="batch_order_159_core", + capsules=[_sample_capsule("turn_order_159_previous")], + ) + record_songryeon_core_source_manifest_ingest( + trace_store=trace_store, + data_store=data_store, + root_path=root, + turn_id="turn_order_159_source", + batch_id="batch_order_159_source", + observed_at=OBSERVED_AT, + ingested_at=INGESTED_AT, + ) + export_result = record_graph_memory_export_packet( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_159_export", + batch_id="batch_order_159_export", + created_at=EXPORTED_AT, + ) + return trace_store, data_store, export_result.packet.packet_id + + +def _sample_repo(tmp_path): + root = tmp_path / "repo" + (root / "Administrative_Reform_1" / "04_Orders").mkdir(parents=True) + (root / "Administrative_Reform_1" / "05_Execution_Records").mkdir(parents=True) + (root / "songryeon_core").mkdir() + (root / "tests").mkdir() + + (root / "AGENTS.md").write_text("# agent rules\n", encoding="utf-8") + (root / "README.md").write_text("# readme\n", encoding="utf-8") + (root / "Administrative_Reform_1" / "04_Orders" / "ORDER_001.md").write_text( + "# order\n", + encoding="utf-8", + ) + (root / "Administrative_Reform_1" / "05_Execution_Records" / "note.md").write_text( + "# note\n", + encoding="utf-8", + ) + (root / "main.py").write_text("print('main')\n", encoding="utf-8") + (root / "songryeon_core" / "module.py").write_text( + "def run():\n return 1\n", + encoding="utf-8", + ) + (root / "tests" / "test_module.py").write_text( + "def test_run():\n assert True\n", + encoding="utf-8", + ) + return root + + +def _sample_capsule(turn_id: str) -> TurnStateCapsule: + return TurnStateCapsule( + turn_id=turn_id, + node_movements=[ + NodeMovement( + movement_id=f"move:{turn_id}:001", + turn_id=turn_id, + step_index=1, + node_id="node_0", + mode="pre_route_report", + input_trace_ids=[f"trace:{turn_id}:user"], + output_trace_ids=[f"trace:{turn_id}:node0"], + status="completed", + ) + ], + trace_event_ids=[ + f"trace:{turn_id}:user", + f"trace:{turn_id}:node0", + f"trace:{turn_id}:final", + ], + user_input_trace_id=f"trace:{turn_id}:user", + final_response_trace_id=f"trace:{turn_id}:final", + ) diff --git a/tests/test_order_160_local_vessel_neo4j_writer.py b/tests/test_order_160_local_vessel_neo4j_writer.py new file mode 100644 index 0000000..347cb29 --- /dev/null +++ b/tests/test_order_160_local_vessel_neo4j_writer.py @@ -0,0 +1,421 @@ +from __future__ import annotations + +from dataclasses import asdict + +import pytest + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.graph_memory import record_graph_memory_for_capsules +from songryeon_core.core.graph_memory_export import record_graph_memory_export_packet +from songryeon_core.core.graph_vessel_adapter import record_graph_vessel_write_plan +from songryeon_core.core.graph_vessel_neo4j import ( + GRAPH_VESSEL_NEO4J_WRITE_RESULT_DATA_TYPE, + GRAPH_VESSEL_NEO4J_WRITER_GENERATOR, + GraphVesselNeo4jConfig, + record_graph_vessel_neo4j_write_result, + write_graph_vessel_plan_to_neo4j, +) +from songryeon_core.core.schemas import NodeMovement, TurnStateCapsule +from songryeon_core.core.trace_store import TraceStore + + +EXPORTED_AT = "2026-07-01T20:00:00" +PLANNED_AT = "2026-07-01T20:00:01" +WRITTEN_AT = "2026-07-01T20:00:02" + + +def test_neo4j_writer_writes_ready_plan_with_fake_driver() -> None: + trace_store, data_store, plan_id = _store_with_write_plan() + fake_driver_factory = FakeNeo4jDriverFactory() + + result = record_graph_vessel_neo4j_write_result( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_160", + plan_id=plan_id, + config=_config(), + created_at=WRITTEN_AT, + driver_factory=fake_driver_factory, + ) + + frame = result.result + plan_payload = data_store.require_record(plan_id).payload + assert frame.generated_by == GRAPH_VESSEL_NEO4J_WRITER_GENERATOR + assert frame.info_class == "absolute" + assert frame.semantic_judgement_status == "not_run" + assert frame.write_status == "written" + assert frame.external_write_status == "written" + assert frame.failure_type is None + assert frame.attempted_operation_count == plan_payload["operation_count"] + assert frame.written_operation_count == plan_payload["operation_count"] + assert frame.neo4j_record_node_count is not None + assert frame.neo4j_graph_edge_count is not None + assert fake_driver_factory.last_driver is not None + assert fake_driver_factory.last_driver.closed is True + + record = data_store.require_record(frame.result_id) + assert record.data_type == GRAPH_VESSEL_NEO4J_WRITE_RESULT_DATA_TYPE + assert record.payload["write_status"] == "written" + assert result.created_data_ids == [frame.result_id] + + +def test_neo4j_writer_missing_password_is_adapter_unavailable_without_driver_call() -> None: + _, data_store, plan_id = _store_with_write_plan() + fake_driver_factory = RaisingDriverFactory() + + result = write_graph_vessel_plan_to_neo4j( + data_store=data_store, + plan_id=plan_id, + config=_config(password=None), + created_at=WRITTEN_AT, + driver_factory=fake_driver_factory, + ) + + assert result.write_status == "adapter_unavailable" + assert result.failure_type == "neo4j_config_missing" + assert result.external_write_status == "not_run" + assert result.attempted_operation_count == 0 + assert fake_driver_factory.called is False + + +def test_neo4j_writer_allow_no_auth_passes_auth_none() -> None: + _, data_store, plan_id = _store_with_write_plan() + fake_driver_factory = FakeNeo4jDriverFactory() + + result = write_graph_vessel_plan_to_neo4j( + data_store=data_store, + plan_id=plan_id, + config=_config(password=None, allow_no_auth=True), + created_at=WRITTEN_AT, + driver_factory=fake_driver_factory, + ) + + assert result.write_status == "written" + assert fake_driver_factory.calls[0]["auth"] is None + + +def test_neo4j_writer_blocks_not_ready_plan() -> None: + trace_store = TraceStore() + data_store = DataStore() + data_store.create_record( + data_id="graph:edge:contains:graph:missing_parent:graph:missing_child", + data_type="graph_memory:edge:CONTAINS", + payload={ + "edge_id": "graph:edge:contains:graph:missing_parent:graph:missing_child", + "edge_kind": "CONTAINS", + "from_node_id": "graph:missing_parent", + "to_node_id": "graph:missing_child", + "source_data_ids": ["graph:missing_parent", "graph:missing_child"], + }, + ) + export = record_graph_memory_export_packet( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_160", + batch_id="batch_order_160_failed_integrity", + created_at=EXPORTED_AT, + ) + plan = record_graph_vessel_write_plan( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_160", + export_packet_id=export.packet.packet_id, + created_at=PLANNED_AT, + ) + + result = write_graph_vessel_plan_to_neo4j( + data_store=data_store, + plan_id=plan.plan.plan_id, + config=_config(), + created_at=WRITTEN_AT, + driver_factory=RaisingDriverFactory(), + ) + + assert result.write_status == "blocked_plan_not_ready" + assert result.failure_type == "plan_status_not_ready" + assert result.attempted_operation_count == 0 + assert result.external_write_status == "not_run" + + +def test_neo4j_writer_records_write_failure() -> None: + _, data_store, plan_id = _store_with_write_plan() + + result = write_graph_vessel_plan_to_neo4j( + data_store=data_store, + plan_id=plan_id, + config=_config(), + created_at=WRITTEN_AT, + driver_factory=FailingNeo4jDriverFactory(), + ) + + assert result.write_status == "write_failed" + assert result.failure_type == "neo4j_write_failed" + assert result.external_write_status == "not_run" + assert result.attempted_operation_count > 0 + assert result.written_operation_count == 0 + + +def test_neo4j_writer_does_not_create_semantic_fields() -> None: + trace_store, data_store, plan_id = _store_with_write_plan() + + result = record_graph_vessel_neo4j_write_result( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_160", + plan_id=plan_id, + config=_config(), + created_at=WRITTEN_AT, + driver_factory=FakeNeo4jDriverFactory(), + ) + + payload = data_store.require_record(result.result.result_id).payload + for forbidden_field in ( + "summary", + "semantic_topic", + "topic_label", + "importance_score", + "relevance_reason", + "meaning_cluster", + "embedding_id", + ): + assert forbidden_field not in payload + assert payload["generated_by"] == GRAPH_VESSEL_NEO4J_WRITER_GENERATOR + assert payload["info_class"] == "absolute" + assert payload["semantic_judgement_status"] == "not_run" + + +def test_neo4j_writer_uses_human_readable_display_vocabulary() -> None: + trace_store, data_store, plan_id = _store_with_write_plan() + fake_driver_factory = FakeNeo4jDriverFactory() + + result = record_graph_vessel_neo4j_write_result( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_161", + plan_id=plan_id, + config=_config(), + created_at=WRITTEN_AT, + driver_factory=fake_driver_factory, + ) + + assert result.result.write_status == "written" + assert fake_driver_factory.last_driver is not None + queries = "\n".join(query for query, _kwargs in fake_driver_factory.last_driver.queries) + assert "SET n:VesselRecord" in queries + assert "REMOVE n:SongRyeonRecord" in queries + assert ":CoreEgo" in queries + assert ":TimeAxis" in queries + assert ":TimeBundle" in queries + assert ":RawCapsule" in queries + assert "r:HAS_AXIS" in queries + assert "r:HAS_BUNDLE" in queries + assert "r:CONTAINS_MEMORY" in queries + + property_sets = [ + kwargs["properties"] + for _query, kwargs in fake_driver_factory.last_driver.queries + if isinstance(kwargs.get("properties"), dict) + ] + assert any(item.get("display_name") == "CoreEgo" for item in property_sets) + assert any(item.get("display_label") == "RawCapsule" for item in property_sets) + assert any( + item.get("display_relationship_type") == "HAS_AXIS" + for item in property_sets + ) + + +def test_neo4j_write_result_is_idempotent_for_same_plan_and_timestamp() -> None: + trace_store, data_store, plan_id = _store_with_write_plan() + + first = record_graph_vessel_neo4j_write_result( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_160", + plan_id=plan_id, + config=_config(), + created_at=WRITTEN_AT, + driver_factory=FakeNeo4jDriverFactory(), + ) + second = record_graph_vessel_neo4j_write_result( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_160", + plan_id=plan_id, + config=_config(), + created_at=WRITTEN_AT, + driver_factory=FakeNeo4jDriverFactory(), + ) + + assert first.created_data_ids == [first.result.result_id] + assert second.created_data_ids == [] + assert second.existing_data_ids == [first.result.result_id] + assert asdict(first.result) == asdict(second.result) + + +def _store_with_write_plan() -> tuple[TraceStore, DataStore, str]: + trace_store = TraceStore() + data_store = DataStore() + record_graph_memory_for_capsules( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_160_core", + batch_id="batch_order_160_core", + capsules=[_sample_capsule("turn_order_160_previous")], + ) + export = record_graph_memory_export_packet( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_160_export", + batch_id="batch_order_160_export", + created_at=EXPORTED_AT, + ) + plan = record_graph_vessel_write_plan( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_160_plan", + export_packet_id=export.packet.packet_id, + created_at=PLANNED_AT, + ) + return trace_store, data_store, plan.plan.plan_id + + +def _sample_capsule(turn_id: str) -> TurnStateCapsule: + return TurnStateCapsule( + turn_id=turn_id, + node_movements=[ + NodeMovement( + movement_id=f"move:{turn_id}:001", + turn_id=turn_id, + step_index=1, + node_id="node_0", + mode="pre_route_report", + input_trace_ids=[f"trace:{turn_id}:user"], + output_trace_ids=[f"trace:{turn_id}:node0"], + status="completed", + ) + ], + trace_event_ids=[ + f"trace:{turn_id}:user", + f"trace:{turn_id}:node0", + f"trace:{turn_id}:final", + ], + user_input_trace_id=f"trace:{turn_id}:user", + final_response_trace_id=f"trace:{turn_id}:final", + ) + + +def _config( + *, + password: str | None = "test-password", + allow_no_auth: bool = False, +) -> GraphVesselNeo4jConfig: + return GraphVesselNeo4jConfig( + uri="bolt://localhost:7687", + user="neo4j", + password=password, + database="songryeon_vessel", + allow_no_auth=allow_no_auth, + ) + + +class FakeNeo4jDriverFactory: + def __init__(self) -> None: + self.calls: list[dict[str, object]] = [] + self.last_driver: FakeNeo4jDriver | None = None + + def __call__(self, uri: str, *, auth: object) -> "FakeNeo4jDriver": + self.calls.append({"uri": uri, "auth": auth}) + self.last_driver = FakeNeo4jDriver() + return self.last_driver + + +class RaisingDriverFactory: + def __init__(self) -> None: + self.called = False + + def __call__(self, *_args, **_kwargs) -> object: + self.called = True + raise AssertionError("driver factory should not be called") + + +class FailingNeo4jDriverFactory: + def __call__(self, uri: str, *, auth: object) -> "FailingNeo4jDriver": + return FailingNeo4jDriver() + + +class FakeNeo4jDriver: + def __init__(self) -> None: + self.record_node_ids: set[str] = set() + self.edge_ids: set[str] = set() + self.queries: list[tuple[str, dict[str, object]]] = [] + self.closed = False + + def session(self, *, database: str) -> "FakeNeo4jSession": + return FakeNeo4jSession(self, database=database) + + def close(self) -> None: + self.closed = True + + +class FailingNeo4jDriver: + def session(self, *, database: str) -> "FailingNeo4jSession": + return FailingNeo4jSession() + + def close(self) -> None: + pass + + +class FakeNeo4jSession: + def __init__(self, driver: FakeNeo4jDriver, *, database: str) -> None: + self.driver = driver + self.database = database + + def __enter__(self) -> "FakeNeo4jSession": + return self + + def __exit__(self, _exc_type, _exc, _tb) -> None: + return None + + def execute_write(self, fn, *args): + return fn(FakeNeo4jTransaction(self.driver), *args) + + def execute_read(self, fn, *args): + return fn(FakeNeo4jTransaction(self.driver), *args) + + +class FailingNeo4jSession: + def __enter__(self) -> "FailingNeo4jSession": + return self + + def __exit__(self, _exc_type, _exc, _tb) -> None: + return None + + def execute_write(self, _fn, *_args): + raise RuntimeError("simulated write failure") + + +class FakeNeo4jTransaction: + def __init__(self, driver: FakeNeo4jDriver) -> None: + self.driver = driver + + def run(self, query: str, **kwargs): + self.driver.queries.append((query, kwargs)) + if "RETURN count(n) AS count" in query: + return FakeNeo4jResult(len(self.driver.record_node_ids)) + if "RETURN count(r) AS count" in query: + return FakeNeo4jResult(len(self.driver.edge_ids)) + data_id = kwargs.get("data_id") + if isinstance(data_id, str): + self.driver.record_node_ids.add(data_id) + edge_id = kwargs.get("edge_id") + if isinstance(edge_id, str) and "SONGRYEON_GRAPH_EDGE" in query: + self.driver.edge_ids.add(edge_id) + return FakeNeo4jResult(0) + + +class FakeNeo4jResult: + def __init__(self, count: int) -> None: + self.count = count + + def single(self): + return {"count": self.count} diff --git a/tests/test_order_162_vessel_readback_verification.py b/tests/test_order_162_vessel_readback_verification.py new file mode 100644 index 0000000..a8b6305 --- /dev/null +++ b/tests/test_order_162_vessel_readback_verification.py @@ -0,0 +1,359 @@ +from __future__ import annotations + +from dataclasses import asdict + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.graph_vessel_neo4j import GraphVesselNeo4jConfig +from songryeon_core.core.graph_vessel_readback import ( + GRAPH_VESSEL_NEO4J_READBACK_GENERATOR, + GRAPH_VESSEL_NEO4J_READBACK_RESULT_DATA_TYPE, + readback_graph_vessel_from_neo4j, + record_graph_vessel_neo4j_readback_result, +) +from songryeon_core.core.trace_store import TraceStore + + +READ_AT = "2026-07-01T21:00:00" + + +def test_vessel_readback_passes_when_core_path_exists() -> None: + fake_driver_factory = FakeNeo4jReadbackDriverFactory() + + result = readback_graph_vessel_from_neo4j( + config=_config(), + batch_id="order_162_pass", + created_at=READ_AT, + driver_factory=fake_driver_factory, + ) + + assert result.generated_by == GRAPH_VESSEL_NEO4J_READBACK_GENERATOR + assert result.info_class == "absolute" + assert result.semantic_judgement_status == "not_run" + assert result.readback_status == "passed" + assert result.failure_type is None + assert result.core_path_exists is True + assert result.core_path_count == 1 + assert result.vessel_record_count == 10 + assert result.vessel_relationship_count == 3 + assert result.core_ego_count == 1 + assert result.time_axis_count == 1 + assert result.time_bundle_count == 1 + assert result.raw_capsule_count == 1 + assert result.has_axis_count == 1 + assert result.has_bundle_count == 1 + assert result.contains_memory_count == 1 + assert result.required_property_missing_count == 0 + assert fake_driver_factory.last_driver is not None + assert fake_driver_factory.last_driver.closed is True + + +def test_vessel_readback_records_result_in_datastore() -> None: + trace_store = TraceStore() + data_store = DataStore() + + recorded = record_graph_vessel_neo4j_readback_result( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_162", + batch_id="order_162_record", + config=_config(), + created_at=READ_AT, + driver_factory=FakeNeo4jReadbackDriverFactory(), + ) + + record = data_store.require_record(recorded.result.result_id) + assert record.data_type == GRAPH_VESSEL_NEO4J_READBACK_RESULT_DATA_TYPE + assert record.payload["readback_status"] == "passed" + assert record.payload["core_path_exists"] is True + assert recorded.created_data_ids == [recorded.result.result_id] + assert trace_store.list_events()[0].actor == "graph_vessel_neo4j_readback" + + +def test_vessel_readback_missing_password_does_not_call_driver() -> None: + fake_driver_factory = RaisingDriverFactory() + + result = readback_graph_vessel_from_neo4j( + config=_config(password=None), + batch_id="order_162_missing_password", + created_at=READ_AT, + driver_factory=fake_driver_factory, + ) + + assert result.readback_status == "adapter_unavailable" + assert result.failure_type == "neo4j_config_missing" + assert result.core_path_count is None + assert fake_driver_factory.called is False + + +def test_vessel_readback_fails_when_core_path_is_missing() -> None: + fake_driver_factory = FakeNeo4jReadbackDriverFactory( + counts={"core_path_count": 0} + ) + + result = readback_graph_vessel_from_neo4j( + config=_config(), + batch_id="order_162_missing_path", + created_at=READ_AT, + driver_factory=fake_driver_factory, + ) + + assert result.readback_status == "failed" + assert result.failure_type == "readback_check_failed" + assert result.core_path_exists is False + assert "path was not found" in (result.failure_reason or "") + + +def test_vessel_readback_fails_when_required_properties_are_missing() -> None: + fake_driver_factory = FakeNeo4jReadbackDriverFactory( + counts={"required_property_missing_count": 2}, + missing_samples=["graph:bad:001", ""], + ) + + result = readback_graph_vessel_from_neo4j( + config=_config(), + batch_id="order_162_missing_properties", + created_at=READ_AT, + driver_factory=fake_driver_factory, + ) + + assert result.readback_status == "failed" + assert result.failure_type == "readback_check_failed" + assert result.required_property_missing_count == 2 + assert result.required_property_missing_samples == ["graph:bad:001", ""] + + +def test_vessel_readback_records_driver_exception() -> None: + result = readback_graph_vessel_from_neo4j( + config=_config(), + batch_id="order_162_read_failed", + created_at=READ_AT, + driver_factory=FailingReadbackDriverFactory(), + ) + + assert result.readback_status == "read_failed" + assert result.failure_type == "neo4j_read_failed" + assert "simulated read failure" in (result.failure_reason or "") + + +def test_vessel_readback_does_not_create_semantic_fields() -> None: + trace_store = TraceStore() + data_store = DataStore() + + recorded = record_graph_vessel_neo4j_readback_result( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_162", + batch_id="order_162_no_semantic", + config=_config(), + created_at=READ_AT, + driver_factory=FakeNeo4jReadbackDriverFactory(), + ) + + payload = data_store.require_record(recorded.result.result_id).payload + for forbidden_field in ( + "summary", + "semantic_topic", + "topic_label", + "importance_score", + "relevance_reason", + "meaning_cluster", + "embedding_id", + ): + assert forbidden_field not in payload + assert payload["generated_by"] == GRAPH_VESSEL_NEO4J_READBACK_GENERATOR + assert payload["info_class"] == "absolute" + assert payload["semantic_judgement_status"] == "not_run" + + +def test_vessel_readback_result_is_idempotent_for_same_batch_and_timestamp() -> None: + trace_store = TraceStore() + data_store = DataStore() + + first = record_graph_vessel_neo4j_readback_result( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_162", + batch_id="order_162_idempotent", + config=_config(), + created_at=READ_AT, + driver_factory=FakeNeo4jReadbackDriverFactory(), + ) + second = record_graph_vessel_neo4j_readback_result( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_162", + batch_id="order_162_idempotent", + config=_config(), + created_at=READ_AT, + driver_factory=FakeNeo4jReadbackDriverFactory(), + ) + + assert first.created_data_ids == [first.result.result_id] + assert second.created_data_ids == [] + assert second.existing_data_ids == [first.result.result_id] + assert asdict(first.result) == asdict(second.result) + + +def _config( + *, + password: str | None = "test-password", + allow_no_auth: bool = False, +) -> GraphVesselNeo4jConfig: + return GraphVesselNeo4jConfig( + uri="bolt://localhost:7687", + user="neo4j", + password=password, + database="neo4j", + allow_no_auth=allow_no_auth, + ) + + +class FakeNeo4jReadbackDriverFactory: + def __init__( + self, + *, + counts: dict[str, int] | None = None, + missing_samples: list[str] | None = None, + ) -> None: + self.counts = { + "core_path_count": 1, + "vessel_record_count": 10, + "vessel_relationship_count": 3, + "core_ego_count": 1, + "time_axis_count": 1, + "time_bundle_count": 1, + "raw_capsule_count": 1, + "has_axis_count": 1, + "has_bundle_count": 1, + "contains_memory_count": 1, + "required_property_missing_count": 0, + } + self.counts.update(counts or {}) + self.missing_samples = list(missing_samples or []) + self.calls: list[dict[str, object]] = [] + self.last_driver: FakeNeo4jReadbackDriver | None = None + + def __call__(self, uri: str, *, auth: object) -> "FakeNeo4jReadbackDriver": + self.calls.append({"uri": uri, "auth": auth}) + self.last_driver = FakeNeo4jReadbackDriver( + counts=self.counts, + missing_samples=self.missing_samples, + ) + return self.last_driver + + +class RaisingDriverFactory: + def __init__(self) -> None: + self.called = False + + def __call__(self, *_args, **_kwargs) -> object: + self.called = True + raise AssertionError("driver factory should not be called") + + +class FailingReadbackDriverFactory: + def __call__(self, uri: str, *, auth: object) -> "FailingReadbackDriver": + return FailingReadbackDriver() + + +class FakeNeo4jReadbackDriver: + def __init__( + self, + *, + counts: dict[str, int], + missing_samples: list[str], + ) -> None: + self.counts = dict(counts) + self.missing_samples = list(missing_samples) + self.queries: list[tuple[str, dict[str, object]]] = [] + self.closed = False + + def session(self, *, database: str) -> "FakeNeo4jReadbackSession": + return FakeNeo4jReadbackSession(self, database=database) + + def close(self) -> None: + self.closed = True + + +class FailingReadbackDriver: + def session(self, *, database: str) -> "FailingReadbackSession": + return FailingReadbackSession() + + def close(self) -> None: + pass + + +class FakeNeo4jReadbackSession: + def __init__(self, driver: FakeNeo4jReadbackDriver, *, database: str) -> None: + self.driver = driver + self.database = database + + def __enter__(self) -> "FakeNeo4jReadbackSession": + return self + + def __exit__(self, _exc_type, _exc, _tb) -> None: + return None + + def execute_read(self, fn, *args): + return fn(FakeNeo4jReadbackTransaction(self.driver), *args) + + +class FailingReadbackSession: + def __enter__(self) -> "FailingReadbackSession": + return self + + def __exit__(self, _exc_type, _exc, _tb) -> None: + return None + + def execute_read(self, _fn, *_args): + raise RuntimeError("simulated read failure") + + +class FakeNeo4jReadbackTransaction: + def __init__(self, driver: FakeNeo4jReadbackDriver) -> None: + self.driver = driver + + def run(self, query: str, **kwargs): + self.driver.queries.append((query, kwargs)) + compact = " ".join(query.split()) + if "collect(coalesce(n.data_id" in compact: + return FakeNeo4jSamplesResult(self.driver.missing_samples) + if "CoreEgo" in compact and "HAS_AXIS" in compact and "CONTAINS_MEMORY" in compact: + return FakeNeo4jCountResult(self.driver.counts["core_path_count"]) + if "WHERE n.data_id IS NULL" in compact: + return FakeNeo4jCountResult(self.driver.counts["required_property_missing_count"]) + if "MATCH (n:VesselRecord" in compact and "RETURN count(n) AS count" in compact: + return FakeNeo4jCountResult(self.driver.counts["vessel_record_count"]) + if "MATCH (n:CoreEgo" in compact: + return FakeNeo4jCountResult(self.driver.counts["core_ego_count"]) + if "MATCH (n:TimeAxis" in compact: + return FakeNeo4jCountResult(self.driver.counts["time_axis_count"]) + if "MATCH (n:TimeBundle" in compact: + return FakeNeo4jCountResult(self.driver.counts["time_bundle_count"]) + if "MATCH (n:RawCapsule" in compact: + return FakeNeo4jCountResult(self.driver.counts["raw_capsule_count"]) + if "MATCH ()-[r:HAS_AXIS" in compact: + return FakeNeo4jCountResult(self.driver.counts["has_axis_count"]) + if "MATCH ()-[r:HAS_BUNDLE" in compact: + return FakeNeo4jCountResult(self.driver.counts["has_bundle_count"]) + if "MATCH ()-[r:CONTAINS_MEMORY" in compact: + return FakeNeo4jCountResult(self.driver.counts["contains_memory_count"]) + if "WHERE r.display_relationship_type IS NOT NULL" in compact: + return FakeNeo4jCountResult(self.driver.counts["vessel_relationship_count"]) + return FakeNeo4jCountResult(0) + + +class FakeNeo4jCountResult: + def __init__(self, count: int) -> None: + self.count = count + + def single(self): + return {"count": self.count} + + +class FakeNeo4jSamplesResult: + def __init__(self, samples: list[str]) -> None: + self.samples = samples + + def single(self): + return {"samples": list(self.samples)} diff --git a/tests/test_order_163_vessel_inspect_manual_walk.py b/tests/test_order_163_vessel_inspect_manual_walk.py new file mode 100644 index 0000000..291f56b --- /dev/null +++ b/tests/test_order_163_vessel_inspect_manual_walk.py @@ -0,0 +1,369 @@ +from __future__ import annotations + +from dataclasses import asdict + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.graph_vessel_inspect import ( + GRAPH_VESSEL_NEO4J_INSPECT_GENERATOR, + GRAPH_VESSEL_NEO4J_INSPECT_RESULT_DATA_TYPE, + inspect_graph_vessel_from_neo4j, + record_graph_vessel_neo4j_inspect_result, +) +from songryeon_core.core.graph_vessel_neo4j import GraphVesselNeo4jConfig +from songryeon_core.core.trace_store import TraceStore +from songryeon_core.runtime.graph_vessel_inspect import render_vessel_inspect_text + + +INSPECTED_AT = "2026-07-01T22:00:00" + + +def test_vessel_inspect_returns_manual_walk_tree() -> None: + fake_driver_factory = FakeInspectDriverFactory() + + result = inspect_graph_vessel_from_neo4j( + config=_config(), + batch_id="order_163_tree", + created_at=INSPECTED_AT, + driver_factory=fake_driver_factory, + ) + + assert result.generated_by == GRAPH_VESSEL_NEO4J_INSPECT_GENERATOR + assert result.info_class == "absolute" + assert result.semantic_judgement_status == "not_run" + assert result.inspect_status == "passed" + assert result.inspected_path_count == 1 + assert result.core_count == 1 + assert result.time_axis_count == 1 + assert result.time_bundle_count == 1 + assert result.raw_capsule_count == 1 + assert result.tree_lines == [ + "CoreEgo [graph:core_ego:root]", + " HAS_AXIS -> Time Axis [graph:axis:time]", + " HAS_BUNDLE -> Time Bundle [graph:time_bundle:manual_vessel_first_write:core]", + " CONTAINS_MEMORY -> Raw Capsule: turn_vessel_first_write_0001:previous [graph:raw_capsule:turn_vessel_first_write_0001:previous]", + ] + assert result.source_data_ids == [ + "graph:core_ego:root", + "graph:axis:time", + "graph:time_bundle:manual_vessel_first_write:core", + "graph:raw_capsule:turn_vessel_first_write_0001:previous", + ] + assert result.source_trace_ids == ["trace:turn_vessel_first_write_0001:previous:final"] + assert fake_driver_factory.last_driver is not None + assert fake_driver_factory.last_driver.closed is True + + +def test_vessel_inspect_records_result_in_datastore() -> None: + trace_store = TraceStore() + data_store = DataStore() + + recorded = record_graph_vessel_neo4j_inspect_result( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_163", + batch_id="order_163_record", + config=_config(), + created_at=INSPECTED_AT, + driver_factory=FakeInspectDriverFactory(), + ) + + record = data_store.require_record(recorded.result.result_id) + assert record.data_type == GRAPH_VESSEL_NEO4J_INSPECT_RESULT_DATA_TYPE + assert record.payload["inspect_status"] == "passed" + assert record.payload["tree_lines"][0] == "CoreEgo [graph:core_ego:root]" + assert recorded.created_data_ids == [recorded.result.result_id] + assert trace_store.list_events()[0].actor == "graph_vessel_neo4j_inspector" + + +def test_vessel_inspect_empty_when_core_axis_path_missing() -> None: + result = inspect_graph_vessel_from_neo4j( + config=_config(), + batch_id="order_163_empty", + created_at=INSPECTED_AT, + driver_factory=FakeInspectDriverFactory(path_rows=[]), + ) + + assert result.inspect_status == "empty" + assert result.failure_type == "no_inspectable_path" + assert result.inspected_path_count == 0 + assert result.tree_lines == [] + + +def test_vessel_inspect_missing_password_does_not_call_driver() -> None: + fake_driver_factory = RaisingDriverFactory() + + result = inspect_graph_vessel_from_neo4j( + config=_config(password=None), + batch_id="order_163_missing_password", + created_at=INSPECTED_AT, + driver_factory=fake_driver_factory, + ) + + assert result.inspect_status == "adapter_unavailable" + assert result.failure_type == "neo4j_config_missing" + assert result.path_items == [] + assert fake_driver_factory.called is False + + +def test_vessel_inspect_records_driver_exception() -> None: + result = inspect_graph_vessel_from_neo4j( + config=_config(), + batch_id="order_163_read_failed", + created_at=INSPECTED_AT, + driver_factory=FailingInspectDriverFactory(), + ) + + assert result.inspect_status == "read_failed" + assert result.failure_type == "neo4j_read_failed" + assert "simulated inspect failure" in (result.failure_reason or "") + + +def test_vessel_inspect_text_renderer_uses_tree_lines() -> None: + result = { + "status": "VESSEL_INSPECT_OK", + "inspect_status": "passed", + "failure_type": None, + "failure_reason": None, + "graph_namespace": "songryeon_core_graph_v0", + "inspected_path_count": 1, + "tree_text": "CoreEgo [graph:core_ego:root]", + } + + rendered = render_vessel_inspect_text(result) + + assert "status: VESSEL_INSPECT_OK" in rendered + assert "inspect_status: passed" in rendered + assert "CoreEgo [graph:core_ego:root]" in rendered + + +def test_vessel_inspect_does_not_create_semantic_fields() -> None: + trace_store = TraceStore() + data_store = DataStore() + + recorded = record_graph_vessel_neo4j_inspect_result( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_163", + batch_id="order_163_no_semantic", + config=_config(), + created_at=INSPECTED_AT, + driver_factory=FakeInspectDriverFactory(), + ) + + payload = data_store.require_record(recorded.result.result_id).payload + for forbidden_field in ( + "summary", + "semantic_topic", + "topic_label", + "importance_score", + "relevance_reason", + "meaning_cluster", + "embedding_id", + ): + assert forbidden_field not in payload + assert payload["generated_by"] == GRAPH_VESSEL_NEO4J_INSPECT_GENERATOR + assert payload["info_class"] == "absolute" + assert payload["semantic_judgement_status"] == "not_run" + + +def test_vessel_inspect_result_is_idempotent_for_same_batch_and_timestamp() -> None: + trace_store = TraceStore() + data_store = DataStore() + + first = record_graph_vessel_neo4j_inspect_result( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_163", + batch_id="order_163_idempotent", + config=_config(), + created_at=INSPECTED_AT, + driver_factory=FakeInspectDriverFactory(), + ) + second = record_graph_vessel_neo4j_inspect_result( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_163", + batch_id="order_163_idempotent", + config=_config(), + created_at=INSPECTED_AT, + driver_factory=FakeInspectDriverFactory(), + ) + + assert first.created_data_ids == [first.result.result_id] + assert second.created_data_ids == [] + assert second.existing_data_ids == [first.result.result_id] + assert asdict(first.result) == asdict(second.result) + + +def _config( + *, + password: str | None = "test-password", + allow_no_auth: bool = False, +) -> GraphVesselNeo4jConfig: + return GraphVesselNeo4jConfig( + uri="bolt://localhost:7687", + user="neo4j", + password=password, + database="neo4j", + allow_no_auth=allow_no_auth, + ) + + +def _default_path_rows() -> list[dict[str, object]]: + return [ + { + "core_data_id": "graph:core_ego:root", + "core_display_name": "CoreEgo", + "axis_data_id": "graph:axis:time", + "axis_display_name": "Time Axis", + "bundle_data_id": "graph:time_bundle:manual_vessel_first_write:core", + "bundle_display_name": "Time Bundle", + "bundle_created_at": "2026-07-01T18:39:54", + "bundle_written_at": "2026-07-01T18:39:54", + "raw_capsule_data_id": "graph:raw_capsule:turn_vessel_first_write_0001:previous", + "raw_capsule_display_name": "Raw Capsule: turn_vessel_first_write_0001:previous", + "raw_capsule_created_at": "2026-07-01T18:39:54", + "raw_capsule_written_at": "2026-07-01T18:39:54", + "raw_capsule_source_trace_id": "trace:turn_vessel_first_write_0001:previous:final", + } + ] + + +class FakeInspectDriverFactory: + def __init__( + self, + *, + counts: dict[str, int] | None = None, + path_rows: list[dict[str, object]] | None = None, + summary_rows: list[dict[str, object]] | None = None, + ) -> None: + self.counts = { + "core_count": 1, + "time_axis_count": 1, + "time_bundle_count": 1, + "raw_capsule_count": 1, + "summary_count": 0, + } + self.counts.update(counts or {}) + self.path_rows = _default_path_rows() if path_rows is None else list(path_rows) + self.summary_rows = [] if summary_rows is None else list(summary_rows) + self.calls: list[dict[str, object]] = [] + self.last_driver: FakeInspectDriver | None = None + + def __call__(self, uri: str, *, auth: object) -> "FakeInspectDriver": + self.calls.append({"uri": uri, "auth": auth}) + self.last_driver = FakeInspectDriver( + counts=self.counts, + path_rows=self.path_rows, + summary_rows=self.summary_rows, + ) + return self.last_driver + + +class RaisingDriverFactory: + def __init__(self) -> None: + self.called = False + + def __call__(self, *_args, **_kwargs) -> object: + self.called = True + raise AssertionError("driver factory should not be called") + + +class FailingInspectDriverFactory: + def __call__(self, uri: str, *, auth: object) -> "FailingInspectDriver": + return FailingInspectDriver() + + +class FakeInspectDriver: + def __init__( + self, + *, + counts: dict[str, int], + path_rows: list[dict[str, object]], + summary_rows: list[dict[str, object]], + ) -> None: + self.counts = dict(counts) + self.path_rows = list(path_rows) + self.summary_rows = list(summary_rows) + self.queries: list[tuple[str, dict[str, object]]] = [] + self.closed = False + + def session(self, *, database: str) -> "FakeInspectSession": + return FakeInspectSession(self, database=database) + + def close(self) -> None: + self.closed = True + + +class FailingInspectDriver: + def session(self, *, database: str) -> "FailingInspectSession": + return FailingInspectSession() + + def close(self) -> None: + pass + + +class FakeInspectSession: + def __init__(self, driver: FakeInspectDriver, *, database: str) -> None: + self.driver = driver + self.database = database + + def __enter__(self) -> "FakeInspectSession": + return self + + def __exit__(self, _exc_type, _exc, _tb) -> None: + return None + + def execute_read(self, fn, *args): + return fn(FakeInspectTransaction(self.driver), *args) + + +class FailingInspectSession: + def __enter__(self) -> "FailingInspectSession": + return self + + def __exit__(self, _exc_type, _exc, _tb) -> None: + return None + + def execute_read(self, _fn, *_args): + raise RuntimeError("simulated inspect failure") + + +class FakeInspectTransaction: + def __init__(self, driver: FakeInspectDriver) -> None: + self.driver = driver + + def run(self, query: str, **kwargs): + self.driver.queries.append((query, kwargs)) + compact = " ".join(query.split()) + if "RETURN core.data_id AS core_data_id" in compact: + return FakePathResult(self.driver.path_rows) + if "MATCH (summary:SummaryGraphNode" in compact: + return FakePathResult(self.driver.summary_rows) + if "MATCH (n:CoreEgo" in compact: + return FakeCountResult(self.driver.counts["core_count"]) + if "MATCH (n:TimeAxis" in compact: + return FakeCountResult(self.driver.counts["time_axis_count"]) + if "MATCH (n:TimeBundle" in compact: + return FakeCountResult(self.driver.counts["time_bundle_count"]) + if "MATCH (n:RawCapsule" in compact: + return FakeCountResult(self.driver.counts["raw_capsule_count"]) + if "MATCH (n:SummaryGraphNode" in compact: + return FakeCountResult(self.driver.counts["summary_count"]) + return FakeCountResult(0) + + +class FakeCountResult: + def __init__(self, count: int) -> None: + self.count = count + + def single(self): + return {"count": self.count} + + +class FakePathResult: + def __init__(self, rows: list[dict[str, object]]) -> None: + self.rows = list(rows) + + def __iter__(self): + return iter(self.rows) diff --git a/tests/test_order_164_source_version_lineage_and_invalidation.py b/tests/test_order_164_source_version_lineage_and_invalidation.py new file mode 100644 index 0000000..ca17ec3 --- /dev/null +++ b/tests/test_order_164_source_version_lineage_and_invalidation.py @@ -0,0 +1,255 @@ +from __future__ import annotations + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.graph_memory import record_graph_memory_for_capsules +from songryeon_core.core.graph_memory_export import record_graph_memory_export_packet +from songryeon_core.core.graph_source_ingest import record_graph_source_kind_ingest +from songryeon_core.core.graph_vessel_adapter import record_graph_vessel_write_plan +from songryeon_core.core.schemas import ( + NodeMovement, + SourceVersionLineageFrame, + SourceObservationLedgerFrame, + SummaryInvalidationLedgerFrame, + TurnStateCapsule, + validate_source_observation_ledger_frame, + validate_source_version_lineage_frame, + validate_summary_invalidation_ledger_frame, +) +from songryeon_core.core.source_version_lineage import ( + SOURCE_OBSERVATION_LEDGER_FRAME_DATA_TYPE, + SOURCE_VERSION_LINEAGE_FRAME_DATA_TYPE, + SUMMARY_INVALIDATION_LEDGER_FRAME_DATA_TYPE, +) +from songryeon_core.core.trace_store import TraceStore + + +def test_changed_source_builds_lineage_and_invalidates_old_summary(tmp_path) -> None: + source = tmp_path / "dynamic_policy.md" + source.write_text("version one\n", encoding="utf-8") + trace_store, data_store = _core_time_axis_store() + + first = record_graph_source_kind_ingest( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_164_source", + batch_id="batch_order_164_source_001", + source_paths_by_kind={"internal_document": [source]}, + observed_at="2026-07-02T10:00:00", + ingested_at="2026-07-02T10:00:01", + ) + _record_fake_summary_for_source( + data_store=data_store, + summary_node_id="graph:summary:order_164:old", + source_graph_node_id=first.raw_source_node_ids[0], + ) + + source.write_text("version two\n", encoding="utf-8") + second = record_graph_source_kind_ingest( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_164_source", + batch_id="batch_order_164_source_002", + source_paths_by_kind={"internal_document": [source]}, + observed_at="2026-07-02T11:00:00", + ingested_at="2026-07-02T11:00:01", + ) + + lineage_record = data_store.require_record(second.source_version_lineage_frame_ids[0]) + assert lineage_record.data_type == SOURCE_VERSION_LINEAGE_FRAME_DATA_TYPE + lineage = SourceVersionLineageFrame(**lineage_record.payload) + validate_source_version_lineage_frame(lineage) + assert lineage.source_identity_key + assert lineage.lineage_status == "content_changed" + assert lineage.version_source_graph_node_ids == [ + first.raw_source_node_ids[0], + second.raw_source_node_ids[0], + ] + assert lineage.active_source_graph_node_id == second.raw_source_node_ids[0] + assert lineage.superseded_source_graph_node_ids == [first.raw_source_node_ids[0]] + assert len(set(lineage.content_sha1_by_version.values())) == 2 + + ledger_record = data_store.require_record(second.summary_invalidation_ledger_frame_id) + assert ledger_record.data_type == SUMMARY_INVALIDATION_LEDGER_FRAME_DATA_TYPE + ledger = SummaryInvalidationLedgerFrame(**ledger_record.payload) + validate_summary_invalidation_ledger_frame(ledger) + assert ledger.ledger_status == "invalidations_recorded" + assert ledger.invalidated_summary_node_ids == ["graph:summary:order_164:old"] + assert ledger.invalidation_records[0]["invalidated_reason_code"] == ( + "source_content_changed" + ) + assert ledger.invalidation_records[0]["validity_status"] == ( + "invalidated_by_source_change" + ) + assert ledger.invalidation_records[0]["superseded_source_graph_node_id"] == ( + first.raw_source_node_ids[0] + ) + assert ledger.invalidation_records[0]["superseding_source_graph_node_id"] == ( + second.raw_source_node_ids[0] + ) + + +def test_same_content_reobserve_reuses_raw_source_and_keeps_summary_ledger_empty(tmp_path) -> None: + source = tmp_path / "stable_policy.md" + source.write_text("same content\n", encoding="utf-8") + trace_store, data_store = _core_time_axis_store() + + first = record_graph_source_kind_ingest( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_164_source", + batch_id="batch_order_164_same_001", + source_paths_by_kind={"internal_document": [source]}, + observed_at="2026-07-02T12:00:00", + ingested_at="2026-07-02T12:00:01", + ) + _record_fake_summary_for_source( + data_store=data_store, + summary_node_id="graph:summary:order_164:same", + source_graph_node_id=first.raw_source_node_ids[0], + ) + second = record_graph_source_kind_ingest( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_164_source", + batch_id="batch_order_164_same_002", + source_paths_by_kind={"internal_document": [source]}, + observed_at="2026-07-02T13:00:00", + ingested_at="2026-07-02T13:00:01", + ) + + lineage = SourceVersionLineageFrame( + **data_store.require_record(second.source_version_lineage_frame_ids[0]).payload + ) + assert first.raw_source_node_ids == second.raw_source_node_ids + assert lineage.lineage_status == "single_version" + assert lineage.version_source_graph_node_ids == [first.raw_source_node_ids[0]] + + observation_record = data_store.require_record( + second.source_observation_ledger_frame_id + ) + assert observation_record.data_type == SOURCE_OBSERVATION_LEDGER_FRAME_DATA_TYPE + observation = SourceObservationLedgerFrame(**observation_record.payload) + validate_source_observation_ledger_frame(observation) + assert observation.observation_status_counts == {"unchanged": 1} + assert observation.observation_records[0]["active_source_graph_node_id"] == ( + first.raw_source_node_ids[0] + ) + + ledger = SummaryInvalidationLedgerFrame( + **data_store.require_record(second.summary_invalidation_ledger_frame_id).payload + ) + validate_summary_invalidation_ledger_frame(ledger) + assert ledger.ledger_status == "no_invalidations" + assert ledger.invalidated_summary_node_ids == [] + assert ledger.invalidation_records == [] + + +def test_lineage_and_invalidation_ledgers_are_exported_as_support_records(tmp_path) -> None: + source = tmp_path / "exported_policy.md" + source.write_text("export me\n", encoding="utf-8") + trace_store, data_store = _core_time_axis_store() + + ingest = record_graph_source_kind_ingest( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_164_source", + batch_id="batch_order_164_export_source", + source_paths_by_kind={"internal_document": [source]}, + observed_at="2026-07-02T14:00:00", + ingested_at="2026-07-02T14:00:01", + ) + export = record_graph_memory_export_packet( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_164_export", + batch_id="batch_order_164_export", + created_at="2026-07-02T14:00:02", + ) + + assert ingest.source_version_lineage_frame_ids[0] in ( + export.packet.source_version_lineage_frame_data_ids + ) + assert ingest.source_observation_ledger_frame_id in ( + export.packet.source_observation_ledger_frame_data_ids + ) + assert ingest.summary_invalidation_ledger_frame_id in ( + export.packet.summary_invalidation_ledger_frame_data_ids + ) + + plan = record_graph_vessel_write_plan( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_164_plan", + export_packet_id=export.packet.packet_id, + created_at="2026-07-02T14:00:03", + ) + support_ids = { + operation.source_data_id + for operation in plan.plan.operations + if operation.operation_kind == "upsert_support_record" + } + assert ingest.source_version_lineage_frame_ids[0] in support_ids + assert ingest.source_observation_ledger_frame_id in support_ids + assert ingest.summary_invalidation_ledger_frame_id in support_ids + + +def _record_fake_summary_for_source( + *, + data_store: DataStore, + summary_node_id: str, + source_graph_node_id: str, +) -> None: + data_store.create_record( + data_id=summary_node_id, + data_type="graph_memory:node:summary", + exists=True, + created_at="2026-07-02T10:30:00", + source_trace_id="trace:order_164:summary", + payload={ + "node_id": summary_node_id, + "node_kind": "summary", + "source_graph_node_ids": [source_graph_node_id], + "source_data_ids": [source_graph_node_id], + "generated_by": "LLM:test:summary", + "info_class": "mixed", + "semantic_judgement_status": "ran", + }, + ) + + +def _core_time_axis_store() -> tuple[TraceStore, DataStore]: + trace_store = TraceStore() + data_store = DataStore() + record_graph_memory_for_capsules( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_164_core", + batch_id="batch_order_164_core", + capsules=[_sample_capsule()], + ) + return trace_store, data_store + + +def _sample_capsule(turn_id: str = "turn_order_164_previous") -> TurnStateCapsule: + return TurnStateCapsule( + turn_id=turn_id, + node_movements=[ + NodeMovement( + movement_id=f"move:{turn_id}:001", + turn_id=turn_id, + step_index=1, + node_id="node_0", + mode="pre_route_report", + input_trace_ids=[f"trace:{turn_id}:user"], + output_trace_ids=[f"trace:{turn_id}:node0"], + status="completed", + ) + ], + trace_event_ids=[ + f"trace:{turn_id}:user", + f"trace:{turn_id}:node0", + f"trace:{turn_id}:final", + ], + user_input_trace_id=f"trace:{turn_id}:user", + final_response_trace_id=f"trace:{turn_id}:final", + ) diff --git a/tests/test_order_165_same_content_observation_ledger.py b/tests/test_order_165_same_content_observation_ledger.py new file mode 100644 index 0000000..5d9f03f --- /dev/null +++ b/tests/test_order_165_same_content_observation_ledger.py @@ -0,0 +1,148 @@ +from __future__ import annotations + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.graph_memory import record_graph_memory_for_capsules +from songryeon_core.core.graph_source_ingest import record_graph_source_kind_ingest +from songryeon_core.core.schemas import ( + NodeMovement, + SourceObservationLedgerFrame, + SourceVersionLineageFrame, + TurnStateCapsule, + validate_source_observation_ledger_frame, + validate_source_version_lineage_frame, +) +from songryeon_core.core.source_version_lineage import ( + SOURCE_OBSERVATION_LEDGER_FRAME_DATA_TYPE, +) +from songryeon_core.core.trace_store import TraceStore + + +def test_same_content_reobserve_does_not_create_new_raw_source_version(tmp_path) -> None: + source = tmp_path / "unchanged_code.py" + source.write_text("VALUE = 1\n", encoding="utf-8") + trace_store, data_store = _core_time_axis_store() + + first = record_graph_source_kind_ingest( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_165", + batch_id="batch_order_165_001", + source_paths_by_kind={"source_code_file": [source]}, + observed_at="2026-07-02T15:00:00", + ingested_at="2026-07-02T15:00:01", + ) + second = record_graph_source_kind_ingest( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_165", + batch_id="batch_order_165_002", + source_paths_by_kind={"source_code_file": [source]}, + observed_at="2026-07-02T16:00:00", + ingested_at="2026-07-02T16:00:01", + ) + + assert first.raw_source_node_ids == second.raw_source_node_ids + assert first.source_file_data_ids != second.source_file_data_ids + assert second.source_observation_status_counts == {"unchanged": 1} + + raw_source_records = [ + record + for record in data_store.list_records() + if record.data_type == "graph_memory:node:raw_source" + ] + assert len(raw_source_records) == 1 + + lineage = SourceVersionLineageFrame( + **data_store.require_record(second.source_version_lineage_frame_ids[0]).payload + ) + validate_source_version_lineage_frame(lineage) + assert lineage.lineage_status == "single_version" + assert lineage.version_source_graph_node_ids == first.raw_source_node_ids + + observation_record = data_store.require_record(second.source_observation_ledger_frame_id) + assert observation_record.data_type == SOURCE_OBSERVATION_LEDGER_FRAME_DATA_TYPE + observation = SourceObservationLedgerFrame(**observation_record.payload) + validate_source_observation_ledger_frame(observation) + assert observation.ledger_status == "recorded" + assert observation.observation_records[0]["observation_status"] == "unchanged" + assert observation.observation_records[0]["previous_active_source_graph_node_id"] == ( + first.raw_source_node_ids[0] + ) + + +def test_content_change_creates_new_raw_source_version_and_observation_record(tmp_path) -> None: + source = tmp_path / "changing_code.py" + source.write_text("VALUE = 1\n", encoding="utf-8") + trace_store, data_store = _core_time_axis_store() + + first = record_graph_source_kind_ingest( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_165", + batch_id="batch_order_165_change_001", + source_paths_by_kind={"source_code_file": [source]}, + observed_at="2026-07-02T17:00:00", + ingested_at="2026-07-02T17:00:01", + ) + source.write_text("VALUE = 2\n", encoding="utf-8") + second = record_graph_source_kind_ingest( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_165", + batch_id="batch_order_165_change_002", + source_paths_by_kind={"source_code_file": [source]}, + observed_at="2026-07-02T18:00:00", + ingested_at="2026-07-02T18:00:01", + ) + + assert first.raw_source_node_ids != second.raw_source_node_ids + assert second.source_observation_status_counts == {"content_changed": 1} + + lineage = SourceVersionLineageFrame( + **data_store.require_record(second.source_version_lineage_frame_ids[0]).payload + ) + validate_source_version_lineage_frame(lineage) + assert lineage.lineage_status == "content_changed" + assert lineage.version_source_graph_node_ids == [ + first.raw_source_node_ids[0], + second.raw_source_node_ids[0], + ] + assert lineage.superseded_source_graph_node_ids == [first.raw_source_node_ids[0]] + + +def _core_time_axis_store() -> tuple[TraceStore, DataStore]: + trace_store = TraceStore() + data_store = DataStore() + record_graph_memory_for_capsules( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_165_core", + batch_id="batch_order_165_core", + capsules=[_sample_capsule()], + ) + return trace_store, data_store + + +def _sample_capsule(turn_id: str = "turn_order_165_previous") -> TurnStateCapsule: + return TurnStateCapsule( + turn_id=turn_id, + node_movements=[ + NodeMovement( + movement_id=f"move:{turn_id}:001", + turn_id=turn_id, + step_index=1, + node_id="node_0", + mode="pre_route_report", + input_trace_ids=[f"trace:{turn_id}:user"], + output_trace_ids=[f"trace:{turn_id}:node0"], + status="completed", + ) + ], + trace_event_ids=[ + f"trace:{turn_id}:user", + f"trace:{turn_id}:node0", + f"trace:{turn_id}:final", + ], + user_input_trace_id=f"trace:{turn_id}:user", + final_response_trace_id=f"trace:{turn_id}:final", + ) diff --git a/tests/test_order_166_night_time_bundle_summary_node.py b/tests/test_order_166_night_time_bundle_summary_node.py new file mode 100644 index 0000000..b0d7f80 --- /dev/null +++ b/tests/test_order_166_night_time_bundle_summary_node.py @@ -0,0 +1,245 @@ +from __future__ import annotations + +import json + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.graph_memory import record_graph_memory_for_capsules +from songryeon_core.core.graph_memory_export import record_graph_memory_export_packet +from songryeon_core.core.graph_vessel_adapter import record_graph_vessel_write_plan +from songryeon_core.core.schemas import ( + NightTimeBundleSummaryFrame, + NodeMovement, + TurnStateCapsule, + validate_night_time_bundle_summary_frame, +) +from songryeon_core.core.trace_store import TraceStore +from songryeon_core.llm.base import LLMRequest, LLMResponse +from songryeon_core.nodes.night_summarize_time_bundle import ( + NIGHT_SUMMARIZE_TIME_BUNDLE_FRAME_DATA_TYPE, + run_night_summarize_time_bundle, +) + + +class SummaryPayloadAdapter: + model_id = "order-166-summary-payload-adapter" + + def __init__(self, payload: dict[str, object]) -> None: + self.payload = payload + self.last_input_payload: dict[str, object] | None = None + + def complete(self, request: LLMRequest) -> LLMResponse: + self.last_input_payload = request.input_payload + return LLMResponse( + text=json.dumps(self.payload, ensure_ascii=False), + model_id=self.model_id, + raw=self.payload, + ) + + +def test_time_bundle_summary_for_multiple_capsules_is_mixed_graph_node() -> None: + trace_store, data_store, time_bundle_id = _store_with_capsules( + [_sample_capsule("turn_order_166_a"), _sample_capsule("turn_order_166_b")], + batch_id="batch_order_166_multi", + ) + adapter = SummaryPayloadAdapter( + {"summary_text": "This TimeBundle contains two raw capsule coordinate records."} + ) + + result = run_night_summarize_time_bundle( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_166", + target_time_bundle_node_id=time_bundle_id, + summary_run_id="001", + adapter=adapter, + ) + + frame = result.frame + validate_night_time_bundle_summary_frame(frame) + assert frame.summary_status == "ran" + assert frame.info_class == "mixed" + assert frame.source_mode == "source_bundle" + assert frame.claim_alignment == "multi_source_bundle" + assert frame.summary_depth == 1 + assert frame.source_leaf_count == 2 + assert frame.generated_by == f"LLM:{adapter.model_id}:night_summarize_time_bundle" + assert frame.semantic_judgement_status == "ran" + assert frame.llm_call_data_id in frame.source_data_ids + + summary_record = data_store.require_record(frame.summary_graph_node_id) + assert summary_record.data_type == "graph_memory:node:summary" + assert summary_record.payload["node_kind"] == "summary" + assert summary_record.payload["target_graph_node_id"] == time_bundle_id + + edge_record = data_store.require_record(result.summary_edge_data_id or "") + assert edge_record.data_type == "graph_memory:edge:SUMMARY_OF" + assert edge_record.payload["edge_kind"] == "SUMMARY_OF" + assert edge_record.payload["from_node_id"] == frame.summary_graph_node_id + assert edge_record.payload["to_node_id"] == time_bundle_id + + original_time_bundle = data_store.require_record(time_bundle_id) + assert original_time_bundle.payload["generated_by"] == "CODE:GRAPH_MEMORY_BUILDER" + assert original_time_bundle.payload["info_class"] == "absolute" + assert original_time_bundle.payload["semantic_judgement_status"] == "not_run" + assert "summary_text" not in original_time_bundle.payload + + assert adapter.last_input_payload is not None + assert adapter.last_input_payload["expected_info_class"] == "mixed" + + +def test_single_leaf_time_bundle_summary_is_relative() -> None: + trace_store, data_store, time_bundle_id = _store_with_capsules( + [_sample_capsule("turn_order_166_single")], + batch_id="batch_order_166_single", + ) + adapter = SummaryPayloadAdapter( + { + "summary_text": "This TimeBundle contains one raw capsule coordinate record.", + "info_class": "relative", + } + ) + + result = run_night_summarize_time_bundle( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_166", + target_time_bundle_node_id=time_bundle_id, + summary_run_id="001", + adapter=adapter, + ) + + frame = result.frame + assert frame.summary_status == "ran" + assert frame.info_class == "relative" + assert frame.source_mode == "single_source" + assert frame.claim_alignment == "single_absolute_record" + assert frame.source_leaf_count == 1 + assert len(frame.source_graph_node_ids) == 1 + + +def test_incompatible_llm_info_class_fails_without_graph_summary_node() -> None: + trace_store, data_store, time_bundle_id = _store_with_capsules( + [_sample_capsule("turn_order_166_c"), _sample_capsule("turn_order_166_d")], + batch_id="batch_order_166_bad_class", + ) + adapter = SummaryPayloadAdapter( + { + "summary_text": "This payload intentionally supplies the wrong class.", + "info_class": "relative", + } + ) + + result = run_night_summarize_time_bundle( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_166", + target_time_bundle_node_id=time_bundle_id, + summary_run_id="001", + adapter=adapter, + ) + + frame = result.frame + assert frame.summary_status == "failed" + assert frame.failure_type == "schema_failed" + assert frame.summary_text == "" + assert frame.info_class == "mixed" + assert frame.semantic_judgement_status == "failed" + assert result.summary_graph_node_data_id is None + assert data_store.get_record(frame.summary_graph_node_id) is None + assert data_store.require_record(frame.frame_id).data_type == ( + NIGHT_SUMMARIZE_TIME_BUNDLE_FRAME_DATA_TYPE + ) + assert frame.llm_call_data_id is not None + assert data_store.require_record(frame.llm_call_data_id).payload["failure_type"] == ( + "schema_failed" + ) + + +def test_summary_node_and_edge_are_exported_and_planned_for_vessel() -> None: + trace_store, data_store, time_bundle_id = _store_with_capsules( + [_sample_capsule("turn_order_166_export_a")], + batch_id="batch_order_166_export_core", + ) + result = run_night_summarize_time_bundle( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_166", + target_time_bundle_node_id=time_bundle_id, + summary_run_id="001", + adapter=SummaryPayloadAdapter({"summary_text": "One raw capsule coordinate record."}), + ) + + export = record_graph_memory_export_packet( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_166_export", + batch_id="batch_order_166_export", + ) + + assert result.summary_graph_node_data_id in export.packet.graph_node_data_ids + assert result.summary_edge_data_id in export.packet.graph_edge_data_ids + + plan = record_graph_vessel_write_plan( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_166_plan", + export_packet_id=export.packet.packet_id, + ) + operations_by_source = { + operation.source_data_id: operation.operation_kind + for operation in plan.plan.operations + } + assert operations_by_source[result.summary_graph_node_data_id or ""] == ( + "upsert_graph_node" + ) + assert operations_by_source[result.summary_edge_data_id or ""] == ( + "upsert_graph_edge" + ) + + +def _store_with_capsules( + capsules: list[TurnStateCapsule], + *, + batch_id: str, +) -> tuple[TraceStore, DataStore, str]: + trace_store = TraceStore() + data_store = DataStore() + record_graph_memory_for_capsules( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_166_graph", + batch_id=batch_id, + capsules=capsules, + ) + time_bundle_records = [ + record + for record in data_store.list_records() + if record.data_type == "graph_memory:node:time_bundle" + ] + assert len(time_bundle_records) == 1 + return trace_store, data_store, time_bundle_records[0].data_id + + +def _sample_capsule(turn_id: str) -> TurnStateCapsule: + return TurnStateCapsule( + turn_id=turn_id, + node_movements=[ + NodeMovement( + movement_id=f"move:{turn_id}:001", + turn_id=turn_id, + step_index=1, + node_id="node_0", + mode="pre_route_report", + input_trace_ids=[f"trace:{turn_id}:user"], + output_trace_ids=[f"trace:{turn_id}:node0"], + status="completed", + ) + ], + trace_event_ids=[ + f"trace:{turn_id}:user", + f"trace:{turn_id}:node0", + f"trace:{turn_id}:final", + ], + user_input_trace_id=f"trace:{turn_id}:user", + final_response_trace_id=f"trace:{turn_id}:final", + ) diff --git a/tests/test_order_168_night_summarize_changed_source_leaves.py b/tests/test_order_168_night_summarize_changed_source_leaves.py new file mode 100644 index 0000000..9df4c4f --- /dev/null +++ b/tests/test_order_168_night_summarize_changed_source_leaves.py @@ -0,0 +1,345 @@ +from __future__ import annotations + +import json + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.graph_memory import record_graph_memory_for_capsules +from songryeon_core.core.graph_memory_export import record_graph_memory_export_packet +from songryeon_core.core.graph_source_ingest import record_graph_source_kind_ingest +from songryeon_core.core.graph_vessel_adapter import record_graph_vessel_write_plan +from songryeon_core.core.schemas import ( + NightSourceLeafSummaryFrame, + NodeMovement, + TurnStateCapsule, + validate_night_source_leaf_summary_frame, +) +from songryeon_core.core.trace_store import TraceStore +from songryeon_core.llm.base import LLMRequest, LLMResponse +from songryeon_core.nodes.night_summarize_source_leaf import ( + NIGHT_SUMMARIZE_SOURCE_LEAF_FRAME_DATA_TYPE, + run_night_summarize_changed_source_leaves, +) + + +class SourceSummaryAdapter: + model_id = "order-168-source-summary-adapter" + + def __init__(self) -> None: + self.input_payloads: list[dict[str, object]] = [] + + def complete(self, request: LLMRequest) -> LLMResponse: + self.input_payloads.append(request.input_payload) + source_text = str(request.input_payload.get("source_text") or "") + payload = { + "summary_text": f"Source leaf summary: {source_text.strip()[:40]}", + } + return LLMResponse( + text=json.dumps(payload, ensure_ascii=False), + model_id=self.model_id, + raw=payload, + ) + + +def test_new_source_versions_are_all_summarized_one_to_one(tmp_path) -> None: + source_a = tmp_path / "policy_a.md" + source_b = tmp_path / "tool_b.py" + source_a.write_text("policy alpha\n", encoding="utf-8") + source_b.write_text("VALUE = 'beta'\n", encoding="utf-8") + trace_store, data_store = _core_time_axis_store() + ingest = record_graph_source_kind_ingest( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_168_ingest", + batch_id="batch_order_168_new", + source_paths_by_kind={ + "internal_document": [source_a], + "source_code_file": [source_b], + }, + observed_at="2026-07-02T20:00:00", + ingested_at="2026-07-02T20:00:01", + store_text_snapshots=True, + ) + adapter = SourceSummaryAdapter() + + batch = run_night_summarize_changed_source_leaves( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_168_summary", + source_observation_ledger_frame_id=ingest.source_observation_ledger_frame_id, + summary_run_id="001", + adapter=adapter, + ) + + assert batch.selected_source_graph_node_ids == ingest.raw_source_node_ids + assert len(batch.results) == 2 + assert len(adapter.input_payloads) == 2 + for result in batch.results: + frame = result.frame + validate_night_source_leaf_summary_frame(frame) + assert frame.summary_status == "ran" + assert frame.info_class == "relative" + assert frame.source_mode == "single_source" + assert frame.claim_alignment == "single_absolute_record" + assert frame.source_graph_node_ids == [frame.target_graph_node_id] + assert frame.generated_by == ( + f"LLM:{adapter.model_id}:night_summarize_source_leaf" + ) + assert result.summary_graph_node_data_id == frame.summary_graph_node_id + assert result.summary_edge_data_id is not None + summary_record = data_store.require_record(frame.summary_graph_node_id) + assert summary_record.data_type == "graph_memory:node:summary" + assert summary_record.payload["target_node_kind"] == "raw_source" + edge_record = data_store.require_record(result.summary_edge_data_id) + assert edge_record.data_type == "graph_memory:edge:SUMMARY_OF" + assert edge_record.payload["from_node_id"] == frame.summary_graph_node_id + assert edge_record.payload["to_node_id"] == frame.target_graph_node_id + + +def test_unchanged_observation_is_not_summarized(tmp_path) -> None: + source = tmp_path / "unchanged.md" + source.write_text("same content\n", encoding="utf-8") + trace_store, data_store = _core_time_axis_store() + record_graph_source_kind_ingest( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_168_ingest", + batch_id="batch_order_168_unchanged_001", + source_paths_by_kind={"internal_document": [source]}, + observed_at="2026-07-02T21:00:00", + ingested_at="2026-07-02T21:00:01", + store_text_snapshots=True, + ) + second = record_graph_source_kind_ingest( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_168_ingest", + batch_id="batch_order_168_unchanged_002", + source_paths_by_kind={"internal_document": [source]}, + observed_at="2026-07-02T22:00:00", + ingested_at="2026-07-02T22:00:01", + store_text_snapshots=True, + ) + adapter = SourceSummaryAdapter() + + batch = run_night_summarize_changed_source_leaves( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_168_summary", + source_observation_ledger_frame_id=second.source_observation_ledger_frame_id, + summary_run_id="001", + adapter=adapter, + ) + + assert batch.selected_source_graph_node_ids == [] + assert batch.skipped_unchanged_source_graph_node_ids == second.raw_source_node_ids + assert batch.results == [] + assert adapter.input_payloads == [] + assert _summary_record_count(data_store) == 0 + + +def test_changed_source_version_summarizes_only_new_active_leaf(tmp_path) -> None: + source = tmp_path / "changing.py" + source.write_text("VALUE = 1\n", encoding="utf-8") + trace_store, data_store = _core_time_axis_store() + first = record_graph_source_kind_ingest( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_168_ingest", + batch_id="batch_order_168_change_001", + source_paths_by_kind={"source_code_file": [source]}, + observed_at="2026-07-02T23:00:00", + ingested_at="2026-07-02T23:00:01", + store_text_snapshots=True, + ) + source.write_text("VALUE = 2\n", encoding="utf-8") + second = record_graph_source_kind_ingest( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_168_ingest", + batch_id="batch_order_168_change_002", + source_paths_by_kind={"source_code_file": [source]}, + observed_at="2026-07-03T00:00:00", + ingested_at="2026-07-03T00:00:01", + store_text_snapshots=True, + ) + + batch = run_night_summarize_changed_source_leaves( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_168_summary", + source_observation_ledger_frame_id=second.source_observation_ledger_frame_id, + summary_run_id="001", + adapter=SourceSummaryAdapter(), + ) + + assert batch.selected_source_graph_node_ids == [second.raw_source_node_ids[0]] + assert batch.results[0].frame.target_graph_node_id == second.raw_source_node_ids[0] + assert batch.results[0].frame.target_graph_node_id != first.raw_source_node_ids[0] + assert _summary_record_count(data_store) == 1 + + +def test_missing_text_snapshot_records_skipped_without_summary_graph_node(tmp_path) -> None: + source = tmp_path / "metadata_only.md" + source.write_text("text exists but snapshot disabled\n", encoding="utf-8") + trace_store, data_store = _core_time_axis_store() + ingest = record_graph_source_kind_ingest( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_168_ingest", + batch_id="batch_order_168_no_snapshot", + source_paths_by_kind={"internal_document": [source]}, + observed_at="2026-07-03T01:00:00", + ingested_at="2026-07-03T01:00:01", + store_text_snapshots=False, + ) + + batch = run_night_summarize_changed_source_leaves( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_168_summary", + source_observation_ledger_frame_id=ingest.source_observation_ledger_frame_id, + summary_run_id="001", + adapter=SourceSummaryAdapter(), + ) + + frame = batch.results[0].frame + assert frame.summary_status == "skipped_no_text_snapshot" + assert frame.info_class == "absolute" + assert frame.semantic_judgement_status == "not_run" + assert batch.results[0].summary_graph_node_data_id is None + assert data_store.get_record(frame.summary_graph_node_id) is None + assert data_store.require_record(frame.frame_id).data_type == ( + NIGHT_SUMMARIZE_SOURCE_LEAF_FRAME_DATA_TYPE + ) + assert _summary_record_count(data_store) == 0 + + +def test_empty_text_snapshot_records_skipped_empty_text(tmp_path) -> None: + source = tmp_path / "empty.md" + source.write_text("", encoding="utf-8") + trace_store, data_store = _core_time_axis_store() + ingest = record_graph_source_kind_ingest( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_168_ingest", + batch_id="batch_order_168_empty", + source_paths_by_kind={"internal_document": [source]}, + observed_at="2026-07-03T02:00:00", + ingested_at="2026-07-03T02:00:01", + store_text_snapshots=True, + ) + + batch = run_night_summarize_changed_source_leaves( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_168_summary", + source_observation_ledger_frame_id=ingest.source_observation_ledger_frame_id, + summary_run_id="001", + adapter=SourceSummaryAdapter(), + ) + + frame = batch.results[0].frame + assert frame.summary_status == "skipped_empty_text" + assert frame.failure_type == "empty_text" + assert frame.text_snapshot_data_id is not None + assert batch.results[0].summary_graph_node_data_id is None + assert _summary_record_count(data_store) == 0 + + +def test_source_leaf_summary_node_and_edge_are_exported_and_planned(tmp_path) -> None: + source = tmp_path / "export_me.md" + source.write_text("exported source text\n", encoding="utf-8") + trace_store, data_store = _core_time_axis_store() + ingest = record_graph_source_kind_ingest( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_168_ingest", + batch_id="batch_order_168_export_source", + source_paths_by_kind={"internal_document": [source]}, + observed_at="2026-07-03T03:00:00", + ingested_at="2026-07-03T03:00:01", + store_text_snapshots=True, + ) + batch = run_night_summarize_changed_source_leaves( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_168_summary", + source_observation_ledger_frame_id=ingest.source_observation_ledger_frame_id, + summary_run_id="001", + adapter=SourceSummaryAdapter(), + ) + result = batch.results[0] + + export = record_graph_memory_export_packet( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_168_export", + batch_id="batch_order_168_export", + ) + assert result.summary_graph_node_data_id in export.packet.graph_node_data_ids + assert result.summary_edge_data_id in export.packet.graph_edge_data_ids + + plan = record_graph_vessel_write_plan( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_168_plan", + export_packet_id=export.packet.packet_id, + ) + operations_by_source = { + operation.source_data_id: operation.operation_kind + for operation in plan.plan.operations + } + assert operations_by_source[result.summary_graph_node_data_id or ""] == ( + "upsert_graph_node" + ) + assert operations_by_source[result.summary_edge_data_id or ""] == ( + "upsert_graph_edge" + ) + + +def _summary_record_count(data_store: DataStore) -> int: + return len( + [ + record + for record in data_store.list_records() + if record.data_type == "graph_memory:node:summary" + ] + ) + + +def _core_time_axis_store() -> tuple[TraceStore, DataStore]: + trace_store = TraceStore() + data_store = DataStore() + record_graph_memory_for_capsules( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_168_core", + batch_id="batch_order_168_core", + capsules=[_sample_capsule()], + ) + return trace_store, data_store + + +def _sample_capsule(turn_id: str = "turn_order_168_previous") -> TurnStateCapsule: + return TurnStateCapsule( + turn_id=turn_id, + node_movements=[ + NodeMovement( + movement_id=f"move:{turn_id}:001", + turn_id=turn_id, + step_index=1, + node_id="node_0", + mode="pre_route_report", + input_trace_ids=[f"trace:{turn_id}:user"], + output_trace_ids=[f"trace:{turn_id}:node0"], + status="completed", + ) + ], + trace_event_ids=[ + f"trace:{turn_id}:user", + f"trace:{turn_id}:node0", + f"trace:{turn_id}:final", + ], + user_input_trace_id=f"trace:{turn_id}:user", + final_response_trace_id=f"trace:{turn_id}:final", + ) diff --git a/tests/test_order_169_night_changed_source_summary_cli.py b/tests/test_order_169_night_changed_source_summary_cli.py new file mode 100644 index 0000000..85d797c --- /dev/null +++ b/tests/test_order_169_night_changed_source_summary_cli.py @@ -0,0 +1,133 @@ +from __future__ import annotations + +import json +import subprocess +import sys + +from songryeon_core.runtime.night_changed_source_summary import ( + run_night_changed_source_summary, +) + + +def test_night_changed_source_summary_first_run_summarizes_manifest_leaves( + tmp_path, +) -> None: + root = _minimal_songryeon_root(tmp_path / "project") + store_dir = tmp_path / "store" + + result = run_night_changed_source_summary( + root_path=root, + store_dir=store_dir, + batch_id="batch_order_169_first", + turn_id="turn_order_169_first", + llm_mode="fake", + ) + + assert result["status"] == "NIGHT_CHANGED_SOURCE_SUMMARY_OK" + assert result["source_observation_status_counts"] == {"new_source_version": 4} + assert result["selected_source_leaf_count"] == 4 + assert result["summary_success_count"] == 4 + assert result["summary_failed_count"] == 0 + assert len(result["summary_graph_node_ids"]) == 4 + assert result["write_plan_status"] == "ready_to_write" + assert (store_dir / "trace_store.json").exists() + assert (store_dir / "data_store.json").exists() + + +def test_night_changed_source_summary_second_run_skips_unchanged(tmp_path) -> None: + root = _minimal_songryeon_root(tmp_path / "project") + store_dir = tmp_path / "store" + run_night_changed_source_summary( + root_path=root, + store_dir=store_dir, + batch_id="batch_order_169_unchanged_001", + turn_id="turn_order_169_unchanged_001", + llm_mode="fake", + ) + + result = run_night_changed_source_summary( + root_path=root, + store_dir=store_dir, + batch_id="batch_order_169_unchanged_002", + turn_id="turn_order_169_unchanged_002", + llm_mode="fake", + ) + + assert result["source_observation_status_counts"] == {"unchanged": 4} + assert result["selected_source_leaf_count"] == 0 + assert result["skipped_unchanged_source_leaf_count"] == 4 + assert result["summary_success_count"] == 0 + assert result["summary_graph_node_ids"] == [] + + +def test_night_changed_source_summary_only_summarizes_changed_file(tmp_path) -> None: + root = _minimal_songryeon_root(tmp_path / "project") + store_dir = tmp_path / "store" + run_night_changed_source_summary( + root_path=root, + store_dir=store_dir, + batch_id="batch_order_169_change_001", + turn_id="turn_order_169_change_001", + llm_mode="fake", + ) + (root / "main.py").write_text("VALUE = 2\n", encoding="utf-8") + + result = run_night_changed_source_summary( + root_path=root, + store_dir=store_dir, + batch_id="batch_order_169_change_002", + turn_id="turn_order_169_change_002", + llm_mode="fake", + ) + + assert result["source_observation_status_counts"] == { + "unchanged": 3, + "content_changed": 1, + } + assert result["selected_source_leaf_count"] == 1 + assert result["summary_success_count"] == 1 + assert len(result["summary_graph_node_ids"]) == 1 + + +def test_night_changed_source_summary_main_cli_runs_with_fake_mode(tmp_path) -> None: + root = _minimal_songryeon_root(tmp_path / "project") + store_dir = tmp_path / "store" + + completed = subprocess.run( + [ + sys.executable, + "main.py", + "night-summarize-changed-sources", + "--root", + str(root), + "--store-dir", + str(store_dir), + "--batch-id", + "batch_order_169_cli", + "--turn-id", + "turn_order_169_cli", + "--llm-mode", + "fake", + ], + check=True, + capture_output=True, + text=True, + encoding="utf-8", + ) + payload = json.loads(completed.stdout) + + assert payload["status"] == "NIGHT_CHANGED_SOURCE_SUMMARY_OK" + assert payload["summary_success_count"] == 4 + assert payload["write_vessel_requested"] is False + assert payload["write_result"] is None + + +def _minimal_songryeon_root(root) -> object: + root.mkdir(parents=True) + (root / "AGENTS.md").write_text("# Agents\n", encoding="utf-8") + (root / "README.md").write_text("# Readme\n", encoding="utf-8") + (root / "main.py").write_text("VALUE = 1\n", encoding="utf-8") + docs = root / "Administrative_Reform_1" / "04_Orders" + docs.mkdir(parents=True) + (docs / "ORDER_TEST.md").write_text("# Test order\n", encoding="utf-8") + return root diff --git a/tests/test_order_170_night_changed_source_one_at_a_time.py b/tests/test_order_170_night_changed_source_one_at_a_time.py new file mode 100644 index 0000000..ed74129 --- /dev/null +++ b/tests/test_order_170_night_changed_source_one_at_a_time.py @@ -0,0 +1,137 @@ +from __future__ import annotations + +import json +import subprocess +import sys + +from songryeon_core.core.data_store import DataStore +from songryeon_core.runtime.night_changed_source_summary import ( + NIGHT_CHANGED_SOURCE_ONE_AT_A_TIME_QUEUE_DATA_TYPE, + run_night_changed_source_summary, +) + + +def test_one_at_a_time_processes_one_leaf_per_run_and_resumes_queue(tmp_path) -> None: + root = _minimal_songryeon_root(tmp_path / "project") + store_dir = tmp_path / "store" + + first = run_night_changed_source_summary( + root_path=root, + store_dir=store_dir, + batch_id="batch_order_170_one", + turn_id="turn_order_170_one_001", + llm_mode="fake", + one_at_a_time=True, + ) + second = run_night_changed_source_summary( + root_path=root, + store_dir=store_dir, + batch_id="batch_order_170_one", + turn_id="turn_order_170_one_002", + llm_mode="fake", + one_at_a_time=True, + ) + + assert first["execution_mode"] == "one_at_a_time" + assert first["one_at_a_time_queue_status"] == "created" + assert first["selected_source_leaf_count"] == 4 + assert first["summary_success_count"] == 1 + assert first["one_at_a_time_processed_this_run"] == 1 + assert first["one_at_a_time_queue_processed_count_after"] == 1 + assert first["one_at_a_time_queue_pending_count_after"] == 3 + + assert second["one_at_a_time_queue_status"] == "existing" + assert second["source_observation_status_counts"] == {"new_source_version": 4} + assert second["summary_success_count"] == 1 + assert second["one_at_a_time_queue_processed_count_before"] == 1 + assert second["one_at_a_time_queue_processed_count_after"] == 2 + assert second["one_at_a_time_queue_pending_count_after"] == 2 + + data_store = DataStore.load_json(store_dir / "data_store.json") + summary_records = [ + record + for record in data_store.list_records() + if record.data_type == "graph_memory:node:summary" + ] + queue_records = [ + record + for record in data_store.list_records() + if record.data_type == NIGHT_CHANGED_SOURCE_ONE_AT_A_TIME_QUEUE_DATA_TYPE + ] + assert len(summary_records) == 2 + assert len(queue_records) == 1 + + +def test_one_at_a_time_reports_complete_after_queue_is_exhausted(tmp_path) -> None: + root = _minimal_songryeon_root(tmp_path / "project") + store_dir = tmp_path / "store" + + results = [ + run_night_changed_source_summary( + root_path=root, + store_dir=store_dir, + batch_id="batch_order_170_complete", + turn_id=f"turn_order_170_complete_{index:03d}", + llm_mode="fake", + one_at_a_time=True, + ) + for index in range(1, 6) + ] + + assert [result["one_at_a_time_processed_this_run"] for result in results] == [ + 1, + 1, + 1, + 1, + 0, + ] + assert results[-1]["one_at_a_time_completion_status"] == "complete" + assert results[-1]["one_at_a_time_queue_processed_count_after"] == 4 + assert results[-1]["one_at_a_time_queue_pending_count_after"] == 0 + assert results[-1]["summary_success_count"] == 0 + + +def test_one_at_a_time_main_cli_runs_single_step_with_fake_mode(tmp_path) -> None: + root = _minimal_songryeon_root(tmp_path / "project") + store_dir = tmp_path / "store" + + completed = subprocess.run( + [ + sys.executable, + "main.py", + "night-summarize-changed-sources", + "--root", + str(root), + "--store-dir", + str(store_dir), + "--batch-id", + "batch_order_170_cli", + "--turn-id", + "turn_order_170_cli", + "--llm-mode", + "fake", + "--one-at-a-time", + ], + check=True, + capture_output=True, + text=True, + encoding="utf-8", + ) + payload = json.loads(completed.stdout) + + assert payload["status"] == "NIGHT_CHANGED_SOURCE_SUMMARY_OK" + assert payload["execution_mode"] == "one_at_a_time" + assert payload["summary_success_count"] == 1 + assert payload["one_at_a_time_processed_this_run"] == 1 + assert payload["one_at_a_time_queue_pending_count_after"] == 3 + + +def _minimal_songryeon_root(root) -> object: + root.mkdir(parents=True) + (root / "AGENTS.md").write_text("# Agents\n", encoding="utf-8") + (root / "README.md").write_text("# Readme\n", encoding="utf-8") + (root / "main.py").write_text("VALUE = 1\n", encoding="utf-8") + docs = root / "Administrative_Reform_1" / "04_Orders" + docs.mkdir(parents=True) + (docs / "ORDER_TEST.md").write_text("# Test order\n", encoding="utf-8") + return root diff --git a/tests/test_order_171_night_token_budget_layer_summary.py b/tests/test_order_171_night_token_budget_layer_summary.py new file mode 100644 index 0000000..17d96a0 --- /dev/null +++ b/tests/test_order_171_night_token_budget_layer_summary.py @@ -0,0 +1,148 @@ +from __future__ import annotations + +import json +import subprocess +import sys + +from songryeon_core.core.data_store import DataStore +from songryeon_core.runtime.night_changed_source_summary import ( + run_night_changed_source_summary, +) +from songryeon_core.runtime.night_token_budget_layer_summary import ( + NIGHT_TOKEN_BUDGET_LAYER_QUEUE_DATA_TYPE, + run_night_token_budget_layer_summary, +) + + +def test_token_budget_layer_processes_one_bundle_per_run(tmp_path) -> None: + root = _minimal_songryeon_root(tmp_path / "project") + store_dir = tmp_path / "store" + run_night_changed_source_summary( + root_path=root, + store_dir=store_dir, + batch_id="batch_order_171_sources", + turn_id="turn_order_171_sources", + llm_mode="fake", + ) + + first = run_night_token_budget_layer_summary( + store_dir=store_dir, + batch_id="batch_order_171_layer", + turn_id="turn_order_171_layer_001", + max_bundle_chars=90, + llm_mode="fake", + ) + second = run_night_token_budget_layer_summary( + store_dir=store_dir, + batch_id="batch_order_171_layer", + turn_id="turn_order_171_layer_002", + max_bundle_chars=90, + llm_mode="fake", + ) + + assert first["status"] == "NIGHT_TOKEN_BUDGET_LAYER_SUMMARY_OK" + assert first["execution_mode"] == "one_bundle_at_a_time" + assert first["budget_unit"] == "characters" + assert first["source_summary_count"] == 4 + assert first["bundle_count"] >= 2 + assert first["processed_this_run"] == 1 + assert first["processed_count_after"] == 1 + assert first["pending_count_after"] == first["bundle_count"] - 1 + assert first["summary_status"] == "ran" + + assert second["queue_status"] == "existing" + assert second["processed_this_run"] == 1 + assert second["processed_count_before"] == 1 + assert second["processed_count_after"] == 2 + + data_store = DataStore.load_json(store_dir / "data_store.json") + queue_records = [ + record + for record in data_store.list_records() + if record.data_type == NIGHT_TOKEN_BUDGET_LAYER_QUEUE_DATA_TYPE + ] + bundle_records = [ + record + for record in data_store.list_records() + if record.data_type == "graph_memory:node:token_budget_summary_bundle" + ] + token_summary_records = [ + record + for record in data_store.list_records() + if record.data_type == "graph_memory:node:summary" + and isinstance(record.payload, dict) + and record.payload.get("data_kind") == "token_budget_bundle_summary" + ] + assert len(queue_records) == 1 + assert len(bundle_records) == 2 + assert len(token_summary_records) == 2 + assert token_summary_records[0].payload["info_class"] == "mixed" + assert token_summary_records[0].payload["semantic_judgement_status"] == "ran" + + +def test_token_budget_layer_reports_no_candidates_without_leaf_summaries(tmp_path) -> None: + store_dir = tmp_path / "empty_store" + + result = run_night_token_budget_layer_summary( + store_dir=store_dir, + batch_id="batch_order_171_empty", + turn_id="turn_order_171_empty", + max_bundle_chars=90, + llm_mode="fake", + ) + + assert result["source_summary_count"] == 0 + assert result["bundle_count"] == 0 + assert result["processed_this_run"] == 0 + assert result["completion_status"] == "no_candidates" + + +def test_token_budget_layer_main_cli_runs_one_step(tmp_path) -> None: + root = _minimal_songryeon_root(tmp_path / "project") + store_dir = tmp_path / "store" + run_night_changed_source_summary( + root_path=root, + store_dir=store_dir, + batch_id="batch_order_171_cli_sources", + turn_id="turn_order_171_cli_sources", + llm_mode="fake", + ) + + completed = subprocess.run( + [ + sys.executable, + "main.py", + "night-summarize-token-layer", + "--store-dir", + str(store_dir), + "--batch-id", + "batch_order_171_cli_layer", + "--turn-id", + "turn_order_171_cli_layer", + "--max-bundle-chars", + "90", + "--llm-mode", + "fake", + ], + check=True, + capture_output=True, + text=True, + encoding="utf-8", + ) + payload = json.loads(completed.stdout) + + assert payload["status"] == "NIGHT_TOKEN_BUDGET_LAYER_SUMMARY_OK" + assert payload["processed_this_run"] == 1 + assert payload["summary_status"] == "ran" + assert payload["summary_graph_node_id"].startswith("graph:summary:token_budget_bundle:") + + +def _minimal_songryeon_root(root) -> object: + root.mkdir(parents=True) + (root / "AGENTS.md").write_text("# Agents\n", encoding="utf-8") + (root / "README.md").write_text("# Readme\n", encoding="utf-8") + (root / "main.py").write_text("VALUE = 1\n", encoding="utf-8") + docs = root / "Administrative_Reform_1" / "04_Orders" + docs.mkdir(parents=True) + (docs / "ORDER_TEST.md").write_text("# Test order\n", encoding="utf-8") + return root diff --git a/tests/test_order_172_night_token_layer_auto_reduce.py b/tests/test_order_172_night_token_layer_auto_reduce.py new file mode 100644 index 0000000..a93834a --- /dev/null +++ b/tests/test_order_172_night_token_layer_auto_reduce.py @@ -0,0 +1,146 @@ +from __future__ import annotations + +import json +import subprocess +import sys + +from songryeon_core.core.data_store import DataStore +from songryeon_core.runtime.night_changed_source_summary import ( + run_night_changed_source_summary, +) +from songryeon_core.runtime.night_token_budget_layer_summary import ( + run_night_token_budget_layer_summary, +) + + +def test_auto_reduce_runs_multiple_layers_until_context_budget(tmp_path) -> None: + root = _minimal_songryeon_root(tmp_path / "project") + store_dir = tmp_path / "store" + run_night_changed_source_summary( + root_path=root, + store_dir=store_dir, + batch_id="batch_order_172_sources", + turn_id="turn_order_172_sources", + llm_mode="fake", + ) + + result = run_night_token_budget_layer_summary( + store_dir=store_dir, + batch_id="batch_order_172_layer", + turn_id="turn_order_172_layer", + max_bundle_chars=200, + llm_mode="fake", + until_context_budget=True, + target_context_chars=80, + max_layer_depth=5, + max_steps=10, + ) + + assert result["status"] == "NIGHT_TOKEN_BUDGET_AUTO_REDUCE_OK" + assert result["execution_mode"] == "auto_reduce_until_context_budget" + assert result["auto_reduce_status"] == "target_context_reached" + assert result["step_count"] >= 2 + assert result["final_layer_depth"] >= 2 + assert result["final_summary_char_count"] <= 80 + assert any(step.get("target_layer_depth") == 3 for step in result["steps"]) + + data_store = DataStore.load_json(store_dir / "data_store.json") + token_summary_depths = sorted( + { + record.payload.get("summary_depth") + for record in data_store.list_records() + if record.data_type == "graph_memory:node:summary" + and isinstance(record.payload, dict) + and record.payload.get("data_kind") == "token_budget_bundle_summary" + } + ) + assert max(token_summary_depths) >= 3 + + +def test_auto_reduce_stops_at_max_steps_without_unbounded_run(tmp_path) -> None: + root = _minimal_songryeon_root(tmp_path / "project") + store_dir = tmp_path / "store" + run_night_changed_source_summary( + root_path=root, + store_dir=store_dir, + batch_id="batch_order_172_max_steps_sources", + turn_id="turn_order_172_max_steps_sources", + llm_mode="fake", + ) + + result = run_night_token_budget_layer_summary( + store_dir=store_dir, + batch_id="batch_order_172_max_steps_layer", + turn_id="turn_order_172_max_steps_layer", + max_bundle_chars=90, + llm_mode="fake", + until_context_budget=True, + target_context_chars=1, + max_layer_depth=5, + max_steps=1, + ) + + assert result["auto_reduce_status"] == "max_steps_reached" + assert result["step_count"] == 1 + assert result["steps"][0]["processed_this_step"] == 1 + + +def test_auto_reduce_main_cli_runs_with_fake_mode(tmp_path) -> None: + root = _minimal_songryeon_root(tmp_path / "project") + store_dir = tmp_path / "store" + run_night_changed_source_summary( + root_path=root, + store_dir=store_dir, + batch_id="batch_order_172_cli_sources", + turn_id="turn_order_172_cli_sources", + llm_mode="fake", + ) + + completed = subprocess.run( + [ + sys.executable, + "main.py", + "night-summarize-token-layer", + "--store-dir", + str(store_dir), + "--batch-id", + "batch_order_172_cli_layer", + "--turn-id", + "turn_order_172_cli_layer", + "--max-bundle-chars", + "200", + "--llm-mode", + "fake", + "--until-context-budget", + "--target-context-chars", + "80", + "--max-layer-depth", + "5", + "--max-steps", + "10", + ], + check=True, + capture_output=True, + text=True, + encoding="utf-8", + ) + payload = json.loads(completed.stdout) + + assert payload["status"] == "NIGHT_TOKEN_BUDGET_AUTO_REDUCE_OK" + assert payload["auto_reduce_status"] == "target_context_reached" + assert payload["final_summary_char_count"] <= 80 + + +def _minimal_songryeon_root(root) -> object: + root.mkdir(parents=True) + (root / "AGENTS.md").write_text("# Agents\n", encoding="utf-8") + (root / "README.md").write_text("# Readme\n", encoding="utf-8") + (root / "main.py").write_text("VALUE = 1\n", encoding="utf-8") + docs = root / "Administrative_Reform_1" / "04_Orders" + docs.mkdir(parents=True) + for index in range(1, 9): + (docs / f"ORDER_TEST_{index:03d}.md").write_text( + f"# Test order {index}\n", + encoding="utf-8", + ) + return root diff --git a/tests/test_order_173_night_checkpointed_long_runner.py b/tests/test_order_173_night_checkpointed_long_runner.py new file mode 100644 index 0000000..0d1c429 --- /dev/null +++ b/tests/test_order_173_night_checkpointed_long_runner.py @@ -0,0 +1,141 @@ +from __future__ import annotations + +import json +import subprocess +import sys + +import pytest + +from songryeon_core.runtime.night_changed_source_summary import ( + run_night_changed_source_summary, +) +from songryeon_core.runtime.night_token_budget_layer_summary import ( + run_night_token_budget_layer_summary, +) + + +def test_checkpointed_runner_writes_progress_jsonl(tmp_path) -> None: + root = _minimal_songryeon_root(tmp_path / "project") + store_dir = tmp_path / "store" + progress_path = tmp_path / "progress" / "night-progress.jsonl" + run_night_changed_source_summary( + root_path=root, + store_dir=store_dir, + batch_id="batch_order_173_sources", + turn_id="turn_order_173_sources", + llm_mode="fake", + ) + + result = run_night_token_budget_layer_summary( + store_dir=store_dir, + batch_id="batch_order_173_layer", + turn_id="turn_order_173_layer", + max_bundle_chars=90, + llm_mode="fake", + until_context_budget=True, + target_context_chars=1, + max_layer_depth=5, + max_steps=2, + progress_jsonl=progress_path, + vessel_write_mode="none", + ) + + assert result["status"] == "NIGHT_TOKEN_BUDGET_AUTO_REDUCE_OK" + assert result["auto_reduce_status"] == "max_steps_reached" + assert result["step_count"] == 2 + assert result["progress_jsonl_path"] == progress_path.resolve().as_posix() + assert result["vessel_write_mode"] == "none" + assert progress_path.exists() + + events = [ + json.loads(line) + for line in progress_path.read_text(encoding="utf-8").splitlines() + if line.strip() + ] + assert len(events) == 2 + assert {event["event_type"] for event in events} == {"auto_reduce_step"} + assert [event["step_index"] for event in events] == [1, 2] + assert all(event["processed_this_step"] == 1 for event in events) + + +def test_checkpointed_runner_rejects_unknown_vessel_write_mode(tmp_path) -> None: + with pytest.raises(ValueError, match="unknown vessel_write_mode"): + run_night_token_budget_layer_summary( + store_dir=tmp_path / "store", + llm_mode="fake", + until_context_budget=True, + vessel_write_mode="sometimes", + ) + + +def test_checkpointed_runner_main_cli_accepts_new_options(tmp_path) -> None: + root = _minimal_songryeon_root(tmp_path / "project") + store_dir = tmp_path / "store" + progress_path = tmp_path / "cli-progress.jsonl" + run_night_changed_source_summary( + root_path=root, + store_dir=store_dir, + batch_id="batch_order_173_cli_sources", + turn_id="turn_order_173_cli_sources", + llm_mode="fake", + ) + + completed = subprocess.run( + [ + sys.executable, + "main.py", + "night-summarize-token-layer", + "--store-dir", + str(store_dir), + "--batch-id", + "batch_order_173_cli_layer", + "--turn-id", + "turn_order_173_cli_layer", + "--max-bundle-chars", + "90", + "--llm-mode", + "fake", + "--until-context-budget", + "--target-context-chars", + "1", + "--max-layer-depth", + "5", + "--max-steps", + "1", + "--max-runtime-minutes", + "10", + "--progress-jsonl", + str(progress_path), + "--vessel-write-mode", + "none", + "--no-stop-on-failure", + ], + check=True, + capture_output=True, + text=True, + encoding="utf-8", + ) + payload = json.loads(completed.stdout) + + assert payload["status"] == "NIGHT_TOKEN_BUDGET_AUTO_REDUCE_OK" + assert payload["auto_reduce_status"] == "max_steps_reached" + assert payload["step_count"] == 1 + assert payload["max_runtime_minutes"] == 10.0 + assert payload["stop_on_failure"] is False + assert payload["vessel_write_mode"] == "none" + assert progress_path.exists() + + +def _minimal_songryeon_root(root) -> object: + root.mkdir(parents=True) + (root / "AGENTS.md").write_text("# Agents\n", encoding="utf-8") + (root / "README.md").write_text("# Readme\n", encoding="utf-8") + (root / "main.py").write_text("VALUE = 1\n", encoding="utf-8") + docs = root / "Administrative_Reform_1" / "04_Orders" + docs.mkdir(parents=True) + for index in range(1, 9): + (docs / f"ORDER_TEST_{index:03d}.md").write_text( + f"# Test order {index}\n", + encoding="utf-8", + ) + return root diff --git a/tests/test_order_174_vessel_inspect_summary_layer_view.py b/tests/test_order_174_vessel_inspect_summary_layer_view.py new file mode 100644 index 0000000..d968f1a --- /dev/null +++ b/tests/test_order_174_vessel_inspect_summary_layer_view.py @@ -0,0 +1,128 @@ +from __future__ import annotations + +import json + +from songryeon_core.core.graph_vessel_inspect import inspect_graph_vessel_from_neo4j +from songryeon_core.runtime.graph_vessel_inspect import render_vessel_inspect_text + +from tests.test_order_163_vessel_inspect_manual_walk import ( + INSPECTED_AT, + FakeInspectDriverFactory, + _config, +) + + +def test_vessel_inspect_reads_summary_layer_counts_and_samples() -> None: + result = inspect_graph_vessel_from_neo4j( + config=_config(), + batch_id="order_174_summary_layer", + created_at=INSPECTED_AT, + driver_factory=FakeInspectDriverFactory( + counts={"summary_count": 2}, + summary_rows=_summary_rows(), + ), + ) + + assert result.inspect_status == "passed" + assert result.summary_count == 2 + assert result.active_summary_count == 2 + assert result.invalidated_summary_count == 0 + assert result.summary_count_by_data_kind == { + "source_leaf_summary": 1, + "token_budget_bundle_summary": 1, + } + assert result.summary_count_by_depth == {"1": 1, "2": 1} + assert len(result.summary_sample_items) == 2 + assert result.summary_sample_items[0]["summary_text_preview"].startswith( + "Leaf summary for code tool file" + ) + assert "graph:summary:source_leaf:code_tools" in result.source_data_ids + assert "graph:raw_source:code_tools" in result.source_data_ids + + +def test_vessel_inspect_text_renderer_prints_summary_section() -> None: + result = { + "status": "VESSEL_INSPECT_OK", + "inspect_status": "passed", + "failure_type": None, + "failure_reason": None, + "graph_namespace": "songryeon_core_graph_v0", + "inspected_path_count": 1, + "summary_count": 2, + "active_summary_count": 2, + "invalidated_summary_count": 0, + "summary_count_by_data_kind": {"source_leaf_summary": 1}, + "summary_count_by_depth": {"1": 1}, + "tree_text": "CoreEgo [graph:core_ego:root]", + "summary_lines": [ + "Summary samples", + " source_leaf_summary(depth=1, info=relative) [graph:summary:source_leaf:code_tools]", + " preview: Leaf summary for code tool file.", + ], + } + + rendered = render_vessel_inspect_text(result) + + assert "summary_count: 2" in rendered + assert "summary_count_by_depth: {'1': 1}" in rendered + assert "Summary samples" in rendered + assert "Leaf summary for code tool file" in rendered + + +def _summary_rows() -> list[dict[str, object]]: + return [ + { + "summary_data_id": "graph:summary:source_leaf:code_tools", + "summary_display_name": "Summary: graph:raw_source:code_tools", + "summary_data_kind": "source_leaf_summary", + "summary_info_class": "relative", + "summary_generated_by": "LLM:fake:night_summarize_source_leaf", + "summary_payload_json": json.dumps( + { + "data_kind": "source_leaf_summary", + "summary_depth": 1, + "summary_status": "ran", + "validity_status": "active", + "review_status": "not_reviewed", + "target_graph_node_id": "graph:raw_source:code_tools", + "target_node_kind": "raw_source", + "source_leaf_count": 1, + "source_summary_count": 0, + "info_class": "relative", + "generated_by": "LLM:fake:night_summarize_source_leaf", + "summary_text": "Leaf summary for code tool file.", + }, + ensure_ascii=False, + ), + "target_graph_node_id": "graph:raw_source:code_tools", + "target_display_name": "Raw Source: code_tools.py", + "target_node_kind": "raw_source", + }, + { + "summary_data_id": "graph:summary:token_bundle:001", + "summary_display_name": "Summary: graph:token_budget_bundle:001", + "summary_data_kind": "token_budget_bundle_summary", + "summary_info_class": "mixed", + "summary_generated_by": "LLM:fake:night_summarize_token_budget_bundle", + "summary_payload_json": json.dumps( + { + "data_kind": "token_budget_bundle_summary", + "summary_depth": 2, + "summary_status": "ran", + "validity_status": "active", + "review_status": "not_reviewed", + "target_graph_node_id": "graph:token_budget_bundle:001", + "target_node_kind": "token_budget_summary_bundle", + "source_leaf_count": 8, + "source_summary_count": 8, + "info_class": "mixed", + "generated_by": "LLM:fake:night_summarize_token_budget_bundle", + "summary_text": "Layer summary over eight source leaf summaries.", + }, + ensure_ascii=False, + ), + "target_graph_node_id": "graph:token_budget_bundle:001", + "target_display_name": "Token Budget Bundle 001", + "target_node_kind": "token_budget_summary_bundle", + }, + ] diff --git a/tests/test_order_175_vessel_backed_r_read_packet.py b/tests/test_order_175_vessel_backed_r_read_packet.py new file mode 100644 index 0000000..9726a6d --- /dev/null +++ b/tests/test_order_175_vessel_backed_r_read_packet.py @@ -0,0 +1,398 @@ +from __future__ import annotations + +import json + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.graph_vessel_neo4j import GraphVesselNeo4jConfig +from songryeon_core.core.r_loop_vessel_read_packet import ( + R_LOOP_VESSEL_READ_PACKET_DATA_TYPE, + R_LOOP_VESSEL_READ_PACKET_GENERATOR, + build_r_loop_vessel_read_packet_from_neo4j, + record_r_loop_vessel_read_packet, +) +from songryeon_core.core.trace_store import TraceStore +from songryeon_core.runtime.r_loop_vessel_read_packet import ( + render_r_loop_vessel_read_packet_text, +) + + +READ_AT = "2026-07-02T19:00:00" + + +def test_r_loop_vessel_read_packet_reads_entries_and_active_summaries_only() -> None: + result = build_r_loop_vessel_read_packet_from_neo4j( + config=_config(), + batch_id="order_175_packet", + created_at=READ_AT, + driver_factory=FakeRLoopVesselDriverFactory( + entry_rows=_entry_rows(), + summary_rows=_summary_rows(), + ), + ) + + assert result.read_status == "passed" + assert result.entry_candidate_count == 2 + assert result.summary_candidate_count == 1 + assert result.total_summary_scanned_count == 3 + assert result.active_summary_candidate_count == 1 + assert result.skipped_summary_count == 2 + assert result.summary_count_by_data_kind == {"source_leaf_summary": 1} + assert result.summary_count_by_depth == {"1": 1} + assert result.entry_candidate_records[0]["candidate_kind"] == "source_ingest_bundle" + assert result.summary_candidate_records[0]["summary_node_id"] == ( + "graph:summary:source_leaf:code_tools" + ) + assert result.summary_candidate_records[0]["summary_text"] == ( + "This source leaf summary is copied from an active LLM summary node." + ) + assert "graph:summary:invalidated:old" not in result.source_data_ids + assert "graph:raw_source:code_tools" in result.source_data_ids + assert result.source_trace_ids == ["trace:source:manifest", "trace:summary:active"] + + +def test_r_loop_vessel_read_packet_marks_absolute_code_generated() -> None: + result = build_r_loop_vessel_read_packet_from_neo4j( + config=_config(), + batch_id="order_175_boundary", + created_at=READ_AT, + driver_factory=FakeRLoopVesselDriverFactory( + entry_rows=_entry_rows(), + summary_rows=_summary_rows(), + ), + ) + + assert result.generated_by == R_LOOP_VESSEL_READ_PACKET_GENERATOR + assert result.info_class == "absolute" + assert result.semantic_judgement_status == "not_run" + assert result.target_consumer == "R_LOOP" + + +def test_r_loop_vessel_read_packet_missing_password_does_not_call_driver() -> None: + fake_driver_factory = RaisingDriverFactory() + + result = build_r_loop_vessel_read_packet_from_neo4j( + config=_config(password=None), + batch_id="order_175_missing_password", + created_at=READ_AT, + driver_factory=fake_driver_factory, + ) + + assert result.read_status == "adapter_unavailable" + assert result.failure_type == "neo4j_config_missing" + assert result.entry_candidate_records == [] + assert fake_driver_factory.called is False + + +def test_r_loop_vessel_read_packet_records_payload_in_datastore() -> None: + trace_store = TraceStore() + data_store = DataStore() + + recorded = record_r_loop_vessel_read_packet( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_175", + batch_id="order_175_record", + config=_config(), + created_at=READ_AT, + driver_factory=FakeRLoopVesselDriverFactory( + entry_rows=_entry_rows(), + summary_rows=_summary_rows(), + ), + ) + + record = data_store.require_record(recorded.packet.packet_id) + assert record.data_type == R_LOOP_VESSEL_READ_PACKET_DATA_TYPE + assert record.payload["read_status"] == "passed" + assert record.payload["target_consumer"] == "R_LOOP" + assert record.payload["summary_candidate_count"] == 1 + assert recorded.created_data_ids == [recorded.packet.packet_id] + assert trace_store.list_events()[0].actor == "r_loop_vessel_read_packet_builder" + + +def test_r_loop_vessel_read_packet_text_renderer_prints_candidate_counts() -> None: + result = { + "status": "R_LOOP_VESSEL_READ_PACKET_OK", + "read_status": "passed", + "failure_type": None, + "failure_reason": None, + "graph_namespace": "songryeon_core_graph_v0", + "target_consumer": "R_LOOP", + "entry_candidate_count": 2, + "summary_candidate_count": 1, + "total_summary_scanned_count": 3, + "skipped_summary_count": 2, + "summary_count_by_data_kind": {"source_leaf_summary": 1}, + "summary_count_by_depth": {"1": 1}, + "packet_text": "\n".join( + [ + "R loop Vessel entry candidates: 2", + "Active summary candidates", + " source_leaf_summary(depth=1, info=relative) [graph:summary:source_leaf:code_tools]", + ] + ), + } + + rendered = render_r_loop_vessel_read_packet_text(result) + + assert "status: R_LOOP_VESSEL_READ_PACKET_OK" in rendered + assert "entry_candidate_count: 2" in rendered + assert "summary_candidate_count: 1" in rendered + assert "Active summary candidates" in rendered + + +def test_r_loop_vessel_read_packet_records_driver_exception() -> None: + result = build_r_loop_vessel_read_packet_from_neo4j( + config=_config(), + batch_id="order_175_read_failed", + created_at=READ_AT, + driver_factory=FailingDriverFactory(), + ) + + assert result.read_status == "read_failed" + assert result.failure_type == "neo4j_read_failed" + assert "simulated R packet read failure" in (result.failure_reason or "") + + +def _config( + *, + password: str | None = "test-password", + allow_no_auth: bool = False, +) -> GraphVesselNeo4jConfig: + return GraphVesselNeo4jConfig( + uri="bolt://localhost:7687", + user="neo4j", + password=password, + database="neo4j", + allow_no_auth=allow_no_auth, + ) + + +def _entry_rows() -> list[dict[str, object]]: + return [ + { + "candidate_node_id": "graph:source_ingest_time_bundle:night_changed_sources", + "display_name": "Source Ingest Bundle", + "node_kind": "source_ingest_time_bundle", + "data_kind": "source_ingest_time_bundle", + "created_at": "2026-07-02T14:02:36", + "written_at": "2026-07-02T14:02:36", + "source_trace_id": "trace:source:manifest", + "payload_json": json.dumps( + { + "node_kind": "source_ingest_time_bundle", + "data_kind": "source_ingest_time_bundle", + "summary_depth": 0, + "source_leaf_count": 552, + "source_summary_count": 0, + }, + ensure_ascii=False, + ), + "labels": ["VesselRecord", "GraphMemoryNode", "SourceIngestBundle"], + }, + { + "candidate_node_id": "graph:time_bundle:manual_vessel_first_write:core", + "display_name": "Time Bundle", + "node_kind": "time_bundle", + "data_kind": "time_bundle", + "created_at": "2026-07-01T18:39:54", + "written_at": "2026-07-01T18:39:54", + "source_trace_id": None, + "payload_json": json.dumps( + { + "node_kind": "time_bundle", + "data_kind": "time_bundle", + "summary_depth": 0, + "source_leaf_count": 1, + "source_summary_count": 0, + }, + ensure_ascii=False, + ), + "labels": ["VesselRecord", "GraphMemoryNode", "TimeBundle"], + }, + ] + + +def _summary_rows() -> list[dict[str, object]]: + return [ + { + "summary_node_id": "graph:summary:source_leaf:code_tools", + "summary_display_name": "Summary: graph:raw_source:code_tools", + "data_kind": "source_leaf_summary", + "info_class": "relative", + "generated_by": "LLM:fake:night_summarize_source_leaf", + "payload_json": json.dumps( + { + "data_kind": "source_leaf_summary", + "summary_depth": 1, + "summary_status": "ran", + "validity_status": "active", + "review_status": "not_reviewed", + "target_graph_node_id": "graph:raw_source:code_tools", + "target_node_kind": "raw_source", + "source_leaf_count": 1, + "source_summary_count": 0, + "source_graph_node_ids": ["graph:raw_source:code_tools"], + "source_data_ids": [ + "graph:raw_source:code_tools", + "source_manifest:file:code_tools.py", + ], + "source_trace_ids": ["trace:summary:active"], + "info_class": "relative", + "generated_by": "LLM:fake:night_summarize_source_leaf", + "summary_text": ( + "This source leaf summary is copied from an active LLM summary node." + ), + }, + ensure_ascii=False, + ), + "target_graph_node_id": "graph:raw_source:code_tools", + "target_display_name": "Raw Source: code_tools.py", + "target_node_kind": "raw_source", + }, + { + "summary_node_id": "graph:summary:invalidated:old", + "summary_display_name": "Summary: old source", + "data_kind": "source_leaf_summary", + "info_class": "relative", + "generated_by": "LLM:fake:night_summarize_source_leaf", + "payload_json": json.dumps( + { + "data_kind": "source_leaf_summary", + "summary_depth": 1, + "summary_status": "ran", + "validity_status": "invalidated_by_source_change", + "summary_text": "Old invalidated summary.", + }, + ensure_ascii=False, + ), + "target_graph_node_id": "graph:raw_source:old", + "target_display_name": "Raw Source: old", + "target_node_kind": "raw_source", + }, + { + "summary_node_id": "graph:summary:failed:empty", + "summary_display_name": "Summary: failed source", + "data_kind": "source_leaf_summary", + "info_class": "relative", + "generated_by": "LLM:fake:night_summarize_source_leaf", + "payload_json": json.dumps( + { + "data_kind": "source_leaf_summary", + "summary_depth": 1, + "summary_status": "failed", + "validity_status": "active", + "summary_text": "", + }, + ensure_ascii=False, + ), + "target_graph_node_id": "graph:raw_source:failed", + "target_display_name": "Raw Source: failed", + "target_node_kind": "raw_source", + }, + ] + + +class FakeRLoopVesselDriverFactory: + def __init__( + self, + *, + entry_rows: list[dict[str, object]], + summary_rows: list[dict[str, object]], + ) -> None: + self.entry_rows = list(entry_rows) + self.summary_rows = list(summary_rows) + self.last_driver: FakeRLoopVesselDriver | None = None + + def __call__(self, uri: str, *, auth: object) -> "FakeRLoopVesselDriver": + self.last_driver = FakeRLoopVesselDriver( + entry_rows=self.entry_rows, + summary_rows=self.summary_rows, + ) + return self.last_driver + + +class RaisingDriverFactory: + def __init__(self) -> None: + self.called = False + + def __call__(self, *_args, **_kwargs) -> object: + self.called = True + raise AssertionError("driver factory should not be called") + + +class FailingDriverFactory: + def __call__(self, uri: str, *, auth: object) -> "FailingDriver": + return FailingDriver() + + +class FakeRLoopVesselDriver: + def __init__( + self, + *, + entry_rows: list[dict[str, object]], + summary_rows: list[dict[str, object]], + ) -> None: + self.entry_rows = list(entry_rows) + self.summary_rows = list(summary_rows) + self.closed = False + + def session(self, *, database: str) -> "FakeRLoopVesselSession": + return FakeRLoopVesselSession(self, database=database) + + def close(self) -> None: + self.closed = True + + +class FailingDriver: + def session(self, *, database: str) -> "FailingSession": + return FailingSession() + + def close(self) -> None: + pass + + +class FakeRLoopVesselSession: + def __init__(self, driver: FakeRLoopVesselDriver, *, database: str) -> None: + self.driver = driver + self.database = database + + def __enter__(self) -> "FakeRLoopVesselSession": + return self + + def __exit__(self, _exc_type, _exc, _tb) -> None: + return None + + def execute_read(self, fn, *args): + return fn(FakeRLoopVesselTransaction(self.driver), *args) + + +class FailingSession: + def __enter__(self) -> "FailingSession": + return self + + def __exit__(self, _exc_type, _exc, _tb) -> None: + return None + + def execute_read(self, _fn, *_args): + raise RuntimeError("simulated R packet read failure") + + +class FakeRLoopVesselTransaction: + def __init__(self, driver: FakeRLoopVesselDriver) -> None: + self.driver = driver + + def run(self, query: str, **kwargs): + compact = " ".join(query.split()) + if "entry.data_id AS candidate_node_id" in compact: + return FakeResult(self.driver.entry_rows) + if "MATCH (summary:SummaryGraphNode" in compact: + return FakeResult(self.driver.summary_rows) + return FakeResult([]) + + +class FakeResult: + def __init__(self, rows: list[dict[str, object]]) -> None: + self.rows = list(rows) + + def __iter__(self): + return iter(self.rows) diff --git a/tests/test_order_176_vessel_r_one_step_traversal.py b/tests/test_order_176_vessel_r_one_step_traversal.py new file mode 100644 index 0000000..f9c7438 --- /dev/null +++ b/tests/test_order_176_vessel_r_one_step_traversal.py @@ -0,0 +1,311 @@ +from __future__ import annotations + +import json + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.r_loop_vessel_read_packet import record_r_loop_vessel_read_packet +from songryeon_core.core.trace_store import TraceStore +from songryeon_core.llm.base import LLMRequest, LLMResponse +from songryeon_core.loops.r_loop_vessel_one_step import ( + R_LOOP_VESSEL_ONE_STEP_GENERATOR, + RLoopVesselOneStepFakeLLMAdapter, + run_r_loop_vessel_one_step, +) +from songryeon_core.runtime.r_loop_vessel_one_step import ( + render_r_loop_vessel_one_step_text, +) + +from tests.test_order_175_vessel_backed_r_read_packet import ( + READ_AT, + FakeRLoopVesselDriverFactory, + _config, + _entry_rows, + _summary_rows, +) + + +def test_vessel_r_one_step_runs_llm_r1_r2_r3_over_read_packet() -> None: + trace_store, data_store, packet_event_id, packet = _record_packet() + + result = run_r_loop_vessel_one_step( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_176", + user_question="송련 Core의 코드 요약 후보를 하나 골라봐", + read_packet=packet, + adapter=RLoopVesselOneStepFakeLLMAdapter(), + frame_label="order_176_success", + input_ref=[packet_event_id], + ) + + assert result.result_frame.one_step_status == "completed" + assert result.r1_goal is not None + assert result.candidate_layer_surface is not None + assert result.surface_selection is not None + assert result.r2_selection is not None + assert result.r3_inspection is not None + assert result.continuation is not None + assert result.return_summary is not None + assert result.r1_goal.generated_by.startswith("LLM:") + assert result.r2_selection.generated_by.startswith("LLM:") + assert result.r3_inspection.generated_by.startswith("LLM:") + assert result.r2_selection.selected_graph_node_id in result.r2_selection.available_graph_node_ids + assert result.r3_inspection.inspected_graph_node_id == result.r2_selection.selected_graph_node_id + assert result.r3_inspection.sufficiency_status == "sufficient" + assert result.continuation.continuation_status == "stop_sufficient" + assert result.return_summary.r_loop_task_status == "sufficient" + assert result.result_frame.generated_by == R_LOOP_VESSEL_ONE_STEP_GENERATOR + assert result.result_frame.info_class == "absolute" + assert result.result_frame.semantic_judgement_status == "not_run" + + +def test_vessel_r_one_step_rejects_r2_selection_outside_packet() -> None: + trace_store, data_store, packet_event_id, packet = _record_packet() + + result = run_r_loop_vessel_one_step( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_176", + user_question="지도 밖 후보를 고르면 실패해야 한다", + read_packet=packet, + adapter=BadR2SelectionAdapter(), + frame_label="order_176_bad_r2", + input_ref=[packet_event_id], + ) + + assert result.result_frame.one_step_status == "failed" + assert result.result_frame.failure_stage == "R2" + assert result.result_frame.failure_type == "schema_failed" + assert "selected_node_ref" in (result.result_frame.failure_reason or "") + assert result.result_frame.r1_goal_frame_id == "R1:order_176_bad_r2:vessel_goal_frame" + assert result.result_frame.r2_selection_frame_id is None + + +def test_vessel_r_one_step_adapter_missing_does_not_select_candidate() -> None: + trace_store, data_store, packet_event_id, packet = _record_packet() + + result = run_r_loop_vessel_one_step( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_176", + user_question="adapter가 없으면 선택하지 마라", + read_packet=packet, + adapter=None, + frame_label="order_176_adapter_missing", + input_ref=[packet_event_id], + ) + + assert result.result_frame.one_step_status == "failed" + assert result.result_frame.failure_stage == "adapter" + assert result.result_frame.failure_type == "adapter_missing" + assert result.result_frame.selected_graph_node_id is None + assert result.result_frame.llm_call_data_ids == [] + + +def test_vessel_r3_child_ids_are_copied_from_packet_not_llm() -> None: + trace_store, data_store, packet_event_id, packet = _record_packet() + + result = run_r_loop_vessel_one_step( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_176", + user_question="R3가 없는 child id를 만들지 못해야 한다", + read_packet=packet, + adapter=InventingR3Adapter(), + frame_label="order_176_r3_child_guard", + input_ref=[packet_event_id], + ) + + assert result.r3_inspection is not None + assert result.r3_inspection.child_node_ids == [] + assert "graph:invented:child" not in result.r3_inspection.child_node_ids + assert result.r3_inspection.recommended_next_action == "deeper" + assert result.continuation is not None + assert result.continuation.continuation_status == "stop_budget_exhausted" + + +def test_vessel_r_one_step_records_expected_frames_in_datastore() -> None: + trace_store, data_store, packet_event_id, packet = _record_packet() + + result = run_r_loop_vessel_one_step( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_176", + user_question="기록되는 frame들을 확인해", + read_packet=packet, + adapter=RLoopVesselOneStepFakeLLMAdapter(), + frame_label="order_176_records", + input_ref=[packet_event_id], + ) + + data_types = { + data_store.require_record(data_id).data_type + for data_id in result.output_data_ids + } + assert "node_output:R1_graph_goal_frame" in data_types + assert "r_loop:vessel_candidate_layer_surface" in data_types + assert "node_output:R_loop_vessel_surface_selection_frame" in data_types + assert "node_output:R2_graph_node_selection_frame" in data_types + assert "node_output:R3_graph_inspection_frame" in data_types + assert "node_output:R_loop_continuation_frame" in data_types + assert "node_output:R_loop_return_summary_frame" in data_types + assert "r_loop:vessel_one_step_result" in data_types + + +def test_vessel_r_one_step_text_renderer_prints_llm_reasons() -> None: + rendered = render_r_loop_vessel_one_step_text( + { + "status": "R_LOOP_VESSEL_ONE_STEP_OK", + "one_step_status": "completed", + "read_packet_status": "passed", + "entry_candidate_count": 2, + "summary_candidate_count": 1, + "selected_graph_node_id": "graph:summary:source_leaf:code_tools", + "inspected_graph_node_id": "graph:summary:source_leaf:code_tools", + "sufficiency_status": "sufficient", + "continuation_status": "stop_sufficient", + "r_loop_task_status": "sufficient", + "r1_goal_frame": { + "graph_search_goal": "Inspect one active Vessel summary candidate.", + "required_information_granularity": "low_summary", + }, + "r2_selection_frame": { + "selection_reason": "Selected the active code summary.", + }, + "r3_inspection_frame": { + "inspection_reason": "The summary is sufficient for one step.", + }, + } + ) + + assert "one_step_status: completed" in rendered + assert "selected_graph_node_id: graph:summary:source_leaf:code_tools" in rendered + assert "R1 goal: Inspect one active Vessel summary candidate." in rendered + assert "R2 selection_reason: Selected the active code summary." in rendered + assert "R3 inspection_reason: The summary is sufficient for one step." in rendered + + +def _record_packet(): + trace_store = TraceStore() + data_store = DataStore() + recorded = record_r_loop_vessel_read_packet( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_176", + batch_id="order_176_packet", + config=_config(), + created_at=READ_AT, + driver_factory=FakeRLoopVesselDriverFactory( + entry_rows=_entry_rows(), + summary_rows=_summary_rows(), + ), + ) + return trace_store, data_store, recorded.trace_event_id, recorded.packet + + +class BadR2SelectionAdapter: + model_id = "bad-r2-selection-adapter" + + def complete(self, request: LLMRequest) -> LLMResponse: + if "R1 Vessel Goal Setter" in request.prompt: + payload = { + "graph_search_goal": "Test invalid R2 selection.", + "user_question_anchor_id": request.input_payload["user_question_anchor"]["anchor_id"], + "required_information_granularity": "low_summary", + "allowed_summary_depth": 1, + "stop_condition": "Stop after one selected candidate inspection.", + } + elif "R2 Vessel Node Selector" in request.prompt: + available_surfaces = request.input_payload.get("available_surface_refs") + surface_id = ( + available_surfaces[0] + if isinstance(available_surfaces, list) and available_surfaces + else None + ) + payload = { + "selection_status": "selected", + "selected_surface_ref": surface_id, + "selected_node_ref": "node_missing", + "selection_reason": "This intentionally violates the allowed ID list.", + "expected_information_granularity": "low_summary", + "expected_source_kind": "summary", + } + else: + payload = {} + return LLMResponse( + text=json.dumps(payload, ensure_ascii=False), + model_id=self.model_id, + raw=payload, + ) + + +class InventingR3Adapter: + model_id = "inventing-r3-adapter" + + def complete(self, request: LLMRequest) -> LLMResponse: + if "R1 Vessel Goal Setter" in request.prompt: + payload = { + "graph_search_goal": "Test R3 child id boundary.", + "user_question_anchor_id": request.input_payload["user_question_anchor"]["anchor_id"], + "required_information_granularity": "low_summary", + "allowed_summary_depth": 1, + "stop_condition": "Stop after one selected candidate inspection.", + } + elif "R2 Vessel Node Selector" in request.prompt: + available_surfaces = request.input_payload.get("available_surface_refs") + records_by_surface = request.input_payload.get("candidate_records_by_surface_ref") + selected_surface_id, selected_id = _first_summary_surface_and_node( + available_surfaces, + records_by_surface, + ) + payload = { + "selection_status": "selected", + "selected_surface_ref": selected_surface_id, + "selected_node_ref": selected_id, + "selection_reason": "Select the first supplied candidate.", + "expected_information_granularity": "low_summary", + "expected_source_kind": "summary", + } + elif "R3 Vessel Inspector" in request.prompt: + payload = { + "current_information_granularity": "low_summary", + "sufficiency_status": "insufficient", + "granularity_problem_status": "needs_lower_granularity", + "branch_problem_status": "none", + "recommended_next_action": "deeper", + "inspection_reason": "Pretend to want a fabricated child, but schema has no child id field.", + "child_node_ids": ["graph:invented:child"], + } + else: + payload = {} + return LLMResponse( + text=json.dumps(payload, ensure_ascii=False), + model_id=self.model_id, + raw=payload, + ) + + +def _first_summary_surface_and_node( + available_surfaces: object, + records_by_surface: object, +) -> tuple[str | None, str | None]: + if not isinstance(available_surfaces, list) or not isinstance(records_by_surface, dict): + return None, None + fallback: tuple[str | None, str | None] = (None, None) + for surface_id in available_surfaces: + if not isinstance(surface_id, str): + continue + records = records_by_surface.get(surface_id) + if not isinstance(records, list): + continue + for record in records: + if not isinstance(record, dict): + continue + node_ref = record.get("node_ref") + if not isinstance(node_ref, str): + continue + if fallback == (None, None): + fallback = (surface_id, node_ref) + if record.get("summary_text"): + return surface_id, node_ref + return fallback diff --git a/tests/test_order_177_r1_candidate_text_blindness.py b/tests/test_order_177_r1_candidate_text_blindness.py new file mode 100644 index 0000000..64d3b3c --- /dev/null +++ b/tests/test_order_177_r1_candidate_text_blindness.py @@ -0,0 +1,175 @@ +from __future__ import annotations + +import json + +from songryeon_core.llm.base import LLMRequest, LLMResponse +from songryeon_core.loops.r_loop_vessel_one_step import run_r_loop_vessel_one_step + +from tests.test_order_176_vessel_r_one_step_traversal import _record_packet + + +def test_r1_input_does_not_receive_candidate_text_or_samples() -> None: + trace_store, data_store, packet_event_id, packet = _record_packet() + adapter = CapturingRLoopAdapter() + + result = run_r_loop_vessel_one_step( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_177", + user_question="source summary와 token layer summary 연결을 한 단계만 봐줘", + read_packet=packet, + adapter=adapter, + frame_label="order_177_blind_r1", + input_ref=[packet_event_id], + ) + + assert result.result_frame.one_step_status == "completed" + r1_payload = adapter.payloads_by_node["R1"] + assert "entry_candidate_samples" not in r1_payload + assert "summary_candidate_samples" not in r1_payload + assert "summary_text" not in json.dumps(r1_payload, ensure_ascii=False) + assert "summary_text_preview" not in json.dumps(r1_payload, ensure_ascii=False) + assert "candidate_text_visibility_policy" in r1_payload + + +def test_r2_receives_core_entry_material_after_r1_blindness() -> None: + trace_store, data_store, packet_event_id, packet = _record_packet() + adapter = CapturingRLoopAdapter() + + result = run_r_loop_vessel_one_step( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_177", + user_question="source summary와 token layer summary 연결을 한 단계만 봐줘", + read_packet=packet, + adapter=adapter, + frame_label="order_177_r2_r3_material", + input_ref=[packet_event_id], + ) + + assert result.result_frame.one_step_status == "completed" + r2_payload = adapter.payloads_by_node["R2"] + r3_payload = adapter.payloads_by_node["R3"] + assert "summary_candidate_records" not in r2_payload + assert "candidate_layer_surface_ref_records" in r2_payload + assert "candidate_records_by_surface_ref" in r2_payload + assert "summary_text" not in json.dumps(r2_payload, ensure_ascii=False) + selected = r3_payload["selected_candidate_record"] + assert isinstance(selected, dict) + assert selected["candidate_node_id"] == "graph:source_ingest_time_bundle:night_changed_sources" + assert "summary_text" not in selected + + +def test_r1_goal_must_preserve_explicit_ascii_question_anchor() -> None: + trace_store, data_store, packet_event_id, packet = _record_packet() + + result = run_r_loop_vessel_one_step( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_177", + user_question="source summary token layer connection", + read_packet=packet, + adapter=DriftingR1Adapter(), + frame_label="order_177_r1_anchor_fail", + input_ref=[packet_event_id], + ) + + assert result.result_frame.one_step_status == "failed" + assert result.result_frame.failure_stage == "R1" + assert result.result_frame.failure_type == "schema_failed" + assert "user_question_anchor_id" in (result.result_frame.failure_reason or "") + + +class CapturingRLoopAdapter: + model_id = "capturing-r-loop-adapter" + + def __init__(self) -> None: + self.payloads_by_node: dict[str, dict[str, object]] = {} + + def complete(self, request: LLMRequest) -> LLMResponse: + if "R1 Vessel Goal Setter" in request.prompt: + self.payloads_by_node["R1"] = request.input_payload + payload = { + "graph_search_goal": "Inspect source summary and token layer summary connection.", + "user_question_anchor_id": request.input_payload["user_question_anchor"]["anchor_id"], + "required_information_granularity": "low_summary", + "allowed_summary_depth": 1, + "stop_condition": "Stop after one selected candidate inspection.", + } + elif "R2 Vessel Node Selector" in request.prompt: + self.payloads_by_node["R2"] = request.input_payload + available_surfaces = request.input_payload.get("available_surface_refs") + records_by_surface = request.input_payload.get("candidate_records_by_surface_ref") + selected_surface_id, selected_id = _first_summary_surface_and_node( + available_surfaces, + records_by_surface, + ) + payload = { + "selection_status": "selected", + "selected_surface_ref": selected_surface_id, + "selected_node_ref": selected_id, + "selection_reason": "Select the first supplied candidate for visibility boundary test.", + "expected_information_granularity": "low_summary", + "expected_source_kind": "summary", + } + elif "R3 Vessel Inspector" in request.prompt: + self.payloads_by_node["R3"] = request.input_payload + payload = { + "current_information_granularity": "low_summary", + "sufficiency_status": "sufficient", + "granularity_problem_status": "none", + "branch_problem_status": "none", + "recommended_next_action": "stop", + "inspection_reason": "The selected candidate material is sufficient for this one-step test.", + } + else: + payload = {} + return LLMResponse( + text=json.dumps(payload, ensure_ascii=False), + model_id=self.model_id, + raw=payload, + ) + + +class DriftingR1Adapter: + model_id = "drifting-r1-adapter" + + def complete(self, request: LLMRequest) -> LLMResponse: + payload = { + "graph_search_goal": "Inspect ORDER 082 W1 triage output.", + "user_question_anchor_id": "r1_user_question_anchor:wrong", + "required_information_granularity": "low_summary", + "allowed_summary_depth": 1, + "stop_condition": "Stop after one selected candidate inspection.", + } + return LLMResponse( + text=json.dumps(payload, ensure_ascii=False), + model_id=self.model_id, + raw=payload, + ) + + +def _first_summary_surface_and_node( + available_surfaces: object, + records_by_surface: object, +) -> tuple[str | None, str | None]: + if not isinstance(available_surfaces, list) or not isinstance(records_by_surface, dict): + return None, None + fallback: tuple[str | None, str | None] = (None, None) + for surface_id in available_surfaces: + if not isinstance(surface_id, str): + continue + records = records_by_surface.get(surface_id) + if not isinstance(records, list): + continue + for record in records: + if not isinstance(record, dict): + continue + node_ref = record.get("node_ref") + if not isinstance(node_ref, str): + continue + if fallback == (None, None): + fallback = (surface_id, node_ref) + if record.get("summary_text"): + return surface_id, node_ref + return fallback diff --git a/tests/test_order_178_r_vessel_candidate_layer_surface.py b/tests/test_order_178_r_vessel_candidate_layer_surface.py new file mode 100644 index 0000000..96741d7 --- /dev/null +++ b/tests/test_order_178_r_vessel_candidate_layer_surface.py @@ -0,0 +1,312 @@ +import json + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.r_loop_vessel_read_packet import ( + build_r_loop_vessel_read_packet_from_neo4j, + record_r_loop_vessel_read_packet, +) +from songryeon_core.core.trace_store import TraceStore +from songryeon_core.llm.base import LLMRequest, LLMResponse +from songryeon_core.loops.r_loop_vessel_one_step import ( + R_LOOP_VESSEL_CANDIDATE_LAYER_SURFACE_DATA_TYPE, + R_LOOP_VESSEL_CANDIDATE_LAYER_SURFACE_GENERATOR, + R_LOOP_VESSEL_SURFACE_SELECTION_DATA_TYPE, + run_r_loop_vessel_one_step, +) + +from tests.test_order_175_vessel_backed_r_read_packet import ( + READ_AT, + FakeRLoopVesselDriverFactory, + _config, + _entry_rows, +) + + +def test_read_packet_balances_summary_candidate_kinds_before_limit() -> None: + result = build_r_loop_vessel_read_packet_from_neo4j( + config=_config(), + batch_id="order_178_balanced_packet", + created_at=READ_AT, + limit=3, + driver_factory=FakeRLoopVesselDriverFactory( + entry_rows=[], + summary_rows=[ + *[ + _active_summary_row( + suffix=f"source_{index:02d}", + data_kind="source_leaf_summary", + summary_depth=1, + ) + for index in range(10) + ], + _active_summary_row( + suffix="token_01", + data_kind="token_budget_bundle_summary", + summary_depth=2, + ), + ], + ), + ) + + data_kinds = { + record["data_kind"] + for record in result.summary_candidate_records + } + assert result.summary_candidate_count == 3 + assert "source_leaf_summary" in data_kinds + assert "token_budget_bundle_summary" in data_kinds + + +def test_candidate_layer_surface_is_absolute_and_text_free() -> None: + trace_store, data_store, packet_event_id, packet = _record_multi_surface_packet() + adapter = CapturingSurfaceAdapter() + + result = run_r_loop_vessel_one_step( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_178", + user_question="source summary와 token layer summary 연결을 한 단계만 봐줘", + read_packet=packet, + adapter=adapter, + frame_label="order_178_surface", + input_ref=[packet_event_id], + ) + + assert result.result_frame.one_step_status == "completed" + assert result.candidate_layer_surface is not None + assert result.candidate_layer_surface.generated_by == ( + R_LOOP_VESSEL_CANDIDATE_LAYER_SURFACE_GENERATOR + ) + assert result.candidate_layer_surface.info_class == "absolute" + assert result.candidate_layer_surface.semantic_judgement_status == "not_run" + assert result.candidate_layer_surface.surface_count >= 2 + assert "summary_text" not in json.dumps( + result.candidate_layer_surface.surface_records, + ensure_ascii=False, + ) + surface_record = data_store.require_record(result.candidate_layer_surface.frame_id) + assert surface_record.data_type == R_LOOP_VESSEL_CANDIDATE_LAYER_SURFACE_DATA_TYPE + + +def test_r2_payload_uses_core_entry_surfaces_not_flat_summaries() -> None: + trace_store, data_store, packet_event_id, packet = _record_multi_surface_packet() + adapter = CapturingSurfaceAdapter() + + result = run_r_loop_vessel_one_step( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_178", + user_question="token layer summary 후보를 먼저 찾아봐", + read_packet=packet, + adapter=adapter, + frame_label="order_178_r2_payload", + input_ref=[packet_event_id], + ) + + assert result.result_frame.one_step_status == "completed" + assert result.surface_selection is not None + r2_payload = adapter.payloads_by_node["R2"] + assert "summary_candidate_records" not in r2_payload + assert "entry_candidate_records" not in r2_payload + assert "candidate_layer_surface_ref_records" in r2_payload + assert "available_surface_refs" in r2_payload + assert "candidate_records_by_surface_ref" in r2_payload + grouped_json = json.dumps( + r2_payload["candidate_records_by_surface_ref"], + ensure_ascii=False, + ) + assert "summary_text" not in grouped_json + assert "source_leaf_summary" not in grouped_json + assert "token_budget_bundle_summary" not in grouped_json + assert "source_ingest_bundle" in grouped_json or "time_bundle" in grouped_json + surface_record = data_store.require_record(result.surface_selection.frame_id) + assert surface_record.data_type == R_LOOP_VESSEL_SURFACE_SELECTION_DATA_TYPE + + +def test_r2_selection_must_stay_inside_selected_surface() -> None: + trace_store, data_store, packet_event_id, packet = _record_multi_surface_packet() + + result = run_r_loop_vessel_one_step( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_178", + user_question="선택 범위 밖 후보를 고르면 실패해야 한다", + read_packet=packet, + adapter=CrossSurfaceR2Adapter(), + frame_label="order_178_cross_surface_fail", + input_ref=[packet_event_id], + ) + + assert result.result_frame.one_step_status == "failed" + assert result.result_frame.failure_stage == "R2" + assert result.result_frame.failure_type == "schema_failed" + assert "selected_graph_node_id" in (result.result_frame.failure_reason or "") + + +def _record_multi_surface_packet(): + trace_store = TraceStore() + data_store = DataStore() + recorded = record_r_loop_vessel_read_packet( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_178", + batch_id="order_178_packet", + config=_config(), + created_at=READ_AT, + driver_factory=FakeRLoopVesselDriverFactory( + entry_rows=_entry_rows(), + summary_rows=[ + _active_summary_row( + suffix="source_leaf", + data_kind="source_leaf_summary", + summary_depth=1, + ), + _active_summary_row( + suffix="token_bundle", + data_kind="token_budget_bundle_summary", + summary_depth=2, + ), + ], + ), + ) + return trace_store, data_store, recorded.trace_event_id, recorded.packet + + +def _active_summary_row( + *, + suffix: str, + data_kind: str, + summary_depth: int, +) -> dict[str, object]: + summary_node_id = f"graph:summary:{suffix}" + target_graph_node_id = f"graph:target:{suffix}" + return { + "summary_node_id": summary_node_id, + "summary_display_name": f"Summary: {suffix}", + "data_kind": data_kind, + "info_class": "mixed" if summary_depth > 1 else "relative", + "generated_by": "LLM:fake:night_summary", + "payload_json": json.dumps( + { + "data_kind": data_kind, + "summary_depth": summary_depth, + "summary_status": "ran", + "validity_status": "active", + "review_status": "not_reviewed", + "target_graph_node_id": target_graph_node_id, + "target_node_kind": "summary_target", + "source_leaf_count": 1 if summary_depth == 1 else 4, + "source_summary_count": 0 if summary_depth == 1 else 4, + "source_graph_node_ids": [target_graph_node_id], + "source_data_ids": [target_graph_node_id], + "source_trace_ids": [f"trace:summary:{suffix}"], + "info_class": "mixed" if summary_depth > 1 else "relative", + "generated_by": "LLM:fake:night_summary", + "summary_text": f"{data_kind} active summary text for {suffix}.", + }, + ensure_ascii=False, + ), + "target_graph_node_id": target_graph_node_id, + "target_display_name": f"Target: {suffix}", + "target_node_kind": "summary_target", + } + + +class CapturingSurfaceAdapter: + model_id = "capturing-surface-adapter" + + def __init__(self) -> None: + self.payloads_by_node: dict[str, dict[str, object]] = {} + + def complete(self, request: LLMRequest) -> LLMResponse: + if "R1 Vessel Goal Setter" in request.prompt: + self.payloads_by_node["R1"] = request.input_payload + payload = { + "graph_search_goal": "Inspect source summary and token layer summary connection.", + "user_question_anchor_id": request.input_payload["user_question_anchor"]["anchor_id"], + "required_information_granularity": "low_summary", + "allowed_summary_depth": 2, + "stop_condition": "Stop after one selected candidate inspection.", + } + elif "R2 Vessel Node Selector" in request.prompt: + self.payloads_by_node["R2"] = request.input_payload + surface_id, graph_node_id = _first_surface_and_node(request.input_payload) + payload = { + "selection_status": "selected", + "selected_surface_ref": surface_id, + "selected_node_ref": graph_node_id, + "selection_reason": "Select the first node inside the first supplied surface.", + "expected_information_granularity": "low_summary", + "expected_source_kind": "summary", + } + elif "R3 Vessel Inspector" in request.prompt: + self.payloads_by_node["R3"] = request.input_payload + payload = { + "current_information_granularity": "low_summary", + "sufficiency_status": "sufficient", + "granularity_problem_status": "none", + "branch_problem_status": "none", + "recommended_next_action": "stop", + "inspection_reason": "The selected candidate is sufficient for the surface test.", + } + else: + payload = {} + return LLMResponse( + text=json.dumps(payload, ensure_ascii=False), + model_id=self.model_id, + raw=payload, + ) + + +class CrossSurfaceR2Adapter(CapturingSurfaceAdapter): + model_id = "cross-surface-r2-adapter" + + def complete(self, request: LLMRequest) -> LLMResponse: + if "R2 Vessel Node Selector" not in request.prompt: + return super().complete(request) + + self.payloads_by_node["R2"] = request.input_payload + available_surfaces = request.input_payload.get("available_surface_refs") + records_by_surface = request.input_payload.get("candidate_records_by_surface_ref") + selected_surface_id = None + cross_surface_node_id = None + if isinstance(available_surfaces, list) and len(available_surfaces) >= 2: + selected_surface_id = available_surfaces[0] + if isinstance(records_by_surface, dict): + other_records = records_by_surface.get(available_surfaces[1]) + if isinstance(other_records, list) and other_records: + first = other_records[0] + if isinstance(first, dict): + cross_surface_node_id = first.get("node_ref") + payload = { + "selection_status": "selected", + "selected_surface_ref": selected_surface_id, + "selected_node_ref": cross_surface_node_id, + "selection_reason": "Intentionally select a node from a different surface.", + "expected_information_granularity": "low_summary", + "expected_source_kind": "summary", + } + return LLMResponse( + text=json.dumps(payload, ensure_ascii=False), + model_id=self.model_id, + raw=payload, + ) + + +def _first_surface_and_node(payload: dict[str, object]) -> tuple[str | None, str | None]: + available_surfaces = payload.get("available_surface_refs") + selected_surface_id = ( + available_surfaces[0] + if isinstance(available_surfaces, list) and available_surfaces + else None + ) + records_by_surface = payload.get("candidate_records_by_surface_ref") + if isinstance(records_by_surface, dict) and selected_surface_id: + records = records_by_surface.get(selected_surface_id) + if isinstance(records, list) and records: + first = records[0] + if isinstance(first, dict): + node_ref = first.get("node_ref") + if isinstance(node_ref, str): + return selected_surface_id, node_ref + return selected_surface_id, None diff --git a/tests/test_order_179_r2_vessel_selection_id_disambiguation.py b/tests/test_order_179_r2_vessel_selection_id_disambiguation.py new file mode 100644 index 0000000..8a75b11 --- /dev/null +++ b/tests/test_order_179_r2_vessel_selection_id_disambiguation.py @@ -0,0 +1,99 @@ +import json + +from songryeon_core.llm.base import LLMRequest, LLMResponse +from songryeon_core.loops.r_loop_vessel_one_step import run_r_loop_vessel_one_step + +from tests.test_order_178_r_vessel_candidate_layer_surface import ( + CapturingSurfaceAdapter, + _record_multi_surface_packet, +) + + +def test_r2_candidate_payload_hides_non_selectable_graph_ids() -> None: + trace_store, data_store, packet_event_id, packet = _record_multi_surface_packet() + adapter = CapturingSurfaceAdapter() + + result = run_r_loop_vessel_one_step( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_179", + user_question="token layer summary 후보를 한 단계만 골라봐", + read_packet=packet, + adapter=adapter, + frame_label="order_179_payload_shape", + input_ref=[packet_event_id], + ) + + assert result.result_frame.one_step_status == "completed" + r2_payload = adapter.payloads_by_node["R2"] + grouped_json = json.dumps( + r2_payload["candidate_records_by_surface_ref"], + ensure_ascii=False, + ) + assert "node_ref" in grouped_json + assert "summary_text" not in grouped_json + assert "graph_node_id" not in grouped_json + assert "target_graph_node_id" not in grouped_json + assert "source_graph_node_ids" not in grouped_json + assert "summary_node_id" not in grouped_json + assert "candidate_node_id" not in grouped_json + + +def test_r2_invalid_target_style_id_reports_failure_payload_summary() -> None: + trace_store, data_store, packet_event_id, packet = _record_multi_surface_packet() + + result = run_r_loop_vessel_one_step( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_179", + user_question="공식 node_ref가 아닌 값을 고르면 실패해야 한다", + read_packet=packet, + adapter=TargetStyleIdR2Adapter(), + frame_label="order_179_target_id_fail", + input_ref=[packet_event_id], + ) + + assert result.result_frame.one_step_status == "failed" + assert result.result_frame.failure_stage == "R2" + assert result.result_frame.failure_type == "schema_failed" + summary = result.result_frame.failure_payload_summary + assert summary is not None + assert summary["selected_surface_ref_in_available"] is True + assert summary["selected_node_ref"] == "node_missing" + assert summary["selected_node_ref_in_available"] is False + + +class TargetStyleIdR2Adapter: + model_id = "target-style-id-r2-adapter" + + def complete(self, request: LLMRequest) -> LLMResponse: + if "R1 Vessel Goal Setter" in request.prompt: + payload = { + "graph_search_goal": "Inspect node_ref selection boundary.", + "user_question_anchor_id": request.input_payload["user_question_anchor"]["anchor_id"], + "required_information_granularity": "low_summary", + "allowed_summary_depth": 2, + "stop_condition": "Stop after one selected candidate inspection.", + } + elif "R2 Vessel Node Selector" in request.prompt: + available_surfaces = request.input_payload.get("available_surface_refs") + selected_surface_id = ( + available_surfaces[0] + if isinstance(available_surfaces, list) and available_surfaces + else None + ) + payload = { + "selection_status": "selected", + "selected_surface_ref": selected_surface_id, + "selected_node_ref": "node_missing", + "selection_reason": "Intentionally choose a made-up node ref.", + "expected_information_granularity": "low_summary", + "expected_source_kind": "summary", + } + else: + payload = {} + return LLMResponse( + text=json.dumps(payload, ensure_ascii=False), + model_id=self.model_id, + raw=payload, + ) diff --git a/tests/test_order_180_r2_prompt_example_id_removal.py b/tests/test_order_180_r2_prompt_example_id_removal.py new file mode 100644 index 0000000..14eb73f --- /dev/null +++ b/tests/test_order_180_r2_prompt_example_id_removal.py @@ -0,0 +1,14 @@ +from pathlib import Path + + +PROMPT_PATH = Path("songryeon_core/prompts/r2_vessel_node_selector_v0.md") + + +def test_r2_prompt_does_not_contain_copyable_fake_graph_id_examples() -> None: + prompt = PROMPT_PATH.read_text(encoding="utf-8") + + assert "graph:summary:source_leaf:example" not in prompt + assert "surface:summary:data:source_leaf_summary:depth:1:info:relative" not in prompt + assert "```json" not in prompt + assert "runtime candidate record `node_ref`" in prompt + assert "The only selectable IDs are in the runtime input payload." in prompt diff --git a/tests/test_order_181_r2_official_selection_ref_map.py b/tests/test_order_181_r2_official_selection_ref_map.py new file mode 100644 index 0000000..0e8cba9 --- /dev/null +++ b/tests/test_order_181_r2_official_selection_ref_map.py @@ -0,0 +1,146 @@ +import json + +from songryeon_core.llm.base import LLMRequest, LLMResponse +from songryeon_core.loops.r_loop_vessel_one_step import run_r_loop_vessel_one_step + +from tests.test_order_178_r_vessel_candidate_layer_surface import ( + _record_multi_surface_packet, +) + + +def test_r2_uses_official_refs_and_code_maps_to_actual_graph_ids() -> None: + trace_store, data_store, packet_event_id, packet = _record_multi_surface_packet() + adapter = OfficialRefAdapter() + + result = run_r_loop_vessel_one_step( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_181", + user_question="공식 ref로 후보를 골라봐", + read_packet=packet, + adapter=adapter, + frame_label="order_181_ref_success", + input_ref=[packet_event_id], + ) + + assert result.result_frame.one_step_status == "completed" + assert result.r2_selection is not None + assert result.r2_selection.selected_graph_node_id == ( + "graph:source_ingest_time_bundle:night_changed_sources" + ) + r2_payload = adapter.payloads_by_node["R2"] + assert r2_payload["available_surface_refs"][0] == "surface_001" + grouped_json = json.dumps( + r2_payload["candidate_records_by_surface_ref"], + ensure_ascii=False, + ) + assert "node_001" in grouped_json + assert "graph:summary:source_leaf" not in grouped_json + assert "graph:source_ingest_time_bundle:night_changed_sources" not in grouped_json + + +def test_r2_invented_short_refs_still_fail_with_diagnostics() -> None: + trace_store, data_store, packet_event_id, packet = _record_multi_surface_packet() + + result = run_r_loop_vessel_one_step( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_181", + user_question="없는 ref를 만들면 실패해야 한다", + read_packet=packet, + adapter=InventedRefAdapter(), + frame_label="order_181_invented_ref_fail", + input_ref=[packet_event_id], + ) + + assert result.result_frame.one_step_status == "failed" + assert result.result_frame.failure_stage == "R2" + assert result.result_frame.failure_type == "schema_failed" + summary = result.result_frame.failure_payload_summary + assert summary is not None + assert summary["selected_surface_ref"] == "surface_999" + assert summary["selected_node_ref"] == "node_999" + assert summary["selected_surface_ref_in_available"] is False + assert summary["selected_node_ref_in_available"] is False + + +class OfficialRefAdapter: + model_id = "official-ref-adapter" + + def __init__(self) -> None: + self.payloads_by_node: dict[str, dict[str, object]] = {} + + def complete(self, request: LLMRequest) -> LLMResponse: + if "R1 Vessel Goal Setter" in request.prompt: + self.payloads_by_node["R1"] = request.input_payload + payload = { + "graph_search_goal": "Inspect official selection refs.", + "user_question_anchor_id": request.input_payload["user_question_anchor"]["anchor_id"], + "required_information_granularity": "low_summary", + "allowed_summary_depth": 2, + "stop_condition": "Stop after one selected candidate inspection.", + } + elif "R2 Vessel Node Selector" in request.prompt: + self.payloads_by_node["R2"] = request.input_payload + surface_ref, candidate = _first_summary_candidate(request.input_payload) + payload = { + "selection_status": "selected", + "selected_surface_ref": surface_ref, + "selected_node_ref": candidate["node_ref"], + "selection_reason": "Select the first official node ref in the first official surface.", + "expected_information_granularity": "low_summary", + "expected_source_kind": "summary", + } + elif "R3 Vessel Inspector" in request.prompt: + self.payloads_by_node["R3"] = request.input_payload + payload = { + "current_information_granularity": "low_summary", + "sufficiency_status": "sufficient", + "granularity_problem_status": "none", + "branch_problem_status": "none", + "recommended_next_action": "stop", + "inspection_reason": "Official ref was mapped to an actual graph node.", + } + else: + payload = {} + return LLMResponse( + text=json.dumps(payload, ensure_ascii=False), + model_id=self.model_id, + raw=payload, + ) + + +def _first_summary_candidate( + payload: dict[str, object], +) -> tuple[str, dict[str, object]]: + available_surface_refs = payload["available_surface_refs"] + records_by_surface = payload["candidate_records_by_surface_ref"] + for surface_ref in available_surface_refs: + records = records_by_surface[surface_ref] + for record in records: + if record.get("summary_text"): + return surface_ref, record + surface_ref = available_surface_refs[0] + return surface_ref, records_by_surface[surface_ref][0] + + +class InventedRefAdapter(OfficialRefAdapter): + model_id = "invented-ref-adapter" + + def complete(self, request: LLMRequest) -> LLMResponse: + if "R2 Vessel Node Selector" not in request.prompt: + return super().complete(request) + self.payloads_by_node["R2"] = request.input_payload + payload = { + "selection_status": "selected", + "selected_surface_ref": "surface_999", + "selected_node_ref": "node_999", + "selection_reason": "Intentionally invent refs.", + "expected_information_granularity": "low_summary", + "expected_source_kind": "summary", + } + return LLMResponse( + text=json.dumps(payload, ensure_ascii=False), + model_id=self.model_id, + raw=payload, + ) diff --git a/tests/test_order_182_r_core_ego_start_surface.py b/tests/test_order_182_r_core_ego_start_surface.py new file mode 100644 index 0000000..009648f --- /dev/null +++ b/tests/test_order_182_r_core_ego_start_surface.py @@ -0,0 +1,186 @@ +import json + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.r_loop_vessel_read_packet import record_r_loop_vessel_read_packet +from songryeon_core.core.trace_store import TraceStore +from songryeon_core.llm.base import LLMRequest, LLMResponse +from songryeon_core.loops.r_loop_vessel_one_step import run_r_loop_vessel_one_step + +from tests.test_order_175_vessel_backed_r_read_packet import ( + READ_AT, + FakeRLoopVesselDriverFactory, + _config, + _entry_rows, + _summary_rows, +) + + +def test_r2_first_view_hides_summary_candidates_even_when_packet_has_them() -> None: + trace_store, data_store, packet_event_id, packet = _record_packet(_entry_rows()) + adapter = CapturingCoreStartAdapter() + + result = run_r_loop_vessel_one_step( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_182", + user_question="CoreEgo에서 한 단계만 내려가 봐", + read_packet=packet, + adapter=adapter, + frame_label="order_182_core_start", + input_ref=[packet_event_id], + ) + + assert packet.summary_candidate_count == 1 + assert result.result_frame.one_step_status == "completed" + assert result.candidate_layer_surface is not None + assert all( + record["surface_kind"] == "entry_candidate_kind" + for record in result.candidate_layer_surface.surface_records + ) + + r2_payload = adapter.payloads_by_node["R2"] + grouped_json = json.dumps( + r2_payload["candidate_records_by_surface_ref"], + ensure_ascii=False, + ) + assert "summary_text" not in grouped_json + assert "summary_node_id" not in grouped_json + assert "source_leaf_summary" not in grouped_json + assert r2_payload["selection_ref_contract"]["current_graph_node_id"] == "graph:core_ego:root" + assert ( + r2_payload["selection_ref_contract"]["first_step_policy"] + == "core_ego_direct_entry_candidates_only" + ) + + +def test_time_axis_entry_is_prioritized_as_core_ego_direct_child() -> None: + entry_rows = [_time_axis_row(), *_entry_rows()] + trace_store, data_store, packet_event_id, packet = _record_packet(entry_rows) + adapter = CapturingCoreStartAdapter() + + result = run_r_loop_vessel_one_step( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_182_axis", + user_question="CoreEgo 바로 아래 시간축을 먼저 봐", + read_packet=packet, + adapter=adapter, + frame_label="order_182_time_axis", + input_ref=[packet_event_id], + ) + + assert result.result_frame.one_step_status == "completed" + assert result.r2_selection is not None + assert result.r2_selection.selected_graph_node_id == "graph:axis:time" + assert result.candidate_layer_surface is not None + assert result.candidate_layer_surface.surface_count == 1 + surface = result.candidate_layer_surface.surface_records[0] + assert surface["candidate_kind"] == "time_axis" + assert surface["candidate_graph_node_ids"] == ["graph:axis:time"] + + +def _record_packet(entry_rows: list[dict[str, object]]): + trace_store = TraceStore() + data_store = DataStore() + recorded = record_r_loop_vessel_read_packet( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_182_packet", + batch_id="order_182_packet", + config=_config(), + created_at=READ_AT, + driver_factory=FakeRLoopVesselDriverFactory( + entry_rows=entry_rows, + summary_rows=_summary_rows(), + ), + ) + return trace_store, data_store, recorded.trace_event_id, recorded.packet + + +def _time_axis_row() -> dict[str, object]: + return { + "candidate_node_id": "graph:axis:time", + "display_name": "Time Axis", + "node_kind": "time_axis", + "data_kind": "time_axis", + "created_at": "2026-07-02T00:00:00", + "written_at": "2026-07-02T00:00:00", + "source_trace_id": None, + "payload_json": json.dumps( + { + "node_kind": "time_axis", + "data_kind": "time_axis", + "summary_depth": 0, + "source_leaf_count": 0, + "source_summary_count": 0, + }, + ensure_ascii=False, + ), + "labels": ["GraphMemoryNode", "TimeAxis"], + } + + +class CapturingCoreStartAdapter: + model_id = "capturing-core-start-adapter" + + def __init__(self) -> None: + self.payloads_by_node: dict[str, dict[str, object]] = {} + + def complete(self, request: LLMRequest) -> LLMResponse: + if "R1 Vessel Goal Setter" in request.prompt: + self.payloads_by_node["R1"] = request.input_payload + payload = { + "graph_search_goal": "Inspect the direct CoreEgo entry layer.", + "user_question_anchor_id": request.input_payload["user_question_anchor"]["anchor_id"], + "required_information_granularity": "raw", + "allowed_summary_depth": 0, + "stop_condition": "Stop after one entry candidate inspection.", + } + elif "R2 Vessel Node Selector" in request.prompt: + self.payloads_by_node["R2"] = request.input_payload + surface_ref, node_ref = _first_visible_node_ref(request.input_payload) + payload = { + "selection_status": "selected", + "selected_surface_ref": surface_ref, + "selected_node_ref": node_ref, + "selection_reason": "Select the first direct CoreEgo entry candidate.", + "expected_information_granularity": "raw", + "expected_source_kind": "entry_candidate", + } + elif "R3 Vessel Inspector" in request.prompt: + self.payloads_by_node["R3"] = request.input_payload + payload = { + "current_information_granularity": "raw", + "sufficiency_status": "sufficient", + "granularity_problem_status": "none", + "branch_problem_status": "none", + "recommended_next_action": "stop", + "inspection_reason": "The selected CoreEgo entry candidate is visible.", + } + else: + payload = {} + return LLMResponse( + text=json.dumps(payload, ensure_ascii=False), + model_id=self.model_id, + raw=payload, + ) + + +def _first_visible_node_ref(payload: dict[str, object]) -> tuple[str | None, str | None]: + surface_refs = payload.get("available_surface_refs") + records_by_surface = payload.get("candidate_records_by_surface_ref") + if not isinstance(surface_refs, list) or not isinstance(records_by_surface, dict): + return None, None + for surface_ref in surface_refs: + if not isinstance(surface_ref, str): + continue + records = records_by_surface.get(surface_ref) + if not isinstance(records, list): + continue + for record in records: + if not isinstance(record, dict): + continue + node_ref = record.get("node_ref") + if isinstance(node_ref, str) and node_ref: + return surface_ref, node_ref + return None, None diff --git a/tests/test_order_183_r_vessel_hierarchical_child_surface.py b/tests/test_order_183_r_vessel_hierarchical_child_surface.py new file mode 100644 index 0000000..594eb69 --- /dev/null +++ b/tests/test_order_183_r_vessel_hierarchical_child_surface.py @@ -0,0 +1,375 @@ +import json + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.r_loop_vessel_read_packet import record_r_loop_vessel_read_packet +from songryeon_core.core.trace_store import TraceStore +from songryeon_core.llm.base import LLMRequest, LLMResponse +from songryeon_core.loops.r_loop_vessel_one_step import run_r_loop_vessel_one_step + +from tests.test_order_175_vessel_backed_r_read_packet import ( + READ_AT, + FakeRLoopVesselDriverFactory, + _config, +) + + +def test_time_axis_selection_records_child_candidate_surface() -> None: + trace_store, data_store, packet_event_id, packet = _record_packet( + entry_rows=[_time_axis_row(), _source_ingest_row(), _time_bundle_row()], + summary_rows=[], + ) + adapter = HierarchyAdapter() + + result = run_r_loop_vessel_one_step( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_183_axis", + user_question="CoreEgo에서 시간축 아래 후보를 봐", + read_packet=packet, + adapter=adapter, + frame_label="order_183_axis", + input_ref=[packet_event_id], + ) + + assert result.result_frame.one_step_status == "completed" + assert result.r2_selection is not None + assert result.r2_selection.selected_graph_node_id == "graph:axis:time" + assert result.r3_inspection is not None + assert set(result.r3_inspection.child_node_ids) == { + "graph:source_ingest_time_bundle:night_changed_sources", + "graph:time_bundle:manual_vessel_first_write:core", + } + assert result.graph_traversal_candidate_surface is not None + assert set(result.graph_traversal_candidate_surface.child_candidate_node_ids) == set( + result.r3_inspection.child_node_ids + ) + record = data_store.require_record(result.graph_traversal_candidate_surface.frame_id) + assert record.data_type == "node_output:R_graph_traversal_candidate_surface_frame" + r3_payload = adapter.payloads_by_node["R3"] + assert r3_payload["hierarchy_child_candidate_count"] == 2 + + +def test_source_ingest_selection_exposes_source_kind_children() -> None: + trace_store, data_store, packet_event_id, packet = _record_packet( + entry_rows=[ + _source_ingest_row( + source_graph_node_ids=[ + "graph:source_kind_bundle:batch:code", + "graph:source_kind_bundle:batch:document", + ], + ), + _source_kind_row("code"), + _source_kind_row("document"), + ], + summary_rows=[], + ) + adapter = HierarchyAdapter(preferred_kind="source_ingest_bundle") + + result = run_r_loop_vessel_one_step( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_183_source_ingest", + user_question="source ingest 아래 자료 종류 후보를 봐", + read_packet=packet, + adapter=adapter, + frame_label="order_183_source_ingest", + input_ref=[packet_event_id], + ) + + assert result.result_frame.one_step_status == "completed" + assert result.r3_inspection is not None + assert result.r3_inspection.child_node_ids == [ + "graph:source_kind_bundle:batch:code", + "graph:source_kind_bundle:batch:document", + ] + assert result.graph_traversal_candidate_surface is not None + assert result.graph_traversal_candidate_surface.candidate_count == 2 + + +def test_token_budget_bundle_selection_exposes_source_summary_children() -> None: + summary_id = "graph:summary:source_leaf:code_tools" + trace_store, data_store, packet_event_id, packet = _record_packet( + entry_rows=[ + _token_budget_bundle_row(source_graph_node_ids=[summary_id]), + ], + summary_rows=[_summary_row(summary_id=summary_id)], + ) + adapter = HierarchyAdapter(preferred_kind="token_budget_summary_bundle") + + result = run_r_loop_vessel_one_step( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_183_token_bundle", + user_question="토큰 묶음 아래 요약 후보를 봐", + read_packet=packet, + adapter=adapter, + frame_label="order_183_token_bundle", + input_ref=[packet_event_id], + ) + + assert result.result_frame.one_step_status == "completed" + assert result.r3_inspection is not None + assert result.r3_inspection.child_node_ids == [summary_id] + assert result.graph_traversal_candidate_surface is not None + assert result.graph_traversal_candidate_surface.child_candidate_node_ids == [summary_id] + + +def _record_packet( + *, + entry_rows: list[dict[str, object]], + summary_rows: list[dict[str, object]], +): + trace_store = TraceStore() + data_store = DataStore() + recorded = record_r_loop_vessel_read_packet( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_183_packet", + batch_id="order_183_packet", + config=_config(), + created_at=READ_AT, + driver_factory=FakeRLoopVesselDriverFactory( + entry_rows=entry_rows, + summary_rows=summary_rows, + ), + ) + return trace_store, data_store, recorded.trace_event_id, recorded.packet + + +def _time_axis_row() -> dict[str, object]: + return { + "candidate_node_id": "graph:axis:time", + "display_name": "Time Axis", + "node_kind": "time_axis", + "data_kind": "time_axis", + "created_at": "2026-07-02T00:00:00", + "written_at": "2026-07-02T00:00:00", + "source_trace_id": None, + "payload_json": json.dumps( + { + "node_kind": "time_axis", + "data_kind": "time_axis", + "summary_depth": 0, + }, + ensure_ascii=False, + ), + "labels": ["GraphMemoryNode", "TimeAxis"], + } + + +def _source_ingest_row( + *, + source_graph_node_ids: list[str] | None = None, +) -> dict[str, object]: + return { + "candidate_node_id": "graph:source_ingest_time_bundle:night_changed_sources", + "display_name": "Source Ingest Bundle", + "node_kind": "source_ingest_time_bundle", + "data_kind": "source_ingest_time_bundle", + "created_at": "2026-07-02T14:02:36", + "written_at": "2026-07-02T14:02:36", + "source_trace_id": "trace:source:manifest", + "payload_json": json.dumps( + { + "node_kind": "source_ingest_time_bundle", + "data_kind": "source_ingest_time_bundle", + "summary_depth": 0, + "source_leaf_count": 2, + "source_summary_count": 0, + "source_graph_node_ids": source_graph_node_ids or [], + }, + ensure_ascii=False, + ), + "labels": ["VesselRecord", "GraphMemoryNode", "SourceIngestBundle"], + "parent_graph_node_ids": ["graph:axis:time"], + } + + +def _time_bundle_row() -> dict[str, object]: + return { + "candidate_node_id": "graph:time_bundle:manual_vessel_first_write:core", + "display_name": "Time Bundle", + "node_kind": "time_bundle", + "data_kind": "time_bundle", + "created_at": "2026-07-01T18:39:54", + "written_at": "2026-07-01T18:39:54", + "source_trace_id": None, + "payload_json": json.dumps( + { + "node_kind": "time_bundle", + "data_kind": "time_bundle", + "summary_depth": 0, + "source_leaf_count": 1, + "source_summary_count": 0, + }, + ensure_ascii=False, + ), + "labels": ["VesselRecord", "GraphMemoryNode", "TimeBundle"], + "parent_graph_node_ids": ["graph:axis:time"], + } + + +def _source_kind_row(source_kind: str) -> dict[str, object]: + return { + "candidate_node_id": f"graph:source_kind_bundle:batch:{source_kind}", + "display_name": f"Source Kind Bundle: {source_kind}", + "node_kind": "source_kind_bundle", + "data_kind": f"{source_kind}_bundle", + "created_at": "2026-07-02T14:02:36", + "written_at": "2026-07-02T14:02:36", + "source_trace_id": None, + "payload_json": json.dumps( + { + "node_kind": "source_kind_bundle", + "data_kind": f"{source_kind}_bundle", + "summary_depth": 0, + "source_leaf_count": 1, + "source_summary_count": 0, + }, + ensure_ascii=False, + ), + "labels": ["VesselRecord", "GraphMemoryNode", "SourceKindBundle"], + "parent_graph_node_ids": ["graph:source_ingest_time_bundle:night_changed_sources"], + } + + +def _token_budget_bundle_row(*, source_graph_node_ids: list[str]) -> dict[str, object]: + return { + "candidate_node_id": "graph:token_budget_summary_bundle:night:0001", + "display_name": "Token Budget Summary Bundle", + "node_kind": "token_budget_summary_bundle", + "data_kind": "token_budget_summary_bundle", + "created_at": "2026-07-02T18:00:00", + "written_at": "2026-07-02T18:00:00", + "source_trace_id": None, + "payload_json": json.dumps( + { + "node_kind": "token_budget_summary_bundle", + "data_kind": "token_budget_summary_bundle", + "summary_depth": 1, + "source_leaf_count": 1, + "source_summary_count": 1, + "source_graph_node_ids": source_graph_node_ids, + }, + ensure_ascii=False, + ), + "labels": ["VesselRecord", "GraphMemoryNode"], + } + + +def _summary_row(*, summary_id: str) -> dict[str, object]: + return { + "summary_node_id": summary_id, + "summary_display_name": "Summary: code_tools", + "data_kind": "source_leaf_summary", + "info_class": "relative", + "generated_by": "LLM:fake:night_summarize_source_leaf", + "payload_json": json.dumps( + { + "data_kind": "source_leaf_summary", + "summary_depth": 1, + "summary_status": "ran", + "validity_status": "active", + "review_status": "not_reviewed", + "target_graph_node_id": "graph:raw_source:code_tools", + "target_node_kind": "raw_source", + "source_leaf_count": 1, + "source_summary_count": 0, + "source_graph_node_ids": ["graph:raw_source:code_tools"], + "source_data_ids": ["graph:raw_source:code_tools"], + "source_trace_ids": ["trace:summary:active"], + "info_class": "relative", + "generated_by": "LLM:fake:night_summarize_source_leaf", + "summary_text": "code_tools.py read-only inspection tool summary.", + }, + ensure_ascii=False, + ), + "target_graph_node_id": "graph:raw_source:code_tools", + "target_display_name": "Raw Source: code_tools", + "target_node_kind": "raw_source", + } + + +class HierarchyAdapter: + model_id = "hierarchy-adapter" + + def __init__(self, *, preferred_kind: str | None = None) -> None: + self.preferred_kind = preferred_kind + self.payloads_by_node: dict[str, dict[str, object]] = {} + + def complete(self, request: LLMRequest) -> LLMResponse: + if "R1 Vessel Goal Setter" in request.prompt: + self.payloads_by_node["R1"] = request.input_payload + user_question = str(request.input_payload.get("user_question") or "") + payload = { + "graph_search_goal": f"Inspect graph hierarchy for: {user_question}", + "user_question_anchor_id": request.input_payload["user_question_anchor"]["anchor_id"], + "required_information_granularity": "raw", + "allowed_summary_depth": 1, + "stop_condition": "Stop after one hierarchy inspection.", + } + elif "R2 Vessel Node Selector" in request.prompt: + self.payloads_by_node["R2"] = request.input_payload + surface_ref, node_ref = _select_node_ref( + request.input_payload, + preferred_kind=self.preferred_kind, + ) + payload = { + "selection_status": "selected", + "selected_surface_ref": surface_ref, + "selected_node_ref": node_ref, + "selection_reason": "Select the hierarchy entry candidate requested by the test.", + "expected_information_granularity": "raw", + "expected_source_kind": self.preferred_kind or "entry_candidate", + } + elif "R3 Vessel Inspector" in request.prompt: + self.payloads_by_node["R3"] = request.input_payload + has_children = bool(request.input_payload.get("hierarchy_child_candidate_count")) + payload = { + "current_information_granularity": "raw", + "sufficiency_status": "insufficient" if has_children else "unknown", + "granularity_problem_status": "needs_lower_granularity" + if has_children + else "unknown", + "branch_problem_status": "none", + "recommended_next_action": "deeper" if has_children else "fail", + "inspection_reason": "Child candidates are available for the next graph layer." + if has_children + else "No hierarchy child candidates are available.", + } + else: + payload = {} + return LLMResponse( + text=json.dumps(payload, ensure_ascii=False), + model_id=self.model_id, + raw=payload, + ) + + +def _select_node_ref( + payload: dict[str, object], + *, + preferred_kind: str | None, +) -> tuple[str | None, str | None]: + surface_refs = payload.get("available_surface_refs") + records_by_surface = payload.get("candidate_records_by_surface_ref") + if not isinstance(surface_refs, list) or not isinstance(records_by_surface, dict): + return None, None + fallback: tuple[str | None, str | None] = (None, None) + for surface_ref in surface_refs: + if not isinstance(surface_ref, str): + continue + records = records_by_surface.get(surface_ref) + if not isinstance(records, list): + continue + for record in records: + if not isinstance(record, dict): + continue + node_ref = record.get("node_ref") + if not isinstance(node_ref, str): + continue + if fallback == (None, None): + fallback = (surface_ref, node_ref) + if preferred_kind and record.get("candidate_kind") == preferred_kind: + return surface_ref, node_ref + return fallback diff --git a/tests/test_order_184_r_vessel_multi_step_traversal.py b/tests/test_order_184_r_vessel_multi_step_traversal.py new file mode 100644 index 0000000..67e3a81 --- /dev/null +++ b/tests/test_order_184_r_vessel_multi_step_traversal.py @@ -0,0 +1,179 @@ +import json + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.r_loop_vessel_read_packet import record_r_loop_vessel_read_packet +from songryeon_core.core.trace_store import TraceStore +from songryeon_core.loops.r_loop_vessel_one_step import ( + RLoopVesselTraverseFakeLLMAdapter, + run_r_loop_vessel_traverse, +) + +from tests.test_order_175_vessel_backed_r_read_packet import ( + READ_AT, + FakeRLoopVesselDriverFactory, + _config, +) +from tests.test_order_183_r_vessel_hierarchical_child_surface import ( + _source_ingest_row, + _time_axis_row, +) + + +SOURCE_INGEST_ID = "graph:source_ingest_time_bundle:night_changed_sources" +SOURCE_KIND_ID = "graph:source_kind_bundle:batch:code" +SUMMARY_ID = "graph:summary:source_kind:code_bundle" + + +def test_r_vessel_traverse_descends_from_core_ego_to_summary() -> None: + trace_store, data_store, packet_event_id, packet = _record_packet( + entry_rows=[ + _time_axis_row(), + _source_ingest_row(source_graph_node_ids=[SOURCE_KIND_ID]), + _source_kind_row(source_graph_node_ids=[SUMMARY_ID]), + ], + summary_rows=[_summary_row()], + ) + + result = run_r_loop_vessel_traverse( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_184_traverse", + user_question="CoreEgo에서 source kind summary까지 계층적으로 내려가", + read_packet=packet, + adapter=RLoopVesselTraverseFakeLLMAdapter(), + frame_label="order_184_traverse", + input_ref=[packet_event_id], + ) + + assert result.result_frame.traverse_status == "completed" + assert result.result_frame.step_count == 4 + assert result.result_frame.selected_graph_node_ids == [ + "graph:axis:time", + SOURCE_INGEST_ID, + SOURCE_KIND_ID, + SUMMARY_ID, + ] + assert result.result_frame.final_graph_node_id == SUMMARY_ID + assert result.result_frame.final_sufficiency_status == "sufficient" + assert result.result_frame.final_continuation_status == "stop_sufficient" + assert result.result_frame.r_loop_task_status == "sufficient" + assert result.result_frame.terminal_material_seen_count == 1 + assert result.result_frame.min_terminal_material_count == 1 + assert result.result_frame.early_stop_guard_trigger_count == 0 + assert len(result.candidate_layer_surfaces) == 4 + assert len(result.graph_traversal_candidate_surfaces) == 4 + assert len(result.r2_selections) == 4 + assert len(result.r3_inspections) == 4 + stored = data_store.require_record(result.result_frame.frame_id) + assert stored.data_type == "r_loop:vessel_traverse_result" + + +def test_r_vessel_traverse_stops_when_budget_runs_out_before_summary() -> None: + trace_store, data_store, packet_event_id, packet = _record_packet( + entry_rows=[ + _time_axis_row(), + _source_ingest_row(source_graph_node_ids=[SOURCE_KIND_ID]), + _source_kind_row(source_graph_node_ids=[SUMMARY_ID]), + ], + summary_rows=[_summary_row()], + ) + + result = run_r_loop_vessel_traverse( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_184_budget", + user_question="예산이 작으면 중간에서 멈춰야 한다", + read_packet=packet, + adapter=RLoopVesselTraverseFakeLLMAdapter(), + frame_label="order_184_budget", + input_ref=[packet_event_id], + max_node_reads=2, + max_traversal_depth=2, + ) + + assert result.result_frame.traverse_status == "completed" + assert result.result_frame.step_count == 2 + assert result.result_frame.final_graph_node_id == SOURCE_INGEST_ID + assert result.result_frame.final_sufficiency_status == "insufficient" + assert result.result_frame.final_continuation_status == "stop_budget_exhausted" + assert result.result_frame.r_loop_task_status == "partial" + + +def _record_packet( + *, + entry_rows: list[dict[str, object]], + summary_rows: list[dict[str, object]], +): + trace_store = TraceStore() + data_store = DataStore() + recorded = record_r_loop_vessel_read_packet( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_184_packet", + batch_id="order_184_packet", + config=_config(), + created_at=READ_AT, + driver_factory=FakeRLoopVesselDriverFactory( + entry_rows=entry_rows, + summary_rows=summary_rows, + ), + ) + return trace_store, data_store, recorded.trace_event_id, recorded.packet + + +def _source_kind_row(*, source_graph_node_ids: list[str]) -> dict[str, object]: + return { + "candidate_node_id": SOURCE_KIND_ID, + "display_name": "Source Kind Bundle: code", + "node_kind": "source_kind_bundle", + "data_kind": "code_bundle", + "created_at": "2026-07-02T14:02:36", + "written_at": "2026-07-02T14:02:36", + "source_trace_id": None, + "payload_json": json.dumps( + { + "node_kind": "source_kind_bundle", + "data_kind": "code_bundle", + "summary_depth": 0, + "source_leaf_count": 1, + "source_summary_count": 0, + "source_graph_node_ids": source_graph_node_ids, + }, + ensure_ascii=False, + ), + "labels": ["VesselRecord", "GraphMemoryNode", "SourceKindBundle"], + "parent_graph_node_ids": [SOURCE_INGEST_ID], + } + + +def _summary_row() -> dict[str, object]: + return { + "summary_node_id": SUMMARY_ID, + "summary_display_name": "Summary: code source kind bundle", + "data_kind": "source_kind_summary", + "info_class": "mixed", + "generated_by": "LLM:fake:night_summarize_token_layer", + "payload_json": json.dumps( + { + "data_kind": "source_kind_summary", + "summary_depth": 1, + "summary_status": "ran", + "validity_status": "active", + "review_status": "not_reviewed", + "target_graph_node_id": SOURCE_KIND_ID, + "target_node_kind": "source_kind_bundle", + "source_leaf_count": 3, + "source_summary_count": 0, + "source_graph_node_ids": [SOURCE_KIND_ID], + "source_data_ids": [SOURCE_KIND_ID], + "source_trace_ids": ["trace:summary:active"], + "info_class": "mixed", + "generated_by": "LLM:fake:night_summarize_token_layer", + "summary_text": "Code source kind bundle summary for R traversal.", + }, + ensure_ascii=False, + ), + "target_graph_node_id": SOURCE_KIND_ID, + "target_display_name": "Source Kind Bundle: code", + "target_node_kind": "source_kind_bundle", + } diff --git a/tests/test_order_185_r_terminal_material_guard.py b/tests/test_order_185_r_terminal_material_guard.py new file mode 100644 index 0000000..6e3edab --- /dev/null +++ b/tests/test_order_185_r_terminal_material_guard.py @@ -0,0 +1,156 @@ +import json + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.r_loop_vessel_read_packet import record_r_loop_vessel_read_packet +from songryeon_core.core.trace_store import TraceStore +from songryeon_core.llm.base import LLMRequest, LLMResponse +from songryeon_core.loops.r_loop_vessel_one_step import run_r_loop_vessel_traverse + +from tests.test_order_175_vessel_backed_r_read_packet import ( + READ_AT, + FakeRLoopVesselDriverFactory, + _config, +) +from tests.test_order_183_r_vessel_hierarchical_child_surface import ( + _source_ingest_row, + _time_axis_row, +) +from tests.test_order_184_r_vessel_multi_step_traversal import ( + SOURCE_INGEST_ID, + SOURCE_KIND_ID, + SUMMARY_ID, + _source_kind_row, + _summary_row, +) + + +def test_terminal_material_guard_rejects_axis_level_sufficient_stop() -> None: + trace_store, data_store, packet_event_id, packet = _record_packet( + entry_rows=[ + _time_axis_row(), + _source_ingest_row(source_graph_node_ids=[SOURCE_KIND_ID]), + _source_kind_row(source_graph_node_ids=[SUMMARY_ID]), + ], + summary_rows=[_summary_row()], + ) + + result = run_r_loop_vessel_traverse( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_185_guard", + user_question="source summary와 token layer summary 연결을 계층적으로 탐색해", + read_packet=packet, + adapter=PrematureSufficientAdapter(), + frame_label="order_185_guard", + input_ref=[packet_event_id], + ) + + assert result.result_frame.traverse_status == "completed" + assert result.result_frame.step_count == 4 + assert result.result_frame.selected_graph_node_ids == [ + "graph:axis:time", + SOURCE_INGEST_ID, + SOURCE_KIND_ID, + SUMMARY_ID, + ] + assert result.result_frame.terminal_material_seen_count == 1 + assert result.result_frame.min_terminal_material_count == 1 + assert result.result_frame.early_stop_guard_trigger_count == 3 + assert result.result_frame.final_graph_node_id == SUMMARY_ID + assert result.result_frame.final_continuation_status == "stop_sufficient" + assert [ + continuation.continuation_reason_code + for continuation in result.continuations[:3] + ] == [ + "CODE_STATUS:r_loop_terminal_material_not_seen", + "CODE_STATUS:r_loop_terminal_material_not_seen", + "CODE_STATUS:r_loop_terminal_material_not_seen", + ] + assert result.continuations[-1].continuation_reason_code == "CODE_STATUS:r3_sufficient" + + +def _record_packet( + *, + entry_rows: list[dict[str, object]], + summary_rows: list[dict[str, object]], +): + trace_store = TraceStore() + data_store = DataStore() + recorded = record_r_loop_vessel_read_packet( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_185_packet", + batch_id="order_185_packet", + config=_config(), + created_at=READ_AT, + driver_factory=FakeRLoopVesselDriverFactory( + entry_rows=entry_rows, + summary_rows=summary_rows, + ), + ) + return trace_store, data_store, recorded.trace_event_id, recorded.packet + + +class PrematureSufficientAdapter: + model_id = "premature-sufficient-adapter" + + def complete(self, request: LLMRequest) -> LLMResponse: + if "R1 Vessel Goal Setter" in request.prompt: + payload = { + "graph_search_goal": "Explore source summary and token layer summary connection.", + "user_question_anchor_id": request.input_payload["user_question_anchor"]["anchor_id"], + "required_information_granularity": "low_summary", + "allowed_summary_depth": 1, + "stop_condition": "Stop after enough terminal material is inspected.", + } + elif "R2 Vessel Node Selector" in request.prompt: + surface_ref, node_ref, candidate_kind = _first_visible_ref(request.input_payload) + payload = { + "selection_status": "selected", + "selected_surface_ref": surface_ref, + "selected_node_ref": node_ref, + "selection_reason": "Select the first supplied candidate for guard testing.", + "expected_information_granularity": "low_summary", + "expected_source_kind": candidate_kind or "candidate", + } + elif "R3 Vessel Inspector" in request.prompt: + payload = { + "current_information_granularity": "low_summary", + "sufficiency_status": "sufficient", + "granularity_problem_status": "none", + "branch_problem_status": "none", + "recommended_next_action": "stop", + "inspection_reason": "Prematurely claims the selected node is sufficient.", + } + else: + payload = {} + return LLMResponse( + text=json.dumps(payload, ensure_ascii=False), + model_id=self.model_id, + raw=payload, + ) + + +def _first_visible_ref(payload: dict[str, object]) -> tuple[str | None, str | None, str | None]: + surface_refs = payload.get("available_surface_refs") + records_by_surface = payload.get("candidate_records_by_surface_ref") + if not isinstance(surface_refs, list) or not isinstance(records_by_surface, dict): + return None, None, None + for surface_ref in surface_refs: + if not isinstance(surface_ref, str): + continue + records = records_by_surface.get(surface_ref) + if not isinstance(records, list): + continue + for record in records: + if not isinstance(record, dict): + continue + node_ref = record.get("node_ref") + if isinstance(node_ref, str) and node_ref: + candidate_kind = record.get("candidate_kind") + return ( + surface_ref, + node_ref, + candidate_kind if isinstance(candidate_kind, str) else None, + ) + return None, None, None diff --git a/tests/test_order_186_r_vessel_exact_child_expansion.py b/tests/test_order_186_r_vessel_exact_child_expansion.py new file mode 100644 index 0000000..d9ca0c8 --- /dev/null +++ b/tests/test_order_186_r_vessel_exact_child_expansion.py @@ -0,0 +1,111 @@ +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.r_loop_vessel_read_packet import ( + R_LOOP_VESSEL_EXACT_CHILD_EXPANSION_POLICY_ID, + build_r_loop_vessel_read_packet_from_neo4j, + record_r_loop_vessel_read_packet, +) +from songryeon_core.core.trace_store import TraceStore +from songryeon_core.loops.r_loop_vessel_one_step import ( + RLoopVesselTraverseFakeLLMAdapter, + run_r_loop_vessel_traverse, +) + +from tests.test_order_175_vessel_backed_r_read_packet import ( + READ_AT, + FakeRLoopVesselDriverFactory, + _config, +) +from tests.test_order_183_r_vessel_hierarchical_child_surface import ( + _source_ingest_row, + _time_axis_row, + _time_bundle_row, +) +from tests.test_order_184_r_vessel_multi_step_traversal import ( + SOURCE_KIND_ID, + SUMMARY_ID, + _source_kind_row, + _summary_row, +) + + +def test_read_packet_expands_exact_child_records_outside_base_limit() -> None: + packet = build_r_loop_vessel_read_packet_from_neo4j( + config=_config(), + batch_id="order_186_exact_child_packet", + created_at=READ_AT, + limit=3, + driver_factory=FakeRLoopVesselDriverFactory( + entry_rows=_entry_rows_with_source_kind_outside_limit(), + summary_rows=[_summary_row()], + ), + ) + + entry_ids = [ + record["candidate_node_id"] + for record in packet.entry_candidate_records + ] + + assert packet.read_status == "passed" + assert packet.exact_child_expansion_policy_id == ( + R_LOOP_VESSEL_EXACT_CHILD_EXPANSION_POLICY_ID + ) + assert packet.base_entry_candidate_count == 3 + assert packet.exact_child_expanded_entry_count == 1 + assert packet.exact_child_expanded_node_ids == [SOURCE_KIND_ID] + assert packet.exact_child_expansion_truncated is False + assert entry_ids == [ + "graph:axis:time", + "graph:source_ingest_time_bundle:night_changed_sources", + "graph:time_bundle:manual_vessel_first_write:core", + SOURCE_KIND_ID, + ] + + +def test_traverse_can_descend_through_expanded_exact_child_record() -> None: + trace_store = TraceStore() + data_store = DataStore() + recorded = record_r_loop_vessel_read_packet( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_186_packet", + batch_id="order_186_traverse_packet", + config=_config(), + created_at=READ_AT, + limit=3, + driver_factory=FakeRLoopVesselDriverFactory( + entry_rows=_entry_rows_with_source_kind_outside_limit(), + summary_rows=[_summary_row()], + ), + ) + + result = run_r_loop_vessel_traverse( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_186_traverse", + user_question="source ingest 아래 source kind와 summary까지 계층적으로 내려가", + read_packet=recorded.packet, + adapter=RLoopVesselTraverseFakeLLMAdapter(), + frame_label="order_186_traverse", + input_ref=[recorded.trace_event_id], + ) + + assert result.result_frame.traverse_status == "completed" + assert result.result_frame.step_count == 4 + assert result.result_frame.selected_graph_node_ids == [ + "graph:axis:time", + "graph:source_ingest_time_bundle:night_changed_sources", + SOURCE_KIND_ID, + SUMMARY_ID, + ] + assert result.result_frame.final_graph_node_id == SUMMARY_ID + assert result.result_frame.terminal_material_seen_count == 1 + assert result.result_frame.final_continuation_status == "stop_sufficient" + + +def _entry_rows_with_source_kind_outside_limit() -> list[dict[str, object]]: + return [ + _time_axis_row(), + _source_ingest_row(source_graph_node_ids=[SOURCE_KIND_ID]), + _time_bundle_row(), + _source_kind_row(source_graph_node_ids=[SUMMARY_ID]), + ] diff --git a/tests/test_order_187_r_vessel_summary_layer_before_raw.py b/tests/test_order_187_r_vessel_summary_layer_before_raw.py new file mode 100644 index 0000000..2e2be0d --- /dev/null +++ b/tests/test_order_187_r_vessel_summary_layer_before_raw.py @@ -0,0 +1,223 @@ +import json + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.r_loop_vessel_read_packet import record_r_loop_vessel_read_packet +from songryeon_core.core.trace_store import TraceStore +from songryeon_core.loops.r_loop_vessel_one_step import ( + RLoopVesselTraverseFakeLLMAdapter, + run_r_loop_vessel_traverse, +) + +from tests.test_order_175_vessel_backed_r_read_packet import ( + READ_AT, + FakeRLoopVesselDriverFactory, + _config, +) +from tests.test_order_183_r_vessel_hierarchical_child_surface import ( + _source_ingest_row, + _time_axis_row, + _time_bundle_row, +) +from tests.test_order_184_r_vessel_multi_step_traversal import ( + SOURCE_KIND_ID, + _source_kind_row, +) + + +RAW_SOURCE_ID = "graph:raw_source:internal_document:order_187" +SECOND_RAW_SOURCE_ID = "graph:raw_source:internal_document:order_187_b" +TOKEN_SUMMARY_ID = "graph:summary:token_budget_bundle:order_187" +LEAF_SUMMARY_ID = "graph:summary:source_leaf:order_187" +TOKEN_BUNDLE_ID = "graph:token_budget_summary_bundle:order_187:0001" + + +def test_source_kind_traversal_prefers_token_summary_layer_before_raw_source() -> None: + trace_store = TraceStore() + data_store = DataStore() + recorded = record_r_loop_vessel_read_packet( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_187_packet", + batch_id="order_187_packet", + config=_config(), + created_at=READ_AT, + driver_factory=FakeRLoopVesselDriverFactory( + entry_rows=[ + _time_axis_row(), + _source_ingest_row(source_graph_node_ids=[SOURCE_KIND_ID]), + _time_bundle_row(), + _source_kind_row( + source_graph_node_ids=[RAW_SOURCE_ID, SECOND_RAW_SOURCE_ID], + ), + _raw_source_row(RAW_SOURCE_ID), + _raw_source_row(SECOND_RAW_SOURCE_ID), + ], + summary_rows=[ + _source_leaf_summary_row(), + _token_budget_summary_row(), + ], + ), + ) + + result = run_r_loop_vessel_traverse( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_187_traverse", + user_question="source kind 아래에서 raw보다 token summary layer를 먼저 봐", + read_packet=recorded.packet, + adapter=RLoopVesselTraverseFakeLLMAdapter(), + frame_label="order_187_traverse", + input_ref=[recorded.trace_event_id], + ) + + assert result.result_frame.traverse_status == "completed" + assert result.result_frame.step_count == 4 + assert result.result_frame.selected_graph_node_ids == [ + "graph:axis:time", + "graph:source_ingest_time_bundle:night_changed_sources", + SOURCE_KIND_ID, + TOKEN_SUMMARY_ID, + ] + assert RAW_SOURCE_ID not in result.result_frame.selected_graph_node_ids + assert result.result_frame.final_graph_node_id == TOKEN_SUMMARY_ID + assert result.result_frame.terminal_material_seen_count == 1 + assert result.result_frame.final_continuation_status == "stop_sufficient" + + +def test_source_kind_traversal_uses_leaf_summary_before_raw_when_token_layer_absent() -> None: + trace_store = TraceStore() + data_store = DataStore() + recorded = record_r_loop_vessel_read_packet( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_187_leaf_packet", + batch_id="order_187_leaf_packet", + config=_config(), + created_at=READ_AT, + driver_factory=FakeRLoopVesselDriverFactory( + entry_rows=[ + _time_axis_row(), + _source_ingest_row(source_graph_node_ids=[SOURCE_KIND_ID]), + _time_bundle_row(), + _source_kind_row(source_graph_node_ids=[RAW_SOURCE_ID]), + _raw_source_row(RAW_SOURCE_ID), + ], + summary_rows=[_source_leaf_summary_row()], + ), + ) + + result = run_r_loop_vessel_traverse( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_187_leaf_traverse", + user_question="source kind 아래에서 leaf summary를 먼저 봐", + read_packet=recorded.packet, + adapter=RLoopVesselTraverseFakeLLMAdapter(), + frame_label="order_187_leaf_traverse", + input_ref=[recorded.trace_event_id], + ) + + assert result.result_frame.traverse_status == "completed" + assert result.result_frame.selected_graph_node_ids == [ + "graph:axis:time", + "graph:source_ingest_time_bundle:night_changed_sources", + SOURCE_KIND_ID, + LEAF_SUMMARY_ID, + ] + assert RAW_SOURCE_ID not in result.result_frame.selected_graph_node_ids + assert result.result_frame.final_graph_node_id == LEAF_SUMMARY_ID + + +def _raw_source_row(raw_source_id: str) -> dict[str, object]: + return { + "candidate_node_id": raw_source_id, + "display_name": "Raw Source: internal_document", + "node_kind": "raw_source", + "data_kind": "internal_document", + "created_at": "2026-07-02T14:02:36", + "written_at": "2026-07-02T14:02:36", + "source_trace_id": None, + "payload_json": json.dumps( + { + "node_kind": "raw_source", + "data_kind": "internal_document", + "summary_depth": 0, + "source_leaf_count": 1, + "source_summary_count": 0, + }, + ensure_ascii=False, + ), + "labels": ["VesselRecord", "GraphMemoryNode", "RawSource"], + "parent_graph_node_ids": [SOURCE_KIND_ID], + } + + +def _source_leaf_summary_row() -> dict[str, object]: + return { + "summary_node_id": LEAF_SUMMARY_ID, + "summary_display_name": "Summary: order_187 raw source", + "data_kind": "source_leaf_summary", + "info_class": "relative", + "generated_by": "LLM:fake:night_summarize_source_leaf", + "payload_json": json.dumps( + { + "data_kind": "source_leaf_summary", + "summary_depth": 1, + "summary_status": "ran", + "validity_status": "active", + "review_status": "not_reviewed", + "target_graph_node_id": RAW_SOURCE_ID, + "target_node_kind": "raw_source", + "source_leaf_count": 1, + "source_summary_count": 0, + "source_graph_node_ids": [RAW_SOURCE_ID], + "source_data_ids": [RAW_SOURCE_ID], + "source_trace_ids": ["trace:summary:order_187_leaf"], + "info_class": "relative", + "generated_by": "LLM:fake:night_summarize_source_leaf", + "summary_text": "Leaf summary for the ORDER 187 raw source.", + }, + ensure_ascii=False, + ), + "target_graph_node_id": RAW_SOURCE_ID, + "target_display_name": "Raw Source: internal_document", + "target_node_kind": "raw_source", + } + + +def _token_budget_summary_row() -> dict[str, object]: + return { + "summary_node_id": TOKEN_SUMMARY_ID, + "summary_display_name": "Summary: token budget bundle order_187", + "data_kind": "token_budget_bundle_summary", + "info_class": "mixed", + "generated_by": "LLM:fake:night_summarize_token_budget_bundle", + "payload_json": json.dumps( + { + "data_kind": "token_budget_bundle_summary", + "summary_depth": 2, + "summary_status": "ran", + "validity_status": "active", + "review_status": "not_reviewed", + "target_graph_node_id": TOKEN_BUNDLE_ID, + "target_node_kind": "token_budget_summary_bundle", + "source_leaf_count": 2, + "source_summary_count": 1, + "source_graph_node_ids": [TOKEN_BUNDLE_ID, LEAF_SUMMARY_ID], + "source_data_ids": [ + TOKEN_BUNDLE_ID, + LEAF_SUMMARY_ID, + RAW_SOURCE_ID, + SECOND_RAW_SOURCE_ID, + ], + "source_trace_ids": ["trace:summary:order_187_token"], + "info_class": "mixed", + "generated_by": "LLM:fake:night_summarize_token_budget_bundle", + "summary_text": "Token budget bundle summary for the ORDER 187 raw source set.", + }, + ensure_ascii=False, + ), + "target_graph_node_id": TOKEN_BUNDLE_ID, + "target_display_name": "Token Budget Summary Bundle", + "target_node_kind": "token_budget_summary_bundle", + } diff --git a/tests/test_order_188_r_vessel_raw_original_cap.py b/tests/test_order_188_r_vessel_raw_original_cap.py new file mode 100644 index 0000000..d039201 --- /dev/null +++ b/tests/test_order_188_r_vessel_raw_original_cap.py @@ -0,0 +1,154 @@ +import json + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.r_loop_vessel_read_packet import record_r_loop_vessel_read_packet +from songryeon_core.core.trace_store import TraceStore +from songryeon_core.loops.r_loop_vessel_one_step import ( + R_TRAVERSE_MAX_RAW_ORIGINAL_MATERIAL_READS, + RLoopVesselTraverseFakeLLMAdapter, + run_r_loop_vessel_traverse, +) + +from tests.test_order_175_vessel_backed_r_read_packet import ( + READ_AT, + FakeRLoopVesselDriverFactory, + _config, +) +from tests.test_order_183_r_vessel_hierarchical_child_surface import ( + _source_ingest_row, + _time_axis_row, + _time_bundle_row, +) +from tests.test_order_184_r_vessel_multi_step_traversal import ( + SOURCE_KIND_ID, + _source_kind_row, +) +from tests.test_order_187_r_vessel_summary_layer_before_raw import ( + _source_leaf_summary_row, + _token_budget_summary_row, +) + + +def test_r_vessel_traverse_stops_after_five_raw_original_reads() -> None: + trace_store = TraceStore() + data_store = DataStore() + raw_ids = [f"graph:raw_source:internal_document:order_188_{index}" for index in range(1, 7)] + recorded = record_r_loop_vessel_read_packet( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_188_packet", + batch_id="order_188_packet", + config=_config(), + created_at=READ_AT, + driver_factory=FakeRLoopVesselDriverFactory( + entry_rows=[ + _time_axis_row(), + _source_ingest_row(source_graph_node_ids=[SOURCE_KIND_ID]), + _time_bundle_row(), + _source_kind_row(source_graph_node_ids=[raw_ids[0]]), + *[ + _raw_source_chain_row( + raw_id=raw_id, + next_raw_id=raw_ids[index + 1] if index + 1 < len(raw_ids) else None, + parent_id=SOURCE_KIND_ID if index == 0 else raw_ids[index - 1], + ) + for index, raw_id in enumerate(raw_ids) + ], + ], + summary_rows=[], + ), + ) + + result = run_r_loop_vessel_traverse( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_188_traverse", + user_question="raw source 원본을 많이 내려가려고 해도 최대 5회까지만 봐", + read_packet=recorded.packet, + adapter=RLoopVesselTraverseFakeLLMAdapter(), + frame_label="order_188_traverse", + input_ref=[recorded.trace_event_id], + max_node_reads=12, + max_traversal_depth=12, + ) + + assert result.result_frame.traverse_status == "completed" + assert result.result_frame.raw_original_material_seen_count == 5 + assert result.result_frame.max_raw_original_material_count == ( + R_TRAVERSE_MAX_RAW_ORIGINAL_MATERIAL_READS + ) + assert result.result_frame.raw_original_read_cap_reached is True + assert result.result_frame.final_continuation_status == "stop_budget_exhausted" + assert result.continuations[-1].continuation_reason_code == ( + "CODE_STATUS:r_loop_raw_original_read_cap_reached" + ) + assert raw_ids[:5] == result.result_frame.selected_graph_node_ids[-5:] + assert raw_ids[5] not in result.result_frame.selected_graph_node_ids + + +def test_summary_layer_material_does_not_count_as_raw_original_read() -> None: + trace_store = TraceStore() + data_store = DataStore() + raw_id = "graph:raw_source:internal_document:order_187" + recorded = record_r_loop_vessel_read_packet( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_188_summary_packet", + batch_id="order_188_summary_packet", + config=_config(), + created_at=READ_AT, + driver_factory=FakeRLoopVesselDriverFactory( + entry_rows=[ + _time_axis_row(), + _source_ingest_row(source_graph_node_ids=[SOURCE_KIND_ID]), + _time_bundle_row(), + _source_kind_row(source_graph_node_ids=[raw_id]), + _raw_source_chain_row(raw_id=raw_id, next_raw_id=None, parent_id=SOURCE_KIND_ID), + ], + summary_rows=[_source_leaf_summary_row(), _token_budget_summary_row()], + ), + ) + + result = run_r_loop_vessel_traverse( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_188_summary_traverse", + user_question="summary layer를 먼저 보면 원본 열람으로 세지 않아야 해", + read_packet=recorded.packet, + adapter=RLoopVesselTraverseFakeLLMAdapter(), + frame_label="order_188_summary_traverse", + input_ref=[recorded.trace_event_id], + ) + + assert result.result_frame.raw_original_material_seen_count == 0 + assert result.result_frame.raw_original_read_cap_reached is False + + +def _raw_source_chain_row( + *, + raw_id: str, + next_raw_id: str | None, + parent_id: str, +) -> dict[str, object]: + return { + "candidate_node_id": raw_id, + "display_name": "Raw Source: internal_document", + "node_kind": "raw_source", + "data_kind": "internal_document", + "created_at": "2026-07-03T00:00:00", + "written_at": "2026-07-03T00:00:00", + "source_trace_id": None, + "payload_json": json.dumps( + { + "node_kind": "raw_source", + "data_kind": "internal_document", + "summary_depth": 0, + "source_leaf_count": 1, + "source_summary_count": 0, + "source_graph_node_ids": [next_raw_id] if next_raw_id else [], + }, + ensure_ascii=False, + ), + "labels": ["VesselRecord", "GraphMemoryNode", "RawSource"], + "parent_graph_node_ids": [parent_id], + } diff --git a/tests/test_order_190_r_vessel_token_summary_deeper_child_expansion.py b/tests/test_order_190_r_vessel_token_summary_deeper_child_expansion.py new file mode 100644 index 0000000..ff32848 --- /dev/null +++ b/tests/test_order_190_r_vessel_token_summary_deeper_child_expansion.py @@ -0,0 +1,281 @@ +import json + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.r_loop_vessel_read_packet import ( + R_LOOP_VESSEL_SUMMARY_CHILD_EXPANSION_POLICY_ID, + build_r_loop_vessel_read_packet_from_neo4j, + record_r_loop_vessel_read_packet, +) +from songryeon_core.core.trace_store import TraceStore +from songryeon_core.llm.base import LLMRequest, LLMResponse +from songryeon_core.loops.r_loop_vessel_one_step import run_r_loop_vessel_traverse + +from tests.test_order_175_vessel_backed_r_read_packet import ( + READ_AT, + FakeRLoopVesselDriverFactory, + _config, +) +from tests.test_order_183_r_vessel_hierarchical_child_surface import ( + _source_ingest_row, + _time_axis_row, + _time_bundle_row, +) +from tests.test_order_184_r_vessel_multi_step_traversal import ( + SOURCE_KIND_ID, + _source_kind_row, +) +from tests.test_order_188_r_vessel_raw_original_cap import _raw_source_chain_row + + +RAW_SOURCE_ID = "graph:raw_source:internal_document:order_190" +PARENT_TOKEN_SUMMARY_ID = "graph:summary:token_budget_bundle:order_190_parent" +UNRELATED_TOKEN_SUMMARY_ID = "graph:summary:token_budget_bundle:order_190_unrelated" +CHILD_TOKEN_SUMMARY_ID = "graph:summary:token_budget_bundle:order_190_child" +PARENT_TOKEN_BUNDLE_ID = "graph:token_budget_summary_bundle:order_190:parent" +UNRELATED_TOKEN_BUNDLE_ID = "graph:token_budget_summary_bundle:order_190:unrelated" +CHILD_TOKEN_BUNDLE_ID = "graph:token_budget_summary_bundle:order_190:child" + + +def test_read_packet_expands_summary_children_for_token_summary_traversal() -> None: + packet = build_r_loop_vessel_read_packet_from_neo4j( + config=_config(), + batch_id="order_190_packet", + created_at=READ_AT, + limit=1, + driver_factory=FakeRLoopVesselDriverFactory( + entry_rows=_entry_rows(), + summary_rows=_summary_rows(), + ), + ) + + summary_ids = [ + record["summary_node_id"] for record in packet.summary_candidate_records + ] + + assert packet.summary_child_expansion_policy_id == ( + R_LOOP_VESSEL_SUMMARY_CHILD_EXPANSION_POLICY_ID + ) + assert packet.base_summary_candidate_count == 1 + assert packet.summary_child_expanded_count == 1 + assert packet.summary_child_expanded_node_ids == [CHILD_TOKEN_SUMMARY_ID] + assert packet.summary_child_expansion_truncated is False + assert PARENT_TOKEN_SUMMARY_ID in summary_ids + assert CHILD_TOKEN_SUMMARY_ID in summary_ids + assert summary_ids == [ + PARENT_TOKEN_SUMMARY_ID, + CHILD_TOKEN_SUMMARY_ID, + ] + + +def test_traverse_continues_from_token_summary_to_lower_summary_layer() -> None: + trace_store = TraceStore() + data_store = DataStore() + recorded = record_r_loop_vessel_read_packet( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_190_packet", + batch_id="order_190_traverse_packet", + config=_config(), + created_at=READ_AT, + limit=2, + driver_factory=FakeRLoopVesselDriverFactory( + entry_rows=_entry_rows(), + summary_rows=_summary_rows(), + ), + ) + + result = run_r_loop_vessel_traverse( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_190_traverse", + user_question="token summary 아래 lower summary layer까지 내려가", + read_packet=recorded.packet, + adapter=TokenSummaryNeedsDeeperAdapter(), + frame_label="order_190_traverse", + input_ref=[recorded.trace_event_id], + ) + + assert result.result_frame.traverse_status == "completed" + assert result.result_frame.final_graph_node_id == CHILD_TOKEN_SUMMARY_ID + assert result.result_frame.final_continuation_status == "stop_sufficient" + assert result.result_frame.r_loop_task_status == "sufficient" + assert result.result_frame.selected_graph_node_ids == [ + "graph:axis:time", + "graph:source_ingest_time_bundle:night_changed_sources", + SOURCE_KIND_ID, + PARENT_TOKEN_SUMMARY_ID, + CHILD_TOKEN_SUMMARY_ID, + ] + assert result.result_frame.raw_original_material_seen_count == 0 + token_surface = result.candidate_layer_surfaces[4] + assert token_surface.total_candidate_count == 1 + assert token_surface.surface_records[0]["candidate_graph_node_ids"] == [ + CHILD_TOKEN_SUMMARY_ID + ] + + +class TokenSummaryNeedsDeeperAdapter: + model_id = "order-190-token-summary-needs-deeper-fake" + + def complete(self, request: LLMRequest) -> LLMResponse: + if "R1 Vessel Goal Setter" in request.prompt: + payload = { + "graph_search_goal": "Traverse from token summary to lower summary layer", + "user_question_anchor_id": request.input_payload["user_question_anchor"]["anchor_id"], + "required_information_granularity": "low_summary", + "allowed_summary_depth": 3, + "stop_condition": "Stop after the source leaf summary is inspected.", + } + elif "R2 Vessel Node Selector" in request.prompt: + surface_ref, node_ref = _first_ref(request.input_payload) + payload = { + "selection_status": "selected" if node_ref else "none_selected", + "selected_surface_ref": surface_ref if node_ref else None, + "selected_node_ref": node_ref, + "selection_reason": "Select the first official child candidate.", + "expected_information_granularity": "low_summary", + "expected_source_kind": "hierarchy_candidate", + } + elif "R3 Vessel Inspector" in request.prompt: + selected = request.input_payload.get("selected_candidate_record") + data_kind = selected.get("data_kind") if isinstance(selected, dict) else None + has_children = request.input_payload.get("hierarchy_child_candidate_count") + should_continue = ( + data_kind == "token_budget_bundle_summary" + and isinstance(has_children, int) + and has_children > 0 + ) + payload = { + "current_information_granularity": "low_summary", + "sufficiency_status": "insufficient" if should_continue else "sufficient", + "granularity_problem_status": ( + "needs_lower_granularity" if should_continue else "none" + ), + "branch_problem_status": "none", + "recommended_next_action": "deeper" if should_continue else "stop", + "inspection_reason": ( + "Token summary has lower summary children, so continue deeper." + if should_continue + else "Source leaf summary is enough for this test." + ), + } + else: + payload = {"error": "unknown prompt"} + return LLMResponse( + text=json.dumps(payload, ensure_ascii=False), + model_id=self.model_id, + raw=payload, + ) + + +def _first_ref(payload: dict[str, object]) -> tuple[str | None, str | None]: + surfaces = payload.get("available_surface_refs") + records_by_surface = payload.get("candidate_records_by_surface_ref") + if not isinstance(surfaces, list) or not isinstance(records_by_surface, dict): + return None, None + for surface in surfaces: + if not isinstance(surface, str): + continue + records = records_by_surface.get(surface) + if not isinstance(records, list): + continue + for record in records: + if isinstance(record, dict) and isinstance(record.get("node_ref"), str): + return surface, record["node_ref"] + return None, None + + +def _entry_rows() -> list[dict[str, object]]: + return [ + _time_axis_row(), + _source_ingest_row(source_graph_node_ids=[SOURCE_KIND_ID]), + _time_bundle_row(), + _source_kind_row(source_graph_node_ids=[RAW_SOURCE_ID]), + _raw_source_chain_row( + raw_id=RAW_SOURCE_ID, + next_raw_id=None, + parent_id=SOURCE_KIND_ID, + ), + ] + + +def _summary_rows() -> list[dict[str, object]]: + return [ + _parent_token_summary_row(), + _unrelated_token_summary_row(), + _child_token_summary_row(), + ] + + +def _parent_token_summary_row() -> dict[str, object]: + return _token_summary_row( + summary_id=PARENT_TOKEN_SUMMARY_ID, + target_id=PARENT_TOKEN_BUNDLE_ID, + source_ids=[PARENT_TOKEN_BUNDLE_ID, CHILD_TOKEN_SUMMARY_ID], + source_data_ids=[PARENT_TOKEN_BUNDLE_ID, CHILD_TOKEN_SUMMARY_ID, RAW_SOURCE_ID], + depth=3, + text="Parent token summary that points at a lower token summary.", + ) + + +def _child_token_summary_row() -> dict[str, object]: + return _token_summary_row( + summary_id=CHILD_TOKEN_SUMMARY_ID, + target_id=CHILD_TOKEN_BUNDLE_ID, + source_ids=[CHILD_TOKEN_BUNDLE_ID], + source_data_ids=[CHILD_TOKEN_BUNDLE_ID, RAW_SOURCE_ID], + depth=2, + text="Child token summary that is enough for this test.", + ) + + +def _unrelated_token_summary_row() -> dict[str, object]: + return _token_summary_row( + summary_id=UNRELATED_TOKEN_SUMMARY_ID, + target_id=UNRELATED_TOKEN_BUNDLE_ID, + source_ids=[UNRELATED_TOKEN_BUNDLE_ID], + source_data_ids=[UNRELATED_TOKEN_BUNDLE_ID], + depth=3, + text="Unrelated token summary used to occupy the base summary window.", + ) + + +def _token_summary_row( + *, + summary_id: str, + target_id: str, + source_ids: list[str], + source_data_ids: list[str], + depth: int, + text: str, +) -> dict[str, object]: + return { + "summary_node_id": summary_id, + "summary_display_name": f"Summary: {target_id}", + "data_kind": "token_budget_bundle_summary", + "info_class": "mixed", + "generated_by": "LLM:fake:night_summarize_token_budget_bundle", + "payload_json": json.dumps( + { + "data_kind": "token_budget_bundle_summary", + "summary_depth": depth, + "summary_status": "ran", + "validity_status": "active", + "review_status": "not_reviewed", + "target_graph_node_id": target_id, + "target_node_kind": "token_budget_summary_bundle", + "source_leaf_count": 1, + "source_summary_count": 1, + "source_graph_node_ids": source_ids, + "source_data_ids": source_data_ids, + "source_trace_ids": [f"trace:summary:{summary_id}"], + "info_class": "mixed", + "generated_by": "LLM:fake:night_summarize_token_budget_bundle", + "summary_text": text, + }, + ensure_ascii=False, + ), + "target_graph_node_id": target_id, + "target_display_name": "Token Budget Summary Bundle", + "target_node_kind": "token_budget_summary_bundle", + } diff --git a/tests/test_order_191_r2_branch_role_surface_stability.py b/tests/test_order_191_r2_branch_role_surface_stability.py new file mode 100644 index 0000000..15f5fce --- /dev/null +++ b/tests/test_order_191_r2_branch_role_surface_stability.py @@ -0,0 +1,181 @@ +import json + +from songryeon_core.core.data_store import DataStore +from songryeon_core.core.r_loop_vessel_read_packet import record_r_loop_vessel_read_packet +from songryeon_core.core.trace_store import TraceStore +from songryeon_core.llm.base import LLMRequest, LLMResponse +from songryeon_core.loops.r_loop_vessel_one_step import run_r_loop_vessel_traverse + +from tests.test_order_175_vessel_backed_r_read_packet import ( + READ_AT, + FakeRLoopVesselDriverFactory, + _config, +) +from tests.test_order_183_r_vessel_hierarchical_child_surface import ( + _source_ingest_row, + _time_axis_row, + _time_bundle_row, +) + + +def test_time_axis_child_surface_exposes_structural_branch_roles() -> None: + trace_store, data_store, packet_event_id, packet = _record_packet() + adapter = BranchRoleCaptureAdapter() + + result = run_r_loop_vessel_traverse( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_191_branch_roles", + user_question="source summary와 token summary 구조를 확인해줘", + read_packet=packet, + adapter=adapter, + frame_label="order_191_branch_roles", + input_ref=[packet_event_id], + max_node_reads=2, + max_traversal_depth=2, + ) + + assert result.result_frame.traverse_status == "completed" + assert len(result.candidate_layer_surfaces) == 2 + + child_surface = result.candidate_layer_surfaces[1] + branch_roles = [ + record["branch_role"] for record in child_surface.surface_records + ] + assert branch_roles == [ + "source_material_ingest", + "conversation_time_memory", + ] + assert child_surface.surface_records[0]["candidate_kind"] == "source_ingest_bundle" + assert child_surface.surface_records[1]["candidate_kind"] == "time_bundle" + + second_r2_payload = adapter.r2_payloads[1] + surface_records = second_r2_payload["candidate_layer_surface_ref_records"] + assert surface_records[0]["branch_role"] == "source_material_ingest" + assert surface_records[1]["branch_role"] == "conversation_time_memory" + + records_by_surface = second_r2_payload["candidate_records_by_surface_ref"] + source_surface_ref = surface_records[0]["surface_ref"] + time_surface_ref = surface_records[1]["surface_ref"] + assert records_by_surface[source_surface_ref][0]["branch_role"] == ( + "source_material_ingest" + ) + assert records_by_surface[time_surface_ref][0]["branch_role"] == ( + "conversation_time_memory" + ) + + +def test_structural_surface_order_keeps_source_ingest_before_time_bundle() -> None: + trace_store, data_store, packet_event_id, packet = _record_packet() + + result = run_r_loop_vessel_traverse( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_191_source_first", + user_question="source summary와 token summary 구조를 확인해줘", + read_packet=packet, + adapter=BranchRoleCaptureAdapter(), + frame_label="order_191_source_first", + input_ref=[packet_event_id], + max_node_reads=2, + max_traversal_depth=2, + ) + + assert result.result_frame.selected_graph_node_ids[:2] == [ + "graph:axis:time", + "graph:source_ingest_time_bundle:night_changed_sources", + ] + assert "graph:time_bundle:manual_vessel_first_write:core" not in ( + result.result_frame.selected_graph_node_ids + ) + + +class BranchRoleCaptureAdapter: + model_id = "order-191-branch-role-capture-fake" + + def __init__(self) -> None: + self.r2_payloads: list[dict[str, object]] = [] + + def complete(self, request: LLMRequest) -> LLMResponse: + if "R1 Vessel Goal Setter" in request.prompt: + payload = { + "graph_search_goal": "Inspect source summary and token summary graph branches", + "user_question_anchor_id": request.input_payload["user_question_anchor"]["anchor_id"], + "required_information_granularity": "low_summary", + "allowed_summary_depth": 1, + "stop_condition": "Stop after the structural branch entry is inspected.", + } + elif "R2 Vessel Node Selector" in request.prompt: + self.r2_payloads.append(request.input_payload) + surface_ref, node_ref = _first_ref(request.input_payload) + payload = { + "selection_status": "selected" if node_ref else "none_selected", + "selected_surface_ref": surface_ref if node_ref else None, + "selected_node_ref": node_ref, + "selection_reason": "Select the first structurally ordered official candidate.", + "expected_information_granularity": "low_summary", + "expected_source_kind": "structural_branch_candidate", + } + elif "R3 Vessel Inspector" in request.prompt: + child_count = request.input_payload.get("hierarchy_child_candidate_count") + has_children = isinstance(child_count, int) and child_count > 0 + payload = { + "current_information_granularity": "low_summary", + "sufficiency_status": "insufficient" if has_children else "unknown", + "granularity_problem_status": ( + "needs_lower_granularity" if has_children else "unknown" + ), + "branch_problem_status": "none", + "recommended_next_action": "deeper" if has_children else "fail", + "inspection_reason": ( + "Child branch candidates are available." + if has_children + else "No child branch candidates are available." + ), + } + else: + payload = {"error": "unknown prompt"} + return LLMResponse( + text=json.dumps(payload, ensure_ascii=False), + model_id=self.model_id, + raw=payload, + ) + + +def _first_ref(payload: dict[str, object]) -> tuple[str | None, str | None]: + surface_refs = payload.get("available_surface_refs") + records_by_surface = payload.get("candidate_records_by_surface_ref") + if not isinstance(surface_refs, list) or not isinstance(records_by_surface, dict): + return None, None + for surface_ref in surface_refs: + if not isinstance(surface_ref, str): + continue + records = records_by_surface.get(surface_ref) + if not isinstance(records, list): + continue + for record in records: + if isinstance(record, dict) and isinstance(record.get("node_ref"), str): + return surface_ref, record["node_ref"] + return None, None + + +def _record_packet(): + trace_store = TraceStore() + data_store = DataStore() + recorded = record_r_loop_vessel_read_packet( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_191_packet", + batch_id="order_191_packet", + config=_config(), + created_at=READ_AT, + driver_factory=FakeRLoopVesselDriverFactory( + entry_rows=[ + _time_axis_row(), + _source_ingest_row(), + _time_bundle_row(), + ], + summary_rows=[], + ), + ) + return trace_store, data_store, recorded.trace_event_id, recorded.packet diff --git a/tests/test_order_192_r1_user_question_anchor_copy.py b/tests/test_order_192_r1_user_question_anchor_copy.py new file mode 100644 index 0000000..38f4113 --- /dev/null +++ b/tests/test_order_192_r1_user_question_anchor_copy.py @@ -0,0 +1,169 @@ +from __future__ import annotations + +import json + +from songryeon_core.llm.base import LLMRequest, LLMResponse +from songryeon_core.loops.r_loop_vessel_one_step import run_r_loop_vessel_one_step + +from tests.test_order_176_vessel_r_one_step_traversal import _record_packet + + +def test_r1_input_payload_supplies_user_question_anchor() -> None: + trace_store, data_store, packet_event_id, packet = _record_packet() + adapter = AnchorCopyAdapter() + + result = run_r_loop_vessel_one_step( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_192_anchor_payload", + user_question="송련 그래프 기억을 한글 질문으로 확인해줘", + read_packet=packet, + adapter=adapter, + frame_label="order_192_anchor_payload", + input_ref=[packet_event_id], + ) + + assert result.result_frame.one_step_status == "completed" + anchor = adapter.r1_input_payload["user_question_anchor"] + assert isinstance(anchor, dict) + assert anchor["copy_required"] is True + assert anchor["source_field"] == "user_question" + assert str(anchor["anchor_id"]).startswith("r1_user_question_anchor:") + assert result.r1_goal is not None + assert result.r1_goal.user_question_anchor_id == anchor["anchor_id"] + + +def test_r1_korean_goal_passes_when_anchor_id_is_copied() -> None: + trace_store, data_store, packet_event_id, packet = _record_packet() + + result = run_r_loop_vessel_one_step( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_192_korean_goal", + user_question="소스 요약과 토큰 묶음 요약의 연결을 한글로 확인해줘", + read_packet=packet, + adapter=AnchorCopyAdapter( + graph_search_goal="한글 목표문으로 그래프 요약 연결을 확인한다.", + ), + frame_label="order_192_korean_goal", + input_ref=[packet_event_id], + ) + + assert result.result_frame.one_step_status == "completed" + assert result.r1_goal is not None + assert result.r1_goal.graph_search_goal == "한글 목표문으로 그래프 요약 연결을 확인한다." + assert "source" not in result.r1_goal.graph_search_goal.lower() + assert "token" not in result.r1_goal.graph_search_goal.lower() + + +def test_r1_missing_user_question_anchor_id_fails() -> None: + trace_store, data_store, packet_event_id, packet = _record_packet() + + result = run_r_loop_vessel_one_step( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_192_missing_anchor", + user_question="소스 요약과 토큰 묶음 요약의 연결을 확인해줘", + read_packet=packet, + adapter=AnchorCopyAdapter(anchor_mode="missing"), + frame_label="order_192_missing_anchor", + input_ref=[packet_event_id], + ) + + assert result.result_frame.one_step_status == "failed" + assert result.result_frame.failure_stage == "R1" + assert result.result_frame.failure_type == "schema_failed" + assert "user_question_anchor_id" in (result.result_frame.failure_reason or "") + + +def test_r1_wrong_user_question_anchor_id_fails() -> None: + trace_store, data_store, packet_event_id, packet = _record_packet() + + result = run_r_loop_vessel_one_step( + trace_store=trace_store, + data_store=data_store, + turn_id="turn_order_192_wrong_anchor", + user_question="소스 요약과 토큰 묶음 요약의 연결을 확인해줘", + read_packet=packet, + adapter=AnchorCopyAdapter(anchor_mode="wrong"), + frame_label="order_192_wrong_anchor", + input_ref=[packet_event_id], + ) + + assert result.result_frame.one_step_status == "failed" + assert result.result_frame.failure_stage == "R1" + assert result.result_frame.failure_type == "schema_failed" + assert "user_question_anchor_id" in (result.result_frame.failure_reason or "") + + +class AnchorCopyAdapter: + model_id = "order-192-anchor-copy-adapter" + + def __init__( + self, + *, + anchor_mode: str = "copy", + graph_search_goal: str = "한글 목표문으로 그래프 경로를 확인한다.", + ) -> None: + self.anchor_mode = anchor_mode + self.graph_search_goal = graph_search_goal + self.r1_input_payload: dict[str, object] = {} + + def complete(self, request: LLMRequest) -> LLMResponse: + if "R1 Vessel Goal Setter" in request.prompt: + self.r1_input_payload = request.input_payload + payload: dict[str, object] = { + "graph_search_goal": self.graph_search_goal, + "required_information_granularity": "low_summary", + "allowed_summary_depth": 1, + "stop_condition": "Stop after one selected candidate inspection.", + } + if self.anchor_mode == "copy": + payload["user_question_anchor_id"] = request.input_payload[ + "user_question_anchor" + ]["anchor_id"] + elif self.anchor_mode == "wrong": + payload["user_question_anchor_id"] = "r1_user_question_anchor:wrong" + elif "R2 Vessel Node Selector" in request.prompt: + surface_ref, node_ref = _first_ref(request.input_payload) + payload = { + "selection_status": "selected" if node_ref else "none_selected", + "selected_surface_ref": surface_ref if node_ref else None, + "selected_node_ref": node_ref, + "selection_reason": "Select the first official candidate for anchor copy test.", + "expected_information_granularity": "low_summary", + "expected_source_kind": "summary_or_entry_candidate", + } + elif "R3 Vessel Inspector" in request.prompt: + payload = { + "current_information_granularity": "low_summary", + "sufficiency_status": "sufficient", + "granularity_problem_status": "none", + "branch_problem_status": "none", + "recommended_next_action": "stop", + "inspection_reason": "Anchor copy test supplied enough material.", + } + else: + payload = {"error": "unknown prompt"} + return LLMResponse( + text=json.dumps(payload, ensure_ascii=False), + model_id=self.model_id, + raw=payload, + ) + + +def _first_ref(payload: dict[str, object]) -> tuple[str | None, str | None]: + surface_refs = payload.get("available_surface_refs") + records_by_surface = payload.get("candidate_records_by_surface_ref") + if not isinstance(surface_refs, list) or not isinstance(records_by_surface, dict): + return None, None + for surface_ref in surface_refs: + if not isinstance(surface_ref, str): + continue + records = records_by_surface.get(surface_ref) + if not isinstance(records, list): + continue + for record in records: + if isinstance(record, dict) and isinstance(record.get("node_ref"), str): + return surface_ref, record["node_ref"] + return None, None