Refactor: clean up repo structure and organize docs#17
Refactor: clean up repo structure and organize docs#17zhangqi-chen merged 1 commit intohw-native-sys:mainfrom
Conversation
- Remove archived directories: junk_models/, junk_tensor_functions/, projects/ - Remove non-essential examples: softmax, paged attention, batch hash lookup - Remove build dump directories from examples/ - Move core reference docs to docs/: coding style, para_for, pto2_rt - Delete temporary session logs and outdated task files - Update file path references in CLAUDE.md, git-commit skill, and issue template
📝 WalkthroughWalkthroughLarge-scale cleanup removing numerous example files, model implementations, tensor function libraries, and supporting documentation. Changes include deletion of multiple paged attention examples, model code for Arctic/DeepSeek/GLM/Qwen3, tensor operation primitives, and project artifacts, alongside minor documentation path updates. Changes
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes
|
There was a problem hiding this comment.
🧹 Nitpick comments (1)
.claude/skills/git-commit/SKILL.md (1)
19-19: Clarify what to run for Python changes outsideexamples/.Line 19 is clear for example files, but the table now leaves
tests//tooling.pychanges unspecified. Add an explicit row for “Python outsideexamples/” with expected checks (e.g., lint/tests) to avoid skipped validation.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In @.claude/skills/git-commit/SKILL.md at line 19, Update the checklist table in .claude/skills/git-commit/SKILL.md by adding an explicit row for "Python (outside `examples/`)" next to the existing "Python (`.py`) in `examples/`" entry; specify the expected verification steps such as running linters (e.g., flake8/ruff), running unit tests in tests/, and validating any tooling or CI config changes so Python changes outside examples aren't left unspecified. Reference the existing table and the "Python (`.py`) in `examples/`" row when adding the new row to match style and columns.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Nitpick comments:
In @.claude/skills/git-commit/SKILL.md:
- Line 19: Update the checklist table in .claude/skills/git-commit/SKILL.md by
adding an explicit row for "Python (outside `examples/`)" next to the existing
"Python (`.py`) in `examples/`" entry; specify the expected verification steps
such as running linters (e.g., flake8/ruff), running unit tests in tests/, and
validating any tooling or CI config changes so Python changes outside examples
aren't left unspecified. Reference the existing table and the "Python (`.py`) in
`examples/`" row when adding the new row to match style and columns.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: bc720b13-d719-4f9b-8556-403184bc97d9
📒 Files selected for processing (129)
.claude/CLAUDE.md.claude/skills/git-commit/SKILL.md.github/ISSUE_TEMPLATE/documentation.ymldeepseek_v3_2_code_review.mddesign_goal.mddocs/para_for.mddocs/pto2_rt.mddocs/pypto-frontend-coding-style.mdexamples/batch_hash_lookup.pyexamples/docs/deepseek_v3_2_aiv_splitting_analysis_pass08.mdexamples/docs/deepseek_v3_2_fusion_scope_notes.mdexamples/docs/deepseek_v3_2_memory_usage_unified_report.mdexamples/docs/deepseek_v3_2_mixed_kernel_local_usage_side_by_side.mdexamples/docs/gen_front_capacity_budget.pyexamples/docs/gen_mixed_kernel_local_usage.pyexamples/docs/qwen3_aiv_splitting_analysis_pass08.mdexamples/docs/seq_lens_handling_analysis.mdexamples/pa.pyexamples/pa1.pyexamples/pa1_build/passes_dump/00_frontend.pyexamples/pa1_build/passes_dump/01_after_UnrollLoops.pyexamples/pa1_build/passes_dump/02_after_ConvertToSSA.pyexamples/pa1_build/passes_dump/03_after_FlattenCallExpr.pyexamples/pa1_build/passes_dump/04_after_SplitChunkedLoops.pyexamples/pa1_build/passes_dump/05_after_InterchangeChunkLoops.pyexamples/pa1_build/passes_dump/06_after_RunVerifier.pyexamples/pa1_build/passes_dump/07_after_OutlineIncoreScopes.pyexamples/pa2.pyexamples/pa3.pyexamples/pa3_build/passes_dump/00_frontend.pyexamples/pa3_build/passes_dump/01_after_UnrollLoops.pyexamples/pa3_build/passes_dump/02_after_ConvertToSSA.pyexamples/pa3_build/passes_dump/03_after_FlattenCallExpr.pyexamples/pa3_build/passes_dump/04_after_SplitChunkedLoops.pyexamples/pa3_build/passes_dump/05_after_InterchangeChunkLoops.pyexamples/pa3_build/passes_dump/06_after_RunVerifier.pyexamples/pa3_build/passes_dump/07_after_OutlineIncoreScopes.pyexamples/pa3_build/passes_dump/08_after_ConvertTensorToBlockOps.pyexamples/pa3_build/passes_dump/09_after_InitMemRef.pyexamples/pa3_build/passes_dump/10_after_MemoryReuse.pyexamples/pa3_build/passes_dump/11_after_InsertSync.pyexamples/pa3_build/passes_dump/12_after_AllocateMemoryAddr.pyexamples/pa4.pyexamples/pa4_build/passes_dump/00_frontend.pyexamples/pa4_build/passes_dump/01_after_UnrollLoops.pyexamples/pa4_build/passes_dump/02_after_ConvertToSSA.pyexamples/pa4_build/passes_dump/03_after_FlattenCallExpr.pyexamples/pa4_build/passes_dump/04_after_SplitChunkedLoops.pyexamples/pa4_build/passes_dump/05_after_InterchangeChunkLoops.pyexamples/pa4_build/passes_dump/06_after_RunVerifier.pyexamples/pa4_build/passes_dump/07_after_OutlineIncoreScopes.pyexamples/pa4_build/passes_dump/08_after_ExpandMixedKernel.pyexamples/pa4_build/passes_dump/09_after_ConvertTensorToBlockOps.pyexamples/pa4_build/passes_dump/10_after_InitMemRef.pyexamples/pa4_build/passes_dump/11_after_MemoryReuse.pyexamples/pa4_build/passes_dump/12_after_InsertSync.pyexamples/pa4_build/passes_dump/13_after_AllocateMemoryAddr.pyexamples/pa5_predicate_build/passes_dump/00_frontend.pyexamples/pa5_predicate_build/passes_dump/01_after_UnrollLoops.pyexamples/pa5_predicate_build/passes_dump/02_after_ConvertToSSA.pyexamples/pa5_predicate_build/passes_dump/03_after_FlattenCallExpr.pyexamples/pa5_predicate_build/passes_dump/04_after_SplitChunkedLoops.pyexamples/pa5_predicate_build/passes_dump/05_after_InterchangeChunkLoops.pyexamples/pa5_predicate_build/passes_dump/06_after_RunVerifier.pyexamples/pa5_predicate_build/passes_dump/07_after_OutlineIncoreScopes.pyexamples/pa5_predicate_build/passes_dump/08_after_ExpandMixedKernel.pyexamples/pa5_predicate_build/passes_dump/09_after_ConvertTensorToBlockOps.pyexamples/pa5_predicate_build/passes_dump/10_after_InitMemRef.pyexamples/pa5_predicate_build/passes_dump/11_after_MemoryReuse.pyexamples/pa5_predicate_build/passes_dump/12_after_InsertSync.pyexamples/pa5_predicate_build/passes_dump/13_after_AllocateMemoryAddr.pyexamples/pa5_predicate_test.pyexamples/paged_attention_example.pyexamples/qwen3_32b_build/report/memory_after_AllocateMemoryAddr.txtexamples/qwen3_32b_prefill_dump/report/prefill_kernel_local_tensor_summary.mdexamples/softmax_example.pyjunk_models/__init__.pyjunk_models/arctic/__init__.pyjunk_models/arctic/sum_lstm.pyjunk_models/deepseek_v32_exp/__init__.pyjunk_models/deepseek_v32_exp/lightning_indexer_prolog_quant.pyjunk_models/deepseek_v32_exp/lightning_indexer_quant.pyjunk_models/deepseek_v32_exp/mla_indexer_prolog_quant.pyjunk_models/deepseek_v32_exp/mla_prolog_quant.pyjunk_models/deepseek_v32_exp/sparse_attention_antiquant.pyjunk_models/deepseek_v32_exp/sparse_flash_attention_quant.pyjunk_models/deepseek_v32_exp/sparse_flash_attention_quant_withscope.pyjunk_models/glm_v4_5/__init__.pyjunk_models/glm_v4_5/glm_attention.pyjunk_models/glm_v4_5/glm_attention_fusion.pyjunk_models/glm_v4_5/glm_attention_pre_quant.pyjunk_models/glm_v4_5/glm_ffn_common_interface.pyjunk_models/glm_v4_5/glm_ffn_shared_expert_quant.pyjunk_models/glm_v4_5/glm_gate.pyjunk_models/glm_v4_5/glm_matmul_allreduce_add_rmsnorm.pyjunk_models/glm_v4_5/glm_moe_fusion.pyjunk_models/glm_v4_5/glm_select_experts.pyjunk_models/qwen3_next/__init__.pyjunk_models/qwen3_next/gated_delta_rule.pyjunk_tensor_functions/__init__.pyjunk_tensor_functions/cast_primitives.pyjunk_tensor_functions/composite.pyjunk_tensor_functions/elementwise_binary.pyjunk_tensor_functions/elementwise_unary.pyjunk_tensor_functions/indexing.pyjunk_tensor_functions/linalg.pyjunk_tensor_functions/reduction.pyjunk_tensor_functions/tiling.pyjunk_tensor_functions/type_layout.pyporting_to_pypto_v3.mdprojects/deepseek_v32_exp_lightning_indexer_prolog_quant/PROJECT_ANALYSIS.mdprojects/deepseek_v32_exp_lightning_indexer_prolog_quant/lightning_indexer_prolog_quant.pyprojects/deepseek_v32_exp_lightning_indexer_prolog_quant/pypto_src/README.mdprojects/deepseek_v32_exp_lightning_indexer_prolog_quant/pypto_src/build_output/kernel_config.pyprojects/deepseek_v32_exp_lightning_indexer_prolog_quant/pypto_src/build_output/kernels/aic/incore_matmul.ptoprojects/deepseek_v32_exp_lightning_indexer_prolog_quant/pypto_src/build_output/kernels/aic/incore_matmul_acc.ptoprojects/deepseek_v32_exp_lightning_indexer_prolog_quant/pypto_src/build_output/kernels/aic/incore_matmul_nn.ptoprojects/deepseek_v32_exp_lightning_indexer_prolog_quant/pypto_src/build_output/kernels/aiv/incore_add.ptoprojects/deepseek_v32_exp_lightning_indexer_prolog_quant/pypto_src/build_output/kernels/aiv/incore_layernorm.ptoprojects/deepseek_v32_exp_lightning_indexer_prolog_quant/pypto_src/build_output/kernels/aiv/incore_mul.ptoprojects/deepseek_v32_exp_lightning_indexer_prolog_quant/pypto_src/build_output/kernels/aiv/incore_neg.ptoprojects/deepseek_v32_exp_lightning_indexer_prolog_quant/pypto_src/build_output/kernels/aiv/incore_softmax.ptoprojects/deepseek_v32_exp_lightning_indexer_prolog_quant/pypto_src/build_output/kernels/aiv/incore_symmetric_quant.ptoprojects/deepseek_v32_exp_lightning_indexer_prolog_quant/pypto_src/build_output/orchestration/LightningIndexerPrologQuant.cppprojects/deepseek_v32_exp_lightning_indexer_prolog_quant/pypto_src/build_output/report/memory_after_AllocateMemoryAddr.txtprojects/deepseek_v32_exp_lightning_indexer_prolog_quant/pypto_src/golden.pyprojects/deepseek_v32_exp_lightning_indexer_prolog_quant/run_a2a3_swimlane.shprojects/paged-attention/paged_attention.pyqwen3_32b_run_log.md
💤 Files with no reviewable changes (123)
- examples/docs/deepseek_v3_2_memory_usage_unified_report.md
- projects/deepseek_v32_exp_lightning_indexer_prolog_quant/pypto_src/build_output/kernels/aiv/incore_softmax.pto
- examples/docs/deepseek_v3_2_mixed_kernel_local_usage_side_by_side.md
- projects/deepseek_v32_exp_lightning_indexer_prolog_quant/pypto_src/build_output/kernels/aiv/incore_add.pto
- junk_tensor_functions/tiling.py
- projects/deepseek_v32_exp_lightning_indexer_prolog_quant/pypto_src/build_output/report/memory_after_AllocateMemoryAddr.txt
- qwen3_32b_run_log.md
- examples/docs/deepseek_v3_2_aiv_splitting_analysis_pass08.md
- projects/deepseek_v32_exp_lightning_indexer_prolog_quant/PROJECT_ANALYSIS.md
- projects/deepseek_v32_exp_lightning_indexer_prolog_quant/pypto_src/build_output/kernels/aiv/incore_mul.pto
- junk_models/deepseek_v32_exp/sparse_attention_antiquant.py
- projects/deepseek_v32_exp_lightning_indexer_prolog_quant/pypto_src/build_output/kernels/aiv/incore_neg.pto
- projects/deepseek_v32_exp_lightning_indexer_prolog_quant/run_a2a3_swimlane.sh
- junk_models/qwen3_next/init.py
- junk_models/glm_v4_5/glm_moe_fusion.py
- examples/pa2.py
- examples/docs/deepseek_v3_2_fusion_scope_notes.md
- examples/pa1_build/passes_dump/05_after_InterchangeChunkLoops.py
- junk_models/deepseek_v32_exp/mla_prolog_quant.py
- examples/pa4_build/passes_dump/05_after_InterchangeChunkLoops.py
- projects/deepseek_v32_exp_lightning_indexer_prolog_quant/pypto_src/build_output/kernels/aic/incore_matmul_nn.pto
- projects/deepseek_v32_exp_lightning_indexer_prolog_quant/pypto_src/README.md
- examples/qwen3_32b_build/report/memory_after_AllocateMemoryAddr.txt
- junk_models/deepseek_v32_exp/sparse_flash_attention_quant_withscope.py
- examples/batch_hash_lookup.py
- junk_models/glm_v4_5/glm_matmul_allreduce_add_rmsnorm.py
- projects/deepseek_v32_exp_lightning_indexer_prolog_quant/pypto_src/build_output/kernels/aic/incore_matmul_acc.pto
- junk_tensor_functions/indexing.py
- examples/pa4.py
- examples/pa3_build/passes_dump/10_after_MemoryReuse.py
- junk_models/deepseek_v32_exp/lightning_indexer_quant.py
- design_goal.md
- examples/pa3_build/passes_dump/00_frontend.py
- projects/deepseek_v32_exp_lightning_indexer_prolog_quant/pypto_src/build_output/kernels/aiv/incore_symmetric_quant.pto
- examples/pa5_predicate_build/passes_dump/02_after_ConvertToSSA.py
- examples/pa5_predicate_build/passes_dump/01_after_UnrollLoops.py
- examples/pa3_build/passes_dump/03_after_FlattenCallExpr.py
- examples/pa3_build/passes_dump/04_after_SplitChunkedLoops.py
- examples/docs/seq_lens_handling_analysis.md
- examples/pa4_build/passes_dump/10_after_InitMemRef.py
- examples/pa1.py
- examples/pa.py
- examples/pa4_build/passes_dump/00_frontend.py
- projects/deepseek_v32_exp_lightning_indexer_prolog_quant/pypto_src/build_output/kernel_config.py
- projects/deepseek_v32_exp_lightning_indexer_prolog_quant/lightning_indexer_prolog_quant.py
- junk_tensor_functions/linalg.py
- examples/pa5_predicate_build/passes_dump/08_after_ExpandMixedKernel.py
- examples/docs/gen_mixed_kernel_local_usage.py
- junk_models/glm_v4_5/glm_gate.py
- examples/pa4_build/passes_dump/04_after_SplitChunkedLoops.py
- examples/pa5_predicate_build/passes_dump/00_frontend.py
- examples/pa5_predicate_build/passes_dump/06_after_RunVerifier.py
- junk_tensor_functions/type_layout.py
- examples/pa5_predicate_build/passes_dump/10_after_InitMemRef.py
- examples/pa3_build/passes_dump/11_after_InsertSync.py
- junk_tensor_functions/cast_primitives.py
- junk_models/glm_v4_5/glm_ffn_shared_expert_quant.py
- porting_to_pypto_v3.md
- examples/qwen3_32b_prefill_dump/report/prefill_kernel_local_tensor_summary.md
- junk_models/arctic/init.py
- projects/deepseek_v32_exp_lightning_indexer_prolog_quant/pypto_src/build_output/kernels/aic/incore_matmul.pto
- examples/paged_attention_example.py
- examples/pa1_build/passes_dump/07_after_OutlineIncoreScopes.py
- projects/deepseek_v32_exp_lightning_indexer_prolog_quant/pypto_src/golden.py
- junk_models/deepseek_v32_exp/sparse_flash_attention_quant.py
- junk_models/glm_v4_5/glm_ffn_common_interface.py
- junk_models/deepseek_v32_exp/init.py
- examples/pa1_build/passes_dump/06_after_RunVerifier.py
- examples/pa5_predicate_build/passes_dump/05_after_InterchangeChunkLoops.py
- junk_models/glm_v4_5/glm_attention_pre_quant.py
- examples/pa3_build/passes_dump/01_after_UnrollLoops.py
- examples/pa5_predicate_build/passes_dump/11_after_MemoryReuse.py
- examples/pa3_build/passes_dump/08_after_ConvertTensorToBlockOps.py
- examples/pa4_build/passes_dump/06_after_RunVerifier.py
- examples/pa5_predicate_build/passes_dump/07_after_OutlineIncoreScopes.py
- examples/pa4_build/passes_dump/02_after_ConvertToSSA.py
- examples/pa5_predicate_build/passes_dump/03_after_FlattenCallExpr.py
- junk_models/glm_v4_5/glm_attention_fusion.py
- examples/pa4_build/passes_dump/09_after_ConvertTensorToBlockOps.py
- examples/docs/qwen3_aiv_splitting_analysis_pass08.md
- projects/deepseek_v32_exp_lightning_indexer_prolog_quant/pypto_src/build_output/kernels/aiv/incore_layernorm.pto
- examples/docs/gen_front_capacity_budget.py
- examples/pa1_build/passes_dump/01_after_UnrollLoops.py
- projects/paged-attention/paged_attention.py
- junk_tensor_functions/elementwise_binary.py
- projects/deepseek_v32_exp_lightning_indexer_prolog_quant/pypto_src/build_output/orchestration/LightningIndexerPrologQuant.cpp
- examples/pa3_build/passes_dump/02_after_ConvertToSSA.py
- junk_models/glm_v4_5/glm_select_experts.py
- junk_models/arctic/sum_lstm.py
- junk_models/init.py
- junk_tensor_functions/reduction.py
- examples/pa1_build/passes_dump/00_frontend.py
- examples/pa3_build/passes_dump/09_after_InitMemRef.py
- examples/pa4_build/passes_dump/13_after_AllocateMemoryAddr.py
- examples/pa5_predicate_test.py
- deepseek_v3_2_code_review.md
- examples/pa4_build/passes_dump/03_after_FlattenCallExpr.py
- examples/pa3_build/passes_dump/12_after_AllocateMemoryAddr.py
- junk_models/glm_v4_5/init.py
- examples/pa4_build/passes_dump/12_after_InsertSync.py
- examples/pa4_build/passes_dump/07_after_OutlineIncoreScopes.py
- examples/softmax_example.py
- junk_tensor_functions/elementwise_unary.py
- examples/pa5_predicate_build/passes_dump/12_after_InsertSync.py
- examples/pa3_build/passes_dump/06_after_RunVerifier.py
- examples/pa5_predicate_build/passes_dump/04_after_SplitChunkedLoops.py
- junk_models/deepseek_v32_exp/mla_indexer_prolog_quant.py
- examples/pa1_build/passes_dump/04_after_SplitChunkedLoops.py
- junk_models/deepseek_v32_exp/lightning_indexer_prolog_quant.py
- junk_tensor_functions/composite.py
- examples/pa1_build/passes_dump/02_after_ConvertToSSA.py
- examples/pa3_build/passes_dump/05_after_InterchangeChunkLoops.py
- junk_tensor_functions/init.py
- examples/pa5_predicate_build/passes_dump/09_after_ConvertTensorToBlockOps.py
- junk_models/qwen3_next/gated_delta_rule.py
- examples/pa3_build/passes_dump/07_after_OutlineIncoreScopes.py
- examples/pa4_build/passes_dump/08_after_ExpandMixedKernel.py
- examples/pa3.py
- examples/pa1_build/passes_dump/03_after_FlattenCallExpr.py
- examples/pa5_predicate_build/passes_dump/13_after_AllocateMemoryAddr.py
- examples/pa4_build/passes_dump/01_after_UnrollLoops.py
- junk_models/glm_v4_5/glm_attention.py
- examples/pa4_build/passes_dump/11_after_MemoryReuse.py
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request significantly refactors the repository's file organization by removing obsolete components and centralizing documentation. The changes aim to enhance the clarity and maintainability of the codebase, making it easier for developers to navigate and understand the project's core elements. Highlights
Changelog
Activity
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
There was a problem hiding this comment.
Code Review
This pull request introduces a significant refactoring to clean up the repository structure. It removes a large number of archived directories, non-essential examples, and outdated documentation files, which greatly improves the project's maintainability. The core documentation has been centralized into a new docs/ directory. The corresponding path updates in configuration files like .claude/CLAUDE.md and issue templates appear to be correct and consistent with these changes. I have found one minor issue regarding a broken relative link within one of the moved documentation files. Given the large scale of this refactoring, it is crucial to ensure that all remaining examples and tests are functional and that no essential code has been inadvertently removed.
Summary
junk_models/,junk_tensor_functions/,projects/pypto-frontend-coding-style.md,para_for.md,pto2_rt.md) intodocs/design_goal.md,porting_to_pypto_v3.md,deepseek_v3_2_code_review.md,qwen3_32b_run_log.md).claude/CLAUDE.md, git-commit skill, and issue templateRetained
examples/: hello_world, qwen3 (4 files), deepseek (4 files)tests/: lint checksdocs/: 3 core reference documentsREADME.md,.claude/,.github/Testing
hello_world --simtest passes (no changes to hello_world.py or CI workflow)Summary by CodeRabbit
Removals
Documentation Updates