Skip to content

WIP: initialize v1.11 Gemma 4 E2B milestone #49

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
gabewillen wants to merge 1 commit into
base: main
Choose a base branch
from
+645 −980
Draft

WIP: initialize v1.11 Gemma 4 E2B milestone #49

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
169 changes: 96 additions & 73 deletions .planning/PROJECT.md
View file
Open in desktop

Large diffs are not rendered by default.

121 changes: 65 additions & 56 deletions .planning/REQUIREMENTS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,105 +1,114 @@
# Requirements: EMEL

**Defined:** 2026-03-31
**Defined:** 2026-04-02
**Core Value:** Prove real end-to-end behavior with explicit SML orchestration and parity-oriented
verification before widening API surface or model scope.

## v1 Requirements

### Fixture And Metadata

- [ ] **FIX-02**: The repo documents one official `LFM2.5-1.2B-Thinking-Q4_K_M.gguf` fixture with
- [ ] **FIX-03**: The repo documents one official `gemma-4-e2b-it-Q8_0.gguf` fixture with
checksum, source, stable maintained path under `tests/models/`, and download URL.
- [ ] **META-01**: The maintained Liquid slice records executable model truth from official
GGUF/config metadata, including `architecture=lfm2` and the official long-context setting,
instead of relying on stale prose-only metadata.
- [ ] **META-02**: The maintained Gemma 4 slice records executable model truth from official
GGUF/config metadata, including `architecture=gemma4`, `context_length=131072`, the text-layer
schedule, and the separate `mmproj` companion file, instead of relying on prose summaries.

### Conditioning

- [ ] **COND-03**: The maintained Liquid slice uses one explicit canonical request-conditioning
contract derived from the primary GGUF `tokenizer.chat_template`, with structured chat-message
input, `tools=none`, `add_generation_prompt=true`, and no implicit raw fallback.
- [ ] **COND-04**: The maintained Gemma 4 slice uses one explicit canonical text-only
request-conditioning contract derived from the official `chat_template`, with structured text
chat-message input, `add_generation_prompt=true`, and no implicit raw fallback.
- [ ] **COND-05**: The maintained Gemma 4 text slice rejects `mmproj`, image, audio, video, and
tool-call request shapes explicitly instead of implying multimodal or tool-use support.

### Runtime Support

- [ ] **RUN-03**: EMEL-owned `src/emel` runtime code truthfully accepts the canonical Liquid
fixture as `lfm2` instead of rejecting it or aliasing it to existing `llama`/`qwen3` paths.
- [ ] **RUN-04**: EMEL can initialize and generate on the canonical Liquid fixture through the
- [ ] **RUN-07**: EMEL-owned `src/emel` runtime code truthfully accepts the canonical Gemma 4
fixture as `gemma4` instead of rejecting it or aliasing it to existing `llama`, `qwen3`, or
`lfm2` paths.
- [ ] **RUN-08**: EMEL can initialize and generate on the canonical Gemma 4 fixture through the
maintained generator path in `src/emel`.
- [ ] **RUN-05**: The maintained Liquid runtime explicitly handles the canonical slice's required
`lfm2` metadata, tensor names, and hybrid block contract instead of pretending broad
Llama-family compatibility.
- [ ] **RUN-06**: The maintained Liquid runtime publishes a truthful quantized-path contract for
the chosen official fixture and does not claim sibling Liquid quant support that is not proven.
- [ ] **RUN-09**: The maintained Gemma 4 runtime explicitly handles the required `gemma4`
metadata, alternating sliding/full attention schedule, shared-KV layers, rope contracts, and
tied embeddings instead of pretending broad compatibility with an existing family.

### Reference Boundary

- [ ] **REF-01**: The pinned `llama.cpp` reference lane used by `tools/paritychecker` and
`tools/bench` is upgraded or confirmed so the canonical Gemma 4 fixture can be loaded for
maintained comparison.

### Verification

- [ ] **PAR-02**: `tools/paritychecker --generation` proves EMEL against `llama.cpp` on the
canonical Liquid slice using the same fixture and conditioning contract.
- [ ] **VER-02**: Regression tests cover the canonical Liquid slice and protect the prior
maintained Llama and Qwen anchors from accidental breakage while support is widened.
- [ ] **PAR-03**: `tools/paritychecker --generation` proves EMEL against `llama.cpp` on the
canonical Gemma 4 text slice using the same fixture and conditioning contract.
- [ ] **VER-03**: Regression tests cover the canonical Gemma 4 text slice and protect the prior
maintained Llama, Qwen, and Liquid anchors from accidental breakage while support is widened.

### Benchmarking

- [ ] **BENCH-08**: `tools/bench` compare output, stored benchmark evidence, and generated docs
publish one truthful canonical Liquid benchmark path aligned with the parity-checked slice.
- [ ] **BENCH-09**: `tools/bench` compare output, stored benchmark evidence, and generated docs
publish one truthful canonical Gemma 4 benchmark path aligned with the parity-checked text-only
slice.

## v2 Requirements

### Broader Liquid Coverage
### Multimodal Support

- **MODEL-03**: Broaden beyond the canonical `LFM2.5-1.2B-Thinking-Q4_K_M.gguf` fixture to
additional Liquid checkpoints or sibling quantizations once the first slice is proven and
benchmarked.
- **MODAL-01**: Add real `mmproj` support plus image input handling for the official Gemma 4 E2B
path once the text slice is proven.
- **MODAL-02**: Add audio/video request support only after the repo has an explicit media encoder
pipeline and maintained multimodal verification.

### Prompt Richness

- **COND-04**: Add richer Liquid system-message, multi-turn thinking-history, or tool-calling
request surfaces only after the canonical maintained slice has an explicit and stable
conditioning contract.
- **COND-06**: Add tool use or broader official template features only after the canonical text
slice has a stable maintained contract.

### Performance
### Broader Model Coverage

- **GEN-04**: Optimize Liquid-specific runtime hot spots after the canonical Liquid slice is
correct, parity-backed, and benchmarked.
- **MODEL-04**: Broaden beyond the canonical `gemma-4-e2b-it-Q8_0.gguf` fixture to `F16` or later
official Gemma 4 variants only after the first text slice is proven and benchmarked.

### Benchmark Policy
### Performance

- **BENCH-07**: Revisit whether noisy benchmark drift should become a blocking repo gate once the
maintained compare surfaces are stable enough to justify it.
- **PERF-01**: Optimize Gemma 4-specific runtime hot spots after the canonical text slice is
correct, parity-backed, and benchmarked.

## Out of Scope

| Feature | Reason |
|---------|--------|
| Broad Liquid-family or multi-model support | Keep v1.9 fixed to one maintained Liquid Thinking GGUF slice first. |
| Sibling Liquid quantizations such as `Q4_0`, `Q5_K_M`, `Q6_K`, `Q8_0`, `BF16`, or `F16` | The first milestone needs one exact proven fixture, not a quant matrix. |
| Tool use or function calling | Liquid documents tool use, but that widens request shape and API scope beyond the first maintained slice. |
| Multi-turn thinking-history replay | The official template supports `keep_past_thinking`, but v1.9 should stay on one narrow maintained request contract. |
| Raw prompt fallback on the maintained Liquid path | Maintained proof should follow one explicit structured contract, not silent raw formatting. |
| Decode extraction or broader generator decomposition | That work is being pursued separately and is not part of this branch's milestone. |
| Benchmark gate hardening | Warning-only benchmark debt is separate from proving the first Liquid slice. |
| `mmproj-gemma-4-e2b-it-f16.gguf` | Keep `v1.11` on one truthful text-generation slice first. |
| Image, audio, or video inputs | The upstream model is multimodal, but EMEL does not yet have a maintained multimodal acceptance surface. |
| Tool use or function calling | The official template supports tools, but that widens request shape and API scope beyond the first maintained text slice. |
| `gemma-4-e2b-it-f16.gguf` as a maintained slice | The first milestone needs one exact proven text fixture, not a mixed precision matrix. |
| Raw prompt fallback on the maintained Gemma 4 path | Maintained proof should follow one explicit structured contract, not silent raw formatting. |
| Re-planning adjacent `v1.8` size-benchmark work | Gemma 4 should integrate with existing maintained benchmark seams, not absorb unrelated benchmark-surface planning. |
| Inflight `v2.0` Bonsai `Q1` kernel work | That kernel work is separate adjacent scope and is not part of Gemma 4 milestone acceptance. |

## Traceability

| Requirement | Phase | Status |
|-------------|-------|--------|
| FIX-02 | Phase 33 | Pending |
| META-01 | Phase 33 | Pending |
| COND-03 | Phase 33 | Pending |
| RUN-03 | Phase 34 | Pending |
| RUN-04 | Phase 35 | Pending |
| RUN-05 | Phase 34 | Pending |
| RUN-06 | Phase 35 | Pending |
| PAR-02 | Phase 36 | Pending |
| VER-02 | Phase 36 | Pending |
| BENCH-08 | Phase 37 | Pending |
| FIX-03 | Phase 38 | Pending |
| META-02 | Phase 38 | Pending |
| COND-04 | Phase 38 | Pending |
| COND-05 | Phase 39 | Pending |

@gabewillen gabewillen Jul 8, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[suggestion] COND-05 (explicit reject of mmproj/image/audio/video/tool-call shapes) is deferred to Phase 39, while Phase 38 only locks the positive text-only contract (COND-04). PROJECT constraints say the maintained path must reject unsupported media shapes explicitly, and Pitfall 2 maps false multimodal claims to Phases 38 and 39. As written, Phase 38 can complete without any rejection success criterion, leaving a window where docs claim text-only honesty without a reject surface.

Suggestion: Move at least a minimal reject-contract requirement into Phase 38 (document/test that unsupported media/tool request shapes are out of contract), or add a Phase 38 success criterion that the maintained formatter/conditioning contract defines explicit rejection outcomes even if model-gate wiring lands in Phase 39.

| RUN-07 | Phase 39 | Pending |
| RUN-08 | Phase 40 | Pending |
| RUN-09 | Phase 39 | Pending |
| REF-01 | Phase 41 | Pending |

@gabewillen gabewillen Jul 8, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[bug] Traceability maps REF-01 -> Phase 41 only, so requirement tracking encodes the same late-ref ordering bug as the roadmap. FEATURES dependency graph correctly shows parity requires reference-lane readiness, but does not force that readiness before runtime bring-up; REQUIREMENTS/ROADMAP therefore diverge from the pitfall mitigation the research itself recommends.

Suggestion: Remap REF-01 to an early phase (38 or 39), or split into REF-01a (pin audit/upgrade before runtime) and REF-01b (parity/bench consumers use the upgraded pin). Update ROADMAP success criteria and STATE “roadmap coverage” bullets to match.

| PAR-03 | Phase 41 | Pending |
| VER-03 | Phase 41 | Pending |
| BENCH-09 | Phase 42 | Pending |

**Coverage:**
- v1 requirements: 10 total
- Mapped to phases: 10
- v1 requirements: 11 total
- Mapped to phases: 11
- Unmapped: 0

---
*Requirements defined: 2026-03-31*
*Last updated: 2026-03-31 after roadmap creation*
*Requirements defined: 2026-04-02*
*Last updated: 2026-04-02 after roadmap creation*
Loading