Skip to content

WIP: initialize v1.11 Gemma 4 E2B milestone#49

Draft
gabewillen wants to merge 1 commit into
mainfrom
feat/gemma4
Draft

WIP: initialize v1.11 Gemma 4 E2B milestone#49
gabewillen wants to merge 1 commit into
mainfrom
feat/gemma4

Conversation

@gabewillen

Copy link
Copy Markdown
Contributor

Summary

  • initialize milestone planning for v1.11 Gemma 4 E2B Text Generation Slice
  • replace the stale Liquid-focused active planning docs with Gemma 4 planning and research
  • define the new requirements, roadmap, and state transition for the branch

Included

  • .planning/PROJECT.md
  • .planning/REQUIREMENTS.md
  • .planning/ROADMAP.md
  • .planning/STATE.md
  • .planning/research/*

Milestone Shape

  • maintained fixture: gemma-4-e2b-it-Q8_0.gguf
  • maintained scope: text generation only
  • explicit out of scope for this milestone: mmproj, image/audio/video inputs, and tool-use surfaces
  • explicit requirement to verify or update the pinned llama.cpp reference lane before Gemma 4 parity/bench claims

Next

  • Phase 38: Fixture, Metadata, And Text-Only Contract Lock
  • follow-up command: $gsd-discuss-phase 38

Notes

  • planning/docs only in this PR
  • scripts/quality_gates.sh was started during planning validation but not carried through to a completed result in this PR context

@gabewillen gabewillen left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

Planning-only PR that rescopes active .planning/ docs from the Liquid v1.9 slice to a narrow Gemma 4 E2B text-generation milestone (v1.11, Phases 38–42). Scope honesty (text-only, no mmproj/tools, explicit gemma4 path, reference-pin risk) is generally strong and metadata claims look coherent. The main merge risks are internal planning contradictions (especially REF-01 scheduled after the runtime phase while pitfalls say never do that), under-specified model-family placement (model/data.* only, no model/gemma4 contract surface), and phase/requirement traceability gaps that would steer implementation the wrong way if followed literally.

Issue counts by severity

  • bugs: 2
  • suggestions: 6
  • nits: 2

Comment thread .planning/ROADMAP.md
**Goal**: The reference lane is Gemma 4-capable, the exact maintained text slice is proven
correct against the reference, and the existing maintained anchors stay green.
**Depends on**: Phase 40
**Requirements**: REF-01, PAR-03, VER-03

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[bug] REF-01 (Gemma 4-capable pinned llama.cpp reference lane) is required only in Phase 41, which depends on Phase 40 runtime bring-up. The same PR’s pitfalls research marks “Ignore the pinned reference commit until parity phase” as never acceptable (.planning/research/PITFALLS.md:107) and Architecture Pattern 4 / Anti-Pattern 3 warn that discovering a non-Gemma4 pin after runtime work is a late dead-end. Following the roadmap as written recreates the documented failure mode: full EMEL runtime work can complete before anyone learns parity/bench cannot load the fixture.

Suggestion: Promote reference-pin audit/upgrade to Phase 38 or a dedicated early phase (or make Phase 40 depend on a completed REF-01 gate). Keep PAR-03/VER-03 in the later proof phase, but do not defer the pin readiness check until after runtime.

Comment thread .planning/REQUIREMENTS.md
| RUN-07 | Phase 39 | Pending |
| RUN-08 | Phase 40 | Pending |
| RUN-09 | Phase 39 | Pending |
| REF-01 | Phase 41 | Pending |

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[bug] Traceability maps REF-01 -> Phase 41 only, so requirement tracking encodes the same late-ref ordering bug as the roadmap. FEATURES dependency graph correctly shows parity requires reference-lane readiness, but does not force that readiness before runtime bring-up; REQUIREMENTS/ROADMAP therefore diverge from the pitfall mitigation the research itself recommends.

Suggestion: Remap REF-01 to an early phase (38 or 39), or split into REF-01a (pin audit/upgrade before runtime) and REF-01b (parity/bench consumers use the upgraded pin). Update ROADMAP success criteria and STATE “roadmap coverage” bullets to match.

Liquid should be added here additively, not via a new harness.
| `tests/models/README.md` + tool constants | Pin one official Gemma 4 truth anchor | Stable path, checksum, download URL, and slug for one maintained file |
| `tools/generation_formatter_contract.hpp` | Resolve one canonical Gemma 4 text contract | Match one supported template subset with no media and no tools |
| `src/emel/model/data.*` | Accept and validate `gemma4` metadata/tensor contracts | Explicit architecture gate plus Gemma 4-specific topology handling |

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[suggestion] Architecture guidance places the Gemma 4 model contract primarily in src/emel/model/data.* and execution in src/emel/generator/..., with no dedicated src/emel/model/gemma4/** family surface. Repo engineering rules keep model-family contracts under model/<family> (not bulk-only data.*, not a top-level gemma4 domain). STATE already claims maintained lfm2 support exists, so a family-local module pattern should be the research default; otherwise implementers are steered toward growing data.* / generator blobs and risk domain-boundary or “runtime choice in detail” violations later.

Suggestion: Explicitly prescribe src/emel/model/gemma4/** (or equivalent family-local contract modules) for topology/metadata validation, with model/data.* only as the architecture gate/dispatch into that contract. Mirror the same wording in STACK.md and SUMMARY.md “Major components.”

| `tools/paritychecker/parity_runner.cpp` | Add one official Liquid maintained fixture path/slug and `lfm2` architecture validation | This keeps parity truthful on one explicit asset instead of broadening to generic Liquid support. |
| `tools/bench/generation_bench.cpp` and [`tools/bench/bench_cases.hpp`](/Users/gabrielwillen/.superset/worktrees/emel.cpp/feat/liquid-ai/tools/bench/bench_cases.hpp) | Add one Liquid benchmark case family using the same maintained fixture | This preserves the existing compare/docs workflow with one new case set instead of a new toolchain. |
| [`tests/models/README.md`](/Users/gabrielwillen/.superset/worktrees/emel.cpp/feat/liquid-ai/tests/models/README.md) | Record official model provenance for the maintained Liquid fixture | The milestone should have the same provenance discipline as Qwen and Llama. |
| `src/emel/model/data.*` | Add `gemma4` to supported execution architectures and define a Gemma 4 text contract | Current code explicitly supports maintained families such as `llama`, `qwen3`, and `lfm2`; the new fixture reports `architecture=gemma4`. |

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[suggestion] Same placement gap as Architecture: “Required Additions In EMEL” lists src/emel/model/data.* for the Gemma 4 text contract and src/emel/generator/... for runtime path, but never names a family-owned model module. That is incomplete scaffolding relative to AGENTS domain rules and will matter as soon as Phase 39 implementation starts from this research.

Suggestion: Add a row for src/emel/model/gemma4/** (metadata, tensor/topology contract, text-only validation helpers) and describe data.* as the architecture allow-list / binding site only.

Comment thread .planning/ROADMAP.md
produces bounded text generation on ARM.
2. Runtime execution for the maintained Gemma 4 slice uses the explicit `gemma4` path and the
maintained text-only acceptance surface rather than broad multimodal or family claims.
3. Maintained runtime evidence publishes a truthful quantized-path contract for the official

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[suggestion] Phase 40 success criterion 3 requires publishing “a truthful quantized-path contract for the official Q8_0 fixture only,” but Phase 40 requirements list only RUN-08 (initialize/generate). There is no RUN/META requirement that owns quantized-path evidence, so closeout can claim criterion 3 without a mapped requirement, or implementers can skip the contract proof while still checking RUN-08.

Suggestion: Either add a Phase 40 requirement (e.g. RUN-10) for the Q8_0 quantized-path/publication contract, or drop success criterion 3 and fold any needed quant-boundary language into RUN-08/RUN-09 text.

Comment thread .planning/ROADMAP.md
**Phase Numbering:**
- Integer phases continue across milestones.
- v1.9 starts at Phase 33 because v1.7 ended at Phase 32.
- `v1.11` starts at Phase 38 because the current visible roadmap ceiling in `.planning/phases/` is

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[suggestion] Milestone phase numbering restarts at Phase 38 because “the current visible roadmap ceiling in .planning/phases/ is Phase 37,” while PROJECT/STATE simultaneously treat adjacent v1.8 size-benchmark work as real concurrent context. In this repo’s planning history, Phase 38+ is already used by adjacent milestone closeout bookkeeping. Reusing 38–42 while those streams remain “adjacent” creates phase-ID collisions when branches merge or when .planning/phases/ directories are reconciled.

Suggestion: Pick the next free global phase ID after all adjacent in-flight milestones (including closeout phases), or namespace phase dirs by milestone (v1.11-phases/38-... style) and document the collision policy in ROADMAP phase-numbering notes.

Comment thread .planning/REQUIREMENTS.md
| FIX-03 | Phase 38 | Pending |
| META-02 | Phase 38 | Pending |
| COND-04 | Phase 38 | Pending |
| COND-05 | Phase 39 | Pending |

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[suggestion] COND-05 (explicit reject of mmproj/image/audio/video/tool-call shapes) is deferred to Phase 39, while Phase 38 only locks the positive text-only contract (COND-04). PROJECT constraints say the maintained path must reject unsupported media shapes explicitly, and Pitfall 2 maps false multimodal claims to Phases 38 and 39. As written, Phase 38 can complete without any rejection success criterion, leaving a window where docs claim text-only honesty without a reject surface.

Suggestion: Move at least a minimal reject-contract requirement into Phase 38 (document/test that unsupported media/tool request shapes are out of contract), or add a Phase 38 success criterion that the maintained formatter/conditioning contract defines explicit rejection outcomes even if model-gate wiring lands in Phase 39.

execute the maintained Liquid slice truthfully.
4. `tools/paritychecker` and `tools/bench`: prove and publish only the same fixture and contract
after runtime support is real.
canonical Gemma 4 text contract; `src/emel/model/data.*` and the generator runtime own explicit

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[suggestion] Summary still states that src/emel/model/data.* and “the generator runtime” own explicit gemma4 acceptance and topology handling, and the phase write-up puts reference readiness only in Phase 41 after Phases 39–40 model/runtime work. That repeats the Architecture placement gap and the late-REF ordering bug in the document most people will skim before planning phases.

Suggestion: Rewrite the architecture/phase paragraphs to (1) name model/gemma4 family ownership, (2) schedule ref-pin readiness before runtime/parity, and (3) restore a short “gaps / confidence” section so known unknowns (exact rope/shared-KV runtime mapping, pin upgrade strategy) are not dropped relative to the Liquid research summary.


**Why it happens:**
Benchmark rows are visible and easy to demo, while parity/runtime bring-up is slower.
Benchmark rows are visible and easy to demo, while reference/runtimе parity is slower.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] Typo uses a Cyrillic “е” in runtimе (reference/runtimе parity), which can break search and look like mojibake in review tooling.

Suggestion: Replace with ASCII runtime.

| `mmproj-gemma-4-e2b-it-f16.gguf` in `v1.11` | It implies image/media execution that the repo cannot currently prove | Keep `v1.11` text only |
| Broad "Gemma 4 support" language | It sounds stronger but hides the multimodal boundary | Name one exact maintained text fixture |
| Tool use or media placeholders on the maintained path | The official template allows them, but EMEL does not yet have maintained support | Reject those request shapes explicitly |
| Assuming the pinned `llama.cpp` reference lane already supports Gemma 4 | The current pinned commit appears not to contain `gemma4` | Make ref-pin readiness explicit in the milestone |

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] Soft claim language “appears not to contain gemma4” is repeated across PROJECT/STATE/research while the pin SHA is known (ecbcb7ea9d3303097519723b264a8b5f1e977028) and a pin-specific raw source URL is already listed. For a milestone that treats ref readiness as first-class, the research should state a hard verified result (present/absent symbols) rather than hedging.

Suggestion: Record an explicit verification result for that commit (e.g. no LLM_ARCH_GEMMA4 / gemma4 architecture path in pin tree) and only then keep the upgrade requirement.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant