feat(catalog): Gemma 4 E4B/E2B uncensored GGUF models (#1836)#1837
Conversation
Adds CPU/llama.cpp + Ollama GGUF catalog entries for the uncensored (abliterated) Gemma 4 E4B and E2B variants (mradermacher uploads). These are GGUF and do NOT run on the Hailo NPU. Multimodal (text/image/video/audio). sha256 left as TODO per xet-backed repos. Docs-Reviewed: add Gemma 4 E4B/E2B uncensored GGUF catalog entries (CPU path) per #1836
Qodo reviews are paused for this user.Troubleshooting steps vary by plan Learn more → On a Teams plan? Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center? |
📝 WalkthroughWalkthroughAdds catalog manifests for Gemma 4 E2B and E4B uncensored GGUF models, including metadata, quantized variants, backend requirements, download information, and hardware-tier recommendations. ChangesGemma GGUF manifests
Estimated code review effort: 2 (Simple) | ~10 minutes 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![kilo-code-bot[bot]](https://avatars.githubusercontent.com/in/2193792?s=60&v=4){kind=small}
| name: Gemma 4 E2B Uncensored (GGUF) | ||
| type: model | ||
| version: 4.0.0 | ||
| description: "Google Gemma 4 E2B (~5B raw / 2.3B effective via PLE), abliterated/uncensored GGUF (no refusals, original Google behaviour). Natively multimodal: text, image, video and audio in one model (mmproj needed for vision/audio). CPU/llama.cpp + Ollama path only; does NOT run on the Hailo NPU." |
There was a problem hiding this comment.
WARNING: context_window: 131072 differs from the sibling Gemma 4 GGUF manifests (which use 32768). Gemma 4 E2B/E4B base context is 32K (with 128K window available via rope scaling in some builds). Confirm the 128K value is valid for these specific mradermacher GGUF uploads; if wrong, this can cause oversized KV-cache allocation/oom. The non-uncensored gemma-4-e2b-gguf uses 32768, so this inconsistency should be reconciled.
Reply with @kilocode-bot fix it to have Kilo Code address this issue.
| description: "Google Gemma 4 E2B (~5B raw / 2.3B effective via PLE), abliterated/uncensored GGUF (no refusals, original Google behaviour). Natively multimodal: text, image, video and audio in one model (mmproj needed for vision/audio). CPU/llama.cpp + Ollama path only; does NOT run on the Hailo NPU." | ||
| homepage: https://huggingface.co/mradermacher/gemma-4-E2B-it-uncensored-GGUF | ||
| license: "Gemma Terms of Use" | ||
| capabilities: [chat, tool-calling, vision, audio] |
There was a problem hiding this comment.
WARNING: capabilities: [chat, tool-calling, vision, audio] advertises multimodal vision/audio, but the manifest provides no mmproj (multimodal projector) download in either variant. The description itself notes "mmproj needed for vision/audio". Without the projector file, these capabilities cannot be fulfilled and UIs may offer vision/audio that silently fails. Either add the mmproj GGUF as a separate asset/variant, or drop vision/audio from capabilities until it's provided.
Reply with @kilocode-bot fix it to have Kilo Code address this issue.
| # TODO: sha256 not yet resolved (xet-backed repo, no LFS pointer). Fill in before release. | ||
| sha256: '' | ||
| requires: | ||
| backends: |
There was a problem hiding this comment.
WARNING: sha256: '' (empty) leaves both variants without integrity verification. The PR documents this as a deliberate TODO for xet-backed repos, but shipping a release manifest with no checksum means downloads are unverified and silently corrupted/partial files are accepted. Add the real sha256 before release, or gate the manifest behind a draft/experimental flag if your schema supports it.
Reply with @kilocode-bot fix it to have Kilo Code address this issue.
| arm-npu-16gb: {recommended: q4_k_m} | ||
| arm-npu-32gb: {recommended: q4_k_m} | ||
| apple-silicon: {recommended: q4_k_m} | ||
| x86-cuda-12gb: {recommended: q4_k_m} |
There was a problem hiding this comment.
SUGGESTION: The E2B manifest's hardware_tiers omits the 6GB tiers (arm-npu-6gb, x86-cuda-6gb, x86-vulkan-6gb) that the E4B manifest includes. Users on 6GB-class hardware get no recommendation for the smaller E2B model. Add the 6GB tier entries (recommending q3_k_m / q4_k_m as appropriate) for parity, since E2B's min_ram is only 4096MB.
Reply with @kilocode-bot fix it to have Kilo Code address this issue.
| name: Gemma 4 E4B Uncensored (GGUF) | ||
| type: model | ||
| version: 4.0.0 | ||
| description: "Google Gemma 4 E4B (~7.5B raw / 4B effective via PLE), abliterated/uncensored GGUF (no refusals, original Google behaviour). Natively multimodal: text, image, video and audio in one model (mmproj needed for vision/audio). CPU/llama.cpp + Ollama path only; does NOT run on the Hailo NPU." |
There was a problem hiding this comment.
WARNING: context_window: 131072 differs from the sibling gemma-4-e4b-gguf manifest (32768). Confirm the 128K context is valid for these mradermacher GGUF uploads; a wrong value can cause oversized KV-cache allocation/oom. Reconcile with the non-uncensored E4B entry.
Reply with @kilocode-bot fix it to have Kilo Code address this issue.
| description: "Google Gemma 4 E4B (~7.5B raw / 4B effective via PLE), abliterated/uncensored GGUF (no refusals, original Google behaviour). Natively multimodal: text, image, video and audio in one model (mmproj needed for vision/audio). CPU/llama.cpp + Ollama path only; does NOT run on the Hailo NPU." | ||
| homepage: https://huggingface.co/mradermacher/gemma-4-E4B-it-uncensored-GGUF | ||
| license: "Gemma Terms of Use" | ||
| capabilities: [chat, tool-calling, vision, audio] |
There was a problem hiding this comment.
WARNING: capabilities: [chat, tool-calling, vision, audio] advertises multimodal vision/audio, but no mmproj projector GGUF is provided in either variant. The description notes "mmproj needed for vision/audio". Without the projector file these capabilities cannot be fulfilled. Add the mmproj asset or drop vision/audio until provided.
Reply with @kilocode-bot fix it to have Kilo Code address this issue.
| # TODO: sha256 not yet resolved (xet-backed repo, no LFS pointer). Fill in before release. | ||
| sha256: '' | ||
| requires: | ||
| backends: |
There was a problem hiding this comment.
WARNING: sha256: '' (empty) leaves both variants without integrity verification. Documented as a deliberate TODO, but a release manifest with no checksum accepts silently corrupted/partial downloads. Add the real sha256 before release.
Reply with @kilocode-bot fix it to have Kilo Code address this issue.
Code Review SummaryStatus: 7 Issues Found | Recommendation: Address before merge Overview
Issue Details (click to expand)WARNING
SUGGESTION
Files Reviewed (2 files)
Fix these issues in Kilo Cloud Reviewed by hy3:free · Input: 38.3K · Output: 3.7K · Cached: 129.3K |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@app-catalog/models/gemma-4-e2b-uncensored-gguf/manifest.yaml`:
- Line 8: Remove vision and audio from the capabilities lists in both
app-catalog/models/gemma-4-e2b-uncensored-gguf/manifest.yaml:8-8 and
app-catalog/models/gemma-4-e4b-uncensored-gguf/manifest.yaml:8-8, leaving only
capabilities supported by the downloaded base text GGUF files.
- Around line 24-26: Add apple-silicon and x86-cuda to the targets list for
every llama-cpp entry: Q4_K_M and Q3_K_M in
app-catalog/models/gemma-4-e2b-uncensored-gguf/manifest.yaml at lines 24-26 and
40-42, and Q4_K_M and Q3_K_M in
app-catalog/models/gemma-4-e4b-uncensored-gguf/manifest.yaml at lines 24-26 and
40-42; retain the existing x86-vulkan, arm-vulkan, and cpu targets.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: 256ddcec-1a40-4d7e-9b23-bb29b887eb4f
📒 Files selected for processing (2)
app-catalog/models/gemma-4-e2b-uncensored-gguf/manifest.yamlapp-catalog/models/gemma-4-e4b-uncensored-gguf/manifest.yaml
| description: "Google Gemma 4 E2B (~5B raw / 2.3B effective via PLE), abliterated/uncensored GGUF (no refusals, original Google behaviour). Natively multimodal: text, image, video and audio in one model (mmproj needed for vision/audio). CPU/llama.cpp + Ollama path only; does NOT run on the Hailo NPU." | ||
| homepage: https://huggingface.co/mradermacher/gemma-4-E2B-it-uncensored-GGUF | ||
| license: "Gemma Terms of Use" | ||
| capabilities: [chat, tool-calling, vision, audio] |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Remove multimodal capabilities until the mmproj projector is included.
The manifests claim vision and audio capabilities, and the description correctly notes that an mmproj file is needed for them. However, the variants only download the base text .gguf file. Without the required projector file, backends will fail to process image or audio inputs.
app-catalog/models/gemma-4-e2b-uncensored-gguf/manifest.yaml#L8-L8: Removevisionandaudiofrom thecapabilitieslist.app-catalog/models/gemma-4-e4b-uncensored-gguf/manifest.yaml#L8-L8: Removevisionandaudiofrom thecapabilitieslist.
🛠️ Proposed fix (apply to both files)
-capabilities: [chat, tool-calling, vision, audio]
+capabilities: [chat, tool-calling]📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| capabilities: [chat, tool-calling, vision, audio] | |
| capabilities: [chat, tool-calling] |
📍 Affects 2 files
app-catalog/models/gemma-4-e2b-uncensored-gguf/manifest.yaml#L8-L8(this comment)app-catalog/models/gemma-4-e4b-uncensored-gguf/manifest.yaml#L8-L8
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@app-catalog/models/gemma-4-e2b-uncensored-gguf/manifest.yaml` at line 8,
Remove vision and audio from the capabilities lists in both
app-catalog/models/gemma-4-e2b-uncensored-gguf/manifest.yaml:8-8 and
app-catalog/models/gemma-4-e4b-uncensored-gguf/manifest.yaml:8-8, leaving only
capabilities supported by the downloaded base text GGUF files.
| - id: llama-cpp | ||
| targets: [x86-vulkan, arm-vulkan, cpu] | ||
| min_ram_mb: 4096 |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Restore missing apple-silicon and x86-cuda targets for llama-cpp.
The llama-cpp backend configuration omits apple-silicon and x86-cuda from its targets, despite these being natively supported by llama.cpp and explicitly enabled for the ollama backend on the same models. This omission will incorrectly prevent Mac and Nvidia users from utilizing the llama-cpp backend for these variants.
app-catalog/models/gemma-4-e2b-uncensored-gguf/manifest.yaml#L24-L26: Addapple-siliconandx86-cudato the Q4_K_Mllama-cpptargets list.app-catalog/models/gemma-4-e2b-uncensored-gguf/manifest.yaml#L40-L42: Addapple-siliconandx86-cudato the Q3_K_Mllama-cpptargets list.app-catalog/models/gemma-4-e4b-uncensored-gguf/manifest.yaml#L24-L26: Addapple-siliconandx86-cudato the Q4_K_Mllama-cpptargets list.app-catalog/models/gemma-4-e4b-uncensored-gguf/manifest.yaml#L40-L42: Addapple-siliconandx86-cudato the Q3_K_Mllama-cpptargets list.
🛠️ Proposed fix (apply to all 4 locations)
- id: llama-cpp
- targets: [x86-vulkan, arm-vulkan, cpu]
+ targets: [apple-silicon, x86-cuda, x86-vulkan, arm-vulkan, cpu]📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| - id: llama-cpp | |
| targets: [x86-vulkan, arm-vulkan, cpu] | |
| min_ram_mb: 4096 | |
| - id: llama-cpp | |
| targets: [apple-silicon, x86-cuda, x86-vulkan, arm-vulkan, cpu] | |
| min_ram_mb: 4096 |
📍 Affects 2 files
app-catalog/models/gemma-4-e2b-uncensored-gguf/manifest.yaml#L24-L26(this comment)app-catalog/models/gemma-4-e2b-uncensored-gguf/manifest.yaml#L40-L42app-catalog/models/gemma-4-e4b-uncensored-gguf/manifest.yaml#L24-L26app-catalog/models/gemma-4-e4b-uncensored-gguf/manifest.yaml#L40-L42
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@app-catalog/models/gemma-4-e2b-uncensored-gguf/manifest.yaml` around lines 24
- 26, Add apple-silicon and x86-cuda to the targets list for every llama-cpp
entry: Q4_K_M and Q3_K_M in
app-catalog/models/gemma-4-e2b-uncensored-gguf/manifest.yaml at lines 24-26 and
40-42, and Q4_K_M and Q3_K_M in
app-catalog/models/gemma-4-e4b-uncensored-gguf/manifest.yaml at lines 24-26 and
40-42; retain the existing x86-vulkan, arm-vulkan, and cpu targets.
1 participant
Summary
gemma-4-e4b-uncensored-gguf(Q4_K_M + Q3_K_M)gemma-4-e2b-uncensored-gguf(Q4_K_M + Q3_K_M)llama-cppandollama(CPU path). They do NOT run on the Hailo NPU, sork-llama/hailobackends are intentionally excluded.Notes
sha256is left as an empty''with a clear TODO comment: the source repos are xet-backed and expose no LFS pointer, so a real sha256 could not be resolved without downloading multi-GB blobs. No fake values were inserted.python3 scripts/check_manifests.py managed-lintpasses.Docs-Reviewed: add Gemma 4 E4B/E2B uncensored GGUF catalog entries (CPU path) per #1836
Summary by CodeRabbit