Création d'un agent d'autonomie systémique : OpenClaw + Gemma 4 et TurboQuant sur Raspberry Pi 4B

[doc62fr]

Bonjour

Je viens de découvrir cela:

https://dev.to/kheai/building-a-systemic-autonomy-agent-openclaw-gemma-4-turboquant-on-raspberry-pi-4b-449l

Je pense que l 'intégration à taOS serait une bonne chose.

[doc62fr]

Voir aussi une éventuelle évolution vers le Raspberry Pi 5...

[doc62fr]

Petit complément d'information :

https://github.com/ericcurtin/inferrs

Merci de vos retours

[jaylfc]

Merci doc62fr. Two good things here. First, Gemma 4 E2B is already in the taOS catalog (GGUF), and the catalog already picks the right variant per hardware tier, so on the CPU path it is one-tap.

But for your setup specifically, a Pi 5 with the Hailo-10H, the more interesting path is not the CPU/TurboQuant route from that Pi 4B article (that technique is mainly for Pis with no accelerator). It is running a Gemma-class model on your Hailo NPU through the hailo-ollama backend we are wiring up in #1771. That offloads to the accelerator and sidesteps the RAM squeeze entirely.

So the concrete question: is Gemma 4 E2B one of the 5 models you already have running on the Hailo, and would you want a Gemma .hef entry added to the Hailo model catalog (that is part of the Hailo backend work)? If a compatible .hef exists we can fold it in as a first-class Hailo catalog model. TurboQuant and inferrs stay useful as the CPU-only fallback for accelerator-less boards.

[jaylfc]

Quick update after digging into the Hailo GenAI Model Zoo. Two useful findings:

1. Gemma is not available as a Hailo .hef. The official Hailo GenAI Model Zoo currently ships 6 LLM .hef models for the 10H, all Qwen family plus Llama and DeepSeek: Qwen2.5 1.5B, Qwen3, Qwen2.5-Coder 1.5B, Qwen2 1.5B, Llama 3.2 1B, and DeepSeek-R1 1.5B (Q4_0). No Gemma yet.

2. We cannot compile Gemma to .hef ourselves. Vision models can be compiled with the Hailo Dataflow Compiler, but the GenAI zoo ships prebuilt LLM .hef only, with no public custom-LLM compilation flow. So for now the Hailo NPU path is limited to those 6 models until Hailo publishes the GenAI compiler or adds Gemma to the zoo.

Plan: taOS will add catalog entries for those 6 Hailo .hef models so they are one-tap on a 10H, alongside the GGUF catalog for the CPU path. One thing you can help with since you have models running already: how does hailo-ollama actually fetch a model on your setup, is it hailo-ollama pull that downloads the .hef by its hef_h10h hash, or a manual .hef placement? That settles how we wire the install step. Merci.

[jaylfc]

Correction to my earlier note, and thanks for the inferrs link, that is a useful pointer.

I was wrong that no compiler exists. The Hailo Dataflow Compiler is public (free with the AI Software Suite) and does compile models to HEF, including LLMs; the zoo models themselves were built with it at W4A8. What actually blocks a custom Gemma today is the runtime, not the compiler: a self-compiled LLM HEF currently will not load as a runnable LLM object via hailo_platform.genai / hailo-ollama. A user hit exactly this in June, compiling a Qwen LoRA, the .hef compiled fine but would not instantiate as an LLM object.

So until Hailo opens the generative runtime to non-zoo HEFs, the 10H path is bounded to their 6 official LLMs (Qwen2.5, Qwen3, Qwen2.5-Coder, Qwen2, Llama 3.2 1B, DeepSeek-R1 1.5B), which is what taOS will ship as one-tap. If that runtime gate lifts, custom compiles like Gemma 4 become possible and we will add them. One thing that would help us wire the install cleanly: on your working setup, does hailo-ollama pull a model by name (resolving its content hash), or do you place the .hef manually? Merci.

[doc62fr]

Bonjour et merci pour ce complément d'information

on va donc faire pression sur hailo afin qu'il autorise l'exécution de n'importe quels LLM pouvant entrer des les 8go de ram de l'Accélérateur

Le client est roi et visiblement hailo la oublié...

[jaylfc]

Haha, bien vu. Le client devrait effectivement pouvoir choisir. En attendant que Hailo ouvre le runtime generatif aux modeles compiles hors zoo, taOS proposera les 6 LLM officiels du 10H en un tap (Qwen2.5, Qwen3, Qwen2.5-Coder, Qwen2, Llama 3.2 1B, DeepSeek-R1 1.5B), et des que la restriction saute nous ajoutons les autres, Gemma compris. Merci pour cet echange, tres utile pour cadrer le support Hailo.

[doc62fr]

Merci de votre retour

J'ai répondu à un post sur hailo Community espérons que ça les fassent réagir

https://community.hailo.ai/t/gemma-4-e4b-4-bit-quantized-model-support/19056

A voir si mon message ne sera pas supprimé ( suis rester poli quand même)