Skip to content

feat(docker): add localai-stack compose for self-hosted gemma-3-4b-it#18

Merged
using-system merged 3 commits into
mainfrom
feat/localai-stack-compose
Apr 17, 2026
Merged

feat(docker): add localai-stack compose for self-hosted gemma-3-4b-it#18
using-system merged 3 commits into
mainfrom
feat/localai-stack-compose

Conversation

@using-system

@using-system using-system commented Apr 17, 2026

Copy link
Copy Markdown
Owner

Summary

  • New docker-compose/localai-stack/ alongside observability-stack/, running a local LocalAI inference server that preloads google/gemma-3-4b-it from the default model gallery.
  • Agent service overrides OPENAI_BASE_URL, LLM_MODEL, and OPENAI_API_KEY so the digest runs end-to-end against the local model with no external LLM provider, gated on a LocalAI /readyz healthcheck.
  • New stack README covers prerequisites, first-start download (~3 GB cached in a named volume), direct API probes, and teardown.
  • Root README.md gets a short "Local LocalAI stack" subsection next to the existing "Local observability stack" one, pointing to the new compose directory.

Test plan

  • docker compose -f docker-compose/localai-stack/docker-compose.yml config validates the file.
  • docker compose up -d localai boots LocalAI, /readyz returns 200, /v1/models exposes gemma-3-4b-it, and a sample /v1/chat/completions call returns a valid response.
  • Full run (up --build --abort-on-container-exit agent): collector (10 posts) → filterer → scorer (10 → 6) → summarizer (6/6) → Telegram delivery (HTTP 200 × 2) with agent exit code 0.

🤖 Generated with Claude Code

Provides a second compose stack alongside observability-stack that runs
the agent against a local LocalAI inference server preloading
google/gemma-3-4b-it from the default gallery, so the digest can be
produced without any external LLM provider.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings April 17, 2026 14:39

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new Docker Compose “LocalAI stack” to run the reddit-digest agent end-to-end against a self-hosted LocalAI server (preloading a Gemma model), with accompanying usage documentation.

Changes:

  • Added docker-compose/localai-stack/docker-compose.yml defining localai + agent, wiring the agent to LocalAI via OpenAI-compatible /v1 and gating startup on /readyz.
  • Added docker-compose/localai-stack/README.md with prerequisites, quickstart, API probe examples, and teardown instructions.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File Description
docker-compose/localai-stack/docker-compose.yml New compose stack running LocalAI and an agent configured to use it via OPENAI_BASE_URL/LLM_MODEL, with a healthcheck gate.
docker-compose/localai-stack/README.md Documentation for running and interacting with the new LocalAI stack.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@@ -0,0 +1,62 @@
# LocalAI Stack

Local Docker Compose stack that runs the reddit-digest-agent against a self-hosted [LocalAI](https://localai.io/) inference server serving `google/gemma-3-4b-it`.

Copilot AI Apr 17, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

README mentions serving google/gemma-3-4b-it, but the rest of this stack (compose MODELS/LLM_MODEL and the example /v1/chat/completions payload) uses gemma-3-4b-it. Please align the model identifier in the docs with the actual model name that LocalAI exposes (either update this line to gemma-3-4b-it or update the compose/examples accordingly).

Suggested change
Local Docker Compose stack that runs the reddit-digest-agent against a self-hosted [LocalAI](https://localai.io/) inference server serving `google/gemma-3-4b-it`.
Local Docker Compose stack that runs the reddit-digest-agent against a self-hosted [LocalAI](https://localai.io/) inference server serving `gemma-3-4b-it`.

Copilot uses AI. Check for mistakes.
Comment on lines +1 to +3
# LocalAI Stack

Local Docker Compose stack that runs the reddit-digest-agent against a self-hosted [LocalAI](https://localai.io/) inference server serving `google/gemma-3-4b-it`.

Copilot AI Apr 17, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR description says documentation was added to the repository root README.md, but this PR adds docs under docker-compose/localai-stack/README.md only. Please either update the PR description or add a short pointer in the root README so users can discover this stack.

Copilot uses AI. Check for mistakes.
Comment on lines +35 to +37
| Service | URL | Description |
|---------|-----|-------------|
| LocalAI | http://localhost:8080 | OpenAI-compatible inference API (`/v1/chat/completions`, `/v1/models`, …) |

Copilot AI Apr 17, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The markdown table under “Services” has an extra leading | on each row (|| ...), which breaks table rendering in most Markdown parsers. Change those lines to start with a single |.

Copilot uses AI. Check for mistakes.
using-system and others added 2 commits April 17, 2026 16:42
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@using-system using-system merged commit 28f0331 into main Apr 17, 2026
8 checks passed
@using-system using-system deleted the feat/localai-stack-compose branch April 17, 2026 14:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants