Skip to content
This repository was archived by the owner on Apr 24, 2026. It is now read-only.

voice-to-mermaid: remove vtm-ollama sidecar, default qwen3:8b #8

Merged
jaewilson07 merged 1 commit into from
Apr 24, 2026
Merged

voice-to-mermaid: remove vtm-ollama sidecar, default qwen3:8b #8

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 12 additions & 5 deletions voice-to-mermaid/backend/config.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,15 +4,22 @@
# Secrets (OLLAMA_URL, OPENAI_API_KEY, etc.) stay in .env / Infisical.

ollama:
# Default model for first-diagram generation. qwen3:8b is chosen for:
# - Fast first-token latency (important for "type and see diagram" UX)
# - Good instruction-following on mermaid grammar
# - Fits in 8-12 GB VRAM (cold-load ~5s, then streams)
# Swap to a 32B class model (qwen2.5-coder:32b, qwen3.5:27b) for complex
# multi-node diagrams at the cost of ~20s cold-load latency.
default_model: "qwen3:8b"
# Models shown in the UI picker — exact names or prefixes (e.g. "qwen2.5:").
# Leave empty to show all installed models.
# Models shown in the UI picker. Must exist in Windows Ollama
# (C:\Users\jaewi\.ollama is not used — see OLLAMA_MODELS=D:\ollama\models).
Comment on lines +14 to +15

Copilot AI Apr 24, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment uses a user-specific Windows path (C:\Users\jaewi...) and a specific drive letter for OLLAMA_MODELS. Consider making this guidance portable (e.g., %USERPROFILE%\.ollama, or describing how to locate/set OLLAMA_MODELS) so it applies to other developers/machines.

Suggested change
# Models shown in the UI picker. Must exist in Windows Ollama
# (C:\Users\jaewi\.ollama is not used — see OLLAMA_MODELS=D:\ollama\models).
# Models shown in the UI picker. Must exist in Ollama's configured models
# directory (by default on Windows, under %USERPROFILE%\.ollama; if
# OLLAMA_MODELS is set, use that location instead).

Copilot uses AI. Check for mistakes.
# Update as needed with: ollama pull <model> (run in Windows PowerShell)
model_filter:
- "qwen3:8b"
- "qwen2.5:14b"
- "qwen2.5-coder:32b"
- "qwen3.5:27b"
- "qwen3:8b"
- "qwen3:14b"
- "llama3.2:3b"
- "gemma3:12b"

openai:
model: "gpt-4o-mini"
Expand Down
43 changes: 21 additions & 22 deletions voice-to-mermaid/backend/docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,21 +1,18 @@
services:
ollama:
image: ollama/ollama:latest
container_name: vtm-ollama
restart: unless-stopped
ports:
- "11434:11434"
volumes:
- ollama-data:/root/.ollama
# GPU support — remove 'deploy' block if no GPU available
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: 1
capabilities: [gpu]
# Voice-to-Mermaid — LOCAL MACHINE ONLY (requires GPU)
# ⚠️ GPU GUARD: This compose file requires NVIDIA GPU hardware.
# ⚠️ DO NOT deploy to the VPS (187.77.216.108) — it has no GPU.

Copilot AI Apr 24, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment hardcodes a public VPS IP address. Even in comments, committing infrastructure IPs can increase attack surface and becomes stale quickly; consider referencing a hostname/alias (e.g. "the VPS") or pointing to an internal inventory/runbook instead of embedding the raw IP.

Suggested change
# ⚠️ DO NOT deploy to the VPS (187.77.216.108) — it has no GPU.
# ⚠️ DO NOT deploy to the VPS — it has no GPU.

Copilot uses AI. Check for mistakes.
# ⚠️ See infrastructure/local/README.md for local machine service registry.

Copilot AI Apr 24, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment references infrastructure/local/README.md, but that path does not exist in the repository. Please update the pointer to an existing doc path or remove the reference to avoid broken guidance.

Suggested change
# ⚠️ See infrastructure/local/README.md for local machine service registry.
# ⚠️ Local machine only: keep service guidance in repository docs up to date.

Copilot uses AI. Check for mistakes.

#
# Architecture (as of 2026-04-24):
# vtm-api → http://host.docker.internal:11434 → Windows-native Ollama
#
# The previous `vtm-ollama` sidecar + `ollama-data` volume were removed —
# they duplicated the Windows Ollama instance (ran empty, wasted resources).
# See .agents/runbooks/troubleshoot-ollama/SKILL.md for the single-instance
# architecture rules.
Comment on lines +12 to +13

Copilot AI Apr 24, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment points to .agents/runbooks/troubleshoot-ollama/SKILL.md, but there is no .agents directory in the repo. Please update to an existing runbook location or remove the reference so readers can actually find the architecture rules.

Suggested change
# See .agents/runbooks/troubleshoot-ollama/SKILL.md for the single-instance
# architecture rules.
# Keep this as a single-instance setup: use the existing Windows-native
# Ollama service and do not reintroduce a Compose-managed Ollama sidecar here.

Copilot uses AI. Check for mistakes.
#
services:
api:
build: .
container_name: vtm-api
Expand All @@ -24,14 +21,16 @@ services:
- "${PORT:-7625}:7625"
env_file: .env
environment:
# Default to the bundled ollama service; override in .env to use a
# host-resident Ollama (http://host.docker.internal:11434) instead.
OLLAMA_URL: ${OLLAMA_URL:-http://ollama:11434}
# Windows-native Ollama via Docker Desktop gateway.
OLLAMA_URL: ${OLLAMA_URL:-http://host.docker.internal:11434}
extra_hosts:
- "host.docker.internal:host-gateway"
volumes:
- ./prompts:/app/prompts # mount prompts so edits don't need rebuild
- ./config.yaml:/app/config.yaml # mount config so edits don't need rebuild
networks:
- local-ai

volumes:
ollama-data:
networks:
local-ai:
name: local-ai
Loading