Skip to content

Migrate to go#40

Merged
openminddev merged 4 commits into
mainfrom
go
Jun 11, 2026
Merged

Migrate to go#40
openminddev merged 4 commits into
mainfrom
go

Conversation

@shicaih

@shicaih shicaih commented Jun 11, 2026

Copy link
Copy Markdown
Contributor

Summary

Rewrites the OM1-OTA agent and updater from Python to Go. The Python package (OTA/) and its tooling (uv/pyproject/ruff/black/pyright/pre-commit) are removed and replaced by a Go module that builds two static binaries: agent and updater.

Net diff is large because it deletes the entire Python tree and uv.lock; the new Go code is ~3k lines.

Layout

Package Responsibility
cmd/agent, cmd/updater Entry points
internal/ota Core engine: ota (WS dispatcher), actions, docker, ecr, filemanager, progress
internal/ws Reconnecting WebSocket client (reader/writer goroutines, send queue)
internal/s3 Artifact download (s3:// + https://), checksum verify, schema/env helpers
internal/agent Container status + info reporting loops, config sync wiring
internal/configsync Periodic config-manifest sync (new)
internal/config, internal/logging Env helpers, zap logger setup

Wire-compatible with the backend

Behavior was kept identical so the existing API/portal need no changes:

  • WS auth via ?api_key_id=&api_key= query params; x-api-key headers on HTTP calls.
  • /info (GET) and /status (POST) payload shapes, and the ota_progress frame schema (incl. ISO-8601 timestamp).
  • S3 and per-tag schema.json URLs; .ota/ file naming ({svc}_{tag}.yaml, {svc}_latest.yaml, {svc}_{tag}.env).
  • Docker work still shells out to the docker / docker-compose CLIs (os/exec); all actions (upgrade/start/stop/pause/unpause/restart) behave the same, including stop = stop and remove the container.

Behavior changes

  • Config syncer (internal/configsync): the agent periodically pulls a config manifest and syncs files for gated accounts; the loop stops cleanly on a 403 (feature not enabled).
  • Info-fetch is now a real periodic loop. The Python equivalent only ran once (a missing loop); this fetches on the documented interval.
  • Structured logging via zap (matching openmind-api), replacing slog.

Build / CI / deploy

  • Multi-stage Dockerfile: builds both binaries, ships them on debian:bookworm-slim; default CMD ["agent"].
  • docker-compose.yml: services now use command: ["agent"] / ["updater"].
  • lint.yml runs gofmt/go vet/go build/go test/go mod tidy (replaces ruff/black/isort/pyright/pre-commit).

Deployment note

When deploying this image, the compose command: must be ["agent"] / ["updater"] (not python -m OTA.*.main), and the image must be rebuilt from the Go Dockerfile. Existing .ota/ state is forward-compatible (same file layout).

Testing

  • go build ./..., go vet ./..., go test ./..., gofmt -l all pass; internal/configsync has unit tests.
  • On-device validation against the OTA backend recommended before rollout (run agent/updater, confirm status reporting and an upgrade/restart round-trip).

🤖 Generated with Claude Code

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR migrates the OM1 OTA agent/updater implementation from Python to Go, updating build/deployment tooling and CI accordingly.

Changes:

  • Replace the Python OTA agent/updater codebase with Go binaries (agent, updater) plus new internal packages for OTA actions, Docker orchestration, WebSocket connectivity, and S3/schema downloads.
  • Update container build and compose runtime to run Go binaries and drive the host Docker daemon via mounted docker / docker-compose CLIs.
  • Replace Python lint/typecheck tooling with Go formatting/vet/build/test checks in GitHub Actions.

Reviewed changes

Copilot reviewed 38 out of 45 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
README.md Updates documentation to reflect Go binaries, new layout, and Go-based build/run steps.
pyrightconfig.json Removes Python type-check configuration as part of Go migration.
pyproject.toml Removes Python packaging/dependency configuration.
OTA/utils/ws_client.py Removes Python WebSocket client implementation.
OTA/utils/s3_utils.py Removes Python S3/HTTP downloader and schema helpers.
OTA/utils/init.py Python package artifact remaining after code removal.
OTA/updater/README.md Removes Python updater README.
OTA/updater/main.py Removes Python updater entrypoint.
OTA/updater/init.py Python package artifact remaining after code removal.
OTA/README.md Removes Python OTA README.
OTA/ota/progress_reporter.py Removes Python progress reporter.
OTA/ota/ota.py Removes Python OTA dispatcher/engine.
OTA/ota/file_manager.py Removes Python OTA state/env file manager.
OTA/ota/ecr_handler.py Removes Python ECR credential handler.
OTA/ota/docker_operations.py Removes Python Docker orchestration implementation.
OTA/ota/action_handlers.py Removes Python action handler implementation.
OTA/ota/init.py Removes Python exports for OTA engine.
OTA/agent/README.md Removes Python agent README.
OTA/agent/main.py Removes Python agent entrypoint and status reporting logic.
OTA/agent/init.py Python package artifact remaining after code removal.
OTA/init.py Python package artifact remaining after code removal.
internal/ws/client.go Adds reconnecting WebSocket client for agent/updater communication.
internal/s3/downloader.go Adds S3/HTTP artifact download + checksum verification + schema caching.
internal/ota/progress.go Adds progress frame encoding and WebSocket progress reporting.
internal/ota/ota.go Adds Go OTA engine wiring + message dispatch for actions.
internal/ota/filemanager.go Adds .ota state management for compose YAML and env files.
internal/ota/ecr.go Adds ECR credential retrieval + docker login support.
internal/ota/docker.go Adds docker/docker-compose CLI orchestration with progress parsing and cleanup.
internal/ota/actions.go Adds action handlers for upgrade/start/stop/pause/unpause/restart.
internal/logging/logging.go Adds zap-based global logger initialization.
internal/configsync/syncer.go Adds config-manifest syncer to download managed config files safely.
internal/configsync/syncer_test.go Adds unit tests for config sync behavior and traversal protection.
internal/config/config.go Adds small env helper for defaults.
internal/agent/agent.go Adds agent wrapper for container status reporting + config sync loop.
go.sum Adds Go dependency checksums.
go.mod Adds Go module definition and dependencies.
Dockerfile Switches container build/runtime from Python to multi-stage Go build.
docker-compose.yml Updates compose commands to run Go binaries instead of Python modules.
cmd/updater/main.go Adds Go updater entrypoint wiring OTA engine to updater endpoint.
cmd/agent/main.go Adds Go agent entrypoint wiring OTA engine + status reporting + config sync.
.python-version Removes Python version pin.
.pre-commit-config.yaml Removes Python-focused pre-commit configuration.
.gitignore Updates ignore rules for Go build artifacts and runtime state.
.github/workflows/lint.yml Replaces Python lint/typecheck steps with gofmt/vet/build/test/tidy verification.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread internal/configsync/syncer.go
Comment thread internal/ota/actions.go
Comment thread internal/ws/client.go Outdated
@shicaih
shicaih requested a review from openminddev June 11, 2026 20:56
@openminddev
openminddev merged commit 566a9a7 into main Jun 11, 2026
4 checks passed
@openminddev
openminddev deleted the go branch June 11, 2026 21:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants