lagotto

Watch for EC2 instance capacity and act when it appears.

Some instance types — particularly high-demand GPU families — aren't always available. Lagotto runs as a serverless Lambda that polls for capacity on a schedule and acts when it appears.

Installation

macOS / Linux (Homebrew)

brew install spore-host/tap/lagotto

Windows (Scoop)

scoop bucket add spore-host https://github.com/spore-host/scoop-bucket
scoop install lagotto

Direct download — pre-built binaries for Linux, macOS, and Windows (amd64/arm64) on the releases page.

Quick Start

# Watch for any p5 instance and notify when available
lagotto watch "p5.*" --action notify --ttl 7d

# Watch and auto-launch when capacity appears
lagotto watch "g5.xlarge" --action spawn --spawn-config my-job.yaml

# Manage watches
lagotto list
lagotto status <watch-id>
lagotto extend <watch-id> --ttl 48h
lagotto cancel <watch-id>          # prompts for confirmation; -y/--yes to skip
lagotto history

Goal-driven fleets

By default a watch fires its action once and retires. A fleet watch instead maintains a target number of workers until an external completion condition is true — relaunching toward the goal each poll, including from zero if the whole fleet is reclaimed:

lagotto watch m8g.8xlarge --action spawn --spawn-config prep.yaml \
  --maintain 4 \                                            # keep ~4 workers alive…
  --until 's3-empty: s3://bucket/manifest minus s3://bucket/prepared/' \  # …until done
  --regions us-west-2 --spot

Each poll: if --until holds, the fleet retires (completed); otherwise lagotto counts the running workers (by a lagotto:watch=<id> tag) and launches enough to reach --maintain. This makes idempotent, spot-friendly batch fire-and-forget: pull-model workers absorb independent reclaim, and the supervisor revives the fleet after a correlated total loss.

--until conditions:

• s3-empty: <wanted-prefix> minus <done-prefix> — done when every wanted S3 key has a corresponding done key (the durable, compute-independent completion state for a pull-model job).
• http-200: <url> — done when a GET returns 2xx.
• shell: <cmd> — done when the command exits 0. Runs only on a local poll --daemon (the hosted poller has no shell sandbox).

Requires --action spawn. Bound cost with a TTL in the spawn config; workers still get spored's TTL/idle lifecycle.

Polling

A watch only fires when something polls it. Two ways:

# Infra-free: poll in the foreground until the watch fires/expires (no Lambda,
# no CloudFormation). Keep it running — or under your own supervisor/cron.
lagotto poll --daemon --interval 5m

# One-off cycle (testing/debugging)
lagotto poll

--daemon runs the same poll loop the hosted Lambda does, so lagotto watch --action spawn works hands-off in your own account with zero extra infrastructure. The hosted, multi-tenant Lambda poller (deployed via CloudFormation) remains the option for teams — see DEPLOYMENT.md.

Scoping a daemon in a shared account

By default poll --daemon services every active watch in the account. In an account shared across projects/people, scope it to your own watches so it doesn't drive (and launch) someone else's (#47):

lagotto watch "g5.12xlarge" --action spawn --spawn-config job.yaml --project fieldwork
lagotto poll --daemon --project fieldwork   # only fieldwork's watches
lagotto poll --daemon --mine                # only watches you created
lagotto poll --daemon --watch w-aaa,w-bbb   # only these watch IDs

--project defaults to $LAGOTTO_PROJECT (on both watch and poll). A scoped daemon exits when its watches are done, not the whole account's.

Before acting on a match, a poller claims a short processing lease on the watch, so two daemons — or a local daemon racing the hosted Lambda — can't both launch the same watch. A crashed poller's lease ages out automatically. Disable with --no-lease (not recommended when more than one poller runs).

Scheduled launches

Where watch fires when capacity appears, lagotto launch fires at a time — once at a clock time, after a delay, or on a recurring cron. The motivating case is launching into an EC2 Capacity Block for ML at its reserved start time.

# Launch into a Capacity Block reliably at its window open — derives the start
# time from the reservation, fires slightly early, and retries through the
# boundary until the instance runs (block.yaml sets reservation_id + capacity_block)
lagotto launch --reservation-id cr-0abc123 --at-reservation-start --spawn-config block.yaml

# Launch at an explicit clock time
lagotto launch --at 2026-07-01T08:00:00Z --az us-east-1a --spawn-config block.yaml

# Launch 6 hours from now
lagotto launch --after 6h --spawn-config job.yaml

# Recurring: every weekday at 09:00 UTC
lagotto launch --cron "0 9 ? * MON-FRI *" --spawn-config nightly.yaml

For a Capacity Block, prefer --reservation-id … --at-reservation-start over a hand-typed --at: you've paid up front for a non-refundable window that ends at a fixed 11:30 UTC, so every minute of late start is wasted. This mode derives the open time from the reservation, fires --fire-early (default 2m) ahead so EventBridge latency doesn't cost you, and retries on --retry-interval (default 30s) through the open — absorbing the transient capacity / not-yet-active blips at the boundary — until an instance is running, then stops.

Scheduled launches are driven by EventBridge Scheduler in the hosted poller stack, so they require lagotto deploy first (the schedule targets the poller Lambda in your account). The launched instance always carries a TTL (#38), and a one-shot's schedule self-deletes after it fires.

Overlap policy. If an instance with the same Name tag already exists when a schedule fires, --if-exists decides what happens:

--if-exists	Behavior	Default for
skip	Don't launch; treat the existing instance as the fulfillment	--at / --after (a Capacity Block must not double-book)
launch	Launch anyway — each fire is a fresh box	--cron
replace	Terminate the existing instance, then launch	—

The dedup key is the instance Name tag (--name, or the spawn config's name).

Actions

Action	Description
notify	Send email/webhook/SNS notification
spawn	Auto-launch instance with config file
hold	Record availability without acting

Deployment

Lagotto deploys as a CloudFormation stack. See DEPLOYMENT.md for setup.

Go Library

import "github.com/spore-host/lagotto/pkg/watcher"

License

Apache 2.0 — Copyright 2025-2026 Scott Friedman.