Skip to content

Named tag support#8

Open
SmartyAdam wants to merge 2 commits into
masterfrom
adam/tagging
Open

Named tag support#8
SmartyAdam wants to merge 2 commits into
masterfrom
adam/tagging

Conversation

@SmartyAdam

Copy link
Copy Markdown

Tagged Versions

Tags are named, manually-managed pointers to specific package versions. They let
a consumer depend on a stable name (for example release or stable) instead of
a concrete version string, so that downstream pipelines do not have to know — or
be updated with — the exact version they should pull. Unlike latest, a tag only
moves when you explicitly move it, which makes it suitable for promoting a vetted
build to production on your own schedule.

Concepts

Each data package has a root manifest at /{package}/manifest.json and one
versioned manifest per version at /{package}/{version}/manifest.json. The
root manifest is a copy of the most recently uploaded version's manifest and is
what latest resolves to.

Tags live only in the root manifest, in a tags array. Each entry maps a tag
name to a version:

{
  "name": "cat-sound-data",
  "version": "2026.02.A",
  "archive": {
    "filename": "archive",
    "size": 2506300011,
    "md5": "34GMLE2LaQI1QHKoHJFRbQ==",
    "contents": [
      { "path": "meows.txt", "size": 16087014133, "md5": "4ksM3tWryJom2O8XrU1l5A==" },
      { "path": "purrs.txt", "size": 39014497,    "md5": "E4LTd7kXWcP6kdAvTy9sHg==" }
    ],
    "compression": "zstd"
  },
  "tags": [
    { "name": "experimental", "version": "2026.01.B" },
    { "name": "stable",       "version": "2026.01.A" }
  ]
}

The versioned manifests are unchanged — they never carry a tags array. The
tags field is omitted entirely from a manifest's JSON when there are no tags, so
packages that never use tags produce byte-for-byte the same manifests as before.
Manifests written to the local download directory also have their tags stripped,
since the tag list is a remote, root-level concern.

Reserved names and validation

  • latest is reserved and cannot be used as a tag name. The download keyword
    latest always takes precedence, so a tag of that name would be unreachable.
  • Tag names may not be blank.
  • Within a single tag-modification request, a name may not appear twice, and a
    name may not appear in both the add and delete lists.

Resolving a tag on download

A download dependency may put a tag name where it would normally put a version:

{
  "dependencies": [
    {
      "package_name": "cat-sound-data",
      "package_version": "stable",
      "remote_address": "gcs://cats-data-dev",
      "local_directory": "."
    }
  ]
}

Resolution is version-first:

  1. satisfy first treats package_version as a literal version and looks for a
    manifest at /{package}/{version}/manifest.json.
  2. Only if no manifest exists there does satisfy fetch the root manifest and look
    the string up in the tags array.

This means a real version always shadows a tag of the same name, and an ordinary
pinned-version download behaves exactly as it did before tags existed — there is
no extra work on the common path. If the string matches neither a version nor a
tag, the download fails (and, importantly, fails before removing any
already-installed local files).

When a tag resolves to the version that is already installed locally, satisfy
runs its normal integrity check and skips the download, just as it does for a
matching pinned version. This is what makes a pipeline pinned to a tag cheap to
re-run: it does not re-download a multi-gigabyte archive every time simply
because the tag name differs from the installed version string.

Note on cost: the existence check in step 1 is an HTTP HEAD (a size
probe), not a full manifest download, so resolving a tag — or confirming a
tag's target is already installed — costs only a header-level round trip.

Tagging during upload

An upload request may list tag names to apply to the version being uploaded:

{
  "package_name": "cat-sound-data",
  "package_version": "2026.02.B",
  "compression_algorithm": "zstd",
  "compression_level": 9,
  "remote_address": "gcs://cats-data-dev",
  "tags": ["ultra-experimental", "experimental"],
  "source_path": ""
}

The versioned manifest is written as usual. For the root manifest, satisfy:

  1. Downloads the current root manifest (if one exists) to read its existing tags.
  2. Preserves every existing tag and points each requested tag name at the
    version just uploaded — adding the name if it is new, updating it if it
    already existed.

So uploading 2026.02.B with "tags": ["experimental"] moves experimental to
2026.02.B while leaving an unrelated stable tag exactly where it was. An
upload with no tags field leaves all existing tags untouched. The resulting tag
list is sorted by name for stable, reviewable diffs.

Modifying tags without uploading

The tags subcommand changes tags on an already-uploaded package without
re-uploading any data. It reads a JSON request from a file (-json <path>) or
from standard input (the default, -json _STDIN_):

satisfy tags -json modify-tags.json

Request format:

{
  "package_name": "cat-sound-data",
  "remote_address": "gcs://cats-data-dev",
  "add": [
    { "name": "stable",         "version": "2026.02.A" },
    { "name": "marks-favorite", "version": "2026.01.B" }
  ],
  "delete": [
    { "name": "active-build-test" },
    { "name": "adam-test" }
  ]
}

Behavior:

  • Add creates the tag if it does not exist, or updates it to the given
    version if it does.
  • Delete removes the named tag. Only the name is consulted; any version
    is ignored. Deleting a tag that does not exist is a no-op, not an error.
  • Before writing anything, satisfy verifies that every version named in add
    actually exists
    on remote storage (again via a HEAD probe of the version's
    manifest). If any target version is missing, the whole request fails and
    nothing is written — there is no partial update. This prevents creating a
    tag that points at a version which cannot be downloaded.

On success, the updated root manifest is uploaded back to
/{package}/manifest.json with its tag list sorted by name.

Flags

Flag Default Meaning
-json _STDIN_ Path to the request file, or _STDIN_ to read stdin.
-max-retry 5 How many times to retry remote storage operations.

Exit code 0 indicates success; exit code 1 indicates a failure (see stderr).

Summary of behavior

Action Effect on root manifest tags
Upload with no tags Existing tags preserved unchanged.
Upload with tags: [...] Existing tags preserved; named tags point at new version.
tags add (new name) Tag created (target version verified to exist first).
tags add (existing name) Tag updated to the new version.
tags delete (existing name) Tag removed.
tags delete (missing name) No-op.
Download by version Literal version path used; tags not consulted.
Download by tag Resolved via the root manifest's tags array.

@smartybryan smartybryan left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good.

@timothy-smarty timothy-smarty left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@cody-smarty

Copy link
Copy Markdown

Like it!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants