feat: virtual-hosted S3 addressing (CLI + in-pod)#83
Open
mostlymaxi wants to merge 4 commits into
Open
Conversation
mostlymaxi
changed the title
mostlymaxi
force-pushed
the
feat/s3-virtual-hosted
branch
from
June 10, 2026 16:43
c5237ed to
4ca0411
Compare
Add `[artifact_repository.s3] virtual_host` (bool, default false = path-style, what MinIO / most S3-compatible stores want). When true the CLI's object_store client uses virtual-hosted-style requests; resolved to a bool on S3Ref so publish / submit / emulate use one style for the binary tarball and load/save_artifact objects alike. CLI-side only for now: the in-pod Argo executor auto-detects until upstream Argo exposes the toggle on its S3 artifact. Regenerates the describe/ls metadata goldens (S3Ref's bool); emitted WorkflowTemplate YAML is unchanged.
mostlymaxi
changed the title
mostlymaxi
force-pushed
the
feat/s3-virtual-hosted
branch
from
June 10, 2026 17:13
4ca0411 to
216af80
Compare
Emit the addressing style onto every S3 artifact so the in-pod executor uses it too, not just the CLI client. api::S3Artifact gains addressing_style; s3_loc maps virtual_host true -> "virtual-hosted", false -> "" (omitted, auto-detect). Skip-when-empty keeps goldens byte-identical and lets pre-#15734 Argo ignore the unknown field. from_template reads the style off the emitted artifact instead of a threaded bool, so the run metadata has one source of truth.
mostlymaxi
changed the title
addressingStyle (PR #15734) is newer than every supported Argo (v4.0.5 / v3.7.14 / v3.6.19), so the in-pod half is inert on all of them today. Reword the caveat and the mirrored comments to say that plainly instead of implying only "older" executors ignore it, and link the support matrix. CLI side is unaffected.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a
virtual_hosttoggle to[artifact_repository.s3]so cargo-athena uses virtual-hosted-style S3 addressing (https://<bucket>.<endpoint>/<key>) for providers that require it, instead of the default path-style.Two halves:
publish/submit/emulate): theobject_storeclient useswith_virtual_hosted_style_request. Works today.addressingStyle(upstream PR #15734) so the executor uses the same style for the binary tarball andload/save_artifact!objects.addressingStyleis newer than every supported Argo (v4.0.5 / v3.7.14 / v3.6.19 all predate it), so this half is inert until a cluster runs an Argo that ships #15734. On the current matrix the emitted key is silently pruned on admission, so it is harmless to set.The config stays a bool;
s3_locmaps it onto Argo's tri-stateaddressingStyle:truebecomesvirtual-hosted,falsebecomes""(auto-detect). The empty default is skip-serialized, which buys three things:virtual_host = trueemits the key),submitdrift against WTs from a pre-virtual_hostbinary (absent field deserializes to"", so"" == ""),from_templatenow reads the style off the emitted artifact instead of a threaded bool, so the describe/ls run metadata has a single source of truth. New unit test locks the mapping and the skip-when-empty serialization.Build / clippy / test / fmt / mdbook green; goldens unchanged.