Configuration_Reference.md

Configuration Reference

This page contains the full configuration and environment reference.

Go Orchestrion version selection

Guided Go bootstrap accepts:

Flag	Default	Description
--dd-trace-go-version	v2.9.0	Go tracer query for bootstrap. Accepts a tag, pseudo-version, branch, or commit SHA and persists the exact resolved versions Bazel should use
--go-binary	go	Go binary used for bootstrap module graph synchronization. Set this to a pinned SDK path named go or go.exe when the repository must match Bazel's Go SDK; do not include shell syntax or arguments
--go-mod-sync	targeted	Local Go module synchronization strategy: targeted updates and verifies only Orchestrion tool packages, tidy also runs go mod tidy, and off skips Go module commands

WORKSPACE snippet mode is read-only. WORKSPACE generation mode writes only local files and managed blocks; it never edits WORKSPACE itself:

Flag	Default	Description
--workspace-mode	false	Enable WORKSPACE-oriented onboarding generation without editing WORKSPACE
--print-workspace-snippet	false	Print WORKSPACE-mode repository wiring and exit without modifying files
--print-bazelrc-snippet	false	Print the recommended .bazelrc block and exit without modifying files
--write-bazelrc	false	Insert or replace the managed .bazelrc block. With --guided, bootstrap also continues the normal guided setup
--write-root-targets	false	Insert or replace managed root BUILD.bazel blocks for doctor, uploader, and root pin exports
--write-orchestrion-files	false	Write orchestrion.tool.go, orchestrion.yml, and pin-file exports for the selected Go module directory
--write-wrapper-template	false	Write a configurable repo-local Go wrapper template with plain and optimized wrapper paths
--bazelrc-path	.bazelrc	Path to the Bazel rc file updated by --write-bazelrc
--bazelrc-config	test-optimization	Config name used by generated common:<config> and test:<config> lines
--sync-repo-name	test_optimization_data	External repository name used by generated doctor, uploader, wrapper, and WORKSPACE sync snippets
--expected-target	repeatable	Local label the generated doctor should require, for example //pkg:go_default_test
--wrapper-package	tools/build	Workspace-relative Bazel package for the generated wrapper template
--wrapper-file	dd_topt_go_test.bzl	Wrapper .bzl filename used by --write-wrapper-template
--plain-wrapper-name	dd_go_test	Plain local wrapper function name generated by the wrapper template
--optimized-wrapper-name	dd_topt_go_test	Optimized local wrapper function name generated by the wrapper template
--datadog-fetch	git	Fetch mode for datadog-rules-test-optimization-go: git or archive
--rules-go-fetch	git	Fetch mode for the Orchestrion-enabled rules_go fork: git or archive
--rules-go-upstream	default	Datadog-managed upstream support line. Omit this to preserve the repository default; the current default resolves to v0_60_0
--rules-go-variant	base	Published fork variant. Only base is supported.
--rules-go-repo-name	io_bazel_rules_go	Bazel repository name that consumers use for rules_go
--rto-commit	empty	Published rules_test_optimization commit used in generated WORKSPACE snippets. Required when either fetch mode is git
--rto-archive-url	empty	Archive URL used when either fetch mode is archive
--rto-archive-sha256	empty	Archive SHA256 used when either fetch mode is archive
--rto-archive-prefix	empty	Archive root prefix used when either fetch mode is archive
--rto-archive-type	tar.gz	Archive type passed to http_archive
--check-go-repositories	false	Check checked-in WORKSPACE go_repository(...) declarations for the Orchestrion and dd-trace-go modules bootstrap owns
--go-repositories-file	repositories.bzl	Workspace-relative file inspected by --check-go-repositories
--print-go-repository-updates	false	Print the expected versions for the checked Orchestrion-related go_repository(...) declarations
--go-repositories-refresh-command	empty	Repository-owned command to refresh go_repository(...) declarations after a successful --go-mod-sync=targeted run; bootstrap rechecks the file after the command

Validation script generation writes a repository-owned operational helper. It does not mutate Bazel rules or force remote output behavior outside the normal .bazelrc/CLI contract:

Flag	Default	Description
--print-validation-script	false	Print the validation script and exit without modifying files
--write-validation-script	false	Write the generated validation script
--validation-script-path	tools/test_optimization/validate_go_pilot.sh	Workspace-relative output path for --write-validation-script
--bazel-command	bazel	Bazel command used by the script, for example bazel, ./bazelw, or bzl; do not include arguments
--bazel-config	test-optimization	Config passed as --config=<name> to sync, test, doctor, and upload commands
--control-target	repeatable	Plain/control test target run before instrumented targets
--expected-target	repeatable	Instrumented target run before doctor; this is also used by generated doctor targets
--doctor-target	//:dd_test_optimization_doctor	Doctor label run after tests pass
--upload-target	//:dd_upload_payloads	Uploader label run only when the script receives --upload
--extra-sync-flag	repeatable	Extra Bazel sync flag
--extra-test-flag	repeatable	Extra Bazel test flag; --test_env=DD_GIT_* is rejected
--extra-run-flag	repeatable	Extra Bazel run flag for doctor/uploader
--large-monorepo	false	Add disk checks before heavy phases and keep phases serialized
--min-free-disk-gb	25	Warning threshold used by --large-monorepo scripts
--shutdown-bazel-on-exit	false	Run <bazel-command> shutdown when the script exits
--default-jobs	0	Add --jobs=<n> to generated test commands when greater than zero

In WORKSPACE mode, bootstrap does not run Go module commands unless --go-mod-sync is explicitly provided. This keeps large monorepo scaffolding safe: operators can generate local files first, review the WORKSPACE snippet, then run --go-mod-sync=targeted only when they are ready to update go.mod and go.sum.

If the repository keeps checked-in go_repository(...) declarations, pair the targeted sync with --check-go-repositories. Bootstrap does not edit repositories.bzl directly. It compares only the Orchestrion-related modules it owns (github.com/DataDog/orchestrion, github.com/DataDog/dd-trace-go/v2, github.com/DataDog/dd-trace-go/contrib/net/http/v2, and github.com/DataDog/dd-trace-go/contrib/log/slog/v2) and either prints the required versions or runs the repository-owned refresh command you provide. The refresh command only runs after targeted sync succeeds, so a stale repositories.bzl cannot mask a broken Go module graph.

Manual Orchestrion wiring in MODULE.bazel accepts:

Setting	Default	Description
orchestrion.from_source(..., dd_trace_go_version = "...")	v2.9.0	Shared canonical tracer version that Bazel validates against the target Go module and uses for synthetic fallback paths
orchestrion.from_source(..., dd_trace_go_versions = {...})	none	Exact canonical per-module tracer versions that Bazel validates against the target Go module for github.com/DataDog/dd-trace-go/v2, github.com/DataDog/dd-trace-go/contrib/net/http/v2, and github.com/DataDog/dd-trace-go/contrib/log/slog/v2

Notes:

• The selected version is workspace-wide for Go. There is no per-test override.
• Bootstrap repins the local Go module to the same effective versions.
• Bootstrap uses targeted module sync by default and does not run go mod tidy unless --go-mod-sync=tidy is selected.
• Bootstrap accepts tags, pseudo-versions, branches, and commit SHAs through --dd-trace-go-version and resolves them to canonical persisted versions.
• Manual orchestrion.from_source(...) tracer settings must already use canonical persisted versions. Branches and commit SHAs are rejected there.
• The Orchestrion tool bootstrap records these versions in dd_trace_go_versions.json, but it no longer rewrites the downloaded Orchestrion source repo's own go.mod.
• Bootstrap writes dd_trace_go_version when all traced modules resolve to one shared version, and dd_trace_go_versions when they resolve to different exact versions.
• If you rerun bootstrap without --dd-trace-go-version, it preserves the existing bootstrap-managed tracer config instead of resetting it.
• Bootstrap refuses to proceed when active tracer settings already exist in orchestrion.from_source(...) calls outside its managed block. Those manual settings must be removed or migrated first.
• Do not set both dd_trace_go_version and dd_trace_go_versions in the same orchestrion.from_source(...) call.
• If the workspace setting and the effective local Go module versions differ, the build fails instead of mixing versions.

Go Test Optimization Orchestrion mode

dd_topt_go_test(...) accepts orchestrion_mode:

Value	Description
general	Default. Preserves the generic Orchestrion behavior for the transitioned raw go_test
test_optimization	Standard Go testing Test Optimization mode. Keeps stdlib/testing, synthetic testmain, helper packagefiles, importcfg, and link support, while leaving customer package compiles and external _test package compiles on the plain rules_go path

test_optimization mode requires package-local pin files or explicit orchestrion_pin_files pointing at the Go module root. It uses a reduced synthetic Orchestrion tool file for action-time module work, so orchestrion.tool.go is not required for the optimized mode even when a generic Orchestrion setup keeps one for broader integrations. Automatic testify/suite instrumentation is intentionally outside this mode.

When orchestrion_mode = "test_optimization", dd_topt_go_test also enables test-binary linker optimization by default in Bazel modes where rules_go is not already stripping. Disable it per target or in a repo-local wrapper with:

enable_test_binary_linker_optimization = False

Go target metadata records the selected mode in bazel.go.orchestrion.mode and whether the linker optimization was active in bazel.go.test_binary_linker_optimization.

Sync extension attributes

Extension tag: test_optimization_sync.test_optimization_sync(...)

Attribute	Type	Default	Description
name	string	required	External repository name to create (examples use test_optimization_data)
out_dir	string	.testoptimization	Base output directory. Must be a non-empty relative path; absolute paths and .. traversal are rejected
service	string	"unnamed-service" or DD_SERVICE	Service name precedence: service attr > DD_SERVICE env > fallback
runtime_name	string	empty	Optional runtime name (examples: go, python, java, nodejs, dotnet, ruby)
runtime_version	string	empty	Optional runtime version (example: 1.25.0)
runtime_arch	string	auto-detected	Optional runtime arch; defaults to detected os.architecture
runtime_module_path	string	empty	Optional module path for the selected runtime. Runtime-specific env vars such as GO_MODULE_PATH still win, then this attr is used, then Go falls back to go.mod autodetection
http_connect_timeout_seconds	int	-1 attr / 10 effective	Optional connect-timeout override for sync HTTP requests (-1 keeps env/default behavior)
http_max_time_seconds	int	-1 attr / 60 effective	Optional per-request max-time override for sync HTTP requests (-1 keeps env/default behavior)
http_retry_attempts	int	-1 attr / 3 effective	Optional retry-attempt override for sync HTTP requests (-1 keeps env/default behavior)
http_retry_delay_seconds	int	-1 attr / 2 effective	Optional retry-delay override for sync HTTP requests (-1 keeps env/default behavior)
http_execute_timeout_buffer_seconds	int	-1 attr / 60 effective	Optional outer execute-timeout buffer override (-1 keeps env/default behavior)
known_tests	bool	True	Local switch for Known Tests request. When False, request is skipped, a minimal stub is written, and settings are mutated to known_tests_enabled=false
test_management	bool	True	Local switch for Test Management request. When False, request is skipped, a minimal stub is written, and settings are mutated to test_management.enabled=false
require_git_metadata	bool	False	Strict local/CI validation for settings-request Git metadata. When True, sync fails before HTTP if repository URL, branch or tag, and commit SHA cannot be resolved
debug	bool	False	Enables verbose repository-rule logging

Notes:

• manifest.txt path is exported via topt_data["manifest_path"] and respects out_dir.
• Parent directories are created automatically for all output paths.
• Omitted optional attributes keep default behavior and avoid unnecessary cache-key churn.
• For HTTP numeric overrides, -1 means "do not pin here"; resolution falls back to environment overrides first, then the rule default.

Multi-sync extension attributes

Extension tag: test_optimization_multi_sync.test_optimization_multi_sync(...)

Attribute	Type	Default	Description
name	string	required	Aggregator repository name (examples use test_optimization_data)
services	string_list	required	Non-empty list of service names; one sync repo is created per service
out_dir	string	.testoptimization	Optional base output directory applied to each per-service sync repo
runtime_name	string	empty	Optional runtime name propagated to each per-service sync repo
runtime_version	string	empty	Optional runtime version propagated to each per-service sync repo
runtime_arch	string	auto-detected	Optional runtime arch propagated to each per-service sync repo
runtime_module_path	string	empty	Optional module path propagated to each per-service sync repo
http_connect_timeout_seconds	int	-1 attr / 10 effective	Optional connect-timeout override propagated to each per-service sync repo (-1 keeps env/default behavior)
http_max_time_seconds	int	-1 attr / 60 effective	Optional per-request max-time override propagated to each per-service sync repo (-1 keeps env/default behavior)
http_retry_attempts	int	-1 attr / 3 effective	Optional retry-attempt override propagated to each per-service sync repo (-1 keeps env/default behavior)
http_retry_delay_seconds	int	-1 attr / 2 effective	Optional retry-delay override propagated to each per-service sync repo (-1 keeps env/default behavior)
http_execute_timeout_buffer_seconds	int	-1 attr / 60 effective	Optional outer execute-timeout buffer override propagated to each per-service sync repo (-1 keeps env/default behavior)
known_tests	bool	True	Known Tests kill-switch propagated to each per-service sync repo
test_management	bool	True	Test Management kill-switch propagated to each per-service sync repo
require_git_metadata	bool	False	Strict Git metadata validation propagated to each per-service sync repo
debug	bool	False	Enables verbose logging for generated per-service sync repos

Uploader rule attributes

Rule: dd_payload_uploader(...)

Attribute	Type	Default	Description
name	string	required	Target name
quiescent_sec	int	10	Seconds to wait for filesystem to settle before uploading
max_wait_sec	int	300	Maximum seconds to wait for payloads (0 skips waiting when no payloads are present)
fail_on_error	bool	False	Exit with error if no payloads found when tests ran
debug	bool	False	Enable debug logging
keep_payloads	bool	False	Keep payload files after successful upload
filter_prefix	bool	False	Only upload files matching span_events_*.json or coverage_*.json
gzip_payloads	bool	False	Gzip test payloads before upload
data	label_list	[]	Data files to include (for example, context.json for enrichment)

Doctor rule attributes

Rule: dd_test_optimization_doctor(...)

Run this target after bazel test and before dd_payload_uploader. It validates local or BEP-staged payload files and metadata; it does not upload, delete, or rewrite source payloads.

Attribute	Type	Default	Description
name	string	required	Target name
data	label_list	["@test_optimization_data//:test_optimization_context"] in examples	Context targets used to validate Git metadata before upload
expected_targets	string_list	[]	Optional strict list of local Bazel test labels to validate. When empty, the doctor validates discovered Test Optimization output directories and ignores plain non-instrumented test outputs
require_git_metadata	bool	True	Require git.repository_url, git.commit.sha, and git.branch or git.tag in synced context data
require_bazel_metadata	bool	True	Require bazel_target_metadata.json next to selected payload outputs
require_json_payloads	bool	True	Require parseable .json payload files
forbid_full_bundle_no_match	bool	True	Fail when Go metadata reports bazel.go.payload_selection = "full_bundle_no_match". Valid selection values are module, module_override, full_bundle_disabled, and full_bundle_no_match; the last one is accepted only when this attr is False
forbid_msgpack_payloads	bool	True	Fail when tests emitted .msgpack or .msgpack.gz payloads instead of Bazel JSON payload files
forbid_dd_git_test_env	bool	True	Fail when workspace .bazelrc files pass sync metadata, upload credentials, or upload endpoints through --test_env; use --repo_env for sync metadata and pass credentials only to the uploader runtime
allowed_payload_selections	string_list	[]	Optional explicit allowlist for bazel.go.payload_selection; empty means module, module_override, and full_bundle_disabled are accepted, with full_bundle_no_match still rejected by default
expected_payload_selection_by_target	string_dict	{}	Optional map from local target label to the exact expected bazel.go.payload_selection value

Doctor notes:

• The doctor validates local files by default. When configured with --artifact-source=bep, it may stage BEP-referenced local files or outputs.zip carriers into a temporary staging directory before validation. HTTP/HTTPS remote outputs.zip carriers can be staged natively; non-HTTP remote providers or HTTP endpoints requiring custom auth still use the downloader contract. It does not upload, delete, or rewrite source payloads.
• When expected_targets is set, each listed test must have run before the doctor. If tests ran remotely, prefer --remote_download_minimal --remote_download_regex=.*test[.]outputs.* plus --zip_undeclared_test_outputs in the test config, and run the doctor with DD_TEST_OPTIMIZATION_ARTIFACT_SOURCE=bep so local outputs.zip carriers are extracted through BEP artifact staging before validation. If BEP references HTTP/HTTPS outputs.zip artifacts that were not downloaded locally, configure BEP artifact resolution with --remote-artifacts=download or required. If BEP references bytestream/CAS/custom-auth artifacts, also configure a downloader.
• The doctor scans versioned .bazelrc files for --test_env=DD_GIT_*, but it cannot see ad-hoc --test_env=DD_GIT_* flags typed directly on the CLI.
• The doctor prints a payload-selection summary so rollout owners can quickly see whether targets used per-module data, module overrides, or a disabled full bundle.

Doctor/uploader convenience macro

Macro: dd_test_optimization_targets(...)

Load from:

load("@datadog-rules-test-optimization//tools/core:test_optimization_targets.bzl", "dd_test_optimization_targets")

Use this helper when a consumer workspace wants the standard doctor/uploader pair without repeating context wiring. The macro can live in any package. In large monorepos, prefer a lightweight package such as //tools/test_optimization so running doctor or uploader does not analyze the workspace root package.

Attribute	Type	Default	Description
name	string	"test_optimization"	Logical helper name used for diagnostics; no target with this name is created
sync_repo_name	string	"test_optimization_data"	Repository exposing :test_optimization_context
doctor_name	string	"dd_test_optimization_doctor"	Generated doctor target name
uploader_name	string	"dd_upload_payloads"	Generated uploader target name
expected_targets	string_list	[]	Strict labels passed to the doctor. List only instrumented runtime test targets that emit payloads
context_data	label_list or None	["@<sync_repo>//:test_optimization_context"]	Explicit context data labels when the default sync repo label is not enough
doctor_kwargs	dict or None	{}	Extra attrs for dd_test_optimization_doctor; cannot override name, data, or expected_targets
uploader_kwargs	dict or None	{}	Extra attrs for dd_payload_uploader; cannot override name or data

Example:

dd_test_optimization_targets(
    name = "test_optimization",
    sync_repo_name = "test_optimization_data_python",
    expected_targets = [
        "//app:service_py_test",
    ],
    uploader_kwargs = {
        "max_wait_sec": 30,
        "quiescent_sec": 1,
    },
)

Python snippet generator

Tool: @datadog-rules-test-optimization-python//tools/dd_topt_py_bootstrap:dd_topt_py_bootstrap

The Python bootstrap is a snippet generator. It does not call Datadog APIs, does not execute tests, does not modify Python lockfiles, and does not declare pytest or ddtrace for the consumer. By default it prints all normal onboarding snippets for the selected mode. It writes files only when --write-bazelrc or --write-targets is passed.

Flag	Default	Description
--mode	required	workspace or bzlmod
--service	required	Datadog service name used by sync metadata fetch
--runtime-version	required	Python runtime version, for example 3.12
--runtime-module-path	required	Python module identifier used for runtime metadata
--sync-repo-name	test_optimization_data	Sync repository name used in generated labels
--rto-commit	empty	Published commit SHA used for git fetch snippets
--rto-remote	GitHub HTTPS remote	Rules repository remote for public git snippets
--datadog-fetch	git	git or archive
--private-repo-fetch	none	none, ssh-git, or authenticated-archive
--rto-archive-url / --rto-archive-sha256 / --rto-archive-prefix	empty	Required for archive fetch modes
--rules-python-repo-name	rules_python	Existing consumer repository name for rules_python
--test-optimization-package	tools/test_optimization	Package used by generated doctor/uploader commands
--doctor-name	dd_test_optimization_doctor	Doctor target name
--uploader-name	dd_upload_payloads	Uploader target name
--bazelrc-config	test-optimization	Config name used in .bazelrc and commands
--expected-target	repeatable	Labels that the doctor must validate; use only payload-emitting test targets
--test-target	repeatable	Labels used by generated bazel test commands; defaults to expected targets when omitted
--runner-mode	managed_pytest	managed_pytest or consumer_runner
--py-test-rule-load-label / --py-test-rule-symbol	empty	Consumer wrapper load used only for consumer_runner snippets
--bazel-command	bazel	Command printed in generated examples, such as bzl or ./bazelw
--print-workspace-snippet, --print-bzlmod-snippet, --print-bazelrc-snippet, --print-targets-snippet, --print-test-snippet, --print-command-snippet	false	Print only the selected snippet groups
--print-refresh-snippet	false	Print the separate force-refresh bazel sync ... --repo_env=FETCH_SALT="$(date +%s)" command
--write-bazelrc / --bazelrc-path	false / .bazelrc	Insert or replace the managed .bazelrc block
--write-targets / --targets-build-path	false / tools/test_optimization/BUILD.bazel	Insert or replace the managed doctor/uploader block

Generated .bazelrc blocks use common:<config> --repo_env=... for sync metadata plus test:<config> --remote_download_minimal and test:<config> --remote_download_regex=.*test[.]outputs.* plus test:<config> --zip_undeclared_test_outputs for local output availability in remote execution/cache setups. They do not include a fixed --build_event_json_file; wrappers should pass a fresh BEP path per Bazel test invocation. They do not include FETCH_SALT, DD_GIT_* test env, DD_API_KEY test env, upload endpoint test env, or DD_CIVISIBILITY_AGENTLESS_ENABLED.

How data is fetched

The sync rule executes HTTP requests with timeouts/retries to:

• Settings: https://api.<DD_SITE>/api/v2/libraries/tests/services/setting
• Known Tests: https://api.<DD_SITE>/api/v2/ci/libraries/tests
• Test Management Tests: https://api.<DD_SITE>/api/v2/test/libraries/test-management/tests

Settings response attributes determine follow-up requests:

• known_tests_enabled -> triggers Known Tests
• test_management.enabled -> triggers Test Management Tests

If a feature is disabled, the rule still writes a minimal stub JSON for the corresponding output file so consumers can always depend on stable filegroups.

Environment variables

All variables in this section are declared in repository-rule environ, so changes can invalidate fetch cache entries as expected.

Primary inputs

Variable	Required	Purpose
DD_API_KEY	Yes	Datadog API key for metadata fetches
DD_SITE	No	Site domain (datadoghq.com, datadoghq.eu, etc.). Leading/trailing ASCII whitespace is trimmed; values like app.<site> normalize to api.<site>
DD_TEST_OPTIMIZATION_AGENTLESS_URL	No	Override sync API base URL (test/dev and mock-server scenarios)
FETCH_SALT	No	Manual refetch trigger (example: --repo_env=FETCH_SALT=<timestamp>)
GO_MODULE_PATH	No	Explicit Go module path override used when emitting export.bzl
PYTHON_MODULE_PATH	No	Explicit Python module path override used when emitting export.bzl
JAVA_MODULE_PATH	No	Explicit Java module path override used when emitting export.bzl
NODEJS_MODULE_PATH	No	Explicit NodeJS module path override used when emitting export.bzl
DOTNET_MODULE_PATH	No	Explicit .NET module path override used when emitting export.bzl
RUBY_MODULE_PATH	No	Explicit Ruby module path override used when emitting export.bzl
DD_TEST_OPTIMIZATION_HTTP_CONNECT_TIMEOUT_SECONDS	No	Sync connect-timeout override
DD_TEST_OPTIMIZATION_HTTP_MAX_TIME_SECONDS	No	Sync per-request max-time override
DD_TEST_OPTIMIZATION_HTTP_RETRY_ATTEMPTS	No	Sync retry-attempt override
DD_TEST_OPTIMIZATION_HTTP_RETRY_DELAY_SECONDS	No	Sync retry-delay override
DD_TEST_OPTIMIZATION_HTTP_EXECUTE_TIMEOUT_BUFFER_SECONDS	No	Sync outer execute-timeout buffer override

DD_TEST_OPTIMIZATION_AGENTLESS_URL is the shared direct URL override used by:

• Sync metadata requests (repository rule phase), and
• Agentless uploader intake requests (bazel run //:dd_upload_payloads).

For Go/Orchestrion Bazel file mode, neither DD_TEST_OPTIMIZATION_AGENTLESS_URL nor DD_TEST_OPTIMIZATION_AGENT_URL belongs in the test sandbox. Put DD_TEST_OPTIMIZATION_AGENTLESS_URL in --repo_env only when the sync metadata fetch or uploader needs the override, and pass DD_TEST_OPTIMIZATION_AGENT_URL only at uploader runtime if you use EVP proxy mode.

Note: FETCH_SALT_TTL is a convenience variable for this repository's ./bazelw wrapper (not a repository-rule input itself). It periodically derives --repo_env=FETCH_SALT=... for local maintainer workflows.

Datadog Git metadata overrides (highest precedence)

When set, these override auto-detected CI/git metadata:

Variable
DD_GIT_REPOSITORY_URL
DD_GIT_BRANCH
DD_GIT_TAG
DD_GIT_COMMIT_SHA
DD_GIT_HEAD_COMMIT
DD_GIT_COMMIT_MESSAGE
DD_GIT_HEAD_MESSAGE
DD_GIT_COMMIT_AUTHOR_NAME
DD_GIT_COMMIT_AUTHOR_EMAIL
DD_GIT_COMMIT_AUTHOR_DATE
DD_GIT_COMMIT_COMMITTER_NAME
DD_GIT_COMMIT_COMMITTER_EMAIL
DD_GIT_COMMIT_COMMITTER_DATE
DD_GIT_HEAD_AUTHOR_NAME
DD_GIT_HEAD_AUTHOR_EMAIL
DD_GIT_HEAD_AUTHOR_DATE
DD_GIT_HEAD_COMMITTER_NAME
DD_GIT_HEAD_COMMITTER_EMAIL
DD_GIT_HEAD_COMMITTER_DATE
DD_GIT_PR_BASE_BRANCH
DD_GIT_PR_BASE_BRANCH_SHA
DD_GIT_PR_BASE_BRANCH_HEAD_SHA
DD_PR_NUMBER

CI provider detection coverage

Auto-detection currently maps CI metadata from:

• GitHub Actions
• GitLab CI
• Jenkins
• CircleCI
• Azure Pipelines
• Buildkite
• Travis CI
• Bitbucket
• AppVeyor
• TeamCity
• Bitrise
• Codefresh
• AWS CodePipeline
• Drone

Provider-name note: AWS CodePipeline is emitted as awscodepipeline in sync metadata (ci.provider.name).

Provider precedence note:

• CI-provider environment detection and event payloads are collected first;
• local Git fallback fills only fields that are still empty;
• explicit DD_GIT_* overrides win last.

When require_git_metadata = True, sync fails before the settings request if repository URL, branch, or commit SHA are still missing. Detached HEAD checkouts must pass DD_GIT_BRANCH explicitly when strict mode is enabled.

Additional mapped metadata inputs include:

• APPVEYOR_PULL_REQUEST_HEAD_REPO_BRANCH
• CI_PROJECT_PATH
• GITHUB_WORKFLOW
• TRAVIS_JOB_WEB_URL
• BUILD_URL

Uploader runtime environment variables

The doctor and/or uploader runtimes read these variables at bazel run time:

Variable	Purpose
DD_API_KEY	Required for agentless uploads
DD_SITE	Required for agentless uploads (intake host derivation)
DD_TEST_OPTIMIZATION_AGENT_URL	Enables EVP proxy mode
DD_TEST_OPTIMIZATION_AGENTLESS_URL	Optional agentless intake base override for test/dev setups
DD_TEST_OPTIMIZATION_KEEP_PAYLOADS	Keep payload files after successful upload
DD_TEST_OPTIMIZATION_FILTER_PREFIX	0 uploads all payloads; 1 restricts to span_events_*.json / coverage_*.json
DD_TEST_OPTIMIZATION_DEBUG	Enable verbose uploader logs
DD_TEST_OPTIMIZATION_GZIP	Gzip test payloads before upload
DD_TEST_OPTIMIZATION_MAX_WAIT_SEC	Override uploader max wait
DD_TEST_OPTIMIZATION_QUIESCENT_SEC	Override uploader quiescence wait
DD_TEST_OPTIMIZATION_MAX_DEPTH	Limit payload discovery depth in large trees
DD_TEST_OPTIMIZATION_CODEOWNERS_FILE	Explicit CODEOWNERS path for enrichment
DD_TEST_OPTIMIZATION_BEP_JSON	BEP JSON file from the matching bazel test --build_event_json_file=... invocation
DD_TEST_OPTIMIZATION_FRESHNESS_SOURCE	Freshness source: auto, bep, or execution_log
DD_TEST_OPTIMIZATION_FRESHNESS_MODE	Freshness mode: auto, required, optional, or disabled
DD_TEST_OPTIMIZATION_DOCTOR_REPORT_JSON	Optional path for the doctor machine-readable diagnostic report
DD_TEST_OPTIMIZATION_UPLOADER_REPORT_JSON	Optional path for the uploader machine-readable diagnostic report
DD_TEST_OPTIMIZATION_REPORT_DIR	Optional wrapper/report-script directory. CI wrappers write doctor-report.json, uploader-dry-run-report.json, and, when upload is enabled, uploader-upload-report.json under this directory unless explicit report paths override the doctor or dry-run uploader path
DD_TEST_OPTIMIZATION_SUPPORT_BUNDLE	Optional doctor or wrapper path for the redacted support diagnostics zip
DD_TEST_OPTIMIZATION_SUPPORT_BUNDLE_COLLECTOR	Optional override for the support bundle collector script. Doctor targets provide this through runfiles; wrappers default to create_support_bundle.py beside the wrapper
DD_TEST_OPTIMIZATION_PYTHON	Optional Python interpreter used by wrapper support-bundle generation and helper scripts before falling back to PYTHON, python3, and python
DD_TEST_OPTIMIZATION_ARTIFACT_SOURCE	Artifact discovery source: local, bep, or auto. Recommended CI with zipped undeclared outputs should set bep
DD_TEST_OPTIMIZATION_REMOTE_ARTIFACTS	Remote BEP artifact handling: disabled, download, or required. HTTP/HTTPS outputs.zip carriers can be staged natively when enabled
DD_TEST_OPTIMIZATION_ARTIFACT_STAGING_DIR	Directory used for per-run staged BEP artifacts
DD_TEST_OPTIMIZATION_BEP_ARTIFACT_DOWNLOADER	Optional executable used to download non-HTTP remote/CAS BEP artifacts or HTTP endpoints requiring custom auth
DD_TEST_OPTIMIZATION_BEP_ARTIFACT_DOWNLOADER_TIMEOUT_SEC	Timeout for the BEP artifact downloader and native HTTP/HTTPS request attempts
DD_TEST_OPTIMIZATION_EXECUTION_LOG_JSON	Optional explicit legacy execution-log fallback path; default execution-log files are not auto-discovered
DD_TEST_OPTIMIZATION_EXECUTION_LOG_MODE	Legacy alias for freshness mode when DD_TEST_OPTIMIZATION_FRESHNESS_MODE is unset
TESTLOGS_DIR	Explicit bazel-testlogs path for non-standard layouts

Uploader CLI flags:

Flag	Purpose
--dry-run	Enrich and validate discovered payloads without uploading or deleting files
--validate-enrichment	In dry-run mode, require key Git and Bazel tags to exist after enrichment
--expected-enriched-tag=<tag>	Add a required enriched tag; repeatable. Defaults cover git.repository_url, git.commit.sha, bazel.target, and bazel.package. Add bazel.go.payload_selection explicitly when a Go rollout must prove per-module selection
--bep-json=<path>	BEP JSON file from the matching Bazel test invocation; repeat for multiple invocations
--freshness-source=<source>	Freshness source: auto, bep, or execution_log
--freshness-mode=<mode>	Freshness mode: auto, required, optional, or disabled
--allow-cached-payload-uploads	Explicitly disables BEP and legacy execution-log freshness filtering
--execution-log-json=<path>	Explicit legacy execution-log fallback path
--execution-log-mode=<mode>	Legacy alias for freshness mode
--artifact-source=<source>	Artifact discovery source: local, bep, or auto. Default local scans local bazel-testlogs; bep stages BEP-referenced test.outputs directories or local outputs.zip carriers before discovery; auto can stage BEP artifacts while preserving local discovery fallback
--remote-artifacts=<mode>	Remote BEP artifact handling: disabled, download, or required. Default disabled never downloads remote artifacts. download stages local/file and native HTTP/HTTPS carriers and invokes a configured downloader for custom remote providers; unresolved remote artifacts are skipped only outside required BEP freshness. In the recommended --freshness-mode=required CI flow, every selected fresh remote-only artifact must be materialized or the command fails. required fails if any selected fresh BEP artifact cannot be materialized
--artifact-staging-dir=<path>	Directory used for per-run staged BEP artifacts; wrapper examples create a temporary staging directory per run
--bep-artifact-downloader=<path>	Executable used to download non-HTTP remote/CAS BEP artifacts or HTTP endpoints requiring custom auth. It receives --uri, --name, and --output, and must write an outputs.zip archive to --output. Plain HTTP/HTTPS outputs.zip carriers do not need this flag
--bep-artifact-downloader-timeout-sec=<seconds>	Timeout for each BEP artifact downloader invocation and each native HTTP/HTTPS request attempt
--report-json=<path>	Write a machine-readable uploader diagnostic report on success, no-op, or controlled uploader failure

Doctor CLI flags:

Flag	Purpose
--bep-json=<path>	BEP JSON file from the matching Bazel test invocation; repeat for multiple invocations
--freshness-source=<source>	Freshness source: auto, bep, or execution_log. Use bep in CI; the doctor only performs BEP freshness validation and rejects required execution-log validation
--freshness-mode=<mode>	Freshness mode: auto, required, optional, or disabled
--artifact-source=<source>	Artifact discovery source: local, bep, or auto
--remote-artifacts=<mode>	Remote BEP artifact handling: disabled, download, or required. HTTP/HTTPS outputs.zip carriers can be staged natively when enabled
--artifact-staging-dir=<path>	Directory used for per-run staged BEP artifacts
--bep-artifact-downloader=<path>	Executable used to download non-HTTP remote/CAS BEP artifacts or HTTP endpoints requiring custom auth
--bep-artifact-downloader-timeout-sec=<seconds>	Timeout for each BEP artifact downloader invocation and each native HTTP/HTTPS request attempt
--report-json=<path>	Write a machine-readable doctor diagnostic report on success or controlled doctor failure
--support-bundle=<path> / DD_TEST_OPTIMIZATION_SUPPORT_BUNDLE	Write a redacted doctor-only support diagnostics zip. This is the simplest support request after a customer has run tests. It includes the doctor report, selected BEP summaries, effective doctor flags, runtime metadata, and no uploader report

Wrapper report options:

Option / variable	Purpose
--report-dir=<path> / -ReportDir <path> / DD_TEST_OPTIMIZATION_REPORT_DIR	Recommended CI artifact directory. Bash and PowerShell wrappers write separate doctor, dry-run uploader, and upload reports under this directory
--doctor-report-json=<path> / -DoctorReportJson <path> / DD_TEST_OPTIMIZATION_DOCTOR_REPORT_JSON	Override only the wrapper doctor report path
--uploader-report-json=<path> / -UploaderReportJson <path> / DD_TEST_OPTIMIZATION_UPLOADER_REPORT_JSON	Override only the wrapper dry-run uploader report path. Real upload still uses <report-dir>/uploader-upload-report.json when --report-dir is set
--support-bundle=<path> / -SupportBundle <path> / DD_TEST_OPTIMIZATION_SUPPORT_BUNDLE	Write a redacted support diagnostics zip containing reports, selected BEP summaries, command metadata, runtime metadata, and a Markdown summary
--support-bundle-collector=<path> / -SupportBundleCollector <path> / DD_TEST_OPTIMIZATION_SUPPORT_BUNDLE_COLLECTOR	Override the support bundle collector path. Defaults to create_support_bundle.py beside the wrapper

The doctor support bundle is built into dd_test_optimization_doctor runfiles, so customers do not need to copy helper scripts just to create a first-pass diagnostics package. The wrapper support bundle still requires the full tools/test_optimization/ helper directory or an explicit DD_TEST_OPTIMIZATION_SUPPORT_BUNDLE_COLLECTOR path. Both modes are opt-in and do not change the doctor, test, dry-run, upload, or final CI exit status. Bundle generation failures are reported as warnings.

Bundle file	Purpose
diagnostics.json	Machine-readable index and primary reason code
summary.md	Human-readable support summary
reports/*.json	Redacted and bounded doctor/uploader reports
bep/*.summary.json	Redacted BEP test-result summaries; raw BEP is not included
command/flags.json	Effective doctor or wrapper targets and Test Optimization flags
environment/runtime.json	OS, Python, and Bazel version metadata
redaction-manifest.json	Description of redaction rules applied

tools/test_optimization/create_support_bundle.py remains available for manual fallback flows that collect one or more doctor or uploader JSON reports without using the doctor or wrapper support-bundle entrypoints. tools/test_optimization/render_report_summary.py is the Markdown-only fallback when a zip bundle is not available. For the support-facing triage order, see docs/Troubleshooting.md.

Uploader execution modes:

Scenario	Test command shape	Doctor/uploader flags
Local development, no cache-safety requirement	Normal local bazel test that leaves bazel-testlogs/**/test.outputs on disk	Defaults are enough, or pass --freshness-mode=disabled / --allow-cached-payload-uploads intentionally
Recommended CI	.bazelrc includes --remote_download_minimal, --remote_download_regex=.*test[.]outputs.*, and --zip_undeclared_test_outputs; wrapper passes a fresh --build_event_json_file=... to each Bazel test invocation	Wrapper passes repeatable --bep-json=<path>, --freshness-source=bep, --freshness-mode=required, --artifact-source=bep, and --artifact-staging-dir=<temp-dir> to doctor/uploader
CI without zipped outputs	Same BEP-producing test command, but no --zip_undeclared_test_outputs	BEP freshness flags are enough; --artifact-source=bep is still valid and stages BEP-referenced local directories
HTTP/HTTPS artifacts not present locally	Same BEP-producing test command; selective download did not materialize test.outputs locally and BEP points at http:// or https:// outputs.zip carriers	Add --artifact-source=bep --remote-artifacts=download; use --remote-artifacts=required for strict all-or-nothing behavior
Custom remote/CAS artifacts not present locally	Same BEP-producing test command; selective download did not materialize test.outputs locally and BEP points at bytestream/CAS/internal APIs or HTTP endpoints requiring custom auth	Add --artifact-source=bep --remote-artifacts=download --bep-artifact-downloader=/path/to/downloader, or use --remote-artifacts=required for strict all-or-nothing behavior
Transitional local+BEP staging	BEP-producing test command with some local outputs and some BEP-stageable carriers	--artifact-source=auto --remote-artifacts=download lets staged BEP outputs win over stale local directories with the same BEP output key

Numeric precision caveat:

• Keep high-cardinality IDs (for example CI job IDs) as strings when possible. Some JSON tooling and runtimes may lose precision for integers above 2^53 - 1.

Integration harness environment variables (maintainer workflows)

These variables are used by repository integration harness scripts (not by the sync repository rule or uploader runtime paths above):

• Linux/macOS harness supports existing shell env overrides documented in tools/tests/integration/run_mock_server_tests.sh.
• Windows harness is PowerShell-only; no Git Bash path override variable is used.