ref(pipedream): group regions into parallel jobs per pipeline#92
Merged
mchen-sentry merged 18 commits intomainfrom Apr 27, 2026
Merged
ref(pipedream): group regions into parallel jobs per pipeline#92mchen-sentry merged 18 commits intomainfrom
mchen-sentry merged 18 commits intomainfrom
Conversation
Optimize env var handling so shared vars stay at stage level while region-specific vars cascade to jobs. Cache pipeline_fn results, extract get_matching_stage helper. Align s4s group to ['s4s2'] only. Add rollback-final-stage-override fixture and regenerate goldens.
…rties GoCD only supports stage-level attributes (approval, fetch_materials, etc.) with no per-job override. When regions in a group define conflicting stage properties, we now assert at build time rather than silently using the first region's values.
In Jsonnet, a missing comma between stage definitions silently merges them into a single object, causing stages to be lost. This assertion catches the issue at build time by validating that each stage object has exactly one key.
s4s is no longer a deployment group. Remove from pipeline_groups and group_order, update all test fixtures and golden files.
s4s2 is a prod region that was incorrectly removed when the s4s test group was dropped. Restore it as a single-region group in pipeline_groups.
…fix README and cleanup - Restore fetch_materials: false on both pipeline-complete stages (trigger and group pipelines) that was accidentally dropped during the rewrite - Fix README example to use s4s2 group name instead of s4s - Remove working notes from .gitignore (files already deleted) - Fix misleading comment in env-vars-precedence test fixture - Regenerate golden files
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
s4s was removed as a region (PR #91) and replaced by s4s2. Update is_st() to check for s4s2 instead of the defunct s4s region. Add s4s2 to the is-st test fixture. Also fix jsonnetfmt formatting in pipedream.libsonnet (indentation and quote style in assertion messages).
s4s no longer exists as a region, and s4s2 uses the SaaS-style multi-container deploy (not ST), so the s4s check was dead code. Simplify is_st to only match customer-* regions.
![sentry[bot]](https://avatars.githubusercontent.com/in/12637?s=60&v=4){kind=small}
Remove reference to specific baseline region since the baseline may not be regions[0] when regions have different stage sets.
Assert that materials, lock_behavior, and other pipeline-level attributes are consistent across all regions in a group, matching the existing stage-level property validation.
mchen-sentry
added a commit
to getsentry/eng-pipes
that referenced
this pull request
Apr 15, 2026
Pipedream v3 collapses deploy-snuba-{py,rs}-customer-{1,2,4,7} into
deploy-snuba-{py,rs}-st. Add the new st pipeline names to Slack feed
filters and Datadog region mappings. Keep old customer-* names for
backward compat during staged rollout.
Also remove dead deploy-snuba-{py,rs}-s4s entries (s4s test region was
removed in gocd-jsonnet).
Ref: getsentry/gocd-jsonnet#92
mchen-sentry
added a commit
to getsentry/eng-pipes
that referenced
this pull request
Apr 15, 2026
Pipedream v3 collapses deploy-snuba-{py,rs}-customer-{1,2,4,7} into
deploy-snuba-{py,rs}-st. Add the new st pipeline names to Slack feed
filters and Datadog region mappings. Keep old customer-* names for
backward compat during staged rollout.
Also remove dead deploy-snuba-{py,rs}-s4s entries (s4s test region was
removed in gocd-jsonnet).
Ref: getsentry/gocd-jsonnet#92
mchen-sentry
added a commit
to getsentry/snuba
that referenced
this pull request
Apr 15, 2026
Bumps gocd-jsonnet from v2.19.0 to v3.0.0 which changes the pipedream
model from one pipeline per region to one pipeline per group. The four
customer region pipelines (customer-1, -2, -4, -7) collapse into a
single deploy-snuba-{py,rs}-st pipeline with parallel jobs.
Also updates PIPELINE_FIRST_STEP from deploy-snuba-{py,rs}-s4s (no
longer exists) to deploy-snuba-{py,rs}-s4s2 (the first group pipeline).
Ref: getsentry/gocd-jsonnet#92
dmajere
approved these changes
Apr 20, 2026
In grouped pipedream, triggering a group pipeline deploys to every region in the group in parallel. This adds a way to target a subset of those regions at trigger time without rendering per-region pipelines. The library now sets PIPEDREAM_GROUP_REGIONS automatically on each group pipeline (a comma-separated list of the regions in that group). Every region-keyed script task in the group pipeline starts with a POSIX-shell preamble that exits 0 cleanly when SENTRY_REGION isn't in the list. To target a subset, override PIPEDREAM_GROUP_REGIONS on GoCD's "Trigger with options" Environment Variables tab; default behavior is unchanged. Notes: - The gate stays POSIX (single-bracket [, no [[) so it works under dash, which is /bin/sh on most Linux GoCD agents. - Only gocdtasks.script(...) tasks are wrapped. exec-shaped tasks (noop etc.) pass through unwrapped. - Trigger and rollback pipelines are not gated.
mchen-sentry
force-pushed
the
iw/grouped-pipedream
branch
from
7408c77 to
f471ace
Compare
# Conflicts: # libs/getsentry.libsonnet # libs/pipedream.libsonnet # test/testdata/goldens/getsentry/regions.jsonnet_output-files.golden
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 1d56a8c. Configure here.
| if std.objectHas(task, 'script') then | ||
| { script: deploy_target_gate + task.script } | ||
| else | ||
| task; |
There was a problem hiding this comment.
wrap_task drops non-script properties from task objects
Medium Severity
wrap_task returns { script: deploy_target_gate + task.script }, a brand-new object containing only the script key. Any other properties on the original task (e.g., run_if, timeout, working_directory) are silently dropped. Using task { script: deploy_target_gate + task.script } would preserve all original properties while prepending the gate.
Reviewed by Cursor Bugbot for commit 1d56a8c. Configure here.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds region grouping support to gocd-jsonnet. Note this change has been deployed and verified.
The pipedream deployment model will be moving from one GoCD pipeline per region to one pipeline per region group. Regions within a group run as parallel jobs in a single pipeline. The single-tenant customer regions (customer-1, customer-2, customer-4, customer-7) will be collapsed into a single
stregion groupdeploy-{service}-stpipeline with four parallel jobs.Current single-region groups like
deandusare unchanged in behavior. They produce one pipeline with one job, same as before. This change is backwards compatible and thepipeline_fn(region)callback contract is unchanged. Services still return a pipeline definition for a given region. The difference is only in how pipedream aggregates those outputs; no service template code changes are required.One item to note is that GoCD stage-level attributes (approval, fetch_materials, etc.) don't support per-job overrides, so all regions in a group must agree on these properties for a given stage. Environment variables identical across all regions stay at the stage level, while region-specific values cascade to the job level. GoCD's native precedence (job > stage > pipeline) means there's no behavioral change from the old model. In practice we don't make use of per-job environment variables, but still worth mentioning here.
This change has been verified in devinfra-example-service. Planning on a staged rollout.