+System tests
this step will then only test that the application starts successfully. See production deployment for an example without tests. ++ If the production zones span multiple cloud providers (e.g., both AWS and GCP), + system tests are run separately for each cloud provider, + using test nodes from that provider. + This ensures the application starts and works correctly on each provider's infrastructure + before production deployment. +
Read more about system tests.
@@ -90,6 +100,10 @@Staging tests
this step will then only test that the application starts successfully before and after the change. See production deployment for an example without tests. ++ Like system tests, staging tests are run separately for each cloud provider + when the production zones span multiple providers. +
Read more about staging tests.
@@ -134,7 +148,6 @@Running tests only
Deployment orchestration
-The deployment orchestration is flexible. One can configure dependencies between deployments to production zones, @@ -143,6 +156,188 @@
Deployment orchestration
+
+Pipeline graph
++ The deployment pipeline is visualized as a graph in the + Vespa Cloud Console. + Each node represents a step in the pipeline, + and edges show dependencies between steps. + Hover over any node to see details and available actions. +
+ +Node shapes
+| Shape | +Step type | +Description | +
|---|---|---|
| + + | +Instance | ++ The application instance. Hover to see target versions, cancel/deploy/pin controls, and block windows. + | +
| + + | +Test | ++ System test, staging test, or production test. + Hover to see run status, versions, and abort/restart actions. + | +
| + + | +Production deployment | ++ A deployment to a production zone. + Hover to see run status, versions, and abort/restart/defer actions. + | +
| + + | +Delay | ++ A configured delay between steps. + | +
Visual indicators
+| Indicator | +Meaning | +Description | +
|---|---|---|
| + + | +Completed | +The step has completed successfully on the current version. The color corresponds to the deployed version. | +
| + + | +Running | +A deployment or test is currently in progress. Shown as an animated gradient between the source and target version colors. | +
| + + | +Failed | +The last run of this step failed. | +
| + + | +Unknown / initial | +No version has been deployed to this step yet. | +
| + + | +Pending change | +A newer version is queued and waiting to be deployed to this step. | +
| + + | +Paused / deferred | +Deployments to this step are temporarily postponed. | +
| + + | +Application blocked | +Application changes are blocked by a block window. Shown as vertical bars. | +
| + + | +Platform blocked | +Platform upgrades are blocked by a block window. Shown as horizontal bars. | +
+ Each version deployed through the pipeline is assigned a distinct color. + This makes it easy to see at a glance which zones are on the same version + and where a rollout is in progress. + A thumbtack icon on a node indicates that the version is + pinned. +
+On a higher level, instances can also depend on each other in the same way. This makes it easy to configure a deployment process @@ -168,6 +363,97 @@
Deployment orchestration
must always be successfully run before the application package is deployed to production zones. + +Version progression
++ The deployment pipeline deploys one revision at a time through the production zones. + When a revision is being deployed, it must complete deployment to all declared production zones + before the next revision begins its production rollout. + System and staging tests for newer revisions may run in parallel, + but production deployment is serialized. +
++ For example, if build 90 is being deployed to the second of two production zones, + build 91 will not start deploying to the first zone until build 90 has completed in all zones — + even if build 91 has already passed system and staging tests. +
+ +Superseding a version
++ To override the currently deploying revision and force a newer build through the pipeline, + hover over the instance node in the pipeline graph and use the TARGET VERSIONS controls. + Select the desired build number from the revision dropdown and click deploy. + This updates the instance's deployment target. + Any running production job for the old revision will be aborted, + and the pipeline will start deploying the new revision from the first production zone. +
+
++ To cancel the currently deploying revision without selecting a new one, + click cancel. + This lets the pipeline pick the next revision automatically. +
+ +Pinning versions
++ Pinning locks the pipeline to a specific platform version or application revision, + preventing automatic upgrades. + This is useful for forcing a downgrade, holding a known-good revision during an incident, + or preventing the system from picking up a new platform version. +
++ To pin a version, hover over the instance node in the pipeline graph. + Under TARGET VERSIONS, select the desired version from the dropdown + and click pin. + A reason is required — enter a description and click submit pin. + Platform and revision can be pinned independently. +
+
++ While pinned, no newer platform versions or revisions will be deployed for the pinned dimension. + The dropdown and deploy button are disabled to prevent accidental changes. + To unpin, hover over the instance node and click unpin, + which allows newer versions to move through the pipeline again. +
++ For example, to roll back to a previous revision: +
+-
+
- Select the older build number from the revision dropdown. +
- Click pin and provide a reason (e.g., "rollback due to regression in build 91"). +
- The pipeline will deploy the pinned build to all production zones. +
- Once the issue is resolved, click unpin to resume normal deployments. +
Cooldown after failures
++ When a production deployment fails repeatedly, an exponential cooldown is applied before + the job is automatically retried. The cooldown period grows with the time between the first + failure and the last completed run. This prevents the system from continuously retrying + a failing deployment. +
++ The cooldown applies only when the target versions match those of the failing runs. + If the target changes (e.g., a new revision is set as the deployment target), + the cooldown resets and the new revision can be deployed immediately. +
++ To manually re-trigger a failed deployment and bypass the cooldown, + hover over the failed zone node in the pipeline graph and click restart. +
+
+
+Pausing deployments to a zone
++ To temporarily hold off deployments to a specific production zone, + hover over the zone node in the pipeline graph and click defer. + This postpones deployments for 72 hours. + Click enable to resume scheduling before the deferral period expires. +
+Source code repository integration
Each new submission is assigned an increasing build number, @@ -207,23 +493,34 @@
