diff --git a/cloudformation/templates/1-main.yml b/cloudformation/templates/1-main.yml index d8ecd86e..c7bff6f8 100644 --- a/cloudformation/templates/1-main.yml +++ b/cloudformation/templates/1-main.yml @@ -58,11 +58,13 @@ Metadata: - ResponseHeadersPolicy - Label: default: App CodePipeline information - Parameters: + Parameters: - GitHubConnectionArn - GitHubRepo - GitHubBranch - CreateS3Bucket + - BundlesGitHubRepo + - BundlesGitHubTriggerBranch - ViteAlternateMatomoUrl - ViteAlternateMatomoSiteId - ViteAlternateMatomoSubdomain @@ -202,6 +204,12 @@ Parameters: GitHubConnectionArn: Description: Can be found or created in CodePipeline console, settings, connections. Type: String + BundlesGitHubRepo: + Type: String + Description: GitHub repo name for the bundles repo (e.g. edanalytics/earthmover_edfi_bundles) + BundlesGitHubTriggerBranch: + Type: String + Description: Branch of the bundles repo whose merges should trigger an executor image rebuild. CreateS3Bucket: Type: String Description: Create the S3 bucket named 'codepipeline-artifact-store-{AWS::Region}-{AWS::AccountId}'? Do not change this value if updating an existing stack. @@ -467,6 +475,8 @@ Resources: GitHubBranch: !Ref GitHubBranch GitHubConnectionArn: !Ref GitHubConnectionArn GitHubRepo: !Ref GitHubRepo + BundlesGitHubRepo: !Ref BundlesGitHubRepo + BundlesGitHubTriggerBranch: !Ref BundlesGitHubTriggerBranch CreateS3Bucket: 'The S3 Bucket already exists' EcsStack: Type: AWS::CloudFormation::Stack @@ -497,4 +507,4 @@ Resources: Outputs: TemplateVersion: Description: The version of the cloudformation template - Value: '2.0.2' + Value: '2.0.3' diff --git a/cloudformation/templates/code-pipeline-executor.yml b/cloudformation/templates/code-pipeline-executor.yml index a6f903f6..197221ea 100644 --- a/cloudformation/templates/code-pipeline-executor.yml +++ b/cloudformation/templates/code-pipeline-executor.yml @@ -14,6 +14,12 @@ Parameters: GitHubBranch: Type: String Description: Name of the branch in the GitHub repo to build from. + BundlesGitHubRepo: + Type: String + Description: GitHub repo name for the bundles repo + BundlesGitHubTriggerBranch: + Type: String + Description: Branch of the bundles repo whose merges should trigger a rebuild. CreateS3Bucket: Type: String Description: Create the S3 bucket named 'job-executor-codepipeline-artifact-store-{AWS::Region}-{AWS::AccountId}'? Do not change this value if updating an existing stack. @@ -119,6 +125,13 @@ Resources: Includes: - 'executor/**' ProviderType: CodeStarSourceConnection + - GitConfiguration: + SourceActionName: BundlesSource + Push: + - Branches: + Includes: + - !Ref BundlesGitHubTriggerBranch + ProviderType: CodeStarSourceConnection Stages: - Name: SourceStage Actions: @@ -137,6 +150,21 @@ Resources: OutputArtifacts: - Name: SourceArtifact Region: !Ref 'AWS::Region' + - Name: BundlesSource + ActionTypeId: + Category: Source + Owner: AWS + Provider: CodeStarSourceConnection + Version: 1 + RunOrder: 1 + Configuration: + BranchName: !Ref BundlesGitHubTriggerBranch + ConnectionArn: !Ref GitHubConnectionArn + FullRepositoryId: !Ref BundlesGitHubRepo + OutputArtifactFormat: CODE_ZIP + OutputArtifacts: + - Name: BundlesArtifact + Region: !Ref 'AWS::Region' - Name: BuildStage Actions: - Name: Build @@ -160,6 +188,7 @@ Resources: - Name: BuildArtifact InputArtifacts: - Name: SourceArtifact + - Name: BundlesArtifact # codebuild requires this be declared Region: !Ref 'AWS::Region' Namespace: BuildVariables diff --git a/cloudformation/templates/x-parameter-values.md b/cloudformation/templates/x-parameter-values.md index 39e8057d..3b57763c 100644 --- a/cloudformation/templates/x-parameter-values.md +++ b/cloudformation/templates/x-parameter-values.md @@ -60,6 +60,8 @@ Please note that descriptions, values, and available parameters may evolve over - **GitHubConnectionArn** - ARN of the GitHub Connection that was created as a prerequisite to deploying Runway. - **GitHubRepo** - Name of the forked Runway Github repository. - **GitHubBranch** - Name of the branch in the GitHub repo to build from. +- **BundlesGitHubRepo** - GitHub repo name for the bundles repo +- **BundlesGitHubTriggerBranch** - Branch of the bundles repo whose merges should trigger a rebuild - **CreateS3Bucket** - Choose whether to create S3 bucket named 'codepipeline-artifact-store-{AWS::Region}-{AWS::AccountId}' in your AWS account. Only choose `Create the S3 bucket` for the very first deployment of Runway in your AWS account. Do not change this value if updating an existing stack. - **ViteAlternateMatomoUrl** - Internal to EA. Not needed for open-source deployments. Leave blank. - **ViteAlternateMatomoSiteId** - Internal to EA. Not needed for open-source deployments. Leave blank. diff --git a/executor/executor/executor.py b/executor/executor/executor.py index a156a82a..3401a51c 100644 --- a/executor/executor/executor.py +++ b/executor/executor/executor.py @@ -208,9 +208,13 @@ def refresh_bundle_code(self): subprocess.run( ["git", "-C", config.BUNDLE_DIR, "pull", "--ff-only"] ).check_returncode() - except subprocess.CalledProcessError: - self.error = error.GitPullError() - raise + except subprocess.CalledProcessError as err: + if self.local_mode: + self.error = error.GitPullError() + raise + self.logger.warning( + f"bundle refresh failed in deployed mode; assuming bundle code is already up to date: {err}" + ) def earthmover_deps(self): """Create the Earthmover runtime environment by installing bundle dependencies"""