[WIP] Add result Lambda function and related resources#436
Closed
can-gaa-hou wants to merge 9 commits intopytorch:crcr-l1from
Closed
[WIP] Add result Lambda function and related resources#436can-gaa-hou wants to merge 9 commits intopytorch:crcr-l1from
can-gaa-hou wants to merge 9 commits intopytorch:crcr-l1from
Conversation
**Summary**: - Add Terraform infrastructure for CRCR (Cross-Repository CI Relay), a GitHub webhook relay service for PyTorch out-of-tree backends that receives upstream webhook events via a GitHub App and forwards `repository_dispatch` events to registered downstream repositories - Infrastructure includes: Lambda function (webhook handler), ElastiCache Redis (allowlist caching), dedicated VPC, IAM roles, and Lambda Function URL - Add two GitHub Actions workflows: `crcr-on-pr.yml` and `crcr-deploy-prod.yml` **Architecture**: GitHub App → Lambda webhook (Function URL) → `repository_dispatch` → downstream repos - Lambda function (`cross_repo_ci_webhook`) with Python 3.10 runtime - ElastiCache Redis replication group (`cache.t3.small`) for allowlist caching - VPC with private subnets for Lambda ↔ Redis connectivity - IAM role with Secrets Manager, VPC networking, and CloudWatch Logs permissions - S3 backend for Terraform state **Test**: Multiple deployments and verifications have been completed on personal AWS environment. Co-authored-by: can-gaa-hou <jiahaochen535@gmail.com>
- Fix some bugs about elasticache - update README.md to correspond to the crcr code.
- creating secret manager via terraform rather than creating manually - Moving REDIS_LOGIN from environment to secret manager - Optimize all terraform label and name to be clearer and more understanding
As the title stated.
- Replaced jeanschmidt/terraform-aws-vpc with the upstream terraform-aws-modules/terraform-aws-vpc (v6.6.1, the latest release). - Since the AWS provider was bumped to >= 6.28 (required by the new VPC module), invoked_via_function_url is now natively supported. Replaced the aws_cloudformation_stack workaround with native aws_lambda_permission resources. - Flattened the directory structure from aws/<account>/<region>/ to aws/ for two reasons: first, CRCR's canary environment is deployed under personal AWS account, and the previous structure required code changes just to switch accounts; second, CRCR doesn't need cross-region deployment, so the nested structure added unnecessary complexity.
12 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.