diff --git a/lib/polygon/.gitignore b/lib/polygon/.gitignore new file mode 100644 index 00000000..f60797b6 --- /dev/null +++ b/lib/polygon/.gitignore @@ -0,0 +1,8 @@ +*.js +!jest.config.js +*.d.ts +node_modules + +# CDK asset staging directory +.cdk.staging +cdk.out diff --git a/lib/polygon/.npmignore b/lib/polygon/.npmignore new file mode 100644 index 00000000..c1d6d45d --- /dev/null +++ b/lib/polygon/.npmignore @@ -0,0 +1,6 @@ +*.ts +!*.d.ts + +# CDK asset staging directory +.cdk.staging +cdk.out diff --git a/lib/polygon/README.md b/lib/polygon/README.md new file mode 100644 index 00000000..147b4209 --- /dev/null +++ b/lib/polygon/README.md @@ -0,0 +1,276 @@ +# Sample AWS Blockchain Node Runner app for Polygon PoS Nodes + +| Contributed by | +|:--------------------:| +| [@snese](https://github.com/snese) | + +## Architecture Overview + +This blueprint has two options for running Polygon PoS nodes using **Erigon**. You can set up a single RPC node or multiple nodes in a highly available setup. Instead of running a local Heimdall node, Erigon connects to Polygon's official Heimdall API endpoint. + +> **Why Erigon?** The traditional Polygon setup requires two containers: Heimdall (consensus) and Bor (execution). The ongoing Heimdall v1→v2 migration makes this dual-container setup unreliable. Erigon provides a simpler, single-container alternative that connects to Polygon's official Heimdall API, avoiding the migration complexity entirely. + +### Single node setup + +The setup deploys a single EC2 instance running one Erigon container. The RPC port (8545) is exposed only to the internal VPC IP range, while P2P port (30303) and snap sync torrent port (42069) allow external access to keep the node synced with the network. + +**Docker image:** `0xpolygon/erigon:v3.4.0` (Polygon's fork of Erigon, supports ARM/Graviton) + +**Ports:** +| Port | Protocol | Access | Purpose | +|:-----|:---------|:-------|:--------| +| 8545 | TCP | VPC internal | JSON-RPC API | +| 30303 | TCP/UDP | Public | P2P networking | +| 42069 | TCP/UDP | Public | Torrent-based snap sync | + +### Highly Available RPC Nodes setup + +The highly available setup deploys multiple Erigon nodes behind an Application Load Balancer (ALB), managed by an Auto Scaling Group (ASG) across multiple Availability Zones. The ALB performs health checks on port 8545 to ensure only healthy nodes receive traffic. If a node fails, the ASG automatically replaces it, maintaining the desired number of healthy RPC endpoints without manual intervention. + +### Hardware requirements + +| Network | Instance | Storage | IOPS | Throughput | Est. monthly cost (us-east-1) | +|:--------|:---------|:--------|:-----|:-----------|:------------------------------| +| Mainnet | m7g.4xlarge (Graviton3) | 8 TB gp3 | 16,000 | 1,000 MB/s | ~$1,100 | +| Amoy testnet | m7g.xlarge (Graviton3) | 1 TB gp3 | 5,000 | default | ~$250 | + +> **NOTE:** *Mainnet full node storage is comparable to the previous Heimdall+Bor setup. Archive node storage will be larger. Syncing from genesis takes time — snapshot restore is recommended for mainnet.* + +## Well-Architected + +
+Review the pros and cons of this solution. + +### Well-Architected Checklist + +This is the Well-Architected checklist for Polygon PoS nodes implementation of the AWS Blockchain Node Runner app. This checklist takes into account questions from the [AWS Well-Architected Framework](https://aws.amazon.com/architecture/well-architected/) which are relevant to this workload. Please feel free to add more checks from the framework if required for your workload. + +| Pillar | Control | Question/Check | Remarks | +|:------------------------|:----------------------------------|:---------------------------------------------------------------------------------|:-----------------| +| Security | Network protection | Are there unnecessary open ports in security groups? | P2P port 30303 and Erigon snap sync port 42069 are open to the public for network participation (same as the Ethereum Erigon blueprint). RPC port 8545 is restricted to VPC internal traffic only. | +| | | Traffic inspection | AWS WAF could be implemented for traffic inspection. Additional charges will apply. | +| | Compute protection | Reduce attack surface | This solution uses Amazon Linux 2 AMI. You may choose to run hardening scripts on it. | +| | | Enable people to perform actions at a distance | This solution uses AWS Systems Manager (SSM) Session Manager for terminal access, not SSH. | +| | Data protection at rest | Use encrypted Amazon Elastic Block Store (Amazon EBS) volumes | This solution uses encrypted Amazon EBS volumes. | +| | Authorization and access control | Use instance profile with Amazon Elastic Compute Cloud (Amazon EC2) instances | This solution uses AWS Identity and Access Management (AWS IAM) role instead of IAM user. | +| | | Following principle of least privilege access | The EC2 instance role has only the permissions required for CloudWatch metrics and SSM access. | +| | Application security | Security focused development practices | cdk-nag is being used with appropriate suppressions. | +| Cost optimization | Service selection | Use cost effective resources | AWS Graviton3-based Amazon EC2 instances (m7g) are used for best price-performance ratio. | +| | Cost awareness | Estimate costs | A single mainnet node with m7g.4xlarge and 8 TB gp3 storage costs ~$1,100/month in us-east-1. Amoy testnet with m7g.xlarge and 1 TB gp3 costs ~$250/month. | +| Reliability | Resiliency implementation | Withstand component failures | Single-node deployment has no HA. For high availability, use the `polygon-ha-nodes` stack which deploys multiple nodes behind an Application Load Balancer with Auto Scaling Group to automatically replace failed nodes. | +| | Data backup | How is data backed up? | Chain data can be restored from public snapshots. No automated S3 backup is configured in this blueprint. | +| | Resource monitoring | How are workload resources monitored? | Amazon CloudWatch custom metrics (ErigonBlockHeight) are published every 5 minutes via cron. | +| Performance efficiency | Compute selection | How is compute solution selected? | Compute solution is selected based on best price-performance — AWS Graviton3-based m7g instances. | +| | Storage selection | How is storage solution selected? | gp3 Amazon EBS volumes with 16,000 IOPS and 1,000 MB/s throughput for mainnet, per Polygon official recommendations. | +| Operational excellence | Workload health | How is health of workload determined? | CloudWatch metrics (ErigonBlockHeight) are published every 5 minutes. Operators can monitor sync progress via CloudWatch dashboards. | +| Sustainability | Hardware & services | Select most efficient hardware for your workload | This solution uses AWS Graviton3-based Amazon EC2 instances which offer the best performance per watt of energy use in Amazon EC2. | + +
+ +## Solution Walkthrough + +### Open AWS CloudShell + +To begin, ensure you login to your AWS account with permissions to create and modify resources in IAM, EC2, EBS, and VPC. + +From the AWS Management Console, open the [AWS CloudShell](https://docs.aws.amazon.com/cloudshell/latest/userguide/welcome.html), a web-based shell environment. If unfamiliar, review the [2-minute YouTube video](https://youtu.be/fz4rbjRaiQM) for an overview and check out [CloudShell with VPC environment](https://docs.aws.amazon.com/cloudshell/latest/userguide/creating-vpc-environment.html) that we'll use to test the node's RPC API from internal IP address space. + +### Clone this repository and install dependencies + +```bash +git clone https://github.com/aws-samples/aws-blockchain-node-runners.git +cd aws-blockchain-node-runners +npm install +``` + +### Prepare AWS account to deploy nodes + +1. Make sure you are in the root directory of the cloned repository + +2. If you have deleted or don't have the default VPC, create default VPC + +```bash +aws ec2 create-default-vpc +``` + +> **NOTE:** *You may see the following error if the default VPC already exists: `An error occurred (DefaultVpcAlreadyExists) when calling the CreateDefaultVpc operation: A Default VPC already exists for this account in this region.`. That means you can just continue with the following steps.* + +> **NOTE:** *The default VPC must have at least two public subnets in different Availability Zones, and public subnet must set `Auto-assign public IPv4 address` to `YES`.* + +### Configure your setup + +Create your own copy of the `.env` file and edit it: + +```bash +# Make sure you are in aws-blockchain-node-runners/lib/polygon +cd lib/polygon +pwd +``` + +**For Mainnet:** +```bash +cp ./sample-configs/.env-mainnet .env +nano .env +``` + +**For Amoy Testnet:** +```bash +cp ./sample-configs/.env-amoy .env +nano .env +``` + +Edit the following values in your `.env` file: +- `AWS_ACCOUNT_ID` — your target AWS account ID +- `AWS_REGION` — your target AWS region +- `POLYGON_ERIGON_IMAGE` — Docker image for Erigon (default: `0xpolygon/erigon:v3.4.0`) +- `POLYGON_NETWORK` — `mainnet` or `amoy` +- `POLYGON_HEIMDALL_API_URL` — Polygon's official Heimdall API endpoint: + - Mainnet: `https://heimdall-api.polygon.technology` + - Amoy: `https://heimdall-api-amoy.polygon.technology` + +**HA-specific parameters (only needed for Highly Available RPC Nodes):** +- `POLYGON_RPC_NUMBER_OF_NODES` — Number of RPC nodes in the Auto Scaling Group (default: `"2"`) +- `POLYGON_RPC_ALB_HEALTHCHECK_GRACE_PERIOD_MIN` — Grace period in minutes before ALB health checks start (default: `"10"`) +- `POLYGON_RPC_HA_NODES_HEARTBEAT_DELAY_MIN` — Delay in minutes between node heartbeat checks (default: `"60"`) + +### Deploy common components + +Deploy common components such as IAM role and security groups: + +```bash +pwd +# Make sure you are in aws-blockchain-node-runners/lib/polygon +npx cdk deploy polygon-common +``` + +### Deploy Single Node + +1. Deploy `polygon-single-node` stack + +```bash +pwd +# Make sure you are in aws-blockchain-node-runners/lib/polygon +npx cdk deploy polygon-single-node --json --outputs-file single-node-deploy.json +``` + +2. After starting the node you need to wait for the initial synchronization process to finish. You can use Amazon CloudWatch to track the progress. A custom metric (`ErigonBlockHeight`) is published every 5 minutes. To see it: + + - Navigate to [CloudWatch service](https://console.aws.amazon.com/cloudwatch/) (make sure you are in the region you have specified for `AWS_REGION`) + - Open `Dashboards` and select the Polygon node dashboard from the list. + +3. Once the initial synchronization is done, you should be able to access the RPC API of that node from within the same VPC. The RPC port is not exposed to the Internet. Run the following to get the node's internal IP: + +```bash +INSTANCE_ID=$(cat single-node-deploy.json | jq -r '..|.node-instance-id? | select(. != null)') +NODE_INTERNAL_IP=$(aws ec2 describe-instances --instance-ids $INSTANCE_ID --query 'Reservations[*].Instances[*].PrivateIpAddress' --output text) +echo "NODE_INTERNAL_IP=$NODE_INTERNAL_IP" +``` + +Copy output from the last `echo` command with `NODE_INTERNAL_IP=` and open a [CloudShell tab with VPC environment](https://docs.aws.amazon.com/cloudshell/latest/userguide/creating-vpc-environment.html) to access internal IP address space. Paste `NODE_INTERNAL_IP=` into the new CloudShell tab. Then query the API: + +```bash +# IMPORTANT: Run from CloudShell VPC environment tab +curl http://$NODE_INTERNAL_IP:8545 -X POST -H "Content-Type: application/json" \ + --data '{"method":"eth_blockNumber","params":[],"id":1,"jsonrpc":"2.0"}' +``` + +The result should be like this (the actual block number will differ): + +```json +{"jsonrpc":"2.0","id":1,"result":"0x3d0975a"} +``` + +### Deploy Highly Available RPC Nodes + +1. Deploy `polygon-ha-nodes` stack + +```bash +pwd +# Make sure you are in aws-blockchain-node-runners/lib/polygon +npx cdk deploy polygon-ha-nodes --json --outputs-file ha-nodes-deploy.json +``` + +2. Give the new RPC nodes some time to initialize, then run the following to get the ALB URL: + +```bash +export POLYGON_RPC_ALB_URL=$(cat ha-nodes-deploy.json | jq -r '..|.alburl? | select(. != null)') +echo POLYGON_RPC_ALB_URL=$POLYGON_RPC_ALB_URL +``` + +3. Test the RPC API from a [CloudShell VPC environment](https://docs.aws.amazon.com/cloudshell/latest/userguide/creating-vpc-environment.html): + +```bash +# IMPORTANT: Run from CloudShell VPC environment tab +curl http://$POLYGON_RPC_ALB_URL:8545 -X POST -H "Content-Type: application/json" \ + --data '{"jsonrpc":"2.0","method":"eth_blockNumber","params":[],"id":1}' +``` + +The result should be like this (the actual block number will differ): + +```json +{"jsonrpc":"2.0","id":1,"result":"0x3d0975a"} +``` + +> **NOTE:** *By default and for security reasons the ALB is only available from within the VPC in the region where it is deployed. It is not available from the Internet and is not open for external connections. Before opening it up please make sure you protect your RPC APIs.* + +### Clearing up and undeploying everything + +```bash +# Setting the AWS account id and region in case local .env file is lost +export AWS_ACCOUNT_ID= +export AWS_REGION= + +pwd +# Make sure you are in aws-blockchain-node-runners/lib/polygon + +# Destroy HA RPC Nodes +npx cdk destroy polygon-ha-nodes + +# Destroy Single Node +npx cdk destroy polygon-single-node + +# Delete all common components like IAM role and Security Group +npx cdk destroy polygon-common +``` + +### FAQ + +1. How to check the logs of the client running on my node? + +> **NOTE:** *This solution uses SSM Session Manager instead of SSH. That allows you to log all sessions in AWS CloudTrail to see who logged into the server and when. If you receive an error similar to `SessionManagerPlugin is not found`, [install Session Manager plugin for AWS CLI](https://docs.aws.amazon.com/systems-manager/latest/userguide/session-manager-working-with-install-plugin.html).* + +```bash +pwd +# Make sure you are in aws-blockchain-node-runners/lib/polygon + +export INSTANCE_ID=$(cat single-node-deploy.json | jq -r '..|.node-instance-id? | select(. != null)') +echo "INSTANCE_ID=" $INSTANCE_ID +aws ssm start-session --target $INSTANCE_ID --region $AWS_REGION + +# Erigon logs: +docker logs erigon -f +``` + +2. How to check the logs from the EC2 user-data script? + +```bash +pwd +# Make sure you are in aws-blockchain-node-runners/lib/polygon + +export INSTANCE_ID=$(cat single-node-deploy.json | jq -r '..|.node-instance-id? | select(. != null)') +echo "INSTANCE_ID=" $INSTANCE_ID +aws ssm start-session --target $INSTANCE_ID --region $AWS_REGION +sudo cat /var/log/cloud-init-output.log +``` + +3. How to restart the node if it gets stuck during syncing? + +```bash +pwd +# Make sure you are in aws-blockchain-node-runners/lib/polygon + +export INSTANCE_ID=$(cat single-node-deploy.json | jq -r '..|.node-instance-id? | select(. != null)') +echo "INSTANCE_ID=" $INSTANCE_ID +aws ssm start-session --target $INSTANCE_ID --region $AWS_REGION +docker-compose down && docker-compose up -d +``` diff --git a/lib/polygon/app.ts b/lib/polygon/app.ts new file mode 100644 index 00000000..f3bb982b --- /dev/null +++ b/lib/polygon/app.ts @@ -0,0 +1,51 @@ +#!/usr/bin/env node +import 'dotenv/config' +import "source-map-support/register"; +import * as cdk from "aws-cdk-lib"; +import * as nag from "cdk-nag"; +import * as config from "./lib/config/node-config"; + +import { PolygonSingleNodeStack } from "./lib/single-node-stack"; +import { PolygonHaNodesStack } from "./lib/ha-nodes-stack"; +import { PolygonCommonStack } from "./lib/common-stack"; + +const app = new cdk.App(); +cdk.Tags.of(app).add("Project", "AWSPolygon"); + +new PolygonCommonStack(app, "polygon-common", { + stackName: `polygon-nodes-common`, + env: { account: config.baseConfig.accountId, region: config.baseConfig.region }, +}); + +new PolygonSingleNodeStack(app, "polygon-single-node", { + stackName: `polygon-single-node-${config.baseConfig.network}`, + env: { account: config.baseConfig.accountId, region: config.baseConfig.region }, + network: config.baseConfig.network, + erigonImage: config.baseConfig.erigonImage, + heimdallApiUrl: config.baseConfig.heimdallApiUrl, + instanceType: config.singleNodeConfig.instanceType, + instanceCpuType: config.singleNodeConfig.instanceCpuType, + dataVolume: config.singleNodeConfig.dataVolumes[0], +}); + +new PolygonHaNodesStack(app, "polygon-ha-nodes", { + stackName: `polygon-ha-nodes-${config.baseConfig.network}`, + env: { account: config.baseConfig.accountId, region: config.baseConfig.region }, + network: config.baseConfig.network, + erigonImage: config.baseConfig.erigonImage, + heimdallApiUrl: config.baseConfig.heimdallApiUrl, + instanceType: config.haNodeConfig.instanceType, + instanceCpuType: config.haNodeConfig.instanceCpuType, + numberOfNodes: config.haNodeConfig.numberOfNodes, + albHealthCheckGracePeriodMin: config.haNodeConfig.albHealthCheckGracePeriodMin, + heartBeatDelayMin: config.haNodeConfig.heartBeatDelayMin, + dataVolume: config.haNodeConfig.dataVolumes[0], +}); + +cdk.Aspects.of(app).add( + new nag.AwsSolutionsChecks({ + verbose: false, + reports: true, + logIgnores: false, + }) +); diff --git a/lib/polygon/cdk.json b/lib/polygon/cdk.json new file mode 100644 index 00000000..83f6c768 --- /dev/null +++ b/lib/polygon/cdk.json @@ -0,0 +1,32 @@ +{ + "app": "npx ts-node --prefer-ts-exts app.ts", + "watch": { + "include": ["**"], + "exclude": ["README.md", "cdk*.json", "**/*.d.ts", "**/*.js", "tsconfig.json", "package*.json", "yarn.lock", "node_modules", "test"] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImdsv2TemplateName": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/core:validateSnapshotRemovalPolicy": true, + "@aws-cdk/aws-codepipeline:crossAccountKeyAliasStackSafeResourceName": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-sns-subscriptions:restrictSqsDescryption": true, + "@aws-cdk/aws-apigateway:disableCloudWatchRole": true, + "@aws-cdk/core:enablePartitionLiterals": true, + "@aws-cdk/aws-events:eventsTargetQueueSameAccount": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-ecs:disableExplicitDeploymentControllerForCircuitBreaker": true, + "@aws-cdk/aws-iam:importedRoleStackSafeDefaultPolicyName": true, + "@aws-cdk/aws-s3:serverAccessLogsUseBucketPolicy": true, + "@aws-cdk/aws-route53-patters:useCertificate": true, + "@aws-cdk/customresources:installLatestAwsSdkDefault": false + } +} diff --git a/lib/polygon/doc/assets/.gitkeep b/lib/polygon/doc/assets/.gitkeep new file mode 100644 index 00000000..e69de29b diff --git a/lib/polygon/jest.config.js b/lib/polygon/jest.config.js new file mode 100644 index 00000000..0bc77ac0 --- /dev/null +++ b/lib/polygon/jest.config.js @@ -0,0 +1,8 @@ +module.exports = { + testEnvironment: 'node', + roots: ['/test'], + testMatch: ['**/*.test.ts'], + transform: { + '^.+\\.tsx?$': 'ts-jest' + }, +}; diff --git a/lib/polygon/lib/assets/user-data/node.sh b/lib/polygon/lib/assets/user-data/node.sh new file mode 100644 index 00000000..e9a9e424 --- /dev/null +++ b/lib/polygon/lib/assets/user-data/node.sh @@ -0,0 +1,147 @@ +#!/bin/bash +set -euo pipefail + +# Variables injected by CDK via Fn::Sub +REGION="${_REGION_}" +ASSETS_S3_PATH="${_ASSETS_S3_PATH_}" +POLYGON_NETWORK="${_POLYGON_NETWORK_}" +POLYGON_ERIGON_IMAGE="${_POLYGON_ERIGON_IMAGE_}" +POLYGON_HEIMDALL_API_URL="${_POLYGON_HEIMDALL_API_URL_}" +STACK_NAME="${_STACK_NAME_}" +DATA_VOLUME_TYPE="${_DATA_VOLUME_TYPE_}" +DATA_VOLUME_SIZE="${_DATA_VOLUME_SIZE_}" + +# Map network name to Erigon chain name +case "$POLYGON_NETWORK" in + mainnet) POLYGON_CHAIN_NAME="bor-mainnet" ;; + amoy) POLYGON_CHAIN_NAME="amoy" ;; + *) POLYGON_CHAIN_NAME="bor-mainnet" ;; +esac + +echo "========== Polygon Node Setup Starting ==========" +echo "Network: $POLYGON_NETWORK (chain: $POLYGON_CHAIN_NAME)" +echo "Erigon Image: $POLYGON_ERIGON_IMAGE" + +# Install dependencies +yum update -y +yum install -y jq aws-cfn-bootstrap amazon-cloudwatch-agent cronie + +# Install Docker from official repo (includes docker-compose-plugin) +yum install -y yum-utils +yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo +yum install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin + +# Start Docker +systemctl enable docker +systemctl start docker + +# Format and mount data volume +# Note: CloudFormation VolumeAttachment may take several minutes after instance launch. +# We poll for up to 10 minutes for the device to appear. +DATA_DIR="/data" +mkdir -p "$DATA_DIR" +if [ "$DATA_VOLUME_TYPE" != "instance-store" ]; then + echo "Waiting for EBS data volume to be attached..." + WAIT_SECONDS=0 + MAX_WAIT=600 + DEVICE="" + while [ $WAIT_SECONDS -lt $MAX_WAIT ]; do + # Look for unformatted nvme device larger than 100GB (skip root volume) + DEVICE=$(lsblk -lnb | awk '{if ($7 == "" && $4 > 100000000000) {print "/dev/"$1}}' | grep nvme | sort | head -1) + if [ -n "$DEVICE" ]; then + echo "Found data volume: $DEVICE after $WAIT_SECONDS seconds" + break + fi + # Also check traditional device names + for dev in /dev/sdf /dev/xvdf; do + if [ -e "$dev" ]; then DEVICE="$dev"; break 2; fi + done + sleep 10 + WAIT_SECONDS=$((WAIT_SECONDS + 10)) + echo "Waiting for data volume... ($WAIT_SECONDS seconds/$MAX_WAIT seconds)" + done + + if [ -n "$DEVICE" ]; then + echo "Using device: $DEVICE" + if ! blkid "$DEVICE" 2>/dev/null; then + mkfs.xfs "$DEVICE" + fi + mount "$DEVICE" "$DATA_DIR" + VOLUME_UUID=$(blkid -s UUID -o value "$DEVICE") + echo "UUID=$VOLUME_UUID $DATA_DIR xfs defaults,nofail 0 2" >> /etc/fstab + else + echo "WARNING: No data volume found after $MAX_WAIT seconds. Using root volume for data." + fi +fi + +# Create data directory with restricted permissions +# Erigon runs as UID 1000 inside the container +mkdir -p "$DATA_DIR/erigon" +chown -R 1000:1000 "$DATA_DIR/erigon" +chmod -R 750 "$DATA_DIR/erigon" + +# Create docker-compose file +cat > /home/ec2-user/docker-compose.yml << COMPOSEOF +services: + erigon: + image: $POLYGON_ERIGON_IMAGE + container_name: erigon + restart: always + command: + - --chain=$POLYGON_CHAIN_NAME + - --bor.heimdall=$POLYGON_HEIMDALL_API_URL + - --datadir=/var/lib/erigon/data + - --http + - --http.api=eth,debug,net,trace,web3,erigon,txpool,bor + - --http.addr=0.0.0.0 + - --http.vhosts=localhost,127.0.0.1 + - --torrent.download.rate=512mb + - --metrics + - --metrics.addr=0.0.0.0 + - --maxpeers=100 + ports: + - "8545:8545" + - "30303:30303/tcp" + - "30303:30303/udp" + - "42069:42069/tcp" + - "42069:42069/udp" + volumes: + - $DATA_DIR/erigon:/var/lib/erigon/data +COMPOSEOF + +# Start services +cd /home/ec2-user +docker compose up -d + +# Setup sync checker cron (publish metrics to CloudWatch every 5 min) +# Uses IMDSv2 token-based authentication +cat > /home/ec2-user/sync-checker.sh << 'CHECKEREOF' +#!/bin/bash +TOKEN=$(curl -s -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600") +INSTANCE_ID=$(curl -s -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/instance-id) + +# Check if Erigon is syncing +SYNCING=$(curl -s -X POST -H "Content-Type: application/json" \ + --data '{"method":"eth_syncing","params":[],"id":1,"jsonrpc":"2.0"}' \ + http://localhost:8545 2>/dev/null | jq -r '.result') + +BLOCK_HEX=$(curl -s -X POST -H "Content-Type: application/json" \ + --data '{"method":"eth_blockNumber","params":[],"id":1,"jsonrpc":"2.0"}' \ + http://localhost:8545 2>/dev/null | jq -r '.result // "0x0"') +BLOCK=$((BLOCK_HEX)) + +IS_SYNCING=1 +if [ "$SYNCING" = "false" ]; then IS_SYNCING=0; fi + +aws cloudwatch put-metric-data --namespace "Polygon/Node" --dimensions InstanceId="$INSTANCE_ID" \ + --metric-data "[{\"MetricName\":\"ErigonBlockHeight\",\"Value\":$BLOCK,\"Unit\":\"Count\"},{\"MetricName\":\"ErigonSyncing\",\"Value\":$IS_SYNCING,\"Unit\":\"Count\"}]" 2>/dev/null || true +CHECKEREOF + +chmod +x /home/ec2-user/sync-checker.sh +echo "*/5 * * * * /home/ec2-user/sync-checker.sh" | crontab - + +# Note: cfn-signal is not used because CreationPolicy is disabled +# (avoids circular dependency with VolumeAttachment). +# Node health is monitored via CloudWatch metrics instead. + +echo "========== Polygon Node Setup Complete ==========" diff --git a/lib/polygon/lib/common-stack.ts b/lib/polygon/lib/common-stack.ts new file mode 100644 index 00000000..25329e61 --- /dev/null +++ b/lib/polygon/lib/common-stack.ts @@ -0,0 +1,52 @@ +import * as cdk from "aws-cdk-lib"; +import * as cdkConstructs from "constructs"; +import * as iam from "aws-cdk-lib/aws-iam"; +import * as s3Assets from "aws-cdk-lib/aws-s3-assets"; +import * as path from "path"; +import * as nag from "cdk-nag"; + +export class PolygonCommonStack extends cdk.Stack { + constructor(scope: cdkConstructs.Construct, id: string, props?: cdk.StackProps) { + super(scope, id, props); + + const region = cdk.Stack.of(this).region; + const asset = new s3Assets.Asset(this, "assets", { + path: path.join(__dirname, "assets"), + }); + + const instanceRole = new iam.Role(this, `node-role`, { + assumedBy: new iam.ServicePrincipal("ec2.amazonaws.com"), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName("AmazonSSMManagedInstanceCore"), + iam.ManagedPolicy.fromAwsManagedPolicyName("CloudWatchAgentServerPolicy"), + ], + }); + + instanceRole.addToPolicy( + new iam.PolicyStatement({ + resources: ["*"], + actions: ["cloudformation:SignalResource"], + }) + ); + + new cdk.CfnOutput(this, "Instance Role ARN", { + value: instanceRole.roleArn, + exportName: "PolygonNodeInstanceRoleArn", + }); + + nag.NagSuppressions.addResourceSuppressions( + this, + [ + { + id: "AwsSolutions-IAM4", + reason: "AmazonSSMManagedInstanceCore and CloudWatchAgentServerPolicy are restrictive enough", + }, + { + id: "AwsSolutions-IAM5", + reason: "Can't target specific stack: https://github.com/aws/aws-cdk/issues/22657", + }, + ], + true + ); + } +} diff --git a/lib/polygon/lib/config/node-config.interface.ts b/lib/polygon/lib/config/node-config.interface.ts new file mode 100644 index 00000000..b082255e --- /dev/null +++ b/lib/polygon/lib/config/node-config.interface.ts @@ -0,0 +1,15 @@ +import * as configTypes from "../../../constructs/config.interface"; + +export type PolygonNetwork = "mainnet" | "amoy"; + +export interface PolygonDataVolumeConfig extends configTypes.DataVolumeConfig {} + +export interface PolygonBaseConfig extends configTypes.BaseConfig { + network: PolygonNetwork; + erigonImage: string; + heimdallApiUrl: string; +} + +export interface PolygonSingleNodeConfig extends configTypes.SingleNodeConfig {} + +export interface PolygonHaNodeConfig extends configTypes.HaNodesConfig {} diff --git a/lib/polygon/lib/config/node-config.ts b/lib/polygon/lib/config/node-config.ts new file mode 100644 index 00000000..d8f23a92 --- /dev/null +++ b/lib/polygon/lib/config/node-config.ts @@ -0,0 +1,61 @@ +import * as ec2 from "aws-cdk-lib/aws-ec2"; +import * as configTypes from "./node-config.interface"; +import * as constants from "../../../constructs/constants"; + +const parseDataVolumeType = (dataVolumeType: string) => { + switch (dataVolumeType) { + case "gp3": + return ec2.EbsDeviceVolumeType.GP3; + case "io2": + return ec2.EbsDeviceVolumeType.IO2; + case "io1": + return ec2.EbsDeviceVolumeType.IO1; + case "instance-store": + return constants.InstanceStoreageDeviceVolumeType; + default: + return ec2.EbsDeviceVolumeType.GP3; + } +} + +const network = process.env.POLYGON_NETWORK || "mainnet"; + +const defaultHeimdallApiUrl = network === "amoy" + ? "https://heimdall-api-amoy.polygon.technology" + : "https://heimdall-api.polygon.technology"; + +export const baseConfig: configTypes.PolygonBaseConfig = { + accountId: process.env.AWS_ACCOUNT_ID || "xxxxxxxxxxx", + region: process.env.AWS_REGION || "us-east-1", + network, + erigonImage: process.env.POLYGON_ERIGON_IMAGE || "0xpolygon/erigon:v3.4.0", + heimdallApiUrl: process.env.POLYGON_HEIMDALL_API_URL || defaultHeimdallApiUrl, +}; + +export const singleNodeConfig: configTypes.PolygonSingleNodeConfig = { + instanceType: new ec2.InstanceType(process.env.POLYGON_INSTANCE_TYPE || "m7g.4xlarge"), + instanceCpuType: process.env.POLYGON_CPU_TYPE?.toLowerCase() == "x86_64" ? ec2.AmazonLinuxCpuType.X86_64 : ec2.AmazonLinuxCpuType.ARM_64, + dataVolumes: [ + { + sizeGiB: process.env.POLYGON_DATA_VOL_SIZE ? parseInt(process.env.POLYGON_DATA_VOL_SIZE) : 8000, + type: parseDataVolumeType(process.env.POLYGON_DATA_VOL_TYPE?.toLowerCase() || "gp3"), + iops: process.env.POLYGON_DATA_VOL_IOPS ? parseInt(process.env.POLYGON_DATA_VOL_IOPS) : 16000, + throughput: process.env.POLYGON_DATA_VOL_THROUGHPUT ? parseInt(process.env.POLYGON_DATA_VOL_THROUGHPUT) : 1000, + } + ], +}; + +export const haNodeConfig: configTypes.PolygonHaNodeConfig = { + instanceType: new ec2.InstanceType(process.env.POLYGON_RPC_INSTANCE_TYPE || "m7g.4xlarge"), + instanceCpuType: process.env.POLYGON_RPC_CPU_TYPE?.toLowerCase() == "x86_64" ? ec2.AmazonLinuxCpuType.X86_64 : ec2.AmazonLinuxCpuType.ARM_64, + numberOfNodes: process.env.POLYGON_RPC_NUMBER_OF_NODES ? parseInt(process.env.POLYGON_RPC_NUMBER_OF_NODES) : 2, + albHealthCheckGracePeriodMin: process.env.POLYGON_RPC_ALB_HEALTHCHECK_GRACE_PERIOD_MIN ? parseInt(process.env.POLYGON_RPC_ALB_HEALTHCHECK_GRACE_PERIOD_MIN) : 10, + heartBeatDelayMin: process.env.POLYGON_RPC_HA_NODES_HEARTBEAT_DELAY_MIN ? parseInt(process.env.POLYGON_RPC_HA_NODES_HEARTBEAT_DELAY_MIN) : 60, + dataVolumes: [ + { + sizeGiB: process.env.POLYGON_RPC_DATA_VOL_SIZE ? parseInt(process.env.POLYGON_RPC_DATA_VOL_SIZE) : 8000, + type: parseDataVolumeType(process.env.POLYGON_RPC_DATA_VOL_TYPE?.toLowerCase() || "gp3"), + iops: process.env.POLYGON_RPC_DATA_VOL_IOPS ? parseInt(process.env.POLYGON_RPC_DATA_VOL_IOPS) : 16000, + throughput: process.env.POLYGON_RPC_DATA_VOL_THROUGHPUT ? parseInt(process.env.POLYGON_RPC_DATA_VOL_THROUGHPUT) : 1000, + } + ], +}; diff --git a/lib/polygon/lib/constructs/polygon-cw-dashboard.ts b/lib/polygon/lib/constructs/polygon-cw-dashboard.ts new file mode 100644 index 00000000..d819d625 --- /dev/null +++ b/lib/polygon/lib/constructs/polygon-cw-dashboard.ts @@ -0,0 +1,119 @@ +export const SingleNodeCWDashboardJSON = { + "widgets": [ + { + "height": 5, + "width": 8, + "y": 0, + "x": 0, + "type": "metric", + "properties": { + "view": "timeSeries", + "stat": "Average", + "period": 300, + "stacked": false, + "yAxis": { "left": { "min": 0 } }, + "region": "${REGION}", + "metrics": [ + [ "AWS/EC2", "CPUUtilization", "InstanceId", "${INSTANCE_ID}", { "label": "${INSTANCE_ID}-${INSTANCE_NAME}" } ] + ], + "title": "CPU Utilization (%)" + } + }, + { + "height": 5, + "width": 8, + "y": 0, + "x": 8, + "type": "metric", + "properties": { + "view": "timeSeries", + "stat": "Average", + "period": 300, + "stacked": false, + "region": "${REGION}", + "metrics": [ + [ "CWAgent", "mem_used_percent", "InstanceId", "${INSTANCE_ID}", { "label": "${INSTANCE_ID}-${INSTANCE_NAME}" } ] + ], + "title": "Memory Utilization (%)" + } + }, + { + "height": 5, + "width": 8, + "y": 0, + "x": 16, + "type": "metric", + "properties": { + "view": "timeSeries", + "stat": "Average", + "period": 300, + "stacked": false, + "yAxis": { "left": { "min": 0 } }, + "region": "${REGION}", + "metrics": [ + [ "AWS/EC2", "NetworkIn", "InstanceId", "${INSTANCE_ID}", { "label": "Network In" } ], + [ "AWS/EC2", "NetworkOut", "InstanceId", "${INSTANCE_ID}", { "label": "Network Out" } ] + ], + "title": "Network In/Out (bytes)" + } + }, + { + "height": 5, + "width": 8, + "y": 5, + "x": 0, + "type": "metric", + "properties": { + "view": "timeSeries", + "stat": "Average", + "period": 300, + "stacked": false, + "yAxis": { "left": { "min": 0 } }, + "region": "${REGION}", + "metrics": [ + [ "AWS/EC2", "EBSReadOps", "InstanceId", "${INSTANCE_ID}", { "label": "Read Ops" } ], + [ "AWS/EC2", "EBSWriteOps", "InstanceId", "${INSTANCE_ID}", { "label": "Write Ops" } ] + ], + "title": "Disk Read/Write (ops)" + } + }, + { + "height": 5, + "width": 8, + "y": 5, + "x": 8, + "type": "metric", + "properties": { + "sparkline": true, + "view": "timeSeries", + "stat": "Maximum", + "period": 60, + "stacked": false, + "region": "${REGION}", + "metrics": [ + [ "Polygon/Node", "ErigonBlockHeight", "InstanceId", "${INSTANCE_ID}", { "label": "${INSTANCE_ID}-${INSTANCE_NAME}" } ] + ], + "title": "Erigon Block Height" + } + }, + { + "height": 5, + "width": 8, + "y": 5, + "x": 16, + "type": "metric", + "properties": { + "sparkline": true, + "view": "timeSeries", + "stat": "Maximum", + "period": 60, + "stacked": false, + "region": "${REGION}", + "metrics": [ + [ "Polygon/Node", "ErigonSyncing", "InstanceId", "${INSTANCE_ID}", { "label": "${INSTANCE_ID}-${INSTANCE_NAME}" } ] + ], + "title": "Erigon Syncing" + } + } + ] +} diff --git a/lib/polygon/lib/constructs/polygon-node-security-group.ts b/lib/polygon/lib/constructs/polygon-node-security-group.ts new file mode 100644 index 00000000..98657815 --- /dev/null +++ b/lib/polygon/lib/constructs/polygon-node-security-group.ts @@ -0,0 +1,46 @@ +import * as cdk from "aws-cdk-lib"; +import * as cdkContructs from 'constructs'; +import * as ec2 from "aws-cdk-lib/aws-ec2"; +import * as nag from "cdk-nag"; + +export interface PolygonNodeSecurityGroupConstructProps { + vpc: cdk.aws_ec2.IVpc; +} + +export class PolygonNodeSecurityGroupConstruct extends cdkContructs.Construct { + public securityGroup: cdk.aws_ec2.ISecurityGroup; + + constructor(scope: cdkContructs.Construct, id: string, props: PolygonNodeSecurityGroupConstructProps) { + super(scope, id); + + const { vpc } = props; + + const sg = new ec2.SecurityGroup(this, `node-security-group`, { + vpc, + description: "Security Group for Polygon PoS nodes (Erigon)", + allowAllOutbound: true, + }); + + // Public P2P ports + sg.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(30303), "Erigon P2P"); + sg.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.udp(30303), "Erigon P2P"); + sg.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(42069), "Erigon torrent sync"); + sg.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.udp(42069), "Erigon torrent sync"); + + // Private RPC restricted to VPC + sg.addIngressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(8545), "Erigon HTTP RPC"); + + this.securityGroup = sg; + + nag.NagSuppressions.addResourceSuppressions( + this, + [ + { + id: "AwsSolutions-EC23", + reason: "Polygon requires wildcard inbound for P2P ports to sync with the network", + }, + ], + true + ); + } +} diff --git a/lib/polygon/lib/ha-nodes-stack.ts b/lib/polygon/lib/ha-nodes-stack.ts new file mode 100644 index 00000000..f3ba6442 --- /dev/null +++ b/lib/polygon/lib/ha-nodes-stack.ts @@ -0,0 +1,128 @@ +import * as cdk from "aws-cdk-lib"; +import * as cdkConstructs from "constructs"; +import * as ec2 from "aws-cdk-lib/aws-ec2"; +import * as iam from "aws-cdk-lib/aws-iam"; +import * as s3Assets from "aws-cdk-lib/aws-s3-assets"; +import * as nag from "cdk-nag"; +import * as path from "path"; +import * as fs from "fs"; +import * as configTypes from "./config/node-config.interface"; +import { PolygonNodeSecurityGroupConstruct } from "./constructs/polygon-node-security-group"; +import { HANodesConstruct } from "../../constructs/ha-rpc-nodes-with-alb"; + +export interface PolygonHaNodesStackProps extends cdk.StackProps { + network: configTypes.PolygonNetwork; + erigonImage: string; + heimdallApiUrl: string; + instanceType: ec2.InstanceType; + instanceCpuType: ec2.AmazonLinuxCpuType; + dataVolume: configTypes.PolygonDataVolumeConfig; + numberOfNodes: number; + albHealthCheckGracePeriodMin: number; + heartBeatDelayMin: number; +} + +export class PolygonHaNodesStack extends cdk.Stack { + constructor(scope: cdkConstructs.Construct, id: string, props: PolygonHaNodesStackProps) { + super(scope, id, props); + + const REGION = cdk.Stack.of(this).region; + const STACK_NAME = cdk.Stack.of(this).stackName; + const lifecycleHookName = STACK_NAME; + const autoScalingGroupName = STACK_NAME; + + const { + instanceType, + network, + erigonImage, + heimdallApiUrl, + instanceCpuType, + dataVolume, + albHealthCheckGracePeriodMin, + heartBeatDelayMin, + numberOfNodes, + } = props; + + // Using default VPC + const vpc = ec2.Vpc.fromLookup(this, "vpc", { isDefault: true }); + + // Setting up the security group for the node + const instanceSG = new PolygonNodeSecurityGroupConstruct(this, "security-group", { + vpc: vpc, + }); + + // Making our scripts and configs from the local "assets" directory available for instance to download + const asset = new s3Assets.Asset(this, "assets", { + path: path.join(__dirname, "assets"), + }); + + // Getting the IAM role ARN from the common stack + const importedInstanceRoleArn = cdk.Fn.importValue("PolygonNodeInstanceRoleArn"); + const instanceRole = iam.Role.fromRoleArn(this, "iam-role", importedInstanceRoleArn); + + // Making sure our instance will be able to read the assets + asset.bucket.grantRead(instanceRole); + + // Parsing user data script and injecting necessary variables + const userData = fs.readFileSync(path.join(__dirname, "assets", "user-data", "node.sh")).toString(); + + const modifiedUserData = cdk.Fn.sub(userData, { + _REGION_: REGION, + _ASSETS_S3_PATH_: `s3://${asset.s3BucketName}/${asset.s3ObjectKey}`, + _POLYGON_NETWORK_: network, + _POLYGON_ERIGON_IMAGE_: erigonImage, + _POLYGON_HEIMDALL_API_URL_: heimdallApiUrl, + _STACK_NAME_: STACK_NAME, + _DATA_VOLUME_TYPE_: dataVolume.type, + _DATA_VOLUME_SIZE_: dataVolume.sizeGiB.toString(), + _LIFECYCLE_HOOK_NAME_: lifecycleHookName, + _AUTOSCALING_GROUP_NAME_: autoScalingGroupName, + }); + + // Setting up the nodes using generic High Availability (HA) Node construct + const rpcNodes = new HANodesConstruct(this, "rpc-nodes", { + instanceType, + dataVolumes: [dataVolume], + machineImage: new ec2.AmazonLinuxImage({ + generation: ec2.AmazonLinuxGeneration.AMAZON_LINUX_2023, + kernel: ec2.AmazonLinuxKernel.KERNEL6_1, + cpuType: instanceCpuType, + }), + role: instanceRole, + vpc, + securityGroup: instanceSG.securityGroup, + userData: modifiedUserData, + numberOfNodes, + rpcPortForALB: 8545, + albHealthCheckGracePeriodMin, + heartBeatDelayMin, + lifecycleHookName: lifecycleHookName, + autoScalingGroupName: autoScalingGroupName, + }); + + // Output the URL of the Application Load Balancer + new cdk.CfnOutput(this, "alb-url", { + value: rpcNodes.loadBalancerDnsName, + }); + + // Adding suppressions to the stack + nag.NagSuppressions.addResourceSuppressions( + this, + [ + { + id: "AwsSolutions-AS3", + reason: "No notifications needed", + }, + { + id: "AwsSolutions-S1", + reason: "No access log needed for ALB logs bucket", + }, + { + id: "AwsSolutions-EC28", + reason: "Using basic monitoring to save costs", + }, + ], + true + ); + } +} diff --git a/lib/polygon/lib/single-node-stack.ts b/lib/polygon/lib/single-node-stack.ts new file mode 100644 index 00000000..43451927 --- /dev/null +++ b/lib/polygon/lib/single-node-stack.ts @@ -0,0 +1,128 @@ +import * as cdk from "aws-cdk-lib"; +import * as cdkConstructs from "constructs"; +import * as ec2 from "aws-cdk-lib/aws-ec2"; +import * as iam from "aws-cdk-lib/aws-iam"; +import * as s3Assets from "aws-cdk-lib/aws-s3-assets"; +import * as path from "path"; +import * as fs from "fs"; +import * as cw from 'aws-cdk-lib/aws-cloudwatch'; +import { SingleNodeConstruct } from "../../constructs/single-node"; +import * as configTypes from "./config/node-config.interface"; +import * as constants from "../../constructs/constants"; +import { PolygonNodeSecurityGroupConstruct } from "./constructs/polygon-node-security-group"; +import * as polygonCwDashboard from "./constructs/polygon-cw-dashboard"; +import * as nag from "cdk-nag"; + +export interface PolygonSingleNodeStackProps extends cdk.StackProps { + network: configTypes.PolygonNetwork; + erigonImage: string; + heimdallApiUrl: string; + instanceType: ec2.InstanceType; + instanceCpuType: ec2.AmazonLinuxCpuType; + dataVolume: configTypes.PolygonDataVolumeConfig; +} + +export class PolygonSingleNodeStack extends cdk.Stack { + constructor(scope: cdkConstructs.Construct, id: string, props: PolygonSingleNodeStackProps) { + super(scope, id, props); + + const REGION = cdk.Stack.of(this).region; + const STACK_NAME = cdk.Stack.of(this).stackName; + const availabilityZones = cdk.Stack.of(this).availabilityZones; + const azIndex = parseInt(process.env.POLYGON_AZ_INDEX || "0"); + const chosenAvailabilityZone = availabilityZones[Math.min(azIndex, availabilityZones.length - 1)]; + + const { + instanceType, + network, + erigonImage, + heimdallApiUrl, + instanceCpuType, + dataVolume, + } = props; + + const vpc = ec2.Vpc.fromLookup(this, "vpc", { isDefault: true }); + + const instanceSG = new PolygonNodeSecurityGroupConstruct(this, "security-group", { + vpc: vpc, + }); + + const asset = new s3Assets.Asset(this, "assets", { + path: path.join(__dirname, "assets"), + }); + + const importedInstanceRoleArn = cdk.Fn.importValue("PolygonNodeInstanceRoleArn"); + const instanceRole = iam.Role.fromRoleArn(this, "iam-role", importedInstanceRoleArn); + + asset.bucket.grantRead(instanceRole); + + const node = new SingleNodeConstruct(this, "single-node", { + instanceName: STACK_NAME, + instanceType, + dataVolumes: [dataVolume], + machineImage: new ec2.AmazonLinuxImage({ + generation: ec2.AmazonLinuxGeneration.AMAZON_LINUX_2023, + kernel: ec2.AmazonLinuxKernel.KERNEL6_1, + cpuType: instanceCpuType, + }), + vpc, + availabilityZone: chosenAvailabilityZone, + role: instanceRole, + securityGroup: instanceSG.securityGroup, + vpcSubnets: { + subnetType: ec2.SubnetType.PUBLIC, + }, + }); + + // Override CreationPolicy: remove cfn-signal requirement. + // SingleNodeConstruct sets a 15-min CreationPolicy that waits for cfn-signal, + // but this creates a circular dependency with VolumeAttachment — CloudFormation + // won't attach the volume until the Instance is CREATE_COMPLETE, but the Instance + // won't signal until user-data finishes (which needs the volume). + // Instead, we let the Instance be CREATE_COMPLETE immediately and rely on + // CloudWatch metrics to monitor node health. + const cfnInstance = node.instance.node.defaultChild as ec2.CfnInstance; + cfnInstance.cfnOptions.creationPolicy = undefined; + + const userData = fs.readFileSync(path.join(__dirname, "assets", "user-data", "node.sh")).toString(); + + const modifiedUserData = cdk.Fn.sub(userData, { + _REGION_: REGION, + _ASSETS_S3_PATH_: `s3://${asset.s3BucketName}/${asset.s3ObjectKey}`, + _POLYGON_NETWORK_: network, + _POLYGON_ERIGON_IMAGE_: erigonImage, + _POLYGON_HEIMDALL_API_URL_: heimdallApiUrl, + _STACK_NAME_: STACK_NAME, + _DATA_VOLUME_TYPE_: dataVolume.type, + _DATA_VOLUME_SIZE_: dataVolume.sizeGiB.toString(), + }); + + node.instance.addUserData(modifiedUserData); + + const dashboardString = cdk.Fn.sub(JSON.stringify(polygonCwDashboard.SingleNodeCWDashboardJSON), { + INSTANCE_ID: node.instanceId, + INSTANCE_NAME: STACK_NAME, + REGION: REGION, + }); + + new cw.CfnDashboard(this, 'single-cw-dashboard', { + dashboardName: `${STACK_NAME}-${node.instanceId}`, + dashboardBody: dashboardString, + }); + + new cdk.CfnOutput(this, "node-instance-id", { + value: node.instanceId, + }); + + nag.NagSuppressions.addResourceSuppressions( + this, + [ + { + id: "AwsSolutions-IAM5", + reason: "Need read access to the S3 assets bucket", + }, + ], + true + ); + } +} diff --git a/lib/polygon/package.json b/lib/polygon/package.json new file mode 100644 index 00000000..b78b53a5 --- /dev/null +++ b/lib/polygon/package.json @@ -0,0 +1,11 @@ +{ + "name": "aws-blockchain-node-runners-polygon", + "version": "0.1.0", + "scripts": { + "build": "npx tsc", + "watch": "npx tsc -w", + "test": "npx jest", + "cdk": "npx cdk", + "scan-cdk": "npx cdk synth" + } +} diff --git a/lib/polygon/sample-configs/.env-amoy b/lib/polygon/sample-configs/.env-amoy new file mode 100644 index 00000000..13bd49bb --- /dev/null +++ b/lib/polygon/sample-configs/.env-amoy @@ -0,0 +1,35 @@ +################################################################ +# Example configuration for Polygon PoS node runner on AWS # +# Network: Amoy Testnet (Erigon) # +################################################################ + +AWS_ACCOUNT_ID="xxxxxxxxxxx" +AWS_REGION="us-east-1" + +# Polygon network +POLYGON_NETWORK="amoy" + +# Erigon (single container replaces Heimdall+Bor) +POLYGON_ERIGON_IMAGE="0xpolygon/erigon:v3.4.0" +POLYGON_HEIMDALL_API_URL="https://heimdall-api-amoy.polygon.technology" + +# Instance configuration (smaller for testnet) +POLYGON_INSTANCE_TYPE="m7g.xlarge" # Graviton3: 4 vCPU, 16 GB RAM +POLYGON_CPU_TYPE="ARM_64" + +# Storage configuration (testnet is much smaller) +POLYGON_DATA_VOL_SIZE="1000" # 1 TB +POLYGON_DATA_VOL_TYPE="gp3" +POLYGON_DATA_VOL_IOPS="5000" +POLYGON_DATA_VOL_THROUGHPUT="250" + +# HA RPC nodes configuration +POLYGON_RPC_INSTANCE_TYPE="m7g.xlarge" +POLYGON_RPC_CPU_TYPE="ARM_64" +POLYGON_RPC_DATA_VOL_SIZE="1000" +POLYGON_RPC_DATA_VOL_TYPE="gp3" +POLYGON_RPC_DATA_VOL_IOPS="5000" +POLYGON_RPC_DATA_VOL_THROUGHPUT="250" +POLYGON_RPC_NUMBER_OF_NODES="2" +POLYGON_RPC_ALB_HEALTHCHECK_GRACE_PERIOD_MIN="10" +POLYGON_RPC_HA_NODES_HEARTBEAT_DELAY_MIN="60" diff --git a/lib/polygon/sample-configs/.env-mainnet b/lib/polygon/sample-configs/.env-mainnet new file mode 100644 index 00000000..9e3125d4 --- /dev/null +++ b/lib/polygon/sample-configs/.env-mainnet @@ -0,0 +1,36 @@ +################################################################ +# Example configuration for Polygon PoS node runner on AWS # +# Network: Mainnet (Erigon) # +# Ref: https://docs.erigon.tech/get-started/easy-nodes/how-to-run-a-polygon-node # +################################################################ + +AWS_ACCOUNT_ID="xxxxxxxxxxx" +AWS_REGION="us-east-1" + +# Polygon network +POLYGON_NETWORK="mainnet" # Options: "mainnet" | "amoy" + +# Erigon (single container replaces Heimdall+Bor) +POLYGON_ERIGON_IMAGE="0xpolygon/erigon:v3.4.0" +POLYGON_HEIMDALL_API_URL="https://heimdall-api.polygon.technology" + +# Instance configuration +POLYGON_INSTANCE_TYPE="m7g.4xlarge" # Graviton3: 16 vCPU, 64 GB RAM +POLYGON_CPU_TYPE="ARM_64" # Options: "ARM_64" | "x86_64" + +# Storage configuration +POLYGON_DATA_VOL_SIZE="8000" # 8 TB (mainnet ~4.5 TB extracted + 2x growth buffer) +POLYGON_DATA_VOL_TYPE="gp3" # Options: "gp3" | "io2" | "io1" +POLYGON_DATA_VOL_IOPS="16000" # Recommended: 16,000 +POLYGON_DATA_VOL_THROUGHPUT="1000" # Recommended: 1,000 MB/s + +# HA RPC nodes configuration +POLYGON_RPC_INSTANCE_TYPE="m7g.4xlarge" +POLYGON_RPC_CPU_TYPE="ARM_64" +POLYGON_RPC_DATA_VOL_SIZE="8000" +POLYGON_RPC_DATA_VOL_TYPE="gp3" +POLYGON_RPC_DATA_VOL_IOPS="16000" +POLYGON_RPC_DATA_VOL_THROUGHPUT="1000" +POLYGON_RPC_NUMBER_OF_NODES="2" +POLYGON_RPC_ALB_HEALTHCHECK_GRACE_PERIOD_MIN="10" +POLYGON_RPC_HA_NODES_HEARTBEAT_DELAY_MIN="60" diff --git a/lib/polygon/test/ha-nodes-stack.test.ts b/lib/polygon/test/ha-nodes-stack.test.ts new file mode 100644 index 00000000..78ca23e2 --- /dev/null +++ b/lib/polygon/test/ha-nodes-stack.test.ts @@ -0,0 +1,56 @@ +import { Match, Template } from "aws-cdk-lib/assertions"; +import * as cdk from "aws-cdk-lib"; +import * as ec2 from "aws-cdk-lib/aws-ec2"; +import { PolygonHaNodesStack } from "../lib/ha-nodes-stack"; + +describe("PolygonHaNodesStack", () => { + test("synthesizes the way we expect", () => { + const app = new cdk.App(); + + const haNodesStack = new PolygonHaNodesStack(app, "polygon-ha-nodes", { + env: { account: "1234567890", region: "us-east-1" }, + stackName: "polygon-ha-nodes", + network: "amoy" as any, + erigonImage: "0xpolygon/erigon:v3.4.0", + heimdallApiUrl: "https://heimdall-api-amoy.polygon.technology", + instanceType: new ec2.InstanceType("m7g.xlarge"), + instanceCpuType: ec2.AmazonLinuxCpuType.ARM_64, + numberOfNodes: 2, + albHealthCheckGracePeriodMin: 10, + heartBeatDelayMin: 60, + dataVolume: { sizeGiB: 1000, type: "gp3", iops: 5000, throughput: 250 }, + }); + + const template = Template.fromStack(haNodesStack); + + // Has Auto Scaling Group. + template.hasResourceProperties("AWS::AutoScaling::AutoScalingGroup", { + DesiredCapacity: "2", + HealthCheckType: "ELB", + VPCZoneIdentifier: Match.anyValue(), + TargetGroupARNs: Match.anyValue(), + }); + + // Has ALB. + template.hasResourceProperties("AWS::ElasticLoadBalancingV2::LoadBalancer", { + Scheme: "internal", + Type: "application", + SecurityGroups: [Match.anyValue()], + }); + + // Has Launch Template. + template.hasResourceProperties("AWS::EC2::LaunchTemplate", { + LaunchTemplateData: { + InstanceType: "m7g.xlarge", + SecurityGroupIds: [Match.anyValue()], + UserData: Match.anyValue(), + }, + }); + + // Has Security Group. + template.hasResourceProperties("AWS::EC2::SecurityGroup", { + GroupDescription: Match.anyValue(), + VpcId: Match.anyValue(), + }); + }); +}); diff --git a/lib/polygon/test/polygon.test.ts b/lib/polygon/test/polygon.test.ts new file mode 100644 index 00000000..d9dedadd --- /dev/null +++ b/lib/polygon/test/polygon.test.ts @@ -0,0 +1,153 @@ +import { Match, Template } from "aws-cdk-lib/assertions"; +import * as cdk from "aws-cdk-lib"; +import * as ec2 from "aws-cdk-lib/aws-ec2"; +import { Aspects } from "aws-cdk-lib"; +import { AwsSolutionsChecks } from "cdk-nag"; +import { PolygonCommonStack } from "../lib/common-stack"; +import { PolygonSingleNodeStack } from "../lib/single-node-stack"; + +describe("PolygonCommonStack", () => { + test("synthesizes the way we expect", () => { + const app = new cdk.App(); + + const commonStack = new PolygonCommonStack(app, "polygon-common", { + env: { account: "1234567890", region: "us-east-1" }, + stackName: "polygon-nodes-common", + }); + + const template = Template.fromStack(commonStack); + + // Has EC2 instance role with SSM and CloudWatch policies. + template.hasResourceProperties("AWS::IAM::Role", { + AssumeRolePolicyDocument: { + Statement: [ + { + Action: "sts:AssumeRole", + Effect: "Allow", + Principal: { + Service: "ec2.amazonaws.com", + }, + }, + ], + }, + ManagedPolicyArns: [ + { + "Fn::Join": [ + "", + [ + "arn:", + { Ref: "AWS::Partition" }, + ":iam::aws:policy/AmazonSSMManagedInstanceCore", + ], + ], + }, + { + "Fn::Join": [ + "", + [ + "arn:", + { Ref: "AWS::Partition" }, + ":iam::aws:policy/CloudWatchAgentServerPolicy", + ], + ], + }, + ], + }); + + // Exports the instance role ARN. + template.hasOutput("InstanceRoleARN", { + Export: { Name: "PolygonNodeInstanceRoleArn" }, + }); + }); + + test("passes cdk-nag AwsSolutions checks", () => { + const app = new cdk.App(); + + const commonStack = new PolygonCommonStack(app, "polygon-common-nag", { + env: { account: "1234567890", region: "us-east-1" }, + stackName: "polygon-nodes-common-nag", + }); + + Aspects.of(app).add(new AwsSolutionsChecks({ verbose: true })); + + // Force synthesis to trigger nag checks — no errors means pass. + const warnings = app.synth().getStackByName(commonStack.stackName).messages + .filter((m) => m.level === "error"); + expect(warnings).toHaveLength(0); + }); +}); + +describe("PolygonSingleNodeStack", () => { + test("synthesizes the way we expect", () => { + const app = new cdk.App(); + + const singleNodeStack = new PolygonSingleNodeStack(app, "polygon-single-node", { + env: { account: "1234567890", region: "us-east-1" }, + stackName: "polygon-single-node", + network: "amoy" as any, + erigonImage: "0xpolygon/erigon:v3.4.0", + heimdallApiUrl: "https://heimdall-api-amoy.polygon.technology", + instanceType: new ec2.InstanceType("m7g.xlarge"), + instanceCpuType: ec2.AmazonLinuxCpuType.ARM_64, + dataVolume: { sizeGiB: 1000, type: "gp3", iops: 5000, throughput: 250 }, + }); + + const template = Template.fromStack(singleNodeStack); + + // Has EC2 instance. + template.hasResourceProperties("AWS::EC2::Instance", { + InstanceType: "m7g.xlarge", + Monitoring: true, + SecurityGroupIds: Match.anyValue(), + SubnetId: Match.anyValue(), + }); + + // Has security group with expected egress. + template.hasResourceProperties("AWS::EC2::SecurityGroup", { + GroupDescription: Match.anyValue(), + VpcId: Match.anyValue(), + SecurityGroupEgress: [ + { + CidrIp: "0.0.0.0/0", + Description: "Allow all outbound traffic by default", + IpProtocol: "-1", + }, + ], + }); + + // Has Erigon P2P port (30303), torrent (42069), and RPC (8545) in security group inline rules. + template.hasResourceProperties("AWS::EC2::SecurityGroup", { + SecurityGroupIngress: Match.arrayWith([ + Match.objectLike({ FromPort: 30303, IpProtocol: "tcp" }), + Match.objectLike({ FromPort: 42069, IpProtocol: "tcp" }), + Match.objectLike({ FromPort: 8545, IpProtocol: "tcp" }), + ]), + }); + + // No Heimdall ports in security group. + const sgResources = template.findResources("AWS::EC2::SecurityGroup"); + const allIngressPorts = Object.values(sgResources).flatMap( + (r: any) => (r.Properties?.SecurityGroupIngress || []).map((i: any) => i.FromPort) + ); + expect(allIngressPorts).not.toContain(26656); + expect(allIngressPorts).not.toContain(26657); + expect(allIngressPorts).not.toContain(1317); + + // Has EBS data volume. + template.hasResourceProperties("AWS::EC2::Volume", { + AvailabilityZone: Match.anyValue(), + Encrypted: true, + Iops: 5000, + Size: 1000, + Throughput: 250, + VolumeType: "gp3", + }); + + // Has EBS data volume attachment. + template.hasResourceProperties("AWS::EC2::VolumeAttachment", { + Device: "/dev/sdf", + InstanceId: Match.anyValue(), + VolumeId: Match.anyValue(), + }); + }); +}); diff --git a/lib/polygon/tsconfig.json b/lib/polygon/tsconfig.json new file mode 100644 index 00000000..f8ec89d3 --- /dev/null +++ b/lib/polygon/tsconfig.json @@ -0,0 +1,23 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["es2020", "dom"], + "declaration": true, + "strict": true, + "noImplicitAny": true, + "strictNullChecks": true, + "noImplicitThis": true, + "alwaysStrict": true, + "noUnusedLocals": false, + "noUnusedParameters": false, + "noImplicitReturns": true, + "noFallthroughCasesInSwitch": false, + "inlineSourceMap": true, + "inlineSources": true, + "experimentalDecorators": true, + "strictPropertyInitialization": false, + "typeRoots": ["../../node_modules/@types"] + }, + "exclude": ["node_modules", "cdk.out"] +} diff --git a/scripts/run-all-cdk-tests.sh b/scripts/run-all-cdk-tests.sh index a8092e63..d8ed90a2 100755 --- a/scripts/run-all-cdk-tests.sh +++ b/scripts/run-all-cdk-tests.sh @@ -26,7 +26,7 @@ run_test(){ } # Run tests for each blueprint -excluded_directories=("besu-private" "includes" "constructs" "wax" "polygon") +excluded_directories=("besu-private" "includes" "constructs" "wax") for dir in */; do # If dir is not in the array of excluded_directories, run test if [[ "${excluded_directories[*]}" =~ ${dir%/} ]]; then diff --git a/website/docs/Blueprints/Polygon.md b/website/docs/Blueprints/Polygon.md new file mode 100644 index 00000000..3a203396 --- /dev/null +++ b/website/docs/Blueprints/Polygon.md @@ -0,0 +1,8 @@ +--- +sidebar_label: Polygon +--- +# + +import Readme from '../../../lib/polygon/README.md'; + +