Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -20,3 +20,5 @@ scripts/prod_testnet/.env
eds/staging_testnet_canton.local.toml
eds/prod_testnet_canton.local.toml
eds/prod_testnet_canton.local.secrets.toml
ccip/devenv/.env.staging.local
ccip/devenv/.env.prod-testnet.local
9 changes: 9 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,15 @@ run-canton2evm-load: ## Canton→EVM WASP load (requires running devenv + env-ca
run-evm2canton-load: ## EVM→Canton WASP load (requires running devenv + env-canton-evm-out.toml).
cd ccip/devenv/tests/load && go test -timeout 15m -v -count 1 -run '^TestEVM2Canton_Load$$'

.PHONY: run-evm2canton-load-prod
run-evm2canton-load-prod: ## EVM→Canton WASP load on prod-testnet (Canton TestNet + Sepolia; set CANTON_* and PRIVATE_KEY).
cd ccip/devenv/tests/load && go test -timeout 45m -v -count 1 -ccip-env=prod-testnet -run '^TestEVM2Canton_Load$$'

.PHONY: run-canton2evm-load-prod
run-canton2evm-load-prod: ## Canton→EVM WASP load on prod-testnet (send-only; set CANTON_* and PRIVATE_KEY).
cd ccip/devenv/tests/load && CANTON_LOAD_SKIP_EXEC_CONFIRM=true go test -timeout 30m -v -count=1 \
-ccip-env=prod-testnet -run '^TestCanton2EVM_Load$$'

.PHONY: run-canton2evm-token-load
run-canton2evm-token-load: ## Canton→EVM token WASP load (requires running devenv + env-canton-evm-out.toml).
cd ccip/devenv/tests/load && go test -timeout 20m -v -count 1 -run '^TestCanton2EVM_TokenLoad$$'
Expand Down
145 changes: 141 additions & 4 deletions ccip/devenv/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,13 +30,54 @@ After the environment is spun up, you can run a test like:
make run-e2e-tests
```

### E2E environment selection (`-ccip-env` / `CCIP_ENV`)

Message e2e tests run against **local devenv** by default or **Canton TestNet + Sepolia** when prod-testnet is selected. Set the environment by name (not TOML path):

| Value | Config file | Remote |
|---|---|---|
| `devenv` (default) | [`env-canton-evm-out.toml`](./env-canton-evm-out.toml) | no |
| `prod-testnet` | [`env-prod-testnet-out.toml`](./env-prod-testnet-out.toml) | yes |

Use the `-ccip-env` flag or `CCIP_ENV` env var (flag wins if both are set):

```bash
# Local devenv (default)
cd ccip/devenv/tests/e2e && go test -v -run 'TestCanton2EVM_Basic/EOA' -count=1

# Prod testnet
CCIP_ENV=prod-testnet \
CANTON_GRPC_URL=... CANTON_PARTY_ID=... CANTON_AUTH_*=... \
PRIVATE_KEY=... \
go test -timeout 8m -v -count=1 -ccip-env=prod-testnet \
-run 'TestEVM2Canton_Basic/message|TestCanton2EVM_Basic/EOA'
```

**Prod prerequisites**

- Canton party wallet funded with at least **50 Amulet units** (message fee)
- Canton auth env vars (`CANTON_GRPC_URL`, `CANTON_PARTY_ID`, `CANTON_AUTH_*`) — see [Prod testnet connection smoke test](#prod-testnet-connection-smoke-test)
- Sepolia gas via `PRIVATE_KEY` (EVM sender / receiver for prod runs)

**Optional instance ID overrides** (defaults: `test-router`, `e2e-ccipsender`, `e2e-receiver`):

| Env var | Default |
|---|---|
| `CANTON_ROUTER_INSTANCE_ID` | `test-router` |
| `CANTON_SENDER_INSTANCE_ID` | `e2e-ccipsender` |
| `CANTON_RECEIVER_INSTANCE_ID` | `e2e-receiver` |

Token e2e subtests are skipped on prod-testnet. A second prod run reuses existing router/sender/receiver contracts on ledger when instance IDs match.

## Load tests

Load tests live in `ccip/devenv/tests/load`. They use [WASP](https://pkg.go.dev/github.com/smartcontractkit/chainlink-testing-framework/wasp) and run sequentially (RPS=1) because Canton holdings are single-flight.

### Canton → EVM load (requires devenv)
### Canton → EVM load

Sequential Canton→EVM messages round-robined across every EVM destination in the env file. Requires `make start-devenv` so `ccip/devenv/env-canton-evm-out.toml` exists.
Sequential Canton→EVM messages round-robined across every EVM destination in the env file.

**Devenv** (requires `make start-devenv` so `ccip/devenv/env-canton-evm-out.toml` exists): pre-mints fee holdings and calls `SetupSend` once before WASP starts. Full send + EVM exec confirmation per message.

Schedule is configured via env vars (defaults are `1/1s` for 90s):

Expand Down Expand Up @@ -64,6 +105,27 @@ cd ccip/devenv/tests/load && go test -timeout 15m -v -count 1 -run '^TestCanton2

If the out file is missing the test skips with a hint.

**Prod-testnet** (Canton TestNet + Sepolia): send-only message load — Canton send + confirm send, no `ConfirmExecOnDest` on EVM (executor not available on prod). Set `CANTON_LOAD_SKIP_EXEC_CONFIRM=true`. Verify delivery via indexer/CCIP ops; the test does not assert EVM execution.

Prerequisites: `CANTON_GRPC_URL`, `CANTON_PARTY_ID`, `CANTON_AUTH_*`, `PRIVATE_KEY` (EVM message receiver wallet), and a pre-funded Canton party (~50 Amulet per message at `CantonToEVMFeeAmount`).

```bash
CCIP_ENV=prod-testnet \
CANTON_LOAD_SKIP_EXEC_CONFIRM=true \
CANTON_GRPC_URL=... CANTON_PARTY_ID=... CANTON_AUTH_CLIENT_ID=... CANTON_AUTH_CLIENT_SECRET=... \
PRIVATE_KEY=0x... \
CANTON_LOAD_MESSAGE_RATE=1/10s \
CANTON_LOAD_DURATION=5m \
go test -timeout 30m -v -count=1 -ccip-env=prod-testnet \
-run '^TestCanton2EVM_Load$' ./ccip/devenv/tests/load/
```

Or from repo root:

```bash
make run-canton2evm-load-prod
```

### Canton → EVM token load (requires devenv)

Separate test from message-only load: `TestCanton2EVM_TokenLoad`. Resolves the token lane declared in [`token_transfer_config.toml`](./tests/token_transfer_config.toml) (see [Token lane configuration](#token-lane-configuration)) against the source chain's `GetTokenTransferConfigs`, validating every destination has the lane. Pre-mints Canton fee + transfer holdings, runs WASP, then asserts EVM receiver token balance delta.
Expand All @@ -78,9 +140,11 @@ Equivalent:
cd ccip/devenv/tests/load && go test -timeout 20m -v -count 1 -run '^TestCanton2EVM_TokenLoad$'
```

### EVM → Canton load (requires devenv)
### EVM → Canton load

Sequential EVM→Canton messages against the Canton destination in the env file. Uses the same schedule env vars as Canton→EVM (`CANTON_LOAD_MESSAGE_RATE`, `CANTON_LOAD_DURATION`). EVM accounts are pre-funded by devenv; no Canton pre-mint.
Sequential EVM→Canton messages against the Canton destination in the env file. Uses the same schedule env vars as Canton→EVM (`CANTON_LOAD_MESSAGE_RATE`, `CANTON_LOAD_DURATION`).

**Devenv** (requires running devenv + `env-canton-evm-out.toml`): EVM accounts are pre-funded by devenv; no Canton pre-mint.

Example — 1 message every 20 seconds for 10 minutes:

Expand All @@ -99,6 +163,27 @@ Equivalent:
cd ccip/devenv/tests/load && go test -timeout 15m -v -count 1 -run '^TestEVM2Canton_Load$'
```

**Prod-testnet** (Canton TestNet + Sepolia): message-only load with full per-message confirmation (send → receipt on EVM → `ConfirmExecOnDest` on Canton). Use a conservative rate — each iteration is synchronous (~30–60s end-to-end on prod), so WASP RPS=1 is effectively bounded by confirm latency. Budget for Sepolia gas per send plus Canton execution fees. Token load remains devenv-only; Canton→EVM message-only prod load is supported send-only (see above).

Prerequisites: `CANTON_GRPC_URL`, `CANTON_PARTY_ID`, `CANTON_AUTH_*`, `PRIVATE_KEY` (Sepolia sender/receiver wallet), and a pre-funded Canton party.

```bash
CCIP_ENV=prod-testnet \
CANTON_GRPC_URL=... CANTON_PARTY_ID=... CANTON_AUTH_CLIENT_ID=... CANTON_AUTH_CLIENT_SECRET=... \
PRIVATE_KEY=0x... \
CANTON_LOAD_MESSAGE_RATE=1/30s \
CANTON_LOAD_DURATION=5m \
CANTON_CONFIRM_EXEC_TIMEOUT=10m \
go test -timeout 45m -v -count=1 -ccip-env=prod-testnet \
-run '^TestEVM2Canton_Load$' ./ccip/devenv/tests/load/
```

Or from repo root:

```bash
make run-evm2canton-load-prod
```

### EVM → Canton token load (requires devenv)

Separate test from message-only load: `TestEVM2Canton_TokenLoad`. Resolves the token lane declared in [`token_transfer_config.toml`](./tests/token_transfer_config.toml) (see [Token lane configuration](#token-lane-configuration)), logs EVM sender balance vs estimated transfer need (devenv pre-funds sender), runs WASP, logs Canton holdings post-run.
Expand Down Expand Up @@ -177,3 +262,55 @@ If you want to build docker images, spin up a new env, and run the test in a sin
# From repo root
make build-run-e2e-tests
```

## Prod testnet connection smoke test

Minimal Canton-only connectivity check against real testnet infrastructure. Uses [`env-prod-testnet-out.toml`](./env-prod-testnet-out.toml) (Canton TestNet ↔ Sepolia message lane: contract refs, indexer URLs, EDS URL, verifier issuers); auth secrets and party identity come from environment variables.

The test is opt-in: it skips unless `CANTON_GRPC_URL` is set (even though the TOML file includes default URLs). This keeps CI green while allowing manual or workflow-triggered runs.

### Environment variables

| Variable | Required | Notes |
|---|---|---|
| `CANTON_PARTY_ID` | yes | Ledger party to query; skips `GetUser` when set |
| `CANTON_AUTH_URL` | yes | OIDC issuer |
| `CANTON_CLIENT_ID` | yes | OAuth2 client ID |
| `CANTON_AUTH_TYPE` | no | `authorizationCode` (local), `clientCredentials` (CI), `static`, `insecureStatic` |
| `CANTON_USER_ID` | no | Required for `clientCredentials`; optional for `authorizationCode` (extracted from token `sub` after login) |
| `CANTON_CLIENT_SECRET` | CI only | Required when `CANTON_AUTH_TYPE=clientCredentials` |
| `CANTON_JWT` | static only | For `static` / `insecureStatic` auth |
| `CANTON_GRPC_URL` | opt-in signal | Must be set to run the test; overrides TOML gRPC URL |
| `CANTON_VALIDATOR_API_URL` | no | Overrides TOML validator API URL |

**Local (browser Okta login):**

```bash
export CANTON_GRPC_URL='testnet.cv1.bcy-v.metalhosts.com:443'
export CANTON_PARTY_ID='u_0e0328cbbcb7::1220c250c23c55120f7c758bccc5cbc739629015ab921594e1c29656981f985bffa7'
export CANTON_AUTH_URL='https://smartcontract.okta.com/oauth2/austsuml9q2WhPBMM5d7'
export CANTON_CLIENT_ID='0oau1l22b1Jv3dcih5d7'
export CANTON_AUTH_TYPE='authorizationCode'
```

**CI (`clientCredentials`, no browser):**

```bash
export CANTON_GRPC_URL='testnet.cv1.bcy-v.metalhosts.com:443'
export CANTON_PARTY_ID='...'
export CANTON_AUTH_URL='https://smartcontract.okta.com/oauth2/austsuml9q2WhPBMM5d7'
export CANTON_CLIENT_ID='0oau1l22b1Jv3dcih5d7'
export CANTON_AUTH_TYPE='clientCredentials'
export CANTON_CLIENT_SECRET='...'
export CANTON_USER_ID='...'
```

### Run command

```bash
cd ccip/devenv/tests/integration && go test -v -run TestIntegration_CantonProdTestnet_Connection -count=1
```

Use `-ccip-env=prod-testnet` (or `CCIP_ENV=prod-testnet`) so the test loads `env-prod-testnet-out.toml`; default devenv config is `env-canton-evm-out.toml` via `-ccip-env=devenv`.

The test connects via `NewCLDF`, asserts `PartyID` is set, and lists holdings for the party (empty balance is OK).
90 changes: 62 additions & 28 deletions ccip/devenv/cldf.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,14 @@ import (
"context"
"fmt"

adminv2 "github.com/digital-asset/dazl-client/v8/go/api/com/daml/ledger/api/v2/admin"
chainsel "github.com/smartcontractkit/chain-selectors"
"github.com/smartcontractkit/chainlink-deployments-framework/chain/canton/provider/authentication"
"github.com/smartcontractkit/chainlink-testing-framework/framework/components/blockchain"
"google.golang.org/grpc"

cldf_chain "github.com/smartcontractkit/chainlink-deployments-framework/chain"
cldf_canton_provider "github.com/smartcontractkit/chainlink-deployments-framework/chain/canton/provider"

"github.com/smartcontractkit/chainlink-canton/commonconfig"
)

func NewCLDF(ctx context.Context, b *blockchain.Input) (cldf_chain.BlockChain, uint64, error) {
Expand All @@ -24,46 +24,80 @@ func NewCLDF(ctx context.Context, b *blockchain.Input) (cldf_chain.BlockChain, u
Participants: make([]cldf_canton_provider.ParticipantConfig, len(b.Out.NetworkSpecificData.CantonData.ExternalEndpoints.Participants)),
}

presetPartyID := resolvePartyID()
internalParticipants := b.Out.NetworkSpecificData.CantonData.InternalEndpoints.Participants

for i, config := range b.Out.NetworkSpecificData.CantonData.ExternalEndpoints.Participants {
authProvider := authentication.NewInsecureStaticProvider(config.JWT)
// Get Primary Party for user
ledgerApiConn, err := grpc.NewClient(
config.GRPCLedgerAPIURL,
grpc.WithTransportCredentials(authProvider.TransportCredentials()),
grpc.WithPerRPCCredentials(authProvider.PerRPCCredentials()),
)
authCfg := resolveAuthConfig(config.JWT, config.UserID)
authProvider, err := newAuthProvider(ctx, authCfg)
if err != nil {
return nil, 0, fmt.Errorf("failed to create gRPC connection to Ledger API for Canton participant %d: %w", i+1, err)
return nil, 0, fmt.Errorf("create auth provider for Canton participant %d: %w", i+1, err)
}
userResp, err := adminv2.NewUserManagementServiceClient(ledgerApiConn).GetUser(context.Background(), &adminv2.GetUserRequest{UserId: config.UserID})

userID := resolveUserID(config.UserID)
if userID == "" && authCfg.Type == commonconfig.AuthTypeAuthorizationCode {
userID, err = userIDFromToken(ctx, authProvider)
if err != nil {
return nil, 0, fmt.Errorf("resolve user id for Canton participant %d: %w", i+1, err)
}
}

grpcURL := resolveGRPCLedgerURL(config.GRPCLedgerAPIURL)
party, err := func() (string, error) {
conn, err := grpc.NewClient(
grpcURL,
grpc.WithTransportCredentials(authProvider.TransportCredentials()),
grpc.WithPerRPCCredentials(authProvider.PerRPCCredentials()),
)
if err != nil {
return "", fmt.Errorf("create gRPC connection to Ledger API for Canton participant %d: %w", i+1, err)
}
defer conn.Close()

party, err := resolveParticipantParty(ctx, conn, userID, presetPartyID)
if err != nil {
return "", fmt.Errorf("resolve party for Canton participant %d: %w", i+1, err)
}

return party, nil
}()
if err != nil {
return nil, 0, fmt.Errorf("failed to get user info for user %s for Canton participant %d: %w", config.UserID, i+1, err)
return nil, 0, err
}
party := userResp.GetUser().GetPrimaryParty()
if party == "" {
return nil, 0, fmt.Errorf("no primary party found for user %s for Canton participant %d", config.UserID, i+1)

var internalEndpoints *cldf_canton_provider.Endpoints
if i < len(internalParticipants) {
internal := internalParticipants[i]
if endpointsNonEmpty(
internal.JSONLedgerAPIURL,
internal.GRPCLedgerAPIURL,
internal.AdminAPIURL,
internal.ValidatorAPIURL,
) {
internalEndpoints = &cldf_canton_provider.Endpoints{
JSONLedgerAPIURL: internal.JSONLedgerAPIURL,
GRPCLedgerAPIURL: internal.GRPCLedgerAPIURL,
AdminAPIURL: internal.AdminAPIURL,
ValidatorAPIURL: internal.ValidatorAPIURL,
}
}
}
_ = ledgerApiConn.Close()

providerConfig.Participants[i] = cldf_canton_provider.ParticipantConfig{
Endpoints: cldf_canton_provider.Endpoints{
JSONLedgerAPIURL: config.JSONLedgerAPIURL,
GRPCLedgerAPIURL: config.GRPCLedgerAPIURL,
GRPCLedgerAPIURL: grpcURL,
AdminAPIURL: config.AdminAPIURL,
ValidatorAPIURL: config.ValidatorAPIURL,
ValidatorAPIURL: resolveValidatorAPIURL(config.ValidatorAPIURL),
},
InternalEndpoints: &cldf_canton_provider.Endpoints{
JSONLedgerAPIURL: b.Out.NetworkSpecificData.CantonData.InternalEndpoints.Participants[i].JSONLedgerAPIURL,
GRPCLedgerAPIURL: b.Out.NetworkSpecificData.CantonData.InternalEndpoints.Participants[i].GRPCLedgerAPIURL,
AdminAPIURL: b.Out.NetworkSpecificData.CantonData.InternalEndpoints.Participants[i].AdminAPIURL,
ValidatorAPIURL: b.Out.NetworkSpecificData.CantonData.InternalEndpoints.Participants[i].ValidatorAPIURL,
},
UserID: config.UserID,
PartyID: party,
AuthProvider: authProvider,
InternalEndpoints: internalEndpoints,
UserID: userID,
PartyID: party,
AuthProvider: authProvider,
}
}
p, err := cldf_canton_provider.NewRPCChainProvider(d.ChainSelector, providerConfig).Initialize(context.TODO())

p, err := cldf_canton_provider.NewRPCChainProvider(d.ChainSelector, providerConfig).Initialize(ctx)
if err != nil {
return nil, 0, err
}
Expand Down
Loading
Loading