diff --git a/README.md b/README.md index c6e673f6..870fa20f 100644 --- a/README.md +++ b/README.md @@ -255,6 +255,21 @@ starforge contract generate-bindings ./my_contract.wasm --lang rust starforge contract generate-bindings ./my_contract.wasm --lang ts ``` +### Rollback safety testing + +```bash +# Validate that an upgraded contract can be rolled back without losing critical state +starforge test \ + --wasm target/wasm32-unknown-unknown/release/my_contract_v2.wasm \ + --rollback \ + --previous-wasm target/wasm32-unknown-unknown/release/my_contract_v1.wasm \ + --rollback-scenario tests/rollback/token-balances.json \ + --rollback-performance-budget-ms 1000 \ + --report json +``` + +The rollback harness checks state preservation, rollback scenarios, data integrity invariants, and rollback performance budgets. See [ROLLBACK_TESTING.md](ROLLBACK_TESTING.md) for scenario schema and CI examples. + ### Environment info ```bash diff --git a/ROLLBACK_TESTING.md b/ROLLBACK_TESTING.md new file mode 100644 index 00000000..957e0dc3 --- /dev/null +++ b/ROLLBACK_TESTING.md @@ -0,0 +1,161 @@ +# Contract Rollback Testing Framework + +StarForge includes a rollback-specific test harness for validating that a Soroban contract upgrade can be safely reverted without losing critical contract state. + +The harness is designed for CI and local pre-release checks. It compares a previous WASM rollback target with an upgraded WASM, applies one or more rollback scenarios against a deterministic mock state model, and fails when preserved keys, state invariants, data integrity checks, or performance budgets are violated. + +## Quick Start + +Run the default rollback scenario: + +```bash +starforge test \ + --wasm target/wasm32-unknown-unknown/release/contract_v2.wasm \ + --rollback \ + --previous-wasm target/wasm32-unknown-unknown/release/contract_v1.wasm \ + --report json +``` + +Run custom scenarios: + +```bash +starforge test \ + --wasm ./contract_v2.wasm \ + --rollback \ + --previous-wasm ./contract_v1.wasm \ + --rollback-scenario ./rollback-scenarios/token-balances.json \ + --rollback-scenario ./rollback-scenarios/admin-controls.json \ + --rollback-performance-budget-ms 500 \ + --report html +``` + +Reports are written under StarForge's reports directory in the local StarForge config folder. + +## What the Harness Validates + +| Acceptance criterion | Harness coverage | +| --- | --- | +| Rollback harness works | `starforge test --rollback` runs a dedicated rollback harness over previous/upgraded WASM pairs. | +| State preservation tested | `preserved_keys` compare values immediately before upgrade with values after rollback. | +| Scenario testing | One JSON file can define a single scenario; another can define an array of scenarios. Pass multiple `--rollback-scenario` flags to compose suites. | +| Integrity checks | `key_exists`, `key_absent`, `equals`, `checksum_unchanged`, `numeric_sum_equals`, and `no_unexpected_keys` checks are supported. | +| Performance testing | Each scenario receives a duration check against `max_duration_ms` or `--rollback-performance-budget-ms`. | +| Documentation | This document defines workflow, schema, and CI usage. | + +## Scenario Model + +A scenario models the storage lifecycle around an upgrade and rollback: + +1. `initial_state` seeds contract storage. +2. `pre_upgrade_mutations` optionally create state that should exist immediately before the upgrade. +3. The harness snapshots this pre-upgrade state. +4. `upgrade_mutations` simulate migration or behavior introduced by the upgraded WASM. +5. `rollback_mutations` simulate the rollback path back to the previous contract version. +6. `preserved_keys`, `expected_after_rollback`, and `integrity_checks` validate the final state. + +This deterministic mock model is intentionally independent of a live chain so that rollback safety tests can run quickly in CI. It should be used alongside integration or testnet rollback drills for final release validation. + +## Scenario Schema Example + +```json +{ + "name": "token_balances_survive_rollback", + "description": "Critical balances and supply remain intact when v2 is rolled back to v1.", + "initial_state": { + "admin": "GADMIN", + "balance:alice": 1000, + "balance:bob": 500, + "total_supply": 1500, + "schema_version": 1 + }, + "pre_upgrade_mutations": [], + "upgrade_mutations": [ + { "operation": "set", "key": "schema_version", "value": 2 }, + { "operation": "set", "key": "feature:new_accounting", "value": true } + ], + "rollback_mutations": [ + { "operation": "set", "key": "schema_version", "value": 1 } + ], + "preserved_keys": [ + "admin", + "balance:alice", + "balance:bob", + "total_supply" + ], + "expected_after_rollback": { + "schema_version": 1, + "balance:alice": 1000, + "balance:bob": 500, + "total_supply": 1500 + }, + "integrity_checks": [ + { "kind": "key_exists", "key": "admin" }, + { + "kind": "checksum_unchanged", + "keys": ["admin", "balance:alice", "balance:bob", "total_supply"] + }, + { + "kind": "numeric_sum_equals", + "keys": ["balance:alice", "balance:bob"], + "expected_sum": 1500 + } + ], + "max_duration_ms": 1000 +} +``` + +A file may also contain an array of scenarios: + +```json +[ + { "name": "scenario_one", "initial_state": {}, "max_duration_ms": 1000 }, + { "name": "scenario_two", "initial_state": {}, "max_duration_ms": 1000 } +] +``` + +## Mutation Operations + +| Operation | Required fields | Behavior | +| --- | --- | --- | +| `set` | `key`, `value` | Writes or replaces a state value. | +| `delete` | `key` | Removes a key from state. | +| `increment` | `key`, integer `value` | Adds the integer value to an existing numeric key, or starts at `0` when absent. | + +## Integrity Check Types + +| Check | Required fields | Purpose | +| --- | --- | --- | +| `key_exists` | `key` | Ensures a critical key remains present after rollback. | +| `key_absent` | `key` | Ensures a temporary or unsafe migration key is removed after rollback. | +| `equals` | `key`, `value` | Ensures a key has an exact final value. | +| `checksum_unchanged` | optional `keys` | Compares a canonical SHA-256 checksum before upgrade and after rollback. If `keys` is omitted, the full state map is compared. | +| `numeric_sum_equals` | `keys`, `expected_sum` | Ensures a set of numeric values preserves a supply or balance total. | +| `no_unexpected_keys` | `allowed_keys` | Fails if rollback leaves any keys outside an allowlist. | + +## Recommended Rollback Test Suite + +For each upgrade, add scenarios that cover: + +- balances, allowances, ownership/admin keys, authorization lists, and supply counters; +- storage schema migrations and reverse migrations; +- rollback after partially completed feature initialization; +- rollback after user activity on the upgraded version; +- removal of temporary migration keys; +- performance budgets for high-volume state maps. + +## CI Example + +```yaml +- name: Rollback safety tests + run: | + cargo run -- test \ + --wasm artifacts/contract_v2.wasm \ + --rollback \ + --previous-wasm artifacts/contract_v1.wasm \ + --rollback-scenario tests/rollback/token-balances.json \ + --rollback-scenario tests/rollback/admin-controls.json \ + --rollback-performance-budget-ms 750 \ + --report json +``` + +A non-zero exit indicates at least one rollback scenario failed and the upgrade should not be shipped until data preservation is fixed. diff --git a/src/commands/test.rs b/src/commands/test.rs index 6acb21d2..93908515 100644 --- a/src/commands/test.rs +++ b/src/commands/test.rs @@ -1,4 +1,4 @@ -use crate::utils::{config, print as p, test_automation, test_runner}; +use crate::utils::{config, print as p, rollback_testing, test_automation, test_runner}; use anyhow::Result; use clap::Args; use std::path::PathBuf; @@ -13,6 +13,22 @@ pub struct TestArgs { #[arg(long)] pub source: Option, + /// Run the rollback safety test harness for a previous/upgraded contract pair + #[arg(long, default_value = "false")] + pub rollback: bool, + + /// Path to the previous compiled wasm used as the rollback target + #[arg(long = "previous-wasm")] + pub previous_wasm: Option, + + /// Rollback scenario JSON file. Can be passed multiple times. + #[arg(long = "rollback-scenario")] + pub rollback_scenario: Vec, + + /// Maximum allowed rollback scenario duration in milliseconds + #[arg(long = "rollback-performance-budget-ms", default_value = "1000")] + pub rollback_performance_budget_ms: u64, + /// Collect coverage analysis (requires --source) #[arg(long, default_value = "false")] pub coverage: bool, @@ -63,6 +79,68 @@ pub async fn handle(args: TestArgs) -> Result<()> { if args.parallel { p::kv("Workers", &args.workers.to_string()); } + if args.rollback { + p::kv("Rollback harness", "enabled"); + p::kv( + "Rollback scenarios", + if args.rollback_scenario.is_empty() { + "default" + } else { + "custom" + }, + ); + } + + if args.rollback { + let previous_wasm = args.previous_wasm.clone().ok_or_else(|| { + anyhow::anyhow!("--rollback requires --previous-wasm ") + })?; + config::validate_file_path(&previous_wasm, Some("wasm"))?; + for scenario in &args.rollback_scenario { + config::validate_file_path(scenario, Some("json"))?; + } + + p::info("Running contract rollback safety harness..."); + let report = rollback_testing::run_rollback_tests(rollback_testing::RollbackTestOptions { + previous_wasm, + upgraded_wasm: args.wasm.clone(), + scenario_paths: args.rollback_scenario.clone(), + performance_budget_ms: args.rollback_performance_budget_ms, + report_format: args.report.clone(), + })?; + + println!(); + p::separator(); + p::kv_accent("Previous SHA256", &report.previous_wasm_hash); + p::kv_accent("Upgraded SHA256", &report.upgraded_wasm_hash); + p::kv("Rollback scenarios", &report.total_scenarios.to_string()); + p::kv("Passed", &report.passed.to_string()); + p::kv("Failed", &report.failed.to_string()); + p::kv("Duration", &format!("{}ms", report.total_duration_ms)); + if let Some(path) = &report.report_path { + p::kv("Rollback report", &path.display().to_string()); + } + + for scenario in &report.scenario_results { + println!(); + p::kv( + &format!("Scenario {}", scenario.scenario_name), + if scenario.passed { "pass" } else { "fail" }, + ); + for check in &scenario.checks { + let marker = if check.passed { "✓" } else { "✗" }; + println!(" {} {:?}: {}", marker, check.category, check.message); + } + } + p::separator(); + + if report.failed > 0 { + anyhow::bail!("Rollback safety checks failed"); + } + + p::success("Rollback safety checks passed"); + return Ok(()); + } // Handle automated test generation if args.generate { diff --git a/src/utils/mod.rs b/src/utils/mod.rs index ce941859..7c967434 100644 --- a/src/utils/mod.rs +++ b/src/utils/mod.rs @@ -23,6 +23,7 @@ pub mod print; pub mod profiler; pub mod registry; pub mod repl; +pub mod rollback_testing; pub mod sandbox; pub mod security; pub mod social; diff --git a/src/utils/rollback_testing.rs b/src/utils/rollback_testing.rs new file mode 100644 index 00000000..79256634 --- /dev/null +++ b/src/utils/rollback_testing.rs @@ -0,0 +1,756 @@ +use anyhow::{Context, Result}; +use serde::{Deserialize, Serialize}; +use serde_json::Value; +use sha2::{Digest, Sha256}; +use std::collections::{BTreeMap, BTreeSet}; +use std::fs; +use std::path::{Path, PathBuf}; +use std::time::Instant; + +pub type StateMap = BTreeMap; + +#[derive(Debug, Clone)] +pub struct RollbackTestOptions { + pub previous_wasm: PathBuf, + pub upgraded_wasm: PathBuf, + pub scenario_paths: Vec, + pub performance_budget_ms: u64, + pub report_format: Option, +} + +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct RollbackScenario { + pub name: String, + #[serde(default)] + pub description: String, + #[serde(default)] + pub initial_state: StateMap, + #[serde(default)] + pub pre_upgrade_mutations: Vec, + #[serde(default)] + pub upgrade_mutations: Vec, + #[serde(default)] + pub rollback_mutations: Vec, + #[serde(default)] + pub preserved_keys: Vec, + #[serde(default)] + pub expected_after_rollback: StateMap, + #[serde(default)] + pub integrity_checks: Vec, + pub max_duration_ms: Option, +} + +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct StateMutation { + pub operation: MutationOperation, + pub key: String, + #[serde(default)] + pub value: Option, +} + +#[derive(Debug, Clone, Copy, Serialize, Deserialize)] +#[serde(rename_all = "snake_case")] +pub enum MutationOperation { + Set, + Delete, + Increment, +} + +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct IntegrityCheck { + pub kind: IntegrityCheckKind, + #[serde(default)] + pub key: Option, + #[serde(default)] + pub value: Option, + #[serde(default)] + pub keys: Option>, + #[serde(default)] + pub allowed_keys: Option>, + #[serde(default)] + pub expected_sum: Option, +} + +#[derive(Debug, Clone, Copy, Serialize, Deserialize)] +#[serde(rename_all = "snake_case")] +pub enum IntegrityCheckKind { + KeyExists, + KeyAbsent, + Equals, + ChecksumUnchanged, + NumericSumEquals, + NoUnexpectedKeys, +} + +#[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq)] +#[serde(rename_all = "snake_case")] +pub enum RollbackCheckCategory { + WasmValidation, + StatePreservation, + ScenarioExpectation, + DataIntegrity, + Performance, +} + +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct RollbackCheckResult { + pub category: RollbackCheckCategory, + pub name: String, + pub passed: bool, + pub message: String, +} + +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct RollbackScenarioResult { + pub scenario_name: String, + pub description: String, + pub passed: bool, + pub duration_ms: u64, + pub checks: Vec, + pub before_upgrade_checksum: String, + pub after_upgrade_checksum: String, + pub after_rollback_checksum: String, +} + +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct RollbackTestReport { + pub previous_wasm_hash: String, + pub upgraded_wasm_hash: String, + pub total_scenarios: u32, + pub passed: u32, + pub failed: u32, + pub total_duration_ms: u64, + pub scenario_results: Vec, + pub report_path: Option, + pub generated_at: String, +} + +pub fn run_rollback_tests(options: RollbackTestOptions) -> Result { + let started = Instant::now(); + let previous_bytes = validate_wasm_file(&options.previous_wasm) + .with_context(|| format!("Invalid previous WASM: {}", options.previous_wasm.display()))?; + let upgraded_bytes = validate_wasm_file(&options.upgraded_wasm) + .with_context(|| format!("Invalid upgraded WASM: {}", options.upgraded_wasm.display()))?; + + let previous_wasm_hash = sha256_hex(&previous_bytes); + let upgraded_wasm_hash = sha256_hex(&upgraded_bytes); + let scenarios = load_scenarios(&options.scenario_paths)?; + + let mut scenario_results = Vec::new(); + for scenario in &scenarios { + let mut result = execute_scenario(scenario, options.performance_budget_ms)?; + result.checks.insert( + 0, + RollbackCheckResult { + category: RollbackCheckCategory::WasmValidation, + name: "wasm_versions_are_distinct".to_string(), + passed: previous_wasm_hash != upgraded_wasm_hash, + message: if previous_wasm_hash != upgraded_wasm_hash { + "previous and upgraded WASM hashes are distinct".to_string() + } else { + "previous and upgraded WASM hashes are identical; rollback safety cannot be proven for a no-op upgrade".to_string() + }, + }, + ); + result.passed = result.checks.iter().all(|check| check.passed); + scenario_results.push(result); + } + + let passed = scenario_results + .iter() + .filter(|result| result.passed) + .count() as u32; + let failed = scenario_results.len() as u32 - passed; + + let mut report = RollbackTestReport { + previous_wasm_hash, + upgraded_wasm_hash, + total_scenarios: scenario_results.len() as u32, + passed, + failed, + total_duration_ms: started.elapsed().as_millis() as u64, + scenario_results, + report_path: None, + generated_at: chrono::Utc::now().to_rfc3339(), + }; + + if let Some(format) = options.report_format.as_deref() { + report.report_path = Some(write_rollback_report(&report, format)?); + } + + Ok(report) +} + +pub fn load_scenarios(paths: &[PathBuf]) -> Result> { + if paths.is_empty() { + return Ok(default_scenarios()); + } + + let mut scenarios = Vec::new(); + for path in paths { + let content = fs::read_to_string(path) + .with_context(|| format!("Failed to read rollback scenario file {}", path.display()))?; + + match serde_json::from_str::(&content) { + Ok(scenario) => scenarios.push(scenario), + Err(single_error) => match serde_json::from_str::>(&content) { + Ok(mut loaded) => scenarios.append(&mut loaded), + Err(array_error) => { + anyhow::bail!( + "Failed to parse rollback scenario file {} as a scenario ({}) or scenario array ({})", + path.display(), + single_error, + array_error + ); + } + }, + } + } + + if scenarios.is_empty() { + anyhow::bail!("No rollback scenarios were loaded"); + } + Ok(scenarios) +} + +pub fn default_scenarios() -> Vec { + let mut initial_state = StateMap::new(); + initial_state.insert("admin".to_string(), Value::String("GADMIN".to_string())); + initial_state.insert("balance:alice".to_string(), Value::from(1000)); + initial_state.insert("balance:bob".to_string(), Value::from(500)); + initial_state.insert("total_supply".to_string(), Value::from(1500)); + initial_state.insert("schema_version".to_string(), Value::from(1)); + + let mut expected_after_rollback = StateMap::new(); + expected_after_rollback.insert("schema_version".to_string(), Value::from(1)); + expected_after_rollback.insert("balance:alice".to_string(), Value::from(1000)); + expected_after_rollback.insert("balance:bob".to_string(), Value::from(500)); + expected_after_rollback.insert("total_supply".to_string(), Value::from(1500)); + + vec![RollbackScenario { + name: "default_balance_state_preservation".to_string(), + description: "Verifies that a rollback preserves critical token-like balances, admin data, and supply invariants.".to_string(), + initial_state, + pre_upgrade_mutations: vec![], + upgrade_mutations: vec![ + StateMutation { + operation: MutationOperation::Set, + key: "schema_version".to_string(), + value: Some(Value::from(2)), + }, + StateMutation { + operation: MutationOperation::Set, + key: "feature:new_accounting".to_string(), + value: Some(Value::Bool(true)), + }, + ], + rollback_mutations: vec![StateMutation { + operation: MutationOperation::Set, + key: "schema_version".to_string(), + value: Some(Value::from(1)), + }], + preserved_keys: vec![ + "admin".to_string(), + "balance:alice".to_string(), + "balance:bob".to_string(), + "total_supply".to_string(), + ], + expected_after_rollback, + integrity_checks: vec![ + IntegrityCheck { + kind: IntegrityCheckKind::KeyExists, + key: Some("admin".to_string()), + value: None, + keys: None, + allowed_keys: None, + expected_sum: None, + }, + IntegrityCheck { + kind: IntegrityCheckKind::ChecksumUnchanged, + key: None, + value: None, + keys: Some(vec![ + "admin".to_string(), + "balance:alice".to_string(), + "balance:bob".to_string(), + "total_supply".to_string(), + ]), + allowed_keys: None, + expected_sum: None, + }, + IntegrityCheck { + kind: IntegrityCheckKind::NumericSumEquals, + key: None, + value: None, + keys: Some(vec!["balance:alice".to_string(), "balance:bob".to_string()]), + allowed_keys: None, + expected_sum: Some(1500), + }, + ], + max_duration_ms: Some(1000), + }] +} + +fn execute_scenario( + scenario: &RollbackScenario, + default_performance_budget_ms: u64, +) -> Result { + let started = Instant::now(); + let mut checks = Vec::new(); + let mut state = scenario.initial_state.clone(); + + apply_mutations(&mut state, &scenario.pre_upgrade_mutations).with_context(|| { + format!( + "pre-upgrade mutation failed in scenario '{}'", + scenario.name + ) + })?; + let before_upgrade = state.clone(); + let before_upgrade_checksum = state_checksum(&before_upgrade, None)?; + + apply_mutations(&mut state, &scenario.upgrade_mutations) + .with_context(|| format!("upgrade mutation failed in scenario '{}'", scenario.name))?; + let after_upgrade_checksum = state_checksum(&state, None)?; + + apply_mutations(&mut state, &scenario.rollback_mutations) + .with_context(|| format!("rollback mutation failed in scenario '{}'", scenario.name))?; + let after_rollback = state; + let after_rollback_checksum = state_checksum(&after_rollback, None)?; + + for key in &scenario.preserved_keys { + let before = before_upgrade.get(key); + let after = after_rollback.get(key); + let passed = before == after; + checks.push(RollbackCheckResult { + category: RollbackCheckCategory::StatePreservation, + name: format!("preserve:{}", key), + passed, + message: if passed { + format!("key '{}' preserved across upgrade and rollback", key) + } else { + format!( + "key '{}' changed or was lost (before: {}, after: {})", + key, + value_for_message(before), + value_for_message(after) + ) + }, + }); + } + + for (key, expected) in &scenario.expected_after_rollback { + let actual = after_rollback.get(key); + let passed = actual == Some(expected); + checks.push(RollbackCheckResult { + category: RollbackCheckCategory::ScenarioExpectation, + name: format!("expect:{}", key), + passed, + message: if passed { + format!("key '{}' matched expected rollback value", key) + } else { + format!( + "key '{}' mismatch after rollback (expected: {}, actual: {})", + key, + expected, + value_for_message(actual) + ) + }, + }); + } + + for integrity_check in &scenario.integrity_checks { + checks.push(evaluate_integrity_check( + integrity_check, + &before_upgrade, + &after_rollback, + )?); + } + + let duration_ms = started.elapsed().as_millis() as u64; + let budget = scenario + .max_duration_ms + .unwrap_or(default_performance_budget_ms) + .max(1); + let performance_passed = duration_ms <= budget; + checks.push(RollbackCheckResult { + category: RollbackCheckCategory::Performance, + name: "rollback_duration_budget".to_string(), + passed: performance_passed, + message: if performance_passed { + format!( + "scenario completed in {}ms within {}ms budget", + duration_ms, budget + ) + } else { + format!( + "scenario took {}ms and exceeded {}ms budget", + duration_ms, budget + ) + }, + }); + + let passed = checks.iter().all(|check| check.passed); + Ok(RollbackScenarioResult { + scenario_name: scenario.name.clone(), + description: scenario.description.clone(), + passed, + duration_ms, + checks, + before_upgrade_checksum, + after_upgrade_checksum, + after_rollback_checksum, + }) +} + +fn apply_mutations(state: &mut StateMap, mutations: &[StateMutation]) -> Result<()> { + for mutation in mutations { + match mutation.operation { + MutationOperation::Set => { + let value = mutation.value.clone().ok_or_else(|| { + anyhow::anyhow!("set mutation for '{}' requires a value", mutation.key) + })?; + state.insert(mutation.key.clone(), value); + } + MutationOperation::Delete => { + state.remove(&mutation.key); + } + MutationOperation::Increment => { + let delta = mutation + .value + .as_ref() + .and_then(Value::as_i64) + .ok_or_else(|| { + anyhow::anyhow!( + "increment mutation for '{}' requires an integer value", + mutation.key + ) + })?; + let current = state + .get(&mutation.key) + .and_then(Value::as_i64) + .unwrap_or(0); + state.insert( + mutation.key.clone(), + Value::Number((current + delta).into()), + ); + } + } + } + Ok(()) +} + +fn evaluate_integrity_check( + check: &IntegrityCheck, + before_upgrade: &StateMap, + after_rollback: &StateMap, +) -> Result { + match check.kind { + IntegrityCheckKind::KeyExists => { + let key = required_key(check, "key_exists")?; + let passed = after_rollback.contains_key(key); + Ok(RollbackCheckResult { + category: RollbackCheckCategory::DataIntegrity, + name: format!("key_exists:{}", key), + passed, + message: if passed { + format!("key '{}' exists after rollback", key) + } else { + format!("key '{}' is missing after rollback", key) + }, + }) + } + IntegrityCheckKind::KeyAbsent => { + let key = required_key(check, "key_absent")?; + let passed = !after_rollback.contains_key(key); + Ok(RollbackCheckResult { + category: RollbackCheckCategory::DataIntegrity, + name: format!("key_absent:{}", key), + passed, + message: if passed { + format!("key '{}' is absent after rollback", key) + } else { + format!("key '{}' should be absent after rollback", key) + }, + }) + } + IntegrityCheckKind::Equals => { + let key = required_key(check, "equals")?; + let expected = check.value.as_ref().ok_or_else(|| { + anyhow::anyhow!("integrity check 'equals' for '{}' requires value", key) + })?; + let actual = after_rollback.get(key); + let passed = actual == Some(expected); + Ok(RollbackCheckResult { + category: RollbackCheckCategory::DataIntegrity, + name: format!("equals:{}", key), + passed, + message: if passed { + format!("key '{}' equals expected value", key) + } else { + format!( + "key '{}' mismatch (expected: {}, actual: {})", + key, + expected, + value_for_message(actual) + ) + }, + }) + } + IntegrityCheckKind::ChecksumUnchanged => { + let keys = check.keys.as_deref(); + let before = state_checksum(before_upgrade, keys)?; + let after = state_checksum(after_rollback, keys)?; + let passed = before == after; + Ok(RollbackCheckResult { + category: RollbackCheckCategory::DataIntegrity, + name: "checksum_unchanged".to_string(), + passed, + message: if passed { + "selected state checksum unchanged after rollback".to_string() + } else { + format!( + "selected state checksum changed (before: {}, after: {})", + before, after + ) + }, + }) + } + IntegrityCheckKind::NumericSumEquals => { + let keys = check.keys.as_ref().ok_or_else(|| { + anyhow::anyhow!("integrity check 'numeric_sum_equals' requires keys") + })?; + let expected_sum = check.expected_sum.ok_or_else(|| { + anyhow::anyhow!("integrity check 'numeric_sum_equals' requires expected_sum") + })?; + let mut actual_sum = 0i64; + let mut non_numeric = Vec::new(); + for key in keys { + match after_rollback.get(key).and_then(Value::as_i64) { + Some(value) => actual_sum += value, + None => non_numeric.push(key.clone()), + } + } + let passed = non_numeric.is_empty() && actual_sum == expected_sum; + Ok(RollbackCheckResult { + category: RollbackCheckCategory::DataIntegrity, + name: "numeric_sum_equals".to_string(), + passed, + message: if passed { + format!("numeric sum over selected keys equals {}", expected_sum) + } else if !non_numeric.is_empty() { + format!( + "non-numeric or missing keys in sum: {}", + non_numeric.join(", ") + ) + } else { + format!( + "numeric sum mismatch (expected: {}, actual: {})", + expected_sum, actual_sum + ) + }, + }) + } + IntegrityCheckKind::NoUnexpectedKeys => { + let allowed_keys = check.allowed_keys.as_ref().ok_or_else(|| { + anyhow::anyhow!("integrity check 'no_unexpected_keys' requires allowed_keys") + })?; + let allowed: BTreeSet<_> = allowed_keys.iter().cloned().collect(); + let unexpected: Vec<_> = after_rollback + .keys() + .filter(|key| !allowed.contains(*key)) + .cloned() + .collect(); + let passed = unexpected.is_empty(); + Ok(RollbackCheckResult { + category: RollbackCheckCategory::DataIntegrity, + name: "no_unexpected_keys".to_string(), + passed, + message: if passed { + "no unexpected keys remain after rollback".to_string() + } else { + format!( + "unexpected keys remain after rollback: {}", + unexpected.join(", ") + ) + }, + }) + } + } +} + +fn required_key<'a>(check: &'a IntegrityCheck, name: &str) -> Result<&'a str> { + check + .key + .as_deref() + .ok_or_else(|| anyhow::anyhow!("integrity check '{}' requires key", name)) +} + +fn state_checksum(state: &StateMap, keys: Option<&[String]>) -> Result { + let mut filtered = StateMap::new(); + match keys { + Some(keys) => { + for key in keys { + if let Some(value) = state.get(key) { + filtered.insert(key.clone(), value.clone()); + } + } + } + None => filtered = state.clone(), + } + + let canonical = serde_json::to_vec(&filtered)?; + Ok(sha256_hex(&canonical)) +} + +fn validate_wasm_file(path: &Path) -> Result> { + let bytes = fs::read(path).with_context(|| format!("Failed to read {}", path.display()))?; + if bytes.len() < 4 || &bytes[..4] != b"\0asm" { + anyhow::bail!( + "{} is not a valid WASM file (missing magic header)", + path.display() + ); + } + Ok(bytes) +} + +fn sha256_hex(bytes: &[u8]) -> String { + hex::encode(Sha256::digest(bytes)) +} + +fn write_rollback_report(report: &RollbackTestReport, format: &str) -> Result { + let dir = crate::utils::config::config_dir().join("reports"); + fs::create_dir_all(&dir)?; + let hash_prefix = &report.upgraded_wasm_hash[..12.min(report.upgraded_wasm_hash.len())]; + let path = dir.join(format!("rollback-test-{}.{}", hash_prefix, format)); + + match format { + "json" => fs::write(&path, serde_json::to_string_pretty(report)?)?, + "html" => fs::write(&path, render_html_report(report))?, + other => anyhow::bail!( + "Unsupported rollback report format '{}'. Use html or json.", + other + ), + } + + Ok(path) +} + +fn render_html_report(report: &RollbackTestReport) -> String { + let scenario_rows: String = report + .scenario_results + .iter() + .map(|scenario| { + let status = if scenario.passed { "PASS" } else { "FAIL" }; + let check_rows: String = scenario + .checks + .iter() + .map(|check| { + format!( + "
  • {} [{}] {}
    • ", + html_escape(&check.name), + if check.passed { "PASS" } else { "FAIL" }, + html_escape(&check.message) + ) + }) + .collect(); + format!( + "

    {} - {}

    {}

    Duration: {}ms

      {}
    ", + html_escape(&scenario.scenario_name), + status, + html_escape(&scenario.description), + scenario.duration_ms, + check_rows + ) + }) + .collect(); + + format!( + "Rollback Test Report

    Contract Rollback Test Report

    Previous WASM: {}

    Upgraded WASM: {}

    Scenarios: {} passed / {} failed

    {}", + report.previous_wasm_hash, + report.upgraded_wasm_hash, + report.passed, + report.failed, + scenario_rows + ) +} + +fn html_escape(input: &str) -> String { + input + .replace('&', "&") + .replace('<', "<") + .replace('>', ">") + .replace('"', """) + .replace('\'', "'") +} + +fn value_for_message(value: Option<&Value>) -> String { + value + .map(|value| value.to_string()) + .unwrap_or_else(|| "".to_string()) +} + +#[cfg(test)] +mod tests { + use super::*; + use serde_json::json; + + fn wasm_file(dir: &Path, name: &str, suffix: &[u8]) -> PathBuf { + let path = dir.join(name); + let mut bytes = b"\0asm\x01\0\0\0".to_vec(); + bytes.extend_from_slice(suffix); + fs::write(&path, bytes).unwrap(); + path + } + + #[test] + fn default_rollback_scenario_passes() { + let dir = tempfile::tempdir().unwrap(); + let previous = wasm_file(dir.path(), "previous.wasm", b"previous"); + let upgraded = wasm_file(dir.path(), "upgraded.wasm", b"upgraded"); + + let report = run_rollback_tests(RollbackTestOptions { + previous_wasm: previous, + upgraded_wasm: upgraded, + scenario_paths: vec![], + performance_budget_ms: 1000, + report_format: None, + }) + .unwrap(); + + assert_eq!(report.total_scenarios, 1); + assert_eq!(report.failed, 0); + assert!(report.scenario_results[0].checks.iter().any(|check| { + check.category == RollbackCheckCategory::StatePreservation && check.passed + })); + assert!(report.scenario_results[0] + .checks + .iter() + .any(|check| check.category == RollbackCheckCategory::Performance && check.passed)); + } + + #[test] + fn preserved_key_loss_fails_scenario() { + let scenario = RollbackScenario { + name: "data_loss".to_string(), + description: "detects missing balance after rollback".to_string(), + initial_state: BTreeMap::from([("balance:alice".to_string(), json!(100))]), + pre_upgrade_mutations: vec![], + upgrade_mutations: vec![StateMutation { + operation: MutationOperation::Delete, + key: "balance:alice".to_string(), + value: None, + }], + rollback_mutations: vec![], + preserved_keys: vec!["balance:alice".to_string()], + expected_after_rollback: BTreeMap::new(), + integrity_checks: vec![], + max_duration_ms: Some(1000), + }; + + let result = execute_scenario(&scenario, 1000).unwrap(); + assert!(!result.passed); + assert!(result.checks.iter().any(|check| { + check.category == RollbackCheckCategory::StatePreservation && !check.passed + })); + } +} diff --git a/tests/contract_rollback_testing.rs b/tests/contract_rollback_testing.rs new file mode 100644 index 00000000..40665bcc --- /dev/null +++ b/tests/contract_rollback_testing.rs @@ -0,0 +1,180 @@ +use serde_json::json; +use starforge::utils::rollback_testing::{ + run_rollback_tests, IntegrityCheck, IntegrityCheckKind, MutationOperation, + RollbackCheckCategory, RollbackScenario, RollbackTestOptions, StateMap, StateMutation, +}; +use std::fs; +use std::path::{Path, PathBuf}; + +fn wasm_file(dir: &Path, name: &str, suffix: &[u8]) -> PathBuf { + let path = dir.join(name); + let mut bytes = b"\0asm\x01\0\0\0".to_vec(); + bytes.extend_from_slice(suffix); + fs::write(&path, bytes).unwrap(); + path +} + +#[test] +fn rollback_harness_runs_default_state_integrity_and_performance_checks() { + let dir = tempfile::tempdir().unwrap(); + let previous = wasm_file(dir.path(), "v1.wasm", b"v1"); + let upgraded = wasm_file(dir.path(), "v2.wasm", b"v2"); + + let report = run_rollback_tests(RollbackTestOptions { + previous_wasm: previous, + upgraded_wasm: upgraded, + scenario_paths: vec![], + performance_budget_ms: 1000, + report_format: None, + }) + .unwrap(); + + assert_eq!(report.total_scenarios, 1); + assert_eq!(report.failed, 0); + let checks = &report.scenario_results[0].checks; + assert!(checks + .iter() + .any(|check| check.category == RollbackCheckCategory::WasmValidation && check.passed)); + assert!(checks + .iter() + .any(|check| check.category == RollbackCheckCategory::StatePreservation && check.passed)); + assert!(checks + .iter() + .any(|check| check.category == RollbackCheckCategory::DataIntegrity && check.passed)); + assert!(checks + .iter() + .any(|check| check.category == RollbackCheckCategory::Performance && check.passed)); +} + +#[test] +fn rollback_harness_fails_when_upgrade_loses_preserved_state() { + let dir = tempfile::tempdir().unwrap(); + let previous = wasm_file(dir.path(), "v1.wasm", b"v1"); + let upgraded = wasm_file(dir.path(), "v2.wasm", b"v2"); + + let scenario = RollbackScenario { + name: "lost_balance".to_string(), + description: "Detects deleted user balances during rollback.".to_string(), + initial_state: StateMap::from([("balance:alice".to_string(), json!(42))]), + pre_upgrade_mutations: vec![], + upgrade_mutations: vec![StateMutation { + operation: MutationOperation::Delete, + key: "balance:alice".to_string(), + value: None, + }], + rollback_mutations: vec![], + preserved_keys: vec!["balance:alice".to_string()], + expected_after_rollback: StateMap::new(), + integrity_checks: vec![IntegrityCheck { + kind: IntegrityCheckKind::KeyExists, + key: Some("balance:alice".to_string()), + value: None, + keys: None, + allowed_keys: None, + expected_sum: None, + }], + max_duration_ms: Some(1000), + }; + + let scenario_path = dir.path().join("scenario.json"); + fs::write( + &scenario_path, + serde_json::to_string_pretty(&scenario).unwrap(), + ) + .unwrap(); + + let report = run_rollback_tests(RollbackTestOptions { + previous_wasm: previous, + upgraded_wasm: upgraded, + scenario_paths: vec![scenario_path], + performance_budget_ms: 1000, + report_format: None, + }) + .unwrap(); + + assert_eq!(report.failed, 1); + assert!(!report.scenario_results[0].passed); + assert!(report.scenario_results[0].checks.iter().any(|check| { + check.category == RollbackCheckCategory::StatePreservation && !check.passed + })); +} + +#[test] +fn rollback_harness_loads_custom_scenario_array_and_validates_expected_state() { + let dir = tempfile::tempdir().unwrap(); + let previous = wasm_file(dir.path(), "v1.wasm", b"v1"); + let upgraded = wasm_file(dir.path(), "v2.wasm", b"v2"); + + let mut initial_state = StateMap::new(); + initial_state.insert("counter".to_string(), json!(7)); + initial_state.insert("owner".to_string(), json!("GALICE")); + + let mut expected_after_rollback = StateMap::new(); + expected_after_rollback.insert("counter".to_string(), json!(7)); + expected_after_rollback.insert("schema_version".to_string(), json!(1)); + + let scenarios = vec![RollbackScenario { + name: "counter_schema_rollback".to_string(), + description: "Rollback restores schema metadata while keeping user counter state." + .to_string(), + initial_state, + pre_upgrade_mutations: vec![], + upgrade_mutations: vec![ + StateMutation { + operation: MutationOperation::Increment, + key: "counter".to_string(), + value: Some(json!(5)), + }, + StateMutation { + operation: MutationOperation::Set, + key: "schema_version".to_string(), + value: Some(json!(2)), + }, + ], + rollback_mutations: vec![ + StateMutation { + operation: MutationOperation::Increment, + key: "counter".to_string(), + value: Some(json!(-5)), + }, + StateMutation { + operation: MutationOperation::Set, + key: "schema_version".to_string(), + value: Some(json!(1)), + }, + ], + preserved_keys: vec!["owner".to_string(), "counter".to_string()], + expected_after_rollback, + integrity_checks: vec![IntegrityCheck { + kind: IntegrityCheckKind::ChecksumUnchanged, + key: None, + value: None, + keys: Some(vec!["owner".to_string(), "counter".to_string()]), + allowed_keys: None, + expected_sum: None, + }], + max_duration_ms: Some(1000), + }]; + + let scenario_path = dir.path().join("scenarios.json"); + fs::write( + &scenario_path, + serde_json::to_string_pretty(&scenarios).unwrap(), + ) + .unwrap(); + + let report = run_rollback_tests(RollbackTestOptions { + previous_wasm: previous, + upgraded_wasm: upgraded, + scenario_paths: vec![scenario_path], + performance_budget_ms: 1000, + report_format: None, + }) + .unwrap(); + + assert_eq!(report.total_scenarios, 1); + assert_eq!(report.failed, 0); + assert!(report.scenario_results[0].checks.iter().any(|check| { + check.category == RollbackCheckCategory::ScenarioExpectation && check.passed + })); +}