diff --git a/doc/measuring-minimum-block-time.md b/doc/measuring-minimum-block-time.md index e6171dc0..f57c90fc 100644 --- a/doc/measuring-minimum-block-time.md +++ b/doc/measuring-minimum-block-time.md @@ -3,7 +3,7 @@ Supercluster provides two missions that measure the minimum ledger target close time (a.k.a. "block time") stellar-core can sustain at a fixed transaction rate while still meeting a latency SLA: * `MinBlockTimeClassic` measures the minimum block time using exclusively classic payments. -* `MinBlockTimeMixed` measures the minimum block time using a configurable mix of soroban invoke, soroban upload, and classic pay transactions. +* `MinBlockTimeMixed` measures the minimum block time using one explicit `MIXED_PREGEN_*` overlay-only loadgen mode: pre-generated classic payments plus the selected synthetic Soroban transaction type. Other than the type of load generated, these two missions are identical. They both spin up a configurable network of stellar-core nodes, then search for the smallest `ledgerTargetCloseTimeMilliseconds` value that the network can sustain, using a binary search over the range `[--min-block-time-ms, --max-block-time-ms]`. For each candidate close time `T`, the missions upgrade the network's SCP timing settings to `T` (with proportionally scaled ballot and nomination timeouts), run ~5 minutes of load at the fixed transaction rate, then check the `ledger.age.closed-histogram` metric on every node against the SLA. If the SLA is met, the missions try again with a smaller `T`; otherwise, they try with a larger `T`. @@ -36,26 +36,24 @@ Parameter settings also have a large impact on run time. Each iteration of the b These parameters affect both `MinBlockTimeClassic` and `MinBlockTimeMixed` missions: -* `--tx-rate`: The fixed transaction rate (TPS) used for every iteration of the search. The mission answers the question "what is the smallest block time the network can sustain at this TPS?" so choosing a TPS the network clearly cannot sustain (e.g., above the network's max TPS at default block time) will result in the mission failing with no block time satisfying the SLA. +* `--tx-rate`: The fixed transaction rate (TPS) used for every iteration of the search. For `MinBlockTimeMixed`, this is used only when neither `--classic-tx-rate` nor `--soroban-tx-rate` is set, in which case the mission splits it evenly between classic and Soroban streams. The mission answers the question "what is the smallest block time the network can sustain at this TPS?" so choosing a TPS the network clearly cannot sustain (e.g., above the network's max TPS at default block time) will result in the mission failing with no block time satisfying the SLA. * `--min-block-time-ms`: Binary search lower bound, in milliseconds. Defaults to `4000`. * `--max-block-time-ms`: Binary search upper bound, in milliseconds. Defaults to `5000`, which is also the protocol's maximum allowed ledger target close time — setting this higher will cause the mission to fail at startup, since validators reject upgrades above the protocol cap. Must be strictly greater than `--min-block-time-ms`. -* `--num-pregenerated-txs`: On small networks (≤30 nodes), the mission automatically switches classic payment load to `PayPregenerated` mode, which requires pregenerating signed transactions at each node. Defaults to `2500000`; lower this if you hit pod boot / liveness-probe issues while pregeneration runs. +* `--num-pregenerated-txs`: Number of pre-generated signed classic transactions to create per loadgen node. `MinBlockTimeClassic` uses these on small networks (≤30 nodes) when it automatically switches classic payment load to `PayPregenerated`; `MinBlockTimeMixed` always uses them for the classic stream in its `MIXED_PREGEN_*` mode. Defaults to `2500000` * `--pubnet-data`: Network topology to use. Defaults to a topology of tier 1 validators. See [Specifying network topologies](#specifying-network-topologies) for details on how to specify a custom topology. * `--netdelay-image`: Helper image providing simulated network delay for latency simulation. SDF provides a public image on dockerhub at `stellar/sdf-netdelay`. -### Additional options for mixed soroban and classic traffic +### Additional options for mixed pre-generated classic and synthetic Soroban traffic -In addition to the parameters in the previous section, `MinBlockTimeMixed` supports the following options to adjust the distribution of various transaction types: +In addition to the parameters in the previous section, `MinBlockTimeMixed` supports: -* `--pay-weight`: Weight of pay transactions. Defaults to 50. -* `--soroban-upload-weight`: Weight of soroban upload transactions. Defaults to 5. -* `--soroban-invoke-weight`: Weight of soroban invoke transactions. Defaults to 45. +* `--min-block-time-mixed-mode`: Exact stellar-core loadgen mode. Must be one of `mixed_pregen_sac_payment`, `mixed_pregen_oz_token_transfer`, or `mixed_pregen_soroswap_swap`. Defaults to `mixed_pregen_sac_payment`. +* `--classic-tx-rate`: Classic payment TPS for the pre-generated classic stream. +* `--soroban-tx-rate`: Soroban TPS for the selected synthetic Soroban stream. -That is, the default distribution produces 50% pay transactions, 5% soroban upload transactions, and 45% soroban invoke transactions. +If neither stream-specific TPS is set, `--tx-rate` is split evenly between classic and Soroban traffic. If either stream-specific TPS is set, any omitted stream defaults to `0`, and the fixed TPS for the mission is the sum of `--classic-tx-rate` and `--soroban-tx-rate`. -#### Specifying soroban transaction distributions - -As with `MaxTPSMixed`, `MinBlockTimeMixed` also allows customization of various low-level transaction details, and defaults to distributions based on real-world data. See the [Specifying soroban transaction distributions](measuring-transaction-throughput.md#specifying-soroban-transaction-distributions) section of the max TPS documentation for the available parameters (`--wasm-bytes` / `--wasm-bytes-weights`, `--data-entries` / `--data-entries-weights`, `--total-kilobytes` / `--total-kilobytes-weights`, `--tx-size-bytes` / `--tx-size-bytes-weights`, `--instructions` / `--instructions-weights`). +Before enabling overlay-only mode, the mission upgrades classic max tx set size from `--classic-tx-rate * 15`, and Soroban network limits from the selected transaction type's per-transaction resources multiplied by `--soroban-tx-rate * 15` (~15 seconds of throughput as leeway). ### More parameters @@ -90,3 +88,9 @@ To run a mission that searches for the minimum block time at 1000 TPS, with the ```bash dotnet run --project src/App/App.fsproj --configuration Release -- mission MinBlockTimeClassic --image=stellar/unsafe-stellar-core: --netdelay-image=stellar/sdf-netdelay:latest --tx-rate=1000 --min-block-time-ms=4000 --max-block-time-ms=5000 ``` + +To run the mixed overlay-only mode at 600 total TPS with Soroswap synthetic Soroban swaps: + +```bash +dotnet run --project src/App/App.fsproj --configuration Release -- mission MinBlockTimeMixed --image=stellar/unsafe-stellar-core: --netdelay-image=stellar/sdf-netdelay:latest --min-block-time-mixed-mode=mixed_pregen_soroswap_swap --classic-tx-rate=300 --soroban-tx-rate=300 --min-block-time-ms=4000 --max-block-time-ms=5000 +``` diff --git a/doc/missions.md b/doc/missions.md index f417d45d..a8b36f0d 100644 --- a/doc/missions.md +++ b/doc/missions.md @@ -157,7 +157,7 @@ Find the minimum ledger target close time a simulated Tier1 network can sustain ## MissionMinBlockTimeMixed -Same as `MissionMinBlockTimeClassic`, but drives a configurable mix of classic payments plus Soroban invoke and upload transactions. +Same as `MissionMinBlockTimeClassic`, but drives an explicit `MIXED_PREGEN_*` overlay-only loadgen mode with pre-generated classic payments plus a selected synthetic Soroban transaction type. ## MissionMixedNominationLeaderElectionWithOldMajority diff --git a/src/App/Program.fs b/src/App/Program.fs index ce4ce79b..495797f2 100644 --- a/src/App/Program.fs +++ b/src/App/Program.fs @@ -133,7 +133,10 @@ type MissionOptions benchmarkDurationSeconds: int, enableTcpTuning: bool, minBlockTimeMs: int, - maxBlockTimeMs: int + maxBlockTimeMs: int, + minBlockTimeMixedMode: string, + minBlockTimeMixedClassicTxRate: int option, + minBlockTimeMixedSorobanTxRate: int option ) = [] @@ -177,7 +180,7 @@ type MissionOptions [] member self.ExportToPrometheus : bool = exportToPrometheus - [] + [] member self.ProbeTimeout = probeTimeout [] @@ -611,6 +614,22 @@ type MissionOptions Default = 5000)>] member self.MaxBlockTimeMs = maxBlockTimeMs + [] + member self.MinBlockTimeMixedMode = minBlockTimeMixedMode + + [] + member self.MinBlockTimeMixedClassicTxRate = minBlockTimeMixedClassicTxRate + + [] + member self.MinBlockTimeMixedSorobanTxRate = minBlockTimeMixedSorobanTxRate + let splitLabel (lab: string) : (string * string option) = match lab.Split ':' |> Array.toList with | [ x ] -> x, None @@ -757,6 +776,9 @@ let main argv = enableTcpTuning = false minBlockTimeMs = 4000 maxBlockTimeMs = 5000 + minBlockTimeMixedMode = "mixed_pregen_sac_payment" + minBlockTimeMixedClassicTxRate = None + minBlockTimeMixedSorobanTxRate = None runForMinBlockTime = false } let nCfg = MakeNetworkCfg ctx [] None @@ -928,6 +950,9 @@ let main argv = enableTcpTuning = mission.EnableTcpTuning minBlockTimeMs = mission.MinBlockTimeMs maxBlockTimeMs = mission.MaxBlockTimeMs + minBlockTimeMixedMode = mission.MinBlockTimeMixedMode + minBlockTimeMixedClassicTxRate = mission.MinBlockTimeMixedClassicTxRate + minBlockTimeMixedSorobanTxRate = mission.MinBlockTimeMixedSorobanTxRate runForMinBlockTime = false } allMissions.[m] missionContext diff --git a/src/FSLibrary.Tests/Tests.fs b/src/FSLibrary.Tests/Tests.fs index 4c0d9581..f7ac2f64 100644 --- a/src/FSLibrary.Tests/Tests.fs +++ b/src/FSLibrary.Tests/Tests.fs @@ -140,6 +140,9 @@ let ctx : MissionContext = enableTcpTuning = false minBlockTimeMs = 4000 maxBlockTimeMs = 5000 + minBlockTimeMixedMode = "mixed_pregen_sac_payment" + minBlockTimeMixedClassicTxRate = None + minBlockTimeMixedSorobanTxRate = None runForMinBlockTime = false } let netdata = __SOURCE_DIRECTORY__ + "/../../../data/public-network-data-2024-08-01.json" diff --git a/src/FSLibrary/MinBlockTimeTest.fs b/src/FSLibrary/MinBlockTimeTest.fs index 2092c609..babb17fe 100644 --- a/src/FSLibrary/MinBlockTimeTest.fs +++ b/src/FSLibrary/MinBlockTimeTest.fs @@ -26,6 +26,151 @@ let private protocolMaxBlockTimeMs = 5000 let private timeoutsFor (targetMs: int) : int = max 500 (targetMs / 5) +let private mixedPregenLedgerMultiplier = 15 + +type private MixedPregenSorobanResources = + { instructions: int64 + readBytes: int + writeBytes: int + readOnlyEntries: int + readWriteEntries: int + txSizeBytes: int + contractEventBytes: int } + +let private usesPregeneratedTxs (mode: LoadGenMode) = mode = PayPregenerated || isMixedPregenMode mode + +let private mixedPregenSorobanResources (mode: LoadGenMode) = + match mode with + | MixedPregenSACPayment -> + { instructions = 250000L + readBytes = 800 + writeBytes = 800 + readOnlyEntries = 2 + readWriteEntries = 2 + txSizeBytes = 350 + contractEventBytes = 200 } + | MixedPregenOZTokenTransfer -> + { instructions = 5000000L + readBytes = 5000 + writeBytes = 5000 + readOnlyEntries = 2 + readWriteEntries = 2 + txSizeBytes = 1000 + contractEventBytes = 200 } + | MixedPregenSoroswapSwap -> + { instructions = 5000000L + readBytes = 5000 + writeBytes = 5000 + readOnlyEntries = 5 + readWriteEntries = 5 + txSizeBytes = 2000 + contractEventBytes = 200 } + | _ -> failwithf "Mode %s is not a MIXED_PREGEN_* mode" (mode.ToString()) + +let private scaleLedgerLimit (name: string) (value: int) (sorobanTxRate: int) = + let scaled = int64 value * int64 sorobanTxRate * int64 mixedPregenLedgerMultiplier + + if scaled > int64 System.Int32.MaxValue then + failwithf "Scaled %s limit %d exceeds supported int range" name scaled + + int scaled + +let private upgradeMixedPregenSorobanLimits + (formation: StellarFormation) + (coreSets: CoreSet list) + (baseLoadGen: LoadGen) + = + let sorobanTxRate = baseLoadGen.sorobanTxRate |> Option.defaultValue 0 + + if sorobanTxRate > 0 then + let resources = mixedPregenSorobanResources baseLoadGen.mode + let footprintEntries = resources.readOnlyEntries + resources.readWriteEntries + + LogInfo + "Upgrading MIXED_PREGEN_* Soroban limits for %s: soroban TPS=%d, ledger multiplier=%d" + (baseLoadGen.mode.ToString()) + sorobanTxRate + mixedPregenLedgerMultiplier + + formation.SetupUpgradeContract coreSets.Head + let peer = formation.NetworkCfg.GetPeer coreSets.Head 0 + + let scaledLedgerMaxTxCount = sorobanTxRate * mixedPregenLedgerMultiplier + + let ledgerMaxInstructions = + max (peer.GetLedgerMaxInstructions()) (resources.instructions * int64 scaledLedgerMaxTxCount) + + let ledgerMaxReadBytes = + max (peer.GetLedgerReadBytes()) (scaleLedgerLimit "ledger read bytes" resources.readBytes sorobanTxRate) + + let ledgerMaxWriteBytes = + max (peer.GetLedgerWriteBytes()) (scaleLedgerLimit "ledger write bytes" resources.writeBytes sorobanTxRate) + + let ledgerMaxReadEntries = + max (peer.GetLedgerReadEntries()) (scaleLedgerLimit "ledger read entries" footprintEntries sorobanTxRate) + + let ledgerMaxWriteEntries = + max + (peer.GetLedgerWriteEntries()) + (scaleLedgerLimit "ledger write entries" resources.readWriteEntries sorobanTxRate) + + let ledgerMaxTxCount = max (peer.GetLedgerMaxTxCount()) scaledLedgerMaxTxCount + + let ledgerMaxTransactionsSizeBytes = + max + (peer.GetLedgerMaxTransactionsSizeBytes()) + (scaleLedgerLimit "ledger tx bytes" resources.txSizeBytes sorobanTxRate) + + let txMaxInstructions = max (peer.GetTxMaxInstructions()) resources.instructions + let txMaxReadBytes = max (peer.GetTxReadBytes()) resources.readBytes + let txMaxWriteBytes = max (peer.GetTxWriteBytes()) resources.writeBytes + let txMaxReadEntries = max (peer.GetTxReadEntries()) footprintEntries + let txMaxWriteEntries = max (peer.GetTxWriteEntries()) resources.readWriteEntries + let txMaxFootprintSize = peer.GetTxMaxFootprintSize() |> Option.map (max footprintEntries) + let txMaxSizeBytes = max (peer.GetMaxTxSize()) resources.txSizeBytes + let txMaxContractEventsSizeBytes = max (peer.GetTxMaxContractEventsSize()) resources.contractEventBytes + + formation.DeployUpgradeEntriesAndArmAfter + coreSets + { LoadGen.GetDefault() with + mode = CreateSorobanUpgrade + ledgerMaxInstructions = Some ledgerMaxInstructions + ledgerMaxReadBytes = Some ledgerMaxReadBytes + ledgerMaxWriteBytes = Some ledgerMaxWriteBytes + ledgerMaxReadLedgerEntries = Some ledgerMaxReadEntries + ledgerMaxWriteLedgerEntries = Some ledgerMaxWriteEntries + ledgerMaxTxCount = Some ledgerMaxTxCount + ledgerMaxTransactionsSizeBytes = Some ledgerMaxTransactionsSizeBytes + txMaxInstructions = Some txMaxInstructions + txMaxReadBytes = Some txMaxReadBytes + txMaxWriteBytes = Some txMaxWriteBytes + txMaxReadLedgerEntries = Some txMaxReadEntries + txMaxWriteLedgerEntries = Some txMaxWriteEntries + txMaxFootprintSize = txMaxFootprintSize + txMaxSizeBytes = Some txMaxSizeBytes + txMaxContractEventsSizeBytes = Some txMaxContractEventsSizeBytes } + (System.TimeSpan.FromSeconds(20.0)) + + peer.WaitForLedgerMaxTxCount ledgerMaxTxCount + peer.WaitForTxMaxInstructions txMaxInstructions + +let private toggleOverlayOnlyMode (formation: StellarFormation) (coreSets: CoreSet list) = + formation.NetworkCfg.EachPeerInSets + (List.toArray coreSets) + (fun peer -> + let res = peer.ToggleOverlayOnlyMode() + LogInfo "Toggled overlay-only mode on %s: %s" peer.ShortName.StringName res) + +let private withOverlayOnlyMode (formation: StellarFormation) (coreSets: CoreSet list) (f: unit -> unit) = + LogInfo "Enabling overlay-only mode" + toggleOverlayOnlyMode formation coreSets + + try + f () + finally + LogInfo "Disabling overlay-only mode" + toggleOverlayOnlyMode formation coreSets + let private readLedgerAgePercentiles (peer: Peer) : float * float = let h = peer.GetMetrics().LedgerAgeClosedHistogram float h.``75``, float h.``99`` @@ -75,7 +220,6 @@ let minBlockTimeTest (context: MissionContext) (baseLoadGen: LoadGen) (setupCfg: // Mirrors MaxTPSTest: on small networks, GeneratePaymentLoad runs out of // source accounts at high TPS, so switch to PayPregenerated which uses // genesis-created accounts and pregenerated signed txs. - // TODO: use 4 tx profiles let baseLoadGen = if List.length allNodes <= 30 && baseLoadGen.mode = GeneratePaymentLoad then { baseLoadGen with mode = PayPregenerated } @@ -87,7 +231,7 @@ let minBlockTimeTest (context: MissionContext) (baseLoadGen: LoadGen) (setupCfg: runForMinBlockTime = true genesisTestAccountCount = Some(context.genesisTestAccountCount |> Option.defaultValue 100000) numPregeneratedTxs = - if baseLoadGen.mode = PayPregenerated then + if usesPregeneratedTxs baseLoadGen.mode then Some(context.numPregeneratedTxs |> Option.defaultValue 2500000) else None } @@ -98,20 +242,40 @@ let minBlockTimeTest (context: MissionContext) (baseLoadGen: LoadGen) (setupCfg: let isLoadGenNode cs = List.exists (fun (cs': CoreSet) -> cs' = cs) loadGenNodes - // For PayPregenerated, partition genesis accounts evenly across loadgen - // nodes and assign non-overlapping offsets so each node signs txs against - // its own slice. Verbatim port of MaxTPSTest's partitioning. + let activeLoadGenNodes = + if isMixedPregenMode baseLoadGen.mode then + let requestedCount = + max + (baseLoadGen.classicTxRate |> Option.defaultValue 0) + (baseLoadGen.sorobanTxRate |> Option.defaultValue 0) + + loadGenNodes + |> List.truncate (min (List.length loadGenNodes) (max 1 requestedCount)) + else + loadGenNodes + + // For pre-generated modes, partition genesis accounts evenly across + // loadgen nodes and assign offsets so each active node signs txs against + // its own slice. Mixed pregen keeps every tier1 core set initialized, but + // partitions accounts by the active loadgen count so low-TPS runs still + // have enough local accounts on the node that generates load. let allNodes = match context.numPregeneratedTxs, context.genesisTestAccountCount, baseLoadGen.mode with - | Some txs, Some accounts, PayPregenerated -> - let loadGenCount = List.length loadGenNodes - let accountsPerNode = accounts / loadGenCount + | Some txs, Some accounts, mode when usesPregeneratedTxs mode -> + let partitionCount = + if isMixedPregenMode mode then + List.length activeLoadGenNodes + else + List.length loadGenNodes + + let accountsPerNode = accounts / partitionCount let mutable j = 0 List.map (fun (cs: CoreSet) -> if isLoadGenNode cs then - let i = j + let i = if isMixedPregenMode mode then j % partitionCount else j + j <- j + 1 { cs with @@ -134,6 +298,12 @@ let minBlockTimeTest (context: MissionContext) (baseLoadGen: LoadGen) (setupCfg: let numAccounts = context.genesisTestAccountCount.Value let fixedTxRate = context.txRate + let classicTxRateForLimits = + if isMixedPregenMode baseLoadGen.mode then + baseLoadGen.classicTxRate |> Option.defaultValue 0 + else + fixedTxRate + // Headroom factor matches MaxTPSTest's `limitMultiplier = 5 * 2`: // 5x for ledgers-per-second at the default 5s close time, 2x for // spike margin. Applied at every (re)boot because pod restart @@ -145,9 +315,11 @@ let minBlockTimeTest (context: MissionContext) (baseLoadGen: LoadGen) (setupCfg: formation.ManualClose coreSets formation.WaitUntilSynced coreSets formation.UpgradeProtocolToLatest coreSets - formation.UpgradeMaxTxSetSize coreSets (fixedTxRate * 10) + formation.UpgradeMaxTxSetSize coreSets (classicTxRateForLimits * 10) - if not isPaymentOnly then + if isMixedPregenMode baseLoadGen.mode then + upgradeMixedPregenSorobanLimits formation coreSets baseLoadGen + elif not isPaymentOnly then MaxTPSTest.upgradeSorobanLedgerLimits context formation coreSets fixedTxRate MaxTPSTest.upgradeSorobanTxLimits context formation coreSets @@ -198,7 +370,15 @@ let minBlockTimeTest (context: MissionContext) (baseLoadGen: LoadGen) (setupCfg: try applySCPUpgrade targetMs formation.clearMetrics allNodes - formation.RunMultiLoadgen loadGenNodes loadGen + + if isMixedPregenMode baseLoadGen.mode then + withOverlayOnlyMode + formation + allNodes + (fun () -> formation.RunMultiLoadgen activeLoadGenNodes loadGen) + else + formation.RunMultiLoadgen activeLoadGenNodes loadGen + formation.CheckNoErrorsAndPairwiseConsistency() formation.EnsureAllNodesInSync allNodes checkLedgerAgeSLA formation allNodes targetMs @@ -224,12 +404,12 @@ let minBlockTimeTest (context: MissionContext) (baseLoadGen: LoadGen) (setupCfg: LogInfo "Starting min block time search: T in [%d, %d] ms, fixed TPS = %d" lo hi fixedTxRate - // Restart-or-sleep between iterations. PayPregenerated requires a - // full restart because the pregenerated txs have baked-in sequence - // numbers that become stale after a partial iteration. Other modes - // just need time for the tx queue to drain. Ported from MaxTPSTest. + // Restart-or-sleep between iterations. Pre-generated modes require + // a full restart because the pregenerated txs have baked-in + // sequence numbers that become stale after a partial iteration. + // Other modes just need time for the tx queue to drain. let restartCoreSetsOrWait () = - if baseLoadGen.mode = PayPregenerated then + if usesPregeneratedTxs baseLoadGen.mode then LogInfo "Restarting all nodes to refresh pregenerated txs" allNodes diff --git a/src/FSLibrary/MissionMinBlockTimeMixed.fs b/src/FSLibrary/MissionMinBlockTimeMixed.fs index deceb377..3a7034b1 100644 --- a/src/FSLibrary/MissionMinBlockTimeMixed.fs +++ b/src/FSLibrary/MissionMinBlockTimeMixed.fs @@ -4,45 +4,47 @@ module MissionMinBlockTimeMixed -// Mirror of MissionMaxTPSMixed for the minimum-block-time search: mixed -// classic + Soroban workload at a fixed TPS, binary-searches for the smallest -// ledger target close time that still keeps ledger.age.closed-histogram within SLA. +// Minimum-block-time search for the new overlay-only mixed-pregen loadgen modes: +// pre-generated classic payments plus one explicit synthetic Soroban tx type. open MinBlockTimeTest -open PubnetData open StellarMissionContext open StellarCoreHTTP let minBlockTimeMixed (baseContext: MissionContext) = + let mode = parseMixedPregenMode baseContext.minBlockTimeMixedMode + + let classicTxRate, sorobanTxRate = + match baseContext.minBlockTimeMixedClassicTxRate, baseContext.minBlockTimeMixedSorobanTxRate with + | None, None -> + let soroban = baseContext.txRate / 2 + baseContext.txRate - soroban, soroban + | Some classic, None -> classic, 0 + | None, Some soroban -> 0, soroban + | Some classic, Some soroban -> classic, soroban + + if classicTxRate < 0 || sorobanTxRate < 0 then + failwith "--classic-tx-rate and --soroban-tx-rate must be non-negative" + + if classicTxRate = 0 && sorobanTxRate = 0 then + failwith "At least one of --classic-tx-rate or --soroban-tx-rate must be non-zero" + let context = { baseContext with coreResources = SimulatePubnetTier1PerfResources installNetworkDelay = Some(baseContext.installNetworkDelay |> Option.defaultValue true) enableTailLogging = false - wasmBytesDistribution = defaultListValue pubnetWasmBytes baseContext.wasmBytesDistribution - dataEntriesDistribution = defaultListValue pubnetDataEntries baseContext.dataEntriesDistribution - totalKiloBytesDistribution = defaultListValue pubnetTotalKiloBytes baseContext.totalKiloBytesDistribution - txSizeBytesDistribution = defaultListValue pubnetTxSizeBytes baseContext.txSizeBytesDistribution - instructionsDistribution = defaultListValue pubnetInstructions baseContext.instructionsDistribution } + txRate = classicTxRate + sorobanTxRate } let baseLoadGen = { LoadGen.GetDefault() with - mode = MixedClassicSoroban + mode = mode spikesize = context.spikeSize spikeinterval = context.spikeInterval offset = 0 maxfeerate = None skiplowfeetxs = false + classicTxRate = Some classicTxRate + sorobanTxRate = Some sorobanTxRate } - wasms = context.numWasms - instances = context.numInstances - - payWeight = Some(baseContext.payWeight |> Option.defaultValue 50) - sorobanUploadWeight = Some(baseContext.sorobanUploadWeight |> Option.defaultValue 5) - sorobanInvokeWeight = Some(baseContext.sorobanInvokeWeight |> Option.defaultValue 45) - - minSorobanPercentSuccess = Some(baseContext.minSorobanPercentSuccess |> Option.defaultValue 60) } - - let invokeSetupCfg = { baseLoadGen with mode = SorobanInvokeSetup } - - minBlockTimeTest context baseLoadGen (Some invokeSetupCfg) + minBlockTimeTest context baseLoadGen None diff --git a/src/FSLibrary/StellarCoreHTTP.fs b/src/FSLibrary/StellarCoreHTTP.fs index 1b2893aa..66d58fae 100644 --- a/src/FSLibrary/StellarCoreHTTP.fs +++ b/src/FSLibrary/StellarCoreHTTP.fs @@ -50,6 +50,9 @@ type LoadGenMode = | MixedClassicSoroban | StopRun | PayPregenerated + | MixedPregenSACPayment + | MixedPregenOZTokenTransfer + | MixedPregenSoroswapSwap override self.ToString() = match self with @@ -62,6 +65,26 @@ type LoadGenMode = | MixedClassicSoroban -> "mixed_classic_soroban" | StopRun -> "stop" | PayPregenerated -> "pay_pregenerated" + | MixedPregenSACPayment -> "mixed_pregen_sac_payment" + | MixedPregenOZTokenTransfer -> "mixed_pregen_oz_token_transfer" + | MixedPregenSoroswapSwap -> "mixed_pregen_soroswap_swap" + +let isMixedPregenMode (mode: LoadGenMode) = + match mode with + | MixedPregenSACPayment + | MixedPregenOZTokenTransfer + | MixedPregenSoroswapSwap -> true + | _ -> false + +let parseMixedPregenMode (mode: string) = + match mode.Trim().ToLowerInvariant() with + | "mixed_pregen_sac_payment" -> MixedPregenSACPayment + | "mixed_pregen_oz_token_transfer" -> MixedPregenOZTokenTransfer + | "mixed_pregen_soroswap_swap" -> MixedPregenSoroswapSwap + | _ -> + failwithf + "Unsupported mixed pregen mode '%s'. Expected one of: mixed_pregen_sac_payment, mixed_pregen_oz_token_transfer, mixed_pregen_soroswap_swap" + mode type LoadGen = { mode: LoadGenMode @@ -108,6 +131,10 @@ type LoadGen = sorobanUploadWeight: int option sorobanInvokeWeight: int option + // Fields for MIXED_PREGEN_* modes + classicTxRate: int option + sorobanTxRate: int option + // Fields for SCP timing configuration ledgerTargetCloseTimeMilliseconds: int option ballotTimeoutIncrementMilliseconds: int option @@ -133,57 +160,43 @@ type LoadGen = | None -> [] let optionalParams = - optionalParam "maxfeerate" self.maxfeerate - @ optionalParam "wasms" self.wasms - @ optionalParam "instances" self.instances - @ optionalParam "minpercentsuccess" self.minSorobanPercentSuccess - @ optionalParam "mxcntrctsz" self.maxContractSizeBytes - @ optionalParam "mxcntrctkeysz" self.maxContractDataKeySizeBytes - @ optionalParam "mxcntrctdatasz" self.maxContractDataEntrySizeBytes - @ optionalParam "ldgrmxinstrc" self.ledgerMaxInstructions - @ optionalParam "txmxinstrc" self.txMaxInstructions - @ optionalParam "txmemlim" self.txMemoryLimit - @ optionalParam "ldgrmxrdntry" self.ledgerMaxReadLedgerEntries - @ optionalParam "ldgrmxrdbyt" self.ledgerMaxReadBytes - @ optionalParam "ldgrmxwrntry" self.ledgerMaxWriteLedgerEntries - @ optionalParam "ldgrmxwrbyt" self.ledgerMaxWriteBytes - @ optionalParam "ldgrmxtxcnt" self.ledgerMaxTxCount - @ optionalParam "txmxrdntry" self.txMaxReadLedgerEntries - @ optionalParam "txmxrdbyt" self.txMaxReadBytes - @ optionalParam "txmxwrntry" self.txMaxWriteLedgerEntries - @ optionalParam "txmxwrbyt" self.txMaxWriteBytes - @ optionalParam "txmxevntsz" self.txMaxContractEventsSizeBytes - @ optionalParam "ldgrmxtxsz" self.ledgerMaxTransactionsSizeBytes - @ optionalParam "txmxsz" self.txMaxSizeBytes - @ optionalParam "wndowsz" self.bucketListSizeWindowSampleSize - @ optionalParam "evctsz" self.evictionScanSize - @ optionalParam "evctlvl" self.startingEvictionScanLevel - @ optionalParam "payweight" self.payWeight - @ optionalParam - "sorobanuploadweight" - self.sorobanUploadWeight - @ optionalParam - "sorobaninvokeweight" - self.sorobanInvokeWeight - @ optionalParam "txmxftprnt" self.txMaxFootprintSize - @ optionalParam - "ldgrclse" - self.ledgerTargetCloseTimeMilliseconds - @ optionalParam - "balinc" - self.ballotTimeoutIncrementMilliseconds - @ optionalParam - "balinit" - self.ballotTimeoutInitialMilliseconds - @ optionalParam - "nominit" - self.nominationTimeoutInitialMilliseconds - @ optionalParam - "nominc" - self.nominationTimeoutIncrementMilliseconds - @ optionalParam - "maxtxclstrs" - self.ledgerMaxDependentTxClusters + List.concat [ optionalParam "maxfeerate" self.maxfeerate + optionalParam "wasms" self.wasms + optionalParam "instances" self.instances + optionalParam "minpercentsuccess" self.minSorobanPercentSuccess + optionalParam "mxcntrctsz" self.maxContractSizeBytes + optionalParam "mxcntrctkeysz" self.maxContractDataKeySizeBytes + optionalParam "mxcntrctdatasz" self.maxContractDataEntrySizeBytes + optionalParam "ldgrmxinstrc" self.ledgerMaxInstructions + optionalParam "txmxinstrc" self.txMaxInstructions + optionalParam "txmemlim" self.txMemoryLimit + optionalParam "ldgrmxrdntry" self.ledgerMaxReadLedgerEntries + optionalParam "ldgrmxrdbyt" self.ledgerMaxReadBytes + optionalParam "ldgrmxwrntry" self.ledgerMaxWriteLedgerEntries + optionalParam "ldgrmxwrbyt" self.ledgerMaxWriteBytes + optionalParam "ldgrmxtxcnt" self.ledgerMaxTxCount + optionalParam "txmxrdntry" self.txMaxReadLedgerEntries + optionalParam "txmxrdbyt" self.txMaxReadBytes + optionalParam "txmxwrntry" self.txMaxWriteLedgerEntries + optionalParam "txmxwrbyt" self.txMaxWriteBytes + optionalParam "txmxevntsz" self.txMaxContractEventsSizeBytes + optionalParam "ldgrmxtxsz" self.ledgerMaxTransactionsSizeBytes + optionalParam "txmxsz" self.txMaxSizeBytes + optionalParam "wndowsz" self.bucketListSizeWindowSampleSize + optionalParam "evctsz" self.evictionScanSize + optionalParam "evctlvl" self.startingEvictionScanLevel + optionalParam "payweight" self.payWeight + optionalParam "sorobanuploadweight" self.sorobanUploadWeight + optionalParam "sorobaninvokeweight" self.sorobanInvokeWeight + optionalParam "txmxftprnt" self.txMaxFootprintSize + optionalParam "ldgrclse" self.ledgerTargetCloseTimeMilliseconds + optionalParam "balinc" self.ballotTimeoutIncrementMilliseconds + optionalParam "balinit" self.ballotTimeoutInitialMilliseconds + optionalParam "nominit" self.nominationTimeoutInitialMilliseconds + optionalParam "nominc" self.nominationTimeoutIncrementMilliseconds + optionalParam "maxtxclstrs" self.ledgerMaxDependentTxClusters + optionalParam "classictxrate" self.classicTxRate + optionalParam "sorobantxrate" self.sorobanTxRate ] mandatoryParams @ optionalParams @@ -225,6 +238,8 @@ type LoadGen = payWeight = None sorobanUploadWeight = None sorobanInvokeWeight = None + classicTxRate = None + sorobanTxRate = None ledgerTargetCloseTimeMilliseconds = None ballotTimeoutIncrementMilliseconds = None ballotTimeoutInitialMilliseconds = None @@ -674,6 +689,12 @@ type Peer with (fun _ -> Http.RequestString(httpMethod = "GET", headers = self.Headers, url = self.URL "clearmetrics")) |> ignore + member self.ToggleOverlayOnlyMode() = + WebExceptionRetry + DefaultRetry + (fun _ -> + Http.RequestString(httpMethod = "GET", headers = self.Headers, url = self.URL "toggleoverlayonlymode")) + member self.GetTestAcc(accName: string) : TestAcc.Root = // NB: work around buggy JSON parser upstream, see // https://github.com/fsharp/FSharp.Data/pull/1262 diff --git a/src/FSLibrary/StellarMissionContext.fs b/src/FSLibrary/StellarMissionContext.fs index 0bb339ae..c64b4cca 100644 --- a/src/FSLibrary/StellarMissionContext.fs +++ b/src/FSLibrary/StellarMissionContext.fs @@ -135,4 +135,7 @@ type MissionContext = enableTcpTuning: bool minBlockTimeMs: int maxBlockTimeMs: int + minBlockTimeMixedMode: string + minBlockTimeMixedClassicTxRate: int option + minBlockTimeMixedSorobanTxRate: int option runForMinBlockTime: bool } diff --git a/src/FSLibrary/StellarStatefulSets.fs b/src/FSLibrary/StellarStatefulSets.fs index 79362114..2e7a5e38 100644 --- a/src/FSLibrary/StellarStatefulSets.fs +++ b/src/FSLibrary/StellarStatefulSets.fs @@ -380,6 +380,26 @@ type StellarFormation with (List.toArray coreSetList) (fun peer -> peer.UpgradeNetworkSetting contractKey upgradeTime) + member self.DeployUpgradeEntriesAndArmAfter + (coreSetList: CoreSet list) + (loadGen: LoadGen) + (delay: System.TimeSpan) + = + let peer = self.NetworkCfg.GetPeer coreSetList.[0] 0 + let resStr = peer.GenerateLoad loadGen + + let contractKey = Loadgen.Parse(resStr).ConfigUpgradeSetKey + + LogInfo "Loadgen: %s" resStr + peer.WaitForLoadGenComplete loadGen + + let upgradeTime = System.DateTime.UtcNow.Add(delay) + + // Arm upgrades on each peer in the core set + self.NetworkCfg.EachPeerInSets + (List.toArray coreSetList) + (fun peer -> peer.UpgradeNetworkSetting contractKey upgradeTime) + member self.clearMetrics(coreSets: CoreSet list) = self.NetworkCfg.EachPeerInSets(coreSets |> List.toArray) (fun peer -> peer.ClearMetrics()) @@ -390,39 +410,60 @@ type StellarFormation with let fractionalLoadGen (i: int) : LoadGen = let getFraction attr = if i + 1 = n then (attr / n + attr % n) else attr / n + let getOptionalFraction attr = Option.map getFraction attr { fullLoadGen with accounts = fullLoadGen.accounts / n txs = fullLoadGen.txs / n spikesize = getFraction fullLoadGen.spikesize - txrate = getFraction fullLoadGen.txrate } + txrate = getFraction fullLoadGen.txrate + classicTxRate = getOptionalFraction fullLoadGen.classicTxRate + sorobanTxRate = getOptionalFraction fullLoadGen.sorobanTxRate } + + let hasNonZeroRate (loadGen: LoadGen) = + match loadGen.classicTxRate, loadGen.sorobanTxRate with + | Some classicRate, Some sorobanRate -> classicRate <> 0 || sorobanRate <> 0 + | Some classicRate, None -> classicRate <> 0 + | None, Some sorobanRate -> sorobanRate <> 0 + | None, None -> true let loadGenPeers = List.map (fun cs -> self.NetworkCfg.GetPeer cs 0) coreSets - for (i, peer) in (List.indexed loadGenPeers) do - let loadGen = fractionalLoadGen i - let offset = loadGen.accounts * i - let peerSpecificLoadgen = { loadGen with offset = offset } + let peerLoadGens = + loadGenPeers + |> List.indexed + |> List.map + (fun (i, peer) -> + let loadGen = fractionalLoadGen i + let offset = loadGen.accounts * i + (peer, { loadGen with offset = offset }, offset)) + |> List.filter (fun (_, loadGen, _) -> hasNonZeroRate loadGen) + + if List.isEmpty peerLoadGens then + failwith "Loadgen failed: no peer has a non-zero tx rate" + + for (peer, peerSpecificLoadgen, offset) in peerLoadGens do LogInfo "Loadgen: %s with offset %d" (peer.GenerateLoad peerSpecificLoadgen) offset while List.exists - (fun (peer: Peer) -> not (peer.IsLoadGenComplete() = Success || peer.IsLoadGenComplete() = Failure)) - loadGenPeers do + (fun (peer: Peer, _, _) -> + not (peer.IsLoadGenComplete() = Success || peer.IsLoadGenComplete() = Failure)) + peerLoadGens do Thread.Sleep(millisecondsTimeout = 3000) - for (i, peer) in (List.indexed loadGenPeers) do - peer.LogLoadGenProgressTowards(fractionalLoadGen i) + for (peer, loadGen, _) in peerLoadGens do + peer.LogLoadGenProgressTowards(loadGen) // Check if any loadGen has failed - if List.exists (fun (peer: Peer) -> peer.IsLoadGenComplete() = Failure) loadGenPeers then + if List.exists (fun (peer: Peer, _, _) -> peer.IsLoadGenComplete() = Failure) peerLoadGens then // Stop all runs - for peer in loadGenPeers do + for (peer, _, _) in peerLoadGens do LogInfo "%s loadgen: %s" (peer.ShortName.ToString()) (peer.StopLoadGen()) failwith "Loadgen failed!" // Final check after the loop completes - if List.exists (fun (peer: Peer) -> peer.IsLoadGenComplete() = Failure) loadGenPeers then + if List.exists (fun (peer: Peer, _, _) -> peer.IsLoadGenComplete() = Failure) peerLoadGens then failwith "Loadgen failed!" // Deploys TCP tuning DaemonSets to configure node-level network settings if enabled.