fsm-benchmarks

This repo contains benchmarks for C++ state machine libraries.

It has been originally created to test various optimizations of the Boost MSM library.

Benchmarks

Compile resources are measured with the command time -f "%e seconds, %M kB RAM". RAM usage is the peak RAM reported by time (in KiB), converted to MB by dividing by 1000. Binary size corresponds to the sum of the text and data segments reported by the size command (Berkeley format), divided by 1000.

Header inclusion:

No instantiation of a state machine, only the headers of the library are included.

Large state machine:

A state machine gets constructed 1,000 times and processes events for 1,000*1,000 iterations. The events trigger transitions and internal transitions in alternation. In total it processes:

• 25,000,000 transitions (25 states with 1 transition event each)

• 25,000,000 internal transitions (all states share a common internal transition event)

• 5,000,000 internal transitions from the event queue/pool (5/25 transition events trigger a process_event while the SM is processing)

• 10,000,000 entry actions (10/25 states have entry actions)

Large hierarchical state machine:

Extension of the large state machine benchmark with a hierarchical structure of 3 state machines:

• fsm0 is the root state machine

• fsm1 is contained in fsm0

• fsm2 is contained in fsm1

One iteration in this benchmark runs as follows:

• process events

• enter fsm1 and process events

• enter fsm2 and process events

• exit fsm2 and exit fsm1

The events handled by each state machine overlap with each other to simulate transition conflicts. Each state machine processes a similar amount of events as in the large state machine benchmark.

Tested libraries

The following libraries are benchmarked:

MSM:

MSM is used with 3 back-ends back, back11 and backmp11. The back-end backmp11 is a recent addition. For more details about the applied optimizations and changes to previous back-ends take a look at the MSM documentation.

Only the functor front-end is used in the benchmarks. The following back-end configurations are tested:

• back back-end with favor_runtime_speed and favor_compile_time compile policies

• back11 back-end

• backmp11 back-end with favor_runtime_speed and favor_compile_time compile policies

Last run was done on commit d0cf98bd39aaf529005f5dff83dbfc0e26a5a468.

SML:

SML is tested with a default configuration.

Last run was done on commit d0b0df7d39d05c6d31db3aebe64990e00cc08bd2.

Results

System used for running the benchmarks:

• Intel i7-6700K @4GHZ, 32 GB RAM

• Clang 19.1.4, Release build, C++20

• CUs are compiled one-by-one except the multi_cu benchmark for MSM with backmp11 & favor_compile_time

Header inclusion

	Compile / sec	RAM / MB
back	2.3	224
back_favor_compile_time	2.3	224
back11	2.3	224
backmp11	0.6	126
backmp11_favor_compile_time	0.6	130
sml	0.1	95

Large state machine

	Compile time / sec	Compile RAM / MB	Binary size / kB	Runtime / sec
back	14	815	68	2.8
back_favor_compile_time	17	775	226	3.5
back11	37	2682	84	2.8
backmp11	3	209	29	0.7
backmp11_favor_compile_time	3	193	43	6.0
sml	5	234	57	0.3

Large hierarchical state machine

	Compile time / sec	Compile RAM / MB	Binary size / kB	Runtime / sec
back	49	2165	230	13.2
back_favor_compile_time	55	1704	911	> 300
backmp11	8	351	85	3.3
backmp11_favor_compile_time	5	256	100	20.4
backmp11_favor_compile_time_multi_cu	5	~863	100	20.8
sml	18	543	422	5.4