Skip to content

Run build-test coverage on Windows#1051

Merged
timsehn merged 8 commits into
masterfrom
windows-build-test-coverage
May 27, 2026
Merged

Run build-test coverage on Windows#1051
timsehn merged 8 commits into
masterfrom
windows-build-test-coverage

Conversation

@timsehn
Copy link
Copy Markdown
Collaborator

@timsehn timsehn commented May 27, 2026

Summary

  • run Build-Test shell/C/testfixture coverage on the Windows matrix leg
  • build Windows test dependencies: sqlite3, testfixture, and Python for analyzer-backed tests
  • keep Linux-only Python LD_PRELOAD and Go integration steps on Ubuntu only

Validation

  • ruby YAML parse for .github/workflows/build-test.yml
  • git diff --check for .github/workflows/build-test.yml

@timsehn timsehn force-pushed the windows-build-test-coverage branch 3 times, most recently from b130c0d to 4d7e848 Compare May 27, 2026 18:19
@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 27, 2026

Sysbench-Style Benchmark: Doltlite vs SQLite

In-Memory

Reads

Test SQLite (us) Doltlite (us) Multiplier
oltp_point_select 24,435 27,937 1.14
oltp_range_select 10,492 11,172 1.06
oltp_sum_range 9,694 10,758 1.11
oltp_order_range 2,700 2,887 1.07
oltp_distinct_range 3,796 3,866 1.02
oltp_index_scan 3,989 5,047 1.27
select_random_points 10,433 11,377 1.09
select_random_ranges 3,011 4,135 1.37
covering_index_scan 4,255 4,230 0.99
groupby_scan 32,810 32,683 1.00
index_join 5,898 7,705 1.31
index_join_scan 3,416 4,782 1.40
types_table_scan 1,108,634 1,183,853 1.07
table_scan 1,294,420 1,321,996 1.02
oltp_read_only 107,529 113,043 1.05
Average 1.13

Writes

Test SQLite (us) Doltlite (us) Multiplier
oltp_bulk_insert 180,221 242,359 1.34
oltp_insert 16,068 27,797 1.73
oltp_update_index 51,965 88,534 1.70
oltp_update_non_index 35,696 60,472 1.69
oltp_delete_insert 46,118 70,135 1.52
oltp_write_only 22,507 44,288 1.97
types_delete_insert 25,831 39,825 1.54
oltp_read_write 66,191 101,146 1.53
Average 1.63

File-Backed

Reads

Test SQLite (us) Doltlite (us) Multiplier
oltp_point_select 117,050 57,067 0.49
oltp_range_select 21,130 14,529 0.69
oltp_sum_range 20,446 13,918 0.68
oltp_order_range 3,723 3,327 0.89
oltp_distinct_range 4,844 4,243 0.88
oltp_index_scan 13,523 8,264 0.61
select_random_points 27,360 14,594 0.53
select_random_ranges 12,304 7,169 0.58
covering_index_scan 12,582 7,457 0.59
groupby_scan 35,928 33,123 0.92
index_join 10,763 9,916 0.92
index_join_scan 4,662 5,429 1.16
types_table_scan 1,283,697 1,185,110 0.92
table_scan 1,526,745 1,313,765 0.86
oltp_read_only 241,582 153,951 0.64
Average 0.76

Writes

Test SQLite (us) Doltlite (us) Multiplier
oltp_bulk_insert 188,583 260,867 1.38
oltp_insert 21,466 34,830 1.62
oltp_update_index 128,427 115,129 0.90
oltp_update_non_index 82,115 80,330 0.98
oltp_delete_insert 93,465 91,265 0.98
oltp_write_only 55,398 65,248 1.18
types_delete_insert 50,061 52,492 1.05
oltp_read_write 120,269 120,882 1.01
Average 1.14

File-Backed (autocommit)

Each statement runs as its own transaction — exposes per-commit
fixed costs that the wrapped-in-BEGIN/COMMIT tests amortize away.
SQLite uses WAL mode with synchronous=FULL in this section so
the comparison uses SQLite's durable WAL autocommit path.

Reads

Reads have no commit cost; these are the same SQL files as the
File-Backed Reads section, included here for symmetry and to
catch any per-statement overhead doltlite pays on the read path.

Test SQLite (us) Doltlite (us) Multiplier
oltp_point_select 61,024 56,389 0.92
oltp_range_select 15,629 14,605 0.93
oltp_sum_range 15,053 14,056 0.93
oltp_order_range 3,206 3,299 1.03
oltp_distinct_range 4,277 4,254 0.99
oltp_index_scan 8,241 8,378 1.02
select_random_points 21,445 14,558 0.68
select_random_ranges 6,928 7,111 1.03
covering_index_scan 7,438 7,573 1.02
groupby_scan 35,485 32,992 0.93
index_join 8,192 10,146 1.24
index_join_scan 4,061 5,401 1.33
types_table_scan 1,290,539 1,184,422 0.92
table_scan 1,526,360 1,323,084 0.87
oltp_read_only 164,612 155,082 0.94
Average 0.99

Writes

Test SQLite (us) Doltlite (us) Multiplier
oltp_bulk_insert_ac 16,112 62,548 3.88
oltp_insert_ac 18,597 82,187 4.42
oltp_update_index_ac 20,255 95,804 4.73
oltp_update_non_index_ac 16,972 70,481 4.15
oltp_delete_insert_ac 18,336 87,071 4.75
oltp_write_only_ac 18,889 82,489 4.37
types_delete_insert_ac 18,275 72,858 3.99
oltp_read_write_ac 25,911 98,332 3.79
Average 4.26

100000 rows, median of 5 invocations per test, workload-only timing via host monotonic clock when available.

Performance Ceiling Check (2.5x individual, 2x average; autocommit writes: 5x / 5x)

All tests within ceilings.

@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 27, 2026

Sysbench-Style Benchmark (BLOB PK): Doltlite vs SQLite

Companion to the classic Sysbench-Style Benchmark. Every workload here
runs against tables with a 16-byte big-endian BLOB PRIMARY KEY.

Individual ratios gated at 2.5×; section averages gated at 2×. Autocommit writes use 5× / 5×.

In-Memory

Reads

Test SQLite (us) Doltlite (us) Multiplier
oltp_point_select 30,916 34,992 1.13
oltp_range_select 14,102 15,638 1.11
oltp_sum_range 12,397 14,820 1.20
oltp_order_range 3,192 3,359 1.05
oltp_distinct_range 4,303 4,445 1.03
oltp_index_scan 4,626 5,999 1.30
select_random_points 18,192 19,904 1.09
select_random_ranges 4,290 5,314 1.24
covering_index_scan 4,702 4,822 1.03
groupby_scan 35,361 37,566 1.06
index_join 6,838 9,703 1.42
index_join_scan 4,274 5,900 1.38
types_table_scan 1,231,303 1,344,574 1.09
table_scan 1,565,763 1,490,249 0.95
oltp_read_only 130,857 138,846 1.06
Average 1.14

Writes

Test SQLite (us) Doltlite (us) Multiplier
oltp_bulk_insert 242,710 332,860 1.37
oltp_insert 20,991 39,194 1.87
oltp_update_index 73,267 135,804 1.85
oltp_update_non_index 51,907 87,503 1.69
oltp_delete_insert 52,700 108,540 2.06
oltp_write_only 30,286 65,773 2.17
types_delete_insert 34,263 55,146 1.61
oltp_read_write 89,387 142,201 1.59
Average 1.78

File-Backed

Reads

Test SQLite (us) Doltlite (us) Multiplier
oltp_point_select 124,175 64,669 0.52
oltp_range_select 24,655 18,993 0.77
oltp_sum_range 24,039 18,460 0.77
oltp_order_range 4,234 3,734 0.88
oltp_distinct_range 5,258 4,812 0.92
oltp_index_scan 14,706 9,147 0.62
select_random_points 36,724 23,785 0.65
select_random_ranges 13,611 8,324 0.61
covering_index_scan 13,820 7,908 0.57
groupby_scan 38,556 38,343 0.99
index_join 14,759 11,947 0.81
index_join_scan 7,391 6,509 0.88
types_table_scan 1,402,834 1,346,180 0.96
table_scan 1,659,339 1,481,658 0.89
oltp_read_only 260,833 180,976 0.69
Average 0.77

Writes

Test SQLite (us) Doltlite (us) Multiplier
oltp_bulk_insert 255,831 356,437 1.39
oltp_insert 36,788 51,543 1.40
oltp_update_index 185,532 173,298 0.93
oltp_update_non_index 113,029 112,718 1.00
oltp_delete_insert 123,699 136,842 1.11
oltp_write_only 74,031 88,737 1.20
types_delete_insert 65,071 73,555 1.13
oltp_read_write 162,901 167,807 1.03
Average 1.15

File-Backed (autocommit)

Each statement runs as its own transaction — exposes per-commit
fixed costs that the wrapped-in-BEGIN/COMMIT tests amortize away.
SQLite uses WAL mode with synchronous=FULL in this section so
the comparison uses SQLite's durable WAL autocommit path.

Reads

Reads have no commit cost; these are the same SQL files as the
File-Backed Reads section, included here for symmetry and to
catch any per-statement overhead doltlite pays on the read path.

Test SQLite (us) Doltlite (us) Multiplier
oltp_point_select 68,586 64,774 0.94
oltp_range_select 19,360 19,117 0.99
oltp_sum_range 18,348 18,522 1.01
oltp_order_range 3,741 3,717 0.99
oltp_distinct_range 4,808 4,857 1.01
oltp_index_scan 9,579 9,168 0.96
select_random_points 31,164 23,589 0.76
select_random_ranges 8,074 8,189 1.01
covering_index_scan 8,450 7,805 0.92
groupby_scan 38,069 38,003 1.00
index_join 12,244 11,747 0.96
index_join_scan 6,729 6,379 0.95
types_table_scan 1,391,482 1,357,501 0.98
table_scan 1,671,180 1,500,368 0.90
oltp_read_only 188,243 186,036 0.99
Average 0.96

Writes

Test SQLite (us) Doltlite (us) Multiplier
oltp_bulk_insert_ac 17,039 71,372 4.19
oltp_insert_ac 20,039 86,343 4.31
oltp_update_index_ac 21,000 96,430 4.59
oltp_update_non_index_ac 18,319 78,038 4.26
oltp_delete_insert_ac 18,950 87,509 4.62
oltp_write_only_ac 20,479 81,647 3.99
types_delete_insert_ac 17,980 78,414 4.36
oltp_read_write_ac 26,684 95,703 3.59
Average 4.24

100000 rows, median of 5 invocations per test, workload-only timing via host monotonic clock when available.

Performance Ceiling Check (2.5x individual, 2x average; autocommit writes: 5x / 5x)

All tests within ceilings.

@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 27, 2026

Sysbench-Style Benchmark (TEXT PK): Doltlite vs SQLite

Companion to the classic Sysbench-Style Benchmark. Every workload here
runs against tables with a 32-char hex TEXT PRIMARY KEY (UUID-shaped).

Individual ratios gated at 2.5×; section averages gated at 2×. Autocommit writes use 5× / 5×.

In-Memory

Reads

Test SQLite (us) Doltlite (us) Multiplier
oltp_point_select 28,792 36,805 1.28
oltp_range_select 12,611 16,106 1.28
oltp_sum_range 11,425 15,835 1.39
oltp_order_range 2,861 3,358 1.17
oltp_distinct_range 3,907 4,461 1.14
oltp_index_scan 4,273 6,065 1.42
select_random_points 17,460 21,241 1.22
select_random_ranges 3,734 5,221 1.40
covering_index_scan 4,343 4,417 1.02
groupby_scan 31,890 35,448 1.11
index_join 6,678 9,036 1.35
index_join_scan 4,366 5,430 1.24
types_table_scan 1,063,150 1,247,955 1.17
table_scan 1,209,980 1,339,471 1.11
oltp_read_only 117,077 142,347 1.22
Average 1.23

Writes

Test SQLite (us) Doltlite (us) Multiplier
oltp_bulk_insert 229,204 339,881 1.48
oltp_insert 21,158 37,081 1.75
oltp_update_index 69,399 129,188 1.86
oltp_update_non_index 46,497 84,121 1.81
oltp_delete_insert 49,896 100,180 2.01
oltp_write_only 27,746 59,291 2.14
types_delete_insert 31,813 53,898 1.69
oltp_read_write 81,560 138,152 1.69
Average 1.80

File-Backed

Reads

Test SQLite (us) Doltlite (us) Multiplier
oltp_point_select 104,286 61,248 0.59
oltp_range_select 22,350 18,789 0.84
oltp_sum_range 21,438 18,496 0.86
oltp_order_range 3,905 3,733 0.96
oltp_distinct_range 4,960 4,850 0.98
oltp_index_scan 12,947 8,913 0.69
select_random_points 34,169 24,974 0.73
select_random_ranges 11,486 7,813 0.68
covering_index_scan 12,222 7,298 0.60
groupby_scan 35,453 36,050 1.02
index_join 14,157 11,344 0.80
index_join_scan 7,743 6,050 0.78
types_table_scan 1,273,561 1,251,516 0.98
table_scan 1,501,924 1,354,221 0.90
oltp_read_only 237,963 179,960 0.76
Average 0.81

Writes

Test SQLite (us) Doltlite (us) Multiplier
oltp_bulk_insert 240,724 365,675 1.52
oltp_insert 49,118 50,200 1.02
oltp_update_index 184,726 163,939 0.89
oltp_update_non_index 143,153 110,207 0.77
oltp_delete_insert 126,004 128,793 1.02
oltp_write_only 90,336 81,442 0.90
types_delete_insert 67,608 72,056 1.07
oltp_read_write 195,102 163,865 0.84
Average 1.00

File-Backed (autocommit)

Each statement runs as its own transaction — exposes per-commit
fixed costs that the wrapped-in-BEGIN/COMMIT tests amortize away.
SQLite uses WAL mode with synchronous=FULL in this section so
the comparison uses SQLite's durable WAL autocommit path.

Reads

Reads have no commit cost; these are the same SQL files as the
File-Backed Reads section, included here for symmetry and to
catch any per-statement overhead doltlite pays on the read path.

Test SQLite (us) Doltlite (us) Multiplier
oltp_point_select 59,874 61,687 1.03
oltp_range_select 17,884 18,794 1.05
oltp_sum_range 16,896 18,354 1.09
oltp_order_range 3,502 3,780 1.08
oltp_distinct_range 4,577 4,789 1.05
oltp_index_scan 8,637 8,875 1.03
select_random_points 28,690 24,953 0.87
select_random_ranges 7,123 7,776 1.09
covering_index_scan 8,260 7,226 0.87
groupby_scan 34,671 36,169 1.04
index_join 12,191 11,243 0.92
index_join_scan 7,520 6,138 0.82
types_table_scan 1,279,087 1,245,288 0.97
table_scan 1,510,549 1,353,381 0.90
oltp_read_only 175,226 179,323 1.02
Average 0.99

Writes

Test SQLite (us) Doltlite (us) Multiplier
oltp_bulk_insert_ac 21,532 74,795 3.47
oltp_insert_ac 27,302 93,111 3.41
oltp_update_index_ac 27,166 106,492 3.92
oltp_update_non_index_ac 24,879 88,185 3.54
oltp_delete_insert_ac 25,226 98,500 3.90
oltp_write_only_ac 24,439 98,433 4.03
types_delete_insert_ac 22,096 95,497 4.32
oltp_read_write_ac 31,070 114,939 3.70
Average 3.79

100000 rows, median of 5 invocations per test, workload-only timing via host monotonic clock when available.

Performance Ceiling Check (2.5x individual, 2x average; autocommit writes: 5x / 5x)

All tests within ceilings.

@timsehn timsehn force-pushed the windows-build-test-coverage branch from 4d7e848 to f0b6e79 Compare May 27, 2026 18:24
@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 27, 2026

Sysbench-Style Benchmark (composite PK): Doltlite vs SQLite

Companion to the classic Sysbench-Style Benchmark. Every workload here
runs against tables with a 2-column INTEGER PRIMARY KEY(a, b) WITHOUT ROWID.

Individual ratios gated at 2.5×; section averages gated at 2×. Autocommit writes use 5× / 5×.

In-Memory

Reads

Test SQLite (us) Doltlite (us) Multiplier
oltp_point_select 29,192 34,788 1.19
oltp_range_select 16,752 20,174 1.20
oltp_sum_range 15,868 19,385 1.22
oltp_order_range 3,140 3,575 1.14
oltp_distinct_range 3,940 4,472 1.14
oltp_index_scan 3,967 5,306 1.34
select_random_points 25,341 29,416 1.16
select_random_ranges 6,562 7,971 1.21
covering_index_scan 3,586 3,848 1.07
groupby_scan 32,769 36,067 1.10
index_join 6,930 9,963 1.44
index_join_scan 3,317 5,385 1.62
types_table_scan 1,006,743 1,137,983 1.13
table_scan 1,239,679 1,255,109 1.01
oltp_read_only 131,810 154,253 1.17
Average 1.21

Writes

Test SQLite (us) Doltlite (us) Multiplier
oltp_bulk_insert 207,417 284,857 1.37
oltp_insert 16,038 29,575 1.84
oltp_update_index 60,604 101,630 1.68
oltp_update_non_index 43,677 69,931 1.60
oltp_delete_insert 43,070 80,422 1.87
oltp_write_only 23,046 47,505 2.06
types_delete_insert 27,892 46,191 1.66
oltp_read_write 88,820 132,318 1.49
Average 1.70

File-Backed

Reads

Test SQLite (us) Doltlite (us) Multiplier
oltp_point_select 64,040 47,233 0.74
oltp_range_select 21,323 21,697 1.02
oltp_sum_range 20,678 20,777 1.00
oltp_order_range 3,700 3,772 1.02
oltp_distinct_range 4,462 4,677 1.05
oltp_index_scan 8,038 6,891 0.86
select_random_points 34,685 31,470 0.91
select_random_ranges 10,626 9,443 0.89
covering_index_scan 6,590 5,255 0.80
groupby_scan 34,065 36,374 1.07
index_join 9,589 10,897 1.14
index_join_scan 4,086 5,584 1.37
types_table_scan 1,106,569 1,140,860 1.03
table_scan 1,296,462 1,262,126 0.97
oltp_read_only 188,084 171,203 0.91
Average 0.98

Writes

Test SQLite (us) Doltlite (us) Multiplier
oltp_bulk_insert 221,556 301,228 1.36
oltp_insert 24,319 36,493 1.50
oltp_update_index 115,721 120,805 1.04
oltp_update_non_index 81,715 84,126 1.03
oltp_delete_insert 82,394 95,928 1.16
oltp_write_only 51,602 61,338 1.19
types_delete_insert 48,770 55,680 1.14
oltp_read_write 130,791 145,438 1.11
Average 1.19

File-Backed (autocommit)

Each statement runs as its own transaction — exposes per-commit
fixed costs that the wrapped-in-BEGIN/COMMIT tests amortize away.
SQLite uses WAL mode with synchronous=FULL in this section so
the comparison uses SQLite's durable WAL autocommit path.

Reads

Reads have no commit cost; these are the same SQL files as the
File-Backed Reads section, included here for symmetry and to
catch any per-statement overhead doltlite pays on the read path.

Test SQLite (us) Doltlite (us) Multiplier
oltp_point_select 44,010 46,073 1.05
oltp_range_select 19,585 21,811 1.11
oltp_sum_range 18,464 21,020 1.14
oltp_order_range 3,647 3,766 1.03
oltp_distinct_range 4,434 4,708 1.06
oltp_index_scan 6,148 6,839 1.11
select_random_points 32,604 31,630 0.97
select_random_ranges 8,695 9,377 1.08
covering_index_scan 4,673 5,216 1.12
groupby_scan 33,970 36,479 1.07
index_join 8,690 10,944 1.26
index_join_scan 4,027 5,596 1.39
types_table_scan 1,117,324 1,142,860 1.02
table_scan 1,314,100 1,265,647 0.96
oltp_read_only 158,427 171,712 1.08
Average 1.10

Writes

Test SQLite (us) Doltlite (us) Multiplier
oltp_bulk_insert_ac 29,329 100,597 3.43
oltp_insert_ac 32,913 112,228 3.41
oltp_update_index_ac 31,646 122,964 3.89
oltp_update_non_index_ac 26,920 109,880 4.08
oltp_delete_insert_ac 29,646 121,124 4.09
oltp_write_only_ac 31,471 116,975 3.72
types_delete_insert_ac 27,327 106,574 3.90
oltp_read_write_ac 34,898 117,881 3.38
Average 3.74

100000 rows, median of 5 invocations per test, workload-only timing via host monotonic clock when available.

Performance Ceiling Check (2.5x individual, 2x average; autocommit writes: 5x / 5x)

All tests within ceilings.

@timsehn timsehn force-pushed the windows-build-test-coverage branch 2 times, most recently from 6fc3b58 to 675f26e Compare May 27, 2026 18:38
@timsehn timsehn force-pushed the windows-build-test-coverage branch from 675f26e to 858b4c5 Compare May 27, 2026 18:43
@timsehn timsehn merged commit 9d9985c into master May 27, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant