Skip to content

feat: configurable high-iteration benchmarks with env var overrides #76

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Copilot wants to merge 2 commits into
base: main
Choose a base branch
from
Open

feat: configurable high-iteration benchmarks with env var overrides #76

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
1793420
Initial plan
Copilot Apr 6, 2026
1f732e6
feat: add high-iteration benchmark support with env var controls and …
Copilot Apr 6, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
67 changes: 67 additions & 0 deletions benchmarks/RESULTS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
# Zario Benchmark Results

Results from running `npm run bench:high` (500,000 iterations/sample, 8 samples, 20% warmup).

Environment: Node.js v24.14.1, `--expose-gc`

---

## Sync Logging

| Benchmark | Iter/sample | Warmup | Median ops/sec | Mean ops/sec | P95 ops/sec | Median ns/op | Median total (ms) |
|---|---:|---:|---:|---:|---:|---:|---:|
| Simple message (sync) | 500,000 | 100,000 | 11,552,893 | 11,509,206 | 11,626,194 | 87 | 43.28 |
| Message with metadata (sync) | 500,000 | 100,000 | 10,572,753 | 10,538,822 | 10,653,341 | 95 | 47.29 |
| Message with deep metadata (sync) | 500,000 | 100,000 | 6,687,858 | 6,655,676 | 6,698,615 | 150 | 74.76 |

## Filtered Logs (Early Exit)

| Benchmark | Iter/sample | Warmup | Median ops/sec | Mean ops/sec | P95 ops/sec | Median ns/op | Median total (ms) |
|---|---:|---:|---:|---:|---:|---:|---:|
| Filtered debug log (level=error) | 500,000 | 100,000 | 1,008,217,655 | 993,001,783 | 1,724,197,386 | 1 | 0.50 |
| Filtered info log (level=error) | 500,000 | 100,000 | 1,005,689,767 | 930,771,268 | 1,597,270,584 | 1 | 0.50 |

## JSON Formatting

| Benchmark | Iter/sample | Warmup | Median ops/sec | Mean ops/sec | P95 ops/sec | Median ns/op | Median total (ms) |
|---|---:|---:|---:|---:|---:|---:|---:|
| Simple JSON log | 500,000 | 100,000 | 11,568,692 | 11,557,255 | 11,661,240 | 86 | 43.22 |
| JSON log with metadata | 500,000 | 100,000 | 10,543,431 | 10,419,380 | 10,687,100 | 95 | 47.43 |

## Async Logging

| Benchmark | Iter/sample | Warmup | Median ops/sec | Mean ops/sec | P95 ops/sec | Median ns/op | Median total (ms) |
|---|---:|---:|---:|---:|---:|---:|---:|
| Simple message (async enqueue) | 500,000 | 100,000 | 5,079,084 | 5,067,134 | 5,096,529 | 197 | 98.44 |
| Message with metadata (async enqueue) | 500,000 | 100,000 | 4,987,750 | 4,915,549 | 5,008,323 | 200 | 100.25 |

## Formatter Direct

| Benchmark | Iter/sample | Warmup | Median ops/sec | Mean ops/sec | P95 ops/sec | Median ns/op | Median total (ms) |
|---|---:|---:|---:|---:|---:|---:|---:|
| Format text (with metadata) | 500,000 | 100,000 | 2,181,644 | 2,179,942 | 2,189,498 | 458 | 229.19 |
| Format text (simple) | 500,000 | 100,000 | 3,790,382 | 3,784,413 | 3,806,556 | 264 | 131.91 |
| Format JSON (with metadata) | 500,000 | 100,000 | 931,265 | 929,513 | 939,866 | 1,074 | 536.90 |
| Format JSON (simple fast path) | 500,000 | 100,000 | 1,390,686 | 1,388,912 | 1,406,143 | 719 | 359.54 |

## Child Logger

| Benchmark | Iter/sample | Warmup | Median ops/sec | Mean ops/sec | P95 ops/sec | Median ns/op | Median total (ms) |
|---|---:|---:|---:|---:|---:|---:|---:|
| Child logger simple message | 500,000 | 100,000 | 10,453,676 | 10,445,592 | 10,480,180 | 96 | 47.83 |
| Child logger with metadata | 500,000 | 100,000 | 10,117,448 | 10,084,576 | 10,169,217 | 99 | 49.42 |

---

## Running the Benchmarks

```sh
# Standard benchmark (50k–200k iterations/sample)
npm run bench

# High-iteration benchmark (500k iterations/sample, 8 samples)
npm run bench:high

# Custom iterations/samples via environment variables
ZARIO_BENCH_ITERATIONS=1000000 ZARIO_BENCH_SAMPLES=10 npm run bench
```
40 changes: 17 additions & 23 deletions benchmarks/logger.bench.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,29 +30,23 @@ interface BenchmarkSummary {
medianTotalMs: number;
}

const DEFAULT_SYNC_CONFIG: BenchmarkConfig = {
iterations: 50_000,
warmupIterations: 10_000,
samples: 6,
};

const FILTERED_CONFIG: BenchmarkConfig = {
iterations: 200_000,
warmupIterations: 40_000,
samples: 6,
};

const DEFAULT_ASYNC_CONFIG: BenchmarkConfig = {
iterations: 20_000,
warmupIterations: 4_000,
samples: 6,
};

const FORMATTER_CONFIG: BenchmarkConfig = {
iterations: 80_000,
warmupIterations: 12_000,
samples: 6,
};
const BENCH_ITERATIONS = Number(process.env["ZARIO_BENCH_ITERATIONS"] ?? 0);
const BENCH_SAMPLES = Number(process.env["ZARIO_BENCH_SAMPLES"] ?? 0);

function makeConfig(baseIterations: number, baseWarmup: number): BenchmarkConfig {
const iterations = BENCH_ITERATIONS > 0 ? BENCH_ITERATIONS : baseIterations;
const warmupIterations = BENCH_ITERATIONS > 0 ? Math.floor(iterations * 0.2) : baseWarmup;
const samples = BENCH_SAMPLES > 0 ? BENCH_SAMPLES : 6;
Comment on lines +33 to +39

Copilot AI Apr 6, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The benchmark config reads iteration/sample counts directly via Number(...) and only checks > 0. This allows non-integers (e.g. 0.5), Infinity (e.g. 1e309/Infinity), or very large values, which can lead to misleading metrics (division by non-integer) or effectively infinite loops/hangs. Consider parsing as an integer and validating with Number.isFinite + Number.isInteger, and either clamping or throwing a clear error when invalid/out-of-range values are provided.

Suggested change
const BENCH_ITERATIONS = Number(process.env["ZARIO_BENCH_ITERATIONS"] ?? 0);
const BENCH_SAMPLES = Number(process.env["ZARIO_BENCH_SAMPLES"] ?? 0);
function makeConfig(baseIterations: number, baseWarmup: number): BenchmarkConfig {
const iterations = BENCH_ITERATIONS > 0 ? BENCH_ITERATIONS : baseIterations;
const warmupIterations = BENCH_ITERATIONS > 0 ? Math.floor(iterations * 0.2) : baseWarmup;
const samples = BENCH_SAMPLES > 0 ? BENCH_SAMPLES : 6;
function parsePositiveIntegerEnv(name: string): number | undefined {
const rawValue = process.env[name];
if (rawValue == null || rawValue.trim() === "") {
return undefined;
}
const parsedValue = Number(rawValue);
if (!Number.isFinite(parsedValue) || !Number.isInteger(parsedValue) || parsedValue <= 0) {
throw new Error(
`Invalid ${name}: expected a finite positive integer, received "${rawValue}".`,
);
}
return parsedValue;
}
const BENCH_ITERATIONS = parsePositiveIntegerEnv("ZARIO_BENCH_ITERATIONS");
const BENCH_SAMPLES = parsePositiveIntegerEnv("ZARIO_BENCH_SAMPLES");
function makeConfig(baseIterations: number, baseWarmup: number): BenchmarkConfig {
const iterations = BENCH_ITERATIONS ?? baseIterations;
const warmupIterations = BENCH_ITERATIONS != null ? Math.floor(iterations * 0.2) : baseWarmup;
const samples = BENCH_SAMPLES ?? 6;

Copilot uses AI. Check for mistakes.
return { iterations, warmupIterations, samples };
}

const DEFAULT_SYNC_CONFIG: BenchmarkConfig = makeConfig(50_000, 10_000);

const FILTERED_CONFIG: BenchmarkConfig = makeConfig(200_000, 40_000);

const DEFAULT_ASYNC_CONFIG: BenchmarkConfig = makeConfig(20_000, 4_000);

const FORMATTER_CONFIG: BenchmarkConfig = makeConfig(80_000, 12_000);

function maybeGC(): void {
const gcFn = (globalThis as { gc?: () => void }).gc;
Expand Down
48 changes: 46 additions & 2 deletions package-lock.json
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 3 additions & 0 deletions package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,8 @@
"dev": "tsup --watch",
"test": "jest",
"lint": "eslint src benchmarks --ext .ts --max-warnings=0",
"bench": "node --expose-gc --import tsx/esm benchmarks/logger.bench.ts",
"bench:high": "ZARIO_BENCH_ITERATIONS=500000 ZARIO_BENCH_SAMPLES=8 node --expose-gc --import tsx/esm benchmarks/logger.bench.ts",
Comment on lines +35 to +36

Copilot AI Apr 6, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The bench:high script sets env vars using POSIX inline assignment (ZARIO_BENCH_ITERATIONS=...). This won’t work in Windows cmd.exe (and differs in PowerShell), so the benchmark scripts become shell-dependent. If cross-platform support is desired, use a cross-platform env setter (e.g., cross-env) or move the preset logic into the benchmark script itself (e.g., via CLI args).

Copilot uses AI. Check for mistakes.
"prepublishOnly": "npm run build"
},
"repository": {
Expand Down Expand Up @@ -83,6 +85,7 @@
"jest": "^30.2.0",
"ts-jest": "^29.4.5",
"tsup": "^8.5.1",
"tsx": "^4.21.0",
"typescript": "^5.9.3",
"typescript-eslint": "^8.49.0"
},
Expand Down
Loading