Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 15 additions & 2 deletions .github/workflows/benchmark.yml
Original file line number Diff line number Diff line change
Expand Up @@ -38,8 +38,21 @@ jobs:
sudo apt-get update
sudo apt-get install -y wrk

- name: Run benchmark suite
run: npm run benchmark:compare -- --duration 20 --output benchmark_summary.md
- name: Run benchmark suite (Express 4 vs uExpress)
run: npm run benchmark:compare -- --duration 20 --output benchmark_v4.md

- name: Run benchmark suite (Express 5 vs uExpress v5)
run: npm run benchmark:compare -- --duration 20 --frameworks express5,ultimate-express-v5 --output benchmark_v5.md

- name: Combine benchmark results
run: |
echo '<!-- benchmark-comment -->' > benchmark_summary.md
echo '' >> benchmark_summary.md
cat benchmark_v4.md | grep -v '<!-- benchmark-comment -->' >> benchmark_summary.md
echo '' >> benchmark_summary.md
echo '## Express 5 Benchmark' >> benchmark_summary.md
echo '' >> benchmark_summary.md
cat benchmark_v5.md | grep -v '<!-- benchmark-comment -->' >> benchmark_summary.md

- name: Upload benchmark markdown artifact
uses: actions/upload-artifact@v4
Expand Down
20 changes: 17 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

The *Ultimate* Express. Fastest http server with **full** Express compatibility, based on µWebSockets.

This library is a very fast re-implementation of Express.js 4.
This library is a very fast re-implementation of Express.js that supports both **Express 4** and **Express 5** behavior.
It is designed to be a drop-in replacement for Express.js, with the same API and functionality, while being much faster. It is not a fork of Express.js.
To make sure µExpress matches behavior of Express in all cases, we run all tests with Express first, and then with µExpress and compare results to make sure they match.

Expand Down Expand Up @@ -92,6 +92,20 @@ app.listen(3000, () => {
- This also applies to non-SSL HTTP too. Do not create http server manually, use `app.listen()` instead.
- Node.JS max header size is 16384 bytes, while uWebSockets by default is 4096 bytes, so if you need longer headers set the env variable `UWS_HTTP_MAX_HEADERS_SIZE` to max byte count you need.

## Express 5 mode

µExpress supports both Express 4 and Express 5 behavior in the same package. By default, it behaves like Express 4. To enable Express 5 behavior, pass `version: 5` to the constructor:

```js
const express = require("ultimate-express");

// Express 4 behavior (default)
const app = express();

// Express 5 behavior
const app = express({ version: 5 });
```

## Performance tips

1. µExpress tries to optimize routing as much as possible, but it's only possible if:
Expand Down Expand Up @@ -132,7 +146,7 @@ const app = express({

## Compatibility

In general, basically all features and options are supported. Use [Express 4.x documentation](https://expressjs.com/en/4x/api.html) for API reference.
In general, basically all features and options are supported. Use [Express 4.x documentation](https://expressjs.com/en/4x/api.html) for API reference (default mode), or [Express 5.x migration guide](https://expressjs.com/en/guide/migrating-5) for v5 mode differences.

✅ - Full support (all features and options are supported)
🚧 - Partial support (some options are not supported)
Expand Down Expand Up @@ -323,7 +337,7 @@ Almost all middlewares that are compatible with Express are compatible with µEx

Middlewares and modules that are confirmed to not work:

- ❌ [express-async-errors](https://npmjs.com/package/express-async-errors) - doesn't work, use `app.set('catch async errors', true)` instead.
- ❌ [express-async-errors](https://npmjs.com/package/express-async-errors) - doesn't work, use `app.set('catch async errors', true)` instead, or use `version: 5` which handles async errors natively.

## Tested view engines

Expand Down
32 changes: 30 additions & 2 deletions benchmark/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Benchmark suite

Run all scenarios and compare `express` vs `ultimate-express`:
Run all scenarios and compare `express` vs `ultimate-express` (default):

```bash
npm run benchmark:compare -- --duration 20 --output benchmark_summary.md
Expand All @@ -12,7 +12,30 @@ Run a single scenario:
npm run benchmark:compare -- --duration 20 --scenario hello-world
```

Scenarios:
## Comparing against Express 5

Compare all four frameworks (Express 4, Express 5, uExpress v4 mode, uExpress v5 mode):

```bash
npm run benchmark:compare -- --duration 20 --frameworks express,express5,ultimate-express,ultimate-express-v5
```

Compare only Express 5 vs uExpress v5:

```bash
npm run benchmark:compare -- --duration 20 --frameworks express5,ultimate-express-v5
```

## Available frameworks

| ID | Description |
|----|-------------|
| `express` | Express 4 (latest-4 tag) |
| `express5` | Express 5 (latest tag) |
| `ultimate-express` | µExpress in v4 mode (default) |
| `ultimate-express-v5` | µExpress in v5 mode (`version: 5`) |

## Scenarios

- `hello-world`
- `routes-1000`
Expand All @@ -25,3 +48,8 @@ Scenarios:
- `streaming-without-content-length`
- `streaming-with-content-length`
- `readable-hash-4mb`

## Requirements

- Linux (wrk is required)
- `wrk` installed and in PATH
169 changes: 95 additions & 74 deletions benchmark/run.js
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,15 @@ const { spawn, spawnSync } = require('child_process');

const SCENARIO_FILES = fs.readdirSync(path.join(__dirname, 'scenarios')).filter((file) => file.endsWith('.js')).map((file) => file.replace('.js', ''));

const FRAMEWORKS = [
{ id: 'express', label: 'Express', port: 3001 },
{ id: 'ultimate-express', label: 'uExpress', port: 3000 }
const ALL_FRAMEWORKS = [
{ id: 'express', label: 'Express 4', port: 3001 },
{ id: 'express5', label: 'Express 5', port: 3002 },
{ id: 'ultimate-express', label: 'uExpress', port: 3000 },
{ id: 'ultimate-express-v5', label: 'uExpress v5', port: 3003 }
];

const DEFAULT_FRAMEWORKS = ['express', 'ultimate-express'];

function parseArgs(argv) {
const args = {};
for (let i = 0; i < argv.length; i++) {
Expand Down Expand Up @@ -155,7 +159,7 @@ function runHttpRequest(port, verifyConfig, timeoutMs = 30000) {
});
}

async function validateScenarioResponses(scenarioName, scenario) {
async function validateScenarioResponses(scenarioName, scenario, frameworks) {
const verify = scenario.verify || {};
let verifyBody = verify.body;
if (verify.bodyRepeat && typeof verify.bodyRepeat === 'object') {
Expand All @@ -170,7 +174,7 @@ async function validateScenarioResponses(scenarioName, scenario) {
};
const results = {};

for (const framework of FRAMEWORKS) {
for (const framework of frameworks) {
const { server, stderrRef } = startScenarioServer(framework, scenarioName);
try {
await waitForReady(framework.port);
Expand All @@ -180,23 +184,35 @@ async function validateScenarioResponses(scenarioName, scenario) {
}
}

const expressResult = results.express;
const ultimateResult = results['ultimate-express'];
const sameStatus = expressResult.statusCode === ultimateResult.statusCode;
const sameBodyHash = expressResult.bodyHash === ultimateResult.bodyHash;
// Compare all frameworks against the first one
const baseId = frameworks[0].id;
const baseResult = results[baseId];
const mismatches = [];

for (let i = 1; i < frameworks.length; i++) {
const fwId = frameworks[i].id;
const fwResult = results[fwId];
const sameStatus = baseResult.statusCode === fwResult.statusCode;
const sameBodyHash = baseResult.bodyHash === fwResult.bodyHash;
if (!sameStatus || !sameBodyHash) {
mismatches.push(
`${fwId}: status=${fwResult.statusCode}, hash=${fwResult.bodyHash}, size=${fwResult.bodySize}`
);
}
}

if (sameStatus && sameBodyHash) {
if (mismatches.length === 0) {
return {
ok: true,
message: `status ${expressResult.statusCode}, body sha256 ${expressResult.bodyHash.slice(0, 12)}`
message: `status ${baseResult.statusCode}, body sha256 ${baseResult.bodyHash.slice(0, 12)}`
};
}

return {
ok: false,
message: [
`express: status=${expressResult.statusCode}, hash=${expressResult.bodyHash}, size=${expressResult.bodySize}`,
`ultimate-express: status=${ultimateResult.statusCode}, hash=${ultimateResult.bodyHash}, size=${ultimateResult.bodySize}`
`${baseId}: status=${baseResult.statusCode}, hash=${baseResult.bodyHash}, size=${baseResult.bodySize}`,
...mismatches
].join(' | ')
};
}
Expand Down Expand Up @@ -313,42 +329,52 @@ async function runScenario(framework, scenarioName, scenario, durationSeconds) {
}
}

function buildMarkdown(results) {
function buildMarkdown(results, frameworks) {
const lines = [];
lines.push('<!-- benchmark-comment -->');
lines.push('## Benchmark Comparison');
lines.push('');
lines.push('| Test | Express req/sec | uExpress req/sec | Express throughput | uExpress throughput | uExpress speedup |');
lines.push('| --- | ---: | ---: | ---: | ---: | ---: |');

// Build header
const headerCols = ['Test'];
for (const fw of frameworks) {
headerCols.push(`${fw.label} req/sec`);
headerCols.push(`${fw.label} throughput`);
}
// Add speedup column (last framework vs first)
if (frameworks.length >= 2) {
headerCols.push(`${frameworks[frameworks.length - 1].label} speedup`);
}
lines.push(`| ${headerCols.join(' | ')} |`);
lines.push(`| ${headerCols.map(() => '---:').join(' | ')} |`);

const failures = [];
for (const row of results) {
const speedup = row.express.ok && row.ultimate.ok && row.express.transferPerSecBytes > 0
? `${(row.ultimate.transferPerSecBytes / row.express.transferPerSecBytes).toFixed(2)}x`
: 'N/A';
const expressReq = row.express.ok ? formatReqPerSec(row.express.requestsPerSec) : 'FAILED';
const ultimateReq = row.ultimate.ok ? formatReqPerSec(row.ultimate.requestsPerSec) : 'FAILED';
const expressTransfer = row.express.ok ? formatBytesPerSec(row.express.transferPerSecBytes) : 'FAILED';
const ultimateTransfer = row.ultimate.ok ? formatBytesPerSec(row.ultimate.transferPerSecBytes) : 'FAILED';

if (!row.express.ok) {
failures.push({
scenario: row.name,
framework: 'express',
message: row.express.error
});
const cols = [row.name];
for (const fw of frameworks) {
const r = row.results[fw.id];
if (r && r.ok) {
cols.push(formatReqPerSec(r.requestsPerSec));
cols.push(formatBytesPerSec(r.transferPerSecBytes));
} else {
cols.push('FAILED');
cols.push('FAILED');
if (r && !r.ok) {
failures.push({ scenario: row.name, framework: fw.id, message: r.error });
}
}
}
if (!row.ultimate.ok) {
failures.push({
scenario: row.name,
framework: 'ultimate-express',
message: row.ultimate.error
});
// Speedup: last vs first
if (frameworks.length >= 2) {
const first = row.results[frameworks[0].id];
const last = row.results[frameworks[frameworks.length - 1].id];
if (first && first.ok && last && last.ok && first.transferPerSecBytes > 0) {
cols.push(`**${(last.transferPerSecBytes / first.transferPerSecBytes).toFixed(2)}x**`);
} else {
cols.push('N/A');
}
}

lines.push(
`| ${row.name} | ${expressReq} | ${ultimateReq} | ${expressTransfer} | ${ultimateTransfer} | **${speedup}** |`
);
lines.push(`| ${cols.join(' | ')} |`);
}

if (failures.length > 0) {
Expand All @@ -373,14 +399,26 @@ async function main() {
const requestedScenario = args.scenario;
const scenarioList = requestedScenario ? [requestedScenario] : SCENARIO_FILES;

// Parse --frameworks flag (comma-separated list of framework ids)
const requestedFrameworks = args.frameworks
? args.frameworks.split(',').map((s) => s.trim())
: DEFAULT_FRAMEWORKS;
const FRAMEWORKS = ALL_FRAMEWORKS.filter((fw) => requestedFrameworks.includes(fw.id));

if (FRAMEWORKS.length === 0) {
throw new Error(`No valid frameworks specified. Available: ${ALL_FRAMEWORKS.map((f) => f.id).join(', ')}`);
}

process.stdout.write(`Frameworks: ${FRAMEWORKS.map((f) => f.label).join(', ')}\n`);

const results = [];
for (const scenarioName of scenarioList) {
const scenario = require(path.join(__dirname, 'scenarios', `${scenarioName}.js`));
process.stdout.write(`Running scenario: ${scenario.name}\n`);
let validation = null;

try {
validation = await validateScenarioResponses(scenarioName, scenario);
validation = await validateScenarioResponses(scenarioName, scenario, FRAMEWORKS);
if (validation.ok) {
process.stdout.write(`[validation] PASS ${scenario.name}: ${validation.message}\n`);
} else {
Expand All @@ -394,54 +432,37 @@ async function main() {
process.stderr.write(`[validation] ERROR ${scenario.name}: ${validation.message}\n`);
}

let expressResult;
try {
const successfulResult = await runScenario(FRAMEWORKS[0], scenarioName, scenario, durationSeconds);
expressResult = {
ok: true,
...successfulResult
};
} catch (error) {
expressResult = {
ok: false,
error: error.stack || error.message || String(error)
};
process.stderr.write(`[benchmark] FAILED express/${scenarioName}\n${expressResult.error}\n`);
}

let ultimateResult;
try {
const successfulResult = await runScenario(FRAMEWORKS[1], scenarioName, scenario, durationSeconds);
ultimateResult = {
ok: true,
...successfulResult
};
} catch (error) {
ultimateResult = {
ok: false,
error: error.stack || error.message || String(error)
};
process.stderr.write(`[benchmark] FAILED ultimate-express/${scenarioName}\n${ultimateResult.error}\n`);
const scenarioResults = {};
for (const framework of FRAMEWORKS) {
try {
const successfulResult = await runScenario(framework, scenarioName, scenario, durationSeconds);
scenarioResults[framework.id] = { ok: true, ...successfulResult };
} catch (error) {
const errorMessage = error.stack || error.message || String(error);
scenarioResults[framework.id] = { ok: false, error: errorMessage };
process.stderr.write(`[benchmark] FAILED ${framework.id}/${scenarioName}\n${errorMessage}\n`);
}
}

results.push({
name: scenario.name,
validation,
express: expressResult,
ultimate: ultimateResult
results: scenarioResults
});

process.stdout.write(`Done: ${scenario.name}\n`);
}

const successfulRows = results.filter((row) => row.express.ok || row.ultimate.ok);
if (successfulRows.length === 0) {
const hasAnySuccess = results.some((row) =>
Object.values(row.results).some((r) => r.ok)
);
if (!hasAnySuccess) {
throw new Error('All benchmark scenarios failed. No summary table generated.');
}

results.sort((a, b) => a.name.localeCompare(b.name));

const markdown = buildMarkdown(results);
const markdown = buildMarkdown(results, FRAMEWORKS);
fs.writeFileSync(outputPath, markdown, 'utf8');
process.stdout.write(markdown);
}
Expand Down
Loading
Loading