Skip to content

feat(disk-storage): support opt-in flush for durable uploads (#1381)#1391

Open
SAY-5 wants to merge 1 commit intoexpressjs:mainfrom
SAY-5:disk-storage-flush
Open

feat(disk-storage): support opt-in flush for durable uploads (#1381)#1391
SAY-5 wants to merge 1 commit intoexpressjs:mainfrom
SAY-5:disk-storage-flush

Conversation

@SAY-5
Copy link
Copy Markdown

@SAY-5 SAY-5 commented Apr 15, 2026

Closes #1381.

Why

As reported in #1381, the disk storage pipes the uploaded file into a vanilla `fs.createWriteStream` and reports the upload as successful the moment the stream finishes. On Linux (and any OS with a write-back page cache) the bytes can still be sitting in kernel buffers when the request handler returns — a power loss or crash at the wrong moment leaves the file truncated or zero-length, but the HTTP response already claimed success.

Node.js 20.10 / 21.0 added a `flush` option to `fs.createWriteStream` that calls `fdatasync()` on the file descriptor before the stream closes, which is exactly what this workflow needs.

Change

Expose `flush` as an opt-in `DiskStorage` option:

```js
multer.diskStorage({
destination: './uploads',
flush: true
})
```

  • Default stays `false`, so existing users get the exact same buffered-write behavior and the same throughput — no perf regression.
  • When `flush` is `true`, the option is forwarded as `fs.createWriteStream(path, { flush: true })`. On runtimes older than Node 20.10, `createWriteStream` silently ignores the option, which matches the documented upstream behavior.
  • No change to `_removeFile` or error handling.

Tests

Added two regression tests in `test/disk-storage.js`:

  1. With `flush: true`, the test wraps `fs.createWriteStream` and asserts it was called with `{ flush: true }` while the upload still round-trips at the expected size.
  2. Without `flush`, `createWriteStream` is still called with no options — verifies the default behavior is unchanged.

Full `npm test` suite: 74/74 passing.

…js#1381)

Disk storage currently pipes the uploaded file into a vanilla
`fs.createWriteStream` and reports success as soon as the stream
finishes. On Linux, and any OS with a write-back page cache, this
means the bytes can still be sitting in kernel buffers when the
request handler returns — a power loss or crash at the wrong
moment leaves the file either truncated or zero-length, but the
HTTP response already claimed success.

Node.js 20.10 / 21.0 added a `flush` option to
`fs.createWriteStream` that calls `fdatasync()` on the underlying
file descriptor before the stream is closed. This commit exposes
it as an opt-in DiskStorage option:

    multer.diskStorage({ destination: dir, flush: true })

Callers who need durability can turn it on; everything else keeps
the current, faster buffered-write behavior so there's no perf
regression for existing users. On runtimes older than Node 20.10
the option is silently ignored by `createWriteStream`, which
matches the documented upstream behavior.

Added two regression tests in `test/disk-storage.js`:

1. With `flush: true`, `createWriteStream` receives
   `{ flush: true }` and the upload still succeeds.
2. Without `flush`, `createWriteStream` is still called with no
   options so the default behavior is unchanged.

Full `npm test` passes: 74/74.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

uploaded files are not flushed to disk

1 participant