From 441cc25944854827d5885d20d557c49d535253a2 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Mon, 16 Mar 2026 11:28:56 +0000 Subject: [PATCH 1/6] Initial plan From eafd73a04d6d7071253b73eff78988e5584ad0a6 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Mon, 16 Mar 2026 11:32:42 +0000 Subject: [PATCH 2/6] feat: add RFC 0050 to exclude node_modules from Time Machine backups Co-authored-by: guillaumekh <1011994+guillaumekh@users.noreply.github.com> --- ...-exclude-node-modules-from-time-machine.md | 203 ++++++++++++++++++ 1 file changed, 203 insertions(+) create mode 100644 accepted/0050-exclude-node-modules-from-time-machine.md diff --git a/accepted/0050-exclude-node-modules-from-time-machine.md b/accepted/0050-exclude-node-modules-from-time-machine.md new file mode 100644 index 00000000..2cbb5906 --- /dev/null +++ b/accepted/0050-exclude-node-modules-from-time-machine.md @@ -0,0 +1,203 @@ +# Exclude `node_modules` from macOS Time Machine backups + +## Summary + +When `npm install` creates a `node_modules` directory on macOS, npm should mark +it with the extended attribute (xattr) that tells Time Machine to skip it. This +prevents `node_modules` — which can be fully reconstructed from `package-lock.json` +— from filling up backup disks and slowing down backup runs. The behaviour can +be opted out of via an `.npmrc` flag for users who do wish to include +`node_modules` in their backups. + +## Motivation + +`node_modules` directories are, in effect, a local cache derived entirely from +`package-lock.json`. They can be recreated at any time with a single `npm install` +and therefore carry no unique, irreplaceable data. + +On macOS, Apple's Time Machine backup tool backs up every file that it can see, +including the often tens-of-thousands of small files inside `node_modules`. This +has several painful side-effects: + +* **Slow incremental backups.** Each `npm install` or `npm update` touches many + files, forcing Time Machine to scan and snapshot all of them on the next backup + run. +* **Rapid backup-disk consumption.** Accumulated snapshots of large + `node_modules` trees eat disk space quickly and can cause backups to fail + entirely when the backup volume fills up. +* **Slower system performance during backups.** The high file count in + `node_modules` can cause noticeable I/O spikes while Time Machine is indexing. + +The xattr-based "sticky exclusion" mechanism provided by macOS lets any tool +flag a directory so that it is permanently skipped by Time Machine — even when +the directory is moved or renamed — without requiring user interaction in System +Preferences. + +## Detailed Explanation + +### The macOS sticky-exclusion mechanism + +macOS exposes a per-item backup exclusion flag through the extended attribute +key `com.apple.metadata:com_apple_backup_excludeItem`. When this attribute is +present on a file or directory with the binary-plist value `com.apple.backupd`, +Time Machine (and the underlying `backupd` daemon) skips that item permanently. +This is the same mechanism used by the Xcode build toolchain for its `DerivedData` +directories. + +The binary plist value is: +``` +62 70 6C 69 73 74 30 30 5F 10 11 63 6F 6D 2E 61 |bplist00_..com.a| +70 70 6C 65 2E 62 61 63 6B 75 70 64 08 00 00 00 |pple.backupd....| +00 00 00 01 01 00 00 00 00 00 00 00 01 00 00 00 |................| +00 00 00 00 00 00 00 00 00 00 00 00 1C |.............| +``` + +This is equivalent to running: +```sh +tmutil addexclusion -p /path/to/node_modules +``` + +### When the attribute is applied + +npm should set this attribute whenever it creates or updates a `node_modules` +directory on macOS (i.e. `process.platform === 'darwin'`). Concretely, this +means applying it after the reify step in `@npmcli/arborist` completes, once +the `node_modules` directory is known to exist on disk. + +### Setting the attribute + +Because Node.js does not provide a built-in API for extended attributes, npm +should shell out to the macOS `xattr` command-line utility, which ships with +every macOS installation: + +```sh +xattr -w -x com.apple.metadata:com_apple_backup_excludeItem \ + "62706c6973743030 5f1011 636f6d2e6170706c652e6261636b757064 08 0000000000000101 0000000000000001 000000000000001c" \ + /path/to/node_modules +``` + +The call is made with `child_process.execFile` (non-blocking, errors suppressed) +so that it does not affect npm's exit code or performance on non-macOS systems +or when the `xattr` binary is unavailable. + +### New configuration option + +A new boolean flag `backup-node-modules` is added to npm's configuration: + +| Key | Default | Description | +|----------------------|----------|-------------| +| `backup-node-modules`| `false` | When `true`, prevents npm from marking `node_modules` with the Time Machine exclusion xattr on macOS. | + +Setting `backup-node-modules=true` in `.npmrc` (or via `npm config set`) gives +users who need `node_modules` in their backups a simple escape hatch. + +## Rationale and Alternatives + +### Alternative 1 – Opt-in (do nothing by default) + +npm could expose the `backup-node-modules=false` flag but leave the default as +`true` (i.e. back up by default, opt out of backup exclusion). This mirrors the +current status quo. + +**Drawback:** Users who suffer from bloated backups or slow backup runs must +first discover the problem, then discover the flag. Many users will never find +it. The downside of the default (disk-consuming, slow backups) is worse than +the downside of the proposed default (no backup of something that is +reconstructible), making opt-in a poorer default. + +### Alternative 2 – Require users to use `tmutil` manually + +Users can already run `tmutil addexclusion -p ~/project/node_modules` themselves. +Some tools (e.g. `Finder`'s "Exclude from backups" checkbox) surface this, but +the exclusion must be re-applied after `node_modules` is deleted and reinstalled. +The "sticky exclusion" set by npm persists even across reinstalls because it +travels with the directory while it exists and is re-applied by npm on the next +install. + +**Drawback:** Relies on the user taking manual action every time they start a +new project; does not scale. + +### Alternative 3 – Use a `.nobackup` file convention + +Some tools place a sentinel file (e.g. `CACHEDIR.TAG`) inside directories to +hint that the contents are caches. This is not recognised by Time Machine and +would have no effect. + +### Chosen approach + +The proposed opt-out default strikes the best balance: the vast majority of +developers do not want `node_modules` in their backups (it is reproducible from +`package-lock.json`), while the minority who do can set one flag. The xattr +approach is the correct macOS-native mechanism, requires no third-party +dependency, and is battle-tested by both Xcode and the Rust toolchain. + +## Implementation + +### Affected repositories / packages + +* **`@npmcli/arborist`** – After the `reify()` call writes `node_modules` to + disk, a new helper (`lib/utils/time-machine-exclude.js` or similar) is invoked + when `process.platform === 'darwin'` and the `backup-node-modules` config + option is `false`. + +* **`npm/cli`** – Adds the `backup-node-modules` configuration key (type: + `Boolean`, default: `false`) to `lib/utils/config/definitions.js` and exposes + it in the docs. + +### Helper implementation sketch + +```js +// lib/utils/time-machine-exclude.js (inside @npmcli/arborist or npm/cli) +const { execFile } = require('child_process') + +const ATTR_NAME = 'com.apple.metadata:com_apple_backup_excludeItem' +const ATTR_VALUE = + '62706c6973743030' + + '5f1011636f6d2e6170706c652e6261636b757064' + + '0800000000000001010000000000000001' + + '000000000000001c' + +function excludeFromTimeMachine (nodeModulesPath) { + if (process.platform !== 'darwin') { + return + } + execFile('xattr', ['-w', '-x', ATTR_NAME, ATTR_VALUE, nodeModulesPath], () => { + // errors intentionally ignored — absence of xattr binary or unsupported FS + // must not break npm + }) +} + +module.exports = { excludeFromTimeMachine } +``` + +### No native add-on required + +The implementation shells out to `/usr/bin/xattr`, which is present on every +supported macOS version. No native Node.js add-on or additional npm dependency +is needed. + +## Prior Art + +* **Rust / Cargo** – Added the same xattr exclusion for `target/` directories in + [cargo#4386](https://github.com/rust-lang/cargo/pull/4386). This is the + primary inspiration for this RFC. +* **Python / Poetry** – Implemented the same exclusion for `.venv` directories + in [poetry#4599](https://github.com/python-poetry/poetry/pull/4599). +* **Xcode** – Uses the same mechanism for `DerivedData`; Apple documents it in + their backup-exclusion guides. + +## Unresolved Questions and Bikeshedding + +* **Config flag naming**: `backup-node-modules` vs `time-machine-exclude` vs + `no-backup`? The name should make the purpose clear and follow npm config + naming conventions. +* **Scope of exclusion**: Should only the top-level `node_modules` be excluded, + or also nested `node_modules` within workspaces? The simplest initial + implementation excludes only the root `node_modules` and each workspace's own + `node_modules`, matching where Arborist writes packages. +* **Other backup tools**: The xattr is specific to Time Machine / `backupd`. It + does not exclude `node_modules` from iCloud Drive sync, Dropbox, or other + tools. Those cases may be addressed in a separate RFC. +* **Removal on `npm uninstall --global` or `rm -rf node_modules`**: The xattr + lives on the directory; when the directory is deleted and recreated, npm + re-applies the attribute on the next install. From 8568d5f08dd27e0b60325526e9202347e055aea5 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Mon, 16 Mar 2026 12:04:05 +0000 Subject: [PATCH 3/6] =?UTF-8?q?fix:=20update=20RFC=200050=20=E2=80=94=20op?= =?UTF-8?q?t-in=20default,=20no=20magic=20bytes,=20remove=20.nobackup=20al?= =?UTF-8?q?ternative?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: guillaumekh <1011994+guillaumekh@users.noreply.github.com> --- ...-exclude-node-modules-from-time-machine.md | 150 ++++++++++-------- 1 file changed, 85 insertions(+), 65 deletions(-) diff --git a/accepted/0050-exclude-node-modules-from-time-machine.md b/accepted/0050-exclude-node-modules-from-time-machine.md index 2cbb5906..305ed200 100644 --- a/accepted/0050-exclude-node-modules-from-time-machine.md +++ b/accepted/0050-exclude-node-modules-from-time-machine.md @@ -2,12 +2,10 @@ ## Summary -When `npm install` creates a `node_modules` directory on macOS, npm should mark -it with the extended attribute (xattr) that tells Time Machine to skip it. This +On macOS, npm should provide an opt-in flag that marks a project's `node_modules` +directory with the extended attribute that tells Time Machine to skip it. This prevents `node_modules` — which can be fully reconstructed from `package-lock.json` -— from filling up backup disks and slowing down backup runs. The behaviour can -be opted out of via an `.npmrc` flag for users who do wish to include -`node_modules` in their backups. +— from filling up backup disks and slowing down backup runs. ## Motivation @@ -39,71 +37,85 @@ Preferences. macOS exposes a per-item backup exclusion flag through the extended attribute key `com.apple.metadata:com_apple_backup_excludeItem`. When this attribute is -present on a file or directory with the binary-plist value `com.apple.backupd`, -Time Machine (and the underlying `backupd` daemon) skips that item permanently. -This is the same mechanism used by the Xcode build toolchain for its `DerivedData` -directories. +present on a file or directory with a binary plist value encoding the string +`"com.apple.backupd"`, Time Machine (and the underlying `backupd` daemon) skips +that item permanently. This is the same mechanism used by the Xcode build +toolchain for its `DerivedData` directories, and is equivalent to running: -The binary plist value is: -``` -62 70 6C 69 73 74 30 30 5F 10 11 63 6F 6D 2E 61 |bplist00_..com.a| -70 70 6C 65 2E 62 61 63 6B 75 70 64 08 00 00 00 |pple.backupd....| -00 00 00 01 01 00 00 00 00 00 00 00 01 00 00 00 |................| -00 00 00 00 00 00 00 00 00 00 00 00 1C |.............| -``` - -This is equivalent to running: ```sh tmutil addexclusion -p /path/to/node_modules ``` ### When the attribute is applied -npm should set this attribute whenever it creates or updates a `node_modules` -directory on macOS (i.e. `process.platform === 'darwin'`). Concretely, this -means applying it after the reify step in `@npmcli/arborist` completes, once -the `node_modules` directory is known to exist on disk. +When the opt-in flag is set, npm should apply this attribute whenever it creates +or updates a `node_modules` directory on macOS (i.e. `process.platform === +'darwin'`). Concretely, this means applying it after the reify step in +`@npmcli/arborist` completes, once the `node_modules` directory is known to +exist on disk. ### Setting the attribute Because Node.js does not provide a built-in API for extended attributes, npm should shell out to the macOS `xattr` command-line utility, which ships with -every macOS installation: +every macOS installation. The attribute value is a binary plist encoding of the +string `"com.apple.backupd"`, generated programmatically — the same approach +used by the Python `plistlib.dumps("com.apple.backupd", fmt=plistlib.FMT_BINARY)` +call in Poetry. In Node.js this can be built with standard `Buffer` operations: -```sh -xattr -w -x com.apple.metadata:com_apple_backup_excludeItem \ - "62706c6973743030 5f1011 636f6d2e6170706c652e6261636b757064 08 0000000000000101 0000000000000001 000000000000001c" \ - /path/to/node_modules +```js +// Construct a minimal binary plist containing a single ASCII string. +// This mirrors what Python's plistlib.dumps(str, fmt=FMT_BINARY) produces. +function binaryPlistString (str) { + const header = Buffer.from('bplist00') + const payload = Buffer.from(str, 'ascii') + // Object descriptor: 0x5F = variable-length ASCII string, + // 0x10 = 1-byte integer follows for the length + const objDesc = Buffer.from([0x5f, 0x10, payload.length]) + // Offset table: the single object starts right after the 8-byte header + const offsetTable = Buffer.from([header.length]) + // 32-byte trailer (Apple binary plist spec): + // 5 unused bytes, 1 sort-version byte, 1 offset-int-size byte, + // 1 object-ref-size byte, 8-byte object count, 8-byte top-object index, + // 8-byte offset-table start + const offsetTableStart = header.length + objDesc.length + payload.length + const trailer = Buffer.alloc(32) + trailer[6] = 1 // offset int size: 1 byte + trailer[7] = 1 // object ref size: 1 byte + trailer.writeBigUInt64BE(BigInt(1), 8) // num objects + trailer.writeBigUInt64BE(BigInt(0), 16) // top object + trailer.writeBigUInt64BE(BigInt(offsetTableStart), 24) // offset table + return Buffer.concat([header, objDesc, payload, offsetTable, trailer]) +} ``` -The call is made with `child_process.execFile` (non-blocking, errors suppressed) -so that it does not affect npm's exit code or performance on non-macOS systems -or when the `xattr` binary is unavailable. +The call to `xattr` is then made with `child_process.execFile` (non-blocking, +errors suppressed) so that it does not affect npm's exit code or performance on +non-macOS systems or when the `xattr` binary is unavailable. ### New configuration option -A new boolean flag `backup-node-modules` is added to npm's configuration: +A new boolean flag `time-machine-exclude` is added to npm's configuration: -| Key | Default | Description | -|----------------------|----------|-------------| -| `backup-node-modules`| `false` | When `true`, prevents npm from marking `node_modules` with the Time Machine exclusion xattr on macOS. | +| Key | Default | Description | +|------------------------|----------|-------------| +| `time-machine-exclude` | `false` | When `true`, marks `node_modules` with the macOS Time Machine exclusion xattr after every install on macOS. | -Setting `backup-node-modules=true` in `.npmrc` (or via `npm config set`) gives -users who need `node_modules` in their backups a simple escape hatch. +Users who want `node_modules` excluded from their backups set +`time-machine-exclude=true` in their `.npmrc` (or via `npm config set`). ## Rationale and Alternatives -### Alternative 1 – Opt-in (do nothing by default) +### Alternative 1 – Opt-out (exclude by default) -npm could expose the `backup-node-modules=false` flag but leave the default as -`true` (i.e. back up by default, opt out of backup exclusion). This mirrors the -current status quo. +npm could apply the xattr on every macOS install and provide a flag to opt back +in to backups. This would protect the majority of users automatically. -**Drawback:** Users who suffer from bloated backups or slow backup runs must -first discover the problem, then discover the flag. Many users will never find -it. The downside of the default (disk-consuming, slow backups) is worse than -the downside of the proposed default (no backup of something that is -reconstructible), making opt-in a poorer default. +**Drawback:** Silently preventing something from being backed up — without the +user asking for it — is surprising behaviour and has caused concern among npm +collaborators. A user who relies on their backup for disaster recovery may not +realise `node_modules` is excluded until they attempt a restore. The opt-in +approach ensures the user has made a deliberate choice. ### Alternative 2 – Require users to use `tmutil` manually @@ -117,19 +129,13 @@ install. **Drawback:** Relies on the user taking manual action every time they start a new project; does not scale. -### Alternative 3 – Use a `.nobackup` file convention - -Some tools place a sentinel file (e.g. `CACHEDIR.TAG`) inside directories to -hint that the contents are caches. This is not recognised by Time Machine and -would have no effect. - ### Chosen approach -The proposed opt-out default strikes the best balance: the vast majority of -developers do not want `node_modules` in their backups (it is reproducible from -`package-lock.json`), while the minority who do can set one flag. The xattr -approach is the correct macOS-native mechanism, requires no third-party -dependency, and is battle-tested by both Xcode and the Rust toolchain. +The opt-in default is the conservative choice: npm does not silently alter backup +behaviour, but provides a first-class, project-level knob for developers who +actively want their `node_modules` excluded. The xattr mechanism is the correct +macOS-native approach, requires no third-party dependency, and is battle-tested +by both Xcode and the Rust toolchain. ## Implementation @@ -137,10 +143,10 @@ dependency, and is battle-tested by both Xcode and the Rust toolchain. * **`@npmcli/arborist`** – After the `reify()` call writes `node_modules` to disk, a new helper (`lib/utils/time-machine-exclude.js` or similar) is invoked - when `process.platform === 'darwin'` and the `backup-node-modules` config - option is `false`. + when `process.platform === 'darwin'` and the `time-machine-exclude` config + option is `true`. -* **`npm/cli`** – Adds the `backup-node-modules` configuration key (type: +* **`npm/cli`** – Adds the `time-machine-exclude` configuration key (type: `Boolean`, default: `false`) to `lib/utils/config/definitions.js` and exposes it in the docs. @@ -151,17 +157,31 @@ dependency, and is battle-tested by both Xcode and the Rust toolchain. const { execFile } = require('child_process') const ATTR_NAME = 'com.apple.metadata:com_apple_backup_excludeItem' -const ATTR_VALUE = - '62706c6973743030' + - '5f1011636f6d2e6170706c652e6261636b757064' + - '0800000000000001010000000000000001' + - '000000000000001c' + +// Construct a minimal binary plist containing a single ASCII string. +// This mirrors what Python's plistlib.dumps(str, fmt=FMT_BINARY) produces. +function binaryPlistString (str) { + const header = Buffer.from('bplist00') + const payload = Buffer.from(str, 'ascii') + // 0x5F = variable-length ASCII string, 0x10 = 1-byte length follows + const objDesc = Buffer.from([0x5f, 0x10, payload.length]) + const offsetTable = Buffer.from([header.length]) + const offsetTableStart = header.length + objDesc.length + payload.length + const trailer = Buffer.alloc(32) + trailer[6] = 1 // offset int size + trailer[7] = 1 // object ref size + trailer.writeBigUInt64BE(BigInt(1), 8) + trailer.writeBigUInt64BE(BigInt(0), 16) + trailer.writeBigUInt64BE(BigInt(offsetTableStart), 24) + return Buffer.concat([header, objDesc, payload, offsetTable, trailer]) +} function excludeFromTimeMachine (nodeModulesPath) { if (process.platform !== 'darwin') { return } - execFile('xattr', ['-w', '-x', ATTR_NAME, ATTR_VALUE, nodeModulesPath], () => { + const attrValue = binaryPlistString('com.apple.backupd').toString('hex') + execFile('xattr', ['-w', '-x', ATTR_NAME, attrValue, nodeModulesPath], () => { // errors intentionally ignored — absence of xattr binary or unsupported FS // must not break npm }) From 4fd0299d14fad39a7f3f53fdadecb7d42e548a83 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Mon, 16 Mar 2026 14:38:48 +0000 Subject: [PATCH 4/6] fix: simplify xattr call to plain string and apply attribute on directory creation Co-authored-by: guillaumekh <1011994+guillaumekh@users.noreply.github.com> --- ...-exclude-node-modules-from-time-machine.md | 125 ++++++++---------- 1 file changed, 58 insertions(+), 67 deletions(-) diff --git a/accepted/0050-exclude-node-modules-from-time-machine.md b/accepted/0050-exclude-node-modules-from-time-machine.md index 305ed200..8a7b3c1a 100644 --- a/accepted/0050-exclude-node-modules-from-time-machine.md +++ b/accepted/0050-exclude-node-modules-from-time-machine.md @@ -37,10 +37,10 @@ Preferences. macOS exposes a per-item backup exclusion flag through the extended attribute key `com.apple.metadata:com_apple_backup_excludeItem`. When this attribute is -present on a file or directory with a binary plist value encoding the string -`"com.apple.backupd"`, Time Machine (and the underlying `backupd` daemon) skips -that item permanently. This is the same mechanism used by the Xcode build -toolchain for its `DerivedData` directories, and is equivalent to running: +present on a file or directory with the value `com.apple.backupd`, Time Machine +(and the underlying `backupd` daemon) skips that item permanently. This is the +same mechanism used by the Xcode build toolchain for its `DerivedData` +directories, and is equivalent to running: ```sh tmutil addexclusion -p /path/to/node_modules @@ -48,50 +48,28 @@ tmutil addexclusion -p /path/to/node_modules ### When the attribute is applied -When the opt-in flag is set, npm should apply this attribute whenever it creates -or updates a `node_modules` directory on macOS (i.e. `process.platform === -'darwin'`). Concretely, this means applying it after the reify step in -`@npmcli/arborist` completes, once the `node_modules` directory is known to -exist on disk. +When the opt-in flag is set, npm should apply this attribute on macOS +(i.e. `process.platform === 'darwin'`) as soon as a `node_modules` directory is +created on disk — that is, at directory-creation time inside +`@npmcli/arborist`, before any package files are written into it. ### Setting the attribute Because Node.js does not provide a built-in API for extended attributes, npm should shell out to the macOS `xattr` command-line utility, which ships with -every macOS installation. The attribute value is a binary plist encoding of the -string `"com.apple.backupd"`, generated programmatically — the same approach -used by the Python `plistlib.dumps("com.apple.backupd", fmt=plistlib.FMT_BINARY)` -call in Poetry. In Node.js this can be built with standard `Buffer` operations: +every macOS installation: -```js -// Construct a minimal binary plist containing a single ASCII string. -// This mirrors what Python's plistlib.dumps(str, fmt=FMT_BINARY) produces. -function binaryPlistString (str) { - const header = Buffer.from('bplist00') - const payload = Buffer.from(str, 'ascii') - // Object descriptor: 0x5F = variable-length ASCII string, - // 0x10 = 1-byte integer follows for the length - const objDesc = Buffer.from([0x5f, 0x10, payload.length]) - // Offset table: the single object starts right after the 8-byte header - const offsetTable = Buffer.from([header.length]) - // 32-byte trailer (Apple binary plist spec): - // 5 unused bytes, 1 sort-version byte, 1 offset-int-size byte, - // 1 object-ref-size byte, 8-byte object count, 8-byte top-object index, - // 8-byte offset-table start - const offsetTableStart = header.length + objDesc.length + payload.length - const trailer = Buffer.alloc(32) - trailer[6] = 1 // offset int size: 1 byte - trailer[7] = 1 // object ref size: 1 byte - trailer.writeBigUInt64BE(BigInt(1), 8) // num objects - trailer.writeBigUInt64BE(BigInt(0), 16) // top object - trailer.writeBigUInt64BE(BigInt(offsetTableStart), 24) // offset table - return Buffer.concat([header, objDesc, payload, offsetTable, trailer]) -} +```sh +xattr -w com.apple.metadata:com_apple_backup_excludeItem \ + com.apple.backupd \ + /path/to/node_modules ``` -The call to `xattr` is then made with `child_process.execFile` (non-blocking, -errors suppressed) so that it does not affect npm's exit code or performance on -non-macOS systems or when the `xattr` binary is unavailable. +Writing the value as a plain UTF-8 string is sufficient — Time Machine respects +the attribute regardless of whether the value is a raw string or a binary plist. +The call is made with `child_process.execFile` (non-blocking, errors suppressed) +so that it does not affect npm's exit code or performance on non-macOS systems +or when the `xattr` binary is unavailable. ### New configuration option @@ -117,7 +95,37 @@ collaborators. A user who relies on their backup for disaster recovery may not realise `node_modules` is excluded until they attempt a restore. The opt-in approach ensures the user has made a deliberate choice. -### Alternative 2 – Require users to use `tmutil` manually +### Alternative 2 – Write the value as a binary plist + +`tmutil addexclusion` writes the value as a binary plist encoding of the string +`com.apple.backupd`. This is technically the more correct form of the attribute. + +**Drawback:** Generating a binary plist in Node.js without a native library +requires manual `Buffer` construction (or shelling out to an additional tool). +In practice, Time Machine respects the attribute whether the value is a raw +string or a binary plist — many projects write it as a plain string with the +same effect. Plain strings are simpler and carry no additional risk. + +### Alternative 3 – Use `tmutil addexclusion` + +`tmutil addexclusion -p /path/to/node_modules` is the official command-line +interface for adding a "sticky" backup exclusion. + +**Drawback:** `tmutil` is significantly slower than writing the xattr directly, +which would noticeably increase install time. + +### Alternative 4 – Use `NSURLIsExcludedFromBackupKey` / `kCFURLIsExcludedFromBackupKey` + +Apple's official URL resource property (`NSURLIsExcludedFromBackupKey` in +Objective-C/Swift, `kCFURLIsExcludedFromBackupKey` in Core Foundation) is the +high-level API for the same underlying mechanism. This is the approach taken by +Cargo ([cargo#4386](https://github.com/rust-lang/cargo/pull/4386)). + +**Drawback:** Node.js and libuv do not expose this API. Using it would require +a native add-on, which significantly increases implementation and maintenance +complexity. + +### Alternative 5 – Require users to use `tmutil` manually Users can already run `tmutil addexclusion -p ~/project/node_modules` themselves. Some tools (e.g. `Finder`'s "Exclude from backups" checkbox) surface this, but @@ -133,17 +141,18 @@ new project; does not scale. The opt-in default is the conservative choice: npm does not silently alter backup behaviour, but provides a first-class, project-level knob for developers who -actively want their `node_modules` excluded. The xattr mechanism is the correct -macOS-native approach, requires no third-party dependency, and is battle-tested -by both Xcode and the Rust toolchain. +actively want their `node_modules` excluded. Writing the xattr value as a plain +string is the simplest correct implementation — no native add-on, no binary plist +generation, no dependency on slow system utilities. ## Implementation ### Affected repositories / packages -* **`@npmcli/arborist`** – After the `reify()` call writes `node_modules` to - disk, a new helper (`lib/utils/time-machine-exclude.js` or similar) is invoked - when `process.platform === 'darwin'` and the `time-machine-exclude` config +* **`@npmcli/arborist`** – The code path that creates a new `node_modules` + directory on disk is updated to immediately call a helper + (`lib/utils/time-machine-exclude.js` or similar) when + `process.platform === 'darwin'` and the `time-machine-exclude` config option is `true`. * **`npm/cli`** – Adds the `time-machine-exclude` configuration key (type: @@ -157,31 +166,13 @@ by both Xcode and the Rust toolchain. const { execFile } = require('child_process') const ATTR_NAME = 'com.apple.metadata:com_apple_backup_excludeItem' - -// Construct a minimal binary plist containing a single ASCII string. -// This mirrors what Python's plistlib.dumps(str, fmt=FMT_BINARY) produces. -function binaryPlistString (str) { - const header = Buffer.from('bplist00') - const payload = Buffer.from(str, 'ascii') - // 0x5F = variable-length ASCII string, 0x10 = 1-byte length follows - const objDesc = Buffer.from([0x5f, 0x10, payload.length]) - const offsetTable = Buffer.from([header.length]) - const offsetTableStart = header.length + objDesc.length + payload.length - const trailer = Buffer.alloc(32) - trailer[6] = 1 // offset int size - trailer[7] = 1 // object ref size - trailer.writeBigUInt64BE(BigInt(1), 8) - trailer.writeBigUInt64BE(BigInt(0), 16) - trailer.writeBigUInt64BE(BigInt(offsetTableStart), 24) - return Buffer.concat([header, objDesc, payload, offsetTable, trailer]) -} +const ATTR_VALUE = 'com.apple.backupd' function excludeFromTimeMachine (nodeModulesPath) { if (process.platform !== 'darwin') { return } - const attrValue = binaryPlistString('com.apple.backupd').toString('hex') - execFile('xattr', ['-w', '-x', ATTR_NAME, attrValue, nodeModulesPath], () => { + execFile('xattr', ['-w', ATTR_NAME, ATTR_VALUE, nodeModulesPath], () => { // errors intentionally ignored — absence of xattr binary or unsupported FS // must not break npm }) From 7127d0880477eafbe0157b65e10183423d7e8451 Mon Sep 17 00:00:00 2001 From: Guillaume Khayat Date: Mon, 16 Mar 2026 16:25:02 +0100 Subject: [PATCH 5/6] edits --- ...-exclude-node-modules-from-time-machine.md | 87 +++++++++---------- 1 file changed, 43 insertions(+), 44 deletions(-) diff --git a/accepted/0050-exclude-node-modules-from-time-machine.md b/accepted/0050-exclude-node-modules-from-time-machine.md index 8a7b3c1a..ca7f9e92 100644 --- a/accepted/0050-exclude-node-modules-from-time-machine.md +++ b/accepted/0050-exclude-node-modules-from-time-machine.md @@ -15,16 +15,13 @@ and therefore carry no unique, irreplaceable data. On macOS, Apple's Time Machine backup tool backs up every file that it can see, including the often tens-of-thousands of small files inside `node_modules`. This -has several painful side-effects: +has several regrettable side-effects: -* **Slow incremental backups.** Each `npm install` or `npm update` touches many +- **Slower backups.** Each `npm install` or `npm update` touches many files, forcing Time Machine to scan and snapshot all of them on the next backup run. -* **Rapid backup-disk consumption.** Accumulated snapshots of large - `node_modules` trees eat disk space quickly and can cause backups to fail - entirely when the backup volume fills up. -* **Slower system performance during backups.** The high file count in - `node_modules` can cause noticeable I/O spikes while Time Machine is indexing. +- **Higher backup-disk consumption.** Accumulated snapshots of large + `node_modules` trees eat disk space quickly. The xattr-based "sticky exclusion" mechanism provided by macOS lets any tool flag a directory so that it is permanently skipped by Time Machine — even when @@ -43,7 +40,7 @@ same mechanism used by the Xcode build toolchain for its `DerivedData` directories, and is equivalent to running: ```sh -tmutil addexclusion -p /path/to/node_modules +tmutil addexclusion /path/to/node_modules ``` ### When the attribute is applied @@ -53,6 +50,8 @@ When the opt-in flag is set, npm should apply this attribute on macOS created on disk — that is, at directory-creation time inside `@npmcli/arborist`, before any package files are written into it. +According to the [2021-10-13 meeting notes](https://github.com/npm/rfcs/blob/816944d6fe9c85130a5a7a92883c29c69e5777e2/meetings/2021-10-13.md?plain=1#L94), it was agreed the setting should be first introduced as opt-in. + ### Setting the attribute Because Node.js does not provide a built-in API for extended attributes, npm @@ -75,9 +74,9 @@ or when the `xattr` binary is unavailable. A new boolean flag `time-machine-exclude` is added to npm's configuration: -| Key | Default | Description | -|------------------------|----------|-------------| -| `time-machine-exclude` | `false` | When `true`, marks `node_modules` with the macOS Time Machine exclusion xattr after every install on macOS. | +| Key | Default | Description | +| ---------------------- | ------- | ----------------------------------------------------------------------------------------------------------- | +| `time-machine-exclude` | `false` | When `true`, marks `node_modules` with the macOS Time Machine exclusion xattr after every install on macOS. | Users who want `node_modules` excluded from their backups set `time-machine-exclude=true` in their `.npmrc` (or via `npm config set`). @@ -89,11 +88,12 @@ Users who want `node_modules` excluded from their backups set npm could apply the xattr on every macOS install and provide a flag to opt back in to backups. This would protect the majority of users automatically. -**Drawback:** Silently preventing something from being backed up — without the -user asking for it — is surprising behaviour and has caused concern among npm -collaborators. A user who relies on their backup for disaster recovery may not -realise `node_modules` is excluded until they attempt a restore. The opt-in -approach ensures the user has made a deliberate choice. +**Drawback:** Silently preventing something from being backed up without the +user asking for it is likely to surprise some users and has some maintainers have +[raised concerns](https://github.com/npm/rfcs/issues/471#issuecomment-943215770). +A user who relies on their backup for disaster recovery may not realise `node_modules` +is excluded until they attempt a restore. The opt-in approach ensures the user +has made a deliberate choice. ### Alternative 2 – Write the value as a binary plist @@ -103,8 +103,9 @@ approach ensures the user has made a deliberate choice. **Drawback:** Generating a binary plist in Node.js without a native library requires manual `Buffer` construction (or shelling out to an additional tool). In practice, Time Machine respects the attribute whether the value is a raw -string or a binary plist — many projects write it as a plain string with the -same effect. Plain strings are simpler and carry no additional risk. +string or a binary plist — [many projects](https://github.com/search?q=%22com.apple.metadata%3Acom_apple_backup_excludeItem%22&type=code) +write it as a plain string with the same effect. Plain strings are simpler +and carry no additional risk. ### Alternative 3 – Use `tmutil addexclusion` @@ -112,7 +113,8 @@ same effect. Plain strings are simpler and carry no additional risk. interface for adding a "sticky" backup exclusion. **Drawback:** `tmutil` is significantly slower than writing the xattr directly, -which would noticeably increase install time. +which would noticeably increase install time. I'm not sure what extra work tmutil +does which explain the longer execution time. ### Alternative 4 – Use `NSURLIsExcludedFromBackupKey` / `kCFURLIsExcludedFromBackupKey` @@ -125,7 +127,7 @@ Cargo ([cargo#4386](https://github.com/rust-lang/cargo/pull/4386)). a native add-on, which significantly increases implementation and maintenance complexity. -### Alternative 5 – Require users to use `tmutil` manually +### Alternative 5 – Advising users to use `tmutil` manually Users can already run `tmutil addexclusion -p ~/project/node_modules` themselves. Some tools (e.g. `Finder`'s "Exclude from backups" checkbox) surface this, but @@ -140,7 +142,7 @@ new project; does not scale. ### Chosen approach The opt-in default is the conservative choice: npm does not silently alter backup -behaviour, but provides a first-class, project-level knob for developers who +behaviour, but provides a first-class, either project-level or global knob for developers who actively want their `node_modules` excluded. Writing the xattr value as a plain string is the simplest correct implementation — no native add-on, no binary plist generation, no dependency on slow system utilities. @@ -149,13 +151,13 @@ generation, no dependency on slow system utilities. ### Affected repositories / packages -* **`@npmcli/arborist`** – The code path that creates a new `node_modules` +- **`@npmcli/arborist`** – The code path that creates a new `node_modules` directory on disk is updated to immediately call a helper (`lib/utils/time-machine-exclude.js` or similar) when `process.platform === 'darwin'` and the `time-machine-exclude` config option is `true`. -* **`npm/cli`** – Adds the `time-machine-exclude` configuration key (type: +- **`npm/cli`** – Adds the `time-machine-exclude` configuration key (type: `Boolean`, default: `false`) to `lib/utils/config/definitions.js` and exposes it in the docs. @@ -163,22 +165,22 @@ generation, no dependency on slow system utilities. ```js // lib/utils/time-machine-exclude.js (inside @npmcli/arborist or npm/cli) -const { execFile } = require('child_process') +const { execFile } = require("child_process"); -const ATTR_NAME = 'com.apple.metadata:com_apple_backup_excludeItem' -const ATTR_VALUE = 'com.apple.backupd' +const ATTR_NAME = "com.apple.metadata:com_apple_backup_excludeItem"; +const ATTR_VALUE = "com.apple.backupd"; -function excludeFromTimeMachine (nodeModulesPath) { - if (process.platform !== 'darwin') { - return +function excludeFromTimeMachine(nodeModulesPath) { + if (process.platform !== "darwin") { + return; } - execFile('xattr', ['-w', ATTR_NAME, ATTR_VALUE, nodeModulesPath], () => { + execFile("xattr", ["-w", ATTR_NAME, ATTR_VALUE, nodeModulesPath], () => { // errors intentionally ignored — absence of xattr binary or unsupported FS // must not break npm - }) + }); } -module.exports = { excludeFromTimeMachine } +module.exports = { excludeFromTimeMachine }; ``` ### No native add-on required @@ -189,26 +191,23 @@ is needed. ## Prior Art -* **Rust / Cargo** – Added the same xattr exclusion for `target/` directories in +- **Rust / Cargo** – Added the same xattr exclusion for `target/` directories in [cargo#4386](https://github.com/rust-lang/cargo/pull/4386). This is the primary inspiration for this RFC. -* **Python / Poetry** – Implemented the same exclusion for `.venv` directories +- **Python / Poetry** – Implemented the same exclusion for `.venv` directories in [poetry#4599](https://github.com/python-poetry/poetry/pull/4599). -* **Xcode** – Uses the same mechanism for `DerivedData`; Apple documents it in +- **Xcode** – Uses the same mechanism for `DerivedData`; Apple documents it in their backup-exclusion guides. ## Unresolved Questions and Bikeshedding -* **Config flag naming**: `backup-node-modules` vs `time-machine-exclude` vs +- **Support broader range of OS and backup software**: The Windows/Linux backup + landscape is much more fragmented, with no shared approach for programmatically + excluding files/directories. But maybe there are some widely deployed tools which + should still be supported? +- **Config flag naming**: `backup-node-modules` vs `time-machine-exclude` vs `no-backup`? The name should make the purpose clear and follow npm config naming conventions. -* **Scope of exclusion**: Should only the top-level `node_modules` be excluded, - or also nested `node_modules` within workspaces? The simplest initial - implementation excludes only the root `node_modules` and each workspace's own - `node_modules`, matching where Arborist writes packages. -* **Other backup tools**: The xattr is specific to Time Machine / `backupd`. It +- **Other backup tools**: The xattr is specific to Time Machine / `backupd`. It does not exclude `node_modules` from iCloud Drive sync, Dropbox, or other tools. Those cases may be addressed in a separate RFC. -* **Removal on `npm uninstall --global` or `rm -rf node_modules`**: The xattr - lives on the directory; when the directory is deleted and recreated, npm - re-applies the attribute on the next install. From c7302e506a7b9432adb3bad527d784058fb274fc Mon Sep 17 00:00:00 2001 From: Guillaume Khayat Date: Mon, 16 Mar 2026 16:54:46 +0100 Subject: [PATCH 6/6] proper tmutil invokation --- accepted/0050-exclude-node-modules-from-time-machine.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/accepted/0050-exclude-node-modules-from-time-machine.md b/accepted/0050-exclude-node-modules-from-time-machine.md index ca7f9e92..35e69b61 100644 --- a/accepted/0050-exclude-node-modules-from-time-machine.md +++ b/accepted/0050-exclude-node-modules-from-time-machine.md @@ -109,7 +109,7 @@ and carry no additional risk. ### Alternative 3 – Use `tmutil addexclusion` -`tmutil addexclusion -p /path/to/node_modules` is the official command-line +`tmutil addexclusion /path/to/node_modules` is the official command-line interface for adding a "sticky" backup exclusion. **Drawback:** `tmutil` is significantly slower than writing the xattr directly, @@ -129,7 +129,7 @@ complexity. ### Alternative 5 – Advising users to use `tmutil` manually -Users can already run `tmutil addexclusion -p ~/project/node_modules` themselves. +Users can already run `tmutil addexclusion ~/project/node_modules` themselves. Some tools (e.g. `Finder`'s "Exclude from backups" checkbox) surface this, but the exclusion must be re-applied after `node_modules` is deleted and reinstalled. The "sticky exclusion" set by npm persists even across reinstalls because it