chore: migrate to pnpm and enable minimum release age#175
Merged
Conversation
Migrates got-scraping from npm to pnpm and adds a 1-day minimum release age supply-chain guard at the package-manager layer (pnpm-workspace.yaml) and at the Renovate layer. Internal `@apify/*` and `@crawlee/*` packages are whitelisted at both layers. This repo previously had no Renovate config; a standard config is added now. Notable changes: - package.json: set packageManager to pnpm@10.24.0; bump engines node >=18 (was >=16; pnpm 10 needs Node 18+); "npm run" -> "pnpm" in scripts - pnpm-workspace.yaml: non-monorepo minimumReleaseAge settings only - .npmrc: node-linker=hoisted + link-workspace-packages=true + prefer-workspace-packages=true + public-hoist-pattern[]=* (last one ensures transitive types are visible to tsc like they were under npm's hoisted layout) - tsconfig.json: skipLibCheck: true (standard, suppresses ow/quick-lru d.ts incompatibilities with TS 5.9 lib types) - src/agent/wrapped-agent.ts + src/index.ts: two small casts to satisfy stricter type resolution under pnpm (npm happened to hide these mismatches via different node_modules layout; the same code paths at runtime) - New .github/actions/pnpm-install composite action (cached pnpm store, year-month + lockfile hash key — pattern from apify/apify-cli#1068) - CI workflows (check.yml, release.yaml, publish-to-npm.yaml): delegate install to the composite; use pnpm/pnpm publish --no-git-checks; Node matrix 18/20/22/24 - New renovate.json with minimumReleaseAge "1 day", internalChecksFilter "strict", and @apify/* + @crawlee/* whitelist Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
github-actions
Bot
added
the
t-tooling
This was referenced Apr 14, 2026
Replaces the local .github/actions/pnpm-install composite copy with the shared one from apify/workflows@main. Identical behavior, less duplication.
…m release-age gate
Silences npm warnings about unknown options like node-linker; pnpm reads the same keys from pnpm-workspace.yaml in camelCase form.
Block accidental npm/yarn install — npm 10.5+ and pnpm 10.x both honor devEngines.packageManager and refuse to run when it doesn't match.
barjin
approved these changes
Apr 16, 2026
Member
barjin
left a comment
barjin
left a comment
There was a problem hiding this comment.
Lgtm, just one curious question ⬇️
|
|
||
| createConnection(options: NetConnectOpts, callback?: (err: Error | null, stream: Duplex) => void): Duplex { | ||
| return this.agent.createConnection(options, callback); | ||
| return this.agent.createConnection(options, callback) as Duplex; |
Member
There was a problem hiding this comment.
Why does the package manager migration require a cast in the code?
Member
Author
There was a problem hiding this comment.
The lockfile regeneration during the pnpm migration resolved a newer @types/node where agent.createConnection() returns Duplex | null | undefined instead of just Duplex. Without the cast, tsc fails:
Type 'Duplex | null | undefined' is not assignable to type 'Duplex'.
The newer typing is more correct (the underlying Node API can return null/undefined), so the cast is the right fix here.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
devEngines.packageManager breaks pnpm commands that delegate to npm (pnpm version, pnpm pkg set, etc). Use the battle-tested only-allow approach instead (same as Vite, Vue, Astro).
barjin
pushed a commit
to apify/impit
that referenced
this pull request
Apr 22, 2026
## Summary Migrate the `impit-node/` Node/TypeScript subpackage from **Yarn 4** to **pnpm 10.24.0** as part of the org-wide supply-chain hardening rollout (aligning with apify/got-scraping#175 and apify/actor-scraper#273). The Rust workspace (`impit/`) and `impit-python/` are untouched. A 24 h `minimumReleaseAge` is enforced via `pnpm-workspace.yaml` and Renovate, with `@apify/*` and `@crawlee/*` excluded so first-party packages flow through immediately. ## Changes - **`impit-node/package.json`**: `packageManager` -> `pnpm@10.24.0`; scripts rewritten so `npm run build:debug` becomes `pnpm build:debug`. `engines.node` was already `>= 20` so no change needed. - **`impit-node/pnpm-workspace.yaml`** (new): `minimumReleaseAge: 1440` with `@apify/*`, `@crawlee/*` in the exclude list. - **`impit-node/.npmrc`** (new): `node-linker=hoisted`, `link-workspace-packages=true`, `prefer-workspace-packages=true`, `public-hoist-pattern[]=*` - needed so `napi-rs`, `typedoc`, and friends can resolve transitive deps without the Yarn PnP / flat layout they implicitly rely on. - **Deleted**: `impit-node/yarn.lock`, `impit-node/.yarnrc.yml`. There was no `.yarn/patches/` directory to preserve. - **Added**: `impit-node/pnpm-lock.yaml` (generated by `pnpm install`). - **`.github/actions/pnpm-install/action.yml`** (new): composite action copied verbatim from `apify/got-scraping` for consistency. The impit workflows don't end up using it (see below) but it's in place for future reuse. - **`.github/workflows/node-test.yaml`**: removed `cache: yarn`, `corepack enable`, `yarn global add pnpm lerna`; replaced `yarn --cwd impit-node X` with `pnpm X` (build job already sets `defaults.run.working-directory: impit-node`); inlined the pnpm setup (`pnpm/action-setup@v4.1.0` + `pnpm install --frozen-lockfile`) rather than using the composite because each job interleaves Rust toolchain setup, `dtolnay/rust-toolchain`, cargo caching, and Docker-based musl builds - approach (a) from the migration guide is cleaner here. Node test matrix is now `[18, 20, 22, 24]` (was `[20, 22]`). Docker-based musl/aarch64 test shells install pnpm via `npm install -g pnpm@10.24.0` before invoking it. - **`.github/workflows/node-release.yaml`**: same pnpm substitution; `yarn version` -> `npm version --no-git-tag-version` + `pnpm copy-version`; `npm publish --provenance` -> `pnpm publish --no-git-checks --provenance`; `yarn install --no-immutable` lockfile refresh -> `pnpm install --lockfile-only`; `npx semver` -> `pnpm dlx semver`. - **`.github/workflows/docs.yaml`**: replaced the Yarn setup in the TypeDoc job with pnpm; `cd impit-node && yarn docs` -> `cd impit-node && pnpm docs`. Rust and Python docs jobs untouched. - **`renovate.json`**: added `minimumReleaseAge: "1 day"` and `internalChecksFilter: "strict"`; added a package rule bypassing the delay for `@apify/*` and `@crawlee/*`; dropped the `constraints.npm` block which no longer reflects reality under pnpm. ## Rust + Node layout notes - `impit-node/` is one crate in a multi-crate Cargo workspace. `cargo metadata` and the napi build both run from `impit-node/` and reach out to the parent `Cargo.toml`, so nothing about the pnpm change affects the Rust side. - The `build` job in `node-test.yaml` uses `defaults.run.working-directory: impit-node` and the inline pnpm setup honours that. The docker-based musl matrix entries still `cd impit-node` explicitly because they run from the mounted workspace root (`/build`). ## Verification - `pnpm install` and `pnpm install --frozen-lockfile` in `impit-node/` both succeed with pnpm 10.24.0. - `pnpm exec napi --version` resolves (`3.6.1`), confirming the napi CLI is reachable through the hoisted layout. - `pnpm build` / `pnpm build:debug` / `pnpm test` require the Rust toolchain, which is not available in this environment; they are exercised by CI which already installs `dtolnay/rust-toolchain@stable`. - No `lint` script exists in `impit-node/package.json`, so nothing to run there. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
barjin
pushed a commit
to apify/camoufox-js
that referenced
this pull request
Apr 22, 2026
## Summary Migrates camoufox-js from Yarn 4 to pnpm and adds a 1-day minimum release age supply-chain guard at both the package-manager layer (`pnpm-workspace.yaml`) and the Renovate layer. Internal `@apify/*` and `@crawlee/*` packages are whitelisted at both layers. Part of the org-wide pnpm migration; follows the same pattern as apify/apify-shared-js#616, apify/actor-scraper#273, and apify/got-scraping#175. ## Changes - `package.json`: set `packageManager` to `pnpm@10.24.0` (was `yarn@4.13.0`); replace `npm run copy-files` with `pnpm copy-files` in the build script - Removed Yarn artifacts: `yarn.lock`, `.yarnrc.yml` - Removed `.yarn/` and `.npmrc` entries from `.gitignore` so pnpm's `.npmrc` is committed - `pnpm-workspace.yaml`: non-monorepo `minimumReleaseAge: 1440` with `@apify/*` + `@crawlee/*` exclusions - `.npmrc`: `node-linker=hoisted`, `link-workspace-packages=true`, `prefer-workspace-packages=true`, `public-hoist-pattern[]=*` - New `.github/actions/pnpm-install` composite action (cached pnpm store, year-month + lockfile hash key) - CI workflows (`test.yml`, `release.yml`): delegate install to the composite; corepack step removed; `yarn X` -> `pnpm X`; `npx camoufox fetch` -> `pnpm exec camoufox fetch`; `npm publish` -> `pnpm publish --no-git-checks`; Node matrix `[20, 22, 24]` (engines.node requires `>=20`, so Node 18 is intentionally omitted) - `renovate.json`: add `minimumReleaseAge: "1 day"`, `internalChecksFilter: "strict"`, and a packageRule whitelisting `@apify/*` + `@crawlee/*` at `0 days` 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
barjin
pushed a commit
to apify/fingerprint-suite
that referenced
this pull request
Apr 22, 2026
## Summary Migrates fingerprint-suite from npm to pnpm as part of an org-wide supply-chain hardening effort, aligning with the patterns established in apify/apify-shared-js#616, apify/actor-scraper#273 and apify/got-scraping#175. Enables a 24-hour `minimumReleaseAge` for third-party packages both at install time (via `pnpm-workspace.yaml`) and at dependency-update time (via `renovate.json`). `@apify/*` and `@crawlee/*` are excluded so our own releases can still be consumed immediately. ## Changes - Replace `package-lock.json` with `pnpm-lock.yaml`. - Move `workspaces` config from `package.json` to `pnpm-workspace.yaml` and add `minimumReleaseAge` settings with an exclude list for `@apify/*` and `@crawlee/*`. - Switch internal cross-package dependencies (`generative-bayesian-network`, `header-generator`, `fingerprint-generator`) to the `workspace:*` protocol so pnpm links the local copies rather than fetching from the registry. - Add `.npmrc` with `node-linker=hoisted`, `link-workspace-packages=true`, `prefer-workspace-packages=true`, `public-hoist-pattern[]=*`; drop `.npmrc` from `.gitignore`. - Add reusable composite action `.github/actions/pnpm-install` and use it from every workflow that installs deps (`test-and-sync`, `publish-to-npm`, `e2e-benchmark`, `model-updater`). - Replace all `npm run X` / `npx` with `pnpm X` / `pnpm exec` in root and package scripts and in workflows. - Bump `engines.node` to `>=18.0.0` in every workspace package and update the CI test matrix to `[18, 20, 22, 24]`. - Bump `packageManager` to `pnpm@10.24.0`. - `scripts/publish.sh` uses `pnpm publish --no-git-checks`. - Update `renovate.json`: add `"minimumReleaseAge": "1 day"` and `"internalChecksFilter": "strict"`, remove the legacy `"constraints": { "npm": "^7.0.0" }`, and add a package rule that sets `minimumReleaseAge: 0 days` for `@apify/*` and `@crawlee/*`. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Part of the org-wide supply-chain hardening + pnpm migration. Migrates got-scraping from npm to pnpm and adds a 1-day minimum release age guard at two layers:
minimumReleaseAge: 1440inpnpm-workspace.yamlblocks installs of versions < 1 day oldminimumReleaseAge: "1 day"delays PR creation@apify/*and@crawlee/*excluded at both layersThis repo previously had no Renovate config; a standard one is added.
Changes
package.json:packageManager: pnpm@10.24.0;engines.node >= 18(pnpm 10 requires Node 18+, was>=16);npm run→pnpmin scriptspnpm-workspace.yaml: non-monorepo minimum release age settings only.npmrc:node-linker=hoisted,link-workspace-packages=true,prefer-workspace-packages=true,public-hoist-pattern[]=*(the last one mirrors npm's transitive-type hoisting so tsc sees the same types)tsconfig.json:skipLibCheck: true(standard; suppresses ow/quick-lru.d.tsincompatibilities with TS 5.9'sMapIteratorlib types)src/agent/wrapped-agent.ts+src/index.ts: two small casts — same runtime behavior, satisfy stricter type resolution under pnpm that npm's hoisting happened to hide.github/actions/pnpm-installcomposite action (cached pnpm store, keyed by year-month + lockfile hash — pattern from chore: move to pnpm from yarn apify-cli#1068)pnpm publish --no-git-checks; Node matrix now 18/20/22/24renovate.jsonwithminimumReleaseAge "1 day",internalChecksFilter "strict",@apify/*+@crawlee/*whitelist🤖 Generated with Claude Code