Conversation
The feature has seen little adoption, and the tier rotation bled into APIs that otherwise had no business knowing about it (`ProxyConfiguration` pulling a `Request`, `newProxyInfo` returning a tier, browser controllers tracking a tier, etc.). In v4 the main rotation unit is the `Session`, so proxy-level tier rotation is redundant. Closes #3597
There was a problem hiding this comment.
Pull request overview
This PR removes the deprecated tieredProxyUrls proxy rotation mechanism (and the related proxyTier plumbing) to simplify the proxy/session surface area in Crawlee v4, aligning rotation around Session rather than proxy tiers.
Changes:
- Removed
tieredProxyUrlssupport fromProxyConfigurationand eliminated tier-tracking logic. - Removed
proxyTierfromProxyInfoand from the BrowserPool/BrowserCrawler plumbing. - Updated v4 upgrading docs and removed tiered-proxy documentation from the proxy management guide.
Reviewed changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| test/core/proxy_configuration.test.ts | Removes tiered-proxy tests and updates imports accordingly. |
| packages/types/src/session.ts | Drops proxyTier from ProxyInfo type. |
| packages/core/src/proxy_configuration.ts | Removes tieredProxyUrls API and tier-tracker implementation; simplifies option validation and URL generation. |
| packages/browser-pool/src/launch-context.ts | Removes proxyTier from launch context options/state. |
| packages/browser-pool/src/browser-pool.ts | Removes tier-aware browser selection/launch plumbing; simplifies capacity picking based on proxy URL only. |
| packages/browser-pool/src/abstract-classes/browser-plugin.ts | Stops passing proxyTier into launch context construction. |
| packages/browser-pool/src/abstract-classes/browser-controller.ts | Removes proxyTier from controller state/docs. |
| packages/browser-crawler/src/internals/browser-crawler.ts | Stops passing proxyTier into BrowserPool page creation. |
| docs/upgrading/upgrading_v4.md | Adds a migration note + example showing how to emulate tier escalation using named sessions in SessionPool. |
| docs/guides/proxy_management.mdx | Removes the “Tiered proxies” section from the guide. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
Sad to see this go but it is understandable. We didn't really have a need for this in the Store team (other than WCC maybe). When I proposed this feature, we are churning lot of Actors that could have used this but the business changed. I think we can implement this in userland if needed and go from there. |
l2ysho
left a comment
There was a problem hiding this comment.
this looks straightforward 👍
The feature has seen little adoption, and the tier rotation bled into APIs that otherwise had no business knowing about it (
ProxyConfigurationpulling aRequest,newProxyInforeturning a tier, browser controllers tracking a tier, etc.). Inv4, the main rotation unit is theSession, so proxy-level tier rotation is redundant.The behaviour can still be emulated idiomatically with
SessionandSessionPoolclasses (see the Upgrading guide).Closes #3597