From e2781b0e42fe47836f95e369e7e9fb4e260d428c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Richard=20Sol=C3=A1r?= Date: Mon, 27 Apr 2026 14:27:01 +0200 Subject: [PATCH 1/4] Generate docs Generated by yarn docusaurus docs:version 1.5 --- .../versioned_docs/version-1.5/changelog.md | 474 +++++ website/versioned_docs/version-1.5/index.md | 36 + .../version-1.5/installation.md | 87 + .../version-1.5/integrating-scrapy.md | 188 ++ .../versioned_docs/version-1.5/quick-start.md | 84 + .../versioned_docs/version-1.5/reference.md | 1560 +++++++++++++++++ .../versioned_docs/version-1.5/telemetry.md | 24 + .../version-1.5/troubleshooting.md | 41 + website/versioned_docs/version-1.5/vars.md | 60 + .../version-1.5-sidebars.json | 50 + .../version-1.5.0-sidebars.json | 50 + website/versions.json | 1 + 12 files changed, 2655 insertions(+) create mode 100644 website/versioned_docs/version-1.5/changelog.md create mode 100644 website/versioned_docs/version-1.5/index.md create mode 100644 website/versioned_docs/version-1.5/installation.md create mode 100644 website/versioned_docs/version-1.5/integrating-scrapy.md create mode 100644 website/versioned_docs/version-1.5/quick-start.md create mode 100644 website/versioned_docs/version-1.5/reference.md create mode 100644 website/versioned_docs/version-1.5/telemetry.md create mode 100644 website/versioned_docs/version-1.5/troubleshooting.md create mode 100644 website/versioned_docs/version-1.5/vars.md create mode 100644 website/versioned_sidebars/version-1.5-sidebars.json create mode 100644 website/versioned_sidebars/version-1.5.0-sidebars.json diff --git a/website/versioned_docs/version-1.5/changelog.md b/website/versioned_docs/version-1.5/changelog.md new file mode 100644 index 000000000..1b88f2de2 --- /dev/null +++ b/website/versioned_docs/version-1.5/changelog.md @@ -0,0 +1,474 @@ +--- +title: Changelog +sidebar_label: Changelog +toc_max_heading_level: 3 +--- +# Changelog + +All notable changes to this project will be documented in this file. + +### [1.5.0](https://github.com/apify/apify-cli/releases/tag/v1.5.0) (2026-04-27) + +#### πŸš€ Features + +* Add execution context to telemetry events ([#1074](https://github.com/apify/apify-cli/pull/1074)) ([9288ff8](https://github.com/apify/apify-cli/commit/9288ff886864acd78b5e06b5ff13096774bd81f6)) by [@patrikbraborec](https://github.com/patrikbraborec) +* Improve --help output for AI agents and humans ([#1080](https://github.com/apify/apify-cli/pull/1080)) ([e9be7ee](https://github.com/apify/apify-cli/commit/e9be7eee232fe4627ee252dc48dc01f2413fbccf)) by [@patrikbraborec](https://github.com/patrikbraborec), closes [#1060](https://github.com/apify/apify-cli/issues/1060) +* Add `apify api` subcommand for direct API access ([#1076](https://github.com/apify/apify-cli/pull/1076)) ([da15ae7](https://github.com/apify/apify-cli/commit/da15ae7bb48c353d1b62928b666fd634ee5c2832)) by [@patrikbraborec](https://github.com/patrikbraborec), closes [#1075](https://github.com/apify/apify-cli/issues/1075) +* Add --yes flag to commands that prompt users ([#1105](https://github.com/apify/apify-cli/pull/1105)) ([3446b68](https://github.com/apify/apify-cli/commit/3446b688d550e95952c07360f8b5d30894fa83f9)) by [@l2ysho](https://github.com/l2ysho) +* Add --user-agent flag for caller-identified telemetry ([#1102](https://github.com/apify/apify-cli/pull/1102)) ([5d1fe06](https://github.com/apify/apify-cli/commit/5d1fe06771a5a60d68c6487de092c4f5bdcb5ef2)) by [@patrikbraborec](https://github.com/patrikbraborec), closes [#1100](https://github.com/apify/apify-cli/issues/1100) + +### [1.4.1](https://github.com/apify/apify-cli/releases/tag/v1.4.1) (2026-04-02) + +#### πŸ› Bug Fixes + +* Correct upgrade script URL for Windows ([#1055](https://github.com/apify/apify-cli/pull/1055)) ([0c5d1ec](https://github.com/apify/apify-cli/commit/0c5d1ec18be2711ed51cd90cc743ef869eb79465)) by [@vladfrangu](https://github.com/vladfrangu) + +### [1.4.0](https://github.com/apify/apify-cli/releases/tag/v1.4.0) (2026-03-26) + +#### πŸš€ Features + +* Add `.actorignore` support ([#1038](https://github.com/apify/apify-cli/pull/1038)) ([cb77da8](https://github.com/apify/apify-cli/commit/cb77da8c012ab4c234a207e7d26858647ba0370b)) by [@l2ysho](https://github.com/l2ysho), closes [#706](https://github.com/apify/apify-cli/issues/706), [#982](https://github.com/apify/apify-cli/issues/982) +* Add `actors search` command to search Apify Store ([#1047](https://github.com/apify/apify-cli/pull/1047)) ([ada9674](https://github.com/apify/apify-cli/commit/ada967424619ed2c1f597bb80153d0b1c77ec009)) by [@patrikbraborec](https://github.com/patrikbraborec) + +#### πŸ› Bug Fixes + +* Throw an error on missing secrets ([#1039](https://github.com/apify/apify-cli/pull/1039)) ([2ac7e3c](https://github.com/apify/apify-cli/commit/2ac7e3ca32d0fe35bfb6be9c2d38513518b16e61)) by [@janbuchar](https://github.com/janbuchar), closes [#921](https://github.com/apify/apify-cli/issues/921) +* **generate-schema-types:** Mark properties with defaults as always present in generated types ([#1048](https://github.com/apify/apify-cli/pull/1048)) ([0af4974](https://github.com/apify/apify-cli/commit/0af497418e32425b9d9235898f6447c553f4e810)) by [@JuanGalilea](https://github.com/JuanGalilea) + +### [1.3.0](https://github.com/apify/apify-cli/releases/tag/v1.3.0) (2026-03-11) + +#### πŸš€ Features + +* Add `--dockerfile` flag to `apify init` ([#989](https://github.com/apify/apify-cli/pull/989)) ([3f2104e](https://github.com/apify/apify-cli/commit/3f2104ef3321728cca07c53b853f668e31c66648)) by [@janbuchar](https://github.com/janbuchar), closes [#76](https://github.com/apify/apify-cli/issues/76) +* Add secrets:ls command to list all secrets ([#990](https://github.com/apify/apify-cli/pull/990)) ([8256cf5](https://github.com/apify/apify-cli/commit/8256cf55995060c8ff845dede46fb1943c9ec179)) by [@janbuchar](https://github.com/janbuchar), closes [#80](https://github.com/apify/apify-cli/issues/80) +* **actors:** Propagates title/description ([#992](https://github.com/apify/apify-cli/pull/992)) ([7285aec](https://github.com/apify/apify-cli/commit/7285aec39a1ecdcf6e21999ddf2195c427335da9)) by [@l2ysho](https://github.com/l2ysho), closes [#723](https://github.com/apify/apify-cli/issues/723) +* **auth:** Introduces auth commands ([#1002](https://github.com/apify/apify-cli/pull/1002)) ([f3e1460](https://github.com/apify/apify-cli/commit/f3e146065f489d0b0c85ff5356a9f070c1a279ab)) by [@l2ysho](https://github.com/l2ysho), closes [#898](https://github.com/apify/apify-cli/issues/898) +* **builds:** Adds build tag commands ([#1004](https://github.com/apify/apify-cli/pull/1004)) ([082f981](https://github.com/apify/apify-cli/commit/082f981a9161c745bbcfa5c89303ba381bff226c)) by [@l2ysho](https://github.com/l2ysho), closes [#997](https://github.com/apify/apify-cli/issues/997) +* **actors:** Enables standby mode via actor.json ([#991](https://github.com/apify/apify-cli/pull/991)) ([519aa43](https://github.com/apify/apify-cli/commit/519aa4327d92117d398fa47b819653012526ebdf)) by [@l2ysho](https://github.com/l2ysho), closes [#913](https://github.com/apify/apify-cli/issues/913) +* Add `--json` flag support to `apify push` ([#1025](https://github.com/apify/apify-cli/pull/1025)) ([baf39bb](https://github.com/apify/apify-cli/commit/baf39bb806ec8a552a1c34784d7bf1f0c76fcda4)) by [@Copilot](https://github.com/Copilot), closes [#1024](https://github.com/apify/apify-cli/issues/1024) +* **actor:** Adds actor types generation ([#1000](https://github.com/apify/apify-cli/pull/1000)) ([ed11261](https://github.com/apify/apify-cli/commit/ed11261f8cfdc1f7ca30d056c7da8411d4258b45)) by [@l2ysho](https://github.com/l2ysho), closes [#993](https://github.com/apify/apify-cli/issues/993) + +#### πŸ› Bug Fixes + +* Improve Python project detection and entrypoint resolution ([#1010](https://github.com/apify/apify-cli/pull/1010)) ([f0704d3](https://github.com/apify/apify-cli/commit/f0704d3767c8e4cab883e3ba5d678f951a3fa378)) by [@vdusek](https://github.com/vdusek) +* Set correct exit code on incorect `apify secrets` usage ([#1021](https://github.com/apify/apify-cli/pull/1021)) ([083c909](https://github.com/apify/apify-cli/commit/083c909990185ca48e88be77dce12ac5a627cf37)) by [@janbuchar](https://github.com/janbuchar), closes [#953](https://github.com/apify/apify-cli/issues/953) +* Enable standby mode on actors during push when usesStandbyMode is set ([#1016](https://github.com/apify/apify-cli/pull/1016)) ([fca7c58](https://github.com/apify/apify-cli/commit/fca7c58ce4659e659e0dc7984467ff43ce8f0949)) by [@MQ37](https://github.com/MQ37) +* Install script TTY handling on Unix ([#1032](https://github.com/apify/apify-cli/pull/1032)) ([f149310](https://github.com/apify/apify-cli/commit/f14931001b4eaf708185e8106bec5e902ad8fd65)) by [@vladfrangu](https://github.com/vladfrangu), closes [#1027](https://github.com/apify/apify-cli/issues/1027) + +#### ⚑ Performance + +* Speed up creating archives when pusing big codebases ([#1011](https://github.com/apify/apify-cli/pull/1011)) ([a873e0c](https://github.com/apify/apify-cli/commit/a873e0cb6c1c1f9e3bca61821de2b8373ec92744)) by [@matyascimbulka](https://github.com/matyascimbulka) + +### [1.2.1](https://github.com/apify/apify-cli/releases/tag/v1.2.1) (2026-01-06) + +#### πŸ”§ CI/CD + +* Fix release flow permissions ([#988](https://github.com/apify/apify-cli/pull/988)) ([a9d0d90](https://github.com/apify/apify-cli/commit/a9d0d9013f42e48bfcdddbc467e995b1655cf853)) by [@vladfrangu](https://github.com/vladfrangu) + +### [1.2.0](https://github.com/apify/apify-cli/releases/tag/v1.2.0) (2026-01-06) + +#### πŸš€ Features + +* Add automatic git repository initialization to `apify create` command ([#919](https://github.com/apify/apify-cli/pull/919)) ([bd1f0a1](https://github.com/apify/apify-cli/commit/bd1f0a1f7be15d807fab1e74951641059783f117)) by [@Copilot](https://github.com/Copilot), closes [#918](https://github.com/apify/apify-cli/issues/918) +* Add calculate-memory command to actor ([#980](https://github.com/apify/apify-cli/pull/980)) ([78eeee7](https://github.com/apify/apify-cli/commit/78eeee79e61b7c300ef669c7c310925539683e90)) by [@danpoletaev](https://github.com/danpoletaev) + +#### πŸ› Bug Fixes + +* Only run `git init` in new actors if not running from a git repository ([#925](https://github.com/apify/apify-cli/pull/925)) ([5c904bb](https://github.com/apify/apify-cli/commit/5c904bbc3819196bae59197729d67854034c0f4b)) by [@vladfrangu](https://github.com/vladfrangu) +* Handle unknown flags with random characters in name correctly ([#920](https://github.com/apify/apify-cli/pull/920)) ([958a3f2](https://github.com/apify/apify-cli/commit/958a3f22a8f714f651a39361c28d83f0fc2bba85)) by [@vladfrangu](https://github.com/vladfrangu) +* Alias `-f` for `--force` in `apify push` ([#940](https://github.com/apify/apify-cli/pull/940)) ([fea16ec](https://github.com/apify/apify-cli/commit/fea16ec6cf706a894d4e3e15165c8770911c1b16)) by [@vladfrangu](https://github.com/vladfrangu) +* **run:** Only match correct INPUT files, not any possible metadatas ([#951](https://github.com/apify/apify-cli/pull/951)) ([a07d513](https://github.com/apify/apify-cli/commit/a07d5130f6212f33f09e6c353875997fa15a8110)) by [@vladfrangu](https://github.com/vladfrangu), closes [#933](https://github.com/apify/apify-cli/issues/933) +* Calculate run memory min/max memory clamping ([#986](https://github.com/apify/apify-cli/pull/986)) ([0101cb1](https://github.com/apify/apify-cli/commit/0101cb18742d3e349d782710ed9fcbaa709f4bc7)) by [@danpoletaev](https://github.com/danpoletaev) + +### [1.1.1](https://github.com/apify/apify-cli/releases/tag/v1.1.1) (2025-08-18) + +#### πŸ› Bug Fixes + +* Unknown errors from tracking ([#895](https://github.com/apify/apify-cli/pull/895)) ([3485f36](https://github.com/apify/apify-cli/commit/3485f366f3a62117ac833e78157c230206c3c60e)) by [@vladfrangu](https://github.com/vladfrangu), closes [#894](https://github.com/apify/apify-cli/issues/894) +* Upgrade command should always check when manually ran ([#897](https://github.com/apify/apify-cli/pull/897)) ([5e0ea9f](https://github.com/apify/apify-cli/commit/5e0ea9ff84012732ca7117d1f68207b5170ffeed)) by [@vladfrangu](https://github.com/vladfrangu) + +### [1.1.0](https://github.com/apify/apify-cli/releases/tag/v1.1.0) (2025-08-13) + +#### πŸš€ Features + +* Upgrade command, install shell script ([#810](https://github.com/apify/apify-cli/pull/810)) ([51ef00a](https://github.com/apify/apify-cli/commit/51ef00ad32a6835c48781b99c6233113cf58d8a4)) by [@vladfrangu](https://github.com/vladfrangu) +* \[**breaking**] Make storage purging default, add `--resurrect` ([#729](https://github.com/apify/apify-cli/pull/729)) ([8dff93a](https://github.com/apify/apify-cli/commit/8dff93a2d769997a96d4a7750fb36c2770b9a61c)) by [@vladfrangu](https://github.com/vladfrangu), closes [#590](https://github.com/apify/apify-cli/issues/590) +* Handle sub-schema validation ([#853](https://github.com/apify/apify-cli/pull/853)) ([5fc2a2f](https://github.com/apify/apify-cli/commit/5fc2a2f6b780a86a250b69375455f3bb2e9a8983)) by [@MFori](https://github.com/MFori) +* Upgrade command upgrading CLI + install command ([#856](https://github.com/apify/apify-cli/pull/856)) ([4252e6c](https://github.com/apify/apify-cli/commit/4252e6cb681deb5f92c654520d0ed03b70e426c3)) by [@vladfrangu](https://github.com/vladfrangu) +* Add signature to KV store URLs where required ([#875](https://github.com/apify/apify-cli/pull/875)) ([a1e9982](https://github.com/apify/apify-cli/commit/a1e998270b5c05cd91280efa144325e2d7a7de0e)) by [@danpoletaev](https://github.com/danpoletaev) + +#### πŸ› Bug Fixes + +* Pretty message for invalid choices ([#805](https://github.com/apify/apify-cli/pull/805)) ([57bd5de](https://github.com/apify/apify-cli/commit/57bd5de9bc5289f151a9083533dc3d2c71f8b9ab)) by [@vladfrangu](https://github.com/vladfrangu) +* Shebangs ([#806](https://github.com/apify/apify-cli/pull/806)) ([1cdc101](https://github.com/apify/apify-cli/commit/1cdc1011f36974708ab91a25d4d6c6a5dc43d989)) by [@vladfrangu](https://github.com/vladfrangu) +* Recognize sh files as text files ([#813](https://github.com/apify/apify-cli/pull/813)) ([ef3e9b0](https://github.com/apify/apify-cli/commit/ef3e9b064483c04cd7bef2143a19e1a6992ddcff)) by [@DaveHanns](https://github.com/DaveHanns) +* **init:** Prompt for a name if an old config does not exist ([#836](https://github.com/apify/apify-cli/pull/836)) ([26fcd66](https://github.com/apify/apify-cli/commit/26fcd660a0f7b4adb4e1a3329705a8ff6d8f43b2)) by [@vladfrangu](https://github.com/vladfrangu), closes [#835](https://github.com/apify/apify-cli/issues/835) +* Pass apify client down to output job log wherever possible ([#839](https://github.com/apify/apify-cli/pull/839)) ([5cdb06c](https://github.com/apify/apify-cli/commit/5cdb06c0e24c2501b2034dbb7339798985b269cc)) by [@vladfrangu](https://github.com/vladfrangu) +* **pull:** Handle private actors correctly ([#865](https://github.com/apify/apify-cli/pull/865)) ([efd7308](https://github.com/apify/apify-cli/commit/efd730855f99a36091ce51d501e5755b5ad79ffb)) by [@vladfrangu](https://github.com/vladfrangu), closes [#855](https://github.com/apify/apify-cli/issues/855) + +#### Chore + +* \[**breaking**] Move from yargs to node:util ([#871](https://github.com/apify/apify-cli/pull/871)) ([482d0b2](https://github.com/apify/apify-cli/commit/482d0b29f285c020320f1f2e3f0fd08a362d57cc)) by [@vladfrangu](https://github.com/vladfrangu), closes [#833](https://github.com/apify/apify-cli/issues/833) +* \[**breaking**] Make opening the actor build results in push opt-in ([#881](https://github.com/apify/apify-cli/pull/881)) ([d842424](https://github.com/apify/apify-cli/commit/d84242421387a9487eef5c07183dd0b8ac7ae67b)) by [@vladfrangu](https://github.com/vladfrangu) + +### [0.21.9](https://github.com/apify/apify-cli/releases/tag/v0.21.9) (2025-07-17) + +#### πŸš€ Features + +* Handle sub-schema validation (#853) ([51cc733](https://github.com/apify/apify-cli/commit/51cc7333ca665a6e0dfab8a6dc296a396b3e40ff)) + +### [0.21.8](https://github.com/apify/apify-cli/releases/tag/v0.21.8) (2025-06-24) + +#### πŸ› Bug Fixes + +* Recognize sh files as text files (#813) ([9c514be](https://github.com/apify/apify-cli/commit/9c514bef27cded91d0d737077e01d8ddb471effb)) +* **init:** Prompt for a name if an old config does not exist (#836) ([4817c1c](https://github.com/apify/apify-cli/commit/4817c1c95ee6ca5d3078add283dd7c54c1444b48)) +* Pass apify client down to output job log wherever possible (#839) ([96ba05f](https://github.com/apify/apify-cli/commit/96ba05fdacf30131ef9173e24359bc98e805f1b1)) + +### [0.21.7](https://github.com/apify/apify-cli/releases/tag/v0.21.7) (2025-05-15) + +#### πŸ› Bug Fixes + +* Pick the start script for node projects before main property ([#787](https://github.com/apify/apify-cli/pull/787)) ([0e2a9d3](https://github.com/apify/apify-cli/commit/0e2a9d375f3b977b6f1a588fe5bbce5527b7505b)) by [@vladfrangu](https://github.com/vladfrangu) +* Non-"migrated" scrapy not working anymore ([#800](https://github.com/apify/apify-cli/pull/800)) ([9eaf76c](https://github.com/apify/apify-cli/commit/9eaf76cfe455c08b93511e985419182a6549d37e)) by [@vladfrangu](https://github.com/vladfrangu), closes [#796](https://github.com/apify/apify-cli/issues/796) + +### [0.21.6](https://github.com/apify/apify-cli/releases/tag/v0.21.6) (2025-04-24) + +#### πŸ› Bug Fixes + +* Windows and python compatibility ([#786](https://github.com/apify/apify-cli/pull/786)) ([eeca37d](https://github.com/apify/apify-cli/commit/eeca37df62461a0ca1fa6f2f72386b18eb65a077)) by [@vladfrangu](https://github.com/vladfrangu) + +### [0.21.5](https://github.com/apify/apify-cli/releases/tag/v0.21.5) (2025-04-23) + +#### πŸ› Bug Fixes + +* Runtime detection on Windows ([#785](https://github.com/apify/apify-cli/pull/785)) ([579d303](https://github.com/apify/apify-cli/commit/579d3036ac5062420abe6c17f478a24ca5da9e93)) by [@vladfrangu](https://github.com/vladfrangu) + +### [0.21.4](https://github.com/apify/apify-cli/releases/tag/v0.21.4) (2025-04-22) + +#### πŸ› Bug Fixes + +* Python project detection and project detection ([#774](https://github.com/apify/apify-cli/pull/774)) ([09a42e4](https://github.com/apify/apify-cli/commit/09a42e48b26793578ce845639f7ca921a94d41ff)) by [@vladfrangu](https://github.com/vladfrangu) + +### [0.21.3](https://github.com/apify/apify-cli/releases/tag/v0.21.3) (2025-03-31) + +#### πŸš€ Features + +* Add signature to actor get-public-url ([#767](https://github.com/apify/apify-cli/pull/767)) ([4641053](https://github.com/apify/apify-cli/commit/464105310d52bb78dbf979cc6d7d31c3ee30003d)) by [@danpoletaev](https://github.com/danpoletaev) + +### [0.21.2](https://github.com/apify/apify-cli/releases/tag/v0.21.2) (2025-02-26) + +#### πŸš€ Features + +* Actor charge command ([#748](https://github.com/apify/apify-cli/pull/748)) ([766c647](https://github.com/apify/apify-cli/commit/766c6470fe8c84fa1be663ee3251b48830f3cc27)) by [@mhamas](https://github.com/mhamas) +* Add get-public-url command ([#756](https://github.com/apify/apify-cli/pull/756)) ([88aadee](https://github.com/apify/apify-cli/commit/88aadee4621438b5df180a27f0ffc2a6b6bee946)) by [@janbuchar](https://github.com/janbuchar) + +#### πŸ› Bug Fixes + +* `actor set-value` with stdin input ([#755](https://github.com/apify/apify-cli/pull/755)) ([14bf6c5](https://github.com/apify/apify-cli/commit/14bf6c50a9e576700a60e1620a73b2e07c9b801e)) by [@vladfrangu](https://github.com/vladfrangu) + +### [0.21.1](https://github.com/apify/apify-cli/releases/tag/v0.21.1) (2025-02-05) + +#### πŸ› Bug Fixes + +* Emit logs to stderr when actors are started with `--json` flag ([#741](https://github.com/apify/apify-cli/pull/741)) ([a50b80e](https://github.com/apify/apify-cli/commit/a50b80e1e8f636b6878de8f8c30032c14daf8105)) by [@vladfrangu](https://github.com/vladfrangu), closes [#740](https://github.com/apify/apify-cli/issues/740) + +### [0.21.0](https://github.com/apify/apify-cli/releases/tag/v0.21.0) (2025-01-16) + +#### πŸš€ Features + +* Builds namespace ([#620](https://github.com/apify/apify-cli/pull/620)) ([6345162](https://github.com/apify/apify-cli/commit/6345162e44a00b404b4f95c2c80c2e437eff1d62)) by [@vladfrangu](https://github.com/vladfrangu) +* `runs ls` ([#640](https://github.com/apify/apify-cli/pull/640)) ([dd84d37](https://github.com/apify/apify-cli/commit/dd84d37c6ea89c64db712c7c94709f3181a7bd1f)) by [@vladfrangu](https://github.com/vladfrangu) +* `runs abort` ([#643](https://github.com/apify/apify-cli/pull/643)) ([96e5a34](https://github.com/apify/apify-cli/commit/96e5a3435cca08d87dc8d39659a7a6524f18be0e)) by [@vladfrangu](https://github.com/vladfrangu) +* `runs resurrect` ([#644](https://github.com/apify/apify-cli/pull/644)) ([7dbf4fb](https://github.com/apify/apify-cli/commit/7dbf4fb06c657246563c1c64e76ad83f283ea275)) by [@vladfrangu](https://github.com/vladfrangu) +* `runs log` ([#645](https://github.com/apify/apify-cli/pull/645)) ([dd6af5e](https://github.com/apify/apify-cli/commit/dd6af5ece79f4399fc5065483b650c71c61114cf)) by [@vladfrangu](https://github.com/vladfrangu) +* `runs rm` & `builds rm` & `actors rm` ([#648](https://github.com/apify/apify-cli/pull/648)) ([566f8c5](https://github.com/apify/apify-cli/commit/566f8c5d1482f150f4d61229524c7672c2af666d)) by [@vladfrangu](https://github.com/vladfrangu) +* `runs info` ([#657](https://github.com/apify/apify-cli/pull/657)) ([827767c](https://github.com/apify/apify-cli/commit/827767cfc988b7d587adceb825765e553deeed77)) by [@vladfrangu](https://github.com/vladfrangu) +* `actors build` ([#661](https://github.com/apify/apify-cli/pull/661)) ([4605cda](https://github.com/apify/apify-cli/commit/4605cda7f3a4f5a35160ba69bf4a454c889dd813)) by [@vladfrangu](https://github.com/vladfrangu) +* `actors pull` ([#662](https://github.com/apify/apify-cli/pull/662)) ([26d5cb3](https://github.com/apify/apify-cli/commit/26d5cb356fbb38a789e9b88f4d4b01468e38bd26)) by [@vladfrangu](https://github.com/vladfrangu) +* `actors call` ([#663](https://github.com/apify/apify-cli/pull/663)) ([a472300](https://github.com/apify/apify-cli/commit/a4723007e65bde8db6eb121a0dc38e2c7bc6caec)) by [@vladfrangu](https://github.com/vladfrangu) +* Check if cli was installed using volta when checking for updates ([#667](https://github.com/apify/apify-cli/pull/667)) ([aee0233](https://github.com/apify/apify-cli/commit/aee023336768e59fd4ff8d6c957f804d315e7bf3)) by [@vladfrangu](https://github.com/vladfrangu) +* `actors start` ([#669](https://github.com/apify/apify-cli/pull/669)) ([45956e2](https://github.com/apify/apify-cli/commit/45956e224305dd040b607d1fc3ff5cbbc8b28f32)) by [@vladfrangu](https://github.com/vladfrangu) +* `actors push` ([#671](https://github.com/apify/apify-cli/pull/671)) ([d77c531](https://github.com/apify/apify-cli/commit/d77c5314d4252a6bbf30718436dd84467aa35d7f)) by [@vladfrangu](https://github.com/vladfrangu) +* `actors ls` ([#675](https://github.com/apify/apify-cli/pull/675)) ([de258cb](https://github.com/apify/apify-cli/commit/de258cb8872857aa559afb4b16ed5a52f4fb2094)) by [@vladfrangu](https://github.com/vladfrangu) +* `key-value-stores get-value` ([#678](https://github.com/apify/apify-cli/pull/678)) ([67cfefe](https://github.com/apify/apify-cli/commit/67cfefef88fac220a1c959aaaecf3d051e482236)) by [@vladfrangu](https://github.com/vladfrangu) +* `datasets get-items` ([#679](https://github.com/apify/apify-cli/pull/679)) ([b521546](https://github.com/apify/apify-cli/commit/b521546df195bab7bedf5534167b6edae6a5e69e)) by [@vladfrangu](https://github.com/vladfrangu) +* `datasets` / `key-value-stores` commands ([#685](https://github.com/apify/apify-cli/pull/685)) ([c7d77e1](https://github.com/apify/apify-cli/commit/c7d77e1cec711edd9996cbb1249e489fbf3db547)) by [@vladfrangu](https://github.com/vladfrangu) +* Key-value-store commands ([#700](https://github.com/apify/apify-cli/pull/700)) ([eb8ff3b](https://github.com/apify/apify-cli/commit/eb8ff3b9c7f1319d0937543f7b0b97cb25d6390a)) by [@vladfrangu](https://github.com/vladfrangu) +* `actors info` ([#701](https://github.com/apify/apify-cli/pull/701)) ([0f4b3f0](https://github.com/apify/apify-cli/commit/0f4b3f08dd5937ca6664342c2510a9f4f3fa52f6)) by [@vladfrangu](https://github.com/vladfrangu) +* `datasets info` / `key-value-stores info` ([#726](https://github.com/apify/apify-cli/pull/726)) ([56e8ffa](https://github.com/apify/apify-cli/commit/56e8ffaf7a7e03dcdd73e4e5472b106b31b0f543)) by [@vladfrangu](https://github.com/vladfrangu) +* Rewrite command descriptions ([#727](https://github.com/apify/apify-cli/pull/727)) ([3eb21d6](https://github.com/apify/apify-cli/commit/3eb21d6ab74ca66f59da2432aadcb40f7ea42440)) by [@TC-MO](https://github.com/TC-MO) + +#### πŸ› Bug Fixes + +* Look for lowercase input schema in default paths ([#647](https://github.com/apify/apify-cli/pull/647)) ([68456e6](https://github.com/apify/apify-cli/commit/68456e63eee3c28e7c0ee7464a2cbc1a00ba9dfa)) by [@mvolfik](https://github.com/mvolfik) +* Emit warning if input.json is modified during run and prefilled with defaults ([#672](https://github.com/apify/apify-cli/pull/672)) ([8a6fd3f](https://github.com/apify/apify-cli/commit/8a6fd3f60523380041309db830a62f52cc60e4d4)) by [@vladfrangu](https://github.com/vladfrangu), closes [#670](https://github.com/apify/apify-cli/issues/670) +* Scrapy wrapping being broken due to ESM migration ([#686](https://github.com/apify/apify-cli/pull/686)) ([e2a7591](https://github.com/apify/apify-cli/commit/e2a7591070a284394643e8dbb03bc020939ff61f)) by [@vladfrangu](https://github.com/vladfrangu) +* **ci:** Make it work + publish with provenances ([#694](https://github.com/apify/apify-cli/pull/694)) ([e41ea72](https://github.com/apify/apify-cli/commit/e41ea728a9177dcec4ea73c25128cddebc00dd79)) by [@vladfrangu](https://github.com/vladfrangu) +* Handle stdin correctly from slower stdout emitting ([#704](https://github.com/apify/apify-cli/pull/704)) ([a5b53de](https://github.com/apify/apify-cli/commit/a5b53de480aad3caf80e1a9439cd5e64648fe312)) by [@vladfrangu](https://github.com/vladfrangu) +* Running commands with spaces on windows ([#715](https://github.com/apify/apify-cli/pull/715)) ([d1c207a](https://github.com/apify/apify-cli/commit/d1c207a703a6948e7b3a6cfe82c5cfa6a3b9222d)) by [@vladfrangu](https://github.com/vladfrangu), closes [#692](https://github.com/apify/apify-cli/issues/692) + +### [0.21.0](https://github.com/apify/apify-cli/releases/tag/v0.21.0) (2025-01-16) + +#### πŸš€ Features + +* Builds namespace ([#620](https://github.com/apify/apify-cli/pull/620)) ([6345162](https://github.com/apify/apify-cli/commit/6345162e44a00b404b4f95c2c80c2e437eff1d62)) by [@vladfrangu](https://github.com/vladfrangu) +* `runs ls` ([#640](https://github.com/apify/apify-cli/pull/640)) ([dd84d37](https://github.com/apify/apify-cli/commit/dd84d37c6ea89c64db712c7c94709f3181a7bd1f)) by [@vladfrangu](https://github.com/vladfrangu) +* `runs abort` ([#643](https://github.com/apify/apify-cli/pull/643)) ([96e5a34](https://github.com/apify/apify-cli/commit/96e5a3435cca08d87dc8d39659a7a6524f18be0e)) by [@vladfrangu](https://github.com/vladfrangu) +* `runs resurrect` ([#644](https://github.com/apify/apify-cli/pull/644)) ([7dbf4fb](https://github.com/apify/apify-cli/commit/7dbf4fb06c657246563c1c64e76ad83f283ea275)) by [@vladfrangu](https://github.com/vladfrangu) +* `runs log` ([#645](https://github.com/apify/apify-cli/pull/645)) ([dd6af5e](https://github.com/apify/apify-cli/commit/dd6af5ece79f4399fc5065483b650c71c61114cf)) by [@vladfrangu](https://github.com/vladfrangu) +* `runs rm` & `builds rm` & `actors rm` ([#648](https://github.com/apify/apify-cli/pull/648)) ([566f8c5](https://github.com/apify/apify-cli/commit/566f8c5d1482f150f4d61229524c7672c2af666d)) by [@vladfrangu](https://github.com/vladfrangu) +* `runs info` ([#657](https://github.com/apify/apify-cli/pull/657)) ([827767c](https://github.com/apify/apify-cli/commit/827767cfc988b7d587adceb825765e553deeed77)) by [@vladfrangu](https://github.com/vladfrangu) +* `actors build` ([#661](https://github.com/apify/apify-cli/pull/661)) ([4605cda](https://github.com/apify/apify-cli/commit/4605cda7f3a4f5a35160ba69bf4a454c889dd813)) by [@vladfrangu](https://github.com/vladfrangu) +* `actors pull` ([#662](https://github.com/apify/apify-cli/pull/662)) ([26d5cb3](https://github.com/apify/apify-cli/commit/26d5cb356fbb38a789e9b88f4d4b01468e38bd26)) by [@vladfrangu](https://github.com/vladfrangu) +* `actors call` ([#663](https://github.com/apify/apify-cli/pull/663)) ([a472300](https://github.com/apify/apify-cli/commit/a4723007e65bde8db6eb121a0dc38e2c7bc6caec)) by [@vladfrangu](https://github.com/vladfrangu) +* Check if cli was installed using volta when checking for updates ([#667](https://github.com/apify/apify-cli/pull/667)) ([aee0233](https://github.com/apify/apify-cli/commit/aee023336768e59fd4ff8d6c957f804d315e7bf3)) by [@vladfrangu](https://github.com/vladfrangu) +* `actors start` ([#669](https://github.com/apify/apify-cli/pull/669)) ([45956e2](https://github.com/apify/apify-cli/commit/45956e224305dd040b607d1fc3ff5cbbc8b28f32)) by [@vladfrangu](https://github.com/vladfrangu) +* `actors push` ([#671](https://github.com/apify/apify-cli/pull/671)) ([d77c531](https://github.com/apify/apify-cli/commit/d77c5314d4252a6bbf30718436dd84467aa35d7f)) by [@vladfrangu](https://github.com/vladfrangu) +* `actors ls` ([#675](https://github.com/apify/apify-cli/pull/675)) ([de258cb](https://github.com/apify/apify-cli/commit/de258cb8872857aa559afb4b16ed5a52f4fb2094)) by [@vladfrangu](https://github.com/vladfrangu) +* `key-value-stores get-value` ([#678](https://github.com/apify/apify-cli/pull/678)) ([67cfefe](https://github.com/apify/apify-cli/commit/67cfefef88fac220a1c959aaaecf3d051e482236)) by [@vladfrangu](https://github.com/vladfrangu) +* `datasets get-items` ([#679](https://github.com/apify/apify-cli/pull/679)) ([b521546](https://github.com/apify/apify-cli/commit/b521546df195bab7bedf5534167b6edae6a5e69e)) by [@vladfrangu](https://github.com/vladfrangu) +* `datasets` / `key-value-stores` commands ([#685](https://github.com/apify/apify-cli/pull/685)) ([c7d77e1](https://github.com/apify/apify-cli/commit/c7d77e1cec711edd9996cbb1249e489fbf3db547)) by [@vladfrangu](https://github.com/vladfrangu) +* Key-value-store commands ([#700](https://github.com/apify/apify-cli/pull/700)) ([eb8ff3b](https://github.com/apify/apify-cli/commit/eb8ff3b9c7f1319d0937543f7b0b97cb25d6390a)) by [@vladfrangu](https://github.com/vladfrangu) +* `actors info` ([#701](https://github.com/apify/apify-cli/pull/701)) ([0f4b3f0](https://github.com/apify/apify-cli/commit/0f4b3f08dd5937ca6664342c2510a9f4f3fa52f6)) by [@vladfrangu](https://github.com/vladfrangu) +* `datasets info` / `key-value-stores info` ([#726](https://github.com/apify/apify-cli/pull/726)) ([56e8ffa](https://github.com/apify/apify-cli/commit/56e8ffaf7a7e03dcdd73e4e5472b106b31b0f543)) by [@vladfrangu](https://github.com/vladfrangu) +* Rewrite command descriptions ([#727](https://github.com/apify/apify-cli/pull/727)) ([3eb21d6](https://github.com/apify/apify-cli/commit/3eb21d6ab74ca66f59da2432aadcb40f7ea42440)) by [@TC-MO](https://github.com/TC-MO) + +#### πŸ› Bug Fixes + +* Look for lowercase input schema in default paths ([#647](https://github.com/apify/apify-cli/pull/647)) ([68456e6](https://github.com/apify/apify-cli/commit/68456e63eee3c28e7c0ee7464a2cbc1a00ba9dfa)) by [@mvolfik](https://github.com/mvolfik) +* Emit warning if input.json is modified during run and prefilled with defaults ([#672](https://github.com/apify/apify-cli/pull/672)) ([8a6fd3f](https://github.com/apify/apify-cli/commit/8a6fd3f60523380041309db830a62f52cc60e4d4)) by [@vladfrangu](https://github.com/vladfrangu), closes [#670](https://github.com/apify/apify-cli/issues/670) +* Scrapy wrapping being broken due to ESM migration ([#686](https://github.com/apify/apify-cli/pull/686)) ([e2a7591](https://github.com/apify/apify-cli/commit/e2a7591070a284394643e8dbb03bc020939ff61f)) by [@vladfrangu](https://github.com/vladfrangu) +* **ci:** Make it work + publish with provenances ([#694](https://github.com/apify/apify-cli/pull/694)) ([e41ea72](https://github.com/apify/apify-cli/commit/e41ea728a9177dcec4ea73c25128cddebc00dd79)) by [@vladfrangu](https://github.com/vladfrangu) +* Handle stdin correctly from slower stdout emitting ([#704](https://github.com/apify/apify-cli/pull/704)) ([a5b53de](https://github.com/apify/apify-cli/commit/a5b53de480aad3caf80e1a9439cd5e64648fe312)) by [@vladfrangu](https://github.com/vladfrangu) +* Running commands with spaces on windows ([#715](https://github.com/apify/apify-cli/pull/715)) ([d1c207a](https://github.com/apify/apify-cli/commit/d1c207a703a6948e7b3a6cfe82c5cfa6a3b9222d)) by [@vladfrangu](https://github.com/vladfrangu), closes [#692](https://github.com/apify/apify-cli/issues/692) + +## 0.6.1 / 2020-05-18 + +* **BREAKING:** Templates are now fully decoupled from this project and + the [templates repository](https://github.com/apify/actor-templates) + serves as the single source of truth. Some templates were replaced + and others were renamed to better clarify their purpose. + +* **BREAKING:** Providing an invalid template in `apify.json` no longer + throws, but rather silently uses a reasonable default configuration. + This is to support regular changes to templates without breaking older + versions of the CLI. + + ## 0.5.3 / 2020-03-03 + +* Moved templates to separate repository + +* Fixed: creating `apify_storage` in root folder after `apify create` command + + ## 0.5.2 / 2020-01-22 + +* Added bot(dependabot.com) to check latest Apify SDK version in all templates + +* Updated apify package in all templates + +* Updated npm packages and fixed all npm audit issues + + ## 0.5.1 / 2019-12-19 + +* Added warning about outdated node.js version + +* Fixed infinite push, when the previous one was interrupted + +* Fixed calling public actors with `apify call` + +* `apify init` create empty INPUT.json file + + ## 0.5.0 / 2019-11-27 + +* Drop support for node 8 and 9 + +* Fix: Pass the --max-http-header-size=80000 to the nodeJs process + + ## 0.4.1 / 2019-10-02 + +* New actor template for Apify projects, you can create it with `apify create --template apify_project` + +* `apify vis` - Using improved schema validator + + ## 0.4.0 / 2019-09-23 + +* Breaking Change - `apify push`: Pushes source code as a "Multiple source files" in case source code is less that 3 MB + + ## 0.3.12 / 2019-09-18 + + Bug fixes: + +* `apify create`: Added validation for actor name + +* `apify init` skips creation of apify.json if already exists + +* `apify run -p` runs actor, if apify\_storage doesn't exist + +* Updated packages + +* Additional minor fixes + + ## 0.3.11 / 2019-07-26 + +* Updated packages + +* Updated Cheerio Crawler template + +* Updated Apify package version in all templates + + ## 0.3.10 / 2019-06-03 + +* Updated packages + + ## 0.3.9 / 2019-05-15 + +* Improved the templates and texts + + ## 0.3.8 / 2019-03-29 + +* Updated all templates regarding the last version of apify SDK. + + ## 0.3.7 / 2019-03-18 + +* Fixed templates to use Apify.getInput(), replaced deprecated function and options, + added debug fields, added .idea to .gitignore + +* Updated packages + +* Fixed bug: Users without username can use push/call command + + ## 0.3.6 / 2019-01-29 + +* Added command `apify vis` that validates actor input schema. + + ## 0.3.5 / 2019-01-25 + +* Upgraded to apify@0.11 in templates + + ## 0.3.3 / 2018-12-12 + +* Omitted CMD command in all templates Dockerfile. + + ## 0.3.2 / 2018-12-05 + +* Updated apify-client package. It fixed bug, when user can not push actor, whe he changed version in apify.json. + + ## 0.3.1 / 2018-11-29 + +* :tada: New commands to manage secret environment variables: `apify secrets:add`, `apify secrets:rm`. + +* New documentation how to set environment variable in `apify.json`, see [documentation](https://github.com/apify/apify-cli/blob/master/README.md#environment-variables). + +* **BREAKING CHANGES**: Simplified `apify.json` structure. It will be updated automatically before execution apify run and push command. + +* Command `apify create` now shows progress bar of npm install. + +* Small bugs fixes + + ## 0.2.7 / 2018-11-27 + +* Updated all templates to latest apify packages + + ## 0.2.6 / 2018-11-09 + +* Added warning if `apify run` reuse old state in storage + +* Fixed issues #70 #65 #68 + + ## 0.2.5 / 2018-10-31 + +* Updated NPM dependencies + +* Upgraded to apify-shared@0.1.6 + +* Fixed templates to use apify/actor-node-chrome Docker image instead of outdated apify/actor-node-puppeteer + + ## 0.2.3 / 2018-09-17 + +* Updated all templates to apify version 0.8.\* + +* Added template named hello\_word + + ## 0.2.1 / 2018-09-17 + +* **BREAKING CHANGES**: The local storage directories have been renamed and package.json files needs a new `start` command. + See [migration guide](/MIGRATIONS.md) for existing projects if you are upgrading from 0.1.\_ to 0.2.\_. + +* You can specified another file that main.js for `apify run` command using npm start script. + + ## 0.2.0 / 2018-09-12 + +* **BREAKING CHANGES**: Version 0.2.0 of Apify CLI supports only version 0.7.0 of API SDK or newer as management of environment variables + has been changed according to Apify SDK version 0.7.0. + +* Dropped support for Node 7 + + ## 0.1.18 / 2018-09-12 + +* Updated NPM dependencies, npm-shrinkwrap.json replaced with package-lock.json + +* Updated NPM dependencies in code templates + + ## 0.1.15 / 2018-07-23 + +* Rename act to actor + + ## 0.1.13 / 2018-07-12 + +* Add environment variables for enable live view for local actors. + + ## 0.1.12 / 2018-06-28 + +* From now `apify call` and `apify push` commands stream live logs from run and build to your terminal + +* Add options -p, --purge, --purge-dataset, --purge-key-value-store, --purge-queue in `apify run` to clean stores before runs actor locally + +* Add option -w, --wait-for-finish=wait-for-finish in `apify push` and `apify call` - command waits x seconds to finish run or build on Apify + +* Fixes #26, #33, #34, #36, #38, #39, #37, #35 + + ## 0.1.11 / 2018-05-30 + +* Use npm-shrinkwrap.json instead of package-lock.json for published module + +* Update template, where we using proxy + +* Fix #30 + + ## 0.1.9 / 2018-04-18 + +* apify run takes APIFY\_USER\_ID and APIFY\_TOKEN as environments variables, if client is logged locally + +* apify call takes input from default local key-value-store + +* Fix: duplicates new lines in log + + ## 0.1.8 / 2018-04-17 + +* Print warning if you have old version of cli + +* apify run - kills all sub processes for SIGINT signal (ctrl+c) - It kills all related browsers in apify run command, related issue: + https://github.com/apify/apify-js/issues/72 + + ## 0.1.7 / 2018-04-12 + +* Readme and templates updates + + ## 0.1.6 / 2018-04-11 + +* Add support for request queue + + ## 0.1.5 / 2018-04-09 + +* Works for windows + +* New command apify info + + ## 0.1.x / 2018-04-01 + +* The first public release + + ## 0.0.x / 2018-03-01 + +* Initial development, lot of new stuff diff --git a/website/versioned_docs/version-1.5/index.md b/website/versioned_docs/version-1.5/index.md new file mode 100644 index 000000000..c18eabb60 --- /dev/null +++ b/website/versioned_docs/version-1.5/index.md @@ -0,0 +1,36 @@ +--- +title: Apify CLI overview +description: An introduction to Apify CLI, a command-line interface for creating, developing, building, and running Apify Actors and managing the Apify cloud platform. +sidebar_label: Overview +--- + +Apify command-line interface (Apify CLI) helps you create, develop, build and run [Apify Actors](https://apify.com/actors), and manage the Apify platform from any computer. + +## What are Apify Actors? + +Actors are cloud programs that can perform arbitrary web scraping, automation or data processing job. They accept input, perform their job and generate output. + +## Why use Apify CLI? + +The Apify CLI enables you to develop Actors locally on your computer using your preferred tools: + +- Your favorite code editor +- Version control system +- Development tools and workflows + +This gives you full control over your development environment and makes it easier to work on complex projects. You can leverage the [Apify SDK](https://github.com/apify/apify-sdk-js) with all its powerful features, then push your Actor to the Apify platform for deployment when ready. + +:::note Actor development environment + +Actors run in Docker containers on the Apify platform. With an appropriate `Dockerfile`, you can build Actors in any programming language. We recommend JavaScript/Node.js and Python, for which we have the most comprehensive libraries and support. + +::: + +## Learn more + +Learn everything you need to use the Apify CLI effectively: + +- Learn how to [install](./installation.md) the CLI on your system +- Get started with your [first Actor project](./quick-start.md) +- See the complete [reference of all CLI commands](./reference.md) and options +- Find [solutions to common issues](./troubleshooting.md) diff --git a/website/versioned_docs/version-1.5/installation.md b/website/versioned_docs/version-1.5/installation.md new file mode 100644 index 000000000..77a3c88b1 --- /dev/null +++ b/website/versioned_docs/version-1.5/installation.md @@ -0,0 +1,87 @@ +--- +title: Installation +description: Learn how to install Apify CLI using installation scripts, Homebrew, or NPM. +--- + +Learn how to install Apify CLI using installation scripts, Homebrew, or NPM. + +--- + +## Installation scripts + +### MacOS / Linux + +```bash +curl -fsSL https://apify.com/install-cli.sh | bash +``` + +### Windows + +```powershell +irm https://apify.com/install-cli.ps1 | iex +``` + +:::tip No need for Node.js + +If you install Apify CLI using our installation scripts, you don't need Node.js. The scripts use [Bun](https://bun.sh/) to create a standalone executable file. + +This approach eliminates Node.js dependency management, which is useful for Python developers or users working in non-Node.js environments. + +::: + +## Homebrew + +```bash +brew install apify-cli +``` + +:::tip Homebrew and Node.js dependency + +When you install Apify CLI using Homebrew, it automatically installs Node.js as a dependency. If you already have Node.js installed through another method (e.g., `nvm`), this may create version conflicts. + +If you experience Node.js version conflicts, modify your `PATH` environment variable to prioritize your preferred Node.js installation over Homebrew's version. + +::: + +## NPM + +First, make sure you have [Node.js](https://nodejs.org) version 22 or higher with NPM installed on your computer: + +```bash showLineNumbers +node --version +npm --version +``` + +Install or upgrade Apify CLI by running: + +```bash +npm install -g apify-cli +``` + +:::tip Troubleshooting + +If you receive a permission error, read npm's [official guide](https://docs.npmjs.com/resolving-eacces-permissions-errors-when-installing-packages-globally) on installing packages globally. + +::: + +## Verify installation + +You can verify the installation process by running the following command: + +```bash +apify --version +``` + +The output should resemble the following (exact details like version or platform may vary): + +```bash +apify-cli/1.0.1 (0dfcfd8) running on darwin-arm64 with bun-1.2.19 (emulating node 24.3.0), installed via bundle +``` + +## Upgrading + +Upgrading Apify CLI is as simple as running the following command: + +```bash +apify upgrade +``` diff --git a/website/versioned_docs/version-1.5/integrating-scrapy.md b/website/versioned_docs/version-1.5/integrating-scrapy.md new file mode 100644 index 000000000..9fcbe2259 --- /dev/null +++ b/website/versioned_docs/version-1.5/integrating-scrapy.md @@ -0,0 +1,188 @@ +--- +title: Integrating Scrapy projects +description: Learn how to run Scrapy projects as Apify Actors and deploy them on the Apify platform. +sidebar_label: Integrating Scrapy projects +--- + +[Scrapy](https://scrapy.org/) is a widely used open-source web scraping framework for Python. Scrapy projects can now be executed on the Apify platform using our dedicated wrapping tool. This tool allows users to transform their Scrapy projects into [Apify Actors](https://docs.apify.com/platform/actors) with just a few simple commands. + +## Prerequisites + +Before you begin, make sure you have the Apify CLI installed on your system. If you haven't installed it yet, follow the [installation guide](./installation.md). + +## Actorization of your existing Scrapy spider + +Assuming your Scrapy project is set up, navigate to the project root where the `scrapy.cfg` file is located. + +```bash +cd your_scraper +``` + +Verify the directory contents to ensure the correct location. + +```bash showLineNumbers +$ ls -R +.: +your_scraper README.md requirements.txt scrapy.cfg + +./your_scraper: +__init__.py items.py __main__.py main.py pipelines.py settings.py spiders + +./your_scraper/spiders: +your_spider.py __init__.py +``` + +To convert your Scrapy project into an Apify Actor, initiate the wrapping process by executing the following command: + +```bash +apify init +``` + +The script will prompt you with a series of questions. Upon completion, the output might resemble the following: + +```bash showLineNumbers +Info: The current directory looks like a Scrapy project. Using automatic project wrapping. +? Enter the Scrapy BOT_NAME (see settings.py): books_scraper +? What folder are the Scrapy spider modules stored in? (see SPIDER_MODULES in settings.py): books_scraper.spiders +? Pick the Scrapy spider you want to wrap: BookSpider (/home/path/to/actor-scrapy-books-example/books_scraper/spiders/book.py) +Info: Downloading the latest Scrapy wrapper template... +Info: Wrapping the Scrapy project... +Success: The Scrapy project has been wrapped successfully. +``` + +For example, here is a [source code](https://github.com/apify/actor-scrapy-books-example) of an actorized Scrapy project, and [here](https://apify.com/vdusek/scrapy-books-example) the corresponding Actor in Apify Store. + +### Run the Actor locally + +Create a Python virtual environment by running: + +```bash +python -m virtualenv .venv +``` + +Activate the virtual environment: + +```bash +source .venv/bin/activate +``` + +Install Python dependencies using the provided requirements file named `requirements_apify.txt`. Ensure these requirements are installed before executing your project as an Apify Actor locally. You can put your own dependencies there as well. + +```bash +pip install -r requirements-apify.txt [-r requirements.txt] +``` + +Finally execute the Apify Actor. + +```bash +apify run [--purge] +``` + +If [ActorDatasetPushPipeline](https://github.com/apify/apify-sdk-python/blob/master/src/apify/scrapy/pipelines.py) is configured, the Actor's output will be stored in the `storage/datasets/default/` directory. + +### Run the scraper as Scrapy project + +The project remains executable as a Scrapy project. + +```bash +scrapy crawl your_spider -o books.json +``` + +## Deploy on Apify + +### Log in to Apify + +You will need to provide your [Apify API Token](https://console.apify.com/settings/integrations) to complete this action. + +```bash +apify login +``` + +### Deploy your Actor + +This command will deploy and build the Actor on the Apify platform. You can find your newly created Actor under [Actors -> My Actors](https://console.apify.com/actors?tab=my). + +```bash +apify push +``` + +## What the wrapping process does + +The initialization command enhances your project by adding necessary files and updating some of them while preserving its functionality as a typical Scrapy project. The additional requirements file, named `requirements_apify.txt`, includes the Apify Python SDK and other essential requirements. The `.actor/` directory contains basic configuration of your Actor. We provide two new Python files [main.py](https://github.com/apify/actor-templates/blob/master/templates/python-scrapy/src/main.py) and [\_\_main\_\_.py](https://github.com/apify/actor-templates/blob/master/templates/python-scrapy/src/__main__.py), where we encapsulate the Scrapy project within an Actor. We also import and use there a few Scrapy components from our [Python SDK](https://github.com/apify/apify-sdk-python/tree/master/src/apify/scrapy). These components facilitate the integration of the Scrapy projects with the Apify platform. Further details about these components are provided in the following subsections. + +### Scheduler + +The [scheduler](https://docs.scrapy.org/en/latest/topics/scheduler.html) is a core component of Scrapy responsible for receiving and providing requests to be processed. To leverage the [Apify request queue](https://docs.apify.com/platform/storage/request-queue) for storing requests, a custom scheduler becomes necessary. Fortunately, Scrapy is a modular framework, allowing the creation of custom components. As a result, we have implemented the [ApifyScheduler](https://github.com/apify/apify-sdk-python/blob/master/src/apify/scrapy/scheduler.py). When using the Apify CLI wrapping tool, the scheduler is configured in the [src/main.py](https://github.com/apify/actor-templates/blob/master/templates/python-scrapy/src/main.py) file of your Actor. + +### Dataset push pipeline + +[Item pipelines](https://docs.scrapy.org/en/latest/topics/item-pipeline.html) are used for the processing of the results produced by your spiders. To handle the transmission of result data to the [Apify dataset](https://docs.apify.com/platform/storage/dataset), we have implemented the [ActorDatasetPushPipeline](https://github.com/apify/apify-sdk-python/blob/master/src/apify/scrapy/pipelines.py). When using the Apify CLI wrapping tool, the pipeline is configured in the [src/main.py](https://github.com/apify/actor-templates/blob/master/templates/python-scrapy/src/main.py) file of your Actor. It is assigned the highest integer value (1000), ensuring its execution as the final step in the pipeline sequence. + +### Retry middleware + +[Downloader middlewares](https://docs.scrapy.org/en/latest/topics/downloader-middleware.html) are a way how to hook into Scrapy's request/response processing. Scrapy comes with various default middlewares, including the [RetryMiddleware](https://docs.scrapy.org/en/latest/topics/downloader-middleware.html#module-scrapy.downloadermiddlewares.retry), designed to handle retries for requests that may have failed due to temporary issues. When integrating with the [Apify request queue](https://docs.apify.com/platform/storage/request-queue), it becomes necessary to enhance this middleware to facilitate communication with the request queue marking the requests either as handled or ready for a retry. When using the Apify CLI wrapping tool, the default `RetryMiddleware` is disabled, and [ApifyRetryMiddleware](https://github.com/apify/apify-sdk-python/blob/master/src/apify/scrapy/middlewares/apify_retry.py) takes its place. Configuration for the middlewares is established in the [src/main.py](https://github.com/apify/actor-templates/blob/master/templates/python-scrapy/src/main.py) file of your Actor. + +### HTTP proxy middleware + +Another default Scrapy [downloader middleware](https://docs.scrapy.org/en/latest/topics/downloader-middleware.html) that requires replacement is [HttpProxyMiddleware](https://docs.scrapy.org/en/latest/topics/downloader-middleware.html#module-scrapy.downloadermiddlewares.httpproxy). To utilize the use of proxies managed through the Apify [ProxyConfiguration](https://github.com/apify/apify-sdk-python/blob/master/src/apify/proxy_configuration.py), we provide [ApifyHttpProxyMiddleware](https://github.com/apify/apify-sdk-python/blob/master/src/apify/scrapy/middlewares/apify_proxy.py). When using the Apify CLI wrapping tool, the default `HttpProxyMiddleware` is disabled, and [ApifyHttpProxyMiddleware](https://github.com/apify/apify-sdk-python/blob/master/src/apify/scrapy/middlewares/apify_proxy.py) takes its place. Additionally, inspect the [.actor/input_schema.json](https://github.com/apify/actor-templates/blob/master/templates/python-scrapy/.actor/input_schema.json) file, where proxy configuration is specified as an input property for your Actor. The processing of this input is carried out together with the middleware configuration in [src/main.py](https://github.com/apify/actor-templates/blob/master/templates/python-scrapy/src/main.py). + +## Known limitations + +There are some known limitations of running the Scrapy projects on Apify platform we are aware of. + +### Asynchronous code in spiders and other components + +Scrapy asynchronous execution is based on the [Twisted](https://twisted.org/) library, not the +[AsyncIO](https://docs.python.org/3/library/asyncio.html), which brings some complications on the table. + +Due to the asynchronous nature of the Actors, all of their code is executed as a coroutine inside the `asyncio.run`. +In order to execute Scrapy code inside an Actor, following the section +[Run Scrapy from a script](https://docs.scrapy.org/en/latest/topics/practices.html?highlight=CrawlerProcess#run-scrapy-from-a-script) +from the official Scrapy documentation, we need to invoke a +[`CrawlProcess.start`](https://github.com/scrapy/scrapy/blob/2.11.0/scrapy/crawler.py#L393:L427) +method. This method triggers Twisted's event loop, also known as a reactor. +Consequently, Twisted's event loop is executed within AsyncIO's event loop. +On top of that, when employing AsyncIO code in spiders or other components, it necessitates the creation of a new +AsyncIO event loop, within which the coroutines from these components are executed. This means there is +an execution of the AsyncIO event loop inside the Twisted event loop inside the AsyncIO event loop. + +We have resolved this issue by leveraging the [nest-asyncio](https://pypi.org/project/nest-asyncio/) library, +enabling the execution of nested AsyncIO event loops. For executing a coroutine within a spider or other component, +it is recommended to use Apify's instance of the nested event loop. Refer to the code example below or derive +inspiration from Apify's Scrapy components, such as the +[ApifyScheduler](https://github.com/apify/apify-sdk-python/blob/v1.5.0/src/apify/scrapy/scheduler.py#L114). + +```python +from apify.scrapy.utils import nested_event_loop +... + +# Coroutine execution inside a spider +nested_event_loop.run_until_complete(my_coroutine()) +``` + +### More spiders per Actor + +It is recommended to execute only one Scrapy spider per Apify Actor. + +Mapping more Scrapy spiders to a single Apify Actor does not make much sense. We would have to create a separate +instace of the [request queue](https://docs.apify.com/platform/storage/request-queue) for every spider. +Also, every spider can produce a different output resulting in a mess in an output +[dataset](https://docs.apify.com/platform/storage/dataset). A solution for this could be to store an output +of every spider to a different [key-value store](https://docs.apify.com/platform/storage/key-value-store). However, +a much more simple solution to this problem would be to just have a single spider per Actor. + +If you want to share common Scrapy components (middlewares, item pipelines, ...) among more spiders (Actors), you +can use a dedicated Python package containing your components and install it to your Actors environment. The +other solution to this problem could be to have more spiders per Actor, but keep only one spider run per Actor. +What spider is going to be executed in an Actor run can be specified in the +[input schema](https://docs.apify.com/academy/deploying-your-code/input-schema). + +## Additional links + +- [Scrapy Books Example Actor](https://apify.com/vdusek/scrapy-books-example) +- [Python Actor Scrapy template](https://apify.com/templates/python-scrapy) +- [Apify SDK for Python](https://docs.apify.com/sdk/python) +- [Apify platform](https://docs.apify.com/platform) +- [Join our developer community on Discord](https://discord.com/invite/jyEM2PRvMU) + +> We welcome any feedback! Please feel free to contact us at [python@apify.com](mailto:python@apify.com). Thank you for your valuable input. diff --git a/website/versioned_docs/version-1.5/quick-start.md b/website/versioned_docs/version-1.5/quick-start.md new file mode 100644 index 000000000..72a4d7f5b --- /dev/null +++ b/website/versioned_docs/version-1.5/quick-start.md @@ -0,0 +1,84 @@ +--- +title: Quick start +description: Learn how to create, run, and manage Actors using Apify CLI. +--- + +Learn how to create, run, and manage Actors using Apify CLI. + +## Prerequisites + +Before you begin, make sure you have the Apify CLI installed on your system. If you haven't installed it yet, follow the [installation guide](./installation.md). + +## Step 1: Create your Actor + +Run the following command in your terminal. It will guide you step by step through the creation process. + +```bash +apify create +``` + +:::info Explore Actor templates + +The Apify CLI will prompt you to choose a template. Browse the [full list of templates](https://apify.com/templates) to find the best fit for your Actor. + +::: + +## Step 2: Run your Actor + +Once the Actor is initialized, you can run it: + +```bash +apify run +``` + +You'll see output similar to this in your terminal: + +```bash +INFO System info {"apifyVersion":"3.4.3","apifyClientVersion":"2.12.6","crawleeVersion":"3.13.10","osType":"Darwin","nodeVersion":"v22.17.0"} +Extracted heading { level: 'h1', text: 'Your full‑stack platform for webΒ scraping' } +Extracted heading { level: 'h3', text: 'TikTok Scraper' } +Extracted heading { level: 'h3', text: 'Google Maps Scraper' } +Extracted heading { level: 'h3', text: 'Instagram Scraper' } +``` + +## Step 3: Push your Actor + +Once you are ready, you can push your Actor to the Apify platform, where you can schedule runs, or make the Actor public for other developers. + +### Login to Apify Console + +```bash +apify login +``` + +:::note Create an Apify account + +Before you can interact with the Apify Console, [create an Apify account](https://console.apify.com/). +When you run `apify login`, you can choose one of the following methods: + +- Sign in via the Apify Console in your browser β€” recommended. +- Provide an [Apify API token](https://console.apify.com/settings/integrations) β€” alternative method. + +The interactive prompt will guide you through either option. + +::: + +### Push to Apify Console + +```bash +apify push +``` + +## Step 4: Call your Actor (optional) + +You can run your Actor on the Apify platform. In the following example, the command runs `apify/hello-world` on the Apify platform. + +```bash +apify call apify/hello-world +``` + +## Next steps + +- Check the [command reference](./reference.md) for more information about individual commands. +- If you have a problem with the Apify CLI, check the [troubleshooting](./troubleshooting.md) guide. +- Learn more about [Actors](https://docs.apify.com/platform/actors). diff --git a/website/versioned_docs/version-1.5/reference.md b/website/versioned_docs/version-1.5/reference.md new file mode 100644 index 000000000..5f9471557 --- /dev/null +++ b/website/versioned_docs/version-1.5/reference.md @@ -0,0 +1,1560 @@ +--- +title: Command reference +sidebar_label: Command reference +toc_max_heading_level: 5 +--- + +The Apify CLI provides tools for managing your Apify projects and resources from the command line. Use these commands to develop Actors locally, deploy them to Apify platform, manage storage, orchestrate runs, and handle account configuration. + +This reference guide documents available commands, their options, and common usage patterns, to efficiently work with Apify platform. + +### General + +The general commands provide basic functionality for getting help and information about the Apify CLI. + + + +##### `apify help` + +```sh +DESCRIPTION + Prints out help about a command, or all available commands. + +USAGE + $ apify help [commandString] + +ARGUMENTS + commandString The command to get help for. +``` + +##### `apify upgrade` + +```sh +DESCRIPTION + Checks that installed Apify CLI version is up to date. + +USAGE + $ apify upgrade [-f] [--version ] + +FLAGS + -f, --force [DEPRECATED] This flag is now + ignored, as running the command manually will always check + for the latest version. + --version= The version of the CLI to upgrade to. If + not provided, the latest version will be used. +``` + +##### `apify api` + +```sh +DESCRIPTION + Makes an authenticated HTTP request to the Apify API and prints the response. + The endpoint can be a relative path (e.g. "acts", "v2/acts", or "/v2/acts"); + the "v2/" prefix is added automatically if omitted. + + Use --list-endpoints to see all available API endpoints. + +USAGE + $ apify api [methodOrEndpoint] [endpoint] [-d ] [-H ] + [-l] [-X GET|POST|PUT|PATCH|DELETE] [-p ] + +ARGUMENTS + methodOrEndpoint The API endpoint path (e.g. "acts", + "v2/acts", "/v2/users/me"), or an HTTP method followed by the + endpoint (e.g. "GET /v2/users/me"). + endpoint The API endpoint path when the first + argument is an HTTP method. + +FLAGS + -d, --body= The request body (JSON string). Use + "-" to read from stdin. + -H, --header= Additional HTTP header(s). Pass a + single "key:value" string, or a JSON object like '{"X-Foo": + "bar", "X-Baz": "qux"}' to send multiple headers. The flag + can only be used once; use the JSON form for multiple + headers. + -l, --list-endpoints List all available Apify API + endpoints. + -X, --method=