Skip to content

Feature: Batch saved-list scraping via CLI and library #66

Description

@MozzamShahid

File / lines: src/gmaps_scraper/scraper.py (scrape_saved_list); src/gmaps_scraper/cli.py (lines 495-498)

Problem: The saved-list workflow is strictly single-URL. scrape_saved_list() takes one URL, and the CLI errors if len(args.urls) != 1 and rejects --input for --kind list. Users with many lists must write their own wrapper, while the place workflow already supports batch input, session reuse, staggered starts, and retries.

Suggested fix:

  1. Add scrape_saved_lists(urls, *, max_concurrency=1, stagger_ms=0, ...) that reuses HttpSessionConfig / BrowserSessionConfig across calls (mirroring scrape_places).
  2. In the CLI, accept multiple URLs or --input when --kind list.
  3. Emit a {"results": [...]} envelope like place batch mode.
  4. Add tests for batch list scraping and input file handling.

Example:

printf %s\n LIST_URL_1 LIST_URL_2 | uv run gmaps-scraper --kind list --input -

Difficulty: medium
Impact: Medium — expands a core workflow and improves parity with place scraping.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions