Skip to content

Replace Apify with free JobSpy library#2

Open
Aryan-Pardeshi wants to merge 3 commits into
mainfrom
replace-apify-with-jobspy
Open

Replace Apify with free JobSpy library#2
Aryan-Pardeshi wants to merge 3 commits into
mainfrom
replace-apify-with-jobspy

Conversation

@Aryan-Pardeshi

@Aryan-Pardeshi Aryan-Pardeshi commented May 26, 2026 •
edited
Loading

Copy link
Copy Markdown
Owner

Changes since last PR update

MCP Server (mcp_server.py)

  • Renamed etchlinkedin → search_jobs_tool (more accurate, searches multiple sites)
  • Added search_jobs_broad_tool — unfiltered second-pass for broader results
  • Added list_supported_sites — returns available job boards
  • Structured logging with log levels
  • Try/except error handling on all tools

Streamlit UI (main.py)

  • Sidebar with app info, how-it-works guide, and tech stack summary
  • Job result cards with color-coded source badges (LinkedIn=blue, Indeed, Google)
  • Expandable job descriptions inline
  • Stats dashboard at a glance (total jobs, remote count, salary ranges, top companies)
  • CSV export button to download results
  • Multi-select for choosing which job boards to search
  • Tab layout (search + how-it-works info)
  • Better empty states, error handling, and user feedback

Job API (src/job_api.py)

  • Renamed ind_job → search_jobs with full parameter set
  • Added search_jobs_broad for unfiltered searches
  • Added get_job_stats — computes metrics from results
  • Added SUPPORTED_SITES constant
  • Reusable parameter mapping dictionaries
  • NaN handling in DataFrame-to-dict conversion

Agent (src/agent.py)

  • Updated SYSTEM_PROMPT for new MCP tool names
  • Structured logging
  • Better error handling with user-facing error messages

README

  • Comprehensive rewrite documenting all new features
  • Updated architecture and project structure tree
  • Added JobSpy data schema reference

Closes #1

@Aryan-Pardeshi
Replace Apify with free JobSpy library
ba8c433 
- Swap apify-client for python-jobspy in src/job_api.py
  (scrape_linkedin -> scrape_jobs with LinkedIn, Indeed, Google)
- Map Apify params (workType, experienceLevel) to JobSpy equivalents
  (is_remote, job_type)
- Remove APIFY_TOKEN from .env and .env.example
- Update pyproject.toml - remove apify-client dependency
- Update README with JobSpy setup instructions
- Update main.py spinner text

Closes #1
@Aryan-Pardeshi Aryan-Pardeshi mentioned this pull request May 26, 2026
Closed
chatgpt-codex-connector[bot]
chatgpt-codex-connector Bot reviewed May 26, 2026
View reviewed changes

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ba8c433445

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/job_api.py Outdated
Comment on lines +20 to +22
hours_old=720,
job_type=jt,
is_remote=is_remote,

@chatgpt-codex-connector chatgpt-codex-connector Bot May 26, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Avoid incompatible Indeed filter combinations

This call always passes hours_old together with is_remote (and for internships also job_type), but JobSpy treats those filters as mutually exclusive for Indeed; as a result, Indeed queries can fail or ignore part of the filter set. In practice this means one of the advertised sources may return empty/incorrect results for normal searches, especially remote/internship requests.

Useful? React with 👍 / 👎.

Comment thread src/job_api.py Outdated
Comment on lines +8 to +9
job_type_map = {"1": "internship"}
jt = job_type_map.get(experience_level, None)

@chatgpt-codex-connector chatgpt-codex-connector Bot May 26, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve experience filtering for non-intern roles

Only experience level "1" is mapped, so levels "2"–"5" become None and are no longer used to constrain results. This is a functional regression from the previous implementation: users selecting Entry, Associate, Mid-Senior, or Director will get unfiltered mixes of seniority, which hurts recommendation quality.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@chatgpt-codex-connector chatgpt-codex-connector[bot] chatgpt-codex-connector[bot] left review comments

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Replace Apify with free JobSpy library

1 participant

@Aryan-Pardeshi