Skip to content

README.md

Latest commit

 

History

History
29 lines (18 loc) · 1.35 KB

README.md

File metadata and controls

29 lines (18 loc) · 1.35 KB
Raw

Reddit Scraper

Scrape Reddit utilising Decodo's Web Scraping API

Dependencies

BeautifulSoup

Authentication

Once you have an active Web Scraping API subscription, you can try sending a request right from the dashboard Web Scraping API > API playground method tab simply by clicking on Send Request. You will also see an example of a curl request generated on the right.

This Pyhton code example uses Base64 encoded user:pass authentication.

Parser type Example location Download
HTML to JSON reddit_python_scraper.py curl https://raw.githubusercontent.com/Decodo/reddit-python-scraper/blob/main/reddit_python_scraper.py > reddit_python_scraper.py

HTML to JSON

This Python script extracts Subreddit details, post data, and comments straight from the HTML of Reddit post pages and saves them to a JSON file.