Skip to content

amanbind898/Full-Stack-Scraper-Challenge

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

1 Commit
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

🧠 Full Stack Scraper Challenge

A full-stack application built with Python (Flask) and Vue.js, which scrapes product card data from the Croma Televisions & Accessories page, stores the structured data in Redis, and renders it dynamically on the frontend.

πŸš€ This project was developed as part of a real-world full-stack engineering challenge.


πŸ›  Tech Stack

  • Backend: Python, Flask, Redis, BeautifulSoup, requests
  • Frontend: Vue.js, Axios, CSS Grid
  • Others: Flask-CORS, Redis CLI, npm, Docker (optional)

πŸ“¦ Features

  • Scrapes <head> and header HTML elements, along with all product cards from Croma
  • Stores cleaned product data as JSON in Redis (scraped_content)
  • Flask API exposes two endpoints:
    • /scraped-content: Serves all scraped data
    • /products: Serves product list only
  • Vue.js SPA fetches and displays product cards dynamically
  • Reusable Vue components and responsive layout with CSS Grid

πŸ”§ Setup Instructions

βœ… Prerequisites

  • Python 3.8+
  • Node.js 14+
  • Redis server running on localhost:6379 (use Docker or local installation)

πŸ§ͺ Backend Setup

# Create and activate virtual environment
python -m venv venv
source venv/bin/activate   # Linux/macOS
venv\Scripts\activate      # Windows

# Navigate and install dependencies
cd backend
pip install -r requirements.txt

# Run the scraper to populate Redis
python scraper.py

# Start the Flask API
python app.py

🌐 Frontend Setup

# In a new terminal
cd frontend

# Install dependencies
npm install

# Run the development server
npm run serve

πŸ“ Folder Structure

.
β”œβ”€β”€ backend/
β”‚   β”œβ”€β”€ app.py              # Flask API
β”‚   β”œβ”€β”€ scraper.py          # BeautifulSoup scraper
β”‚   └── requirements.txt
β”œβ”€β”€ frontend/
β”‚   β”œβ”€β”€ src/
β”‚   β”‚   β”œβ”€β”€ components/
β”‚   β”‚   β”‚   └── Product.vue
β”‚   β”‚   └── App.vue
β”‚   └── package.json
└── README.md

βœ… Tasks Completed

  • Implemented scraper.py to extract <head> and product details
  • Stored JSON data in Redis with proper serialization
  • Built /scraped-content and /products endpoints using Flask + CORS
  • Created Vue frontend with dynamic product grid
  • Used Axios and reusable components for rendering

πŸ§ͺ Example Output

JSON stored in Redis under key scraped_content:

{
  "head": "<title>Televisions - Croma</title>",
  "header": "<div class='bs-header'>...</div>",
  "products": [
    {
      "id": "12345",
      "title": "Samsung 43\" UHD Smart TV",
      "link": "https://www.croma.com/...",
      "price": "β‚Ή34,999",
      "original_price": "β‚Ή42,000",
      "discount": "16%",
      "tags": ["Best Seller", "4K"]
    },
    ...
  ]
}

πŸ§‘β€πŸ’» Author

Aman Kumar Bind
πŸ“§ amanbind898@gmail.com
πŸ”— LinkedIn | GitHub


πŸ“„ License

This project is licensed under the MIT License.

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published