Vulnerability Scanner API

This Go service scans a velancio/vulnerability_scans/ repo for JSON CVE reports, stores them in an SQLite database, and provides a query API to retrieve vulnerabilities based on severity.

---

Table of Contents

1. Features
2. Prerequisites
3. Installation
4. Running the Service
5. Dockerfile
6. API Usage
   • Scan API (POST /scan)
   • Query API (POST /query)
7. Running Tests
8. Project Structure
9. Environment Variables
10. Contributing
11. License
12. Authoe

---

Features

• Scan API (/scan): Fetches JSON files from a GitHub repository and stores vulnerability data.
• Query API (/query): Retrieves stored vulnerabilities based on severity.
• SQLite Database: Persistent storage for scanned vulnerabilities.
• Parallel File Processing: Scans multiple files concurrently.
• Error Handling & Retries: Handles API failures with automatic retries.
• Docker Support: Runs inside a Docker container.
• Unit Tests with High Coverage: Ensures code reliability.

---

Prerequisites

Ensure the following dependencies are installed:

• Go 1.21+
• Docker
• Git

---

Installation

1. Clone the Repository:

   git clone https://github.com/soubhi/CVEScanner.git
   cd CVEScanner
   

2. Install Dependencies:

   go mod tidy
   

---

Running the Service

1. Run Locally (Without Docker)

go run main.go

The service will start on http://localhost:8081.

2. Run with Docker

1. Build the Docker Image:

   docker build -t vuln-scanner .
   

2. Run the Container:

   docker run -p 8081:8081 vuln-scanner
   

Now your service should be up and running at http://localhost:8081.

---

Dockerfile

Below is a production-ready Dockerfile that installs all necessary dependencies to run this scanner (including CA certificates, tzdata, and libsqlite3-dev for SQLite support).

FROM golang:1.21 AS builder

# Create and switch to the /app directory
WORKDIR /app

# Copy go.mod and go.sum first for caching
COPY go.mod go.sum ./
RUN go mod tidy

# Copy the entire project
COPY . .

# Build the Go application
RUN CGO_ENABLED=1 GOOS=linux GOARCH=amd64 go build -o server main.go

# Final minimal image
FROM ubuntu:22.04

# Install required libraries
RUN apt-get update && apt-get install -y \
    ca-certificates \
    tzdata \
    libsqlite3-dev && \
    rm -rf /var/lib/apt/lists/*

WORKDIR /root/

# Copy the compiled binary from the builder stage
COPY --from=builder /app/server .

# Expose port 8081
EXPOSE 8081

# Run the server
CMD ["./server"]

---

API Usage

1. Scan API (POST /scan)

Fetches JSON vulnerability files from a GitHub repository and stores them.

• Example Request:

  curl -X POST http://localhost:8081/scan \
       -H "Content-Type: application/json" \
       -d '{
             "repo": "velancio/vulnerability_scans",
             "files": ["vulnscan1.json", "vulnscan2.json"]
           }'
  

• Example Response:

  {
    "message": "Scan completed successfully",
    "repo": "velancio/vulnerability_scans",
    "files": ["vulnscan1.json", "vulnscan2.json"],
    "total_vulnerabilities": 10
  }
  

---

2. Query API (POST /query)

Retrieves vulnerabilities matching the given severity.

• Example Request:

  curl -X POST http://localhost:8081/query \
       -H "Content-Type: application/json" \
       -d '{
             "filters": {
               "severity": "HIGH"
             }
           }'
  

• Example Response:

  [
    {
      "id": "CVE-2024-1234",
      "severity": "HIGH",
      "cvss": 8.5,
      "status": "fixed",
      "package_name": "openssl",
      "current_version": "1.1.1t-r0",
      "fixed_version": "1.1.1u-r0",
      "description": "Buffer overflow vulnerability in OpenSSL",
      "published_date": "2024-01-15T00:00:00Z",
      "link": "https://nvd.nist.gov/vuln/detail/CVE-2024-1234",
      "risk_factors": [
        "Remote Code Execution",
        "High CVSS Score",
        "Public Exploit Available"
      ]
    }
  ]
  

---

Running Tests

1. Run All Tests (with coverage):

   go test ./... -cover
   

2. Run Specific Test File:

   go test ./services/scan_test.go
   

3. Check Coverage:

   go test -coverprofile=coverage.out ./...
   go tool cover -func=coverage.out
   # Generate an HTML coverage report
   go tool cover -html=coverage.out -o coverage.html
   

---

Project Structure

vuln-scanner
├── api
│   ├── scan.go         # Handles /scan API
│   ├── query.go        # Handles /query API
│   └── route.go                # Defines routes
├── database
│   ├── database.go             # SQLite setup & queries
├── services
│   ├── scan.go                 # Fetch & store vulnerabilities
│   ├── query.go                # Query vulnerabilities
├── main.go                     # Entry point
├── Dockerfile                  # Docker setup
├── go.mod                      # Go module dependencies
├── go.sum                      # Go dependency versions
└── README.md                   # This documentation

---

Environment Variables

You can add a .env file (optional) for environment-specific configs:

PORT=8081
DB_FILE=./vulnerabilities.db
GITHUB_TOKEN=your_github_personal_access_token

To load environment variables, run:

source .env
go run main.go

---

Contributing

1. Fork this repository
2. Create a feature branch
3. Commit your changes
4. Open a Pull Request

---

License

MIT License

---

📌 Author

Soubhagya Akkena
📧 soubhagyancsu@gmail.com
🌐 https://www.linkedin.com/in/soubhgya-akkena

---

Enjoy scanning for vulnerabilities! If you have any questions or need further assistance, please open an issue or reach out to the maintainer. Thank you for using Vulnerability Scanner API.