Skip to content

README.md

Latest commit

 

History

History
186 lines (135 loc) · 4.56 KB

README.md

File metadata and controls

186 lines (135 loc) · 4.56 KB

LinkTracker

.NET Docker Image CI License Release .NET 9 PostgreSQL Docker Tests Platform

LinkTracker is a .NET 9 microservice application for tracking GitHub repositories and StackOverflow questions. The Bot service handles Telegram interaction, and the Scrapper service stores subscriptions, checks links on a schedule, and sends update notifications back to the Bot.

Project Structure

src/LinkTracker.Bot       Telegram bot HTTP service
src/LinkTracker.Scrapper  Subscription storage and scheduled update checker
src/LinkTracker.Shared    Shared DTOs
migrations/               SQL migrations applied by Scrapper on startup

Prerequisites

  • .NET 9 SDK
  • Docker Desktop
  • Telegram bot token from BotFather

Configuration

Create .env in the repository root:

TELEGRAM_BOT_TOKEN=your-telegram-bot-token

POSTGRES_DB=linktracker
POSTGRES_USER=linktracker
POSTGRES_PASSWORD=linktracker

.env is ignored by git. Use .env.example as the template.

Scrapper database settings are in src/LinkTracker.Scrapper/appsettings.json:

"Database": {
  "AccessType": "SQL",
  "ConnectionString": "Host=localhost;Port=5433;Database=linktracker;Username=linktracker;Password=linktracker",
  "RunMigrations": true
}

AccessType can be:

SQL  raw SQL repositories via Npgsql
ORM  EF Core repositories

Scheduler settings:

"Scrapper": {
  "CheckIntervalSeconds": 30,
  "BatchSize": 100,
  "Parallelism": 4,
  "GitHubBaseUrl": "https://api.github.com/",
  "StackOverflowBaseUrl": "https://api.stackexchange.com/2.3/"
}

BatchSize is clamped by the application to 50..500. Parallelism controls how many links are processed concurrently.

Run With Docker Compose

Run all services:

docker compose up --build

Endpoints:

Bot API:       http://localhost:5100
Scrapper API: http://localhost:5000
PostgreSQL:   localhost:5433

Inside Docker, Scrapper connects to PostgreSQL by service name:

Host=postgres;Port=5432

Run From IDE

Start PostgreSQL first:

docker compose up postgres -d

Then run the services from IDE or terminal:

dotnet run --project src\LinkTracker.Scrapper
dotnet run --project src\LinkTracker.Bot

Scrapper applies SQL migrations from migrations/ automatically when Database:RunMigrations is true.

Useful Manual Checks

List database tables:

docker exec -e PGPASSWORD=linktracker linktracker-postgres psql -U linktracker -d linktracker -c "\dt"

Expected domain tables:

chats
links
chat_links
tags
chat_link_tags

DbUp also creates:

schemaversions

Open Scrapper Swagger:

http://localhost:5000/swagger

Basic API flow:

POST   /tg-chat/{id}
POST   /links      with Tg-Chat-Id header
GET    /links      with Tg-Chat-Id header
DELETE /links      with Tg-Chat-Id header
GET    /tags
POST   /tags
PUT    /tags/{id}
DELETE /tags/{id}

Update Checking

Scrapper uses Quartz to periodically process tracked links in batches.

For GitHub links, it detects new:

Issue
Pull request

For StackOverflow links, it detects new:

Answer
Question comment
Answer comment

Notifications include:

type of update
title
user name
creation time
text preview limited to 200 characters

The notification sender is abstracted behind IMessageSender. The current implementation is HTTP from Scrapper to Bot; another implementation such as Kafka can be added later without changing the scheduler business logic.