[Backend] Build Enterprise Webhook System (Discord Integration, Events, Security, Retries, Idempotency)

[abhishek-nexgen-dev]

We are building a complete webhook + event-driven system in MERN stack:

Events happen (member join / repo push / event created)
→ backend captures event
→ event pushed to queue
→ processed asynchronously
→ webhook triggered
→ Discord / external systems notified
→ UI shows logs & status

---

🔗 Related Issues (IMPORTANT)

This issue depends on:

👉 Webhook Setup System (Configuration Layer)

• Create webhook
• Select events
• Store secrets
• Enable/disable

👉 Use that issue as foundation before delivery system.

---

🎯 Goals

System must be:

• Real-time (event-driven)
• Secure (signed webhooks + GitHub verification)
• Scalable (queue-based workers)
• Observable (logs + dashboard)
• Full-stack (backend + React UI)

---

🧱 SYSTEM ARCHITECTURE

---

Tech Stack

Frontend → React (Dashboard)
Backend → Node.js + Express
Database → MongoDB
Queue → Redis + BullMQ

---

High-Level Flow

Event occurs
→ Event Bus (emitEvent)
→ Queue (BullMQ)
→ Worker processes job
→ Webhook delivery
→ Discord / external system notified
→ Store delivery logs
→ Display in dashboard

---

🧠 CORE MODULES

---

1️⃣ EVENT SYSTEM (CORE)

---

Events

member.created
member.activated
event.created
hackathon.created
community.approved
github.push
github.pr.opened

---

Event Bus

emitEvent(type, payload)

---

Requirements

• decoupled system
• async event publishing
• multiple consumers

---

---

2️⃣ WEBHOOK SETUP SYSTEM (FROM PREVIOUS ISSUE)

---

APIs

POST /api/v1/webhooks
GET /api/v1/webhooks
PATCH /api/v1/webhooks/:id
DELETE /api/v1/webhooks/:id
POST /api/v1/webhooks/:id/test

---

Features

• event selection
• secret config
• URL validation (SSRF protected)
• enable/disable toggle
• RBAC protected

---

👉 This must be implemented first (dependency)

---

---

3️⃣ WEBHOOK DELIVERY SYSTEM

---

Flow

event triggered
→ find matching webhooks
→ push job to queue
→ worker sends HTTP POST
→ store result

---

Payload

{
  "event": "member.created",
  "timestamp": "ISO_DATE",
  "eventId": "unique-id",
  "data": {}
}

---

Headers

X-Webhook-Event
X-Webhook-Id
X-Webhook-Signature

---

---

4️⃣ DISCORD INTEGRATION

---

Use Cases

• new member joined
• new event created
• hackathon launched
• GitHub PR opened

---

Payload

{
  "content": "🚀 New event created!"
}

---

Features

• message templates
• per-event formatting
• emoji support
• multiple channels/webhooks

---

---

5️⃣ GITHUB WEBHOOK INTEGRATION

---

Endpoint

POST /api/v1/webhooks/github

---

Verify Signature

X-Hub-Signature-256

---

Flow

receive webhook
→ verify signature
→ parse event
→ emitEvent()
→ process internally

---

Supported Events

push
pull_request
issues
release

---

---

6️⃣ SECURITY 🔐

---

Signature

HMAC-SHA256(secret, payload)

---

SSRF Protection

• block localhost
• block private IP ranges
• allow HTTPS only

---

Rate Limiting

Webhook receive → 100/min
Webhook delivery → 10/sec/community

---

Secret Storage

• hash + encrypt
• never return in API

---

---

7️⃣ RETRY SYSTEM

---

Strategy

1 → immediate
2 → 1 min
3 → 5 min
4 → 15 min
5 → fail

---

Features

• exponential backoff
• dead letter queue
• manual retry support

---

---

8️⃣ IDEMPOTENCY

---

Problem

• GitHub duplicate events
• retry duplicates

---

Solution

eventId
deliveryId

---

DB Check

• ignore already processed events

---

---

9️⃣ DATABASE DESIGN

---

Webhook

communityId
url
events
secretHash
active

---

WebhookDelivery

webhookId
event
payload
status
attempts
responseCode
responseBody

---

EventStore

eventId
type
payload
processed

---

---

🔟 FRONTEND (REACT DASHBOARD)

---

Pages

---

Webhook List

• list all webhooks
• status (active/inactive)

---

Create Webhook

• URL input
• event selector
• secret field

---

Delivery Logs

• show logs

• filter:

  • success/failed
  • event type

---

---

🎨 UI FEATURES

---

• toggle enable/disable
• retry delivery button
• view payload
• error logs display
• test webhook button

---

---

🧪 TESTING

---

Backend

• webhook delivery
• retry logic
• signature validation

---

Frontend

• form validation
• API calls

---

Security

• SSRF
• invalid signature
• replay attack

---

---

📊 OBSERVABILITY

---

Logs

• event triggered
• delivery status

---

Metrics

success rate
failure rate
retry count
queue size

---

Alerts

• high failure rate
• queue backlog

---

---

🧨 EDGE CASES

---

duplicate GitHub events
invalid webhook URL
Discord downtime
queue crash
timeout
retry storm

---

---

⚙️ PERFORMANCE

---

• async processing
• queue workers scaling
• batch job handling

---

---

🌍 ENVIRONMENT

---

DEV
STAGING
PROD

---

---

✅ ACCEPTANCE CRITERIA

---

✔ Webhook setup works
✔ Event system works
✔ Delivery system works
✔ GitHub integration works
✔ Discord notifications work
✔ Retry system works
✔ Duplicate handling works
✔ Logs visible in UI
✔ Secure system
✔ Scalable architecture

---

🔥 FINAL SUMMARY

This system is:

Stripe + GitHub + Slack webhook system (MERN implementation)