From f8a67464b9a9e8d5a011520f17c1c10ff8a6fcb8 Mon Sep 17 00:00:00 2001 From: "google-labs-jules[bot]" <161369871+google-labs-jules[bot]@users.noreply.github.com> Date: Fri, 27 Mar 2026 17:12:47 +0000 Subject: [PATCH] feat(arch): initialize specialized documentation for event-driven architecture Co-authored-by: beginwebdev2002 <102213457+beginwebdev2002@users.noreply.github.com> --- .../event-driven-architecture/data-flow.md | 86 +++++++++ .../folder-structure.md | 94 +++++++++ .../implementation-guide.md | 180 ++++++++++++++++++ .../event-driven-architecture/readme.md | 51 +++++ .../event-driven-architecture/trade-offs.md | 51 +++++ architectures/readme.md | 1 + 6 files changed, 463 insertions(+) create mode 100644 architectures/event-driven-architecture/data-flow.md create mode 100644 architectures/event-driven-architecture/folder-structure.md create mode 100644 architectures/event-driven-architecture/implementation-guide.md create mode 100644 architectures/event-driven-architecture/readme.md create mode 100644 architectures/event-driven-architecture/trade-offs.md diff --git a/architectures/event-driven-architecture/data-flow.md b/architectures/event-driven-architecture/data-flow.md new file mode 100644 index 0000000..fcc2568 --- /dev/null +++ b/architectures/event-driven-architecture/data-flow.md @@ -0,0 +1,86 @@ +--- +description: Vibe coding guidelines for the asynchronous request and data flow lifecycle in an Event-Driven Architecture (EDA). +technology: Event-Driven Architecture +domain: Architecture +complexity: Architect +last_evolution: 2026-03-27 +vibe_coding_ready: true +tags: [eda, data-flow, sequence-diagram, asynchronous, messaging, event-lifecycle] +topic: Event-Driven Data Flow +--- + +
+ # 📊 EDA Data Flow (Sequence Blueprint) +
+ +--- + +This document illustrates the execution lifecycle of a distributed, asynchronous event-driven system. It defines the path an initial synchronous request takes as it propagates across independent microservices via a message broker. + +## Mental Model & Asynchronous Lifecycle + +The architectural contract is simple: +- The **Ingress Gateway (API)** accepts the synchronous HTTP request from the User. +- The **API Gateway** immediately validates the request and queues a Command/Event on the **Message Broker (Kafka/RabbitMQ)**. It returns HTTP 202 Accepted. +- Downstream **Consumers (Subscribers)** independently poll/listen to the broker, performing background work without blocking the UI. +- Finally, the UI relies on WebSocket, Server-Sent Events (SSE), or polling for real-time completion status. + +> [!IMPORTANT] +> **Data Flow Constraint:** A microservice handling an event MUST NOT synchronously invoke another microservice. It must process the event, update its localized database, and optionally emit a subsequent domain event. + +### Sequence Diagram: Distributed E-Commerce Checkout + +```mermaid +sequenceDiagram + autonumber + actor Client as User (Frontend) + participant API as API Gateway (REST) + participant Broker as Message Broker (Kafka) + participant OrderSvc as Order Service + participant PaySvc as Payment Service + participant NotifySvc as Notification Service + + Client->>API: POST /checkout (Cart DTO) + API-->>Broker: Publish [CheckoutInitiatedEvent] + API-->>Client: HTTP 202 Accepted (Order Pending) + + Broker-->>OrderSvc: Consume [CheckoutInitiatedEvent] + OrderSvc->>OrderSvc: Create Pending Order (DB) + OrderSvc-->>Broker: Publish [OrderCreatedEvent] + + Broker-->>PaySvc: Consume [OrderCreatedEvent] + PaySvc->>PaySvc: Process Stripe Payment + + alt Payment Success + PaySvc-->>Broker: Publish [PaymentSucceededEvent] + Broker-->>OrderSvc: Consume [PaymentSucceededEvent] + OrderSvc->>OrderSvc: Update Order Status -> Confirmed + Broker-->>NotifySvc: Consume [PaymentSucceededEvent] + NotifySvc->>Client: Push Notification / Email (Success) + else Payment Failure + PaySvc-->>Broker: Publish [PaymentFailedEvent] + Broker-->>OrderSvc: Consume [PaymentFailedEvent] + OrderSvc->>OrderSvc: Update Order Status -> Failed + Broker-->>NotifySvc: Consume [PaymentFailedEvent] + NotifySvc->>Client: Push Notification / Email (Failure) + end +``` + +--- + +## The Outbox Pattern (Reliable Publishing) + +To ensure dual-write safety (saving state in the local DB and publishing the event to Kafka simultaneously), EDA relies on the **Transactional Outbox Pattern**. + +1. The service begins a local DB transaction. +2. The service saves business entity data (e.g., `orders` table). +3. The service inserts an event record in an `outbox` table in the SAME transaction. +4. The service commits the transaction. +5. A background process (e.g., Debezium, CDC) reads the `outbox` table and publishes the messages to Kafka, ensuring "at-least-once" delivery. + +--- + +
+ [Back to Main Blueprint](./readme.md)

+ Master the event lifecycle to prevent distributed monoliths! 🌊 +
diff --git a/architectures/event-driven-architecture/folder-structure.md b/architectures/event-driven-architecture/folder-structure.md new file mode 100644 index 0000000..cee5ff5 --- /dev/null +++ b/architectures/event-driven-architecture/folder-structure.md @@ -0,0 +1,94 @@ +--- +description: Vibe coding guidelines for the folder structure and structural hierarchy of Event-Driven Architecture (EDA) projects. +technology: Event-Driven Architecture +domain: Architecture +complexity: Architect +last_evolution: 2026-03-27 +vibe_coding_ready: true +tags: [eda, folder-structure, architecture-hierarchy, backend, microservices] +topic: Event-Driven Folder Structure +--- + +
+ # 📁 EDA Folder Structure (Hierarchy Blueprint) +
+ +--- + +This document outlines the optimal 2026-grade folder structure for an Event-Driven microservice (or bounded context). This hierarchy enforces the segregation between business logic and message-broker infrastructure. + +## Folder Hierarchy (Mental Model) + +A robust EDA microservice separates its core domain from its external adapters (Publishers and Subscribers). The overarching directory aligns closely with DDD or Clean Architecture, where Event handlers act as secondary entry points (instead of HTTP controllers). + +> [!NOTE] +> **Constraint:** Domain layers MUST NOT depend on the specific message broker (Kafka, AWS EventBridge). Infrastructure dependencies (like `@nestjs/microservices` or `kafkajs`) are strictly confined to the `infrastructure/` or `adapters/` layer. + +### System Diagram: Layered Hierarchy + +```mermaid +graph TD + Root[Microservice Root] --> Domain[core/domain] + Root --> App[core/application] + Root --> Infra[infrastructure] + + Infra --> DB[database (Adapters)] + Infra --> Msg[messaging (Broker Integrations)] + + Msg --> Pub[publishers (Producers)] + Msg --> Sub[subscribers (Consumers)] + Msg --> Config[kafka-config] + + App --> Handlers[Command/Event Handlers] + Handlers -.-> Pub + + %% Apply strict styling tokens + classDef default fill:#e1f5fe,stroke:#03a9f4,stroke-width:2px,color:#000; + classDef component fill:#e8f5e9,stroke:#4caf50,stroke-width:2px,color:#000; + classDef layout fill:#f3e5f5,stroke:#9c27b0,stroke-width:2px,color:#000; + + class Root layout; + class Domain,App,Handlers component; + class Infra,DB,Msg,Pub,Sub,Config default; +``` + +--- + +## Detailed Directory Tree + +```text +src/ +├── 📁 core/ # Pure business logic (No infra dependencies) +│ ├── 📁 domain/ # Aggregates, Value Objects, Domain Events +│ │ ├── events/ # Internal domain event types (e.g., OrderCreated) +│ │ └── models/ # Business entities +│ └── 📁 application/ # Use cases orchestration +│ ├── commands/ # Sync logic executed before emitting events +│ └── handlers/ # Logic that responds to consumed events +│ +├── 📁 infrastructure/ # Framework and Broker integrations +│ ├── 📁 messaging/ # The Event-Driven core +│ │ ├── 📁 config/ # Kafka client configuration, schemas +│ │ ├── 📁 publishers/ # Outbound adapters (Emit events to Broker) +│ │ │ └── OrderPublisher.ts # Implements IEventPublisher from Core +│ │ ├── 📁 subscribers/ # Inbound adapters (Listen to Broker queues) +│ │ │ └── PaymentConsumer.ts # Routes Kafka messages to Application handlers +│ │ └── 📁 schemas/ # AsyncAPI/Avro/Protobuf message schemas +│ └── 📁 database/ # DB adapters (Repositories, Outbox pattern) +│ +└── main.ts # Application bootstrap (Starts HTTP + Consumers) +``` + +### Explanation of Key Directories + +1. **`core/domain/events/`**: These are internal representations of an event. They are purely business-focused (e.g., `OrderPlacedDomainEvent`). They know nothing about Kafka serialization. +2. **`infrastructure/messaging/publishers/`**: This directory contains implementations of your output ports. It serializes the internal domain event into a payload (JSON/Avro) and publishes it to the external topic. +3. **`infrastructure/messaging/subscribers/`**: This directory acts exactly like HTTP Controllers. A consumer listens to a Kafka topic, deserializes the message, and hands it off to a `core/application/handlers/` class to perform the actual business logic. +4. **`infrastructure/messaging/schemas/`**: Strongly-typed schemas (like Protobuf or Avro) defining the contract for events passing through the broker. + +--- + +
+ [Back to Main Blueprint](./readme.md)

+ A clean directory tree prevents tightly-coupled broker dependencies! 📁 +
diff --git a/architectures/event-driven-architecture/implementation-guide.md b/architectures/event-driven-architecture/implementation-guide.md new file mode 100644 index 0000000..6c17199 --- /dev/null +++ b/architectures/event-driven-architecture/implementation-guide.md @@ -0,0 +1,180 @@ +--- +description: Vibe coding implementation guidelines, strict rules, code patterns, and constraints for implementing Event-Driven Architecture (EDA) using 2026 standards. +technology: Event-Driven Architecture +domain: Architecture +complexity: Architect +last_evolution: 2026-03-27 +vibe_coding_ready: true +tags: [eda, implementation-guide, kafka, microservices, typescript, nestjs, architecture-patterns] +topic: Event-Driven Implementation Guide +--- + +
+ # 🛠️ EDA Implementation Guide (Code Blueprint) +
+ +--- + +This blueprint details strict coding patterns and anti-patterns for implementing Event-Driven Architecture, ensuring "at-least-once" delivery, schema registry compliance, and robust idempotency. + +> [!IMPORTANT] +> **Implementation Contract:** All code must adhere to 2026 modern backend standards (Node.js 24+, TypeScript 5.5+, strict types, decorators, or class-based dependency injection). Services must integrate safely with message brokers (Kafka) without tightly coupling business logic. + +## Entity & Handler Relationships + +```mermaid +classDiagram + class DomainEvent { + +String eventId + +String aggregateId + +Date occurredOn + +Object payload + } + class EventPublisher { + <> + +publish(DomainEvent) void + } + class KafkaAdapter { + -Producer producer + +publish(DomainEvent) void + } + class EventHandler { + +handle(DomainEvent) void + } + + EventPublisher <|-- KafkaAdapter + DomainEvent <-- EventHandler + DomainEvent <-- EventPublisher +``` + +--- + +## 1. Idempotent Consumers (Crucial) + +Because Kafka or RabbitMQ may deliver the same message twice (e.g., during a consumer rebalance), handlers must be purely idempotent. Processing the exact same `eventId` twice MUST NOT duplicate the business outcome (e.g., charging a credit card twice). + +### ❌ Bad Practice +```typescript +class PaymentEventHandler { + async handle(event: OrderCreatedEvent) { + // ❌ Blindly processing the payment every time the event is received! + // A duplicate Kafka message will charge the user again. + await this.stripeService.charge(event.payload.amount); + await this.db.payments.insert({ orderId: event.aggregateId, status: 'PAID' }); + } +} +``` + +### ✅ Best Practice +```typescript +class PaymentEventHandler { + async handle(event: OrderCreatedEvent) { + // ✅ 1. Check if we've already processed this specific event ID + const alreadyProcessed = await this.db.processedEvents.exists(event.eventId); + if (alreadyProcessed) { + this.logger.warn(`Event ${event.eventId} already processed. Skipping.`); + return; + } + + // ✅ 2. Execute business logic idempotently + await this.db.transaction(async (tx) => { + await this.stripeService.charge(event.payload.amount); + await tx.payments.insert({ orderId: event.aggregateId, status: 'PAID' }); + + // ✅ 3. Record the event ID to prevent duplicate processing + await tx.processedEvents.insert({ id: event.eventId, processedAt: new Date() }); + }); + } +} +``` + +--- + +## 2. The Transactional Outbox Pattern + +To solve the "Dual-Write Problem" (saving state to the DB and publishing to Kafka reliably), we use an Outbox table. If the application crashes after saving to the DB but before publishing to Kafka, the message is permanently lost. + +### ❌ Bad Practice +```typescript +class OrderService { + async createOrder(data: CreateOrderDto) { + // ❌ Dual-write problem! + const order = await this.db.orders.insert(data); // 1. Save to DB + + // If the server crashes HERE, the event is never published, + // and downstream services never know the order was created. + + await this.kafkaProducer.send('orders.created', order); // 2. Publish to Broker + } +} +``` + +### ✅ Best Practice +```typescript +class OrderService { + async createOrder(data: CreateOrderDto) { + // ✅ The Outbox Pattern: Save BOTH the business entity and the event + // in the exact same ACID database transaction. + await this.db.transaction(async (tx) => { + const order = await tx.orders.insert(data); + + const outboxEvent = { + aggregateType: 'Order', + aggregateId: order.id, + eventType: 'OrderCreated', + payload: JSON.stringify(order), + createdAt: new Date(), + }; + + await tx.outbox.insert(outboxEvent); // Saves strictly to a local DB table + }); + + // A separate background process (e.g., Debezium or a Polling Worker) + // reads the 'outbox' table and safely publishes to Kafka. + } +} +``` + +--- + +## 3. Strictly Typed Schemas (Schema Registry) + +Microservices evolve independently. If a publisher changes the shape of a JSON event payload, all downstream subscribers will break. Always enforce a Schema Registry (Avro, Protobuf, JSON Schema) for all events. + +### ✅ Best Practice (Avro Example) +```typescript +// 1. Define a strict Avro schema for the event +const orderCreatedSchema = { + type: 'record', + name: 'OrderCreated', + fields: [ + { name: 'orderId', type: 'string' }, + { name: 'amount', type: 'double' }, + { name: 'customerId', type: 'string' } + // Enforces backward compatibility rules via Confluent Schema Registry + ] +}; + +class OrderKafkaPublisher { + async publish(event: DomainEvent) { + // 2. The payload is validated and serialized against the Schema Registry + // before it ever reaches the Kafka topic. + const encodedPayload = await this.schemaRegistry.encode( + 'orders.created-value', + event.payload + ); + + await this.producer.send({ + topic: 'orders.created', + messages: [{ key: event.aggregateId, value: encodedPayload }] + }); + } +} +``` + +--- + +
+ [Back to Main Blueprint](./readme.md)

+ Master these implementation constraints to guarantee asynchronous consistency! 🛠️ +
diff --git a/architectures/event-driven-architecture/readme.md b/architectures/event-driven-architecture/readme.md new file mode 100644 index 0000000..88e86f4 --- /dev/null +++ b/architectures/event-driven-architecture/readme.md @@ -0,0 +1,51 @@ +--- +description: Vibe coding guidelines and architectural constraints for Event-Driven Architecture (EDA) within the Architecture domain. +technology: Event-Driven Architecture +domain: Architecture +complexity: Architect +last_evolution: 2026-03-27 +vibe_coding_ready: true +tags: [eda, event-driven, architecture, pub-sub, asynchronous, messaging, kafka, rabbitmq, system-design] +topic: Event-Driven Architecture +--- + +
+ Kafka Logo + + # 📨 Event-Driven Architecture (EDA) Blueprint +
+ +--- + +This engineering directive contains strict architectural guidelines and 2026-grade patterns for using Event-Driven Architecture (EDA) to build highly scalable, decoupled, and fault-tolerant backend systems. + +## Context & Scope +- **Primary Goal:** Provide a determinist structural blueprint for managing asynchronous communication across autonomous microservices or domains using event streams and message brokers. +- **Target Tooling:** AI Agents (Cursor, Copilot) and Senior/Architect Developers. +- **Tech Stack Version:** Agnostic (Kafka, RabbitMQ, AWS EventBridge, Redis Pub/Sub, Node.js, Spring Boot). + +> [!NOTE] +> **Architectural Contract:** System components MUST NOT depend on one another synchronously for state mutations. Components merely emit events (Publish) and react to events (Subscribe) via an intermediary broker, maintaining strict decoupling. + +## Specialized Modules (Map of Patterns) +To deeply understand the nuances of EDA, consult the following specialized modules: + +- 📊 [**Data Flow:** Request and Event Lifecycle](./data-flow.md) +- 📁 [**Folder Structure:** Layering Publisher/Subscriber logic](./folder-structure.md) +- ⚖️ [**Trade-offs:** Pros, Cons, and System Constraints](./trade-offs.md) +- 🛠️ [**Implementation Guide:** Code patterns and Anti-patterns](./implementation-guide.md) + +--- + +## Core Principles + +1. **Asynchronous by Default:** Synchronous RPC (REST/gRPC) is restricted only to immediate read-queries or initial gateway ingress. All inter-service state mutations must occur asynchronously. +2. **Event Sourcing (Optional but Recommended):** State is derived from an immutable, append-only log of events rather than overwriting records in a database. +3. **Idempotency is Mandatory:** Because message brokers can guarantee "at least once" delivery, every subscriber/consumer must be idempotent to handle duplicate events safely. + +--- + +
+ [Back to Architecture Map](../readme.md)

+ Adhere to these EDA principles to establish a relentlessly scalable, highly-decoupled system ecosystem! 🚀 +
diff --git a/architectures/event-driven-architecture/trade-offs.md b/architectures/event-driven-architecture/trade-offs.md new file mode 100644 index 0000000..030c1d2 --- /dev/null +++ b/architectures/event-driven-architecture/trade-offs.md @@ -0,0 +1,51 @@ +--- +description: Vibe coding guidelines and constraints for the trade-offs, pros, cons, and anti-patterns of Event-Driven Architecture (EDA). +technology: Event-Driven Architecture +domain: Architecture +complexity: Architect +last_evolution: 2026-03-27 +vibe_coding_ready: true +tags: [eda, trade-offs, architecture, messaging, kafka] +topic: Event-Driven Trade-offs +--- + +
+ # ⚖️ EDA Trade-offs (Pros & Cons Blueprint) +
+ +--- + +This document outlines the high-level trade-offs associated with Event-Driven Architecture. EDA introduces incredible scalability and loose coupling but incurs extreme operational complexity, eventual consistency, and distributed debugging challenges. + +## 1. High-Level Comparison + +| 🌟 **Pros (Advantages)** | ⚠️ **Cons (Disadvantages)** | +| ------------------------ | --------------------------- | +| **Loose Coupling:** Services act independently without knowing about the existence of other services. | **Eventual Consistency:** Systems cannot rely on immediate strong consistency (ACID across services). | +| **High Availability:** If a downstream service crashes, the broker queues the message until the service recovers. | **Complex Debugging:** Tracing a single user request across 5 microservices requires Distributed Tracing (Jaeger, OpenTelemetry). | +| **Scalability:** You can scale consumers horizontally (Kafka Consumer Groups) based on queue lag. | **Operational Overhead:** Managing Kafka clusters, ZooKeeper/KRaft, schema registries, and dead-letter queues is difficult. | +| **Extensibility:** Adding a new feature (e.g., a new Notification Service) requires zero changes to the Publisher. | **Duplicate Events:** Brokers guarantee "at-least-once" delivery. Consumers MUST be strictly idempotent. | +| **Polyglot Systems:** Microservices can be written in any language as long as they adhere to the broker protocol and schema. | **Dual-Write Problem:** Guaranteeing a local DB commit and a Kafka publish simultaneously requires the Outbox Pattern. | + +--- + +## 2. Distributed Anti-Patterns + +### ❌ The "Distributed Monolith" +**Symptom:** Microservices communicate via events, but they expect an immediate asynchronous response via a "reply queue" (RPC over Kafka). The system halts if the response event is not received within a timeout. +**Resolution:** Services must rely on Choreography or Orchestration (Saga Pattern) rather than synchronous-style request/reply over a message broker. + +### ❌ Event Sourcing Abuse +**Symptom:** Storing absolutely every state change as an immutable event in Kafka indefinitely, leading to massive storage costs and complex snapshotting logic for simple CRUD applications. +**Resolution:** Use Event Sourcing ONLY for core financial or audit-heavy domains. Use standard State-Oriented CRUD for basic entities. + +### ❌ Shared Database Integration +**Symptom:** Two microservices communicate via Kafka, but both services still connect to the same PostgreSQL database directly to read/write state. +**Resolution:** Strictly adhere to the "Database-per-Service" pattern. Services must rely on their own materialized views built from consuming events. + +--- + +
+ [Back to Main Blueprint](./readme.md)

+ Master these trade-offs to avoid engineering distributed chaos! ⚖️ +
diff --git a/architectures/readme.md b/architectures/readme.md index 1f0e9ad..390f847 100644 --- a/architectures/readme.md +++ b/architectures/readme.md @@ -308,6 +308,7 @@ src/ Kafka Logo RabbitMQ **Description:** System components know nothing about each other (Low Coupling). They merely "publish" events and "subscribe" to them, reacting asynchronously. Ideal for high-load, highly-scalable backend systems. +**📖 Map of Patterns:** [Go to EDA Guidelines](./event-driven-architecture/readme.md) **Architecture Diagram & Folder Tree:** ```mermaid