Skip to content

Jobs Division

Jump to bottom
PCBZ edited this page Nov 9, 2025 · 3 revisions

Jobs Division

Project Overview

Objective: Compare Push vs Pull vs Hybrid fan-out algorithms in a distributed social media system
Team Size: 4 developers

Service Ownership

👤 Person 1: User Service Owner

Primary Responsibilities:

  • PostgreSQL database design and operations
  • User add and get
  • API implementation: POST /api/users, GET /api/users

Technology Stack:

  • PostgreSQL/RDS
  • Go + Gin framework

🌐 Person 2: Social Graph Service Owner

Primary Responsibilities:

  • Neptune graph database design
  • Follow relationship management
  • API implementation: POST /api/follow

Technology Stack:

  • AWS Neptune
  • Gremlin query language
  • Go + TinkerPop driver

📝 Person 3: Post Service Owner

Primary Responsibilities:

  • DynamoDB content storage design
  • Post creation, storage, and retrieval
  • API implementation: POST /api/posts

Technology Stack:

  • DynamoDB
  • Go + AWS SDK

Key Deliverables:

  • Post creation system with 3 different fan-out implementations
  • Content storage and retrieval

📱 Person 4: Timeline Service Owner

Primary Responsibilities:

  • Critical: Implementation of 3 fan-out algorithms (Push, Pull, Hybrid)
  • Timeline generation and caching strategies
  • API implementation: GET /api/timeline/{user_id}

Technology Stack:

  • DynamoDB (timeline storage)
  • Redis (caching layer)
  • Go + microservice calls
  • 3 fan-out algorithm implementations

Key Deliverables:

  • Push Model: Pre-computed timeline system
  • Pull Model: Real-time timeline aggregation
  • Hybrid Model: Smart strategy selection

📊 Algorithm Implementation Differences

Push Model (Fan-out on Write)

Service Implementation
Post Service Store post + trigger fan-out to all followers
Timeline Service Receive batch writes + store in follower timelines

Pull Model (Fan-out on Read)

Service Implementation
Post Service Store post only (no fan-out)
Timeline Service Real-time aggregation from followed users

Hybrid Model (Smart Strategy)

Service Implementation
Post Service Conditional fan-out based on user type
Timeline Service Mixed approach: pre-computed + real-time

🗃️ Test Data Generation

👤 Person 1: User Data Generation

Responsibilities:

  • Generate 5K/25K/100K user accounts
  • Create user type distribution (Regular 85%, Influencer 14%, Celebrity 1%)
  • Generate realistic user profiles
  • Bulk user registration via POST /api/users

🌐 Person 2: Relationship Data Generation

Responsibilities:

  • Generate power-law distributed follow relationships
  • Regular Users: 10-100 followers
  • Influencers: 1K-10K followers
  • Celebrities: 50K-500K followers
  • Implement Zipfian distribution algorithm

🔄 Collaboration Workflow

Phase 1: Parallel Development (Week 1-2)

  • Each person develops their individual service
  • Define inter-service API contracts

Phase 2: Integration & Testing (Week 3)

  • Person 4 coordinates inter-service calls
  • Service mesh setup and testing

Phase 3: Experiment Execution (Week 4)

📈 Experiment Coordination

Data Generation Sequence

1. Person 1: Generate users → PostgreSQL
2. Person 3: Generate relationships → Neptune  
3. Person 2 & 4: Use existing user/relationship data
4. Execute fan-out algorithm performance tests

Clone this wiki locally