Skip to content

[AWS] Story 3: Compute Layer (ECS Fargate) #178

Description

@mfittko

Problem Statement

The deployment needs ECS services that start reliably, consume config safely,
run migrations once, and remain healthy under normal lifecycle events.

Scope

In scope:

  • ECS cluster and Fargate services
  • Proxy and dispatcher task definitions
  • Init-container behavior for config fetch and migrations
  • Health checks, scaling, and ECS Exec

Out of scope:

  • Public ingress routing
  • Final alerting and dashboard policy
  • Release automation

Technical Approach

  • Define ECS services and task definitions.
  • Use ECS-native secret and parameter injection.
  • Use an init container for provider config fetch and migration execution.
  • Use a DynamoDB lock to serialize migration execution.

Dependencies

Hard dependencies:

  • CDK foundation and config contract
  • Data layer
    Blocks:
  • Networking layer
  • Observability layer
  • CI/CD pipeline
  • Production readiness gate

Acceptance Criteria

  • Proxy service runs on Fargate with ARM64
  • Dispatcher service runs on Fargate
  • ECS injects env vars from Secrets Manager + SSM
  • Init container fetches api_providers.yaml from SSM
  • Migrations run with DynamoDB lock (tasks WAIT)
  • Services connect to Aurora and Redis
  • Health checks pass consistently
  • Auto-scaling responds to CPU load
  • ECS Exec works for debugging
  • Graceful shutdown handles SIGTERM

Proposed Definition Of Done

  • Service startup logs are attached.
  • Migration lock behavior is proven with concurrent startup evidence.
  • Connectivity proof for Aurora and Redis is attached.
  • Scaling and SIGTERM behavior evidence is attached.

Validation Plan

  1. Deploy proxy and dispatcher services and confirm steady state.
  2. Verify env var and provider-config injection paths.
  3. Start multiple tasks and verify only one migration runner proceeds.
  4. Execute connectivity checks for Aurora and Redis.
  5. Apply controlled load and confirm scale-out and scale-in behavior.
  6. Send SIGTERM and confirm graceful shutdown behavior.

Risks And Mitigations

  • Risk: Migration locking fails under concurrent startup.
    • Mitigation: Require concurrent-start evidence before closure.
  • Risk: Runtime config fetch succeeds but main containers read stale config.
    • Mitigation: Validate init completion and mounted file contents.

Handoff Notes

This story is the runtime contract for networking, observability,
and pipeline work.

AC/DoD Coverage Matrix

Item Type (AC/DoD/Non-goal) Status (Met/Partial/Unmet/Unverified) Evidence (spec/tests/behavior) Notes
Proxy service runs on Fargate with ARM64 AC Unverified ECS service evidence Source AC
Dispatcher service runs on Fargate AC Unverified ECS service evidence Source AC
ECS injects env vars from Secrets Manager + SSM AC Unverified Task def and runtime proof Source AC
Init container fetches api_providers.yaml from SSM AC Unverified Init logs Source AC
Migrations run with DynamoDB lock (tasks WAIT) AC Unverified Concurrent startup proof Source AC
Services connect to Aurora and Redis AC Unverified Connectivity logs Source AC
Health checks pass consistently AC Unverified Health and soak logs Source AC
Auto-scaling responds to CPU load AC Unverified Scaling timeline Source AC
ECS Exec works for debugging AC Unverified ECS Exec transcript Source AC
Graceful shutdown handles SIGTERM AC Unverified Shutdown test logs Source AC
Service startup logs are attached. DoD Unverified Issue evidence Proposed DoD
Migration lock behavior is proven with concurrent startup evidence. DoD Unverified Issue evidence Proposed DoD
Connectivity proof for Aurora and Redis is attached. DoD Unverified Issue evidence Proposed DoD
Scaling and SIGTERM behavior evidence is attached. DoD Unverified Issue evidence Proposed DoD

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions