DynamoDB Local Sync

This project allows you to synchronize your production DynamoDB data to a local DynamoDB instance. This enables safe local development without the risk of accidentally modifying or deleting production data ("dropping the database and running away").

prerequisites

1. Install Docker Desktop

Please refer to the official documentation for installation instructions:

• Docker Desktop for Mac
• Docker Desktop for Windows
• Docker Engine on Ubuntu

2. Install & Configure AWS CLI

Installation

Please refer to the official documentation:

• Install or update the latest version of the AWS CLI

How to get your Access Keys

If you don't have your access keys (or lost them), you need to generate new ones:

1. Log in to the AWS Console.
2. Click your username in the top right -> Security credentials.
3. Scroll down to the Access keys section.
4. Click Create access key.
   • Select Command Line Interface (CLI) if prompted.
   • Acknowledge the recommendation box and click Next.
5. Important: Copy the Access key ID and Secret access key immediately or download the .csv file. You will not be able to see the secret key again.

Configuration (Login)

After installation, configure your credentials:

aws configure

You will be prompted for:

• AWS Access Key ID: Your access key.
• AWS Secret Access Key: Your secret key.
• Default region name: e.g., us-west-1 (Match your production region).
• Default output format: json.

Usage

1. Configure .env

Copy the example configuration file and fill in your details:

cp .env.example .env

Edit .env and set:

• MFA_DEVICE_ARN: Your MFA Device ARN (see below how to find it)
• TABLES: List of tables you want to sync, e.g. "Table1 Table2 Table3"
• REGION: Your AWS region, e.g. "us-west-1"
• LOCAL_ENDPOINT: Local DynamoDB URL, default http://localhost:4567

How to find your MFA Device ARN

1. Log in to the AWS Console.
2. Click your username in the top right -> Security credentials.
3. Scroll to Multi-factor authentication (MFA).
4. Copy the Identifier (ARN) of your assigned device.
   • Format: arn:aws:iam::123456789012:mfa/deviceName

2. Authenticate with MFA (Recommended)

Since your account requires MFA, you must obtain a session token before running the sync script.

Run the auth script using source so it can export variables to your current shell:

# You will be prompted for your 6-digit MFA code
source ./aws-env.sh

# Or provide the code as an argument
source ./aws-env.sh 123456

Why do I need to enter the code every time? MFA tokens are temporary credentials (valid for a short session). For security reasons, they expire quickly, requiring you to re-authenticate with a new code for each new session. You can also manually clear credentials by running: source ./aws-env.sh unauth

3. Start Local DynamoDB

Start the local DynamoDB container using Docker Compose:

docker compose up -d

4. Sync Data

Run the sync script (it will load tables and region from .env):

# Source the script to run it
# This ensures it can automatically clear your shell's AWS credentials after running
source ./sync-dynamodb.sh

Note: The script will automatically unset the high-privilege AWS credentials from your current terminal session after finishing to prevent accidental operations on production.

Arguments (optional if .env is set):

• -t: Override tables, Space-separated list of tables to sync, e.g. "Table1 Table2 Table3"
• -r: Override AWS region, e.g. "us-west-1"
• -e: Override endpoint, e.g. "http://localhost:4567"

3. View Data with DynamoDB Admin

dynamodb-admin provides a GUI to view and manage your local DynamoDB tables.

Installation (requires Node.js):

npm install -g dynamodb-admin

Start the Admin Tool: Since we mapped our local DynamoDB to port 4567, start it with:

dynamodb-admin --dynamo-endpoint=http://localhost:4567

Access the GUI: Open your browser and visit: http://localhost:8001