Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -142,7 +142,7 @@ Configuration in `src/envs-constants.ts`:
- `CONCURRENCY_LIMIT` - Number of concurrent requests (defaults to ~95% of rate limit)
- Override defaults via `.env` file with `RATE_LIMIT` or `CONCURRENCY_LIMIT`

The script uses **p-limit for concurrency control** across all API calls.
The tool uses **p-limit for concurrency control** across all API calls.

**Retry logic**:

Expand Down Expand Up @@ -205,7 +205,7 @@ This is necessary because Clerk's API only accepts one primary identifier per cr

### Environment Variable Detection

The script auto-detects instance type from `CLERK_SECRET_KEY`:
The tool auto-detects instance type from `CLERK_SECRET_KEY`:

- Checks if key contains `"live"` → production
- Otherwise → development
Expand Down
26 changes: 13 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# Clerk User Migration Script
# Clerk User Migration Tool

## Description

This repository contains a script that takes a JSON file as input, containing a list of users, and creates a user in Clerk using Clerk's backend API. The script respects rate limits and handles errors.
This repository contains a tool that takes a JSON file as input, containing a list of users, and creates a user in Clerk using Clerk's backend API. The tool respects rate limits and handles errors.

## Table of Contents

Expand All @@ -25,18 +25,18 @@ This repository contains a script that takes a JSON file as input, containing a
Clone the repository and install the dependencies.

```bash
git clone git@github.com:clerk/migration-script
git clone git@github.com:clerk/migration-tool

cd migration-script
cd migration-tool

bun install
```

### Users file

The script is designed to import from multiple sources, including moving users from one Clerk instance to another. You may need to edit the transformer for your source. Please see below for more information on that.
The tool is designed to import from multiple sources, including moving users from one Clerk instance to another. You may need to edit the transformer for your source. Please see below for more information on that.

The script will import from a CSV or JSON. It accounts for empty fields in a CSV and will remove them when converting from CSV to a javascript object.
The tool will import from a CSV or JSON. It accounts for empty fields in a CSV and will remove them when converting from CSV to a javascript object.

The only required fields are `userId` and an identifier (one of `email`, `phone` or `username`).

Expand Down Expand Up @@ -75,25 +75,25 @@ If no key is found, the interactive CLI will prompt you to enter one and optiona

You can find your secret key in the [Clerk Dashboard](https://dashboard.clerk.dev/) under **API Keys**.

### Run the script
### Run the tool

```bash
bun migrate
```

The script will begin processing users and attempting to import them into Clerk. The script respects rate limits for the Clerk Backend API. If the script hits a rate limit, it will wait 10 seconds and retry (up to 5 times). Any errors will be logged to timestamped log files in the `./logs` folder.
The tool will begin processing users and attempting to import them into Clerk. The tool respects rate limits for the Clerk Backend API. If the tool hits a rate limit, it will wait 10 seconds and retry (up to 5 times). Any errors will be logged to timestamped log files in the `./logs` folder.

The script can be run on the same data multiple times. Clerk automatically uses the email as a unique key so users won't be created again.
The tool can be run on the same data multiple times. Clerk automatically uses the email as a unique key so users won't be created again.

**Error Handling & Resuming**: If the migration stops for any reason (error, interruption, etc.), the script will display the last processed user ID. You can resume the migration from that point by providing the user ID when prompted, or by using:
**Error Handling & Resuming**: If the migration stops for any reason (error, interruption, etc.), the tool will display the last processed user ID. You can resume the migration from that point by providing the user ID when prompted, or by using:

```bash
bun migrate --resume-after="user_xxx"
```

## CLI Reference

The migration script supports both interactive and non-interactive modes.
The migration tool supports both interactive and non-interactive modes.

### Usage

Expand Down Expand Up @@ -218,15 +218,15 @@ You could add a column in your user table inside of your database called `ClerkI

## Configuration

The script can be configured through the following environment variables:
The tool can be configured through the following environment variables:

| Variable | Description |
| ------------------- | ------------------------------------------------------------------------- |
| `CLERK_SECRET_KEY` | Your Clerk secret key |
| `RATE_LIMIT` | Rate limit in requests/second (auto-configured: 100 for prod, 10 for dev) |
| `CONCURRENCY_LIMIT` | Number of concurrent requests (auto-configured: ~9 for prod, ~1 for dev) |

The script automatically detects production vs development instances from your `CLERK_SECRET_KEY` and sets appropriate rate limits and concurrency:
The tool automatically detects production vs development instances from your `CLERK_SECRET_KEY` and sets appropriate rate limits and concurrency:

- **Production** (`sk_live_*`):
- Rate limit: 100 requests/second (Clerk's limit: 1000 requests per 10 seconds)
Expand Down
2 changes: 1 addition & 1 deletion docs/schema-fields.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Supported Schema Fields

The migration script validates all user data against a Zod schema defined in `src/migrate/validator.ts`. Below is a complete list of supported fields.
The migration tool validates all user data against a Zod schema defined in `src/migrate/validator.ts`. Below is a complete list of supported fields.

## Required Fields

Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"name": "clerk-user-migration",
"name": "clerk-user-migration-tool",
"version": "0.0.1",
"description": "## Description",
"main": "index.ts",
Expand Down
4 changes: 2 additions & 2 deletions prompts/migration-prompt.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ Use this prompt with an AI assistant to analyze your user data file and run the
Copy and paste the following prompt, replacing `[YOUR FILE PATH]` with the path to your user data file:

````
I want to migrate users to Clerk using the migration script. Please help me import the following file:
I want to migrate users to Clerk using the migration tool. Please help me import the following file:

[YOUR FILE PATH]

Expand Down Expand Up @@ -78,7 +78,7 @@ After migration completes:
- **Development instances** (`sk_test_*`): 10 requests/second
- **Production instances** (`sk_live_*`): 100 requests/second

The script handles rate limiting and retries automatically. All operations are logged to `./logs/`.
The tool handles rate limiting and retries automatically. All operations are logged to `./logs/`.
````

---
Expand Down
2 changes: 1 addition & 1 deletion prompts/transformer-prompt.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ Use this prompt with an AI assistant to generate a custom transformer from your
Copy and paste the following prompt, replacing `[YOUR SAMPLE DATA]` with a sample of your user JSON or CSV data:

````
I need to create a custom transformer for the Clerk user migration script. Please analyze my sample user data and generate a transformer file.
I need to create a custom transformer for the Clerk user migration tool. Please analyze my sample user data and generate a transformer file.

## Environment Setup

Expand Down
6 changes: 3 additions & 3 deletions src/migrate/cli.ts
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ import type {
} from '../types';

/**
* Parsed command-line arguments for the migration script
* Parsed command-line arguments for the migration tool
*/
export type CLIArgs = {
transformer?: string;
Expand Down Expand Up @@ -111,7 +111,7 @@ ENVIRONMENT VARIABLES:
NOTES:
- In non-interactive mode (-y), --transformer and --file are required
- Firebase migrations require all four --firebase-* options
- The script auto-detects dev/prod instance from CLERK_SECRET_KEY
- The tool auto-detects dev/prod instance from CLERK_SECRET_KEY
`);
}

Expand Down Expand Up @@ -257,7 +257,7 @@ async function ensureClerkSecretKey(
/**
* Parses command-line arguments into a CLIArgs object
*
* @param argv - Array of command-line arguments (without node/bun and script path)
* @param argv - Array of command-line arguments (without node/bun and tool path)
* @returns Parsed CLI arguments
*/
export function parseArgs(argv: string[]): CLIArgs {
Expand Down
2 changes: 1 addition & 1 deletion src/migrate/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ import * as p from '@clack/prompts';
import color from 'picocolors';

/**
* Main entry point for the user migration script
* Main entry point for the user migration tool
*
* Workflow:
* 1. Parses CLI arguments to determine mode (interactive vs non-interactive)
Expand Down