Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
87 changes: 36 additions & 51 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,41 +1,41 @@
# tdc

`tdc` is the command-line interface for TiDB Cloud Filesystem and TiDB Cloud Starter.
tdc ([TiDB Cloud](https://tidbcloud.com) CLI) is a unified tool to manage your TiDB Cloud Filesystem (FS) and Starter services.

> tdc is currently in Preview. Its features and command-line interface might change without prior notice.
- TiDB Cloud Filesystem is a serverless distributed file system designed specifically for AI coding agent workloads.
- TiDB Cloud Starter provides serverless distributed database clusters that are fully compatible with MySQL.

- TiDB Cloud Filesystem is a distributed file system designed specifically for AI coding agent workloads, with zero infrastructure.
- TiDB Cloud Starter provides distributed database clusters that are fully compatible with MySQL, with zero infrastructure.
> `tdc` is currently in preview. Subcommands labeled as preview are subject to change without prior notice.

## Your Agent's Toolbelt
## 3-Command Superpower for Your Agent

### Always-on, zero infrastructure file system for sandboxesThe 3-Command Superpower
### Always-On File System for SandboxesZero Infrastructure Required

An agent can persist state between sessions, share files across sandboxes, snapshot its workspace before attempting a risky operation, and roll back on failure — all through a CLI with POSIX compatibility.
With `tdc`, an agent can persist state between sessions, share files across sandboxes, snapshot its workspace before attempting a risky operation, and roll back on failure — all through a CLI with POSIX compatibility.

1. Create a filesystem resource and get the returning token (one-time, out of the sandbox)
1. Create a file system and obtain the file system token (performed once, outside the sandbox).

```shell
export TDC_FS_TOKEN="$(tdc fs create-file-system --file-system-name agent-workspace --region <REGION_CODE> --wait --query fs_token --output text)"
```

2. Mount the filesystem and use just like any regular POSIX-compliant filesystem (inside the sandbox environment)
2. Mount the filesystem to a local path and use it as a normal POSIX-compliant filesystem (performed within the sandbox)

```shell
export TDC_FS_TOKEN="<FS_TOKEN>"
tdc fs mount-file-system --file-system-name agent-workspace --mount-path /path_to_workspace --region <REGION_CODE>
echo "Hello Sandbox Workspace!" >> /path_to_workspace/hello.txt
tdc fs mount-file-system --file-system-name agent-workspace --mount-path /path-to-workspace --region <REGION_CODE>
echo "Hello Sandbox Workspace!" >> /path-to-workspace/hello.txt
```

3. Unmount to safely release the workspace before handing off to another sandbox (inside the sandbox environment)
3. Unmount the file system to release the workspace before passing it to another sandbox (performed within the sandbox).

```shell
tdc fs unmount-file-system --mount-path /path_to_workspace --region <REGION_CODE>
tdc fs unmount-file-system --mount-path /path-to-workspace --region <REGION_CODE>
```

### Always-on, zero infrastructure MySQL — The 3-Command Superpower
### Always-On MySQL — Zero Infrastructure Required

An agent can go from zero to live HTAP SQL (Hybrid Transaction / Analytical Processing) in three commands:
With `tdc`, an agent can go from zero to live HTAP SQL (Hybrid Transaction / Analytical Processing) in three commands:

1. Provision a serverless MySQL-compatible cluster, wait until it is active, and capture its ID

Expand Down Expand Up @@ -93,28 +93,31 @@ Add `$HOME\.tdc\bin` to your user `PATH` to keep tdc available in new PowerShell

### Configure

Configure `tdc` with a TiDB Cloud Public Key and Private Key from the [TiDB Cloud](https://tidbcloud.com/org-settings/api-keys) console. Supported region codes are `aws-us-east-1`, `aws-us-west-2`, `aws-eu-central-1`, `aws-ap-northeast-1`, `aws-ap-southeast-1`, and `ali-ap-southeast-1`.
- Authentication: a TiDB Cloud Public Key and a Private Key from the [TiDB Cloud API Keys](https://tidbcloud.com/org-settings/api-keys) console.
- Default region: one of aws-us-east-1, aws-us-west-2, aws-eu-central-1, aws-ap-northeast-1, aws-ap-southeast-1, or ali-ap-southeast-1.
- Regions support TiDB Cloud Filesystem: aws-us-east-1, aws-ap-southeast-1.
- Regions support TiDB Cloud Starter: aws-us-east-1, aws-us-west-2, aws-eu-central-1, aws-ap-northeast-1, aws-ap-southeast-1, or ali-ap-southeast-1.

Set up a default profile with one command:

```shell
tdc configure --non-interactive --region-code <TDC_REGION_CODE> --tdc-public-key <TDC_PUBLIC_KEY> --tdc-private-key <TDC_PRIVATE_KEY>
```

Configure verifies the API key by listing all accessible projects, requires exactly one project with `type = "tidbx_virtual"`, and stores its ID as the profile's default `project_id` in `~/.tdc/config`. API credentials remain in `~/.tdc/credentials`. Configuration fails without changing the profile when project discovery fails.
Alternatively, set up a default profile interactively by running the command below. You will be prompted to enter your TiDB Cloud Public Key, Private Key, and the default region:

```toml
[default]
region_code = "aws-us-east-1"
project_id = "1372813089454645969"
```shell
tdc configure
```

### TiDB Cloud Filesystem
`tdc configure` stores non-sensitive settings in `~/.tdc/config` and API credentials to `~/.tdc/credentials`.

Supported regions: `aws-us-east-1` and `aws-ap-southeast-1`.
### TiDB Cloud Filesystem

```shell
mkdir ~/my-workspace
tdc fs create-file-system --file-system-name my-workspace --wait
tdc fs mount-file-system --mount-path ~/my-workspace
tdc fs mount-file-system --file-system-name my-workspace --mount-path ~/my-workspace
```

Automatic mounting uses FUSE on Linux and WebDAV on macOS and Windows. macOS users can install macFUSE and explicitly add `--driver fuse` for the full FUSE experience.
Expand All @@ -133,45 +136,27 @@ tdc fs describe-file-system --file-system-name scratch
export TDC_FS_TOKEN="$(tdc fs create-file-system --file-system-name agent-workspace --wait --query fs_token --output text)"
```

Without `--wait`, file system creation returns after Drive9 accepts provisioning. With the flag, tdc waits up to 10 minutes until the file system root is readable through the public Drive9 data-plane CLI. A timeout or interruption leaves the file system and its locally stored credentials intact.

An agent sandbox can then use that existing file system without running `tdc configure` or providing TiDB Cloud API keys:

```shell
export TDC_FS_TOKEN="<FS_TOKEN>"
tdc fs mount-file-system --file-system-name agent-workspace --mount-path /path_to_workspace --region aws-us-east-1
```

> **Preview Note:** Creating a new file system automatically provisions and manages a TiDB Cloud Starter database cluster (name prefix `tidbcloud-fs-`) in your TiDB Cloud organization. This is temporary behavior; in future releases, this backend database cluster will no longer be displayed or count against your TiDB Cloud Starter slot limits.

### TiDB Cloud Starter

```shell
tdc db create-db-cluster --db-cluster-name my-distributed-mysql --db-cluster-type starter --wait
```

Cluster creation uses the configured `project_id` by default. Use optional `--project-id <project-id>` to create in another accessible project. An explicit empty `--project-id` is rejected instead of falling back to the profile.

Without `--wait`, cluster creation returns as soon as TiDB Cloud accepts the asynchronous create request. With the flag, tdc waits up to 12 minutes and returns the final `ACTIVE` cluster. A timeout or interruption leaves the created cluster intact and reports its ID for inspection.

Branch creation and cluster deletion have equivalent explicit wait modes:

```shell
tdc db create-db-cluster-branch --db-cluster-id <CLUSTER_ID> --db-cluster-branch-name development --wait
tdc db delete-db-cluster --db-cluster-id <CLUSTER_ID> --wait
```

Branch waiting lasts up to 5 minutes. Cluster deletion waiting lasts up to 12 minutes and succeeds when the API reports `DELETED` or the deleted cluster is no longer accessible.

### Organization Projects

```shell
tdc organization list-projects
```

Each project includes a `type`: `tidbx` identifies a regular project and `tidbx_virtual` identifies a virtual project.

## Commands
## Get Help

Running `tdc` without a command returns a usage error and a compact two-level command synopsis. Run `tdc help`, `tdc <command> help`, or `tdc <command> <subcommand> help` for the full command list, flags, and examples. Help displays flag value types in angle brackets and marks required flags with `(required)`.
- `tdc`
- `tdc help`
- `tdc <command> help`
- `tdc <command> <subcommand> help`

<details>
<summary>All commands</summary>
Expand Down Expand Up @@ -267,9 +252,9 @@ tdc update --target-version v0.1.1

## Documentation

- [English Preview documentation](docs/pingcap-docs/docs/ai/tdc/tdc-overview.md)
- [Preview Documentation](docs/pingcap-docs/docs/ai/tdc/tdc-overview.md)

## Build from source
## Build From Source

Requirements:

Expand Down
60 changes: 30 additions & 30 deletions internal/cli/commands.go
Original file line number Diff line number Diff line change
Expand Up @@ -903,7 +903,7 @@ func addFSSelectorFlags(commands []*cobra.Command, excluded ...string) {
continue
}
if command.Flags().Lookup("file-system-name") == nil {
command.Flags().String("file-system-name", "", "tdc fs resource name; defaults to the profile default or only configured resource")
command.Flags().String("file-system-name", "", "The name of the file system.")
}
}
}
Expand All @@ -918,7 +918,7 @@ func addFSAuthFlags(commands []*cobra.Command, excluded ...string) {
continue
}
if command.Flags().Lookup("fs-token") == nil {
command.Flags().String("fs-token", "", "tdc fs owner token; prefer TDC_FS_TOKEN for automation")
command.Flags().String("fs-token", "", "File system user token. Default: value taken from the TDC_FS_TOKEN environment variable if not provided.")
}
}
}
Expand Down Expand Up @@ -981,9 +981,9 @@ func newFSCreateFileSystemCommand(info version.Info) *cobra.Command {
})
},
}, info)
cmd.Flags().String("file-system-name", "", "File system name.")
cmd.Flags().Bool("set-default", false, "make the created file system the profile default")
cmd.Flags().Bool("wait", false, "wait until the created file system data plane is ready")
cmd.Flags().String("file-system-name", "", "The name of the file system.")
cmd.Flags().Bool("set-default", false, "Make the created file system the profile default.")
cmd.Flags().Bool("wait", false, "Wait until the created file system is active.")
markUsageRequired(cmd, "file-system-name")
return cmd
}
Expand Down Expand Up @@ -1170,19 +1170,19 @@ func newFSCopyFileCommand(info version.Info) *cobra.Command {
},
}, info)
cmd.Flags().String("from-local", "", "The local source path.")
cmd.Flags().String("from-remote", "", "The TiDB Cloud file system source path.")
cmd.Flags().String("to-local", "", "The local target path.")
cmd.Flags().String("to-remote", "", "The TiDB Cloud file system target path.")
cmd.Flags().Bool("from-stdin", false, "Read from stdin and upload to --to-remote.")
cmd.Flags().String("from-remote", "", "The source path in the TiDB Cloud file system.")
cmd.Flags().String("to-local", "", "The local destination path.")
cmd.Flags().String("to-remote", "", "The destination path in the TiDB Cloud file system.")
cmd.Flags().Bool("from-stdin", false, "Read from stdin and write to --to-remote.")
cmd.Flags().Bool("to-stdout", false, "Write --from-remote to stdout.")
cmd.Flags().Bool("overwrite", false, "Replace an existing target.")
cmd.Flags().Bool("overwrite", false, "Replace an existing destination file.")
cmd.Flags().Bool("create-parents", false, "Create missing local parent directories when copying from a TiDB Cloud file system.")
cmd.Flags().Bool("append", false, "Append a local file to a remote file in TiDB Cloud file system.")
cmd.Flags().Bool("recursive", false, "Copy directory contents recursively.")
cmd.Flags().Bool("resume", false, "Resume an active local-to-remote upload or a partial remote-to-local download.")
cmd.Flags().String("layer-id", "", "Write the copy target into a file system layer instead of the base file system.")
cmd.Flags().StringArray("tag", nil, "Create tag(s) key=value for uploads; repeatable.")
cmd.Flags().String("description", "", "The file description for local or stdin uploads.")
cmd.Flags().Bool("append", false, "Append a local file content to a file in the TiDB Cloudfile system.")
cmd.Flags().Bool("recursive", false, "Copy directory structure recursively.")
cmd.Flags().Bool("resume", false, "Resume an active copy operation.")
cmd.Flags().String("layer-id", "", "Write the copied file content into a file system layer instead of the base file system.")
cmd.Flags().StringArray("tag", nil, "Create tag(s) key=value for --to-remote operation; repeatable.")
cmd.Flags().String("description", "", "The file description for --to-remote operation.")
return cmd
}

Expand Down Expand Up @@ -1386,8 +1386,8 @@ func newFSCreateDirectoryCommand(info version.Info) *cobra.Command {
})
},
}, info)
cmd.Flags().String("path", "", "tdc fs directory path")
cmd.Flags().String("mode", "", "directory mode as an octal value such as 0755")
cmd.Flags().String("path", "", "The file system path of the directory to create.")
cmd.Flags().String("mode", "", "The directory mode as an octal value such as 0755.")
markUsageRequired(cmd, "path")
return cmd
}
Expand Down Expand Up @@ -1473,8 +1473,8 @@ func newFSHardlinkFileCommand(info version.Info) *cobra.Command {
return service.HardlinkFile(ctx.cmd.Context(), tdcfs.HardlinkFileOptions{Profile: profile, Source: source, Link: link})
},
}, info)
cmd.Flags().String("source-path", "", "existing tdc fs source file path")
cmd.Flags().String("link-path", "", "tdc fs path for the created hard link")
cmd.Flags().String("source-path", "", "The existing file path in the TiDB Cloud file system.")
cmd.Flags().String("link-path", "", "The file path for the hard link being created in the TiDB Cloud file system.")
markUsageRequired(cmd, "source-path", "link-path")
return cmd
}
Expand Down Expand Up @@ -1559,7 +1559,7 @@ func newFSFindFilesCommand(info version.Info) *cobra.Command {
func newFSCreateLayerCommand(info version.Info) *cobra.Command {
cmd := newControlPlaneCommand(controlPlaneCommandSpec{
Use: "create-layer",
Short: "Create a file system layer.",
Short: "Create a file system layer. (preview)",
Mutation: mutatingCommand,
Permission: authz.FSFileWrite,
Run: func(ctx commandContext) (any, error) {
Expand Down Expand Up @@ -1597,8 +1597,8 @@ func newFSCreateLayerCommand(info version.Info) *cobra.Command {
return service.DryRunLayerMutation(ctx.cmd.Context(), ctx.CommandPath(), "create_layer", "POST", "/v1/layers", body, profile, authz.FSFileWrite)
},
}, info)
cmd.Flags().String("layer-id", "", "optional stable layer id")
cmd.Flags().String("base-root-path", "", "base tdc fs root path for the layer")
cmd.Flags().String("layer-id", "", "Stable layer ID")
cmd.Flags().String("base-root-path", "", "Base TiDB Cloud file system root path for the layer.")
cmd.Flags().String("layer-name", "", "human-readable layer name")
cmd.Flags().StringArray("tag", nil, "layer tag key=value; repeatable")
cmd.Flags().String("durability-mode", "", "layer durability mode, for example restore-safe")
Expand All @@ -1610,7 +1610,7 @@ func newFSCreateLayerCommand(info version.Info) *cobra.Command {
func newFSListLayersCommand(info version.Info) *cobra.Command {
return newControlPlaneCommand(controlPlaneCommandSpec{
Use: "list-layers",
Short: "List file system layers for a specific file system.",
Short: "List file system layers for a specific file system. (preview)",
Mutation: readOnlyCommand,
Permission: authz.FSFileRead,
Run: func(ctx commandContext) (any, error) {
Expand All @@ -1626,7 +1626,7 @@ func newFSListLayersCommand(info version.Info) *cobra.Command {
func newFSDescribeLayerCommand(info version.Info) *cobra.Command {
cmd := newControlPlaneCommand(controlPlaneCommandSpec{
Use: "describe-layer",
Short: "Describe a specified file system layer.",
Short: "Describe a specified file system layer. (preview)",
Mutation: readOnlyCommand,
Permission: authz.FSFileRead,
Run: func(ctx commandContext) (any, error) {
Expand All @@ -1641,15 +1641,15 @@ func newFSDescribeLayerCommand(info version.Info) *cobra.Command {
return service.DescribeLayer(ctx.cmd.Context(), tdcfs.DescribeLayerOptions{Profile: profile, LayerID: layerID})
},
}, info)
cmd.Flags().String("layer-id", "", "tdc fs layer id")
cmd.Flags().String("layer-id", "", "The ID of the specified file system layer.")
markUsageRequired(cmd, "layer-id")
return cmd
}

func newFSDiffLayerCommand(info version.Info) *cobra.Command {
cmd := newControlPlaneCommand(controlPlaneCommandSpec{
Use: "diff-layer",
Short: "Show changed entries in a file system layer.",
Short: "Show changed entries in a file system layer. (preview)",
Mutation: readOnlyCommand,
Permission: authz.FSFileRead,
Run: func(ctx commandContext) (any, error) {
Expand All @@ -1673,7 +1673,7 @@ func newFSDiffLayerCommand(info version.Info) *cobra.Command {
func newFSCreateLayerCheckpointCommand(info version.Info) *cobra.Command {
cmd := newControlPlaneCommand(controlPlaneCommandSpec{
Use: "create-layer-checkpoint",
Short: "Create a layer checkpoint.",
Short: "Create a layer checkpoint. (preview)",
Mutation: mutatingCommand,
Permission: authz.FSFileWrite,
Run: func(ctx commandContext) (any, error) {
Expand Down Expand Up @@ -1746,7 +1746,7 @@ func newFSRollbackLayerCommand(info version.Info) *cobra.Command {
func newFSCommitLayerCommand(info version.Info) *cobra.Command {
cmd := newControlPlaneCommand(controlPlaneCommandSpec{
Use: "commit-layer",
Short: "Commit a layer into the base file system.",
Short: "Commit a layer into the base file system. (preview)",
Mutation: mutatingCommand,
Permission: authz.FSFileWrite,
Run: func(ctx commandContext) (any, error) {
Expand Down Expand Up @@ -1883,7 +1883,7 @@ func newFSMountFileSystemCommand(info version.Info) *cobra.Command {
return service.DryRunMountFileSystem(ctx.cmd.Context(), ctx.CommandPath(), opts)
},
}, info)
cmd.Flags().String("file-system-name", "", "tdc fs resource name; defaults to the profile default or only configured resource")
cmd.Flags().String("file-system-name", "", "The name of the file system. Default: the name of the default file system in the profile.")
cmd.Flags().String("mount-path", "", "local mount path")
cmd.Flags().String("remote-path", "/", "tdc fs remote root path to expose")
cmd.Flags().String("driver", "auto", "mount driver: auto, fuse, or webdav")
Expand Down
2 changes: 1 addition & 1 deletion internal/cli/root.go
Original file line number Diff line number Diff line change
Expand Up @@ -399,7 +399,7 @@ func newControlPlaneCommand(spec controlPlaneCommandSpec, info version.Info) *co
},
}, info)
if spec.Mutation == mutatingCommand {
cmd.Flags().Bool("dry-run", false, "validate the request without creating, updating, or deleting remote resources")
cmd.Flags().Bool("dry-run", false, "Validate the request without doing the actual changes.")
}
return cmd
}
Expand Down