Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
API_HOST=127.0.0.1
API_PORT=8787
WEB_HOST=0.0.0.0
WEB_PORT=3000

# LLM_* values are optional server-side defaults. Models can also be created in the Web UI after deploy.
LLM_PROVIDER=openai-compatible
LLM_MODEL=qwen-plus
LLM_BASE_URL=https://dashscope.aliyuncs.com/compatible-mode/v1
Expand Down
31 changes: 31 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -156,6 +156,37 @@ jobs:
done
test "$status" != "000"

native-deploy-smoke:
name: Native Deploy Smoke
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- name: Checkout
uses: actions/checkout@v5

- name: Setup Node.js
uses: actions/setup-node@v5
with:
node-version: "22"
cache: npm

- name: Install dependencies
run: npm ci

- name: Run deployment unit tests
run: npm run test:deploy

- name: Run native deployment smoke
run: npm run smoke:native-deploy

- name: Upload redacted deployment logs
if: failure()
uses: actions/upload-artifact@v4
with:
name: native-deploy-logs
path: storage/logs/
if-no-files-found: ignore

docs:
name: Docs
runs-on: ubuntu-latest
Expand Down
69 changes: 26 additions & 43 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,71 +71,54 @@ See the [v0.2.0 release notes](docs/en/releases/v0.2.0.md) for the complete capa

## 🚀 Formal deploy

Default path is the **formal** stack: `password` auth + `build` / `start` (do **not** run `npm run dev`). No business database required — built-in demo data sources, including the DTC Growth Review case, work out of the box.
Formal mode has two paths (do **not** run `npm run dev`). Docker / Compose is not provided in this release.

Two formal environments share the **same startup commands**:
### Recommended: Ubuntu / Debian one-click

| Environment | Use for | Email | Public URL |
| --- | --- | --- | --- |
| **Formal test** | Local / private acceptance | `AUTH_EMAIL_DELIVERY=test` (links in API console) | `http://127.0.0.1:3000` |
| **Real production** | Public service | `AUTH_EMAIL_DELIVERY=smtp` | Public HTTPS + reverse proxy |
`./deploy.sh` generates configuration, installs dependencies, builds (Web, API, and TUI), and starts Web + API as a detached background process — closing the terminal does not stop the stack. The TUI is built and ready but is a foreground client: start it in another terminal with `./deploy.sh tui` (or `npm run start:tui`); it does **not** stay running with the stack. DataLink stays off by default, and no model key is required during deploy — create and enable a model profile in the Web UI after login. Does **not** support native Windows / macOS.

```bash
git clone https://github.com/datagallery-lab/datafoundry.git
cd datafoundry
npm install
cp .env.example .env
cp apps/web/.env.example apps/web/.env.local
./deploy.sh
```

Configure any OpenAI-compatible model in the root `.env`, plus auth for formal test or real production (sample defaults lean formal test):
If a complete `.env` already exists, interactive deploy skips the configuration questions. To change ports, DataLink, or the public URL (secrets are preserved; `.env` is backed up first):

```bash
LLM_PROVIDER=openai-compatible
LLM_MODEL=qwen-plus # or deepseek-chat, gpt-4o, ...
LLM_BASE_URL=https://dashscope.aliyuncs.com/compatible-mode/v1
LLM_API_KEY=replace-with-your-key

# Optional: launch the built-in semantic service with the main stack.
# Requires Python 3.10+ and uv; then run `npm run install:datalink` once.
DATALINK_ENABLED=true

DATAFOUNDRY_AUTH_MODE=password
AUTH_SESSION_SECRET=replace-with-at-least-32-random-characters
AUTH_PUBLIC_BASE_URL=http://127.0.0.1:3000 # real production: https://your.domain
AUTH_EMAIL_DELIVERY=test # real production: smtp + AUTH_SMTP_*
./deploy.sh deploy --reconfigure
```

`apps/web/.env.local`:
Common management commands:

```bash
NEXT_PUBLIC_DATAFOUNDRY_AUTH_MODE=password
NEXT_PUBLIC_AGENT_RUNTIME_URL=
NEXT_PUBLIC_CONFIG_API_URL=
API_PROXY_TARGET=http://127.0.0.1:8787
./deploy.sh status
./deploy.sh start
./deploy.sh stop
./deploy.sh logs
./deploy.sh doctor
./deploy.sh tui # optional: foreground TUI client (API must already be up)
```

Build and start:
Open `http://127.0.0.1:3000/login`, register or sign in, create an OpenAI-compatible model profile, then go to `/data-tasks`. For remote hosts set `AUTH_PUBLIC_BASE_URL`. Re-running `./deploy.sh deploy` uses a maintenance window (stop the managed process group before install/build).

### Windows / macOS / other: manual npm

On native Windows, macOS, or other non-Ubuntu/Debian hosts, use manual npm. Install and run in the same environment; on Windows, do not share `node_modules` between Windows and WSL.

```bash
git clone https://github.com/datagallery-lab/datafoundry.git
cd datafoundry
npm install
cp .env.example .env
cp apps/web/.env.example apps/web/.env.local
# Edit .env / apps/web/.env.local (auth + optional model; see Quick Start)
npm run build
npm run build:web
npm run start # Web :3000 + API :8787 (+ DataLink :8080/:8081 when enabled)
npm run start # Web :3000 + API :8787
```

For split-process deployment, `start:api` and `start:web` remain available; start DataLink separately with `start:datalink:mcp` and `start:datalink:api` when enabled.

Open `http://127.0.0.1:3000/login`, register or sign in, go to `/data-tasks`, and ask:

```text
Show me the tables in this datasource and explain the main fields of each.
```

You will see the full chain: schema inspection → read-only SQL → SQL audit → table output → replayable run history.

For real production, also configure SMTP and a reverse proxy: [`deploy/nginx.datafoundry.conf.example`](deploy/nginx.datafoundry.conf.example) (gzip/brotli static assets; keep `/api/copilotkit` uncompressed and unbuffered for SSE).

> Full steps and the two-environment matrix: [Quick Start](docs/en/quick-start.md). Contributor hot-reload (`npm run dev`) is appendix-only. Connect your own PostgreSQL / MySQL / CSV and more: [Data Sources guide](docs/en/guides/data-sources.md).
> Full steps, the two-environment matrix, and optional DataLink notes: [Quick Start](docs/en/quick-start.md). Connect your own PostgreSQL / MySQL / CSV and more: [Data Sources guide](docs/en/guides/data-sources.md).

## 🆚 How It Differs From Coding Agents And SQL Chatbots

Expand Down
69 changes: 26 additions & 43 deletions README_zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,71 +71,54 @@ DataFoundry 0.2 在首个可用版本上,进一步补齐了有状态、可追

## 🚀 正式态跑通

默认按**正式态**部署:`password` 认证 + `build` / `start`(不要跑 `npm run dev`)。不需要准备业务数据库,包含 DTC 增长经营复盘在内的内置 demo 数据源开箱即用
正式态有两条路径(都不要跑 `npm run dev`)。本版本不提供 Docker / Compose

正式态分两种环境,**启动命令相同**:
### 推荐:Ubuntu / Debian 一键部署

| 环境 | 用途 | 邮箱 | 公网地址 |
| --- | --- | --- | --- |
| **正式测试** | 本机 / 内网验收 | `AUTH_EMAIL_DELIVERY=test`(链接打控制台) | `http://127.0.0.1:3000` |
| **真实生产** | 对外服务 | `AUTH_EMAIL_DELIVERY=smtp` | 公网 HTTPS + 反代 |
`./deploy.sh` 自动生成配置、安装依赖、构建(含 Web、API 与 TUI),并以 detached 后台进程启动 Web + API——关闭终端一般不会停止服务。TUI 会在部署时构建就绪,但它是前台交互客户端:请另开终端执行 `./deploy.sh tui`(或 `npm run start:tui`),**不会**随 stack 后台常驻。默认关闭 DataLink,部署时不要求填写模型 Key——登录后在 Web 中创建并启用模型即可。**不支持**原生 Windows / macOS。

```bash
git clone https://github.com/datagallery-lab/datafoundry.git
cd datafoundry
npm install
cp .env.example .env
cp apps/web/.env.example apps/web/.env.local
./deploy.sh
```

在根目录 `.env` 配置模型,并按正式测试或真实生产填好认证(样例默认偏正式测试):
若已有完整 `.env`,交互部署会跳过配置问答。要改端口、DataLink 或公开访问地址时(会保留密钥并先备份 `.env`):

```bash
LLM_PROVIDER=openai-compatible
LLM_MODEL=qwen-plus # 或 deepseek-chat、gpt-4o……
LLM_BASE_URL=https://dashscope.aliyuncs.com/compatible-mode/v1
LLM_API_KEY=replace-with-your-key

# 可选:随主应用启动内置语义服务。
# 需要 Python 3.10+ 与 uv,并先执行一次 `npm run install:datalink`。
DATALINK_ENABLED=true

DATAFOUNDRY_AUTH_MODE=password
AUTH_SESSION_SECRET=replace-with-at-least-32-random-characters
AUTH_PUBLIC_BASE_URL=http://127.0.0.1:3000 # 真实生产改为 https://你的域名
AUTH_EMAIL_DELIVERY=test # 真实生产改为 smtp,并填 AUTH_SMTP_*
./deploy.sh deploy --reconfigure
```

`apps/web/.env.local`
常用管理命令

```bash
NEXT_PUBLIC_DATAFOUNDRY_AUTH_MODE=password
NEXT_PUBLIC_AGENT_RUNTIME_URL=
NEXT_PUBLIC_CONFIG_API_URL=
API_PROXY_TARGET=http://127.0.0.1:8787
./deploy.sh status
./deploy.sh start
./deploy.sh stop
./deploy.sh logs
./deploy.sh doctor
./deploy.sh tui # 可选:前台启动 TUI(需 API 已在运行)
```

构建并启动:
打开 `http://127.0.0.1:3000/login` 注册登录,在模型配置中创建 OpenAI-compatible Profile,然后进入 `/data-tasks`。远程部署请设置 `AUTH_PUBLIC_BASE_URL`;重复执行 `./deploy.sh deploy` 会进入维护窗口(先停止旧进程再安装/构建)。

### Windows / macOS / 其他:手动 npm

原生 Windows、macOS 或其他非 Ubuntu/Debian 环境请用手动 npm。请在同一环境内安装和运行;Windows 用户不要在 Windows 和 WSL 之间共用 `node_modules`。

```bash
git clone https://github.com/datagallery-lab/datafoundry.git
cd datafoundry
npm install
cp .env.example .env
cp apps/web/.env.example apps/web/.env.local
# 编辑 .env / apps/web/.env.local(认证与可选模型,见快速开始)
npm run build
npm run build:web
npm run start # Web :3000 + API :8787(启用后含 DataLink :8080/:8081)
npm run start # Web :3000 + API :8787
```

需要拆分进程部署时,仍可使用 `start:api` 与 `start:web`;开启 DataLink 后再分别运行 `start:datalink:mcp` 和 `start:datalink:api`。

打开 `http://127.0.0.1:3000/login` 注册登录后进入 `/data-tasks`,提问:

```text
帮我查看数据源里有哪些表,并说明每张表的主要字段。
```

你会看到完整的一条链路:schema 检查 → 只读 SQL → SQL 审计 → 表格产出 → 可回放的 run history。

真实生产请再配 SMTP 与反代:[`deploy/nginx.datafoundry.conf.example`](deploy/nginx.datafoundry.conf.example)(静态 gzip/brotli;`/api/copilotkit` 关压缩与缓冲以保护 SSE)。

> 完整步骤与两种环境对照见 [快速开始](docs/zh/quick-start.md);贡献者热更新(`npm run dev`)仅见该文档附录。接入自己的 PostgreSQL / MySQL / CSV 等见 [数据源指南](docs/zh/guides/data-sources.md)。
> 完整步骤、两种环境对照与 DataLink 可选说明见 [快速开始](docs/zh/quick-start.md)。接入自己的 PostgreSQL / MySQL / CSV 等见 [数据源指南](docs/zh/guides/data-sources.md)。

## 🆚 和 Coding Agent、SQL Chatbot 有什么不同

Expand Down
4 changes: 4 additions & 0 deletions apps/web/.env.example
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
# DataFoundry Web — formal deploy defaults (password + same-origin BFF).
#
# Native one-click deploy (`./deploy.sh`) generates `apps/web/.env.local` for you.
# It uses same-origin BFF variables (empty NEXT_PUBLIC_* URLs) and sets
# API_PROXY_TARGET from the selected API host/port.
#
# Keep in sync with root `.env` → DATAFOUNDRY_AUTH_MODE.
# NEXT_PUBLIC_* values are baked in at `npm run build:web`.
#
Expand Down
16 changes: 16 additions & 0 deletions deploy.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
#!/usr/bin/env bash
set -Eeuo pipefail

ROOT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
# shellcheck source=scripts/deploy/bootstrap.sh
source "${ROOT_DIR}/scripts/deploy/bootstrap.sh"

main() {
check_supported_system
ensure_node_22 "$@"
exec node "$ROOT_DIR/scripts/deploy/cli.mjs" "$@"
}

if [[ "${BASH_SOURCE[0]}" == "$0" ]]; then
main "$@"
fi
2 changes: 1 addition & 1 deletion docs/en/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ Review a realistic growth spike with schema inspection, read-only SQL, and trace
## Recommended path

1. Read [Product overview](overview.md) to confirm the problem space and capability boundaries.
2. Follow [Quick start](quick-start.md) to configure a model API key and run the built-in DTC Growth Review.
2. Follow [Quick start](quick-start.md): on Ubuntu / Debian prefer `./deploy.sh`; on Windows / macOS and other hosts use manual npm (`npm install` → configure → `build` / `start`). Then create a model profile in the Web UI and run the built-in DTC Growth Review.
3. Read [Capabilities](capabilities.md) to see coverage across Web, TUI, and backend API.
4. Choose [Web workbench guide](guides/web-workbench.md) or [TUI guide](guides/tui.md) based on your entry point.
5. When you need your own data, read [Data sources guide](guides/data-sources.md).
Expand Down
Loading
Loading