Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 8 additions & 1 deletion .env.example
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
API_HOST=127.0.0.1
API_PORT=8787
WEB_HOST=0.0.0.0
WEB_PORT=3000

# LLM_* values are optional server-side defaults. Models can also be created in the Web UI after deploy.
LLM_PROVIDER=openai-compatible
LLM_MODEL=qwen-plus
LLM_BASE_URL=https://dashscope.aliyuncs.com/compatible-mode/v1
Expand Down Expand Up @@ -76,7 +79,11 @@ SECRET_MASTER_KEY=replace-with-a-long-random-local-key
DATAFOUNDRY_AUTH_MODE=password
AUTH_SESSION_SECRET=replace-with-at-least-32-random-characters
AUTH_PUBLIC_BASE_URL=http://127.0.0.1:3000
# test = log verification/reset links to console; smtp = send real email.
# Required in password mode (open|closed). open = self-register; closed = REGISTRATION_CLOSED.
# Exposed via GET /api/v1/auth/status (no secrets). Do not use AUTH_EMAIL_DELIVERY=test
# with a non-loopback AUTH_PUBLIC_BASE_URL.
AUTH_REGISTRATION_MODE=open
# Required when set: only smtp|test. test = log links to console; smtp = send real email.
AUTH_EMAIL_DELIVERY=test
AUTH_EMAIL_FROM=DataFoundry <no-reply@example.com>
AUTH_SMTP_HOST=smtp.example.com
Expand Down
40 changes: 40 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,12 @@ jobs:
name: Core Smoke Tests
runs-on: ubuntu-latest
timeout-minutes: 25
env:
DATAFOUNDRY_AUTH_MODE: password
AUTH_PUBLIC_BASE_URL: http://127.0.0.1:3000
AUTH_REGISTRATION_MODE: open
AUTH_EMAIL_DELIVERY: test
AUTH_SESSION_SECRET: ci-auth-session-secret-with-32-bytes!!
steps:
- name: Checkout
uses: actions/checkout@v5
Expand All @@ -65,6 +71,9 @@ jobs:
- name: Build TypeScript workspaces
run: npm run build

- name: Run formal auth foundation tests
run: npm run test:auth-foundation

- name: Generate built-in demo fixture
env:
SKIP_DTC_GROWTH_REGISTER: "1"
Expand Down Expand Up @@ -97,6 +106,37 @@ jobs:
- name: Run built-in DTC Growth regression
run: node --test scripts/test-builtin-dtc-growth-datasource.mjs

native-deploy-smoke:
name: Native Deploy Smoke
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- name: Checkout
uses: actions/checkout@v5

- name: Setup Node.js
uses: actions/setup-node@v5
with:
node-version: "22"
cache: npm

- name: Install dependencies
run: npm ci

- name: Run deployment unit tests
run: npm run test:deploy

- name: Run native deployment smoke
run: npm run smoke:native-deploy

- name: Upload redacted deployment logs
if: failure()
uses: actions/upload-artifact@v4
with:
name: native-deploy-logs
path: storage/logs/
if-no-files-found: ignore

docs:
name: Docs
runs-on: ubuntu-latest
Expand Down
64 changes: 26 additions & 38 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,66 +71,54 @@ See the [v0.2.0 release notes](docs/en/releases/v0.2.0.md) for the complete capa

## 🚀 Formal deploy

Default path is the **formal** stack: `password` auth + `build` / `start` (do **not** run `npm run dev`). No business database required — built-in demo data sources, including the DTC Growth Review case, work out of the box.
Formal mode has two paths (do **not** run `npm run dev`). Docker / Compose is not provided in this release.

Two formal environments share the **same startup commands**:
### Recommended: Ubuntu / Debian one-click

| Environment | Use for | Email | Public URL |
| --- | --- | --- | --- |
| **Formal test** | Local / private acceptance | `AUTH_EMAIL_DELIVERY=test` (links in API console) | `http://127.0.0.1:3000` |
| **Real production** | Public service | `AUTH_EMAIL_DELIVERY=smtp` | Public HTTPS + reverse proxy |
`./deploy.sh` generates configuration, installs dependencies, builds (Web, API, and TUI), and starts Web + API as a detached background process — closing the terminal does not stop the stack. The TUI is built and ready but is a foreground client: start it in another terminal with `./deploy.sh tui` (or `npm run start:tui`); it does **not** stay running with the stack. No model key is required during deploy — create and enable a model profile in the Web UI after login. Does **not** support native Windows / macOS.

```bash
git clone https://github.com/datagallery-lab/datafoundry.git
cd datafoundry
npm install
cp .env.example .env
cp apps/web/.env.example apps/web/.env.local
./deploy.sh
```

Configure any OpenAI-compatible model in the root `.env`, plus auth for formal test or real production (sample defaults lean formal test):
If a complete `.env` already exists, interactive deploy skips the configuration questions. To change ports or the public URL (secrets are preserved; `.env` is backed up first):

```bash
LLM_PROVIDER=openai-compatible
LLM_MODEL=qwen-plus # or deepseek-chat, gpt-4o, ...
LLM_BASE_URL=https://dashscope.aliyuncs.com/compatible-mode/v1
LLM_API_KEY=replace-with-your-key

DATAFOUNDRY_AUTH_MODE=password
AUTH_SESSION_SECRET=replace-with-at-least-32-random-characters
AUTH_PUBLIC_BASE_URL=http://127.0.0.1:3000 # real production: https://your.domain
AUTH_EMAIL_DELIVERY=test # real production: smtp + AUTH_SMTP_*
./deploy.sh deploy --reconfigure
```

`apps/web/.env.local`:
Common management commands:

```bash
NEXT_PUBLIC_DATAFOUNDRY_AUTH_MODE=password
NEXT_PUBLIC_AGENT_RUNTIME_URL=
NEXT_PUBLIC_CONFIG_API_URL=
API_PROXY_TARGET=http://127.0.0.1:8787
./deploy.sh status
./deploy.sh start
./deploy.sh stop
./deploy.sh logs
./deploy.sh doctor
./deploy.sh tui # optional: foreground TUI client (API must already be up)
```

Build and start:
Open `http://127.0.0.1:3000/login`, register or sign in, create an OpenAI-compatible model profile, then go to `/data-tasks`. For remote hosts set `AUTH_PUBLIC_BASE_URL`. Re-running `./deploy.sh deploy` uses a maintenance window (stop the managed process group before install/build).

### Windows / macOS / other: manual npm

On native Windows, macOS, or other non-Ubuntu/Debian hosts, use manual npm. Install and run in the same environment; on Windows, do not share `node_modules` between Windows and WSL.

```bash
git clone https://github.com/datagallery-lab/datafoundry.git
cd datafoundry
npm install
cp .env.example .env
cp apps/web/.env.example apps/web/.env.local
# Edit .env / apps/web/.env.local (auth + optional model; see Quick Start)
npm run build
npm run build:web
npm run start:api # :8787 — /healthz liveness, /ready readiness
npm run start:web # :3000 — password mode via same-origin BFF
```

Open `http://127.0.0.1:3000/login`, register or sign in, go to `/data-tasks`, and ask:

```text
Show me the tables in this datasource and explain the main fields of each.
npm run start # Web :3000 + API :8787
```

You will see the full chain: schema inspection → read-only SQL → SQL audit → table output → replayable run history.

For real production, also configure SMTP and a reverse proxy: [`deploy/nginx.datafoundry.conf.example`](deploy/nginx.datafoundry.conf.example) (gzip/brotli static assets; keep `/api/copilotkit` uncompressed and unbuffered for SSE).

> Full steps and the two-environment matrix: [Quick Start](docs/en/quick-start.md). Contributor hot-reload (`npm run dev`) is appendix-only. Connect your own PostgreSQL / MySQL / CSV and more: [Data Sources guide](docs/en/guides/data-sources.md).
> Full steps, the two-environment matrix,: [Quick Start](docs/en/quick-start.md). Connect your own PostgreSQL / MySQL / CSV and more: [Data Sources guide](docs/en/guides/data-sources.md).

## 🆚 How It Differs From Coding Agents And SQL Chatbots

Expand Down
64 changes: 26 additions & 38 deletions README_zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,66 +71,54 @@ DataFoundry 0.2 在首个可用版本上,进一步补齐了有状态、可追

## 🚀 正式态跑通

默认按**正式态**部署:`password` 认证 + `build` / `start`(不要跑 `npm run dev`)。不需要准备业务数据库,包含 DTC 增长经营复盘在内的内置 demo 数据源开箱即用
正式态有两条路径(都不要跑 `npm run dev`)。本版本不提供 Docker / Compose

正式态分两种环境,**启动命令相同**:
### 推荐:Ubuntu / Debian 一键部署

| 环境 | 用途 | 邮箱 | 公网地址 |
| --- | --- | --- | --- |
| **正式测试** | 本机 / 内网验收 | `AUTH_EMAIL_DELIVERY=test`(链接打控制台) | `http://127.0.0.1:3000` |
| **真实生产** | 对外服务 | `AUTH_EMAIL_DELIVERY=smtp` | 公网 HTTPS + 反代 |
`./deploy.sh` 自动生成配置、安装依赖、构建(含 Web、API 与 TUI),并以 detached 后台进程启动 Web + API——关闭终端一般不会停止服务。TUI 会在部署时构建就绪,但它是前台交互客户端:请另开终端执行 `./deploy.sh tui`(或 `npm run start:tui`),**不会**随 stack 后台常驻。部署时不要求填写模型 Key——登录后在 Web 中创建并启用模型即可。**不支持**原生 Windows / macOS。

```bash
git clone https://github.com/datagallery-lab/datafoundry.git
cd datafoundry
npm install
cp .env.example .env
cp apps/web/.env.example apps/web/.env.local
./deploy.sh
```

在根目录 `.env` 配置模型,并按正式测试或真实生产填好认证(样例默认偏正式测试):
若已有完整 `.env`,交互部署会跳过配置问答。要改端口或公开访问地址时(会保留密钥并先备份 `.env`):

```bash
LLM_PROVIDER=openai-compatible
LLM_MODEL=qwen-plus # 或 deepseek-chat、gpt-4o……
LLM_BASE_URL=https://dashscope.aliyuncs.com/compatible-mode/v1
LLM_API_KEY=replace-with-your-key

DATAFOUNDRY_AUTH_MODE=password
AUTH_SESSION_SECRET=replace-with-at-least-32-random-characters
AUTH_PUBLIC_BASE_URL=http://127.0.0.1:3000 # 真实生产改为 https://你的域名
AUTH_EMAIL_DELIVERY=test # 真实生产改为 smtp,并填 AUTH_SMTP_*
./deploy.sh deploy --reconfigure
```

`apps/web/.env.local`
常用管理命令

```bash
NEXT_PUBLIC_DATAFOUNDRY_AUTH_MODE=password
NEXT_PUBLIC_AGENT_RUNTIME_URL=
NEXT_PUBLIC_CONFIG_API_URL=
API_PROXY_TARGET=http://127.0.0.1:8787
./deploy.sh status
./deploy.sh start
./deploy.sh stop
./deploy.sh logs
./deploy.sh doctor
./deploy.sh tui # 可选:前台启动 TUI(需 API 已在运行)
```

构建并启动:
打开 `http://127.0.0.1:3000/login` 注册登录,在模型配置中创建 OpenAI-compatible Profile,然后进入 `/data-tasks`。远程部署请设置 `AUTH_PUBLIC_BASE_URL`;重复执行 `./deploy.sh deploy` 会进入维护窗口(先停止旧进程再安装/构建)。

### Windows / macOS / 其他:手动 npm

原生 Windows、macOS 或其他非 Ubuntu/Debian 环境请用手动 npm。请在同一环境内安装和运行;Windows 用户不要在 Windows 和 WSL 之间共用 `node_modules`。

```bash
git clone https://github.com/datagallery-lab/datafoundry.git
cd datafoundry
npm install
cp .env.example .env
cp apps/web/.env.example apps/web/.env.local
# 编辑 .env / apps/web/.env.local(认证与可选模型,见快速开始)
npm run build
npm run build:web
npm run start:api # :8787 — /healthz 存活,/ready 就绪
npm run start:web # :3000 — password 模式走同源 BFF
```

打开 `http://127.0.0.1:3000/login` 注册登录后进入 `/data-tasks`,提问:

```text
帮我查看数据源里有哪些表,并说明每张表的主要字段。
npm run start # Web :3000 + API :8787
```

你会看到完整的一条链路:schema 检查 → 只读 SQL → SQL 审计 → 表格产出 → 可回放的 run history。

真实生产请再配 SMTP 与反代:[`deploy/nginx.datafoundry.conf.example`](deploy/nginx.datafoundry.conf.example)(静态 gzip/brotli;`/api/copilotkit` 关压缩与缓冲以保护 SSE)。

> 完整步骤与两种环境对照见 [快速开始](docs/zh/quick-start.md);贡献者热更新(`npm run dev`)仅见该文档附录。接入自己的 PostgreSQL / MySQL / CSV 等见 [数据源指南](docs/zh/guides/data-sources.md)。
> 完整步骤与两种环境对照见 [快速开始](docs/zh/quick-start.md)。接入自己的 PostgreSQL / MySQL / CSV 等见 [数据源指南](docs/zh/guides/data-sources.md)。

## 🆚 和 Coding Agent、SQL Chatbot 有什么不同

Expand Down
Loading