Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 5 additions & 4 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

## Project Description

Scion is a copy-paste code library for Go backend development. It contains 11 self-contained modules in `registry/` — each is a standalone Go package. Modules are standard-library only by default; security-sensitive modules may be declared as `stdlibOnly:false` in `registry/index.json` and copied in standalone mode. Modules are meant to be copied into a user's project and adapted, not imported as a dependency.
Scion is a copy-paste code library for Go backend development. It contains 12 self-contained modules in `registry/` — each is a standalone Go package. Modules are standard-library only by default; security-sensitive modules may be declared as `stdlibOnly:false` in `registry/index.json` and copied in standalone mode. Modules are meant to be copied into a user's project and adapted, not imported as a dependency.

## Coding Standards

Expand Down Expand Up @@ -63,7 +63,7 @@ cd registry/<module>/src/go && go test -v -count=1 ./...

```bash
# PowerShell
$modules = @('middleware','auth','crud','rbac','ratelimit','validation','file-upload','health','cache','pagination','mail')
$modules = @('middleware','auth','crud','database','rbac','ratelimit','validation','file-upload','health','cache','pagination','mail')
foreach ($m in $modules) { Push-Location "registry/$m/src/go"; go test ./...; Pop-Location }
```

Expand All @@ -89,7 +89,7 @@ You are working on Scion, a copy-paste code library for Go backend development.
Project location: <path-to-scion>

Architecture:
- 11 modules in registry/ — each is a standalone Go package
- 12 modules in registry/ — each is a standalone Go package
- Module path pattern: registry/<module>/src/go/
- Go 1.22+, standard library by default, gofmt mandatory

Expand Down Expand Up @@ -125,9 +125,10 @@ Task: <describe your task here>

```
scion/
├── registry/ # 11 copy-paste modules
├── registry/ # 12 copy-paste modules
│ ├── auth/ # JWT auth + bcrypt
│ ├── crud/ # Generic CRUD + pagination
│ ├── database/ # database/sql helpers
│ ├── middleware/ # 9 HTTP middlewares
│ ├── rbac/ # Role-based access control
│ ├── ratelimit/ # 3 rate limiting algorithms
Expand Down
9 changes: 5 additions & 4 deletions AGENTS_zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

## 项目描述

Scion 是一个面向 Go 后端开发的复制粘贴代码库。`registry/` 目录下有 11 个自包含模块,每个都是独立的 Go 包。模块默认仅使用标准库;安全敏感模块可以在 `registry/index.json` 中显式标记为 `stdlibOnly:false`,并以 standalone 模式复制。模块的设计目的是被复制到用户项目中并适配,而非作为依赖导入。
Scion 是一个面向 Go 后端开发的复制粘贴代码库。`registry/` 目录下有 12 个自包含模块,每个都是独立的 Go 包。模块默认仅使用标准库;安全敏感模块可以在 `registry/index.json` 中显式标记为 `stdlibOnly:false`,并以 standalone 模式复制。模块的设计目的是被复制到用户项目中并适配,而非作为依赖导入。

## 编码标准

Expand Down Expand Up @@ -63,7 +63,7 @@ cd registry/<module>/src/go && go test -v -count=1 ./...

```bash
# PowerShell
$modules = @('middleware','auth','crud','rbac','ratelimit','validation','file-upload','health','cache','pagination','mail')
$modules = @('middleware','auth','crud','database','rbac','ratelimit','validation','file-upload','health','cache','pagination','mail')
foreach ($m in $modules) { Push-Location "registry/$m/src/go"; go test ./...; Pop-Location }
```

Expand All @@ -89,7 +89,7 @@ foreach ($m in $modules) { Push-Location "registry/$m/src/go"; go test ./...; Po
项目路径: <scion-项目路径>

架构:
- registry/ 下有 11 个模块 — 每个是独立的 Go 包
- registry/ 下有 12 个模块 — 每个是独立的 Go 包
- 模块路径模式: registry/<模块名>/src/go/
- Go 1.22+,默认仅使用标准库,gofmt 强制

Expand Down Expand Up @@ -125,9 +125,10 @@ foreach ($m in $modules) { Push-Location "registry/$m/src/go"; go test ./...; Po

```
scion/
├── registry/ # 11 个复制粘贴模块
├── registry/ # 12 个复制粘贴模块
│ ├── auth/ # JWT 认证 + bcrypt
│ ├── crud/ # 泛型 CRUD + 分页
│ ├── database/ # database/sql 工具
│ ├── middleware/ # 9 个 HTTP 中间件
│ ├── rbac/ # 角色权限控制
│ ├── ratelimit/ # 3 种限流算法
Expand Down
4 changes: 3 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,7 @@ Backend modules such as auth, CRUD, file upload, and rate limiting share most of
|--------|-------------|-------------------|
| [auth](registry/auth/) | JWT email/password auth + bcrypt | Rate limiting, user enumeration prevention, JTI, aud/iss validation |
| [crud](registry/crud/) | Generic CRUD with pagination | Sort/filter whitelist, SQL injection prevention, pagination ceiling |
| [database](registry/database/) | `database/sql` setup + transactions | DSN-safe errors, whitelisted SQL fragments, parameterized values |
| [middleware](registry/middleware/) | Recovery, CORS, logging, timeout, etc. | CRLF injection prevention, trusted proxy, body size limit |
| [rbac](registry/rbac/) | Role-based access control | Wildcard permissions, cycle detection, hierarchy inheritance |
| [ratelimit](registry/ratelimit/) | Fixed window / sliding window / token bucket | Memory exhaustion protection, LRU eviction, key length limit |
Expand Down Expand Up @@ -109,6 +110,7 @@ scion/
| |-- auth/ # Authentication module
| |-- cache/ # TTL + LRU cache
| |-- crud/ # CRUD operations module
| |-- database/ # database/sql helpers
| |-- file-upload/ # File upload handler
| |-- health/ # Health check probes
| |-- mail/ # SMTP email sender
Expand Down Expand Up @@ -143,7 +145,7 @@ go run ./cmd/scion doctor --strict
Run tests for all registry modules in PowerShell:

```powershell
$modules = @('middleware','auth','crud','rbac','ratelimit','validation','file-upload','health','cache','pagination','mail')
$modules = @('middleware','auth','crud','database','rbac','ratelimit','validation','file-upload','health','cache','pagination','mail')
foreach ($m in $modules) { Push-Location "registry/$m/src/go"; go test ./...; Pop-Location }
```

Expand Down
4 changes: 3 additions & 1 deletion README_zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,7 @@ Get-FileHash .\scion_v0.1.3_windows_amd64.zip -Algorithm SHA256
|------|------|----------|
| [auth](registry/auth/) | JWT 邮箱/密码认证 + bcrypt | 限流、用户枚举防护、JTI、aud/iss 校验 |
| [crud](registry/crud/) | 泛型 CRUD + 分页 | 排序/过滤白名单、SQL 注入防护、分页上限 |
| [database](registry/database/) | `database/sql` 设置 + 事务 | DSN 安全错误、SQL 片段白名单、参数化值 |
| [middleware](registry/middleware/) | Recovery、CORS、日志、超时等 | CRLF 注入防护、可信代理、请求体大小限制 |
| [rbac](registry/rbac/) | 基于角色的访问控制 | 通配符权限、循环检测、层级继承 |
| [ratelimit](registry/ratelimit/) | 固定窗口 / 滑动窗口 / 令牌桶 | 内存耗尽防护、LRU 驱逐、key 长度限制 |
Expand Down Expand Up @@ -109,6 +110,7 @@ scion/
| |-- auth/ # 认证模块
| |-- cache/ # TTL + LRU 缓存
| |-- crud/ # CRUD 模块
| |-- database/ # database/sql 工具
| |-- file-upload/ # 文件上传模块
| |-- health/ # 健康检查模块
| |-- mail/ # SMTP 邮件模块
Expand Down Expand Up @@ -143,7 +145,7 @@ go run ./cmd/scion doctor --strict
PowerShell 中运行所有 registry 模块测试:

```powershell
$modules = @('middleware','auth','crud','rbac','ratelimit','validation','file-upload','health','cache','pagination','mail')
$modules = @('middleware','auth','crud','database','rbac','ratelimit','validation','file-upload','health','cache','pagination','mail')
foreach ($m in $modules) { Push-Location "registry/$m/src/go"; go test ./...; Pop-Location }
```

Expand Down
2 changes: 2 additions & 0 deletions docs/.vitepress/config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ export default defineConfig({
{ text: 'Overview', link: '/modules/' },
{ text: 'Auth', link: '/modules/auth' },
{ text: 'CRUD', link: '/modules/crud' },
{ text: 'Database', link: '/modules/database' },
{ text: 'Middleware', link: '/modules/middleware' },
{ text: 'RBAC', link: '/modules/rbac' },
{ text: 'Rate Limit', link: '/modules/ratelimit' },
Expand Down Expand Up @@ -85,6 +86,7 @@ export default defineConfig({
{ text: '概览', link: '/zh/modules/' },
{ text: 'Auth 认证', link: '/zh/modules/auth' },
{ text: 'CRUD 增删改查', link: '/zh/modules/crud' },
{ text: 'Database 数据库', link: '/zh/modules/database' },
{ text: 'Middleware 中间件', link: '/zh/modules/middleware' },
{ text: 'RBAC 权限控制', link: '/zh/modules/rbac' },
{ text: 'Rate Limit 限流', link: '/zh/modules/ratelimit' },
Expand Down
2 changes: 1 addition & 1 deletion docs/contributing.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ Add your module to `registry/index.json`.
cd registry/<module>/src/go && go test -v ./...

# Test all modules
$modules = @('middleware','auth','crud','rbac','ratelimit','validation','file-upload','health','cache','pagination','mail')
$modules = @('middleware','auth','crud','database','rbac','ratelimit','validation','file-upload','health','cache','pagination','mail')
foreach ($m in $modules) { Push-Location "registry/$m/src/go"; go test ./...; Pop-Location }
```

Expand Down
4 changes: 4 additions & 0 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,10 @@ features:
title: CRUD
details: Generic CRUD operations with pagination, sort/filter whitelist, SQL injection prevention.
link: /modules/crud
- icon: SQL
title: Database
details: database/sql setup, transactions, and whitelisted query fragments.
link: /modules/database
- icon: 🛡️
title: Middleware
details: Recovery, CORS, logging, timeout, request ID, body size limit.
Expand Down
91 changes: 91 additions & 0 deletions docs/modules/database.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
# Database Module

`database/sql` setup, transaction helpers, and safe query fragments.

## What's Included

- Bounded dynamic connection pool defaults
- Dynamic pool sizing based on CPU and IO capacity
- `FromEnv()` for `DATABASE_*` settings
- Startup ping with timeout and DSN-safe errors
- `WithinTx` helper with rollback on error or panic
- `DBTX` interface for repositories
- Whitelisted `WHERE` and `ORDER BY` fragment builders
- Maximum 32 filter conditions per builder/query

## Quick Copy

```bash
cp -r registry/database/src/go/* yourproject/internal/database/
```

## Usage

```go
opts := database.FromEnv()
db, err := database.Open(ctx, opts)
if err != nil {
return err
}
defer db.Close()

err = database.WithinTx(ctx, db, nil, func(ctx context.Context, tx *sql.Tx) error {
_, err := tx.ExecContext(ctx, "UPDATE users SET active = ? WHERE id = ?", true, 123)
return err
})
```

## Safe Fragments

```go
columns := database.ColumnMap{
"email": "users.email",
"created_at": "users.created_at",
}

where, args, err := database.WhereEqual(
map[string]string{"email": "ada@example.com"},
columns,
database.DollarPlaceholder,
)
order, err := database.OrderBy("-created_at", columns)
```

## Dynamic Pool Sizing

`Defaults()` uses a balanced dynamic pool based on `GOMAXPROCS`. For IO-heavy
services, opt into a larger bounded pool:

```go
opts := database.ApplyPoolStrategy(database.FromEnv(), database.PoolStrategy{
Profile: database.PoolIOHeavy,
CPUCores: 8,
IOParallelism: 4,
MaxOpenLimit: 96,
})
```

Environment knobs:

| Variable | Purpose |
|----------|---------|
| `DATABASE_POOL_PROFILE` | `conservative`, `balanced`, or `io-heavy` |
| `DATABASE_POOL_CPU_CORES` | CPU cores used by the formula |
| `DATABASE_IO_PARALLELISM` | extra independent IO capacity |
| `DATABASE_POOL_MAX_OPEN_LIMIT` | hard cap for derived max open connections |

## Security Features

- Rejects CRLF and null bytes in string inputs
- Enforces length limits on driver names, DSNs, fields, and values
- Requires whitelist mapping for SQL identifiers
- Keeps filter values parameterized
- Caps generated filter conditions to avoid unbounded memory growth
- Does not include DSNs in open or ping errors

## Tests

```bash
cd registry/database/src/go
go test -v ./...
```
3 changes: 2 additions & 1 deletion docs/modules/index.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,14 @@
# Modules Overview

Scion provides 11 production-ready, copy-paste Go modules. Each module is self-contained. Modules are standard-library only by default; declared security exceptions are marked in the registry.
Scion provides 12 production-ready, copy-paste Go modules. Each module is self-contained. Modules are standard-library only by default; declared security exceptions are marked in the registry.

## Available Modules

| Module | Description | Security Features |
|--------|-------------|-------------------|
| [Auth](/modules/auth) | JWT email/password auth + bcrypt | Rate limiting, user enumeration prevention, JTI |
| [CRUD](/modules/crud) | Generic CRUD with pagination | Sort/filter whitelist, SQL injection prevention |
| [Database](/modules/database) | `database/sql` setup + transactions | DSN-safe errors, whitelisted SQL fragments |
| [Middleware](/modules/middleware) | Recovery, CORS, logging, timeout | CRLF injection prevention, body size limit |
| [RBAC](/modules/rbac) | Role-based access control | Wildcard permissions, cycle detection |
| [Rate Limit](/modules/ratelimit) | Fixed/sliding window, token bucket | Memory exhaustion protection, LRU eviction |
Expand Down
2 changes: 1 addition & 1 deletion docs/zh/contributing.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ go test -v ./...
cd registry/<module>/src/go && go test -v ./...

# 测试所有模块
$modules = @('middleware','auth','crud','rbac','ratelimit','validation','file-upload','health','cache','pagination','mail')
$modules = @('middleware','auth','crud','database','rbac','ratelimit','validation','file-upload','health','cache','pagination','mail')
foreach ($m in $modules) { Push-Location "registry/$m/src/go"; go test ./...; Pop-Location }
```

Expand Down
4 changes: 4 additions & 0 deletions docs/zh/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,10 @@ features:
title: CRUD 增删改查
details: 通用 CRUD 操作 + 分页,排序/过滤白名单,SQL注入防护。
link: /zh/modules/crud
- icon: SQL
title: Database 数据库
details: database/sql 设置、事务封装和 SQL 片段白名单。
link: /zh/modules/database
- icon: 🛡️
title: Middleware 中间件
details: Recovery、CORS、日志、超时、请求ID、请求体大小限制。
Expand Down
91 changes: 91 additions & 0 deletions docs/zh/modules/database.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
# Database 数据库模块

`database/sql` 设置、事务封装和安全 SQL 片段构建。

## 包含内容

- 显式连接池默认值
- 基于 CPU 和 IO 能力的动态连接池 sizing
- 基于 `DATABASE_*` 的 `FromEnv()`
- 带超时的启动 ping,错误不暴露 DSN
- `WithinTx`,在错误或 panic 时回滚
- 供 repository 使用的 `DBTX` 接口
- 基于白名单的 `WHERE` 和 `ORDER BY` 片段构建
- 每个 builder/query 最多 32 个过滤条件

## 快速复制

```bash
cp -r registry/database/src/go/* yourproject/internal/database/
```

## 使用方式

```go
opts := database.FromEnv()
db, err := database.Open(ctx, opts)
if err != nil {
return err
}
defer db.Close()

err = database.WithinTx(ctx, db, nil, func(ctx context.Context, tx *sql.Tx) error {
_, err := tx.ExecContext(ctx, "UPDATE users SET active = ? WHERE id = ?", true, 123)
return err
})
```

## 安全 SQL 片段

```go
columns := database.ColumnMap{
"email": "users.email",
"created_at": "users.created_at",
}

where, args, err := database.WhereEqual(
map[string]string{"email": "ada@example.com"},
columns,
database.DollarPlaceholder,
)
order, err := database.OrderBy("-created_at", columns)
```

## 动态连接池策略

`Defaults()` 默认使用基于 `GOMAXPROCS` 的 balanced 动态连接池。IO-heavy
服务可以显式使用更大的有界连接池:

```go
opts := database.ApplyPoolStrategy(database.FromEnv(), database.PoolStrategy{
Profile: database.PoolIOHeavy,
CPUCores: 8,
IOParallelism: 4,
MaxOpenLimit: 96,
})
```

环境变量:

| 变量 | 用途 |
|------|------|
| `DATABASE_POOL_PROFILE` | `conservative`、`balanced` 或 `io-heavy` |
| `DATABASE_POOL_CPU_CORES` | sizing 公式使用的 CPU 核数 |
| `DATABASE_IO_PARALLELISM` | 额外独立 IO 能力 |
| `DATABASE_POOL_MAX_OPEN_LIMIT` | 推导出的最大连接数硬上限 |

## 安全特性

- 拒绝字符串输入中的 CRLF 和 null 字节
- 对 driver 名称、DSN、字段和值设置长度限制
- SQL 标识符必须来自白名单映射
- 过滤值只作为参数返回,不拼接进 SQL
- 限制生成的过滤条件数量,避免无界内存增长
- open 或 ping 错误不包含 DSN

## 测试

```bash
cd registry/database/src/go
go test -v ./...
```
3 changes: 2 additions & 1 deletion docs/zh/modules/index.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,14 @@
# 模块概览

Scion 提供 11 个生产就绪、可复制粘贴的 Go 模块。每个模块自包含。模块默认仅使用标准库;安全例外会在 registry 中显式标记。
Scion 提供 12 个生产就绪、可复制粘贴的 Go 模块。每个模块自包含。模块默认仅使用标准库;安全例外会在 registry 中显式标记。

## 可用模块

| 模块 | 描述 | 安全特性 |
|------|------|---------|
| [Auth](/zh/modules/auth) | JWT 认证 + bcrypt | 限流,用户枚举防护,JTI |
| [CRUD](/zh/modules/crud) | 通用 CRUD + 分页 | 排序/过滤白名单,SQL注入防护 |
| [Database](/zh/modules/database) | `database/sql` 设置 + 事务 | DSN 安全错误,SQL 片段白名单 |
| [Middleware](/zh/modules/middleware) | Recovery、CORS、日志、超时 | CRLF 注入防护,请求体大小限制 |
| [RBAC](/zh/modules/rbac) | 基于角色的访问控制 | 通配符权限,循环检测 |
| [Rate Limit](/zh/modules/ratelimit) | 固定/滑动窗口、令牌桶 | 内存耗尽防护,LRU 淘汰 |
Expand Down
Loading
Loading