Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/gateway.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ aitori run -c configs/conversations.yaml \

Notes:

- A **path** on `gateway.url` (e.g. `/api/llm/tf-edge-proxy/`) is sent verbatim as
- A **path** on `gateway.url` (e.g. `/api/llm/ai-proxy/`) is sent verbatim as
the endpoint — the gateway routes to the real upstream from `x-tfy-original-url`,
not from this path.
- `gateway.headers` adds static headers on the agent→gateway leg only. The three
Expand All @@ -49,7 +49,7 @@ Notes:
## Example: TrueFoundry AI Gateway

For the TrueFoundry AI Gateway, the exact steps — where to copy the base URL and
API key, the required `tf-edge-proxy/` path suffix, and where to save the token —
API key, the required `ai-proxy/` path suffix, and where to save the token —
are in [truefoundry_gateway.md](truefoundry_gateway.md).

## The reroute contract (gateway side)
Expand Down
2 changes: 1 addition & 1 deletion docs/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -112,7 +112,7 @@ block (or pass `--gateway-url` / `--token-file`) and LLM/MCP calls reroute throu
it instead of going straight upstream.

- **TrueFoundry AI Gateway** — where to copy the base URL + API key, and the
required `tf-edge-proxy/` suffix: [truefoundry_gateway.md](truefoundry_gateway.md).
required `ai-proxy/` suffix: [truefoundry_gateway.md](truefoundry_gateway.md).
- **Any gateway** — the reroute contract a gateway must implement, and all the
overrides: [gateway.md](gateway.md).

Expand Down
29 changes: 18 additions & 11 deletions docs/truefoundry_gateway.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,17 +18,17 @@ The base URL looks like:
https://<host>/api/llm
```

## 2. Append `tf-edge-proxy/` to the URL
## 2. Append `ai-proxy/` to the URL

> **Required.** aitori must target the gateway's edge-proxy route, not the bare
> base URL. Take the base URL from the UI and **append `tf-edge-proxy/`** (keep
> base URL. Take the base URL from the UI and **append `ai-proxy/`** (keep
> the trailing slash):
>
> ```
> https://<host>/api/llm/tf-edge-proxy/
> https://<host>/api/llm/ai-proxy/
> ```
>
> This `…/api/llm/tf-edge-proxy/` value is what you pass to aitori as the
> This `…/api/llm/ai-proxy/` value is what you pass to aitori as the
> gateway URL.

## 3. Save the token on the device
Expand All @@ -48,16 +48,16 @@ Point aitori at the gateway URL and token via flags (overriding the config), and
stop the local mock gateway if it's running:

```bash
sudo ./bin/aitori up -v -c configs/conversations.yaml \
--gateway-url "https://<host>/api/llm/tf-edge-proxy/" \
sudo aitori up -v \
--gateway-url "https://<host>/api/llm/ai-proxy/" \
--token-file ~/.aitori/tf_token
```

Or set them directly in the config instead of flags:
Or set them directly in the config `config.yaml` instead of flags:

```yaml
gateway:
url: https://<host>/api/llm/tf-edge-proxy/ # base URL + tf-edge-proxy/
url: https://<host>/api/llm/ai-proxy/ # base URL + ai-proxy/
on_error: fail_open
auth:
token_file: ~/.aitori/tf_token
Expand All @@ -67,12 +67,19 @@ gateway:
x-tfy-logging-config: '{"enabled": true}'
```

and run:

```bash
sudo aitori up -v \
-c config.yaml --ui
```

## 5. Verify

```bash
./bin/aitori status -c configs/conversations.yaml \
--gateway-url "https://<host>/api/llm/tf-edge-proxy/" --token-file ~/.aitori/tf_token
# gateway: https://<host>/api/llm/tf-edge-proxy/ [reachable]
aitori status \
--gateway-url "https://<host>/api/llm/ai-proxy/" --token-file ~/.aitori/tf_token
# gateway: https://<host>/api/llm/ai-proxy/ [reachable]
# token: ~/.aitori/tf_token (ok)
```

Expand Down
2 changes: 1 addition & 1 deletion internal/router/router.go
Original file line number Diff line number Diff line change
Expand Up @@ -208,7 +208,7 @@ func (r *Router) Rewrite(req *http.Request, appID, category string, pid int) (re

// 5. Point the upstream dial at the gateway. Scheme/host come from the
// gateway URL; when gateway.url includes a path (e.g.
// https://gw/api/llm/tf-edge-proxy) that path is used VERBATIM as the
// https://gw/api/llm/ai-proxy) that path is used VERBATIM as the
// endpoint — the request's own path is NOT appended to it. The real
// destination travels in x-tfy-original-url, which the gateway reads. A
// root/empty gateway path preserves the original request path (back-compat
Expand Down
4 changes: 2 additions & 2 deletions internal/router/router_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -85,8 +85,8 @@ func TestRewriteGatewayPathVerbatim(t *testing.T) {
for _, tc := range []struct{ gw, wantURI string }{
// The gateway path is used verbatim as the endpoint; the request's own
// path (/v1/messages) is NOT appended. Query is preserved.
{"https://gateway.truefoundry.example/api/llm/tf-edge-proxy", "/api/llm/tf-edge-proxy?beta=1"},
{"https://gateway.truefoundry.example/tf-edge-proxy/", "/tf-edge-proxy/?beta=1"}, // trailing slash kept as-is
{"https://gateway.truefoundry.example/api/llm/ai-proxy", "/api/llm/ai-proxy?beta=1"},
{"https://gateway.truefoundry.example/ai-proxy/", "/ai-proxy/?beta=1"}, // trailing slash kept as-is
// A query on gateway.url is preserved, merged ahead of the request's query.
{"https://gateway.truefoundry.example/edge?tenant=acme", "/edge?tenant=acme&beta=1"},
} {
Expand Down
Loading