Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ Use the following methods to implement AI Transport message-per-response streami
- [`channel.subscribe()`](/docs/channels#subscribe): subscribes to messages, handling `message.create`, `message.append`, and `message.update` actions.
- [`channel.setOptions()`](/docs/channels/options) with [`rewind`](/docs/channels/options/rewind): enables seamless message recovery during reconnections, delivering historical messages as `message.update` events.

Find out more about [AI Transport](/docs/ai-transport) and [message appending](/docs/ai-transport/features/token-streaming/message-per-response).
Find out more about [AI Transport](/docs/ai-transport) and [message appending](/docs/ai-transport/token-streaming/message-per-response).

## Getting started

Expand Down
4 changes: 2 additions & 2 deletions examples/ai-transport-message-per-token/javascript/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ Use the following methods to implement AI Transport token streaming:
- [`channel.publish()`](/docs/channels#publish): publishes individual tokens as they arrive from the LLM service with response tracking headers.
- [`channel.history()`](/docs/channels/history) with [`untilAttach`](/docs/channels/options#attach): enables seamless message recovery during reconnections, ensuring no tokens are lost.

Find out more about [AI Transport](/docs/ai-transport) and [message history](/docs/channels/history).
Find out more about [AI Transport](/docs/ai-transport), [token streaming](/docs/ai-transport/token-streaming), and [message history](/docs/storage-history/history).

## Getting started

Expand Down Expand Up @@ -57,4 +57,4 @@ Find out more about [AI Transport](/docs/ai-transport) and [message history](/do

## Open in CodeSandbox

In CodeSandbox, rename the `.env.example` file to `.env.local` and update the value of your `VITE_ABLY_KEY` variable to use your Ably API key.
In CodeSandbox, rename the `.env.example` file to `.env.local` and update the value of your `VITE_ABLY_KEY` variable to use your Ably API key.
22 changes: 13 additions & 9 deletions src/pages/docs/ai-transport/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ meta_description: "Learn more about Ably's AI Transport and the features that en

AI Transport enables you to add a realtime delivery layer to your application, providing the infrastructure required to deliver modern, stateful AI experiences to users. It works seamlessly with any AI model or framework, such as OpenAI, Anthropic, Vercel or LangChain.

AI Transport runs on Ably's [fault-tolerant](/docs/platform/architecture/fault-tolerance) and highly-available platform. The platform enables data to be streamed between all internet-connected devices at [low latencies](/docs/platform/architecture/latency) across the globe. Its elastic global infrastructure delivers enterprise-scale messaging that [effortlessly scales](/docs/platform/architecture/platform-scalability) to meet demand.
AI Transport runs on Ably's [fault-tolerant](/docs/platform/architecture/fault-tolerance) and highly-available platform. The platform supports streaming data between all internet-connected devices at [low latencies](/docs/platform/architecture/latency) across the globe. Its elastic global infrastructure delivers enterprise-scale messaging that [effortlessly scales](/docs/platform/architecture/platform-scalability) to meet demand.

Drop AI Transport into your applications to transform them into modern, bi-directional AI experiences that keep users engaged. AI Transport provides the building blocks to deliver reliable, resumable token streams with robust session management and state hydration to always keep your users and agents in sync.

Expand All @@ -18,6 +18,8 @@ Start learning the basics of AI Transport right away with a getting started guid

### OpenAI

Use the following guides to get started with OpenAI:

<Tiles>
{[
{
Expand All @@ -37,6 +39,8 @@ Start learning the basics of AI Transport right away with a getting started guid

### Anthropic

Use the following guides to get started with Anthropic:

<Tiles>
{[
{
Expand Down Expand Up @@ -65,17 +69,17 @@ Token streaming is the core of how LLMs deliver their responses to users. Tokens

Using AI Transport, your token streams are reliable and persistent. They survive modern environments where users change browser tabs, refresh the page or switch devices, and common interruptions such as temporary network loss. Your users can always reconnect and continue where they left off without having to start over.

[Read more about token streaming](/docs/ai-transport/features/token-streaming).
[Read more about token streaming](/docs/ai-transport/token-streaming).

### Bi-directional communication <a id="communication"/>

AI Transport supports rich, bi-directional communication patterns between users and agents.

Build sophisticated AI experiences with features like accepting user input for interactive conversations, streaming chain-of-thought reasoning for transparency, attaching citations to responses for verifiability, implementing human-in-the-loop workflows for sensitive operations, and exposing tool calls for generative UI and visibility.

These messaging features work seamlessly with token streaming to create complete, interactive AI experiences.
These messaging features work seamlessly with [token streaming](/docs/ai-transport/token-streaming) to create complete, interactive AI experiences.

[Read more about messaging features](/docs/ai-transport/features/messaging).
[Read more about messaging features](/docs/ai-transport/messaging/accepting-user-input).

### Durable sessions <a id="sessions"/>

Expand All @@ -85,25 +89,25 @@ Communication shouldn't be tied to the connection state of either party. If a us

Your users can start a conversation on their mobile and seamlessly continue it on their desktop. Similarly, multiple users can participate in the same conversation with a single agent and they will all remain in sync, in realtime.

[Read more about sessions and identity](/docs/ai-transport/features/sessions-identity).
[Read more about sessions and identity](/docs/ai-transport/sessions-identity).

### Automatic catch-up <a id="catch-up"/>

AI Transport enables clients to hydrate conversation and session state from the channel, including message history and in-progress responses.
AI Transport enables clients to hydrate conversation and session state from the [channel](/docs/channels), including [message history](/docs/storage-history/history) and in-progress responses.

Whether a user is briefly disconnected when they drive through a tunnel, or they're rejoining a conversation the following day of work, AI Transport allows clients to resynchronise the full conversation state, including both historical messages and in-progress responses. Your users are always up to date with the full conversation, in order, anywhere.

[Read more about client hydration](/docs/ai-transport/features/token-streaming/message-per-response#hydration).
[Read more about client hydration](/docs/ai-transport/token-streaming/message-per-response#hydration).

### Background processing <a id="background"/>

AI Transport allows agents to process jobs in the background while users go offline, with full awareness of their online status through realtime presence tracking.

Users can work asynchronously by prompting an agent to perform a task without having to monitor its progress. They can go offline and receive a push notification when the agent has completed the task, or reconnect at any time to seamlessly resume and see all progress made while they were away using [state hydration](#hydration).
Users can work asynchronously by prompting an agent to perform a task without having to monitor its progress. They can go offline and receive a push notification when the agent has completed the task, or reconnect at any time to seamlessly resume and see all progress made while they were away using [state hydration](#catch-up).

It also puts you in control of how you manage your application when there aren't any users online. For example, you can choose whether to pause a conversation when a user exits their browser tab, or allow the agent to complete its response for the user to view when they return.

[Read more about status-aware cost controls](/docs/ai-transport/features/sessions-identity/online-status).
[Read more about status-aware cost controls](/docs/ai-transport/sessions-identity/online-status).

### Enterprise controls <a id="enterprise"/>

Expand Down
16 changes: 8 additions & 8 deletions src/pages/docs/ai-transport/messaging/chain-of-thought.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -26,15 +26,15 @@ As an application developer, you decide how to surface chain-of-thought reasonin

### Inline pattern <a id="inline"/>

In the inline pattern, reasoning messages are published to the same channel as model output messages.
In the inline pattern, agents publish reasoning messages to the same channel as model output messages.

By publishing all content to a single channel, the inline pattern:

- Simplifies channel management by consolidating all conversation content in one place
- Maintains relative order of reasoning and model output messages as they are generated
- Maintains relative order of reasoning and model output messages as the model generates them
- Supports retrieving reasoning and response messages together from history

#### Publishing
#### Publish

Publish both reasoning and model output messages to a single channel.

Expand Down Expand Up @@ -86,7 +86,7 @@ To learn how to stream individual tokens as they are generated, see the [token s
Set [`echoMessages`](/docs/api/realtime-sdk/types#client-options) to `false` on the agent's Ably client to prevent the agent from receiving its own reasoning and output messages, avoiding billing for [echoed messages](/docs/pub-sub/advanced#echo).
</Aside>

#### Subscribing
#### Subscribe

Subscribe to both reasoning and model output messages on the same channel.

Expand Down Expand Up @@ -140,19 +140,19 @@ To learn about hydrating responses from channel history, including using `rewind

### Threading pattern <a id="threading"/>

In the threading pattern, reasoning messages are published to a separate channel from model output messages. The reasoning channel name is communicated to clients, allowing them to discover where to obtain reasoning content on demand.
In the threading pattern, agents publish reasoning messages to a separate channel from model output messages. The reasoning channel name is communicated to clients, allowing them to discover where to obtain reasoning content on demand.

By separating reasoning into its own channel, the threading pattern:

- Keeps the main channel clean and focused on final responses without reasoning output cluttering the conversation history
- Reduces bandwidth usage by delivering reasoning messages only when users choose to view them
- Works well for long reasoning threads, where not all the detail needs to be immediately surfaced to the user, but is helpful to see on demand

#### Publishing
#### Publish

Publish model output messages to the main conversation channel and reasoning messages to a dedicated reasoning channel. The reasoning channel name includes the response ID, creating a unique reasoning channel per response.

In the example below, a `start` control message is sent on the main channel at the beginning of each response, which includes the response ID in the message [`extras`](/docs/api/realtime-sdk/messages#extras). Clients can derive the reasoning channel name from the response ID, allowing them to discover and subscribe to the stream of reasoning messages on demand:
In the example below, the agent sends a `start` control message on the main channel at the beginning of each response, which includes the response ID in the message [`extras`](/docs/api/realtime-sdk/messages#extras). Clients can derive the reasoning channel name from the response ID, allowing them to discover and subscribe to the stream of reasoning messages on demand:

<Aside data-type="note">
Ably channels are [created when used](/docs/channels#use), so you can dynamically create unique reasoning channels on demand.
Expand Down Expand Up @@ -212,7 +212,7 @@ To learn how to stream individual tokens as they are generated, see the [token s
Set [`echoMessages`](/docs/api/realtime-sdk/types#client-options) to `false` on the agent's Ably client to prevent the agent from receiving its own reasoning and output messages, avoiding billing for [echoed messages](/docs/pub-sub/advanced#echo).
</Aside>

#### Subscribing
#### Subscribe

Subscribe to the main conversation channel to receive control messages and model output. Subscribe to the reasoning channel on demand, for example in response to a click event.

Expand Down
Loading