Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion examples/ai-transport-message-per-response/react/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ Use the following components to implement AI Transport message-per-response stre
- [`rewind`](/docs/channels/options/rewind) channel option: enables seamless message recovery during reconnections, delivering historical messages as `message.update` events.
- [`appendMessage()`](/docs/api/realtime-sdk/channels#append-message): appends tokens to an existing message using its serial.

Find out more about [AI Transport](/docs/ai-transport) and [message-per-response](/docs/ai-transport/features/token-streaming/message-per-response).
Find out more about [AI Transport](/docs/ai-transport) and [message-per-response](/docs/ai-transport/token-streaming/message-per-response).

## Getting started

Expand Down
2 changes: 1 addition & 1 deletion src/pages/docs/ai-transport/messaging/citations.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -140,7 +140,7 @@ When streaming response tokens using the [message-per-response](/docs/ai-transpo
</Aside>

<Aside data-type="note">
Identify the agent with a [`clientId`](/docs/messages#properties) in order to attribute a citation to a specific agent. This is useful in multi-agent architectures where multiple agents may contribute citations to the same response. For more information, see [Agent identity](/docs/ai-transport/features/sessions-identity/identifying-users-and-agents#agent-identity).
Identify the agent with a [`clientId`](/docs/messages#properties) in order to attribute a citation to a specific agent. This is useful in multi-agent architectures where multiple agents may contribute citations to the same response. For more information, see [Agent identity](/docs/ai-transport/sessions-identity/identifying-users-and-agents#agent-identity).
</Aside>

<Aside data-type="note">
Expand Down
2 changes: 1 addition & 1 deletion src/pages/docs/ai-transport/sessions-identity/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ AI Transport uses a channel-oriented model where sessions persist independently

In this model, sessions are associated with the channel, enabling seamless reconnection, background agent work, and multi-device access without additional complexity.

![AI Transport architecture](../../../../../images/content/diagrams/ai-transport-before-and-after.png)
![AI Transport architecture](../../../../images/content/diagrams/ai-transport-before-and-after.png)


The channel-oriented model provides key benefits for modern AI applications: sessions maintain continuity in the face of disconnections, users can refresh or navigate back to the ongoing session, multiple users or devices can participate in the same session, and agents can continue long-running or asynchronous workloads even when clients disconnect.
Expand Down
2 changes: 1 addition & 1 deletion src/pages/docs/ai-transport/token-streaming/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ Ably AI Transport solves this by decoupling token delivery from connection state
2. Server responds with a unique ID for the session, which is used to identify the channel
3. All further communication happens over the channel

![Ably AIT network diagram](../../../../../images/content/diagrams/ai-transport-before-and-after.png)
![Ably AIT network diagram](../../../../images/content/diagrams/ai-transport-before-and-after.png)


Dropping in AI Transport to handle the token stream completely changes the user's experience of device switching and failures. You do not need to add complex failure-handling code to your application or deploy additional infrastructure.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ The limits in the second category, however, cannot be increased arbitrarily and

## Message-per-response <a id="per-response"/>

The [message-per-response](/docs/ai-transport/features/token-streaming/message-per-response) pattern includes automatic rate limit protection. AI Transport prevents a single response stream from reaching the message rate limit for a connection by rolling up multiple appends into a single published message:
The [message-per-response](/docs/ai-transport/token-streaming/message-per-response) pattern includes automatic rate limit protection. AI Transport prevents a single response stream from reaching the message rate limit for a connection by rolling up multiple appends into a single published message:

1. Your agent streams tokens to the channel at the model's output rate
2. Ably publishes the first token immediately, then automatically rolls up subsequent tokens on receipt
Expand Down Expand Up @@ -56,7 +56,7 @@ If you configure the `appendRollupWindow` to allow a single response to use more

## Message-per-token <a id="per-token"/>

The [message-per-token](/docs/ai-transport/features/token-streaming/message-per-token) pattern requires you to manage rate limits directly. Each token publishes as a separate message, so high-speed model output can cause per-connection or per-channel rate limits to be hit, as well as consuming overall message allowances quickly.
The [message-per-token](/docs/ai-transport/token-streaming/message-per-token) pattern requires you to manage rate limits directly. Each token publishes as a separate message, so high-speed model output can cause per-connection or per-channel rate limits to be hit, as well as consuming overall message allowances quickly.

To stay within limits:

Expand All @@ -70,5 +70,5 @@ If your application requires higher message rates than your current package allo
## Next steps <a id="next-steps"/>

- Review [Ably platform limits](/docs/platform/pricing/limits) to understand rate limit thresholds for your package
- Learn about the [message-per-response](/docs/ai-transport/features/token-streaming/message-per-response) pattern for automatic rate limit protection
- Learn about the [message-per-token](/docs/ai-transport/features/token-streaming/message-per-token) pattern for fine-grained control
- Learn about the [message-per-response](/docs/ai-transport/token-streaming/message-per-response) pattern for automatic rate limit protection
- Learn about the [message-per-token](/docs/ai-transport/token-streaming/message-per-token) pattern for fine-grained control