diff --git a/.github/workflows/quality-checks.yml b/.github/workflows/quality-checks.yml new file mode 100644 index 0000000..7753550 --- /dev/null +++ b/.github/workflows/quality-checks.yml @@ -0,0 +1,36 @@ +name: Quality Checks PR +concurrency: + group: Quality-Checks-PR-${{ github.workflow }}-${{ github.head_ref || github.event.workflow_run.head_branch || github.run_id }} + cancel-in-progress: true + +on: + merge_group: + pull_request: null + push: + branches: + - main + +permissions: + contents: read + +jobs: + quality-checks: + runs-on: ubuntu-latest + timeout-minutes: 15 + steps: + - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # ratchet:actions/checkout@v6 + with: + fetch-depth: 0 + persist-credentials: false + - uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # ratchet:actions/setup-python@v6 + with: + python-version: "3.11" + - uses: j178/prek-action@cbc2f23eb5539cf20d82d1aabd0d0ecbcc56f4e3 + with: + prek-version: '0.3.4' + extra-args: ${{ github.event_name == 'pull_request' && format('--from-ref {0} --to-ref {1}', github.event.pull_request.base.sha, github.event.pull_request.head.sha) || github.event_name == 'merge_group' && format('--from-ref {0} --to-ref {1}', github.event.merge_group.base_sha, github.event.merge_group.head_sha) || github.ref_name == 'main' && '--all-files' || '' }} + - name: Check Actions + uses: giner/check-actions@28d366c7cbbe235f9624a88aa31a628167eee28c # ratchet:giner/check-actions@v1.0.1 + with: + check_permissions: false + check_versions: false diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml new file mode 100644 index 0000000..994a233 --- /dev/null +++ b/.pre-commit-config.yaml @@ -0,0 +1,39 @@ +default_install_hook_types: + - pre-commit +repos: + - repo: https://github.com/pre-commit/pre-commit-hooks + rev: 3e8a8703264a2f4a69428a0aa4dcb512790b2c8c # frozen: v6.0.0 + hooks: + - id: check-added-large-files + name: Check for added large files + args: ["--maxkb=10000"] + + - repo: https://github.com/rhysd/actionlint + rev: a443f344ff32813837fa49f7aa6cbc478d770e62 # frozen: v1.7.9 + hooks: + - id: actionlint + + - repo: https://github.com/pre-commit/mirrors-prettier + rev: ffb6a759a979008c0e6dff86e39f4745a2d9eac4 # frozen: v3.1.0 + hooks: + - id: prettier + types_or: [json, css] + language_version: system + + - repo: https://github.com/sirwart/ripsecrets + rev: 7d94620933e79b8acaa0cd9e60e9864b07673d86 # frozen: v0.1.11 + hooks: + - id: ripsecrets + args: + - --strict-ignore + - --additional-pattern + - ^sk-[A-Za-z0-9_\-]{20,}$ + + - repo: local + hooks: + - id: format-docs + name: format_docs.py --check + entry: python3 scripts/format_docs.py --check --paths + language: system + types_or: [markdown, mdx] + require_serial: true diff --git a/admins/actions/mcp.mdx b/admins/actions/mcp.mdx index 1f5189f..1a4f4d1 100644 --- a/admins/actions/mcp.mdx +++ b/admins/actions/mcp.mdx @@ -27,9 +27,9 @@ You can configure Onyx to be an MCP client and allow your Agents to retrieve dat Add MCP Server dialog in Onyx Admin Panel - **Server Name** — A human-readable name for this MCP server. - Typically the name of the service you're connecting to (e.g. "Slack", "Jira"). + Typically the name of the service you're connecting to (e.g. "Slack", "Jira"). - **Description** — Any additional information that helps identify the server. - Useful for distinguishing multiple configured MCP server connections that talk to the same service. + Useful for distinguishing multiple configured MCP server connections that talk to the same service. - **MCP Server URL** — The URL of the MCP server, which must be reachable from your Onyx instance. Most URLs end in `/mcp`, or `/sse` for older servers. - **Managed MCP server:** The URL is typically found in the provider's documentation describing how to connect. - **Self-hosted MCP server:** You'll need to run your server in HTTP transport mode — Onyx doesn't currently support connecting via stdio. If the server is running on the same machine as Onyx, use `http://127.0.0.1:`. Otherwise, use the URL and port the server is exposed at. @@ -56,12 +56,12 @@ You can configure Onyx to be an MCP client and allow your Agents to retrieve dat There are two sub-modes: - **Shared Key** — The admin configures a single API key used for all user requests. - Similar to No Auth in that users won't need any additional configuration and will all get the same level of access - (whatever is provided by the API key you choose). + Similar to No Auth in that users won't need any additional configuration and will all get the same level of access + (whatever is provided by the API key you choose). - **Individual Key** — Users authenticate in the chat flow by providing their own API key(s) - to connect to the server. Useful when you want users to have differing access to an external service by identity. - This mode is also the most configurable — Onyx allows the admin to specify arbitrary header patterns to send to - the MCP server. + to connect to the server. Useful when you want users to have differing access to an external service by identity. + This mode is also the most configurable — Onyx allows the admin to specify arbitrary header patterns to send to the + MCP server. ### OAuth diff --git a/admins/advanced_configs/opensearch_document_index_migration.mdx b/admins/advanced_configs/opensearch_document_index_migration.mdx index 40b8adb..2b0bceb 100644 --- a/admins/advanced_configs/opensearch_document_index_migration.mdx +++ b/admins/advanced_configs/opensearch_document_index_migration.mdx @@ -6,82 +6,71 @@ icon: "arrow-right-arrow-left" ## Overview -Onyx has gradually been working towards switching our backing document index -database from Vespa to OpenSearch. The motivating factors behind this decision -are to reduce the baseline resources required to run Onyx, and to switch to a -widely used and supported documented index provider. +Onyx has gradually been working towards switching our backing document index database from Vespa to OpenSearch. +The motivating factors behind this decision are to reduce the baseline resources required to run Onyx, +and to switch to a widely used and supported documented index provider. - This is a living page and more details will be added over the coming weeks. + This is a living page and more details will be added over the coming weeks. ## Migration -As of `v3`, Onyx uses both Vespa and OpenSearch. Any documents which Onyx -indexes while using this version are indexed into both Vespa and OpenSearch, and -there is no migration required. +As of `v3`, Onyx uses both Vespa and OpenSearch. +Any documents which Onyx indexes while using this version are indexed into both Vespa and OpenSearch, +and there is no migration required. -Also as of `v3`, Onyx automatically runs a task to migrate your existing -documents from Vespa to OpenSearch. This task makes checkpointed progress so -Onyx does not necessarily need to be running continuously until the migration -completes in order for it to complete successfully. No explicit input is needed -from admins other than upgrading to `v3` in order for this task to run. +Also as of `v3`, Onyx automatically runs a task to migrate your existing documents from Vespa to OpenSearch. +This task makes checkpointed progress so Onyx does not necessarily need to be running continuously until the migration +completes in order for it to complete successfully. +No explicit input is needed from admins other than upgrading to `v3` in order for this task to run. -The migration's progress can be monitored in the Admin Panel, in the Document -Index Migration sidebar tab. +The migration's progress can be monitored in the Admin Panel, in the Document Index Migration sidebar tab. - The migration is best-effort. Although the vast majority of documents are - not expected to have migration issues, some documents (in particular - documents which were indexed a long time ago in versions of Onyx which - enforced weaker/different contracts for indexed content) may have - irreconcilable migration issues. In this event the only solution is a - re-index of connectors for those documents, where they will be processed in - such a way that they can then be indexed into OpenSearch. + The migration is best-effort. Although the vast majority of documents are not expected to have migration issues, + some documents (in particular documents which were indexed a long time ago in versions of Onyx which enforced + weaker/different contracts for indexed content) may have irreconcilable migration issues. + In this event the only solution is a re-index of connectors for those documents, + where they will be processed in such a way that they can then be indexed into OpenSearch. ## Deprecation of Vespa -`v4` of Onyx will represent the first major version to use OpenSearch entirely -without Vespa. +`v4` of Onyx will represent the first major version to use OpenSearch entirely without Vespa. - In order to preserve existing indexed content in Vespa without requiring - re-indexing, it will be necessary to upgrade to `v3` of Onyx to allow - migrations to run before upgrading to `v4`, when released. - - Although it may be technically feasible without breaking the product for a - determined admin to upgrade Onyx directly from some version < `v3` directly - to some version > `v3` skipping `v3` entirely, this will result in complete - loss of indexed data and will require re-indexing all connectors in order to - be able to search over documents. + In order to preserve existing indexed content in Vespa without requiring re-indexing, + it will be necessary to upgrade to `v3` of Onyx to allow migrations to run before upgrading to `v4`, when released. + + Although it may be technically feasible without breaking the product for a determined admin to upgrade Onyx directly + from some version < `v3` directly to some version > `v3` skipping `v3` entirely, + this will result in complete loss of indexed data and will require re-indexing all connectors in order to be able to + search over documents. ## Retrieval -During `v3`, document retrieval can be done via either Vespa (the default) or -OpenSearch, and which one is used can be toggled in the Document Index Migration -sidebar tab on the Admin Panel. Naturally, if the migration is not yet complete, -retrieval via OpenSearch may not yield every expected document. +During `v3`, document retrieval can be done via either Vespa (the default) or OpenSearch, +and which one is used can be toggled in the Document Index Migration sidebar tab on the Admin Panel. Naturally, +if the migration is not yet complete, retrieval via OpenSearch may not yield every expected document. ## cloud.onyx.app -The migration for cloud users began in March 2026, and by April 2026 virtually -all tenants' migrations indicated completion. +The migration for cloud users began in March 2026, +and by April 2026 virtually all tenants' migrations indicated completion. -As of `v3.2.0-cloud.7` in April, all tenants in the cloud have been swapped over -to OpenSearch for retrieval, and Vespa is no longer in the loop with Onyx, so -all new documents indexed are indexed only using OpenSearch. +As of `v3.2.0-cloud.7` in April, all tenants in the cloud have been swapped over to OpenSearch for retrieval, +and Vespa is no longer in the loop with Onyx, so all new documents indexed are indexed only using OpenSearch. - Due to a unique failure mode involving automatic updates in Vespa Cloud, - there may be some migrations which report successful completion despite - there being some existing documents which did not exist from the perspective - of Vespa at the time of migration for that particular document, and which - therefore might not have been migrated into OpenSearch. - - Similarly to the best-effort warning above, the recommended solution to this - is a re-index for connectors suspected of having missing documents. - Unfortunately due to the nature of this issue, there is little visibility - into which specific documents may have experienced this. + Due to a unique failure mode involving automatic updates in Vespa Cloud, + there may be some migrations which report successful completion despite there being some existing documents which did + not exist from the perspective of Vespa at the time of migration for that particular document, + and which therefore might not have been migrated into OpenSearch. + + Similarly to the best-effort warning above, + the recommended solution to this is a re-index for connectors suspected of having missing documents. + Unfortunately due to the nature of this issue, + there is little visibility into which specific documents may have experienced this. diff --git a/changelog.mdx b/changelog.mdx index 441005f..9970bb9 100644 --- a/changelog.mdx +++ b/changelog.mdx @@ -10,9 +10,9 @@ rss: true ## v3.2.0 - Multi-model chat lets you run a prompt across several models at once and pick - the best response, a generic OpenAI-compatible provider opens the door to any - LLM endpoint, the indexing pipeline gets a significant performance rework, + Multi-model chat lets you run a prompt across several models at once and pick the best response, + a generic OpenAI-compatible provider opens the door to any LLM endpoint, + the indexing pipeline gets a significant performance rework, and the Hook system matures with a second hook point and admin status views. Thank you to our contributors for helping make this release possible! @@ -20,27 +20,24 @@ rss: true ### Multi-Model Chat - You can now send a single prompt to multiple models in parallel and compare - their responses side by side. Pick the response you like best to continue - the conversation with that model, or deselect to keep exploring. Multi-model - is wired through the chat UI and the Chrome extension. + You can now send a single prompt to multiple models in parallel and compare their responses side by side. + Pick the response you like best to continue the conversation with that model, or deselect to keep exploring. + Multi-model is wired through the chat UI and the Chrome extension. Multi-Modal Chat ### Generic OpenAI-Compatible Provider - Onyx now ships a generic "OpenAI Compatible" provider that lets you point at - any OpenAI-compatible endpoint — local inference servers, gateway proxies, - or self-hosted model runners — without needing a dedicated integration. The - Custom LLM provider configuration was also reworked to support this cleanly. + Onyx now ships a generic "OpenAI Compatible" provider that lets you point at any OpenAI-compatible endpoint — local + inference servers, gateway proxies, or self-hosted model runners — without needing a dedicated integration. + The Custom LLM provider configuration was also reworked to support this cleanly. ### Hooks: status, logs, and document ingestion - The Hook system introduced in v3.1.0 has matured. A new admin page lets you - create, edit, and monitor hooks with live status and logs, and a second - integration point — document ingestion — joins query processing in the set - of hookable lifecycle events. Connection-lost states surface clearly in the - UI when a hook becomes unreachable. + The Hook system introduced in v3.1.0 has matured. A new admin page lets you create, edit, + and monitor hooks with live status and logs, + and a second integration point — document ingestion — joins query processing in the set of hookable lifecycle events. + Connection-lost states surface clearly in the UI when a hook becomes unreachable. Hook Extensions Page @@ -90,8 +87,7 @@ rss: true eliminating duplicate fetches and stabilizing revalidation behavior - Removed unnecessary user-memory eager loads from auth paths - Indexing pipeline streams chunks through a unified iterable-based - abstraction with batched processing, improving memory usage and throughput - on large corpora + abstraction with batched processing, improving memory usage and throughput on large corpora ### Deployment and Observability @@ -115,8 +111,7 @@ rss: true --- - Our [GitHub - Releases](https://github.com/onyx-dot-app/onyx/releases?utm_source=docs&utm_content=changelog) + Our [GitHub Releases](https://github.com/onyx-dot-app/onyx/releases?utm_source=docs&utm_content=changelog) page has the full list of changes and bug fixes for each release. @@ -125,20 +120,17 @@ rss: true ## v3.1.0 - Voice mode lands, the new Hook system opens up Onyx for extension, the Admin - Users and Groups pages get a full rewrite, and we've added two new LLM - gateway providers. Thank you to our contributors for helping make this - release possible! + Voice mode lands, the new Hook system opens up Onyx for extension, + the Admin Users and Groups pages get a full rewrite, and we've added two new LLM gateway providers. + Thank you to our contributors for helping make this release possible! ## [Update] Defaults that Changed - Two default behaviors changed in v3.1.0 that operators should be aware of - before upgrading: + Two default behaviors changed in v3.1.0 that operators should be aware of before upgrading: - **OpenSearch admin password default removed.** The standard - `docker-compose.yml` no longer ships a fallback for - `OPENSEARCH_ADMIN_PASSWORD`. If you previously relied on the default, set - the password explicitly in your environment before upgrading. + `docker-compose.yml` no longer ships a fallback for `OPENSEARCH_ADMIN_PASSWORD`. + If you previously relied on the default, set the password explicitly in your environment before upgrading. ## Key Improvements @@ -147,38 +139,35 @@ rss: true [Onyx now supports voice conversations end-to-end](https://docs.onyx.app/overview/core_features/voice_mode#voice-mode). [Speech-to-text and text-to-speech are configurable from the Admin - Panel](https://docs.onyx.app/admins/actions/voice_mode#voice-mode), and chats - can be revisited with their voice playback intact. + Panel](https://docs.onyx.app/admins/actions/voice_mode#voice-mode), + and chats can be revisited with their voice playback intact. Voice Admin Page ### Hooks - Onyx now ships a first-party Hook system for extending behavior at - well-defined points in the request lifecycle. [Hooks are managed from a new - admin - page](https://docs.onyx.app/admins/advanced_configs/hook_extensions#hooks), - configured through a connect/manage modal, and currently integrate with the - query-processing pipeline, with more hook points planned. + Onyx now ships a first-party Hook system for extending behavior at well-defined points in the request lifecycle. + [Hooks are managed from a new admin page](https://docs.onyx.app/admins/advanced_configs/hook_extensions#hooks), + configured through a connect/manage modal, and currently integrate with the query-processing pipeline, + with more hook points planned. ### Redesigned Admin Users and Groups - The admin Users page has been rebuilt from the ground up with a stats bar, - server-side pagination, role/group/status filters, inline role editing, an - invite flow, and edit-group-membership. The Groups page is also new, with - list and create/edit pages, inline renaming, shared resources, and - token-limit configuration. + The admin Users page has been rebuilt from the ground up with a stats bar, server-side pagination, + role/group/status filters, inline role editing, an invite flow, and edit-group-membership. + The Groups page is also new, with list and create/edit pages, inline renaming, shared resources, + and token-limit configuration. Users and Requests Admin Page ### LiteLLM and Bifrost gateway providers Two new LLM gateway integrations join the provider list: - [**LiteLLM**](https://docs.onyx.app/admins/ai_models/litellm_proxy), which - auto-fetches available models from your LiteLLM proxy, and - [**Bifrost**](https://docs.onyx.app/admins/ai_models/bifrost#bifrost), - configurable from a dedicated provider modal. Both make it easier to - centralize model access and routing through your existing infrastructure. + [**LiteLLM**](https://docs.onyx.app/admins/ai_models/litellm_proxy), + which auto-fetches available models from your LiteLLM proxy, + and [**Bifrost**](https://docs.onyx.app/admins/ai_models/bifrost#bifrost), + configurable from a dedicated provider modal. + Both make it easier to centralize model access and routing through your existing infrastructure. ## Contributor Highlights @@ -208,12 +197,13 @@ rss: true implementation - Custom tools have improved error handling and a refreshed timeline UI - Configurable file upload size limit, with a frontend precheck for oversized - uploads. This can be configured in the Admin Panel, under Advanced Options - for Chat Preferences. + uploads. This can be configured in the Admin Panel, under Advanced Options for Chat Preferences. + File Attachment Size Limit + - More file types can be attached to chats, and the file preview supports - significantly more formats. File types which are not explicitly - supported/recognized are parsed using a text-detection heuristic. + significantly more formats. + File types which are not explicitly supported/recognized are parsed using a text-detection heuristic. - Slackbot now resolves channel references and can filter search results by channel - Search state machine consolidated for more predictable behavior @@ -235,9 +225,8 @@ rss: true - Stop deep-copying during indexing - Plaintext file results are now cached in chat - OpenSearch tuning continues with configurable shards/replicas, configurable - Vespa migration page size, configurable hybrid subquery hits, vectors - excluded from hybrid/random search payloads, and Prometheus metrics for - retrieval + Vespa migration page size, configurable hybrid subquery hits, vectors excluded from hybrid/random search payloads, + and Prometheus metrics for retrieval ### Deployment @@ -256,8 +245,7 @@ rss: true --- - Our [GitHub - Releases](https://github.com/onyx-dot-app/onyx/releases?utm_source=docs&utm_content=changelog) + Our [GitHub Releases](https://github.com/onyx-dot-app/onyx/releases?utm_source=docs&utm_content=changelog) page has the full list of changes and bug fixes for each release. @@ -267,86 +255,77 @@ rss: true ## v3.0.0 The Vespa to OpenSearch migration is here, SCIM 2.0 lands for Enterprise, - search and chat are unified into a single experience, Code Interpreter ships - by default, and Onyx Lite and the Onyx CLI debut. v3.0.0 is the largest - release we've ever cut. Thank you to our contributors for helping make this - release possible! + search and chat are unified into a single experience, Code Interpreter ships by default, + and Onyx Lite and the Onyx CLI debut. v3.0.0 is the largest release we've ever cut. + Thank you to our contributors for helping make this release possible! ## [Update] OpenSearch Migration - v3.0.0 ships the full Vespa to OpenSearch migration toolchain. OpenSearch now - runs alongside Vespa in the standard `docker-compose.yml` and Helm chart, and - admins get a new Migration tab in the Admin Panel for kicking off and - monitoring the migration with live progress. - - Vespa continues to serve all queries throughout the migration by default. Once - the migration completes, admins can switch Onyx to read from OpenSearch. Setup - details and guidance live in the [OpenSearch deployment - guide](https://docs.onyx.app/deployment/local/opensearch), and the full - migration walkthrough is in the [Vespa to OpenSearch migration + v3.0.0 ships the full Vespa to OpenSearch migration toolchain. + OpenSearch now runs alongside Vespa in the standard `docker-compose.yml` and Helm chart, + and admins get a new Migration tab in the Admin Panel for kicking off and monitoring the migration with live progress. + + Vespa continues to serve all queries throughout the migration by default. Once the migration completes, + admins can switch Onyx to read from OpenSearch. + Setup details and guidance live in the [OpenSearch deployment + guide](https://docs.onyx.app/deployment/local/opensearch), + and the full migration walkthrough is in the [Vespa to OpenSearch migration guide](https://docs.onyx.app/admins/advanced_configs/opensearch_document_index_migration). ## Upgrade Path to v4.0.0 - v3.0.0 starts the final stretch of the Vespa to OpenSearch migration. **Every - deployment must run a v3.x.x release long enough for the migration to complete - before upgrading to v4.x.x, where Vespa will be fully removed.** If you skip - v3 entirely, you will lose all of your indexed data and have to re-index from - scratch — including re-uploading any Project files. + v3.0.0 starts the final stretch of the Vespa to OpenSearch migration. + **Every deployment must run a v3.x.x release long enough for the migration to complete before upgrading to v4.x.x, + where Vespa will be fully removed.** If you skip v3 entirely, + you will lose all of your indexed data and have to re-index from scratch — including re-uploading any Project files. ## Key Improvements ### SCIM 2.0 User and Group Provisioning - Onyx Enterprise now ships a complete SCIM 2.0 implementation. Identity - providers like Okta and Entra ID can create, update, and deactivate users and - groups against Onyx via the standard `/scim/v2/*` endpoints with bearer-token - auth. Admins can mint and rotate SCIM tokens from a new section in the Admin - Panel, and we've passed Okta's Runscope spec test for OIN submission. [More - details can be found in our - documentation.](https://docs.onyx.app/deployment/authentication/scim#scim) + Onyx Enterprise now ships a complete SCIM 2.0 implementation. Identity providers like Okta and Entra ID can create, + update, and deactivate users and groups against Onyx via the standard `/scim/v2/*` endpoints with bearer-token auth. + Admins can mint and rotate SCIM tokens from a new section in the Admin Panel, + and we've passed Okta's Runscope spec test for OIN submission. + [More details can be found in our documentation.](https://docs.onyx.app/deployment/authentication/scim#scim) ### Unified Search and Chat - Search and chat are now driven from the same input bar, with smooth animated - transitions between modes. Whether you're asking a question of an agent or - running a quick keyword search across your knowledge base, you start in the - same place. + Search and chat are now driven from the same input bar, with smooth animated transitions between modes. + Whether you're asking a question of an agent or running a quick keyword search across your knowledge base, + you start in the same place. ### Code Interpreter, by default - Code Interpreter is now part of the default deployment for both Docker and - Helm — no extra setup required. Sessions stream output as they run, accept - file inputs, embed generated images directly into chat, and ship with rich - preview modals for CSV, PDF, and DOCX outputs. The Python tool is also enabled - by default on the default agent. [More information can be found - here.](https://docs.onyx.app/admins/actions/code_interpreter) + Code Interpreter is now part of the default deployment for both Docker and Helm — no extra setup required. + Sessions stream output as they run, accept file inputs, embed generated images directly into chat, + and ship with rich preview modals for CSV, PDF, and DOCX outputs. + The Python tool is also enabled by default on the default agent. + [More information can be found here.](https://docs.onyx.app/admins/actions/code_interpreter) ### Onyx CLI - A new command-line tool for working with Onyx, distributed as a PyPI package - and a Docker image. Look for `onyx-cli` to grow over the next few releases. + A new command-line tool for working with Onyx, distributed as a PyPI package and a Docker image. + Look for `onyx-cli` to grow over the next few releases. [Get started here.](https://docs.onyx.app/overview/onyx_anywhere/cli#cli) ### Onyx Lite - A new "lite" deployment mode for installations that don't need a vector - database. Onyx Lite runs without the indexing stack and is suitable for - chat-only and tooling-focused use cases. + A new "lite" deployment mode for installations that don't need a vector database. + Onyx Lite runs without the indexing stack and is suitable for chat-only and tooling-focused use cases. ## Contributor Highlights - [@victoriamreese](https://github.com/victoriamreese) added a configurable - MCP host setting and continued to harden Helm and KEDA scaling for - self-hosted deployments! + MCP host setting and continued to harden Helm and KEDA scaling for self-hosted deployments! ## Additional Improvements ### Connectors - SharePoint connector received a major scalability rework, including delta - sync, more resilient credential refresh, better permission sync, and - substantially better throughput on large multi-site tenants + sync, more resilient credential refresh, better permission sync, + and substantially better throughput on large multi-site tenants - Slack connector can now optionally include bot messages during indexing - Slack bot responses convert markdown tables to a Slack-friendly format and sanitize HTML and broken citation links @@ -408,8 +387,7 @@ rss: true --- - Our [GitHub - Releases](https://github.com/onyx-dot-app/onyx/releases?utm_source=docs&utm_content=changelog) + Our [GitHub Releases](https://github.com/onyx-dot-app/onyx/releases?utm_source=docs&utm_content=changelog) page has the full list of changes and bug fixes for each release. diff --git a/deployment/authentication/oauth.mdx b/deployment/authentication/oauth.mdx index 86526bb..4b29fca 100644 --- a/deployment/authentication/oauth.mdx +++ b/deployment/authentication/oauth.mdx @@ -121,11 +121,11 @@ providing a seamless login experience through existing Google accounts. ## Customizing requested scopes -By default, Onyx requests `openid`, `email`, and `profile` from Google during -login — the minimum needed to identify the user. You can override this list -with `GOOGLE_OAUTH_SCOPE_OVERRIDE`, a comma-separated list of scopes to request -instead. This is primarily useful when the access token issued at login should -be passed through to tool calls that need additional Google API access. +By default, Onyx requests `openid`, `email`, +and `profile` from Google during login — the minimum needed to identify the user. +You can override this list with `GOOGLE_OAUTH_SCOPE_OVERRIDE`, a comma-separated list of scopes to request instead. +This is primarily useful when the access token issued at login should be passed through to tool calls that need +additional Google API access. ```bash .env GOOGLE_OAUTH_SCOPE_OVERRIDE=openid,email,profile,https://www.googleapis.com/auth/drive.readonly @@ -137,14 +137,12 @@ GOOGLE_OAUTH_SCOPE_OVERRIDE=openid,email,profile,https://www.googleapis.com/auth - Any scopes you add here must also be enabled on the OAuth client in Google - Cloud Console (consent screen + client configuration). Onyx only changes - what is sent in the authorize request; Google still rejects scopes that are - not configured for the client. + Any scopes you add here must also be enabled on the OAuth client in Google Cloud Console (consent screen + client + configuration). Onyx only changes what is sent in the authorize request; + Google still rejects scopes that are not configured for the client. These scopes apply only to the **app login** and pass-through OAuth flows. - The Google Drive and Gmail **connectors** use their own scopes and OAuth - flow, which are not affected by this setting. + The Google Drive and Gmail **connectors** use their own scopes and OAuth flow, which are not affected by this setting. diff --git a/deployment/authentication/oidc.mdx b/deployment/authentication/oidc.mdx index f966fb9..c213a38 100644 --- a/deployment/authentication/oidc.mdx +++ b/deployment/authentication/oidc.mdx @@ -82,18 +82,16 @@ Please contact us if you need help with a different identity provider. ## Customizing requested scopes -By default, Onyx uses the standard OIDC base scopes when redirecting users to -the identity provider. You can override this list with `OIDC_SCOPE_OVERRIDE`, -a comma-separated list of scopes to request instead. This is primarily useful -when the access token issued at login should be passed through to tool calls -that need additional scopes from the identity provider. +By default, Onyx uses the standard OIDC base scopes when redirecting users to the identity provider. +You can override this list with `OIDC_SCOPE_OVERRIDE`, a comma-separated list of scopes to request instead. +This is primarily useful when the access token issued at login should be passed through to tool calls that need +additional scopes from the identity provider. ```bash .env OIDC_SCOPE_OVERRIDE=openid,email,profile,groups ``` -Onyx will always also request `offline_access` so refresh tokens are issued, -even if it is not in the override list. +Onyx will always also request `offline_access` so refresh tokens are issued, even if it is not in the override list. The override **replaces** the default scopes — make sure `openid`, `email`, @@ -101,15 +99,14 @@ even if it is not in the override list. - Any scopes you add here must also be enabled on the application in your - identity provider. Onyx only changes what is sent in the authorize request; + Any scopes you add here must also be enabled on the application in your identity provider. + Onyx only changes what is sent in the authorize request; the IdP still rejects scopes that are not configured for the client. ## Enabling PKCE -PKCE is disabled by default to preserve backwards compatibility with existing -OIDC deployments. To enable it, set: +PKCE is disabled by default to preserve backwards compatibility with existing OIDC deployments. To enable it, set: ```bash .env OIDC_PKCE_ENABLED=true diff --git a/deployment/configuration/configuration.mdx b/deployment/configuration/configuration.mdx index 13f8c21..986d768 100644 --- a/deployment/configuration/configuration.mdx +++ b/deployment/configuration/configuration.mdx @@ -306,20 +306,18 @@ LANGFUSE_BASE_URL="https://cloud.langfuse.com" # Or "https://us.cloud.langfuse.c `OPENID_CONFIG_URL`: For OIDC. - `GOOGLE_OAUTH_SCOPE_OVERRIDE`: For Google OAuth login (`AUTH_TYPE=google_oauth` - or `cloud`). Comma-separated list of scopes to request from Google instead - of the defaults (`openid,email,profile`). Useful when the access token from - login needs to be passed through to tool calls that require additional - Google API scopes (i.e. when using Pass-Through OAuth for an MCP server). - Any scopes added here must also be enabled on the OAuth client in Google - Cloud Console. - - `OIDC_SCOPE_OVERRIDE`: For OIDC login (`AUTH_TYPE=oidc`). Comma-separated - list of scopes to request from the OIDC provider instead of the defaults. + `GOOGLE_OAUTH_SCOPE_OVERRIDE`: For Google OAuth login (`AUTH_TYPE=google_oauth` or `cloud`). + Comma-separated list of scopes to request from Google instead of the defaults (`openid,email,profile`). + Useful when the access token from login needs to be passed through to tool calls that require additional Google API + scopes (i.e. when using Pass-Through OAuth for an MCP server). + Any scopes added here must also be enabled on the OAuth client in Google Cloud Console. + + `OIDC_SCOPE_OVERRIDE`: For OIDC login (`AUTH_TYPE=oidc`). + Comma-separated list of scopes to request from the OIDC provider instead of the defaults. Same use case as `GOOGLE_OAUTH_SCOPE_OVERRIDE`. - `OIDC_PKCE_ENABLED`: For OIDC login. Set to `true` to enable PKCE in the - OIDC login flow. Disabled by default for backwards compatibility. + `OIDC_PKCE_ENABLED`: For OIDC login. Set to `true` to enable PKCE in the OIDC login flow. + Disabled by default for backwards compatibility. diff --git a/deployment/local/desktop_app.mdx b/deployment/local/desktop_app.mdx index 3debf9e..d428638 100644 --- a/deployment/local/desktop_app.mdx +++ b/deployment/local/desktop_app.mdx @@ -16,7 +16,8 @@ Features include: Onyx Desktop App - The desktop app connects to an existing Onyx deployment. Users will need access to this Onyx instance URL. + The desktop app is used for connecting to an Onyx server backend. Users will need access to the Onyx instance URL. + It cannot connect directly to a local inference backend. ## Download diff --git a/deployment/local/opensearch.mdx b/deployment/local/opensearch.mdx index bf01603..0b41ecb 100644 --- a/deployment/local/opensearch.mdx +++ b/deployment/local/opensearch.mdx @@ -8,44 +8,40 @@ icon: "magnifying-glass" OpenSearch is the document index Onyx uses for retrieval. -In Onyx’s **Helm** chart, we maintain a section which defines the desired setup -for an instance of OpenSearch, and another section which defines auth for that -instance. If these are enabled (the default as of `v3`), when you start your -Onyx instance using Helm, Helm will also start the OpenSearch instance -(containing one or more containers) and seed the appropriate auth values -throughout the application. By default the OpenSearch instance will have only -one container. - -Additionally, the Onyx application itself needs to know that OpenSearch is -enabled, and how to connect to the OpenSearch instance. In the configmap -template that the Onyx Helm chart tracks, if the OpenSearch section mentioned -above is enabled (the default as of `v3`), the environment variables for the -OpenSearch host, port, and whether it is enabled at all, will be set throughout -the application. - -In Onyx's **docker-compose** file, the `opensearch` service defines both the -desired setup and auth. Rather than a configmap as in Helm, the application just -reads environment variables that you can set in a `.env` file. +In Onyx’s **Helm** chart, we maintain a section which defines the desired setup for an instance of OpenSearch, +and another section which defines auth for that instance. If these are enabled (the default as of `v3`), +when you start your Onyx instance using Helm, +Helm will also start the OpenSearch instance (containing one or more containers) +and seed the appropriate auth values throughout the application. +By default the OpenSearch instance will have only one container. + +Additionally, the Onyx application itself needs to know that OpenSearch is enabled, +and how to connect to the OpenSearch instance. In the configmap template that the Onyx Helm chart tracks, +if the OpenSearch section mentioned above is enabled (the default as of `v3`), +the environment variables for the OpenSearch host, port, and whether it is enabled at all, +will be set throughout the application. + +In Onyx's **docker-compose** file, the `opensearch` service defines both the desired setup and auth. +Rather than a configmap as in Helm, the application just reads environment variables that you can set in a `.env` file. ## Auth -Unless explicitly disabled, OpenSearch requires setting initial credentials for -the cluster which are used to authenticate all requests. The password must meet -certain complexity requirements: minimum 8 chars, at least one uppercase, -lowercase, digit, and special character. +Unless explicitly disabled, +OpenSearch requires setting initial credentials for the cluster which are used to authenticate all requests. +The password must meet certain complexity requirements: minimum 8 chars, at least one uppercase, lowercase, digit, +and special character. -If using Onyx's **docker-compose**, there is no action necessarily required as -the compose file supplies a default initial password which matches the default -value Onyx falls back to in the absence of the relevant env var -`OPENSEARCH_ADMIN_PASSWORD`. +If using Onyx's **docker-compose**, +there is no action necessarily required as the compose file supplies a default initial password which matches the +default value Onyx falls back to in the absence of the relevant env var `OPENSEARCH_ADMIN_PASSWORD`. -If you wish to set this password to some other value, set an env var -`OPENSEARCH_ADMIN_PASSWORD` to your desired string, ensure `docker` references -this env var when running `docker-compose`, and also ensure this env var is in a -`.env` file Onyx references as well. +If you wish to set this password to some other value, set an env var `OPENSEARCH_ADMIN_PASSWORD` to your desired string, +ensure `docker` references this env var when running `docker-compose`, +and also ensure this env var is in a `.env` file Onyx references as well. -If using **Helm**, there is no default password and some password must be -specified in the `values.yaml` file specified in your Helm command as shown: +If using **Helm**, +there is no default password and some password must be specified in the `values.yaml` file specified in your Helm +command as shown: ```yaml values.yaml # Other overrides you may have. @@ -65,10 +61,9 @@ auth: ## If using your own provisioned managed OpenSearch -[If you provisioned a managed -instance](/deployment/local/terraform#opensearch-optional), you do not need Helm -to start an OpenSearch container/cluster for you. You can disable this in your -Helm values `yaml` file. +[If you provisioned a managed instance](/deployment/local/terraform#opensearch-optional), +you do not need Helm to start an OpenSearch container/cluster for you. +You can disable this in your Helm values `yaml` file. ```yaml values.yaml # Other overrides you may have. @@ -83,9 +78,8 @@ auth: enabled: false ``` -You also need to specify what the intended environment variables should be, -based on the OpenSearch cluster. These variables will be propagated throughout -the Onyx application. +You also need to specify what the intended environment variables should be, based on the OpenSearch cluster. +These variables will be propagated throughout the Onyx application. ```yaml values.yaml # Other overrides you may have. diff --git a/deployment/local/terraform.mdx b/deployment/local/terraform.mdx index c49fc09..95bc696 100644 --- a/deployment/local/terraform.mdx +++ b/deployment/local/terraform.mdx @@ -221,13 +221,12 @@ The composed `onyx` module exposes the inputs you'll most often want to tune. ### OpenSearch (optional) Managed OpenSearch is **off by default**. -Enable it if you want a managed document index backend instead of the in-cluster -Vespa node group. +Enable it if you want a managed document index backend instead of the in-cluster Vespa node group. -If provisioning a managed document index, you should disable starting an -OpenSearch container in your relevant docker-compose/Helm as Onyx will not use -this container. You should also set the relevant environment variables. See how -to [configure Onyx with OpenSearch](/deployment/local/opensearch). +If provisioning a managed document index, +you should disable starting an OpenSearch container in your relevant docker-compose/Helm as Onyx will not use this +container. You should also set the relevant environment variables. +See how to [configure Onyx with OpenSearch](/deployment/local/opensearch). Provision an Amazon OpenSearch domain inside the VPC. diff --git a/overview/core_features/voice_mode.mdx b/overview/core_features/voice_mode.mdx index 9f12825..325d94b 100644 --- a/overview/core_features/voice_mode.mdx +++ b/overview/core_features/voice_mode.mdx @@ -41,8 +41,8 @@ You can configure your voice preferences in the user settings panel: Voice Mode Settings - **Auto-send**: When enabled, your message is automatically sent when you pause speaking, - so you don't need to click send. - When disabled, pausing does not send your message. Instead, click the microphone button to finish recording and send. + so you don't need to click send. When disabled, pausing does not send your message. Instead, + click the microphone button to finish recording and send. - **Auto-playback**: When enabled, the microphone will automatically start listening again as soon as the response finishes speaking, enabling a continuous hands-free flow. When disabled, the response is generated as text only and must be played manually.