Skip to content

feat: Add Azure OpenAI Realtime API support for transcription (STT)#4476

Open
jack-jackhui wants to merge 7 commits intofastrepl:mainfrom
jack-jackhui:pr/azure-openai-support
Open

feat: Add Azure OpenAI Realtime API support for transcription (STT)#4476
jack-jackhui wants to merge 7 commits intofastrepl:mainfrom
jack-jackhui:pr/azure-openai-support

Conversation

@jack-jackhui
Copy link

@jack-jackhui jack-jackhui commented Mar 10, 2026

This PR adds support for Azure OpenAI Realtime transcription API (speech-to-text).

This complements PR #4222 which added Azure support for LLM (chat/summarization).

Changes

  • Detect Azure OpenAI endpoints by host (.openai.azure.com)
  • Use transcription_session.update message format for Azure (instead of session.update)
  • Do not include deployment in WebSocket URL (Azure rejects it with 400)
  • Add api-key header support for Azure authentication
  • Store Azure state in adapter for proper message formatting

Technical Details

Azure OpenAI Realtime API differs from standard OpenAI in several ways:

  1. Session message format: Uses transcription_session.update instead of session.update
  2. WebSocket URL: Must not include deployment name in path
  3. Authentication: Uses api-key header instead of Bearer token

Tested with Azure AI Foundry deployment.


Note

Medium Risk
Medium risk because it changes release automation (tags, artifacts, prereleases, npm publish) and modifies API request tracing/error reporting, which could impact production deploys and debugging if misconfigured.

Overview
Release automation is reworked for cli and desktop pipelines. cli_cd.yaml now supports nightly vs stable channels (channel-scoped concurrency), computes prerelease versions, renames artifacts to char, marks GitHub releases as prereleases for nightly, and adds an npm publish job that ships a wrapper package under apps/cli/npm.

Desktop release/publish workflows drop Linux artifacts and improve nightly release notes. desktop_cd.yaml/desktop_publish.yaml remove Linux build/download expectations and generate release bodies that link to either the website changelog (stable) or a GitHub compare view (nightly), while controlling makeLatest based on channel.

API and tooling updates. apps/api adds richer tracing fields derived from forwarded headers and marks spans as errors on 5xx/failures; OpenAPI generation is updated with new request bodies/schemas (calendar list requires connection_id, CreateSessionRequest gains mode/optional connection_id, and adds GET /nango/whoami). Changelog and blog automation are pointed from apps/web/content/changelog to packages/changelog/content, a Porter workflow is removed, and STT E2E now runs provider-matrix tests for both direct providers and transcribe-proxy suites.

Written by Cursor Bugbot for commit b14ca37. This will update automatically on new commits. Configure here.

@netlify
Copy link

netlify bot commented Mar 10, 2026

👷 Deploy request for hyprnote pending review.

Visit the deploys page to approve it

Name Link
🔨 Latest commit f648331

@netlify
Copy link

netlify bot commented Mar 10, 2026

Deploy Preview for hyprnote-storybook canceled.

Name Link
🔨 Latest commit f648331
🔍 Latest deploy log https://app.netlify.com/projects/hyprnote-storybook/deploys/69b7715829b68700090ae890


pub async fn write_text(output: Option<&Path>, text: String) -> CliResult<()> {
write_bytes_to(output, (text + "\n").into_bytes()).await
}
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Double newline when writing text to stdout

Low Severity

write_text appends "\n" to the text before passing it to write_bytes_to, which then appends another b"\n" when writing to stdout. This produces a double trailing newline for text output to stdout. The old code used println! which added only a single newline. The write_json path is unaffected because serde_json::to_vec doesn't include a trailing newline.

Additional Locations (1)
Fix in Cursor Fix in Web

Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

There are 2 total unresolved issues (including 1 from previous review).

Fix All in Cursor

Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

.write_all(b"\n")
.map_err(|e| CliError::operation_failed("write output", e.to_string()))?;
}
Ok(())
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

File output missing trailing newline in write helper

Low Severity

write_bytes_to ignores the add_newline parameter when writing to a file (the Some(path) branch on line 18-23 returns early without checking add_newline). The old code explicitly appended "\n" when writing text to a file via tokio::fs::write(path, transcript + "\n"). Now, write_text and write_json both pass add_newline: true, but that flag only takes effect for the stdout path. Output files produced via --output will be missing their trailing newline, breaking POSIX text file conventions.

Fix in Cursor Fix in Web

@cursor
Copy link

cursor bot commented Mar 16, 2026

You have used all of your free Bugbot PR reviews.

To receive reviews on all of your PRs, visit the Cursor dashboard to activate Pro and start your 14-day free trial.

Jack Hui and others added 7 commits March 16, 2026 13:56
- Detect Azure OpenAI endpoints by host (.openai.azure.com)
- Use transcription_session.update message format for Azure
- Do not include deployment in WebSocket URL (Azure rejects it)
- Add api-key header support for Azure authentication
- Store Azure state in adapter for proper message formatting
Bugbot correctly identified that write_text was adding a newline to the text,
and then write_bytes_to was adding another newline when writing to stdout.

Fixed by:
- Removing the newline concatenation in write_text
- Adding an add_newline parameter to write_bytes_to for explicit control
- Both write_text and write_json now add exactly one trailing newline
The file output path was ignoring the add_newline parameter.
Now both stdout and file output respect the add_newline flag,
ensuring POSIX text file conventions are followed.
@jack-jackhui jack-jackhui force-pushed the pr/azure-openai-support branch from 3c00a48 to f648331 Compare March 16, 2026 02:56
@cursor
Copy link

cursor bot commented Mar 16, 2026

You have used all of your free Bugbot PR reviews.

To receive reviews on all of your PRs, visit the Cursor dashboard to activate Pro and start your 14-day free trial.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants