Enable internal text embedding API by ajtiwari07 · Pull Request #3441 · Azure/data-api-builder

ajtiwari07 · 2026-04-12T16:18:30Z

Summary
This PR adds configurable text chunking capabilities to the embeddings API, enabling automatic text segmentation before embedding generation. This feature supports both single-text and multi-document batch processing with runtime configuration and query parameter overrides.

Changes
Configuration
Added EmbeddingsChunkingOptions.cs - Configuration model for chunking behavior
Enabled (bool) - Enable/disable chunking
SizeChars (int) - Chunk size in characters (default: 1000)
OverlapChars (int) - Overlap between chunks (default: 250)
EffectiveSizeChars property ensures minimum valid chunk size
Modified EmbeddingsOptions.cs - Added Chunking property and IsChunkingEnabled helper
Removed EmbeddingsCacheOptions.cs - Simplified configuration by removing unused cache feature

API Enhancements
Modified Controllers/EmbeddingController.cs
Auto-detects request type (single text vs. document array)
Implements overlapping text chunking algorithm
Supports query parameter overrides: $chunking.enabled, $chunking.size-chars, $chunking.overlap-chars
Returns multiple embeddings per document when chunking is enabled
Added Models/EmbedDocumentRequest.cs - Request model for document arrays
Added Models/EmbedDocumentResponse.cs - Response model with chunked embeddings
Schema (schemas)
Modified dab.draft.schema.json - Added chunking configuration schema with validation rules
Testing (UnitTests)
Added EmbeddingsChunkingOptionsTests.cs (13 tests) - Configuration validation
Added ChunkTextTests.cs (21 tests) - Chunking algorithm validation including edge cases
Modified EmbeddingControllerTests.cs (+18 tests) - API endpoint tests for chunking and document arrays
Total Test Coverage: 72 tests (48 existing + 24 new) - All passing## Why make this change?

Testing
All 72 unit tests passing
Edge cases covered: empty text, very small chunks, overlap larger than chunk size, Unicode text
Query parameter parsing validated
Backward compatibility verified

Breaking Changes
None - This is a backward-compatible addition. Existing single-text requests continue to work without modification.

Co-authored-by: JerryNixon <1749983+JerryNixon@users.noreply.github.com>

…configuration Co-authored-by: JerryNixon <1749983+JerryNixon@users.noreply.github.com>

Co-authored-by: JerryNixon <1749983+JerryNixon@users.noreply.github.com>

… and telemetry integration Co-authored-by: JerryNixon <1749983+JerryNixon@users.noreply.github.com>

Co-authored-by: JerryNixon <1749983+JerryNixon@users.noreply.github.com>

…ate empty embeddings Co-authored-by: JerryNixon <1749983+JerryNixon@users.noreply.github.com>

…oint/health sub-objects Co-authored-by: JerryNixon <1749983+JerryNixon@users.noreply.github.com>

…orization Co-authored-by: JerryNixon <1749983+JerryNixon@users.noreply.github.com>

Co-authored-by: JerryNixon <1749983+JerryNixon@users.noreply.github.com>

…lthCheckConfig in converter

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

…dding controller tests, switching the dab schema for the embedding system to default to false

…icitly set

embeddings endpoint is now permanently fixed to /embed with no user-configurable path option. This removes unnecessary configuration surface since the feature has not been released yet, eliminating the need for backward compatibility. Changes: - Remove path property from dab.draft.schema.json - Remove Path, UserProvidedPath, and EffectivePath from EmbeddingsEndpointOptions - Remove EffectiveEndpointPath from EmbeddingsOptions - Remove path deserialization from EmbeddingsOptionsConverterFactory - Remove --runtime.embeddings.endpoint.path CLI option - Remove path configuration logic from ConfigGenerator - Remove endpoint path validation from RuntimeConfigValidator - Update Startup.cs logging to use DEFAULT_PATH constant - Update all tests to remove path references

…e keys

…onal

This reverts commit a3f0b1d.

JimRoberts-MS · 2026-04-16T16:52:08Z

+        // For batch, check cache for each text individually
+        string[] cacheKeys = texts.Select(CreateCacheKey).ToArray();
+        float[]?[] results = new float[texts.Length][];
+        List<int> uncachedIndices = new();


Would it be better to use a Dictionary in case there are some duplicate keys to avoid duplicate calls to the api?

Can you elaborate? We do a GET on each chunk

JimRoberts-MS · 2026-04-20T21:10:16Z

+            {
+                Stopwatch stopwatch = new();
+                stopwatch.Start();
+                EmbeddingResult result = await _embeddingService.TryEmbedAsync(testText);


I wonder if it would be useful to separate out errors from the cache from errors calling the underlying embedding api.. eg you might have cache hits for the testText but be unable to try getting embeddings for new strings from the service, or you might be able to get new embeddings but not able to cache them causing perf issues

Can you revisit this again with the latest iteration?

souvikghosh04

Posting comments so far. My review is still in progress. also, waiting for existing review comments to be addressed.

souvikghosh04 · 2026-04-23T12:23:56Z

+
+                if (!result.Success)
+                {
+                    errorMessage = result.ErrorMessage ?? "Embedding request failed.";


may add appropriate, exponential retries upto 3?

Good point, the reason to avoid doing retry is to avoid incurring additional latencies especially when we expect embedding creation as a precursor to SQL query.

souvikghosh04 · 2026-04-23T12:25:27Z

+
+                int responseTimeMs = (int)stopwatch.ElapsedMilliseconds;
+                bool isResponseTimeWithinThreshold = responseTimeMs <= thresholdMs;
+                bool isDimensionsValid = true;


should be false by default?

Validation logic verifies dimension of the embeddings returned by the API, so it doesn't really matter.

souvikghosh04

posting additional comments. there are several inconsistencies between the schema JSON and the internal C# code files, including tests. e.g. threshold ms, API version, roles etc. are few to name which differes in schema JSON and the internal C# files. I will wait for these to get addressed, including pending comments.

souvikghosh04 · 2026-04-24T08:59:44Z

+        ""runtime"": {
+            ""embeddings"": {
+                ""provider"": ""azure-openai"",
+                ""base-url"": ""https://my-openai.openai.azure.com"",


urls and other configurations, including repeated code should be put in a common place. assertions in below tests also has similar hardcoded values. keep a single source of truth and reference.

…om/ajtiwari07/data-api-builder into add-internal-text-embedding-system

ajtiwari07 · 2026-04-24T17:29:20Z

posting additional comments. there are several inconsistencies between the schema JSON and the internal C# code files, including tests. e.g. threshold ms, API version, roles etc. are few to name which differes in schema JSON and the internal C# files. I will wait for these to get addressed, including pending comments.

I have revisited the config defaults and made them consistent across repo.

Copilot AI and others added 30 commits February 3, 2026 21:06

Initial plan

0dc589e

Add EmbeddingsOptions and EmbeddingProviderType configuration models

d3187bd

Co-authored-by: JerryNixon <1749983+JerryNixon@users.noreply.github.com>

Add CLI configure options for embeddings and register embedding service

6064826

Co-authored-by: JerryNixon <1749983+JerryNixon@users.noreply.github.com>

Add unit tests for embeddings and update JSON schema with embeddings …

0653f15

…configuration Co-authored-by: JerryNixon <1749983+JerryNixon@users.noreply.github.com>

Simplify HttpClient registration for embedding service

21e81b9

Co-authored-by: JerryNixon <1749983+JerryNixon@users.noreply.github.com>

Plan for embedding service enhancements

0cd8e53

Co-authored-by: JerryNixon <1749983+JerryNixon@users.noreply.github.com>

Refactor embedding code into dedicated namespaces

7fa1c49

Co-authored-by: JerryNixon <1749983+JerryNixon@users.noreply.github.com>

Fix property renames and update tests

c3f6937

Co-authored-by: JerryNixon <1749983+JerryNixon@users.noreply.github.com>

Add EmbeddingsOptionsConverter and fix all tests

1e18c25

Co-authored-by: JerryNixon <1749983+JerryNixon@users.noreply.github.com>

Address code review feedback

857203a

Co-authored-by: JerryNixon <1749983+JerryNixon@users.noreply.github.com>

Address PR feedback: Add Azure OpenAI validation, cache key security,…

89cb2d9

… and telemetry integration Co-authored-by: JerryNixon <1749983+JerryNixon@users.noreply.github.com>

Optimize ProviderName to avoid repeated string allocations

64b592e

Co-authored-by: JerryNixon <1749983+JerryNixon@users.noreply.github.com>

Fix schema mismatch, remove unused field, add enabled handling, valid…

e8d7238

…ate empty embeddings Co-authored-by: JerryNixon <1749983+JerryNixon@users.noreply.github.com>

Add embedding health check execution and update JSON schema with endp…

d9c8a29

…oint/health sub-objects Co-authored-by: JerryNixon <1749983+JerryNixon@users.noreply.github.com>

Add EmbeddingController for /embed REST endpoint with role-based auth…

3e02c0f

…orization Co-authored-by: JerryNixon <1749983+JerryNixon@users.noreply.github.com>

Address code review feedback for EmbeddingController

5c05464

Co-authored-by: JerryNixon <1749983+JerryNixon@users.noreply.github.com>

fix: manually deserialize EmbeddingsEndpointOptions and EmbeddingsHea…

c9eba20

…lthCheckConfig in converter

feat: add embeddings config validation and unit tests

d3a5209

Update src/Core/Services/Embeddings/EmbeddingService.cs

203e232

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update src/Config/Converters/EmbeddingsOptionsConverterFactory.cs

2cb3999

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update src/Service/Startup.cs

b432e4f

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Taking care of feedback

a8442f4

Merge branch 'main' into add-internal-text-embedding-system

a301290

Taking care of copilot feedback, adding embedding service tests, embe…

5f12542

…dding controller tests, switching the dab schema for the embedding system to default to false

Adding default return of application/json and only text/plain if expl…

bd84c84

…icitly set

feat(embeddings): require L2 cache and include provider/model in cach…

7100b8a

…e keys

fix(embeddings): keep L1 allow distributed cache when configured/opti…

73658de

…onal

Adding max text-count validation for embedding requests

ece97ae

Phase 1 Added Embedding Support with Chunking

d065f4b

prshri-msft reviewed Apr 15, 2026

View reviewed changes

Fix PR comments.

a3f0b1d

sayalikudale reviewed Apr 20, 2026

View reviewed changes

ajtiwari07 added 3 commits April 21, 2026 06:47

Revert "Fix PR comments."

1d6f9ea

This reverts commit a3f0b1d.

Pull latest Merge

8a00bad

Address review comments

f2261f6

sayalikudale reviewed Apr 22, 2026

View reviewed changes

Comment thread src/Service/Controllers/EmbeddingController.cs Outdated

Comment thread src/Service/Controllers/EmbeddingController.cs Outdated

Standardize API Error response

1ece78a

JimRoberts-MS reviewed Apr 22, 2026

View reviewed changes

ajtiwari07 added 2 commits April 22, 2026 10:08

Post review commit

896a111

Parameter Validation Check

d3e0413

Aniruddh25 added the assign-for-review label Apr 22, 2026

github-actions Bot assigned RubenCerna2079 and souvikghosh04 Apr 22, 2026

github-actions Bot removed the assign-for-review label Apr 22, 2026

Aniruddh25 assigned Aniruddh25 and unassigned RubenCerna2079 Apr 22, 2026

souvikghosh04 added this to Data API builder Apr 23, 2026

github-project-automation Bot moved this to Todo in Data API builder Apr 23, 2026

souvikghosh04 reviewed Apr 23, 2026

View reviewed changes

ajtiwari07 and others added 4 commits April 23, 2026 10:38

Uniform API response with tests

4c073aa

Fix tests

17414c6

Update the config defaults

d8f66b0

Merge branch 'main' into add-internal-text-embedding-system

17fba18

souvikghosh04 reviewed Apr 24, 2026

View reviewed changes

ajtiwari07 added 3 commits April 24, 2026 08:18

Post review test refactoring

d61b340

Merge branch 'add-internal-text-embedding-system' of https://github.c…

0d97b0b

…om/ajtiwari07/data-api-builder into add-internal-text-embedding-system

Ensure consistency across config defaults

da2bc83

Aniruddh25 mentioned this pull request Apr 24, 2026

Adding semantic caching with Azure Managed Redis #3024

Closed

Conversation

ajtiwari07 commented Apr 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

souvikghosh04 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

souvikghosh04 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

ajtiwari07 commented Apr 24, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

10 participants

ajtiwari07 commented Apr 12, 2026 •

edited

Loading