Skip to content

Optimizations for nidx extracted text API#3597

Open
jotare wants to merge 7 commits intomainfrom
store-extracted-texts-in-nidx-3
Open

Optimizations for nidx extracted text API#3597
jotare wants to merge 7 commits intomainfrom
store-extracted-texts-in-nidx-3

Conversation

@jotare
Copy link
Copy Markdown
Contributor

@jotare jotare commented Apr 16, 2026

Description

Describe the proposed changes made in this PR.

How was this PR tested?

Describe how you tested this PR.

jotare added 2 commits April 16, 2026 12:09
- Directly get paragraph text instead of cloning the whole field text and then cutting
- Reuse the same chars iterator for multiple paragraphs on the same field
@jotare jotare requested a review from a team April 16, 2026 12:43
@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 16, 2026

Codecov Report

❌ Patch coverage is 59.80861% with 84 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.06%. Comparing base (dc87887) to head (3c9931b).
⚠️ Report is 2 commits behind head on main.

Files with missing lines Patch % Lines
nidx/nidx_text/src/reader.rs 64.76% 68 Missing ⚠️
nidx/src/searcher/shard_text.rs 0.00% 15 Missing ⚠️
nidx/nidx_text/src/lib.rs 0.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3597      +/-   ##
==========================================
- Coverage   85.10%   85.06%   -0.04%     
==========================================
  Files         552      552              
  Lines       47151    47321     +170     
  Branches    13467    13637     +170     
==========================================
+ Hits        40126    40255     +129     
- Misses       6434     6477      +43     
+ Partials      591      589       -2     
Flag Coverage Δ
nidx 79.73% <59.80%> (-0.05%) ⬇️
nucliadb 73.07% <ø> (ø)
nucliadb-ingest 43.59% <ø> (-0.02%) ⬇️
nucliadb-reader 43.70% <ø> (+0.01%) ⬆️
nucliadb-search 54.09% <ø> (ø)
nucliadb-standalone 46.06% <ø> (ø)
nucliadb-train 44.72% <ø> (ø)
nucliadb-writer 46.99% <ø> (-0.01%) ⬇️
nucliadb_dataset 73.76% <ø> (ø)
nucliadb_models 71.63% <ø> (ø)
nucliadb_sdk 83.49% <ø> (ø)
nucliadb_telemetry 71.45% <ø> (ø)
nucliadb_utils 80.61% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Comment thread nidx/nidx_text/src/reader.rs Outdated
Comment thread nidx/nidx_text/src/reader.rs Outdated
Comment thread nidx/nidx_text/src/reader.rs Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants