Skip to content

Reduce tensor-scoring inference memory in Hugging Face test translation path#1073

Draft
Copilot wants to merge 16 commits into
masterfrom
copilot/fix-out-of-memory-error
Draft

Reduce tensor-scoring inference memory in Hugging Face test translation path#1073
Copilot wants to merge 16 commits into
masterfrom
copilot/fix-out-of-memory-error

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented May 20, 2026 •
edited by ddaspit
Loading

  • Inspect current translation batching implementation and related tests
  • Attempt baseline tests before edits (pytest unavailable in environment)
  • Remove tensor-specific batch size cap from _translate_sentences
  • Rename _translate_sentence_helper to _translate_batch and update call sites/tests
  • Rename downstream sentences parameters to batch where called by _translate_sentences
  • Stop passing batch_size into translation pipeline calls
  • Run syntax checks
  • Run parallel validation

This change is Reviewable

Copilot AI linked an issue May 20, 2026 that may be closed by this pull request
Out of memory error during the experiment.py test step #1072
Open
Copilot AI and others added 4 commits May 20, 2026 17:26
@benjaminking
fix: cap tensor inference batch size to reduce HF OOM risk
1623e4b 
Agent-Logs-Url: https://github.com/sillsdev/silnlp/sessions/1d43fe05-c924-4848-872d-3ad357553aef

Co-authored-by: benjaminking <1214233+benjaminking@users.noreply.github.com>
@benjaminking
test: harden tensor batch-size cap smoke test setup
d122e90 
Agent-Logs-Url: https://github.com/sillsdev/silnlp/sessions/1d43fe05-c924-4848-872d-3ad357553aef

Co-authored-by: benjaminking <1214233+benjaminking@users.noreply.github.com>
@benjaminking
refactor: simplify tensor batch size cap calculation
e5b944d 
Agent-Logs-Url: https://github.com/sillsdev/silnlp/sessions/1d43fe05-c924-4848-872d-3ad357553aef

Co-authored-by: benjaminking <1214233+benjaminking@users.noreply.github.com>
@benjaminking
refactor: remove unnecessary cast in tensor batch cap logic
27fa7d4 
Agent-Logs-Url: https://github.com/sillsdev/silnlp/sessions/1d43fe05-c924-4848-872d-3ad357553aef

Co-authored-by: benjaminking <1214233+benjaminking@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix out of memory error during experiment.py test step Reduce tensor-scoring inference memory in Hugging Face test translation path May 20, 2026
Copilot AI requested a review from benjaminking May 20, 2026 17:30
Copilot AI and others added 4 commits May 20, 2026 17:55
@benjaminking
feat: add adaptive OOM batch retry for translation inference
a92abd1 
Agent-Logs-Url: https://github.com/sillsdev/silnlp/sessions/318d55d1-ef9f-41ea-962b-48430ef1e230

Co-authored-by: benjaminking <1214233+benjaminking@users.noreply.github.com>
@benjaminking
refactor: tighten adaptive OOM handler and test constant
a22b5e8 
Agent-Logs-Url: https://github.com/sillsdev/silnlp/sessions/318d55d1-ef9f-41ea-962b-48430ef1e230

Co-authored-by: benjaminking <1214233+benjaminking@users.noreply.github.com>
@benjaminking
docs: clarify adaptive translation batch retry helper contract
60a3d3e 
Agent-Logs-Url: https://github.com/sillsdev/silnlp/sessions/318d55d1-ef9f-41ea-962b-48430ef1e230

Co-authored-by: benjaminking <1214233+benjaminking@users.noreply.github.com>
@benjaminking
refactor: handle adaptive retry OOM through RuntimeError path
0658859 
Agent-Logs-Url: https://github.com/sillsdev/silnlp/sessions/318d55d1-ef9f-41ea-962b-48430ef1e230

Co-authored-by: benjaminking <1214233+benjaminking@users.noreply.github.com>
Copilot AI and others added 2 commits May 20, 2026 18:15
Copilot AI and others added 2 commits May 20, 2026 18:28
@benjaminking
refactor: extract shared translate batching helpers
2864bd6 
Agent-Logs-Url: https://github.com/sillsdev/silnlp/sessions/e0d13ac1-561a-44de-b3a9-3a9f1f3dab80

Co-authored-by: benjaminking <1214233+benjaminking@users.noreply.github.com>
@benjaminking
perf: remove redundant sentence list conversion in helper
7ca4455 
Agent-Logs-Url: https://github.com/sillsdev/silnlp/sessions/e0d13ac1-561a-44de-b3a9-3a9f1f3dab80

Co-authored-by: benjaminking <1214233+benjaminking@users.noreply.github.com>
Copilot AI and others added 2 commits May 20, 2026 19:38
@benjaminking
refactor: rename translate batch helper and drop tensor cap/pipeline …
e7cc3ee 
…batch_size

Agent-Logs-Url: https://github.com/sillsdev/silnlp/sessions/06f3a41e-075a-43a6-87b1-2887369bed21

Co-authored-by: benjaminking <1214233+benjaminking@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@benjaminking benjaminking Awaiting requested review from benjaminking

At least 1 approving review is required to merge this pull request.

Assignees

@benjaminking benjaminking

Copilot code review Copilot

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Out of memory error during the experiment.py test step

2 participants

@benjaminking