Skip to content

Add NLLB 600M web suggestion prototype with server-side autocomplete#1075

Draft
Copilot wants to merge 8 commits into
masterfrom
copilot/add-translation-suggestions-functionality
Draft

Add NLLB 600M web suggestion prototype with server-side autocomplete#1075
Copilot wants to merge 8 commits into
masterfrom
copilot/add-translation-suggestions-functionality

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented May 20, 2026

This PR introduces a prototype web app for interactive translation drafting with NLLB 600M. It adds a two-pane UI (source on left, translation on right), language-code selectors, and server-driven suggestions that can be accepted with Tab.

  • Prototype HTTP app + inference service

    • Added silnlp/nmt/nllb_web_demo.py to serve the UI and a suggestion API.
    • Loads facebook/nllb-200-distilled-600M once at startup and runs suggestion generation server-side.
    • Computes completion text as the remaining suffix beyond the user’s current partial translation.
  • UI behavior for translation drafting

    • Added source/target language dropdowns populated from the model tokenizer’s supported language codes.
    • Added source sentence input and target translation input in side-by-side panes.
    • Added debounced suggestion fetch on typing/language changes and Tab acceptance flow.
  • API contract

    • Added POST /api/suggest accepting source text, partial target text, and language codes.
    • Added validation for language codes against the active model’s supported set.
  • CLI + discoverability

    • Registered silnlp-nmt-web-demo script entry in pyproject.toml.
    • Added README usage section for running the prototype.
# silnlp/nmt/nllb_web_demo.py (core inference path)
generated = self.model.generate(
    **model_inputs,
    forced_bos_token_id=self.tokenizer.convert_tokens_to_ids(tgt_lang),
    max_new_tokens=64,
    num_beams=4,
)
full_translation = self.tokenizer.batch_decode(generated, skip_special_tokens=True)[0].strip()
return _remaining_completion(partial_translation, full_translation)
Original prompt

This project contains code to train translation models based on encoder-decoder neural models. It also allows for translating text from these models. I'd like to add new functionality that gives a user translation suggestions as they're producing a translation. Assume that you have access to an encoder-decoder translation model, a source text, and an incomplete translation that a user is in the process of typing. Run the source text and the partial translation through the model to predict what the rest of the translation might look like. When there is a high probability word or phrase that the model predicts could come next, return that suggestion to the user (but never more than one suggestion). One thing that will make this task very tricky is that the user might be in the middle of typing a word and the tokenization of that word could change as they type more. For example, if the user has only typed "cra" so far, it will be tokenized as "cra", but if they type further, it might end up being tokenized as "crass" or "crab" or "cram", so don't just predict based on the current tokenization of the most recent word -- also consider other tokenizations that are possible based on different ways that the word could be completed. Feel free to look up how text completion has been implemented on other Hugging Face models and use that as a basis for your work.

Created from VS Code.


This change is Reviewable

Copilot AI and others added 3 commits May 20, 2026 19:23
Agent-Logs-Url: https://github.com/sillsdev/silnlp/sessions/f94a6383-61bd-4dc9-a2a1-16c0f2c96de1

Co-authored-by: benjaminking <1214233+benjaminking@users.noreply.github.com>
Agent-Logs-Url: https://github.com/sillsdev/silnlp/sessions/f94a6383-61bd-4dc9-a2a1-16c0f2c96de1

Co-authored-by: benjaminking <1214233+benjaminking@users.noreply.github.com>
Agent-Logs-Url: https://github.com/sillsdev/silnlp/sessions/f94a6383-61bd-4dc9-a2a1-16c0f2c96de1

Co-authored-by: benjaminking <1214233+benjaminking@users.noreply.github.com>
Copilot AI changed the title [WIP] Add translation suggestions for incomplete translations Add interactive translation suggestions for Hugging Face NMT models May 20, 2026
Copilot AI requested a review from benjaminking May 20, 2026 19:28
Copilot AI changed the title Add interactive translation suggestions for Hugging Face NMT models Refactor NMT suggestions to reusable TranslationSuggester for low-latency interactive typing May 27, 2026
Copilot AI changed the title Refactor NMT suggestions to reusable TranslationSuggester for low-latency interactive typing refactor: make TranslationSuggester reusable across sentences May 28, 2026
Copilot AI changed the title refactor: make TranslationSuggester reusable across sentences Add NLLB 600M web suggestion prototype with server-side autocomplete May 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants