Localization: stage translations into the String Catalog (manual)#25713
Draft
jkmassel wants to merge 7 commits into
Draft
Localization: stage translations into the String Catalog (manual)#25713jkmassel wants to merge 7 commits into
jkmassel wants to merge 7 commits into
Conversation
Collaborator
Generated by 🚫 Danger |
…lane) Adds a regular-string reverse fold — the catalog analogue of the plural fold — that populates Localizable.xcstrings (the future regular-string backing store) with GlotPress human translations plus AI machine translations, as human ?? existing-machine ?? AI ?? English (human => translated; machine / English => needs_review). - CatalogStrings.fold_translations! (pure, 12 tests): reuse-aware — a valid existing needs_review machine cell is kept, so the catalog's needs_review state is the persistence (no side-store) and re-runs only translate genuinely-new gaps. Humans supersede machine cells on the next fold. - download_localized_catalog lane: generate the English catalog, fold the downloaded .strings humans + AI-fill (translate_all), commit. Gated on ANTHROPIC_API_KEY. STAGED, NOT SHIPPED: the catalog isn't the runtime store yet, so this changes nothing users see. MANUAL ONLY: deliberately not wired into download_localized_strings or CI — it runs xcstringstool extraction, calls the API, and commits a large catalog, so it's run on demand.
… -> AI-fill Make the lane mirror the pipeline order: (1) generate_strings_catalog scans the code, (2) it now downloads the current GlotPress translations itself (ios_download_strings_files_from_glotpress) and folds them in, (3) CatalogStrings AI-fills the rest. Adds a locales:fr,de scope (and skip_download:true) so a run can be kept cheap. Uploading the AI drafts back to GlotPress (the eventual step 4) stays out — it builds on the existing GlotPress import integration (gp_update_metadata_source), not done here.
Run a download then a localize as separate fastlane invocations: - download_catalog_strings — pull GlotPress translations into the *.lproj dirs (scoped by locales:) - localize_catalog — scan + fold the downloaded strings + AI-fill into Localizable.xcstrings Replaces the single download_localized_catalog lane (and its skip_download flag): download once, re-localize as many times as you like without re-downloading.
localize_catalog no longer re-runs generate_strings_catalog — scanning the code is its own lane (generate_strings_catalog), so you can refresh the English catalog without ever invoking the AI. The three stages are now independent fastlane invocations: generate_strings_catalog (scan) -> download_catalog_strings (download) -> localize_catalog (fold + AI-fill) localize_catalog errors if the catalog hasn't been generated yet.
The extract/sync sh calls echo enormous commands (up to 400 file paths per extract batch; a --stringsdata <path> pair per file on sync), burying the output. Pass log: false and emit concise progress lines instead. No fastlane/release-toolkit action wraps xcstringstool, so sh is the invocation; log: false is its clean-output option.
fastlane's sh treats each call as an action: even with log: false it prints a '--- Step: shell command ---' banner per call and lists it in the run summary. extract is chunked into ~7 calls and sync runs twice, so that wrapped the real progress in a wall of banners. Run xcstringstool through Open3.capture2e instead (argv — safe for paths with spaces; captured output surfaced only on failure), via a run_xcstringstool helper. The scan now prints just its own progress lines.
jkmassel
force-pushed
the
jkmassel/catalog-strings-translation
branch
from
d5697f8 to
a45aaf1
Compare
CatalogStrings.fold_translations! and its AI-translator wiring had the same two latent bugs #25710's adversarial review caught and fixed on the plural side: - A whitespace-only GlotPress value was accepted as a valid human translation, when it isn't real content and should fall through to the AI/English path instead. - catalog_ai_translator only rescued LoadError around AITranslator setup, so a StandardError there (e.g. a malformed ANTHROPIC_BASE_URL) would crash the whole localize_catalog lane instead of degrading to English-only. Ports over the same fixes: a strip-aware blank check, and a broadened LoadError, StandardError rescue.
jkmassel
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fills the app's String Catalog (
Localizable.xcstrings) with translations — the human translations we already have from GlotPress, plus machine translations for whatever is still untranslated. This gets the catalog ready ahead of moving the app over to String Catalogs. Nothing users see changes.Two things to know
Staged, not shipped. The catalog file is generated, but the app doesn't read it at runtime yet — regular strings still come from the existing
Localizable.strings. This only pre-fills the catalog for the eventual switch-over. Machine translations are markedneeds_review, so they stay clearly separate from reviewed human translations and can sit in the catalog without shipping to anyone.Manual, not automated. These lanes are run by hand, not wired into any release or CI step: a run extracts strings, calls a paid translation API, and commits a large file, so it shouldn't happen on its own. The unit tests still run in CI.
How it works
Three steps, each runnable on its own:
generate_strings_catalog— extracts the English strings from the code into the catalog.download_catalog_strings— downloads the current translations from GlotPress.localize_catalog— adds the human translations, machine-translates whatever's left, and commits.Each string gets the best translation available in each language: a human translation if there is one, otherwise a machine translation, otherwise the English text. Human translations are marked
translated; machine translations and English fallbacks are markedneeds_review.Re-running is cheap. A machine translation already in the catalog is left alone instead of being requested again, and a human translation always replaces a machine one on the next run. Machine translation only happens when an API key is set — without one, the catalog fills from human translations and English. A machine translation is kept only if its format placeholders (
%1$@,%2$d, …) match the English, so a mangled one falls back to English rather than shipping a broken string.Test plan
fastlane/lanes/*_test.rb) — these run in CIrubocopcleanPlurals are handled the same way in a separate PR (#25710), since they live in their own catalog file.