Skip to content

Add fiftyone.pipeline.translation base engine#165

Merged
Automation51D merged 2 commits into
mainfrom
feature/country-translation
Jul 9, 2026
Merged

Add fiftyone.pipeline.translation base engine#165
Automation51D merged 2 commits into
mainfrom
feature/country-translation

Conversation

@Jamesr51d

Copy link
Copy Markdown
Contributor

Background

51Degrees on-premise IP Intelligence returns country information as weighted
lists of ISO country codes. Turning those into localized, ordered country
names (Runbook 2) needs a generic translation flow element, mirroring the .NET
FiftyOne.Pipeline.Translation package. pipeline-node had no translation
element for this.

Objectives

  • Add a reusable translation flow element that the ip-intelligence-node country
    engines build on.
  • Reproduce the .NET base behaviour: evidence precedence, locale resolution
    with the English short-circuit, weighted-list translation that preserves the
    weights, and the Original missing-translation behaviour.

Key Decisions

  • New standalone package fiftyone.pipeline.translation, mirroring the .NET
    separate base package, rather than inlining the logic into the IP engines.
  • The base extends the core FlowElement (not an engine), matching .NET where
    TranslationEngineBase extends FlowElementBase. It depends only on
    fiftyone.pipeline.core and js-yaml.
  • The translator duck-types AspectPropertyValue and the weighted value shape
    ({ value, weight, rawWeight }) rather than relying on instanceof, so it
    works regardless of which copy of core produced the value.
  • YAML sources are parsed inside the base with js-yaml, keeping the consumer
    builders thin and faithful to the .NET design.

Changes

  • New package fiftyone.pipeline.translation:
    • translationEngine.js: the flow element. Reads a source element, resolves
      the language (fixed or from query.translation, query.accept-language,
      header.accept-language), translates the configured properties.
    • translator.js: string, string-list, weighted-list and
      AspectPropertyValue translation with weights preserved.
    • languages.js: Accept-Language parsing and locale resolution with the
      English base-language short-circuit and the two-letter fallback.
    • translationData.js, missingTranslationBehavior.js, index.js,
      readme.md, tests/base.test.js.
  • Added the package to ci/build-package.ps1, ci/build-project.ps1 and
    ci/update-packages.ps1.

Testing

  • npm test (jest) in fiftyone.pipeline.translation: 8 tests pass, covering
    Accept-Language ordering, exact and two-letter locale resolution, the English
    short-circuit, preferred-before-lower-priority resolution, weighted-list
    weight preservation, the Original miss behaviour and no-value message
    preservation.

Notes

  • This is the base for the companion ip-intelligence-node PR. The country
    engines there depend on this package.
  • No new data files are shipped.

Adds a generic translation flow element that reads string based
properties from an upstream element and writes translated values under
its own element data key. Supports strings, string lists and weighted
string lists wrapped in an AspectPropertyValue, keeping the weights
unchanged. Resolves the target language from a fixed locale or from the
query.translation, query.accept-language and header.accept-language
evidence keys in that order, treating English as the base language so no
translation is performed when it is the preferred language. Parses YAML
translation sources with js-yaml. Adds the package to the build,
build-project and update-packages CI lists.
@Jamesr51d Jamesr51d removed the request for review from justadreamer July 7, 2026 15:37
@Automation51D Automation51D merged commit a6bf537 into main Jul 9, 2026
1 check passed
@Automation51D Automation51D deleted the feature/country-translation branch July 9, 2026 13:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants