Skip to content

Add WisdomAI domain converter (both directions)#13

Open
mwu-wisdom wants to merge 3 commits into
apache:mainfrom
mwu-wisdom:wisdom-converter
Open

Add WisdomAI domain converter (both directions)#13
mwu-wisdom wants to merge 3 commits into
apache:mainfrom
mwu-wisdom:wisdom-converter

Conversation

@mwu-wisdom

@mwu-wisdom mwu-wisdom commented Jul 10, 2026

Copy link
Copy Markdown

What

Adds converters/wisdom, a bidirectional converter between WisdomAI domain exports (format 1.0, the JSON produced by wisdom's exportDomain API and consumed by importDomain) and Ossie semantic model YAML. It follows the layout and conventions of the dbt converter (apache-ossie dependency, ConverterResult/ConverterIssue loss reporting, argparse CLI). Also registers WISDOM as a well-known custom-extension vendor in the spec documents and OSIVendor enum.

ossie-wisdom wisdom-to-osi -i domain-export.json -o semantic_model.yaml
ossie-wisdom osi-to-wisdom -i semantic_model.yaml -o domain-export.json

Import mapping (wisdom → Ossie)

Ossie Wisdom
semantic_model[].name / description domain ref.name / description
ai_context (model level) domainSystemInstructions + domain knowledge contents as a bulleted list
datasets[] tables (location.database.schema.dbTablesource; primaryKey.columns or isPrimaryKey flags → primary_key)
datasets[].fields[] columns (bare, dialect-quoted names) and formulas (verbatim expressions); displayNamelabel; DATE/DATETIME/TIMESTAMP → dimension.is_time
relationships[] relationship graph edges — cardinality folded into edge direction (from = many side); compound AND-joins flattened to composite key arrays; OR/non-equi joins dropped with a warning
metrics[] per-table measures hoisted to the model level, cross-table name collisions resolved by table-name prefixing

Expressions are emitted verbatim under the Ossie dialect mapped from the table's connection (snowflake → SNOWFLAKE, databricks → DATABRICKS); other dialects fall back to ANSI_SQL verbatim with an UNSUPPORTED_DIALECT warning.

Export mapping (Ossie → wisdom)

The exact inverse, so round-trips are stable in both directions:

  • Model ai_context splits back into domainSystemInstructions (leading text) + one knowledge item per - bullet.
  • A field whose expression is just its own (possibly quoted) name becomes a column; anything else becomes a formula. dimension.is_time maps to a TIMESTAMP data type (wisdom re-derives exact types from the warehouse).
  • Relationships become MANY_TO_ONE edges; the ai_context notes written by the import direction restore ONE_TO_ONE/MANY_TO_MANY, and composite keys become compound AND join conditions.
  • Metrics attach to the table their expression references (fallback to the first dataset with a warning).
  • IDs are derived deterministically from element names so re-runs are reproducible; connections are per-dialect placeholders expected to be remapped at import.
  • Not representable in wisdom — dropped with warnings: semantic models beyond the first, unique_keys, custom_extensions, and ai_context on fields/metrics (plus synonyms/examples anywhere).

Testing

  • 27 tests against a synthetic fixture covering all four cardinalities, compound AND-join flattening, OR-join dropping, dialect fallback, hidden columns, stale measures, duplicate fields, metric name collisions, full round-trip equality (OSI → wisdom → OSI yields an identical OSIDocument), and determinism of the emitted export.
  • Verified end-to-end against two real wisdom domain exports (Snowflake and BigQuery connections): the Snowflake output passes validation/validate.py fully (schema, uniqueness, reference integrity, sqlglot SQL parsing), and OSI → wisdom → OSI round-trips are byte-identical YAML for both.

🤖 Generated with Claude Code

mwu-wisdom and others added 2 commits July 10, 2026 15:47
Adds converters/wisdom, a one-way converter from a WisdomAI domain
export JSON (format 1.0) to an Ossie semantic model YAML, following
the layout of the dbt converter.

Mapping: domain -> semantic model; domain knowledge and system
instructions -> model-level ai_context; tables/columns/formulas ->
datasets/fields; relationship graph -> relationships (cardinality
folded into edge direction, compound AND-joins flattened to composite
keys); per-table measures -> model-level metrics.

Expressions are emitted verbatim under the dialect mapped from the
table's connection (snowflake, databricks), falling back to ANSI_SQL
with a warning for unsupported dialects. Information loss is reported
through the same ConverterResult/ConverterIssue pattern as ossie-dbt.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds WISDOM to the OSIVendor enum, the vendor examples in the spec
documents and JSON schema, and the supported-vendors table in the
converter guide.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@mwu-wisdom

Copy link
Copy Markdown
Author

@jbonofre Would appreciate if you could help take a look for our team to land the converter changes.

Adds osi-to-wisdom, the reverse of the wisdom converter: an Ossie
document becomes a wisdom domain export (format 1.0) consumable by
wisdom's importDomain API.

The mapping inverts wisdom-to-osi so round-trips are stable in both
directions: model ai_context splits back into system instructions and
knowledge items, fields split into columns and formulas, relationship
direction is read as many-to-one with ai_context notes restoring
one-to-one/many-to-many and composite keys becoming compound AND join
conditions, and metrics attach to the table their expression
references. IDs are deterministic (derived from names) and connections
are per-dialect placeholders remapped at import time.

Verified: Ossie -> wisdom -> Ossie round-trips of two real domain
exports are byte-identical.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@mwu-wisdom mwu-wisdom changed the title Add WisdomAI domain export converter Add WisdomAI domain converter (both directions) Jul 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant