Add Ossie → wisdom domain export direction

converters/index.md

@@ -73,6 +73,7 @@ The Ossie specification currently defines extensions for the following vendors:
 | `SALESFORCE` | Salesforce / Tableau semantic layer |
 | `DBT` | dbt semantic models |
 | `DATABRICKS` | Databricks semantic layer |
+| `WISDOM` | WisdomAI domain |
 
 Each vendor may define custom extensions (via the `custom_extensions` field in the Ossie spec) to carry vendor-specific metadata that does not have an equivalent in the core specification.
 

converters/wisdom/README.md

@@ -0,0 +1,113 @@
+<!--
+  Licensed to the Apache Software Foundation (ASF) under one
+  or more contributor license agreements.  See the NOTICE file
+  distributed with this work for additional information
+  regarding copyright ownership.  The ASF licenses this file
+  to you under the Apache License, Version 2.0 (the
+  "License"); you may not use this file except in compliance
+  with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+  Unless required by applicable law or agreed to in writing,
+  software distributed under the License is distributed on an
+  "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+  KIND, either express or implied.  See the License for the
+  specific language governing permissions and limitations
+  under the License.
+-->
+
+# WisdomAI ↔ Apache Ossie Converter
+
+Converts between a [WisdomAI](https://wisdom.ai) domain export (format `1.0`, produced by
+wisdom's `exportDomain` API and consumed by `importDomain`) and an Ossie semantic model
+YAML document.
+
+## Usage
+
+```bash
+pip install -e ../../python -e .
+ossie-wisdom wisdom-to-osi -i domain-export.json -o semantic_model.yaml
+ossie-wisdom osi-to-wisdom -i semantic_model.yaml -o domain-export.json
+```
+
+Conversion warnings (information loss) are printed to stderr; the output YAML validates
+against the [Ossie JSON Schema](../../core-spec/osi-schema.json):
+
+```bash
+python ../../validation/validate.py semantic_model.yaml --schema ../../core-spec/osi-schema.json
+```
+
+## Field mapping
+
+| Ossie | Wisdom |
+|-------|--------|
+| `semantic_model[].name` | domain `ref.name` |
+| `semantic_model[].description` | domain `description` |
+| `semantic_model[].ai_context` | `domainSystemInstructions` + each domain `knowledge[].content` as a bulleted list |
+| `datasets[].name` | table `ref.name` |
+| `datasets[].source` | table `location.database.schema.dbTable` |
+| `datasets[].description` | table `description` |
+| `datasets[].primary_key` | table `primaryKey.columns`, else columns flagged `isPrimaryKey` |
+| `datasets[].fields[]` | table `columns[]` (expression = bare column name) and `formulas[]` (expression verbatim) |
+| `fields[].label` | column/formula `properties.displayName` |
+| `fields[].dimension.is_time` | set when `properties.dataType` is `DATE`, `DATETIME`, or `TIMESTAMP` |
+| `relationships[]` | domain `relationshipGraph.relationships[]` (see cardinality below) |
+| `metrics[]` | every table's `measures[]`, hoisted to the model level |
+
+Expressions are emitted verbatim under the Ossie dialect mapped from the table's connection
+(`snowflake → SNOWFLAKE`, `databricks → DATABRICKS`). Connections with any other dialect fall
+back to `ANSI_SQL` verbatim with an `UNSUPPORTED_DIALECT` warning.
+
+### Relationship cardinality
+
+Ossie encodes cardinality by direction (`from` = many side, `to` = one side), so wisdom's
+`relationshipType` is folded into the edge direction:
+
+| Wisdom `relationshipType` | Ossie |
+|---------------------------|-------|
+| `MANY_TO_ONE` | `from` = left, `to` = right |
+| `ONE_TO_MANY` | `from` = right, `to` = left |
+| `ONE_TO_ONE` | `from` = left, plus an `ai_context` note (direction is arbitrary) |
+| `MANY_TO_MANY` | `from` = left, plus an `ai_context` note and a `CARDINALITY_LOSS` warning |
+
+Compound join conditions that are an `AND` of equality conditions are flattened into
+positional `from_columns`/`to_columns` arrays; any other compound condition (e.g. `OR`)
+drops the relationship with a `RELATIONSHIP_DROPPED` warning.
+
+## Ossie → wisdom (export direction)
+
+`osi-to-wisdom` emits a domain export mirroring wisdom's `exportDomain` JSON, inverting
+the mapping above so that wisdom → Ossie → wisdom and Ossie → wisdom → Ossie round-trips
+are stable:
+
+- Model `ai_context` splits back into `domainSystemInstructions` (leading text) and one
+  knowledge item per `- ` bullet.
+- A field whose expression is just its own (possibly quoted) name becomes a column;
+  anything else becomes a formula. `dimension.is_time` becomes a `TIMESTAMP` data type
+  (wisdom re-derives exact types from the warehouse).
+- Relationships become `MANY_TO_ONE` edges (Ossie's `from` is the many side); the
+  `ai_context` notes written by `wisdom-to-osi` restore `ONE_TO_ONE`/`MANY_TO_MANY`, and
+  composite keys become compound `AND` join conditions.
+- Metrics attach to the first dataset their expression references (a
+  `METRIC_TABLE_UNRESOLVED` warning falls back to the first dataset).
+- Connections are per-dialect placeholders (`et-connection-snowflake`, ...) expected to be
+  remapped when the domain is imported; all IDs are derived deterministically from names,
+  so re-runs produce identical output.
+- Not representable in wisdom (dropped with warnings): extra semantic models beyond the
+  first, `unique_keys`, `custom_extensions`, and `ai_context` on fields/metrics (plus
+  synonyms/examples anywhere).
+
+## Known limitations
+
+- Hidden columns and stale measures are converted anyway (with a `STALE_MEASURE` warning
+  for the latter), so the output may reference columns wisdom itself hides from querying.
+- Out of scope for now: reviewed queries, recommended questions, synonym sets, row-level
+  security config, per-knowledge schema annotations, column enum values, and LLM prompts.
+
+## Development
+
+```bash
+pip install -e ../../python -e . pytest
+pytest tests/
+```

converters/wisdom/pyproject.toml

@@ -0,0 +1,51 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+[build-system]
+requires = ["hatchling"]
+build-backend = "hatchling.build"
+
+[project]
+name = "apache-ossie-wisdom"
+version = "0.2.0.dev0"
+description = "WisdomAI domain export -> Apache Ossie converter"
+requires-python = ">=3.11"
+classifiers = [
+    "License :: OSI Approved :: Apache Software License",
+    "Programming Language :: Python :: 3",
+]
+dependencies = [
+    "apache-ossie>=0.2.0.dev0",
+    "PyYAML>=6.0",
+]
+
+[project.license]
+text = "Apache-2.0"
+
+[project.scripts]
+ossie-wisdom = "ossie_wisdom.cli:main"
+
+[tool.hatch.build.targets.wheel]
+packages = ["src/ossie_wisdom"]
+
+[tool.uv]
+dev-dependencies = [
+    "pytest>=8.0",
+]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]

converters/wisdom/src/ossie_wisdom/__init__.py

@@ -0,0 +1,28 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from ossie_wisdom.converter_issues import ConverterIssue, ConverterIssueType, ConverterResult
+from ossie_wisdom.osi_to_wisdom import OSIToWisdomConverter
+from ossie_wisdom.wisdom_to_osi import WisdomToOSIConverter
+
+__all__ = [
+    "ConverterIssue",
+    "ConverterIssueType",
+    "ConverterResult",
+    "OSIToWisdomConverter",
+    "WisdomToOSIConverter",
+]

converters/wisdom/src/ossie_wisdom/cli.py

@@ -0,0 +1,116 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""CLI entry point for the ossie-wisdom converter.
+
+Usage:
+    ossie-wisdom wisdom-to-osi -i domain-export.json -o output.yaml
+    ossie-wisdom osi-to-wisdom -i input.yaml -o domain-export.json
+"""
+
+import argparse
+import json
+import sys
+from pathlib import Path
+
+import yaml
+
+from ossie import OSIDocument
+from ossie_wisdom.converter_issues import ConverterIssueType
+from ossie_wisdom.osi_to_wisdom import OSIToWisdomConverter
+from ossie_wisdom.wisdom_to_osi import WisdomToOSIConverter
+
+_ISSUE_REASON: dict[ConverterIssueType, str] = {
+    ConverterIssueType.UNSUPPORTED_DIALECT: "the connection dialect has no Ossie equivalent; expressions were emitted verbatim as ANSI_SQL",
+    ConverterIssueType.CARDINALITY_LOSS: "Ossie relationships encode cardinality by direction and cannot represent many-to-many",
+    ConverterIssueType.RELATIONSHIP_DROPPED: "the relationship uses a join that cannot be represented (non-table source, OR/non-equi condition, or unknown dataset)",
+    ConverterIssueType.METRIC_NAME_COLLISION: "another table defines a measure with the same name; this one was prefixed with its table name",
+    ConverterIssueType.STALE_MEASURE: "wisdom marked this measure stale; it was converted anyway",
+    ConverterIssueType.DUPLICATE_FIELD_DROPPED: "the dataset already has a field with this name",
+    ConverterIssueType.EXTRA_MODEL_DROPPED: "a wisdom domain export holds a single domain; only the first semantic model was converted",
+    ConverterIssueType.AI_CONTEXT_DROPPED: "wisdom has no equivalent for ai_context at this level (or for synonyms/examples)",
+    ConverterIssueType.METRIC_TABLE_UNRESOLVED: "the metric expression references no known dataset; it was attached to the first dataset",
+    ConverterIssueType.MISSING_DIALECT_EXPRESSION: "no expression was available in the dataset's dialect or ANSI_SQL; the first available dialect was used",
+    ConverterIssueType.UNIQUE_KEYS_DROPPED: "wisdom has no unique-key construct",
+    ConverterIssueType.CUSTOM_EXTENSION_DROPPED: "wisdom cannot store Ossie custom extensions",
+}
+
+_DROPPED_ISSUE_TYPES = {
+    ConverterIssueType.RELATIONSHIP_DROPPED,
+    ConverterIssueType.DUPLICATE_FIELD_DROPPED,
+    ConverterIssueType.EXTRA_MODEL_DROPPED,
+    ConverterIssueType.AI_CONTEXT_DROPPED,
+    ConverterIssueType.UNIQUE_KEYS_DROPPED,
+    ConverterIssueType.CUSTOM_EXTENSION_DROPPED,
+}
+
+
+def _print_issues(result) -> None:
+    for issue in result.issues:
+        verb = "was dropped" if issue.issue_type in _DROPPED_ISSUE_TYPES else "was converted with loss"
+        reason = _ISSUE_REASON[issue.issue_type]
+        print(f"[WARNING] {issue.issue_type.value}: {issue.element_name} {verb} during conversion because {reason}", file=sys.stderr)
+
+
+def _cmd_wisdom_to_osi(args: argparse.Namespace) -> None:
+    input_path = Path(args.input)
+    output_path = Path(args.output)
+
+    export = json.loads(input_path.read_text())
+    result = WisdomToOSIConverter().convert(export)
+
+    _print_issues(result)
+    output_path.write_text(result.output.to_osi_yaml())
+    print(f"Written to {output_path}", file=sys.stderr)
+
+
+def _cmd_osi_to_wisdom(args: argparse.Namespace) -> None:
+    input_path = Path(args.input)
+    output_path = Path(args.output)
+
+    document = OSIDocument.model_validate(yaml.safe_load(input_path.read_text()))
+    result = OSIToWisdomConverter().convert(document)
+
+    _print_issues(result)
+    output_path.write_text(json.dumps(result.output, indent=2) + "\n")
+    print(f"Written to {output_path}", file=sys.stderr)
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser(
+        prog="ossie-wisdom",
+        description="Convert a WisdomAI domain export JSON to Ossie YAML.",
+    )
+    subparsers = parser.add_subparsers(dest="command", required=True)
+
+    wisdom_to_osi = subparsers.add_parser("wisdom-to-osi", help="Convert domain-export.json → Ossie YAML")
+    wisdom_to_osi.add_argument("-i", "--input", required=True, metavar="FILE", help="Path to wisdom domain export JSON")
+    wisdom_to_osi.add_argument("-o", "--output", required=True, metavar="FILE", help="Path for output Ossie YAML")
+
+    osi_to_wisdom = subparsers.add_parser("osi-to-wisdom", help="Convert Ossie YAML → domain-export.json")
+    osi_to_wisdom.add_argument("-i", "--input", required=True, metavar="FILE", help="Path to Ossie YAML")
+    osi_to_wisdom.add_argument("-o", "--output", required=True, metavar="FILE", help="Path for output wisdom domain export JSON")
+
+    args = parser.parse_args()
+    if args.command == "wisdom-to-osi":
+        _cmd_wisdom_to_osi(args)
+    elif args.command == "osi-to-wisdom":
+        _cmd_osi_to_wisdom(args)
+
+
+if __name__ == "__main__":
+    main()

converters/wisdom/src/ossie_wisdom/converter_issues.py

@@ -0,0 +1,56 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from dataclasses import dataclass
+from enum import Enum
+from typing import Generic, List, TypeVar
+
+
+class ConverterIssueType(Enum):
+    """Identifies the kind of information loss that occurred during conversion."""
+
+    UNSUPPORTED_DIALECT = "UNSUPPORTED_DIALECT"
+    CARDINALITY_LOSS = "CARDINALITY_LOSS"
+    RELATIONSHIP_DROPPED = "RELATIONSHIP_DROPPED"
+    METRIC_NAME_COLLISION = "METRIC_NAME_COLLISION"
+    STALE_MEASURE = "STALE_MEASURE"
+    DUPLICATE_FIELD_DROPPED = "DUPLICATE_FIELD_DROPPED"
+    EXTRA_MODEL_DROPPED = "EXTRA_MODEL_DROPPED"
+    AI_CONTEXT_DROPPED = "AI_CONTEXT_DROPPED"
+    METRIC_TABLE_UNRESOLVED = "METRIC_TABLE_UNRESOLVED"
+    MISSING_DIALECT_EXPRESSION = "MISSING_DIALECT_EXPRESSION"
+    UNIQUE_KEYS_DROPPED = "UNIQUE_KEYS_DROPPED"
+    CUSTOM_EXTENSION_DROPPED = "CUSTOM_EXTENSION_DROPPED"
+
+
+@dataclass(frozen=True)
+class ConverterIssue:
+    """Records a single instance of information loss during conversion."""
+
+    issue_type: ConverterIssueType
+    element_name: str
+
+
+T = TypeVar("T")
+
+
+@dataclass(frozen=True)
+class ConverterResult(Generic[T]):
+    """Return value of a converter's convert() method, pairing the output with any conversion issues."""
+
+    output: T
+    issues: List[ConverterIssue]