Make stable identifiers explicit rather than reusing name

[cpimhoff]

Encoding “name” as the top-level, unique, stable identifier for entities in the OSI has some risks.

1. The stable identifier of an OSI resource likely changes when imported/exported from the OSI spec into a consumer format with naming constraints, which feels risky.

The code-addressable identifier for a resource can have unexpected constraints in different environments that OSI resource names may run into often. These include not allowing special characters, not leading with numbers, or even small details like capitalization varies greatly for identifiers across SQL databases.

Any importer into a system which has any of these constraints would have to rewrite not just the name, but any references to the name, including inside all expressions in all dialects, which can be complex.

I would suggest the OSI develop some constraints on stable names so that integrations into external systems can be implemented more smoothly, without translation in most cases. I would suggest a minimum and maximum character length, with no mixed casing, and no spaces or special characters.

2. The user/LLM-facing semantic name cannot be changed without a breaking change to the model.

OSI models will be imported and stored in consumer systems, and these systems will have features which bind application resources to OSI entities. For example, a BI app could import an OSI dataset sales and then build a dashboard that links to sales. The only stable identifier that the OSI entity provides is the name, so if the name changes, the dashboard would break when the OSI models are next ingested.

Changing the user/LLM-facing semantic name of a resource should not break downstream consumption of resources.

I would suggest separating this concept into id: string and name: string. id would be the strict identifier. name would be a loose string suitable for display. This split would more clearly reflect the constraints around identifiers when exporting/importing.

[sfc-gh-wpugh]

I like having a separate ID/NAME from display-name.

However does the character restriction potentially cause problems with existing table names or column names that may already be quoted in DBs? Would this be an issue for languages that use non-ASCII characters?

I can see how it could be nice to have simple names, although, I don't think we can get away from defining some quoting behaviour for referencing the underlying tables/columns.

We also probably want to be careful about how we would handle non-ASCII characters in that case.

[QMalcolm]

In my mind being loose on constraints, allowing for non-ASCII characters and etc, actually makes portability harder 🤔 If the constraints aren't tight then every consumption platform needs to handle every possible edge case. If we have a narrow standard on what a name is, then systems consuming OSI documents only need to know how to go from that narrow definition to what's valid in their system. It guarantees some amount of work is necessary, but not something very difficult. Whereas if the name could be anything, you do get the ability to guarantee you don't need to change anythign going from your platform to OSI and back, but suddenly you need to build an incredibly robust system to handle every possibility from going from system x/y/z -> OSI -> your system

[khush-bhatia]

+1, I do like the idea of having a separate name and ID. But even with ID, it poses new challenges since different systems will require different guarantees of uniqueness within a scope. Unless, we align that ID will have to be a UUID.

[magerdaniel]

I would love to have this column. To have the same metrik name in multiple Power User folders is very common. In MSTR We're using GUID with 32 charackters. I'm relaxed with the uniqueness. The source system is resbonsible for this...

[QMalcolm]

@cpimhoff Thank you for creating this topic ❤️

I absolutely agree that we need to differentiate between identifying name and display name. Additionally, a change in the identifying name constitutes that the semantic model / metric is no longer what it was. We could perhaps imagine a system akin to git mv which allows for tracking a semantic model / metric across identifying name changes, though I think that is very much out of scope for now.

I agree with the concept that we should put constraints on how the identifying name can be specified. The constraints that come to my mind are essentially what is allowed for URL slugs, but slightly stricter: lowercase, no spaces (hyphens or underscores instead), no special characters, no leading numbers, and a minimum + maximum length.

I'd like, if deemed reasonable, to steer us away from using a UUID or GUID. In my mind OSI is not just a compiled intermediary, but also a direct authoring layer. I should be able to write OSI directly instead of having to transpiling from dbt SL definitions to OSI if I want. In such a case, how do I author a UUID or GUID? It isn't exactly human friendly, and I'm not sure how much it gives us over an identifying name that has proper constraints. Additionally, if the authorship is happening somewhere else and being transpiled to OSI, that system can always use a UUID or GUID for the identifying name, but I don't think we should require that.

With all that in mind I see there being two possible paths.

1. name and display-name
2. id and name

I personally gravitate towards option (1). When I'm naming something, I think of it as "naming", not "ID-ing" (though the latter is also valid). However, that's more personal preference than anything concrete. Where I do think option (1) is better is display-name, it's more descriptive than name. That is option (1) has "what is this thing called" and "what is the human display of what this thing is called", while option (2) reads more as "what this thing is called" and another "what this thing is called".

What do y'all think? cc: @magerdaniel @khush-bhatia @sfc-gh-wpugh @cpimhoff

[cpimhoff]

I am in agreement with your assessment, constraints, and choice to avoid uuids. name/display-name seems fine to me, though I generally think the term id implies the semantics of "this shouldn't change or things might break" better than name does.

If it provides any utility, attached is an adapted snippet from our styleguide at Hex for our semantic layer which outlines our rules around the stable identifier key for our resources (we use id/name, but the same general advice applies to name/display-name):
https://gist.github.com/cpimhoff/fb52bc93703ff4f2d0e14e3f644d5547

[jochenchrist]

+1 for id

When we discuss importing from RDF, the ID would be an IRI (e.g. https://onerecord.iata.org/ns/cargo#Carrier).

Do we need both, an local unique id and a universal unique iri?

Regarding name: We discussed language-strings to support different translations for the display name. We would probably recommend a well-known pattern to translate the displayName, e.g. using annotations array.

Option A: Use IRI as ID

id: "https://onerecord.iata.org/ns/cargo#Carrier"
name: "carrier"
description: "Company details of carriers"
annotations:
- name: "displayName"
  value: "Carrier"
  lang: "en"
- name: "displayName"
  value: "Beförderer"
  lang: "de"

Option B: id + iri as a separate field

id: "carrier"
iri: "https://onerecord.iata.org/ns/cargo#Carrier"
name: "Carrier"
description: "Company details of carriers"
annotations:
- name: "displayName"
  value: "Carrier"
  lang: "en"
- name: "displayName"
  value: "Beförderer"
  lang: "de"

Follow-up question

When we introduce id, what do we use in the relationships and formula to refer to a concept? The id or the name property?

[QMalcolm]

I personally think discussion of a URI or an IRI should be a separate discussion, it's a whole nother rabbit hole 😅. As for what to use for relationships and formula to refer to a concept, if we go the route of id being the unchanging identifier and name being the display name, id is what should be used. In such an end state name should only be used for display purposes, not for making references.