Remove references to the Host name in the PostgreSQL database.

[rohit-joy]

Today, a CTID reference in the PostgreSQL DB for the Registry is prefixed by a domain name such as https://www.credentialregistry.org/resources/ce-GUID.

This is a bug rather than a feature which couples and encumbers us to keep the data always under the same domain making it difficult to move to a different domain. In addition, a database migration is nearly impossible without hacky scripts to update the data.

Some things to clean up the data in the PostgreSQL DB.

1. In the PostgreSQL database, clear the prefixed Domain names and Paths wherever a JSON-LD property has a reference to another JSON-LD using a URL. Instead use only the CTID.

2. In the Credential Registry API implementation, prefix the domain name to all the CTID references in the JSON-LD before it is returned to the user.

2a. The default output should include the Domain name and Path of the host that is serving the JSON-LD.

2b. Include support for a query string parameter to turn ON/OFF the prefixing of the Domain name and URL as a customization switch.

Please let me know if you have questions.

[jeannekitchens]

@edgarf who is going to handle this?

[edgarf]

@excelsior will work on it.

[rohit-joy]

Need to meet @excelsior @rohit-joy

[jeannekitchens]

@excelsior can this move forward?

[jeannekitchens]

@edgarf @excelsior need a timeline. This was posted back in September 2025.

[edgarf]

@jeannekitchens this requires a meeting between Rohit and Alex

[rohit-joy]

@edgarf @excelsior Are you guys looking to meet to figure out the steps to do this? In my mind, here's what I expect to happen:

1. Add an entry into the configuration to take the host URL like https://www.credentialengineregistry.org/. This must not be null or empty.
2. Add an entry into the configuration to take a list of all the fields that may need to be turned into FQDN URLs to the resources. This list is a list of JSON-path strings like [ "$.ceterms:owns", ... ].
3. Whenever a request comes in to https://www.credentialengineregistry.org/... for a resource with CTID, get the JSON-LD from the database. Whichever field has a URL type, generate a URL and replace the field value with https://www.credentialengineregistry.org/resource/{CTID}.
4. Update the database to not have the FQDN anymore for the fields in your list. In case someone checks in a URL by mistake from our Publisher, just add a temporary fallback for now. Add a Warning log entry so that we are notified that someone posted a FQDN URL for that field at publishing time even though it should only have been the CTID. It would be a bug that we would fix in our Publisher.
5. Return the modified JSON-LD to user with FQDN URLs for each resource or graph.

For Step 2, you might be wondering which fields are URL type. You can find a couple to start with by eyeballing the JSON-LDs. Generally speaking, these are referring to CTIDs only.

I recommend using the Ruby Set type for this. For example, this JSON-LD https://credentialengineregistry.org/resources/ce-9bd8c615-9f3c-40e6-9c20-6d9f811844e6 has a couple of fields like $.ceterms:owns which would be truncated to ce-e3a50d8c-fbca-4bbd-8d34-775c01c79664 in the database, but when it is returned to user would returned as https://credentialengineregistry.org/resources/ce-e3a50d8c-fbca-4bbd-8d34-775c01c79664.

Estimated cost I expect is 5 days including thorough testing. Please let me know if it could take more time.

[rohit-joy]

@edgarf @excelsior Let me know how much work has been done for this item. If it has not reached a point where it can be completed, then please pause on this item. Let's see whether we can fix this in the S3 buckets directly with a simple Argo Workflow that cleans the JSON-LDs.

Again, I don't want the entire list of JSON-LDs to be evaluated for cleaning every single time the Workflow is executed. This should happen only ONCE. After that, only the items that have been changed should be cleaned.

[mparsons-ce]

@rohit-joy If the objective is to store the data without the host, should we just start publishing with just the CTID?
Seems like it would not make sense to send a URL when the host will be stripped.

So:
"ceterms:ownedBy": [
"ce-a588a18b-2616-4c74-bdcd-6507c0693d0e"
],

Instead of:
"ceterms:ownedBy": [
"https://sandbox.credentialengineregistry.org/resources/ce-a588a18b-2616-4c74-bdcd-6507c0693d0e"
]

Of course this would be implemented after the conversion.

[jeannekitchens]

This is on hold.

[excelsior]

@rohit-joy I implemented most of the functiionality described above as part of the workflow uploading resources into S3. That guarantees that it is only performed once.

We don't have a dynamically mutable config for the new publishing API yet, but once we do, we'll need to decide how to update the previously uploaded resources when the list of URL fields is updated.

The changes I made so far haven't been integrated into the Ruby API, and no such plans exist.