Skip to content

Fix/issue 26775 yaml parse error#27018

Closed
Esvanth wants to merge 12 commits intoopen-metadata:mainfrom
Esvanth:fix/issue-26775-yaml-parse-error
Closed

Fix/issue 26775 yaml parse error#27018
Esvanth wants to merge 12 commits intoopen-metadata:mainfrom
Esvanth:fix/issue-26775-yaml-parse-error

Conversation

@Esvanth
Copy link
Copy Markdown

@Esvanth Esvanth commented Apr 3, 2026

This PR addresses critical configuration issues and UI translation errors found in the OpenMetadata deployment and signup flows:

Docker YAML Parsing Fix (#26775): Replaced fragile environment variable interpolation (e.g., ${OIDC_CUSTOM_PARAMS:-{}}) with literal strings "{}" in 8 Docker Compose files. This ensures cross-platform compatibility for container runtimes like Podman that struggle with nested curly braces in YAML.
French Signup Translation Fix (#25711): Introduced specific translation keys label.first-name and label.last-name. Updated the BasicSignup component to use these keys, resolving the incorrect "Nom de Premier" phrasing in French and replacing it with the correct "Prénom".
Release Announcement Refinement (#26859): Updated the "What's New" announcement text to "What's new in this release" to prevent confusion during fresh deployments.
Testing performed:

Verified YAML syntax integrity.
Manually checked the updated i18n keys in the localization files.
Updated and ran WhatsNewAlert.test.tsx to confirm the announcement text change.
Type of change:
Bug fix
Improvement
New feature
Breaking change (fix or feature that would cause existing functionality to not work as expected)
Documentation
Checklist:
I have read the CONTRIBUTING document.
My PR title is Fixes #26775, #25711, #26859: Resolve Docker YAML parsing and French translation errors
I have commented on my code, particularly in hard-to-understand areas.

@Esvanth Esvanth requested review from a team, akash-jain-10, harshach and tutte as code owners April 3, 2026 10:04
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 3, 2026

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

Comment thread ingestion/src/metadata/profiler/orm/registry.py
Comment thread docker/development/docker-compose-gcp.yml Outdated
@harshach
Copy link
Copy Markdown
Collaborator

harshach commented Apr 3, 2026

@Esvanth the right way of fixing is this PR #26205 if you want to take that PR up and add mroe testing and. manual testing , please do. We can merge that PR in

@Esvanth
Copy link
Copy Markdown
Author

Esvanth commented Apr 3, 2026 via email

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 5, 2026

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

Comment on lines +62 to +76
static boolean isInlineSubstitution(TextStringBuilder buf, int startPos, int endPos) {
if (startPos > 0) {
char before = buf.charAt(startPos - 1);
if (before != ' ' && before != '\t' && before != '\n' && before != '\r') {
return true;
}
}
if (endPos < buf.length()) {
char after = buf.charAt(endPos);
if (after != ' ' && after != '\t' && after != '\n' && after != '\r') {
return true;
}
}
return false;
}
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Edge Case: isInlineSubstitution only checks immediate neighbors

The isInlineSubstitution check looks at the single character before startPos and after endPos. If the YAML value line is key: ${VAR}, the character before startPos is a space (from key: ), so isInlineSubstitution returns false and the value gets quoted. This is the correct behavior.

However, for patterns like key:${VAR} (no space after colon), the character before startPos is :, so it returns true and quoting is skipped — even though the variable is the entire value semantically. This edge case is unlikely in practice (Dropwizard YAML configs always have key: value format), but worth documenting.

Was this helpful? React with 👍 / 👎 | Reply gitar fix to apply this suggestion

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 5, 2026

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

@gitar-bot
Copy link
Copy Markdown

gitar-bot Bot commented Apr 5, 2026

Code Review 👍 Approved with suggestions 4 resolved / 5 findings

Fixes YAML parsing errors in configuration and Docker compose by handling reserved words and guarding metrics against complex types. One minor suggestion: the isInlineSubstitution check could be more robust by examining broader context beyond immediate neighbors.

💡 Edge Case: isInlineSubstitution only checks immediate neighbors

📄 openmetadata-service/src/main/java/org/openmetadata/service/util/YamlSafeSubstitutor.java:62-76

The isInlineSubstitution check looks at the single character before startPos and after endPos. If the YAML value line is key: ${VAR}, the character before startPos is a space (from key: ), so isInlineSubstitution returns false and the value gets quoted. This is the correct behavior.

However, for patterns like key:${VAR} (no space after colon), the character before startPos is :, so it returns true and quoting is skipped — even though the variable is the entire value semantically. This edge case is unlikely in practice (Dropwizard YAML configs always have key: value format), but worth documenting.

✅ 4 resolved
Bug: NOT_COMPUTE reduction without updating core.py filter lets complex types through

📄 ingestion/src/metadata/profiler/orm/registry.py:119-121
Types like ARRAY, JSON, SQAMap, SQAStruct, SQASet, SQAUnion, SQASGeography, and GEOMETRY were removed from NOT_COMPUTE and placed into COLLECTION_SET, STRUCT_SET, and COMPLEX_SET. However, processor/core.py:380 filters columns exclusively via NOT_COMPUTE. This means complex-type columns that were previously excluded from ALL metric computation now pass through the filter and will have metrics dispatched to them.

While some individual metrics (distinct_count, unique_count, regex counts) add their own is_complex_type guards, this is an incomplete defense. Any metric that does NOT have an explicit is_complex_type guard will now be executed against complex-type columns, likely causing SQL errors or incorrect results.

The safest fix is to also filter complex types in the central column preparation method, so all metrics benefit from the guard.

Bug: Docker OIDC_CUSTOM_PARAMS hardcoded, removing env var override

📄 docker/development/docker-compose-gcp.yml:122 📄 docker/development/docker-compose-gcp.yml:336
Replacing ${OIDC_CUSTOM_PARAMS:-{}} with the literal "{}" fixes the YAML parsing issue but also removes the ability for users to configure custom OIDC parameters via environment variables. This parameter is documented across multiple SSO provider guides (Google, Auth0, Azure, etc.) and used in conf/openmetadata.yaml as customParams: ${OIDC_CUSTOM_PARAMS:-}. Users who set OIDC_CUSTOM_PARAMS in their environment (e.g., {"hd": "company.com"} for Google Workspace domain restriction) will silently have their configuration ignored.

A better fix would preserve override capability while avoiding the YAML parsing issue with nested braces.

Bug: Test has wrong expected value for yamlDoubleQuote(">value")

📄 openmetadata-service/src/test/java/org/openmetadata/service/util/YamlSafeSubstitutorTest.java:60
The parameterized test case at line 60 expects yamlDoubleQuote(">value") to return "> value" (with 16 spaces between > and value). However, the yamlDoubleQuote method (lines 180-190) only escapes \, ", , \r, and wraps in double quotes — it would produce ">value" with no extra spaces. This test case will fail.

It looks like the spaces may have been introduced by an editor or copy-paste artifact treating > as a YAML folded block scalar indicator.

Edge Case: needsYamlQuoting misses YAML boolean/null reserved words

📄 openmetadata-service/src/main/java/org/openmetadata/service/util/YamlSafeSubstitutor.java:89-103
The needsYamlQuoting method checks for YAML structural indicators but does not handle YAML reserved words like true, false, yes, no, null, on, off, etc. If an environment variable value is literally true or yes, YAML parsers may interpret it as a boolean rather than a string. For example, DB_FLAG=yes would be parsed as boolean true by SnakeYAML.

This may be acceptable if Dropwizard's config deserialization handles type coercion correctly, but it's worth noting as a potential edge case for string-typed config fields.

🤖 Prompt for agents
Code Review: Fixes YAML parsing errors in configuration and Docker compose by handling reserved words and guarding metrics against complex types. One minor suggestion: the isInlineSubstitution check could be more robust by examining broader context beyond immediate neighbors.

1. 💡 Edge Case: isInlineSubstitution only checks immediate neighbors
   Files: openmetadata-service/src/main/java/org/openmetadata/service/util/YamlSafeSubstitutor.java:62-76

   The `isInlineSubstitution` check looks at the single character before `startPos` and after `endPos`. If the YAML value line is `key: ${VAR}`, the character before `startPos` is a space (from `key: `), so `isInlineSubstitution` returns false and the value gets quoted. This is the correct behavior.
   
   However, for patterns like `key:${VAR}` (no space after colon), the character before `startPos` is `:`, so it returns true and quoting is skipped — even though the variable *is* the entire value semantically. This edge case is unlikely in practice (Dropwizard YAML configs always have `key: value` format), but worth documenting.

Options

Display: compact → Showing less information.

Comment with these commands to change:

Compact
gitar display:verbose         

Was this helpful? React with 👍 / 👎 | Gitar

@Esvanth
Copy link
Copy Markdown
Author

Esvanth commented Apr 5, 2026

@harshach can you please say ellaboratrtely the change where i should make it and also please make it safe to check label so it can bypass checks respectively.

@PubChimps
Copy link
Copy Markdown
Contributor

Hi @Esvanth , thanks for your pr here! We had a PR submitted for this already here. Please make sure that you are assigned an issue and that there is not a PR out for it already before submitting something. We will reopen this PR if necessary.

@PubChimps PubChimps closed this Apr 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Issues with QuickStart

3 participants