Skip to content

HDDS-15963. Fix EC degraded read NPE caused by CodecRegistry depending on TCCL in isolated ClassLoader#10861

Open
hiwuya wants to merge 1 commit into
apache:masterfrom
hiwuya:HDDS-15963
Open

HDDS-15963. Fix EC degraded read NPE caused by CodecRegistry depending on TCCL in isolated ClassLoader#10861
hiwuya wants to merge 1 commit into
apache:masterfrom
hiwuya:HDDS-15963

Conversation

@hiwuya

@hiwuya hiwuya commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

This PR fixes a NullPointerException during EC degraded reads when the Ozone client runs inside an isolated plugin class loader (e.g. Trino Hive connector).

Normal reads succeed, but when a DataNode is unavailable, the EC degraded read path fails with:

java.lang.NullPointerException:
Cannot read the array length because "<local3>" is null
    at org.apache.ozone.erasurecode.rawcoder.util.CodecUtil
        .createRawDecoderWithFallback(CodecUtil.java:90)

Root cause: CodecRegistry loads coder factories via ServiceLoader.load(RawErasureCoderFactory.class), which uses the thread context class loader (TCCL). In Trino's isolated plugin class loader, the TCCL cannot see the Ozone META-INF/services providers, so getCoderNames("rs") returns null, causing the NPE in CodecUtil. Spark works because its TCCL can see the Ozone providers.

Fix: Use CodecRegistry's own class loader instead of the TCCL:

ServiceLoader.load(RawErasureCoderFactory.class,
    CodecRegistry.class.getClassLoader());

No changes to the singleton pattern, registration logic, native coder priority, fallback order, or public API.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-15963

How was this patch tested?

  • Added TestCodecRegistryTcclIsolation#testRegistryLoadsWithoutTccl in a separate test class to ensure it runs in its own JVM as the first test to touch CodecRegistry, avoiding vacuous passes when the eager singleton is already initialized under the normal TCCL by other tests. Sets TCCL to an empty ClassLoader(null) before first CodecRegistry initialization, then verifies getCoderNames("rs") is non-null, non-empty, and contains rs_java. Restores TCCL in finally.
  • Confirmed the test fails when the fix is reverted.

@chihsuan chihsuan left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fix! @hiwuya The fix itself looks correct. But this test can pass vacuously. I tried this locally. With the fix reverted and a different method order forced, all 8 tests pass, so a regression would go undetected.

Would it make sense to move this test to a separate class?

}

@Test
public void testRegistryLoadsWithoutTccl() {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CodecRegistry.instance is a static singleton, so if any other test in this class runs first, the registry is already initialized, and this test passes even with the fix reverted.

private static CodecRegistry instance = new CodecRegistry();

@hiwuya

hiwuya commented Jul 25, 2026

Copy link
Copy Markdown
Contributor Author

@chihsuan Good catch. I've moved the test to a separate class TestCodecRegistryTcclIsolation so it runs in its own JVM and is guaranteed to be the first test touching CodecRegistry.

@chihsuan chihsuan left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the update! Left a non-blocking nit. Other than that, LGTM.

.getCoderNames(ECReplicationConfig.EcCodec.RS.name().toLowerCase());
assertThat(rsCoderNames).isNotNull();
assertThat(rsCoderNames).isNotEmpty();
assertThat(rsCoderNames).contains(RSRawErasureCoderFactory.CODER_NAME);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: assertThat(rsCoderNames).contains(...) alone already implies non-null and non-empty.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants