Skip to content

[Common] Synchronize HadoopUtils YarnClient and HDFS access#4405

Closed
shangeyao wants to merge 1 commit into
apache:devfrom
shangeyao:fix/hadoop-utils-yarn-client-lock
Closed

[Common] Synchronize HadoopUtils YarnClient and HDFS access#4405
shangeyao wants to merge 1 commit into
apache:devfrom
shangeyao:fix/hadoop-utils-yarn-client-lock

Conversation

@shangeyao

Copy link
Copy Markdown
Contributor

Summary

  • Add hadoopLock to synchronize UGI, HDFS, YarnClient, and config access
  • Run Kerberos refresh closeHadoop() inside the same lock

Fixes #4404

Test plan

  • ./mvnw compile -pl streampark-common
  • Yarn submit + watcher polling under Kerberos refresh

AI Disclosure

  • Model: Claude Opus 4.6
  • Platform/Tool: Cursor
  • Human Oversight: partially reviewed
  • Prompt Summary: HadoopUtils concurrency fix from dev branch scan

Made with Cursor

Guard YarnClient, HDFS, and Kerberos refresh with a shared lock to
prevent concurrent use during credential rotation.

Generated-by: Cursor
Co-authored-by: Cursor <cursoragent@cursor.com>
@sonarqubecloud

Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Common] Synchronize HadoopUtils YarnClient and HDFS access

1 participant