Skip to content

[Console] Deduplicate Flink/Spark HTTP watcher polling per application#4387

Closed
shangeyao wants to merge 1 commit into
apache:devfrom
shangeyao:fix/watcher-single-flight-dedup
Closed

[Console] Deduplicate Flink/Spark HTTP watcher polling per application#4387
shangeyao wants to merge 1 commit into
apache:devfrom
shangeyao:fix/watcher-single-flight-dedup

Conversation

@shangeyao

Copy link
Copy Markdown
Contributor

Summary

  • Add per-application WATCH_IN_FLIGHT single-flight guard in Flink and Spark HTTP watchers
  • Skip enqueueing a new watch task when the previous poll is still running
  • Remove in-flight markers when an app stops being watched

Fixes #4386

Test plan

  • Start a Flink app and verify state updates still work
  • Under slow REST responses, confirm watcher does not pile up executor tasks

AI Disclosure

  • Model: Claude Opus 4.6
  • Platform/Tool: Cursor
  • Human Oversight: partially reviewed
  • Prompt Summary: Implement watcher deduplication from dev branch optimization scan

Made with Cursor

Use per-app single-flight guards so scheduled watcher ticks do not
enqueue overlapping REST polling tasks for the same application.

Generated-by: Cursor
Co-authored-by: Cursor <cursoragent@cursor.com>
@shangeyao

Copy link
Copy Markdown
Contributor Author

Closing this PR because the StreamPark Console frontend is about to undergo a major refactor. These console-side changes would likely conflict with or require rework after the refactor. Will revisit relevant improvements once the new frontend architecture is in place.

@shangeyao shangeyao closed this Jun 30, 2026
@sonarqubecloud

Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Console] Deduplicate Flink/Spark HTTP watcher polling per application

1 participant