[Core] Publish platform events via Ray Event Recorder by richabanker · Pull Request #63329 · ray-project/ray

richabanker · 2026-05-13T22:30:35Z

Description

Add support for publishing Platform events via the python ray event exporter framework

gemini-code-assist

Code Review

This pull request introduces the PlatformEventBuilder class to support infrastructure platform events, such as those from Kubernetes, and integrates it into the Ray dashboard's observability module. The changes include initializing the EventRecorder in the dashboard head and emitting events during processing callbacks. Feedback suggests allowing the EventRecorder to generate unique IDs for event updates to avoid deduplication issues, refactoring environment variable checks for efficiency, and moving imports out of the event processing hot path.

gemini-code-assist · 2026-05-13T22:32:12Z

+        if os.environ.get("RAY_ENABLE_PYTHON_RAY_EVENT", "False").lower() in (
+            "true",
+            "1",
+        ):
+            try:
+                from ray._common.observability.platform_events import (
+                    PlatformEventBuilder,
+                )
+                from ray._raylet import EventRecorder


Several modules are imported inside _process_event_callback for every event. Since this callback can be invoked frequently, these imports introduce unnecessary overhead. It is recommended to move these imports to the top of the file or at least outside the hot path of the callback.

richabanker · 2026-05-14T00:44:25Z

@sampan-s-nayak could you please help take a pass at this PR whenever possible for you? Thanks!

richabanker · 2026-05-14T00:45:21Z

@@ -69,32 +103,56 @@ def thread_safe_callback(ray_event: RayEvent):
            )

    def _process_event_callback(self, ray_event: RayEvent):
-        """Callback running in the main asyncio loop to cache events."""
+        """Thread-safe entry point that dispatches event caching to the main asyncio loop."""


change driven by this comment

edoakes · 2026-05-15T15:40:09Z

@sampan-s-nayak PTAL

richabanker · 2026-05-28T00:12:43Z

@sampan-s-nayak I think all comments are now addressed, any other concerns or feedback you have for this?

sampan-s-nayak

thanks for working on this, LGTM

sampan-s-nayak · 2026-05-28T19:17:13Z

@richabanker looks like there are a couple of CI failures

cursor

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

There are 2 total unresolved issues (including 1 from previous review).

^{Reviewed by Cursor Bugbot for commit 961765aa39c29585ca1a20a9ca786f02368da2bb. Configure here.}

cursor · 2026-06-01T19:02:35Z

+                )
+                EventRecorder.emit(cython_event)
+            except Exception as e:
+                logger.warning(f"Failed to emit platform event via EventRecorder: {e}")


Missing initialization guard causes repeated emit failures per event

Low Severity

If EventRecorder.initialize() is never called (because head_node_id_hex is None or the agent address lookup returns None), _process_event_internal still attempts EventRecorder.emit() on every single incoming event. Each call hits the except branch and logs a warning. In a busy Kubernetes cluster this could produce hundreds of warning-level log messages per minute with no mechanism to stop retrying.

Additional Locations (1)

python/ray/dashboard/modules/platform_events/platform_event_head.py#L52-L81

^{Reviewed by Cursor Bugbot for commit 961765aa39c29585ca1a20a9ca786f02368da2bb. Configure here.}

Signed-off-by: Richa Banker <richabanker@google.com>

richabanker · 2026-06-02T18:19:39Z

@richabanker looks like there are a couple of CI failures

Fixed them now, thanks!

) ## Description Add support for publishing Platform events via the python ray event exporter framework --------- Signed-off-by: Richa Banker <richabanker@google.com>

richabanker requested review from a team, MengjinYan, dayshah and edoakes as code owners May 13, 2026 22:30

gemini-code-assist Bot reviewed May 13, 2026

View reviewed changes

cursor Bot reviewed May 13, 2026

View reviewed changes

Comment thread python/ray/dashboard/modules/aggregator/tests/test_ray_platform_events.py

richabanker force-pushed the platform-events-ray-event-recorder branch from 8718785 to b35d418 Compare May 13, 2026 22:40

cursor Bot reviewed May 13, 2026

View reviewed changes

Comment thread python/ray/dashboard/modules/aggregator/tests/test_ray_platform_events.py Outdated

richabanker force-pushed the platform-events-ray-event-recorder branch from b35d418 to bac7c35 Compare May 13, 2026 23:25

cursor Bot reviewed May 13, 2026

View reviewed changes

Comment thread python/ray/dashboard/modules/platform_events/platform_event_head.py

richabanker force-pushed the platform-events-ray-event-recorder branch from bac7c35 to 271db30 Compare May 13, 2026 23:36

cursor Bot reviewed May 13, 2026

View reviewed changes

Comment thread python/ray/dashboard/modules/platform_events/tests/test_platform_event_head.py Outdated

richabanker force-pushed the platform-events-ray-event-recorder branch 3 times, most recently from a8a923c to 1e59f09 Compare May 14, 2026 00:21

cursor Bot reviewed May 14, 2026

View reviewed changes

Comment thread python/ray/dashboard/modules/platform_events/platform_event_head.py

richabanker commented May 14, 2026

View reviewed changes

ray-gardener Bot added core Issues that should be addressed in Ray Core observability Issues related to the Ray Dashboard, Logging, Metrics, Tracing, and/or Profiling community-contribution Contributed by the community labels May 14, 2026

edoakes assigned sampan-s-nayak May 15, 2026

sampan-s-nayak reviewed May 18, 2026

View reviewed changes

Comment thread python/ray/_private/ray_constants.py Outdated

sampan-s-nayak reviewed May 19, 2026

View reviewed changes

Comment thread python/ray/dashboard/modules/platform_events/platform_event_head.py Outdated

sampan-s-nayak reviewed May 19, 2026

View reviewed changes

Comment thread python/ray/dashboard/modules/platform_events/platform_event_head.py Outdated

richabanker force-pushed the platform-events-ray-event-recorder branch 2 times, most recently from 64e7752 to 07e69ed Compare May 19, 2026 22:58

richabanker force-pushed the platform-events-ray-event-recorder branch 2 times, most recently from 8a3ecd9 to 61c73d8 Compare May 28, 2026 00:02

cursor Bot reviewed May 28, 2026

View reviewed changes

Comment thread python/ray/dashboard/modules/platform_events/tests/test_platform_event_head.py

richabanker force-pushed the platform-events-ray-event-recorder branch 2 times, most recently from 38f99f7 to 7286bd3 Compare May 28, 2026 00:12

sampan-s-nayak approved these changes May 28, 2026

View reviewed changes

richabanker force-pushed the platform-events-ray-event-recorder branch from ccd1cc8 to 961765a Compare June 1, 2026 18:56

cursor Bot reviewed Jun 1, 2026

View reviewed changes

richabanker added 3 commits June 1, 2026 14:08

[Core] Publish platform events via Ray Event Recorder

e3afa83

Signed-off-by: Richa Banker <richabanker@google.com>

[Core] Publish platform events via Ray Event Recorder

70b74ce

Signed-off-by: Richa Banker <richabanker@google.com>

[Core] Publish platform events via Ray Event Recorder

5936b11

Signed-off-by: Richa Banker <richabanker@google.com>

richabanker force-pushed the platform-events-ray-event-recorder branch from e9beaf2 to 5936b11 Compare June 1, 2026 21:08

edoakes added the go add ONLY when ready to merge, run all tests label Jun 2, 2026

edoakes approved these changes Jun 2, 2026

View reviewed changes

edoakes enabled auto-merge (squash) June 2, 2026 19:02

edoakes merged commit 080c195 into ray-project:master Jun 2, 2026
9 checks passed

richabanker deleted the platform-events-ray-event-recorder branch June 4, 2026 18:39

Conversation

richabanker commented May 13, 2026

Description

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

gemini-code-assist Bot May 13, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

richabanker commented May 14, 2026

Uh oh!

richabanker May 14, 2026

Choose a reason for hiding this comment

Uh oh!

edoakes commented May 15, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

richabanker commented May 28, 2026

Uh oh!

sampan-s-nayak left a comment

Choose a reason for hiding this comment

Uh oh!

sampan-s-nayak commented May 28, 2026

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

cursor Bot Jun 1, 2026

Choose a reason for hiding this comment

Missing initialization guard causes repeated emit failures per event

Uh oh!

richabanker commented Jun 2, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants