[release-4.22] OCPBUGS-83556: Fix EventOutCh deadlock by making publish non-blocking#664
Conversation
PublishEventViaAPI sends to EventOutCh with a blocking channel send. When ProcessOutChannel (the consumer) is slow due to HTTP POST failures (e.g., unreachable subscribers), the 100-slot buffer fills up and the next publish blocks forever. This deadlocks the processMessages goroutine, freezing socket I/O and causing cloud-event-proxy to hang silently. Fix by using select with default to drop events when the channel is full instead of blocking indefinitely. Dropped events are logged as warnings and counted as failures in metrics. Signed-off-by: Jack Ding <jackding@gmail.com>
jzding
changed the title
|
@openshift-cherrypick-robot: This pull request references Jira Issue OCPBUGS-83556, which is invalid:
Comment The bug has been updated to refer to the pull request using the external bug tracker. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
jzding
changed the title
|
/jira refresh |
|
@jzding: This pull request references Jira Issue OCPBUGS-83556, which is valid. The bug has been moved to the POST state. 7 validation(s) were run on this bug
No GitHub users were found matching the public email listed for the QA contact in Jira (bblock@redhat.com), skipping review request. The bug has been updated to refer to the pull request using the external bug tracker. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: nocturnalastro, openshift-cherrypick-robot The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
@openshift-cherrypick-robot: Jira Issue OCPBUGS-83556: All pull requests linked via external trackers have merged: Jira Issue OCPBUGS-83556 has been moved to the MODIFIED state. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
/cherry-pick release-4.21 |
|
@jzding: new pull request created: #665 DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
4 participants
This is an automated cherry-pick of #663
/assign jzding