-
-
Notifications
You must be signed in to change notification settings - Fork 100
Fix race conditions and improve robustness during socket I/O #779
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -4,6 +4,8 @@ | |
| import _pyio as io | ||
| import socket | ||
|
|
||
| from . import errors as _errors | ||
|
|
||
|
|
||
| # Write only 16K at a time to sockets | ||
| SOCK_WRITE_BLOCKSIZE = 16384 | ||
|
|
@@ -32,8 +34,45 @@ | |
| n = self.raw.write(bytes(self._write_buf)) | ||
| except io.BlockingIOError as e: | ||
| n = e.characters_written | ||
|
|
||
| if n == 0: | ||
| # Non-blocking socket can't write right now — stop flushing. | ||
| break | ||
| if not n: | ||
| # Defensive: write() returned None or other falsy value, | ||
| # which shouldn't happen but could cause an infinite loop. | ||
| break | ||
Check warningCode scanning / CodeQL Unreachable code Warning
This statement is unreachable.
|
||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The tests never hit this line. We should come up with a new regression test. |
||
|
|
||
| del self._write_buf[:n] | ||
|
|
||
| def close(self): | ||
| """ | ||
| Close the stream and its underlying file object. | ||
|
|
||
| This method is designed to be idempotent (it can be called multiple | ||
| times without side effects). It gracefully handles a race condition | ||
| where the underlying socket may have already been closed by the remote | ||
| client or another thread. | ||
|
|
||
| A :exc:`ConnectionError` or :exc:`OSError` with | ||
| :data:`~errno.EBADF` or :data:`~errno.ENOTCONN` is caught | ||
| and ignored, as these indicate a normal, expected connection teardown. | ||
| Other exceptions are re-raised. | ||
| """ | ||
| # pylint incorrectly flags inherited self.closed property as constant | ||
| if self.closed: # pylint: disable=using-constant-test | ||
| return | ||
|
|
||
| try: | ||
| super().close() | ||
| except ConnectionError: | ||
| return | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Looks like tests never hit this. |
||
| except OSError as err: | ||
| # Handle EBADF and other acceptable socket shutdown errors | ||
| if err.errno in _errors.acceptable_sock_shutdown_error_codes: | ||
| return | ||
|
Comment on lines
+71
to
+73
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @julianz- do you know if error codes listed there contain things that aren't represented by
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Also, do we know what's causing the tests to always hit this line? It's best to make sure that this isn't due to some mocks but a real simulated scenario. |
||
| raise | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I wish we have proper logging infra here to see what can actually happen. The coverage report shows that this line is never exercised in the tests.
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Perhaps, we could have a warning emitted, though.. |
||
|
|
||
|
|
||
| class StreamReader(io.BufferedReader): | ||
| """Socket stream reader.""" | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| Socket I/O is now resilient to race conditions happening during connection teardown | ||
| due to sockets dying independently or being closed externally. | ||
|
|
||
| -- by :user:`julianz-` |
|
julianz- marked this conversation as resolved.
|
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -67,6 +67,7 @@ subpackages | |
| symlinked | ||
| syscall | ||
| systemd | ||
| teardown | ||
| threadpool | ||
| Tidelift | ||
| TLS | ||
|
|
||
Uh oh!
There was an error while loading. Please reload this page.