FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

gh-109370: Support closing Connection and PipeConnection from other thread by serhiy-storchaka · Pull Request #109397 · python/cpython · GitHub

/ cpython Public

gh-109370: Support closing Connection and PipeConnection from other thread - #109397

Closed
serhiy-storchaka wants to merge 1 commit into
python:mainfrom
serhiy-storchaka:multiprocessing-connection-close
Closed

gh-109370: Support closing Connection and PipeConnection from other thread#109397
serhiy-storchaka wants to merge 1 commit into
python:mainfrom
serhiy-storchaka:multiprocessing-connection-close

Conversation

serhiy-storchaka commented Sep 14, 2023
edited by bedevere-app Bot
Loading

Copy link
Copy Markdown
Member

Copy link
Copy Markdown
Member Author

I do not like this.

  1. It is too complicated and verbose.
  2. There is still a chance for race condition, if between pushing self.handler on the stack and calling the corresponding OS function, the handler was closed and a new file was open that reuses the same fileno.

I have not tested the code on Windows yet, very likely more try/exept or return code checks should be added in Windows specific code.

_winapi.PeekNamedPipe(self._handle)[0] != 0):
return True
except OSError as err:
if err.errno == errno.EBADF:

vstinner Sep 14, 2023
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

We should not get EBADF. We should just not call functions on closed handle. If we land in this case, we failed badly before, no?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

How can we know that it was closed in other thread without using locks?

You can get EBADF in the following scenario:

  • Thread A reads self._handle and puts in on the stack.
  • Thread B closes it.
  • Thread A calls function with an already closed file descriptor on the stack.
  • That function fails with EBADF errno.

It is the bast case. The worst case is:

  • Thread A reads self._handle and puts in on the stack.
  • Thread B closes it.
  • Thread B or C opens a new file descriptor which reuses the same integer value.
  • Thread A calls function with a file descriptor which now refers to unrelated file or socket.
  • Unpredictable consequences.

raise
except TypeError:
self._check_closed()
raise

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Would it be possible to split this change in two parts? Keep this change for _check_closed() change which now raises BrokenPipeError, and you changes around close(): these ones look straightforward are correct. But write a separated PR to add these try/except?

I dislike these try/except. If there is a risk to land into EBADF case with the current code, maybe some kind of locking is needed to not call the Windows API with a handle, while another thread call close() which invalidates the handle.

Copy link
Copy Markdown
Member Author

The more I look at this, the more I dislike it. I do not see how is it possible to safely close a file descriptor on other thread without locking. Making Connection and PipeConnection truly thread-safe in a task on completely different level.

I opened #109780 instead.

vstinner commented Sep 25, 2023
edited
Loading

Copy link
Copy Markdown
Member

Making Connection and PipeConnection truly thread-safe in a task on completely different level.

Maybe we should just give up and document that they are not thread-safe.

I recall that I did that once on some asyncio classes.

This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants


Back | FazBrowse Home | New Git URL