FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

bpo-35721: Close socket pair if Popen in _UnixSubprocessTransport fails by niklasf · Pull Request #11553 · python/cpython · GitHub

  1. python
  2. cpython

Repository navigation

bpo-35721: Close socket pair if Popen in _UnixSubprocessTransport fails #11553

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
miss-islington merged 1 commit into from
May 20, 2019
Conversation 16 Commits 1 Checks 0 Files changed
Merged

bpo-35721: Close socket pair if Popen in _UnixSubprocessTransport fails #11553

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension .py  (2) .rst  (1) No extension  (1) All 3 file types selected
Viewed files
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Unified
Split
Hide whitespace
Diff view
Unified
Split
Hide whitespace
18 changes: 12 additions & 6 deletions Lib/asyncio/unix_events.py
Show comments
View file
Open in desktop
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Original file line number Diff line number Diff line change
Expand Up @@ -757,12 +757,18 @@ def _start(self, args, shell, stdin, stdout, stderr, bufsize, **kwargs):
# other end). Notably this is needed on AIX, and works
# just fine on other platforms.
stdin, stdin_w = socket.socketpair()
self._proc = subprocess.Popen(
args, shell=shell, stdin=stdin, stdout=stdout, stderr=stderr,
universal_newlines=False, bufsize=bufsize, **kwargs)
if stdin_w is not None:
stdin.close()
self._proc.stdin = open(stdin_w.detach(), 'wb', buffering=bufsize)
try:
self._proc = subprocess.Popen(
args, shell=shell, stdin=stdin, stdout=stdout, stderr=stderr,
universal_newlines=False, bufsize=bufsize, **kwargs)
if stdin_w is not None:
stdin.close()

asvetlov Jan 16, 2019

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Please move the line before open() call.
It saves 1 open file descriptor.
Not a very big deal but why not?

1st1 Jan 16, 2019

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

I'm not sure I even understand this code :) What's it for? Why does monkeypatching self._proc.stdin work?

asvetlov Jan 16, 2019

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

@1st1 please read a code comment at the begin of the method.
AIX requires to make stdin a socket pair (I love all Unix flavors and their differences).
proc.stdin should be a writable file-like object.
open() creates a file object from write-fd of created socket pair.

It exists in asyncio starting from very old times.

niklasf Jan 16, 2019

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

@asvetlov: Good point. I suppose since open(stdin_w.detach(), ...) cannot fail I should have kept the order.

1st1 Jan 16, 2019

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

@1st1 please read a code comment at the begin of the method.

Yeah, that's what I did first. The comment doesn't explain why we pass the read end of the pipe to Popen initially (I see that we then monkey-patch it to the write end).

It exists in asyncio starting from very old times.

I've no problem with the code or this PR, but I wish that the comment is a bit more elaborate.

asvetlov Jan 16, 2019

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

@1st1 you know, I'm bad in comment writing, sorry.
Would you propose exact text?
self._proc.stdin = open(stdin_w.detach(), 'wb', buffering=bufsize)
stdin_w = None
finally:
if stdin_w is not None:
stdin.close()
stdin_w.close()


class AbstractChildWatcher:
Expand Down
17 changes: 13 additions & 4 deletions Lib/test/test_asyncio/test_subprocess.py
Show comments
View file
Open in desktop
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Original file line number Diff line number Diff line change
Expand Up @@ -468,9 +468,7 @@ async def kill_running():
isinstance(self, SubprocessFastWatcherTests)):
asyncio.get_child_watcher()._callbacks.clear()

def test_popen_error(self):
# Issue #24763: check that the subprocess transport is closed
# when BaseSubprocessTransport fails
def _test_popen_error(self, stdin):
if sys.platform == 'win32':
target = 'asyncio.windows_utils.Popen'
else:
Expand All @@ -480,12 +478,23 @@ def test_popen_error(self):
popen.side_effect = exc

create = asyncio.create_subprocess_exec(sys.executable, '-c',
'pass', loop=self.loop)
'pass', stdin=stdin,
loop=self.loop)
with warnings.catch_warnings(record=True) as warns:
with self.assertRaises(exc):
self.loop.run_until_complete(create)
self.assertEqual(warns, [])

def test_popen_error(self):
# Issue #24763: check that the subprocess transport is closed
# when BaseSubprocessTransport fails
self._test_popen_error(stdin=None)

def test_popen_error_with_stdin_pipe(self):
# Issue #35721: check that newly created socket pair is closed when
# Popen fails
self._test_popen_error(stdin=subprocess.PIPE)

def test_read_stdout_after_process_exit(self):

async def execute():
Expand Down
1 change: 1 addition & 0 deletions Misc/ACKS
Show comments
View file
Open in desktop
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Original file line number Diff line number Diff line change
Expand Up @@ -483,6 +483,7 @@ Florian Festi
John Feuerstein
Carl Feynman
Vincent Fiack
Niklas Fiekas
Anastasia Filatova
Tomer Filiba
Segev Finer
Expand Down
3 changes: 3 additions & 0 deletions Misc/NEWS.d/next/Library/2019-01-18-16-23-00.bpo-35721.d8djAJ.rst
Show comments
View file
Open in desktop
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Fix :meth:`asyncio.SelectorEventLoop.subprocess_exec()` leaks file descriptors
if ``Popen`` fails and called with ``stdin=subprocess.PIPE``.
Patch by Niklas Fiekas.
Back | FazBrowse Home | New Git URL