GH-84850: Remove urllib.request.URLopener and FancyURLopener - #125739
Merged
hugovk merged 5 commits intoNov 19, 2024
Merged
Conversation
Copy link
Copy Markdown
Contributor
Sorry, something went wrong.
These classes have been deprecated since Python 3.3.
vstinner
reviewed
Oct 20, 2024
Comment thread
Lib/urllib/request.py
Copy link
Copy Markdown
Member
|
A code search on PyPI top 8,000 projects (at 2024-10-16) on URLopener finds 21 projects:
I didn't check the code to see if imports are conditionals on Python <= 3.3 or not. I ignored six.py copies and .pyi files. DetailsPYPI-2024-10-16/arelle_release-2.31.6.tar.gz: arelle_release-2.31.6/arelle/WebCache.py: class WebCacheUrlOpener(request.FancyURLopener):
PYPI-2024-10-16/bandit-1.7.10.tar.gz: bandit-1.7.10/bandit/blacklists/calls.py: | | | - urllib.URLopener | |
PYPI-2024-10-16/bandit-1.7.10.tar.gz: bandit-1.7.10/bandit/blacklists/calls.py: | | | - urllib.request.URLopener | |
PYPI-2024-10-16/bandit-1.7.10.tar.gz: bandit-1.7.10/bandit/blacklists/calls.py: | | | - urllib.FancyURLopener | |
PYPI-2024-10-16/bandit-1.7.10.tar.gz: bandit-1.7.10/bandit/blacklists/calls.py: | | | - urllib.request.FancyURLopener | |
PYPI-2024-10-16/bandit-1.7.10.tar.gz: bandit-1.7.10/bandit/blacklists/calls.py: | | | .URLopener | |
PYPI-2024-10-16/bandit-1.7.10.tar.gz: bandit-1.7.10/bandit/blacklists/calls.py: | | | .FancyURLopener | |
PYPI-2024-10-16/bandit-1.7.10.tar.gz: bandit-1.7.10/bandit/blacklists/calls.py: "urllib.request.URLopener",
PYPI-2024-10-16/bandit-1.7.10.tar.gz: bandit-1.7.10/bandit/blacklists/calls.py: "urllib.request.FancyURLopener",
PYPI-2024-10-16/bandit-1.7.10.tar.gz: bandit-1.7.10/examples/urlopen.py: opener = urllib.request.URLopener()
PYPI-2024-10-16/bandit-1.7.10.tar.gz: bandit-1.7.10/examples/urlopen.py: opener = urllib.request.FancyURLopener()
PYPI-2024-10-16/cinemagoer-2023.5.1.tar.gz: cinemagoer-2023.5.1/imdb/parser/http/__init__.py: class IMDbURLopener:
PYPI-2024-10-16/cinemagoer-2023.5.1.tar.gz: cinemagoer-2023.5.1/imdb/parser/http/__init__.py: self.urlOpener = IMDbURLopener(*arguments, **keywords)
PYPI-2024-10-16/cinemagoer-2023.5.1.tar.gz: cinemagoer-2023.5.1/tests/conftest.py: from imdb.parser.http import IMDbURLopener
PYPI-2024-10-16/cinemagoer-2023.5.1.tar.gz: cinemagoer-2023.5.1/tests/conftest.py: retrieve_unicode_orig = IMDbURLopener.retrieve_unicode
PYPI-2024-10-16/cinemagoer-2023.5.1.tar.gz: cinemagoer-2023.5.1/tests/conftest.py: IMDbURLopener.retrieve_unicode = retrieve_unicode_cached
PYPI-2024-10-16/cinemagoer-2023.5.1.tar.gz: cinemagoer-2023.5.1/tests/conftest.py: IMDbURLopener.retrieve_unicode = retrieve_unicode_orig
PYPI-2024-10-16/eventlet-0.37.0.tar.gz: eventlet-0.37.0/eventlet/green/urllib/request.py: URLopener.open_ftp = patcher.patch_function(URLopener.open_ftp, *to_patch_in_functions)
PYPI-2024-10-16/fissix-24.4.24.tar.gz: fissix-24.4.24/fissix/fixes/fix_urllib.py: "URLopener",
PYPI-2024-10-16/fissix-24.4.24.tar.gz: fissix-24.4.24/fissix/fixes/fix_urllib.py: "FancyURLopener",
PYPI-2024-10-16/flake8_tidy_imports-4.10.0.tar.gz: flake8_tidy_imports-4.10.0/src/flake8_tidy_imports/__init__.py: "urllib.FancyURLopener": (
PYPI-2024-10-16/flake8_tidy_imports-4.10.0.tar.gz: flake8_tidy_imports-4.10.0/src/flake8_tidy_imports/__init__.py: "urllib.URLopener": (
PYPI-2024-10-16/future-1.0.0.tar.gz: future-1.0.0/src/future/moves/urllib/request.py: # URLopener,
PYPI-2024-10-16/future-1.0.0.tar.gz: future-1.0.0/src/future/moves/urllib/request.py: # FancyURLopener,
PYPI-2024-10-16/future-1.0.0.tar.gz: future-1.0.0/src/libpasteurize/fixes/fix_imports2.py: u'ContentTooShortError', u'FancyURLopener',u'URLopener',
PYPI-2024-10-16/future-1.0.0.tar.gz: future-1.0.0/tests/test_future/test_urllib.py: # Shortcut for testing FancyURLopener
PYPI-2024-10-16/future-1.0.0.tar.gz: future-1.0.0/tests/test_future/test_urllib.py: opener = urllib_request.FancyURLopener(proxies=proxies)
PYPI-2024-10-16/future-1.0.0.tar.gz: future-1.0.0/tests/test_future/test_urllib.py: ('FancyURLopener style of invoking requests is deprecated.',
PYPI-2024-10-16/future-1.0.0.tar.gz: future-1.0.0/tests/test_future/test_urllib.py: opener = urllib_request.FancyURLopener()
PYPI-2024-10-16/future-1.0.0.tar.gz: future-1.0.0/tests/test_future/test_urllib.py: def test_URLopener_deprecation(self):
PYPI-2024-10-16/future-1.0.0.tar.gz: future-1.0.0/tests/test_future/test_urllib.py: urllib_request.URLopener()
PYPI-2024-10-16/future-1.0.0.tar.gz: future-1.0.0/tests/test_future/test_urllib.py: class URLopener_Tests(unittest.TestCase):
PYPI-2024-10-16/future-1.0.0.tar.gz: future-1.0.0/tests/test_future/test_urllib.py: """Testcase to test the open method of URLopener class."""
PYPI-2024-10-16/future-1.0.0.tar.gz: future-1.0.0/tests/test_future/test_urllib.py: class DummyURLopener(urllib_request.URLopener):
PYPI-2024-10-16/future-1.0.0.tar.gz: future-1.0.0/tests/test_future/test_urllib.py: ('DummyURLopener style of invoking requests is deprecated.',
PYPI-2024-10-16/future-1.0.0.tar.gz: future-1.0.0/tests/test_future/test_urllib.py: self.assertEqual(DummyURLopener().open(
PYPI-2024-10-16/future-1.0.0.tar.gz: future-1.0.0/tests/test_future/test_urllib.py: self.assertEqual(DummyURLopener().open(
PYPI-2024-10-16/future-1.0.0.tar.gz: future-1.0.0/tests/test_future/test_urllib_toplevel.py: # Shortcut for testing FancyURLopener
PYPI-2024-10-16/future-1.0.0.tar.gz: future-1.0.0/tests/test_future/test_urllib_toplevel.py: opener = urllib_request.FancyURLopener(proxies=proxies)
PYPI-2024-10-16/future-1.0.0.tar.gz: future-1.0.0/tests/test_future/test_urllib_toplevel.py: ('FancyURLopener style of invoking requests is deprecated.',
PYPI-2024-10-16/future-1.0.0.tar.gz: future-1.0.0/tests/test_future/test_urllib_toplevel.py: opener = urllib_request.FancyURLopener()
PYPI-2024-10-16/future-1.0.0.tar.gz: future-1.0.0/tests/test_future/test_urllib_toplevel.py: def test_URLopener_deprecation(self):
PYPI-2024-10-16/future-1.0.0.tar.gz: future-1.0.0/tests/test_future/test_urllib_toplevel.py: urllib_request.URLopener()
PYPI-2024-10-16/future-1.0.0.tar.gz: future-1.0.0/tests/test_future/test_urllib_toplevel.py: class URLopener_Tests(unittest.TestCase):
PYPI-2024-10-16/future-1.0.0.tar.gz: future-1.0.0/tests/test_future/test_urllib_toplevel.py: """Testcase to test the open method of URLopener class."""
PYPI-2024-10-16/future-1.0.0.tar.gz: future-1.0.0/tests/test_future/test_urllib_toplevel.py: class DummyURLopener(urllib_request.URLopener):
PYPI-2024-10-16/future-1.0.0.tar.gz: future-1.0.0/tests/test_future/test_urllib_toplevel.py: ('DummyURLopener style of invoking requests is deprecated.',
PYPI-2024-10-16/future-1.0.0.tar.gz: future-1.0.0/tests/test_future/test_urllib_toplevel.py: self.assertEqual(DummyURLopener().open(
PYPI-2024-10-16/future-1.0.0.tar.gz: future-1.0.0/tests/test_future/test_urllib_toplevel.py: self.assertEqual(DummyURLopener().open(
PYPI-2024-10-16/future-1.0.0.tar.gz: future-1.0.0/tests/test_future/test_urllibnet.py: open_url = urllib_request.FancyURLopener().open(URL)
PYPI-2024-10-16/jplephem-2.22.tar.gz: jplephem-2.22/jplephem/excerpter.py: from urllib.request import URLopener
PYPI-2024-10-16/jplephem-2.22.tar.gz: jplephem-2.22/jplephem/excerpter.py: from urllib import URLopener
PYPI-2024-10-16/jplephem-2.22.tar.gz: jplephem-2.22/jplephem/excerpter.py: self.opener = URLopener()
PYPI-2024-10-16/m2crypto-0.42.0.tar.gz: m2crypto-0.42.0/src/M2Crypto/m2urllib.py: URLopener.open_https = open_https
PYPI-2024-10-16/newrelic-10.2.0.tar.gz: newrelic-10.2.0/newrelic/hooks/external_urllib.py: if hasattr(module, 'URLopener'):
PYPI-2024-10-16/newrelic-10.2.0.tar.gz: newrelic-10.2.0/newrelic/hooks/external_urllib.py: wrap_function_wrapper(module, 'URLopener.open',
PYPI-2024-10-16/newrelic-10.2.0.tar.gz: newrelic-10.2.0/tests/external_httplib/test_urllib.py: opener = urllib.URLopener()
PYPI-2024-10-16/newrelic-10.2.0.tar.gz: newrelic-10.2.0/tests/external_httplib/test_urllib.py: opener = urllib.URLopener()
PYPI-2024-10-16/newrelic-10.2.0.tar.gz: newrelic-10.2.0/tests/external_httplib/test_urllib.py: opener = urllib.URLopener()
PYPI-2024-10-16/newrelic-10.2.0.tar.gz: newrelic-10.2.0/tests/external_httplib/test_urllib.py: opener = urllib.URLopener()
PYPI-2024-10-16/newrelic-10.2.0.tar.gz: newrelic-10.2.0/tests/external_httplib/test_urllib.py: opener = urllib.URLopener()
PYPI-2024-10-16/newrelic-10.2.0.tar.gz: newrelic-10.2.0/tests/external_httplib/test_urllib.py: opener = urllib.URLopener()
PYPI-2024-10-16/pscript-0.7.7.tar.gz: pscript-0.7.7/translate_to_legacy.py: 'ContentTooShortError', 'FancyURLopener', 'URLopener',
PYPI-2024-10-16/py-backwards-0.7.tar.gz: py-backwards-0.7/py_backwards/transformers/six_moves.py: MovedAttribute("URLopener", "urllib", "urllib.request"),
PYPI-2024-10-16/py-backwards-0.7.tar.gz: py-backwards-0.7/py_backwards/transformers/six_moves.py: MovedAttribute("FancyURLopener", "urllib", "urllib.request"),
PYPI-2024-10-16/pygments-2.18.0.tar.gz: pygments-2.18.0/pygments/lexers/_sourcemod_builtins.py: from urllib.request import FancyURLopener
PYPI-2024-10-16/pygments-2.18.0.tar.gz: pygments-2.18.0/pygments/lexers/_sourcemod_builtins.py: class Opener(FancyURLopener):
PYPI-2024-10-16/reportlab-4.2.5.tar.gz: reportlab-4.2.5/src/reportlab/lib/utils.py: #copied here from urllib.URLopener.open_data because
PYPI-2024-10-16/ruff-0.6.9.tar.gz: ruff-0.6.9/crates/ruff_linter/src/rules/flake8_bandit/rules/suspicious_function_call.rs: // URLOpen (`URLopener`, `FancyURLopener`)
PYPI-2024-10-16/ruff-0.6.9.tar.gz: ruff-0.6.9/crates/ruff_linter/src/rules/flake8_bandit/rules/suspicious_function_call.rs: ["urllib", "request", "URLopener" | "FancyURLopener"] |
PYPI-2024-10-16/tf_keras-2.17.0.tar.gz: tf_keras-2.17.0/tf_keras/src/utils/data_utils.py: Under Python 2, `urlretrieve` relies on `FancyURLopener` from legacy
PYPI-2024-10-16/tf_keras_nightly-2.19.0.dev2024101609.tar.gz: tf_keras_nightly-2.19.0.dev2024101609/tf_keras/src/utils/data_utils.py: Under Python 2, `urlretrieve` relies on `FancyURLopener` from legacy
PYPI-2024-10-16/wagon-1.0.1.tar.gz: wagon-1.0.1/wagon.py: from urllib.request import URLopener
PYPI-2024-10-16/wagon-1.0.1.tar.gz: wagon-1.0.1/wagon.py: from urllib import URLopener
PYPI-2024-10-16/wagon-1.0.1.tar.gz: wagon-1.0.1/wagon.py: f = URLopener()
PYPI-2024-10-16/wandb-0.18.3.tar.gz: wandb-0.18.3/wandb/vendor/pygments/lexers/_sourcemod_builtins.py: from urllib import FancyURLopener
PYPI-2024-10-16/wandb-0.18.3.tar.gz: wandb-0.18.3/wandb/vendor/pygments/lexers/_sourcemod_builtins.py: from urllib.request import FancyURLopener
PYPI-2024-10-16/wandb-0.18.3.tar.gz: wandb-0.18.3/wandb/vendor/pygments/lexers/_sourcemod_builtins.py: class Opener(FancyURLopener):
PYPI-2024-10-16/wxPython-4.2.2.tar.gz: wxPython-4.2.2/wx/lib/pubsub/py2and3.py: MovedAttribute("URLopener", "urllib", "urllib.request"),
PYPI-2024-10-16/wxPython-4.2.2.tar.gz: wxPython-4.2.2/wx/lib/pubsub/py2and3.py: MovedAttribute("FancyURLopener", "urllib", "urllib.request"),
PYPI-2024-10-16/yt-4.3.1.tar.gz: yt-4.3.1/yt/funcs.py: class MyURLopener(urllib.request.FancyURLopener):
PYPI-2024-10-16/yt-4.3.1.tar.gz: yt-4.3.1/yt/funcs.py: fn, h = MyURLopener().retrieve(url, filename)
cc @hugovk |
Sorry, something went wrong.
Copy link
Copy Markdown
Contributor
Author
|
Thanks Victor. From that list, I think about 6 packages are using URLopener in anger, while the remainder are naming coincidences, linter rules, or vendoring of one of affected packages. |
Sorry, something went wrong.
Copy link
Copy Markdown
Contributor
Author
|
I believe only four packages would break for users if we removed these classes: eventlet, jplephem, wagon, and yt. It looks like eventlet is best fixed with a Python version conditional, so we can't fix that til this patch lands. I've opened issues for the other packages:
I'll work on PRs if the package authors seem interested. |
Sorry, something went wrong.
Copy link
Copy Markdown
Contributor
Author
|
I've submitted PRs against the three projects, one of which has been accepted already. They're all simple changes - in Python 2 terms, we're upgrading from urllib to urllib2. Given that 3.14 is still around a year away, and only 3 projects in 8,000 are affected and possible to remedy at this stage, I submit that we don't need to wait for the other two PRs. Thoughts? |
Sorry, something went wrong.
Copy link
Copy Markdown
Member
Agreed, this is definitely on the low side. |
Sorry, something went wrong.
barneygale
commented
Oct 25, 2024
Comment thread
Lib/test/test_urllib.py
hugovk
approved these changes
Nov 17, 2024
vstinner
reviewed
Nov 18, 2024
Comment thread
Doc/whatsnew/3.14.rst
ebonnal
pushed a commit
to ebonnal/cpython
that referenced
this pull request
Jan 12, 2025
allhailwesttexas
added a commit
to allhailwesttexas/eventlet
that referenced
this pull request
Jun 18, 2025
The class urllib.request.URLopener has been removed in Python 3.14: python/cpython#125739 Because of this, trying to run the tests in `test__greenness.py` fails with the following on Python 3.14: ``` ERROR tests/test__greenness.py - NameError: name 'URLopener' is not defined ``` Update the patching so that we only do it for Python versions lower than 3.14.
allhailwesttexas
added a commit
to allhailwesttexas/eventlet
that referenced
this pull request
Jun 18, 2025
The class urllib.request.URLopener has been removed in Python 3.14: python/cpython#125739 Because of this, trying to run the tests in `test__greenness.py` fails with the following on Python 3.14: ``` ERROR tests/test__greenness.py - NameError: name 'URLopener' is not defined ``` Update the patching so that we only do it for Python versions lower than 3.14.
4383
pushed a commit
to eventlet/eventlet
that referenced
this pull request
Jun 19, 2025
The class urllib.request.URLopener has been removed in Python 3.14: python/cpython#125739 Because of this, trying to run the tests in `test__greenness.py` fails with the following on Python 3.14: ``` ERROR tests/test__greenness.py - NameError: name 'URLopener' is not defined ``` Update the patching so that we only do it for Python versions lower than 3.14. More details: https://docs.python.org/3/library/urllib.request.html#urllib.request.URLopener
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
These classes have been deprecated since Python 3.3. They originate in Python 2's urllib, whereas most of the rest of this module comes from urllib2.
📚 Documentation preview 📚: https://cpython-previews--125739.org.readthedocs.build/