Repository navigation
test_peg_generator.test_c_parser emits DeprecationWarning due to distutils #87591
Closed as not planned
Description
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = None
created_at = <Date 2021-03-07.05:07:09.546>
labels = ['type-bug', 'tests', '3.10']
title = 'test_peg_generator.test_c_parser emits DeprecationWarning due to distutils'
updated_at = <Date 2021-09-15.13:15:47.651>
user = 'https://github.com/tirkarthi'bugs.python.org fields:
activity = <Date 2021-09-15.13:15:47.651>
actor = 'vstinner'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Tests']
creation = <Date 2021-03-07.05:07:09.546>
creator = 'xtreak'
dependencies = []
files = []
hgrepos = []
issue_num = 43425
keywords = ['patch']
message_count = 13.0
messages = ['388222', '388242', '388243', '388572', '388780', '395716', '395768', '396805', '396808', '396837', '396838', '397716', '401831']
nosy_count = 8.0
nosy_names = ['terry.reedy', 'vstinner', 'lukasz.langa', 'steve.dower', 'corona10', 'pablogsal', 'miss-islington', 'xtreak']
pr_nums = ['26693', '26713', '26968', '26969', '26978', '26981', '26986']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue43425'
versions = ['Python 3.10']
Activity
tirkarthi commented on Mar 7, 2021
distutils was deprecated for removal in Python 3.10. It is used in test_peg_generator.test_c_parser which emits a deprecation warning. It also seems to be used in test_support for missing_compiler_executable that will emit a deprecation warning.
./python -Wall -m test test_peg_generator test_c_parser
0:00:00 load avg: 0.02 Run tests sequentially
0:00:00 load avg: 0.02 [1/2] test_peg_generator
/root/cpython/Lib/test/test_peg_generator/test_c_parser.py:4: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP-632 for potential alternatives
from distutils.tests.support import TempdirManager
Other places on grep :
rg 'from distutils' | rg -v 'Lib/distutils|rst'
Modules/_decimal/tests/formathelper.py:from distutils.spawn import find_executable
Doc/includes/setup.py:from distutils.core import setup, Extension
Doc/includes/test.py:from distutils.util import get_platform
setup.py:from distutils import log
setup.py:from distutils.command.build_ext import build_ext
setup.py:from distutils.command.build_scripts import build_scripts
setup.py:from distutils.command.install import install
setup.py:from distutils.command.install_lib import install_lib
setup.py:from distutils.core import Extension, setup
setup.py:from distutils.errors import CCompilerError, DistutilsError
setup.py:from distutils.spawn import find_executable
Lib/_osx_support.py: from distutils import log
Lib/_osx_support.py: Currently called from distutils.sysconfig
Lib/test/support/init.py: from distutils import ccompiler, sysconfig, spawn, errors
Lib/test/test_importlib/test_windows.py:from distutils.util import get_platform
Lib/test/test_peg_generator/test_c_parser.py:from distutils.tests.support import TempdirManager
Tools/peg_generator/pegen/build.py: from distutils.core import Distribution, Extension
Tools/peg_generator/pegen/build.py: from distutils.command.clean import clean # type: ignore
Tools/peg_generator/pegen/build.py: from distutils.command.build_ext import build_ext # type: ignore
Tools/peg_generator/pegen/build.py: from distutils.tests.support import fixup_build_ext # type: ignore
Tools/test2to3/setup.py:from distutils.core import setup
Tools/test2to3/setup.py: from distutils.command.build_py import build_py_2to3 as build_py
Tools/test2to3/setup.py: from distutils.command.build_py import build_py
Tools/test2to3/setup.py: from distutils.command.build_scripts import build_scripts_2to3 as build_scripts
Tools/test2to3/setup.py: from distutils.command.build_scripts import build_scripts
Tools/test2to3/test/runtests.py: from distutils.util import copydir_run_2to3
Misc/HISTORY:- Issue bpo-5394: removed > 2.3 syntax from distutils.msvc9compiler.
pablogsal commented on Mar 7, 2021
We just need to suppress the warning unless I am missing somey. Distutils will still be used internally by CPython.
tirkarthi commented on Mar 7, 2021
Thanks for the clarification. As per bpo-41282 I thought disutils will be removed. The accepted PEP-632 states this scenario so I guess silencing the warning is good enough.
terryjreedy commented on Mar 13, 2021
Steve, PEP-632 lacks a reference to the SC acceptance thereof. Could you add one?
https://mail.python.org/archives/list/python-dev@python.org/thread/TXU6TVOMBLQU3SV57DMMOA5Y2E67AW7P/
And can you verify my interpretation below?
The warning is correct as to removal. PEP-632 says, perhaps a bit tersely, that when the 3.11 branch is established for 3.11.0b1 and master/main becomes the 3.12.0a development branch, disutils/* be no longer be installed as part of the stdlib (ie, in /Lib). At that point, 'import disutils' will cease working in installed CPython, outside of a cpython repository. Although this could be worked around by catching the import error and skipping disutils-dependent tests, the dependency should better be removed, if possible, before 3.12.0a.
The PEP gives no timetable for doing so, and anticipates separate PRs like this.
It further says that when "the CPython build process no longer depends on distutils being in the standard library, the entire Lib/distutils directory and Lib/test/test_distutils.py file will be removed from the repository." Leaving it deprecated but present was rejected.
Part of the extensive discussion at https://discuss.python.org/t/pep-632-deprecate-distutils-module/5134/128 was about possibly putting parts of disutiles under /Tools and replacing 'import disutils' with a reference to <python dir>/Tools/distutiles/whatever. If this is done for tests, the access and use has to be conditional. Our Windows installer only installs 4 or 21 Tools subdirectories thought appropriate for uses and others may include less, and maybe omit Tools altogether.
Karthikeyan posted the opening of this and another issue 5 days ago on bpo-41281, with no comment so far.
zooba commented on Mar 15, 2021
The link appears to be in the Resolution header of PEP-632?
But yes, your analysis is correct. If/when we move distutils elsewhere, we'll probably have to update all of these sections.
Ideally we'd have a helper function to do the initial "import distutils", handle failure (because it's not in Tools), and suppress the warning. But I don't know that there's anywhere we could put it reliably anyway, so probably just need to suppress the warnings directly and we'll handle failures if/when we move it into Tools.
corona10 commented on Jun 12, 2021
New changeset 736ed6f by Dong-hee Na in branch 'main':
bpo-43425: Update test_c_parser not to use TempdirManager (GH-26693)
736ed6f
miss-islington commented on Jun 14, 2021
New changeset 2b57ad3 by Miss Islington (bot) in branch '3.10':
bpo-43425: Update test_c_parser not to use TempdirManager (GH-26693)
2b57ad3
corona10 commented on Jul 1, 2021
New changeset c8979f7 by Dong-hee Na in branch 'main':
bpo-43425: Update _osx_support not to use distutils.log (GH-26968)
c8979f7
miss-islington commented on Jul 1, 2021
New changeset 94a4136 by Miss Islington (bot) in branch '3.10':
bpo-43425: Update _osx_support not to use distutils.log (GH-26968)
94a4136
corona10 commented on Jul 2, 2021
New changeset a7e251b by Dong-hee Na in branch 'main':
bpo-43425: Update setup.py not to use distutils.log (GH-26969)
a7e251b
miss-islington commented on Jul 2, 2021
New changeset 956f1fc by Miss Islington (bot) in branch '3.10':
bpo-43425: Update setup.py not to use distutils.log (GH-26969)
956f1fc
ambv commented on Jul 17, 2021
New changeset d0b2b00 by Dong-hee Na in branch 'main':
bpo-43425: Remove test2to3 from Tools (bpo-26981)
d0b2b00
vstinner commented on Sep 15, 2021
test_peg_generator still emits DeprecationWarning warnings. Example:
vstinner@apu$ ./python -m test test_peg_generator -v
== CPython 3.11.0a0 (heads/main:b49263b698, Sep 15 2021, 14:44:10) [GCC 11.2.1 20210728 (Red Hat 11.2.1-1)]
== Linux-5.13.14-200.fc34.x86_64-x86_64-with-glibc2.33 little-endian
== cwd: /home/vstinner/python/main/build/test_python_193641æ
== CPU count: 8
== encodings: locale=UTF-8, FS=utf-8
0:00:00 load avg: 15.94 Run tests sequentially
0:00:00 load avg: 15.94 [1/1] test_peg_generator
test_advanced_left_recursive (test.test_peg_generator.test_c_parser.TestCParser) ...
/home/vstinner/python/main/Lib/test/support/init.py:1713: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP-632 for potential alternatives
from distutils import ccompiler, sysconfig, spawn, errors
/home/vstinner/python/main/Lib/test/support/init.py:1713: DeprecationWarning: The distutils.sysconfig module is deprecated, use sysconfig instead
from distutils import ccompiler, sysconfig, spawn, errors
(...)
iritkatriel commented on Apr 6, 2023
Not anymore - it's skipped since 0faa0ba.