FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

Add upgrade_pythoncapi.py and reorganize tests · python/pythoncapi-compat@f9e807c · GitHub

  1. python
  2. pythoncapi-compat

Repository navigation

Commit f9e807c

Browse files
vstinner
committed
Add upgrade_pythoncapi.py and reorganize tests
1 parent 12724dd commit f9e807c

10 files changed

Lines changed: 1162 additions & 81 deletions

‎.gitignore‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,4 @@
11
*.swp
22
tests/build/
3+
*.py[cod]
4+
__pycache__

‎README.rst‎

Lines changed: 143 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -6,11 +6,17 @@ Python C API compatibility
66
:alt: Build status of pyperf on Travis CI
77
:target: https://travis-ci.com/github/pythoncapi/pythoncapi_compat
88

9-
Header file providing new functions of the Python C API to Python 3.6.
9+
The Python C API compatibility project is made of two parts:
1010

11-
Python 3.6 to Python 3.10 are supported. It requires a subset of C99 like
12-
``static inline`` functions:
13-
see `PEP 7 <https://www.python.org/dev/peps/pep-0007/>`_.
11+
* ``pythoncapi_compat.h``: Header file providing new functions of the Python C
12+
API to Python 3.6.
13+
* ``upgrade_pythoncapi.py``: Script upgrading C extension modules to newer
14+
Python API without losing support for Python 3.6. It relies on
15+
``pythoncapi_compat.h``.
16+
17+
Python 3.6 to Python 3.10 are supported. A subset of C99 is required, like
18+
``static inline`` functions: see `PEP 7
19+
<https://www.python.org/dev/peps/pep-0007/>`_.
1420

1521
Homepage:
1622
https://github.com/pythoncapi/pythoncapi_compat
@@ -27,13 +33,114 @@ This project is covered by the `PSF Code of Conduct
2733
Usage
2834
=====
2935

30-
Copy the header file in your project and include it using::
36+
Run upgrade_pythoncapi.py
37+
-------------------------
3138

32-
#include "pythoncapi_compat.h"
39+
Upgrade ``mod.c`` file::
3340

41+
python3 upgrade_pythoncapi.py mod.c
3442

35-
Functions
36-
=========
43+
Upgrade all ``.c`` and ``.h`` files of a project::
44+
45+
python3 upgrade_pythoncapi.py directory/
46+
47+
WARNING: files are modified in-place! If a file is modified, the original file
48+
is saved as ``<filename>.old``.
49+
50+
To see command line options and list available operations, run it with no
51+
arguments::
52+
53+
python3 upgrade_pythoncapi.py
54+
55+
For example, to only replace ``op->ob_type`` with ``Py_TYPE(op)``, use::
56+
57+
python3 upgrade_pythoncapi.py -o Py_TYPE mod.c
58+
59+
Or the opposite, to apply all operations but leave ``op->ob_type`` unchanged,
60+
use::
61+
62+
python3 upgrade_pythoncapi.py -o all,-Py_TYPE mod.c
63+
64+
Copy pythoncapi_compat.h
65+
------------------------
66+
67+
Most upgrade_pythoncapi.py operations add ``#include "pythoncapi_compat.h"``.
68+
You may have to copy the ``pythoncapi_compat.h`` header file to your project.
69+
It can be copied from::
70+
71+
https://raw.githubusercontent.com/pythoncapi/pythoncapi_compat/master/pythoncapi_compat.h
72+
73+
74+
Upgrade Operations
75+
==================
76+
77+
``upgrade_pythoncapi.py`` implements the following operations:
78+
79+
* ``Py_TYPE``:
80+
81+
* Replace ``op->ob_type`` with ``Py_TYPE(op)``.
82+
83+
* ``Py_SIZE``:
84+
85+
* Replace ``op->ob_size`` with ``Py_SIZE(op)``.
86+
87+
* ``Py_REFCNT``:
88+
89+
* Replace ``op->ob_refcnt`` with ``Py_REFCNT(op)``.
90+
91+
* ``Py_SET_TYPE``:
92+
93+
* Replace ``obj->ob_type = type;`` with ``Py_SET_TYPE(obj, type);``.
94+
* Replace ``Py_TYPE(obj) = type;`` with ``Py_SET_TYPE(obj, type);``.
95+
96+
* ``Py_SET_SIZE``:
97+
98+
* Replace ``obj->ob_size = size;`` with ``Py_SET_SIZE(obj, size);``.
99+
* Replace ``Py_SIZE(obj) = size;`` with ``Py_SET_SIZE(obj, size);``.
100+
101+
* ``Py_SET_REFCNT``:
102+
103+
* Replace ``obj->ob_refcnt = refcnt;`` with ``Py_SET_REFCNT(obj, refcnt);``.
104+
* Replace ``Py_REFCNT(obj) = refcnt;`` with ``Py_SET_REFCNT(obj, refcnt);``.
105+
106+
* ``PyObject_NEW``:
107+
108+
* Replace ``PyObject_NEW(...)`` with ``PyObject_New(...)``.
109+
* Replace ``PyObject_NEW_VAR(...)`` with ``PyObject_NewVar(...)``.
110+
111+
* ``PyMem_MALLOC``:
112+
113+
* Replace ``PyMem_MALLOC(n)`` with ``PyMem_Malloc(n)``.
114+
* Replace ``PyMem_REALLOC(ptr, n)`` with ``PyMem_Realloc(ptr, n)``.
115+
* Replace ``PyMem_FREE(ptr)``, ``PyMem_DEL(ptr)`` and ``PyMem_Del(ptr)`` .
116+
with ``PyMem_Free(n)``.
117+
118+
* ``PyObject_MALLOC``:
119+
120+
* Replace ``PyObject_MALLOC(n)`` with ``PyObject_Malloc(n)``.
121+
* Replace ``PyObject_REALLOC(ptr, n)`` with ``PyObject_Realloc(ptr, n)``.
122+
* Replace ``PyObject_FREE(ptr)``, ``PyObject_DEL(ptr)``
123+
and ``PyObject_Del(ptr)`` . with ``PyObject_Free(n)``.
124+
125+
* ``PyFrame_GetBack``:
126+
127+
* Replace ``frame->f_back`` with ``_PyFrame_GetBackBorrow(frame)``.
128+
129+
* ``PyFrame_GetCode``:
130+
131+
* Replace ``frame->f_code`` with ``_PyFrame_GetCodeBorrow(frame)``.
132+
133+
* ``PyThreadState_GetInterpreter``:
134+
135+
* Replace ``tstate->interp`` with ``PyThreadState_GetInterpreter(tstate)``.
136+
137+
* ``PyThreadState_GetFrame``:
138+
139+
* Replace ``tstate->frame`` with ``_PyThreadState_GetFrameBorrow(tstate)``.
140+
141+
142+
pythoncapi_compat.h functions
143+
=============================
37144

38145
Borrow variant
39146
--------------
@@ -105,16 +212,38 @@ Run tests
105212

106213
Run the command::
107214

108-
python3 tests/test_matrix.py
109-
# add -v option for verbose mode
215+
python3 runtests.py
216+
217+
Verbose mode::
110218

111-
To test one specific Python executable::
219+
python3 runtests.py -v
112220

113-
python3.6 tests/run_tests.py
114-
# add -v option for verbose mode
221+
See tests in the ``tests/`` subdirectory.
222+
223+
224+
Links
225+
=====
115226

227+
* `pythoncapi_compat.h header
228+
<https://github.com/pythoncapi/pythoncapi_compat>`_:
229+
Header file providing new functions of the Python C API for old Python
230+
versions.
231+
* `Py_SET_TYPE() function documentation
232+
<https://docs.python.org/dev/c-api/structures.html#c.Py_SET_TYPE>`_
233+
(Python 3.9)
234+
* `Py_SET_SIZE() function documentation
235+
<https://docs.python.org/dev/c-api/structures.html#c.Py_SET_SIZE>`_
236+
(Python 3.9)
237+
* `Py_SET_REFCNT() function documentation
238+
<https://docs.python.org/dev/c-api/structures.html#c.Py_SET_REFCNT>`_
239+
(Python 3.9)
240+
* `bpo-39573: [C API] Make PyObject an opaque structure in the limited C API
241+
<https://bugs.python.org/issue39573>`_
242+
* `PEP 620 -- Hide implementation details from the C API
243+
<https://www.python.org/dev/peps/pep-0620/>`_
116244

117245
Changelog
118246
=========
119247

120-
* 2020-06-04: Creation of the project.
248+
* 2020-11-30: Creation of the upgrade_pythoncapi.py script.
249+
* 2020-06-04: Creation of the pythoncapi_compat.h header file.

‎runtests.py‎

Lines changed: 77 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,77 @@
1+
#!/usr/bin/python3
2+
"""
3+
Run the test suite on multiple Python versions.
4+
5+
Usage::
6+
7+
python3 test_matrix.py
8+
python3 test_matrix.py -v # verbose mode
9+
"""
10+
import os.path
11+
import subprocess
12+
import sys
13+
import shutil
14+
15+
16+
TEST_DIR = os.path.join(os.path.dirname(__file__), 'tests')
17+
TEST_COMPAT = os.path.join(TEST_DIR, "test_pythoncapi_compat.py")
18+
TEST_UPGRADE = os.path.join(TEST_DIR, "test_upgrade_pythoncapi.py")
19+
20+
PYTHONS = (
21+
"python3.6",
22+
"python3.7",
23+
"python3.8",
24+
"python3.9",
25+
"python3.10",
26+
"python3",
27+
"python3-debug",
28+
)
29+
30+
31+
def run_command(cmd):
32+
proc = subprocess.Popen(cmd)
33+
proc.wait()
34+
exitcode = proc.returncode
35+
if exitcode:
36+
sys.exit(exitcode)
37+
38+
39+
def run_tests_exe(executable, verbose, tested):
40+
executable = os.path.realpath(executable)
41+
if executable in tested:
42+
return
43+
44+
cmd = [executable, TEST_COMPAT]
45+
if verbose:
46+
cmd.append('-v')
47+
run_command(cmd)
48+
tested.add(executable)
49+
50+
51+
def run_tests(python, verbose, tested):
52+
executable = shutil.which(python)
53+
if not executable:
54+
print(f"Ignore missing: {python}")
55+
return
56+
run_tests_exe(executable, verbose, tested)
57+
58+
59+
def main():
60+
verbose = "-v" in sys.argv[1:]
61+
62+
cmd = [sys.executable, TEST_UPGRADE]
63+
if verbose:
64+
cmd.append('-v')
65+
run_command(cmd)
66+
67+
tested = set()
68+
for python in PYTHONS:
69+
run_tests(python, verbose, tested)
70+
run_tests_exe(sys.executable, verbose, tested)
71+
72+
print()
73+
print(f"Tested: {len(tested)} Python executables")
74+
75+
76+
if __name__ == "__main__":
77+
main()

‎runtests.sh‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
set -e -x
2+
python3 tests/test_upgrade_pythoncapi.py -v
3+
python3 tests/test_matrix.py

‎tests/setup.py‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,8 +3,8 @@
33
def main():
44
from distutils.core import setup, Extension
55

6-
ext = Extension('test_pythoncapi_compat',
7-
sources=['test_pythoncapi_compat.c'])
6+
ext = Extension('test_pythoncapi_compat_cext',
7+
sources=['test_pythoncapi_compat_cext.c'])
88

99
setup(name="test_pythoncapi_compat", ext_modules=[ext])
1010

‎tests/test_matrix.py‎

Lines changed: 0 additions & 60 deletions
This file was deleted.

tests/run_tests.py renamed to tests/test_pythoncapi_compat.py

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -54,14 +54,16 @@ def build_ext():
5454

5555

5656
def import_tests():
57-
pythonpath = ''
57+
pythonpath = None
5858
for name in os.listdir("build"):
5959
if name.startswith('lib.'):
6060
pythonpath = os.path.join("build", name)
6161

62+
if not pythonpath:
63+
raise Exception("Failed to find the build directory")
6264
sys.path.append(pythonpath)
63-
import test_pythoncapi_compat
64-
return test_pythoncapi_compat
65+
import test_pythoncapi_compat_cext
66+
return test_pythoncapi_compat_cext
6567

6668

6769
def _run_tests(tests, verbose):

tests/test_pythoncapi_compat.c renamed to tests/test_pythoncapi_compat_cext.c

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -231,13 +231,13 @@ static struct PyMethodDef methods[] = {
231231

232232
static struct PyModuleDef module = {
233233
PyModuleDef_HEAD_INIT,
234-
.m_name = "test_pythoncapi_compat",
234+
.m_name = "test_pythoncapi_compat_cext",
235235
.m_methods = methods,
236236
};
237237

238238

239239
PyMODINIT_FUNC
240-
PyInit_test_pythoncapi_compat(void)
240+
PyInit_test_pythoncapi_compat_cext(void)
241241
{
242242
return PyModuleDef_Init(&module);
243243
}

0 commit comments

Comments
 (0)
Back | FazBrowse Home | New Git URL