FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

bpo-43244: Remove ast.h, asdl.h, Python-ast.h headers by vstinner · Pull Request #24933 · python/cpython · GitHub

/ cpython Public

bpo-43244: Remove ast.h, asdl.h, Python-ast.h headers - #24933

Merged
vstinner merged 1 commit into
python:masterfrom
vstinner:internal_pycore_ast
Mar 23, 2021
Merged

bpo-43244: Remove ast.h, asdl.h, Python-ast.h headers#24933
vstinner merged 1 commit into
python:masterfrom
vstinner:internal_pycore_ast

Conversation

vstinner commented Mar 19, 2021
edited
Loading

Copy link
Copy Markdown
Member

These fuctions were undocumented and excluded from the limited C API.

Most names defined by these header files were not prefixed by "Py"
and so could create names conflicts. For example, Python-ast.h
defined a "Yield" macro which was conflict with the "Yield" name used
by the Windows <winbase.h> header.

Use the Python ast module instead.

  • Move Include/asdl.h to Include/internal/pycore_asdl.h.
  • Move Include/Python-ast.h to Include/internal/pycore_ast.h.
  • Remove ast.h header file.
  • pycore_symtable.h no longer includes Python-ast.h.

https://bugs.python.org/issue43244

Copy link
Copy Markdown
Member Author

I searched for asdl.h in the top PyPI 4000 projects, I found 3 projects using it: 2 can be ignored, the only interesting one is typed_ast. IMO typed_ast should not block this PR.

typed_ast-1.4.2.tar.gz: this project contains copies of C and header files of CPython.

ast3/Include/Python-ast.h
ast3/Include/asdl.h
ast3/Include/ast.h
ast3/Include/bitset.h
ast3/Include/errcode.h
ast3/Include/graminit.h
ast3/Include/grammar.h
ast3/Include/node.h
ast3/Include/parsetok.h
ast3/Include/pgenheaders.h
ast3/Include/token.h
ast3/Parser/acceler.c
ast3/Parser/bitset.c
ast3/Parser/grammar.c
ast3/Parser/grammar1.c
ast3/Parser/node.c
ast3/Parser/parser.c
ast3/Parser/parser.h
ast3/Parser/parsetok.c
ast3/Parser/tokenizer.c
ast3/Parser/tokenizer.h
ast3/Python/Python-ast.c
ast3/Python/asdl.c
ast3/Python/ast.c
ast3/Python/graminit.c

This project can be modified to copy Python 3.10 internal header files and be built with Py_BUILD_CORE_MODULE macro defined.

Pygments-2.8.0.tar.gz: it's used in example files to test the project, these files are not built, Pygments colorizes text files.

./tests/examplefiles/make/Makefile:		Include/asdl.h \
./tests/examplefiles/make/Makefile.output:'Include/asdl.h' Text

odfpy-1.4.1.tar.gz: the tarball contains many virtual environment in an hidden .tox/ subdirectory.

Copy link
Copy Markdown
Member Author

I searched for ast.h using \bast\.h\b regex and for Python-ast.h in the top PyPI 4000 projects, I found:

  • the 3 same projects than when I search for asdl.h (typed_ast, Pygments, odfpy)
  • jsonnet which contains its own ast.h file: core/ast.h

In short, in the 4000 projects, typed_ast should be the only project impacted by this PR.

typed_ast copies/paste code directly from CPython, so I'm not surprised that it's impacted by changes in the CPython ASDL and AST implementation :-) typed_ast can continue copying code from CPython, but may need to be built with Py_BUILD_CORE_MODULE macro defined.

Copy link
Copy Markdown
Member Author

Ah, test_asyncio failed on the Windows x64 CI, known random test failure: https://bugs.python.org/issue43539

pablogsal requested a review from gvanrossum March 19, 2021 12:59

vstinner left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

pycore_ast.h now contains definitions of functions defined in Python-ast.c and in ast.c. Is that a good thing, or do you prefer to keep two header files separated? I failed to find a better name of the generated header file (old Python-ast.h).

Copy link
Copy Markdown
Member

pycore_ast.h now contains definitions of functions defined in Python-ast.c and in ast.c. Is that a good thing, or do you prefer to keep two header files separated? I failed to find a better name of the generated header file (old Python-ast.h).

Hm, Can we merge pycore_ast_state.h in pycore_ast.h? There just only have a ast_state in pycore_ast_state.h.

Copy link
Copy Markdown
Member Author

Hm, Can we merge pycore_ast_state.h in pycore_ast.h? There just only have a ast_state in pycore_ast_state.h.

Yes, but I would prefer to not do that.

pycore_ast_state.h is included by pycore_interp.h and this header file is included by many C files. I would prefer to avoid including pycore_ast.h in C files since this header file is problematic, for example it requires #undef Yield to avoid conflict with <winbase.h>.

Copy link
Copy Markdown
Member Author

PR rebased on master to fix a conflict on compile.h.

Copy link
Copy Markdown
Member Author

Removed functions are undocumented. They are also excluded from the limited C API (and from the stable ABI).

I updated the PR to mention that explicitly.

I cannot find removed functions in Doc/c-api/:

  • asdl.h: asdl
  • Python-ast.h: PyAST_obj2mod, PyAST_Check, PyAST_mod2obj, _Py_Module, mod_ty
  • ast.h: PyAST_Validate, PyAST_FromNode, PyAST_FromNodeObject, _PyAST_ExprAsUnicode, _PyAST_GetDocString

These functions were undocumented and excluded from the limited C
API.

Most names defined by these header files were not prefixed by "Py"
and so could create names conflicts. For example, Python-ast.h
defined a "Yield" macro which was conflict with the "Yield" name used
by the Windows <winbase.h> header.

Use the Python ast module instead.

* Move Include/asdl.h to Include/internal/pycore_asdl.h.
* Move Include/Python-ast.h to Include/internal/pycore_ast.h.
* Remove ast.h header file.
* pycore_symtable.h no longer includes Python-ast.h.

Copy link
Copy Markdown
Member Author

Oops, I fixed a typo: fuctions => functions.

This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants


Back | FazBrowse Home | New Git URL