FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

gh-127303: Add docs for token.EXACT_TOKEN_TYPES by lubaskinc0de · Pull Request #127304 · python/cpython · GitHub

/ cpython Public
7 changes: 7 additions & 0 deletions Doc/library/token.rst
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,13 @@ the :mod:`tokenize` module.
``type_comments=True``.


.. data:: EXACT_TOKEN_TYPES

A dictionary mapping the string representation of a token to its numeric code.

.. versionadded:: 3.8

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Can we have two blank lines here? (as above)


.. versionchanged:: 3.5
Added :data:`!AWAIT` and :data:`!ASYNC` tokens.

Expand Down
3 changes: 2 additions & 1 deletion Lib/token.py

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions Misc/ACKS
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Original file line number Diff line number Diff line change
Expand Up @@ -1154,6 +1154,7 @@ Mark Lutz
Taras Lyapun
Jim Lynch
Mikael Lyngvig
Ilya Lyubavski
Jeff MacDonald
John Machin
Andrew I MacIntyre
Expand Down
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Publicly expose :data:`~token.EXACT_TOKEN_TYPES` in :attr:`!token.__all__`.
3 changes: 2 additions & 1 deletion Tools/build/generate_token.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Original file line number Diff line number Diff line change
Expand Up @@ -226,7 +226,8 @@ def make_rst(infile, outfile='Doc/library/token-list.inc'):
# {AUTO_GENERATED_BY_SCRIPT}
'''
token_py_template += '''
__all__ = ['tok_name', 'ISTERMINAL', 'ISNONTERMINAL', 'ISEOF']
__all__ = ['tok_name', 'ISTERMINAL', 'ISNONTERMINAL', 'ISEOF',
'EXACT_TOKEN_TYPES']

%s
N_TOKENS = %d
Expand Down

Back | FazBrowse Home | New Git URL