[3.12] gh-127303: Add docs for token.EXACT_TOKEN_TYPES (GH-127304) (#… · python/cpython@91399c3 · GitHub

python / cpython Public

• Notifications You must be signed in to change notification settings
• Fork 35.3k
• Star 75k

• Code
• Issues 5k+
• Pull requests 2.6k
• Actions
• Projects
• Security and quality
• Insights

Additional navigation options

Commit 91399c3

Browse filesBrowse the repository at this point in the historyBrowse files

authored

[3.12] gh-127303: Add docs for token.EXACT_TOKEN_TYPES (GH-127304) (#127391)

gh-127303: Add docs for token.EXACT_TOKEN_TYPES (GH-127304) --------- (cherry picked from commit dd3a87d) Co-authored-by: Илья Любавский <100635212+lubaskinc0de@users.noreply.github.com> Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu> Co-authored-by: Tomas R. <tomas.roun8@gmail.com> Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>

1 parent 7d175ca commit 91399c3
Copy full SHA for 91399c3

5 files changed

+13-2Lines changed: 13 additions & 2 deletions

File tree

Expand file treeCollapse file tree

Open diff view settings

Filter options

• Doc/library
  • token.rst
• Lib
  • token.py
• Misc
  • ACKS
  • NEWS.d/next/Library
    • 2024-11-27-16-06-10.gh-issue-127303.asqkgh.rst
• Tools/build
  • generate_token.py

Expand file treeCollapse file tree

Open diff view settings

Collapse file

‎Doc/library/token.rst‎

Copy file name to clipboardExpand all lines: Doc/library/token.rst

+7Lines changed: 7 additions & 0 deletions

• Display the source diff
• Display the rich diff

Original file line number	Diff line number	Diff line change
@@ -79,6 +79,13 @@ the :mod:`tokenize` module.
79	79	``type_comments=True``.
80	80
81	81
	82	+ .. data:: EXACT_TOKEN_TYPES
	83	+
	84	+ A dictionary mapping the string representation of a token to its numeric code.
	85	+
	86	+ .. versionadded:: 3.8
	87	+
	88	+
82	89	.. versionchanged:: 3.5
83	90	Added :data:`AWAIT` and :data:`ASYNC` tokens.
84	91

Collapse file

‎Lib/token.py‎

Copy file name to clipboardExpand all lines: Lib/token.py

+2-1Lines changed: 2 additions & 1 deletion

Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

Collapse file

‎Misc/ACKS‎

Copy file name to clipboardExpand all lines: Misc/ACKS

+1Lines changed: 1 addition & 0 deletions

Original file line number	Diff line number	Diff line change
@@ -1139,6 +1139,7 @@ Mark Lutz
1139	1139	Taras Lyapun
1140	1140	Jim Lynch
1141	1141	Mikael Lyngvig
	1142	+ Ilya Lyubavski
1142	1143	Jeff MacDonald
1143	1144	John Machin
1144	1145	Andrew I MacIntyre

Collapse file

‎Misc/NEWS.d/next/Library/2024-11-27-16-06-10.gh-issue-127303.asqkgh.rst‎

Copy file name to clipboard

+1Lines changed: 1 addition & 0 deletions

• Display the source diff
• Display the rich diff

Original file line number	Diff line number	Diff line change
@@ -0,0 +1 @@
	1	+ Publicly expose :data:`~token.EXACT_TOKEN_TYPES` in :attr:`!token.__all__`.

Collapse file

‎Tools/build/generate_token.py‎

Copy file name to clipboardExpand all lines: Tools/build/generate_token.py

+2-1Lines changed: 2 additions & 1 deletion

Original file line number	Diff line number	Diff line change
@@ -226,7 +226,8 @@ def make_rst(infile, outfile='Doc/library/token-list.inc'):
226	226	# {AUTO_GENERATED_BY_SCRIPT}
227	227	'''
228	228	token_py_template += '''
229		- __all__ = ['tok_name', 'ISTERMINAL', 'ISNONTERMINAL', 'ISEOF']
	229	+ __all__ = ['tok_name', 'ISTERMINAL', 'ISNONTERMINAL', 'ISEOF',
	230	+ 'EXACT_TOKEN_TYPES']
230	231
231	232	%s
232	233	N_TOKENS = %d

0 commit comments

Comments

0 (0)