FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

[3.10] bpo-44010: IDLE: colorize pattern-matching soft keywords (GH-25851) by miss-islington · Pull Request #26237 · python/cpython · GitHub

  1. python
  2. cpython

Repository navigation

[3.10] bpo-44010: IDLE: colorize pattern-matching soft keywords (GH-25851) #26237

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
miss-islington merged 1 commit into from
May 19, 2021
+345 −73
Conversation 2 Commits 1 Checks 0 Files changed 6
Merged

[3.10] bpo-44010: IDLE: colorize pattern-matching soft keywords (GH-25851) #26237

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension .html  (1) .py  (2) .rst  (3) All 3 file types selected
Viewed files
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Unified
Split
Hide whitespace
Diff view
Unified
Split
Hide whitespace
6 changes: 6 additions & 0 deletions Doc/library/idle.rst
Show comments
View file
Open in desktop
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Original file line number Diff line number Diff line change
Expand Up @@ -613,6 +613,12 @@ keywords, builtin class and function names, names following ``class`` and
``def``, strings, and comments. For any text window, these are the cursor (when
present), found text (when possible), and selected text.

IDLE also highlights the :ref:`soft keywords <soft-keywords>` :keyword:`match`,
:keyword:`case <match>`, and :keyword:`_ <wildcard-patterns>` in
pattern-matching statements. However, this highlighting is not perfect and
will be incorrect in some rare cases, including some ``_``-s in ``case``
patterns.

Text coloring is done in the background, so uncolorized text is occasionally
visible. To change the color scheme, use the Configure IDLE dialog
Highlighting tab. The marking of debugger breakpoint lines in the editor and
Expand Down
6 changes: 6 additions & 0 deletions Doc/whatsnew/3.10.rst
Show comments
View file
Open in desktop
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Original file line number Diff line number Diff line change
Expand Up @@ -1030,6 +1030,12 @@ Terry Jan Reedy in :issue:`37892`.)
We expect to backport these shell changes to a future 3.9 maintenance
release.

Highlight the new :ref:`soft keywords <soft-keywords>` :keyword:`match`,
:keyword:`case <match>`, and :keyword:`_ <wildcard-patterns>` in
pattern-matching statements. However, this highlighting is not perfect
and will be incorrect in some rare cases, including some ``_``-s in
``case`` patterns. (Contributed by Tal Einat in bpo-44010.)

importlib.metadata
------------------

Expand Down
135 changes: 90 additions & 45 deletions Lib/idlelib/colorizer.py
Show comments
View file
Open in desktop
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,32 @@ def any(name, alternates):

def make_pat():
kw = r"\b" + any("KEYWORD", keyword.kwlist) + r"\b"
match_softkw = (
r"^[ \t]*" + # at beginning of line + possible indentation
r"(?P<MATCH_SOFTKW>match)\b" +
r"(?![ \t]*(?:" + "|".join([ # not followed by ...
r"[:,;=^&|@~)\]}]", # a character which means it can't be a
# pattern-matching statement
r"\b(?:" + r"|".join(keyword.kwlist) + r")\b", # a keyword
]) +
r"))"
)
case_default = (
r"^[ \t]*" + # at beginning of line + possible indentation
r"(?P<CASE_SOFTKW>case)" +
r"[ \t]+(?P<CASE_DEFAULT_UNDERSCORE>_\b)"
)
case_softkw_and_pattern = (
r"^[ \t]*" + # at beginning of line + possible indentation
r"(?P<CASE_SOFTKW2>case)\b" +
r"(?![ \t]*(?:" + "|".join([ # not followed by ...
r"_\b", # a lone underscore
r"[:,;=^&|@~)\]}]", # a character which means it can't be a
# pattern-matching case
r"\b(?:" + r"|".join(keyword.kwlist) + r")\b", # a keyword
]) +
r"))"
)
builtinlist = [str(name) for name in dir(builtins)
if not name.startswith('_') and
name not in keyword.kwlist]
Expand All @@ -27,12 +53,29 @@ def make_pat():
sq3string = stringprefix + r"'''[^'\\]*((\\.|'(?!''))[^'\\]*)*(''')?"
dq3string = stringprefix + r'"""[^"\\]*((\\.|"(?!""))[^"\\]*)*(""")?'
string = any("STRING", [sq3string, dq3string, sqstring, dqstring])
return (kw + "|" + builtin + "|" + comment + "|" + string +
"|" + any("SYNC", [r"\n"]))
prog = re.compile("|".join([
builtin, comment, string, kw,
match_softkw, case_default,
case_softkw_and_pattern,
any("SYNC", [r"\n"]),
]),
re.DOTALL | re.MULTILINE)
return prog


prog = re.compile(make_pat(), re.S)
idprog = re.compile(r"\s+(\w+)", re.S)
prog = make_pat()
idprog = re.compile(r"\s+(\w+)")
prog_group_name_to_tag = {
"MATCH_SOFTKW": "KEYWORD",
"CASE_SOFTKW": "KEYWORD",
"CASE_DEFAULT_UNDERSCORE": "KEYWORD",
"CASE_SOFTKW2": "KEYWORD",
}


def matched_named_groups(re_match):
"Get only the non-empty named groups from an re.Match object."
return ((k, v) for (k, v) in re_match.groupdict().items() if v)


def color_config(text):
Expand Down Expand Up @@ -231,14 +274,10 @@ def recolorize(self):
def recolorize_main(self):
"Evaluate text and apply colorizing tags."
next = "1.0"
while True:
item = self.tag_nextrange("TODO", next)
if not item:
break
head, tail = item
self.tag_remove("SYNC", head, tail)
item = self.tag_prevrange("SYNC", head)
head = item[1] if item else "1.0"
while todo_tag_range := self.tag_nextrange("TODO", next):
self.tag_remove("SYNC", todo_tag_range[0], todo_tag_range[1])
sync_tag_range = self.tag_prevrange("SYNC", todo_tag_range[0])
head = sync_tag_range[1] if sync_tag_range else "1.0"

chars = ""
next = head
Expand All @@ -256,23 +295,8 @@ def recolorize_main(self):
return
for tag in self.tagdefs:
self.tag_remove(tag, mark, next)
chars = chars + line
m = self.prog.search(chars)
while m:
for key, value in m.groupdict().items():
if value:
a, b = m.span(key)
self.tag_add(key,
head + "+%dc" % a,
head + "+%dc" % b)
if value in ("def", "class"):
m1 = self.idprog.match(chars, b)
if m1:
a, b = m1.span(1)
self.tag_add("DEFINITION",
head + "+%dc" % a,
head + "+%dc" % b)
m = self.prog.search(chars, m.end())
chars += line
self._add_tags_in_section(chars, head)
if "SYNC" in self.tag_names(next + "-1c"):
head = next
chars = ""
Expand All @@ -291,6 +315,40 @@ def recolorize_main(self):
if DEBUG: print("colorizing stopped")
return

def _add_tag(self, start, end, head, matched_group_name):
"""Add a tag to a given range in the text widget.

This is a utility function, receiving the range as `start` and
`end` positions, each of which is a number of characters
relative to the given `head` index in the text widget.

The tag to add is determined by `matched_group_name`, which is
the name of a regular expression "named group" as matched by
by the relevant highlighting regexps.
"""
tag = prog_group_name_to_tag.get(matched_group_name,
matched_group_name)
self.tag_add(tag,
f"{head}+{start:d}c",
f"{head}+{end:d}c")

def _add_tags_in_section(self, chars, head):
"""Parse and add highlighting tags to a given part of the text.

`chars` is a string with the text to parse and to which
highlighting is to be applied.

`head` is the index in the text widget where the text is found.
"""
for m in self.prog.finditer(chars):
for name, matched_text in matched_named_groups(m):
a, b = m.span(name)
self._add_tag(a, b, head, name)
if matched_text in ("def", "class"):
if m1 := self.idprog.match(chars, b):
a, b = m1.span(1)
self._add_tag(a, b, head, "DEFINITION")

def removecolors(self):
"Remove all colorizing tags."
for tag in self.tagdefs:
Expand All @@ -299,27 +357,14 @@ def removecolors(self):

def _color_delegator(parent): # htest #
from tkinter import Toplevel, Text
from idlelib.idle_test.test_colorizer import source
from idlelib.percolator import Percolator

top = Toplevel(parent)
top.title("Test ColorDelegator")
x, y = map(int, parent.geometry().split('+')[1:])
top.geometry("700x250+%d+%d" % (x + 20, y + 175))
source = (
"if True: int ('1') # keyword, builtin, string, comment\n"
"elif False: print(0)\n"
"else: float(None)\n"
"if iF + If + IF: 'keyword matching must respect case'\n"
"if'': x or'' # valid keyword-string no-space combinations\n"
"async def f(): await g()\n"
"# All valid prefixes for unicode and byte strings should be colored.\n"
"'x', '''x''', \"x\", \"\"\"x\"\"\"\n"
"r'x', u'x', R'x', U'x', f'x', F'x'\n"
"fr'x', Fr'x', fR'x', FR'x', rf'x', rF'x', Rf'x', RF'x'\n"
"b'x',B'x', br'x',Br'x',bR'x',BR'x', rb'x', rB'x',Rb'x',RB'x'\n"
"# Invalid combinations of legal characters should be half colored.\n"
"ur'x', ru'x', uf'x', fu'x', UR'x', ufr'x', rfu'x', xf'x', fx'x'\n"
)
top.geometry("700x550+%d+%d" % (x + 20, y + 175))

text = Text(top, background="white")
text.pack(expand=1, fill="both")
text.insert("insert", source)
Expand Down
27 changes: 19 additions & 8 deletions Lib/idlelib/help.html
Show comments
View file
Open in desktop
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>IDLE &#8212; Python 3.10.0a6 documentation</title>
<title>IDLE &#8212; Python 3.11.0a0 documentation</title>
<link rel="stylesheet" href="../_static/pydoctheme.css" type="text/css" />
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" />

Expand All @@ -18,7 +18,7 @@
<script src="../_static/sidebar.js"></script>

<link rel="search" type="application/opensearchdescription+xml"
title="Search within Python 3.10.0a6 documentation"
title="Search within Python 3.11.0a0 documentation"
href="../_static/opensearch.xml"/>
<link rel="author" title="About these documents" href="../about.html" />
<link rel="index" title="Index" href="../genindex.html" />
Expand Down Expand Up @@ -71,7 +71,7 @@ <h3>Navigation</h3>


<li id="cpython-language-and-version">
<a href="../index.html">3.10.0a6 Documentation</a> &#187;
<a href="../index.html">3.11.0a0 Documentation</a> &#187;
</li>

<li class="nav-item nav-item-1"><a href="index.html" >The Python Standard Library</a> &#187;</li>
Expand Down Expand Up @@ -102,7 +102,7 @@ <h3>Navigation</h3>

<div class="section" id="idle">
<span id="id1"></span><h1>IDLE<a class="headerlink" href="#idle" title="Permalink to this headline">¶</a></h1>
<p><strong>Source code:</strong> <a class="reference external" href="https://github.com/python/cpython/tree/master/Lib/idlelib/">Lib/idlelib/</a></p>
<p><strong>Source code:</strong> <a class="reference external" href="https://github.com/python/cpython/tree/main/Lib/idlelib/">Lib/idlelib/</a></p>
<hr class="docutils" id="index-0" />
<p>IDLE is Python’s Integrated Development and Learning Environment.</p>
<p>IDLE has the following features:</p>
Expand Down Expand Up @@ -581,6 +581,11 @@ <h3>Text colors<a class="headerlink" href="#text-colors" title="Permalink to thi
keywords, builtin class and function names, names following <code class="docutils literal notranslate"><span class="pre">class</span></code> and
<code class="docutils literal notranslate"><span class="pre">def</span></code>, strings, and comments. For any text window, these are the cursor (when
present), found text (when possible), and selected text.</p>
<p>IDLE also highlights the <a class="reference internal" href="../reference/lexical_analysis.html#soft-keywords"><span class="std std-ref">soft keywords</span></a> <a class="reference internal" href="../reference/compound_stmts.html#match"><code class="xref std std-keyword docutils literal notranslate"><span class="pre">match</span></code></a>,
<a class="reference internal" href="../reference/compound_stmts.html#match"><code class="xref std std-keyword docutils literal notranslate"><span class="pre">case</span></code></a>, and <a class="reference internal" href="../reference/compound_stmts.html#wildcard-patterns"><code class="xref std std-keyword docutils literal notranslate"><span class="pre">_</span></code></a> in
pattern-matching statements. However, this highlighting is not perfect and
will be incorrect in some rare cases, including some <code class="docutils literal notranslate"><span class="pre">_</span></code>-s in <code class="docutils literal notranslate"><span class="pre">case</span></code>
patterns.</p>
<p>Text coloring is done in the background, so uncolorized text is occasionally
visible. To change the color scheme, use the Configure IDLE dialog
Highlighting tab. The marking of debugger breakpoint lines in the editor and
Expand Down Expand Up @@ -685,7 +690,7 @@ <h3>Running user code<a class="headerlink" href="#running-user-code" title="Perm
directly with Python in a text-mode system console or terminal window.
However, the different interface and operation occasionally affect
visible results. For instance, <code class="docutils literal notranslate"><span class="pre">sys.modules</span></code> starts with more entries,
and <code class="docutils literal notranslate"><span class="pre">threading.activeCount()</span></code> returns 2 instead of 1.</p>
and <code class="docutils literal notranslate"><span class="pre">threading.active_count()</span></code> returns 2 instead of 1.</p>
<p>By default, IDLE runs user code in a separate OS process rather than in
the user interface process that runs the shell and editor. In the execution
process, it replaces <code class="docutils literal notranslate"><span class="pre">sys.stdin</span></code>, <code class="docutils literal notranslate"><span class="pre">sys.stdout</span></code>, and <code class="docutils literal notranslate"><span class="pre">sys.stderr</span></code>
Expand Down Expand Up @@ -939,7 +944,7 @@ <h3>This Page</h3>
<ul class="this-page-menu">
<li><a href="../bugs.html">Report a Bug</a></li>
<li>
<a href="https://github.com/python/cpython/blob/master/Doc/library/idle.rst"
<a href="https://github.com/python/cpython/blob/main/Doc/library/idle.rst"
rel="nofollow">Show Source
</a>
</li>
Expand Down Expand Up @@ -971,7 +976,7 @@ <h3>Navigation</h3>


<li id="cpython-language-and-version">
<a href="../index.html">3.10.0a6 Documentation</a> &#187;
<a href="../index.html">3.11.0a0 Documentation</a> &#187;
</li>

<li class="nav-item nav-item-1"><a href="index.html" >The Python Standard Library</a> &#187;</li>
Expand All @@ -997,13 +1002,19 @@ <h3>Navigation</h3>
<div class="footer">
&copy; <a href="../copyright.html">Copyright</a> 2001-2021, Python Software Foundation.
<br />
This page is licensed under the Python Software Foundation License Version 2.
<br />
Examples, recipes, and other code in the documentation are additionally licensed under the Zero Clause BSD License.
<br />
See <a href="">History and License</a> for more information.
<br /><br />

The Python Software Foundation is a non-profit corporation.
<a href="https://www.python.org/psf/donations/">Please donate.</a>
<br />
<br />

Last updated on Mar 29, 2021.
Last updated on May 11, 2021.
<a href="https://docs.python.org/3/bugs.html">Found a bug</a>?
<br />

Expand Down
Loading
Back | FazBrowse Home | New Git URL