FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

gh-141778: add missing validation in ast.literal_eval() for non-string input by skirpichev · Pull Request #142969 · python/cpython · GitHub

  1. python
  2. cpython

Repository navigation

gh-141778: add missing validation in ast.literal_eval() for non-string input #142969

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
skirpichev wants to merge 12 commits into from
Conversation 5 Commits 12 Checks 52 Files changed
Closed

gh-141778: add missing validation in ast.literal_eval() for non-string input #142969

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
7c90e20
gh-141778: add missing validation in ast.literal_eval() for non-strin…
skirpichev Dec 19, 2025
b7e1f26
Merge branch 'master' into validation-of-Constants/141778
skirpichev May 24, 2026
e3c4a80
revert unrelated change
skirpichev May 24, 2026
53e9620
simplify check for permitted types
skirpichev May 24, 2026
fc46e56
don't misuse ast.Constant() in the inspect module
skirpichev May 24, 2026
45cd1f6
Merge branch 'master' into validation-of-Constants/141778
skirpichev May 24, 2026
fdefe31
w/a for pdb
skirpichev May 24, 2026
18cd785
Merge branch 'master' into validation-of-Constants/141778
skirpichev Aug 18, 2026
7c09f9f
simplify Lib/inspect.py
skirpichev Aug 18, 2026
5927385
Support sentinel and ast.Compare in ast.literal_eval()
skirpichev Aug 18, 2026
37df249
mark _thread.interrupt_main signature unsupported
skirpichev Aug 18, 2026
457aac1
revert Compare support
skirpichev Aug 18, 2026
File filter

Filter by extension

Filter by extension .py  (4) .rst  (1) All 2 file types selected
Viewed files
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Unified
Split
Hide whitespace
Diff view
Unified
Split
Hide whitespace
23 changes: 19 additions & 4 deletions Lib/ast.py
Show comments
View file
Open in desktop
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Original file line number Diff line number Diff line change
Expand Up @@ -59,17 +59,25 @@ def literal_eval(node_or_string):
"""
if isinstance(node_or_string, str):
node_or_string = parse(node_or_string.lstrip(" \t"), mode='eval').body
return _convert_literal(node_or_string, True)
elif isinstance(node_or_string, Expression):
node_or_string = node_or_string.body
return _convert_literal(node_or_string)


def _convert_literal(node):
_permitted_literal_types = (str, bytes, int, float, complex,
bool, type(None), type(...))


def _convert_literal(node, omit_validation=False):

JelleZijlstra Dec 22, 2025

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Why do we need this change? It may make life harder for some users.

skirpichev Dec 22, 2025

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Could you be more precise, any example?

We need this, because literal_eval() accepts invalid AST for non-string input. Per documentation:

The string or node provided may only consist of the following Python literal structures: strings, bytes, numbers, tuples, lists, dicts, sets, booleans, None and Ellipsis.

skirpichev Aug 18, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Based on case from stdlib (_signature_fromstr()), I think that possible cases of using this undocumented feature are rare, if exist at all.

Though, we can properly deprecate old behavior for extra safety.
"""
Used by `literal_eval` to convert an AST node into a value.
"""
if isinstance(node, Constant):
return node.value
if omit_validation:
return node.value
if type(value := node.value) in _permitted_literal_types:
return value
if isinstance(node, Dict) and len(node.keys) == len(node.values):
return dict(zip(
map(_convert_literal, node.keys),
Expand All @@ -83,9 +91,16 @@ def _convert_literal(node):
return set(map(_convert_literal, node.elts))
if (
isinstance(node, Call) and isinstance(node.func, Name)
and node.func.id == 'set' and node.args == node.keywords == []
):
return set()
if node.func.id == 'set' and node.args == node.keywords == []:
return set()
elif (
node.func.id == 'sentinel' and len(node.args) == 1
and node.keywords == []
and isinstance(arg := node.args[0], Constant)
and isinstance(name := arg.value, str)
):
return sentinel(name)
if (
isinstance(node, UnaryOp)
and isinstance(node.op, (UAdd, USub))
Expand Down
6 changes: 4 additions & 2 deletions Lib/inspect.py
Show comments
View file
Open in desktop
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Original file line number Diff line number Diff line change
Expand Up @@ -2215,9 +2215,11 @@ def wrap_value(s):
except NameError:
raise ValueError

if isinstance(value, (str, int, float, bytes, bool, type(None),
sentinel)):
if isinstance(value, (str, int, float, bytes, bool, type(None))):
return ast.Constant(value)
elif isinstance(value, sentinel):
return ast.Call(func=ast.Name(id='sentinel'),
args=[ast.Constant(value.__name__)])
raise ValueError

class RewriteSymbolics(ast.NodeTransformer):
Expand Down
7 changes: 7 additions & 0 deletions Lib/test/test_ast/test_ast.py
Show comments
View file
Open in desktop
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Original file line number Diff line number Diff line change
Expand Up @@ -2032,6 +2032,9 @@ def test_literal_eval(self):
self.assertEqual(ast.literal_eval('{1, 2, 3}'), {1, 2, 3})
self.assertEqual(ast.literal_eval('b"hi"'), b"hi")
self.assertEqual(ast.literal_eval('set()'), set())
val = ast.literal_eval('sentinel("xyz")')
self.assertTrue(isinstance(val, sentinel))
self.assertEqual(val.__name__, "xyz")
self.assertRaises(ValueError, ast.literal_eval, 'foo()')
self.assertEqual(ast.literal_eval('6'), 6)
self.assertEqual(ast.literal_eval('+6'), 6)
Expand All @@ -2043,6 +2046,10 @@ def test_literal_eval(self):
self.assertRaises(ValueError, ast.literal_eval, '++6')
self.assertRaises(ValueError, ast.literal_eval, '+True')
self.assertRaises(ValueError, ast.literal_eval, '2+3')
# gh-141778: reject values of invalid types
node = ast.Expression(body=ast.Constant(object()))
ast.fix_missing_locations(node)
self.assertRaises(ValueError, ast.literal_eval, node)

def test_literal_eval_str_int_limit(self):
with support.adjust_int_max_str_digits(4000):
Expand Down
4 changes: 3 additions & 1 deletion Lib/test/test_inspect/test_inspect.py
Show comments
View file
Open in desktop
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Original file line number Diff line number Diff line change
Expand Up @@ -6375,7 +6375,9 @@ def test_threading_module_has_signatures(self):
def test_thread_module_has_signatures(self):
import _thread
no_signature = {'RLock'}
self._test_module_has_signatures(_thread, no_signature)
unsupported_signature = {'interrupt_main'}
self._test_module_has_signatures(_thread, no_signature,
unsupported_signature=unsupported_signature)

def test_time_module_has_signatures(self):
no_signature = {
Expand Down
2 changes: 2 additions & 0 deletions Misc/NEWS.d/next/Library/2025-12-19-07-09-02.gh-issue-141778.VdSWcy.rst
Show comments
View file
Open in desktop
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Validate value types of :class:`ast.Constant` nodes in the
:func:`ast.literal_eval`. Patch by Sergey B Kirpichev.
Loading
Back | FazBrowse Home | New Git URL