gh-90095: Make .pdbrc work properly and add some reasonable tests by gaogaotiantian · Pull Request #110496 · python/cpython · GitHub

pdb -c and .pdbrc works in a very weird way now. cmd.Cmd has cmdqueue which is perfect for loading user defined commands yet we decided to make our own path to hack it. This makes the special commands like commands or ;; fail because they rely on some cmd.Cmd mechanics.

This PR utilizes cmdqueue (and fixed where it was used wrong) which fixed the problem mentioned in #90095. With this implementation, the commands in .pdbrc and passed in with -c are treated almost exactly the same as the commands the users type in.

Two existing tests related to .pdbrc are removed as ... they were not testing they thought they were.

test_read_pdbrc_with_ascii_encoding simply does not make sense - it basically tests that system with ascii encoding can't print unicode - it has almost nothing to do with pdb. It has nothing to do with enforcing .pdbrc to be utf-8.
test_readrc_kwarg passes only because the commands in .pdbrc does not execute - I don't understand what's the point of the test.

Reasonable tests are added with the PR to test .pdbrc. The testing framework is improved a bit to make writing .pdbrc tests easier.

Issue: Pdb can't use the commands through -c or .pdbrc files #90095

I'd really like to echo that this become included. I myself wanted to take a crack this and am excited to see a PR open for it. Defining commands in .pdbrc would be a great use.

Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com>

Do we consider this as a bug @iritkatriel ? (backport?)

Could backporting break working code?

@iritkatriel thank you reviewing this and getting it merged. @gaogaotiantian thank you for taking up this implementation.

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 Debian root 3.x has failed when building commit 44f9a84.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/345/builds/7401) and take a look at the build logs.
Check if the failure is related to this commit (44f9a84) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/345/builds/7401

Failed tests:

test_tokenize
test_io
test.test_concurrent_futures.test_thread_pool

Failed subtests:

test_normalization - test.test_unicodedata.NormalizationTest.test_normalization
test_choices_algorithms - test.test_random.MersenneTwister_TestBasicOps.test_choices_algorithms
test_bug_9025 - test.test_random.MersenneTwister_TestBasicOps.test_bug_9025
test_framing_many_objects - test.test_pickletools.OptimizedPickleTests.test_framing_many_objects
test_threads - test.test_io.PyBufferedReaderTest.test_threads
test_sample_counts_equivalence - test.test_random.MersenneTwister_TestBasicOps.test_sample_counts_equivalence

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/test_random.py", line 882, in test_choices_algorithms
    c = self.gen.choices(range(n), cum_weights=range(1, n+1), k=10000)
        ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/random.py", line 492, in choices
    return [population[bisect(cum_weights, random() * total, 0, hi)]
                                           ~~~~~~^^
TypeError: 'Random' object is not callable


Traceback (most recent call last):
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/test_io.py", line 1574, in test_threads
    random.shuffle(l)
    ~~~~~~~~~~~~~~^^^
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/random.py", line 360, in shuffle
    j = randbelow(i + 1)
        ~~~~~~~~~^^^^^^^
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/random.py", line 251, in _randbelow_with_getrandbits
    while r >= n:
          ^^^^^^
TypeError: '>=' not supported between instances of 'Random' and 'Random'


Traceback (most recent call last):
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/test_random.py", line 409, in test_bug_9025
    k = sum(randrange(6755399441055744) % 3 == 2 for i in range(n))
        ~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/test_random.py", line 409, in <genexpr>
    k = sum(randrange(6755399441055744) % 3 == 2 for i in range(n))
            ~~~~~~~~~^^^^^^^^^^^^^^^^^^
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/random.py", line 313, in randrange
    raise ValueError("empty range for randrange()")
ValueError: empty range for randrange()


Traceback (most recent call last):
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/test_io.py", line 1574, in test_threads
    random.shuffle(l)
    ~~~~~~~~~~~~~~^^^
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/random.py", line 360, in shuffle
    j = randbelow(i + 1)
        ~~~~~~~~~^^^^^^^
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/random.py", line 252, in _randbelow_with_getrandbits
    r = getrandbits(k)
        ~~~~~~~~~~~^^^
TypeError: 'Random' object is not callable


Traceback (most recent call last):
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/pickletester.py", line 2617, in test_framing_many_objects
    self.check_frame_opcodes(pickled)
    ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/pickletester.py", line 2566, in check_frame_opcodes
    for op, arg, pos in pickletools.genops(pickled):
    ...<27 lines>...
            frame_end = pos + 9 + arg
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/pickletools.py", line 2285, in _genops
    raise ValueError("at position %s, opcode %r unknown" % (
                     "<unknown>" if pos is None else pos,
                     code))
ValueError: at position 141121, opcode b'M' unknown


Traceback (most recent call last):
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/runpy.py", line 198, in _run_module_as_main
    return _run_code(code, main_globals, None,
           ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^
                     "__main__", mod_spec)
                     ^^^^^^^^^^^^^^^^^^^^^
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/runpy.py", line 88, in _run_code
    exec(code, run_globals)
    ~~~~^^^^^^^^^^^^^^^^^^^
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/libregrtest/worker.py", line 6, in <module>
    from test.support import os_helper, Py_DEBUG
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/support/__init__.py", line 15, in <module>
    import textwrap
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/textwrap.py", line 416, in <module>
    _whitespace_only_re = re.compile('^[ \t]+$', re.MULTILINE)
                          ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/re/__init__.py", line 289, in compile
    return _compile(pattern, flags)
           ~~~~~~~~^^^^^^^^^^^^^^^^
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/re/__init__.py", line 350, in _compile
    p = _compiler.compile(pattern, flags)
        ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/re/_compiler.py", line 743, in compile
    p = _parser.parse(p, flags)
        ~~~~~~~~~~~~~^^^^^^^^^^
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/re/_parser.py", line 980, in parse
    p = _parse_sub(source, state, flags & SRE_FLAG_VERBOSE, 0)
        ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/re/_parser.py", line 459, in _parse_sub
    itemsappend(_parse(source, state, verbose, nested + 1,
                ~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                       not nested and not items))
                       ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/re/_parser.py", line 886, in _parse
    op, av = subpattern[i]
    ^^^^^^
ValueError: not enough values to unpack (expected 2, got 0)


Traceback (most recent call last):
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/test_random.py", line 963, in test_sample_counts_equivalence
    s1 = sample(colors, counts=counts, k=k)
         ~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/random.py", line 429, in sample
    selections = self.sample(range(total), k=k)
                 ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/random.py", line 445, in sample
    result[i] = pool[j]
                ~~~~^^^
TypeError: list indices must be integers or slices, not Random


Traceback (most recent call last):
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/test_unicodedata.py", line 393, in test_normalization
    self.run_normalization_tests(testdata)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/test_unicodedata.py", line 426, in run_normalization_tests
    self.assertTrue(c5 ==  NFD(c4) ==  NFD(c5), line)
                                       ~~~^^^^
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/unittest/case.py", line 690, in __call__
    return self.run(*args, **kwds)
           ~~~~~~~~^^^^^^^^^^^^^^^
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/unittest/case.py", line 609, in run
    result.startTest(self)
    ^^^^^^^^^^^^^^^^
AttributeError: 'str' object has no attribute 'startTest'

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot ARM64 MacOS M1 Refleaks NoGIL 3.x has failed when building commit 44f9a84.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/1368/builds/460) and take a look at the build logs.
Check if the failure is related to this commit (44f9a84) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/1368/builds/460

Failed tests:

test.test_asyncio.test_server

Failed subtests:

test_abort_clients - test.test_asyncio.test_server.TestServer2.test_abort_clients

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/Users/ec2-user/buildbot/buildarea/3.x.itamaro-macos-arm64-aws.macos-with-brew.refleak.nogil/build/Lib/unittest/async_case.py", line 93, in _callTestMethod
    if self._callMaybeAsync(method) is not None:
       ~~~~~~~~~~~~~~~~~~~~^^^^^^^^
  File "/Users/ec2-user/buildbot/buildarea/3.x.itamaro-macos-arm64-aws.macos-with-brew.refleak.nogil/build/Lib/unittest/async_case.py", line 115, in _callMaybeAsync
    return self._asyncioRunner.run(
           ~~~~~~~~~~~~~~~~~~~~~~~^
        func(*args, **kwargs),
        ^^^^^^^^^^^^^^^^^^^^^^
        context=self._asyncioTestContext,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/Users/ec2-user/buildbot/buildarea/3.x.itamaro-macos-arm64-aws.macos-with-brew.refleak.nogil/build/Lib/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/Users/ec2-user/buildbot/buildarea/3.x.itamaro-macos-arm64-aws.macos-with-brew.refleak.nogil/build/Lib/asyncio/base_events.py", line 721, in run_until_complete
    return future.result()
           ~~~~~~~~~~~~~^^
  File "/Users/ec2-user/buildbot/buildarea/3.x.itamaro-macos-arm64-aws.macos-with-brew.refleak.nogil/build/Lib/test/test_asyncio/test_server.py", line 231, in test_abort_clients
    s_sock = s_wr.get_extra_info('socket')
             ^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'get_extra_info'

Could backporting break working code?

I don't think so. This change only affects .pdbrc and -c. The changes it made will not break any reasonable code. I can't think of any case that can be broken even if it works in a weird way.

Thanks @gaogaotiantian for the PR, and @iritkatriel for merging it 🌮🎉.. I'm working now to backport this PR to: 3.11.
🐍🍒⛏🤖 I'm not a witch! I'm not a witch!

Thanks @gaogaotiantian for the PR, and @iritkatriel for merging it 🌮🎉.. I'm working now to backport this PR to: 3.12.
🐍🍒⛏🤖

Sorry, @gaogaotiantian and @iritkatriel, I could not cleanly backport this to 3.11 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 44f9a84b67c97c94f0d581ffd63b24b73fb79610 3.11

Sorry, @gaogaotiantian and @iritkatriel, I could not cleanly backport this to 3.12 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 44f9a84b67c97c94f0d581ffd63b24b73fb79610 3.12

GH-116660 is a backport of this pull request to the 3.11 branch.

…ble tests (pythonGH-110496) (cherry picked from commit 44f9a84) Co-authored-by: Tian Gao <gaogaotiantian@hotmail.com>

GH-116661 is a backport of this pull request to the 3.12 branch.

#116661) [3.12] gh-90095: Make .pdbrc work properly and add some reasonable tests (GH-110496) (cherry picked from commit 44f9a84)

#116660) [3.11] gh-90095: Make .pdbrc work properly and add some reasonable tests (GH-110496) (cherry picked from commit 44f9a84)

…ts (python#110496)

gaogaotiantian added 2 commits October 6, 2023 22:19

Make .pdbrc work properly and add some reasonable tests

fdd36d2

Add comment test

3f3cd7d

bedevere-app Bot added the awaiting review label Oct 7, 2023

bedevere-app Bot mentioned this pull request Oct 7, 2023

Pdb can't use the commands through -c or .pdbrc files #90095

Closed

gaogaotiantian requested review from iritkatriel and methane October 7, 2023 05:37

gaogaotiantian and others added 3 commits October 6, 2023 23:14

Fix homesave issue

33225ee

📜🤖 Added by blurb_it.

2e1bc86

Merge branch 'main' into pdb-rc-commands

78f953a

brandtbucher self-assigned this Oct 26, 2023

brandtbucher added type-bug An unexpected behavior, bug, or error stdlib Standard Library Python modules in the Lib/ directory labels Oct 26, 2023

gaogaotiantian added 3 commits February 8, 2024 13:43

Merge branch 'main' into pdb-rc-commands

6761844

Revert emplty line change

e66eb76

Fix a bare return

f07e770

iritkatriel reviewed Mar 10, 2024

View reviewed changes

Comment thread Lib/test/test_pdb.py

Comment thread Lib/test/test_pdb.py

Comment thread Lib/pdb.py

gaogaotiantian and others added 2 commits March 10, 2024 17:34

Clean up logic for remove_home

0b1c0e8

Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com>

Check homesave to restore HOME

75d1361

iritkatriel approved these changes Mar 11, 2024

View reviewed changes

bedevere-app Bot added awaiting merge and removed awaiting review labels Mar 11, 2024

Merge branch 'main' into pdb-rc-commands

a5187a5

iritkatriel merged commit 44f9a84 into python:main Mar 11, 2024

bedevere-app Bot removed the awaiting merge label Mar 11, 2024

gaogaotiantian deleted the pdb-rc-commands branch March 11, 2024 21:56

iritkatriel added needs backport to 3.11 only security fixes needs backport to 3.12 only security fixes labels Mar 12, 2024

miss-islington-app Bot assigned iritkatriel and unassigned brandtbucher Mar 12, 2024

gaogaotiantian mentioned this pull request Mar 12, 2024

[3.11] gh-90095: Make .pdbrc work properly and add some reasonable te… #116660

Merged

bedevere-app Bot removed the needs backport to 3.11 only security fixes label Mar 12, 2024

gaogaotiantian mentioned this pull request Mar 12, 2024

[3.12] gh-90095: Make .pdbrc work properly and add some reasonable te… #116661

Merged

bedevere-app Bot removed the needs backport to 3.12 only security fixes label Mar 12, 2024

iritkatriel pushed a commit that referenced this pull request Mar 12, 2024

[3.12] gh-90095 : Make .pdbrc work properly and add some reasonable te… (

d19c2eb

#116661) [3.12] gh-90095: Make .pdbrc work properly and add some reasonable tests (GH-110496) (cherry picked from commit 44f9a84)

iritkatriel pushed a commit that referenced this pull request Mar 12, 2024

[3.11] gh-90095 : Make .pdbrc work properly and add some reasonable te… (

b17d347

#116660) [3.11] gh-90095: Make .pdbrc work properly and add some reasonable tests (GH-110496) (cherry picked from commit 44f9a84)

adorilson pushed a commit to adorilson/cpython that referenced this pull request Mar 25, 2024

pythongh-90095 : Make .pdbrc work properly and add some reasonable tes…

3582718

…ts (python#110496)

diegorusso pushed a commit to diegorusso/cpython that referenced this pull request Apr 17, 2024

pythongh-90095 : Make .pdbrc work properly and add some reasonable tes…

10b2399

…ts (python#110496)

gh-90095: Make .pdbrc work properly and add some reasonable tests by gaogaotiantian · Pull Request #110496 · python/cpython · GitHub

Uh oh!

Uh oh!

gh-90095: Make .pdbrc work properly and add some reasonable tests - #110496

gh-90095: Make .pdbrc work properly and add some reasonable tests#110496iritkatriel merged 11 commits intopython:mainpython/cpython:mainfrom gaogaotiantian:pdb-rc-commandsgaogaotiantian/cpython:pdb-rc-commandsCopy head branch name to clipboard

Conversation

gaogaotiantian commented Oct 7, 2023 • edited by bedevere-app Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Delengowski commented Mar 9, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

gaogaotiantian commented Mar 11, 2024

Uh oh!

iritkatriel commented Mar 11, 2024

Uh oh!

Delengowski commented Mar 11, 2024

Uh oh!

bedevere-bot commented Mar 11, 2024

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Uh oh!

bedevere-bot commented Mar 11, 2024

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Uh oh!

gaogaotiantian commented Mar 12, 2024

Uh oh!

miss-islington-app Bot commented Mar 12, 2024

Uh oh!

miss-islington-app Bot commented Mar 12, 2024

Uh oh!

miss-islington-app Bot commented Mar 12, 2024

Uh oh!

miss-islington-app Bot commented Mar 12, 2024

Uh oh!

bedevere-app Bot commented Mar 12, 2024

Uh oh!

bedevere-app Bot commented Mar 12, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

gh-90095: Make .pdbrc work properly and add some reasonable tests#110496
iritkatriel merged 11 commits into
python:mainfrom
gaogaotiantian:pdb-rc-commands

gaogaotiantian commented Oct 7, 2023 •
edited by bedevere-app Bot
Loading