Automatically dedent docstring constants by default · Issue #81283 · python/cpython · GitHub

BPO	37102
Nosy	@gpshead, @methane, @Carreau, @pablogsal, @remilapeyre

Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2019-05-30.18:00:00.547>
labels = ['interpreter-core', 'type-feature', '3.9']
title = 'Automatically dedent docstring constants by default'
updated_at = <Date 2019-06-06.14:36:45.161>
user = 'https://github.com/gpshead'

bugs.python.org fields:

activity = <Date 2019-06-06.14:36:45.161>
actor = 'remi.lapeyre'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Interpreter Core']
creation = <Date 2019-05-30.18:00:00.547>
creator = 'gregory.p.smith'
dependencies = []
files = []
hgrepos = []
issue_num = 37102
keywords = []
message_count = 4.0
messages = ['343990', '344128', '344801', '344819']
nosy_count = 5.0
nosy_names = ['gregory.p.smith', 'methane', 'mbussonn', 'pablogsal', 'remi.lapeyre']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue37102'
versions = ['Python 3.9']

Linked PRs

gh-81283: compiler: remove indent from docstring #106411

I'm spawning this issue of as a separate feature from https://bugs.python.org/issue36906 (adding string dedent method and an optimization to do it at compile timer on constants).

It'd be great if docstrings were given a similar treatment. Right now we carry the whitespace burden within the str constants for __doc__ stored in all code objects in the process. This adds up.

This is not _quite_ the same as calling textwrap.dedent() or the upcoming str.dedent method on them at compile time. We need to special case the first line of docstrings. inspect.getdoc() is our library function that does this for us today.

Chance of breaking things with this change? not impossible, but extremely minor. Something using docstrings as data _and_ depending on leading indentation whitespace preservation would not like this. I am not aware of anything that would ever do that.

Hi, I'm working on a PR. It should be ready in a couple of days. It's more involved than what I thought as to avoid importing inspect during compilation I will probably need to port cleandoc() in C.

How about do inspect.cleandoc() instead of just a dedent?
Some method has docstring like this:

"""First line

blah blah blah blah

    example code here
...
"""

In such docstring, dedent can not strip indent well.

There is existing attempt (in Japanese):

https://qiita.com/hhatto/items/3da6c6820817395f2c39#%E6%84%9A%E7%9B%B4%E3%81%AB%E5%AE%9F%E8%A3%85%E3%81%97%E3%81%9F%E3%82%B3%E3%83%BC%E3%83%89

This is the function I inlined and as far as I can tell, my approach as been similar to the one you linked.

I'm still need to fix some issues as doctest was expecting to find the string before dedenting though.

cleandoc is not idempotent. If we cleandoc on compile time, pydoc and inspect.getdoc() shouldn't cleandoc(doc).

And if user write to obj.__doc__, user need to cleandoc. Otherwise, pydoc will have broken layout.
This is backward incompatibile change.

import inspect

s = """
  first line
    second line
      third line
"""

while True:
    print('---')
    print(s)
    t = inspect.cleandoc(s)
    if t == s:
        break
    s = t

print('---')

output:

---

  first line
    second line
      third line

---
first line
  second line
    third line
---
first line
second line
  third line
---

In the PR, compile time cleandoc doesn't remove leading and trailing newlines.

code:

import inspect

def foo():
    """
    first line
      second
        third
    """

print('---')
print(foo.__doc__)
print('---')
print(inspect.getdoc(foo))
print('---')

current behavior:

---

    first line
      second
        third

---
first line
  second
    third
---

c-cleandoc (#106066)

---

first line
  second
    third

---
first line
  second
    third
---

By this way, I think the PR is minimize incompatibility.
Only one small change to docstring is needed to pass test suite.

https://github.com/python/cpython/pull/106411/files#diff-2752e009dea2759fb0a5a0866446e8bbd3f5ed30c030b26d252d7a1178b413e7R1290

gh-81283: compiler: remove indent from docstring (#106411)

Make UtilsTests.test_filter_params Python 3.13+ compatible

Chance of breaking things with this change? not impossible, but extremely minor. Something using docstrings as data and depending on leading indentation whitespace preservation would not like this. I am not aware of anything that would ever do that.

Tests often do that.

So far I saw:

_________________________________ test_excepts _________________________________

    def test_excepts():
        # These are descriptors, make sure this works correctly.
        assert excepts.__name__ == 'excepts'
>       assert (
            'A wrapper around a function to catch exceptions and\n'
            '    dispatch to a handler.\n'
        ) in excepts.__doc__
E       AssertionError: assert 'A wrapper around a function to catch exceptions and\n    dispatch to a handler.\n' in 'A wrapper around a function to catch exceptions and\ndispatch to a handler.\n\nThis is like a functional try/except block, in the same way that\nifexprs are functional if/else blocks.\n\nExamples\n--------\n>>> excepting = excepts(\n...     ValueError,\n...     lambda a: [1, 2].index(a),\n...     lambda _: -1,\n... )\n>>> excepting(1)\n0\n>>> excepting(3)\n-1\n\nMultiple exceptions and default except clause.\n>>> excepting = excepts((IndexError, KeyError), lambda a: a[0])\n>>> excepting([])\n>>> excepting([1])\n1\n>>> excepting({})\n>>> excepting({0: 1})\n1\n'
E        +  where 'A wrapper around a function to catch exceptions and\ndispatch to a handler.\n\nThis is like a functional try/except block, in the same way that\nifexprs are functional if/else blocks.\n\nExamples\n--------\n>>> excepting = excepts(\n...     ValueError,\n...     lambda a: [1, 2].index(a),\n...     lambda _: -1,\n... )\n>>> excepting(1)\n0\n>>> excepting(3)\n-1\n\nMultiple exceptions and default except clause.\n>>> excepting = excepts((IndexError, KeyError), lambda a: a[0])\n>>> excepting([])\n>>> excepting([1])\n1\n>>> excepting({})\n>>> excepting({0: 1})\n1\n' = excepts.__doc__

toolz/tests/test_functoolz.py:741: AssertionError

(toolz)

And:

________________________ UtilsTests.test_filter_params _________________________

self = <tests.oauth1.rfc5849.test_utils.UtilsTests testMethod=test_filter_params>

    def test_filter_params(self):
    
        # The following is an isolated test function used to test the filter_params decorator.
        @filter_params
        def special_test_function(params, realm=None):
            """ I am a special test function """
            return 'OAuth ' + ','.join(['='.join([k, v]) for k, v in params])
    
        # check that the docstring got through
>       self.assertEqual(special_test_function.__doc__, " I am a special test function ")
E       AssertionError: 'I am a special test function ' != ' I am a special test function '
E       - I am a special test function 
E       +  I am a special test function 
E       ? +

tests/oauth1/rfc5849/test_utils.py:60: AssertionError

(oauthlib)

Fix detection of attributes from ccData docstring

Dedent help string in breeze configuration (#43555)

binman: Do not dedent docstrings twice on Python 3.13+

Dedent help string in breeze configuration (#43555)

binman: Do not dedent docstrings twice on Python 3.13+

Automatically dedent docstring constants by default · Issue #81283 · python/cpython · GitHub

Repository navigation

Automatically dedent docstring constants by default #81283

Linked PRs

42 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Automatically dedent docstring constants by default · Issue #81283 · python/cpython · GitHub

Repository navigation

Automatically dedent docstring constants by default #81283

Description

Linked PRs

Activity

gpshead commented on May 30, 2019

remilapeyre commented on May 31, 2019

methane commented on Jun 6, 2019

remilapeyre commented on Jun 6, 2019

methane commented on Jul 4, 2023

methane commented on Jul 4, 2023

hroncok commented on Oct 24, 2023

42 remaining items

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions