bpo-29636: Add --indent / --no-indent arguments to json.tool by dhimmel · Pull Request #345 · python/cpython · GitHub

From #201 which is being split into two pull requests.

Tagging @serhiy-storchaka, @berkerpeksag, @methane who helped review #201.

Closes http://bugs.python.org/issue29636.

https://bugs.python.org/issue29636

Note to rebase on #346 which should be merged first.

I would prefer a --compact option which use indent=None but also separators without spaces: http://bugs.python.org/issue29540

It's not the same than what you propose.

@Haypo I agree the most compact representation would be useful.

One option would be add a third argument (--compact) to the mutually exclusive group. This would provide access to three options:

--indent: pretty (with newlines)
--no-indent: with indent=None in json.dump
--compact: compact, with separators=(',', ':') in json.dump

Alternatively, are you proposing to only support options 1 & 3?

I'm happy to implement this. Let's get a second opinion, so I don't go down the wrong track. Tagging @serhiy-storchaka, @berkerpeksag, @methane.

Is --no-indent differen than --indent=0?

Would it be possible to use a default of 4 spaces when using --indent with no argument?

Is --no-indent differen than --indent=0?

Yes, it is. --indent=0 adds new lines, --no-indent writes all in one line.

Would it be possible to use a default of 4 spaces when using --indent with no argument?

This is the default behavior.

@Haypo the following illustrates the three options

import json
obj = [1, 2]

print('--indent=4')
print(json.dumps(obj, indent=4))

print('--no-indent')
print(json.dumps(obj, indent=None))

print('--compact')
print(json.dumps(obj, separators=(',', ':')))

which produces the following output

--indent=4
[
    1,
    2
]
--no-indent
[1, 2]
--compact
[1,2]

Spaces after comma and semicolon are not just for readability. They make JSON compatible with YAML 1.0.

@serhiy-storchaka great point. I think this is a strong argument for not replacing --no-indent with --compact. However, should we add --compact as a third option? I can see the utility for users who wan't the smallest file sizes.

Spaces after comma and semicolon are not just for readability. They make JSON compatible with YAML 1.0.

Yes, it is. --indent=0 adds new lines, --no-indent writes all in one line.

"write all in one line": it sounds like the use case is to produce smaller file, so it seems redundant and less efficient than --compact. I expect that --compact removes all spaces and newlines to produce the smallest file.

This is not the right place for the design discussion. Lets continue it on the bug tracker.

This is not the right place for the design discussion. Lets continue it on the bug tracker.

@serhiy-storchaka and @Haypo I updated bpo-29636 with a summary of the design discussion thus far.

I propose continuing this pull request with only --indent and --no-indent. If we end up reaching consensus to add --compact, that can be added in the future. I think it would fit in nicely alongside --indent and --no-indent. In other words, it's not pressing to decide on --compact right now.

#346 has been merged and this PR rebased.

@serhiy-storchaka, @berkerpeksag, @methane and @Haypo: open for review. No more outstanding changes on my end.

Originall from python#2720 Set default option for infile and outfile as per python#2720 (review) Use --sort-keys help message based on the json.tool docs at https://docs.python.org/3.6/library/json.html#basic-usage: Remove commens as per https://bugs.python.org/msg298692. Code was descriptive without the comments.

From python#201 which is being split into two pull requests. See http://bugs.python.org/issue29636

@dhimmel Please add NEWS entry.

Thanks @methane for ba16891 which merges in the changes from #5315 (dict order guaranteed as of Python 3.7).

I'll get working on the NEWS entry.

Use `blurb add` command to assist with adding the NEWS entry.

Ready from my end.

As of b8fee34, --help prints the following message:

usage: python -m json.tool [-h] [--sort-keys]
                           [--indent INDENT | --tab | --no-indent | --compact]
                           [infile] [outfile]

A simple command line interface for json module to validate and pretty-print
JSON objects.

positional arguments:
  infile           a JSON file to be validated or pretty-printed
  outfile          write the output of infile to outfile

optional arguments:
  -h, --help       show this help message and exit
  --sort-keys      sort the output of dictionaries by key
  --indent INDENT  separate items with newlines and use this number of spaces
                   for indentation
  --tab            separate items with newlines and use tabs for indentation
  --no-indent      separate items with spaces rather than newlines
  --compact        suppress all whitespace separation (most compact)

What if pass conflicting options? --indent=4 and --tab? --indent=4 and --compact?

What is the benefit of setting default to sys.stdin instead of keeping it None?

This way a suggestion from another contributor at #2720 (comment):

it is easier to read when we process all options in one place, and then use the process options.

By adding default=sys.stdin, we remove the following line later on:

infile = options.infile or sys.stdin

Why removed "alphabetically"?

I believe my intention was to make the help message more closely mirror the json.dump documentation:

cpython/Lib/json/__init__.py

Lines 157 to 158 in cfa797c

    
               If *sort_keys* is true (default: ``False``), then the output of 
        
               dictionaries will be sorted by key.

Happy to re-add "alphabetically" if that was helpful.

What if pass conflicting options? --indent=4 and --tab? --indent=4 and --compact?

The intention is to disallow users from passing multiple whitespace options with argparse's add_mutually_exclusive_group. The implementation will, in most cases, detect and raise an error if multiple whitespace arguments are provided. However, there is an argparse issue (bpo-18943) where mutual exclusivity is not enforced when --indent=4 is specified (since 4 is default).

For example, --compact --tab or --indent=3 --compact or --indent=4 --compact --tab will all throw errors of the form:

python -m json.tool: error: argument --tab: not allowed with argument --compact

--indent=4 --compact (or --indent=4 --tab) does not throw an error. In the case that --indent=4 is specified with --compact, --tab, or no-indent, the later will take precedent.

I hope this feature can be merged ASAP, it's really helpful as a cross-platform command line utility.

I hope this feature can be merged ASAP

@mya12321 the most effective way for you to support this PR would be to chime in on the issue. At present, I think there is some hesitancy to add additional arguments, so if you explain your use case that may be helpful. But as far as I understand, this discussion should go on the bug tracker and not on GitHub.

Would you resolve the conflict?

Well done @dhimmel and thanks for your tenacity!

Bumps [gidgethub](https://github.com/brettcannon/gidgethub) from 4.1.0 to 4.1.1. - [Release notes](https://github.com/brettcannon/gidgethub/releases) - [Changelog](https://github.com/brettcannon/gidgethub/blob/master/docs/changelog.rst) - [Commits](gidgethub/gidgethub@v4.1.0...v4.1.1) Signed-off-by: dependabot-preview[bot] <support@dependabot.com> Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>

the-knights-who-say-ni added the CLA signed label Feb 27, 2017

vstinner reviewed Mar 2, 2017

View reviewed changes

vstinner added the type-feature A feature request or enhancement label Mar 2, 2017

serhiy-storchaka mentioned this pull request Mar 3, 2017

Allow setting indent width or character in json.tool #426

Closed

dhimmel mentioned this pull request Mar 10, 2017

bpo-27413: add --no-ensure-ascii argument to json.tool #201

Closed

dhimmel force-pushed the indent branch from 011ae48 to 562f2ef Compare March 16, 2017 12:39

dhimmel force-pushed the indent branch 4 times, most recently from b53a918 to 56a3851 Compare April 13, 2017 18:58

modulitos mentioned this pull request Jun 15, 2017

bpo-30669: add --indent flag to json.tool #2207

Closed

dhimmel mentioned this pull request Jul 15, 2017

bpo-30971: Improve code readability of json.tool #2720

Closed

dhimmel added 4 commits July 20, 2017 11:40

Add --indent / --no-indent arguments to json.tool

3c67514

From python#201 which is being split into two pull requests. See http://bugs.python.org/issue29636

json.tool: --tab identation option

70cc07b

json.tool: --compact identation option

a098b5d

dhimmel force-pushed the indent branch from 56a3851 to a098b5d Compare July 20, 2017 15:51

brettcannon added the awaiting review label Feb 21, 2018

Merge branch 'master' into indent

ba16891

dhimmel added 2 commits February 22, 2018 11:24

Add NEWS entry

3d3477d

Use `blurb add` command to assist with adding the NEWS entry.

Update whitespace option documentation

b8fee34

serhiy-storchaka reviewed Feb 22, 2018

View reviewed changes

wimglenn mentioned this pull request Oct 9, 2018

bpo-29636: improve CLI of json.tool #9765

Closed

Merge master & resolve --json-lines conflicts

292fc1c

methane merged commit 0325794 into python:master Dec 4, 2019

bedevere-bot removed the awaiting review label Dec 4, 2019

jacobneiltaylor pushed a commit to jacobneiltaylor/cpython that referenced this pull request Dec 5, 2019

bpo-29636: Add --(no-)indent arguments to json.tool (pythonGH-345 )

3a6bd64

dhimmel mentioned this pull request Dec 6, 2019

bpo-29636: json.tool: Add document for indentation options. #17482

Merged

shihai1991 pushed a commit to shihai1991/cpython that referenced this pull request Jan 31, 2020

bpo-29636: Add --(no-)indent arguments to json.tool (pythonGH-345 )

4dd5be6

                                  'to validate and pretty-print JSON objects.')
                   parser = argparse.ArgumentParser(prog=prog, description=description)
                   parser.add_argument('infile', nargs='?', type=argparse.FileType(),
+                                      default=sys.stdin,

-                  parser.add_argument('--sort-keys', action='store_true', default=False,
-                                      help='sort the output of dictionaries alphabetically by key')
+                  parser.add_argument('--sort-keys', action='store_true',
+                                      help='sort the output of dictionaries by key')

	If sort_keys is true (default: ``False``), then the output of
	dictionaries will be sorted by key.

bpo-29636: Add --indent / --no-indent arguments to json.tool by dhimmel · Pull Request #345 · python/cpython · GitHub

Uh oh!

Uh oh!

bpo-29636: Add --indent / --no-indent arguments to json.tool - #345

bpo-29636: Add --indent / --no-indent arguments to json.tool#345methane merged 8 commits intopython:masterpython/cpython:masterfrom dhimmel:indentdhimmel/cpython:indentCopy head branch name to clipboard

Conversation

dhimmel commented Feb 27, 2017 • edited by bedevere-bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dhimmel commented Feb 27, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dhimmel Mar 3, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

serhiy-storchaka commented Mar 3, 2017

Uh oh!

vstinner commented Mar 3, 2017 via email

Uh oh!

serhiy-storchaka commented Mar 3, 2017

Uh oh!

dhimmel commented Mar 3, 2017

Uh oh!

dhimmel commented Mar 16, 2017

Uh oh!

methane commented Feb 22, 2018

Uh oh!

dhimmel commented Feb 22, 2018

Uh oh!

dhimmel commented Feb 22, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

serhiy-storchaka commented Feb 22, 2018

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dhimmel commented Feb 22, 2018

Uh oh!

mya12321 commented Apr 8, 2018

Uh oh!

dhimmel commented Apr 8, 2018

Uh oh!

methane commented Dec 1, 2019

Uh oh!

vstinner commented Dec 4, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

bpo-29636: Add --indent / --no-indent arguments to json.tool#345
methane merged 8 commits into
python:masterfrom
dhimmel:indent

dhimmel commented Feb 27, 2017 •
edited by bedevere-bot
Loading

dhimmel Mar 3, 2017 •
edited
Loading

dhimmel commented Feb 22, 2018 •
edited
Loading