bpo-31553: add --json-lines option to json.tool by hongweipeng · Pull Request #10051 · python/cpython · GitHub

https://bugs.python.org/issue31553

The change LGTM, but I'm not 100% sure about the usefulness of the feature :-) See discussion at https://bugs.python.org/issue31553

Instead of loading all lines and only later dump all objects, it may be better to: load on object, dump it, then load the second object, etc.

The behavior difference is that first JSONs are written into outfile if they are valid, but if the last one is invalid. With the current code, it's all-or-nothing. Maybe it's done on purpose?

The other advantage of working on a single object is to reduce the memory footprint when working with large JSON files.

Good idea.

Just make objs an iterator:

objs = (json.loads(line) for line in infile)

or

objs = map(json.loads, infile)

Wow, that's great. Thank you for your guidance.

I suggest to name the option --json-lines or just --lines.

--json-lines named similar to the existing --sort-keys,it will be more suitable.

Needed the documentation and an entry in What's New.

Maybe 'parse every input line as separate JSON object'?

Write whatsnew in 3.8?

Good idea.

Just make objs an iterator:

objs = (json.loads(line) for line in infile)

or

objs = map(json.loads, infile)

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

Add an empty line after "---". And the number of minuses should match the length of the header.

I have limited knowledge about Doc.Thank you for your patience.

Add "(Contributed by ... in :issue:`...`.)"

I have made the requested changes; please review again.

Thanks for making the requested changes!

@serhiy-storchaka: please review the changes made to this pull request.

You can add your full name if you prefer.

@matrixise It's my full name :)

ping

@serhiy-storchaka: Please replace # with GH- in the commit message next time. Thanks!

Damn it! I have pressed Enter by mistake while edited commit message.

Thank you for your contribution @hongweipeng!

@serhiy-storchaka Oh!It is a beautiful mistake for me. Will you revoke this merger?

It can't be revoked without making larger harm.

add jsonlines option to json.tool

f593057

the-knights-who-say-ni added the CLA signed label Oct 23, 2018

bedevere-bot added the awaiting review label Oct 23, 2018

vstinner reviewed Oct 24, 2018

View reviewed changes

serhiy-storchaka reviewed Oct 25, 2018

View reviewed changes

serhiy-storchaka requested changes Oct 25, 2018

View reviewed changes

bedevere-bot removed the awaiting review label Oct 25, 2018

bedevere-bot added the awaiting changes label Oct 25, 2018

hongweipeng added 2 commits October 25, 2018 17:13

code review

8d31c4f

fix:avoid read infile after it close

546b095

serhiy-storchaka reviewed Oct 26, 2018

View reviewed changes

improve doc in whatsnew 3.8

95933e2

bedevere-bot added awaiting change review and removed awaiting changes labels Oct 26, 2018

hongweipeng changed the title bpo-31553: add jsonlines option to json.tool bpo-31553: add --json-lines option to json.tool Oct 26, 2018

matrixise reviewed Oct 26, 2018

View reviewed changes

serhiy-storchaka approved these changes Nov 7, 2018

View reviewed changes

bedevere-bot added awaiting merge and removed awaiting change review labels Nov 7, 2018

serhiy-storchaka merged commit f194479 into python:master Nov 7, 2018

bedevere-bot removed the awaiting merge label Nov 7, 2018

dhimmel mentioned this pull request Dec 6, 2019

bpo-29636: json.tool: Add document for indentation options. #17482

Merged

hongweipeng deleted the jsonlines-for-json.tool branch March 16, 2020 03:13

                       try:
-                          obj = json.load(infile)
+                          if jsonlines:
+                              objs = [json.loads(line) for line in infile]

                                       help='write the output of infile to outfile')
                   parser.add_argument('--sort-keys', action='store_true', default=False,
                                       help='sort the output of dictionaries alphabetically by key')
+                  parser.add_argument('--jsonlines', action='store_true', default=False,

                   parser.add_argument('--sort-keys', action='store_true', default=False,
                                       help='sort the output of dictionaries alphabetically by key')
+                  parser.add_argument('--jsonlines', action='store_true', default=False,
+                                      help='parse input using the jsonlines format')

                       try:
-                          obj = json.load(infile)
+                          if jsonlines:
+                              objs = [json.loads(line) for line in infile]

+              json.tool
+              -------
+              Add option ``--json-lines`` to parse every input line as separate JSON object.

bpo-31553: add --json-lines option to json.tool by hongweipeng · Pull Request #10051 · python/cpython · GitHub

Repository navigation

bpo-31553: add --json-lines option to json.tool - #10051

bpo-31553: add --json-lines option to json.tool#10051serhiy-storchaka merged 4 commits intopython:masterpython/cpython:masterfrom hongweipeng:jsonlines-for-json.toolhongweipeng/cpython:jsonlines-for-json.toolCopy head branch name to clipboard

Conversation

hongweipeng commented Oct 23, 2018 • edited by bedevere-bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vstinner left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

serhiy-storchaka left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bedevere-bot commented Oct 25, 2018

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

hongweipeng commented Oct 26, 2018

Uh oh!

bedevere-bot commented Oct 26, 2018

Uh oh!

matrixise left a comment

Choose a reason for hiding this comment

Uh oh!

hongweipeng commented Oct 26, 2018

Uh oh!

hongweipeng commented Nov 5, 2018

Uh oh!

bedevere-bot commented Nov 7, 2018

Uh oh!

serhiy-storchaka commented Nov 7, 2018

Uh oh!

hongweipeng commented Nov 7, 2018

Uh oh!

serhiy-storchaka commented Nov 7, 2018

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

bpo-31553: add --json-lines option to json.tool#10051
serhiy-storchaka merged 4 commits into
python:masterfrom
hongweipeng:jsonlines-for-json.tool

hongweipeng commented Oct 23, 2018 •
edited by bedevere-bot
Loading