| FazBrowse GitHub Viewer | Trending | | Home |
| Tools: [Download Repo ZIP] [Original HTTPS Page] |
Sorry, something went wrong.
…uments json.loads() and json.dumps() enter through Python wrappers that run on every call: decode() scanned for leading and trailing whitespace with a regex, and iterencode() built a float-formatting closure even when dispatching to the C encoder. Skip the whitespace scan when there is none, and build the float helper only on the Python encoding path. Both wrappers stay in Python, so the wins apply to the default build; output is byte-identical.
|
+1 on the change to floatstr (this is also part of my branch with a C version of the encoder main...eendebakpt:cpython:json_floatstr, in my benchmarks this makes a difference) For the whitespace we should decide on #117397 first. For that reason maybe split the PR into two (note: please avoid force pushes, they make reviews harder) |
Sorry, something went wrong.
|
@gaborbernat Did you check pyperformance json_loads? I checked this variant once (worse than yours; one more branch): diff --git a/Lib/json/decoder.py b/Lib/json/decoder.py
index 364e44d40cc..37ab3f5cf66 100644
--- a/Lib/json/decoder.py
+++ b/Lib/json/decoder.py
@@ -355,8 +355,13 @@ def decode(self, s, _w=WHITESPACE.match):
containing a JSON document).
"""
- obj, end = self.raw_decode(s, idx=_w(s, 0).end())
- end = _w(s, end).end()
+ if s and s[0] in WHITESPACE_STR:
+ idx = _w(s, 0).end()
+ else:
+ idx = 0
+ obj, end = self.raw_decode(s, idx=idx)
+ if end < len(s) and s[end] in WHITESPACE_STR:
+ end = _w(s, end).end()
if end != len(s):
raise JSONDecodeError("Extra data", s, end)
return objThat's what I've got json_loads result: patch (afe23af): json_loads: Mean +- std dev: 11.5 us +- 0.3 us main (2043886): json_loads: Mean +- std dev: 11.1 us +- 0.2 us Using: import json
import pyperf
def sized(target):
n = target // 12
while True:
s = json.dumps({str(i): i for i in range(n)})
if len(s) >= target:
return s
n += target // 24
PAYLOADS = {
"no_ws_tiny": '{"a":1}', # fast-path target
"leading_ws": ' {"a":1}', # slow path
"trailing_ws": '{"a":1} ', # slow path
"both_ws": ' {"a":1} ', # slow path
"no_ws_1kb": sized(1 * 1024), # amortization
"no_ws_4kb": sized(4 * 1024),
"no_ws_8kb": sized(8 * 1024),
"no_ws_16kb": sized(16 * 1024),
"no_ws_32kb": sized(32 * 1024),
"no_ws_64kb": sized(64 * 1024),
"no_ws_100kb": sized(100 * 1024), # sanity
}
if __name__ == "__main__":
runner = pyperf.Runner()
for name, payload in PAYLOADS.items():
runner.timeit(
name=f"json_loads/{name}",
stmt="loads(s)",
setup=f"from json import loads; s = {payload!r}",
)The results:
|
Sorry, something went wrong.
Keep only the encoder floatstr deferral here; the decoder whitespace skip is contentious (depends on pythongh-117397 and shows mixed results on large documents) and moves to its own PR.
|
Yes — on the pyperformance json_loads dataset (EMPTY/SIMPLE/NESTED/HUGE) it's 1.16x faster (5.57 ms → 4.78 ms): that workload is dominated by small documents with no surrounding whitespace, which is exactly the case this targets. The regression you saw came from the extra branch in your variant — if end < len(s) and s[end] in WHITESPACE_STR adds a per-call membership test. The version here instead skips the trailing match only when the parse already consumed the whole string (end == len(s)), so it avoids that check on the common path. One thing to note: I've split this PR. The whitespace-skip change you're benchmarking now lives in #150861; this PR (#150827) is just the encoder floatstr change. Happy to continue the json_loads discussion over there. |
Sorry, something went wrong.
There was a problem hiding this comment.
Thanks for splitting. Two nits, but +1 for me.
Sorry, something went wrong.
Per review: the comment and the implementation detail in the NEWS entry are not relevant to end users.
|
Thanks for the review and the split suggestion. Applied both: dropped the comment and trimmed the NEWS entry to Speed up :func:json.dumps for small documents. |
Sorry, something went wrong.
|
Could a maintainer kindly re-run the failed job when you get a chance? The failure is in the iOS check (-[TestbedTests testPython] failed to execute), which looks like an unrelated testbed flake — this PR only changes pure-Python Lib/json/encoder.py. Thank you for your time reviewing this. |
Sorry, something went wrong.
Please be patient. Maintainers have many pr's to review and it can take some time. Re-running the test is not required for a review. (although it is often done after approval) |
Sorry, something went wrong.
There was a problem hiding this comment.
Did you run benchmarks for debug build or for optimized build?
Anyways, this is just rearranging of the lines of the code, so the cost is very cheap. LGTM. 👍
Sorry, something went wrong.
…150827) Only create the float formatting helper when the pure Python encoder is used.
| Back | FazBrowse Home | New Git URL |
json.dumps() runs through the pure-Python iterencode() wrapper on every call before the C encoder does the heavy lifting, and that wrapper builds a float-formatting helper closure each time — even though the common C fast path never uses it. For the small documents that dominate real traffic (an API response, a single record, a config fragment), that per-call closure construction is a measurable slice of the total work.
This builds the float helper only inside the branch that runs the pure-Python encoder, so the C fast path no longer allocates a closure it never calls. The wrapper stays in Python, so the win lands on the default build; output is byte-identical.
Over 20 small documents, json.dumps() is about 8% faster, consistently across float-heavy, mixed, and no-float payloads:
Run base vs patched by swapping Lib/json/encoder.py on the same interpreter. The script is self-contained.
The decoder whitespace-skip change that originally accompanied this has moved to a separate PR, since it is more contentious (it depends on the outcome of gh-117397 and shows mixed results on large documents); this PR is now the encoder change alone.
Resolves #150820.