Speed up float encoding in `json` · Issue #159702 · python/cpython · GitHub

Feature or enhancement

I ran a throughput benchmark for different json libraries.

Float encoding is a particularly weak area for the json library, especially for large-magnitude floats. Would it be possible to optimize this by adding a special case to PyUnicodeWriter_WriteRepr(), similar to _PyLong_FormatWriter() for integers?

encode	json	jsonyx	fastsimdjson	msgspec	orjson	fastest time
0.0	12.68x	14.10x	1.32x	1.00x	1.11x	7.32 ns
1e-05	13.05x	13.47x	1.12x	1.81x	1.00x	29.43 ns
5e-324	25.71x	26.07x	1.07x	1.00x	1.33x	24.83 ns
1.7976931348623157e+308	79.89x	80.30x	1.00x	1.28x	1.03x	30.29 ns

cc @vstinner

Has this already been discussed elsewhere?

This is a minor feature, which does not need previous discussion elsewhere

Links to previous discussion of this feature:

No response

Linked PRs

gh-159702: Add fast-path for float in _json encoder #159826

A fast path in PyUnicodeWriter_WriteRepr seems reasonable, but would not help a lot (I did a quick benchmark, only about 10% depending on the type of floats). A faster float-to-str would help more. There are algorithms for it, but not readily available in C. See https://discuss.python.org/t/faster-float-to-string-conversions/107397

@eendebakpt @nineteendo

@vstinner and I discussed this at the sprint that we are going to measure general benchmark how much this call paths are executed and adding the fast path.

I added the int fast-path in #125214 which contains benchmark results: it was between 1.54x and 1.93x faster on repr(list) depending on the list size (between 3 and 50 items).

FYI I added the str fast-path in #133969 which also contains benchmark results (1.18x faster in average, up to 1.61x faster in the best case).

In the past, I considered to add a type slot (like PyTypeObject.tp_str_writer and/or PyTypeObject.tp_repr_writer) instead of using hardcoded fast paths. If we add multiple fast paths in the future, we might reconsider adding a type slot?

pyperformance would not be a good benchmark for adding this fast path: 0% of calls.
But even in the standard library test suite:

Argument type	Calls	Share	Note
exact int	2,924,088	86.7%	existing fast path (_PyLong_FormatWriter)
None	181,066	5.4%
exact list	69,501	2.1%
other types	58,946	1.7%	user classes, type objects, etc.
exact str	58,158	1.7%
exact dict	40,537	1.2%
bool	20,759	0.6%
exact float	10,657	0.32%	proposed fast path
exact bytes	5,700	0.2%
exact tuple	2,244	0.07%
total	3,371,663	100%

So I am -1 unless we can provide a benchmark that justifies this fast path, and even then float would not be the first candidate, since it is lower than str and dict.

pyperformance would not be a good benchmark for adding this fast path: 0% of calls.

If floats are more common in json than in general in Python, json can add a fast-path for float using the internal C API.

The _json extension already has a code path for float which gets a PyUnicodeWriter:

    else if (PyFloat_Check(obj)) {
        PyObject *encoded = encoder_encode_float(s, obj);
        if (encoded == NULL)
            return -1;
        return _steal_accumulate(writer, encoded);
    }

PyUnicode_Format() calls formatfloat() for %f (and other similar formats): it calls PyOS_double_to_string() + _PyUnicodeWriter_WriteASCIIString().

float.__float__() calls _PyFloat_FormatAdvancedWriter() which is implemented with PyOS_double_to_string() + _PyUnicodeWriter_WriteASCIIString() for the common case.

PyUnicode_FromFormat() doesn't support %f format.

pythongh-159702: Add fast-path for float in _json encoder

If floats are more common in json than in general in Python, json can add a fast-path for float using the internal C API.

I wrote PR gh-159826 to do that. It calls PyOS_double_to_string() + PyUnicodeWriter_WriteASCII() (functions of the public C API).

@nineteendo: Would it be possible for you to re-run your benchmark on my PR?

Oh crap, I wrote #156897 (JSON float optimization for json.loads()) last month, but I used git push --force and so lost my work :-(

Speed up float encoding in `json` · Issue #159702 · python/cpython · GitHub

Repository navigation

Speed up float encoding in json #159702

Feature or enhancement

Has this already been discussed elsewhere?

Links to previous discussion of this feature:

Linked PRs

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Speed up float encoding in `json` · Issue #159702 · python/cpython · GitHub

Repository navigation

Speed up float encoding in json #159702

Description

Feature or enhancement

Has this already been discussed elsewhere?

Links to previous discussion of this feature:

Linked PRs

Activity

eendebakpt commented on Oct 19, 2026

corona10 commented on Oct 19, 2026

vstinner commented on Oct 19, 2026

vstinner commented on Oct 19, 2026

corona10 commented on Oct 19, 2026

vstinner commented on Oct 20, 2026

vstinner commented on Oct 20, 2026

vstinner commented on Oct 20, 2026

vstinner commented on Oct 20, 2026

h-vetinari commented on Oct 21, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions