src: improve windows1252 decoding speed - #61120
Conversation
Sorry, something went wrong.
| if (!tWindows1252x2) { | ||
| tWindows1252x2 = (uint32_t *) malloc(256 * 256 * 4); // 256 KiB | ||
| for (uint16_t i = 0; i < 256; i++) { | ||
| for (uint16_t j = 0; j < 256; j++) { | ||
| tWindows1252x2[(i << 8) + j] = (((uint32_t) table[i]) << 16) + table[j]; | ||
| } | ||
| } | ||
| } |
There was a problem hiding this comment.
We’re allocating this table (tWindows1252x2) here for the first time, but we never free it anywhere
This could lead to unnecessary memory usage in long-running processes
I think we should add a cleanup hook right after the allocation, so the table gets freed automatically when the environment shuts down
Sorry, something went wrong.
There was a problem hiding this comment.
this PR is a crude concept demo
Sorry, something went wrong.
|
This pull request has been marked as stale due to 90 days of inactivity. |
Sorry, something went wrong.
|
This pull request has been automatically closed after 30 days of inactivity following its stale status (no activity for a total of 120 days). |
Sorry, something went wrong.
Tracking: #61041
This builds on top of #61093 and #61118 and gives an additional ~1.4x improvement by using iteration in pairs for this single encoding, as it's commonly used (it's aliased as latin1)
Combined, this is ~114x faster than main on ASCII (due to #61093 + #61118) and ~71x faster than main on non-ASCII
This could be improved further with latin1 checks and by moving ascii checks to prefixes instead, but let #61119 land first
I'm not sure if this even makes sense at this point, it comes at a cost of 128 KiB cache (even though allocated at the first large use)
Perhaps there is some other way or we could ignore this
Warning
Very crude, just a concept demonstration at this point
See #61118 for previous benchmarks