| FazBrowse GitHub Viewer | Trending | | Home |
| Tools: [Download Repo ZIP] [Original HTTPS Page] |
Sanitizing user-authored response fields ran multiple allocating passes over every string regardless of content: FilterInvisibleCharacters converted the whole input to []rune and back, FilterCodeFenceMetadata split and rejoined every line, and bluemonday ran unconditionally. On comment- and issue-heavy responses this dominated conversion CPU and allocation. Three changes, none of which alter output or widen what the policy allows: - FilterInvisibleCharacters scans first and copies only from the first filtered rune, skipping ASCII runs without decoding them. Invalid UTF-8 is still re-encoded to U+FFFD, matching the []rune round trip it replaces. - FilterCodeFenceMetadata walks lines in place and returns the input when no line changes. - FilterHTMLTags skips bluemonday for input that is provably a fixed point of the policy: printable ASCII, TAB and LF, with none of the five characters html.EscapeString rewrites. Sanitize also skips the second invisible/code-fence pass when HTML normalization returned its input unchanged, since both filters are fixed points there. Equivalence is pinned by a verbatim copy of the previous pipeline: the new code is diffed against it over a corpus of ~22k deterministic cases plus two fuzz targets, and the fast path is checked byte by byte against the live bluemonday policy. Benchmarks (Intel Ultra 9 185H, n=6): Sanitize/TitleASCII 5.35µs -> 114ns 1 -100% allocs Sanitize/Comment1KiB 45.3µs -> 1.27µs 1 -100% allocs Sanitize/Body64KiB 2.47ms -> 85.9µs 1 -100% allocs 30 issues x 2KiB body 3.00ms -> 88.6µs 1.55MiB -> 1.9KiB 100 comments x 1KiB 5.04ms -> 169µs 2.19MiB -> 6.3KiB Content that genuinely needs rewriting still pays for it, and non-ASCII text still goes through bluemonday by design. Fixes #3117 Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Optimizes sanitization hot paths while preserving existing filtering behavior.
Changes:
| File | Description |
|---|---|
| pkg/sanitize/sanitize.go | Implements sanitizer fast paths. |
| pkg/sanitize/equivalence_test.go | Verifies behavioral equivalence and invariants. |
| pkg/sanitize/bench_test.go | Benchmarks sanitizer workloads. |
| pkg/github/minimal_types_bench_test.go | Benchmarks representative converters. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Sorry, something went wrong.
| Back | FazBrowse Home | New Git URL |
Fixes #3117
Sanitization ran several allocating passes over every user-authored field regardless of content. FilterInvisibleCharacters converted the whole input to []rune and back, FilterCodeFenceMetadata split and rejoined every line, and bluemonday ran unconditionally — so benign text paid full price on every conversion.
Changes
FilterInvisibleCharacters is scan-first / copy-on-first-match. It skips runs of ASCII without decoding them (no filtered rune is ASCII, and no variation selector is either), then copies into a builder only from the first rune that actually changes. Clean input is returned as the original string with zero allocations. Contextual variation-sequence handling is unchanged, and invalid UTF-8 is still re-encoded to U+FFFD to match the []rune round trip it replaces.
FilterCodeFenceMetadata walks lines in place instead of strings.Split/Join, and returns the input untouched when no line changes. sanitizeCodeFenceLine also stops rebuilding a line whose info string is already normalized.
FilterHTMLTags skips bluemonday for provably inert input. The policy tokenizes as HTML and re-emits text through html.EscapeString, so anything it can rewrite must contain one of the five characters EscapeString touches (&, ', ", <, > — also the only way to open a tag, comment, doctype or entity), a byte the tokenizer rewrites (NUL → U+FFFD, CR folded into LF), or a byte outside ASCII that could be malformed UTF-8. The fast path accepts printable ASCII plus TAB and LF minus those five characters, which excludes all three. A loose <>& check would not be sufficient: " and ' are escaped, and CR/NUL are rewritten by the tokenizer.
Sanitize skips the second invisible/code-fence pass when HTML normalization returned its input byte for byte. HTML processing is the only stage that can introduce a character its input did not contain (entity decoding), so if it is the identity there is nothing new to find, and both filters are fixed points on the first pass's output.
Nothing here weakens filtering or broadens allowed HTML. Content that genuinely needs rewriting still goes through the full pipeline, and non-ASCII text still goes through bluemonday by design.
Equivalence evidence
pkg/sanitize/equivalence_test.go keeps a verbatim copy of the previous pipeline and diffs the new code against it:
Benchmarks
Committed in pkg/sanitize/bench_test.go and pkg/github/minimal_types_bench_test.go. Intel Ultra 9 185H, -count=6, benchstat:
All differences p=0.002 (n=6). The residual converter allocations are the struct and slice fields, not sanitization.
Projected overhead at 1,250 RPS
Per the issue's shapes, sanitizing one response:
At 1,250 RPS:
Sanitization stops being a GC-bound cost centre for ordinary content. Non-ASCII bodies still pay for bluemonday (~1.0 ms per 64 KiB, down from 2.7 ms); extending the fast path past strict ASCII is deliberately left out of this change.
Validation
script/lint clean, script/test (race) green, script/generate-docs produces no diff.