| FazBrowse GitHub Viewer | Trending | | Home |
| Tools: [Download Repo ZIP] [Original HTTPS Page] |
The sysmon core needs the multiline map (line -> first line of its multi-line statement) to resolve branch events. It got it by running a full PythonParser.parse_source() per file — an ast.parse, a full tokenization, a compile() via ByteParser, and AST walks — behind a module-level functools.lru_cache(maxsize=20). Test suites tracing more than 20 files thrash that cache and re-parse files repeatedly (the pyca/cryptography suite traces 192 files; see issue coveragepy#2172 for a report of sysmon branch mode being slow on such suites). Three changes: - Make multiline_map_from_tokens() in parser.py the one place the map is computed: PythonParser._raw_parse tokenizes once into a list, gets the map from the shared builder, and its own token loop keeps only the exclusion and indent bookkeeping (its in-flight first_line tracking stays, since the exclusion logic needs the statement start before the map entry for the current statement exists). - Use multiline_map_from_text() in the sysmon core instead of a full parse. This is ~4x cheaper (0.34s vs 1.48s for the 192 files of the cryptography suite) and produces identical maps (verified on 445 files: coverage's own source and tests, the stdlib, cryptography and its tests). - Cache the maps in a plain dict on the SysMonitor instance, unbounded, so each traced file is tokenized at most once per run. The cache dies with the tracer, which also removes the cross-run staleness a module-level cache can have. Measured on the cryptography suite (Python 3.14.2, branch mode, wall time best of 3, base 35.44s): 47.90s (+35.2%) before, 46.07s (+30.0%) after, with byte-identical coverage data. The parse/report phase pays one extra pure-Python pass over the already-materialized token list (parse_source() over the 192 files: 1.75s -> 1.95s, ~+1ms per file), which the reporting-phase caches already amortize. Most of the remaining measurement overhead is the branch_trails() analysis, addressed separately by the sysmon-lazy-branch-resolver branch; the two changes compose to ~0-2%. Verified: parser results (statements, excluded, raw_excluded, multiline_map) identical to released main on 445 files with a realistic exclusion regex; coverage's own test suite failure set identical to released main. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0137DLbSXfm5v5bhCcz7xEKM
Add directed tests of multiline_map_from_text() for each statement shape, plus a parametrized check that it always matches the map PythonParser produces, since branch arcs are attributed with one and reports are keyed by the other. Also test compute_multiline_map()'s fallbacks for missing, non-Python, and badly indented files, and that SysMonitor computes each file's map at most once per tracer instance. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0137DLbSXfm5v5bhCcz7xEKM
|
This is now released as part of coverage 7.15.1. |
Sorry, something went wrong.
| Back | FazBrowse Home | New Git URL |
(This PR was generated via claude but has undergone significant review and changes based on that review before submission)
When measuring branch coverage under sys.monitoring, the tracer needs each file's multiline map to resolve branch events. It currently gets it by running a full PythonParser.parse_source(). This change extracts the map computation into a shared multiline_map_from_tokens(). This is refactored to be the single place the map is built, used by both PythonParser._raw_parse and the sysmon core.
On the cryptography test suite (Python 3.14, branch mode), sysmon overhead drops from +35.2% to +30.0%; the remaining overhead is mostly branch_trails() analysis, which is the next PR. 😄