| FazBrowse GitHub Viewer | Trending | | Home |
| Tools: [Download Repo ZIP] [Original HTTPS Page] |
|
@Extrodox is attempting to deploy a commit to the shashankss1205's projects Team on Vercel. A member of the Team first needs to authorize it. |
Sorry, something went wrong.
|
Overlaps merged #1071 watcher sync and conflicts with current watcher.py. |
Sorry, something went wrong.
# Conflicts: # src/codegraphcontext/core/watcher.py
main added _update_lock/_timers_lock after this branch was cut (CodeGraphContext#1424, serialising concurrent watcher updates). Tests that build the handler via __new__ must set them, as the neighbouring tests in this file already do.
|
Rebased onto current main and pushed to your branch — it was 64 days stale and watcher.py has had 15 commits since you branched from v0.4.13. Suite is green (1111 passed, 7 skipped). Two things I had to resolve:
Why I have not merged itThe parallel parsing hits an open segfault bug — #1370. That issue reports non-deterministic Fatal Python error: Segmentation fault when multiple worker threads call tree_sitter_manager.execute_query() concurrently on different languages. Its traceback shows three threads inside execute_query() at once (TypeScript, Python, HTML). Your ThreadPoolExecutor runs graph_builder.parse_file across affected files, and in a mixed-language repo the affected set spans languages — the exact condition. And execute_query is still unsynchronised on main: # utils/tree_sitter_manager.py:322
def execute_query(language, query_string, node):
query = Query(language, query_string) # per call, no lock
cursor = QueryCursor(query)
res = cursor.captures(node)tree_sitter_manager locks its language cache (_cache_lock) but nothing guards query execution. So merging this would move a known crash from cgc index into cgc watch, where it is worse: watch runs unattended for hours, and a segfault takes the whole process down mid-session. What would unblock itEither is fine, and either would be a genuinely valuable PR on its own:
The rest of the PR is good — reusing the already-parsed changed file is a clear win regardless, and if you wanted to split that out as its own PR it would merge immediately. Keeping this open as a draft. |
Sorry, something went wrong.
|
#1370 is now fixed — #1593 merged, adding a process-wide lock around tree-sitter query construction and execution. That was the blocker on this PR. Measured cost of the lock is ~1% (3.48s vs 3.45s median over 120 parses on 8 threads), because only the query step is serialised — parser.parse and all Python-level work stay parallel, so your ThreadPoolExecutor still buys what it was meant to buy. One caveat I want to be straight about: I could not reproduce the segfault locally, so #1593 is a guard against the reported crash rather than one I demonstrated. It is not proof your parallelism is now safe — it removes the specific unsynchronised path the report pointed at. Your branch is already rebased onto main and green. Since it is still marked draft, I have left it for you to mark ready. Worth re-running your own workload against it first if you have the mixed-language repo that motivated the change. |
Sorry, something went wrong.
|
This is still marked as a draft, so I've left it alone — flagging that it's now conflict-free against main if you'd like to mark it ready for review. Parallelising the incremental parse step is worth having; happy to review once you take it out of draft. |
Sorry, something went wrong.
| Back | FazBrowse Home | New Git URL |
Summary
Why
Incremental indexing was effectively serial for affected-file parsing, and the changed file could be parsed twice in one update pass. This makes large affected sets slower than necessary even when parsing work is CPU-bound and independent per file.
Validation
Note: the default Homebrew Python in this shell is externally managed and lacked project dependencies, so validation used a temporary venv under /tmp/cgc-pr-venv.