Crates:

• The main syntect_server executable
• crates/tree-sitter-all-languages/: All the grammars and parsers live here to make shipping parsers with the same tree-sitter version (and associated build tooling) very easy. All new grammars and parsers should be added here.
• crates/scip-syntax/: Command line tool that produces a SCIP index file from the analysis performed by other scip-* crates in this project
• crates/syntax-analysis/: local navigation calculation (and some other utilities) live here.

See queries

This is an HTTP server that exposes the Rust Syntect syntax highlighting library for use by other services. Send it some code, and it'll send you syntax-highlighted code in response. This service is horizontally scalable, but please give #21942 and #32359 a read before scaling it up.

cargo run --bin syntect_server

You can set the SRC_SYNTECT_SERVER environment var to whatever port this connects to and test against local Sourcegraph instance.

docker run --detach --name=syntax-highlighter -p 9238:9238 sourcegraph/syntax-highlighter

You can then e.g. GET http://localhost:9238/health or http://host.docker.internal:9238/health to confirm it is working.

See API

By default on startup, syntect_server will list all file types it supports. This can be disabled by setting QUIET=true in the environment.

1. Use cargo test --workspace to run all the tests. To update snapshots, run cargo insta review.
2. Use cargo run --bin syntect_server to run the server locally.
3. You can change the SRC_SYNTECT_SERVER option in your sg.config.yaml to point to whatever port you're running on (usually 8000) and test against that without building the docker image.

For formatting, run:

bazel run @rules_rust//:rustfmt

You can run a subset of tests for syntect_scip using the ONLY environment variable. Example:

cd crates/syntax-analysis
ONLY=.java cargo test test_all_files -- --nocapture

./build.sh will build your current repository checkout into a final Docker image. You DO NOT need to do this when you push to get it publish. But, you should do this to make sure that it is possible to build the image 😄.

AGAIN NOTE: The docker image will be published automatically via CI.

Once published, the image version will need to be updated in the following locations to make Sourcegraph use it:

• sourcegraph/sourcegraph > cmd/server/Dockerfile
• sourcegraph/sourcegraph > sg.config.yaml

Additionally, it's worth doing a search for other uses in case this list is stale.

See tree-sitter-all-languages

https://github.com/sourcegraph/Packages is the package registry we use which holds all of the syntax definitions we use in syntect_server and Sourcegraph. Send a PR there by following these steps

We use a temporary fork of syntect as a hack to get our Packages registry into the binary. Update it by creating a PR with two commits like:

• https://github.com/slimsag/syntect/commit/9976d2095e49fd91607026364466cd7b389b938e
• https://github.com/slimsag/syntect/commit/1182dd3bd7c82b6655d8466c9896a1e4f458c71e

Run the following in this directory.

$ cargo update -p syntect

Run: cargo run --bin syntect_server to see supported languages.