Commit graph

1651 commits

Author SHA1 Message Date
Will Lillis 69723ca40e fix(query): correct last_child_step_index in cases where a new step
wasn't created.

This fixes an OOB access to `self.steps` when a last child anchor
immediately follows a predicate.

(cherry picked from commit b1d2b7cfb8)
2025-05-04 00:08:10 +02:00
Will Lillis 97131b4a73 fix(rust): address new clippy lint
(cherry picked from commit cc634236b1)
2025-05-03 22:00:55 +02:00
Amaan Qureshi 41413e7a71 fix(generate): mark url as a Windows-only dependency
(cherry picked from commit 3056dc5be4)
2025-04-29 09:20:34 +02:00
Will Lillis a876fff5ba fix(parse): explicitly move temporaries in the logger callback
This fixes problems where these stack-local temporaries are used after their scope ends.

(cherry picked from commit dcdd5bc372)
2025-04-28 10:12:37 +02:00
Will Lillis 7ddcc7b20b perf(highlight): use BTreeMap over IndexMap for highlight configs
(cherry picked from commit c7475e4bf3)
2025-04-20 07:30:24 -04:00
Daniel Jalkut 779d613941 docs(cli): improve documentation for the edits argument when parsing code
(cherry picked from commit 4514751803)
2025-04-19 12:22:46 +02:00
MichiRecRoom d44d0f94da docs(rust): improve bindings' crate doc
(cherry picked from commit 853ca46899)
2025-04-19 12:01:25 +02:00
Paul Gey 264684d31d Make highlighting more deterministic when themes are ambiguous
(cherry picked from commit b341073192)
2025-04-11 10:20:43 +02:00
Jon Shea e295c99eca fix(rust): clarify error message for non-token reserved words
Improve the `NonTokenReservedWord` error message by including the
specific reserved word that was not used as a token.

(cherry picked from commit 92c5d3b8e2)
2025-04-10 01:10:25 -04:00
Edgar Onghena d2914ca243 chore(generate): add @generated to parser.c header (#4338)
This makes `parser.c` follow the https://generated.at/ convention for generated files. This potentially allows any compatible IDE to discourage editing it directly.

(cherry picked from commit 52d2865365)
2025-04-08 11:20:25 +02:00
Will Lillis 4619261da0 fix(cli): display "N/A" in parse stats where appropriate when no parsing
took place

(cherry picked from commit 0f949168ef)
2025-04-06 17:13:43 +02:00
Will Lillis 14d930d131 fix(highlight): account for multiple rows in highlight testing assertions
(cherry picked from commit 71941d8bda)
2025-04-06 17:13:43 +02:00
Amaan Qureshi ff8bf05def fix(rust): adapt to new clippy lints
(cherry picked from commit 74d7ca8582)
2025-04-06 16:12:21 +02:00
WillLillis fae24b6da6 fix(rust): address new nightly lint for pointer comparisons
(cherry picked from commit 521da2b0a7)
2025-03-28 09:41:19 +01:00
Peter Oliver d25e5d48ea fix(build): make install shouldn’t fail when a parser bundles no queries (#4284)
(cherry picked from commit 17471bdfcc)
2025-03-14 10:06:40 +01:00
WillLillis 979e5ecec0 fix(cli): properly escape invisible characters in parse error output
(cherry picked from commit efd212ee46)
2025-03-12 11:33:28 +01:00
dependabot[bot] b1a9a827d6 build(deps): bump emscripten to 4.0.4
(cherry picked from commit 12aff698b9)
2025-03-12 10:57:58 +01:00
WillLillis c313be63b2 fix(rust): adapt to new nightly lint
(cherry picked from commit 11071ed682)
2025-03-06 18:25:24 -05:00
Max Brunsfeld 2a835ee029 0.25.3 2025-03-04 16:03:16 -08:00
tree-sitter-ci-bot[bot] 3ad1c7d4e1
Fix cases where error recovery could infinite loop (#4257) (#4262)
* Rename corpus test functions to allow easy filtering by language

* Use usize for seed argument

* Avoid retaining useless stack versions when reductions merge

We found this problem when debugging an infinite loop that happened
during error recovery when using the Zig grammar. The large number of
unnecessary paused stack versions were preventing the correct recovery
strategy from being tried.

* Fix leaked lookahead token when reduction results in a merged stack

* Enable running PHP tests in CI

* Fix possible infinite loop during error recovery at EOF

* Account for external scanner state changes when detecting changed ranges in subtrees

(cherry picked from commit 066fd77d39)

Co-authored-by: Max Brunsfeld <maxbrunsfeld@gmail.com>
2025-03-04 15:38:59 -08:00
polazarus b1a7074010 fix(generate): mark TSCharacterRange as static (#4255)
Problem: Linking different parsers into one executable fails due to duplicate symbols.

Solution: Mark `TSCharacterRange` as `static` when generating parsers.

fixes #4209

(cherry picked from commit 8138dba800)
2025-03-04 16:52:58 +01:00
WillLillis 1f64036d87 fix(test): update expected tree-sitter-rust supertypes
(cherry picked from commit 998fb34d15)
2025-03-02 23:20:08 +01:00
WillLillis 4eb46b493f fix(rust): adapt to some new nightly lints
(cherry picked from commit cb30ec5b17)
2025-03-02 23:20:08 +01:00
Max Brunsfeld 6e0618704a 0.25.2 2025-02-17 18:54:23 -08:00
Amaan Qureshi f5afe475de
build: bump version to 0.25.1
Some checks failed
CI / checks (push) Waiting to run
CI / sanitize (push) Failing after 11s
CI / build (push) Failing after 14s
Deploy Docs / deploy-docs (push) Has been cancelled
Check WASM Exports / check-wasm-exports (push) Has been cancelled
2025-02-02 02:07:36 -05:00
Amaan Qureshi eed662df98
fix(bindings): correct Zig bindings to expose a language function
Instead of having users declare the extern function themselves, they can
pass in the language to `Language.create` in the zig bindings. If they
really want, they can always opt into the `extern fn tree_sitter_LANG()
*const ts.Language` approach.
2025-02-02 02:07:36 -05:00
Riley Bruins 9ad096ef22 fix(lib): prevent finished_tree assertion failure
Some checks failed
CI / checks (push) Waiting to run
CI / sanitize (push) Failing after 12s
CI / build (push) Failing after 14s
**Problem:** When resetting the parser during subtree balancing, an
error is thrown:

```
parser.c:2198: ts_parser_parse: Assertion `self->finished_tree.ptr' failed.
```

**Solution:** Reset `canceled_balancing` to false in
`ts_parser_reset()`.
2025-02-01 16:19:14 -05:00
ObserverOfTime c2221f2732 feat(cli): specify abi version via env var 2025-02-01 13:57:53 -05:00
Amaan Qureshi 16aaed78ae build: update authors
Some checks failed
CI / checks (push) Waiting to run
CI / sanitize (push) Failing after 12s
CI / build (push) Failing after 13s
2025-01-31 21:11:10 -05:00
Amaan Qureshi dc64bb5395 chore: adapt to latest clippy lints 2025-01-31 20:41:11 -05:00
ObserverOfTime 51acdf9723
feat(bindings)!: update swift bindings
Some checks failed
CI / checks (push) Waiting to run
CI / sanitize (push) Failing after 11s
CI / build (push) Failing after 13s
2025-01-31 10:16:01 -05:00
ObserverOfTime 1b3a9a4563 chore(bindings): include license file in crate 2025-01-31 16:22:25 +02:00
ObserverOfTime 032af925fc build(cli): explicitly include files 2025-01-28 19:30:43 -05:00
Gabriel Holodak f5e9680b75 fix(cli): propagate error flag in parse summary 2025-01-28 18:17:31 -05:00
Amaan Qureshi 2cf18f5ac2 build: bump dependencies 2025-01-26 14:15:01 -05:00
Amaan Qureshi 959f096a89 feat(generate): defer to ABI 14 if tree-sitter.json doesn't exist, rather than hard failing
Some checks failed
CI / checks (push) Waiting to run
CI / sanitize (push) Failing after 11s
CI / build (push) Failing after 14s
2025-01-26 13:12:58 -05:00
Amaan Qureshi 016dd7afbf feat(bindings): use cc 1.2 for Rust 2025-01-26 13:12:58 -05:00
Amaan Qureshi 4940d2c29b fix(cli): correct traversal behavior for finding error nodes
Some checks failed
CI / checks (push) Waiting to run
CI / sanitize (push) Failing after 12s
CI / build (push) Failing after 14s
2025-01-26 01:10:01 -05:00
Amaan Qureshi 4b6723192e refactor(cli): replace embedded xterm colors file with ansi_colours
Some checks failed
CI / checks (push) Waiting to run
CI / sanitize (push) Failing after 11s
CI / build (push) Failing after 14s
Deploy Docs / deploy-docs (push) Has been cancelled
2025-01-25 17:20:18 -05:00
Amaan Qureshi c01bf6ea4a fix(cli): minor discrepancies with pretty printer
Some checks failed
CI / checks (push) Waiting to run
Deploy Docs / deploy-docs (push) Waiting to run
CI / sanitize (push) Failing after 11s
CI / build (push) Failing after 13s
Check Bindgen Output / check-bindgen (push) Has been cancelled
Check WASM Exports / check-wasm-exports (push) Has been cancelled
2025-01-25 03:47:26 -05:00
ObserverOfTime 5cfeba9c0d feat(bindings): add opt-in zig bindings 2025-01-25 03:47:19 -05:00
Amaan Qureshi 46f8d1267c feat(init): add a title field to grammars 2025-01-25 03:20:43 -05:00
Amaan Qureshi 4820d50336 fix(init): handle parser name replacements better when a kebab-case name is expected 2025-01-25 03:20:43 -05:00
Amaan Qureshi bde94aed4d feat(init): add a class-name field to grammars 2025-01-25 03:20:43 -05:00
Amaan Qureshi 6dbcfdf282 feat(init): add an optional funding field 2025-01-25 03:20:43 -05:00
Amaan Qureshi 4ab78ae00d fix(init): don't prepend the default filetype with a . 2025-01-25 03:20:43 -05:00
Allan Clements cda634a1c4 feat: add error information in the progress callback
This allows users to bail parsing if an error was *definitely* detected
using the progress callback, as all possible stack versions have a
non-zero error cost.

Co-authored-by: Amaan Qureshi <amaanq12@gmail.com>
2025-01-25 02:47:39 -05:00
Amaan Qureshi 8bb1448a6f feat: add the semantic version to TSLanguage, and expose an API for retrieving it 2025-01-25 01:14:30 -05:00
Yuri Astrakhan 3e7721e554 chore: a few more minor lints 2025-01-23 00:10:16 -05:00
Amaan Qureshi 3e72969ce4 fix(playground): work around lack of module support in mdbook
Some checks failed
CI / checks (push) Waiting to run
CI / sanitize (push) Failing after 11s
CI / build (push) Failing after 13s
Deploy Docs / deploy-docs (push) Has been cancelled
2025-01-22 04:09:08 -05:00
Amaan Qureshi 692332ed1c feat!: update playground with new web bindings 2025-01-21 12:36:15 -05:00
Amaan Qureshi 6941497c7a test: improve test coverage
Some checks failed
CI / checks (push) Waiting to run
CI / sanitize (push) Failing after 11s
CI / build (push) Failing after 13s
2025-01-21 01:17:03 -05:00
Yuri Astrakhan 9dbe165296 chore: a few minor lints
* do not use `&` for the format args as it cannot (yet) be optimized by the compiler
* a few format inlining
2025-01-21 00:57:58 -05:00
Yuri Astrakhan 48059b72a8 feat: remove lazy_static in favor of LazyLock
This switches to the built-in `std::sync::LazyLock`
2025-01-21 00:21:35 -05:00
Amaan Qureshi 9365586cc3 feat: allow parser balancing to be cancellable 2025-01-20 23:52:19 -05:00
dependabot[bot] 0e226561b1 build(deps): bump emscripten to 4.0.1 2025-01-20 23:27:25 -05:00
Scorg 40eb26e580 build(bindings): move header to tree_sitter subdirectory
Some checks failed
CI / checks (push) Waiting to run
Deploy Docs / deploy-docs (push) Waiting to run
CI / sanitize (push) Failing after 13s
CI / build (push) Failing after 14s
This patch allows users to include the parser by the same path from
local build as well as installed location. Previously it was not
possible to include the header prior to installing the built parser.
2025-01-20 13:17:24 -05:00
ObserverOfTime a9dbb7257c feat(bindings): support free-threaded python build 2025-01-19 12:45:08 -05:00
ObserverOfTime 7bf51ae08a feat(bindings): drop python 3.9 support 2025-01-19 12:45:08 -05:00
WillLillis 3a85d4d5f3 feat(cli): improve readability of parse debug output
Some checks failed
CI / sanitize (push) Failing after 14s
CI / build (push) Failing after 16s
CI / checks (push) Has been cancelled
2025-01-13 01:13:31 -05:00
WillLillis 23e0891cd5 fix(cli): improve error message for nonterminals used in token rule 2025-01-13 01:12:20 -05:00
Amaan Qureshi 24f51518d1 feat(cli)!: remove migration code for tree-sitter.json
Some checks failed
CI / checks (push) Waiting to run
CI / sanitize (push) Failing after 14s
CI / build (push) Failing after 16s
Check Bindgen Output / check-bindgen (push) Has been cancelled
Check WASM Exports / check-wasm-exports (push) Has been cancelled
2025-01-12 22:11:30 -05:00
Amaan Qureshi b26adf4265 feat(generate): add an extra field for extra nodes in node-types.json 2025-01-12 18:41:25 -05:00
Amaan Qureshi e389d54868 chore: readd skipped test 2025-01-12 13:04:10 -05:00
Amaan Qureshi 5de314833f feat(query): structurally verify supertype queries 2025-01-12 13:04:10 -05:00
Amaan Qureshi ac8bb1b777 build: bump other crates' MSRV to 1.82
Some checks failed
CI / checks (push) Waiting to run
CI / sanitize (push) Failing after 12s
CI / build (push) Failing after 14s
2025-01-12 04:47:59 -05:00
Amaan Qureshi 52ef2992aa feat(generate): properly filter out unused rules 2025-01-12 04:47:59 -05:00
Amaan Qureshi 9d9c76e693 feat(generate): explicitly disallow non-terminals in non-terminals 2025-01-12 04:47:59 -05:00
WillLillis d65a74a667 fix(cli): correct test update option 2025-01-12 01:44:58 -05:00
WillLillis ea9c85fb94 fix(cli): fix error display, considering structured data returned from
generate command
2025-01-12 01:44:51 -05:00
WillLillis dcfc95e563 fix(cli): only remove test input's trailing '\r' if running on windows
Some checks failed
CI / checks (push) Waiting to run
CI / sanitize (push) Failing after 14s
CI / build (push) Failing after 15s
platform
2025-01-11 17:52:54 -05:00
WillLillis f47319212b feat(cli): allow test subcommand to include/exclude by corpus test file names 2025-01-11 00:58:26 -05:00
WillLillis 329dcc92a1 fix(rust): adapt to a few new nightly lints 2025-01-10 22:00:23 -05:00
Amaan Qureshi d38aa596e1 fix(generate): improve error message when a duplicate token is used as the word token 2025-01-10 02:58:38 -05:00
Amaan Qureshi 4d6740980c fix(generate): improve display of token sets with escape characters 2025-01-10 02:58:38 -05:00
WillLillis c8bd78a29c feat(playground): provide colored highlight for captures in code editor 2025-01-10 00:45:08 -05:00
Guilherme Soares 207ef9796e
fix(wasm): check docker presence without arguments 2025-01-08 13:27:02 -05:00
WillLillis cbf960ff20 fix(cli): remove double print of generate errors
Some checks failed
CI / checks (push) Waiting to run
CI / sanitize (push) Failing after 13s
CI / build (push) Failing after 15s
2025-01-08 01:13:22 -05:00
Amaan Qureshi 68e707eb4f feat: support passing in a Rust regex in the grammar dsl
Some checks failed
CI / checks (push) Waiting to run
CI / sanitize (push) Failing after 14s
CI / build (push) Failing after 16s
Deploy Docs / deploy-docs (push) Has been cancelled
2025-01-06 20:52:19 -05:00
Damien Guard 4170f71dbc
feat(generate): add .exp and .lib files to gitignore template
Some checks failed
CI / checks (push) Waiting to run
CI / sanitize (push) Failing after 14s
CI / build (push) Failing after 15s
On Windows, tree-sitter-cli creates parser.exp and parser.lib so ideally we'd exclude those automatically.
2025-01-06 16:13:39 -05:00
Amaan Qureshi ba19fe31be test: ignore flaky test for now
Some checks failed
CI / checks (push) Waiting to run
CI / sanitize (push) Failing after 14s
CI / build (push) Failing after 15s
We'll re-enable post-release & when grammars are updated
2025-01-06 01:55:15 -05:00
WillLillis 867433afd7 feat(rust): use thiserror for generate crate
Some checks failed
CI / checks (push) Waiting to run
CI / sanitize (push) Failing after 13s
CI / build (push) Failing after 16s
Check Bindgen Output / check-bindgen (push) Has been cancelled
Co-authored-by: Amaan Qureshi <amaanq12@gmail.com>
2025-01-05 00:27:12 -05:00
WillLillis 5a825a0930 feat(cli): add json summary of parsing 2025-01-05 00:27:12 -05:00
Riley Bruins 19482834bd feat: add Supertype API
Introduces a new function that takes in a supertype symbol and returns
all associated subtypes. Can be used by query.c to give better errors
for invalid subtypes, as well as downstream applications like the query
LSP to give better diagnostics.
2025-01-05 00:14:09 -05:00
WillLillis 07c08432ca fix(rust): use PathBuf for --query-paths highlight option
Some checks failed
CI / checks (push) Waiting to run
CI / sanitize (push) Failing after 12s
CI / build (push) Failing after 13s
2025-01-04 04:12:24 -05:00
WillLillis 619d347f95 chore: fix doc comment 2025-01-04 04:12:24 -05:00
Amaan Qureshi efc51a596c fix(lib): don't consider unfinished captures definite when the following step is immediate
Some checks failed
CI / checks (push) Waiting to run
CI / sanitize (push) Failing after 12s
CI / build (push) Failing after 14s
When collecting captures, we were treating unfinished ones as definite
even if they had pending immediate steps that weren't yet satisfied. Now
we only mark a capture as definite if the pattern is guaranteed and
there are no pending immediate steps to check.
2025-01-04 02:03:41 -05:00
Amaan Qureshi 5f379da544 fix(lib): prevent wildcards from incorrectly marking child patterns as infallible
When a pattern appears under a wildcard parent (like "(_ (expr))"), we
were incorrectly marking it as infallible. The parent_pattern_guaranteed
flag only means the pattern will match after finding the right wildcard
parent, not that any wildcard parent will work.
2025-01-03 23:09:49 -05:00
Amaan Qureshi a7e6d01144 fix(lib): propagate last_child status to pattern alternatives in queries
Previously, when a pattern was marked as the last child in a query, its
alternatives weren't marked similarly, causing incorrect matching
behavior. Now, the `last_child` status is properly propagated through
all alternatives.
2025-01-03 21:13:29 -05:00
Will Lillis f8e77aa99d
style: move command help strings to doc comments
Some checks failed
CI / checks (push) Waiting to run
CI / sanitize (push) Failing after 13s
CI / build (push) Failing after 15s
Deploy Docs / deploy-docs (push) Has been cancelled
2025-01-03 17:29:07 -05:00
Amaan Qureshi 16be945cdd feat(cli): rework tags to use new input handler
Co-authored-by: Will Lillis <will.lillis24@gmail.com>
2025-01-03 16:16:17 -05:00
Amaan Qureshi 55fda55b9b feat(cli): rework highlight to use new input handler
Co-authored-by: Will Lillis <will.lillis24@gmail.com>
2025-01-03 16:16:17 -05:00
Amaan Qureshi 88d2f010f5 feat(cli): rework query to use new input handler
Co-authored-by: Will Lillis <will.lillis24@gmail.com>
2025-01-03 16:16:17 -05:00
Amaan Qureshi 6bad1bc6c5 feat(cli): rework parse to use new input handler
Co-authored-by: Will Lillis <will.lillis24@gmail.com>
2025-01-03 16:16:17 -05:00
Amaan Qureshi b3183363a2 feat(loader): add a way to get the cwd's language config
Co-authored-by: Will Lillis <will.lillis24@gmail.com>
2025-01-03 16:16:17 -05:00
Amaan Qureshi cc449ad965 feat(cli): make input handling agnostic
Co-authored-by: Will Lillis <will.lillis24@gmail.com>
2025-01-03 16:16:17 -05:00
Amaan Qureshi 3456330fe9 fix: update outdated links 2025-01-03 15:09:33 -05:00
dependabot[bot] 2c064039c7 build(deps): bump emscripten to 3.1.74
Some checks failed
CI / checks (push) Waiting to run
CI / sanitize (push) Failing after 14s
CI / build (push) Failing after 16s
2025-01-02 22:10:49 -06:00
Owen Shepherd 77f74a05c7
chore(generate): remove unused fields 2025-01-02 22:11:04 -05:00
WillLillis cd3d967f3c fix(cli): correct range in cst pretty printer
Some checks failed
CI / checks (push) Waiting to run
CI / sanitize (push) Failing after 15s
CI / build (push) Failing after 16s
2025-01-01 23:55:16 -05:00