* feat(wasm): make syntax trees sendable
* test(wasm): transfer trees across workers
* test(wasm): use JSON grammar for tree transfer
* test(wasm): edit trees across workers
* test(wasm): share dlmalloc with tree-sitter
* test(wasm): simplify worker tree exchange
* test(wasm): drive tree exchange from Rust
* test(wasm): split sendable-tree xtask
* test(wasm): generalize Rust web fixture
* test(wasm): exercise parallel Rust tree access
* fix(wasm): use Rust global allocator for C core
* test(wasm): use default Rust allocator
* feat(wasm): support external scanners in Rust web apps
* Simplify example further, add a readme
* Regenerate wasm-stdlib
* Fix wasm_stdlib check script
* Vendor the Wasm standard library subset
* Test Unicode Ruby scanner behavior in Wasm
* Make Wasm tree languages instance-aware
* Test multi-threaded use of queries in wasm32-unknown
* Refactor reference-counted language storage
* Check ABI version compat before loading rest of language
* 🎨 Remove redundant #ifdef block
* Reject unsupported Rust Wasm builds on 0.26
The early `ts_parser__recover` dispatch for `ERROR_STATE` in
`ts_parser__advance` was dropped in `201b41cf1`. Without it, tokens
with no valid action re-enter `ts_parser__handle_error` per token
and fragment the tree instead of extending one ERROR node. Restore
it to recover the old (pre-0.25) error recovery behavior.
parser can reuse a node.
Lookahead bytes can be used to decide what a node is parsed as, so it's
resaonable to consider this as part of a node's "range" when deciding
which edits affect it.
The `LookaheadIterator` visits symbols in group order for small parse
states, which can produce `reduce_actions` in a different order than the
original linear symbol scan (pre c1379718). Since reductions are applied
sequentially and the last reduction version survives, different orderings
lead to different error recovery outcomes (e.g. losing nodes from ERROR
trees).
Sort the reduce_actions array by symbol (descending) after collection.
The array is typically 1-5 entries, so the insertion sort cost is
negligible and the full optimization speedup is preserved.
In `ts_parser__do_all_potential_reductions`, when `lookahead_symbol` is
0 (error recovery), the code scanned every symbol from 1 to
`token_count` calling ts_language_table_entry for each with most returning
empty. Replace with `LookaheadIterator` which efficiently visits only symbols
with valid actions, yielding 7-20x fewer lookups for typical grammars.
Error recovery throughput improves ~29% (JS) / ~39% (C). Valid-code
parsing is unaffected.
**Problem:** After `ts_parser_parse_with_options()`, the parser options
are still stored in the parser object, meaning that a successive call to
`ts_parser_parse()` will actually behave like
`ts_parser_parse_with_options()`, which is not obvious and can have
unintended consequences.
**Solution:** Reset to empty options state after
`ts_parser_parse_with_options()`.
* Rename corpus test functions to allow easy filtering by language
* Use usize for seed argument
* Avoid retaining useless stack versions when reductions merge
We found this problem when debugging an infinite loop that happened
during error recovery when using the Zig grammar. The large number of
unnecessary paused stack versions were preventing the correct recovery
strategy from being tried.
* Fix leaked lookahead token when reduction results in a merged stack
* Enable running PHP tests in CI
* Fix possible infinite loop during error recovery at EOF
* Account for external scanner state changes when detecting changed ranges in subtrees
**Problem:** When resetting the parser during subtree balancing, an
error is thrown:
```
parser.c:2198: ts_parser_parse: Assertion `self->finished_tree.ptr' failed.
```
**Solution:** Reset `canceled_balancing` to false in
`ts_parser_reset()`.
This allows users to bail parsing if an error was *definitely* detected
using the progress callback, as all possible stack versions have a
non-zero error cost.
Co-authored-by: Amaan Qureshi <amaanq12@gmail.com>
Currently, if a predicate is hard to match on the Rust side, a sizable
query against a very large file can take forever, and ends up hanging.
This commit adds an API function `ts_query_cursor_set_timeout_micros` to
limit how long query execution is allowed to take, thereby negating the
chance of a hang to occur.
* In WASM, use a custom, simple malloc implementation that lets us
expicitly reset the heap with a new start location.
* When a WASM call traps or errors, propagate that as a parse failure.
* Reset the WASM heap after every parse.
Co-authored-by: Conrad <conrad@zed.dev>
Replace non-mutating `ts_parser_wasm_store` function with
`ts_parser_take_wasm_store`, which removes and returns the wasm
store, in order to facilitate single ownership.
