* feat(wasm): make syntax trees sendable
* test(wasm): transfer trees across workers
* test(wasm): use JSON grammar for tree transfer
* test(wasm): edit trees across workers
* test(wasm): share dlmalloc with tree-sitter
* test(wasm): simplify worker tree exchange
* test(wasm): drive tree exchange from Rust
* test(wasm): split sendable-tree xtask
* test(wasm): generalize Rust web fixture
* test(wasm): exercise parallel Rust tree access
* fix(wasm): use Rust global allocator for C core
* test(wasm): use default Rust allocator
* feat(wasm): support external scanners in Rust web apps
* Simplify example further, add a readme
* Regenerate wasm-stdlib
* Fix wasm_stdlib check script
* Vendor the Wasm standard library subset
* Test Unicode Ruby scanner behavior in Wasm
* Make Wasm tree languages instance-aware
* Test multi-threaded use of queries in wasm32-unknown
* Refactor reference-counted language storage
* Check ABI version compat before loading rest of language
* 🎨 Remove redundant #ifdef block
* Reject unsupported Rust Wasm builds on 0.26
`Node::kind`, `Node::grammar_name`, the field name accessors,
`TreeCursor::field_name`, and the `Language` name lookups returned `&'static
str` while pointing into storage owned by the `TSLanguage`. Releasing the last
handle to a Wasm language frees that storage.
`Node` and `TreeCursor` now return `&'tree str`, which is ok because
`ts_tree_new` takes a reference to the language via `ts_language_copy` and
holds it until `ts_tree_delete`. The `Language` lookups return strings borrowed
from `&self`.
`current_symbol` and `current_symbol_name` return an `Option` (`None` when the
iterator is not positioned on a symbol). Change `iter_names`'s item from
`&'static str` to `&str`.
Track the iterator's phase so exhaustion is sticky, and gate the symbol name on
it, so `NULL` means "not positioned on a symbol".
`ts_lookahead_iterator_new` was also the only language consuming constructor
that did not `ts_language_copy`, so an iterator outliving a wasm language read
freed memory. Retain in `_new` and `_reset`, release in `_delete`.
Previously:
- `ts_lookahead_iterator__next` left a small parse state's cursor one past its
group end, so re-advancing an exhausted iterator resumed returning `true` and
walked through the rest of `ts_small_parse_table` and off the end of it.
- `ts_lookahead_iterator_current_symbol_name` returned `NULL` only when the
exhausted symbol index happened to fall outside the names table, so a grammar
with aliases returned a real but wrong name instead.
`QueryMatches` and `QueryCaptures` retain query cursor options while they are
being iterated. However, their types did not preserve the lifetime of the
options' progress callback, allowing the callback to be dropped while it
was still referenced by the cursor.
Add an explicit options lifetime to both iterator types and propagate it
through `matches_with_options` and `captures_with_options`. Use a static
options lifetime for iterators created without options.
`Parser::set_logger` boxes the logger callback into a type-erased C
pointer, but `Parser` carries no lifetime parameter. This allows for
two kinds of UB:
- Use-after-free: a logger could capture a non-`'static` borrow, let the
borrowed value drop, and then dangle the next time the parser logged.
- Data race: a logger could capture a `!Send` value such as an `Rc`. The
parser could then be moved to another thread and logged from there
while the original thread still held a clone, racing on the reference
count.
As a fix, we just require that logger is `Send + 'static`. The
alternative here is to attach a lifetime parameter to `Parser`, but this
is highly breaking for what is mostly a debugging utility. As an escape
hatch `set_logger_unchecked` is added to the public API as an `unsafe fn`
to correctly communicate the risks and invariants that must be held.
object rather than 4 separate optional function pointers.
Helps prevent misuse via mixing different allocators. Also update the
doc comment with relevant safety information.
This allows duplicating a query so that it can be modified by using
disable_capture or disable_pattern, without re-parsing the query. The
new `ts_query_copy` creates a deep copy of the entire query object.
A motivation for this is tags.scm code navigation queries. You might
want to have one version of the compiled query capture only definitions
and the other only capture references, since references are much much
more common than definitions in a typical codebase. Instead of
re-parsing and analyzing the query and then calling
`ts_query_disable_capture`, we can clone the built query all-at-once and
then disable captures / patterns.
- Cache tree byte range length to avoid redundant FFI calls per test
- Combine 5 separate XML attribute write! calls into one
- Pre-allocate format_sexp output buffer to avoid repeated growth
- One has to think about lifetimes if a type has one:
- `<&'a Node<'tree>>::language` now returns `LanguageRef<'tree>` instead of
`LanguageRef<'a>`, as it should;
- Remove explicit "outlives" requirements from `QueryMatches`, `QueryCaptures`,
and their impl blocks, because they're inferred
- Removed unnecessary `&mut` from `cst_render_node`'s `cursor` parameter
System endian conversion macros are gated behind this feature flag for
older versions of GLIBC. `_BSD_SOURCE` and `_SVID_SOURCE` were
deprecated and replaced with `_DEFAULT_SOURCE` starting with GLIBC 2.19.
Instead of returning an undocumented boolean flag, use a
core::ops::ControlFlow object. At the expense of being a bit more
verbose, this is a type that should be self-explanatory in the context
of a callback, as an indication of whether to continue processing or
stop.
Problem: When using alternations, the `#eq?` predicate does not always use the same capture name.
Solution: Iterate the left and right captured nodes more independently.
**Problem:** When encountering an invalid symbol at the beginning of the
file, the rust bindings attempt to index the character at position -1 of
the query source, which leads to an overflow and thus invalid character
index which causes a panic.
**Solution:** Bounds check the offset before performing the subtraction.
Since cargo 1.63, $CARGO_PKG_RUST_VERSION is set in the build
environment to the value of the rust-version Cargo.toml field.
This removes the need to manually invoke cargo from build.rs during a
build of the tree-sitter crate with the bindgen feature enabled.
Removing the cargo invocation also ensures the build doesn't write to
the current directory when the target directory has been redirected
elsewhere. "cargo metadata" will attempt to update Cargo.lock, which
will fail if the source tree is read-only.
