Commit graph

59 commits

Author SHA1 Message Date
Will Lillis 072f68c829 feat(generate)!: derive PartialEq + Eq on GenerateError
Some checks failed
CI / checks (push) Waiting to run
CI / sanitize (push) Failing after 15s
CI / build (push) Failing after 16s
CI / check-wasm-stdlib (push) Failing after 15s
Also use `IoError` more consistently throughout the rest of the project
where its straightforward to do so.

BREAKING CHANGE: Changes public error types for config, generate, and
loader crates.
2026-09-07 17:53:09 -04:00
Will Lillis d9cb739430 fix(toml): drop align_entries = true from .taplo.toml
This key was being applied inconsistently depending on nesting level.
2026-08-29 11:10:37 +02:00
Will Lillis 7a2dd61a7c fix(xtask): preserve format padding when updating Cargo.toml files 2026-08-29 11:10:37 +02:00
Max Brunsfeld 43623ec9bf
Upgrade to latest wasmtime, improve robustness and perf of wasm-based parsing (#5847)
* build(deps): upgrade wasmtime C API to 48.0.0

Upgrade the Rust and Zig Wasmtime dependencies and enable reference values with the null GC collector.

Wasmtime 48 requires a newer Rust toolchain, whose Clippy version identifies three item helpers that can be const. Mark them const so the workspace continues to pass Clippy with warnings denied.

* feat(benchmark): support Wasm grammars

* perf(wasm): cache language function handles

* fix(wasm): improve validation and failure cleanup

Bounds-check dylink metadata parsing, require exact import and export names, and restore memory and function-table allocation offsets when language loading fails.

* fix(wasm): copy the complete supertype map
2026-08-20 21:02:11 +00:00
Max Brunsfeld 0e2af0d8d1
Allow using Tree-sitter rust lib in multi-threaded web apps compiled to wasm32-unknown-unknown (#5851)
* feat(wasm): make syntax trees sendable

* test(wasm): transfer trees across workers

* test(wasm): use JSON grammar for tree transfer

* test(wasm): edit trees across workers

* test(wasm): share dlmalloc with tree-sitter

* test(wasm): simplify worker tree exchange

* test(wasm): drive tree exchange from Rust

* test(wasm): split sendable-tree xtask

* test(wasm): generalize Rust web fixture

* test(wasm): exercise parallel Rust tree access

* fix(wasm): use Rust global allocator for C core

* test(wasm): use default Rust allocator

* feat(wasm): support external scanners in Rust web apps

* Simplify example further, add a readme

* Regenerate wasm-stdlib

* Fix wasm_stdlib check script

* Vendor the Wasm standard library subset

* Test Unicode Ruby scanner behavior in Wasm

* Make Wasm tree languages instance-aware

* Test multi-threaded use of queries in wasm32-unknown

* Refactor reference-counted language storage

* Check ABI version compat before loading rest of language

* 🎨 Remove redundant #ifdef block

* Reject unsupported Rust Wasm builds on 0.26
2026-08-14 17:05:42 -07:00
Will Lillis c279989118 fix(xtask): check for vitest rather than chai and mocha
`test::run_wasm` gated dependency installation on `node_modules/chai` and
`node_modules/mocha`. Neither is a dependency (vitest is used instead),
so the check never passed and every `cargo xtask test-wasm`
reran `npm install`.
2026-08-14 00:13:54 -04:00
Will Lillis 57ea5340c6 fix(xtask): eliminate silent errors in zig fetch 2026-06-30 19:37:55 -04:00
Will Lillis 3ed12db8f0 fix(rust): use more accurate types for various IoError structs 2026-06-13 09:35:12 -04:00
Wilco Kruijer fb348c3493 wasm: add Language.loadSync for synchronous loading
Co-authored-by: Amaan Qureshi <git@amaanq.com>
2026-04-26 18:58:50 -04:00
Will Lillis cf8719f132 fix(rust): correct various typos 2026-03-28 05:23:10 -04:00
Amaan Qureshi 67f848cac9 loader: invalidate cached wasm toolchains on version mismatch
Previously, `get_existing_tool` checked only for the existence of
binaries in the cache directory without verifying their version, meaning bumping the version files had no effect until users manually deleted the cache directories. This commit writes a `.version` marker file after downloading and checks it on subsequent runs, removing stale caches automatically when the expected version changes.
2026-02-23 22:50:20 -05:00
Amaan Qureshi b8fcf27661 build(rust): update deps 2026-02-18 00:59:34 -05:00
Amaan Qureshi ebd2a56b5d treewide: add taplo config and reformat toml files 2026-02-18 00:59:34 -05:00
Amaan Qureshi 921af361cf xtask: fix clippy warnings from edition bump 2026-02-18 00:59:34 -05:00
Will Lillis eab567a02d fix(rust): place std::env::set_var inside unsafe blocks
This is optional for the 2021 but required for 2024. The function is
unsafe either way and should be marked as such.
2026-02-02 04:29:13 -05:00
Will Lillis 55fdc50e81 feat(xtask): allow alternate branch for fixture grammars
Set tree-sitter-php fixture to `upstream_test_fixture` branch.
Also remove the `--update` flag, as it does nothing.
2026-01-24 14:18:40 -05:00
Will Lillis d736b259de wasm: build wctype.c as part of stdlib 2026-01-24 14:19:00 +01:00
Will Lillis aea4ef2338 feat(xtask): give wasm-opt the wasi-sdk treatment 2026-01-21 21:30:13 -05:00
Will Lillis aefae11c0d fix(build): define _BSD_SOURCE
System endian conversion macros are gated behind this feature flag for
older versions of GLIBC. `_BSD_SOURCE` and `_SVID_SOURCE` were
deprecated and replaced with `_DEFAULT_SOURCE` starting with GLIBC 2.19.
2026-01-12 19:41:58 -05:00
Piotr Osiewicz c0b1710f8a Add containing range APIs to query cursor
Co-authored-by: Kirill Bulatov <mail4score@gmail.com>
Co-authored-by: Max Brunsfeld <maxbrunsfeld@gmail.com>
Co-authored-by: dino <dinojoaocosta@gmail.com>
Co-authored-by: John Tur <john-tur@outlook.com>
Co-authored-by: Piotr Osiewicz <24362066+osiewicz@users.noreply.github.com>
Co-authored-by: dino <dinojoaocosta@gmail.com>
Co-authored-by: Will Lillis <will.lillis24@gmail.com>
2025-12-03 18:06:16 -05:00
WillLillis df8b62fc50 feat(xtask): bring wasi-sdk treatment up to par with the loader
The loader package's `ensure_wasi_sdk_exists` private method checks for
the wasi-sdk, fetching it if it can't be found. This logic was
re-implemented in xtask for `build-wasm-stdlib`, but without the
fetching functionality. We can have nice things in xtask too! Rather
than make this function a public member of `tree-sitter-loader`, we
just re-implement and leave a nice comment asking people to keep the
two in sync.
2025-11-27 16:06:25 -05:00
Will Lillis c7b5f89392 feat(xtask): generate JSON schema for cli TestSummary 2025-11-02 21:08:55 -05:00
WillLillis a2f2b16acb fix(xtask): require version argument for bump-version command 2025-10-24 18:13:59 -04:00
Amaan Qureshi 92efd26380 fix(loader): allow parallel compilation on windows 2025-10-01 02:57:04 -04:00
Amaan Qureshi 95ab17e444 build: define _DARWIN_C_SOURCE 2025-09-22 19:50:43 +03:00
ObserverOfTime 6dfa79013f feat(loader): respect NM env var 2025-09-20 18:08:07 -04:00
Amaan Qureshi 22553b3372 feat: support compiling to wasm32-unknown-unknown 2025-09-17 04:57:49 -04:00
Amaan Qureshi 04cfee5664 build(rust): remove unused dependencies 2025-09-16 18:57:06 -04:00
Amaan Qureshi b863b16454 fix(xtask): make building the wasm stdlib work again
Co-authored-by: Will Lillis <will.lillis24@gmail.com>
2025-09-13 22:29:47 -04:00
RedCMD ca8b944b53
build(web): fix cjs build 2025-09-13 05:51:42 -04:00
Amaan Qureshi d60ef9ad0a feat(rust)!: remove deprecated functions 2025-09-12 02:50:30 -04:00
kilo52 d1160cb820 build(cmake)!: move CMakeLists.txt to the source root 2025-09-11 01:59:37 -04:00
Amaan Qureshi 40a8678989 refactor(xtask): don't use docker to run wasi-sdk's clang
This now relies on TREE_SITTER_WASI_SDK_PATH being set
2025-09-09 03:13:35 -04:00
Amaan Qureshi e9e4316569 feat(xtask): update nix 2025-09-09 04:45:57 +02:00
Amaan Qureshi 0719bd6ffb feat(xtask): bump versions in package-lock.json 2025-09-09 04:45:57 +02:00
Amaan Qureshi 3eabba637c fix(xtask): run all tests with the --all flag 2025-09-04 04:22:32 -04:00
Amaan Qureshi f95a52df4a fix(xtask): reset fixtures when checking out new version 2025-09-04 00:59:01 -04:00
Amaan Qureshi 3b0273fc61 build: bump emscripten to 4.0.12 2025-09-04 00:59:01 -04:00
Amaan Qureshi f44cbd407f perf(xtask); check out the tag directly for fixtures 2025-09-03 02:43:32 -04:00
Amaan Qureshi 5cd6e747a0 refactor(xtask): use the git binary to perform git operations 2025-09-02 16:07:37 -04:00
ObserverOfTime 6e8ad7e5cc fix(xtask): update paths in bump-version
and remove the toml dependency
2025-09-02 14:22:01 -04:00
Amaan Qureshi f2e71ec95c fix(xtask): restore stripped sourcesContent when building the wasm module 2025-09-01 23:14:20 -04:00
Boris Verkhovskiy 9b23cd5394
feat: use compileStreaming when loading wasm parsers 2025-09-01 22:08:20 -04:00
Will Lillis 62f93e221d feat(test): pin fixture grammars to specific commits 2025-08-31 01:05:42 -04:00
ObserverOfTime 07986471b3 feat(xtask): automate edits to emscripten generated d.ts 2025-08-30 18:14:42 -04:00
ObserverOfTime 0c5cdcb161 fix(xtask): add heap symbols to emscripten exported methods 2025-08-30 18:14:42 -04:00
ObserverOfTime d188bf6352 chore: remove emscripten xtask & workflow 2025-08-30 16:52:22 -04:00
ObserverOfTime e6fe91e2e7 build(zig): support wasmtime in more archs 2025-08-30 16:47:47 +03:00
ObserverOfTime 88e323ca43 fix(xtask): commit Cargo.lock in upgrade_wasmtime
And add missing zig dependency update
2025-08-30 16:47:47 +03:00
Boris Verkhovskiy a1211d3fbd
feat(web): inline C source code in source map 2025-08-29 14:26:27 -04:00