Commit graph

509 commits

Author SHA1 Message Date
Yuta Saito bf094bd98a fix: exclude APIs that dup given file descriptors from WASI builds
WASI doesn't support `dup(2)` system call, so we cannot implement the
`print_dot_graph` and `print_dot_graphs` functions with exactly the same
semantics as in other platforms.

(cherry picked from commit 94a8262110)
2024-09-30 09:21:54 -04:00
Ron Panduwana d10308528d fix: handle more cases of editing subtrees that depend on column values
(cherry picked from commit a83b893016)
2024-09-29 20:37:24 -04:00
Amaan Qureshi 865f6595e7 fix(lib): correct descendant-for-range behavior with zero-width tokens
(cherry picked from commit 0c43988a5e)
2024-09-27 09:21:17 -04:00
Amaan Qureshi a6b248c1ad fix(lib): peek at the next sibling when iterating to find the child that contains a given descendant
This issue shows up when we have a zero-width token that is the target
descendant node, previously the previous sibling would be returned as
the child that contains the descendant, which is incorrect.

(cherry picked from commit 0a85744eba)
2024-09-17 17:22:13 +02:00
Amaan Qureshi 43e16dd75c fix(lib): backtrack to the last relevant iterator if no child was found
(cherry picked from commit 9b398c2b84)
2024-09-05 18:03:50 -04:00
Amaan Qureshi 1a3f486059 fix(lib): correct extra node creation from non-zero root-alias cursors
(cherry picked from commit ee06325f67)
2024-09-01 16:21:33 -04:00
Ryan Patterson b5e4ef6d9a
clone wasm store engine (#3542)
Some checks failed
CI / checks (push) Waiting to run
CI / sanitize (push) Failing after 12s
CI / build (push) Failing after 13s
This resolves https://github.com/tree-sitter/tree-sitter/issues/3454.

This brings the usage of wasmtime::Engine in line with how wasmtime
intends it to be used. All wasmtime functions that receive an Engine
always receive an `&Engine`, never an owned `Engine`.  They are always
responsible for cloning the reference if they need it.

This brings the usage of wasmtime::Engine in line with how TSParser
treats TSLanguages: when setting a language to the parser, the parser is
responsible for cloning the reference to the TSLanguage. It is
counterintuitive for TSParser to have different behavior when receiving
wasmtime_engine_t.

C API users also expect this behavior, see "Memory Management"
[here](https://docs.wasmtime.dev/c-api/wasm_8h.html). Talking about the
C API: without this change, failing to clone the `wasmtime_engine_t`
(which, again, is never something API users need to do in wasmtime) and
then reusing the engine in multiple TSLanguages results in a use after
free. With this change, failing to call `wasm_engine_delete` on your
owned Engine results in a memory leak. Memory leaks are safer than
use-after-free.
2024-08-22 08:01:37 -07:00
Amaan Qureshi 6dd459b4ab fix(lib): an empty root node should not precede an empty range
Some checks failed
CI / checks (push) Waiting to run
CI / sanitize (push) Failing after 12s
CI / build (push) Failing after 13s
The problem is, given an empty file, the root node of this file spans 0
bytes. As such, the logic for determining whether or not the node
precedes the range fails, and is true when it should be false.
2024-08-17 17:05:26 -04:00
Ron Panduwana 2bb20fe2fe
feat: allow external scanners to use the logger
Co-authored-by: Amaan Qureshi <amaanq12@gmail.com>
2024-08-17 14:46:28 -04:00
Ryan Patterson 779566f588
Reset language when resetting wasm store (#3495)
Some checks failed
CI / sanitize (push) Failing after 12s
CI / build (push) Failing after 12s
CI / checks (push) Has been cancelled
* Reset language when resetting wasm store

* test behavior of language copying
2024-07-31 10:30:58 -07:00
Amaan Qureshi 25c7189180 feat(lib): add ts_query_end_byte_for_pattern 2024-07-07 20:29:09 -04:00
Quentin LE DILAVREC 9610a84600
fix(lib): restrict pattern_map optimization when a wildcard step has an immediate first child
Co-authored-by: Amaan Qureshi <amaanq12@gmail.com>
2024-07-05 03:35:24 -04:00
Davis Vaughan 218a071d33 Swap sprintf() for snprintf() 2024-06-24 09:56:24 -04:00
Amaan Qureshi 7f4a57817d build(deps): bump wasmtime to v21 2024-05-29 18:57:26 -04:00
Amaan Qureshi ae35a36747 fix(lib): a null clock must have tv_nsec be 0 as well 2024-05-19 02:19:37 -04:00
Amaan Qureshi 4c083252ec fix(lib): advance the lookahead end byte by 4 when there's an invalid code point
This helps in the case where an edit was made in the middle of a code
point, but bytes 1-3 are valid, thus we could advance by at most 4 bytes
2024-04-30 20:55:43 -04:00
Amaan Qureshi 61d0395543 fix(lib): do not return field names for extras 2024-04-30 19:23:11 -04:00
Max Brunsfeld 37a919184f Position serialization buffer after language statics in wasm linear memory 2024-04-25 14:21:28 -07:00
Max Brunsfeld 78efd96bbe Crash earlier if native external scanner overflows serialization buffer 2024-04-25 13:55:47 -07:00
Max Brunsfeld e9fd4ccf0c Guard against serialization buffer overflows from wasm languages 2024-04-25 13:51:05 -07:00
Max Brunsfeld d7a23cf017 fix: Correctly load field data from wasm languages
Previously, if the last production id in a language did not have
a unique set of fields, the field maps weren't loaded correctly
from wasm.

Co-authored-by: Marshall <marshall@zed.dev>
2024-04-24 16:29:44 -07:00
vanaigr 90e0e28b95
feat: reverse iteration through node parents (#3214) 2024-04-23 15:19:57 +01:00
Max Brunsfeld 7ec40b0ab4 Implement single-char state transitions using a static array and for loop
This reduces compile time, compared to generating many individual if statements.
2024-04-12 14:40:11 -07:00
Max Brunsfeld be6e6d3708 Merge branch 'master' into simpler-large-char-set-code 2024-04-11 16:03:20 -07:00
ObserverOfTime 818cd8c291
refactor(parser): make REDUCE macro non-variadic 2024-04-11 20:47:08 +03:00
Torsten Schmits cdd00defdb
fix(lib): use correct format specifier in log message 2024-04-04 14:51:15 -04:00
ObserverOfTime 78b6067a5d fix(parser): fix variadic macro 2024-04-02 03:18:05 -04:00
Max Brunsfeld 39be6972fe Use static arrays and a fixed binary search for large char set checks 2024-03-29 23:00:48 -07:00
Max Brunsfeld 0fc92c9a7d Add strncat to wasm stdlib
Co-authored-by: Marshall <marshall@zed.dev>
2024-03-25 09:51:54 -07:00
Max Brunsfeld 8945e3a111 Fix sorting of wasm stdlib symbols 2024-03-20 18:23:44 -07:00
Amaan Qureshi 09d2b23a64 fix(lib): account for the root node of a tree cursor being an alias 2024-03-20 19:23:08 -04:00
Amaan Qureshi 24a68697ac fix(lib): properly account for aliased root nodes and root nodes with
children in `ts_subtree_string`
2024-03-20 19:23:08 -04:00
Amaan Qureshi ab485da756 fix(lib): do not consider childless nodes for ts_node_parent 2024-03-20 19:23:08 -04:00
Max Brunsfeld f415f0c6f2 Add strncmp to wasm stdlib 2024-03-20 15:49:47 -07:00
Max Brunsfeld 09b18fad5b
Merge pull request #3181 from tree-sitter/handle-wasm-oom
When loading languages via WASM, gracefully handle memory errors and leaks in external scanners
2024-03-18 13:15:06 -07:00
Max Brunsfeld 4676cd4200 Defer creation of external scanner until the beginning of parse 2024-03-17 10:20:25 -07:00
Max Brunsfeld fed9b98d6c Clean up wasm store construction, avoid hard-coding initial memory size
Retrieve the initial memory size from the stdlib's memory import type.
2024-03-17 10:20:18 -07:00
Max Brunsfeld 7a9b3076ef Handle memory errors occurring in wasm scanners
* In WASM, use a custom, simple malloc implementation that lets us
  expicitly reset the heap with a new start location.
* When a WASM call traps or errors, propagate that as a parse failure.
* Reset the WASM heap after every parse.

Co-authored-by: Conrad <conrad@zed.dev>
2024-03-17 10:19:42 -07:00
ObserverOfTime 4bbaee2f56 fix(lib): allow hiding symbols 2024-03-17 07:21:06 -04:00
Amaan Qureshi 30fd71f5ac fix(lib): avoid possible UB of calling memset on a null ptr when 0 is passed into array_grow_by 2024-03-13 00:02:53 -04:00
Amaan Qureshi 304f8b7c04 fix: don't use __declspec(dllexport) on windows 2024-03-04 13:23:06 -05:00
geekvest b43e8ce902
docs: remove duplicate the's
Signed-off-by: geekvest <cuimoman@sohu.com>
2024-03-03 10:30:21 -05:00
Daumantas Kavolis ce5afeb4b3
Fix regression in subtree_compare 2024-02-29 10:15:58 +02:00
ObserverOfTime 22c85c6139 refactor(array): use pragma GCC in clang too 2024-02-29 01:50:04 -05:00
ObserverOfTime 799833f9cf build: use c11 everywhere
And improve the makefiles
2024-02-27 15:54:38 -05:00
Amaan Qureshi e75a36232b fix(lib): include api.h so ts_set_allocator is visible 2024-02-27 09:21:17 -05:00
Max Brunsfeld 9b485924d6 Fix invalid parse stack recursive merging with mismatched error cost
Allowing this invalid merge caused an invariant to be violated
later on during parsing, when handling a later error.

Co-authored-by: Amaan Qureshi <amaanq12@gmail.com>
Co-authored-by: Marshall <marshall@zed.dev>
2024-02-26 20:16:21 -05:00
Max Brunsfeld 527848d7c0
Merge pull request #3084 from tree-sitter/no-vendored-wasmtime-headers
Remove vendored wasmtime headers
2024-02-26 15:36:10 -08:00
Max Brunsfeld dfc6d93bae Rename wasm C sources to avoid conflicts with standard wasm headers
Co-authored-by: Marshall <marshall@zed.dev>
2024-02-26 13:08:30 -08:00
Amaan Qureshi a31f084b1b fix: ignore unused variables in the array macros 2024-02-26 14:05:16 -05:00