libs-code/tree-sitter.tree-sitter
1
0
Fork 0
mirror of https://github.com/tree-sitter/tree-sitter.git synced 2026-09-10 07:26:23 -04:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
5199 commits 19 branches 114 tags 51 MiB
Commit graph

1368 commits

Author SHA1 Message Date
Max Brunsfeld 63fa0f23f2 Include 2-character ranges in array-based state transitions 2024-04-12 16:40:04 -07:00
Max Brunsfeld 056237f18b Fix sorting of transitions within a lex state 2024-04-12 16:21:57 -07:00
Max Brunsfeld a3d338781b 0.22.3 2024-04-12 15:10:49 -07:00
Max Brunsfeld 7ec40b0ab4 Implement single-char state transitions using a static array and for loop 
This reduces compile time, compared to generating many individual if statements.
 2024-04-12 14:40:11 -07:00
Max Brunsfeld 3210c7e21f Avoid using a large character set constant when it doesn't reduce code size 2024-04-12 12:01:23 -07:00
Max Brunsfeld 3498498449 Merge branch 'master' into simpler-large-char-set-code 2024-04-12 10:03:46 -07:00
Max Brunsfeld 15fe07a20e Clean up code generation for lexer state transitions 2024-04-12 09:02:33 -07:00
Amaan Qureshi dcb7acede4 build: update emscripten version 2024-04-11 22:35:43 -04:00
Amaan Qureshi 96d18408a3 refactor(js): misc fixes & tidying 2024-04-11 22:35:43 -04:00
Amaan Qureshi abc7910381 refactor(rust): misc fixes & tidying 2024-04-11 22:35:43 -04:00
Amaan Qureshi 5825e24d56 style: wrap comments 2024-04-11 22:35:43 -04:00
Amaan Qureshi b35efa8f33 style: format imports 2024-04-11 22:35:43 -04:00
Amaan Qureshi a48054f1ae
fix: wrong flag check in build.rs 2024-04-11 22:31:10 -04:00
Max Brunsfeld 1f0707e1ac Fix clippy warnings 2024-04-11 16:29:59 -07:00
Max Brunsfeld b8701fcf18 Check EOF when checking a large char set that contains the null character 2024-04-11 16:19:21 -07:00
Max Brunsfeld be6e6d3708 Merge branch 'master' into simpler-large-char-set-code 2024-04-11 16:03:20 -07:00
Max Brunsfeld 18ea74ee12
Merge pull request #3280 from ObserverOfTime/reduce 
refactor(parser): make REDUCE macro non-variadic
 2024-04-11 15:39:28 -07:00
Ron Panduwana c051a90ac5 fix: insert "tree-sitter" section in current directory's package.json if it exists 
Co-authored-by: ObserverOfTime <chronobserver@disroot.org>
 2024-04-11 17:09:35 -04:00
ObserverOfTime 63babea301 fix: proper function prototypes 2024-04-11 16:28:21 -04:00
ObserverOfTime 50dfd409f6 fix(bindings): generate parser.c if missing 2024-04-11 16:28:21 -04:00
ObserverOfTime 7830877f63 fix(bindings): add utf-8 flag to python & node 2024-04-11 16:27:50 -04:00
ObserverOfTime 818cd8c291
refactor(parser): make REDUCE macro non-variadic 2024-04-11 20:47:08 +03:00
Amaan Qureshi 8bfe4e1f6b fix: add back build-wasm temporarily 2024-04-11 10:48:30 -04:00
Ron Panduwana 33de5ef849 fix: default output directory for build --wasm should use current_dir 2024-04-11 10:48:30 -04:00
Amaan Qureshi 895c7680e7 fix(windows): add /utf-8 flag for parsers using unicode symbols 2024-04-11 09:44:51 -04:00
Sebastian Lackner 5dc62cc828
fix(cli): fix mismatched parenthesis when accounting for && 2024-04-11 09:01:56 -04:00
Max Brunsfeld 3d088888f5 Derive large character sets from lex states for individual tokens 2024-04-10 16:53:39 -07:00
Max Brunsfeld be34bc9430 Identify large char sets for lexer using NFA transitions 2024-04-09 17:53:37 -07:00
Amaan Qureshi 21c06101aa fix(bindings): remove required platforms for swift 2024-04-08 17:56:05 -04:00
Amaan Qureshi 0d2dea9e0e fix(test): allow colons in test names 2024-04-08 17:56:05 -04:00
Amaan Qureshi a9172e0caa fix: add a semicolon after SKIP macros 2024-04-08 17:56:05 -04:00
Amaan Qureshi abed43a169 chore: clippy fix 2024-04-08 17:56:05 -04:00
ObserverOfTime 78b6067a5d fix(parser): fix variadic macro 2024-04-02 03:18:05 -04:00
Max Brunsfeld 39be6972fe Use static arrays and a fixed binary search for large char set checks 2024-03-29 23:00:48 -07:00
Amaan Qureshi 62cd13a3de test: add regression test for node parent + string bug 2024-03-20 19:23:08 -04:00
Max Brunsfeld 09b18fad5b
Merge pull request #3181 from tree-sitter/handle-wasm-oom 
When loading languages via WASM, gracefully handle memory errors and leaks in external scanners
 2024-03-18 13:15:06 -07:00
Max Brunsfeld fed9b98d6c Clean up wasm store construction, avoid hard-coding initial memory size 
Retrieve the initial memory size from the stdlib's memory import type.
 2024-03-17 10:20:18 -07:00
Max Brunsfeld 7a9b3076ef Handle memory errors occurring in wasm scanners 
* In WASM, use a custom, simple malloc implementation that lets us
  expicitly reset the heap with a new start location.
* When a WASM call traps or errors, propagate that as a parse failure.
* Reset the WASM heap after every parse.

Co-authored-by: Conrad <conrad@zed.dev>
 2024-03-17 10:19:42 -07:00
Amaan Qureshi fc15f62133
0.22.2 2024-03-17 07:31:58 -04:00
ObserverOfTime 4bbaee2f56 fix(lib): allow hiding symbols 2024-03-17 07:21:06 -04:00
ObserverOfTime 4b12eab927 feat(lib): implement Display for Node 2024-03-17 06:12:28 -04:00
Amaan Qureshi e8dbe011d3 test: fix header writes 2024-03-17 05:36:30 -04:00
Amaan Qureshi 754aa62e89 chore: turbofish styling 2024-03-17 05:36:30 -04:00
Amaan Qureshi 99b93d83a1 feat(cli)!: add a separate build command to compile parsers 
This allows users to build parsers without having to run `test` or
`parse` to invoke the compilation process, and allows them to output the
object file to wherever they like. The `build-wasm` command was merged
into this by just specifying the `--wasm` flag.
 2024-03-17 05:36:30 -04:00
ObserverOfTime 1ff40f5571 fix(lib): makefile installation 2024-03-12 18:14:58 -04:00
Amaan Qureshi 81d90c7cd8 fix(lib): makefile installation on macOS 2024-03-12 15:53:31 -04:00
Max Brunsfeld 2ff746742f
Merge pull request #3172 from tree-sitter/remove-which-crate 
Remove dependency on which crate
 2024-03-12 12:38:48 -07:00
Max Brunsfeld 99a720c968 Remove dependency on which crate 2024-03-12 12:19:27 -07:00
Amaan Qureshi ad07fa8a9e feat(bindings/rust): expose Parser::included_ranges 2024-03-12 02:29:35 -04:00
Amaan Qureshi 1c38d34dea
0.22.1 2024-03-10 17:59:12 -04:00
Amaan Qureshi cdc9fbb240
fix: cli build script behavior on release 2024-03-10 17:53:35 -04:00
Amaan Qureshi 0a5a564ea7
0.22.0 2024-03-10 17:15:08 -04:00
ObserverOfTime e9b3f65ceb fix(bindings): fix template oversights 2024-03-10 15:48:23 -04:00
Christopher Durham 78cc77e7b2 refactor!: remove redundant escape regex & curly brace regex preprocessing 
The regex-syntax crate now natively supports literal escapes for all
ASCII characters except those in [0-9A-Za-z<>].
 2024-03-10 14:47:11 -04:00
ObserverOfTime b60b2489aa feat(bindings): remove dsl types file 2024-03-10 10:06:17 -04:00
ObserverOfTime 072865e450 feat(bindings): add prebuildify to node 
Co-Authored-By: Amaan Qureshi <amaanq12@gmail.com>
 2024-03-10 10:06:17 -04:00
ObserverOfTime 69cf13bc05 feat(bindings): add peerDependencies for npm 2024-03-10 10:06:17 -04:00
Amaan Qureshi 17e50c4c57 fix: allow the regex v flag 2024-03-10 09:50:29 -04:00
Amaan Qureshi 2e56e94648 build: remove unused deps, bump deps, and bump MSRV to 1.74.1 2024-03-10 09:47:39 -04:00
Amaan Qureshi 4b578a3649 fix(generate): camel case name in Cargo.toml description 2024-03-06 19:53:51 -05:00
ObserverOfTime 2d652c90a7 fix(cli): only output the sources with --no-bindings 2024-03-06 13:56:40 -05:00
Amaan Qureshi 92675117a6 fix(generate): extern allocator functions for the template don't need to be "exported" 2024-03-05 11:19:06 -05:00
Amaan Qureshi 54a31069af fix: parsers should export the language function on windows 2024-03-05 11:19:06 -05:00
Amaan Qureshi 304f8b7c04 fix: don't use __declspec(dllexport) on windows 2024-03-04 13:23:06 -05:00
Amaan Qureshi f8c41f74f8 fix(bindings): insert types after main if it exists 2024-03-03 11:17:30 -05:00
Amaan Qureshi 57d01335c0 chore: delete binding_files.rs 2024-03-03 10:46:37 -05:00
Amaan Qureshi 27f52266ac feat: replace nan with node-addon-api and conditionally print logs 2024-03-03 10:46:37 -05:00
Segev Finer 30bb44f48a feat: add typings for the node bindings 
Fixes #3072
 2024-03-03 10:46:37 -05:00
geekvest b43e8ce902
docs: remove duplicate the's 
Signed-off-by: geekvest <cuimoman@sohu.com>
 2024-03-03 10:30:21 -05:00
Segev Finer 1dd35fe0e0
feat(node): type tag the language 
Co-authored-by: ObserverOfTime <chronobserver@disroot.org>
 2024-03-02 20:59:20 -05:00
ObserverOfTime 924c2b4425 chore(bindings): ignore more artifacts 2024-03-02 20:13:39 -05:00
ObserverOfTime f7c6dcadaa feat(bindings): add more make targets 2024-03-02 20:13:39 -05:00
Amaan Qureshi c66ed6a141
fix: and binding.gyp... 2024-03-01 11:02:33 -05:00
Amaan Qureshi 274d0cda0a fix: binding.cc overwrite should replace PARSER_NAME 2024-03-01 10:59:25 -05:00
Amaan Qureshi 6b2879144f build(deps): bump deps & lockfile 2024-02-29 02:17:30 -05:00
Amaan Qureshi 5a4765ca01 style: clippy lints 2024-02-29 02:17:30 -05:00
Amaan Qureshi 60a935139b refactor!: remove top-level corpus dir for tests 
It's confusing to have tests in two different top-level directories when working between different grammars, and most of them use `test/corpus` which is more fitting, so time to go.
 2024-02-29 02:17:30 -05:00
Amaan Qureshi 4ff33e81bb fix(assertions): case shouldn't matter for comment node detection 2024-02-29 02:17:30 -05:00
Amaan Qureshi 8c4861c186 feat(test): add attributes for corpus tests 2024-02-29 02:17:30 -05:00
ObserverOfTime b3c14c9295 build(bindings): make everything c11 2024-02-29 01:50:04 -05:00
ObserverOfTime b4b2d9cecc refactor: remove extern/const where possible 2024-02-29 01:50:04 -05:00
Amaan Qureshi d92a7a9690 test: add test case for parse stack merging with incorrect error cost bug 2024-02-28 08:51:28 -05:00
Segev Finer e87cf7ef15
refactor(bindings)!: convert node bindings to NAPI 
Co-authored-by: Boris Verkhovskiy <boris.verk@gmail.com>
Co-authored-by: Amaan Qureshi <amaanq12@gmail.com>
Co-authored-by: ObserverOfTime <chronobserver@disroot.org>
 2024-02-27 17:06:25 -05:00
Amaan Qureshi eb332ffa4d style(bindings): consistent wording 2024-02-27 16:07:38 -05:00
ObserverOfTime 502dacf220 refactor: rename TS_REUSE_ALLOCATOR flag 
TREE_SITTER_REUSE_ALLOCATOR is more consistent
 2024-02-27 15:54:38 -05:00
ObserverOfTime 46520eda58 build(bindings): metadata improvements 2024-02-27 15:54:38 -05:00
Amaan Qureshi f7c13406c9 build: add uninstall command 2024-02-27 15:54:38 -05:00
ObserverOfTime 799833f9cf build: use c11 everywhere 
And improve the makefiles
 2024-02-27 15:54:38 -05:00
ObserverOfTime 412a6a774e refactor(bindings): remove npmignore 
Replace with package.json files
 2024-02-27 06:22:00 -05:00
Amaan Qureshi eb35502f06 test: don't use TS_REUSE_ALLOCATOR on Darwin systems 
For some reason, the linker seems to behave a bit differently with `-exported_symbols_list` on macOS vs other operating systems, so we'll disable this for now
 2024-02-26 20:15:36 -05:00
Amaan Qureshi 8ae3870b51 fix(generate): add .npmignore, populate Swift's exclude list 2024-02-26 14:30:48 -05:00
ObserverOfTime 721eca2394 fix(bindings): editorconfig and setup.py fixes 2024-02-26 12:00:16 -05:00
Amaan Qureshi f1eecf9786 feat: expose the allocator and array header files for external scanners 2024-02-26 09:43:13 -05:00
Amaan Qureshi 9e5bf6591f feat: improve language bindings 
Co-authored-by: ObserverOfTime <chronobserver@disroot.org>
 2024-02-25 19:06:01 -05:00
Max Brunsfeld 29d3583bdf Use workspace dependencies for internal crates 2024-02-25 11:14:29 -08:00
Amaan Qureshi 068e29c265 build: unify crate versions via workspace 2024-02-24 18:30:04 -05:00
Amaan Qureshi 32c23b6c90 fix: wrap || comparison in parenthesis when && is used 2024-02-24 01:35:40 -05:00
Max Brunsfeld 1f751bbd5f
Merge pull request #2841 from segevfiner/patch-1 
Make Node.js language bindings context aware
 2024-02-23 11:23:11 -08:00
Amaan Qureshi f6b46440c2
fix: bump config 2024-02-21 15:26:25 -05:00
Amaan Qureshi 4408c1570e
fix: publish in the right order now 2024-02-21 15:24:47 -05:00
Amaan Qureshi a5b7c2a584
fix: publish 0.21.0 to registries 2024-02-21 15:11:37 -05:00
Max Brunsfeld 1c55abb530 0.21.0 2024-02-21 10:32:29 -08:00
Will Lillis f894a5350c
feat(cli): add optional config-path argument 2024-02-21 12:02:12 -05:00
Amaan Qureshi 58a4fcc792 chore: get rid of github_issue_test file 2024-02-21 12:01:25 -05:00
Amaan Qureshi 38efefd8bd style: cleaner cast 2024-02-21 12:01:25 -05:00
Amaan Qureshi 62578b8c6e feat(loader): add more commonly used default parser directories 2024-02-21 12:01:25 -05:00
Amaan Qureshi 167855b289 fix(test): edge case when parsing UNEXPECTED/MISSING nodes with an indentation level greater than 0 2024-02-21 12:01:25 -05:00
Amaan Qureshi a1a3903c10 fix(test): allow writing updates to tests without erroneous nodes instead of denying all of them if a single error is found 2024-02-21 12:01:25 -05:00
Amaan Qureshi b97208704b
fix(cli): don't use long for grammar_path 
Co-authored-by: buckynbrocko <77247638+buckynbrocko@users.noreply.github.com>
 2024-02-19 16:05:08 -05:00
Amaan Qureshi b40839cd72 style: prefer turbofish syntax where possible 2024-02-19 16:00:50 -05:00
Amaan Qureshi fd91404ab0 style: tidying 2024-02-19 16:00:50 -05:00
Amaan Qureshi d95fcc83b9 fix: remove redundant imports 2024-02-19 06:36:48 -05:00
Amaan Qureshi f526be8061 test: update html tests 2024-02-19 06:36:48 -05:00
Amaan Qureshi 5b299eafe5
fix: don't throw an error if the user uses map in the grammar 2024-02-18 00:26:14 -05:00
Amaan Qureshi da1f890752 chore: error out when multiple arguments are passed to token/token.immediate 2024-02-17 01:26:44 -05:00
Amaan Qureshi 8dd65ccbc0 refactor: &PathBuf -> &Path 2024-02-17 00:45:28 -05:00
Amaan Qureshi 03c5a8540d feat: better error info when a scanner is missing required symbols 2024-02-17 00:45:28 -05:00
Amaan Qureshi 6c0643f295
fix: always push the default files if there's no externals 2024-02-16 22:42:40 -05:00
Amaan Qureshi ef1b714e08 fix(cli): don't update tests automatically if parse errors are detected 2024-02-16 20:23:39 -05:00
Amaan Qureshi e32a7f3998 chore: clippy lints 2024-02-16 16:32:07 -05:00
Amaan Qureshi 4342efd57e feat: allow specifying an external scanner's files 2024-02-16 16:31:56 -05:00
Amaan Qureshi b6c75ccec1 chore: update relevant rust tests 2024-02-16 12:34:12 -05:00
Amaan Qureshi e996c32108 refactor!: remove the apply-all-captures flag, make last-wins precedence the default 2024-02-16 12:34:12 -05:00
Nickolay 6895b7a1e1
Add some documentation to the playground page 
The minimal UI of the Playground could benefit from some documentation to make it easier for the newer users to understand what's going on. Also added a link to the new documentation from the local playground.

Closes https://github.com/tree-sitter/tree-sitter/issues/1305
 2024-02-15 16:24:21 +01:00
Amaan Qureshi 74812ced1b chore: deprecate C++ scanners 
C++ has been a headache to deal with throughout the ecosystem and for
several downstream projects. It is difficult to get working with WASM,
and induces potential issues with compilation on Windows. It has been
proven that writing scanners in C is a much better alternative, and is
the recommended way to write scanners now. C++ support will likely be
removed in 0.21.0
 2024-02-14 15:43:53 -05:00
Amaan Qureshi d80d101e34 build: move common Cargo.toml keys into the workspace and inherit them 2024-02-14 14:56:23 -05:00
Amaan Qureshi d989b26587 chore: print out full compiler arguments ran when it fails 2024-02-14 13:19:06 -05:00
Amaan Qureshi 73f56bffa9 fix: sexp format edge case with quoted closed parenthesis 2024-02-13 18:31:28 -05:00
Amaan Qureshi 4303ab99c9 fix: properly handle Query.matches when filtering out results 2024-02-13 16:40:05 -05:00
Amaan Qureshi 7e0dd7b9c1 feat(cli): add an optional grammar-path argument for the playground 2024-02-13 16:28:54 -05:00
Amin Yahyaabadi 51c147053e
feat: error out if an empty string is in the extras array 
This prevents never-ending loops in the parser
 2024-02-13 15:00:48 -05:00
Amaan Qureshi 5ea0dbf77a chore: some more clippy lints 2024-02-13 03:33:07 -05:00
Amaan Qureshi a07f988905 refactor: extract regex check into a function and lower its precedence 2024-02-13 03:33:07 -05:00
Bedis Nbiba 7dd096c5f7
feat: implement first-line-regex 2024-02-13 01:51:41 -05:00
Amaan Qureshi 21f25a5305 feat: improve error message for files with an unknown grammar path 2024-02-12 17:17:37 -05:00
Amaan Qureshi 48deb309db chore(cli): warn users when a query path needed for a subcommand isn't specified in a grammar's package.json 2024-02-12 17:17:37 -05:00
Amaan Qureshi 3da79ba2b6 fix: update schema for regex flags 2024-02-12 15:39:14 -05:00
Amaan Qureshi 39df8e2833 chore(test): use different languages for async tests 2024-02-12 06:30:36 -05:00
Amaan Qureshi 7233c2f26f feat: use lockfiles to dedup recompilation 2024-02-12 06:30:36 -05:00
Amaan Qureshi e6d67ecde0 chore(wasm): use SIDE_MODULE=2 to silence warning 2024-02-11 20:18:30 -05:00
Yingdong Yang 0b7fc43828
fix(cli): installation via a HTTP tunnel proxy 2024-02-11 03:58:14 -05:00
Amaan Qureshi 4149ed4149 chore: provide a CLI flag to open log.html 
On macOS, this was done by default regardless of what the user wants.
This was also not done on Windows or Linux. Instead, we now provide a
`--open-log` flag to open the log file in the default browser, and it
works on all platforms.
 2024-02-11 02:28:34 -05:00
Amaan Qureshi ce5f5fffc1 chore(generate): dedup and warn about duplicate or invalid rules 2024-02-11 01:28:31 -05:00
Amaan Qureshi 78ccc39ca9
fix: properly error out when the word property is an invalid rule 2024-02-11 00:37:23 -05:00
Amaan Qureshi cca814afd2 chore: add java and tsx to corpus tests 2024-02-10 18:25:35 -05:00
Amaan Qureshi 3d68ca4583 fix: inherited grammar generation 2024-02-10 18:25:35 -05:00
Amaan Qureshi e4894ac8b4 chore(cli): use spawn to display emcc's stdout and stderr 
Co-authored-by: Nbiba Bedis <bedisnbiba@gmail.com>
 2024-02-09 14:17:34 +01:00
Amaan Qureshi dcb34b25ec fix: prettify xml output and add node position info 2024-02-08 18:09:10 -05:00
Amaan Qureshi a969fc428c chore: move dependencies into the workspace and inherit from there 2024-02-08 15:24:09 -05:00
Amaan Qureshi 7980669267 feat: support dot graph output on windows 2024-02-08 15:21:47 -05:00