libs-code/tree-sitter.tree-sitter
1
0
Fork 0
mirror of https://github.com/tree-sitter/tree-sitter.git synced 2026-09-13 17:06:26 -04:00
Code Issues Actions 4 Packages Projects Releases Wiki Activity
5714 commits 19 branches 114 tags 52 MiB
Commit graph

547 commits

Author SHA1 Message Date
Amaan Qureshi 75e1bcf70a
refactor!: rename proc_macro to avoid conflicts when building std with -Z build-std 2023-07-24 00:44:44 -04:00
Amaan Qureshi c521e9c18e
chore: improve error message in some spots loading grammar.json 2023-07-24 00:44:44 -04:00
Samuel Moelius d9d4116105
Add test_consecutive_zero_or_modifiers 2023-07-19 03:27:43 -04:00
Max Brunsfeld 8310fd423e Update node unit test to reflect C grammar change 2023-07-18 17:58:19 -07:00
Max Brunsfeld 40f7b2ec97 Fix parsing of queries that start with repetitions followed by alternatives 2023-07-18 17:57:52 -07:00
Amaan Qureshi e77405841e
refactor: make a subdirectory for the scratch dir depending on the machine's features 2023-07-18 20:14:21 -04:00
Amaan Qureshi 76f5f81aee
chore: update test 2023-07-18 19:29:40 -04:00
Amaan Qureshi c436897662
test: bump start_time limit for sparc64 2023-07-18 19:07:51 -04:00
Amaan Qureshi 975f3f4686 refactor!: use AtomicUsize for AllocationRecorder for portability 2023-07-16 05:14:11 -04:00
Andrew Hlynskyi 0558f1e0bf test: add retries for all timeout tests 2023-07-16 05:05:36 +03:00
Max Brunsfeld 356f68293a Fix false positive query match bug, introduced in #2085 2023-07-10 16:12:59 -04:00
Amaan Qureshi 3504aa3260 fix(deps): bump proc-macro2 as an unstable feature became stabilized 2023-07-10 16:12:19 -04:00
Amaan Qureshi b192200008 fix: update tests from python grammar changes 2023-07-10 14:40:18 -04:00
Max Brunsfeld 4922602369
Merge pull request #2316 from tree-sitter/node-descendant-index 
Add APIs for retrieving descendant counts, seeking to nodes by their descendant index
 2023-06-12 15:13:43 -07:00
Max Brunsfeld 3375527a89 Fix bug in ts_tree_cursor_goto_descendant 2023-06-12 14:45:30 -07:00
Max Brunsfeld 9dd725b4e3 Start work on exposing APIs for node descendant counts and indices 2023-06-09 08:51:47 -07:00
Samuel Moelius 67f4f2328c Add test_query_error_does_not_oob 2023-05-19 21:31:01 -04:00
Andrew Hlynskyi da269ac350 test: add a test case to demonstrate the #2107 issue 2023-05-13 19:40:49 +03:00
Andrew Hlynskyi ceac2380ed chore(test): reorganize test helpers 2023-05-12 19:29:51 +03:00
Andrew Hlynskyi 85a588b5b0 Fix tests, adapt to tree-sitter-json 40a81c01a40ac48744e0c8ccabbaba1920441199 2023-04-22 12:08:50 +03:00
Andrew Hlynskyi d4d5e29c91 feat(lib): ts_query_cursor_set_max_start_depth - use 0 to reset 2023-04-17 11:16:04 +03:00
Lewis Russell 1e81a1b67f feat(lib): add ts_query_cursor_set_max_start_depth query API 
This allows configuring cursors from traversing too deep into a tree.
 2023-04-17 11:15:13 +03:00
Andrew Hlynskyi 0d326824d2 test: add a reproducing test for #2162 2023-04-06 04:49:39 +03:00
Andrew Hlynskyi c1f784498f chore: nit picking in internal proc_macro crate 2023-04-04 13:51:10 +03:00
Andrew Hlynskyi da894afef5 cargo: specify minimum supported Rust version 2023-04-04 03:42:55 +03:00
Andrew Hlynskyi ddb0af9509 test: use random SEED numbers 
This is needed to omit occurrences of the same seed in a sequence of
following seeds due to the reason of that two initial seed are very
close if based on unix epoch seconds.
 2023-03-16 15:17:21 +02:00
Andrew Hlynskyi 3aeef44eb6 test: run all corpus tests with 10 retries 2023-03-16 15:17:21 +02:00
Andrew Hlynskyi 588549c093 test: run test_parsing_with_a_timeout with 10 retries 2023-03-16 15:17:21 +02:00
Andrew Hlynskyi 62f8c431ae test: add retry and test_with_seed proc macros 2023-03-16 15:17:21 +02:00
Andrew Hlynskyi ca152a93dd ci: fix tests 2023-03-14 20:06:31 +02:00
Max Brunsfeld 8dcf851739 Add unit test for querying within a range of a long top-level repetition 2023-02-16 12:03:51 -08:00
Max Brunsfeld 837899e456 Add API for checking if a pattern in a query is non-local 2023-02-16 11:59:34 -08:00
Max Brunsfeld 40703f110c Fix bug in maintenance of query cursor's tree depth 2023-02-16 11:59:34 -08:00
Max Brunsfeld bd63fb2a0d Tweak query tests 2023-02-16 11:59:34 -08:00
Max Brunsfeld d47713ee4a Integrate WASM compilation into the CLI's Loader 2022-11-15 17:14:33 -08:00
Max Brunsfeld 8e3dc7cd7a Add tests that randomly edit files with disjoint included ranges 2022-11-14 16:04:37 -08:00
Max Brunsfeld d07f864815 Fix parse error when reusing a node at the end of an included range 2022-11-11 16:34:57 -08:00
Max Brunsfeld da6e24de17 Fix adjustment of trees' included ranges on edits 
Previously, when an included range started or ended *inside* of
an edit, that range did not get updated correctly.

Co-authored-by: Mikayla Maki <mikayla@zed.dev>
 2022-11-07 16:52:19 -08:00
Max Brunsfeld f0177f216e Use library-configured allocator to free intermediate included ranges 2022-11-07 10:48:32 -08:00
Max Brunsfeld 1b86ccecc8
Merge pull request #1845 from tree-sitter/root-node-with-offset 
Add API for applying a positional offset when accessing a tree's nodes
 2022-08-25 10:51:20 -07:00
Max Brunsfeld 6b87326470
Merge pull request #1787 from kianmeng/fix-typos 
Fix typos
 2022-08-25 10:25:39 -07:00
Max Brunsfeld 53ed4cf037 Tolerate tree edits whose old range extends beyond the end of the tree 2022-08-25 10:20:40 -07:00
Max Brunsfeld 477b667753 Add ts_tree_root_node_with_offset API 2022-08-18 13:48:47 -07:00
Max Brunsfeld 548c12fb88 Fix bug where patterns with top-level alternatives were not considered 'rooted' 2022-07-07 17:53:54 -07:00
Max Brunsfeld 1401767689 query: Don't attempt to match top-level sibling patterns directly in ERROR nodes 
Co-authored-by: Keith Simmons <keith@zed.dev>
 2022-07-07 15:27:00 -07:00
Kian-Meng Ang b8552ec6c4 Fix typos 2022-06-28 19:57:42 +08:00
Max Brunsfeld 8b5ff2ec8f Loosen randomized changed range assertions on newline characters 
For now, don't error if the scope changes for a newline character.
 2022-06-25 17:13:46 -07:00
Max Brunsfeld 58b719541b Fix failure to match queries with wildcard at root with range restrictions 2022-06-22 15:54:06 -07:00
Aleksei Bavshin b27b4665ac
test: fix incorrect uses of i8 instead of c_char 
Fixes build on aarch64, ppc64le and other platforms that have c_char
defined as u8.
 2022-03-23 00:00:26 -07:00
Max Brunsfeld 7170ec7c96 Improve randomized testing setup 
* Allow iterations to be specified via an env var
* Randomly decide the edit count, with a maximum
  specified via an env var.
* Instead of separate env vars for starting seed + trial, just accept a seed
* Remove some noisy output
 2022-03-02 17:12:25 -08:00
Max Brunsfeld a494d6aa28
Merge pull request #1668 from tree-sitter/remember-lookahead-bytes-on-error-detection 
Remember lookahead bytes on error detection
 2022-02-26 14:59:40 -08:00
Max Brunsfeld c697ebfb27 Add explicit unit test for error detection lookahead bug 2022-02-26 14:33:09 -08:00
Max Brunsfeld 5eb0a3090f
Merge pull request #1547 from the-mikedavis/md-test-tags 
test tags queries in 'tree-sitter test'
 2022-02-24 15:22:15 -08:00
Max Brunsfeld cb4317ba8e Change goto_first_child_for_{byte,point} to compare nodes' ranges inclusively 
Co-Authored-By: Antonio Scandurra <me@as-cii.com>
 2022-02-04 12:38:33 -08:00
Michael Davis 02abc2a063
add test for removals in eager query matches 2022-01-18 20:54:55 -06:00
Hendrik van Antwerpen ec9b00e5c6 Handle multiple top-level alternations correctly 2022-01-11 18:33:36 +01:00
Hendrik van Antwerpen a1a241b013 Expose quantifiers per pattern, instead of merging for all patterns in a query 2022-01-11 18:33:36 +01:00
Hendrik van Antwerpen 1f1a449c76 Improve capture quantifier computation 
Compute quantifiers in a bottom-up manner, which allows more precise
results for alternations, where the quantifiers are now precisly joined.
 2022-01-11 18:33:36 +01:00
Max Brunsfeld 622359b400 Simplify allocation-recording in test suite using new ts_set_allocator API 2021-12-30 16:09:07 -08:00
Michael Davis e7f0cc4e24
add python test case for tag testing 2021-12-18 11:07:46 -06:00
Michael Davis 47dc88b647
add tags config fixture test helper 2021-12-18 11:07:18 -06:00
Max Brunsfeld 26dac9b2dd Fix query bugs revealed by randomized tests 
* Fix bugs related to named wildcard patterns vs regular wildcard patterns.
* Fix handling of extra nodes during query analysis. Previously, the
expected child_index was updated incorrectly after an extra node,
leading to false "impossible pattern" errors.
* Refine logic for avoiding unnecessary state-splitting due to fallible steps.
Compute *two* different analysis results related to step fallibility:
  * `root_pattern_guaranteed` which, like before, summarizes whether the
    entire pattern is guaranteed to match once this step is reached.
  * `parent_pattern_guaranteed` - which just indicates whether the
    immediate parent pattern is guaranteed. This is now used when
    deciding whether it's necessary to split a match state.
 2021-11-21 12:02:58 -08:00
Max Brunsfeld 142f4b6438 Rename Query::step_is_definite -> is_pattern_guaranteed_at_step 2021-11-21 11:37:52 -08:00
Max Brunsfeld f69c4861c3 Add a randomized test for query matching 2021-11-21 11:29:36 -08:00
Max Brunsfeld 2f9b7ac465 Fix logging of seed in randomized test 2021-11-19 12:16:54 -08:00
Max Brunsfeld d05c665863 Convert some of the fixture grammars from JSON to JS 
These tests are easier to write and maintain if the grammars are just JS,
like grammars normally are. It doesn't slow the tests down significantly
to shell out to `node` for each of these grammars.
 2021-10-22 18:47:23 -06:00
Max Brunsfeld ddb12dc0c6 query: Return error on unclosed tree pattern in alternation 
Fixes #1436
 2021-10-12 09:20:43 -07:00
Max Brunsfeld e78413832b Restructure test suite's allocation recording so that tests can run in parallel 2021-10-11 17:24:37 -07:00
Andrew Helwer e8e4dafa92 Fixed cargo fmt suggestions 2021-09-28 18:31:04 -04:00
Andrew Helwer 3d1f760e31 Simplified corpus test output comparisons 2021-09-28 18:27:25 -04:00
Andrew Helwer 0c3fb6e6a2 Fixed cargo fmt issues 2021-09-24 22:49:25 -04:00
Andrew Helwer 0dc1bd806f Fixed some test failures due to CRLF line endings 2021-09-24 22:42:38 -04:00
Max Brunsfeld 8fa875b1a4 Fix possible infinite loop when running syntax highlighting tests 
Fixes #1347
 2021-08-29 15:04:23 -07:00
Andrew Hlynskyi 7f538170bf fix(parser): count rows in the debug log from 0 2021-07-15 11:47:14 +03:00
Max Brunsfeld 9cb732859f Update unit test to reflect HTML grammar change 2021-07-12 12:21:10 -07:00
Cameron Forbis 1ea797fbd2 run fmt 2021-06-29 13:59:21 -07:00
Cameron Forbis ae5340901e address code review comments 2021-06-29 13:49:43 -07:00
Cameron Forbis 9182ebef86 update set_included_ranges to modify extent if the current position is at the very beginning of the included range 2021-06-17 16:42:25 -07:00
Edgar 9feca80b08
Make HighlightConfiguration::configure use into iterator instead of a Vec<String> 2021-06-16 19:14:45 +02:00
Douglas Creager e841fcfa1b cli: Extract CLI configuration into separate crate 
This patch adds the `tree-sitter-config` crate, which manages
tree-sitter's configuration file.  This new setup allows different
components to define their own serializable configuration types, instead
of having to create a single monolithic configuration type.  But the
configuration itself is still stored in a single JSON file.

Before, the default location for the configuration file was
`~/.tree-sitter/config.json`.  This patch updates the default location
to follow the XDG Base Directory spec (or other relevant platform-
specific spec).  So on Linux, for instance, the new default location is
`~/.config/tree-sitter/config.json`.  We will look in the new location
_first_, and fall back on reading from the legacy location if we can't
find anything.
 2021-06-10 10:21:22 -04:00
Douglas Creager 66c30648c2 rust: Extract runtime language detection into separate crate 
This patch adds a new `tree-sitter-loader` crate, which holds the CLI's
logic for finding and building local grammar definitions at runtime.
This allows other command-line tools to use this logic too!
 2021-06-09 17:42:30 -04:00
Douglas Creager d2d01e77e3 cli: Use anyhow and thiserror for errors 
This patch updates the CLI to use anyhow and thiserror for error
management.  The main feature that our custom `Error` type was providing
was a _list_ of messages, which would allow us to annotate "lower-level"
errors with more contextual information.  This is exactly what's
provided by anyhow's `Context` trait.

(This is setup work for a future PR that will pull the `config` and
`loader` modules out into separate crates; by using `anyhow` we wouldn't
have to deal with a circular dependency between with the new crates.)
 2021-06-09 16:17:23 -04:00
Max Brunsfeld f3ea60e23f Merge branch 'master' into query-cursor-api 2021-06-02 11:51:26 -07:00
Douglas Creager cd96552448 query: Allow configurable match limit 
The default is now a whopping 64K matches, which "should be enough for
everyone".  You can use the new `ts_query_cursor_set_match_limit`
function to set this to a lower limit, such as the previous default of
32.
 2021-06-02 11:30:55 -04:00
Max Brunsfeld d72771a19f Make ::set_{byte,point}_range methods take a Range 
Co-Authored-By: Nathan Sobo <nathan@zed.dev>
 2021-05-28 14:21:11 -07:00
Max Brunsfeld 851f55afce Report non-rooted matches that intersect cursor's range restriction 
Co-Authored-By: Nathan Sobo <nathan@zed.dev>
 2021-05-28 11:58:38 -07:00
Max Brunsfeld 919e9745a6 Add ts_tree_cursor_goto_first_child_for_point function 
This function (and the similar `ts_tree_cursor_goto_first_child_for_byte`)
allows you to efficiently seek the tree cursor to a given position,
exploiting the tree's internal balancing, without having to visit
all of the preceding siblings of each node.
 2021-05-27 12:30:19 -07:00
Max Brunsfeld fda35894d4 Stop matching new patterns past the end of QueryCursor's range 
This restores the original signatures of the `set_byte_range` and
`set_point_range` functions. Now, the QueryCursor will properly report
matches that intersect, but are not fully contained by its range.

Co-Authored-By: Nathan Sobo <nathan@zed.dev>
 2021-05-25 18:02:35 -07:00
Max Brunsfeld f597cc6a75 Preserve matches that contain the QueryCursor's start byte 
Co-Authored-By: Nathan Sobo <nathan@zed.dev>
Co-Authored-By: Antonio Scandurra <me@as-cii.com>
 2021-05-25 13:06:24 -07:00
Max Brunsfeld a61f25bc58 Add APIs for advancing a QueryCursor to an arbitrary position 2021-05-24 21:07:59 -07:00
Douglas Creager 78010722a4 query: Allow unlimited pending matches 
Well, not completely unlimited — we're still using a 16-bit counter to
keep track of them.  But we longer have a static maximum of 32 pending
matches when executing a query.
 2021-05-24 11:02:58 -04:00
Max Brunsfeld 8c3d1466ec Allow QueryCursor's text callback to return an iterator 2021-05-23 21:05:26 -07:00
Niranjan Hasabnis c0e2b43d34 Fixing Rust formatting issue 2021-05-21 15:43:10 +00:00
Niranjan Hasabnis c31acb8fec Changing API name; Adding unit test and Rust bindings 2021-05-21 01:50:10 +00:00
Andrew Hlynskyi 3c0152a331 chore(fmt): Apply 'cargo fmt' to the whole code base 2021-05-19 23:21:43 +03:00
Andrew Hlynskyi b57266cbb8 test(binding_rust): Use some redundancy to trigger more clear comliler error on the master 2021-04-22 21:00:58 +03:00
Andrew Hlynskyi 2d9a2a1855 test(binding_rust): test query captures/matches lifetime is separate from nodes lifetime 2021-04-22 21:00:58 +03:00
Max Brunsfeld 72143b17ff Fix handling of queries with field names at the roots of patterns 
Fixes #1018
 2021-03-27 09:36:13 -07:00
Max Brunsfeld 89e1157a29 Fix handling of repetitions in query analysis 
Fixes #1007
 2021-03-19 11:00:31 -07:00
Andrew Hlynskyi 192f51324e Added more test for byte and point ranges 2021-03-17 19:11:20 +02:00
Max Brunsfeld bc0ef5f373 Add negated field patterns to queries 2021-03-12 14:38:02 -08:00
Max Brunsfeld a40045a419 When editing, properly invalidate trees that depend on get_column 2021-03-11 14:46:13 -08:00
Max Brunsfeld 9dc88061d5 query: Fix last child operator with multiple children 
Fixes #806
 2021-03-09 13:40:15 -08:00
Max Brunsfeld 6ba11d7778 Fix query analysis with top-level non-terminal extras 
Fixes https://github.com/tree-sitter/tree-sitter/issues/968#issuecomment-792314281
 2021-03-08 09:33:35 -08:00
Max Brunsfeld 5a2534ac4a Fix crash on unterminated alternation in query 
Fixes #968
 2021-03-07 08:48:03 -08:00
Max Brunsfeld f18c36ca62 Fix handling of extra nodes in query analysis 
Fixes #967
 2021-03-06 14:35:41 -08:00
Max Brunsfeld db6e1d9bdc Add ts_query_cursor_did_exceed_match_limit API 2021-03-05 15:02:49 -08:00
Max Brunsfeld 047d33eecf Remove bad logic for detecting later siblings with the same field 
Co-authored-by: Douglas Creager <dcreager@dcreager.net>
 2021-03-05 14:50:16 -08:00
hvithrafn 05f79f0f90 Update dependencies; fix breakage 2021-02-26 10:55:20 -07:00
Max Brunsfeld c25fa9910e Fix query match failure when indefinite steps have no captures 
Fixes #937
 2021-02-25 21:41:49 -08:00
Max Brunsfeld e49a56ea4c
Merge pull request #939 from tree-sitter/partial-order-precedence 
Allow precedences to be specified using strings and a partial ordering relation
 2021-02-25 13:16:01 -08:00
Max Brunsfeld 344797c110 Implement named precedence comparison 2021-02-24 16:02:56 -08:00
Douglas Creager a29c8d9264 Move allocation tracking into lib crate 
We have several test cases defined in the `cli` crate that depend on the
`lib` crate's `allocation-tracking` feature.  The implementation of the
actual allocation tracker used to live in the `cli` crate, close to the
test cases that use it.  The `allocation-tracking` feature in the `lib`
crate was just used to tell the tree-sitter implementation to expect
that the allocation tracker exists, and to use it.

That pattern meant that we had a circular dependency: `cli` depends on
`lib`, but `lib` required some code that was implemented in `cli`.
That, in turn, caused linker errors — but only when compiling in certain
configurations! [1]

This patch moves all of the allocation tracking implementation into the
`lib` crate, gated on the existing `allocation-tracking` feature, which
fixes the circular dependency.

Note that this patch does **not** fix the fact that feature unification
causes the `lib` crate to be built with the `allocation-tracking`
feature enabled, even though it's not a default.  Fixing that depends on
the forthcoming version 2 feature resolver [2], or using the `dev_dep`
workaround [3] in the meantime.

[1] https://github.com/tree-sitter/tree-sitter/issues/919
[2] https://doc.rust-lang.org/nightly/cargo/reference/features.html#feature-resolver-version-2
[3] https://github.com/tree-sitter/tree-sitter/issues/919#issuecomment-777107086
 2021-02-23 09:16:37 -05:00
Max Brunsfeld 73e252e39b Add unit test for child_by_field_name w/ hidden nodes 2021-02-10 16:14:24 -08:00
Max Brunsfeld 6dbe6a3a90 Merge branch 'master' into actions-ci 2021-02-05 10:19:05 -08:00
Max Brunsfeld 391fc8c340 Update unit tests to expect ruby grammar change 2020-12-20 19:34:45 -08:00
Max Brunsfeld 026231e93d Merge branch 'master' into HEAD 2020-12-03 09:44:33 -08:00
Max Brunsfeld b661050a61 Simplify setup for enabling/disabling allocation recording in the C lib 2020-12-02 15:35:13 -08:00
Patrick Thomson 0b4661e401 Really fix the tests. 2020-11-23 11:41:16 -05:00
Patrick Thomson c1a8985712 Merge remote-tracking branch 'origin/master' into query-testy 2020-11-23 11:10:53 -05:00
Max Brunsfeld b267f90e64 Update unit tests to reflect python and ruby grammar changes 2020-11-16 10:51:08 -08:00
Patrick Thomson c9c886d971 Merge remote-tracking branch 'origin/master' into query-testy 2020-11-10 16:16:53 -05:00
Max Brunsfeld 99cd283e39 query: Fix detection of repeated field names 
Fixes #790
 2020-11-02 14:07:39 -08:00
Max Brunsfeld c2c63baf5b query: Fix escape sequence parsing in anonymous node patterns 
Fixes #776
Fixes #760
 2020-10-28 13:55:13 -07:00
Max Brunsfeld a2d760e426 Ensure nodes are aliased consistently within syntax error nodes 
Co-Authored-By: Rick Winfrey <rewinfrey@github.com>
 2020-10-27 15:46:09 -07:00
Patrick Thomson 1aee60a7c0 propitiate the tests 2020-10-26 14:35:18 -04:00
Max Brunsfeld 0a46033391 Remove duplication of LossyUtf8 helper 2020-10-14 11:35:50 -07:00
Max Brunsfeld 857a9ed07b query: Handle captured wildcard nodes at the root of patterns 2020-10-08 12:34:08 -07:00
Max Brunsfeld d1c95193c1 query: Fix invalid use of slice::from_raw_parts 2020-10-05 12:08:53 -07:00
Max Brunsfeld ba239ce4ab Make query error line numbers consistently display 1-indexed 2020-09-24 15:03:51 -07:00
Max Brunsfeld 518916f221 Return correct path and line in query errors from the CLI 2020-09-24 13:47:27 -07:00
Max Brunsfeld 297e2bcb28 static query analysis: Fix handling of fields in hidden nodes 2020-09-23 16:55:48 -07:00
Max Brunsfeld cb343cad5e Avoid reusing the root node during incremental parsing 
Fixes #712
 2020-09-23 12:59:27 -07:00
Max Brunsfeld 21c3bbc4b4 Account for supertypes during query analysis 2020-09-23 10:55:31 -07:00
Max Brunsfeld 8835dfda99 Fix test for supertypes in queries 2020-09-21 13:11:54 -07:00
Max Brunsfeld b5a9adb555 Allow queries to match on supertypes 
Co-authored-by: Ayman Nadeem <aymannadeem@github.com>
 2020-09-21 12:34:48 -07:00
Max Brunsfeld 36a8821f3e Fix behavior of the last child operator in tree queries 2020-09-02 12:10:04 -07:00
Max Brunsfeld 4aba684d66 Control recursion depth explicitly during query analysis 2020-08-24 16:55:56 -07:00
Max Brunsfeld 2eb04094f8 Handle aliased parent nodes in query analysis 2020-08-21 14:12:04 -07:00
Max Brunsfeld 456b1f6771 Fix handling of alternations and optional nodes in query analysis 2020-08-20 16:28:54 -07:00
Max Brunsfeld 9daec9cb22 Tweak impossible pattern error messages 2020-08-20 13:35:11 -07:00
Max Brunsfeld 4301110c12 query: Indicate specific step that's impossible 2020-08-20 13:06:38 -07:00
Max Brunsfeld bd42729a41 query: Avoid early-returning captures due to predicates 2020-08-18 13:01:45 -07:00
Max Brunsfeld 604f9e8148 query: Assign is_definite correctly for steps within nested sub-patterns 2020-08-18 10:55:03 -07:00
Max Brunsfeld 91fc9f5399 Use is_definite flag in ts_query_cursor_next_capture 2020-08-17 16:50:59 -07:00
Max Brunsfeld 228a9e28e1 Add tests for impossible queries 2020-08-17 13:27:17 -07:00
Max Brunsfeld c3f9b2b377 Fix query analysis bugs found in ruby tags query 2020-08-17 09:57:06 -07:00
Max Brunsfeld 1ea29053e1 Merge branch 'master' into query-pattern-is-definite 2020-08-14 09:31:55 -07:00
Patrick Thomson 94ab884ee4 Add a test. 2020-08-05 12:16:09 -04:00
Patrick Thomson 5c86a9c654 Fix the tests 2020-08-05 11:52:07 -04:00
Max Brunsfeld af655547e5 Fix handling of queries with many patterns with leading repetitions 2020-07-31 12:47:58 -07:00
Max Brunsfeld e89a19a158 tags: Add @ignore capture 2020-07-29 15:33:48 -07:00
Max Brunsfeld 1ae5cbc851 query: Handle #not-match? in rust, wasm bindings 2020-07-24 12:15:23 -07:00
Max Brunsfeld de2b71d465 Fix query bug when max permutations are exceeded 2020-07-23 16:05:50 -07:00
Tuấn-Anh Nguyễn 740d864e67 Add '.' as a valid start of a predicate, in addition to '#' 
See https://github.com/ubolonton/emacs-tree-sitter/issues/38
 2020-07-19 15:46:39 +07:00
Max Brunsfeld 4535efce69 query: Prevent dropping of matches when exceeding range maximum 
Fixes #685
 2020-07-17 09:39:06 -07:00
Max Brunsfeld fef72fb434 Merge branch 'tags-utf16-columns' into tagging-improvements 2020-07-09 10:40:46 -07:00
Max Brunsfeld 255cf0a9cf tags: Add utf16 column ranges to tags 
Also, ensure that line ranges contain only valid UTF8.

Co-authored-by: Tim Clem <tclem@github.com>
Co-authored-by: Beka Valentine <bekavalentine@github.com>
 2020-07-08 16:13:58 -07:00
Max Brunsfeld d614c14c2c tags: Make spans refer to name, not entire def/ref 
Co-authored-by: Tim Clem <tclem@github.com>
Co-authored-by: Beka Valentine <bekavalentine@github.com>
 2020-07-08 12:37:01 -07:00
Max Brunsfeld 0bf2450b4a Always enforce stack version limit during reductions 
Fixes #669
 2020-07-06 15:58:33 -07:00
Max Brunsfeld cc37da7457 Query analysis: fix propagation of uncertainty from later siblings 2020-06-26 16:46:12 -07:00
Max Brunsfeld 19baa5fd5e Clean up and document query analysis code 2020-06-25 17:56:43 -07:00
Max Brunsfeld 891de051e2 Fix population of subgraph nodes when analyzing queries 2020-06-25 15:06:27 -07:00
Max Brunsfeld 7f955419a8 Start work on recognizing impossible patterns 2020-06-25 15:06:27 -07:00
Max Brunsfeld 4c2f36a07b Mark steps as definite on query construction 
* Add a ts_query_pattern_is_definite API, just for debugging this
* Store state_count on TSLanguage structs, to allow for scanning parse tables
 2020-06-25 15:06:27 -07:00
Max Brunsfeld deeeb67a3b
query: Fix handling of alternations under field names (#661) 2020-06-24 14:20:56 -07:00
Timothy Clem f166947abb Test updates, definition/reference prefix is now required 2020-06-18 15:05:27 -07:00
Timothy Clem 75724698f0 Fix up tests 2020-06-18 14:42:41 -07:00
Timothy Clem 3bcb1f8c94 Assert line trimming 2020-06-18 10:48:33 -07:00
Timothy Clem 80f5c52259 Tests compile 2020-06-16 17:19:35 -07:00
Max Brunsfeld 0e5ff14976
Requery the parse table when breaking down the parse stack on invalid lookahead (#636) 
* Requery parse table after breaking down parse stack due to invalid lookahead

* Include Ruby parser in randomized test suite

Ruby and PHP are our only two languages that use non-terminal extras.
Adding Ruby uncovered some bugs.

* Print edited source code when running parse --edit w/ debug flag

* Recompute lookahead when breaking down stack on invalid lookahead

* Fix stack summary leak when there are two discontinuities on a stack version
 2020-06-04 13:40:04 -07:00
Max Brunsfeld 47d607da8d
Add alternative syntax in queries (#630) 
* Add alternative syntax in queries

* Add tests and tweak error handling for alternatives in queries
 2020-06-01 13:23:07 -07:00
Max Brunsfeld 462c86903f
Improve tree queries' ability to handle large numbers of nested matches (#624) 
* query: Acquire capture lists lazily, allow more concurrent states

* Fix some static analysis warnings
 2020-05-18 13:40:24 -07:00
Алексей Пастухов 38d32c018b
add Rust into languages list for corpus tests (#619) 2020-05-15 16:02:39 -07:00
Max Brunsfeld 85c998d572 Change the wildcard syntax in tree queries 
1. Use '_' instead of '*'.
2. Add '*' as a postfix operator for zero-or-more repetitions

Signed-off-by: Patrick Thomson <patrickt@github.com>
 2020-05-11 13:04:04 -07:00
Max Brunsfeld 40262483a9 Change query syntax for predicates 
Signed-off-by: Patrick Thomson <patrickt@github.com>
 2020-05-11 12:35:51 -07:00
Max Brunsfeld b14f564550 Add a test for nested optionals in queries 2020-05-11 10:42:48 -07:00
Max Brunsfeld 9c0535cea6 Fix logic for aborting failed matches 2020-05-08 14:15:25 -07:00
Max Brunsfeld b47c170c75 Query: fix bugs and add tests for top-level and nested repetitions 2020-05-08 12:10:01 -07:00
Max Brunsfeld 3ad71625dd Fix query bugs, expand and clean up query tests 2020-05-07 14:22:15 -07:00
Max Brunsfeld 3456a21f0d Start work on restructuring query implementation to deal w/ optionals and repeats better 2020-05-07 12:41:25 -07:00
Max Brunsfeld 35f82ce301 Fix incorrect parent values after call to child_by_field_name 
Refs tree-sitter/node-tree-sitter#61
Refs tree-sitter/tree-sitter-javascript#127
 2020-04-03 11:21:51 -07:00
Max Brunsfeld 8eac81b8df Merge branch 'master' into tags 2020-03-25 12:49:15 -07:00
Max Brunsfeld 783c087aec tags: Handle cancellation 2020-03-25 11:26:52 -07:00
Max Brunsfeld a003e5f6bd generate: Avoid duplicate string tokens in unique symbol map 2020-03-20 11:35:11 -07:00
Max Brunsfeld ae1c51051a Fix tag order in JS tags test 2020-03-20 11:34:33 -07:00
Max Brunsfeld aedab72afa tags: Start work on handling local variables for ruby support 2020-03-20 10:13:03 -07:00
Max Brunsfeld 651fa38c93 Add unit test for tagging via C API. Fix docs handling 2020-03-18 10:40:15 -07:00
Max Brunsfeld 94a60b8e13 tags: Start adapting Tag struct for use in C API 2020-03-17 11:19:30 -07:00
Max Brunsfeld d0325579ad Use Arc to avoid use-after-free in threaded cancellation unit test 
Fixes #579
 2020-03-16 14:46:05 -07:00
Max Brunsfeld 94bbf14d0e tags: Add test where no comments are adjacent to definition 2020-03-16 14:28:58 -07:00
Max Brunsfeld 65f2874b9e query: Optimize handling of patterns with a wildcard at the root 
Avoid adding and removing states for these patterns on every node in the tree
by just skipping the wildcard step of the matching process
 2020-03-16 14:02:31 -07:00
Max Brunsfeld b5483c67ab query: allow repetition operator to be used on non-terminal nodes 2020-03-13 16:12:39 -07:00
Max Brunsfeld b5f2ed83fe tags: Implement select-adjacent! predicate 2020-03-13 13:02:56 -07:00
Max Brunsfeld 0457736766 rust: add handling of arbitrary predicate operators 2020-03-13 13:02:34 -07:00
Max Brunsfeld 6e2df06dc2 Start proving out tags support for JavaScript 2020-03-12 16:33:52 -07:00
Max Brunsfeld 6f636a0357 query: Add postfix '+' operator for token repetition 
Co-Authored-By: Patrick Thomson <patrickt@users.noreply.github.com>
 2020-03-12 15:10:58 -07:00
Max Brunsfeld 05c1d44e80 Merge branch 'master' into tags 2020-03-11 13:15:26 -07:00
Max Brunsfeld e3aad995f6 query: Fix handling of patterns with wildcards at the root 2020-03-11 13:14:16 -07:00
Patrick Thomson d798bd6bd9 Slice out the line associated with a tag. 2020-03-10 20:39:04 -04:00
Max Brunsfeld 0e02ead0de Update tags test to reflect new handling of escapes in queries 2020-03-10 15:53:42 -07:00
Max Brunsfeld 4531130b44 Merge branch 'master' into tags 2020-03-10 15:50:27 -07:00
Max Brunsfeld 741eed01b7 query: Handle escape sequences and escaped quotes in string literals 2020-03-10 15:50:06 -07:00
Max Brunsfeld fc4d5c3a33 🔥 Dead test code 2020-03-10 12:23:13 -07:00
Max Brunsfeld 17cc38678c Get generate_tags with the new iterator API 2020-03-10 12:05:09 -07:00
Max Brunsfeld 0eb162c685 wip: converting generate_tags to return an iterator 2020-03-10 11:45:31 -07:00
Max Brunsfeld 157258d881 tags: Implement strip regex for docs processing 
Co-Authored-By: Patrick Thomson <patrickt@users.noreply.github.com>
 2020-03-10 10:43:23 -07:00
Max Brunsfeld 7f4828254f Fix criteria for detecting when an aborted parse is resuming 2020-03-09 11:30:08 -07:00
Patrick Thomson 00dcc1eaa6 Need to use expression_statement here. 2020-03-06 17:48:55 -05:00
Max Brunsfeld 680a9e0531 wip 2020-03-06 13:24:03 -08:00
Max Brunsfeld a3f0087b11 Start work on tagging unit test 
Co-Authored-By: Patrick Thomson <patrickt@users.noreply.github.com>
 2020-03-05 13:04:49 -08:00
Max Brunsfeld ee46218a73 Fix incremental parsing problem with non-terminal extras 
Also add PHP grammar as a fixture to test against.
 2020-03-02 14:17:12 -08:00
Max Brunsfeld 570b83e2b2 query: Add immediate child operator 2020-02-19 11:47:52 -08:00
Max Brunsfeld 950a89a525 query: Differentiate between wildcard '*' and named wildcard '(*)' 2020-02-19 09:42:29 -08:00
Max Brunsfeld de8e3ee188 query: Allow multiple captures on a single node 2020-02-11 16:02:32 -08:00
Max Brunsfeld e23f518915 highlight: add built-in support for carriage-return highlight 2020-01-28 14:47:21 -08:00
Max Brunsfeld d06407aca2 Update highlight test for JSDoc grammar changes 2020-01-27 13:08:30 -08:00
Max Brunsfeld 3f109a3cb5 highlight: Fix logic for handling empty injections with no highlights 2020-01-27 12:32:37 -08:00
Max Brunsfeld 9f63139a10 Fix error when set_included_ranges is called with an invalid range list 2020-01-17 10:31:28 -08:00
Max Brunsfeld 9dfd03e79a highlight: Sipmlify injection API w/ new injection.combined property 2020-01-16 12:43:31 -08:00
Max Brunsfeld f3747863df Add ts_query_disable_pattern API 2020-01-15 17:08:55 -08:00
Max Brunsfeld c2e253a4c8 Update unit test to reflect javascript query change 2019-12-10 13:24:36 -08:00
Max Brunsfeld 0cb2ef1082 Fix code paths that still conflated null characters with EOF 2019-12-06 15:29:03 -08:00
Max Brunsfeld d426f46087 Add unit test for symbols respecting simple aliases 2019-12-06 12:47:27 -08:00
Max Brunsfeld d6c7b243a7
Merge pull request #499 from tree-sitter/highlight-test 
Add a system for testing syntax highlighting queries
 2019-12-05 15:55:24 -08:00
Max Brunsfeld 4b9feccd13 Run syntax highlighting tests as part of the main test command 
Also, allow `corpus` to be in the `test/corpus` directory, and expect 
highlighting tests to be in the `test/highlight` directory.
 2019-12-05 15:28:16 -08:00
Max Brunsfeld 6dbfbaed07 Start work on a system for testing syntax highlighting 2019-12-05 09:45:54 -08:00
Max Brunsfeld c957a5b8e7 Add test for handling of too many states in query captures 2019-11-22 11:54:32 -08:00
Max Brunsfeld 1909ae2cee binding_rust: Improve node children APIs 
Closes #480
 2019-11-08 14:20:10 -08:00
Max Brunsfeld 39ceff1908 Restore json corpus tests, which accidentally weren't running 2019-10-31 14:32:50 -07:00
Max Brunsfeld a62b7a70f3 Lexer: track EOF state without relying on null character as lookahead 2019-10-31 14:11:52 -07:00
Max Brunsfeld e14e285a10 cli: Check queries when running tree-sitter test 2019-10-18 14:44:16 -07:00
Max Brunsfeld 64c6cf4473 Implicitly reset parser's state if language is changed after a timeout 2019-10-18 11:28:59 -07:00
Max Brunsfeld 084406148b Tweak highlight name matching semantics, add doc comments 2019-10-17 12:03:34 -07:00
Max Brunsfeld b79295e1a0 🔥 Everything related to property sheets 2019-10-17 11:00:31 -07:00
Max Brunsfeld fa43ce01a6 Allow queries to capture ERROR nodes 2019-10-16 11:54:32 -07:00
Max Brunsfeld 060e00463d Implement include-children directive in injection queries 2019-10-14 17:38:42 -07:00
Max Brunsfeld b3809274f0 Load highlight queries correctly in highlight unit tests 2019-10-14 17:24:16 -07:00
Max Brunsfeld 324c259cbb rust: Create readable messages for query syntax errors 2019-10-14 12:30:22 -07:00
Max Brunsfeld 4c17af3ecd Allow queries with no patterns 2019-10-14 12:30:22 -07:00
Max Brunsfeld f4903578f8 Start reimplementing highlight crate with tree queries 2019-10-14 12:30:22 -07:00
Max Brunsfeld 02b6397aa3 Add test for advancing query cursor after has already finished 2019-10-03 12:55:10 -07:00
Max Brunsfeld 9872a083b7 rust: Change QueryCursor::captures to expose the full match 2019-10-03 12:45:58 -07:00
Max Brunsfeld 27149902f8 Handle is?, is-not?, and set! predicate functions in queries 2019-09-25 10:21:20 -07:00
Max Brunsfeld b15e90bd26 Handle set! predicate function in queries 2019-09-24 11:54:24 -07:00
Max Brunsfeld ff9a2c1f53 Make queries work in languages with simple aliases 2019-09-24 11:54:24 -07:00
Max Brunsfeld 070f11b8bf Use ptr::NonNull in Rust bindings 2019-09-23 17:00:52 -07:00
Max Brunsfeld 598e4e4cf4 Add a unit test to verify captures are sorted by start and end position 2019-09-20 10:12:45 -07:00
Max Brunsfeld a6b6a681ec Fix a bug that prevented early termination of query matches 2019-09-18 16:13:10 -07:00
Max Brunsfeld d9b8bae629 rust: Include pattern indices when iterating query captures 2019-09-18 14:22:55 -07:00