libs-code/tree-sitter.tree-sitter
1
0
Fork 0
mirror of https://github.com/tree-sitter/tree-sitter.git synced 2026-09-13 00:46:18 -04:00
Code Issues Actions 3 Packages Projects Releases Wiki Activity
5714 commits 19 branches 114 tags 51 MiB
Commit graph

152 commits

Author SHA1 Message Date
Amaan Qureshi 56870fdda2
chore: remove unneeded include 2023-07-24 00:44:45 -04:00
Amaan Qureshi 13f6ec2b0c
fix: rename shadowed variables from -Wshadow warnings and apply some useful clang-tidy warnings 2023-07-19 18:12:26 -04:00
Jille Timmermans 9a74f5cbf5 Add ts_node_language() that returns the language of the node 
Since an input might have a nested sublanguage (like Javascript inside
Vue) and symbols are per-language, we need to know which language a node
is.
 2023-07-13 11:21:18 -04:00
Max Brunsfeld a2119cb691 Add APIs for retrieving tree cursor's depth and descendant index 2023-06-12 11:50:44 -07:00
Max Brunsfeld 9dd725b4e3 Start work on exposing APIs for node descendant counts and indices 2023-06-09 08:51:47 -07:00
Andrew Hlynskyi d4d5e29c91 feat(lib): ts_query_cursor_set_max_start_depth - use 0 to reset 2023-04-17 11:16:04 +03:00
Lewis Russell 1e81a1b67f feat(lib): add ts_query_cursor_set_max_start_depth query API 
This allows configuring cursors from traversing too deep into a tree.
 2023-04-17 11:15:13 +03:00
Max Brunsfeld 837899e456 Add API for checking if a pattern in a query is non-local 2023-02-16 11:59:34 -08:00
Max Brunsfeld 97fd990822 Add --dot flag to parse subcommand, for printing tree as DOT graph 2023-02-13 12:33:34 -08:00
Max Brunsfeld 555277a102 Allow testing highlight and tag queries when testing wasm build 
Replace non-mutating `ts_parser_wasm_store` function with
`ts_parser_take_wasm_store`, which removes and returns the wasm
store, in order to facilitate single ownership.
 2023-01-23 11:46:59 -08:00
Max Brunsfeld 042e6f9d57 Restucture wasm module, return a normal TSLanguage from load_language 2022-11-15 17:14:33 -08:00
Max Brunsfeld 3f1a7f9cd4 Start work on ability to load wasm languages from native lib, via wasmtime 2022-11-15 17:14:33 -08:00
Max Brunsfeld 618146260e Add doc comments for tree included ranges getter 2022-11-08 17:47:57 -08:00
Max Brunsfeld 1848d0bc36 Add tree included ranges getter 
Co-authored-by: Nathan Sobo <nathan@zed.dev>
 2022-11-04 15:24:07 -07:00
Max Brunsfeld 477b667753 Add ts_tree_root_node_with_offset API 2022-08-18 13:48:47 -07:00
Max Brunsfeld 548c12fb88 Fix bug where patterns with top-level alternatives were not considered 'rooted' 2022-07-07 17:53:54 -07:00
Alex Pinkus 858ea5782b Fix back compat by moving primary_field_ids to the end 
Due to an oversight in #1589, I added `primary_field_ids` into the
`TSLanguage` struct in a place that wasn't the end. This is not actually
backwards compatible and causes downstream failures :(
 2022-01-17 17:23:02 -08:00
Max Brunsfeld 516fd6f6de Add --abi flag to generate command, generate version 13 by default 2022-01-17 14:50:47 -08:00
Max Brunsfeld aaf4572727
Merge pull request #1589 from alex-pinkus/deduplicate-core-ids 
Ignore duplicate states when initializing subgraphs in `ts_query__analyze_patterns`
 2022-01-17 13:54:31 -08:00
Alex Pinkus eaf9b170f1 Don't start with duplicate states in ts_query__analyze_patterns 
This change exposes a new `primary_state_ids` field on the `TSLanguage`
struct, and populates it by tracking the first encountered state with a
given `core_id`. (For posterity: the initial change just exposed
`core_id` and deduplicated within `ts_analyze_query`).

With this `primary_state_ids` field in place, the
`ts_query__analyze_patterns` function only needs to populate its
subgraphs with starting states that are _primary_, since non-primary
states behave identically to primary ones. This leads to large savings
across the board, since most states are not primary.
 2022-01-16 11:17:47 -08:00
Hendrik van Antwerpen acd3d32c36 Remove reference to strings from quantifier-only function 2022-01-11 18:33:36 +01:00
Hendrik van Antwerpen e338726cde Prefix globally visible TSquantifier values 2022-01-11 18:33:36 +01:00
Hendrik van Antwerpen a1a241b013 Expose quantifiers per pattern, instead of merging for all patterns in a query 2022-01-11 18:33:36 +01:00
Hendrik van Antwerpen 1f1a449c76 Improve capture quantifier computation 
Compute quantifiers in a bottom-up manner, which allows more precise
results for alternations, where the quantifiers are now precisly joined.
 2022-01-11 18:33:36 +01:00
Hendrik van Antwerpen 9bac066330 Deal with quantifiers appearing on capture's enclosing patterns 
- Use a proper enum type for quantifiers.
- Drop quantifiers from `TSQueryStep`, which was not used.
- Keep track of the captures introduced during a pattern parse, and
  apply the quantifier for the pattern to the captures that were
  introduced by the pattern or any sub patterns.
- Use 'quantifier' instead of 'suffix'.
 2022-01-11 18:33:36 +01:00
Hendrik van Antwerpen ae7869d1a6 Expose capture suffixes in queries 2022-01-11 18:33:36 +01:00
Max Brunsfeld 622359b400 Simplify allocation-recording in test suite using new ts_set_allocator API 2021-12-30 16:09:07 -08:00
Mingkai Dong 8e4d4ef8b9 Replace allocator struct with function pointers 2021-12-24 09:28:23 +08:00
Mingkai Dong 486ea2569d Avoid allocator from being switched more than once 2021-12-18 16:45:18 +08:00
Mingkai Dong b9b051e933 Remove ts_allocator from api.h 
Since we have exposed the function to set allocator, we don't need to
expose the global variable in `api.h`.
 2021-12-18 11:26:37 +08:00
Mingkai Dong 578bf74bf3 Add TSAllocator and ts_set_allocator in api.h 2021-12-18 09:53:58 +08:00
Max Brunsfeld 142f4b6438 Rename Query::step_is_definite -> is_pattern_guaranteed_at_step 2021-11-21 11:37:52 -08:00
Max Brunsfeld fe29bc8c19 Tweak comment formatting in api.h so bindgen doesn't create a doctest 2021-10-11 17:23:44 -07:00
Alon Hershenhorn b239583510 Fix typo in API documentation 
Fix small typo in ts_tree_cursor_current_field_id documentation.
 2021-09-10 12:28:23 -07:00
Andrew Hlynskyi 52e6c900c3 fix(lib): fix segfault on ts_query_new with incompatible grammar version, close #1318 2021-09-03 14:24:18 +03:00
Max Brunsfeld f3ea60e23f Merge branch 'master' into query-cursor-api 2021-06-02 11:51:26 -07:00
Douglas Creager 1f6eac555c query: Use uint32_t for capture list IDs 2021-06-02 13:19:52 -04:00
Douglas Creager cd96552448 query: Allow configurable match limit 
The default is now a whopping 64K matches, which "should be enough for
everyone".  You can use the new `ts_query_cursor_set_match_limit`
function to set this to a lower limit, such as the previous default of
32.
 2021-06-02 11:30:55 -04:00
Max Brunsfeld 919e9745a6 Add ts_tree_cursor_goto_first_child_for_point function 
This function (and the similar `ts_tree_cursor_goto_first_child_for_byte`)
allows you to efficiently seek the tree cursor to a given position,
exploiting the tree's internal balancing, without having to visit
all of the preceding siblings of each node.
 2021-05-27 12:30:19 -07:00
Max Brunsfeld fda35894d4 Stop matching new patterns past the end of QueryCursor's range 
This restores the original signatures of the `set_byte_range` and
`set_point_range` functions. Now, the QueryCursor will properly report
matches that intersect, but are not fully contained by its range.

Co-Authored-By: Nathan Sobo <nathan@zed.dev>
 2021-05-25 18:02:35 -07:00
Max Brunsfeld f597cc6a75 Preserve matches that contain the QueryCursor's start byte 
Co-Authored-By: Nathan Sobo <nathan@zed.dev>
Co-Authored-By: Antonio Scandurra <me@as-cii.com>
 2021-05-25 13:06:24 -07:00
Max Brunsfeld a61f25bc58 Add APIs for advancing a QueryCursor to an arbitrary position 2021-05-24 21:07:59 -07:00
Niranjan Hasabnis c31acb8fec Changing API name; Adding unit test and Rust bindings 2021-05-21 01:50:10 +00:00
Niranjan Hasabnis e2b8130f62 Merge branch 'tree-sitter:master' into nhasabni/ts_node_child_field_name 2021-05-20 16:35:52 -07:00
Markus F.X.J. Oberhumer cc519b3121 cli: Improve const-correctness of the generated parsers (part 2 of 2). 
This is a follow-up to my previous commit 1badd131f9 .

I've made this an extra patch as it requires a minor
API change in <tree_sitter/parser.h>.

This commit moves the remaining generated tables into
the read-only segment.

Before:
  $ for f in bash c cpp go html java javascript jsdoc json php python ruby rust; do \
       gcc -o $f.o -O2 -Ilib/include -c test/fixtures/grammars/$f/src/parser.c; \
    done
  $ size --totals *.o
      text    data     bss     dec     hex filename
   5353477   24472       0 5377949  520f9d (TOTALS)

After:
  $ for f in bash c cpp go html java javascript jsdoc json php python ruby rust; do \
       gcc -o $f.o -O2 -Ilib/include -c test/fixtures/grammars/$f/src/parser.c; \
    done
  $ size --totals *.o
   5378147       0       0 5378147  521063 (TOTALS)
 2021-05-19 12:49:57 +02:00
Niranjan Hasabnis dffee22736 Adding API to get field name of a TSNode 
This PR adds an API to get name of the field of TSNode's child.
It uses same set of arguments as that of ts_node_child, but returns
field name if it is found, otherwise it returns NULL.

This API is useful to implement custom printing of S-expressions such
as following:

"(binary_expression
   (binary_expression_left (identifier))
   (binary_expression_operator ("+"))
   (binary_expression_right (identifier)
)"

Currently, ts_node_string does not allow any customization for printing.
 2021-05-15 00:20:18 +00:00
Max Brunsfeld db6e1d9bdc Add ts_query_cursor_did_exceed_match_limit API 2021-03-05 15:02:49 -08:00
Max Brunsfeld 592fd8678d Organize TSLanguage fields 
Due to the breaking ABI change in #943, this is our chance
to reorder the fields in a more logical way.
 2021-03-01 10:27:22 -08:00
Max Brunsfeld c1639cc456 Add production_id_count field to Language objects 
I think this is the last additional field that's needed so
that every array member of TSLanguage has a length that
can be calculated at runtime.
 2021-02-25 16:32:05 -08:00
Max Brunsfeld 774ae5e3d5 In parse tables, store production ids as 16 bits 
Also remove the use of bitfields from the parse table format.
In all cases, bitfields were not necessary to achieve the
current binary sizes. Avoiding them makes the binaries more
portable.

There was no way to make this change backward-compatible,
so we have finally dropped support for parsers generated
with an earlier version of Tree-sitter.

At some point, when Atom adopts this version of Tree-sitter,
this change will affect Atom users who have installed packages
using third-party Tree-sitter parsers. The packages will need
to be updated to use a regenerated version of the parsers.
 2021-02-25 16:12:31 -08:00
Hansraj Das 000455ee79 Multiple typo fixes 
* This is a patch from neovim PR: https://github.com/neovim/neovim/pull/13063
 2020-10-11 13:02:40 +05:30
Max Brunsfeld b5a9adb555 Allow queries to match on supertypes 
Co-authored-by: Ayman Nadeem <aymannadeem@github.com>
 2020-09-21 12:34:48 -07:00
Max Brunsfeld 2eb04094f8 Handle aliased parent nodes in query analysis 2020-08-21 14:12:04 -07:00
Max Brunsfeld 9daec9cb22 Tweak impossible pattern error messages 2020-08-20 13:35:11 -07:00
Max Brunsfeld 4301110c12 query: Indicate specific step that's impossible 2020-08-20 13:06:38 -07:00
Max Brunsfeld cc37da7457 Query analysis: fix propagation of uncertainty from later siblings 2020-06-26 16:46:12 -07:00
Max Brunsfeld 7f955419a8 Start work on recognizing impossible patterns 2020-06-25 15:06:27 -07:00
Max Brunsfeld 4c2f36a07b Mark steps as definite on query construction 
* Add a ts_query_pattern_is_definite API, just for debugging this
* Store state_count on TSLanguage structs, to allow for scanning parse tables
 2020-06-25 15:06:27 -07:00
Riccardo Schirone 780e9cecc9 Do not use multiple unnamed structs inside of unions 2020-04-29 20:42:45 +02:00
Max Brunsfeld 7de36a33eb Remove halt_on_error API 2020-01-27 15:36:09 -08:00
Max Brunsfeld 9f63139a10 Fix error when set_included_ranges is called with an invalid range list 2020-01-17 10:31:28 -08:00
Max Brunsfeld f3747863df Add ts_query_disable_pattern API 2020-01-15 17:08:55 -08:00
Max Brunsfeld 56c620c005 Store a mapping to ensure no two symbols map to the same metadata 2019-12-05 17:21:46 -08:00
Max Brunsfeld ce633a85c6 Improve ts_language_symbol_for_name function 2019-11-15 14:21:13 -08:00
Max Brunsfeld d68dfa78b2 Add misssing docs to rust binding 2019-11-08 12:29:26 -08:00
Max Brunsfeld a7824cf59d Fix docs for ts_node_is_extra 
Fixes #446
 2019-11-05 09:49:16 -08:00
Max Brunsfeld d765332c61 Don't rely on new eof ABI in parsers unless --next-abi is passed 2019-10-31 14:32:50 -07:00
Max Brunsfeld d3b7caa565 Add a TSLexer.eof() API, use it in generated parsers 2019-10-31 14:11:52 -07:00
Max Brunsfeld 42dfba29c6 Make ts_tree_get_changed_ranges less confusing 2019-10-28 15:33:41 -07:00
Björn Linse 124ae30138 fix invalid docs for ts_tree_get_changed_ranges 2019-10-25 21:19:33 +02:00
Max Brunsfeld f490befcde Add ts_query_disable_capture API 2019-10-14 12:30:22 -07:00
Max Brunsfeld 9872a083b7 rust: Change QueryCursor::captures to expose the full match 2019-10-03 12:45:58 -07:00
Björn Linse 15e3bc7fd2 Fix some compiler warnings regarding function prototypes 2019-09-22 11:49:44 +02:00
Max Brunsfeld 82955759c0 Add an API for getting a pattern's start offset in the source code 2019-09-17 16:19:58 -07:00
Max Brunsfeld 1af85dc3f7 Remove unused APIs, expand docs for predicate API 2019-09-16 15:00:32 -07:00
Max Brunsfeld 096126d039 Allow predicates in queries, to match on nodes' text 2019-09-15 22:06:51 -07:00
Max Brunsfeld a1fec71b19 Tweak QueryCursor to allow iterating either matches or captures 
For syntax highlighting, we want to iterate over all of the captures in 
order, and don't care about grouping the captures by pattern.
 2019-09-13 15:19:04 -07:00
Max Brunsfeld c71de5bd81 Tweak query interface 
* Rename TSQueryContext -> TSQueryCursor
* Remove the permanent association between the cursor and its query. The 
cursor can now be used again for a different query.
 2019-09-11 17:33:48 -07:00
Max Brunsfeld c8c75782e3 Allow tree queries to execute within a limited range 2019-09-11 16:49:29 -07:00
Max Brunsfeld fe7c74e7aa Start work on an API for querying trees 2019-09-10 20:53:57 -07:00
Max Brunsfeld 09a2755399 Store parse states with few lookahead symbols in a more compact way 2019-08-29 15:52:23 -07:00
Max Brunsfeld e4873191d6 Refactor generated lex function to use fewer instructions per state 2019-06-20 09:57:38 -07:00
Max Brunsfeld 28011b1e60 Add ts_node_is_extra API 2019-06-19 15:58:29 -07:00
Andrey Zaytsev 7f2233ddaf Ts tree cursor copy (#363) 
* add ts_tree_cursor_copy fn

* indentation
 2019-06-16 10:52:37 -07:00
Max Brunsfeld f30485f9f4 Fix accidentally-removed functions in api.h 2019-03-29 23:05:02 -07:00
Max Brunsfeld 00968db983 Add documentation to api.h 
Fixes #166
 2019-03-29 16:50:05 -07:00
Max Brunsfeld 5035e194ff Merge branch 'master' into node-fields 2019-03-26 11:58:21 -07:00
Max Brunsfeld 82358d3f2f
Merge pull request #307 from tree-sitter/pointer-sized-cancellation-flag 
Make the cancellation flag a size_t, not a uint32_t
 2019-03-22 12:30:39 -07:00
Max Brunsfeld 5a59f19b69 Use explicit syntax for functions with no parameters 2019-03-21 16:06:06 -07:00
Max Brunsfeld 0ccb910922 Use a size_t instead of a uint32_t for cancellation flag 2019-03-21 11:26:05 -07:00
Max Brunsfeld 0ae304f582 Lib: Rework the API for cancelling a parse 
Also, use beta on CI until atomic::AtomicU32 lands in stable.
 2019-03-18 09:51:21 -07:00
Max Brunsfeld 88e3907cc0 Use QueryPerformanceFrequency as clock on windows 2019-03-14 13:42:31 -07:00
Max Brunsfeld cddb3e416d Replace operation limit API with a clock-based timeout API 2019-03-14 11:13:38 -07:00
Max Brunsfeld 56309a1c28 Generate node-fields.json file 2019-02-12 11:06:18 -08:00
Max Brunsfeld 79d90f0d3e Restore naming of alias sequence lengths 
Fields aren't stored in sequences now, so the max length
is back to being just for aliases.
 2019-02-08 16:14:18 -08:00
Max Brunsfeld d8a2c0dda2 Use a separate type for storing field map headers 2019-02-08 16:06:29 -08:00
Max Brunsfeld 1d1674811c Fully implement ts_node_child_by_field_id 2019-02-08 15:16:56 -08:00
Max Brunsfeld 18a13b457d Get basic field API working 2019-02-08 15:16:56 -08:00
Max Brunsfeld bf4e1304f8 Start work on new ref API, for giving names to nodes' children 
Co-Authored-By: Ayman Nadeem <aymannadeem@gmail.com>
 2019-02-08 15:16:56 -08:00
Max Brunsfeld 0f2347b318 Just call the C lib 'the library' everywhere, don't call it a 'runtime' 2019-01-10 15:22:39 -08:00