emacs/gnu.emacs
1
0
Fork 0
mirror of https://git.savannah.gnu.org/git/emacs.git synced 2026-09-10 07:46:51 -04:00
Code Issues Actions Packages Projects Releases Wiki Activity

Update tree-sitter manual for embed level

Browse source 
* doc/lispref/parsing.texi (Using Parser): Mention embed level.
(Multiple Languages): Explain embed level, and local vs
non-local parsers.
This commit is contained in:
Yuan Fu 2026-07-12 19:26:36 -07:00
parent 0f6d977a88
commit 6b670907fa
No known key found for this signature in database
GPG key ID: 56E19BC57664A442
2 changed files with 93 additions and 39 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

131
doc/lispref/parsing.texi
View file

@ -455,20 +455,19 @@ is non-@code{nil}, this function always creates a new parser.
@code{nil}. Different parsers can have the same tag.
@end defun
Given a parser, we can query information about it.
@defun treesit-parser-buffer parser
This function returns the buffer associated with @var{parser}.
@end defun
@defun treesit-parser-language parser
This function returns the language used by @var{parser}.
@end defun
@defun treesit-parser-p object
This function checks if @var{object} is a tree-sitter parser, and
returns non-@code{nil} if it is, and @code{nil} otherwise.
@end defun
@findex treesit-parser-buffer
@findex treesit-parser-language
@findex treesit-parser-embed-level
@findex treesit-parser-p
Given a parser, we can query information about it:
@code{treesit-parser-buffer} returns the buffer associated with a
parser, and @code{treesit-parser-language} returns the language used by
the parser. A parser's tag can be retrieved by
@code{treesit-parser-tag}. Each parser also optionally carries a embed
level that can be set by @code{treesit-parser-set-embed-level} and
retrieved by @code{treesit-parser-embed-level} (for embed level,
@pxref{parser-embed-level}). Finally, @code{treesit-parser-p} tells you
whether an object is a tree-sitter parser.
There is no need to explicitly parse a buffer, because parsing is done
automatically and lazily. A parser only parses when a Lisp program
@ -1808,10 +1807,11 @@ Again, @var{thing} can be either a symbol or a predicate.
If @var{parser} is non-@code{nil}, only use that parser's parse tree.
Otherwise try each parser covering point, from the most specific
(deepest-embedded) to the least specific. If there are multiple parsers with
the same embed level at @var{position}, which parser is tried first is
undefined. If @var{parser} is a language symbol, the function limits
the parsers it tries to the ones for that language.
(deepest-embedded) to the least specific. If there are multiple parsers
with the same embed level (@pxref{parser-embed-level}) at
@var{position}, which parser is tried first is undefined. If
@var{parser} is a language symbol, the function limits the parsers it
tries to the ones for that language.
@end defun
@defun treesit-thing-next position thing &optional parser
@ -1875,8 +1875,9 @@ position must be strictly greater than @var{position}.
If @var{parser} is non-nil, only use that parser's parse tree.
Otherwise try each parser covering point, from the most specific
(deepest-embedded) to the least specific. If there are multiple parsers
with the same embed level at @var{position}, which parser is tried first is
undefined. @var{parser} can also be a language symbol.
with the same embed level (@pxref{parser-embed-level}) at
@var{position}, which parser is tried first is undefined. @var{parser}
can also be a language symbol.
@end defun
@findex treesit-beginning-of-thing
@ -2032,11 +2033,12 @@ for nodes in a region.
@defun treesit-language-at pos
This function returns the language of the text at buffer position
@var{pos}. Under the hood it calls
@code{treesit-language-at-point-function} and returns its return
value. If @code{treesit-language-at-point-function} is @code{nil},
this function returns the language of the deepest parser by embed level
among parsers returned by @code{treesit-parsers-at}. If there is no
parser at that buffer position, it returns @code{nil}.
@code{treesit-language-at-point-function} and returns its return value.
If @code{treesit-language-at-point-function} is @code{nil}, this
function returns the language of the deepest parser by embed level
(@pxref{parser-embed-level}) among parsers returned by
@code{treesit-parsers-at}. If there is no parser at that buffer
position, it returns @code{nil}.
@end defun
@heading Supporting multiple languages in major modes
@ -2153,10 +2155,38 @@ a @var{query} preceded by zero or more @var{keyword}/@var{value}
pairs. Each @var{query} is a tree-sitter query in either the string,
s-expression, or compiled form, or a function.
@code{treesit-update-ranges} uses @var{query} to figure out how to set
the ranges for parsers for the embedded language. It queries
@var{query} in a host language parser, computes the ranges which the
captured nodes span, and applies these ranges to embedded language
parsers.
If @var{query} is a tree-sitter query, it should be preceded by two
@var{keyword}/@var{value} pairs, where the @code{:embed} keyword
specifies the embedded language, and the @code{:host} keyword
specifies the host language.
specifies the embedded language, and the @code{:host} keyword specifies
the host language. The captured nodes determines the range of embedded
parsers. The capture name doesn't matter, but names that starts with an
underscore are ignored, and the @code{@@language} capture name is
reserved.
If the embedded language is dynamic, then @code{:embed} can specify a
function rathar than a static language. The function takes the node
captured with @code{@@language} capturename. Care have to be taken that
the @code{@@language} capture and the range capture are in the same
match group. The function should return either a language symbol, or nil
if no valid language can be found.
If the query is given a @code{:offset} keyword whose value is a pair
of numbers, each captured range is offset by those numbers. For
example, an offset of @code{(1 . -1)} will update a captured range of
@code{(2 . 8)} to be @code{(3 . 7)}. This can be used to exclude things
like surrounding delimiters from being included in the range covered by
an embedded parser.
By default, a parser sees its ranges as a continuum, rather than
treating them as separate independent segments. Therefore, if the
embedded ranges are semantically independent segments, they should be
processed by local parsers, described below.
@cindex local parser
If the query is given the @code{:local} keyword whose value is
@ -2164,19 +2194,44 @@ If the query is given the @code{:local} keyword whose value is
otherwise the range shares a parser with other ranges for the same
language.
By default, a parser sees its ranges as a continuum, rather than
treating them as separate independent segments. Therefore, if the
embedded ranges are semantically independent segments, they should be
processed by local parsers, described below.
Local parser set to a range can be retrieved by
Local parsers set to a range can be retrieved by
@code{treesit-local-parsers-at} and @code{treesit-local-parsers-on}.
@code{treesit-update-ranges} uses @var{query} to figure out how to set
the ranges for parsers for the embedded language. It queries
@var{query} in a host language parser, computes the ranges which the
captured nodes span, and applies these ranges to embedded language
parsers.
Emacs treats local embedded parsers and non-local embedded parsers quite
differently:
@itemize @bullet
@item
Local parsers are managed by Emacs. Emacs create new local parsers when
they are needed and deletes them when they are not. Non-local parsers
must be created by major modes, and Emacs only make use of them if one
exists.
@item
Because local parsers can be freely created by Emacs, they are suitable
for recursively nested embedded parsers, where the embed level can be
arbitrarily deep. On the other hand, since non-local parsers are major
mode created, they can only handle a fix number of embed levels.
@end itemize
@anchor{parser-embed-level}
@cindex embed level, tree-sitter
The embed level of the top-level parsers is 0, or omitted, the first
level of embedded parsers get embed level 1, and so on. For local
parsers, each embed level creates a new parser; for non-local parsers,
since Emacs cannot create non-local parsers freely, it tries to find a
usable parser at the right embed level, if none exists, Emacs just gives
up.
If the query is given a @code{:range-fn} keyword whose value is a
function, Emacs uses that function to compute the ranges to use for the
embedded parser. The function takes captured node and the offset given
by the @code{:offset} keyword, and returns a list of ranges, where each
range is a cons of the start and end position. For local parsers, each
@var{query} creates one local parser, and the ranges set by the
@code{:range-fn} is applied to that parser; for non-local parsers, all
the ranges (including ranges created by @code{:range-fn}) in that embed
level are joined together and applied to the same parser.
If @var{query} is a function, it doesn't need any @var{keyword} and
@var{value} pair. It should be a function that takes 2 arguments,

1
src/treesit.c
View file

@ -2565,7 +2565,6 @@ embedding increments the embed level by 1. */)
return XTS_PARSER (parser)->embed_level;
}
/* TODO: Mention in manual, once the API stabilizes. */
DEFUN ("treesit-parser-set-embed-level",
Ftreesit_parser_set_embed_level, Streesit_parser_set_embed_level,
2, 2, 0,