mirror of
https://github.com/tree-sitter/tree-sitter.git
synced 2026-09-10 07:26:23 -04:00
0f6780b9a3
`ts_decode_utf16_le` and `ts_decode_utf16_be` passed a byte length to `U16_NEXT_LE` and `U16_NEXT_BE`, but those macros count `uint16_t` code units. If a lead surrogate was the last code unit in a chunk, the decoder could peek past the chunk and combine it with adjacent memory. This commit passes the code-unit length to the UTF-16 macros, and fails with `TS_DECODE_ERROR` when a chunk is too short to contain even one full code unit. This lets the lexer retry with a fresh chunk or advance through the invalid byte as it already does. Co-authored-by: Will Lillis <will.lillis24@gmail.com> Co-authored-by: Amaan Qureshi <git@amaanq.com> |
||
|---|---|---|
| .. | ||
| error_corpus | ||
| grammars | ||
| template_corpus | ||
| test_grammars | ||
| fixtures.json | ||