tree-sitter.tree-sitter/lib/include
Will Lillis ea90489d7a
perf(generate): add CSR-compressed parse tables (ABI 16)
Apply Compressed Sparse Row (CSR) compression to the parse table for
grammars that benefit, replacing the original dense + small split with
three flat arrays:

  uint32_t parse_table_row_offsets[STATE_COUNT + 1]
  uint16_t parse_table_columns[TOTAL_NNZ]
  uint16_t parse_table_values[TOTAL_NNZ]

Heuristic for enabling CSR (per grammar, all three must hold):

  1. LARGE_STATE_COUNT * SYMBOL_COUNT > STATE_COUNT * 40
     Ensures the dense table is large enough relative to total state
     count that savings outweigh the small-state grouping penalty.

  2. dense table density < 45%
     Ensures CSR actually saves space. Above ~50% density, CSR's
     per-entry column indices cost more than the zeros they eliminate.
     45% adds margin below the theoretical crossover.

  3. LARGE_STATE_COUNT * SYMBOL_COUNT > 50,000
     Avoids applying a format change to tiny grammars where fixed
     overhead dominates.

Co-authored-by: Tuomas Hietanen <thorium@iki.fi>
2026-05-19 17:57:49 -04:00
..
tree_sitter perf(generate): add CSR-compressed parse tables (ABI 16) 2026-05-19 17:57:49 -04:00