A Tree-sitter grammar gives Nix structured syntax tooling
The new tree-sitter-nix grammar gives tools a structural view of Nix source. Incremental parsing and error recovery matter because an editor usually sees unfinished programs.
An editor spends much of its time looking at source that does not yet form a complete program. A missing delimiter can be a normal intermediate state while someone types. The new Tree-sitter grammar for Nix, announced on March 22 by cstrahan, brings that problem into view. The announcement suggests future syntax highlighting in nix repl and points to Tree-sitter's attempt to produce a useful parse tree even when the input has errors.
The available work is the grammar. REPL highlighting is a possible use described by its author, not an integration that the announcement says has already shipped. That distinction keeps the result concrete while explaining why a parser designed for editing is relevant to a declarative configuration language.
Nix's declarative model is what interests me, but the language still makes readers work to understand nested expressions. Better syntax tooling can reduce that reading effort. A structural view can identify an expression's parts without pretending to know the package or system configuration that evaluating it will produce.
Incremental parsing fits an editor's input
The Tree-sitter README from March 22 describes both a parser generator and an incremental parsing library. It builds a concrete syntax tree for a file and updates that tree as the source changes. Its stated goals include operation on every keystroke and useful results in the presence of syntax errors.
Those goals address different aspects of editing. Incremental updates concern the amount of work needed after an edit. Error tolerance concerns whether the tool can retain useful structure before the programmer has finished repairing the text. An editor needs both if its syntax understanding is to remain available throughout ordinary typing.
The README also identifies a runtime library written in C with no dependencies. That is an embedding property, rather than a promise that every editor already exposes the parser. A Nix grammar and an embeddable runtime make integration possible, while a host application still needs code that uses them.
The announcement's REPL idea follows from this arrangement. An interactive input buffer can contain a partial expression. A parser that requires the final valid program before providing any structure would be less useful during entry. Recovering a sensible partial tree can support highlighting of the text already present.
The grammar names language structure
The grammar published that morning contains rules for function expressions, conditional expressions and operator expressions. It also distinguishes application from attribute selection. These categories give a consumer more information than a sequence of colored character spans.
For example, the grammar assigns precedence levels to multiplication and addition. It also specifies associativity for several operators. Those rules determine the structure of an expression with more than one operator, so a consuming tool need not guess the grouping from text patterns alone.
A simple expression illustrates the structural question.
1 + 2 * 3
The grammar's multiplication precedence is higher than its addition precedence. That gives a parser the information needed to group the multiplication as part of the addition's right operand. The example concerns syntax structure. It does not require evaluating a package set or resolving an external dependency.
The rules also distinguish comments as extra material between grammatical elements. That lets a syntax consumer retain an understanding of the surrounding expression even when comments appear in the source. It is another reason a tree can support operations that need to know where a complete expression begins and ends.
Strings need more than one token rule
The initial grammar separates ordinary string content from indented-string content and names both as external scanner inputs. Within both string forms it recognizes an interpolation opening followed by a Nix expression and a closing brace. That is a direct example of nested language structure in ordinary configuration text.
The parser must distinguish the surrounding string from the expression embedded inside it. Treating the whole string as undifferentiated text loses the embedded structure. Treating every brace identically loses the relationship between the string and its interpolation. Named rules provide a place for that relationship in the tree.
This also shows why publishing a grammar does not prove that every edge case is already handled. The source is an initial implementation with its own scanner and rule choices. The announcement does not present a conformance proof or a complete editor feature set. Consumers have something concrete to exercise and report against.
In the same day's discussion, wmertens describes the limits of an existing regex-based Nix grammar and expresses interest in replacing it. That is a statement from a tooling author about the current approach. It supports the demand for a structural parser without establishing that the replacement is already integrated.
A syntax tree does not evaluate Nix
The published rules describe how source constructs combine. They do not provide the meanings of imported values or identify which package an attribute ultimately denotes. A parser can recognize an application expression without knowing what the function returns.
That boundary matters when describing future capabilities. Syntax highlighting and structural selection can use grammatical categories. Accurate dependency resolution or evaluation-aware diagnostics require more information. A parse tree is useful input to those tools, but it is not their entire implementation.
I welcome that narrower contribution because it addresses a real part of the language's usability. Tools can improve how a configuration is read and edited without having to evaluate a complete system on every edit. The distinction also gives users a clear expectation of what the new project supplies today.
The next work belongs in integrations
The March 22 announcement leaves REPL highlighting as a future possibility, and the discussion asks about editor use. Those are natural next steps because a grammar only becomes visible to most users through a host tool. The host decides how to use the tree and how to present incomplete syntax.
For now, the concrete addition is an independently inspectable Nix grammar for an incremental parsing system. Its expression rules and separate string scanner establish the structure that an integration can consume. A tool can begin with that structure while the author is still typing, including at points when the input is temporarily incomplete.