SwiftSyntax separates package consumption from compiler development

Using a syntax library in a tool should not require adopting the entire compiler's development workflow. SwiftSyntax's new Swift Package Manager arrangement moves consumption toward an ordinary package dependency. It also exposes a remaining relationship with the compiler, because syntax-tree layout definitions still generate from the main Swift repository.

The August 31 announcement introduces SwiftSyntax's separate repository and package-based workflow. Clients can declare the repository as a dependency and make their target depend on SwiftSyntax. Contributors building the newest source still need the main Swift checkout for code generation.

That is the useful distinction in this change. A package consumer selects a prepared version. A contributor changing the library may need the machinery that prepares it. Putting the project in its own repository does not mean those workflows have identical requirements.

I find Swift's build times the least pleasant part of working with it. A tooling library that can be consumed through a package declaration is therefore appealing. The benefit is clearest when the instructions explain which work has already been done for a selected tag and which work a contributor still needs to perform.

A package dependency points to a toolchain version

The announcement identifies tags associated with compiler releases and development snapshots. At publication, it names the August 25 Swift 4.2 development snapshot as the latest tag for that line. It also gives a corresponding Swift 5 development snapshot tag.

The README for the named Swift 4.2 snapshot shows the package declaration. The following manifest excerpt uses that documented snapshot selection.

import PackageDescription

let package = Package(
    name: "MyTool",
    dependencies: [
        .package(
            url: "https://github.com/apple/swift-syntax.git",
            .branch("swift-4.2-DEVELOPMENT-SNAPSHOT-2018-08-25-a")
        )
    ],
    targets: [
        .target(name: "MyTool", dependencies: ["SwiftSyntax"])
    ]
)

This uses the branch-style requirement spelling shown in the contemporary instructions, even though the supplied name identifies a release tag. The spelling deserves attention because package resolution rules distinguish this kind of dependency from semantic-version requirements.

Ankit Aggarwal points out in the announcement thread that branch-based dependencies are restricted to the root package. That creates a composition concern for someone writing a library that itself depends on SwiftSyntax. A direct dependency in an executable's package is a different situation from distributing that dependency through another versioned package.

The same-day discussion considers semantic versioning. Jordan Rose notes the value of retaining the Swift version in the release name because SwiftSyntax is not yet stable. Later that day, Alex Hoppen says the project has decided to use a version-number encoding Rose suggested. The exchange shows that packaging the library immediately raises a version-identity question, beyond moving files into a new repository.

Generated layout remains tied to compiler source

The snapshot README explains why a contributor still needs a Swift checkout. SwiftSyntax uses definitions in the main repository to generate the syntax tree's layout through gyb. The separate repository does not independently define every input needed to build its newest source.

The documented development arrangement places the Swift and SwiftSyntax checkouts side by side and runs SwiftSyntax's build script. That script builds the library with the compiler installed on the system. These are contributor instructions for the newest source, not requirements that every tool embedding a prepared package must repeat.

The announcement says Swift CI will perform the generation step for toolchain releases and development snapshots and create corresponding tags. Selecting those tags is the recommended dependency workflow. The generated preparation belongs to the toolchain publication process, while consumers identify the prepared result by its toolchain-related name.

Separating the workflows also makes a bug report more precise. A consumer can identify the selected tag, while a contributor can identify a source change and the generation step used to build it. Those descriptions distinguish package integration problems from work on the generated library itself.

This gives the tags a practical role beyond labeling an arbitrary point in a repository. They connect a SwiftSyntax revision to a compiler publication and its generated definitions. A moving development branch does not offer the same evidence about that preparation.

Syntax transformation becomes accessible to packages

The snapshot README describes SwiftSyntax as Swift bindings for libSyntax, supporting inspection, generation and transformation of Swift source. Its example subclasses SyntaxRewriter and visits tokens. When it encounters an integer literal token, it constructs a replacement token with a changed literal spelling.

That example demonstrates why a source tool wants syntax nodes instead of a text substitution alone. The visitor can distinguish the integer-literal token kind before changing it. Other tokens pass through unchanged. The transformation operates on a representation of the source's syntax.

The example is deliberately small. It filters literal text down to decimal digits and force-converts it to Int before adding one. That makes it an illustration of the rewriting API, rather than a complete specification for transforming every possible Swift numeric literal. A production transformation needs to decide which spellings and values it accepts.

My preference is for tool examples to make that boundary visible. A working visitor explains how to reach and replace a token. Correctly defining a transformation over all relevant source programs remains the tool author's responsibility. Packaging makes the library easier to reach without settling the semantics of every tool built on it.

Test and platform support remain specific

Local testing has additional requirements in the snapshot README. Some utilities must be built as part of the compiler project. The documented path builds SwiftSyntax and its dependencies through the main Swift build process, then runs the SwiftSyntax tests while skipping unrelated suites.

At the announcement, the main Swift repository's CI test command already exercises SwiftSyntax. CI testing directly on the SwiftSyntax repository is described as forthcoming. The post also says building SwiftSyntax on Linux is not yet supported and invites contributions to add it. A SwiftPM package should therefore not be mistaken for an immediate cross-platform support guarantee.

The announcement further plans to stop including SwiftSyntax in open-source toolchains from Swift 5 onward. It presents that as a way to align toolchains distributed through Xcode and swift.org. As of this announcement, the usable change is the separate package workflow with named snapshot tags. Contributor code generation, test utilities and Linux support each retain the specific limits documented by the project.