Go's SIMD experiment leaves vector width out of the algorithm

A vector width can become an accidental part of an algorithm's design. Code written around one register size needs another implementation when the hardware offers a different size. Go's September 24 SIMD explanation describes an experiment that removes that size from the source type. simd.Float32s describes a vector of floating-point elements without committing the algorithm to a fixed lane count.

David Chase and Junyang Shao explain the portable API available in Go 1.27, following architecture-specific SIMD support introduced in Go 1.26. The portable package remains experimental and requires GOEXPERIMENT=simd at build time. It aims to let the same algorithm use available hardware instructions or an emulation where hardware support is absent.

I am interested in this because Go's readability is a practical advantage in performance-sensitive code too. An optimization becomes easier to maintain when its relationship to the original computation remains visible. The portable API can express repeated arithmetic directly while moving decisions about hardware width into the compiler. That is useful for algorithms that do not inherently depend on a particular number of lanes.

The proposal distinguishes width-independent algorithms

Proposal 78902, opened on April 22, describes the portable package as a second level above the architecture-specific API. Its initial operation set comes from the intersection of existing hardware interfaces, with emulation filling gaps. Starting with a small operation set makes additional architectures easier to accommodate without promising an instruction that every implementation handles poorly.

The proposal discussion raises an instructive counterexample. A commenter asks about algorithms whose logical data structure contains a fixed number of elements, such as a fixed group of hash codes. That is a different requirement from processing an arbitrary-length slice in whatever groups the machine supports. A fixed logical vector may need several physical registers on one target and one register on another.

That distinction explains what portability means here. The package does not make every register-oriented algorithm independent of its chosen layout. It provides an abstraction suitable for computations that can vary their lane count. A reduction or a scan over input data can often express that flexibility, while a representation deliberately arranged around a fixed lane count needs a different design decision.

The tradeoff is a restriction on the algorithm's relationship to its vector. The program can use the vector's length to advance through data, but cannot assume that the length equals a fixed constant chosen during development. That restriction is what permits one source implementation to follow different hardware configurations.

Values enter and leave through slices

The Go 1.27 package documentation defines a valid zero vector and corresponding types for primitive numeric values except complex numbers. Full loads require a sufficiently long slice. Partial loads consume available elements, fill remaining lanes with zero, and return the number loaded. Full and partial stores provide the corresponding transfer back to slices.

These operations give the final short segment of a computation an explicit representation. The main loop handles full vectors, while the last partial vector handles the remainder. Zero filling is useful when zero is neutral for the operation, but the algorithm must establish that property. The same padding choice does not automatically work for every comparison or transformation.

For example, a multiplication followed by summation can tolerate a padded zero product. A selection that counts matching zero values needs to distinguish real lanes from padding. The partial-load count provides information that such an algorithm may need. Portability of the vector type does not remove the ordinary requirement to define what happens at the end of the input.

The package also provides mask types corresponding to element widths. Their purpose is to describe lane selection without requiring the source program to use the mask representation of a particular instruction set. The Go 1.27 source documentation explains that signed-vector conversion to a mask compares against zero. It is not simply a reinterpretation of the vector's bits.

That distinction prevents an appealing but incorrect assumption about conversion. A type conversion used as an operation can have semantic work to do even when its hardware representation looks related. In vector code, naming the conversion's meaning is more useful than reasoning from the apparent similarity of two register layouts.

Padding also changes the reasoning for a minimum operation. If every real value in the final segment is positive, zero-filled lanes can introduce a smaller value that never appeared in the input. A correct implementation needs a way to exclude those lanes or use a representation appropriate to the operation. This follows from the documented load behavior without depending on any particular vector width. The scalar specification of the computation therefore remains useful as the definition of the result, even when most execution happens through vector operations. The tail is part of that definition and should remain visible in the algorithm.

The compiler creates specialized implementations

The proposal describes rewriting SIMD-dependent code into several variants. Functions whose bodies depend on SIMD but whose signatures do not expose SIMD types become dispatch points. Calls within a specialized variant target other functions of the same variant. The intention is to avoid repeated dispatch inside the computation and still allow ordinary inlining.

The September explanation describes the resulting implementation names with a SIMD suffix for a particular width or emulation. This accounts for unfamiliar names in stack traces. The source program presents one algorithm, while generated implementations represent the selected machine operation. The dispatch location is part of how the compiler balances code duplication against runtime overhead.

This is a useful compromise for a package intended to remain readable. A maintainer can follow the computation in one source implementation while the compiler handles variants. The source still needs a sensible function boundary. When dispatch surrounds an entire operation, the operation can perform many vector steps after the selection. A boundary that forces repeated selection around tiny work would undermine that advantage.

That observation also affects performance explanations. A profile entry in a specialized function is evidence about one generated variant. It should not become an undocumented assumption that every deployment selects the same width. A report that names the selected configuration gives another engineer a way to relate the result to the source algorithm.

Portable operations are a deliberate subset

The Go 1.27 release notes distinguish the portable package from simd/archsimd. Architecture-specific operations support amd64, arm64 NEON, and WebAssembly in this release. Portable operations form a scalable subset and use hardware or emulation across architectures.

Some operations fit that subset naturally. Others expose differences in hardware that the portable API cannot erase cheaply. This is why an architecture-specific package remains useful alongside the portable one. A computation that requires an unsupported operation can isolate that part behind a target-specific implementation instead of replacing every surrounding portable operation.

The package documentation provides ToArch and conversion functions for that boundary. Code using the resulting architecture-specific values must use appropriate build constraints and handle the concrete types. That is an explicit increase in maintenance responsibility. The benefit is access to an operation the portable subset does not currently provide, while the cost is owning the cases the portable implementation previously selected.

My preference is to keep such a boundary narrow. Go's advantage diminishes when one data-processing function becomes several unrelated implementations that happen to share a name. An isolated target-specific operation gives the reviewer a smaller equivalence question and preserves the surrounding algorithm as common source.

Emulation is an implementation path with its own value

The experiment's runtime controls include GODEBUG=simd=0 to force emulation. Other settings select vector widths, subject to hardware support. That makes the scalar path available even on development hardware with SIMD. The distinction is useful for checking whether an algorithm depends accidentally on the width selected by one machine.

The emulated path also gives a concrete meaning to the package's portability claim. A supported source algorithm has a way to execute without the native vector facility. It does not imply identical speed across machines. Hardware acceleration remains a property of the selected implementation and workload.

This matters most for libraries distributed to callers whose processors the author does not control. The portable implementation can keep one algorithm available across those targets. A library's documentation can then distinguish its semantic guarantee from its performance expectations. For a width-independent computation, that is a clearer contract than silently assuming the register configuration used during development.