Embedding binary resources without generating C source

A binary resource can become expensive long before a program reads it. Converting an image into a generated C array asks the compiler to process individual integer literals for data that already exists in its final representation. JeanHeyd Meneide's N2470 proposal, listed in the WG14 document log for January, proposes #embed to give that operation its own preprocessing directive. The important design question is how much ordinary language machinery a compiler needs to apply to an uninterpreted resource.

C was my first language, and I value the small language beneath its implementation details. A facility for binary resources fits that preference when it replaces a recurring piece of external build machinery with an explicit declaration. Its usefulness depends on whether the specification lets implementations retain the simplicity of the original data.

Representing data as source

The existing workaround starts with a file and generates source containing its values. That produces something a conventional compiler already understands, but the representation introduces work unrelated to the resource's meaning. A byte in an image becomes a token in a program. A sequence of bytes becomes an initializer that must pass through compiler machinery intended to handle expressions.

Consider the division of responsibility in such a build. The resource generator understands the input file and emits valid source. The compiler then checks that source. A change to the generator can affect formatting or declaration names without changing the underlying resource. A change to the resource requires regenerating an intermediate file whose purpose is to convey the same bytes through a different grammar. These are consequences of the conversion scheme, even when each tool performs its individual task correctly.

The proposal's alternative gives the compiler an explicit indication that the input is resource data. Its initial syntax supplies the whole initializer, including braces. The following declaration illustrates that proposed form, not a facility available in standard C today.

const unsigned char icon_data[] =
#embed "icon.bin"
;

The placement of the semicolon matters. The directive occupies its own preprocessing line, while the declaration continues after it. Writing another pair of initializer braces around this early proposal would describe a different program. A language proposal needs examples that match its actual revision, particularly when its syntax is still under discussion.

Why the representation matters

A compiler that receives general expressions needs machinery capable of representing general expressions. Resource bytes have fewer requirements. They need a representation that preserves their values and lets the compiler place them in the output. My interpretation of the proposal is that its main implementation opportunity comes from communicating this restricted intent before expensive expression processing begins.

That distinction also explains why a shorter generated source file would only address part of the problem. Removing whitespace reduces text size without removing the individual values that the parser must consume. A special directive can communicate that the entire input belongs to a simpler class of data. The potential saving follows from avoiding unnecessary representation, rather than changing what the program stores.

There is a related distinction between observable behavior and implementation technique. An implementation can present resource inclusion to the program through initializer semantics while storing the input internally in a compact form. Programmers need a definition of what their declarations mean. Compiler developers need freedom to implement that meaning without materializing every intermediate form suggested by a literal source expansion.

I find that separation appropriate for C. A small source construct can still require a substantial compiler implementation, but the source should express the operation the programmer needs. Requiring application authors to encode compiler input conventions in generators transfers implementation concerns into every consuming build.

Resource inclusion still needs a build contract

A resource named in source becomes an input to compilation. Consequently, a reproducible build needs to identify the resource as precisely as it identifies the source file. This is an implication of compile-time inclusion, not an additional guarantee supplied by the directive. Two builds cannot be expected to embed identical data if they resolve the same name to different contents.

The existing November working draft describes source inclusion through implementation-defined search behavior. N2470 follows that general model for resource lookup. That makes the design familiar, although familiarity does not determine which directories a particular build will search.

For build authors, the useful question is whether the resource appears among the compiler's dependencies. If a build records only the translation unit, changing the resource alone may leave an existing object untouched. Conversely, an explicit dependency lets an incremental build recognize that the object no longer corresponds to all its inputs. The directive therefore needs a practical relationship with dependency reporting as well as parsing.

A generated-header solution has the same underlying dependency requirement, with an additional conversion step. Removing that step can reduce the number of artifacts that need names and regeneration rules. It does not remove the requirement to track the original resource. This is why the build-system consequences belong in the language discussion instead of appearing as an afterthought to attractive syntax.

Choosing what belongs in the executable

Compile-time inclusion makes the resource part of the program's build inputs. That has a different operational consequence from opening a separately deployed file. Updating the embedded resource requires producing a corresponding executable. For a resource intended to match a particular program revision, that coupling can be desirable. For data that changes independently, the same coupling adds release work.

These are application decisions. A directive cannot decide whether an icon, a script, or a firmware image should have the same release lifecycle as the executable that contains it. It can make the chosen relationship explicit in source. The resource still needs an owner and a defined update process.

The ordinary declaration around the directive is useful for the same reason. The proposal concentrates on obtaining an initializer, leaving the surrounding program to express how it uses the resulting object. Resource inclusion and resource interpretation remain separate responsibilities. Embedding a file does not establish that its bytes describe a valid image or that an application understands its format.

The practical gain I see is a more direct compiler input for a common operation. The remaining implementation work concerns resource lookup and dependency reporting, along with a representation that avoids the cost of generated expression lists. Those details determine whether the directive removes build complexity for programs that already carry binary resources.