March 26, 2021·4 min read·engineeringrubyrailsdependencieslicensing

Rails replaces Marcel's indirect MIME data dependency

Rails updates Active Storage to Marcel 1.0.0, which directly packages signature data adapted from Apache Tika. The change shows why bundled data needs provenance review alongside the code using it.

A dependency's own license does not describe every dataset it uses. The March 26 Rails releases address that distinction by updating Active Storage's Marcel dependency to version 1.0.0. Rails publishes the change in versions 5.2.5, 6.0.3.6 and 6.1.3.1, covering several existing framework lines.

George Claghorn's announcement explains that earlier Marcel versions indirectly depended on MIME data under the GNU General Public License, which the announcement describes as incompatible with Marcel's MIT distribution. Marcel 1.0.0 directly packages replacement data adapted from Apache Tika under Apache License 2.0. That is the project's stated reason for the coordinated framework updates.

I find Ruby's readable object model useful for understanding application behavior. Dependency data introduces another kind of input that is less visible in an application method. The MIME detector's code can remain familiar while a dataset underneath it changes, affecting both distribution provenance and the classifications it can make.

Detection depends on a body of signature data

The Marcel 1.0.0 README describes a detector that considers file contents, names and declared types. Signature data lets it recognize characteristic byte patterns. File extensions and declared media types provide other clues when the data alone does not determine a useful specific answer.

The data therefore participates in executable behavior even though it is not Ruby source implementing the control flow. A magic signature entry can determine whether a byte sequence matches a type. Type relationships can determine whether a suggested more specific classification is acceptable. Reviewing only the few methods that call the detector would omit the information those methods interpret.

This is why replacing the dataset deserves more precise description than simply removing a troublesome dependency. The replacement has a source, a distribution license and a role in behavior. Marcel's README separately identifies its MIT code, portions adapted from mimemagic, and magic signature data adapted from Tika. That description makes the different origins explicit.

The 1.0.0 gem specification lists both MIT and Apache-2.0 in its licenses. It also lists the relevant license files among packaged material. The published package metadata thus reflects the distinction between Marcel's own implementation and the adapted data it distributes.

The implementation combines evidence with an explicit fallback

The release's MimeType implementation begins by identifying a type from file data. It separately constructs a fallback from a declared type, a supplied name, an explicit extension and finally application/octet-stream, in that order. This is the ordering in the tagged implementation.

When data supplies a type, the detector compares it with the fallback through the type relationship model. If their root type sets overlap, it can return the fallback. Otherwise, the data-derived type wins. When data supplies no type, the fallback is the result.

The distinction matters because a filename can supply useful specificity without being allowed to override arbitrary content evidence. The README illustrates an Illustrator document that also has PDF characteristics. A compatible filename can help select a more specific document type. A filename suggesting an unrelated type does not have the same relationship to the content evidence.

Here is a small example from the documented interface that requires no file fixture. It asks for a type based on an extension.

require "marcel"

Marcel::MimeType.for(extension: ".pdf")
# => "application/pdf"

This does not establish that a particular file is a PDF. No bytes were supplied. The returned value describes the extension mapping in Marcel's data. Treating an extension-only lookup as equivalent to inspecting content would attribute evidence to the detector that the caller never gave it.

Declared types remain inputs to interpretation

The implementation normalizes a declared type by lowercasing it and taking the initial portion before separators such as a semicolon or whitespace. It accepts that portion only when it contains a slash. It also excludes the generic binary type from being treated as a useful declared fallback.

These rules show that MIME detection is an interpretation of several inputs. The result does not prove that a document is safe to execute or render in every context. It identifies a media type according to the available evidence and the detector's rules. An application's later policy for handling that type remains its own responsibility.

That is also why the data replacement can matter to applications beyond the packaging issue. A different signature set can classify some inputs differently. The release announcement establishes the provenance change, but it does not promise that every possible byte sequence receives an identical answer. Compatibility claims need to be about actual behavior, not inferred solely from an unchanged method name.

Existing Rails lines receive the new data through Marcel

The announcement updates Active Storage's Marcel requirement in several Rails releases. This lets applications on those lines take the dependency change without treating it as a new major framework feature. The relevant object of the update is the MIME detection dependency used by Active Storage.

The patch should therefore be understood at two levels. Rails supplies the integration and release path. Marcel supplies the detector and its newly packaged data. A dependency inventory that stops at the Rails version would obscure the component responsible for interpreting file types and the source of the replacement signatures.

Marcel's README also distinguishes fixtures recognized by magic numbers from fixtures recognized only by name. Both groups are intended to contain valid examples of the claimed file type. That organization gives behavioral review a concrete basis. It separates successful recognition from bytes from a correct filename fallback, which would otherwise look identical if a test checked only the final MIME string.

I regard the explicit data attribution as the useful maintenance improvement here. Package metadata needs to describe what is actually distributed, including material interpreted by the code. Marcel 1.0.0 and the coordinated Rails updates make that origin visible while retaining the documented interface for requesting MIME classification.