REXML removes ambiguous zeros from numeric-reference matching

Permitting extra leading zeros before a hexadecimal marker gave REXML more than an overly generous XML grammar. It also gave its regular expression several ways to divide a long run of digits between neighboring parts of the pattern. On affected Ruby runtimes, crafted numeric references could make that matching take excessive time.

The October 28 advisory for CVE-2024-49761 recommends REXML 3.3.9 or later. It identifies REXML 3.3.8 and earlier with Ruby 3.1 or earlier as affected. Ruby 3.2 and later do not exhibit the reported issue, making Ruby 3.1 the affected maintained interpreter line at this date.

I find Ruby easier to read than the dense Perl expressions I came from, but a short pattern still needs a precise grammar. Here the fix improves both parsing correctness and the amount of work needed to reject a malformed numeric reference.

The old pattern allowed an extra prefix

The parser correction removes 0* immediately after &# in the CHARACTER_REFERENCES pattern. The remainder still allows a decimal digit sequence or an x followed by hexadecimal digits, ending at a semicolon.

That old leading-zero component accepted a form such as &#0x61;, even though the hexadecimal marker belongs immediately after &#. The REXML 3.3.9 notes identify acceptance of that form as a parse bug. The corrected pattern expresses the two intended numeric forms without the independent zero prefix.

The ambiguity is visible in the decimal branch too. A run of zeros can satisfy both 0* and the following digit repetition. On a failing match, a backtracking matcher can revisit choices about where the prefix stops and the next component starts. This is an interpretation of the removed pattern and the advisory's malformed-input condition, rather than a claim that the patch publishes a universal complexity bound.

The notice describes inputs with many digits between &# and an x...; suffix. That is different from an ordinary hexadecimal reference whose digits follow x. The malformed prefix is part of the reported condition, so describing all hexadecimal references as dangerous would lose the cause.

Numeric conversion now selects the base explicitly

The patch also changes what happens after a reference matches. Previously, the implementation prepended a zero when the captured text began with x, then passed the result to Integer without an explicit base. The corrected implementation selects hexadecimal conversion for the x form and decimal conversion for the other branch.

if m.start_with?("x")
  code_point = Integer(m[1..-1], 16)
else
  code_point = Integer(m, 10)
end

This is an excerpt of the corrected conversion step. The surrounding parser has already matched the numeric reference and obtained its captured content in m. The hexadecimal marker selects the base, while only the digits after it pass to hexadecimal conversion.

The explicit decimal base also makes the intended interpretation of decimal references visible. Matching and conversion now describe the same distinction, a decimal digit sequence or a hexadecimal sequence introduced by x. The conversion no longer depends on manufacturing a different-looking integer string to select the base implicitly.

After conversion, the implementation packs the code point into the output representation. That is a separate step from recognizing the reference grammar. Keeping these responsibilities distinct makes it easier to see which input the pattern accepts and how the accepted digits acquire their numerical meaning.

The regression distinguishes similar-looking references

The added test uses the pull parser on text containing both a and &#0x61;. It expects the first to become a while the second remains literal text at that parser stage. This directly checks that the erroneous extra-zero form no longer receives the same numeric-reference treatment.

The assertion is more precise than saying the parser rejects every document containing such text. Different parser stages expose different representations and validation obligations. The regression specifically verifies the returned text event after numeric-reference processing.

That also illustrates why correctness and performance meet in this fix. The unwanted input form came from a grammar component that expanded the matcher's choices. Removing that component narrows accepted references and eliminates the ambiguity identified by the correction. The change is justified by the intended syntax as well as the security report.

Runtime version is part of the affected range

The advisory explicitly excludes Ruby 3.2 and later from this denial of service. That qualification should remain attached to the claim even though the gem's parsing correction is useful across interpreters. A dependency scanner reporting REXML 3.3.8 alone has less information than the full affected-version statement.

Conversely, using a maintained Ruby 3.1 interpreter does not make an old REXML gem immune. The advisory recommends the corrected gem for that combination. An application's resolved library version and its executing interpreter jointly determine whether the reported condition applies.

The repair is therefore available without requiring an immediate interpreter-series migration. REXML can be updated as a gem in the affected application environment. That is a narrower maintenance change than moving an entire application to another Ruby series, while still removing the parser behavior identified in this advisory.

The release notes date REXML 3.3.9 to October 24, before this disclosure. Today's post uses the disclosure date because it supplies the security interpretation and affected-runtime qualification. The earlier release also contains a correction for text-only invalid XML and a separate source-reading performance improvement.

This is a new condition with a specific repair

The July REXML advisory addressed other long-running parse cases and recommended 3.3.2. That older corrected version is below today's 3.3.9 requirement. A prior security update cannot be treated as resolving a condition disclosed later with a different affected range.

My interest is in the small grammar decision that explains the new correction. Removing an unnecessary zero prefix makes the numeric-reference pattern correspond more closely to the intended forms. Explicit conversion then preserves the same decimal-or-hexadecimal distinction, while the runtime-qualified advisory identifies which deployments face the reported excessive matching time.