SQLite makes subpage reads an explicit VFS capability

Reading part of a database page seems harmless when the underlying file behaves like an ordinary file. A non-standard SQLite VFS can have stricter requirements. SQLite 3.47.1 adds an explicit capability for unaligned subpage reads, correcting a compatibility problem exposed when direct overflow reads became enabled by default earlier this year.

The November 25 release also repairs Windows line-ending handling in sqlite3_rsync, restores installation handling for DESTDIR, and corrects some IN query results. Several fixes concern assumptions around an operation rather than the SQL text itself. The library needs to know what a storage implementation supports, and a binary copying protocol needs its bytes to survive process streams unchanged.

I value SQLite's portability because embedding it should not require rebuilding a database service around the application. These changes show some of the work behind that portability. A familiar file or stream interface can hide platform behavior that a correct implementation must make explicit.

The storage implementation declares what it supports

The versioned public header defines SQLITE_IOCAP_SUBPAGE_READ. Its documented meaning is that a database file permits reads whose size is not a multiple of the page size and whose starting position is not a page boundary. A VFS reports this property through its device characteristics.

Without the capability, SQLite restricts itself to aligned full-page operations, apart from the documented subpage read of the first page for the database header. That exception matters because the capability should not be read as a promise that no short read occurs anywhere. The header access remains part of the stated contract.

The pager implementation uses the new property in deciding whether a direct page read is permissible. It checks that the file is open, that the VFS advertises the required support, that the page cache has no dirty pages, and that the relevant page does not currently come from the WAL. These conditions belong to different layers of the same decision.

A storage capability alone is insufficient if the newest relevant content lives elsewhere. Conversely, a clean cache and suitable database state do not establish that a custom VFS accepts an unaligned read. The added check closes that second gap. It prevents an optimization from assuming a behavior the storage implementation has not declared.

A changed default exposed an implicit assumption

The release traces the compatibility issue to direct overflow reads becoming the default in 3.45.0. An optimization that had previously required a particular configuration can reach more applications when enabled by default. Non-standard VFS implementations are among the affected combinations named in the release.

The new capability gives those implementations a direct way to express whether the optimization's access pattern is valid. A VFS that supports it can advertise the property. An implementation with stricter alignment requirements can omit it, allowing SQLite to choose the compatible path.

I prefer that explicit contract to an assumption based on how conventional operating-system files behave. SQLite's VFS abstraction exists to support different storage implementations. Optimization policy should use the capabilities of the selected implementation, rather than treating every implementation as equivalent in details the interface had not previously named.

Binary transport needs binary streams

The copy utility's source contains the Windows correction. Its child-process pipe streams use binary read and write modes. When the program operates as the remote side over standard input and output, it also switches those streams to binary mode.

This is consequential because the utility's protocol carries page contents as well as protocol messages. Text-mode treatment is inappropriate for arbitrary bytes. A line-ending conversion belongs to a textual representation, while a database page must arrive with its content preserved.

The release introduced sqlite3_rsync as an experimental utility in 3.47.0. November's fix therefore concerns a recently added program's platform integration. It does not announce a different replication model or a new SQL-level export format. The protocol remains a database-copying operation, with corrected Windows stream behavior.

That distinction helps explain why a successful process launch is not enough evidence that a transport implementation is correct. Both sides can start and exchange data while the runtime changes bytes between them. Selecting the right stream mode is part of preserving the protocol's meaning.

Installation and query results have separate corrections

The release also restores DESTDIR handling for installation. The versioned makefile passes the destination directory into the Tcl extension installation tooling. Staged installation needs the requested destination to reach the component that actually writes the installed files.

For packagers, the difference is where the build places its output. It is independent of database-file compatibility or how queries run. Including build-system fixes in the maintenance release matters because a library reaches applications through packaging workflows, not only through its execution engine.

The IN query correction addresses incorrect answers introduced by optimizations in 3.47.0. The release describes the affected queries as obscure but does not provide a universal claim that all IN expressions are wrong. Its scope should remain tied to the identified optimization regression.

These fixes therefore reach different users. A custom storage implementation needs the VFS capability contract, a Windows user of the new copy tool needs binary transport behavior, and a packager needs installation paths honored. Applications affected by the query regression need corrected execution results. Version 3.47.1 repairs each boundary at the component responsible for implementing it.