D1 public beta raises limits on its new storage backend

A managed SQLite service needs room for more than a prototype before an application can evaluate its data model seriously. The September 28 D1 release entry opens public beta and raises the paid-plan database size limit from 500 MB to 2 GB. It also records an increase from ten databases per account to 25 at this point in the rollout.

The release requires the new storage subsystem for the increased limits. Workers Free accounts retain the 500 MB per-database limit and up to ten databases. A plan change and a storage-backend change therefore answer different questions. The plan determines the announced capacity, while the backend determines whether a particular database can use it.

I like SQLite's ability to keep data in a database that matches an application's actual scale. D1 extends that design into a managed service, where the database count and capacity limits become part of the application architecture. The beta matters because those limits now accommodate a wider range of layouts.

Capacity is more than the largest database

The beta announcement explains Cloudflare's interest in database-per-customer applications. Separate databases can keep customer data apart and allow placement decisions to follow where customers access that data. That is an application design option, not a requirement to split every dataset.

A larger per-database limit and a larger count serve different designs. An application with one growing relational dataset benefits from more space in the same database. An application with many independent customer datasets benefits from being allowed to create more databases. Increasing one limit does not substitute for increasing the other.

The announcement also explains why raising size limits needs care. A larger database is useful only if the service can make it available with acceptable startup behavior. Cloudflare presents readiness and performance as constraints on opening access to larger databases. The stated capacity should therefore be read as a supported service boundary, rather than an estimate of SQLite's theoretical file-size limit.

The dated release entry is the clearest source for the initial beta limits. It also says those limits will continue increasing during beta. An application design needs the limits that apply to its account at the time, even when the project's announced direction is to permit larger deployments.

The backend determines recovery behavior too

The new subsystem also supports Time Travel, made available in July. The September announcement describes a bookmark as a representation of a database's state at a particular point in time. A caller can select a timestamp or a bookmark to request restoration.

The connection to database layout is practical. Restoring one customer's database can limit the data affected by that operation to that database. A shared database has a different recovery scope. That observation does not make either layout universally preferable, because the application also has to manage the relationships its data requires.

The announcement says a restore does not prevent selecting an earlier point afterward. It illustrates an incorrect update followed by a restore and a check of the repaired result. This is more specific than a general claim that backups exist. It describes how an operator can choose a historical state after an application mistake.

Time Travel still needs an application-level explanation of the intended state. A database restore cannot decide which later writes should be preserved as valid business actions. The useful property is retained historical state, with the restore scope determined by the database that the application chooses to operate on.

Rows make query work visible

Another substantial part of the beta is its row-based usage model. Cloudflare explains that this replaces the earlier proposal based on byte units. It also exposes the number of rows read and written by a query, giving application developers a way to relate SQL execution to the announced accounting model.

The August row-count release entry documents rows_read and rows_written in the client API's returned metadata. That is query metadata, not a count inferred from the number of objects returned to application code. A filter may read more rows than it returns.

Cloudflare demonstrates the distinction with a query on the Northwind order data. Its example filters by shipment date and initially reports 16,818 rows read. Adding an index on the filtered date column reduces the reported read count to 417 for the same query. Those are the company's example measurements, not a benchmark of another application's schema.

The SQL behind the index is conventional.

CREATE INDEX IF NOT EXISTS idx_orders_date
ON [Order](ShippedDate);

The index gives the engine an access structure corresponding to the predicate. Whether another query benefits depends on its schema, data, and chosen plan. The important addition in D1 is that the returned work measurement lets the caller inspect the consequence in the service's own accounting.

Index choice affects two costs

An index that reduces scanned data can improve query execution and reduce metered reads. That connects an ordinary SQL design decision to the service's usage model. Returning fewer rows without changing the access method does not necessarily have the same effect.

Indexes also require maintenance when indexed data changes. The beta's row accounting should therefore be interpreted as part of a workload with both reads and writes. The Northwind example establishes a read-side benefit for a particular predicate. It does not justify adding an index for every column without considering how the application changes its data.

My preference is for service accounting to expose work that developers can connect to a concrete query. The row metadata makes that connection more direct than an opaque database-hours charge. It also gives a reason to inspect a query's plan when its output is small but its read count is large.

Cloudflare also provides analytics through the dashboard and GraphQL API. Per-query metadata helps during investigation of one statement, while database analytics provide activity over time. They represent different views of the same application's use of the managed database.

The beta increases the scale available to applications while making that use more observable. Database capacity remains conditional on the new backend and account plan. Within those boundaries, row accounting provides a concrete way to evaluate how an application's SQL uses the service.