July 7, 2018·4 min read·engineeringsqlitedatasettehttp

Datasette repairs CSV access and row URLs

Datasette 0.23.2 makes CSV exports respect the CORS option and repairs row pages for table names containing slashes. Both fixes concern the HTTP representation of SQLite data.

A published database has more than one interface. A person may follow a row link in a browser, while JavaScript elsewhere fetches an export. Both can reach the same data through different paths, so a working table page does not establish that every representation works. Datasette 0.23.2 repairs two such inconsistencies at the boundary between SQLite names and HTTP.

The versioned changelog dates the release to 7 July 2018. The release notes identify a fix that makes CSV export respect the server's CORS option and another for row pages belonging to tables whose names contain slashes. Neither changes SQLite's query engine. Each makes an existing publishing feature work for a case that its surrounding interface already implies should be supported.

I value SQLite's ability to carry a dataset in a file. A publishing tool needs to preserve that usefulness when the dataset acquires URLs and response headers. These fixes are interesting because the failures arise after the database has already done its work.

CSV needs the same cross-origin policy as JSON

A browser can display a resource through direct navigation while refusing to let JavaScript on another origin read the response. CORS supplies the HTTP permission for that second operation. A CSV export that works as a download can therefore fail as the input to a browser application.

The CSV response code in the release now adds Access-Control-Allow-Origin: * when the Datasette instance enables CORS. That condition matters. This patch makes CSV honor an explicit server choice instead of enabling cross-origin access unconditionally.

The code also shows why this behavior needed attention in the export implementation. CSV has a response path that streams output and constructs its own headers. The same module has separate response handling for JSON. A header applied in one branch does not automatically become a property of every response produced by the application.

For a consumer, the expected result is consistency. If a publisher chooses to allow cross-origin reads, changing the requested representation from JSON to CSV should not unexpectedly remove that permission. The data query can remain identical while the browser's ability to expose the response changes with the headers.

This is an HTTP policy decision, not a database privilege. The header describes which browser origins can read a response. It does not grant SQL write access or alter the permissions of the SQLite file. Keeping those distinctions clear makes the fix easier to understand without overstating its security implications.

Slashes inside names must remain data

A slash has structural meaning in a URL path. A slash inside a SQLite table name is instead part of an identifier. When a publisher maps that identifier into a path, it must preserve the distinction through encoding and link generation.

The CSV documentation at the release tag includes a table named table/with/slashes.csv. Its URL encodes the internal slashes as %2F. That example also illustrates a separate ambiguity caused by names that end with a format suffix.

Datasette normally selects a representation by appending .csv or .json to a table URL. A table whose actual name ends in one of those strings makes the suffix ambiguous. The documentation provides the _format query parameter as an alternative way to select the output format.

These rules become particularly important for row pages. A table page and a row page must agree on where the table identifier ends and where the row identifier begins. The release's row-page fix extends support to names containing slashes, preserving a linkable row beneath a table name that is already valid in the dataset.

Renaming imported tables to avoid URL punctuation would impose a web transport restriction on the database schema. Correct URL construction is preferable when the purpose of the tool is to publish an existing SQLite file. The HTTP layer should represent the identifier the database actually contains.

Export size and export access remain independent

The tagged CSV documentation describes export for tables, views and custom SQL queries. It also distinguishes a page-sized export from the option to stream all matching rows. Streaming uses pagination internally to continue through the result set while returning one CSV response.

That streaming option has its own configured size limit. The documented default cap is 100 MB, with a truncation error message if output exceeds it. The CORS repair does not change that resource policy. A browser can gain permission to read an export that still stops at the server's configured limit.

CSV can also include labels for foreign key references. The documentation describes extra label columns beside the original identifier columns. That preserves the identifier while making the export easier to interpret outside the database, where the referenced table may not be immediately available.

I regard these details as part of a usable data interface. The row URL must identify the intended record, and the response must carry the access policy selected by the publisher. Version 0.23.2 repairs those paths while retaining the existing export controls and the database's original table names.