Python 3.11.4 adds a policy boundary to tar extraction

A tar archive can describe more than a collection of file contents. Paths, links and permission metadata tell an extractor how to reconstruct parts of a filesystem. That is useful for backups and potentially surprising for an application accepting an archive as ordinary input data. Python 3.11.4 introduces an extraction filter argument that lets callers distinguish those uses.

The June 6 release is the fourth maintenance update in the 3.11 series. Its versioned changelog records the new argument on tarfile extraction methods and shutil.unpack_archive(). The tarfile documentation shipped in the release explains the policies and their limits.

I appreciate a standard-library change that makes an application decision explicit. Extracting a filesystem backup and unpacking data received from elsewhere have different requirements. A single implicit policy makes that difference hard to see in the calling code.

Archive members can affect later members

Checking a filename for a parent-directory component does not account for every way extraction can escape its destination. A symbolic link created by an earlier archive member can change where a later path resolves. An absolute path can also name a location outside the intended directory. The archive format permits behavior that a data importer may never need.

Extraction filters operate immediately before each member is extracted. A callable receives the member metadata and the destination directory, so it can take the current disk state into account. It may return modified metadata, skip the member by returning None, or raise an exception. This puts policy at the point where the extractor is about to perform the filesystem operation.

The timing is part of the design. A preliminary scan sees names and metadata, but extraction itself changes the destination. Evaluating members as they are processed allows policy to consider effects that earlier members have already produced. It also allows metadata to be reduced to the subset the application intends to preserve.

The data policy removes filesystem-specific authority

The named data filter builds on the tar filter. The latter strips leading slashes, refuses paths that remain absolute and rejects resolved destinations outside the extraction directory. It also clears high permission bits and group or other write permissions.

The data policy further rejects links whose targets are absolute or outside the destination. Device files and pipes are refused. User and group metadata become None, so the extraction methods skip applying ownership information. Directory permission metadata is also omitted.

For regular files, the policy sets owner read and write permissions. It removes group and other execute permissions when the owner execute bit is absent. These rules aim to retain useful data while avoiding several Unix filesystem behaviors that a cross-platform data archive does not require.

The release supports selecting the policy at the extraction call. This example requests the data policy explicitly on Python 3.11.4.

import tarfile

with tarfile.open("incoming.tar.gz", "r:gz") as archive:
    archive.extractall("staging", filter="data")

The destination still matters. A fresh temporary directory is the documented recommendation, partly because pre-existing links can complicate extraction and partly because failed work becomes easier to remove. The example names a staging location, but the filter does not create an isolation boundary around the entire process.

Compatibility keeps the old default

Leaving filter as None consults TarFile.extraction_filter. If that is also unset, this release uses fully_trusted. That policy returns member metadata unchanged. Upgrading to 3.11.4 therefore makes restricted extraction available without silently selecting it for existing callers.

This is an important limitation when reading the changelog. The presence of the new argument does not establish that an existing application now rejects escaping paths. The actual call or configured extraction policy determines that behavior. A maintenance release can provide a safer mechanism while retaining compatibility for applications using the full tar format.

I prefer the explicit argument for code that treats an archive as data. It documents the intended restriction next to the operation that writes files. Code requiring a complete filesystem restoration can make its different trust assumption explicit as well.

Refusing a member does not roll back extraction

Filter errors normally abort extraction when errorlevel is at least one. At zero, the error is logged and extraction continues without that member. An aborted operation can leave earlier members on disk, and tarfile does not clean them up automatically.

The filters also do not prevent resource exhaustion. An archive can contain too many files or too much data while satisfying the path and metadata rules. The release documentation calls for separate limits on resources and notes that later archive members may overwrite earlier ones. An application importing a bounded dataset therefore still needs a definition of acceptable size and contents.

These limits describe the scope of the mechanism. A filter constrains the filesystem actions represented by each member. It does not turn extraction into a transaction or decide whether the resulting data is meaningful to the application. That separation gives callers a reusable restriction mechanism without pretending that every archive consumer has the same validation requirements.