Cassandra storage ports and the move to one internode listener

Cassandra 4.0 allows encrypted and unencrypted internode connections to use the same storage port. That changes how a deployment configures its network rules, but the port number alone does not tell an operator whether connections require encryption. The distinction matters when translating an older configuration into a network policy for a 4.0 cluster.

Chris Thornett's January 14 explanation of configurable storage ports describes the change from separate plaintext and encrypted listeners. The practical detail is in the Cassandra 4.0.1 configuration file. Its comments separate the port used for communication from the rules governing outgoing encryption and incoming connections. Reading those settings together gives a more useful account of the change than treating port 7000 as synonymous with plaintext traffic.

I find Cassandra's active-active design compelling when the application needs it. The network configuration deserves the same attention as the data model, because the participating nodes must communicate through the addresses and ports that the deployment actually makes reachable.

Separate the client listener from node communication

The tagged configuration assigns storage_port the value 7000. It describes this as the TCP port for commands and data between nodes. The native transport used by CQL clients has a separate setting, native_transport_port, whose default is 9042. Moving the storage port therefore does not move the client listener. These are independent configuration decisions with different callers.

For example, a deployment that has a reason to move the internode listener can use the following excerpt from its node configuration. The value 7100 is an illustrative choice, not a recommended alternative default.

storage_port: 7100
native_transport_port: 9042

This fragment specifies port numbers only. It does not establish a usable node address or enable encryption. The distinction also matters to firewall rules. A rule allowing application clients to reach the native transport port says nothing about whether peer nodes can reach the storage listener. Conversely, permitting the storage port is not an instruction to expose it to application clients.

The configuration explicitly warns against exposing either listener to the internet. A deployment can preserve that separation even when both kinds of traffic cross the same physical network. The relevant question for each rule is which process needs to initiate the connection and which Cassandra listener must accept it.

Listening and advertising are different settings

The same configuration file distinguishes listen_address from broadcast_address. For internode communication, Cassandra binds the first address. Peers receive the second address through its advertisements. Leaving broadcast_address unset makes it use the listening address.

That default is straightforward when the address on the server is also reachable by every peer. Networks with different internal and external addressing need a more explicit description. The presence of a listening socket proves that the local process has bound an address. It does not establish that another machine can route to the address Cassandra advertises.

There are constraints even before routing enters the discussion. The tagged file says to choose either listen_address or listen_interface, not both. It also says that 0.0.0.0 is not a valid choice for listen_address. A wildcard binding convention copied from another service's configuration is therefore inappropriate here. Cassandra needs an address that has meaning to its peers.

The listen_on_broadcast_address setting addresses another case. It can make Cassandra listen on the broadcast address in addition to the listening address when multiple physical interfaces are involved. The comments distinguish that case from environments whose networking already routes between public and private networks. These controls describe different parts of connectivity, so changing one does not automatically resolve a problem in another.

Encryption policy survives the port consolidation

The 4.0.1 configuration retains ssl_storage_port, with a default of 7001, for legacy encrypted communication. Its comment calls the setting deprecated because one port can now serve secure and insecure connections. The separate legacy listener opens only when the corresponding option under server_encryption_options enables it.

The following excerpt shows the legacy-listener setting in context. It is a fragment to merge into the existing encryption configuration, not a complete TLS setup.

server_encryption_options:
    enable_legacy_ssl_storage_port: false

The tagged comments reserve that legacy listener for the upgrade to 4.0. Disabling it removes that compatibility listener. It does not specify which peer connections require encryption. That policy lives in internode_encryption, whose documented choices include none, dc, rack, and all. The comments define these choices in terms of outgoing connections, including whether peers in other datacenters or racks require encryption.

Incoming acceptance has its own control. The optional setting allows encrypted and unencrypted connections on storage_port when true. The configuration limits that mixed mode to unencrypted or transitional operation. A port that accepts TLS can consequently still accept plaintext if the incoming policy permits it. Network rules that recognize only a port number cannot express this Cassandra setting.

The distinction explains the documented transition procedure. The comments first direct the operator to configure the intended encryption scope with optional: true and restart all nodes. They then direct the operator to remove mixed acceptance by setting optional: false, or removing it as appropriate, and restarting the nodes again. The intermediate state gives the cluster a period in which both forms of incoming connection work while the outgoing configuration changes.

I prefer this configuration to a permanent dependency on separate encrypted and unencrypted port numbers. It makes the transport policy explicit, although it also means that a firewall rule naming the storage port cannot stand in for a review of the encryption settings.

Certificate verification remains explicit as well. The configuration has separate controls for requiring client authentication and verifying that a peer's hostname matches its certificate. Its keystore and truststore settings supply the associated material. Consolidating listeners does not remove any of those decisions. The final configuration must describe the intended peer authentication and encryption behavior as well as the address on which Cassandra listens.