Strimzi adds separate controls for container resources and JVM heap

A Java heap limit does not describe all the memory a Kafka broker needs, and a Kubernetes memory request does not configure the Java heap. Strimzi 0.4 exposes these controls separately. Its May 15 release adds resource requests and limits alongside JVM memory configuration for the deployed components. That makes the distinction part of the declared cluster configuration.

The release also renames the Cluster Controller and Topic Controller to operators and updates Kafka to 1.1.0. More consequentially for existing declarations, several dedicated configuration fields move into general Kafka and Connect configuration objects. This is an interface change that accompanies the new controls, so an unchanged old ConfigMap cannot be assumed to express the same settings to the new operator.

I value Kafka's durable log because it lets consumers maintain independent positions and derive independent results. The broker providing that history still runs inside a process with resource requirements. Expressing its scheduling needs and heap policy separately makes the deployment's assumptions easier to inspect.

Scheduling resources and heap resources have different consumers

The 0.4 Cluster Operator documentation defines kafka-resources as a JSON string containing Kubernetes resource requests and limits. Each can contain CPU and memory values using Kubernetes quantity syntax. The operator maps those settings into the broker container's resource configuration.

Requests participate in placement and resource allocation. Limits bound consumption, with a memory overrun potentially causing termination and a CPU limit capping CPU use. These settings describe how the scheduler and runtime treat the container. They do not directly state how much heap the Java process should allocate.

The separate kafka-jvmOptions field configures supported JVM options, including -Xmx for maximum heap and -Xms for initial heap. Corresponding resource and JVM fields exist for ZooKeeper, with similar controls for Connect. The distinction between container and JVM configuration therefore applies to more than the broker alone.

An excerpt from a cluster ConfigMap's data section can express both kinds of input. These values illustrate the syntax and separation, rather than prescribing a broker size.

data:
  kafka-resources: |-
    {
      "requests": {"memory": "5Gi", "cpu": "1"},
      "limits": {"memory": "5Gi", "cpu": "2"}
    }
  kafka-jvmOptions: |-
    {"-Xmx": "1g", "-Xms": "1g"}

The units differ deliberately. The documented JVM accepts g or G for binary gigabytes, while Gi is not a valid JVM suffix. Kubernetes distinguishes G from Gi. Copying a quantity unchanged between the fields can therefore change its meaning or make it invalid even when both fields concern memory.

Heap size leaves other memory requirements

The documentation explains that the JVM uses memory beyond its maximum heap. It also identifies operating system page cache as relevant for containers performing substantial disk I/O, including Kafka brokers. Assigning the entire container memory budget to -Xmx would leave those other uses unaccounted for.

Strimzi's default heap selection depends on whether the container has a memory limit and on the component being deployed. Explicitly configuring the heap overrides part of that automatic choice. The resource configuration and JVM configuration therefore need to be read together to understand the resulting process.

Matching initial and maximum heap sizes avoids later heap expansion, with the cost of allocating more heap initially than may be needed. The documentation discusses that tradeoff in terms of latency for Kafka and ZooKeeper and aggregate allocation for distributed Connect workers. It does not make one pair of heap values a universal optimum for every deployed component.

I find the separation more useful than a single memory setting that conceals the other assumptions. An operator can identify the amount requested from Kubernetes and the maximum heap requested from Java. Those are two inspectable decisions, and the remaining process and page-cache needs have an explicit place in the capacity calculation.

Configuration becomes broader while retaining managed fields

The new kafka-config object replaces the earlier dedicated replication-factor fields. connect-config similarly replaces several environment-style Connect keys. The release notes enumerate these incompatible changes, including the Connect bootstrap and converter settings. Existing definitions need to express those values in the new object format. The values can be strings, numbers or booleans, so the configuration describes the property value directly instead of encoding every individual option as a separate top-level field.

The broader object does not permit arbitrary replacement of every operator-managed property. The documentation excludes listener and broker identity settings, data directories and other properties that the operator controls. This maintains a boundary between application-level broker tuning and the configuration needed to construct the managed deployment.

The controller-to-operator rename also affects roles, bindings and service accounts. The release explicitly says that those existing resources cannot be reused from previous deployments. This is why the naming change belongs in the deployment account, even though it sounds cosmetic in isolation. The objects that grant the management process its permissions are part of the installation interface, alongside the ConfigMap fields used to describe its workload.