// This module is included in the following assemblies:
//
// upgrading/assembly_upgrade-kafka-versions.adoc

[id='proc-upgrading-brokers-newer-kafka-{context}']

= Upgrading Kafka brokers and client applications

This procedure describes how to upgrade a Strimzi Kafka cluster to the latest supported Kafka version.

Compared to your current Kafka version, the new version might support a higher _log message format version_ or _inter-broker protocol version_, or both.
Follow the steps to upgrade these versions, if required. 
For more information, see xref:ref-kafka-versions-{context}[].

You should also choose a xref:con-strategies-for-upgrading-clients-{context}[strategy for upgrading clients]. 
Kafka clients are upgraded in step 6 of this procedure.

.Prerequisites

For the `Kafka` resource to be upgraded, check that:

* The Cluster Operator, which supports both versions of Kafka, is up and running.
* The `Kafka.spec.kafka.config` does _not_ contain options that are not supported in the new Kafka version.

.Procedure

. Update the Kafka cluster configuration:
+
[source,shell,subs=+quotes]
----
kubectl edit kafka _my-cluster_
----

. If the `log.message.format.version` and `inter.broker.protocol.version` of the _current_ Kafka version are _the same_ as the new Kafka version, go to step 3.
+
Otherwise, ensure that `Kafka.spec.kafka.config` has the `log.message.format.version` and `inter.broker.protocol.version` configured to the defaults for the _current_ Kafka version.
+
For example, if upgrading from Kafka version {KafkaVersionLower} to {KafkaVersionHigher}:
+
[source,yaml,subs=attributes+]
----
kind: Kafka
spec:
  # ...
  kafka:
    version: {KafkaVersionLower}
    config:
      log.message.format.version: "{LogMsgVersLower}"
      inter.broker.protocol.version: "{LogMsgVersLower}"
      # ...
----
+
NOTE: The value of `log.message.format.version` and `inter.broker.protocol.version` must be strings to prevent them from being interpreted as floating point numbers.

. Change the `Kafka.spec.kafka.version` to specify the new Kafka version; leave the `log.message.format.version` and `inter.broker.protocol.version` at the defaults for the _current_ Kafka version.
+
[NOTE]
====
Changing the `kafka.version` ensures that all brokers in the cluster will be upgraded to start using the new broker binaries. 
During this process, some brokers are using the old binaries while others have already upgraded to the new ones. 
Leaving the `inter.broker.protocol.version` unchanged ensures that the brokers can continue to communicate with each other throughout the upgrade. 
====
+
For example, if upgrading from Kafka {KafkaVersionLower} to {KafkaVersionHigher}:
+
[source,yaml,subs=attributes+]
----
apiVersion: {KafkaApiVersion}
kind: Kafka
spec:
  # ...
  kafka:
    version: {KafkaVersionHigher} <1>
    config:
      log.message.format.version: "{LogMsgVersLower}" <2>
      inter.broker.protocol.version: "{LogMsgVersLower}" <3>
      # ...
----
<1> Kafka version is changed to the new version.
<2> Message format version is unchanged.
<3> Inter-broker protocol version is unchanged.
+
WARNING: You cannot downgrade Kafka if the `inter.broker.protocol.version` for the new Kafka version changes. The inter-broker protocol version determines the schemas used for persistent metadata stored by the broker, including messages written to `__consumer_offsets`. The downgraded cluster will not understand the messages.

. If the image for the Kafka cluster is defined in the Kafka custom resource, in `Kafka.spec.kafka.image`, update the `image` to point to a container image with the new Kafka version.
+
See xref:con-versions-and-images-str[Kafka version and image mappings]

. Save and exit the editor, then wait for rolling updates to complete.
+
Check the progress of the rolling updates by watching the pod state transitions:
+
[source,shell,subs=+quotes]
----
kubectl get pods my-cluster-kafka-0 -o jsonpath='{.spec.containers[0].image}'
----
+
The rolling updates:
+
* Ensure each pod is using the broker binaries for the new version of Kafka
* Configure the brokers to send messages using the inter-broker protocol version of the new version of Kafka

. Depending on your chosen xref:con-strategies-for-upgrading-clients-{context}[strategy for upgrading clients], upgrade all client applications to use the new version of the client binaries.
+
If required, set the `version` property for Kafka Connect and MirrorMaker as the new version of Kafka:
+
.. For Kafka Connect, update `KafkaConnect.spec.version`.
.. For MirrorMaker, update `KafkaMirrorMaker.spec.version`.
.. For MirrorMaker 2.0, update `KafkaMirrorMaker2.spec.version`.

. _Optional step to reduce rolling updates_:
+
You can now perform one or more further product upgrades, starting with xref:proc-upgrading-the-co-{context}[]. 
After reaching this step for the last upgrade you want to perform, you can complete this procedure.
+
Alternatively, skip this step and go to step 8.
+
. If the `log.message.format.version` and `inter.broker.protocol.version` of the _new_ Kafka version are _different from_ the old Kafka version, update the Kafka resource to use the new default versions. Otherwise, go to step 9.
+
In `Kafka.spec.kafka.config`, change the `log.message.format.version` and `inter.broker.protocol.version`.
+
For example, if upgrading to Kafka {KafkaVersionHigher}:
+
[source,yaml,subs=attributes+]
----
apiVersion: {KafkaApiVersion}
kind: Kafka
spec:
  # ...
  kafka:
    version: {KafkaVersionHigher}
    config:
      log.message.format.version: "{LogMsgVersHigher}"
      inter.broker.protocol.version: "{InterBrokVersHigher}"
      # ...
----

. Wait for the Cluster Operator to update the cluster.
+
The Kafka cluster and clients are now using the new Kafka version.

Following the Kafka upgrade, if required, you can:

* xref:con-upgrade-listeners-{context}[Update listeners to the `GenericKafkaListener` schema]
* xref:proc-upgrading-consumers-streams-cooperative-rebalancing_{context}[Upgrade consumers to use the incremental cooperative rebalance protocol]
* xref:assembly-upgrade-resources-{context}[Update existing custom resources] 