// Module included in the following assemblies:
//
// configuring/assembly-scaling-clusters.adoc

[id='con-partition-reassignment-{context}']

= Partition reassignment tool

[role="_abstract"]
The Topic Operator does not currently support reassigning replicas to different brokers, so it is necessary to connect directly to broker pods to reassign replicas to brokers.

Within a broker pod, the `kafka-reassign-partitions.sh` tool allows you to reassign partitions to different brokers.

It has three different modes:

`--generate`::
Takes a set of topics and brokers and generates a _reassignment JSON file_ which will result in the partitions of those topics being assigned to those brokers.
Because this operates on whole topics, it cannot be used when you only want to reassign some partitions of some topics.

`--execute`::
Takes a _reassignment JSON file_ and applies it to the partitions and brokers in the cluster.
Brokers that gain partitions as a result become followers of the partition leader.
For a given partition, once the new broker has caught up and joined the ISR (in-sync replicas) the old broker will stop being a follower and will delete its replica.

`--verify`::
Using the same _reassignment JSON file_ as the `--execute` step, `--verify` checks whether all the partitions in the file have been moved to their intended brokers.
If the reassignment is complete, `--verify` also removes any traffic throttles (`--throttle`) that are in effect.
Unless removed, throttles will continue to affect the cluster even after the reassignment has finished.

It is only possible to have one reassignment running in a cluster at any given time, and it is not possible to cancel a running reassignment.
If you need to cancel a reassignment, wait for it to complete and then perform another reassignment to revert the effects of the first reassignment.
The `kafka-reassign-partitions.sh` will print the reassignment JSON for this reversion as part of its output.
Very large reassignments should be broken down into a number of smaller reassignments in case there is a need to stop in-progress reassignment.

== Partition reassignment JSON file

The _reassignment JSON file_ has a specific structure:

[source,subs=+quotes]
----
{
  "version": 1,
  "partitions": [
    _<PartitionObjects>_
  ]
}
----

Where _<PartitionObjects>_ is a comma-separated list of objects like:

[source,subs=+quotes]
----
{
  "topic": _<TopicName>_,
  "partition": _<Partition>_,
  "replicas": [ _<AssignedBrokerIds>_ ]
}
----

NOTE: Although Kafka also supports a `"log_dirs"` property this should not be used in Strimzi.

The following is an example reassignment JSON file that assigns partition `4` of topic `topic-a` to brokers `2`, `4` and `7`, and partition `2` of topic `topic-b` to brokers `1`, `5` and `7`:

.Example partition reassignment file
[source,json]
----
{
  "version": 1,
  "partitions": [
    {
      "topic": "topic-a",
      "partition": 4,
      "replicas": [2,4,7]
    },
    {
      "topic": "topic-b",
      "partition": 2,
      "replicas": [1,5,7]
    }
  ]
}
----

Partitions not included in the JSON are not changed.

== Partition reassignment between JBOD volumes

When using JBOD storage in your Kafka cluster, you can choose to reassign the partitions between specific volumes and their log directories (each volume has a single log directory).
To reassign a partition to a specific volume, add the `log_dirs` option to _<PartitionObjects>_ in the reassignment JSON file.

[source,subs=+quotes]
----
{
  "topic": _<TopicName>_,
  "partition": _<Partition>_,
  "replicas": [ _<AssignedBrokerIds>_ ],
  "log_dirs": [ _<AssignedLogDirs>_ ]
}
----

The `log_dirs` object should contain the same number of log directories as the number of replicas specified in the `replicas` object.
The value should be either an absolute path to the log directory, or the `any` keyword.

.Example partition reassignment file specifying log directories
[source,subs=+quotes]
----
{
      "topic": "topic-a",
      "partition": 4,
      "replicas": [2,4,7].
      "log_dirs": [ "/var/lib/kafka/data-0/kafka-log2", "/var/lib/kafka/data-0/kafka-log4", "/var/lib/kafka/data-0/kafka-log7" ]
}
----

[discrete]
= Partition reassignment throttles

Partition reassignment can be a slow process because it involves transferring large amounts of data between brokers.
To avoid a detrimental impact on clients, you can throttle the reassignment process.
Use the `--throttle` parameter with the `kafka-reassign-partitions.sh` tool to throttle a reassignment.
You specify a maximum threshold in bytes per second for the movement of partitions between brokers.
For example, `--throttle 5000000` sets a maximum threshold for moving partitions of 50 MBps.

Throttling might cause the reassignment to take longer to complete.

* If the throttle is too low, the newly assigned brokers will not be able to keep up with records being published and the reassignment will never complete.
* If the throttle is too high, clients will be impacted.

For example, for producers, this could manifest as higher than normal latency waiting for acknowledgment.
For consumers, this could manifest as a drop in throughput caused by higher latency between polls.
