假设所有具有分区leader和副本的节点都崩溃了,并且节点不可恢复。
如何选择另一个Kafka broker作为该分区的leader ?(我很好与数据丢失)
我尝试了"kafka-preferred-replica-election.sh"one_answers"kafka- reassignment -partitions.sh"但是没有成功。
设置:
- 双节点kafka集群,代理id为10和11。
- 主题"test",复制因子为1,分区为1。Broker 11是主题"test"分区0的领导者
- Broker 11崩溃 Kafka版本- kafka_2.11-0.10.1.0
主题描述
Topic:test PartitionCount:1 ReplicationFactor:1 Configs:
Topic: test Partition: 0 Leader: -1 Replicas: 11 Isr:
kafka-reassign-partitions.sh输出:
bin/kafka-reassign-partitions.sh --zookeeper myvm:2181 --reassignment-json-file manual-assignment.json --execute
Partitions reassignment failed due to The proposed assignment contains non-existent partitions: List([test,10])
kafka.common.AdminCommandFailedException: The proposed assignment contains non-existent partitions: List([test,10])
at kafka.admin.ReassignPartitionsCommand$.parseAndValidate(ReassignPartitionsCommand.scala:201)
at kafka.admin.ReassignPartitionsCommand$.executeAssignment(ReassignPartitionsCommand.scala:153)
at kafka.admin.ReassignPartitionsCommand$.executeAssignment(ReassignPartitionsCommand.scala:149)
at kafka.admin.ReassignPartitionsCommand$.main(ReassignPartitionsCommand.scala:46)
at kafka.admin.ReassignPartitionsCommand.main(ReassignPartitionsCommand.scala)
cat manual-assignment.json
{
"partitions":
[
{"topic": "test", "partition": 10, "replicas": [10]}
],
"version":1
}
kafka-preferred-replica-election.sh
bin/kafka-preferred-replica-election.sh --zookeeper myvm:2181 --path-to-json-file topicPartitionList.json
[2016-11-08 03:27:01,993] ERROR Skipping preferred replica leader election for partition [test,10] since it doesn't exist (kafka.admin.PreferredReplicaLeaderElectionCommand)
Successfully started preferred replica election for partitions Set([test,10])
cat topicPartitionList.json
{
"partitions":
[
{"topic": "test", "partition": 10}
]
}
Kafka只从ISR和分配的副本中选举leader(当启用不洁leader选举时)。在复制因子设置为1的情况下,在领导代理崩溃后,ISR中没有剩余的代理。没有候选人可以当选为新的领导人,所以这是一个设计。