kafka单节点安装

Kafka 2.8.0 版本移除了对Zookeeper的依赖，该版本将依赖于 ZooKeeper 的控制器改造成了基于 Kafka Raft 的 Quorm 控制器。

[TOC]

安装jdk

• 安装 jdk 1.8.0_391

curl -sL http://iso.sqlfans.cn/jdk/install_jdk_8u391.sh | bash
source /etc/bashrc
java -version

安装 kafka

• 1.下载 kafka 3.5.0

curl -sL http://iso.sqlfans.cn/linux/kafka_2.12-3.5.0.tgz -o /opt/kafka_2.12-3.5.0.tgz
tar -xzf /opt/kafka_2.12-3.5.0.tgz -C /opt/
mv /opt/kafka_2.12-3.5.0 /data/kafka_9092

• 2.配置 server.properties

cp /data/kafka_9092/config/kraft/server.properties /data/kafka_9092/config/kraft/server.properties.bak.$(date +%Y%m%d%H%M%S)
sed -i 's/^process.roles=.*/process.roles=broker,controller/' /data/kafka_9092/config/kraft/server.properties
sed -i 's/^controller.quorum.voters=.*/controller.quorum.voters=1@10.30.3.231:9093/' /data/kafka_9092/config/kraft/server.properties
sed -i 's/^log.dirs=.*/log.dirs=\/data\/kafka_9092\/logs/' /data/kafka_9092/config/kraft/server.properties

sed -i 's/^node.id=.*/node.id=1/' /data/kafka_9092/config/kraft/server.properties
sed -i 's/^listeners=.*/listeners=PLAINTEXT:\/\/10.30.3.231:9092,CONTROLLER:\/\/10.30.3.231:9093/' /data/kafka_9092/config/kraft/server.properties
sed -i 's/^advertised.listeners=.*/advertised.listeners=PLAINTEXT:\/\/10.30.3.231:9092/' /data/kafka_9092/config/kraft/server.properties
cat /data/kafka_9092/config/kraft/server.properties | grep -Ev '^$|#' | egrep "(process.|node.id|controller.|listeners|log.dir|sadvertised.)"

• 3.启动进程

KAFKA_CLUSTER_ID="$(/data/kafka_9092/bin/kafka-storage.sh random-uuid)"
/data/kafka_9092/bin/kafka-storage.sh format -t $KAFKA_CLUSTER_ID -c /data/kafka_9092/config/kraft/server.properties
nohup /data/kafka_9092/bin/kafka-server-start.sh /data/kafka_9092/config/kraft/server.properties > /data/kafka_9092/logs/kafka_startup.log 2>&1 &
sleep 5 && netstat -lnpt | egrep "(9092|9093)"

• 4.测试

# 创建topic：指定broker及1个partition及3个副本
/data/kafka_9092/bin/kafka-topics.sh --create --bootstrap-server 10.30.3.231:9092 --replication-factor 1 --partitions 1 --topic test1
/data/kafka_9092/bin/kafka-topics.sh --list --bootstrap-server 10.30.3.231:9092
/data/kafka_9092/bin/kafka-topics.sh --describe --topic test1 --bootstrap-server 10.30.3.231:9092

# 生产消息：写入2条测试消息
echo "message123" | /data/kafka_9092/bin/kafka-console-producer.sh --topic test1 --bootstrap-server 10.30.3.231:9092
echo "message456" | /data/kafka_9092/bin/kafka-console-producer.sh --topic test1 --bootstrap-server 10.30.3.231:9092

# 消费消息：设定从头消费，执行 Ctrl-C 随时停止
/data/kafka_9092/bin/kafka-console-consumer.sh --topic test1 --from-beginning --bootstrap-server 10.30.3.231:9092

# 删除topic
/data/kafka_9092/bin/kafka-topics.sh --bootstrap-server 10.30.3.231:9092 --delete --topic test1

如何彻底卸载kafka

cd /opt/
ps aux | grep kafka | grep -v grep | awk '{print $2}' | xargs kill -9 2> /dev/null
rm -rf /data/kafka_9092