Kafka 高可用部署
解压并放到对应目录
1
2
| tar xf kafka_2.12-2.8.0.tgz
mv kafka_2.12-2.8.0 /opt
|
zookeeper 相关配置
1
2
3
4
5
6
7
8
9
10
11
| vim /opt/kafka_2.12-2.8.0/config/zookeeper.properties
dataDir=/data/zookeeper
clientPort=2181
maxClientCnxns=0
tickTime=2000
initLimit=20
syncLimit=10
server.1=192.168.8.204:2888:3888
server.2=192.168.8.205:2888:3888
server.3=192.168.8.206:2888:3888
|
kafka 相关配置
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
| vim /opt/kafka_2.12-2.8.0/config/server.properties
#
broker.id=1
host.name=192.168.8.204
listeners=PLAINTEXT://192.168.8.204:9092
log.dirs=/data/kafka-logs
log.retention.check.interval.ms=300000
log.retention.hours=1
log.segment.bytes=1073741824
num.io.threads=4
default.replication.factor=2
num.network.threads=3
num.partitions=30
num.recovery.threads.per.data.dir=1
port=9092
socket.receive.buffer.bytes=102400
socket.request.max.bytes=104857600
socket.send.buffer.bytes=102400
zookeeper.connect=192.168.8.204:2181,192.168.8.205:2181,192.168.8.206:2181
zookeeper.connection.timeout.ms=10000
fetch.message.max.bytes=52428700
replica.fetch.max.bytes=52428700
|
启动时加入到/etc/rc.local
1
2
3
4
5
| vim /etc/rc.local
cd /var/tmp;nohup /opt/kafka_2.12-2.8.0/bin/zookeeper-server-start.sh /opt/kafka_2.12-2.8.0/config/zookeeper.properties &
cd /tmp;nohup /opt/kafka_2.12-2.8.0/bin/kafka-server-start.sh /opt/kafka_2.12-2.8.0/config/server.properties &
touch /var/lock/subsys/local
|
创建 zookeeper myid 文件
依次创建zookeeper的数据目录/data/zookeeper和文件/data/zookeeper/myid,此时我们需要对应的Zookeeper服务器上在dataDir的目录下面新建myid文件，同时把对应的1、2、3写进去。
1
2
3
| mkdir -p /data/zookeeper
vim /data/zookeeper/myid
1
|
启动 zookeeper
依次启动集群的kafka1、kafka2、kafka3 的zookeeper
1
| nohup /opt/kafka_2.12-2.8.0/bin/zookeeper-server-start.sh /opt/kafka_2.12-2.8.0/config/zookeeper.properties &
|
启动 kafka
依次启动集群的kafka1、kafka2、kafka3 的kafka服务
1
| nohup /opt/kafka_2.12-2.8.0/bin/kafka-server-start.sh /opt/kafka_2.12-2.8.0/config/server.properties &
|
检查
使用 jps 还有 netstat 检查服务
