centos安装zookeeper和kafka

一、前言

为了学习kafka,最好是先用单机kafka,先不在docker集群中做。

kafka依赖zookeeper和java jdk,因此需要先弄好java环境。

我这里,会直接在虚拟机的centos系统里安装jdk、zookeeper、kafka,不用docker安装了,这样出现问题也更方便排查。

 

二、安装java jdk

详见文章 centos7安装Java

 

三、安装zookeeper

1、下载安装包
cd /usr/local
wget https://dlcdn.apache.org/zookeeper/zookeeper-3.6.3/apache-zookeeper-3.6.3-bin.tar.gz
2、解压
tar zxf apache-zookeeper-3.6.3-bin.tar.gz
rm -rf apache-zookeeper-3.6.3-bin.tar.gz
mv apache-zookeeper-3.6.3-bin/ zookeeper
3、zookeeper配置文件
cd /usr/local/zookeeper/conf
cp zoo_sample.cfg zoo.cfg

目前zookeeper没什么需要配置的,如果要配置,就修改zoo.cfg

这里面要配置的就两个参数,一个是dataDir=/tmp/zookeeper,另一个是clientPort=2181,不过我们这里就采用默认的了,不去修改。

4、启动zookeeper
cd /usr/local/zookeeper/bin
./zkServer.sh start
5、验证是否安装并启动成功
ps -ef | grep zookeeper
[root@v2 bin]# ps -ef | grep zookeeper
root       7499      1  1 10:43 pts/2    00:00:01 /usr/local/jdk-17/bin/java -Dzookeeper.log.dir=/usr/local/zookeeper/bin/../logs -Dzookeeper.log.file=zookeeper-root-server-v2.log -Dzookeeper.root.logger=INFO,CONSOLE -XX:+HeapDumpOnOutOfMemoryError -XX:OnOutOfMemoryError=kill -9 %p -cp /usr/local/zookeeper/bin/../zookeeper-server/target/classes:/usr/local/zookeeper/bin/../build/classes:/usr/local/zookeeper/bin/../zookeeper-server/target/lib/*.jar:/usr/local/zookeeper/bin/../build/lib/*.jar:/usr/local/zookeeper/bin/../lib/zookeeper-prometheus-metrics-3.6.3.jar:/usr/local/zookeeper/bin/../lib/zookeeper-jute-3.6.3.jar:/usr/local/zookeeper/bin/../lib/zookeeper-3.6.3.jar:/usr/local/zookeeper/bin/../lib/snappy-java-1.1.7.jar:/usr/local/zookeeper/bin/../lib/slf4j-log4j12-1.7.25.jar:/usr/local/zookeeper/bin/../lib/slf4j-api-1.7.25.jar:/usr/local/zookeeper/bin/../lib/simpleclient_servlet-0.6.0.jar:/usr/local/zookeeper/bin/../lib/simpleclient_hotspot-0.6.0.jar:/usr/local/zookeeper/bin/../lib/simpleclient_common-0.6.0.jar:/usr/local/zookeeper/bin/../lib/simpleclient-0.6.0.jar:/usr/local/zookeeper/bin/../lib/netty-transport-native-unix-common-4.1.63.Final.jar:/usr/local/zookeeper/bin/../lib/netty-transport-native-epoll-4.1.63.Final.jar:/usr/local/zookeeper/bin/../lib/netty-transport-4.1.63.Final.jar:/usr/local/zookeeper/bin/../lib/netty-resolver-4.1.63.Final.jar:/usr/local/zookeeper/bin/../lib/netty-handler-4.1.63.Final.jar:/usr/local/zookeeper/bin/../lib/netty-common-4.1.63.Final.jar:/usr/local/zookeeper/bin/../lib/netty-codec-4.1.63.Final.jar:/usr/local/zookeeper/bin/../lib/netty-buffer-4.1.63.Final.jar:/usr/local/zookeeper/bin/../lib/metrics-core-3.2.5.jar:/usr/local/zookeeper/bin/../lib/log4j-1.2.17.jar:/usr/local/zookeeper/bin/../lib/json-simple-1.1.1.jar:/usr/local/zookeeper/bin/../lib/jline-2.14.6.jar:/usr/local/zookeeper/bin/../lib/jetty-util-ajax-9.4.39.v20210325.jar:/usr/localzookeeper/bin/../lib/jetty-util-9.4.39.v20210325.jar:/usr/local/zookeeper/bin/../lib/jetty-servlet-9.4.39.v20210325.jar:/usr/local/zookeeper/bin/../lib/jetty-server-9.4.39.v20210325.jar:/usr/local/zookeeper/bin/../lib/jetty-security-9.4.39.v20210325.jar:/usr/local/zookeeper/bin/../lib/jetty-io-9.4.39.v20210325.jar:/usr/local/zookeeper/bin/../lib/jetty-http-9.4.39.v20210325.jar:/usr/local/zookeeper/bin/../lib/javax.servlet-api-3.1.0.jar:/usr/local/zookeeper/bin/../lib/jackson-databind-2.10.5.1.jar:/usr/local/zookeeper/bin/../lib/jackson-core-2.10.5.jar:/usr/local/zookeeper/bin/../lib/jackson-annotations-2.10.5.jar:/usr/local/zookeeper/bin/../lib/commons-cli-1.2.jar:/usr/local/zookeeper/bin/../lib/audience-annotations-0.5.0.jar:/usr/local/zookeeper/bin/../zookeeper-*.jar:/usr/local/zookeeper/bin/../zookeeper-server/src/main/resources/lib/*.jar:/usr/local/zookeeper/bin/../conf: -Xmx1000m -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.local.only=false org.apache.zookeeper.server.quorum.QuorumPeerMain /usr/local/zookeeper/bin/../conf/zoo.cfg
root       7640   1912  0 10:46 pts/2    00:00:00 grep --color zookeeper

 

四、安装kafka

1、下载kafka压缩包
cd /usr/local
wget https://dlcdn.apache.org/kafka/3.0.0/kafka_2.12-3.0.0.tgz
2、解压缩
tar zxf kafka_2.12-3.0.0.tgz
rm -rf kafka_2.12-3.0.0.tgz
mv kafka_2.12-3.0.0/ kafka
3、修改配置文件
cd /usr/local/kafka/config/
vi server.properties
# 添加下面两行
listeners=PLAINTEXT://192.168.78.102:9092
advertised.listeners=PLAINTEXT://192.168.78.102:9092

# 修改zookeeper.connect=localhost:2181,这行也可以不修改,因为我们现在zookeeper和kafka安装在一个主机上了
zookeeper.connect=192.168.78.102:2181

正常还有一个比较重要的参数是kafka的日志目录,log.dirs=/tmp/kafka-logs,但我们这里只是拿这个kafka做体验,因此这里就不特意改了。

4、启动kafka
cd /usr/local/kafka
bin/kafka-server-start.sh config/server.properties &

会出来一大堆kafka的启动日志,没报错就说明启动成功了,直接control + c 就可以了,因为我们加了&启动的,所以关了这个界面也没关系

5、验证kafka启动成功
ps -ef | grep kafka

出现下面界面说明启动成功

[root@v2 kafka]# ps -ef | grep kafka
root       8122   1912  4 11:02 pts/2    00:00:07 /usr/local/jdk-17/bin/java -Xmx1G -Xms1G -server -XX:+UseG1GC -XX:MaxGCPauseMillis=20 -XX:InitiatingHeapOccupancyPercent=35 -XX:+ExplicitGCInvokesConcurrent -XX:MaxInlineLevel=15 -Djava.awt.headless=true -Xlog:gc*:file=/usr/local/kafka/bin/../logs/kafkaServer-

到这里,就完成了单机jdk、zookeeper、kafka的安装