【原创】《从0开始学RocketMQ》

内容目录

1. RocketMQ是什么？

2. 下载并解压

3. 启动NameServer

4. 启动 Broker

5. 关闭消息队列

1. RocketMQ是什么？

RocketMQ是一种消息队列。何为消息队列？即数据结构中一种“先进先出”的数据结构。在微服务中，分布式消息队列可以解决什么问题？应用解耦、流量削峰、消息分发、保证最终一致性、方便动态扩容等。

RocketMQ中不可不知的四个角色：Producer（消息生产者）、Consumer（消息消费者）、Broker（消息暂存者）、NameServer（消息协调者）。由这四个角色的作用可知，启动 RocketMQ 时，应该先启动 NameServer，然后再启动 Broker，后续需要发送消息就用 Producer，需要接收消息就用 Consumer。

2. 下载并解压

cd /usr/local

wget -c http://mirrors.shu.edu.cn/apache/rocketmq/4.3.2/rocketmq-all-4.3.2-bin-release.zip

unzip rocketmq-all-4.3.-bin-release.zip -d .

cd rocketmq-all-4.3.-bin-release

ls

[root@- rocketmq-all-4.3.-bin-release]# ls

benchmark  bin  conf  lib  LICENSE  NOTICE  README.md

3. 启动NameServer

启动NameServer：nohup sh bin/mqnamesrv &，然而发现不能正常启动，如下：

[root@- rocketmq-all-4.3.-bin-release]# nohup sh bin/mqnamesrv &

[]

[root@- rocketmq-all-4.3.-bin-release]# nohup: ignoring input and appending output to ‘nohup.out’

[]+  Exit                   nohup sh bin/mqnamesrv

查看nohup.out：

[root@- rocketmq-all-4.3.-bin-release]# cat nohup.out

ERROR: Please set the JAVA_HOME variable in your environment, We need java(x64)! !!

可是，本机明明已经安装jdk并且已经配置了 JAVA_HOME 环境变量呀：

[root@- rocketmq-all-4.3.-bin-release]# java -version

java version "1.8.0_171"

Java(TM) SE Runtime Environment (build 1.8.0_171-b11)

Java HotSpot(TM) -Bit Server VM (build 25.171-b11, mixed mode)

环境变量设置：

#####################java env###############

JAVA_HOME=/opt/jdk1..0_171

CLASSPATH=.:$JAVA_HOME/lib/tools.jar:$JAVA_HOME/lib/dt.jar

PATH=$JAVA_HOME/bin:$HOME/bin:$HOME/.local/bin:$PATH

查看 runserver.sh 脚本，发现有这么三行命令：

[ ! -e "$JAVA_HOME/bin/java" ] && JAVA_HOME=$HOME/jdk/java

[ ! -e "$JAVA_HOME/bin/java" ] && JAVA_HOME=/usr/java

[ ! -e "$JAVA_HOME/bin/java" ] && error_exit "Please set the JAVA_HOME variable in your environment, We need java(x64)!"

将其注释之，继续往下看，发现下面两行：

export JAVA_HOME

export JAVA="$JAVA_HOME/bin/java"

嗯，果断修改 JAVA_HOME 的值如下：

export JAVA_HOME=/opt/jdk1..0_171

再次启动 NameServer ：

[root@- rocketmq-all-4.3.-bin-release]# nohup sh bin/mqnamesrv &

[]

[root@- rocketmq-all-4.3.-bin-release]# nohup: ignoring input and appending output to ‘nohup.out’

[root@- rocketmq-all-4.3.-bin-release]# ls

benchmark  bin  conf  lib  LICENSE  nohup.out  NOTICE  README.md

[root@- rocketmq-all-4.3.-bin-release]#  cat nohup.out

Java HotSpot(TM) -Bit Server VM warning: Using the DefNew young collector with the CMS collector is deprecated and will likely be removed in a future release

Java HotSpot(TM) -Bit Server VM warning: UseCMSCompactAtFullCollection is deprecated and will likely be removed in a future release.

The Name Server boot success. serializeType=JSON

此处可以看到 NameServer 已成功启动，查看 NameServer 日志：

[root@- rocketmq-all-4.3.-bin-release]# tail -f ~/logs/rocketmqlogs/namesrv.log

-- :: INFO main - tls.client.certPath = null

-- :: INFO main - tls.client.authServer = false

-- :: INFO main - tls.client.trustCertPath = null

-- :: INFO main - Using OpenSSL provider

-- :: INFO main - SSLContext created for server

-- :: INFO NettyEventExecutor - NettyEventExecutor service started

-- :: INFO FileWatchService - FileWatchService service started

-- :: INFO main - The Name Server boot success. serializeType=JSON

-- :: INFO NSScheduledThread1 - --------------------------------------------------------

-- :: INFO NSScheduledThread1 - configTable SIZE:

NameServer 启动成功

4. 启动 Broker

启动Broker：nohup sh bin/mqbroker -n localhost:9876&，然而发现不能正常启动，如下：

[root@- rocketmq-all-4.3.-bin-release]# nohup sh bin/mqbroker -n localhost:&

[]

[root@- rocketmq-all-4.3.-bin-release]# nohup: ignoring input and appending output to ‘nohup.out’

[]+  Exit                   nohup sh bin/mqbroker -n localhost:

查看nohup.out，发现里面多了这么几行错误信息：

Java HotSpot(TM) -Bit Server VM warning: INFO: os::commit_memory(0x00000005c0000000, , ) failed; error='Cannot allocate memory' (errno=)

#

# There is insufficient memory for the Java Runtime Environment to continue.

# Native memory allocation (mmap) failed to map  bytes for committing reserved memory.

# An error report file with more information is saved as:

# /usr/local/rocketmq-all-4.3.-bin-release/hs_err_pid28580.log

遂，查看 hs_err_pid28580.log：

[root@- rocketmq-all-4.3.-bin-release]# tail -1000f /usr/local/rocketmq-all-4.3.-bin-release/hs_err_pid28580.log

#

# There is insufficient memory for the Java Runtime Environment to continue.

# Native memory allocation (mmap) failed to map  bytes for committing reserved memory.

# Possible reasons:

#   The system is out of physical RAM or swap space

#   In  bit mode, the process size limit was hit

# Possible solutions:

#   Reduce memory load on the system

#   Increase physical memory or swap space

#   Check if swap backing store is full

#   Use  bit Java on a  bit OS

#   Decrease Java heap size (-Xmx/-Xms)

#   Decrease number of Java threads

#   Decrease Java thread stack sizes (-Xss)

#   Set larger code cache with -XX:ReservedCodeCacheSize=

# This output file may be truncated or incomplete.

#

#  Out of Memory Error (os_linux.cpp:), pid=, tid=0x00007fbc93a6c700

#

# JRE version:  (.0_171-b11) (build )

# Java VM: Java HotSpot(TM) -Bit Server VM (25.171-b11 mixed mode linux-amd64 compressed oops)

# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again

........

VM Arguments:

jvm_args: -Xms8g -Xmx8g -Xmn4g -XX:+UseG1GC -XX:G1HeapRegionSize=16m -XX:G1ReservePercent= -XX:InitiatingHeapOccupancyPercent= -XX:SoftRefLRUPolicyMSPerMB= -XX:SurvivorRatio= -verbose:gc -Xloggc:/dev/shm/mq_gc_%p.log -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCApplicationStoppedTime -XX:+PrintAdaptiveSizePolicy -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles= -XX:GCLogFileSize=30m -XX:-OmitStackTraceInFastThrow -XX:+AlwaysPreTouch -XX:MaxDirectMemorySize=15g -XX:-UseLargePages -XX:-UseBiasedLocking -Djava.ext.dirs=/opt/jdk1..0_171/jre/lib/ext:/usr/local/rocketmq-all-4.3.-bin-release/bin/../lib

java_command: org.apache.rocketmq.broker.BrokerStartup -n localhost:

java_class_path (initial): .:/usr/local/rocketmq-all-4.3.-bin-release/bin/../conf:.:/opt/jdk1..0_171/lib/tools.jar:/opt/jdk1..0_171/lib/dt.jar

Launcher Type: SUN_STANDARD

......

大概意思呢，就是说，启动broker时内存不足，为什么内存不足呢？因为启动时候指定了JVM最大可用内存和最小可用内存均为8G，年轻代大小为2G。
查看Broker启动脚本，发现果然指定了这三个jvm启动参数，然后意外发现对于JAVA_HOME的设置，runbroker.sh与runserver.sh是一样一样的：

[ ! -e "$JAVA_HOME/bin/java" ] && JAVA_HOME=$HOME/jdk/java

[ ! -e "$JAVA_HOME/bin/java" ] && JAVA_HOME=/usr/java

[ ! -e "$JAVA_HOME/bin/java" ] && error_exit "Please set the JAVA_HOME variable in your environment, We need java(x64)!"

export JAVA_HOME

export JAVA="$JAVA_HOME/bin/java"

export BASE_DIR=$(dirname $)/..

export CLASSPATH=.:${BASE_DIR}/conf:${CLASSPATH}

#===========================================================================================

# JVM Configuration

#===========================================================================================

JAVA_OPT="${JAVA_OPT} -server -Xms8g -Xmx8g -Xmn4g"

遂，修改启动脚本中JAVA_HOME的设置以及JVM启动脚本如下：

#[ ! -e "$JAVA_HOME/bin/java" ] && JAVA_HOME=$HOME/jdk/java

#[ ! -e "$JAVA_HOME/bin/java" ] && JAVA_HOME=/usr/java

#[ ! -e "$JAVA_HOME/bin/java" ] && error_exit "Please set the JAVA_HOME variable in your environment, We need java(x64)!"

export JAVA_HOME=/opt/jdk1..0_171

export JAVA="$JAVA_HOME/bin/java"

export BASE_DIR=$(dirname $)/..

export CLASSPATH=.:${BASE_DIR}/conf:${CLASSPATH}

#===========================================================================================

# JVM Configuration

#===========================================================================================

JAVA_OPT="${JAVA_OPT} -server -Xms256m -Xmx256m -Xmn128m"

再次启动Broker：

[root@- rocketmq-all-4.3.-bin-release]# nohup sh bin/mqbroker -n localhost:&

[]

[root@- rocketmq-all-4.3.-bin-release]# nohup: ignoring input and appending output to ‘nohup.out’

[root@- rocketmq-all-4.3.-bin-release]# cat nohup.out

Java HotSpot(TM) -Bit Server VM warning: INFO: os::commit_memory(0x00000005c0000000, , ) failed; error='Cannot allocate memory' (errno=)

#

# There is insufficient memory for the Java Runtime Environment to continue.

# Native memory allocation (mmap) failed to map  bytes for committing reserved memory.

# An error report file with more information is saved as:

# /usr/local/rocketmq-all-4.3.-bin-release/hs_err_pid28580.log

The broker[-, 39.107.153.215:] boot success. serializeType=JSON and name server is localhost:

此处可以看到Broker已启动成功，查看Broker日志：

[root@- rocketmq-all-4.3.-bin-release]# tail -1000f ~/logs/rocketmqlogs/broker.log

-- :: INFO main - rocketmqHome=/usr/local/rocketmq-all-4.3.-bin-release

-- :: INFO main - namesrvAddr=localhost:

......

-- :: INFO FileWatchService - FileWatchService service started

-- :: INFO PullRequestHoldService - PullRequestHoldService service started

-- :: INFO brokerOutApi_thread_1 - register broker to name server localhost: OK

-- :: INFO main - Start transaction service!

-- :: INFO main - The broker[-, 39.107.157.89:] boot success. serializeType=JSON and name server is localhost:

5. 关闭消息队列

关闭 Broker：

[root@- rocketmq-all-4.3.-bin-release]# sh bin/mqshutdown broker

The mqbroker() is running...

Send shutdown request to mqbroker() OK

关闭 NameServer：

[root@- rocketmq-all-4.3.-bin-release]# sh bin/mqshutdown namesrv

The mqnamesrv() is running...

Send shutdown request to mqnamesrv() OK