StarRocks 运维工具 StarGo

注：本文主要内容均来源 StarRocks 官网 https://docs.starrocks.com/zh-cn/main/administration/stargo

StarGo 是一个用于管理多个 StarRocks 集群的命令行工具。通过 StarGo，您可以使用简单的命令行实现多集群的部署、查看、升级、启动与停止等操作。

StarGo 使用 go 开发，github 地址 https://github.com/wangtianyi2004/starrocks-controller

部署 StarGo

在当前用户路径下下载并解压 StarGo 二进制安装包。

wget https://raw.githubusercontent.com/wangtianyi2004/starrocks-controller/main/stargo-pkg.tar.gz

tar -xzvf stargo-pkg.tar.gz

安装包包含以下文件。

stargo：StarGo 二进制文件，无需安装。

deploy-template.yaml：部署配置文件模板。

repo.yaml：指定 StarRocks 安装包下载库的配置文件。

说明：您可以在 http://cdn-thirdparty.starrocks.com 地址下获取相应的版本的安装 index 文件以及安装包。

StarGo 支持的版本参见： http://starrocks-thirdparty.oss-cn-zhangjiakou.aliyuncs.com/packageVersion.list

目前支持版本如下：

v2.0.1

v2.1.3

v2.1.6

v2.2.0

v2.2.1

v2.2.2

v2.2.3

v2.3.0

部署集群

前提条件

待部署集群至少需要一个中控机节点和三个部署机节点，所有节点可以混合部署于同一台机器。
中控机上需部署 StarGo。
中控机与部署机间需创建 SSH 互信。

以下示例创建了中控机 sr-dev@r0 与部署机 starrocks@r1、starrocks@r2 以及 starrocks@r3 间的 SSH 互信。

创建 sr-dev@r0 到 starrocks@r1、r2、r3 的 ssh 互信。

[sr-dev@r0 ~]$ ssh-keygen

[sr-dev@r0 ~]$ ssh-copy-id starrocks@r1

[sr-dev@r0 ~]$ ssh-copy-id starrocks@r2

[sr-dev@r0 ~]$ ssh-copy-id starrocks@r3

## 验证 sr-dev@r0 到 starrocks@r1、r2、r3 的 ssh 互信。

[sr-dev@r0 ~]$ ssh starrocks@r1 date

[sr-dev@r0 ~]$ ssh starrocks@r2 date

[sr-dev@r0 ~]$ ssh starrocks@r3 date

创建配置文件

根据以下 YAML 模板，创建部署 StarRocks 集群的拓扑文件。具体配置项参考参数配置。

[starrocks@bigdata11 stargo-pkg]$ cat deploy-star3.yaml

global:

    user: "starrocks"

    ssh_port: 22

fe_servers:

  - host: xx.xx.xx.229

    ssh_port: 22

    http_port: 38030

    rpc_port: 39020

    query_port: 39030

    edit_log_port: 39010

    deploy_dir: /opt/StarRocks1/fe

    meta_dir: /data/starrocks1/fe/meta

    log_dir: /data/starrocks1/fe/log

    priority_networks: xx.xx.xx.229

    config:

      sys_log_level: "INFO"

be_servers:

  - host: xx.xx.xx.229

    ssh_port: 22

    be_port: 39060

    webserver_port: 38040

    heartbeat_service_port: 39050

    brpc_port: 38060

    deploy_dir : /opt/StarRocks1/be

    storage_dir: /data/starrocks1/be/storage

    log_dir: /data/starrocks1/be/log

    priority_networks: xx.xx.xx.229/24

    config:

      create_tablet_worker_count: 3

注：服务器上已经有一套 StarRocks，再部署一套单机的，修改了下程序、端口

创建部署目录（可选）

如果您在配置文件中设定的部署路径不存在，且您有创建该路径的权限，StarGo 将根据配置文件自动创建部署目录。如果路径已存在，请确保您有在该路径下拥有写入的权限。您也可以通过以下命令，在各部署节点分别创建部署路径。

在 FE 节点安装目录下上创建 meta 路径。

mkdir -p /opt/StarRocks1/be

在 BE 节点安装目录下上创建 storage 路径。

mkdir -p /data/starrocks1/be/storage

注意: 请确保以上创建的路径与配置文件中的 meta_dir 和 storage_dir 相同。

部署 StarRocks

通过以下命令部署 StarRocks 集群。

./stargo cluster deploy <cluster_name> <version> <topology_file>

参数	描述
cluster_name	创建的集群名
version	StarRocks 的版本
topology_file	配置文件名

创建成功后，集群将会自动启动。当返回 beStatus 和feStatus 为 true 时，集群部署启动成功。

示例：

[starrocks@bigdata11 stargo-pkg]$ ./stargo cluster deploy star3 v2.2.2 deploy-star3.yaml

[20220812-113235  OUTPUT] Deploy cluster [clusterName = star3, clusterVersion = v2.2.2, metaFile = deploy-star3.yaml]

[20220812-113236  OUTPUT] PRE CHECK DEPLOY ENV:

PreCheck FE:

server id                  ssh auth         meta dir                        deploy dir                      http port        rpc port         query port       edit log port    open files count

--------------------       ---------------  ------------------------------  ------------------------------  ---------------  ---------------  ---------------  ---------------  ---------------

xx.xx.xx.229:39010         PASS             PASS                            PASS                            PASS             PASS             PASS             PASS             PASS           

PreCheck BE:

server id                  ssh auth         storage dir                     deploy dir                      webSer port      heartbeat port   brpc port        be port          open files count

--------------------       ---------------  ------------------------------  ------------------------------  ---------------  ---------------  ---------------  ---------------  ---------------

xx.xx.xx.229:39060         PASS             PASS                            PASS                            PASS             PASS             PASS             PASS             PASS           

[20220812-113236  OUTPUT] PreCheck successfully. RESPECT

[20220812-113236  OUTPUT] Create the deploy folder ...

[20220812-113237  OUTPUT] Download StarRocks package & jdk ...

[20220812-113322    INFO] The file starrocks-2.2.2-quickstart.tar.gz [1695364308] download successfully

[20220812-113322  OUTPUT] Download done.

[20220812-113322  OUTPUT] Decompress StarRocks pakcage & jdk ...

[20220812-113325    INFO] The tar file /home/starrocks/.stargo/download/starrocks-2.2.2-quickstart.tar.gz has been decompressed under /home/starrocks/.stargo/download

[20220812-113349    INFO] The tar file /home/starrocks/.stargo/download/StarRocks-2.2.2.tar.gz has been decompressed under /home/starrocks/.stargo/download

[20220812-113354    INFO] The tar file /home/starrocks/.stargo/download/jdk-8u301-linux-x64.tar.gz has been decompressed under /home/starrocks/.stargo/download

[20220812-113354  OUTPUT] Distribute FE Dir ...

[20220812-113402    INFO] Upload dir feSourceDir = [/home/starrocks/.stargo/download/StarRocks-2.2.2/fe] to feTargetDir = [/opt/StarRocks1/fe] on FeHost = [xx.xx.xx.229]

[20220812-113406    INFO] Upload dir JDKSourceDir = [/home/starrocks/.stargo/download/jdk1.8.0_301] to JDKTargetDir = [/opt/StarRocks1/fe/jdk] on FeHost = [xx.xx.xx.229]

[20220812-113406    INFO] Modify JAVA_HOME: host = [xx.xx.xx.229], filePath = [/opt/StarRocks1/fe/bin/start_fe.sh]

[20220812-113406  OUTPUT] Distribute BE Dir ...

[20220812-113417    INFO] Upload dir BeSourceDir = [/home/starrocks/.stargo/download/StarRocks-2.2.2/be] to BeTargetDir = [/opt/StarRocks1/be] on BeHost = [xx.xx.xx.229]

[20220812-113417  OUTPUT] Modify configuration for FE nodes & BE nodes ...

############################################# START FE CLUSTER #############################################

############################################# START FE CLUSTER #############################################

[20220812-113417    INFO] Starting leader FE node [host = xx.xx.xx.229, editLogPort = 39010]

[20220812-113438    WARN] The FE node doesn't start, wait for 10s [FeHost = xx.xx.xx.229, FeQueryPort = 39030, error = Process exited with status 1]

[20220812-113438    INFO] Starting leader FE node [host = xx.xx.xx.229, editLogPort = 39010]

[20220812-113454    INFO] The FE node start succefully [host = xx.xx.xx.229, queryPort = 39030]

[20220812-113454    INFO] List all FE status:

                                        feHost = xx.xx.xx.229        feQueryPort = 39030     feStatus = true

############################################# START BE CLUSTER #############################################

############################################# START BE CLUSTER #############################################

[20220812-113454    INFO] Starting BE node [BeHost = xx.xx.xx.229 HeartbeatServicePort = 39050]

[20220812-113515    INFO] The BE node start succefully [host = xx.xx.xx.229, heartbeatServicePort = 39050]

[20220812-113515  OUTPUT] List all BE status:

                                        beHost = xx.xx.xx.229        beHeartbeatServicePort = 39050	beStatus = true

如果执行检查失败，会有相应提示，按提示操作即可

[starrocks@bigdata12 stargo-pkg]$ ./stargo cluster deploy star3 v2.2.2 deploy-star3.yaml

[20220812-102653  OUTPUT] Deploy cluster [clusterName = star3, clusterVersion = v2.2.2, metaFile = deploy-star3.yaml]

[20220812-102658  OUTPUT] PRE CHECK DEPLOY ENV:

PreCheck FE:

server id                  ssh auth         meta dir                        deploy dir                      http port        rpc port         query port       edit log port    open files count

--------------------       ---------------  ------------------------------  ------------------------------  ---------------  ---------------  ---------------  ---------------  ---------------

xx.xx.xx.228:39010         PASS             FAILED: Priv failed             FAILED: Priv failed             PASS             PASS             PASS             PASS             PASS           

PreCheck BE:

server id                  ssh auth         storage dir                     deploy dir                      webSer port      heartbeat port   brpc port        be port          open files count

--------------------       ---------------  ------------------------------  ------------------------------  ---------------  ---------------  ---------------  ---------------  ---------------

xx.xx.xx.228:39060         PASS             FAILED: Dir exist/Priv failed   FAILED: Dir exist/Priv failed   PASS             PASS             PASS             PASS             PASS

xx.xx.xx.229:39060         PASS             FAILED: Dir exist/Priv failed   FAILED: Dir exist/Priv failed   PASS             PASS             PASS             PASS             PASS

xx.xx.xx.230:39060         PASS             FAILED: Dir exist               FAILED: Dir exist               PASS             PASS             PASS             PASS             PASS           

[20220812-102658   ERROR] Please use bellowing promption to fix the issue for FE servers:

Detect the FE META FOLDER exist or no privilege. Use bellowing command to check or fix the issue:

  [Host = xx.xx.xx.228]  chown -R starrocks /data/starrocks1/fe/meta

Detect the FE DEPLOY FOLDER exist or no privilege. Use bellowing command to check or fix the issue:

  [Host = xx.xx.xx.228]  chown -R starrocks /opt/StarRocks1/fe

[20220812-102658   ERROR] Please use bellowing promption to fix the issue for BE servers:

Detect the BE STORAGE FOLDER exist or no privilege. Use bellowing command to check or fix the issue:

  [Host = xx.xx.xx.228]  mkdir /data/starrocks1/be/storage.bak && mv /data/starrocks1/be/storage/* /data/starrocks1/be/storage.bak/ && chown -R starrocks /data/starrocks1/be/storage

  [Host = xx.xx.xx.229]  mkdir /data/starrocks1/be/storage.bak && mv /data/starrocks1/be/storage/* /data/starrocks1/be/storage.bak/ && chown -R starrocks /data/starrocks1/be/storage

  [Host = xx.xx.xx.230]  mkdir /data/starrocks1/be/storage.bak && mv /data/starrocks1/be/storage/* /data/starrocks1/be/storage.bak/

Detect the BE DEPLOY FOLDER exist or no privilege. Use bellowing command to check or fix the issue:

  [Host = xx.xx.xx.228]  mkdir /opt/StarRocks1/be.bak && mv /opt/StarRocks1/be/* /opt/StarRocks1/be.bak/ && chown -R starrocks /opt/StarRocks1/be

  [Host = xx.xx.xx.229]  mkdir /opt/StarRocks1/be.bak && mv /opt/StarRocks1/be/* /opt/StarRocks1/be.bak/ && chown -R starrocks /opt/StarRocks1/be

  [Host = xx.xx.xx.230]  mkdir /opt/StarRocks1/be.bak && mv /opt/StarRocks1/be/* /opt/StarRocks1/be.bak/

[20220812-102658   ERROR] PreCheck failed.

查看集群信息

[starrocks@bigdata11 stargo-pkg]$ ./stargo cluster list

[20220812-115246  OUTPUT] List all clusters

ClusterName      Version     User        CreateDate                 MetaPath                                                      PrivateKey

---------------  ----------  ----------  -------------------------  ------------------------------------------------------------  --------------------------------------------------

star3            v2.2.2      starrocks   2022-08-12 11:34:17        /home/starrocks/.stargo/cluster/star3                         /home/starrocks/.ssh/id_rsa

查看指定集群信息

[starrocks@bigdata11 stargo-pkg]$ ./stargo cluster display star3

[20220812-143023  OUTPUT] Display cluster [clusterName = star3]

clusterName = star3

clusterVerison = v2.2.2

ID                          ROLE    HOST                  PORT             STAT        DATADIR                                             DEPLOYDIR

--------------------------  ------  --------------------  ---------------  ----------  --------------------------------------------------  --------------------------------------------------

xx.xx.xx.229:39010          FE      xx.xx.xx.229          39010/39030      UP/L        /opt/StarRocks1/fe                                  /data/starrocks1/fe/meta

xx.xx.xx.229:39060          BE      xx.xx.xx.229          39060/39050      UP          /opt/StarRocks1/be                                  /data/starrocks1/be/storage

[starrocks@bigdata11 stargo-pkg]$

升级版本

starrocks 2.2.0 升级到 2.3.0

[starrocks@bigdata11 stargo-pkg]$ ./stargo cluster upgrade star3 v2.3.0

[20220812-143227  OUTPUT] Upgrade cluster. [ClusterName = star3, TargetVersion = v2.3.0]

[20220812-143227  OUTPUT] Upgrade StarRocks Cluster star3, from version v2.2.2 to version v2.3.0

[20220812-143227  OUTPUT] Download StarRocks package & jdk ...

[20220812-143340    INFO] The file starrocks-2.3.0-quickstart.tar.gz [1726845129] download successfully

[20220812-143340  OUTPUT] Download done.

[20220812-143340  OUTPUT] Decompress StarRocks pakcage & jdk ...

[20220812-143346    INFO] The tar file /home/starrocks/.stargo/download/starrocks-2.3.0-quickstart.tar.gz has been decompressed under /home/starrocks/.stargo/download

[20220812-143411    INFO] The tar file /home/starrocks/.stargo/download/StarRocks-2.3.0.tar.gz has been decompressed under /home/starrocks/.stargo/download

[20220812-143411    INFO] The tar file /home/starrocks/.stargo/download/jdk-8u301-linux-x64.tar.gz has been decompressed under /home/starrocks/.stargo/download

[20220812-143413  OUTPUT] Starting upgrade BE node. [beId = 0]

[20220812-143413    INFO] upgrade be node - backup be lib. [host = xx.xx.xx.229, sourceDir = /opt/StarRocks1/be/lib, targetDir = /opt/StarRocks1/be/lib.bak-20220812143413]

[20220812-143423    INFO] upgrade be node - upload new be lib. [host = xx.xx.xx.229, sourceDir = /home/starrocks/.stargo/download/StarRocks-2.3.0/be/lib, targetDir = /opt/StarRocks1/be/lib]

[20220812-143423    INFO] Waiting for stoping BE node [BeHost = xx.xx.xx.229]

[20220812-143429    INFO] upgrade be node - stop be node. [host = xx.xx.xx.229, beDeployDir = /opt/StarRocks1/be]

[20220812-143430    INFO] upgrade be node - start be node. [host = xx.xx.xx.229, beDeployDir = /opt/StarRocks1/be]

[20220812-143443    INFO] upgrade be node - start be node. [host = xx.xx.xx.229, beDeployDir = /opt/StarRocks1/be]

[20220812-143443  OUTPUT] The Be node upgrade successfully. [beId = 0, currentVersion = v2.3.0-a9bdb09]

[20220812-143443  OUTPUT] Starting upgrade FE node. [feId = 0]

[20220812-143445    INFO] upgrade FE node - backup FE lib. [host = xx.xx.xx.229, sourceDir = /opt/StarRocks1/fe/lib, targetDir = /opt/StarRocks1/fe/lib.bak-20220812143443]

[20220812-143448    INFO] upgrade FE node - upload new FE lib. [host = xx.xx.xx.229, sourceDir = /home/starrocks/.stargo/download/StarRocks-2.3.0/fe/lib, targetDir = /opt/StarRocks1/fe/lib]

[20220812-143448    INFO] Waiting for stoping FE node [FeHost = xx.xx.xx.229]

[20220812-143451    INFO] upgrade FE node - stop FE node. [host = xx.xx.xx.229, feDeployDir = /opt/StarRocks1/fe]

[20220812-143452    INFO] upgrade FE node - start FE node. [host = xx.xx.xx.229, feDeployDir = /opt/StarRocks1/fe]

[20220812-143452   ERROR] Error in ping db [dbPath = root:@tcp(xx.xx.xx.229:39030)/], error = dial tcp xx.xx.xx.229:39030: connect: connection refused

[20220812-143502    INFO] upgrade FE node - start FE node. [host = xx.xx.xx.229, feDeployDir = /opt/StarRocks1/fe]

[20220812-143503   ERROR] The FE node upgrade failed.  [feId = 0, targetVersion = v2.3.0, currentVersion = v]

注：是有报错，但是升级成功了

升级后查看集群



[20220812-143555  OUTPUT] List all clusters

ClusterName      Version     User        CreateDate                 MetaPath                                                      PrivateKey

---------------  ----------  ----------  -------------------------  ------------------------------------------------------------  --------------------------------------------------

star3            v2.3.0      starrocks   2022-08-12 14:35:03        /home/starrocks/.stargo/cluster/star3                         /home/starrocks/.ssh/id_rsa

已知缺陷

StarGo 不会部署 Broker
StarGo 安装包包含 jdk，不会替换安装

欢迎关注Flink菜鸟公众号，会不定期更新Flink（开发技术）相关的推文