k8s之helm部署mysql集群

一、简介

• Helm
  
  Helm 是 Kubernetes 的包管理器。
• Chart
  
  Helm使用的包格式称为 chart。chart存储在Chart Repository。
  
  chart就是一个描述Kubernetes相关资源的文件集合。单个chart可以用来部署一些简单的， 类似于memcache pod，或者某些复杂的HTTP服务器以及web全栈应用、数据库、缓存等等。
• Release
  
  基于Chart运行的实例叫做Release
  
  在 Kubernetes 集群上运行的 Chart 的一个实例。在同一个集群上，一个 Chart 可以安装很多次。每次安装都会创建一个新的 release。
  
  例如一个 MySQL Chart，如果想在服务器上运行两个数据库，就可以把这个 Chart 安装两次。每次安装都会生成自己的 Release，会有自己的 Release 名称。
• Repository
  
  用于发布和存储 Chart 的仓库。

二、chart的组成

chart主要包含的资源配置文件，但这些文件通常不是原生格式的配置，而内嵌了大量的模板字串，以给用户提供自定义配置的接口;

为了简化了需要配置模板字串，Chart还会为这些模板字串提供默认值，存一个yaml格式的文件中“values.yml”，因此，也称其为值文件;

• Chart 文件结构
  
  chart是一个组织在文件目录中的集合。目录名称就是chart名称（没有版本信息）。因而描述WordPress的chart可以存储在wordpress/目录中。
  
  在这个目录中，Helm 期望可以匹配以下结构：

  wordpress/
  Chart.yaml          # 文件包含了该chart的描述。你可以从模板中访问它。charts/目录 可以包含其他的chart(称之为 子chart)。
  LICENSE             # 可选: 包含chart许可证的纯文本文件
  README.md           # 可选: 可读的README文件
  values.yaml         # chart 默认的配置值，这个文件包含了chart的默认值 。这些值会在用户执行helm install 或 helm upgrade时被覆盖。
  values.schema.json  # 可选: 一个使用JSON结构的values.yaml文件
  charts/             # 包含chart依赖的其他chart
  crds/               # 自定义资源的定义
  templates/          # 模板目录， 当和values 结合时，可生成有效的Kubernetes manifest文件，当Helm评估chart时，会通过模板渲染引擎将所有文件发送到templates/目录中。 然后收集模板的结果并发送给Kubernetes。
  templates/NOTES.txt # 可选: 包含简要使用说明的纯文本文件
  

• chart获取
  
  可从社区中获取 (https://artifacthub.io/)，也可自行创建，并通过社区分发
  
  需要先将Chart Repository的信息配置在Helm上才能从中下载
  
  ChartChart保存在本地缓存中

三、安装Helm

https://github.com/helm/helm

root@k8s-master01:~# curl -LO https://get.helm.sh/helm-v3.14.0-linux-amd64.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 15.3M  100 15.3M    0     0  47832      0  0:05:37  0:05:37 --:--:--  111k
root@k8s-master01:~# tar xf helm-v3.14.0-linux-amd64.tar.gz
root@k8s-master01:~# ls
helm-v3.14.0-linux-amd64.tar.gz  jiva-operator.yaml  kube-flannel.yml  learning-k8s  linux-amd64  nfs-pvc.yaml  snap  storageclass-nfs.yaml  test
root@k8s-master01:~# cd linux-amd64/
root@k8s-master01:~/linux-amd64# ls
LICENSE  README.md  helm
root@k8s-master01:~/linux-amd64# mv helm /usr/local/bin/
root@k8s-master01:~/linux-amd64# cd ..
root@k8s-master01:~# ls
helm-v3.14.0-linux-amd64.tar.gz  jiva-operator.yaml  kube-flannel.yml  learning-k8s  linux-amd64  nfs-pvc.yaml  snap  storageclass-nfs.yaml  test
root@k8s-master01:~# rm -rf linux-amd64/

四、Helm命令

Helm的一般操作：

helm search:     搜索chart
helm pull:       下载chart到本地目录查看
helm install:    上传chart到Kubernetes
helm list:       列出已发布的chart

从本地的仓库中查找应用，由于本地仓库没有配置故无法搜索到

root@k8s-master01:~# helm search repo mysql
Error: no repositories configured

从Helm Hub中查找Chart，这些Chart来自于注册到Helm Hub中的各个仓库。

helm search hub mysql
URL                                                     CHART VERSION   APP VERSION             DESCRIPTION
https://artifacthub.io/packages/helm/saber/mysql        8.8.21          8.0.27                  Chart to create a Highly available MySQL cluster
https://artifacthub.io/packages/helm/ygqygq2/mysql      9.5.0           8.0.32                  MySQL is a fast, reliable, scalable, and easy t...
https://artifacthub.io/packages/helm/mysql/mysql        2.1.3           8.0.26                  deploy mysql standalone or group-replication He...
https://artifacthub.io/packages/helm/kubegems/m...      8.9.6           8.0.29                  MySQL is a fast, reliable, scalable, and easy t...
https://artifacthub.io/packages/helm/kvalitetsi...      9.10.4          8.0.33                  MySQL is a fast, reliable, scalable, and easy t...
https://artifacthub.io/packages/helm/groundhog2...      1.1.1           8.2.0                   A Helm chart for MySQL on Kubernetes
https://artifacthub.io/packages/helm/cloudnativ...      5.0.1           8.0.16                  Chart to create a Highly available MySQL cluster
https://artifacthub.io/packages/helm/kubesphere...      1.0.2           5.7.33                  High Availability MySQL Cluster, Open Source.
https://artifacthub.io/packages/helm/bitnami/mysql      9.18.0          8.0.36                  MySQL is a fast, reliable, scalable, and easy t...
https://artifacthub.io/packages/helm/bitnami-ak...      9.4.3           8.0.31                  MySQL is a fast, reliable, scalable, and easy t...
...

五、基于Helm部署mysql

1、社区网站搜索mysql chart





2、准备安装mysql依赖的存储类

存储类的环境准备参考[https://www.cnblogs.com/OpenSourceSite/p/17969118],需要准备：

openebs-hostpath：提供本地卷存储

openebs-jiva-csi：提供复制卷

openebs-rwx：提供多路读写的卷

root@k8s-master01:~# kubectl get pod -n openebs
NAME                                                              READY   STATUS    RESTARTS         AGE
jiva-operator-5d8846bc49-5x2d2                                    1/1     Running   0                9d
nfs-pvc-62db4f63-df95-4815-b7f0-7f41c116aa31-7c575fd444-2k9rc     1/1     Running   0                8d
openebs-jiva-csi-controller-0                                     5/5     Running   201 (142m ago)   9d
openebs-jiva-csi-node-5t6pt                                       3/3     Running   0                9d
openebs-jiva-csi-node-cv6x8                                       3/3     Running   0                5d18h
openebs-jiva-csi-node-xqqbl                                       3/3     Running   0                9d
openebs-localpv-provisioner-6787b599b9-ns4nt                      1/1     Running   198 (142m ago)   9d
openebs-ndm-cluster-exporter-7bfd5746f4-v97jr                     1/1     Running   0                9d
openebs-ndm-h9vkc                                                 1/1     Running   0                9d
openebs-ndm-node-exporter-4hplz                                   1/1     Running   0                5d18h
openebs-ndm-node-exporter-7zrl2                                   1/1     Running   0                9d
openebs-ndm-node-exporter-bt6kb                                   1/1     Running   0                9d
openebs-ndm-operator-845b8858db-9b6m5                             1/1     Running   0                9d
openebs-ndm-qqwv4                                                 1/1     Running   0                9d
openebs-ndm-vkcxc                                                 1/1     Running   0                5d18h
openebs-nfs-provisioner-5b595f4798-2ft9w                          1/1     Running   93 (142m ago)    9d
pvc-4134b8fc-8918-4076-88fa-89da76dead3b-jiva-ctrl-d77d67dhdppw   2/2     Running   0                8d
pvc-4134b8fc-8918-4076-88fa-89da76dead3b-jiva-rep-0               1/1     Running   0                8d
pvc-4134b8fc-8918-4076-88fa-89da76dead3b-jiva-rep-1               1/1     Running   0                8d
root@k8s-master01:~# kubectl get sc
NAME               PROVISIONER                    RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
local              kubernetes.io/no-provisioner   Delete          WaitForFirstConsumer   false                  3d19h
nfs-csi            nfs.csi.k8s.io                 Delete          Immediate              false                  2d20h
openebs-device     openebs.io/local               Delete          WaitForFirstConsumer   false                  9d
openebs-hostpath   openebs.io/local               Delete          WaitForFirstConsumer   false                  9d
openebs-jiva-csi   jiva.csi.openebs.io            Delete          Immediate              true                   9d
openebs-rwx        openebs.io/nfsrwx              Delete          Immediate              false                  9d

3、基于dockerhub上的oci仓库部署mysql

helm install mysql  \
        --set auth.rootPassword='mysql' \
        --set global.storageClass=openebs-hostpath \
        --set architecture=replication \
        --set auth.database=wpdb \
        --set auth.username=wpuser \
        --set auth.password='mysql' \
        --set secondary.replicaCount=1 \
        --set auth.replicationPassword='replpass' \
        oci://registry-1.docker.io/bitnamicharts/mysql \
        -n app --create-namespace

root@k8s-master01:~# helm install mysql  \
        --set auth.rootPassword='mysql' \
        --set global.storageClass=openebs-hostpath \
        --set architecture=replication \
        --set auth.database=wpdb \
        --set auth.username=wpuser \
        --set auth.password='mysql' \
        --set secondary.replicaCount=1 \
        --set auth.replicationPassword='replpass' \
        oci://registry-1.docker.io/bitnamicharts/mysql \
        -n app --create-namespace
Pulled: registry-1.docker.io/bitnamicharts/mysql:9.18.0
Digest: sha256:464bb1d8027b7650472d363c234df2af9210410961c64b18753200367c794e5e
NAME: mysql
LAST DEPLOYED: Sun Jan 28 17:40:03 2024
NAMESPACE: app
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
CHART NAME: mysql
CHART VERSION: 9.18.0
APP VERSION: 8.0.36

** Please be patient while the chart is being deployed **

Tip:

  Watch the deployment status using the command: kubectl get pods -w --namespace app

Services:

  echo Primary: mysql-primary.app.svc.cluster.local:3306
  echo Secondary: mysql-secondary.app.svc.cluster.local:3306

Execute the following to get the administrator credentials:

  echo Username: root
  MYSQL_ROOT_PASSWORD=$(kubectl get secret --namespace app mysql -o jsonpath="{.data.mysql-root-password}" | base64 -d)

To connect to your database:

  1. Run a pod that you can use as a client:

      kubectl run mysql-client --rm --tty -i --restart='Never' --image  docker.io/bitnami/mysql:8.0.36-debian-11-r0 --namespace app --env MYSQL_ROOT_PASSWORD=$MYSQL_ROOT_PASSWORD --command -- bash

  2. To connect to primary service (read/write):

      mysql -h mysql-primary.app.svc.cluster.local -uroot -p"$MYSQL_ROOT_PASSWORD"

  3. To connect to secondary service (read-only):

      mysql -h mysql-secondary.app.svc.cluster.local -uroot -p"$MYSQL_ROOT_PASSWORD"
root@k8s-master01:~#

• 查看部署的版本

root@k8s-master01:~# helm list -n app
NAME    NAMESPACE       REVISION        UPDATED                                 STATUS          CHART           APP VERSION
mysql   app             1               2024-01-28 17:40:03.544271842 +0800 CST deployed        mysql-9.18.0    8.0.36

• 查看默认使用模板渲染之后的配置文件

root@k8s-master01:~# helm get manifest mysql -n app
---
# Source: mysql/templates/serviceaccount.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: mysql
  namespace: "app"
  labels:
    app.kubernetes.io/instance: mysql
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: mysql
    app.kubernetes.io/version: 8.0.36
    helm.sh/chart: mysql-9.18.0
automountServiceAccountToken: false
secrets:
  - name: mysql
---
# Source: mysql/templates/secrets.yaml
apiVersion: v1
kind: Secret
metadata:
  name: mysql
  namespace: "app"
  labels:
    app.kubernetes.io/instance: mysql
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: mysql
    app.kubernetes.io/version: 8.0.36
    helm.sh/chart: mysql-9.18.0
type: Opaque
data:
  mysql-root-password: "bXlzcWw="
  mysql-password: "bXlzcWw="
  mysql-replication-password: "cmVwbHBhc3M="

• 查看用户自定义的参数的值

root@k8s-master01:~# helm get values  mysql -n app
USER-SUPPLIED VALUES:
architecture: replication
auth:
  database: wpdb
  password: mysql
  replicationPassword: replpass
  rootPassword: mysql
  username: wpuser
global:
  storageClass: openebs-hostpath
secondary:
  replicaCount: 1

• 查看部署的mysql

root@k8s-master01:~# kubectl get pod -n app
NAME                READY   STATUS    RESTARTS      AGE
mysql-primary-0     1/1     Running   0             3m23s
mysql-secondary-0   1/1     Running   0             3m23s

root@k8s-master01:~# kubectl exec -it mysql-primary-0 -n app bash
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
I have no name!@mysql-primary-0:/$ 

I have no name!@mysql-primary-0:/$ mysql -uroot -hlocalhost -p
Enter password:
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 290
Server version: 8.0.36 Source distribution

Copyright (c) 2000, 2024, Oracle and/or its affiliates.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> SHOW DATABASES;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
| sys                |
| wpdb               |
+--------------------+
5 rows in set (0.01 sec)