Consul初体验

 

Preface

 

    Today I'm gonna implement a consul in my environment to discover service of MySQL database and check whether the master-slave is working normally.

 

Introduce

 

    Consul is a tool like zookeeper which can discover services we've registered on it.It's just a simple binary command which can run with simple configuration file(json format).

 

official website:

https://www.consul.io/

 

Framework

 

Hostname	IP	Port	OS Version	MySQL Version	MySQL Role	Consul Version	Consul Role
zlm1	192.168.56.100	3306	CentOS 7.0	5.7.21	master	Consul v1.2.2	server
zlm2	192.168.56.101	3306	CentOS 7.0	5.7.21	slave	Consul v1.2.2	client

 

Procedure

 

Download and install consul.

 [root@zlm1 :: /vagrant]
 #wget https://releases.hashicorp.com/consul/1.2.2/consul_1.2.2_linux_amd64.zip
 ---- ::--  https://releases.hashicorp.com/consul/1.2.2/consul_1.2.2_linux_amd64.zip
 Resolving releases.hashicorp.com (releases.hashicorp.com)... 151.101.229.183, 2a04:4e42:::
 Connecting to releases.hashicorp.com (releases.hashicorp.com)|151.101.229.183|:... connected.
 HTTP request sent, awaiting response...  OK
 Length:  (18M) [application/zip]
 Saving to: ‘consul_1..2_linux_amd64.zip’
 
 %[===========================================================================================================>] ,,  .8KB/s   in 6m 12s 
 
 -- :: (48.3 KB/s) - ‘consul_1..2_linux_amd64.zip’ saved [/]
 
 [root@zlm1 :: /vagrant/consul_1..2_linux_amd64]
 #ls -l
 total
 -rwxrwxrwx  vagrant vagrant  Jul  : consul //There's only a binary command in the unziped directory of zip file.
 
 [root@zlm1 :: /vagrant/consul_1..2_linux_amd64]
 #mkdir /etc/consul.d
 
 [root@zlm1 :: /vagrant/consul_1..2_linux_amd64]
 #mkdir /data/consul
 
 [root@zlm1 :: /vagrant/consul_1..2_linux_amd64]
 #cp consul ~
 
 [root@zlm1 :: ~]
 #cd /usr/local/bin
 
 [root@zlm1 :: /usr/local/bin]
 #ls -l|grep consul
 lrwxrwxrwx  root root       Aug  : consul -> /root/consul
 
 [root@zlm1 :: /usr/local/bin]
 #consul --help
 Usage: consul [--version] [--help] <command> [<args>]
 
 Available commands are:
     agent          Runs a Consul agent
     catalog        Interact with the catalog
     connect        Interact with Consul Connect
     event          Fire a new event
     exec           Executes a command on Consul nodes
     force-leave    Forces a member of the cluster to enter the "left" state
     info           Provides debugging information for operators.
     intention      Interact with Connect service intentions
     join           Tell Consul agent to join cluster
     keygen         Generates a new encryption key
     keyring        Manages gossip layer encryption keys
     kv             Interact with the key-value store
     leave          Gracefully leaves the Consul cluster and shuts down
     lock           Execute a command holding a lock
     maint          Controls node or service maintenance mode
     members        Lists the members of a Consul cluster
     monitor        Stream logs from a Consul agent
     operator       Provides cluster-level tools for Consul operators
     reload         Triggers the agent to reload configuration files
     rtt            Estimates network round trip time between nodes
     snapshot       Saves, restores and inspects snapshots of Consul server state
     validate       Validate config files/directories
     version        Prints the Consul version
     watch          Watch for changes in Consul

Start agent of consul by "dev" mode.

 [root@zlm1 :: /usr/local/bin]
 #consul agent -dev
 ==> Starting Consul agent...
 ==> Consul agent running!
            Version: 'v1.2.2'
            Node ID: '7c839914-8a47-ab36-8920-1a9da54fc6c3'
          Node name: 'zlm1'
         Datacenter: 'dc1' (Segment: '<all>')
             Server: true (Bootstrap: false)
        Client Addr: [127.0.0.1] (HTTP: , HTTPS: -, DNS: )
       Cluster Addr: 127.0.0.1 (LAN: , WAN: )
            Encrypt: Gossip: false, TLS-Outgoing: false, TLS-Incoming: false
 
 ==> Log data will now stream in as it occurs:
 
     // :: [DEBUG] agent: Using random ID "7c839914-8a47-ab36-8920-1a9da54fc6c3" as node ID
     // :: [INFO] raft: Initial configuration (index=): [{Suffrage:Voter ID:7c839914-8a47-ab36--1a9da54fc6c3 Address:127.0.0.1:}]
     // :: [INFO] serf: EventMemberJoin: zlm1.dc1 127.0.0.1
     // :: [INFO] serf: EventMemberJoin: zlm1 127.0.0.1
     // :: [INFO] raft: Node at 127.0.0.1: [Follower] entering Follower state (Leader: "")
     // :: [INFO] consul: Adding LAN server zlm1 (Addr: tcp/127.0.0.1:) (DC: dc1)
     // :: [INFO] consul: Handled member-join event for server "zlm1.dc1" in area "wan"
     // :: [DEBUG] agent/proxy: managed Connect proxy manager started
     // :: [WARN] agent/proxy: running as root, will not start managed proxies
     // :: [INFO] agent: Started DNS server 127.0.0.1: (tcp)
     // :: [INFO] agent: Started DNS server 127.0.0.1: (udp)
     // :: [INFO] agent: Started HTTP server on 127.0.0.1: (tcp)
     // :: [INFO] agent: started state syncer
     // :: [WARN] raft: Heartbeat timeout from "" reached, starting election
     // :: [INFO] raft: Node at 127.0.0.1: [Candidate] entering Candidate state in term
     // :: [DEBUG] raft: Votes needed:
     // :: [DEBUG] raft: Vote granted from 7c839914-8a47-ab36--1a9da54fc6c3 in term . Tally:
     // :: [INFO] raft: Election won. Tally:
     // :: [INFO] raft: Node at 127.0.0.1: [Leader] entering Leader state
     // :: [INFO] consul: cluster leadership acquired
     // :: [INFO] consul: New leader elected: zlm1
     // :: [INFO] connect: initialized CA with provider "consul"
     // :: [DEBUG] consul: Skipping self join check for "zlm1" since the cluster is too small
     // :: [INFO] consul: member 'zlm1' joined, marking health alive
     // :: [DEBUG] agent: Skipping remote check "serfHealth" since it is managed automatically
     // :: [INFO] agent: Synced node info
     // :: [DEBUG] agent: Skipping remote check "serfHealth" since it is managed automatically
     // :: [DEBUG] agent: Node info in sync
     // :: [DEBUG] agent: Node info in sync
     // :: [DEBUG] consul: Skipping self join check for "zlm1" since the cluster is too small
     // :: [DEBUG] agent: Skipping remote check "serfHealth" since it is managed automatically
     // :: [DEBUG] agent: Node info in sync
     // :: [DEBUG] manager: Rebalanced  servers, next active server is zlm1.dc1 (Addr: tcp/127.0.0.1:) (DC: dc1)
     // :: [DEBUG] consul: Skipping self join check for "zlm1" since the cluster is too small
     // :: [DEBUG] consul: Skipping self join check for "zlm1" since the cluster is too small
     // :: [DEBUG] agent: Skipping remote check "serfHealth" since it is managed automatically
     // :: [DEBUG] agent: Node info in sync
 
 //Now the consul cluser has only one node "zlm1".
 
 [root@zlm1 :: ~]
 #consul members
 Node  Address         Status  Type    Build  Protocol  DC   Segment
 zlm1  127.0.0.1:  alive   server  1.2.           dc1  <all>
 
 //Type "Ctrl+C" can exit consul gracefully.
 ^C    // :: [INFO] agent: Caught signal:  interrupt
     // :: [INFO] agent: Graceful shutdown disabled. Exiting
     // :: [INFO] agent: Requesting shutdown
     // :: [WARN] agent: dev mode disabled persistence, killing all proxies since we can't recover them
     // :: [DEBUG] agent/proxy: Stopping managed Connect proxy manager
     // :: [INFO] consul: shutting down server
     // :: [WARN] serf: Shutdown without a Leave
     // :: [WARN] serf: Shutdown without a Leave
     // :: [INFO] manager: shutting down
     // :: [INFO] agent: consul server down
     // :: [INFO] agent: shutdown complete
     // :: [INFO] agent: Stopping DNS server 127.0.0.1: (tcp)
     // :: [INFO] agent: Stopping DNS server 127.0.0.1: (udp)
     // :: [INFO] agent: Stopping HTTP server 127.0.0.1: (tcp)
     // :: [INFO] agent: Waiting for endpoints to shut down
     // :: [INFO] agent: Endpoints down
     // :: [INFO] agent: Exit code: 
 
 [root@zlm1 :: /usr/local/bin]
 #ps aux|grep consul
 root        0.0  0.0     pts/    R+   :   : grep --color=auto consul
 
 [root@zlm1 :: /usr/local/bin]
 #consul members
 Error retrieving members: Get http://127.0.0.1:8500/v1/agent/members?segment=_all: dial tcp 127.0.0.1:8500: connect: connection refused

Prepare the server configuration file and start it again.

 [root@zlm1 :: /etc/consul.d]
 #cat server.json
 {
     "data_dir":"/data/consul",
     "datacenter":"dc1",
     "log_level":"INFO",
     "server":true,
     "bootstrap_expect":,
     "bind_addr":"192.168.56.100",
     "client_addr":"192.168.56.100",
     "ports":{
     },
     "ui":true,
     "retry_join":[],
     "retry_interval":"3s",
     "raft_protocol":,
     "rejoin_after_leave":true
 }
 
 [root@zlm1 :: /etc/consul.d]
 #consul agent --config-dir=/etc/consul.d/ > /data/consul/consul.log >& &
 [] 
 
 [root@zlm1 :: /etc/consul.d]
 #ps aux|grep consul
 root        1.3  2.0    pts/    Sl   :   : consul agent --config-dir=/etc/consul.d/
 root        0.0  0.0     pts/    R+   :   : grep --color=auto consul
 
 [root@zlm1 :: /etc/consul.d]
 #tail -f /data/consul/consul.log
     // :: [INFO] agent: Started HTTP server on 192.168.56.100: (tcp)
     // :: [INFO] agent: started state syncer
     // :: [WARN] raft: Heartbeat timeout from "" reached, starting election
     // :: [INFO] raft: Node at 192.168.56.100: [Candidate] entering Candidate state in term
     // :: [INFO] raft: Election won. Tally:
     // :: [INFO] raft: Node at 192.168.56.100: [Leader] entering Leader state
     // :: [INFO] consul: cluster leadership acquired
     // :: [INFO] consul: New leader elected: zlm1
     // :: [INFO] consul: member 'zlm1' joined, marking health alive
     // :: [INFO] agent: Synced node info
 
 [root@zlm1 :: ~]
 #consul members
 Error retrieving members: Get http://127.0.0.1:8500/v1/agent/members?segment=_all: dial tcp 127.0.0.1:8500: connect: connection refused
 
 [root@zlm1 :: ~]
 #consul members --http-addr=192.168.56.100:
 Node  Address              Status  Type    Build  Protocol  DC   Segment
 zlm1  192.168.56.100:  alive   server  1.2.           dc1  <all>

Add a client on zlm2.

 [root@zlm1 :: /etc/consul.d]
 #scp /root/consul zlm2:~
 consul                                                                                                             %   80MB  .3MB/s   :
 
 [root@zlm2 :: ~]
 #mkdir /etc/consul.d
 
 [root@zlm2 :: ~]
 #mkdir /data/consul
 
 [root@zlm2 :: ~]
 #cd /etc/consul.d/
 
 [root@zlm2 :: /etc/consul.d]
 #ls -l
 total
 -rw-r--r--  root root  Aug  : client.json
 
 [root@zlm2 :: /etc/consul.d]
 #cat client.json
 {
     "data_dir": "/data/consul",
     "enable_script_checks": true,
     "bind_addr": "192.168.56.101",
     "retry_join": ["192.168.56.100"],
     "retry_interval": "30s",
     "rejoin_after_leave": true,
     "start_join": ["192.168.56.100"]
 }
 
 [root@zlm2 :: /etc/consul.d]
 #consul agent -client 192.168.56.101 -bind 192.168.56.101 --config-dir=/etc/consul.d
 ==> Starting Consul agent...
 ==> Joining cluster...
     Join completed. Synced with  initial agents
 ==> Consul agent running!
            Version: 'v1.2.2'
            Node ID: 'a69eae21-4e31-7edf-1f1a-3ec285a8fb3b'
          Node name: 'zlm2'
         Datacenter: 'dc1' (Segment: '')
             Server: false (Bootstrap: false)
        Client Addr: [192.168.56.101] (HTTP: , HTTPS: -, DNS: )
       Cluster Addr: 192.168.56.101 (LAN: , WAN: )
            Encrypt: Gossip: false, TLS-Outgoing: false, TLS-Incoming: false
 
 ==> Log data will now stream in as it occurs:
 
     // :: [INFO] serf: EventMemberJoin: zlm2 192.168.56.101
     // :: [INFO] agent: Started DNS server 192.168.56.101: (udp)
     // :: [WARN] agent/proxy: running as root, will not start managed proxies
     // :: [INFO] agent: Started DNS server 192.168.56.101: (tcp)
     // :: [INFO] agent: Started HTTP server on 192.168.56.101: (tcp)
     // :: [INFO] agent: (LAN) joining: [192.168.56.100]
     // :: [INFO] agent: Retry join LAN is supported for: aliyun aws azure digitalocean gce os packet scaleway softlayer triton vsphere
     // :: [INFO] agent: Joining LAN cluster...
     // :: [INFO] agent: (LAN) joining: [192.168.56.100]
     // :: [INFO] serf: EventMemberJoin: zlm1 192.168.56.100
     // :: [INFO] agent: (LAN) joined:  Err: <nil>
     // :: [INFO] agent: started state syncer
     // :: [INFO] consul: adding server zlm1 (Addr: tcp/192.168.56.100:) (DC: dc1)
     // :: [INFO] agent: (LAN) joined:  Err: <nil>
     // :: [INFO] agent: Join LAN completed. Synced with  initial agents
     // :: [INFO] agent: Synced node info
 
 [root@zlm2 :: /etc/consul.d]
 #consul members --http-addr=192.168.56.100:
 Node  Address              Status  Type    Build  Protocol  DC   Segment
 zlm1  192.168.56.100:  alive   server  1.2.           dc1  <all>
 zlm2  192.168.56.101:  alive   client  1.2.           dc1  <default>

Add two services on node zlm2.

 [root@zlm2 :: /etc/consul.d]
 #cat service_master_check.json
 {
     "service":
        {
             "name": "w_db3306",
             "tags":  [
                           "Master Satus Check"
                         ],
             "address": "192.168.56.100",
             "port": ,
             "check":
                             {
                                 "args": [
                                              "/data/consul/script/CheckMaster.py",
                                              ""
                                             ],
                                 "interval": "15s"
                             }
         }
 }
 
 [root@zlm2 :: /etc/consul.d]
 #cat service_slave_check.json
 {
     "service":
         {
             "name": "r_db3306",
             "tags": [
                          "Slave Satus Check"
                         ],
             "address": "192.168.56.101",
             "port": ,
             "check":
                             {
                                 "args": [
                                              "/data/consul/script/CheckSlave.py",
                                              ""
                                             ],
                                 "interval": "15s"
                             }
         }
 }
 
 [root@zlm2 :: /etc/consul.d]
 #ls -l
 total
 -rw-r--r--  root root  Aug  : client.json
 -rw-r--r--  root root  Aug  : service_master_check.json
 -rw-r--r--  root root  Aug  : servi_slave_check.json

Check CheckMaster.py script.

 [root@zlm2 :: /data/consul/script]
 #cat CheckMaster.py
 #!/usr/bin/python
 import sys
 import os
 import pymysql
 
 port = int(sys.argv[])
 var={}
 conn = pymysql.connect(host='192.168.56.100',port=port,user='zlm',passwd='zlmzlm')
 #cur = conn.cursor(pymysql.cursor.DictCursor)
 cur = conn.cursor()
 
 cur.execute("show global variables like \"%read_only%\"")
 rows = cur.fetchall()
 
 for r in rows:
     var[r[]]=r[]
 if var['read_only']=='OFF' and var['super_read_only']=='OFF':
         print "MySQL %d master instance." % port
 else:
         print "This is read only instance."
         sys.exit()
 
 sys.exit()
 cur.close()
 conn.close()

Check CheckSlave.py script.

 [root@zlm2 :: /data/consul/script]
 #cat CheckSlave.py
 #!/usr/bin/python
 import sys
 import os
 import pymysql
 
 port = int(sys.argv[])
 var={}
 conn = pymysql.connect(host='192.168.56.101',port=port,user='zlm',passwd='zlmzlm')
 cur = conn.cursor
 cur.execute("show global variables like \"%read_only%\"")
 rows = cur.fetchall()
 
 for r in rows:
     var[r[]]=r[]
 if var['read_only']=='OFF' and var['super_read_only']=='OFF':
         print "MySQL %d master instance." % port
         sys.exit()
 else:
         print "MySQL %d is read only instance." % port
 
 cur = conn.cursor(pymysql.cursors.DictCursor)
 cur.execute("show slave status")
 slave_status = cur.fetchone()
 
 if len(slave_status)<:
         print "Slave replication setup error.";
         sys.exit()
 
 if slave_status['Slave_IO_Running'] !='Yes' or slave_status['Slave_SQL_Running'] !='Yes':
         print "Replication error: replication from host=%s, port=%s, io_thread=%s, sql_thread=%s, error info %s %s" % (slave_status['Master_Host'],slave_status['Master_Port'],slave_status['Slave_IO_Running'],slave_status['Slave_SQL_Running'],slave_status['Last_IO_Error'],slave_status['Last_SQL_Error'])
         sys.exit()
 print slave_status
 
 sys.exit()
 cur.close()
 conn.close()

Restartt consul(or reload).

 [root@zlm2 :: /etc/consul.d]
 #consul members --http-addr=192.168.56.100:
 Node  Address              Status  Type    Build  Protocol  DC   Segment
 zlm1  192.168.56.100:  alive   server  1.2.           dc1  <all>
 zlm2  192.168.56.101:  alive   client  1.2.           dc1  <default>
 
 [root@zlm2 :: /etc/consul.d]
 #consul leave --http-addr=192.168.56.101:
 Graceful leave complete
 
 [root@zlm2 :: /etc/consul.d]
 #consul members --http-addr=192.168.56.100:
 Node  Address              Status  Type    Build  Protocol  DC   Segment
 zlm1  192.168.56.100:  alive   server  1.2.           dc1  <all>
 zlm2  192.168.56.101:  left    client  1.2.           dc1  <default>
 
 [root@zlm2 :: /etc/consul.d]
 #consul agent --config-dir=/etc/consul.d/ -client 192.168.56.101 > /data/consul/consul.log >& &
 [] 
 
 [root@zlm2 :: /etc/consul.d]
 #!ps
 ps aux|grep consul
 root        2.3  2.0    pts/    Sl   :   : consul agent --config-dir=/etc/consul.d/ -client 192.168.56.101
 root        0.0  0.0     pts/    R+   :   : grep --color=auto consul
 
 [root@zlm2 :: /etc/consul.d]
 #consul reload -http-addr=192.168.56.101:
 Configuration reload triggered

Well,now we can login the dashbord(GUI) of consul to check MySQL working status.