Zipkin Server Configuration Using Docker and MySQL[转]
Zipkin is a used for capturing timing data, it also has a centralized repository, and a microweb server that allows you to display, and search through spans and traces of your distributed programs or websites.
We can configure Zipkin by deploying it in a Docker container. Using this approach, we can match the latest version of Zipkin by just pulling down the latest images. You can Isolate the Docker service and dependencies to just the container(s), and chose where you want your data to persist. Most importantly, by using Docker, you can spend more time focused on data analysis, instead of spending time on configuring a Zipkin as a service.
The Target Scenario
Our main task is setting up a Zipkin server with MySQL, so that the spans/traces persist on the host file.
Zipkin Server Configuration
Install Package Dependencies
Log into your Zipkin host machine and make sure your system is up to date:
sudo dnf update && sudo dnf upgrade
Add the Docker repository:
sudo dnf install dnf-plugins-core
sudo dnf config-manager --add-repo https://download.docker.com/linux/fedora/docker-ce.repo
Install Docker CE:
sudo dnf install docker-ce
Enable Docker as a service:
sudo systemctl enable docker.service
sudo systemctl start docker.service
(Optional) Add your limited user account to the
docker
group, so that you can run Docker commands without usingsudo
:sudo usermod -aG docker username
You can test your Docker installation by running
docker run hello-world
.Install Docker Compose:
curl -LO https://github.com/docker/compose/releases/download/1.16.1/docker-compose-`uname -s`-`uname -m`
sudo mv docker-compose-Linux-x86_64 /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose
You can test the installation with
docker-compose --version
.Note
The current stable version of Docker Compose is 1.16.1. Check for the latest version at the releases pageand update the version in thecurl
command accordingly.Install git:
sudo dnf install git
Use git to retrieve the Zipkin Docker-compose YAML files at openzipkin/docker-zipkin. This is one of the powerful features of Docker, these files hold all of the system level configurations we need, to run several different Zipkin configurations. Like, Zipkin with MySQL, Zipkin with elasticsearch, and Zipkin with Kakfa.
cd ~
git clone https://github.com/openzipkin/docker-zipkin.git
Install MySQL:
sudo dnf install mysql
Configure Docker
The Docker service will manage your containers, the container’s host, Zipkin services, and your MYSQL server.
Docker is in charge of starting and stopping these services automatically when the host system is rebooted. it’ll help us to map the ports from the container to the host’s ports and it’ll manage exporting the MySQL database files onto the host system. Docker can check to see if the container has failed, and restart it for us too. The host is in charge of running the actual Docker service and setting the firewall correctly.
Notice that the Zipkin container will expose port 9411 for its service, and the MySQL container will expose port 3306. We’ll use the Docker-compose YAML files to forward port 9411 to the host’s port 9411, so that the container will be accessible on the internet.
Zipkin Server Firewall Concepts
To avoid the server being compromised, limit the exposure of our Zipkin server to just our analyst and client machines.
The default Fedora 26 firewall rules block all ports as a safety precaution. Create a new firewall zone to handle the Zipkin services without exposing too much of the system:
Our goal is to set up the Zipkin Server for:
- Thrift service: receiving data from clients
- Web service: showing searches of time data
- lock down access to only our web and analyst machines.
Zipkin Server Firewall
Create a new zone in our firewall called zipkin
sudo firewall-cmd --new-zone=zipkin --permanent
Reload the firewall and refresh your zone list.
sudo firewall-cmd --reload
Add an analyst machine IP (If you forget to define any source IPs, you will have no filtering on your IPs. You need at least 1 source IP to start filtering on IPs. If there are no source IPs defined, any machine can connect to your server.)
sudo firewall-cmd --zone=zipkin --add-source=203.0.113.0/32 --permanent
Open a port through your firewall.
sudo firewall-cmd --zone=zipkin --add-port=9411/tcp --permanent
(Optional) Since we may want to access our machine from the analyst machine, it may be a good idea to add an ssh port.
sudo firewall-cmd --zone=zipkin --add-service=ssh --permanent
Reload your firewall rules to activate them in your new zone.
sudo firewall-cmd --reload
View your new zone:
sudo firewall-cmd --zone=zipkin --list-all
Docker-Compose Configuration
The docker-compose yml files will control which system configuration we can use. We’re going to select a MySQL configuration for storage.
Copy the MySQL docker-compose YAML file to your home directory and rename it docker-init.yml as we’re going to need to make a few changes:
cd ~
cp docker-zipkin/docker-compose.yml docker-init.yml
Open
docker-init.yml
in a text editor and edit the content as follows:- ~/docker-init.yml
-
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
version: '2' services:
storage:
image: openzipkin/zipkin-mysql
container_name: mysql
# Uncomment to expose the storage port for testing
ports:
- 3306:3306
volumes:
- dbfiles:/mysql/data
restart: unless-stopped zipkin:
image: openzipkin/zipkin
container_name: zipkin
# Environment settings are defined here https://github.com/openzipkin/zipkin/tree/1.19.0/zipkin-server#environment-variables
environment:
- STORAGE_TYPE=mysql
# Point the zipkin at the storage backend
- MYSQL_HOST=mysql
# Uncomment to enable scribe
# - SCRIBE_ENABLED=true
# Uncomment to enable self-tracing
# - SELF_TRACING_ENABLED=true
# Uncomment to enable debug logging
# - JAVA_OPTS=-Dlogging.level.zipkin=DEBUG
ports:
# Port used for the Zipkin UI and HTTP Api
- 9411:9411
depends_on:
- storage
restart: unless-stopped dependencies:
image: openzipkin/zipkin-dependencies
container_name: dependencies
entrypoint: crond -f
environment:
- STORAGE_TYPE=mysql
- MYSQL_HOST=mysql
# Add the baked-in username and password for the zipkin-mysql image
- MYSQL_USER=zipkin
- MYSQL_PASS=zipkin
# Uncomment to adjust memory used by the dependencies job
- JAVA_OPTS=-verbose:gc -Xms512m -Xmx512m
depends_on:
- storage
restart: unless-stopped volumes:
dbfiles:
In the MySQL container section in the docker-init.yml, export the MySQL data directory, forward the MySQL port to the host, and add the restart command so that this service is automatically restarted if it goes down.
In the Zipkin container section in the docker-init.yml, make sure the port 9411 is forwarded to the host machine and add the restart command so that this service is automatically restarted if it goes down.
In the dependencies container, we uncomment the JAVA_OPTS and set it to at least 512M. This setting is optimized for a 1G Linode. However, if in the future this container needs more memory, you can increase this value. Add a restart command to the end of this section.
You can now update your Zipkin Docker images by performing a
docker pull
command. This will check the web for the images we need (the first time), and all other times it’ll update the images to the latest version if need be.docker-compose -f docker-init.yml pull
Run your Docker services by using the
docker-compose up
command. Conversely there is also adocker-compose down
command that can be used to shutdown your Zipkin services.docker-compose -f docker-init.yml up -d
Notice the -d flag at the end of the command, this detaches the container. Now it’s running as its own process. If we just left the machine, it would continue to run, even if we rebooted the machine, it would run when the Docker service started it, because we didn’t explicitly issue a
docker-compose down
command. Just for reference, to shutdown the Zipkin services:docker-compose -f docker-init.yml down
Backup Span/Trace Data
There are 2 different backup methods: using MySQL , and using sysadmin.
MySQL Backup
Ensure that the MySQL service is running on a container. You can check this with a
docker ps
command. Thedocker ps
command displays the active containers:- docker -ps
-
1
2
3
4
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
023d14e6193d openzipkin/zipkin-dependencies "crond -f" 3 days ago Up 3 days dependencies
ee0c255b7765 openzipkin/zipkin "/bin/sh -c 'test ..." 3 days ago Up 3 days 9410/tcp, 0.0.0.0:9411->9411/tcp zipkin
43f659b36f17 openzipkin/zipkin-mysql "/bin/sh -c /mysql..." 3 days ago Up 3 days 0.0.0.0:3306->3306/tcp mysql
If isn’t running, make sure you start the Zipkin services with the
docker-compose up
command first. Then issue the MySQLdump with the following parameters from your Zipkin host machine.mysqldump --protocol=tcp -A -pzipkin -uzipkin > ~/database.bak
This command will dump the entire MySQL database from your MySQL container into the file called database.bak in your home directory. Alternatively, you can just dump your Zipkin span/trace data with:
mysqldump --protocol=tcp -pzipkin -uzipkin zipkin > ~/db_zipkin.bak
Database Backups
We can just zip or tar the exported database files on the host system. Since we don’t know if the container is writing information to these files at any given time, we need to make sure that the container is stopped.
We can check the status with a
docker ps
command or just perform adocker down
command.docker-compose -f docker-init.yml down
After the
docker down
command, we can perform adocker ps
command and see that there are no containers running. That should look like this:CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
At this point, we can create a zip backup of your files. The db files will be prepended with the name of your user. If you are running as root it would be root_dbfiles, but running as root isn’t recommended.
sudo zip -r ~/db_files.zip /var/lib/docker/volumes/_dbfiles/
Remember to start your Zipkin services if they’re still needed. They will not restart even on a reboot because we have explicitly shut them down.
Testing the Zipkin Service
Easiest way to do this is by using your web browser on your analyst machine. Log into your analyst machine, bring up your browser, and type in the following URL:
http://192.0.2.0:9411/zipkin/
If you see the Zipkin web page, you’re done.
If you don’t see a web page, log into the Zipkin host machine, and make sure your containers are up by running either
docker ps
command ordocker-compose up
command. If the containers are not all running, it’s possible that your Linode has run out of memory.Make sure your firewall port is open by typing:
sudo firewall-cmd --add-port 9411/tcp --permanent
At this point, what might have happened is that we added the wrong IP address of our analyst machine. We can check this by logging into our analyst machine. If our analyst machine is a Fedora workstation, we can install nmap and perform a network port status check to our Zipkin host machine like so:
sudo dnf install -y nmap
nmap 192.0.2.0 -p 9411 -Pn
A good return has an open for the STATE of the port, anything else and we probably don’t have the right analyst machine IP address in our firewall rules:
Starting Nmap 7.40 (https://nmap.org) at 2017-09-24 18:34 MDT
Nmap scan report for zipkin (192.0.2.0)
Host is up (0.10s latency).
PORT STATE SERVICE
9411/tcp open unknown
Review your firewall rules and try again.
More Information
You may wish to consult the following resources for additional information on this topic. While these are provided in the hope that they will be useful, please note that we cannot vouch for the accuracy or timeliness of externally hosted materials.
Zipkin Server Configuration Using Docker and MySQL[转]的更多相关文章
- docker nginx mysql
docker run -p 9000:9000 --name myphp -v /docker/www/:/var/www/html/ -v /docker/php/php.ini:/usr/loca ...
- docker安装MySQL 8.0及初始化错误处理
Preface Several days ago,I've implement a docker environmnet,I'm gonna Install a MySQL 8. ...
- linux记录-docker配置mysql
docker部署mysql 1.拉取镜像 docker pull mysql 2.docker rm containerID 删除镜像iD 3.创建镜像 docker run --name=m ...
- 阿里云服务器centos7,docker部署mysql+Redis+vue+springboot+Nginx+fastdfs,亲测可用
一.购买云服务器 我是今年双十一期间在阿里云购买的服务器, 简单配置2核_4G_40G_3M,三年用了不到800块,不过当时我记得腾讯云更便宜,个人感觉,阿里的云服务器更加的稳定, 毕竟身经百战, 经 ...
- docker 部署mysql连接问题
发现windows上有一个docker descktop(虽然不怎么好用), 安装之后准备直接用docker搭本地测试环境的基础设施(比如MySQL,Redis,MongoDB,ES啥的), 虽然比去 ...
- springboot多数据源配合docker部署mysql主从实现读写分离
本篇主要有两部分: 1.使用docker部署mysql主从 实现主从复制 2.springboot项目多数据源配置,实现读写分离 一.使用docker部署mysql主从 实现主从复制 此次使用的是wi ...
- Docker 安装 MySQL、Redis
1 Docker 中安装 Redis 1.1 创建目录 在硬盘上创建 redis 的数据目录: mkdir -p /Users/yygnb/dockerMe/redis/data 为该目录添加权限: ...
- Docker搭建MySQL服务
Docker开源镜像 前面我们已经安装好了Docker,也简单了解了Docker.那么我们可以尝试搭建一个MySQL服务. 要搭建服务就要启动服务容器,要创建容易就要有镜像,Docker提供了一个类似 ...
- 使用Docker安装Mysql
最近使用阿里云服务器,学习一下Docker,今天学着使用Docker安装MySQL. 首先,从阿里云的Docker Hub 上pull一个MySQL的image. [centos@loovelj~]$ ...
随机推荐
- 【原创】数据库基础之Mysql(3)mysql删除历史binlog
mysql开启binlog后会在/var/lib/mysql下创建binlog文件,如果手工删除,则下次mysql启动会报错: mysqld: File './master-bin.000001' n ...
- java结合testng,利用excel做数据源的数据驱动实例
数据驱动部分,是自动化测试常用部分,也是参数化设计的重要环节,前面分享了,mysql.yaml做数据源,那么再来分享下excel做数据驱动 思路: 先用POI读取excel.解析读取数据,返回list ...
- Python-WEB -VUE初识
走进Vue_渐进式 JavaScript 框架 通过对框架的了解与运用程度,来决定其在整个项目中的应用范围,最终可以独立以框架方式完成整个web前端项目 what -- 什么是Vue 可以独立完成前后 ...
- flex下部固定高,上部不固定,而且超过内容要滚动
<!DOCTYPE html> <html lang="zh"> <head> <meta charset="UTF-8&quo ...
- layui前端框架
项目中需要弹出层效果,使用了layui前端框架,主要使用了里面的弹出层特效(可以移动) html代码 要给这个标签绑定click方法 <a href='javascript:;' data-me ...
- HTML5-长按事件
<!DOCTYPE html> <html> <head> <meta charset="utf-8"/> <title> ...
- 判断ie浏览器7、8、9三个版本
判断ie浏览器7.8.9三个版本 上午的时候,本来是想做一个position:fixed在各个浏览器下兼容的方案的,但是发现ie7/8下面的position:fixed只支持一个屏幕,如果内容高度 ...
- django----Form详细信息
Form类: 创建Form类时,主要涉及到 [字段] 和 [插件],字段用于对用户请求数据的验证,插件用于自动生成HTML; Django内置字段 Field required=True, 是否允许为 ...
- cf861D 字典树+时间戳
好久没碰字典树之类的题了,搞起来有点生疏 /* 把所有母串的后缀加入字典树中 然后再扫一次所有母串的后缀,把后缀放到字典树中查询,找到第一个访问次数为1的结点返回即可 num在计数时,同一个母串的子串 ...
- bzoj 2761
神题... 其实这题巨水,用各种诡异的方法都能A,包括STL等等 我之所以写题解,是因为我发现了一个bug:bz和luogu时限有问题! 这题我用了两种做法: ①:直接使用STL-map(不能直接用数 ...