第四模块 ：微服务调用链监控CAT架构和实践

采样率：每一个请求为都进行记录，或者100次请求为记录50次

各个开源框架都满足opentracing的标准，只要使用opentracing标准埋点的客户端，可以使用不同的客户端去展示，opentracing对客户端调用的标准、tracing的标准进行了规范化

69.调用链监控产品和比较~1.mp4

1、调用链的可视化，上面三款产品都支持

2、服务之间的依赖关系，Pinpoint做的最好，图形化展示直观形象

3、埋点 cat和zipnkin都是侵入式的，Pinpoint的是aop的不需要修改代码

4、实例的cpu、内存的情况，Pinpoint有

71.CAT 典型报表~1.mp4

cat的 报表丰富是cat的最大的优点

cat会把它接入的应用程序对于的报错情况，实时的做成一个报错大盘，以1分钟为单位展示出来

cat是支持业务监控的，例如支付成功次数等，但是对于业务指标，最好使用专门的metris监控工具去做，cat最大的长处是调用链监控

cat除了全链路追踪功能之外，还有其他功能

transaction报表

整个应用在1小时内发生了多少次调用，有多少次是失败的，整个应用的最小延迟，最大延迟，平均值，如果95line和99line和平均值相差不大，说明该系统很稳定，qps就是服务的调用频率

点击该应用可以看到该应用下接口的性能统计情况

例如一个url请求超过了50毫秒。我们可以进行搜索，查看对于的具体信息

72.CAT 告警简介~1.mp4

73.CAT 架构设计~1.mp4

原始的调用链trace数据存储在hdfs中，聚合的报表数据存储在mysql中，按照分钟、小时的报表聚合数据是存储在mysql中的

74.【实验】CAT 本地部署~1.mp4

cat的安装，第一步首先安装cat的源码

这里按照视频上的，我们安装2.0.0版本

进入改目录执行下面的操作：进行maven打包的操作

mvn clean install -DskipTests

打包成功之后war包位于cat-home目录下，我们将cat-alpha-2.0.0.war重新命名为cat.war

打包完成之后接下来要进行数据库的操作

数据库的脚本在这个目录下

创建一个cat的数据库

然后执行cat.sql 的脚本

我们cat运行在d盘，这里需要强调的是在tomcat所在的盘根目录需要有data文件夹，这里我们把tomcat放在d盘目录，因此我们在d盘建立下面的目录D:\data\appdatas\cat

我们将下面的三个xml文件拷贝到D:\data\appdatas\cat目录下

接下来修改client.xml的配置文件

<?xml version="1.0" encoding="utf-8"?>

<config mode="client" xmlns:xsi="http://www.w3.org/2001/XMLSchema" xsi:noNamespaceSchemaLocation="config.xsd">
    <servers>
        <!-- Local mode for development -->
        <server ip="192.168.43.80" port="" http-port="" />
        <!-- If under production environment, put actual server address as list. -->
        <!--
            <server ip="192.168.7.71" port="" />
            <server ip="192.168.7.72" port="" />
        -->
    </servers>
</config>

这里client要配置server的地址，port是上传文件到server的TCP端口是2280，2281是cat页面访问的端口，这里要写本地机器的IP地址，不要写成127.0.0.1

<!-- ip：部署CAT应用的服务器IP

port：CAT服务端接收客户端数据的端口（不允许更改）

http-port：CAT应用部署到的容器的端口（tomcat的端口）

-->

<server ip="10.10.10.121" port="2280" http-port="8080" />

浏览器打开http://localhost:2281/cat 可以看到cat监控的界面

只有客户端配置了这个配置文件，客户端才能正确连接到CAT服务器端

接下来配置datasources.xml，这里是cat访问mysql数据的配置

<?xml version="1.0" encoding="utf-8"?>

<data-sources>
<data-source id="cat">
<maximum-pool-size>3</maximum-pool-size>
<connection-timeout>1s</connection-timeout>
<idle-timeout>10m</idle-timeout>
<statement-cache-size>1000</statement-cache-size>
<properties>
<driver>com.mysql.jdbc.Driver</driver>
<url><![CDATA[jdbc:mysql://192.168.43.80:3306/cat]]></url>
<user>root</user>
<password>123456</password>
<connectionProperties><![CDATA[useUnicode=true&autoReconnect=true]]></connectionProperties>
</properties>
</data-source>
<data-source id="app">
<maximum-pool-size>3</maximum-pool-size>
<connection-timeout>1s</connection-timeout>
<idle-timeout>10m</idle-timeout>
<statement-cache-size>1000</statement-cache-size>
<properties>
<driver>com.mysql.jdbc.Driver</driver>
<url><![CDATA[jdbc:mysql://192.168.43.80:3306/cat]]></url>
<user>root</user>
<password>123456</password>
<connectionProperties><![CDATA[useUnicode=true&autoReconnect=true]]></connectionProperties>
</properties>
</data-source>
</data-sources>

接下来我们要配置server.xml

<?xml version="1.0" encoding="utf-8"?>

<!-- local-mode：是否为本地开发模式。建议在开发环境以及生产环境都设置为false

hdfs-machine：是否启用HDFS存储

job-machine：是否为报告工作机（开启生成汇总报告和统计报告，只需要一台服务机开启此功能）

alert-machine：是否为报警机（开启各类报警监听, 只需要一台服务机开启此功能）

-->

控制台 - 提供给业务人员进行数据查看【默认所有的cat节点都可以作为控制台，不可配置】
消费机 - 实时接收业务数据，实时处理，提供实时分析报表【默认所有的cat节点都可以作为消费机，不可配置】
任务机(job-machine) - 做一些离线的任务，合并天、周、月等报表 【可以配置】
告警端(alert-machine) - 启动告警线程，进行规则匹配，发送告警（目前仅支持单点部署）【可以配置】

local-mode: 建议在开发环境以及生产环境时，都设置为false
hdfs-machine: 定义是否启用 HDFS 存储方式，默认为 false
job-machine: 定义当前服务是否为报告工作机（开启生成汇总报告和统计报告的任务，只需要一台服务机开启此功能），默认为 false
alert-machine: 定义当前服务是否为报警机（开启各类报警监听，只需要一台服务机开启此功能），默认为 false；
storage: 定义数据存储配置信息
local-report-storage-time: 定义本地报告文件存放时长，单位为（天）
local-logivew-storage-time: 定义本地日志文件存放时长，单位为（天）
local-base-dir: 定义本地数据存储目录，建议直接使用/data/appdatas/cat/bucket目录
hdfs: 定义HDFS配置信息
server-uri: 定义HDFS服务地址
console: 定义服务控制台信息
remote-servers: 定义HTTP服务列表，（远程监听端同步更新服务端信息即取此值）

<?xml version="1.0" encoding="utf-8"?>

<!-- Configuration for development environment-->
<config local-mode="false" hdfs-machine="false" job-machine="true" alert-machine="true">

    <storage  local-base-dir="/data/appdatas/cat/bucket/" max-hdfs-storage-time="" local-report-storage-time="" local-logivew-storage-time="">

    </storage>

    <console default-domain="Cat" show-cat-domain="true">
        <remote-servers>192.168.43.80:</remote-servers>
    </console>

</config>

这里本地开发模式，不适用hdfs存储，启动聚合，启动报警，远程页面来访问server进行数据查询的短路时2281

接下来我们要将打包出来的cat.war包部署到tomcat中

这里因为上面配置的页面访问http-port端口是2281，这里tomcat中server.xml的端口地址也要配置为2281

接下来，我们把cat.war包部署到webapps目录下

接下来我们启动tomcat，我们在D:\data\applogs\cat下可以看到启动对于的目录，这个日志目录一定要保证具有可读写的权限

启动tomcat的时候要查看日志里面没有错误的信息

[- ::28.598] [INFO] [ServerConfigManager] Loading configuration file(D:\data\appdatas\cat\server.xml) ...
[- ::28.696] [INFO] [ServerConfigManager] CAT server is running with hdfs,false
[- ::31.585] [INFO] [ServerConfigManager] CAT server is running with alert,false
[- ::31.585] [INFO] [ServerConfigManager] CAT server is running with job,true
[- ::31.590] [INFO] [ServerConfigManager] <?xml version="1.0" encoding="utf-8"?>
<config local-mode="false" job-machine="true" alert-machine="false" hdfs-machine="false" send-machine="true">
   <storage local-base-dir="/data/appdatas/cat/bucket/" upload-thread="" max-hdfs-storage-time="" local-report-storage-time="" local-logivew-storage-time="">
   </storage>
   <consumer>
      <long-config default-url-threshold="" default-sql-threshold="" default-service-threshold="">
      </long-config>
   </consumer>
   <console default-domain="Cat" show-cat-domain="true">
      <remote-servers>192.168.43.80:</remote-servers>
   </console>
</config>

[- ::33.287] [INFO] [TcpSocketReceiver] start netty server!
[- ::33.295] [INFO] [CatClientModule] Current working directory is D:\apache-tomcat-8.5.\bin
[- ::33.340] [INFO] [DefaultClientConfigManager] Global config file(\data\appdatas\cat\client.xml) found.
[- ::33.344] [INFO] [DefaultClientConfigManager] Find domain name cat from app.properties.
[- ::33.381] [INFO] [DefaultTransportManager] Remote CAT servers: [/127.0.0.1:]
[- ::35.683] [INFO] [ChannelManager] Connected to CAT server at /127.0.0.1:
[- ::35.686] [INFO] [ChannelManager] success when init CAT server, new active holderactive future :/127.0.0.1: index: ip:127.0.0.1 server config:null
[- ::35.690] [INFO] [CatThreadListener] Thread group(cat) created.
[- ::35.692] [INFO] [TcpSocketSender] Starting thread(cat-TcpSocketSender) ...
[- ::35.693] [INFO] [TcpSocketSender] Starting thread(cat-TcpSocketSender-ChannelManager) ...
[- ::35.695] [INFO] [TcpSocketSender] Starting thread(cat-merge-atomic-task) ...
[- ::35.737] [INFO] [CatClientModule] Starting thread(cat-StatusUpdateTask) ...
[- ::36.008] [INFO] [DefaultDataSourceProvider] Loading data sources from \data\appdatas\cat\datasources.xml ...
[- ::36.707] [INFO] [JdbcDataSource] Connecting to JDBC data source(cat) with properties(driver=com.mysql.jdbc.Driver, url=jdbc:mysql://127.0.0.1:3306/cat?useUnicode=true&autoReconnect=true, user=root) ...
[- ::37.729] [INFO] [JdbcDataSource] Connected to JDBC data source(cat).
[- ::37.932] [INFO] [ServerFilterConfigManager] Starting thread(cat-Server-Filter-Config-Reload) ...
[- ::38.290] [INFO] [LocalMessageBucketManager] Starting thread(cat-LocalMessageBucketManager-BlockDumper) ...
[- ::38.294] [INFO] [LocalMessageBucketManager] Starting thread(cat-LocalMessageBucketManager-OldMessageMover) ...
[- ::38.304] [INFO] [LocalMessageBucketManager] Starting thread(cat-Message-Gzip-) ...
[- ::38.304] [INFO] [LocalMessageBucketManager] Starting thread(cat-Message-Gzip-) ...
[- ::38.305] [INFO] [LocalMessageBucketManager] Starting thread(cat-Message-Gzip-) ...
[- ::38.306] [INFO] [LocalMessageBucketManager] Starting thread(cat-Message-Gzip-) ...
[- ::38.306] [INFO] [LocalMessageBucketManager] Starting thread(cat-Message-Gzip-) ...
[- ::38.306] [INFO] [LocalMessageBucketManager] Starting thread(cat-Message-Gzip-) ...
[- ::38.307] [INFO] [LocalMessageBucketManager] Starting thread(cat-Message-Gzip-) ...
[- ::38.307] [INFO] [LocalMessageBucketManager] Starting thread(cat-Message-Gzip-) ...
[- ::38.308] [INFO] [LocalMessageBucketManager] Starting thread(cat-Message-Gzip-) ...
[- ::38.308] [INFO] [LocalMessageBucketManager] Starting thread(cat-Message-Gzip-) ...
[- ::38.308] [INFO] [LocalMessageBucketManager] Starting thread(cat-Message-Gzip-) ...
[- ::38.309] [INFO] [LocalMessageBucketManager] Starting thread(cat-Message-Gzip-) ...
[- ::38.309] [INFO] [LocalMessageBucketManager] Starting thread(cat-Message-Gzip-) ...
[- ::38.310] [INFO] [LocalMessageBucketManager] Starting thread(cat-Message-Gzip-) ...
[- ::38.310] [INFO] [LocalMessageBucketManager] Starting thread(cat-Message-Gzip-) ...
[- ::38.311] [INFO] [LocalMessageBucketManager] Starting thread(cat-Message-Gzip-) ...
[- ::38.311] [INFO] [LocalMessageBucketManager] Starting thread(cat-Message-Gzip-) ...
[- ::38.312] [INFO] [LocalMessageBucketManager] Starting thread(cat-Message-Gzip-) ...
[- ::38.312] [INFO] [LocalMessageBucketManager] Starting thread(cat-Message-Gzip-) ...
[- ::38.313] [INFO] [LocalMessageBucketManager] Starting thread(cat-Message-Gzip-) ...
[- ::38.845] [INFO] [Period] Starting  tasks in period [-- ::, -- ::]
[- ::38.846] [INFO] [CatThreadListener] Thread group(Cat-RealtimeConsumer) created.
[- ::38.848] [INFO] [Period] Starting thread(Cat-RealtimeConsumer-ProblemAnalyzer--) ...
[- ::38.850] [INFO] [Period] Starting thread(Cat-RealtimeConsumer-TopAnalyzer--) ...
[- ::38.850] [INFO] [Period] Starting thread(Cat-RealtimeConsumer-MetricAnalyzer--) ...
[- ::38.851] [INFO] [Period] Starting thread(Cat-RealtimeConsumer-HeartbeatAnalyzer--) ...
[- ::38.852] [INFO] [Period] Starting thread(Cat-RealtimeConsumer-ProblemAnalyzer--) ...
[- ::38.860] [INFO] [Period] Starting thread(Cat-RealtimeConsumer-CrossAnalyzer--) ...
[- ::38.860] [INFO] [Period] Starting thread(Cat-RealtimeConsumer-StorageAnalyzer--) ...
[- ::38.861] [INFO] [Period] Starting thread(Cat-RealtimeConsumer-TransactionAnalyzer--) ...
[- ::38.861] [INFO] [Period] Starting thread(Cat-RealtimeConsumer-CrossAnalyzer--) ...
[- ::38.861] [INFO] [Period] Starting thread(Cat-RealtimeConsumer-DumpAnalyzer--) ...
[- ::38.862] [INFO] [Period] Starting thread(Cat-RealtimeConsumer-StateAnalyzer--) ...
[- ::38.862] [INFO] [Period] Starting thread(Cat-RealtimeConsumer-EventAnalyzer--) ...
[- ::38.862] [INFO] [Period] Starting thread(Cat-RealtimeConsumer-DumpAnalyzer--) ...
[- ::38.862] [INFO] [Period] Starting thread(Cat-RealtimeConsumer-EventAnalyzer--) ...
[- ::38.862] [INFO] [Period] Starting thread(Cat-RealtimeConsumer-TransactionAnalyzer--) ...
[- ::38.863] [INFO] [RealtimeConsumer] Starting thread(cat-RealtimeConsumer-PeriodManager) ...
[- ::38.961] [INFO] [CatHomeModule] Starting thread(cat-Config-Reload) ...
[- ::39.891] [INFO] [CatHomeModule] Starting thread(cat-Task-Consumer) ...
[- ::39.897] [INFO] [MVC] MVC is starting at /cat
[- ::40.131] [INFO] [CatThreadListener] Thread pool(Cat-ModelService) created.
[- ::41.129] [INFO] [TopologyGraphManager] Starting thread(cat-TopologyGraphReload) ...
[- ::41.358] [INFO] [UrlPatternConfigManager] Starting thread(cat-UrlPattern-Config-Reload) ...
[- ::41.960] [INFO] [AggregationConfigManager] Starting thread(cat-Aggreation-Config-Reload) ...
[- ::42.434] [INFO] [MVC] MVC started at /cat
[- ::00.515] [INFO] [Worker] Starting thread(Cat-ModelService-) ...
[- ::45.770] [WARN] [TcpSocketSender] Netty write buffer is full! Attempts:
[- ::00.501] [INFO] [Worker] Starting thread(Cat-ModelService-) ...
[- ::05.745] [INFO] [TcpSocketSender] success when init CAT server, new active holderactive future :/127.0.0.1: index: ip:127.0.0.1 server config:127.0.0.1:;
[- ::00.502] [INFO] [Worker] Starting thread(Cat-ModelService-) ...
[- ::00.502] [INFO] [Worker] Starting thread(Cat-ModelService-) ...
[- ::00.503] [INFO] [Worker] Starting thread(Cat-ModelService-) ...

上面没有错误的信息

我们在页面上可以看到访问的信息为：

这样cat就已经启动成功了，接下来我们要进行全局路由的设置

点击页面上的配置按钮

需要进行登录，登录的密码默认是用户名是catadmin，密码是admin

点击客户端路由

这里客户端能够访问的server地址只有一台，我们只需要将127.0.0.1替换成当前机器的IP地址就可以了

<?xml version="1.0" encoding="utf-8"?>
<router-config backup-server="192.168.43.80" backup-server-port="">
   <default-server id="192.168.43.80" weight="1.0" port="" enable="true"/>
</router-config>

点击cat'上面的state

如果看到

这才说明我们的cat服务器安装成功。