查找了很多文档,没有完全达到我要求的,

于是,作了一定的调整,成现在这样。

操作步骤如下:

一,准备好两个文件。

jmx_prometheus_javaagent-0.3.1.jar

jmx_exporter.yml

jmx_exporter.yml作最简单配置如下:

---
rules:
- pattern: ".*"

二,tomcat的启动项,一直在bin下单独的setenv.sh里操作,新增如下启动选项,将jar包及yaml加入启动选项。

JAVA_OPTS="$JAVA_OPTS -javaagent:/xxx/bin/jmx_prometheus_javaagent-0.3.1.jar=1234:/xxx/bin/jmx_exporter.yml"

三,制作dockerfile时,要将这两个东东也CP到上面指定目录,并EXPOSE一个新的端口。

...
COPY --chown=a:a tomcat /xxx/tomcat
COPY --chown=a:a jdk /xxx/jdk
COPY --chown=a:a jmx_prometheus_javaagent-0.3.1.jar /xxx/tomcat/bin/
COPY --chown=a:a jmx_exporter.yml /xxx/tomcat/bin/

WORKDIR ${CATALINA_HOME}

USER a
EXPOSE 8080 1234

四,更新prometheus的配置文件configmap,新增一个job。

- job_name: 'tomcat-pods'
        tls_config:
          ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
        bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
        kubernetes_sd_configs:
        - role: endpoints
        relabel_configs:
        - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_scrape, __meta_kubernetes_service_annotation_prometheus_io_jvm_scrape]
          regex: true;true
          action: keep
        - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_app_metrics_patn]
          action: replace
          target_label: __metrics_path__
          regex: (.+)
        - source_labels: [__meta_kubernetes_pod_ip, __meta_kubernetes_service_annotation_prometheus_io_app_metrics_port]
          action: replace
          target_label: __address__
          regex: (.+);(.+)

        - source_labels: [__meta_kubernetes_namespace]
          action: replace
          target_label: kubernetes_namespace
        - source_labels: [__meta_kubernetes_pod_name]
          action: replace
          target_label: kubernetes_pod_name

五,在对应的需要监控JVM的应用时,新增一个SERVICE。

apiVersion: v1
kind: Service
metadata:
  labels:
    name: xxx
  name: xxx
  namespace: default
  annotations:
        prometheus.io/scrape: "true"
        prometheus.io/jvm-scrape: "true"
        prometheus.io/jvm-port: "1234"
        prometheus.io/jvm-path: "/metrics"
spec:
  ports:
  - port: 1234
    targetPort: 1234
  selector:
    name: xxx

六,最后,导入grafana的插件文件。

{
"__inputs": [
    {
      "name": "DS_PROM",
      "label": "prom",
      "description": "A prometheus datasource with JMX Exporter scraping",
      "type": "datasource",
      "pluginId": "prometheus",
      "pluginName": "Prometheus"
    }
  ],
  "__requires": [
    {
      "type": "grafana",
      "id": "grafana",
      "name": "Grafana",
      "version": "4.5.0-pre1"
    },
    {
      "type": "panel",
      "id": "graph",
      "name": "Graph",
      "version": ""
    },
    {
      "type": "datasource",
      "id": "prometheus",
      "name": "Prometheus",
      "version": "1.0.0"
    }
  ],
  "annotations": {
    "list": []
  },
  "description": " tomcat JVM Dashboard for JVM metrics with Prometheus / JMX Exporter",
  "editable": true,
  "gnetId": 3066,
  "graphTooltip": 0,
  "id": 10,
  "iteration": 1544507474291,
  "links": [],
  "panels": [
    {
      "aliasColors": {},
      "bars": false,
      "dashLength": 10,
      "dashes": false,
      "datasource": "${DS_PROM}",
      "fill": 1,
      "gridPos": {
        "h": 7,
        "w": 24,
        "x": 0,
        "y": 0
      },
      "id": 2,
      "legend": {
        "alignAsTable": true,
        "avg": true,
        "current": true,
        "max": true,
        "min": true,
        "rightSide": true,
        "show": true,
        "total": false,
        "values": true
      },
      "lines": true,
      "linewidth": 1,
      "links": [],
      "nullPointMode": "null",
      "percentage": false,
      "pointradius": 5,
      "points": false,
      "renderer": "flot",
      "repeat": null,
      "seriesOverrides": [],
      "spaceLength": 10,
      "stack": false,
      "steppedLine": false,
      "targets": [
        {
          "expr": "jvm_memory_bytes_used{kubernetes_pod_name=~\"$kubernetes_pod_name\"}",
          "format": "time_series",
          "interval": "",
          "intervalFactor": 5,
          "legendFormat": "{{area}} memory [{{instance}}]",
          "metric": "jvm_memory_bytes_used",
          "refId": "A",
          "step": 5
        }
      ],
      "thresholds": [],
      "timeFrom": null,
      "timeShift": null,
      "title": "Memory used",
      "tooltip": {
        "shared": true,
        "sort": 0,
        "value_type": "individual"
      },
      "type": "graph",
      "xaxis": {
        "buckets": null,
        "mode": "time",
        "name": null,
        "show": true,
        "values": []
      },
      "yaxes": [
        {
          "format": "bytes",
          "label": null,
          "logBase": 1,
          "max": null,
          "min": null,
          "show": true
        },
        {
          "format": "short",
          "label": null,
          "logBase": 1,
          "max": null,
          "min": null,
          "show": true
        }
      ],
      "yaxis": {
        "align": false,
        "alignLevel": null
      }
    },
    {
      "aliasColors": {},
      "bars": false,
      "dashLength": 10,
      "dashes": false,
      "datasource": "${DS_PROM}",
      "fill": 1,
      "gridPos": {
        "h": 7,
        "w": 12,
        "x": 0,
        "y": 7
      },
      "id": 3,
      "legend": {
        "avg": false,
        "current": false,
        "max": false,
        "min": false,
        "show": true,
        "total": false,
        "values": false
      },
      "lines": true,
      "linewidth": 1,
      "links": [],
      "nullPointMode": "null",
      "percentage": false,
      "pointradius": 5,
      "points": false,
      "renderer": "flot",
      "seriesOverrides": [],
      "spaceLength": 10,
      "stack": false,
      "steppedLine": false,
      "targets": [
        {
          "expr": "jvm_threads_current{kubernetes_pod_name=~\"$kubernetes_pod_name\"}",
          "format": "time_series",
          "intervalFactor": 5,
          "legendFormat": "current [{{instance}}]",
          "metric": "jvm_threads_current",
          "refId": "A",
          "step": 10
        },
        {
          "expr": "jvm_threads_daemon{kubernetes_pod_name=~\"$kubernetes_pod_name\"}",
          "format": "time_series",
          "intervalFactor": 5,
          "legendFormat": "daemon [{{instance}}]",
          "metric": "jvm_threads_daemon",
          "refId": "B",
          "step": 10
        }
      ],
      "thresholds": [],
      "timeFrom": null,
      "timeShift": null,
      "title": "Threads used",
      "tooltip": {
        "shared": true,
        "sort": 0,
        "value_type": "individual"
      },
      "type": "graph",
      "xaxis": {
        "buckets": null,
        "mode": "time",
        "name": null,
        "show": true,
        "values": []
      },
      "yaxes": [
        {
          "decimals": 0,
          "format": "short",
          "label": null,
          "logBase": 1,
          "max": null,
          "min": null,
          "show": true
        },
        {
          "format": "short",
          "label": null,
          "logBase": 1,
          "max": null,
          "min": null,
          "show": true
        }
      ],
      "yaxis": {
        "align": false,
        "alignLevel": null
      }
    },
    {
      "aliasColors": {},
      "bars": false,
      "dashLength": 10,
      "dashes": false,
      "datasource": "${DS_PROM}",
      "fill": 1,
      "gridPos": {
        "h": 7,
        "w": 12,
        "x": 12,
        "y": 7
      },
      "id": 4,
      "legend": {
        "avg": false,
        "current": false,
        "max": false,
        "min": false,
        "show": true,
        "total": false,
        "values": false
      },
      "lines": true,
      "linewidth": 1,
      "links": [],
      "nullPointMode": "null",
      "percentage": false,
      "pointradius": 5,
      "points": false,
      "renderer": "flot",
      "seriesOverrides": [],
      "spaceLength": 10,
      "stack": false,
      "steppedLine": false,
      "targets": [
        {
          "expr": "jvm_classes_loaded{kubernetes_pod_name=~\"$kubernetes_pod_name\"}",
          "format": "time_series",
          "interval": "",
          "intervalFactor": 5,
          "legendFormat": "loaded [{{instance}}]",
          "metric": "jvm_classes_loaded",
          "refId": "A",
          "step": 10
        }
      ],
      "thresholds": [],
      "timeFrom": null,
      "timeShift": null,
      "title": "Class loading",
      "tooltip": {
        "shared": true,
        "sort": 0,
        "value_type": "individual"
      },
      "type": "graph",
      "xaxis": {
        "buckets": null,
        "mode": "time",
        "name": null,
        "show": true,
        "values": []
      },
      "yaxes": [
        {
          "decimals": 0,
          "format": "short",
          "label": "",
          "logBase": 1,
          "max": null,
          "min": null,
          "show": true
        },
        {
          "format": "short",
          "label": null,
          "logBase": 1,
          "max": null,
          "min": null,
          "show": true
        }
      ],
      "yaxis": {
        "align": false,
        "alignLevel": null
      }
    },
    {
      "aliasColors": {},
      "bars": false,
      "dashLength": 10,
      "dashes": false,
      "datasource": "${DS_PROM}",
      "fill": 1,
      "gridPos": {
        "h": 7,
        "w": 12,
        "x": 0,
        "y": 14
      },
      "id": 5,
      "legend": {
        "avg": false,
        "current": false,
        "max": false,
        "min": false,
        "show": true,
        "total": false,
        "values": false
      },
      "lines": true,
      "linewidth": 1,
      "links": [],
      "nullPointMode": "null",
      "percentage": false,
      "pointradius": 5,
      "points": false,
      "renderer": "flot",
      "seriesOverrides": [],
      "spaceLength": 10,
      "stack": false,
      "steppedLine": false,
      "targets": [
        {
          "expr": "rate(jvm_gc_collection_seconds_sum{kubernetes_pod_name=~\"$kubernetes_pod_name\"}[1m])",
          "format": "time_series",
          "interval": "",
          "intervalFactor": 5,
          "legendFormat": "{{gc}} [{{instance}}]",
          "metric": "jvm_gc_collection_seconds_sum",
          "refId": "A",
          "step": 10
        }
      ],
      "thresholds": [],
      "timeFrom": null,
      "timeShift": null,
      "title": "GC time / 1 min. rate",
      "tooltip": {
        "shared": true,
        "sort": 0,
        "value_type": "individual"
      },
      "type": "graph",
      "xaxis": {
        "buckets": null,
        "mode": "time",
        "name": null,
        "show": true,
        "values": []
      },
      "yaxes": [
        {
          "format": "s",
          "label": null,
          "logBase": 1,
          "max": null,
          "min": null,
          "show": true
        },
        {
          "format": "short",
          "label": null,
          "logBase": 1,
          "max": null,
          "min": null,
          "show": true
        }
      ],
      "yaxis": {
        "align": false,
        "alignLevel": null
      }
    },
    {
      "aliasColors": {},
      "bars": false,
      "dashLength": 10,
      "dashes": false,
      "datasource": "${DS_PROM}",
      "fill": 1,
      "gridPos": {
        "h": 7,
        "w": 12,
        "x": 12,
        "y": 14
      },
      "id": 6,
      "legend": {
        "avg": false,
        "current": false,
        "max": false,
        "min": false,
        "show": true,
        "total": false,
        "values": false
      },
      "lines": true,
      "linewidth": 1,
      "links": [],
      "nullPointMode": "null",
      "percentage": false,
      "pointradius": 5,
      "points": false,
      "renderer": "flot",
      "seriesOverrides": [],
      "spaceLength": 10,
      "stack": false,
      "steppedLine": false,
      "targets": [
        {
          "expr": "jvm_gc_collection_seconds_count{kubernetes_pod_name=~\"$kubernetes_pod_name\"}",
          "format": "time_series",
          "interval": "",
          "intervalFactor": 5,
          "legendFormat": "{{gc}} [{{instance}}]",
          "metric": "",
          "refId": "A",
          "step": 10
        }
      ],
      "thresholds": [],
      "timeFrom": null,
      "timeShift": null,
      "title": "GC count",
      "tooltip": {
        "shared": true,
        "sort": 0,
        "value_type": "individual"
      },
      "type": "graph",
      "xaxis": {
        "buckets": null,
        "mode": "time",
        "name": null,
        "show": true,
        "values": []
      },
      "yaxes": [
        {
          "decimals": 0,
          "format": "short",
          "label": null,
          "logBase": 1,
          "max": null,
          "min": null,
          "show": true
        },
        {
          "format": "short",
          "label": null,
          "logBase": 1,
          "max": null,
          "min": null,
          "show": true
        }
      ],
      "yaxis": {
        "align": false,
        "alignLevel": null
      }
    }
  ],
  "refresh": "30s",
  "schemaVersion": 16,
  "style": "dark",
  "tags": [
    "JVM",
    "prometheus",
    "jmx_exporter"
  ],
  "templating": {
    "list": [
      {
        "allValue": "",
        "current": {
          "tags": [],
          "text": "All",
          "value": [
            "$__all"
          ]
        },
        "datasource": "${DS_PROM}",
        "hide": 0,
        "includeAll": true,
        "label": "kubernetes_pod_name",
        "multi": true,
        "name": "kubernetes_pod_name",
        "options": [],
        "query": "label_values(jvm_memory_bytes_max,kubernetes_pod_name)",
        "refresh": 1,
        "regex": "",
        "skipUrlSync": false,
        "sort": 0,
        "tagValuesQuery": "",
        "tags": [],
        "tagsQuery": "",
        "type": "query",
        "useTags": false
      }
    ]
  },
  "time": {
    "from": "now-30m",
    "to": "now"
  },
  "timepicker": {
    "refresh_intervals": [
      "5s",
      "10s",
      "30s",
      "1m",
      "5m",
      "15m",
      "30m",
      "1h",
      "2h",
      "1d"
    ],
    "time_options": [
      "5m",
      "15m",
      "1h",
      "6h",
      "12h",
      "24h",
      "2d",
      "7d",
      "30d"
    ]
  },
  "timezone": "",
  "title": " tomcat JVM overview - Prometheus",
  "uid": "PVQFY0Pmz",
  "version": 2
}

在k8s集群中,利用prometheus的jmx_exporter进行tomcat的JVM性能监控,并用grafana作前端展示的更多相关文章

  1. k8s集群中部署prometheus server

    1.概述 本文档主要介绍如何在k8s集群中部署prometheus server用来作为监控的数据采集服务器,这样做可以很方便的对k8s集群中的指标.pod的.节点的指标进行采集和监控. 2.下载镜像 ...

  2. 在 Nebula K8s 集群中使用 nebula-spark-connector 和 nebula-algorithm

    本文首发于 Nebula Graph Community 公众号 解决思路 解决 K8s 部署 Nebula Graph 集群后连接不上集群问题最方便的方法是将 nebula-algorithm / ...

  3. 【K8S学习笔记】Part2:获取K8S集群中运行的所有容器镜像

    本文将介绍如何使用kubectl列举K8S集群中运行的Pod内的容器镜像. 注意:本文针对K8S的版本号为v1.9,其他版本可能会有少许不同. 0x00 准备工作 需要有一个K8S集群,并且配置好了k ...

  4. k8s 集群中的etcd故障解决

    一次在k8s集群中创建实例发现etcd集群状态出现连接失败状况,导致创建实例失败.于是排查了一下原因. 问题来源 下面是etcd集群健康状态: [root@docker01 ~]# cd /opt/k ...

  5. 将 master 节点服务器从 k8s 集群中移除并重新加入

    背景 1 台 master 加入集群后发现忘了修改主机名,而在 k8s 集群中修改节点主机名非常麻烦,不如将 master 退出集群改名并重新加入集群(前提是用的是高可用集群). 操作步骤 ssh 登 ...

  6. k8s集群中遇到etcd集群故障的排查思路

    一次在k8s集群中创建实例发现etcd集群状态出现连接失败状况,导致创建实例失败.于是排查了一下原因. 问题来源 下面是etcd集群健康状态: 1 2 3 4 5 6 7 8 9 10 11 [roo ...

  7. 实操教程丨如何在K8S集群中部署Traefik Ingress Controller

    注:本文使用的Traefik为1.x的版本 在生产环境中,我们常常需要控制来自互联网的外部进入集群中,而这恰巧是Ingress的职责. Ingress的主要目的是将HTTP和HTTPS从集群外部暴露给 ...

  8. 终于解决 k8s 集群中部署 nodelocaldns 的问题

    自从开始在 kubernetes 集群中部署 nodelocaldns 以提高 dns 解析性能以来,一直被一个问题困扰,只要一部署 nodelocaldns ,在 coredns 中添加的 rewr ...

  9. 在k8s集群中安装rook-ceph 1.8版本步骤

    官方文档地址:https://rook.io/docs/rook/v1.8/quickstart.html Kubernetes 最小版本号 Kubernetes 最小版本号:Kubernetes v ...

随机推荐

  1. POJ 1182 食物链 (带权并查集)

    食物链 Time Limit: 1000MS   Memory Limit: 10000K Total Submissions: 78551   Accepted: 23406 Description ...

  2. tomcat 启用NIO

    从Tomcat6.0以后, Java开发者很容易就可以是用NIO的技术来提升tomcat的并发处理能力. <Connector port="8080" protocol=&q ...

  3. 浅入 dancing links x(舞蹈链算法)

    abastract:利用dancing links 解决精确覆盖问题,例如数独,n皇后问题:以及重复覆盖问题. 要学习dacning links 算法,首先要先了解该算法适用的问题,精确覆盖问题和重复 ...

  4. java生成棋盘

    第一步:新建java项目,具体的命名,看下面的文件结构. 第二步:代码区 DrawChessBoard类: package com.hp.chenyanlong; import java.awt.Gr ...

  5. Kubernetes HPA

    简介 通过手工执行 kubectl scale 命令或者通过修改deployment的replicas数量,可以实现 Pod 扩容或缩容.但如果仅止于此,显然不符合 Google 对 Kubernet ...

  6. Go_22: Golang 命令行 test 应用

    1. 测试某一个包下的所有测试用例 cd /myGoProject/src/go-test/utils go test -v "-v" 参数 go test -v ... 表示无论 ...

  7. Swift学习笔记2

    1.函数参数都有一个外部参数名(external parameter name)和一个局部参数名(local parameter name).外部参数名用于在函数调用时标注传递给函数的参数,局部参数名 ...

  8. Centos 7和 Centos 6开放查看端口 防火墙关闭打开

    Centos 7 firewall 命令: 查看已经开放的端口: firewall-cmd --list-ports 开启端口 firewall-cmd --zone=public --add-por ...

  9. javascript构造函数模块

    var Person = (function(){ var Constr; Constr = function(){ this.name = 'carl'; } Constr.prototype = ...

  10. asp.net分页之AJAX 分页

    查询功能是开发中最重要的一个功能,大量数据的显示,我们用的最多的就是分页. 在ASP.NET 中有很多数据展现的控件,比如Repeater.GridView,用的最多的GridView,它同时也自带了 ...