查找了很多文档,没有完全达到我要求的,

于是,作了一定的调整,成现在这样。

操作步骤如下:

一,准备好两个文件。

jmx_prometheus_javaagent-0.3.1.jar

jmx_exporter.yml

jmx_exporter.yml作最简单配置如下:

---
rules:
- pattern: ".*"

二,tomcat的启动项,一直在bin下单独的setenv.sh里操作,新增如下启动选项,将jar包及yaml加入启动选项。

JAVA_OPTS="$JAVA_OPTS -javaagent:/xxx/bin/jmx_prometheus_javaagent-0.3.1.jar=1234:/xxx/bin/jmx_exporter.yml"

三,制作dockerfile时,要将这两个东东也CP到上面指定目录,并EXPOSE一个新的端口。

...
COPY --chown=a:a tomcat /xxx/tomcat
COPY --chown=a:a jdk /xxx/jdk
COPY --chown=a:a jmx_prometheus_javaagent-0.3.1.jar /xxx/tomcat/bin/
COPY --chown=a:a jmx_exporter.yml /xxx/tomcat/bin/

WORKDIR ${CATALINA_HOME}

USER a
EXPOSE 8080 1234

四,更新prometheus的配置文件configmap,新增一个job。

- job_name: 'tomcat-pods'
        tls_config:
          ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
        bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
        kubernetes_sd_configs:
        - role: endpoints
        relabel_configs:
        - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_scrape, __meta_kubernetes_service_annotation_prometheus_io_jvm_scrape]
          regex: true;true
          action: keep
        - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_app_metrics_patn]
          action: replace
          target_label: __metrics_path__
          regex: (.+)
        - source_labels: [__meta_kubernetes_pod_ip, __meta_kubernetes_service_annotation_prometheus_io_app_metrics_port]
          action: replace
          target_label: __address__
          regex: (.+);(.+)

        - source_labels: [__meta_kubernetes_namespace]
          action: replace
          target_label: kubernetes_namespace
        - source_labels: [__meta_kubernetes_pod_name]
          action: replace
          target_label: kubernetes_pod_name

五,在对应的需要监控JVM的应用时,新增一个SERVICE。

apiVersion: v1
kind: Service
metadata:
  labels:
    name: xxx
  name: xxx
  namespace: default
  annotations:
        prometheus.io/scrape: "true"
        prometheus.io/jvm-scrape: "true"
        prometheus.io/jvm-port: "1234"
        prometheus.io/jvm-path: "/metrics"
spec:
  ports:
  - port: 1234
    targetPort: 1234
  selector:
    name: xxx

六,最后,导入grafana的插件文件。

{
"__inputs": [
    {
      "name": "DS_PROM",
      "label": "prom",
      "description": "A prometheus datasource with JMX Exporter scraping",
      "type": "datasource",
      "pluginId": "prometheus",
      "pluginName": "Prometheus"
    }
  ],
  "__requires": [
    {
      "type": "grafana",
      "id": "grafana",
      "name": "Grafana",
      "version": "4.5.0-pre1"
    },
    {
      "type": "panel",
      "id": "graph",
      "name": "Graph",
      "version": ""
    },
    {
      "type": "datasource",
      "id": "prometheus",
      "name": "Prometheus",
      "version": "1.0.0"
    }
  ],
  "annotations": {
    "list": []
  },
  "description": " tomcat JVM Dashboard for JVM metrics with Prometheus / JMX Exporter",
  "editable": true,
  "gnetId": 3066,
  "graphTooltip": 0,
  "id": 10,
  "iteration": 1544507474291,
  "links": [],
  "panels": [
    {
      "aliasColors": {},
      "bars": false,
      "dashLength": 10,
      "dashes": false,
      "datasource": "${DS_PROM}",
      "fill": 1,
      "gridPos": {
        "h": 7,
        "w": 24,
        "x": 0,
        "y": 0
      },
      "id": 2,
      "legend": {
        "alignAsTable": true,
        "avg": true,
        "current": true,
        "max": true,
        "min": true,
        "rightSide": true,
        "show": true,
        "total": false,
        "values": true
      },
      "lines": true,
      "linewidth": 1,
      "links": [],
      "nullPointMode": "null",
      "percentage": false,
      "pointradius": 5,
      "points": false,
      "renderer": "flot",
      "repeat": null,
      "seriesOverrides": [],
      "spaceLength": 10,
      "stack": false,
      "steppedLine": false,
      "targets": [
        {
          "expr": "jvm_memory_bytes_used{kubernetes_pod_name=~\"$kubernetes_pod_name\"}",
          "format": "time_series",
          "interval": "",
          "intervalFactor": 5,
          "legendFormat": "{{area}} memory [{{instance}}]",
          "metric": "jvm_memory_bytes_used",
          "refId": "A",
          "step": 5
        }
      ],
      "thresholds": [],
      "timeFrom": null,
      "timeShift": null,
      "title": "Memory used",
      "tooltip": {
        "shared": true,
        "sort": 0,
        "value_type": "individual"
      },
      "type": "graph",
      "xaxis": {
        "buckets": null,
        "mode": "time",
        "name": null,
        "show": true,
        "values": []
      },
      "yaxes": [
        {
          "format": "bytes",
          "label": null,
          "logBase": 1,
          "max": null,
          "min": null,
          "show": true
        },
        {
          "format": "short",
          "label": null,
          "logBase": 1,
          "max": null,
          "min": null,
          "show": true
        }
      ],
      "yaxis": {
        "align": false,
        "alignLevel": null
      }
    },
    {
      "aliasColors": {},
      "bars": false,
      "dashLength": 10,
      "dashes": false,
      "datasource": "${DS_PROM}",
      "fill": 1,
      "gridPos": {
        "h": 7,
        "w": 12,
        "x": 0,
        "y": 7
      },
      "id": 3,
      "legend": {
        "avg": false,
        "current": false,
        "max": false,
        "min": false,
        "show": true,
        "total": false,
        "values": false
      },
      "lines": true,
      "linewidth": 1,
      "links": [],
      "nullPointMode": "null",
      "percentage": false,
      "pointradius": 5,
      "points": false,
      "renderer": "flot",
      "seriesOverrides": [],
      "spaceLength": 10,
      "stack": false,
      "steppedLine": false,
      "targets": [
        {
          "expr": "jvm_threads_current{kubernetes_pod_name=~\"$kubernetes_pod_name\"}",
          "format": "time_series",
          "intervalFactor": 5,
          "legendFormat": "current [{{instance}}]",
          "metric": "jvm_threads_current",
          "refId": "A",
          "step": 10
        },
        {
          "expr": "jvm_threads_daemon{kubernetes_pod_name=~\"$kubernetes_pod_name\"}",
          "format": "time_series",
          "intervalFactor": 5,
          "legendFormat": "daemon [{{instance}}]",
          "metric": "jvm_threads_daemon",
          "refId": "B",
          "step": 10
        }
      ],
      "thresholds": [],
      "timeFrom": null,
      "timeShift": null,
      "title": "Threads used",
      "tooltip": {
        "shared": true,
        "sort": 0,
        "value_type": "individual"
      },
      "type": "graph",
      "xaxis": {
        "buckets": null,
        "mode": "time",
        "name": null,
        "show": true,
        "values": []
      },
      "yaxes": [
        {
          "decimals": 0,
          "format": "short",
          "label": null,
          "logBase": 1,
          "max": null,
          "min": null,
          "show": true
        },
        {
          "format": "short",
          "label": null,
          "logBase": 1,
          "max": null,
          "min": null,
          "show": true
        }
      ],
      "yaxis": {
        "align": false,
        "alignLevel": null
      }
    },
    {
      "aliasColors": {},
      "bars": false,
      "dashLength": 10,
      "dashes": false,
      "datasource": "${DS_PROM}",
      "fill": 1,
      "gridPos": {
        "h": 7,
        "w": 12,
        "x": 12,
        "y": 7
      },
      "id": 4,
      "legend": {
        "avg": false,
        "current": false,
        "max": false,
        "min": false,
        "show": true,
        "total": false,
        "values": false
      },
      "lines": true,
      "linewidth": 1,
      "links": [],
      "nullPointMode": "null",
      "percentage": false,
      "pointradius": 5,
      "points": false,
      "renderer": "flot",
      "seriesOverrides": [],
      "spaceLength": 10,
      "stack": false,
      "steppedLine": false,
      "targets": [
        {
          "expr": "jvm_classes_loaded{kubernetes_pod_name=~\"$kubernetes_pod_name\"}",
          "format": "time_series",
          "interval": "",
          "intervalFactor": 5,
          "legendFormat": "loaded [{{instance}}]",
          "metric": "jvm_classes_loaded",
          "refId": "A",
          "step": 10
        }
      ],
      "thresholds": [],
      "timeFrom": null,
      "timeShift": null,
      "title": "Class loading",
      "tooltip": {
        "shared": true,
        "sort": 0,
        "value_type": "individual"
      },
      "type": "graph",
      "xaxis": {
        "buckets": null,
        "mode": "time",
        "name": null,
        "show": true,
        "values": []
      },
      "yaxes": [
        {
          "decimals": 0,
          "format": "short",
          "label": "",
          "logBase": 1,
          "max": null,
          "min": null,
          "show": true
        },
        {
          "format": "short",
          "label": null,
          "logBase": 1,
          "max": null,
          "min": null,
          "show": true
        }
      ],
      "yaxis": {
        "align": false,
        "alignLevel": null
      }
    },
    {
      "aliasColors": {},
      "bars": false,
      "dashLength": 10,
      "dashes": false,
      "datasource": "${DS_PROM}",
      "fill": 1,
      "gridPos": {
        "h": 7,
        "w": 12,
        "x": 0,
        "y": 14
      },
      "id": 5,
      "legend": {
        "avg": false,
        "current": false,
        "max": false,
        "min": false,
        "show": true,
        "total": false,
        "values": false
      },
      "lines": true,
      "linewidth": 1,
      "links": [],
      "nullPointMode": "null",
      "percentage": false,
      "pointradius": 5,
      "points": false,
      "renderer": "flot",
      "seriesOverrides": [],
      "spaceLength": 10,
      "stack": false,
      "steppedLine": false,
      "targets": [
        {
          "expr": "rate(jvm_gc_collection_seconds_sum{kubernetes_pod_name=~\"$kubernetes_pod_name\"}[1m])",
          "format": "time_series",
          "interval": "",
          "intervalFactor": 5,
          "legendFormat": "{{gc}} [{{instance}}]",
          "metric": "jvm_gc_collection_seconds_sum",
          "refId": "A",
          "step": 10
        }
      ],
      "thresholds": [],
      "timeFrom": null,
      "timeShift": null,
      "title": "GC time / 1 min. rate",
      "tooltip": {
        "shared": true,
        "sort": 0,
        "value_type": "individual"
      },
      "type": "graph",
      "xaxis": {
        "buckets": null,
        "mode": "time",
        "name": null,
        "show": true,
        "values": []
      },
      "yaxes": [
        {
          "format": "s",
          "label": null,
          "logBase": 1,
          "max": null,
          "min": null,
          "show": true
        },
        {
          "format": "short",
          "label": null,
          "logBase": 1,
          "max": null,
          "min": null,
          "show": true
        }
      ],
      "yaxis": {
        "align": false,
        "alignLevel": null
      }
    },
    {
      "aliasColors": {},
      "bars": false,
      "dashLength": 10,
      "dashes": false,
      "datasource": "${DS_PROM}",
      "fill": 1,
      "gridPos": {
        "h": 7,
        "w": 12,
        "x": 12,
        "y": 14
      },
      "id": 6,
      "legend": {
        "avg": false,
        "current": false,
        "max": false,
        "min": false,
        "show": true,
        "total": false,
        "values": false
      },
      "lines": true,
      "linewidth": 1,
      "links": [],
      "nullPointMode": "null",
      "percentage": false,
      "pointradius": 5,
      "points": false,
      "renderer": "flot",
      "seriesOverrides": [],
      "spaceLength": 10,
      "stack": false,
      "steppedLine": false,
      "targets": [
        {
          "expr": "jvm_gc_collection_seconds_count{kubernetes_pod_name=~\"$kubernetes_pod_name\"}",
          "format": "time_series",
          "interval": "",
          "intervalFactor": 5,
          "legendFormat": "{{gc}} [{{instance}}]",
          "metric": "",
          "refId": "A",
          "step": 10
        }
      ],
      "thresholds": [],
      "timeFrom": null,
      "timeShift": null,
      "title": "GC count",
      "tooltip": {
        "shared": true,
        "sort": 0,
        "value_type": "individual"
      },
      "type": "graph",
      "xaxis": {
        "buckets": null,
        "mode": "time",
        "name": null,
        "show": true,
        "values": []
      },
      "yaxes": [
        {
          "decimals": 0,
          "format": "short",
          "label": null,
          "logBase": 1,
          "max": null,
          "min": null,
          "show": true
        },
        {
          "format": "short",
          "label": null,
          "logBase": 1,
          "max": null,
          "min": null,
          "show": true
        }
      ],
      "yaxis": {
        "align": false,
        "alignLevel": null
      }
    }
  ],
  "refresh": "30s",
  "schemaVersion": 16,
  "style": "dark",
  "tags": [
    "JVM",
    "prometheus",
    "jmx_exporter"
  ],
  "templating": {
    "list": [
      {
        "allValue": "",
        "current": {
          "tags": [],
          "text": "All",
          "value": [
            "$__all"
          ]
        },
        "datasource": "${DS_PROM}",
        "hide": 0,
        "includeAll": true,
        "label": "kubernetes_pod_name",
        "multi": true,
        "name": "kubernetes_pod_name",
        "options": [],
        "query": "label_values(jvm_memory_bytes_max,kubernetes_pod_name)",
        "refresh": 1,
        "regex": "",
        "skipUrlSync": false,
        "sort": 0,
        "tagValuesQuery": "",
        "tags": [],
        "tagsQuery": "",
        "type": "query",
        "useTags": false
      }
    ]
  },
  "time": {
    "from": "now-30m",
    "to": "now"
  },
  "timepicker": {
    "refresh_intervals": [
      "5s",
      "10s",
      "30s",
      "1m",
      "5m",
      "15m",
      "30m",
      "1h",
      "2h",
      "1d"
    ],
    "time_options": [
      "5m",
      "15m",
      "1h",
      "6h",
      "12h",
      "24h",
      "2d",
      "7d",
      "30d"
    ]
  },
  "timezone": "",
  "title": " tomcat JVM overview - Prometheus",
  "uid": "PVQFY0Pmz",
  "version": 2
}

在k8s集群中,利用prometheus的jmx_exporter进行tomcat的JVM性能监控,并用grafana作前端展示的更多相关文章

  1. k8s集群中部署prometheus server

    1.概述 本文档主要介绍如何在k8s集群中部署prometheus server用来作为监控的数据采集服务器,这样做可以很方便的对k8s集群中的指标.pod的.节点的指标进行采集和监控. 2.下载镜像 ...

  2. 在 Nebula K8s 集群中使用 nebula-spark-connector 和 nebula-algorithm

    本文首发于 Nebula Graph Community 公众号 解决思路 解决 K8s 部署 Nebula Graph 集群后连接不上集群问题最方便的方法是将 nebula-algorithm / ...

  3. 【K8S学习笔记】Part2:获取K8S集群中运行的所有容器镜像

    本文将介绍如何使用kubectl列举K8S集群中运行的Pod内的容器镜像. 注意:本文针对K8S的版本号为v1.9,其他版本可能会有少许不同. 0x00 准备工作 需要有一个K8S集群,并且配置好了k ...

  4. k8s 集群中的etcd故障解决

    一次在k8s集群中创建实例发现etcd集群状态出现连接失败状况,导致创建实例失败.于是排查了一下原因. 问题来源 下面是etcd集群健康状态: [root@docker01 ~]# cd /opt/k ...

  5. 将 master 节点服务器从 k8s 集群中移除并重新加入

    背景 1 台 master 加入集群后发现忘了修改主机名,而在 k8s 集群中修改节点主机名非常麻烦,不如将 master 退出集群改名并重新加入集群(前提是用的是高可用集群). 操作步骤 ssh 登 ...

  6. k8s集群中遇到etcd集群故障的排查思路

    一次在k8s集群中创建实例发现etcd集群状态出现连接失败状况,导致创建实例失败.于是排查了一下原因. 问题来源 下面是etcd集群健康状态: 1 2 3 4 5 6 7 8 9 10 11 [roo ...

  7. 实操教程丨如何在K8S集群中部署Traefik Ingress Controller

    注:本文使用的Traefik为1.x的版本 在生产环境中,我们常常需要控制来自互联网的外部进入集群中,而这恰巧是Ingress的职责. Ingress的主要目的是将HTTP和HTTPS从集群外部暴露给 ...

  8. 终于解决 k8s 集群中部署 nodelocaldns 的问题

    自从开始在 kubernetes 集群中部署 nodelocaldns 以提高 dns 解析性能以来,一直被一个问题困扰,只要一部署 nodelocaldns ,在 coredns 中添加的 rewr ...

  9. 在k8s集群中安装rook-ceph 1.8版本步骤

    官方文档地址:https://rook.io/docs/rook/v1.8/quickstart.html Kubernetes 最小版本号 Kubernetes 最小版本号:Kubernetes v ...

随机推荐

  1. BZOJ2721 Violet5樱花(数论)

    有(x+y)n!=xy.套路地提出x和y的gcd,设为d,令ad=x,bd=y.则有(a+b)n!=abd.此时d已是和a.b无关的量.由a与b互质,得a+b与ab互质,于是将a+b除过来得n!=ab ...

  2. Simple Cycles Edges CodeForces - 962F(点双连通分量)

    题意: 求出简单环的所有边,简单环即为边在一个环内 解析: 求出点双连通分量,如果一个连通分量的点数和边数相等,则为一个简单环 点双连通分量  任意两个点都至少存在两条点不重复的路径  即任意两条边都 ...

  3. Valid BFS? CodeForces - 1037D(思维 bfs)

    我真是一只菜狗......emm... 题意: 判断一个从1开始的队列是否可以按照bfs的顺序 进行遍历..必须从1开始...然后后边依次是bfs顺序 解析: 看代码能看懂吧...emm...就是把每 ...

  4. 【BZOJ1449】[JSOI2009]球队收益(网络流,费用流)

    [BZOJ1449][JSOI2009]球队收益(网络流,费用流) 题面 BZOJ 洛谷 题解 首先对于一支队伍而言,总共进行多少场比赛显然是已知的,假设是\(n_i\)场,那么它的贡献是:\(C_i ...

  5. Mask RCNN 原理

    转自:https://blog.csdn.net/ghw15221836342/article/details/80084861 https://blog.csdn.net/ghw1522183634 ...

  6. Java基础-SSM之mybatis一对多和多对一关系映射

    Java基础-SSM之mybatis一对多和多对一关系映射 作者:尹正杰 版权声明:原创作品,谢绝转载!否则将追究法律责任. 一.准备测试环境(创建数据库表)  1>.创建customers表: ...

  7. Hadoop基础-MapReduce的Combiner用法案例

    Hadoop基础-MapReduce的Combiner用法案例 作者:尹正杰 版权声明:原创作品,谢绝转载!否则将追究法律责任. 一.编写年度最高气温统计 如上图说所示:有一个temp的文件,里面存放 ...

  8. bzoj千题计划292:bzoj2244: [SDOI2011]拦截导弹

    http://www.lydsy.com/JudgeOnline/problem.php?id=2244 每枚导弹成功拦截的概率 = 包含它的最长上升子序列个数/最长上升子序列总个数 pre_len ...

  9. spark RDD 常见操作

    fold 操作 区别 与 co 1.mapValus 2.flatMapValues 3.comineByKey 4.foldByKey 5.reduceByKey 6.groupByKey 7.so ...

  10. 《Two Dozen Short Lessons in Haskell》(二十四)代数类型

    这是<Two Dozen Short Lessons in Haskell>这本书的最后一章,第23章没有习题. 这一章里介绍了Haskell如果自定义一种类型,并且用一个双人博弈游戏为例 ...