1.prometheus安装
wget https://github.com/prometheus/prometheus/releases/download/v1.5.2/prometheus-1.5.2.linux-amd64.tar.gztar -zxvf prometheus-1.5.2.linux-amd64.tar.gz -C /opt/prometheus --strip-components=1cd /opt/prometheusmv prometheus.yml prometheus.yml-bak# vi prometheus.ymlglobal:scrape_interval: 10sevaluation_interval: 10sscrape_configs:- job_name: linuxstatic_configs:- targets: ['192.168.0.8:9100']labels:instance: db-0.8- job_name: mysqlstatic_configs:- targets: ['192.168.0.8:9104']labels:instance: db-0.8
启动 nohup /opt/prometheus/prometheus &
web界面 http://192.168.0.15:9090/graph
2.grafana安装
wget https://s3-us-west-2.amazonaws.com/grafana-releases/release/grafana-4.0.1-1480694114.x86_64.rpm yum localinstall grafana-4.0.1-1480694114.x86_64.rpmservice grafana-server start
至此安装完成。
浏览器打开 http://192.168.0.15:3000 ，输入默认用户名密码 (admin/admin) 可以进入 Grafana 。
然后配置数据源：
Prometheus： URL： http://192.168.0.15:9090/
即可完成 Prometheus 和 Grafana 的对接。
3.替换grafana的dashboards
Grafana 并没有太多的配置好的图表模板，除了 Percona 开源的一些外，很多需要自行配置。
下载dashboards(https://github.com/percona/grafana-dashboards)git clone https://github.com/percona/grafana-dashboards.gitcp -r grafana-dashboards/dashboards /var/lib/grafana/编辑 Grafana configvi /etc/grafana/grafana.ini[dashboards.json]enabled = truepath = /var/lib/grafana/dashboardssystemctl restart grafana-server
4.客户端安装
(1)mysql:在需要监控的mysql上安装 node_exporter和 mysqld_exporter
wget https://github.com/prometheus/node_exporter/releases/download/0.13.0/node_exporter-0.13.0.linux-amd64.tar.gzwget https://github.com/prometheus/mysqld_exporter/releases/download/v0.9.0/mysqld_exporter-0.9.0.linux-amd64.tar.gztar -zxvf node_exporter-0.13.0.linux-amd64.tar.gz -C /opt/prometheus_exporters --strip-components=1tar -zxvf mysqld_exporter-0.9.0.linux-amd64.tar.gz -C /opt/prometheus_exporters --strip-components=1运行node_exporter ：nohup /opt/prometheus_exporters/node_exporter & mysqld_exporter需要连接到Mysql，所以需要Mysql的权限，我们先为它创建用户并赋予所需的权限：mysql> GRANT REPLICATION CLIENT, PROCESS ON *.* TO 'prom'@'localhost' identified by 'amnt@#*IK<1qaz';mysql> GRANT SELECT ON performance_schema.* TO 'prom'@'localhost';mysql> flush privileges;创建.my.cnf文件并运行mysqld_exporter：$ cd /usr/local/services/prometheus_exporters$ cat << EOF > .my.cnf[client]user=prompassword=abc123EOF$ nohup /opt/prometheus_exporters/mysqld_exporter -config.my-cnf=".my.cnf" &
(2).redis 在redis服务器安装node_exporter和redis_exporter
wget https://github.com/oliver006/redis_exporter/releases/download/v0.10.8/redis_exporter-v0.10.8.linux-amd64.tar.gz
wget https://github.com/prometheus/node_exporter/releases/download/0.13.0/node_exporter-0.13.0.linux-amd64.tar.gz
tar -zxvf node_exporter-0.13.0.linux-amd64.tar.gz -C /opt/prometheus_exporters --strip-components=1
tar -zxvf redis_exporter-v0.10.8.linux-amd64.tar.gz -C /opt/prometheus_exporters --strip-components=1启动
nohup /opt/prometheus_exporters/node_exporter &
nohup /opt/prometheus_exporters/redis_exporter redis//192.168.0.17:6379 &
配置prometheus.yml 加入
- job_name: redis_exporter static_configs: - targets: ['192.168.0.17:9121']
下载grafana的redis的prometheus-redis_rev1.json模板
wget https://grafana.com/api/dashboards/763/revisions/1/download
在grafana中导入json模板
过一段时间就能看到图形了
(3).
kubernetes 模板配置
因为prometheus和kubernetes是结合的，所以导入模板后，直接配置prometheus.yml即可
模板下载：https://grafana.com/dashboards/315
prometheus.yml 加入以下配置- job_name: kubernetes-nodes-cadvisorstatic_configs:- targets: ['192.168.0.19:4194','192.168.0.21:4194']labels:instance: kubernetes-nodes-cadvisorkubernetes_sd_configs:- role: noderelabel_configs:- action: labelmapregex: __meta_kubernetes_node_label_(.+)metric_relabel_configs:- action: replacesource_labels: [id]regex: '^/machine\.slice/machine-rkt\\x2d([^\\]+)\\.+/([^/]+)\.service$'target_label: rkt_container_namereplacement: '${2}-${1}'- action: replacesource_labels: [id]regex: '^/system\.slice/(.+)\.service$'target_label: systemd_service_namereplacement: '${1}' 等待片刻可见图形：
最终prometheus配置：
# cat prometheus.ymlglobal: scrape_interval: 10s evaluation_interval: 10sscrape_configs: - job_name: node static_configs: - targets: ['192.168.0.8:9100','192.168.0.19:9100','192.168.0.21:9100','192.168.0.17:9100'] labels: instance: node - job_name: mysql static_configs: - targets: ['192.168.0.8:9104'] labels: instance: db-0.8 - job_name: redis_exporter static_configs: - targets: ['192.168.0.17:9121'] - job_name: kubernetes-nodes-cadvisor static_configs: - targets: ['192.168.0.19:4194','192.168.0.21:4194'] labels: instance: kubernetes-nodes-cadvisor kubernetes_sd_configs: - role: node relabel_configs: - action: labelmap regex: __meta_kubernetes_node_label_(.+) metric_relabel_configs: - action: replace source_labels: [id] regex: '^/machine\.slice/machine-rkt\\x2d([^\\]+)\\.+/([^/]+)\.service$' target_label: rkt_container_name replacement: '${2}-${1}' - action: replace source_labels: [id] regex: '^/system\.slice/(.+)\.service$' target_label: systemd_service_name replacement: '${1}'
参考文档:
https://segmentfault.com/a/1190000007040144
http://www.tuicool.com/articles/vEVjai
https://github.com/prometheus
dashboards模板下载:https://grafana.com/dashboards redis模板:https://github.com/oliver006/redis_exporter启动 nohup /opt/prometheus_exporters/redis_exporter redis//192.168.0.17:6379 &
Prometheus监控 - Alertmanager报警模块：http://blog.csdn.net/y_xiao_/article/details/50818451