title: 2.7.监控rabbitmq
order: 12
icon: lightbulb
一、环境
主机名 | IP地址 | 系统 | 说明 |
localhost | 192.168.11.61 | Ubuntu 20.04 | docker安装的prometheus |
test | 192.168.11.62 | Ubuntu 20.04 | rabbitmq版本3.7.15,Docker 版本 23.0.1 |
1、环境搭建
docker安装
略
docker-compose安装
略
rabbitmq
已在监控nginx那节课安装好
cd /data/rabbitmq
使用cat创建文件
cat >>docker-compose.yaml<<"EOF"
version: '3'
services:
rabbitmq:
image: rabbitmq:3.7.15-management
container_name: rabbitmq
restart: always
volumes:
- /data/rabbitmq/data:/var/lib/rabbitmq
- /data/rabbitmq/log:/var/log/rabbitmq
ports:
- 5672:5672
- 15672:15672
EOF
运行
docker-compose up -d
二、监控rabbitmq
1、二进制安装(二选一)
rabbit_exporter下载地址: https://github.com/kbudde/rabbitmq_exporter/releases
下载二进制包解压并放入/opt目录
wget https://github.com/kbudde/rabbitmq_exporter/releases/download/v1.0.0-RC19/rabbitmq_exporter_1.0.0-RC19_linux_amd64.tar.gz
mkdir /opt/prometheus/rabbitmq_exporter -p
tar xvf rabbitmq_exporter_1.0.0-RC19_linux_amd64.tar.gz -C /opt/prometheus/rabbitmq_exporter
创建用户
useradd -M -s /usr/sbin/nologin prometheus
更改exporter文件夹权限
chown prometheus:prometheus -R /opt/prometheus
创建 systemd 服务
使用cat创建
cat > /etc/systemd/system/rabbitmq_exporter.service <<"EOF"
[Unit]
Description=prometheus rabbitmq exporter
After=network.target
[Service]
Environment=RABBIT_USER=guest
Environment=RABBIT_PASSWORD=guest
Environment=RABBIT_URL=http://localhost:15672
OUTPUT_FORMAT=JSON
Type=simple
User=prometheus
Group=prometheus
Restart=always
ExecStart=/opt/prometheus/rabbitmq_exporter/rabbitmq_exporter
[Install]
WantedBy=multi-user.target
EOF
启动
systemctl daemon-reload
systemctl start rabbitmq_exporter
加入到开机自启动
systemctl enable rabbitmq_exporter
检查
systemctl status rabbitmq_exporter
启动不了检查日志
journalctl -u rabbitmq_exporter.service -f
2、docker安装(二选一)
docker直接运行
docker run -d --restart=always -p 9419:9419 --name rabbitmq_exporter -e RABBIT_URL=http://192.168.11.62:15672 -e RABBIT_USER=guest -e RABBIT_PASSWORD=guest kbudde/rabbitmq-exporter
docker-compose方式
cat >docker-compose.yaml <<EOF
version: '3.3'
services:
rabbitmq_exporter:
image: kbudde/rabbitmq-exporter
container_name: rabbitmq_exporter
restart: always
environment:
RABBIT_URL: "http://192.168.11.62:15672"
RABBIT_USER: "guest"
RABBIT_PASSWORD: "guest"
PUBLISH_PORT: "9419"
OUTPUT_FORMAT: "JSON"
ports:
- "9419:9419"
EOF
启动
docker-compose up -d
检查
docker ps
或:
docker logs -f rabbitmq_exporter
3、参数解释
Environment variable | default | description |
RABBIT_URL | rabbitMQ管理插件的url(必须以http(s)://开头) | |
RABBIT_USER | guest | rabbitMQ 管理插件的用户名。 |
RABBIT_PASSWORD | guest | rabbitMQ 管理插件的密码。 |
OUTPUT_FORMAT | JSON | 输出格式 |
PUBLISH_PORT | 9419 | 运行端口(监听端口) |
4、metrics地址
注:安装好Exporter后会暴露一个http://ip:端口/metrics
的HTTP服务
名称 | 地址 | 备注 |
rabbitmq_exporter |
5、Prometheus配置
配置prometheus去采集(拉取)rabbitmq_exporter的监控样本数据
cd /data/docker-prometheus
#在scrape_configs(搜刮配置):下面增加如下配置:
cat >> prometheus/prometheus.yml << "EOF"
- job_name: 'rabbitmq_exporter'
static_configs:
- targets: ['192.168.11.62:9419']
labels:
instance: test服务器
EOF
重新加载配置
curl -X POST http://localhost:9090/-/reload
检查
6、常用的监控指标
rabbitmq_queue_messages_unacknowledged_global 队列中有未确认的消息总数(未被消费的消息)
rabbitmq_node_disk_free_limit 使用磁盘大小
rabbitmq_node_disk_free 磁盘总大小
rabbitmq_node_mem_used 使用内存大小
rabbitmq_node_mem_limit 内存总大小
rabbitmq_sockets_used 使用sockets的数量
rabbitmq_sockets_available 可用的sockets总数
rabbitmq_fd_used 使用文件描述符的数量
rabbitmq_fd_available 可用的文件描述符总数
7、触发器配置(告警规则)
Prometheus配置
# 报警(触发器)配置
rule_files:
- "alert.yml"
- "rules/*.yml"
Rabbitmq触发器(告警规则)
也是单节点,未配置集群触发器
cat > prometheus/rules/rabbitmq.yml <<"EOF"
groups:
- name: Rabbitmq
rules:
- alert: RabbitMQDown
expr: rabbitmq_up != 1
labels:
severity: High
annotations:
summary: "Rabbitmq Down,实例:{{ $labels.instance }}"
description: "Rabbitmq_exporter连不上RabbitMQ! ! !"
- alert: RabbitMQ有未确认消息
expr: rabbitmq_queue_messages_unacknowledged_global > 0
for: 1m
labels:
severity: critical
annotations:
summary: "RabbitMQ有未确认消息,实例:{{ $labels.instance }}"
description: 'RabbitMQ未确认消息>0,当前值为:{{ $value }}'
- alert: RabbitMQ可用磁盘空间不足告警
expr: rabbitmq_node_disk_free_alarm != 0
#expr: rabbitmq_node_disk_free_limit / rabbitmq_node_disk_free *100 > 90
for: 0m
labels:
severity: critical
annotations:
summary: "RabbitMQ可用磁盘空间不足,实例:{{ $labels.instance }}"
description: "RabbitMQ可用磁盘空间不足,请检查"
- alert: RabbitMQ可用内存不足告警
expr: rabbitmq_node_mem_alarm != 0
#expr: rabbitmq_node_mem_used / rabbitmq_node_mem_limit * 100 > 90
for: 0m
labels:
severity: critical
annotations:
summary: "RabbitMQ可用内存不足,实例:{{ $labels.instance }}"
description: "RabbitMQ可用内存不足,请检查"
- alert: RabbitMQ_socket连接数使用过高告警
expr: rabbitmq_sockets_used / rabbitmq_sockets_available * 100 > 60
for: 0m
labels:
severity: critical
annotations:
summary: "RabbitMQ_socket连接数使用过高,实例:{{ $labels.instance }}"
description: 'RabbitMQ_sockets使用>60%,当前值为:{{ $value }}'
- alert: RabbitMQ文件描述符使用过高告警
expr: rabbitmq_fd_used / rabbitmq_fd_available * 100 > 60
for: 0m
labels:
severity: critical
annotations:
summary: "RabbitMQ文件描述符使用过高,实例:{{ $labels.instance }}"
description: 'RabbitMQ文件描述符使用>60%,当前值为:{{ $value }}'
EOF
检查:
vim prometheus/alert.yml
检查配置
docker exec -it prometheus promtool check config /etc/prometheus/prometheus.yml
重新加载配置
curl -X POST http://localhost:9090/-/reload
检查
http://192.168.11.61:9090/alerts?search=
或:
http://192.168.11.61:9090/rules
8、dashboard
grafana展示prometheus从rabbitmq_exporter收集到的的数据
id:4279
三、我的微信
如果碰到问题,可以随时加我微信,谢谢
评论区