alert manager not working properly in prometheus

0 votes

I have some IoT devices connected to a server being monitored by Prometheus. I have it configured in a way that whenever a device shuts it sends an alert. But right now, whenever a device goes offline the alert manager is sending alert of all the devices which are offline and not a single device at a time.

ALERT IoT_online
  IF IoT_online == 0
  LABELS {severity="critical"}
  ANNOTATIONS {description="This device is offline  {{ $value }}.", summary="Instance {{ $labels.instance }} IoT device is offline"}

I just want it to send alerts of the only device which went offline at that particular time. Help Please...

Apr 10, 2018 in DevOps & Agile by Atul
• 10,230 points

1 answer to this question.

0 votes

This is the default functionality of the alert manager. Whenever an alert group sends an alert, it includes all alerts that are currently active.

What you can try is to separate alert group into smaller groups by labeling group_by. This may increase the amount of notifications that you'll get. You can also try and throttle how often these alerts are sent because of group changes using grpoup_interval.

answered Apr 10, 2018 by DareDev
• 6,890 points

Related Questions In DevOps & Agile

+1 vote
1 answer

How to add a node as a manager in swarm cluster?

Here is what you can try and ...READ MORE

answered Dec 10, 2018 in DevOps & Agile by Damon Salvatore
• 5,980 points
0 votes
2 answers

How to configure Prometheus server in linux?

Hi there , Setup Prometheus Configuration All the prometheus ...READ MORE

answered Jul 9, 2020 in DevOps & Agile by Suhana
• 340 points
0 votes
1 answer

How to add Node Exporter in Prometheus server?

Hi@akhtar, To add Node Exporter in Prometheus server, you have to ...READ MORE

answered May 28, 2020 in DevOps & Agile by MD
• 95,300 points
+15 votes
2 answers

Git management technique when there are multiple customers and need multiple customization?

Consider this - In 'extended' Git-Flow, (Git-Multi-Flow, ...READ MORE

answered Mar 27, 2018 in DevOps & Agile by DragonLord999
• 8,450 points
+2 votes
1 answer
0 votes
1 answer

High availability clustering (HA-C) in nexus repo manager oss

It's a paid feature available in Nexus ...READ MORE

answered Apr 26, 2018 in DevOps & Agile by DareDev
• 6,890 points
+1 vote
8 answers

How can I implement process.env in Angular 5 environment?

Users do not have access to process.env ...READ MORE

answered Apr 3, 2018 in DevOps & Agile by DareDev
• 6,890 points