Hi all when I add a rule file in prometheus yml file the promethus service will be stopped I checked the log but couldnt figure it out

0 votes

This is the prometheus.yml file. Before I configure any rules, It was running perfectly


# my global config
global:
  scrape_interval: 15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
  evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
  # scrape_timeout is set to the global default (10s).
#  query_log_file: /var/log/prometheus/query.log

# Alertmanager configuration
alerting:
  alertmanagers:
    - static_configs:
        - targets:
          - 'localhost:9093'

# Load rules once and periodically evaluate them according to the global 'evaluation_interval'.
rule_files:
  - prometheus_rules.yml

This is the log

Aug  5 05:38:03 localhost prometheus: level=info ts=2021-08-05T09:38:03.562Z caller=main.go:842 msg="TSDB started"
Aug  5 05:38:03 localhost prometheus: level=info ts=2021-08-05T09:38:03.562Z caller=main.go:969 msg="Loading configuration file" filename=/usr/local/bin/prometheus/prometheus.yml
Aug  5 05:38:03 localhost prometheus: level=error ts=2021-08-05T09:38:03.571Z caller=manager.go:956 component="rule manager" msg="loading groups failed" err="/usr/local/bin/prometheus/prometheus_rules.yml: group \"instance_down_alert_rule\", rule 1, \"InstanceDown\": invalid annotation name: d:wqescription"
Aug  5 05:38:03 localhost prometheus: level=error ts=2021-08-05T09:38:03.571Z caller=main.go:995 msg="Failed to apply configuration" err="error loading rules, previous rule set restored"
Aug  5 05:38:03 localhost prometheus: level=info ts=2021-08-05T09:38:03.571Z caller=main.go:685 msg="Stopping scrape discovery manager..."
Aug  5 05:38:03 localhost prometheus: level=info ts=2021-08-05T09:38:03.571Z caller=main.go:699 msg="Stopping notify discovery manager..."
Aug  5 05:38:03 localhost prometheus: level=info ts=2021-08-05T09:38:03.571Z caller=main.go:721 msg="Stopping scrape manager..."
Aug  5 05:38:03 localhost prometheus: level=info ts=2021-08-05T09:38:03.571Z caller=main.go:695 msg="Notify discovery manager stopped"
Aug  5 05:38:03 localhost prometheus: level=info ts=2021-08-05T09:38:03.571Z caller=main.go:681 msg="Scrape discovery manager stopped"
Aug  5 05:38:03 localhost prometheus: level=info ts=2021-08-05T09:38:03.571Z caller=manager.go:934 component="rule manager" msg="Stopping rule manager..."
Aug  5 05:38:03 localhost prometheus: level=info ts=2021-08-05T09:38:03.571Z caller=manager.go:944 component="rule manager" msg="Rule manager stopped"
Aug  5 05:38:03 localhost prometheus: level=info ts=2021-08-05T09:38:03.572Z caller=notifier.go:601 component=notifier msg="Stopping notification manager..."
Aug  5 05:38:03 localhost prometheus: level=info ts=2021-08-05T09:38:03.573Z caller=main.go:896 msg="Notifier manager stopped"
Aug  5 05:38:03 localhost prometheus: level=info ts=2021-08-05T09:38:03.573Z caller=main.go:715 msg="Scrape manager stopped"
Aug  5 05:38:03 localhost prometheus: level=error ts=2021-08-05T09:38:03.573Z caller=main.go:905 err="error loading config from \"/usr/local/bin/prometheus/prometheus.yml\": one or more errors occurred while applying the new configuration (--config.file=\"/usr/local/bin/prometheus/prometheus.yml\")"
Aug  5 05:38:03 localhost systemd: prometheus.service: main process exited, code=exited, status=1/FAILURE
Aug  5 05:38:03 localhost systemd: Unit prometheus.service entered failed state.
Aug  5 05:38:03 localhost systemd: prometheus.service failed.

"I am getting this same error in Docker prometheus container too"

Aug 5, 2021 in DevOps Tools by Khopithan

edited Mar 4 15 views

No answer to this question. Be the first to respond.

Your answer

Your name to display (optional):
Privacy: Your email address will only be used for sending these notifications.
webinar REGISTER FOR FREE WEBINAR X
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP