instancemonitor fails if hypervisors are registered with their fqdn

Bug #1866638 reported by Liam Young on 2020-03-09
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Ubuntu Cloud Archive
Medium
Unassigned
Stein
Medium
Unassigned
Train
Medium
Unassigned
Ussuri
Medium
Unassigned
masakari-monitors
Undecided
Tushar Patil
masakari-monitors (Ubuntu)
Medium
Unassigned
Eoan
Medium
Unassigned
Focal
Medium
Unassigned

Bug Description

When instance-monitor sends a notification that an instance is down it uses the bare hostname which will result in a host not found error being returned by the masakari api service. The code that sends notifications of instances being down will always return the bare hostname ( https://github.com/openstack/masakari-monitors/blob/master/masakarimonitors/instancemonitor/libvirt_handler/eventfilter.py#L48 ).

Example of error:

2020-03-09 15:06:13.395 13338 ERROR masakarimonitors.ha.masakari Traceback (most recent call last):
2020-03-09 15:06:13.395 13338 ERROR masakarimonitors.ha.masakari File "/usr/lib/python3/dist-packages/masakarimonitors/ha/masakari.py", line 70, in send_notification
2020-03-09 15:06:13.395 13338 ERROR masakarimonitors.ha.masakari payload=event['notification']['payload'])
2020-03-09 15:06:13.395 13338 ERROR masakarimonitors.ha.masakari File "/usr/lib/python3/dist-packages/openstack/instance_ha/v1/_proxy.py", line 65, in create_notification
2020-03-09 15:06:13.395 13338 ERROR masakarimonitors.ha.masakari return self._create(_notification.Notification, **attrs)
2020-03-09 15:06:13.395 13338 ERROR masakarimonitors.ha.masakari File "/usr/lib/python3/dist-packages/openstack/proxy.py", line 224, in _create
2020-03-09 15:06:13.395 13338 ERROR masakarimonitors.ha.masakari return res.create(self, base_path=base_path)
2020-03-09 15:06:13.395 13338 ERROR masakarimonitors.ha.masakari File "/usr/lib/python3/dist-packages/openstack/resource.py", line 1116, in create
2020-03-09 15:06:13.395 13338 ERROR masakarimonitors.ha.masakari self._translate_response(response)
2020-03-09 15:06:13.395 13338 ERROR masakarimonitors.ha.masakari File "/usr/lib/python3/dist-packages/openstack/resource.py", line 962, in _translate_response
2020-03-09 15:06:13.395 13338 ERROR masakarimonitors.ha.masakari exceptions.raise_from_response(response, error_message=error_message)
2020-03-09 15:06:13.395 13338 ERROR masakarimonitors.ha.masakari File "/usr/lib/python3/dist-packages/openstack/exceptions.py", line 229, in raise_from_response
2020-03-09 15:06:13.395 13338 ERROR masakarimonitors.ha.masakari http_status=http_status, request_id=request_id
2020-03-09 15:06:13.395 13338 ERROR masakarimonitors.ha.masakari openstack.exceptions.BadRequestException: BadRequestException: 400: Client Error for url: https://172.20.0.101:15868/v1/a5ef30411ba9493aa7e713f941b6529a/notifications, Host with name juju-afe8c1-zaza-b4b90df200e5-17 could not be found.

Fix proposed to branch: master
Review: https://review.opendev.org/711954

Changed in masakari-monitors:
assignee: nobody → Liam Young (gnuoy)
status: New → In Progress
James Page (james-page) on 2020-03-24
Changed in masakari-monitors (Ubuntu Eoan):
status: New → Triaged
Changed in masakari-monitors (Ubuntu Focal):
status: New → Triaged
Changed in masakari-monitors (Ubuntu Eoan):
importance: Undecided → Medium
Changed in masakari-monitors (Ubuntu Focal):
importance: Undecided → Medium
Changed in masakari-monitors:
assignee: Liam Young (gnuoy) → Tushar Patil (tpatil)

Reviewed: https://review.opendev.org/711954
Committed: https://git.openstack.org/cgit/openstack/masakari-monitors/commit/?id=85dda1b0ebab75a8f9997a616cf1a759cad79558
Submitter: Zuul
Branch: master

commit 85dda1b0ebab75a8f9997a616cf1a759cad79558
Author: Liam Young <email address hidden>
Date: Mon Mar 9 17:10:59 2020 +0000

    Check config file for hostname

    When sending an alert from the instancemonitor check the monitors
    config file for the hostname before sending the alert.

    Change-Id: If11aa1abb1142941d6dcd00c46063d9015644978
    Closes-Bug: #1866638

Changed in masakari-monitors:
status: In Progress → Fix Released
Brian Murray (brian-murray) wrote :

The Eoan Ermine has reached end of life, so this bug will not be fixed for that release

Changed in masakari-monitors (Ubuntu Eoan):
status: Triaged → Won't Fix

Change abandoned by Radosław Piliszek (<email address hidden>) on branch: stable/train
Review: https://review.opendev.org/727980
Reason: see my last comment

Change abandoned by Radosław Piliszek (<email address hidden>) on branch: stable/stein
Review: https://review.opendev.org/727981
Reason: see my last comment

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers