Ironic inspector fails with permission denied dhcp-hostsdir

[Mark Goddard]

If you run with enable_ironic set to true and ironic_inspector_pxe_filter set to dnsmasq (the default in stein), ironic inspector can fail with the following in the logs:

2019-06-07 17:37:56.636 7 ERROR oslo_service.service Traceback (most recent call last):
2019-06-07 17:37:56.636 7 ERROR oslo_service.service File "/var/lib/kolla/venv/lib/python2.7/site-packages/oslo_service/service.py", line 796, in run_service
2019-06-07 17:37:56.636 7 ERROR oslo_service.service service.start()
2019-06-07 17:37:56.636 7 ERROR oslo_service.service File "/var/lib/kolla/venv/lib/python2.7/site-packages/ironic_inspector/common/rpc_service.py", line 40, in start
2019-06-07 17:37:56.636 7 ERROR oslo_service.service self.manager.init_host()
2019-06-07 17:37:56.636 7 ERROR oslo_service.service File "/var/lib/kolla/venv/lib/python2.7/site-packages/ironic_inspector/conductor/manager.py", line 76, in init_host
2019-06-07 17:37:56.636 7 ERROR oslo_service.service driver.init_filter()
2019-06-07 17:37:56.636 7 ERROR oslo_service.service File "/var/lib/kolla/venv/lib/python2.7/site-packages/ironic_inspector/pxe_filter/base.py", line 81, in inner
2019-06-07 17:37:56.636 7 ERROR oslo_service.service return method(self, *args, **kwargs)
2019-06-07 17:37:56.636 7 ERROR oslo_service.service File "/var/lib/kolla/venv/lib/python2.7/site-packages/ironic_inspector/pxe_filter/dnsmasq.py", line 144, in init_filter
2019-06-07 17:37:56.636 7 ERROR oslo_service.service self._sync(ironic)
2019-06-07 17:37:56.636 7 ERROR oslo_service.service File "/var/lib/kolla/venv/lib/python2.7/site-packages/ironic_inspector/pxe_filter/dnsmasq.py", line 103, in _sync
2019-06-07 17:37:56.636 7 ERROR oslo_service.service _blacklist_mac(mac)
2019-06-07 17:37:56.636 7 ERROR oslo_service.service File "/var/lib/kolla/venv/lib/python2.7/site-packages/ironic_inspector/pxe_filter/dnsmasq.py", line 287, in _blacklist_mac
2019-06-07 17:37:56.636 7 ERROR oslo_service.service if _exclusive_write_or_pass(path, '%s,ignore\n' % mac):
2019-06-07 17:37:56.636 7 ERROR oslo_service.service File "/var/lib/kolla/venv/lib/python2.7/site-packages/ironic_inspector/pxe_filter/dnsmasq.py", line 202, in _exclusive_write_or_pass
2019-06-07 17:37:56.636 7 ERROR oslo_service.service with open(path, 'w', 1) as f:
2019-06-07 17:37:56.636 7 ERROR oslo_service.service IOError: [Errno 13] Permission denied: u'/var/lib/ironic-inspector/dhcp-hostsdir/00:11:22:33:44:55'

A node must be registered with a port for this to happen.

Weirdly this happens on centos/source, ubuntu/source, and ubuntu/binary, but not centos/binary.

Workaround:
chown ironic-inspector:ironic-inspector /var/lib/ironic-inspector/dhcp-hostsdir/

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/664011

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/664011
Committed: https://git.openstack.org/cgit/openstack/kolla/commit/?id=41bcd7b4961fd52066b79804fdce9f81cc129735
Submitter: Zuul
Branch: master

commit 41bcd7b4961fd52066b79804fdce9f81cc129735
Author: Mark Goddard <email address hidden>
Date: Fri Jun 7 18:49:46 2019 +0100

    Fix ownership of ironic inspector DHCP hostsdir

    If you run with enable_ironic set to true and
    ironic_inspector_pxe_filter set to dnsmasq (the default in stein),
    ironic inspector can fail with the following in the logs:

    Permission denied: u'/var/lib/ironic-inspector/dhcp-hostsdir/<MAC address>'

    A node must be registered with a port for this to happen.

    Weirdly this happens on centos/source, ubuntu/source, and ubuntu/binary,
    but not centos/binary.

    This change changes the ownership of
    /var/lib/ironic-inspector/dhcp-hostsdir to ironic-inspector user to make
    it writeable.

    Change-Id: I19447727f19dbd9c0a3e17d218b48ddc4c253587
    Closes-Bug: #1832026

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/666077

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/666077
Committed: https://git.openstack.org/cgit/openstack/kolla/commit/?id=6acca35f5b3a7517881b5d494500e3b67152285d
Submitter: Zuul
Branch: stable/stein

commit 6acca35f5b3a7517881b5d494500e3b67152285d
Author: Mark Goddard <email address hidden>
Date: Fri Jun 7 18:49:46 2019 +0100

    Fix ownership of ironic inspector DHCP hostsdir

    If you run with enable_ironic set to true and
    ironic_inspector_pxe_filter set to dnsmasq (the default in stein),
    ironic inspector can fail with the following in the logs:

    Permission denied: u'/var/lib/ironic-inspector/dhcp-hostsdir/<MAC address>'

    A node must be registered with a port for this to happen.

    Weirdly this happens on centos/source, ubuntu/source, and ubuntu/binary,
    but not centos/binary.

    This change changes the ownership of
    /var/lib/ironic-inspector/dhcp-hostsdir to ironic-inspector user to make
    it writeable.

    Change-Id: I19447727f19dbd9c0a3e17d218b48ddc4c253587
    Closes-Bug: #1832026
    (cherry picked from commit 41bcd7b4961fd52066b79804fdce9f81cc129735)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/kolla 8.0.0.0rc2 release candidate.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/kolla 9.0.0.0rc1 release candidate.