Undercloud: pike -> queens upgrade break introspection

[Harald Jensås]

In https://review.openstack.org/523944 we switch the ironic inspector filter driver from iptables to dnsmasq.

The old iptables driver created a firewall chain, and will in most cases configure a REJECT rule[1] to block any introspection unless the operator start introspection of nodes.

On the upgraded undercloud we have these left-over rules still there:
  357 183K ironic-inspector udp -- br-ctlplane any anywhere anywhere udp dpt:bootps

Chain ironic-inspector (1 references) pkts bytes target prot opt in out source destination
357 183K REJECT all -- any any anywhere anywhere reject-with icmp-port-unreachable

[1] https://github.com/openstack/ironic-inspector/blob/master/ironic_inspector/pxe_filter/iptables.py#L186

[Harald Jensås (harald-jensas)]

Clean up:
iptables -D INPUT -i br-ctlplane -p udp --dport 67 -j ironic-inspector
iptables -F ironic-inspector
iptables -X ironic-inspector

We may also need to add a generic accept rule for DHCP request on the introspection interface.
iptables -I INPUT -i br-ctlplane -p udp --dport 67 -j ACCEPT

[Harald Jensås (harald-jensas)]

Hmm, looks like the iptables filter should have cleaned these up.

https://github.com/openstack/ironic-inspector/blob/master/ironic_inspector/wsgi_service.py#L163
  -> https://github.com/openstack/ironic-inspector/blob/master/ironic_inspector/pxe_filter/base.py#L164
  -> https://github.com/openstack/ironic-inspector/blob/master/ironic_inspector/pxe_filter/iptables.py#L61
  -> https://github.com/openstack/ironic-inspector/blob/master/ironic_inspector/pxe_filter/iptables.py#L168

[Harald Jensås (harald-jensas)]

I have opened a story for ironic-inspector, and proposed a fix there. The fix would have to be backported to stable/pike.

https://storyboard.openstack.org/#!/story/2001890

https://review.openstack.org/563335

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/563461

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/563467

[Harald Jensås (harald-jensas)]

Undercloud is persisting iptables rules in two places.

- First in puppet class tripleo::firewall
- Then in /usr/libexec/os-refresh-config/post-configure.d/80-seedstack-masquerade

In tripleo::firewall there is code to ensure neutron firewall rules are not persisted.
But ... the neutron rules are then persisted when 80-seedstack-masquerade is executed making the tripleo::firewall's attempts to filter these useless on the undercloud.

Since the Ironi Inspector rules are ephemeral as well, it would make sense to filter these as ell.

a) Implement filtering of ironic-inspector pxe_filter rules in tripleo::firewall:
   Patch: https://review.openstack.org/563461
b) Implement similar filtering in ``80-seedstack-masquerade`` to make sure neutron and ironic-inspector pxe_filter iptables rules are not persisted.
   Patch: https://review.openstack.org/563467

[OpenStack Infra (hudson-openstack)]

Change abandoned by Harald Jensås (<email address hidden>) on branch: stable/queens
Review: https://review.openstack.org/563581
Reason: Missing cherry-pick in commit message.
Will cherry pick again once the master commit is merged.

[OpenStack Infra (hudson-openstack)]

Change abandoned by Harald Jensås (<email address hidden>) on branch: stable/queens
Review: https://review.openstack.org/563580
Reason: Missing cherry-pick in commit message. Will cherry pick again once the master commit is merged.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/563461
Committed: https://git.openstack.org/cgit/openstack/puppet-tripleo/commit/?id=94ca328e5d309a5617043b2944e65df79271e0cd
Submitter: Zuul
Branch: master

commit 94ca328e5d309a5617043b2944e65df79271e0cd
Author: Harald Jensås <email address hidden>
Date: Sat Apr 21 13:26:44 2018 +0200

    Firewall: NOT persist ephemetal ironic-inspector rules

    When Ironic Inspector is configured to use the iptables
    pxe_filter driver it uses ephemeral firewall rules. This
    change ensures that these rules are not persisted.

    Ironic Inspector iptables filter driver does not set
    comments on it's rules. This patch use the string:
    '-m comment --comment' to ensure rules created by
    puppet-tripleo firewall is not accidentally removed from
    the persisted firewall rules.

    Change-Id: Ic90804db8dd74fce78169d9857a150d5ebf40cbb
    Related-Bug: #1765700

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/563467
Committed: https://git.openstack.org/cgit/openstack/instack-undercloud/commit/?id=e49688be9844b9ae32e14747ad95a07be0fa142c
Submitter: Zuul
Branch: master

commit e49688be9844b9ae32e14747ad95a07be0fa142c
Author: Harald Jensås <email address hidden>
Date: Sun Apr 22 13:12:01 2018 +0200

    Masqeurading: NOT persist ephemeral firewall rules

    Puppet class tripleo::firewall makes an effort to not
    persist ephemeral firewall rules created by neutron and
    ironic-inspector. In instack-undercloud the rules are
    persisted anyway because we run iptables-save when
    configuring masquerading.

    This changes the masquerading to also filter the rules,
    similar to what we do in tripleo::firewall.

    Additionally filtering of the Ironic Inspector iptables
    pxe_filter rules are implemented.

    Closes-Bug: #1765700
    Change-Id: I0cebfe4177981958c6e1a3b4b772f0a365f79e39

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/563581
Committed: https://git.openstack.org/cgit/openstack/instack-undercloud/commit/?id=77f557416a6a4fd912ef8ab6e5154ef784741ccf
Submitter: Zuul
Branch: stable/queens

commit 77f557416a6a4fd912ef8ab6e5154ef784741ccf
Author: Harald Jensås <email address hidden>
Date: Sun Apr 22 13:12:01 2018 +0200

    Masquerading, do not persist ephemeral firewall rules

    Puppet class tripleo::firewall makes an effort to not
    persist ephemeral firewall rules created by neutron and
    ironic-inspector. In instack-undercloud the rules are
    persisted anyway because we run iptables-save when
    configuring masquerading.

    This changes the masquerading to also filter the rules,
    similar to what we do in tripleo::firewall.

    Additionally filtering of the Ironic Inspector iptables
    pxe_filter rules are implemented.

    Closes-Bug: #1765700
    Change-Id: I0cebfe4177981958c6e1a3b4b772f0a365f79e39
    (cherry picked from commit e49688be9844b9ae32e14747ad95a07be0fa142c)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/563580
Committed: https://git.openstack.org/cgit/openstack/puppet-tripleo/commit/?id=2e66aa9bc2710d62fdacd068afa5ce58c648a0fe
Submitter: Zuul
Branch: stable/queens

commit 2e66aa9bc2710d62fdacd068afa5ce58c648a0fe
Author: Harald Jensås <email address hidden>
Date: Sat Apr 21 13:26:44 2018 +0200

    Firewall: NOT persist ephemetal ironic-inspector rules

    When Ironic Inspector is configured to use the iptables
    pxe_filter driver it uses ephemeral firewall rules. This
    change ensures that these rules are not persisted.

    Ironic Inspector iptables filter driver does not set
    comments on it's rules. This patch use the string:
    '-m comment --comment' to ensure rules created by
    puppet-tripleo firewall is not accidentally removed from
    the persisted firewall rules.

    Change-Id: Ic90804db8dd74fce78169d9857a150d5ebf40cbb
    Related-Bug: #1765700
    (cherry picked from commit 94ca328e5d309a5617043b2944e65df79271e0cd)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/instack-undercloud 8.4.2 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/instack-undercloud 9.1.0 release.