Nova role may leave libvirt default network enabled

Bug #2042369 reported by Damian Dąbrowski
264
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack-Ansible
Fix Released
Undecided
Damian Dąbrowski

Bug Description

Short description
=================

Currently, nova playbook disables autostart for default libvirt network only if this network is currently active[1].
In some cases, this network may not be active during playbook execution, but it is still required to disable autostart to prevent accidentally enabling it in the future(host reboot, package upgrade etc.)

If all 3 requirements are met, then virtual machine will have broken networking:
- libvirt default network is active
- neutron is using iptables firewall driver
- VM's subnet is the same as libvirt default subnet(192.168.122.0/24)

Detailed description
====================

As mentioned above, it's super important to keep libvirt default network disabled, especially when iptables firewall driver is used in neutron.

Disabling libvirt default network was working fine in os_nova role if dnsmasq-base was available during nova playbook execution. In that case, libvirt default network was automatically started, hence nova was able to disable it[1].
It was a default behavior to install dnsmasq-base during nova playbook execution because it's a recommended package of libvirt-daemon-system and we were installing recommended packages by default.
We disabled installation of recommended packages in Yoga[2].
After this change, this is what happens during typical compute node deployment:

1. [os_nova_role] installs libvirt-daemon-system but not dnsmasq-base.
2. [libvirt] is not able to start default network because dnsmasq-base package is missing
3. [os_nova_role] does not disable autostart on libvirt default network because this network is not active(it's not visible on `virsh net-list` output)[1]
4. [os_neutron_role] installs dnsmasq-base[3]
5. User reboots the server
6. [libvirt] Default libvirt network is getting activated because it has autostart enabled and dnsmasq-base is available now
7. [libvirt] Creates masquerade iptables rules for 192.168.122.0/24

Steps to reproduce the issue
============================

(tested on ubuntu 22.04 and 20.04)
```
# apt install libvirt-daemon-system
# virsh net-list # should return empty list
# reboot
# virsh net-list # should return empty list
# apt install dnsmasq-base
# reboot
# virsh net-list # default virsh network is active
```

Security Risk
=============

If default libvirt network is active, neutron is using iptables firewall driver and VM's subnet is the same as libvirt default subnet(192.168.122.0/24), then masquerade is enabled for all traffic with source ip 192.168.122.0/24[4].
It opens potential security risk. For ex. it may allow a VM to communicate with openstack's mgmt network.

Proposed solution
=================

Always disable autostart for default libvirt network, even if network is not active at the moment.

Links
=====

[1] https://opendev.org/openstack/openstack-ansible-os_nova/src/commit/f372c88a093ce8a48cc3b7e0a26fd5ea06aec59a/tasks/drivers/kvm/nova_compute_kvm_virsh_net_remove.yml
[2] https://review.opendev.org/c/openstack/openstack-ansible-openstack_hosts/+/838855
[3] https://opendev.org/openstack/openstack-ansible-os_neutron/src/commit/ef4d3278556c6e34f79b195cfb3e0d300f7184f7/vars/debian.yml#L58
[4] https://paste.opendev.org/raw/bf798llzQNQ5iD552eEj/

Changed in openstack-ansible:
assignee: nobody → Damian Dąbrowski (damiandabrowski)
Changed in openstack-ansible:
status: New → In Progress
information type: Private Security → Public Security
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to openstack-ansible-os_nova (stable/2023.1)

Reviewed: https://review.opendev.org/c/openstack/openstack-ansible-os_nova/+/900190
Committed: https://opendev.org/openstack/openstack-ansible-os_nova/commit/5af16d68655a3ee8c2c32808311be4c480aa824c
Submitter: "Zuul (22348)"
Branch: stable/2023.1

commit 5af16d68655a3ee8c2c32808311be4c480aa824c
Author: Damian Dabrowski <email address hidden>
Date: Wed Nov 1 00:38:01 2023 +0100

    Always disable libvirt default network

    Currently, autostart for libvirt default network is disabled only when
    this network is active during nova playbook execution.
    It's an incorrect behavior because in some cases this network may not be
    active from the beginning.
    Autostart should be always disabled to ensure that this network will not
    be unexpectedly marked as active in the future(during package upgrade,
    host reboot etc.).

    Closes-Bug: #2042369
    Change-Id: I697234bda1601b534ce1b6ab186fa98f83179ee8
    (cherry picked from commit feb15af75b1d38fdfa2c670c55a6d0f4c95c5ad7)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to openstack-ansible-os_nova (stable/xena)

Reviewed: https://review.opendev.org/c/openstack/openstack-ansible-os_nova/+/900193
Committed: https://opendev.org/openstack/openstack-ansible-os_nova/commit/af5fc8ae964090ffc47dadc840837849d72ab71a
Submitter: "Zuul (22348)"
Branch: stable/xena

commit af5fc8ae964090ffc47dadc840837849d72ab71a
Author: Damian Dabrowski <email address hidden>
Date: Wed Nov 1 00:38:01 2023 +0100

    Always disable libvirt default network

    Currently, autostart for libvirt default network is disabled only when
    this network is active during nova playbook execution.
    It's an incorrect behavior because in some cases this network may not be
    active from the beginning.
    Autostart should be always disabled to ensure that this network will not
    be unexpectedly marked as active in the future(during package upgrade,
    host reboot etc.).

    Closes-Bug: #2042369
    Change-Id: I697234bda1601b534ce1b6ab186fa98f83179ee8
    (cherry picked from commit feb15af75b1d38fdfa2c670c55a6d0f4c95c5ad7)

tags: added: in-stable-xena
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to openstack-ansible-os_nova (stable/yoga)

Reviewed: https://review.opendev.org/c/openstack/openstack-ansible-os_nova/+/900191
Committed: https://opendev.org/openstack/openstack-ansible-os_nova/commit/9b933ec9aa74ab94aea2bcd453a6a8edd8b3acbe
Submitter: "Zuul (22348)"
Branch: stable/yoga

commit 9b933ec9aa74ab94aea2bcd453a6a8edd8b3acbe
Author: Damian Dabrowski <email address hidden>
Date: Wed Nov 1 00:38:01 2023 +0100

    Always disable libvirt default network

    Currently, autostart for libvirt default network is disabled only when
    this network is active during nova playbook execution.
    It's an incorrect behavior because in some cases this network may not be
    active from the beginning.
    Autostart should be always disabled to ensure that this network will not
    be unexpectedly marked as active in the future(during package upgrade,
    host reboot etc.).

    Closes-Bug: #2042369
    Change-Id: I697234bda1601b534ce1b6ab186fa98f83179ee8
    (cherry picked from commit feb15af75b1d38fdfa2c670c55a6d0f4c95c5ad7)

tags: added: in-stable-yoga
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to openstack-ansible-os_nova (stable/zed)

Reviewed: https://review.opendev.org/c/openstack/openstack-ansible-os_nova/+/900192
Committed: https://opendev.org/openstack/openstack-ansible-os_nova/commit/4750a1597850f762af9c9176aa3ed345690c9ff2
Submitter: "Zuul (22348)"
Branch: stable/zed

commit 4750a1597850f762af9c9176aa3ed345690c9ff2
Author: Damian Dabrowski <email address hidden>
Date: Wed Nov 1 00:38:01 2023 +0100

    Always disable libvirt default network

    Currently, autostart for libvirt default network is disabled only when
    this network is active during nova playbook execution.
    It's an incorrect behavior because in some cases this network may not be
    active from the beginning.
    Autostart should be always disabled to ensure that this network will not
    be unexpectedly marked as active in the future(during package upgrade,
    host reboot etc.).

    Closes-Bug: #2042369
    Change-Id: I697234bda1601b534ce1b6ab186fa98f83179ee8
    (cherry picked from commit feb15af75b1d38fdfa2c670c55a6d0f4c95c5ad7)

tags: added: in-stable-zed
Changed in openstack-ansible:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/openstack-ansible-os_nova yoga-eom

This issue was fixed in the openstack/openstack-ansible-os_nova yoga-eom release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/openstack-ansible-os_nova xena-eom

This issue was fixed in the openstack/openstack-ansible-os_nova xena-eom release.

To post a comment you must log in.
This report contains Public Security information  
Everyone can see this security related information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.