Bug #1287194 “Docs recommend insecure configuration option” : Bugs : openstack-manuals

Thierry Carrez (ttx) on 2014-03-03

Changed in ossa:
status:	New → Incomplete

Revision history for this message

Thierry Carrez (ttx) wrote on 2014-03-03:

#1

This is not something that gets fixed by upgrading, so not OSSA territory. I would very much like to see an OSSN about this though.

I would also argue that making this public ASAP would help more than it hurts.

Changed in ossa:
status:	Incomplete → Won't Fix

Revision history for this message

Tom Fifield (fifieldt) wrote on 2014-03-03:

#2

does anyone subscribed to the bug have time to make a test system to verify instructions for one of the other configuration methods?

Revision history for this message

Anne Gentle (annegentle) wrote on 2014-03-03: Re: [Bug 1287194] Re: Docs recommend insecure configuration option

#3

Do you think that step-by-step instructions are required? I don't since
there are considerations based on environment that may cause people to
choose. I've submitted three patches that are not step-by-step.

On Mon, Mar 3, 2014 at 9:14 AM, Tom Fifield <email address hidden>wrote:

> does anyone subscribed to the bug have time to make a test system to
> verify instructions for one of the other configuration methods?
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1287194
>
> Title:
> Docs recommend insecure configuration option
>
> Status in OpenStack Manuals:
> New
> Status in OpenStack Security Advisories:
> Won't Fix
> Status in OpenStack Security Notes:
> New
>
> Bug description:
> From Daniel Berrange:
>
> ---------
> I just came across the following docs for configuring migration with
> nova+libvirt+kvm
>
>
> http://docs.openstack.org/trunk/config-reference/content//section_configuring-compute-migrations.html
>
> http://docs.openstack.org/grizzly/openstack-compute/admin/content//configuring-migrations.html
>
> At point 7 the docs say
>
> 7. Update the libvirt configurations. Modify the
> /etc/libvirt/libvirtd.conf file:
>
> before : #listen_tls = 0
> after : listen_tls = 0
> before : #listen_tcp = 1
> after : listen_tcp = 1
> add: auth_tcp = "none"
>
> Modify the /etc/init/libvirt-bin.conf file:
>
> before : exec /usr/sbin/libvirtd -d
> after : exec /usr/sbin/libvirtd -d -l
>
> What this does is tell the libvirt daemon listen for client TCP
> connections
> on all network interfaces, and accept any clients performing absolutely
> zero authentication.
>
> Sure this works for migration, but it also allows anyone of the network
> to be able to completely own all your compute hosts, by invoking whatever
> libvirt API calls they like.
>
> This is equivalent to telling someone to configure SSH to allow root
> logins with no passwords or keys at all. Actually it is worse because
> as well as killing authentication, it kills any encryption too.
>
> Libvirt has a choice of 4 secure options for remote access over TCP
>
> - SSH tunnel to libvirtd's UNIX socket
> - libvirtd TCP socket, with GSSAPI/Kerberos for auth+data encryption
> - libvirtd TCP socket, with TLS for encryption and x509 client
> certs for authentication
> - libvirtd TCP socket, with TLS for encryption and Kerberos for
> authentication
>
> documenting any of these setups would be better than what's there
> now, which needs to be removed asap.
>
> I don't know whether these docs are the current supported / preferred
> docs for this - they're just what I found via google. If we have other
> docs covering migration setup, they should be checked too.
> ----------------
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/openstack-manuals/+bug/1287194/+subscriptions
>

Do you think that step-by-step instructions are required? I don't since
there are considerations based on environment that may cause people to
choose. I've submitted three patches that are not step-by-step.

On Mon, Mar 3, 2014 at 9:14 AM, Tom Fifield <1287194@bugs.launchpad.net>wrote:

> does anyone subscribed to the bug have time to make a test system to
> verify instructions for one of the other configuration methods?
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1287194
>
> Title:
>   Docs recommend insecure configuration option
>
> Status in OpenStack Manuals:
>   New
> Status in OpenStack Security Advisories:
>   Won't Fix
> Status in OpenStack Security Notes:
>   New
>
> Bug description:
>   From Daniel Berrange:
>
>   ---------
>   I just came across the following docs for configuring migration with
>   nova+libvirt+kvm
>
>
> http://docs.openstack.org/trunk/config-reference/content//section_configuring-compute-migrations.html
>
> http://docs.openstack.org/grizzly/openstack-compute/admin/content//configuring-migrations.html
>
>   At point 7 the docs say
>
>    7. Update the libvirt configurations. Modify the
>   /etc/libvirt/libvirtd.conf file:
>
>     before : #listen_tls = 0
>     after : listen_tls = 0
>     before : #listen_tcp = 1
>     after : listen_tcp = 1
>     add: auth_tcp = "none"
>
>     Modify the /etc/init/libvirt-bin.conf file:
>
>     before : exec /usr/sbin/libvirtd -d
>     after : exec /usr/sbin/libvirtd -d -l
>
>   What this does is tell the libvirt daemon listen for client TCP
> connections
>   on all network interfaces, and accept any clients performing absolutely
>   zero authentication.
>
>   Sure this works for migration, but it also allows anyone of the network
>   to be able to completely own all your compute hosts, by invoking whatever
>   libvirt API calls they like.
>
>   This is equivalent to telling someone to configure SSH to allow root
>   logins with no passwords or keys at all. Actually it is worse because
>   as well as killing authentication, it kills any encryption too.
>
>   Libvirt has a choice of 4 secure options for remote access over TCP
>
>    - SSH tunnel to libvirtd's UNIX socket
>    - libvirtd TCP socket, with GSSAPI/Kerberos for auth+data encryption
>    - libvirtd TCP socket, with TLS for encryption and x509 client
>      certs for authentication
>    - libvirtd TCP socket, with TLS for encryption and Kerberos for
>      authentication
>
>   documenting any of these setups would be better than what's there
>   now, which needs to be removed asap.
>
>   I don't know whether these docs are the current supported / preferred
>   docs for this - they're just what I found via google. If we have other
>   docs covering migration setup, they should be checked too.
>   ----------------
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/openstack-manuals/+bug/1287194/+subscriptions
>

Revision history for this message

Nathan Kinder (nkinder) wrote on 2014-03-04:

#4

@fifieldt
I did set up Havana (RDO distribution) with two compute nodes configured for live migration. I configured libvirtd to use the SASL GSSAPI mechanism to perform Kerberos authentication over an normal TCP connection (no TLS). Performing a live migration with the "nova live-migration" command works fine with this approach. It's not ideal, as you need to ensure that root has a Kerberos ticket first, which requires one to run kinit to get a valid ticket. This could likely be partially automated by setting the KRB5CCNAME environment variable in /etc/sysconfig/libvirtdand using a keytab to call kinit from a script, but one would still have to deal with ticket expiration.

I haven't yet tried the X.509 approach, but I don't expect any big issues there. In addition to the libvirtd configuration changes to use TLS for remote connections and X.509 client certificate authentication, the "live_migration_uri" setting in /etc/nova/nova.conf will need to be changed as follows:

live_migration_uri=qemu+tls://%s/system

This change would also be needed if one were using GSSAPI/Kerberos over TLS.

One advantage with the X.509 approach is that certificates usually have a long lifetime. This makes expiration issues less of an issue since it's infrequent (usually every few years in most PKI deployments). One would simply be able to drop the client public and private key files onto each compute node (protected by the appropriate permissions), which would then be used for authentication between compute nodes during live migration. I will test this approach tomorrow, but I think this is enough to go on for putting together an OSSN.

Nathan Kinder (nkinder) on 2014-03-04

Changed in ossn:
assignee:	nobody → Nathan Kinder (nkinder)
importance:	Undecided → High
status:	New → In Progress

Revision history for this message

Nathan Kinder (nkinder) wrote on 2014-03-04:

#5

Download full text (7.3 KiB)

Here is the draft for an OSSN for this issue. Please review it and provide any feedback.

------------------------------------------------------------------------------------------

Live migration instructions recommend unsecured libvirt remote access
---

### Summary ###
When using the KVM hypervisor with libvirt on OpenStack Compute nodes, live
migration of instances from one Compute server to another requires that the
libvirt daemon is configured for remote network connectivity. The libvirt
daemon configuration recommended in the OpenStack Configuration Reference
manual configures libvirtd to listen for incoming TCP connections on all
network interfaces without requiring any authentication or using any
encryption. This insecure configuration allows for anyone with network access
to the libvirt daemon TCP port on OpenStack Compute nodes to control the
hypervisor through the libvirt API.

### Affected Services / Software ###
Nova, Compute, KVM, libvirt, Grizzly, Havana, Icehouse

### Discussion ###
The default configuration of the libvirt daemon is to not allow remote access.
Live migration of running instances between OpenStack Compute nodes requires
libvirt daemon remote access between OpenStack Compute nodes.

The libvirt daemon should not be configured to allow unauthenticated remote
access. The libvirt daemon has a choice of 4 secure options for remote access
over TCP. These options are:

- SSH tunnel to libvirtd's UNIX socket
- libvirtd TCP socket, with GSSAPI/Kerberos for auth+data encryption
- libvirtd TCP socket, with TLS for encryption and x.509 client certificates
for authentication
- libvirtd TCP socket, with TLS for encryption and Kerberos for authentication

It is not necessary for the libvirt daemon to listen for remote TCP connections
on all interfaces. Remote network connectivity to the libvirt daemon should be
restricted as much as possible. Remote access is only needed between the
OpenStack Compute nodes, so the libvirt daemon only needs to listen for remote
TCP connections on the interface that is used for this communication. A
firewall can be configured to lock down access to the TCP port that the libvirt
daemon listens on, but this does not sufficiently protect access to the libvirt
API. Other processes on a remote OpenStack Compute node might have network
access, but should not be authorized to remotely control the hypervisor on
another OpenStack Compute node.

### Recommended Actions ###
If you are using the KVM hypervisor with libvirt on OpenStack Compute nodes,
you should review your libvirt daemon configuration to ensure that it is not
allowing unauthenticated remote access.

Remote access to the libvirt daemon via TCP is configured by the "listen_tls",
"listen_tcp", and "auth_tcp" configuration directives. By default, these
directives are all commented out. This results in remote access via TCP being
disabled.

If you do not need remote libvirt daemon access, you should ensure that the
following configuration directives are set as follows in the
/etc/libvirt/libvirtd.conf configuration file. Commenting out these directives
will have the same effect, as these values match the internal defaults...

Here is the draft for an OSSN for this issue.  Please review it and provide any feedback.

------------------------------------------------------------------------------------------

Live migration instructions recommend unsecured libvirt remote access
---

### Summary ###
When using the KVM hypervisor with libvirt on OpenStack Compute nodes, live
migration of instances from one Compute server to another requires that the
libvirt daemon is configured for remote network connectivity.  The libvirt
daemon configuration recommended in the OpenStack Configuration Reference
manual configures libvirtd to listen for incoming TCP connections on all
network interfaces without requiring any authentication or using any
encryption.  This insecure configuration allows for anyone with network access
to the libvirt daemon TCP port on OpenStack Compute nodes to control the
hypervisor through the libvirt API.

### Affected Services / Software ###
Nova, Compute, KVM, libvirt, Grizzly, Havana, Icehouse

### Discussion ###
The default configuration of the libvirt daemon is to not allow remote access.
Live migration of running instances between OpenStack Compute nodes requires
libvirt daemon remote access between OpenStack Compute nodes.

The libvirt daemon should not be configured to allow unauthenticated remote
access.  The libvirt daemon  has a choice of 4 secure options for remote access
over TCP.  These options are:

- SSH tunnel to libvirtd's UNIX socket
 - libvirtd TCP socket, with GSSAPI/Kerberos for auth+data encryption
 - libvirtd TCP socket, with TLS for encryption and x.509 client certificates
   for authentication
 - libvirtd TCP socket, with TLS for encryption and Kerberos for authentication

It is not necessary for the libvirt daemon to listen for remote TCP connections
on all interfaces.  Remote network connectivity to the libvirt daemon should be
restricted as much as possible.   Remote access is only needed between the
OpenStack Compute nodes, so the libvirt daemon only needs to listen for remote
TCP connections on the interface that is used for this communication.  A
firewall can be configured to lock down access to the TCP port that the libvirt
daemon listens on, but this does not sufficiently protect access to the libvirt
API.   Other processes on a remote OpenStack Compute node might have network
access, but should not be authorized to remotely control the hypervisor on
another OpenStack Compute node.

### Recommended Actions ###
If you are using the KVM hypervisor with libvirt on OpenStack Compute nodes,
you should review your libvirt daemon configuration to ensure that it is not
allowing unauthenticated remote access.

Remote access to the libvirt daemon via TCP is configured by the "listen_tls",
"listen_tcp", and "auth_tcp" configuration directives.  By default, these
directives are all commented out.  This results in remote access via TCP being
disabled.

If you do not need remote libvirt daemon access, you should ensure that the
following configuration directives are set as follows in the
/etc/libvirt/libvirtd.conf configuration file.  Commenting out these directives
will have the same effect, as these values match the internal defaults:

---- begin example libvirtd.conf snippet ----
listen_tls = 1
listen_tcp = 0
auth_tcp = "sasl"
---- end example libvirtd.conf snippet ----

If you need to allow remote access to the libvirt daemon between OpenStack
Compute nodes for live migration, you should ensure that authentication is
required.  Additionally, you should consider enabling TLS to allow remote
connections to be encrypted.

The following libvirt daemon configuration directives will allow for
unencrypted remote connections that use SASL for authentication:

---- begin example libvirtd.conf snippet ----
listen_tls = 0
listen_tcp = 1
auth_tcp = "sasl"
---- end example libvirtd.conf snippet ----

If you want to require TLS encrypted remote connections, you will have to
obtain X.509 certificates and configure the libvirt daemon to use them to use
TLS.  Details on this configuration are in the libvirt daemon documentation.
Once the certificates are configured, you should set the following libvirt
daemon configuration directives:

---- begin example libvirtd.conf snippet ----
listen_tls = 1
listen_tcp = 0
auth_tls = "none"
---- end example libvirtd.conf snippet ----

When using TLS, setting the "auth_tls" configuration directive to "none" uses
X.509 client certificates for authentication.  You can additionally require
SASL authentication by setting the following libvirt daemon configuration
directives:

---- begin example libvirtd.conf snippet ----
listen_tls = 1
listen_tcp = 0
auth_tls = "sasl"
---- end example libvirtd.conf snippet ----

When using TLS, it is also necessary to configure the OpenStack Compute nodes
to use a non-default URI for live migration.  This is done by setting the
following configuration directive in /etc/nova/nova.conf:

---- begin example nova.conf snippet ----
live_migration_uri=qemu+tls://%s/system
---- end example nova.conf snippet ----

For more details on libvirt daemon remote URI formats, please see the following
libvirt daemon documentation:

http://libvirt.org/remote.html#Remote_URI_reference

For details on configuring SASL authentication and X.509 certificates for the
libvirt daemon, please consult the libvirt daemon documentation at the
following locations:

http://libvirt.org/remote.html
  http://libvirt.org/auth.html

When configuring the libvirt daemon for authentication, it is also important to
configure authorization to restrict remote access to your OpenStack Compute
nodes.  For example, if you don't configure authorization, any X.509 client
certificate that is signed by your issuing CA will be allowed access.  When
using SASL/GSSAPI for Kerberos authentication, any client with a valid TGT will
be granted access.  Lack of authorization can allow unintended remote access.
The libvirt daemon documentation should be consulted for details on configuring
authorization.

In addition to requiring authentication for remote access to the libvirt daemon
on your OpenStack Compute nodes, it is also recommended to restrict network
access such that connectivity is only allowed between the Compute nodes.

The first thing that should be done is to restrict the network interfaces that
the libvirt daemon listens on for remote connections.  By default, the libvirt
daemon listens on all interfaces when remote access is enabled.  This can be
restricted by setting the following configuration directive in
/etc/libvirt/libvirtd.conf:

---- begin example libvirtd.conf snippet ----
listen_addr = <IP address or hostname>
---- end example libvirtd.conf snippet ----

It is also recommended to configure the firewall on each OpenStack Compute node
to only allow other Compute nodes to access the ports that are used for remote
access to the libvirt daemon.  By default, this is port 16514 for TLS, 16509
for unencrypted TCP, and an ephemeral port range of 49152-49261.  Please
consult the documentation for your firewall software for instructions on
configuring the appropriate firewall rules.

### Contacts / References ###
This OSSN : https://bugs.launchpad.net/ossn/+bug/1287194
Original LaunchPad Bug : https://bugs.launchpad.net/openstack-manuals/+bug/1287194
OpenStack Security ML : openstack-security@lists.openstack.org
OpenStack Security Group : https://launchpad.net/~openstack-ossg

Revision history for this message

Nathan Kinder (nkinder) wrote on 2014-03-04:

#6

I've made some slight changes to the last section of the "Recommended Actions" section of the OSSN draft. Specifically, I corrected an error in the ephemeral migration port range. I also added details on how to look up what ports are configured for libvirt remote access and live migration in case one is not using the defaults. Here is the new content:

-----------------------------------------------------------------------------
It is also recommended to configure the firewall on each OpenStack Compute node
to only allow other Compute nodes to access the ports that are used for remote
access to the libvirt daemon. By default, this is port 16514 for TLS, 16509
for unencrypted TCP, and an ephemeral port range of 49152-49215. You can check
what ports you have configured for the libvirt daemon by looking at the
following configuration directives:

  tls_port (libvirtd.conf)
  tcp_port (libvirtd.conf)
  migration_port_min (qemu.conf)
  migration_port_max (qemu.conf)

Please consult the documentation for your firewall software for instructions on
configuring the appropriate firewall rules.
-----------------------------------------------------------------------------

Revision history for this message

Daniel Berrange (berrange) wrote on 2014-03-05:

#7

This looks like a pretty comprehensive and accurate writeup of the situation from libvirt POV

Revision history for this message

Nathan Kinder (nkinder) wrote on 2014-03-05:

#8

Download full text (4.1 KiB)

I added some additional content to the "Recommended Actions" section. The additional content covers configuring the list address for the epehmeral migration ports, the ability to tunnel migration traffic over the remote access port, and details about securing the live block migration traffic.

I believe this gives a more complete picture of how live migration can/should be secured. It also will allow this single OSSN to cover the issues described in bug 1240554 instead of having two separte (but similar) OSSNs.

The new/reworked sections are below. Feedback would be appreciated.

----------------------------------------------------------------------
In addition to requiring authentication for remote access to the libvirt daemon
on your OpenStack Compute nodes, it is also recommended to restrict network
access such that connectivity is only allowed between the Compute nodes.

The first thing that should be done is to restrict the network interfaces that
the libvirt daemon listens on for remote connections. By default, the libvirt
daemon listens on all interfaces when remote access is enabled. This can be
restricted by setting the following configuration directive in
/etc/libvirt/libvirtd.conf:

---- begin example libvirtd.conf snippet ----
listen_addr = <IP address or hostname>
---- end example libvirtd.conf snippet ----

Migration in the libvirt daemon also uses a range of ephemeral ports by
default. The connections to these ephemeral ports are not authenticated or
encrypted. It is possible to tunnel the migration traffic over the regular
libvirt daemon remote access port, which will use the authentication and
encryption settings that you have defined for that port. It is recommended
that you do this for the additional security that it provides. To enable
tunneling of the migration traffic, you must tell your OpenStack Compute
nodes to set the VIR_MIGRATE_TUNNELLED flag for live migration. This is done
by setting the following directive in /etc/nova/nova.conf:

---- begin example nova.conf snippet ----
live_migration_flag=VIR_MIGRATE_UNDEFINE_SOURCE, VIR_MIGRATE_PEER2PEER, VIR_MIGRATE_TUNNELLED
---- end example nova.conf snippet ----

The tunneling of migration traffic described above does not apply to live block
migration. Live block migration is currently only possible over ephemeral
ports.

If you choose to use the ephemeral migration ports, there are a few things that
you should be aware of. Unfortunately, there is no way to restrict the network
interfaces that these ephemeral ports will listen on in libvirt versions prior
to 1.1.4. If you are running version 1.1.4 or later of the libvirt daemon, you
can set the following directive in /etc/libvirt/qemu.conf to specify what
interfaces are used for the ephemeral migration ports:

---- begin example qemu.conf snippet ----
migration_address = <IP address>
---- end example qemu.conf snippet ----

It is also recommended to configure the firewall on each OpenStack Compute node
to only allow other Compute nodes to access the ports that are used for remote
access to the libvirt daemon. By default, this is port 16514 for TLS and 16509
for unencrypted TCP.

Additionally, migration over...

I added some additional content to the "Recommended Actions" section.  The additional content covers configuring the list address for the epehmeral migration ports, the ability to tunnel migration traffic over the remote access port, and details about securing the live block migration traffic.

I believe this gives a more complete picture of how live migration can/should be secured.  It also will allow this single OSSN to cover the issues described in bug 1240554 instead of having two separte (but similar) OSSNs.

The new/reworked sections are below.  Feedback would be appreciated.

----------------------------------------------------------------------
In addition to requiring authentication for remote access to the libvirt daemon
on your OpenStack Compute nodes, it is also recommended to restrict network
access such that connectivity is only allowed between the Compute nodes.

The first thing that should be done is to restrict the network interfaces that
the libvirt daemon listens on for remote connections.  By default, the libvirt
daemon listens on all interfaces when remote access is enabled.  This can be
restricted by setting the following configuration directive in
/etc/libvirt/libvirtd.conf:

---- begin example libvirtd.conf snippet ----
listen_addr = <IP address or hostname>
---- end example libvirtd.conf snippet ----

Migration in the libvirt daemon also uses a range of ephemeral ports by
default.  The connections to these ephemeral ports are not authenticated or
encrypted.  It is possible to tunnel the migration traffic over the regular
libvirt daemon remote access port, which will use the authentication and
encryption settings that you have defined for that port.  It is recommended
that you do this for the additional security that it provides.  To enable
tunneling of the migration traffic, you must tell your OpenStack Compute
nodes to set the VIR_MIGRATE_TUNNELLED flag for live migration.  This is done
by setting the following directive in /etc/nova/nova.conf:

---- begin example nova.conf snippet ----
live_migration_flag=VIR_MIGRATE_UNDEFINE_SOURCE, VIR_MIGRATE_PEER2PEER, VIR_MIGRATE_TUNNELLED
---- end example nova.conf snippet ----

The tunneling of migration traffic described above does not apply to live block
migration.  Live block migration is currently only possible over ephemeral
ports.

If you choose to use the ephemeral migration ports, there are a few things that
you should be aware of.  Unfortunately, there is no way to restrict the network
interfaces that these ephemeral ports will listen on in libvirt versions prior
to 1.1.4.  If you are running version 1.1.4 or later of the libvirt daemon, you
can set the following directive in /etc/libvirt/qemu.conf to specify what
interfaces are used for the ephemeral migration ports:

---- begin example qemu.conf snippet ----
migration_address = <IP address>
---- end example qemu.conf snippet ----

It is also recommended to configure the firewall on each OpenStack Compute node
to only allow other Compute nodes to access the ports that are used for remote
access to the libvirt daemon.  By default, this is port 16514 for TLS and 16509
for unencrypted TCP.

Additionally, migration over ephemeral ports uses a port range of 49152-49215.
You will need to allow your OpenStack Compute nodes to communicate with each
other over these ports if you choose not to tunnel the migration traffic over
the libvirt daemon remote access port.

You can check what ports you have configured for the libvirt daemon by looking
at the following configuration directives:

tls_port (libvirtd.conf)
  tcp_port (libvirtd.conf)
  migration_port_min (qemu.conf)
  migration_port_max (qemu.conf)

If you are running a version of the libvirt daemon older than 1.1.4 and you
want to perform live block migration, you will need to allow your OpenStack
Compute nodes to communicate over port range 5900-49151.  If you are running
version 1.1.4 or later of the libvirt daemon, the regular ephemeral migration
port range is used for live block migration.

Please consult the documentation for your firewall software for instructions on
configuring the appropriate firewall rules.

Revision history for this message

Anne Gentle (annegentle) wrote on 2014-03-05:

#9

Download full text (7.2 KiB)

I've read all the drafts and changes and I don't have any suggestions,
looks great!

Is the doc change sufficient, or should the doc also roll this much detail
into the migration instructions? We've merged all the changes that avoid
the too-open recommendation, but should we also not consider this bug
closed until the instructions are in the doc itself?

On Wed, Mar 5, 2014 at 2:43 PM, Nathan Kinder <email address hidden> wrote:

> I added some additional content to the "Recommended Actions" section.
> The additional content covers configuring the list address for the
> epehmeral migration ports, the ability to tunnel migration traffic over
> the remote access port, and details about securing the live block
> migration traffic.
>
> I believe this gives a more complete picture of how live migration
> can/should be secured. It also will allow this single OSSN to cover the
> issues described in bug 1240554 instead of having two separte (but
> similar) OSSNs.
>
> The new/reworked sections are below. Feedback would be appreciated.
>
> ----------------------------------------------------------------------
> In addition to requiring authentication for remote access to the libvirt
> daemon
> on your OpenStack Compute nodes, it is also recommended to restrict network
> access such that connectivity is only allowed between the Compute nodes.
>
> The first thing that should be done is to restrict the network interfaces
> that
> the libvirt daemon listens on for remote connections. By default, the
> libvirt
> daemon listens on all interfaces when remote access is enabled. This can
> be
> restricted by setting the following configuration directive in
> /etc/libvirt/libvirtd.conf:
>
> ---- begin example libvirtd.conf snippet ----
> listen_addr = <IP address or hostname>
> ---- end example libvirtd.conf snippet ----
>
> Migration in the libvirt daemon also uses a range of ephemeral ports by
> default. The connections to these ephemeral ports are not authenticated or
> encrypted. It is possible to tunnel the migration traffic over the regular
> libvirt daemon remote access port, which will use the authentication and
> encryption settings that you have defined for that port. It is recommended
> that you do this for the additional security that it provides. To enable
> tunneling of the migration traffic, you must tell your OpenStack Compute
> nodes to set the VIR_MIGRATE_TUNNELLED flag for live migration. This is
> done
> by setting the following directive in /etc/nova/nova.conf:
>
> ---- begin example nova.conf snippet ----
> live_migration_flag=VIR_MIGRATE_UNDEFINE_SOURCE, VIR_MIGRATE_PEER2PEER,
> VIR_MIGRATE_TUNNELLED
> ---- end example nova.conf snippet ----
>
> The tunneling of migration traffic described above does not apply to live
> block
> migration. Live block migration is currently only possible over ephemeral
> ports.
>
> If you choose to use the ephemeral migration ports, there are a few things
> that
> you should be aware of. Unfortunately, there is no way to restrict the
> network
> interfaces that these ephemeral ports will listen on in libvirt versions
> prior
> to 1.1.4. If you are running version 1.1.4 or later of the libvirt
> da...

I've read all the drafts and changes and I don't have any suggestions,
looks great!

Is the doc change sufficient, or should the doc also roll this much detail
into the migration instructions? We've merged all the changes that avoid
the too-open recommendation, but should we also not consider this bug
closed until the instructions are in the doc itself?

On Wed, Mar 5, 2014 at 2:43 PM, Nathan Kinder <nkinder@redhat.com> wrote:

> I added some additional content to the "Recommended Actions" section.
> The additional content covers configuring the list address for the
> epehmeral migration ports, the ability to tunnel migration traffic over
> the remote access port, and details about securing the live block
> migration traffic.
>
> I believe this gives a more complete picture of how live migration
> can/should be secured.  It also will allow this single OSSN to cover the
> issues described in bug 1240554 instead of having two separte (but
> similar) OSSNs.
>
> The new/reworked sections are below.  Feedback would be appreciated.
>
> ----------------------------------------------------------------------
> In addition to requiring authentication for remote access to the libvirt
> daemon
> on your OpenStack Compute nodes, it is also recommended to restrict network
> access such that connectivity is only allowed between the Compute nodes.
>
> The first thing that should be done is to restrict the network interfaces
> that
> the libvirt daemon listens on for remote connections.  By default, the
> libvirt
> daemon listens on all interfaces when remote access is enabled.  This can
> be
> restricted by setting the following configuration directive in
> /etc/libvirt/libvirtd.conf:
>
> ---- begin example libvirtd.conf snippet ----
> listen_addr = <IP address or hostname>
> ---- end example libvirtd.conf snippet ----
>
> Migration in the libvirt daemon also uses a range of ephemeral ports by
> default.  The connections to these ephemeral ports are not authenticated or
> encrypted.  It is possible to tunnel the migration traffic over the regular
> libvirt daemon remote access port, which will use the authentication and
> encryption settings that you have defined for that port.  It is recommended
> that you do this for the additional security that it provides.  To enable
> tunneling of the migration traffic, you must tell your OpenStack Compute
> nodes to set the VIR_MIGRATE_TUNNELLED flag for live migration.  This is
> done
> by setting the following directive in /etc/nova/nova.conf:
>
> ---- begin example nova.conf snippet ----
> live_migration_flag=VIR_MIGRATE_UNDEFINE_SOURCE, VIR_MIGRATE_PEER2PEER,
> VIR_MIGRATE_TUNNELLED
> ---- end example nova.conf snippet ----
>
> The tunneling of migration traffic described above does not apply to live
> block
> migration.  Live block migration is currently only possible over ephemeral
> ports.
>
> If you choose to use the ephemeral migration ports, there are a few things
> that
> you should be aware of.  Unfortunately, there is no way to restrict the
> network
> interfaces that these ephemeral ports will listen on in libvirt versions
> prior
> to 1.1.4.  If you are running version 1.1.4 or later of the libvirt
> daemon, you
> can set the following directive in /etc/libvirt/qemu.conf to specify what
> interfaces are used for the ephemeral migration ports:
>
> ---- begin example qemu.conf snippet ----
> migration_address = <IP address>
> ---- end example qemu.conf snippet ----
>
> It is also recommended to configure the firewall on each OpenStack Compute
> node
> to only allow other Compute nodes to access the ports that are used for
> remote
> access to the libvirt daemon.  By default, this is port 16514 for TLS and
> 16509
> for unencrypted TCP.
>
> Additionally, migration over ephemeral ports uses a port range of
> 49152-49215.
> You will need to allow your OpenStack Compute nodes to communicate with
> each
> other over these ports if you choose not to tunnel the migration traffic
> over
> the libvirt daemon remote access port.
>
> You can check what ports you have configured for the libvirt daemon by
> looking
> at the following configuration directives:
>
>   tls_port (libvirtd.conf)
>   tcp_port (libvirtd.conf)
>   migration_port_min (qemu.conf)
>   migration_port_max (qemu.conf)
>
> If you are running a version of the libvirt daemon older than 1.1.4 and you
> want to perform live block migration, you will need to allow your OpenStack
> Compute nodes to communicate over port range 5900-49151.  If you are
> running
> version 1.1.4 or later of the libvirt daemon, the regular ephemeral
> migration
> port range is used for live block migration.
>
> Please consult the documentation for your firewall software for
> instructions on
> configuring the appropriate firewall rules.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1287194
>
> Title:
>   Docs recommend insecure configuration option
>
> Status in OpenStack Manuals:
>   New
> Status in OpenStack Security Advisories:
>   Won't Fix
> Status in OpenStack Security Notes:
>   In Progress
>
> Bug description:
>   From Daniel Berrange:
>
>   ---------
>   I just came across the following docs for configuring migration with
>   nova+libvirt+kvm
>
>
> http://docs.openstack.org/trunk/config-reference/content//section_configuring-compute-migrations.html
>
> http://docs.openstack.org/grizzly/openstack-compute/admin/content//configuring-migrations.html
>
>   At point 7 the docs say
>
>    7. Update the libvirt configurations. Modify the
>   /etc/libvirt/libvirtd.conf file:
>
>     before : #listen_tls = 0
>     after : listen_tls = 0
>     before : #listen_tcp = 1
>     after : listen_tcp = 1
>     add: auth_tcp = "none"
>
>     Modify the /etc/init/libvirt-bin.conf file:
>
>     before : exec /usr/sbin/libvirtd -d
>     after : exec /usr/sbin/libvirtd -d -l
>
>   What this does is tell the libvirt daemon listen for client TCP
> connections
>   on all network interfaces, and accept any clients performing absolutely
>   zero authentication.
>
>   Sure this works for migration, but it also allows anyone of the network
>   to be able to completely own all your compute hosts, by invoking whatever
>   libvirt API calls they like.
>
>   This is equivalent to telling someone to configure SSH to allow root
>   logins with no passwords or keys at all. Actually it is worse because
>   as well as killing authentication, it kills any encryption too.
>
>   Libvirt has a choice of 4 secure options for remote access over TCP
>
>    - SSH tunnel to libvirtd's UNIX socket
>    - libvirtd TCP socket, with GSSAPI/Kerberos for auth+data encryption
>    - libvirtd TCP socket, with TLS for encryption and x509 client
>      certs for authentication
>    - libvirtd TCP socket, with TLS for encryption and Kerberos for
>      authentication
>
>   documenting any of these setups would be better than what's there
>   now, which needs to be removed asap.
>
>   I don't know whether these docs are the current supported / preferred
>   docs for this - they're just what I found via google. If we have other
>   docs covering migration setup, they should be checked too.
>   ----------------
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/openstack-manuals/+bug/1287194/+subscriptions
>

Revision history for this message

Nathan Kinder (nkinder) wrote on 2014-03-06:

#10

@annegentle
Thanks for you review! I think that the docs should be improved. I had planned to submit a patch for the docs to add a lot of the detail I've written up for the OSSN once this issue is made public.

You've fixed the major issue in the docs to prevent people from newly creating an insecure setup, so that's great for people who have not yet configured live migration. the next step is a OSSN to raise awareness for people who may have already followed the old instructions. These two items should deal with the "security issue" here.

The third step should be to improve the docs to show how you can configure things securely to help people who are setting up live migration instead of requiring them to go hunting for information. It's up to you if want to deal with that in this bug or another bug. The important thing is that this issue be made public and the OSSN is published first.

Revision history for this message

Robert Clark (robert-clark) wrote on 2014-03-06:

#11

Can we get the OSSN published as soon as possible please?

The anticipation with the security guide was that we'd inter-weave OSSNs throughout, we've done that in a few places but there are also significant gaps. I hope that we can look to include this as a standard part of the process as the new OSSN process rolls out in the future.

Revision history for this message

Nathan Kinder (nkinder) wrote on 2014-03-06:

#12

@robert-clark
The OSSN is ready to publish if nobody has any further comments about it. I was hoping to get a review of the content by another OSSG member. Does anyone have any last comments before I publish it and make this bug public?

Revision history for this message

Anne Gentle (annegentle) wrote on 2014-03-06:

#13

+1 to this Robert, thanks.

On Thu, Mar 6, 2014 at 9:56 AM, Robert Clark <email address hidden>wrote:

> Can we get the OSSN published as soon as possible please?
>
> The anticipation with the security guide was that we'd inter-weave OSSNs
> throughout, we've done that in a few places but there are also
> significant gaps. I hope that we can look to include this as a standard
> part of the process as the new OSSN process rolls out in the future.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1287194
>
> Title:
> Docs recommend insecure configuration option
>
> Status in OpenStack Manuals:
> New
> Status in OpenStack Security Advisories:
> Won't Fix
> Status in OpenStack Security Notes:
> In Progress
>
> Bug description:
> From Daniel Berrange:
>
> ---------
> I just came across the following docs for configuring migration with
> nova+libvirt+kvm
>
>
> http://docs.openstack.org/trunk/config-reference/content//section_configuring-compute-migrations.html
>
> http://docs.openstack.org/grizzly/openstack-compute/admin/content//configuring-migrations.html
>
> At point 7 the docs say
>
> 7. Update the libvirt configurations. Modify the
> /etc/libvirt/libvirtd.conf file:
>
> before : #listen_tls = 0
> after : listen_tls = 0
> before : #listen_tcp = 1
> after : listen_tcp = 1
> add: auth_tcp = "none"
>
> Modify the /etc/init/libvirt-bin.conf file:
>
> before : exec /usr/sbin/libvirtd -d
> after : exec /usr/sbin/libvirtd -d -l
>
> What this does is tell the libvirt daemon listen for client TCP
> connections
> on all network interfaces, and accept any clients performing absolutely
> zero authentication.
>
> Sure this works for migration, but it also allows anyone of the network
> to be able to completely own all your compute hosts, by invoking whatever
> libvirt API calls they like.
>
> This is equivalent to telling someone to configure SSH to allow root
> logins with no passwords or keys at all. Actually it is worse because
> as well as killing authentication, it kills any encryption too.
>
> Libvirt has a choice of 4 secure options for remote access over TCP
>
> - SSH tunnel to libvirtd's UNIX socket
> - libvirtd TCP socket, with GSSAPI/Kerberos for auth+data encryption
> - libvirtd TCP socket, with TLS for encryption and x509 client
> certs for authentication
> - libvirtd TCP socket, with TLS for encryption and Kerberos for
> authentication
>
> documenting any of these setups would be better than what's there
> now, which needs to be removed asap.
>
> I don't know whether these docs are the current supported / preferred
> docs for this - they're just what I found via google. If we have other
> docs covering migration setup, they should be checked too.
> ----------------
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/openstack-manuals/+bug/1287194/+subscriptions
>

+1 to this Robert, thanks.

On Thu, Mar 6, 2014 at 9:56 AM, Robert Clark <1287194@bugs.launchpad.net>wrote:

> Can we get the OSSN published as soon as possible please?
>
> The anticipation with the security guide was that we'd inter-weave OSSNs
> throughout, we've done that in a few places but there are also
> significant gaps. I hope that we can look to include this as a standard
> part of the process as the new OSSN process rolls out in the future.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1287194
>
> Title:
>   Docs recommend insecure configuration option
>
> Status in OpenStack Manuals:
>   New
> Status in OpenStack Security Advisories:
>   Won't Fix
> Status in OpenStack Security Notes:
>   In Progress
>
> Bug description:
>   From Daniel Berrange:
>
>   ---------
>   I just came across the following docs for configuring migration with
>   nova+libvirt+kvm
>
>
> http://docs.openstack.org/trunk/config-reference/content//section_configuring-compute-migrations.html
>
> http://docs.openstack.org/grizzly/openstack-compute/admin/content//configuring-migrations.html
>
>   At point 7 the docs say
>
>    7. Update the libvirt configurations. Modify the
>   /etc/libvirt/libvirtd.conf file:
>
>     before : #listen_tls = 0
>     after : listen_tls = 0
>     before : #listen_tcp = 1
>     after : listen_tcp = 1
>     add: auth_tcp = "none"
>
>     Modify the /etc/init/libvirt-bin.conf file:
>
>     before : exec /usr/sbin/libvirtd -d
>     after : exec /usr/sbin/libvirtd -d -l
>
>   What this does is tell the libvirt daemon listen for client TCP
> connections
>   on all network interfaces, and accept any clients performing absolutely
>   zero authentication.
>
>   Sure this works for migration, but it also allows anyone of the network
>   to be able to completely own all your compute hosts, by invoking whatever
>   libvirt API calls they like.
>
>   This is equivalent to telling someone to configure SSH to allow root
>   logins with no passwords or keys at all. Actually it is worse because
>   as well as killing authentication, it kills any encryption too.
>
>   Libvirt has a choice of 4 secure options for remote access over TCP
>
>    - SSH tunnel to libvirtd's UNIX socket
>    - libvirtd TCP socket, with GSSAPI/Kerberos for auth+data encryption
>    - libvirtd TCP socket, with TLS for encryption and x509 client
>      certs for authentication
>    - libvirtd TCP socket, with TLS for encryption and Kerberos for
>      authentication
>
>   documenting any of these setups would be better than what's there
>   now, which needs to be removed asap.
>
>   I don't know whether these docs are the current supported / preferred
>   docs for this - they're just what I found via google. If we have other
>   docs covering migration setup, they should be checked too.
>   ----------------
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/openstack-manuals/+bug/1287194/+subscriptions
>

Revision history for this message

Bryan D. Payne (bdpayne) wrote on 2014-03-06:

#14

I have reviewed the proposed OSSN and it looks good to me. Thanks for the nice writeup Nate!

Revision history for this message

Nathan Kinder (nkinder) wrote on 2014-03-06:

#15

Download full text (9.5 KiB)

Ok, I'm going to publish the following OSSN and make this bug public:

--------------------------------------------

Live migration instructions recommend unsecured libvirt remote access
---

### Summary ###
When using the KVM hypervisor with libvirt on OpenStack Compute nodes,
live migration of instances from one Compute server to another requires
that the libvirt daemon is configured for remote network connectivity.
The libvirt daemon configuration recommended in the OpenStack
Configuration Reference manual configures libvirtd to listen for
incoming TCP connections on all network interfaces without requiring any
authentication or using any encryption. This insecure configuration
allows for anyone with network access to the libvirt daemon TCP port on
OpenStack Compute nodes to control the hypervisor through the libvirt
API.

### Affected Services / Software ###
Nova, Compute, KVM, libvirt, Grizzly, Havana, Icehouse

### Discussion ###
The default configuration of the libvirt daemon is to not allow remote
access. Live migration of running instances between OpenStack Compute
nodes requires libvirt daemon remote access between OpenStack Compute
nodes.

The libvirt daemon should not be configured to allow unauthenticated
remote access. The libvirt daemon has a choice of 4 secure options for
remote access over TCP. These options are:

- SSH tunnel to libvirtd's UNIX socket
- libvirtd TCP socket, with GSSAPI/Kerberos for auth+data encryption
- libvirtd TCP socket, with TLS for encryption and x.509 client
certificates for authentication
- libvirtd TCP socket, with TLS for encryption and Kerberos for
authentication

It is not necessary for the libvirt daemon to listen for remote TCP
connections on all interfaces. Remote network connectivity to the
libvirt daemon should be restricted as much as possible. Remote
access is only needed between the OpenStack Compute nodes, so the
libvirt daemon only needs to listen for remote TCP connections on the
interface that is used for this communication. A firewall can be
configured to lock down access to the TCP port that the libvirt daemon
listens on, but this does not sufficiently protect access to the libvirt
API. Other processes on a remote OpenStack Compute node might have
network access, but should not be authorized to remotely control the
hypervisor on another OpenStack Compute node.

### Recommended Actions ###
If you are using the KVM hypervisor with libvirt on OpenStack Compute
nodes, you should review your libvirt daemon configuration to ensure
that it is not allowing unauthenticated remote access.

Remote access to the libvirt daemon via TCP is configured by the
"listen_tls", "listen_tcp", and "auth_tcp" configuration directives. By
default, these directives are all commented out. This results in remote
access via TCP being disabled.

If you do not need remote libvirt daemon access, you should ensure that
the following configuration directives are set as follows in the
/etc/libvirt/libvirtd.conf configuration file. Commenting out these
directives will have the same effect, as these values match the internal
defaults:

---- begin example libvirtd.conf snippet ----
listen_tls = 1
l...

Ok, I'm going to publish the following OSSN and make this bug public:

--------------------------------------------

Live migration instructions recommend unsecured libvirt remote access
---

### Summary ###
When using the KVM hypervisor with libvirt on OpenStack Compute nodes,
live migration of instances from one Compute server to another requires
that the libvirt daemon is configured for remote network connectivity.
The libvirt daemon configuration recommended in the OpenStack
Configuration Reference manual configures libvirtd to listen for
incoming TCP connections on all network interfaces without requiring any
authentication or using any encryption.  This insecure configuration
allows for anyone with network access to the libvirt daemon TCP port on
OpenStack Compute nodes to control the hypervisor through the libvirt
API.

### Affected Services / Software ###
Nova, Compute, KVM, libvirt, Grizzly, Havana, Icehouse

### Discussion ###
The default configuration of the libvirt daemon is to not allow remote
access.  Live migration of running instances between OpenStack Compute
nodes requires libvirt daemon remote access between OpenStack Compute
nodes.

The libvirt daemon should not be configured to allow unauthenticated
remote access.  The libvirt daemon  has a choice of 4 secure options for
remote access over TCP.  These options are:

- SSH tunnel to libvirtd's UNIX socket
 - libvirtd TCP socket, with GSSAPI/Kerberos for auth+data encryption
 - libvirtd TCP socket, with TLS for encryption and x.509 client
   certificates for authentication
 - libvirtd TCP socket, with TLS for encryption and Kerberos for
   authentication

It is not necessary for the libvirt daemon to listen for remote TCP
connections on all interfaces.  Remote network connectivity to the
libvirt daemon should be restricted as much as possible.  Remote
access is only needed between the OpenStack Compute nodes, so the
libvirt daemon only needs to listen for remote TCP connections on the
interface that is used for this communication.  A firewall can be
configured to lock down access to the TCP port that the libvirt daemon
listens on, but this does not sufficiently protect access to the libvirt
API.  Other processes on a remote OpenStack Compute node might have
network access, but should not be authorized to remotely control the
hypervisor on another OpenStack Compute node.

### Recommended Actions ###
If you are using the KVM hypervisor with libvirt on OpenStack Compute
nodes, you should review your libvirt daemon configuration to ensure
that it is not allowing unauthenticated remote access.

Remote access to the libvirt daemon via TCP is configured by the
"listen_tls", "listen_tcp", and "auth_tcp" configuration directives.  By
default, these directives are all commented out.  This results in remote
access via TCP being disabled.

If you do not need remote libvirt daemon access, you should ensure that
the following configuration directives are set as follows in the
/etc/libvirt/libvirtd.conf configuration file.  Commenting out these
directives will have the same effect, as these values match the internal
defaults:

---- begin example libvirtd.conf snippet ----
listen_tls = 1
listen_tcp = 0
auth_tcp = "sasl"
---- end example libvirtd.conf snippet ----

If you need to allow remote access to the libvirt daemon between
OpenStack Compute nodes for live migration, you should ensure that
authentication is required.  Additionally, you should consider enabling
TLS to allow remote connections to be encrypted.

The following libvirt daemon configuration directives will allow for
unencrypted remote connections that use SASL for authentication:

---- begin example libvirtd.conf snippet ----
listen_tls = 0
listen_tcp = 1
auth_tcp = "sasl"
---- end example libvirtd.conf snippet ----

If you want to require TLS encrypted remote connections, you will have
to obtain X.509 certificates and configure the libvirt daemon to use
them to use TLS.  Details on this configuration are in the libvirt
daemon documentation.  Once the certificates are configured, you should
set the following libvirt daemon configuration directives:

---- begin example libvirtd.conf snippet ----
listen_tls = 1
listen_tcp = 0
auth_tls = "none"
---- end example libvirtd.conf snippet ----

When using TLS, setting the "auth_tls" configuration directive to "none"
uses X.509 client certificates for authentication.  You can additionally
require SASL authentication by setting the following libvirt daemon
configuration directives:

---- begin example libvirtd.conf snippet ----
listen_tls = 1
listen_tcp = 0
auth_tls = "sasl"
---- end example libvirtd.conf snippet ----

When using TLS, it is also necessary to configure the OpenStack Compute
nodes to use a non-default URI for live migration.  This is done by
setting the following configuration directive in /etc/nova/nova.conf:

---- begin example nova.conf snippet ----
live_migration_uri=qemu+tls://%s/system
---- end example nova.conf snippet ----

For more details on libvirt daemon remote URI formats, please see the
following libvirt daemon documentation:

http://libvirt.org/remote.html#Remote_URI_reference

For details on configuring SASL authentication and X.509 certificates
for the libvirt daemon, please consult the libvirt daemon documentation
at the following locations:

http://libvirt.org/remote.html
  http://libvirt.org/auth.html

When configuring the libvirt daemon for authentication, it is also
important to configure authorization to restrict remote access to your
OpenStack Compute nodes.  For example, if you don't configure
authorization, any X.509 client certificate that is signed by your
issuing CA will be allowed access.  When using SASL/GSSAPI for Kerberos
authentication, any client with a valid TGT will be granted access.
Lack of authorization can allow unintended remote access.  The libvirt
daemon documentation should be consulted for details on configuring
authorization.

In addition to requiring authentication for remote access to the libvirt
daemon on your OpenStack Compute nodes, it is also recommended to
restrict network access such that connectivity is only allowed between
the Compute nodes.

The first thing that should be done is to restrict the network
interfaces that the libvirt daemon listens on for remote connections.
By default, the libvirt daemon listens on all interfaces when remote
access is enabled.  This can be restricted by setting the following
configuration directive in /etc/libvirt/libvirtd.conf:

---- begin example libvirtd.conf snippet ----
listen_addr = <IP address or hostname>
---- end example libvirtd.conf snippet ----

Migration in the libvirt daemon also uses a range of ephemeral ports by
default.  The connections to these ephemeral ports are not authenticated
or encrypted.  It is possible to tunnel the migration traffic over the
regular libvirt daemon remote access port, which will use the
authentication and encryption settings that you have defined for that
port.  It is recommended that you do this for the additional security
that it provides.  To enable tunneling of the migration traffic, you
must tell your OpenStack Compute nodes to set the VIR_MIGRATE_TUNNELLED
flag for live migration.  This is done by setting the following
directive in /etc/nova/nova.conf:

---- begin example nova.conf snippet ----
live_migration_flag=VIR_MIGRATE_UNDEFINE_SOURCE, VIR_MIGRATE_PEER2PEER, VIR_MIGRATE_TUNNELLED
---- end example nova.conf snippet ----

The tunneling of migration traffic described above does not apply to
live block migration.  Live block migration is currently only possible
over ephemeral ports.

If you choose to use the ephemeral migration ports, there are a few
things that you should be aware of.  Unfortunately, there is no way to
restrict the network interfaces that these ephemeral ports will listen
on in libvirt versions prior to 1.1.4.  If you are running version 1.1.4
or later of the libvirt daemon, you can set the following directive in
/etc/libvirt/qemu.conf to specify what interfaces are used for the
ephemeral migration ports:

---- begin example qemu.conf snippet ----
migration_address = <IP address>
---- end example qemu.conf snippet ----

It is also recommended to configure the firewall on each OpenStack
Compute node to only allow other Compute nodes to access the ports that
are used for remote access to the libvirt daemon.  By default, this is
port 16514 for TLS and 16509 for unencrypted TCP.

Additionally, migration over ephemeral ports uses a port range of
49152-49215.  You will need to allow your OpenStack Compute nodes to
communicate with each other over these ports if you choose not to tunnel
the migration traffic over the libvirt daemon remote access port.

You can check what ports you have configured for the libvirt daemon by
looking at the following configuration directives:

tls_port (libvirtd.conf)
  tcp_port (libvirtd.conf)
  migration_port_min (qemu.conf)
  migration_port_max (qemu.conf)

If you are running a version of the libvirt daemon older than 1.1.4 and
you want to perform live block migration, you will need to allow your
OpenStack Compute nodes to communicate over port range 5900-49151.  If
you are running version 1.1.4 or later of the libvirt daemon, the
regular ephemeral migration port range is used for live block migration.

Please consult the documentation for your firewall software for
instructions on configuring the appropriate firewall rules.

### Contacts / References ###
This OSSN: https://wiki.openstack.org/wiki/OSSN/OSSN-0007
Original LaunchPad Bug : https://bugs.launchpad.net/openstack-manuals/+bug/1287194
OpenStack Security ML : openstack-security@lists.openstack.org
OpenStack Security Group : https://launchpad.net/~openstack-ossg

information type:

Private Security → Public Security

Revision history for this message

Nathan Kinder (nkinder) wrote on 2014-03-06:

#16

The OSSN has been published to the following locations:

- Wiki (https://wiki.openstack.org/wiki/OSSN/OSSN-0007)
- <email address hidden>
- <email address hidden>

Changed in ossn:
status:	In Progress → Fix Released

Andreas Jaeger (jaegerandi) on 2014-03-07

Changed in openstack-manuals:
importance:	Undecided → Critical

Thierry Carrez (ttx) on 2014-03-19

information type:

Public Security → Public

Anne Gentle (annegentle) on 2014-03-21

Changed in openstack-manuals:
status:	New → Fix Released
assignee:	nobody → Anne Gentle (annegentle)

Anne Gentle (annegentle) on 2014-11-20

Changed in openstack-manuals:
status:	Fix Released → In Progress

Revision history for this message

Anne Gentle (annegentle) wrote on 2014-11-20:

#17

We moved the Configuration Reference away from /trunk/ back in January, but the files were not deleted. I have deleted the files on the FTP server so this page should no longer come up in search results as soon as the next Google index spider does its work.

Changed in openstack-manuals:
status:	In Progress → Fix Released

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2017-03-24: Fix included in openstack/openstack-manuals 15.0.0

#18

This issue was fixed in the openstack/openstack-manuals 15.0.0 release.

openstack-manuals

Docs recommend insecure configuration option

Bug Description

Other bug subscribers

Remote bug watches

	Status	Importance	Assigned to
OpenStack Security Advisory	Won't Fix	Undecided	Unassigned
OpenStack Security Notes	Fix Released	High	Nathan Kinder
openstack-manuals	Fix Released	Critical	Anne Gentle