Libvirt KVM: Huge pages need to be mapped shared to allow vhostuser access

[Daniel Berrange]

By default, when a KVM guest is setup using huge pages, the pages are mapped using the MAP_PRIVATE flag to mmap().

The vhostuser VIF backend is designed to allow an external process to provide the QEMU network driver functionality. For some usecases of vhostuser, this requires that the external process be able to access the QEMU guest's memory pages directly. This is not possible when the hugepages are mapped with MAP_PRIVATE - they must use MAP_SHARED instead.

The result is that current Nova hugepages config doesn't work too well with vhostuser VIF backend.

The fix to this is to tell libvirt to use a shared mapping for huge pages

http://libvirt.org/formatdomain.html#elementsCPU

eg:

  ...
  <cpu>
    ...
    <numa>
      <cell id='0' cpus='0-3' memory='512000' unit='KiB' memAccess='shared'/>
      <cell id='1' cpus='4-7' memory='512000' unit='KiB' memAccess='shared'/>
    </numa>
    ...
  </cpu>
  ...

notice the addition of the memAccess attribute.

There is no serious downside to using shared mappings, so Nova might as well just unconditionally request them all the time.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/176360

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/176360
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=25df2fae79c03a0335413e7a5de6b54c127e7926
Submitter: Jenkins
Branch: master

commit 25df2fae79c03a0335413e7a5de6b54c127e7926
Author: Daniel P. Berrange <email address hidden>
Date: Wed Apr 22 11:40:29 2015 +0100

    libvirt: mark NUMA huge page mappings as shared access

    When a guest is configured to use huge page backed memory,
    these mappings should be marked as shared. This allows a
    external process connected to a vhostuser VIF type to
    access the guest RAM segment.

    Closes-bug: #1447079
    Change-Id: I1136929d921cdb915de6553e6f0f93e7b29b547e