[lucid] merge libvirt to 0.7.7

[Jamie Strandboge]

For LTS support and tons of bugfixes, it would be good to merge libvirt 0.7.7 to Lucid. This is a placeholder bug and is being discussed and evaluated by the server team (in ubuntu-server mailing list), particularly Dustin Kirkland.

There are mainly two problems so far:
1. scsi attach is broken due to moving to device_add. These changes could be backed out with some work or upstream could patch it soon and we cherrypick it. This is a blocker because it breaks eucalyptus volume attachment. This is upstream bug https://bugzilla.redhat.com/show_bug.cgi?id=578975.

2. 0.7.7 uses a stacked security driver which means the DAC security driver is on by default and changes the permissions of the files it uses to that of the user running qemu. On Ubuntu this is root. This may be unexpected behavior for people who store images outside of /var/lib. This only happens with qemu:///system.

'2' we can live with, but '1' is a blocker.

[Jamie Strandboge (jdstrand)]

As it turns out, https://bugzilla.redhat.com/show_bug.cgi?id=578975 is not a bug, but it is a functional regression. From the upstream bug:

09:07 < aliguori> jdstrand, you're adding a scsi disk to an existing scsi
                  adapter
09:07 < aliguori> lspci isn't going to change
09:07 < aliguori> but you won't see anything in /proc/partitions until you
                  rescan the scsi bus
09:07 < aliguori> which is not something that happens automatically
09:07 < aliguori> it's *not* pci hot add
09:08 < aliguori> it's scsi hot add
09:08 < aliguori> http://kbase.redhat.com/faq/docs/DOC-3942
09:08 < aliguori> this is different than what older libvirts did
09:08 < aliguori> older libvirt added a new scsi controller for every disk add
09:08 < aliguori> which was incorrect
09:14 < jdstrand> aliguori: I'm just used to seeing this show up in the guest,
                  so the new behavior, while I understand it, needs to be dealt
                  with in applications that depended on the previously faulty
                  behavior
09:14 < aliguori> yup

So, after using attach-device, I perform the following:
echo "- - -" > /sys/class/scsi_host/host0/scan

Then the disk shows up in the disk.

[Jamie Strandboge (jdstrand)]

From IRC from #ubuntu-server (excerpted):
09:36 < jdstrand> kirkland, smoser: in the past, libvirt created a new scsi
                  controller for every new device. this was a pci add which is
                  why it showed up magically
09:37 < jdstrand> kirkland, smoser: this is incorrect behavior and now libvirt
                  adds disks to an existing bus, like what would happen in the
                  real world
09:39 < jdstrand> smoser: also, aligouri (upstream qemu) reiterated that we
                  should not be using scsi hot add
09:47 < jdstrand> smoser: virtio hot add stills works
09:43 < smoser> it will absolutely break guests if devices don't "just show up"
09:48 < jdstrand> smoser: of course, kirkland mentioned euco is resistent to
                  vd* devices
10:05 < jdstrand> kirkland, smoser: so the decision comes down to: do we stick
                  with the old 0.7.5, lose the bug fixes but maintain
                  deprecated scsi behavior and not touch euca for an LTS, or
                  move to the new 0.7.7 with its fixes, but new scsi behavior
                  and updating euca
10:11 < smoser> the primary one, in my opinion, is that EBS (the functionality
                exposed by attach/detach) is supposed to promise that your data
                is SAFE
10:11 < smoser> and as we know, with buggy scsi, that is probably not the case.
10:13 < smoser> i'm open to a tested virtio solution because it is the
                "accepted solution" for hotplug
10:15 < jdstrand> I'm betting you will see a performance gain over scsi as well
10:15 < kirkland> jdstrand: oh, the performance gain is huge
10:16 < kirkland> jdstrand: we're going to insist on it [Ed. virtio] for lucid+1
10:20 < smoser> kirkland, jdstrand i think, for maintainability that this is
                probably somethign worth testing

So bottom line, for stability, integrity, maintainability and performance reasons, kirkland and smoser will test virtio attach and decide if it is worth pursuing. If so, 0.7.7 could still make it for Lucid.

[Jamie Strandboge (jdstrand)]

The server team after discussing with upstream eucalyptus have decided this is too risky, but that eucalyptus will be updated to use virtio in Lucid+1.

[Launchpad Janitor (janitor)]

This bug was fixed in the package libvirt - 0.8.1-2ubuntu1

---------------
libvirt (0.8.1-2ubuntu1) maverick; urgency=low

  * Merge from debian unstable. Remaining changes:
    - Fixes:
      LP: #522845
      LP: #553737
      LP: #520386
    - debian/control:
      + Build-Depends on qemu-kvm, not qemu
      + Build-Depends on open-iscsi-utils, not open-iscsi
      + Build-Depends on libxml2-utils
      + Build-Depends on libapparmor-dev and Suggests apparmor
      + Bump bridge-utils, dnsmasq-base, netcat-openbsd, and iptables
        to Depends of libvirt-bin
      + Drop qemu-kvm and qemu to Suggests
      + We call libxen-dev libxen3-dev, so change all references
      + Rename Vcs-* to XS-Debian-Vcs-*
    - debian/libvirt-bin.postinst:
      + rename the libvirt group to libvirtd
      + add each admin user to the libvirtd group
      + reload apparmor profiles
    - debian/libvirt-bin.postrm:
      + rename the libvirt group to libvirtd
      + remove apparmor symlinks on purge
    - debian/README.Debian: add AppArmor section based on the upstream
      documentation
    - debian/rules:
      + update DEB_DH_INSTALLINIT_ARGS for upstart
      + add DEB_MAKE_CHECK_TARGET := check
      + use --with-apparmor
      + copy apparmor and apport hook to debian/tmp
    - add debian/libvirt-bin.upstart
    - debian/libvirt-bin.dirs: add /etc/apparmor.d/abstractions,
      /etc/apparmor.d/disable, /etc/apparmor.d/force-complain,
      /etc/apparmor.d/libvirt, /etc/cron.daily and
      /usr/share/apport/package-hooks
    - add debian/libvirt-bin.cron.daily
    - add debian/libvirt-bin.apport
    - debian/libvirt-bin.install: install apparmor profiles, abstractions
      and apport hook
    - debian/apparmor:
      - add TEMPLATE
      - add libvirt-qemu abstraction
      - add usr.lib.libvirt.virt-aa-helper
      - add usr.sbin.libvirtd
    - debian/patches/series:
      + don't apply 0002-qemu-disable-network.diff.patch
      + don't apply 0005-Terminate-nc-on-EOF.patch. Use
        9010-autodetect-nc-params.patch instead
      + 9000-delayed_iff_up_bridge.patch (refreshed)
      + 9001-dont_clobber_existing_bridges.patch
      + 9002-better_default_uri_virsh.patch (updated)
      + 9004-better-default-arch.patch
      + 9005-libvirtd-group-name.patch
      + 9006-increase-unix-socket-timeout.patch (refreshed)
      + 9007-default-config-test-case.patch (updated)
      + 9008-fix-daemon-conf-ftbfs.patch (rewritten)
      + 9009-run-as-root-by-default.patch (refreshed)
      + 9010-autodetect-nc-params.patch (refreshed, formerly 9015)
      + 9011-dont-disable-ipv6.patch (updated)
  * Dropped following packaging changes, no longer required with upgrades
    from Lucid:
    - debian/control:
      + versioned Conflicts/Replaces to libvirt0 for libvirt0-dbg
      + remove Build-Depends on libcap-ng-dev
    - debian/libvirt-bin.postinst: virt-aa-helper profile migration to
      /usr/lib/libvirt
    - debian/libvirt-bin.preinst: added to force complain on certain
      upgrades
  * Dropped the following patches, included upstream:
    - 0010-Use-base-16-for-product-vendor.patch
    - 9003-increase-logoutput-timeout.patch
    - 9010-apparmor-ftbfs...