| 2018-06-13 09:20:23 |
Christian Ehrhardt |
description |
Hi,
on the discussions and tests around bug 1769053 it was identified to long term it would be nice to expose host-pyhs-bits and phys-bits through libvirt.
That is what bug 1769053 and the related upstream BZ will continue to track.
But there are two shortcomings on this for now:
1. it takes way too long to get that done for people wanting to use this right now
2. even if it would be implemented in libvirt, the stacks above do not yet exploit the theoretical pyhs-bits related attributes.
To get Ubuntu Users with huge systems a rather quick access to this feature including usability in higher parts of the virtualization stack we want to somewhat follow what RedHat/CentOs do at [1]
There they just switch the default to "on" in the default machine type.
This has some potential drawbacks if you want to migrate across different generations of machines, therefore we don't want it to be the default in Ubuntu.
Instead we would prefer to add a type with a -large suffix could carry that.
Note: Bikeshedding on the name is appreciated until the change is done, then it is as it is.
I have tested code that does so and it makes maintaining Ubuntu machine types not much harder while at the same time providing our users mentioned benefits for huge guests.
Eventually this will lower the pain for those running this huge guests, and by that increase the time until someone does the exposure through libvirt and all the upper stack exploitation. But that is acceptable as price to get this fast and more easy at least to Cosmic and Bionic releases without breaking too much and the mentioned instant exploitation by components higher in the virt stack.
[1]: http://mirrors.ibiblio.org/ovirt/pub/ovirt-4.0/src/qemu-kvm-ev/kvm-target-i386-Enable-host-phys-bits-on-RHEL.patch |
[Impact]
* Qemu supports running guests >1TB but currently the virtualization
stack above has no good way to control that (e.g. libvirt/openstack).
* Long term we'd want to see bug 1769053 (this is where all started)
implemented in libvirt and exploited by at least the major upper
stacks. But until then we want/need to make a way available that lets
users run huge guests.
* Following the example of CentOS (but not switching the default for
better compatibility) we provide a machine type which has the host-
phys-bit setting of qemu default-on. Since upper layers of the virt
stack already can control machine types that allows most use cases to
work until the long term solution can take over down the road.
[Test Case]
* You can do this on swap, but it will be so slow that you will suffer.
So better get a machine with a lot of memory - if possible >1TB
* Spawn a guest with uvtool and ensure that you are running fine
* 'virsh edit' the guest and increase the memory consumption to >1TB
* When this guest is started it works at first but later on
initialization from the guest will touch unreachable adressing bits and
crash - this will appear as hard freeze to the guest and as KVM
emulation failure in the host log of guest execution.
* With the fix you can modify the default type (which should be pc-
i440fx-bionic) to use the -hpb suffix so it would be:
pc-i440fx-bionic-hpb
Use virsh edit to do so.
* Start the guest again, now the guest will work (be aware that
initializing so much memory can take some time)
* This can be tested from PPA build at https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/3292
[Regression Potential]
* We were tempted to change the default on the normal type which would
have had the easiest "ease of use" but then huge regression potential.
We decided against it so we have a new type that people need to select,
this means two things:
1. no change => no regression for users of existing types
2. users using the new type might find new issues we have never seen
(tests are good thou) but that is not a regression, but a
theoretical set of issues with the new function.
[Other Info]
* The SRUability in my opinion falls under BOTH paragraphs from the SRU
page
* For Long Term Support releases we regularly want to enable new
hardware. Such changes are appropriate provided that we can ensure not
to affect upgrades on existing hardware. For example, modaliases of
newly introduced drivers must not overlap with previously shipped
drivers. This also includes updating hardware description data such as
udev's keymaps, media-player-info, mobile broadband vendors, or PCI
vendor/product list updates.
=> This is ensuring that - as machines grow - Bionuc users
can put all their ram to good use in KVM guests.
* For Long Term Support releases we sometimes want to introduce new
features. They must not change the behaviour on existing installations
(e. g. entirely new packages are usually fine). If existing software
needs to be modified to make use of the new feature, it must be
demonstrated that these changes are unintrusive, have a minimal
regression potential, and have been tested properly. To avoid
regressions on upgrade, any such feature must then also be added to any
newer supported Ubuntu release. Once a new feature/package has been
introduced, subsequent changes to it are subject to the usual
requirements of SRUs to avoid regressions.
=> This is exactly what we achieved by adding new types instead of
modifying an existing ones.
---
Hi,
on the discussions and tests around bug 1769053 it was identified to long term it would be nice to expose host-pyhs-bits and phys-bits through libvirt.
That is what bug 1769053 and the related upstream BZ will continue to track.
But there are two shortcomings on this for now:
1. it takes way too long to get that done for people wanting to use this right now
2. even if it would be implemented in libvirt, the stacks above do not yet exploit the theoretical pyhs-bits related attributes.
To get Ubuntu Users with huge systems a rather quick access to this feature including usability in higher parts of the virtualization stack we want to somewhat follow what RedHat/CentOs do at [1]
There they just switch the default to "on" in the default machine type.
This has some potential drawbacks if you want to migrate across different generations of machines, therefore we don't want it to be the default in Ubuntu.
Instead we would prefer to add a type with a -large suffix could carry that.
Note: Bikeshedding on the name is appreciated until the change is done, then it is as it is.
I have tested code that does so and it makes maintaining Ubuntu machine types not much harder while at the same time providing our users mentioned benefits for huge guests.
Eventually this will lower the pain for those running this huge guests, and by that increase the time until someone does the exposure through libvirt and all the upper stack exploitation. But that is acceptable as price to get this fast and more easy at least to Cosmic and Bionic releases without breaking too much and the mentioned instant exploitation by components higher in the virt stack.
[1]: http://mirrors.ibiblio.org/ovirt/pub/ovirt-4.0/src/qemu-kvm-ev/kvm-target-i386-Enable-host-phys-bits-on-RHEL.patch |
|
