Activity log for bug #1457643

Date Who What changed Old value New value Message
2015-05-21 20:17:54 Vladimir Kozhukalov bug added bug
2015-05-21 20:19:02 Mike Scherbakov tags fuel-docs docs
2015-05-21 20:20:33 Vladimir Kozhukalov description We need to mention this in our documentation. As far as we are building target OS images for Ubuntu on the master node, and it requires quite high-performance disk subsystem, it is strictly recommended for KVM users to use cache='unsafe' attribute in their libvirt disks definitions when they use KVM/QEMU VM for their master node. <disk type='file' device='disk'> <driver name='qemu' type='qcow2' cache='unsafe' io='native'/> <source file='/var/lib/libvirt/images/fuel-master.qcow2'/> <target dev='vda' bus='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/> </disk> We need to mention this in our documentation, otherwise building image could take more than an hour and so deployment could fail due to provision timeout. As far as we are building target OS images for Ubuntu on the master node, and it requires quite high-performance disk subsystem, it is strictly recommended for KVM users to use cache='unsafe' attribute in their libvirt disks definitions when they use KVM/QEMU VM for their master node. <disk type='file' device='disk'>       <driver name='qemu' type='qcow2' cache='unsafe' io='native'/>       <source file='/var/lib/libvirt/images/fuel-master.qcow2'/>       <target dev='vda' bus='virtio'/>       <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>     </disk>
2015-05-21 20:21:46 Vladimir Kozhukalov description We need to mention this in our documentation, otherwise building image could take more than an hour and so deployment could fail due to provision timeout. As far as we are building target OS images for Ubuntu on the master node, and it requires quite high-performance disk subsystem, it is strictly recommended for KVM users to use cache='unsafe' attribute in their libvirt disks definitions when they use KVM/QEMU VM for their master node. <disk type='file' device='disk'>       <driver name='qemu' type='qcow2' cache='unsafe' io='native'/>       <source file='/var/lib/libvirt/images/fuel-master.qcow2'/>       <target dev='vda' bus='virtio'/>       <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>     </disk> We need to mention this in our documentation. As far as we are building target OS images for Ubuntu on the master node, and it requires quite high-performance disk subsystem, it is strictly recommended for KVM users to use cache='unsafe' attribute in their libvirt disks definitions when they use KVM/QEMU VM for their master node. Otherwise, building target OS image could take more than an hour and so the deployment could fail due to provision timeout. <disk type='file' device='disk'>       <driver name='qemu' type='qcow2' cache='unsafe' io='native'/>       <source file='/var/lib/libvirt/images/fuel-master.qcow2'/>       <target dev='vda' bus='virtio'/>       <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>     </disk>
2015-07-01 11:50:46 Evgeny Konstantinov fuel: assignee Fuel Documentation Team (fuel-docs) Evgeny Konstantinov (evkonstantinov)
2015-07-01 12:27:40 OpenStack Infra fuel: status Triaged In Progress
2015-07-03 07:05:09 Alexei Sheplyakov summary Use cache='unsafe' for KVM based master node IBP images build is way too slow
2015-07-03 07:15:35 Alexei Sheplyakov tags docs
2015-07-03 07:15:57 Alexei Sheplyakov fuel: assignee Evgeny Konstantinov (evkonstantinov) Alexei Sheplyakov (asheplyakov)
2015-07-03 07:16:03 Alexei Sheplyakov fuel: status In Progress Confirmed
2015-07-11 17:43:42 Mike Scherbakov nominated for series fuel/7.0.x
2015-07-11 17:43:42 Mike Scherbakov bug task added fuel/7.0.x
2015-07-11 17:43:42 Mike Scherbakov nominated for series fuel/6.1.x
2015-07-11 17:43:42 Mike Scherbakov bug task added fuel/6.1.x
2015-07-11 17:44:15 Mike Scherbakov fuel/6.1.x: milestone 6.1-updates
2015-07-11 17:44:22 Mike Scherbakov fuel/7.0.x: milestone 6.1 7.0
2015-07-13 07:55:35 Alexei Sheplyakov description We need to mention this in our documentation. As far as we are building target OS images for Ubuntu on the master node, and it requires quite high-performance disk subsystem, it is strictly recommended for KVM users to use cache='unsafe' attribute in their libvirt disks definitions when they use KVM/QEMU VM for their master node. Otherwise, building target OS image could take more than an hour and so the deployment could fail due to provision timeout. <disk type='file' device='disk'>       <driver name='qemu' type='qcow2' cache='unsafe' io='native'/>       <source file='/var/lib/libvirt/images/fuel-master.qcow2'/>       <target dev='vda' bus='virtio'/>       <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>     </disk> Due to a very suboptimal IO layering building the target OS (Ubuntu) IBP images takes an unreasonably long time (more than an hour!) if the master node runs in the KVM VM. fuel_agent (which builds Ubuntu images) runs in the mcollective container. A container filesystem is backed by dm-snapshot which is known to have a poor write performance, especially for large files (like OS images). To add an insult to injury dm-snapshot storage is a loopback file residing in /var/lib/docker. Thus the data written to the OS image being built goes through a number of layers: ext4 -> loop -> dm-snapshot (/var/lib/docker) -> loop -> -> ext4 (/var/lib/docker) -> dm-linear (os-varlibdocker) -> -> hard drive If the master node runs on a VM rather than a physcial hardware the above chain gets even more complicated (and write performance gets really awful) ext4 -> loop -> dm-snapshot (/var/lib/docker) -> loop -> -> ext4 (/var/lib/docker) -> dm-linear (os-varlibdocker) -> -> virtual hard drive (qemu) -> qcow2 -> ext4 (host) We need to remove the extra layers to make IO performance reasonable.
2015-07-13 09:57:07 Aleksandr Didenko fuel/7.0.x: status Confirmed Fix Committed
2015-07-13 09:57:42 Aleksandr Didenko fuel/6.1.x: assignee MOS Sustaining (mos-sustaining)
2015-07-13 09:57:47 Aleksandr Didenko fuel/6.1.x: importance Undecided High
2015-07-13 09:57:51 Aleksandr Didenko fuel/6.1.x: status New Confirmed
2015-07-20 21:03:55 Vitaly Sedelnik fuel/6.1.x: status Confirmed Won't Fix
2015-09-28 10:26:40 Maksym Strukov fuel/7.0.x: status Fix Committed Fix Released
2015-09-28 10:26:46 Maksym Strukov fuel: status Fix Committed Fix Released