[qa] [sriov] To validate SR-IOV we need ixgbevf drivers in Cirros

Bug #1558598 reported by Vladimir Eremin on 2016-03-17
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Low
Albert Syriy

Bug Description

When instance is launched on SR-IOV interface, host nic will be passed to it, so we need appropriate drivers. Cirros contains only libvirt drivers.

We need to rebuild Cirros with ixgbevf

Igor Yozhikov (iyozhikov) wrote :

According to responsibility matrix - moving this to mos-linux

Changed in fuel:
assignee: MOS Packaging Team (mos-packaging) → MOS Linux (mos-linux)
Dmitry Teselkin (teselkin-d) wrote :

Please use more appropriate OS for such tests [0]

[0] http://cloud-images.ubuntu.com/

Changed in fuel:
status: New → Invalid
Artem Panchenko (apanchenko-8) wrote :

Dmitry,

AFAIK we already have customized Cirros image in MOS:

https://review.fuel-infra.org/gitweb?p=packages/trusty/cirros-testvm.git;a=tree;h=refs/heads/master;hb=refs/heads/master

In 9.0 we are going to add health checks for SR-IOV feature to Fuel, so end users will be able to automatically test it after cloud deployment. It will be nice to have ability to use Cirros image for that, because we don't ship Ubuntu image with MOS (manual actions will be required to run the test).

Please let us know is it technically possible to rebuild Cirros with ixgbevf driver?

Changed in fuel:
status: Invalid → Confirmed
importance: Undecided → Medium
milestone: none → 9.0
tags: added: feature-sr-iov
removed: feature-security
summary: - [packaging] [qa] [sriov] To validate SR-IOV we need ixgbevf drivers in
- Cirros
+ [qa] [sriov] To validate SR-IOV we need ixgbevf drivers in Cirros
Changed in fuel:
assignee: MOS Linux (mos-linux) → Albert Syriy (asyriy)
Albert Syriy (asyriy) wrote :

Could you, please check the image

Albert Syriy (asyriy) wrote :

This image is going to be packed, but could you test the requested drivers first&
The drivers e1000e, igbevf, ixgbedvf were added to the image.

Albert Syriy (asyriy) on 2016-03-21
Changed in fuel:
status: Confirmed → In Progress

Fix proposed to branch: master
Change author: Albert Syriy <email address hidden>
Review: https://review.fuel-infra.org/18261

Vladimir Eremin (yottatsa) wrote :

This image crashed my host machine, but I has wrong-bios-hw. Please re-check on correct hw.

Vladimir, I afraid we don't have such HW (network card).
Is it possible to upgrade your "wrong bios-HW" ?

Regards,
Albert

Albert Syriy,

Software Engineer,
Mirantis

On Mon, Mar 21, 2016 at 8:04 PM, Vladimir Eremin <email address hidden>
wrote:

> This image crashed my host machine, but I has wrong-bios-hw. Please re-
> check on correct hw.
>
> --
> You received this bug notification because you are a bug assignee.
> https://bugs.launchpad.net/bugs/1558598
>
> Title:
> [qa] [sriov] To validate SR-IOV we need ixgbevf drivers in Cirros
>
> Status in Fuel for OpenStack:
> In Progress
>
> Bug description:
> When instance is launched on SR-IOV interface, host nic will be passed
> to it, so we need appropriate drivers. Cirros contains only libvirt
> drivers.
>
> We need to rebuild Cirros with ixgbevf
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/fuel/+bug/1558598/+subscriptions
>

Vladimir Eremin (yottatsa) wrote :

mchernik to validate

Changed in fuel:
assignee: Albert Syriy (asyriy) → Mikhail Chernik (mchernik)
Vladimir Eremin (yottatsa) wrote :

Cirros fails with next in messages on dmesg on HOST

[Wed Mar 23 16:38:13 2016] pci-stub 0000:03:11.5: claimed by stub
[Wed Mar 23 16:38:13 2016] ixgbe 0000:03:00.1: setting MAC fa:16:3e:6d:a9:ce on VF 6
[Wed Mar 23 16:38:13 2016] ixgbe 0000:03:00.1: Reload the VF driver to make this change effective.
[Wed Mar 23 16:38:13 2016] ixgbe 0000:03:00.1: Setting VLAN 1804, QOS 0x0 on VF 6
[Wed Mar 23 16:38:13 2016] type=1400 audit(1458751100.769:59): apparmor="STATUS" operation="profile_load" profile="unconfined" name="libvirt-f6258012-ebbb-4cf6-9fc8-fac31f819b21" pid=31962 comm="apparmor_parser"
[Wed Mar 23 16:38:13 2016] type=1400 audit(1458751100.769:60): apparmor="STATUS" operation="profile_load" profile="unconfined" name="qemu_bridge_helper" pid=31962 comm="apparmor_parser"
[Wed Mar 23 16:38:13 2016] pci-stub 0000:03:11.5: enabling device (0000 -> 0002)
[Wed Mar 23 16:38:14 2016] pci-stub 0000:03:11.5: kvm assign device
[Wed Mar 23 16:38:19 2016] kvm: zapping shadow pages for mmio generation wraparound
[Wed Mar 23 16:38:21 2016] intel_iommu_map: iommu width (48) is not sufficient for the mapped address (fffffffffe001000)
[Wed Mar 23 16:38:21 2016] kvm_iommu_map_address:iommu failed to map pfn=90820
[Wed Mar 23 16:38:22 2016] type=1400 audit(1458751109.429:61): apparmor="STATUS" operation="profile_remove" profile="unconfined" name="libvirt-f6258012-ebbb-4cf6-9fc8-fac31f819b21" pid=32210 comm="apparmor_parser"
[Wed Mar 23 16:38:22 2016] ixgbevf 0000:03:11.5: enabling device (0000 -> 0002)
[Wed Mar 23 16:38:22 2016] ixgbe 0000:03:00.1 ens3f1: VF Reset msg received from vf 6
[Wed Mar 23 16:38:22 2016] ixgbevf 0000:03:11.5: irq 329 for MSI/MSI-X
[Wed Mar 23 16:38:22 2016] ixgbevf 0000:03:11.5: irq 330 for MSI/MSI-X

Vladimir Eremin (yottatsa) wrote :

Console log if the instance ends on
[ 0.000000] Linux version 3.2.0-80-virtual (buildd@batsu) (gcc version 4.6.3 (Ubuntu/Linaro 4.6.3-1ubuntu5) ) #116-Ubuntu SMP Mon Mar 23 17:28:52 UTC 2015 (Ubuntu 3.2.0-80.116-virtual 3.2.68)
...
[ 0.059703] pci_root PNP0A03:00: host bridge window [io 0x0000-0x0cf7]
[ 0.060005] pci_root PNP0A03:00: host bridge window [io 0x0d00-0xffff]
[ 0.060716] pci_root PNP0A03:00: host bridge window [mem 0x000a0000-0x000bffff]
[ 0.061644] pci_root PNP0A03:00: host bridge window [mem 0x80000000-0xfebfffff]
[ 0.078033] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI
[ 0.078985] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB

And KVM drops on
kvm_set_phys_mem: error registering slot: Bad address

Vladimir Eremin (yottatsa) wrote :

According to https://bugs.launchpad.net/qemu/+bug/1353947, problematic is guests with linux 3.2.0

Changed in fuel:
assignee: Mikhail Chernik (mchernik) → Albert Syriy (asyriy)
Changed in fuel:
importance: Medium → Low
Albert Syriy (asyriy) wrote :

The CirrOS image has been rebuilt against the current latest kernel version 3.13.0-83.
Please check it on HW.

Albert Syriy (asyriy) wrote :

Sorry, the image doesn't contain partition table.
Update the file

Vladimir Eremin (yottatsa) wrote :

This one is works fully.

Albert Syriy (asyriy) wrote :

Image with DHCP client, sudo, ssh ... etc was built.
Please check again

Albert Syriy (asyriy) wrote :
Albert Syriy (asyriy) wrote :
Vladimir Eremin (yottatsa) wrote :

1. Strange
WARN: failed: route add -net "0.0.0.0/0" gw "192.168.111.1"

2. mkdir error
checking http://169.254.169.254/2009-04-04/instance-id
successful after 1/20 tries: up 1.18. iid=i-00000003
failed to get http://169.254.169.254/2009-04-04/meta-data/public-keys
warning: no ec2 metadata for public-keys
failed to get http://169.254.169.254/2009-04-04/user-data
warning: no ec2 metadata for user-data
mkdir: can't create directory '.ssh': No space left on device
failed to make .ssh
Starting dropbear sshd: OK
GROWROOT: CHANGED: partition=1 start=2048 old: size=49152 end=51200 new: size=2095071,end=2097119
/run/cirros/datasource/data/user-data was not '#!' or executable

Albert Syriy (asyriy) wrote :

Hello Vladimir,

Could we check together the issue tomorrow?
(Please don't sweep away the deployed environment).

Albert Syriy,

Software Engineer,
Mirantis

On Thu, Mar 31, 2016 at 8:00 PM, Vladimir Eremin <email address hidden>
wrote:

> 1. Strange
> WARN: failed: route add -net "0.0.0.0/0" gw "192.168.111.1"
>
> 2. mkdir error
> checking http://169.254.169.254/2009-04-04/instance-id
> successful after 1/20 tries: up 1.18. iid=i-00000003
> failed to get http://169.254.169.254/2009-04-04/meta-data/public-keys
> warning: no ec2 metadata for public-keys
> failed to get http://169.254.169.254/2009-04-04/user-data
> warning: no ec2 metadata for user-data
> mkdir: can't create directory '.ssh': No space left on device
> failed to make .ssh
> Starting dropbear sshd: OK
> GROWROOT: CHANGED: partition=1 start=2048 old: size=49152 end=51200 new:
> size=2095071,end=2097119
> /run/cirros/datasource/data/user-data was not '#!' or executable
>
> --
> You received this bug notification because you are a bug assignee.
> https://bugs.launchpad.net/bugs/1558598
>
> Title:
> [qa] [sriov] To validate SR-IOV we need ixgbevf drivers in Cirros
>
> Status in Fuel for OpenStack:
> In Progress
>
> Bug description:
> When instance is launched on SR-IOV interface, host nic will be passed
> to it, so we need appropriate drivers. Cirros contains only libvirt
> drivers.
>
> We need to rebuild Cirros with ixgbevf
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/fuel/+bug/1558598/+subscriptions
>

Albert Syriy (asyriy) wrote :

The BVT test with the cirros image successfully passed.

Reviewed: https://review.fuel-infra.org/18261
Submitter: Pkgs Jenkins <email address hidden>
Branch: master

Commit: b0a0f15dfaa205161a7fcb20cf1b8cd4948c2ef3
Author: Albert Syriy <email address hidden>
Date: Fri Apr 1 19:09:45 2016

The cirros image was rebuilt against the 3.13.0-83 kernel, drivers e1000e, igbvf, ixgbevf were added.

The cirros image were rebuilt angainst Ubuntu 14.04 and 3.13.0-83 kernel
since the project has been frozen (the last version 0.3.4 was issued about
a year ago). The Intel drivers e1000e, igbvf, ixgbevf were added to
the cirros image. The sources were taken from the link
http://bazaar.launchpad.net/~cirros-dev/cirros/trunk
and put into the cirros-testvm/src-cirros folders, the
cirros-testvm/download folder is used for keeping compressed
cirros-testvm/src-cirros/build folder 2015.05 version.
Both src-cirros and download folder were excluded from copying
during the build in the debian/source/option file.
Please consult the README file how to rebuilt the cirros image.

Change-Id: I7561a2b8c43c7c6b737aa917ea048c38237f31eb
Closes-Bug: 1558598

Changed in fuel:
status: In Progress → Fix Committed

Fix proposed to branch: 9.0
Change author: Albert Syriy <email address hidden>
Review: https://review.fuel-infra.org/19553

Alexander Gubanov (ogubanov) wrote :

Vefified on MOS 9.0 (build 201) - I've successfully created instance from this image - modules are presented/uploaded.
Proof: http://pastebin.com/PQ0HPCNL

Vladimir, could you please also verify it and move to "Fix Release" ?

tags: added: keep-in-9.0

Reviewed: https://review.fuel-infra.org/19553
Submitter: Pkgs Jenkins <email address hidden>
Branch: 9.0

Commit: f9e8ed6f4d16953cf0e4fb97ea168c8d0ab0e561
Author: Albert Syriy <email address hidden>
Date: Wed Apr 13 07:18:40 2016

The cirros image was rebuilt against the 3.13.0-83 kernel, drivers e1000e, igbvf, ixgbevf were added.

The cirros image were rebuilt angainst Ubuntu 14.04 and 3.13.0-83 kernel
since the project has been frozen (the last version 0.3.4 was issued about
a year ago). The Intel drivers e1000e, igbvf, ixgbevf were added to
the cirros image. The sources were taken from the link
http://bazaar.launchpad.net/~cirros-dev/cirros/trunk
and put into the cirros-testvm/src-cirros folders, the
cirros-testvm/download folder is used for keeping compressed
cirros-testvm/src-cirros/build folder 2015.05 version.
Both src-cirros and download folder were excluded from copying
during the build in the debian/source/option file.
Please consult the README file how to rebuilt the cirros image.

Change-Id: I7561a2b8c43c7c6b737aa917ea048c38237f31eb
Closes-Bug: 1558598
(cherry picked from commit b0a0f15dfaa205161a7fcb20cf1b8cd4948c2ef3)

Mikhail Chernik (mchernik) wrote :

Verified on MOS9.0 ISO 232, drivers are present.

Changed in fuel:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers