Scsi device can't be recognized in Guest in aarch64

Bug #1591827 reported by Kevin Zhao
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Fix Released
Medium
Kevin Zhao
Newton
Fix Committed
Medium
Lee Yarwood

Bug Description

Description
===========
Using nova to create an instance in Aarch64,the disk.config is the 'cdrom' and has the type of 'scsi'.After instance creation ,log into the instance and can't see the cdrom device.

Steps to reproduce
==================
1.Using devstack to deploy openstack. Using default local.conf.

2.Upload the aarch64 image with glance.
$ source ~/devstack/openrc admin admin
$ glance image-create --name image-arm64.img --disk-format qcow2 --container-format bare --visibility public --file images/image-arm64-wily.qcow2 --progress
$ glance image-create --name image-arm64.vmlinuz --disk-format aki --container-format aki --visibility public --file images/image-arm64-wily.vmlinuz --progress
$ glance image-create --name image-arm64.initrd --disk-format ari --container-format ari --visibility public --file images/image-arm64-wily.initrd --progress
$ IMAGE_UUID=$(glance image-list | grep image-arm64.img | awk '{ print $2 }')
$ IMAGE_KERNEL_UUID=$(glance image-list | grep image-arm64.vmlinuz | awk '{ print $2 }')
$ IMAGE_INITRD_UUID=$(glance image-list | grep image-arm64.initrd | awk '{ print $2 }')
$ glance image-update --kernel-id ${IMAGE_KERNEL_UUID} --ramdisk-id ${IMAGE_INITRD_UUID} ${IMAGE_UUID}
3.Set the scsi model:
$ glance image-update --property hw_disk_bus --property hw_scsi_model=virtio-scsi ${IMAGE_UUID}

4.nova add keypair
$ nova keypair-add default --pub-key ~/.ssh/id_rsa.pub

5.Launch the instance:
$ image=$(nova image-list | egrep "image-arm64.img"'[^-]' | awk '{ print $2 }')
$ nova boot --flavor m1.small--image ${image} --key-name default test-arm64

6.After creation,use ssh to login into the instance.In the guest:
$ ls -l /dev
The we can see that cdrom doesn't exist.

Expected result
===============
After spawningn the instance, login into the guest :
$ ls -l /dev
We can see the cdrom is here.

Actual result
=============
The xml file of the disk.config is generated by nova ,but can't work.

Environment
===========
1. Exact version of OpenStack you are running. See the following
  list for all releases: http://docs.openstack.org/releases/
   Nova development, commit code: 279f1a9bf65c4b904e01d26f0619a62ed99fc4d3

2. Which hypervisor did you use?
    Libvirt+KVM
    $ kvm --version
    QEMU emulator version 2.5.0 (Debian 1:2.5+dfsg-5ubuntu10.1), Copyright (c) 2003-2008 Fabrice Bellard
    $ libvirtd --version
    libvirtd (libvirt) 1.3.1

2. Which storage type did you use?
   In the host file system,all in one physics machine.
stack@u202154:/opt/stack/nova$ df -hl
Filesystem Size Used Avail Use% Mounted on
udev 7.8G 0 7.8G 0% /dev
tmpfs 1.6G 61M 1.6G 4% /run
/dev/sda2 917G 41G 830G 5% /
tmpfs 7.9G 0 7.9G 0% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 7.9G 0 7.9G 0% /sys/fs/cgroup
/dev/sda1 511M 888K 511M 1% /boot/efi
cgmfs 100K 0 100K 0% /run/cgmanager/fs
tmpfs 1.6G 0 1.6G 0% /run/user/1002
tmpfs 1.6G 0 1.6G 0% /run/user/1000
tmpfs 1.6G 0 1.6G 0% /run/user/0

3. Which networking type did you use?
   nova-network

4. Environment information:
   Architecture : AARCH64
   OS: Ubuntu 16.04

Detailed log info is in the accessory.
The guest xml is also in the log info.

Revision history for this message
Kevin Zhao (kevin-zhao) wrote :

This is the xml file that generated by nova to create the instance.We can see there is a scsi controller which has the model type: virtio-scsi.

Changed in nova:
assignee: nobody → Kevin Zhao (kevin-zhao)
Kevin Zhao (kevin-zhao)
Changed in nova:
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/329403

Revision history for this message
Kashyap Chamarthy (kashyapc) wrote :

Kevin, you seemed to have mixed up the file you intended to add on comment#1 -- you've added some snippet of log file, while you're talking about the Nova-generated guest XML definition.

Revision history for this message
Kevin Zhao (kevin-zhao) wrote :

Hi Kashyap,
Here is the xml that Nova generated.
<domain type="kvm">
  <uuid>b0540cfd-2477-47fb-82d9-a91d7b8e37d7</uuid>
  <name>instance-00000023</name>
  <memory>2097152</memory>
  <vcpu>1</vcpu>
  <metadata>
    <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.0">
      <nova:package version="14.0.0"/>
      <nova:name>test-arm64-1</nova:name>
      <nova:creationTime>2016-06-07 08:18:04</nova:creationTime>
      <nova:flavor name="m1.small">
        <nova:memory>2048</nova:memory>
        <nova:disk>20</nova:disk>
        <nova:swap>0</nova:swap>
        <nova:ephemeral>0</nova:ephemeral>
        <nova:vcpus>1</nova:vcpus>
      </nova:flavor>
      <nova:owner>
        <nova:user uuid="d06750c5a5094ebebcc8c20bdb243b1a">admin</nova:user>
        <nova:project uuid="40757abc778346698c52718f921e9853">admin</nova:project>
      </nova:owner>
      <nova:root type="image" uuid="c6f4f941-7d4b-4a6c-b9c7-2688924826ca"/>
    </nova:instance>
  </metadata>
  <os>
    <type machine="virt">hvm</type>
    <kernel>/opt/stack/data/nova/instances/b0540cfd-2477-47fb-82d9-a91d7b8e37d7/kernel</kernel>
    <initrd>/opt/stack/data/nova/instances/b0540cfd-2477-47fb-82d9-a91d7b8e37d7/ramdisk</initrd>
    <cmdline>root=/dev/vda console=tty0 console=ttyS0</cmdline>
  </os>
  <features>
    <acpi/>
    <apic/>
  </features>
  <cputune>
    <shares>1024</shares>
  </cputune>
  <clock offset="utc">
    <timer name="pit" tickpolicy="delay"/>
    <timer name="rtc" tickpolicy="catchup"/>
  </clock>
  <cpu mode="host-model" match="exact">
    <topology sockets="1" cores="1" threads="1"/>
  </cpu>
  <devices>
    <disk type="file" device="disk">
      <driver name="qemu" type="qcow2" cache="none"/>
      <source file="/opt/stack/data/nova/instances/b0540cfd-2477-47fb-82d9-a91d7b8e37d7/disk"/>
      <target bus="virtio" dev="vda"/>
    </disk>
    <disk type="file" device="cdrom">
      <driver name="qemu" type="raw" cache="none"/>
      <source file="/opt/stack/data/nova/instances/b0540cfd-2477-47fb-82d9-a91d7b8e37d7/disk.config"/>
      <target bus="scsi" dev="sdz"/>
    </disk>
    <controller type="scsi" model="virtio-scsi"/>
    <interface type="bridge">
      <mac address="fa:16:3e:38:8c:51"/>
      <model type="virtio"/>
      <source bridge="br100"/>
      <filterref filter="nova-instance-instance-00000023-fa163e388c51"/>
    </interface>
    <serial type="file">
      <source path="/opt/stack/data/nova/instances/b0540cfd-2477-47fb-82d9-a91d7b8e37d7/console.log"/>
    </serial>
    <serial type="pty"/>
    <memballoon model="virtio">
      <stats period="10"/>
    </memballoon>
  </devices>
</domain>

Kevin Zhao (kevin-zhao)
description: updated
Revision history for this message
Kevin Zhao (kevin-zhao) wrote :

Hi Kashyap,
   Pls ignore the comment#4.
   Here is the Nova generated xml here. We can see that the virtio-scsi controller is added :
<controller type="scsi" model="virtio-scsi"/>
   The controller.index=0 by default.
   index=0 controller worked for sda->sdg
But for sdz(cdrom), the controller bus can not be recognized.
   The next attachment is the xml file by "virsh dumpxml instance-00000006"

Revision history for this message
Kevin Zhao (kevin-zhao) wrote :
Revision history for this message
Kevin Zhao (kevin-zhao) wrote :

The xml which Nova generated for spawning the instance

Revision history for this message
Daniel Berrange (berrange) wrote :

We really need to just fix nova so that it doesn't hardcode a device name of sdz. It should be using the normal block device dev assignment code for the config drive, which would result in it gettig a sensible name of 'sda' (if there's no other SCSI devs attached before it)

Revision history for this message
Kevin Zhao (kevin-zhao) wrote :

Hi Daniel,
    Nova has hardcode the config drive --> cdrom as the last device name sdz. With your method, we can change the cdrom dev name to the front of the devname such as sda,sdb,,, That will work due to add controller:
<controller type="scsi" model="virtio-scsi"/>
The default index for controller virtio-scsi is index=0, which corresponding to sda to sdg. But for example, sdh, it needs virtio-scsi controller that has index=1 and so as sdz need index=3. So...
    Actually I meet another problem by virtio-scsi mode here. When I set hw_scsi_model=virtio-scsi and hw_disk_bus=scsi, then I use "nova volume-attach" to add the volume to an existing instanc, I have add device one by one. Adding volume as sdb, sdc,sde are all OK, they can be hotplug. When the adding device has devname "sdh", it can't be recognized by the instance, due to the controller index problem.
    So I think maybe add the index calculator according to its devname and add the controller xml corresponding devname can solve the problem.
    Thanks~

Changed in nova:
assignee: Kevin Zhao (kevin-zhao) → Zhenyu Zheng (zhengzhenyu)
Changed in nova:
assignee: Zhenyu Zheng (zhengzhenyu) → Kevin Zhao (kevin-zhao)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (master)

Reviewed: https://review.openstack.org/329403
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=d442c1aa31b0a0cbfa0d057e8b6574002ee2e2fc
Submitter: Jenkins
Branch: master

commit d442c1aa31b0a0cbfa0d057e8b6574002ee2e2fc
Author: KevinZhao <email address hidden>
Date: Mon Jun 13 02:14:51 2016 +0000

    libvirt: Delete the lase_device of find_disk_dev_for_disk_bus

    Delete the last_device parameters of find_disk_dev_for_disk_bus.
    Just pick the first available letter on the bus as the device,
    instead of hardcoding on the last device for config drive. Also
    tweak the test case for this.

    Closes-bug: #1591827

    Change-Id: I6343d7ab6f1b0aa58f29805e24db7e2c61d1409e
    Signed-off-by: Kevin Zhao <email address hidden>

Changed in nova:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (stable/newton)

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/403256

Matt Riedemann (mriedem)
Changed in nova:
importance: Undecided → Medium
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (stable/newton)

Reviewed: https://review.openstack.org/403256
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=3d0ec34865df871f448525fca2beca4f83accae3
Submitter: Jenkins
Branch: stable/newton

commit 3d0ec34865df871f448525fca2beca4f83accae3
Author: KevinZhao <email address hidden>
Date: Mon Jun 13 02:14:51 2016 +0000

    libvirt: Delete the lase_device of find_disk_dev_for_disk_bus

    Delete the last_device parameters of find_disk_dev_for_disk_bus.
    Just pick the first available letter on the bus as the device,
    instead of hardcoding on the last device for config drive. Also
    tweak the test case for this.

    Closes-bug: #1591827
    Change-Id: I6343d7ab6f1b0aa58f29805e24db7e2c61d1409e
    Signed-off-by: Kevin Zhao <email address hidden>
    (cherry picked from commit d442c1aa31b0a0cbfa0d057e8b6574002ee2e2fc)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/nova 15.0.0.0b2

This issue was fixed in the openstack/nova 15.0.0.0b2 development milestone.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/nova 14.0.3

This issue was fixed in the openstack/nova 14.0.3 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.