Unable to mount more than 7 volumes when using virtio-scsi

[Vladyslav Drok]

Customer is experiencing issue in their Mitaka using scsi as hw_bus and virtio-scsi as default hw_scsi_model in all their public images based on the following bug: https://bugs.launchpad.net/nova/+bug/1686116

They are using ceph. Here is a excerpt of the syslog when attempting to mount a 8th volume:
Aug 9 13:53:12 c3e1 nova-compute[68086]: 2017-08-09 13:53:12.084 68086 INFO nova.compute.manager [req-2d5c1506-69ed-4cc2-9111-256aa2e51609 k2k:8e81125511504d0abc946f30c0fd0161 19424077745e45ff8b9d62b7dfbe78d4 - - -] [instance: 4c8c7d55-92b1-47b9-a7b7-c5b608d32d65] Attaching volume a95384a8-7ba2-4b71-9b83-25a5fad523e2 to /dev/sdi
Aug 9 13:53:12 c3e1 snmpd[191582]: error on subcontainer 'ia_addr' insert (-1)
Aug 9 13:53:13 c3e1 kernel: [7320163.790896] audit: type=1400 audit(1502286793.056:8200): apparmor="DENIED" operation="open" profile="libvirt-4c8c7d55-92b1-47b9-a7b7-c5b608d32d65" name="/etc/ceph/ceph.client.nova.keyring" pid=25937 comm="qemu-system-x86" requested_mask="r" denied_mask="r" fsuid=64055 ouid=0
Aug 9 13:53:12 c3e1 snmpd[191582]: message repeated 3 times: [ error on subcontainer 'ia_addr' insert (-1)]
Aug 9 13:53:18 c3e1 libvirtd[7500]: invalid argument: invalid path: sdj
Aug 9 13:53:18 c3e1 collectd[7740]: libvirt: QEMU Driver error : invalid argument: invalid path: sdj
Aug 9 13:53:19 c3e1 nova-compute[68086]: 2017-08-09 13:53:19.507 68086 WARNING nova.compute.manager [req-4be01fde-9e3b-4727-9ead-d58284ac8b82 - - - - -] While synchronizing instance power states, found 22 instances in the database and 33 instances on the hypervisor.
Aug 9 13:53:23 c3e1 libvirtd[7500]: internal error: missing storage backend for network files using rbd protocol
Aug 9 13:53:23 c3e1 libvirtd[7500]: message repeated 3 times: [ internal error: missing storage backend for network files using rbd protocol]

[Fuel Devops McRobotson (fuel-devops-robot)]

Reviewed: https://review.fuel-infra.org/36274
Submitter: Pkgs Jenkins <email address hidden>
Branch: 9.0/mitaka

Commit: 7adaadea0981241b5a89bc3af483b5b59eb214de
Author: Sahid Orentino Ferdjaoui <email address hidden>
Date: Tue Aug 29 21:17:25 2017

Properly support address section in the disk description

This change is a squash of 6 commits that add parsing and proper
use of address section of disk devices in the domain XMLs for the
libvirt driver. It also enables libvirt driver to properly increment
'unit' counter if multiple scsi devices requested.

PROD ticket: https://mirantis.jira.com/browse/PROD-13763

Upstream changes squashed:
I9620feffa74b8f7bc932eed03e3a60f46533681d
Id796d9129ff850abd209f4e9e4e54ea77c477acf
I03b03962748db7081abd5aebf3578ddc4e30fe68
I98e53b378cc99747765066001a0b51880543d2dd
I701145abc0e300711a01889e8d62b1f7887da120
I1a8023ee6e8c85eed1c7c55a21f996371a0dd80a

Closes-Bug: #1713824
Change-Id: Ie50dca3e1c56c0114ee38dcfea1f2209a29da035

[Fuel Devops McRobotson (fuel-devops-robot)]

Fix proposed to branch: mcp/1.0/mitaka
Change author: Sahid Orentino Ferdjaoui <email address hidden>
Review: https://review.fuel-infra.org/36373

[Fuel Devops McRobotson (fuel-devops-robot)]

Change abandoned by Vladyslav Drok <email address hidden> on branch: mcp/1.0/mitaka
Review: https://review.fuel-infra.org/36373

[Fuel Devops McRobotson (fuel-devops-robot)]

Change restored by Vladyslav Drok <email address hidden> on branch: mcp/1.0/mitaka
Review: https://review.fuel-infra.org/36373

[Fuel Devops McRobotson (fuel-devops-robot)]

Reviewed: https://review.fuel-infra.org/36373
Submitter: Pkgs Jenkins <email address hidden>
Branch: mcp/1.0/mitaka

Commit: 657fdd197fa824a6efe3eab2cf3924194f660ea7
Author: Sahid Orentino Ferdjaoui <email address hidden>
Date: Wed Aug 30 18:45:15 2017

Properly support address section in the disk description

This change is a squash of 6 commits that add parsing and proper
use of address section of disk devices in the domain XMLs for the
libvirt driver. It also enables libvirt driver to properly increment
'unit' counter if multiple scsi devices requested.

PROD ticket: https://mirantis.jira.com/browse/PROD-13763

Upstream changes squashed:
I9620feffa74b8f7bc932eed03e3a60f46533681d
Id796d9129ff850abd209f4e9e4e54ea77c477acf
I03b03962748db7081abd5aebf3578ddc4e30fe68
I98e53b378cc99747765066001a0b51880543d2dd
I701145abc0e300711a01889e8d62b1f7887da120
I1a8023ee6e8c85eed1c7c55a21f996371a0dd80a

Closes-Bug: #1713824
(cherry picked from commit 4cde3638c26645be95f52f74890363e6e810e434)
Change-Id: Ie50dca3e1c56c0114ee38dcfea1f2209a29da035

[Ilya Bumarskov (ibumarskov)]

Verified on Fuel 9.2 MU-3 (snapshots/9.0-2017-10-09-094324):
All 8 volumes were attached to virtio-scsi controller:

-------------------------------------------------------------------------------------
  <devices>
    <emulator>/usr/bin/qemu-system-x86_64</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2' cache='none'/>
      <source file='/var/lib/nova/instances/cd76e051-130e-480a-9eb5-af80e2337dab/disk'/>
      <backingStore type='file' index='1'>
        <format type='raw'/>
        <source file='/var/lib/nova/instances/_base/6ccc5ff78b132fdee6214c3d2b4f233ea1684f56'/>
        <backingStore/>
      </backingStore>
      <target dev='sda' bus='scsi'/>
      <alias name='scsi0-0-0-0'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <disk type='network' device='disk'>
      <driver name='qemu' type='raw' cache='writeback'/>
      <auth username='compute'>
        <secret type='ceph' uuid='a5d0dd94-57c4-ae55-ffe0-7e3732a24455'/>
      </auth>
      <source protocol='rbd' name='volumes/volume-9704a041-696c-4871-9969-a363c49c2aa2'>
        <host name='10.109.2.2' port='6789'/>
      </source>
      <backingStore/>
      <target dev='sdb' bus='scsi'/>
      <serial>9704a041-696c-4871-9969-a363c49c2aa2</serial>
      <alias name='scsi0-0-0-2'/>
      <address type='drive' controller='0' bus='0' target='0' unit='2'/>
    </disk>
    <disk type='network' device='disk'>
      <driver name='qemu' type='raw' cache='writeback'/>
      <auth username='compute'>
        <secret type='ceph' uuid='a5d0dd94-57c4-ae55-ffe0-7e3732a24455'/>
      </auth>
      <source protocol='rbd' name='volumes/volume-2a762181-acb8-4325-a768-44ee111b8567'>
        <host name='10.109.2.2' port='6789'/>
      </source>
      <backingStore/>
      <target dev='sdc' bus='scsi'/>
      <serial>2a762181-acb8-4325-a768-44ee111b8567</serial>
      <alias name='scsi0-0-0-3'/>
      <address type='drive' controller='0' bus='0' target='0' unit='3'/>
    </disk>
    <disk type='network' device='disk'>
      <driver name='qemu' type='raw' cache='writeback'/>
      <auth username='compute'>
        <secret type='ceph' uuid='a5d0dd94-57c4-ae55-ffe0-7e3732a24455'/>
      </auth>
      <source protocol='rbd' name='volumes/volume-54a2e9c2-ab00-409b-8d7b-340823c89b70'>
        <host name='10.109.2.2' port='6789'/>
      </source>
      <backingStore/>
      <target dev='sdd' bus='scsi'/>
      <serial>54a2e9c2-ab00-409b-8d7b-340823c89b70</serial>
      <alias name='scsi0-0-0-4'/>
      <address type='drive' controller='0' bus='0' target='0' unit='4'/>
    </disk>
    <disk type='network' device='disk'>
      <driver name='qemu' type='raw' cache='writeback'/>
      <auth username='compute'>
        <secret type='ceph' uuid='a5d0dd94-57c4-ae55-ffe0-7e3732a24455'/>
      </auth>
      <source protocol='rbd' name='volumes/volume-7de91944-5da7-4128-9978-91d7b8ba3c16'>
        <host name='10.109.2.2' port='6789'/>
      </source>
      <backingStore/>
      <target dev='sde' bus='scsi'/>
      <serial>7de91944-5da7-4128-9978-91d7b8ba3c16</serial>
      <alias name='scsi0-0-0-5'/>
      <address type='drive' controller='0' ...