Hi Sofia, Thanks for reaching out, The VM is not allowed to read/write into the space that is bigger than the volume. Example: Message at 2.986 : Here the machine finishes booting and cloud-init takes over installing the user-data script. I focused on vda / xfs / iscsi and network errors in dmesg, giving me the following : [ 2.986795] XFS (vda1): Ending clean mount [ 284.439066] blk_update_request: I/O error, dev vda, sector 2641440 [ 284.443818] blk_update_request: I/O error, dev vda, sector 2642448 [ 284.445159] blk_update_request: I/O error, dev vda, sector 2643456 [ 284.446351] blk_update_request: I/O error, dev vda, sector 2644464 [ 284.447578] blk_update_request: I/O error, dev vda, sector 2645472 [ 284.448825] blk_update_request: I/O error, dev vda, sector 2646480 [ 284.450015] blk_update_request: I/O error, dev vda, sector 2647488 [ 284.451242] blk_update_request: I/O error, dev vda, sector 2648496 [ 284.452505] blk_update_request: I/O error, dev vda, sector 2649504 [ 284.942258] XFS (vda1): writeback error on sector 2556768 and-so-on. (note: No iscsi-errors in the virtual server log, none expected either but good to note.) On the hypervisor , I do see iscsi-errors , namely: [14153.160954] iSCSI/iqn.1994-05.com.redhat:44b41107f44: Unsupported SCSI Opcode 0xa3, sending CHECK_CONDITION. [14158.338313] iSCSI/iqn.1994-05.com.redhat:44b41107f44: Unsupported SCSI Opcode 0xa3, sending CHECK_CONDITION. [14162.583635] iSCSI/iqn.1994-05.com.redhat:44b41107f44: Unsupported SCSI Opcode 0xa3, sending CHECK_CONDITION. [17015.456426] iSCSI/iqn.1994-05.com.redhat:8565d64f61b1: Unsupported SCSI Opcode 0xa3, sending CHECK_CONDITION. [17086.182440] iSCSI/iqn.1994-05.com.redhat:8565d64f61b1: Unsupported SCSI Opcode 0xa3, sending CHECK_CONDITION. [17089.104229] iSCSI/iqn.1994-05.com.redhat:8565d64f61b1: Unsupported SCSI Opcode 0xa3, sending CHECK_CONDITION. As to question 2: I have tried with 10g, 20g, 40g 50g, 4T and 9T volumes. I have also tried with a simple AIO-cluster of 9T image, it worked fine when Wallaby was the newest, (cluster built about 2 months before Xena came out) The biggest problem is when I am trying to attach a 'data' volume to a vm and then create a filesystem on it, because both xfs and ext4 try to write a copy of the partition table to the last block. So far I have solved that by using lvm and simply creating a logical volume max 95% of the reported disk-size. Normal glance-image backed volumes are less affected, because it "grows" the filesystem from qcow2 and move on. so as long as disks dont run above 95% I am safe. Exception: When installing kubernetes-cluster through a heat-template , the "master" node also reports disk-errors whereas the 2 slave nodes attach cleanly. I would love to debug this issue because I recently started hosting kubernetes classrooms and the students complain sometimes. Kind Regards, Mark. Sofia Enriquez schreef op 2022-01-12 16:49: > Greetings Mark Olie, > I'd like to ask you the next questions: > - Is the vm allowed to read/write into the "extra" space? > - Is this scenario happening only for <20G or have you only tried with 20G? > Thanks in advance > > -- > You received this bug notification because you are subscribed to the bug > report. > https://bugs.launchpad.net/bugs/1956887 > > Title: > LVM backed ISCSI device not reporting same size to nova > > Status in Cinder: > New > > Bug description: > Environment: > > - Compute nodes: CentOS 8 Stream (latest) (Supermicro) > - Storage nodes: CentOS 8 Stream (latest)(Supermicro hardware with 18TB Storage hardware raid5) > - Controller nodes: CentOS 8 Stream (latest) (Supermicro) > > Openstack version: Wallaby, deployed by kolla-ansible not using > Ironic. > > When using LVM backed devices exposed to nova through the iscsi > protocol, the blocksize of the device differs. > > Example: > Using horizon, create a volume 20GB or bigger and attach it to a Virtual Machine. > > Exact device size in bytes gotten by running on the storage-node: > fdisk -l /dev/cinder-volumes/volume-UUID > 4398046511104 (Example 4 Terabyte volume on storage-node, in bytes) > > However, when using fdisk /dev/vda (attached to nova-instance on > compute node, inside the virtual machine) > > 4398066466816 (Example 4 Terabyte volume as seen by the virtual > machine, in bytes) > > Now if the sizes would be the other way around this would not be a > problem, but the VM disksize is bigger than the real disk size on the > iscsi-backed lvm volume. > > This is from a 20GB backed volume vm, because smaller vm disks are > affected sooner rather than later: > > Thus resulting in the following messages in the kernel log on the vm: > [111761.391344] blk_update_request: I/O error, dev vda, sector 17777976 > [111761.394839] blk_update_request: I/O error, dev vda, sector 17778984 > [111761.396241] blk_update_request: I/O error, dev vda, sector 17779992 > [111761.397782] blk_update_request: I/O error, dev vda, sector 17781000 > [111761.399343] blk_update_request: I/O error, dev vda, sector 17782008 > [111761.400929] blk_update_request: I/O error, dev vda, sector 17783016 > [111761.402189] blk_update_request: I/O error, dev vda, sector 17784024 > [111761.403377] blk_update_request: I/O error, dev vda, sector 17785032 > [111761.404569] blk_update_request: I/O error, dev vda, sector 17786040 > [111761.406165] blk_update_request: I/O error, dev vda, sector 17787048 > > Double-checked, by creating an All-in-One node from a storage-node so > network issues can be ruled out. > > Issue did not go away. > > To manage notifications about this bug go to: > https://bugs.launchpad.net/cinder/+bug/1956887/+subscriptions