ubuntu_nbd_smoke_test failed on some B-AWS instances

[Po-Hsu Lin]

Running '/home/ubuntu/autotest/client/tests/ubuntu_nbd_smoke_test/ubuntu_nbd_smoke_test.sh'
 done

 --------------------------------------------------------------------------------
 Image path: /tmp/nbd_image.img
 Mount point: /mnt/nbd-test-7666
 Date: Tue Nov 13 06:20:46 UTC 2018
 Host: ip-172-31-14-117
 Kernel: 4.15.0-1027-aws #27-Ubuntu SMP Fri Nov 2 15:14:20 UTC 2018
 Machine: ip-172-31-14-117 x86_64 x86_64
 CPUs online: 32
 CPUs total: 32
 Page size: 4096
 Pages avail: 15126865
 Pages total: 15454667
 --------------------------------------------------------------------------------

 tcp6 0 0 :::9999 :::* LISTEN 7884/nbd-server
 mount: /mnt/nbd-test-7666: wrong fs type, bad option, bad superblock on /dev/nbd0, missing codepage or helper program, or other error.
 mount: /mnt/nbd-test-7666: wrong fs type, bad option, bad superblock on /dev/nbd0, missing codepage or helper program, or other error.
 mount: /mnt/nbd-test-7666: wrong fs type, bad option, bad superblock on /dev/nbd0, missing codepage or helper program, or other error.
 mount: /mnt/nbd-test-7666: wrong fs type, bad option, bad superblock on /dev/nbd0, missing codepage or helper program, or other error.
 mount: /mnt/nbd-test-7666: wrong fs type, bad option, bad superblock on /dev/nbd0, missing codepage or helper program, or other error.
 mount: /mnt/nbd-test-7666: wrong fs type, bad option, bad superblock on /dev/nbd0, missing codepage or helper program, or other error.
 mount: /mnt/nbd-test-7666: wrong fs type, bad option, bad superblock on /dev/nbd0, missing codepage or helper program, or other error.
 mount: /mnt/nbd-test-7666: wrong fs type, bad option, bad superblock on /dev/nbd0, missing codepage or helper program, or other error.
 mount: /mnt/nbd-test-7666: wrong fs type, bad option, bad superblock on /dev/nbd0, missing codepage or helper program, or other error.
 mount: /mnt/nbd-test-7666: wrong fs type, bad option, bad superblock on /dev/nbd0, missing codepage or helper program, or other error.
 mount: /mnt/nbd-test-7666: wrong fs type, bad option, bad superblock on /dev/nbd0, missing codepage or helper program, or other error.
 mount: /mnt/nbd-test-7666: wrong fs type, bad option, bad superblock on /dev/nbd0, missing codepage or helper program, or other error.
 mount: /mnt/nbd-test-7666: wrong fs type, bad option, bad superblock on /dev/nbd0, missing codepage or helper program, or other error.
 mount: /mnt/nbd-test-7666: wrong fs type, bad option, bad superblock on /dev/nbd0, missing codepage or helper program, or other error.
 mount: /mnt/nbd-test-7666: wrong fs type, bad option, bad superblock on /dev/nbd0, missing codepage or helper program, or other error.
 mount: /mnt/nbd-test-7666: wrong fs type, bad option, bad superblock on /dev/nbd0, missing codepage or helper program, or other error.
 mount: /mnt/nbd-test-7666: wrong fs type, bad option, bad superblock on /dev/nbd0, missing codepage or helper program, or other error.
 mount: /mnt/nbd-test-7666: wrong fs type, bad option, bad superblock on /dev/nbd0, missing codepage or helper program, or other error.
 mount: /mnt/nbd-test-7666: wrong fs type, bad option, bad superblock on /dev/nbd0, missing codepage or helper program, or other error.
 mount: /mnt/nbd-test-7666: wrong fs type, bad option, bad superblock on /dev/nbd0, missing codepage or helper program, or other error.
 mount: /mnt/nbd-test-7666: wrong fs type, bad option, bad superblock on /dev/nbd0, missing codepage or helper program, or other error.
 mount: /mnt/nbd-test-7666: wrong fs type, bad option, bad superblock on /dev/nbd0, missing codepage or helper program, or other error.
 mount on /dev/nbd0 failed
 umount: /mnt/nbd-test-7666: not mounted.
 disconnect, sock, done

Failed on:
 * c3.8xlarge
 * g2.8xlarge
 * m4.16xlarge
 * r3.8xlarge
 * r4.16xlarge
 * x1.16xlarge
 * x1.32xlarge

[Colin Ian King (colin-king)]

I can't reproduce this when running on a cleanly provisioned x1.32xlarge box, so there must be something else triggering this when running the regression tests.

[Sean Feole (sfeole)]

Hi Colin/Sam.. This bug is an older one, and since this bug has been filed we have updated our instance storage types with the introduction of multi-region testing. I believe this may have been discovered by Colin that the bug was failing due to particular instances running out of disk space.

We no longer test many of the instance listed in this bug.
These instances all used a default 8GB EBS storage type as their root filesystem, that can be adjusted if required.

Anything extra discovered attached to these instances is most likely added after the fact. although I'm still trying to make sense of such a large filesystem discovered (/mnt ~~300GB).

I also have not seen this test fail in the last SRU Cycle (aws : 4.15.0-1027.27 -- Bionic ) on any of our newly proposed instances. I think we can safely close this for now.

[Sean Feole (sfeole)]

When I mentioned this bug was older, I meant, prior to our Major changes introduced last week.

[Colin Ian King (colin-king)]

On 19/11/2018 18:45, Sean Feole wrote:
> Hi Colin/Sam.. This bug is an older one, and since this bug has been
> filed we have updated our instance storage types with the introduction
> of multi-region testing. I believe this may have been discovered by
> Colin that the bug was failing due to particular instances running out
> of disk space.
>
> We no longer test many of the instance listed in this bug.
> These instances all used a default 8GB EBS storage type as their root filesystem, that can be adjusted if required.

Bumping up the space is indeed a workable short term solution. I still
think it's prudent to ensure large repos cloned by tests are removed
after a test has completed to make sure we don't run out of space in the
future.

>
> Anything extra discovered attached to these instances is most likely
> added after the fact. although I'm still trying to make sense of such a
> large filesystem discovered (/mnt ~~300GB).
>
>
> I also have not seen this test fail in the last SRU Cycle (aws : 4.15.0-1027.27 -- Bionic ) on any of our newly proposed instances. I think we can safely close this for now.

OK - that's +1 from me too.

>