intermittent generic back end extend/shrink share failures

Bug #1816430 reported by Tom Barron on 2019-02-18
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Manila
Low
Unassigned

Bug Description

generic back end jobs are failing intermittently on extend and shrink tests. manila share service log shows tracebacks like the following:

Feb 18 12:20:09.801776 ubuntu-bionic-inap-mtl01-0002671090 manila-share[31033]: ERROR manila.share.manager [None req-21f46801-420e-43d4-be60-3eae44d46406 None None] Extend share failed.: ProcessExecutionError: Unexpected error while running command.
Feb 18 12:20:09.801776 ubuntu-bionic-inap-mtl01-0002671090 manila-share[31033]: Command: sudo fsck -pf /dev/vdd
Feb 18 12:20:09.801776 ubuntu-bionic-inap-mtl01-0002671090 manila-share[31033]: Exit code: 8
Feb 18 12:20:09.801776 ubuntu-bionic-inap-mtl01-0002671090 manila-share[31033]: Stdout: u'fsck from util-linux 2.31.1\nPossibly non-existent device?\n'
Feb 18 12:20:09.801776 ubuntu-bionic-inap-mtl01-0002671090 manila-share[31033]: Stderr: u'fsck.ext2: No such file or directory while trying to open /dev/vdd\n'
Feb 18 12:20:09.801776 ubuntu-bionic-inap-mtl01-0002671090 manila-share[31033]: ERROR manila.share.manager Traceback (most recent call last):
Feb 18 12:20:09.801776 ubuntu-bionic-inap-mtl01-0002671090 manila-share[31033]: ERROR manila.share.manager File "/opt/stack/new/manila/manila/share/manager.py", line 3626, in extend_share
Feb 18 12:20:09.801776 ubuntu-bionic-inap-mtl01-0002671090 manila-share[31033]: ERROR manila.share.manager share_instance, new_size, share_server=share_server)
Feb 18 12:20:09.801776 ubuntu-bionic-inap-mtl01-0002671090 manila-share[31033]: ERROR manila.share.manager File "/opt/stack/new/manila/manila/share/drivers/generic.py", line 117, in wrap
Feb 18 12:20:09.801776 ubuntu-bionic-inap-mtl01-0002671090 manila-share[31033]: ERROR manila.share.manager return f(self, context, *args, **kwargs)
Feb 18 12:20:09.801776 ubuntu-bionic-inap-mtl01-0002671090 manila-share[31033]: ERROR manila.share.manager File "/opt/stack/new/manila/manila/share/drivers/generic.py", line 670, in extend_share
Feb 18 12:20:09.801776 ubuntu-bionic-inap-mtl01-0002671090 manila-share[31033]: ERROR manila.share.manager self._resize_filesystem(server_details, volume)
Feb 18 12:20:09.801776 ubuntu-bionic-inap-mtl01-0002671090 manila-share[31033]: ERROR manila.share.manager File "/opt/stack/new/manila/manila/share/drivers/generic.py", line 727, in _resize_filesystem
Feb 18 12:20:09.801776 ubuntu-bionic-inap-mtl01-0002671090 manila-share[31033]: ERROR manila.share.manager self._ssh_exec(server_details, check_command)
Feb 18 12:20:09.801776 ubuntu-bionic-inap-mtl01-0002671090 manila-share[31033]: ERROR manila.share.manager File "/opt/stack/new/manila/manila/share/drivers/generic.py", line 171, in _ssh_exec
Feb 18 12:20:09.801776 ubuntu-bionic-inap-mtl01-0002671090 manila-share[31033]: ERROR manila.share.manager check_exit_code=check_exit_code)
Feb 18 12:20:09.801776 ubuntu-bionic-inap-mtl01-0002671090 manila-share[31033]: ERROR manila.share.manager File "/usr/local/lib/python2.7/dist-packages/oslo_concurrency/processutils.py", line 570, in ssh_execute
Feb 18 12:20:09.801776 ubuntu-bionic-inap-mtl01-0002671090 manila-share[31033]: ERROR manila.share.manager cmd=sanitized_cmd)
Feb 18 12:20:09.801776 ubuntu-bionic-inap-mtl01-0002671090 manila-share[31033]: ERROR manila.share.manager ProcessExecutionError: Unexpected error while running command.
Feb 18 12:20:09.801776 ubuntu-bionic-inap-mtl01-0002671090 manila-share[31033]: ERROR manila.share.manager Command: sudo fsck -pf /dev/vdd
Feb 18 12:20:09.801776 ubuntu-bionic-inap-mtl01-0002671090 manila-share[31033]: ERROR manila.share.manager Exit code: 8
Feb 18 12:20:09.803921 ubuntu-bionic-inap-mtl01-0002671090 manila-share[31033]: ERROR manila.share.manager Stdout: u'fsck from util-linux 2.31.1\nPossibly non-existent device?\n'
Feb 18 12:20:09.803921 ubuntu-bionic-inap-mtl01-0002671090 manila-share[31033]: ERROR manila.share.manager Stderr: u'fsck.ext2: No such file or directory while trying to open /dev/vdd\n'

Tom Barron (tpb) on 2019-02-18
tags: added: driver generic
Jason Grosso (jgrosso) on 2019-03-14
Changed in manila:
status: New → Triaged
Jason Grosso (jgrosso) on 2019-03-21
Changed in manila:
importance: Undecided → Medium
Tom Barron (tpb) on 2019-09-19
Changed in manila:
importance: Medium → Low
Tom Barron (tpb) wrote :

Setting Importance to Low since we aren't getting reports on this issue from the field and we haven't seen it in CI lately. If you hit this issue please update this Bug and bump the priority if it's justified.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers