flaky log ERRORs in c-vol after successful tempest run

Bug #1243840 reported by David Kranz
14
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Cinder
Fix Released
Undecided
John Griffith

Bug Description

These showed up 9 times out of 156 runs and are of several varieties. Here is a sample of each:

https://review.openstack.org/#/c/51966/7
http://logs.openstack.org/66/51966/7/check/check-tempest-devstack-vm-postgres-full/a884365
2013-10-22 21:06:08.354 | Log File: c-vol
2013-10-22 21:06:08.355 | 2013-10-22 20:57:49.993 24174 ERROR cinder.brick.iscsi.iscsi [req-077902c7-a5e6-414a-b241-2f429766a569 c6dbf14b4d794430bb3b20395dad67d7 44b8cb996d984ea0aba77a30fa9531d0] Failed to create iscsi target for volume id:volume-52749038-fe54-4f93-9be1-50c480df4b6a: Unexpected error while running command.

https://review.openstack.org/#/c/47712/4
http://logs.openstack.org/12/47712/4/check/check-tempest-devstack-vm-neutron/843f8e8
2013-10-22 23:35:40.881 | Log File: c-vol
2013-10-22 23:35:40.882 | 2013-10-22 23:29:57.004 6308 ERROR cinder.brick.local_dev.lvm [req-4e6c087e-c345-4072-9c18-0f39861680d9 75f6a750c42745ca9440629f79dbaa18 21a7f4942ba7490d8198acf0510fe4fe] Error reported running lvremove: CMD: sudo cinder-rootwrap /etc/cinder/rootwrap.conf lvremove -f stack-volumes/volume-472b303e-7d40-4956-859e-e91506947641, RESPONSE: /dev/dm-1: stat failed: No such file or directory

https://review.openstack.org/#/c/53255/1
http://logs.openstack.org/55/53255/1/check/check-tempest-devstack-vm-neutron-pg/2319aee
2013-10-23 00:38:37.188 | Log File: c-vol
2013-10-23 00:38:37.188 | 2013-10-23 00:32:36.192 4619 ERROR cinder.brick.local_dev.lvm [req-025aa642-42ac-4a14-8b5c-0165c6c2c268 6105fe9f840049fba93305e06d8cab2c c085cabe4d8b46a19b35708f4ec1c21a] Error reported running lvremove: CMD: sudo cinder-rootwrap /etc/cinder/rootwrap.conf lvremove -f stack-volumes/volume-f0845e0f-2fbb-4d68-879d-ea21a10d152a, RESPONSE: Can't remove open logical volume "volume-f0845e0f-2fbb-4d68-879d-ea21a10d152a"

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to cinder (master)

Fix proposed to branch: master
Review: https://review.openstack.org/53440

Changed in cinder:
assignee: nobody → John Griffith (john-griffith)
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to cinder (master)

Reviewed: https://review.openstack.org/53440
Committed: http://github.com/openstack/cinder/commit/587fd9021982e3b1fb0c5765df6bad2723701031
Submitter: Jenkins
Branch: master

commit 587fd9021982e3b1fb0c5765df6bad2723701031
Author: John Griffith <email address hidden>
Date: Wed Oct 23 20:00:35 2013 +0000

    Downgrade target create failure mesg to warning

    Since this will occur more often than we would like and
    almost always recovers, change the log level to warning
    instead of error.

    This is to comply with the desire of the QA team to eradicate
    error messages from succesful tempest runs.

    Change-Id: If1a841bbeb9a36ff7e103a0e9aff67fda66a7f3b
    Closes-Bug: #1243840

Changed in cinder:
status: In Progress → Fix Committed
Revision history for this message
Matt Riedemann (mriedem) wrote :

Seeing "stat failed: No such file or directory" here:

http://logs.openstack.org/26/54026/1/gate/gate-tempest-devstack-vm-full/9ba5443/logs/screen-c-vol.txt.gz

ERROR cinder.brick.local_dev.lvm [req-c9fa047b-de14-416c-a1bd-2c6783ac3608 755dd31acc6f4eaba3da5f1cd85a4de2 ef8d0a0996f6403a853b19e7ae48b38b] Error reported running lvremove: CMD: sudo cinder-rootwrap /etc/cinder/rootwrap.conf lvremove -f stack-volumes/volume-8529daaa-1aee-4d02-97b0-3ca7ffa2b77e, RESPONSE: /dev/dm-1: stat failed: No such file or directory

Thierry Carrez (ttx)
Changed in cinder:
milestone: none → icehouse-1
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in cinder:
milestone: icehouse-1 → 2014.1
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.