| 2010-06-07 19:18:31 |
Chris Cheney |
bug |
|
|
added bug |
| 2010-06-07 19:20:46 |
Chris Cheney |
description |
When testing out bug 586134 I found another issue and although similar to bug 517086 appears not to be the same issue.
If you create for example 200 1GB volumes like this:
for i in `seq 1 200` ; do euca-create-volume -z cluster1 -s 1 ; done
You will end up with quite a few failed. If you look in the logs it is usually caused by vgcreate failing for unknown reasons as the output of the command is not logged. If you run the same command on the command line that failed it will then work.
Also if you then delete all the volumes like this:
for i in `euca-describe-volumes | awk '{print $2}'` ; do euca-delete-volume $i ; done
You will end up with a huge number that do not get removed. If you look in the logs it is usually caused by lvremove failing for unknown reasons like with vgcreate. Again if you run the same command on the command line that failed it will then work.
Chris |
When testing out bug 586134 I found another issue and although similar to bug 517086 appears not to be the same issue.
If you create for example 200 1GB volumes like this:
for i in `seq 1 200` ; do euca-create-volume -z cluster1 -s 1 ; done
You will end up with quite a few failed. If you look in the logs it is usually caused by vgcreate failing for unknown reasons as the output of the command is not logged. If you run the same command on the command line that failed it will then work.
Also if you then delete all the volumes like this:
for i in `euca-describe-volumes | awk '{print $2}'` ; do euca-delete-volume $i ; done
You will end up with a huge number that do not get removed. If you look in the logs it is usually caused by lvremove failing for unknown reasons like with vgcreate. Again if you run the same command on the command line that failed it will then work.
Also despite the failure to properly delete the volume euca-describe-volumes will have invalid information. For example if you delete the volume it will no longer show up in the list but it will still be there as seen by losetup and vgscan, and will still be set back up when rebooted by eucalyptus but it will not show up in the euca-describe-volume list.
Chris
|
|
| 2010-06-07 19:21:42 |
Chris Cheney |
description |
When testing out bug 586134 I found another issue and although similar to bug 517086 appears not to be the same issue.
If you create for example 200 1GB volumes like this:
for i in `seq 1 200` ; do euca-create-volume -z cluster1 -s 1 ; done
You will end up with quite a few failed. If you look in the logs it is usually caused by vgcreate failing for unknown reasons as the output of the command is not logged. If you run the same command on the command line that failed it will then work.
Also if you then delete all the volumes like this:
for i in `euca-describe-volumes | awk '{print $2}'` ; do euca-delete-volume $i ; done
You will end up with a huge number that do not get removed. If you look in the logs it is usually caused by lvremove failing for unknown reasons like with vgcreate. Again if you run the same command on the command line that failed it will then work.
Also despite the failure to properly delete the volume euca-describe-volumes will have invalid information. For example if you delete the volume it will no longer show up in the list but it will still be there as seen by losetup and vgscan, and will still be set back up when rebooted by eucalyptus but it will not show up in the euca-describe-volume list.
Chris
|
When testing out bug 586134 I found another issue and although similar to bug 517086 appears not to be the same issue.
If you create for example 200 1GB volumes like this:
for i in `seq 1 200` ; do euca-create-volume -z cluster1 -s 1 ; done
You will end up with quite a few failed. If you look in the logs it is usually caused by vgcreate failing for unknown reasons as the output of the command is not logged. If you run the same command on the command line that failed it will then work.
Also if you then delete all the volumes like this:
for i in `euca-describe-volumes | awk '{print $2}'` ; do euca-delete-volume $i ; done
You will end up with a huge number that do not get removed. If you look in the logs it is usually caused by lvremove failing for unknown reasons like with vgcreate. Again if you run the same command on the command line that failed it will then work.
Also despite the failure to properly delete the volume euca-describe-volumes will have invalid information. For example if you delete the volume it will no longer show up in the list but it will still be there as seen by losetup and vgscan, and will still be set back up by eucalyptus when rebooted but it will not show up in the euca-describe-volume list.
Chris
|
|
| 2010-06-07 19:22:18 |
C de-Avillez |
eucalyptus (Ubuntu): importance |
Undecided |
Medium |
|
| 2010-06-07 19:22:18 |
C de-Avillez |
eucalyptus (Ubuntu): status |
New |
Confirmed |
|
| 2010-06-07 19:22:38 |
Chris Cheney |
nominated for series |
|
Ubuntu Lucid |
|
| 2010-06-07 19:22:38 |
Chris Cheney |
bug task added |
|
eucalyptus (Ubuntu Lucid) |
|
| 2010-06-07 19:22:53 |
Chris Cheney |
eucalyptus (Ubuntu Lucid): importance |
Undecided |
Medium |
|
| 2010-06-07 19:22:56 |
Chris Cheney |
eucalyptus (Ubuntu Lucid): status |
New |
Confirmed |
|
| 2010-07-07 20:15:23 |
mahmoh |
bug |
|
|
added subscriber Massimo Morana |
| 2010-08-18 16:10:43 |
Dustin Kirkland |
bug task added |
|
eucalyptus |
|
| 2010-08-18 16:52:01 |
mahmoh |
bug task added |
|
jaguarundi |
|
| 2010-08-18 16:52:24 |
mahmoh |
jaguarundi: status |
New |
Confirmed |
|
| 2010-08-18 17:46:12 |
Neil Soman |
eucalyptus: status |
New |
Invalid |
|
| 2010-08-19 14:18:09 |
mahmoh |
jaguarundi: importance |
Undecided |
High |
|
| 2010-08-19 14:18:18 |
mahmoh |
jaguarundi: milestone |
|
sales-training |
|
| 2010-08-19 14:21:16 |
mahmoh |
jaguarundi: milestone |
sales-training |
jaguar-ga |
|
| 2010-08-19 14:37:33 |
mahmoh |
summary |
eucalyptus create and delete volumes sometimes fail on lvm commands |
eucalyptus create and delete volumes sometimes fail on lvm commands (POC + ENT configs) |
|
| 2010-08-29 18:02:03 |
Neil Soman |
branch linked |
|
lp:eucalyptus |
|
| 2010-08-29 18:02:09 |
Neil Soman |
eucalyptus: status |
Invalid |
In Progress |
|
| 2010-08-29 18:05:36 |
Neil Soman |
nominated for series |
|
eucalyptus/1.6.2 |
|
| 2010-08-29 18:05:36 |
Neil Soman |
bug task added |
|
eucalyptus/1.6.2 |
|
| 2010-08-29 18:05:49 |
Neil Soman |
eucalyptus/1.6.2: status |
In Progress |
Fix Committed |
|
| 2010-08-29 18:06:04 |
Neil Soman |
eucalyptus/1.6.2: assignee |
|
Neil Soman (neilsoman) |
|
| 2010-09-01 23:10:10 |
Dustin Kirkland |
eucalyptus (Ubuntu): status |
Confirmed |
Fix Released |
|
| 2010-09-01 23:16:27 |
Dustin Kirkland |
eucalyptus (Ubuntu Lucid): status |
Confirmed |
Triaged |
|
| 2010-09-01 23:16:36 |
Dustin Kirkland |
eucalyptus (Ubuntu Lucid): assignee |
|
Dustin Kirkland (kirkland) |
|
| 2010-09-01 23:16:43 |
Dustin Kirkland |
jaguarundi: assignee |
|
Dustin Kirkland (kirkland) |
|
| 2010-09-01 23:16:47 |
Dustin Kirkland |
jaguarundi: status |
Confirmed |
Triaged |
|
| 2010-09-02 20:15:39 |
Dustin Kirkland |
jaguarundi: status |
Triaged |
In Progress |
|
| 2010-09-02 20:15:45 |
Dustin Kirkland |
eucalyptus (Ubuntu Lucid): status |
Triaged |
In Progress |
|
| 2010-09-02 23:10:16 |
Dustin Kirkland |
summary |
eucalyptus create and delete volumes sometimes fail on lvm commands (POC + ENT configs) |
[SRU] eucalyptus create and delete volumes sometimes fail on lvm commands (POC + ENT configs) |
|
| 2010-09-02 23:19:35 |
Dustin Kirkland |
description |
When testing out bug 586134 I found another issue and although similar to bug 517086 appears not to be the same issue.
If you create for example 200 1GB volumes like this:
for i in `seq 1 200` ; do euca-create-volume -z cluster1 -s 1 ; done
You will end up with quite a few failed. If you look in the logs it is usually caused by vgcreate failing for unknown reasons as the output of the command is not logged. If you run the same command on the command line that failed it will then work.
Also if you then delete all the volumes like this:
for i in `euca-describe-volumes | awk '{print $2}'` ; do euca-delete-volume $i ; done
You will end up with a huge number that do not get removed. If you look in the logs it is usually caused by lvremove failing for unknown reasons like with vgcreate. Again if you run the same command on the command line that failed it will then work.
Also despite the failure to properly delete the volume euca-describe-volumes will have invalid information. For example if you delete the volume it will no longer show up in the list but it will still be there as seen by losetup and vgscan, and will still be set back up by eucalyptus when rebooted but it will not show up in the euca-describe-volume list.
Chris
|
When testing out bug 586134 I found another issue and although similar to bug 517086 appears not to be the same issue.
If you create for example 200 1GB volumes like this:
for i in `seq 1 200` ; do euca-create-volume -z cluster1 -s 1 ; done
You will end up with quite a few failed. If you look in the logs it is usually caused by vgcreate failing for unknown reasons as the output of the command is not logged. If you run the same command on the command line that failed it will then work.
Also if you then delete all the volumes like this:
for i in `euca-describe-volumes | awk '{print $2}'` ; do euca-delete-volume $i ; done
You will end up with a huge number that do not get removed. If you look in the logs it is usually caused by lvremove failing for unknown reasons like with vgcreate. Again if you run the same command on the command line that failed it will then work.
Also despite the failure to properly delete the volume euca-describe-volumes will have invalid information. For example if you delete the volume it will no longer show up in the list but it will still be there as seen by losetup and vgscan, and will still be set back up by eucalyptus when rebooted but it will not show up in the euca-describe-volume list.
Chris
===============
SRU INFORMATION
IMPACT:
This bug affects users of UEC, trying to create EBS volumes using the euca-create-volume command. In the current packages, this command will fail from time to time, for no apparent or reported reason, and Eucalyptus does not recover gracefully and retry the volume creation. This is a problem that heavy UEC users will probably encounter at some point in time.
HOW ADDRESSED:
This bug has been addressed by the upstream Eucalyptus maintainer of this code. The Eucalyptus 2.0 branch (in Maverick) does not suffer from this bug, as much of the storage related code has been refactored in the interest of scalability. Neil Soman has graciously backported the minimal subset of those changes necessary to address this bug. Still the patch is a few hundred lines. But it's all in the interest of solving this bug.
PATCH:
See bzr revisions r1235 and r1236 of lp:eucalyptus (upstream Eucalyptus branch).
REPRODUCE:
To reproduce this bug, simply:
for i in $(seq 1 40); do euca-create-volume -z cluster1 -s 1; done
sleep 10
euca-describe-volumes | grep fail
euca-describe-volumes | awk '{print $2}' | xargs -i euca-delete-volume {}
I see failures trying to create a few as 40 volumes. (Note that I chose 40, as I have 50GB of disk free). We would expect being able to do the above ad nauseum, without unhandled errors.
REGRESSION POTENTIAL:
There is perhaps some chance of regression in the storage related functionality in Eucalyptus. For this reason, this particular SRU package will be heavily tested by Canonical OEM Server QA as well as Ubuntu Platform QA and Ubuntu Platform Server team members.
=============== |
|
| 2010-09-02 23:22:09 |
Dustin Kirkland |
eucalyptus (Ubuntu Lucid): milestone |
|
ubuntu-10.04.2 |
|
| 2010-09-02 23:22:23 |
Dustin Kirkland |
bug |
|
|
added subscriber Ubuntu Stable Release Updates Team |
| 2010-09-08 10:13:01 |
Colin Watson |
eucalyptus (Ubuntu Lucid): status |
In Progress |
Fix Committed |
|
| 2010-09-08 10:13:10 |
Colin Watson |
bug |
|
|
added subscriber SRU Verification |
| 2010-09-08 10:13:14 |
Colin Watson |
tags |
|
verification-needed |
|
| 2010-09-08 11:16:20 |
Launchpad Janitor |
branch linked |
|
lp:ubuntu/lucid-proposed/eucalyptus |
|
| 2010-09-09 12:18:08 |
Sylvain Pineau |
eucalyptus (Ubuntu): status |
Fix Released |
Confirmed |
|
| 2010-09-09 12:19:08 |
Sylvain Pineau |
jaguarundi: status |
In Progress |
Confirmed |
|
| 2010-09-14 16:06:54 |
mahmoh |
jaguarundi: status |
Confirmed |
Fix Released |
|
| 2010-09-14 16:14:21 |
mahmoh |
tags |
verification-needed |
verification-done |
|
| 2010-09-16 14:13:25 |
Dustin Kirkland |
eucalyptus (Ubuntu): status |
Confirmed |
Fix Released |
|
| 2010-09-16 16:28:55 |
Launchpad Janitor |
eucalyptus (Ubuntu Lucid): status |
Fix Committed |
Fix Released |
|
| 2011-10-06 12:37:54 |
Andy Grimm |
eucalyptus: status |
Fix Committed |
Fix Released |
|
| 2011-10-28 20:53:12 |
graziano obertelli |
eucalyptus/1.6.2: status |
Fix Committed |
Fix Released |
|
