VMBuilder

The directory ubuntu-kvm disappeared after creating vms failed

Bug #1090223 reported by whuang
30
This bug affects 5 people
Affects Status Importance Assigned to Milestone
VMBuilder
New
Undecided
Unassigned
vm-builder (Ubuntu)
Confirmed
High
Unassigned

Bug Description

Previously I have several VMs created by vmbuilder. Their disks are all in the directory "ubuntu-kvm".

Yesterday, I upgraded the server from 10.04 to 12.04. Then I tried to create a new VM by vmbuilder with the below cmd, but after executing it, the directory "ubuntu-kvm" disappered! Does it mean that all my previous VMs are deleted? This is severe! Is there a way to recover the VMs.

Logs:
whuang:~/virtfs$ ls
ubuntu-kvm vmbuilder.partition
whuang:~/virtfs$ sudo vmbuilder kvm ubuntu --suite precise --flavour virtual --arch i386 -o \
> --libvirt qemu:///system --ip 10.65.31.90 --mask 255.255.255.0 --gw 10.65.31.1 --hostname vm180-vm \
> --dns 10.64.96.32 --bridge virbr0 \
> --part vmbuilder.partition --user aaa --name aaa --pass aaa \
> --addpkg openssh-server \
> --addpkg unattended-upgrades --addpkg acpid --ppa nijaba
[sudo] password for whuang:
2012-12-13 18:41:29,568 INFO : Calling hook: preflight_check
2012-12-13 18:41:29,574 INFO : Calling hook: set_defaults
2012-12-13 18:41:29,575 INFO : Calling hook: bootstrap
2012-12-13 19:51:34,628 INFO : Calling hook: configure_os
2012-12-13 19:55:17,035 INFO : W: GPG error: http://ppa.launchpad.net precise Release: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 1A64983BDB2D6F62
2012-12-13 19:55:17,035 INFO : W: Failed to fetch bzip2:/var/lib/apt/lists/partial/archive.ubuntu.com_ubuntu_dists_precise-updates_universe_binary-i386_Packages Hash Sum mismatch
2012-12-13 19:55:17,035 INFO :
2012-12-13 19:55:17,035 INFO : E: Some index files failed to download. They have been ignored, or old ones used instead.
2012-12-13 19:55:17,036 INFO : Cleaning up
2012-12-13 19:55:17,036 ERROR : Process (['chroot', '/tmp/tmpzhjUVq', 'apt-get', 'update']) returned 100. stdout: Ign http://security.ubuntu.com precise-security InRelease
Ign http://ppa.launchpad.net precise InRelease
Get:1 http://security.ubuntu.com precise-security Release.gpg [198 B]
Get:2 http://ppa.launchpad.net precise Release.gpg [316 B]
Get:3 http://security.ubuntu.com precise-security Release [49.6 kB]
Get:4 http://ppa.launchpad.net precise Release [11.9 kB]
Ign http://ppa.launchpad.net precise Release
Get:5 http://ppa.launchpad.net precise/main i386 Packages [801 B]
Ign http://ppa.launchpad.net precise/main TranslationIndex
Get:6 http://security.ubuntu.com precise-security/main i386 Packages [215 kB]
Ign http://ppa.launchpad.net precise/main Translation-en
Ign http://archive.ubuntu.com precise InRelease
Ign http://archive.ubuntu.com precise-updates InRelease
Hit http://archive.ubuntu.com precise Release.gpg
Get:7 http://archive.ubuntu.com precise-updates Release.gpg [198 B]
Hit http://archive.ubuntu.com precise Release
Get:8 http://archive.ubuntu.com precise-updates Release [49.6 kB]
Get:9 http://security.ubuntu.com precise-security/restricted i386 Packages [3968 B]
Get:10 http://security.ubuntu.com precise-security/universe i386 Packages [59.1 kB]
Hit http://archive.ubuntu.com precise/main i386 Packages
Get:11 http://archive.ubuntu.com precise/restricted i386 Packages [8431 B]
Get:12 http://archive.ubuntu.com precise/universe i386 Packages [4796 kB]
Get:13 http://security.ubuntu.com precise-security/main TranslationIndex [74 B]
Get:14 http://security.ubuntu.com precise-security/restricted TranslationIndex [71 B]
Get:15 http://security.ubuntu.com precise-security/universe TranslationIndex [73 B]
Get:16 http://security.ubuntu.com precise-security/main Translation-en [102 kB]
Get:17 http://security.ubuntu.com precise-security/restricted Translation-en [978 B]
Get:18 http://security.ubuntu.com precise-security/universe Translation-en [35.6 kB]
Get:19 http://archive.ubuntu.com precise/main TranslationIndex [3706 B]
Get:20 http://archive.ubuntu.com precise/restricted TranslationIndex [2596 B]
Get:21 http://archive.ubuntu.com precise/universe TranslationIndex [2922 B]
Get:22 http://archive.ubuntu.com precise-updates/main i386 Packages [449 kB]
Get:23 http://archive.ubuntu.com precise-updates/restricted i386 Packages [8374 B]
Get:24 http://archive.ubuntu.com precise-updates/main TranslationIndex [3564 B]
Get:25 http://archive.ubuntu.com precise-updates/restricted TranslationIndex [2461 B]
Get:26 http://archive.ubuntu.com precise-updates/universe TranslationIndex [2850 B]
Get:27 http://archive.ubuntu.com precise/restricted Translation-en [2395 B]
Get:28 http://archive.ubuntu.com precise/universe Translation-en [3341 kB]
Get:29 http://archive.ubuntu.com precise-updates/universe i386 Packages [161 kB]
Get:30 http://archive.ubuntu.com precise-updates/restricted Translation-en [2082 B]
Get:31 http://archive.ubuntu.com precise/main Translation-en [726 kB]
Get:32 http://archive.ubuntu.com precise-updates/universe Translation-en [96.3 kB]
Get:33 http://archive.ubuntu.com precise-updates/main Translation-en [216 kB]
Fetched 10.4 MB in 3min 42s (46.6 kB/s)
, stderr: W: GPG error: http://ppa.launchpad.net precise Release: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 1A64983BDB2D6F62
W: Failed to fetch bzip2:/var/lib/apt/lists/partial/archive.ubuntu.com_ubuntu_dists_precise-updates_universe_binary-i386_Packages Hash Sum mismatch

E: Some index files failed to download. They have been ignored, or old ones used instead.

Traceback (most recent call last):
  File "/usr/bin/vmbuilder", line 24, in <module>
    cli.main()
  File "/usr/lib/python2.7/dist-packages/VMBuilder/contrib/cli.py", line 216, in main
    distro.build_chroot()
  File "/usr/lib/python2.7/dist-packages/VMBuilder/distro.py", line 84, in build_chroot
    self.call_hooks('configure_os')
  File "/usr/lib/python2.7/dist-packages/VMBuilder/distro.py", line 67, in call_hooks
    call_hooks(self, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/VMBuilder/util.py", line 165, in call_hooks
    getattr(context, func, log_no_such_method)(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/VMBuilder/plugins/ubuntu/distro.py", line 140, in configure_os
    self.suite.install_sources_list()
  File "/usr/lib/python2.7/dist-packages/VMBuilder/plugins/ubuntu/dapper.py", line 241, in install_sources_list
    self.run_in_target('apt-get', 'update', ignore_fail=final)
  File "/usr/lib/python2.7/dist-packages/VMBuilder/plugins/ubuntu/dapper.py", line 327, in run_in_target
    return self.context.run_in_target(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/VMBuilder/plugins/__init__.py", line 86, in run_in_target
    return util.run_cmd('chroot', self.chroot_dir, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/VMBuilder/util.py", line 120, in run_cmd
    raise VMBuilderException, "Process (%s) returned %d. stdout: %s, stderr: %s" % (args.__repr__(), status, mystdout.buf, mystderr.buf)
VMBuilder.exception.VMBuilderException: Process (['chroot', '/tmp/tmpzhjUVq', 'apt-get', 'update']) returned 100. stdout: Ign http://security.ubuntu.com precise-security InRelease
Ign http://ppa.launchpad.net precise InRelease
Get:1 http://security.ubuntu.com precise-security Release.gpg [198 B]
Get:2 http://ppa.launchpad.net precise Release.gpg [316 B]
Get:3 http://security.ubuntu.com precise-security Release [49.6 kB]
Get:4 http://ppa.launchpad.net precise Release [11.9 kB]
Ign http://ppa.launchpad.net precise Release
Get:5 http://ppa.launchpad.net precise/main i386 Packages [801 B]
Ign http://ppa.launchpad.net precise/main TranslationIndex
Get:6 http://security.ubuntu.com precise-security/main i386 Packages [215 kB]
Ign http://ppa.launchpad.net precise/main Translation-en
Ign http://archive.ubuntu.com precise InRelease
Ign http://archive.ubuntu.com precise-updates InRelease
Hit http://archive.ubuntu.com precise Release.gpg
Get:7 http://archive.ubuntu.com precise-updates Release.gpg [198 B]
Hit http://archive.ubuntu.com precise Release
Get:8 http://archive.ubuntu.com precise-updates Release [49.6 kB]
Get:9 http://security.ubuntu.com precise-security/restricted i386 Packages [3968 B]
Get:10 http://security.ubuntu.com precise-security/universe i386 Packages [59.1 kB]
Hit http://archive.ubuntu.com precise/main i386 Packages
Get:11 http://archive.ubuntu.com precise/restricted i386 Packages [8431 B]
Get:12 http://archive.ubuntu.com precise/universe i386 Packages [4796 kB]
Get:13 http://security.ubuntu.com precise-security/main TranslationIndex [74 B]
Get:14 http://security.ubuntu.com precise-security/restricted TranslationIndex [71 B]
Get:15 http://security.ubuntu.com precise-security/universe TranslationIndex [73 B]
Get:16 http://security.ubuntu.com precise-security/main Translation-en [102 kB]
Get:17 http://security.ubuntu.com precise-security/restricted Translation-en [978 B]
Get:18 http://security.ubuntu.com precise-security/universe Translation-en [35.6 kB]
Get:19 http://archive.ubuntu.com precise/main TranslationIndex [3706 B]
Get:20 http://archive.ubuntu.com precise/restricted TranslationIndex [2596 B]
Get:21 http://archive.ubuntu.com precise/universe TranslationIndex [2922 B]
Get:22 http://archive.ubuntu.com precise-updates/main i386 Packages [449 kB]
Get:23 http://archive.ubuntu.com precise-updates/restricted i386 Packages [8374 B]
Get:24 http://archive.ubuntu.com precise-updates/main TranslationIndex [3564 B]
Get:25 http://archive.ubuntu.com precise-updates/restricted TranslationIndex [2461 B]
Get:26 http://archive.ubuntu.com precise-updates/universe TranslationIndex [2850 B]
Get:27 http://archive.ubuntu.com precise/restricted Translation-en [2395 B]
Get:28 http://archive.ubuntu.com precise/universe Translation-en [3341 kB]
Get:29 http://archive.ubuntu.com precise-updates/universe i386 Packages [161 kB]
Get:30 http://archive.ubuntu.com precise-updates/restricted Translation-en [2082 B]
Get:31 http://archive.ubuntu.com precise/main Translation-en [726 kB]
Get:32 http://archive.ubuntu.com precise-updates/universe Translation-en [96.3 kB]
Get:33 http://archive.ubuntu.com precise-updates/main Translation-en [216 kB]
Fetched 10.4 MB in 3min 42s (46.6 kB/s)
, stderr: W: GPG error: http://ppa.launchpad.net precise Release: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 1A64983BDB2D6F62
W: Failed to fetch bzip2:/var/lib/apt/lists/partial/archive.ubuntu.com_ubuntu_dists_precise-updates_universe_binary-i386_Packages Hash Sum mismatch

E: Some index files failed to download. They have been ignored, or old ones used instead.

whuang:~/virtfs$ ls ubuntu-kvm
ls: cannot access ubuntu-kvm: No such file or directory

Then I checked the help page and found I'm using the python-vm-builder. (python version is 2.7)
whuang$ aptitude search vm-builder
i python-vm-builder - VM builder
p python-vm-builder-ec2 - EC2 Ubuntu VM builder
p ubuntu-vm-builder - Ubuntu VM builder

Revision history for this message
whuang (davhuang) wrote :

Sorry, it should not be a bug, but a misleading option.

According to the official virtualization guide:
http://doc.ubuntu.com/ubuntu/serverguide/C/jeos-and-vmbuilder.html
"the -o tells vmbuilder to overwrite the previous version of the VM"
The users should suppose to overwrite ONLY the VM.

But the manpages of vmbuilder tells
" -o, --overwrite
              Force overwrite of destination directory if it already exist. [default: False]"
It will delete the whole directory which may contains many VMs.

Is it too dangerous?
Will it make more sense if the "-o" option only overwrite the VM with the same "--hostname"?

Revision history for this message
Robie Basak (racb) wrote :

Thank you for taking the time to report this bug and helping to make Ubuntu better.

It sounds to that at the very least the documentation could be made clearer here. So at the minimum, this could be a documentation bug that could lead to data loss, so marking this as Importance: High.

Changed in vm-builder (Ubuntu):
importance: Undecided → High
Revision history for this message
whuang (davhuang) wrote :

Hi Robie, I'd like to help to report bugs or fixing bugs.

Back to the beginning of this problem, if vmbuilder didn't create a directory containing a single VM image, users won't put other VM images in the directory.
So is it really necessary to have a directory? IMHO maybe just a VM image is enough for the output.

TJ (tj)
Changed in vm-builder (Ubuntu):
status: New → Confirmed
Revision history for this message
jlord87 (moro-87) wrote :

Today is the 10th of december 2013.
This bug on 2012-12-14 has been marked as "High" importance and no bugfix has been released.

Today I've lost a virtual machine in my production enviroment using che -o option.

This is really frustrating.

Revision history for this message
Benjamin Kircher (bkircher) wrote :

Well, today is Oct 8th 2014 and I just lost three VMs because of this.

That the tyrget folder (/var/lib/libvirt/images) wasn't backed-up is my fault but the documentation SHOULD BE CLEARER about what happens when you pass the -o option to vmbuilder.

And yes, overwriting the whole directory is not that clever when overwriting the disk image with the same name is enough.

Just lost hours (even days) worth of hard work :-(

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.