dos.py is failed to create a new environment in fuel-devops@2.9.18

[Dennis Dmitriev]

Steps to reproduce:

1) Install fuel-devops from the tag 2.9.18 into a python virtual environment:

  pip install git+https://github.com/openstack/fuel-devops.git@2.9.18 --upgrade

2) Try to create a new environment:

  export ISO_PATH=<path to Fuel ISO file>
  export ENV_NAME=env_1
  dos.py create --iso-path $ISO_PATH $ENV_NAME

Expected result: environment is created

Actual result: environment is failed to create with the following messages:
...
libvirt: Network Driver error : operation failed: network 'test_env_storage' already exists with uuid 3eee1a90-f466-466f-8a1a-7e311e6aa7fd
...

This is happened because some networks are defined from shell.py ; and they are failed to define at the second time with Environment.define()

Affected fuel-devops branches : release/2.9 , master

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: release/2.9
Review: https://review.openstack.org/282007

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/282007
Committed: https://git.openstack.org/cgit/openstack/fuel-devops/commit/?id=20367d634bb1bcb76d357e7e02d98f8e86d0e198
Submitter: Jenkins
Branch: release/2.9

commit 20367d634bb1bcb76d357e7e02d98f8e86d0e198
Author: Dennis Dmitriev <email address hidden>
Date: Thu Feb 18 20:51:56 2016 +0200

    Fix 'dos.py create' command

    - do not define network objects twice
    - define admin node in shell.py instead of in the Environment
      (like other slave nodes are defined)

    Change-Id: I5dfefad9991fde315c1e94e90eeed2a22dc89cd3
    Closes-Bug:#1546635

[Alex Schultz (alex-schultz)]

This is still happening on master

[Sergio Aguilar (chekolyn)]

@Alex Schultz

I can confirm this is happening on master.

I've made two changes to make it work.

Reorder the do_create (moved self.env.define() before adding slaves)

    def do_create(self):
        env_name = self.params.name
        for env in Environment.list_all():
            if env.name == env_name:
                print("Please, set another environment name")
                raise SystemExit()
        self.env = Environment.create(env_name)
        networks, prefix = self.params.net_pool.split(':')
        Network.default_pool = Network.create_network_pool(
            networks=[ipaddr.IPNetwork(networks)],
            prefix=int(prefix))
        networks = Network.create_networks(environment=self.env)
        # We need to define the networks here because they are quieried by the
        # admin_add function.
        for network in networks:
            network.define()
        admin_node = self.admin_add(networks=networks)
        self.env.define()
        # define the slave nodes since the Environment define no longer defines
        # the networks, volumes and nodes by default and we need them to be
        # created for this function
        self.do_slave_add(force_define=True)
        admin_node.disk_devices.get(device='cdrom').volume.upload(
            self.params.iso_path)
        for net in self.env.get_networks():
            net.start()

And environment.py is trying to define the networks again.

    def define(self, skip=True):
        # 'skip' param is a temporary workaround.
        # It will be removed with introducing the new database schema
        # See the task QA-239 for details.
        #for network in self.get_networks():
        # network.define()
        if not skip:
            for volume in self.get_volumes():
                volume.define()
        for node in self.get_nodes():
            node.define()

Anyways I know probably a git pull request is better and this is perhaps not the best to way to communicate things but just adding additional info for debugging purposes.

[Alex Schultz (alex-schultz)]

So it appears to be fixed on the release/2.9 branch, as master is behind. The branches should be documented better or cleaned up

[Dennis Dmitriev (ddmitriev)]

@Alex, it will be fixed in master separatelly, for fuel-devops3.0

[Anton Studenov (astudenov)]

Works ok on master. Looks like it was fixed in
https://review.openstack.org/#/c/274755/
https://github.com/openstack/fuel-devops/commit/f4e184a5c1861677030500f397f90eb3cae7fe4f

[Bogdan Dobrelya (bogdando)]

I still can reproduce it with the 29815fb44af73d57dc8db2153afee1cc7c3c4633

[Bogdan Dobrelya (bogdando)]

For my case, the issue was related to the "orhpaned" artifacts listed in the virsh nwfilter-list

[Bogdan Dobrelya (bogdando)]

Nope, I still can reproduce it with the master branch.
libvirt.libvirtError: operation failed: network 'foo' already exists with uuid bar

[Bogdan Dobrelya (bogdando)]

Also, after I erase the env, I have a local and a private network remained

[Bogdan Dobrelya (bogdando)]

And also, there is undeleted *_slave-0_system in the /var/lib/libvirt/images/

[Dennis Dmitriev (ddmitriev)]

@Bogdan, can you reproduce the bug with fuel-devops from 'master' branch?

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/344257

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/344257
Committed: https://git.openstack.org/cgit/openstack/fuel-devops/commit/?id=d62e43f3e39eaa37e8321a4bfaee577aeb0d7cbd
Submitter: Jenkins
Branch: master

commit d62e43f3e39eaa37e8321a4bfaee577aeb0d7cbd
Author: Dennis Dmitriev <email address hidden>
Date: Tue Jul 19 17:05:03 2016 +0300

    Fix for qcow2 volume resize

    If qcow2 volume has bigger size than specified for the volume with
    the attribute 'capacity', qemu-img fails to shrink such image.

    - do not fail if the image cannot be shrinked to specified size

    Change-Id: I8560e04657eb6570c43ebe07ddd5aa4975de9aec
    Closes-Bug:#1546635