Heats agent for diskimage-builder in Ubuntu 16.04

[Volodymyr Litovka]

Dears,

Summary: I wasn't be able to create xenial-based custom image with Heat integration.

There are few bugs in two areas actually:

1) Documentation to Heat incorrectly describers procedure of creating custom image
2) Installation of Heat's agents don't work when creating Xenial-based custom image

==== Documentation ===
a) https://docs.openstack.org/developer/heat/template_guide/software_deployment.html#custom-image-script says that the following command

git clone https://git.openstack.org/openstack/diskimage-builder.git

is enough to launch diskimage-builder. That's wrong at the moment - just git'ing diskimage-builder's tree don't provide access to required executables, one need to install it (preferably in virtualenv) in the way like this :

$ virtualenv dib
$ source dib/bin/activate
$ git clone https://git.openstack.org/openstack/diskimage-builder
$ cd diskimage-builder
$ pip install -e .

and just then all next steps are possible:

git clone https://git.openstack.org/openstack/tripleo-image-elements.git
git clone https://git.openstack.org/openstack/heat-agents.git
[ ... ]

b) I guess, it makes sense to note there are different branches of tripleo-image-elements / heat-agents and one need to distinguish between them: master, stable/ocata or stable/newton, e.g.

git clone -b stable/ocata https://git.openstack.org/openstack/tripleo-image-elements.git
git clone -b stable/ocata https://git.openstack.org/openstack/heat-agents.git

c) to proper work of os-*-config tools, it's required to have dib-run-parts tool on host VM, which need to be added to BASE_ELEMENTS variable (last entry in the following row):

export BASE_ELEMENTS="ubuntu selinux-permissive dib-run-parts"

==== Code ====
Few fixes required to diskimage-builder's elements to work with Xenial distribution:

d) in Xenial, dib-run-parts tool resides in 'python-dib-utils' package, which need to be reflected in diskimage-builder/diskimage_builder/elements/dib-run-parts/package-installs.yaml instead of 'dib-utils' (see (c) above)

e) heat-config-notify (used for notifying Heat about changes on VM) require keystoneclient, so this package need to be added to heat-agents/heat-config/install.d/heat-config-source-install/50-heat-config-soure (last entry in the following row):

pip install python-heatclient python-zaqarclient python-keystoneclient

f) there is broken dependency in os-collect-config's systemd integration, which prevents it starting upon VM start/reboot. This can be fixed using the following change in tripleo-image-elements/elements/os-collect-config/install.d/os-collect-config-source-install/10-os-collect-config:

[Install]
WantedBy=cloud-init.target

instead of "WantedBy=multi-user.target"

After all these changes, I was be able to create and launch xenial-based custom image using diskimage-builder tool. Possibly, it can break Trusty-based installation, this needs to be checked when considering changes above.

Hope this'll help. Thank you.

[Job (jobcespedes)]

Information about second point (f):
os-collect-config.service is defined to start in multi-user.target after cloud-final.service but
cloud-final.service is defined to start after multi-user.target (deadlock)
That's why the following works:

sed -i "s@^WantedBy=.*@WantedBy=cloud-init.target@" /lib/systemd/system/cloud-final.service
systemctl reenable os-collect-config.service

I found some dependency issues when using tripleo-images-elements (os-<xxx>-config) and building a Xenial image, so I used apt and python-os-<xxx>-config in custom elements
Thanks,

[Job (jobcespedes)]

On correction:

sed -i "s@^WantedBy=.*@WantedBy=cloud-init.target@" /lib/systemd/system/os-collect-config.service
systemctl reenable os-collect-config.service

Sorry