diskimage-builder: Software Deployments Broken In Xenial

[Nenad Radojevic]

I am trying to build an image for ubuntu trusty and xenial that works with OS::Heat::SoftwareDeployment using diskimage-builder. I installed diskimage-builde with

pip install diskimage-builder

an run the following scripts to build the images:

export DISTRIBUTION="ubuntu"
export DIB_RELEASE="trusty"

git clone https://git.openstack.org/openstack/tripleo-image-elements.git
git clone https://git.openstack.org/openstack/heat-agents.git

export ELEMENTS_PATH=tripleo-image-elements/elements:heat-agents/
export BASE_ELEMENTS="$DISTRIBUTION selinux-permissive"
export AGENT_ELEMENTS="os-collect-config os-refresh-config os-apply-config"
export DEPLOYMENT_BASE_ELEMENTS="heat-config heat-config-script heat-config-ansible"
export DEPLOYMENT_TOOL=""

TIMESTAMP=$(date +%Y%m%d%H%M%S)
export IMAGE_NAME="$DISTRIBUTION-$DIB_RELEASE_$TIMESTAMP"

disk-image-create vm $BASE_ELEMENTS $AGENT_ELEMENTS \
     $DEPLOYMENT_BASE_ELEMENTS $DEPLOYMENT_TOOL -t raw -o $IMAGE_NAME.raw

openstack image create --disk-format raw --container-format bare \
    $IMAGE_NAME < $IMAGE_NAME.raw

To build xenial I just set DIB_RELEASE to "xenial". The build process runs fine and the images a uploaded. Then I deploy instances from both images and check the os-collect-config service. On trusty the service is running and executing os-collect-config manually results in a successful run as expected.

On xenial though, this is what I see:

root@test:~# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04.2 LTS
Release: 16.04
Codename: xenial
root@test:~# os-collect-config --debug --force
Source [heat_local] Unavailable.
Traceback (most recent call last):
  File "/usr/local/bin/os-collect-config", line 11, in <module>
    sys.exit(__main__())
  File "/opt/stack/venvs/os-collect-config/lib/python3.5/site-packages/os_collect_config/collect.py", line 262, in __main__
    collector_kwargs_map=collector_kwargs_map)
  File "/opt/stack/venvs/os-collect-config/lib/python3.5/site-packages/os_collect_config/collect.py", line 166, in collect_all
    content = module.Collector(**collector_kwargs).collect()
  File "/opt/stack/venvs/os-collect-config/lib/python3.5/site-packages/os_collect_config/ec2.py", line 75, in collect
    md = config_drive.get_metadata()
  File "/opt/stack/venvs/os-collect-config/lib/python3.5/site-packages/os_collect_config/config_drive.py", line 173, in get_metadata
    bd = config_drive()
  File "/opt/stack/venvs/os-collect-config/lib/python3.5/site-packages/os_collect_config/config_drive.py", line 166, in config_drive
    for bd in all_block_devices():
  File "/opt/stack/venvs/os-collect-config/lib/python3.5/site-packages/os_collect_config/config_drive.py", line 155, in all_block_devices
    for device in out.split('\n\n'):
TypeError: a bytes-like object is required, not 'str'
root@test:~#

I have no idea what this means or where it might come from. Any help is very much appreciated.

Got the hint that it might be an os-collect-config bug where it seems it is not python3 compatible because xenial only ships with python3 by default. Therfore, I exported DIB_PYTHON_VERSION=2 before running the diskimage-create command, but the image is still broken. Here is the output: https://gist.github.com/anonymous/f413b6f76194f6ef3391495670250655

Any ideas?

[Jacob Schramm (xfxian)]

I think it is an issue with the default python version running on your local system when the virtualenvs for os-* are created. I fixed this by specifing a Python 2 virtualenv environment variable before calling disk-image-create like this:

export VIRTUALENV_PYTHON="/usr/bin/python2"

[Nenad Radojevic (nradojevic)]

Hi Jacob,

thanks for your input, but unfortunately it still doesn't work with your solution.
I have added the evnvironment variable and now I'm getting this error:

root@image-test:~# os-collect-config --force --debug

Source [heat_local] Unavailable.
Source [request] Unavailable.
/var/lib/os-collect-config/local-data not found. Skipping
No local metadata found (['/var/lib/os-collect-config/local-data'])
No auth_url configured.
[2017-05-15 09:50:01,452] (os-refresh-config) [INFO] Starting phase pre-configure
Traceback (most recent call last):
  File "/usr/local/bin/os-refresh-config", line 11, in <module>
    sys.exit(main())
  File "/opt/stack/venvs/os-refresh-config/local/lib/python2.7/site-packages/os_refresh_config/os_refresh_config.py", line 149, in main
    subprocess.check_call(args, close_fds=True)
  File "/usr/lib/python2.7/subprocess.py", line 536, in check_call
    retcode = call(*popenargs, **kwargs)
  File "/usr/lib/python2.7/subprocess.py", line 523, in call
    return Popen(*popenargs, **kwargs).wait()
  File "/usr/lib/python2.7/subprocess.py", line 711, in __init__
    errread, errwrite)
  File "/usr/lib/python2.7/subprocess.py", line 1343, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory
Command failed, will not cache new data. Command 'os-refresh-config' returned non-zero exit status 1