cc_mcollective.py fails on Ubuntu16.04

Bug #1597699 reported by Sergii Golovatiuk on 2016-06-30
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
cloud-init
High
Unassigned
cloud-init (Ubuntu)
High
Unassigned
Xenial
High
Unassigned

Bug Description

---- Begin SRU Template ----
[Impact]
mcollective usage via cloud-init is not possible.

[Test Case]
launch instance with cloud-config containing 'mcollective' entry.

$ cat >user-data <<"EOF"
#cloud-config
mcollective:
  conf:
    main_collective: mcollective
    collectives: mcollective
    libdir: /usr/share/mcollective/plugins
    logfile: /var/log/mcollective.log
    loglevel: debug
    daemonize: 1
    direct_addressing: 1
    ttl: 4294957
    securityprovider: psk
    plugin.psk: unset
    identity: 2

    connector: rabbitmq
    plugin.rabbitmq.vhost: mcollective
    plugin.rabbitmq.pool.size: 1
    plugin.rabbitmq.pool.1.host: 10.10.0.2
    plugin.rabbitmq.pool.1.port: 61613
    plugin.rabbitmq.pool.1.user: mcollective
    plugin.rabbitmq.pool.1.password: ScwpVo8egrZ0OmT6sRmp9zEA
    plugin.rabbitmq.heartbeat_interval: 30

    factsource: yaml
    plugin.yaml: /etc/mcollective/facts.yaml
EOF

$ keyname=brickies; image=$image;
$ openstack server create \
   --key-name=$keyname --flavor=m1.small \
   --image=$image --user-data=user-data mcollective-test0

# then ssh in and
a.) verify mcollective package is installed
$ dpkg-query --show mcollective
mcollective 2.6.0+dfsg-2.1

b.) verify mcollective service is running
$ systemctl status mcollective

c.) grep WARN /var/log/cloud-init.log && echo FAIL

[Regression Potential]
low to none. This has been unfortunately broken since wily.

[Other Info]
---- End SRU Template ----

How to reproduce:

#cat /etc/issue.net
Ubuntu 16.04 LTS
#( cd /var/lib/cloud/ && rm -rf * )
#cloud-init -d init

#grep ^mcollective: -A25 instance/user-data.txt
mcollective:
  conf:
    main_collective: mcollective
    collectives: mcollective
    libdir: /usr/share/mcollective/plugins
    logfile: /var/log/mcollective.log
    loglevel: debug
    daemonize: 1
    direct_addressing: 1
    ttl: 4294957
    securityprovider: psk
    plugin.psk: unset
    identity: 2

    connector: rabbitmq
    plugin.rabbitmq.vhost: mcollective
    plugin.rabbitmq.pool.size: 1
    plugin.rabbitmq.pool.1.host: 10.10.0.2
    plugin.rabbitmq.pool.1.port: 61613
    plugin.rabbitmq.pool.1.user: mcollective
    plugin.rabbitmq.pool.1.password: ScwpVo8egrZ0OmT6sRmp9zEA
    plugin.rabbitmq.heartbeat_interval: 30

    factsource: yaml
    plugin.yaml: /etc/mcollective/facts.yaml

#cloud-init -d single --name mcollective

The log will contain

Jun 30 08:26:43 node-2 [CLOUDINIT] util.py[DEBUG]: Running module mcollective (<module 'cloudinit.config.cc_mcollective' from '/usr/lib/python3/dist-packages/cloudinit/config/cc_mcollective.py'>) failed#012Traceback (most recent call last):#012 File "/usr/lib/python3/dist-packages/cloudinit/stages.py", line 739, in _run_modules#012 freq=freq)#012 File "/usr/lib/python3/dist-packages/cloudinit/cloud.py", line 70, in run#012 return self._runners.run(name, functor, args, freq, clear_on_fail)#012 File "/usr/lib/python3/dist-packages/cloudinit/helpers.py", line 199, in run#012 results = functor(*args)#012 File "/usr/lib/python3/dist-packages/cloudinit/config/cc_mcollective.py", line 83, in handle#012 mcollective_config.write(contents)#012 File "/usr/lib/python3/dist-packages/configobj.py", line 2126, in write#012 outfile.write(output_bytes)#012TypeError: string argument expected, got 'bytes'

Related branches

Scott Moser (smoser) on 2016-07-11
Changed in cloud-init:
status: New → Confirmed
importance: Undecided → High
Scott Moser (smoser) on 2016-07-14
description: updated
Scott Moser (smoser) on 2016-07-15
description: updated
Changed in cloud-init (Ubuntu):
status: New → Fix Released
importance: Undecided → High
Changed in cloud-init (Ubuntu Xenial):
status: New → Confirmed
importance: Undecided → High
Changed in cloud-init:
status: Confirmed → Fix Committed

Hello Sergii, or anyone else affected,

Accepted cloud-init into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/0.7.7~bzr1256-0ubuntu1~16.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in cloud-init (Ubuntu Xenial):
status: Confirmed → Fix Committed
tags: added: verification-needed
Scott Moser (smoser) on 2016-07-18
description: updated
Scott Moser (smoser) wrote :
Download full text (3.6 KiB)

I've verified this in an lxc container using the method described here.

# launch instance (lxc)
$ name="mcollective0"
$ cat > user-data <<EOF
#cloud-config
mcollective:
  conf:
    main_collective: mcollective
    collectives: mcollective
    libdir: /usr/share/mcollective/plugins
    logfile: /var/log/mcollective.log
    loglevel: debug
    daemonize: 1
    direct_addressing: 1
    ttl: 4294957
    securityprovider: psk
    plugin.psk: unset
    identity: 2

    connector: rabbitmq
    plugin.rabbitmq.vhost: mcollective
    plugin.rabbitmq.pool.size: 1
    plugin.rabbitmq.pool.1.host: 10.10.0.2
    plugin.rabbitmq.pool.1.port: 61613
    plugin.rabbitmq.pool.1.user: mcollective
    plugin.rabbitmq.pool.1.password: ScwpVo8egrZ0OmT6sRmp9zEA
    plugin.rabbitmq.heartbeat_interval: 30

    factsource: yaml
    plugin.yaml: /etc/mcollective/facts.yaml
EOF
$ lxc launch ubuntu-daily:xenial $name "--config=user.user-data=$(cat user-data)"

# wait for boot
$ while ! lxc exec $name -- test -e /run/cloud-init/result.json; do echo .; sleep 1; done

# get in the instance (first boot, unpatched)
$ lxc exec "$name" /bin/bash
# PS1="% "
% cat /etc/cloud/build.info
build_name: server
serial: 20160715

% grep FAIL /var/log/cloud-init.log
Jul 19 17:01:10 ubuntu [CLOUDINIT] handlers.py[DEBUG]: finish: modules-config/config-mcollective: FAIL: running config-mcollective with frequency once-per-instance
Jul 19 17:01:10 ubuntu [CLOUDINIT] handlers.py[DEBUG]: finish: modules-config: FAIL: running modules for config

% dpkg-query --show cloud-init
cloud-init 0.7.7~bzr1246-0ubuntu1~16.04.1

# The failed path results in installation of mcollective
# but with the installed config file moved to '.old'. Just clean
# up to make this more pristine.
% apt-get --purge remove mcollective; rm -Rf /etc/mcollective/

# enable proposed and update
% cat > enable-proposed <<"EOF"
#!/bin/sh
set -e
rel=$(lsb_release -sc)
awk '$1 == "deb" && $2 ~ /ubuntu.com/ {
  printf("%s %s %s-proposed main universe\n", $1, $2, rel); exit(0) };
  ' "rel=$rel" /etc/apt/sources.list |
    tee /etc/apt/sources.list.d/proposed.list
EOF

% sh ./enable-proposed
deb http://archive.ubuntu.com/ubuntu xenial-proposed main universe
% sudo apt-get update -q && sudo apt-get install cloud-init
% dpkg-query --show cloud-init
cloud-init 0.7.7~bzr1256-0ubuntu1~16.04.1

# similate first boot on pristine image.
# avoid the 'seed' dir in /var/lib/cloud as it is used by lxd
% sudo rm -Rf /var/log/cloud-init
% rm -Rf /var/lib/cloud/{data,handlers,instance,instances,scripts,sem}
% sudo reboot

### wait and re-attach to instance
$ while ! lxc exec $name -- test -e /run/cloud-init/result.json; do echo .; sleep 1; done
$ lxc exec "$name" /bin/bash
# PS1="% "

% dpkg-query --show mcollective
mcollective 2.6.0+dfsg-2.1
% grep WARN /var/log/cloud-init.log || echo no WARN
no WARN

% systemctl status mcollective
● mcollective.service - The Marionette Collective
   Loaded: loaded (/lib/systemd/system/mcollective.service; enabled; vendor pres
   Active: active (running) since Tue 2016-07-19 17:03:14 UTC; 3min 8s ago
 Main PID: 739 (ruby)
    Tasks: 3
   Memory: 9.8M
      CPU: 248ms
   CGroup: /system.slice/mcollective.service
 ...

Read more...

tags: added: verification-done
removed: verification-needed

The verification of the Stable Release Update for cloud-init has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Launchpad Janitor (janitor) wrote :

This bug was fixed in the package cloud-init - 0.7.7~bzr1256-0ubuntu1~16.04.1

---------------
cloud-init (0.7.7~bzr1256-0ubuntu1~16.04.1) xenial-proposed; urgency=medium

  * New upstream snapshot.
    - distros/debian.py: fix eni renderer to not render .link files
    - fixes for execution in python2.6.
    - ConfigDrive: fix writing of 'injected' files and legacy networking
      (LP: #1602373)
    - improvements to /etc/network/interfaces rendering including rendering
      of 'lo' devices and sorting attributes within a interface section.
    - fix mcollective module that was completely broken if using python3
      (LP: #1597699)

 -- Scott Moser <email address hidden> Fri, 15 Jul 2016 13:27:04 -0400

Changed in cloud-init (Ubuntu Xenial):
status: Fix Committed → Fix Released
Scott Moser (smoser) wrote :

This is fixed in cloud-init 0.7.7

Changed in cloud-init:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers