Activity log for bug #1100491

Date Who What changed Old value New value Message
2013-01-16 21:50:38 Adam Gandelman bug added bug
2013-01-16 21:50:38 Adam Gandelman attachment added cloud-init.log https://bugs.launchpad.net/bugs/1100491/+attachment/3485133/+files/cloud-init.log
2013-01-16 21:51:16 Adam Gandelman attachment added user-data.txt https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1100491/+attachment/3485134/+files/user-data.txt
2013-01-16 22:22:21 Launchpad Janitor branch linked lp:~smoser/ubuntu/precise/cloud-init/lp-1100491
2013-01-16 22:26:30 Scott Moser cloud-init (Ubuntu): importance Undecided Critical
2013-01-16 22:26:30 Scott Moser cloud-init (Ubuntu): status New In Progress
2013-01-16 22:26:30 Scott Moser cloud-init (Ubuntu): assignee Scott Moser (smoser)
2013-01-16 23:27:17 Adam Gandelman description Noticed juju deployed services were failing to come up using Juju+MAAS and the newly released cloud-init SRU for 12.04. The node provisioned thru MAAS okay, but after first boot did not have the required agents installed. On further investigation, it looks like the required juju packages were never installed. /var/log/cloud-init-output.log shows the following traceback: The key fingerprint is: b4:83:ef:99:6e:65:15:a3:a3:a2:83:0b:f1:3c:62:7f root@test-08 The key's randomart image is: +--[ECDSA 256]---+ | | | o | | . . o | | o .o . | |. . S. o | | + ....o | |o.+. . ..o | |.oo.oE ..o | | .o.. o= | +-----------------+ Generating locales... en_US.UTF-8... up-to-date Generation complete. 2013-01-16 16:32:56,919 - __init__.py[WARNING]: Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/cloudinit/CloudConfig/__init__.py", line 117, in run_cc_modules cc.handle(name, run_args, freq=freq) File "/usr/lib/python2.7/dist-packages/cloudinit/CloudConfig/__init__.py", line 78, in handle [name, self.cfg, self.cloud, cloudinit.log, args]) File "/usr/lib/python2.7/dist-packages/cloudinit/__init__.py", line 326, in sem_and_run func(*args) File "/usr/lib/python2.7/dist-packages/cloudinit/CloudConfig/cc_apt_update_upgrade.py", line 75, in handle errors = add_sources(cloud, cfg['apt_sources'], params) TypeError: add_sources() takes at most 2 arguments (3 given) 2013-01-16 16:32:56,919 - __init__.py[ERROR]: config handling of apt-update-upgrade, None, [] failed 2013-01-16 16:32:56,998 - cloud-init-cfg[ERROR]: errors running cloud_config [config]: ['apt-update-upgrade'] errors running cloud_config [config]: ['apt-update-upgrade'] Attached are /var/log/cloud-init.log and the MAAS provided user-data from /var/lib/cloud/instance/user-data.txt [Impact] cloud-init can no longer process cloud-config that specify a 'packages' value. This affects any tools (Juju, MAAS) that rely on cloud-init to install various packages as part of its first-boot. This seems to trigger when specifying apt_sources in cloud-init. [Test Case] Create a userdata file: cat >>/tmp/ud.txt #cloud-config apt_sources: - {source: 'ppa:openstack-ubuntu-testing/folsom-trunk-testing'} apt_update: true apt_upgrade: false packages: [python-novaclient] <<END Boot a quantal or precise Ubuntu Cloud image, specifying the ud.txt as user-data: euca-run-instances -k adam -t m1.tiny -f /tmp/ud.txt ami-04d Allow instance to boot. Since MAAS provisiongs nodes on-the-fly, first boot will have the affected cloud-init version and the attached traceback in /var/log/cloud-init-output.log. For cloud instances, its most likely running an older version of cloud-init and the cloud-config was processed okay on first boot. To test the affected version there, ssh to the machine: # revert changes to system from initial cloud-init run. $ sudo dpkg -P python-novaclient ;\ sudo rm -rf /etc/apt/sources.list.d/openstack-ubuntu-testing-folsom-trunk-testing-precise.list ;\ sudo rm -rf /var/lib/cloud/* # Install cloud-init from precise-updates $ sudo apt-get install cloud-init=0.6.3-0ubuntu1.3 # re-run cloud-init $ sudo cloud-init start-local ; sudo cloud-init start; sudo cloud-init-cfg all config ; sudo cloud-init-cfg all final Observe traceback ending in "TypeError: add_sources() takes at most 2 arguments (3 given)", the expected package was not reinstalled. Apply fix to /usr/share/pyshared/cloudinit/CloudConfig/cc_apt_update_upgrade.py and re-run cloud-init. Works as expected, package is reinstalled. [Regression Potential] Minimal, simple one-line. [Original Report] Noticed juju deployed services were failing to come up using Juju+MAAS and the newly released cloud-init SRU for 12.04. The node provisioned thru MAAS okay, but after first boot did not have the required agents installed. On further investigation, it looks like the required juju packages were never installed. /var/log/cloud-init-output.log shows the following traceback: The key fingerprint is: b4:83:ef:99:6e:65:15:a3:a3:a2:83:0b:f1:3c:62:7f root@test-08 The key's randomart image is: +--[ECDSA 256]---+ | | | o | | . . o | | o .o . | |. . S. o | | + ....o | |o.+. . ..o | |.oo.oE ..o | | .o.. o= | +-----------------+ Generating locales...   en_US.UTF-8... up-to-date Generation complete. 2013-01-16 16:32:56,919 - __init__.py[WARNING]: Traceback (most recent call last):   File "/usr/lib/python2.7/dist-packages/cloudinit/CloudConfig/__init__.py", line 117, in run_cc_modules     cc.handle(name, run_args, freq=freq)   File "/usr/lib/python2.7/dist-packages/cloudinit/CloudConfig/__init__.py", line 78, in handle     [name, self.cfg, self.cloud, cloudinit.log, args])   File "/usr/lib/python2.7/dist-packages/cloudinit/__init__.py", line 326, in sem_and_run     func(*args)   File "/usr/lib/python2.7/dist-packages/cloudinit/CloudConfig/cc_apt_update_upgrade.py", line 75, in handle     errors = add_sources(cloud, cfg['apt_sources'], params) TypeError: add_sources() takes at most 2 arguments (3 given) 2013-01-16 16:32:56,919 - __init__.py[ERROR]: config handling of apt-update-upgrade, None, [] failed 2013-01-16 16:32:56,998 - cloud-init-cfg[ERROR]: errors running cloud_config [config]: ['apt-update-upgrade'] errors running cloud_config [config]: ['apt-update-upgrade'] Attached are /var/log/cloud-init.log and the MAAS provided user-data from /var/lib/cloud/instance/user-data.txt
2013-01-16 23:27:29 Adam Gandelman summary cloud-init 0.6.3-0ubuntu1.3 failing to process juju-generated userdata [SRU] cloud-init 0.6.3-0ubuntu1.3 failing to process juju-generated userdata
2013-01-17 00:35:24 Scott Moser description [Impact] cloud-init can no longer process cloud-config that specify a 'packages' value. This affects any tools (Juju, MAAS) that rely on cloud-init to install various packages as part of its first-boot. This seems to trigger when specifying apt_sources in cloud-init. [Test Case] Create a userdata file: cat >>/tmp/ud.txt #cloud-config apt_sources: - {source: 'ppa:openstack-ubuntu-testing/folsom-trunk-testing'} apt_update: true apt_upgrade: false packages: [python-novaclient] <<END Boot a quantal or precise Ubuntu Cloud image, specifying the ud.txt as user-data: euca-run-instances -k adam -t m1.tiny -f /tmp/ud.txt ami-04d Allow instance to boot. Since MAAS provisiongs nodes on-the-fly, first boot will have the affected cloud-init version and the attached traceback in /var/log/cloud-init-output.log. For cloud instances, its most likely running an older version of cloud-init and the cloud-config was processed okay on first boot. To test the affected version there, ssh to the machine: # revert changes to system from initial cloud-init run. $ sudo dpkg -P python-novaclient ;\ sudo rm -rf /etc/apt/sources.list.d/openstack-ubuntu-testing-folsom-trunk-testing-precise.list ;\ sudo rm -rf /var/lib/cloud/* # Install cloud-init from precise-updates $ sudo apt-get install cloud-init=0.6.3-0ubuntu1.3 # re-run cloud-init $ sudo cloud-init start-local ; sudo cloud-init start; sudo cloud-init-cfg all config ; sudo cloud-init-cfg all final Observe traceback ending in "TypeError: add_sources() takes at most 2 arguments (3 given)", the expected package was not reinstalled. Apply fix to /usr/share/pyshared/cloudinit/CloudConfig/cc_apt_update_upgrade.py and re-run cloud-init. Works as expected, package is reinstalled. [Regression Potential] Minimal, simple one-line. [Original Report] Noticed juju deployed services were failing to come up using Juju+MAAS and the newly released cloud-init SRU for 12.04. The node provisioned thru MAAS okay, but after first boot did not have the required agents installed. On further investigation, it looks like the required juju packages were never installed. /var/log/cloud-init-output.log shows the following traceback: The key fingerprint is: b4:83:ef:99:6e:65:15:a3:a3:a2:83:0b:f1:3c:62:7f root@test-08 The key's randomart image is: +--[ECDSA 256]---+ | | | o | | . . o | | o .o . | |. . S. o | | + ....o | |o.+. . ..o | |.oo.oE ..o | | .o.. o= | +-----------------+ Generating locales...   en_US.UTF-8... up-to-date Generation complete. 2013-01-16 16:32:56,919 - __init__.py[WARNING]: Traceback (most recent call last):   File "/usr/lib/python2.7/dist-packages/cloudinit/CloudConfig/__init__.py", line 117, in run_cc_modules     cc.handle(name, run_args, freq=freq)   File "/usr/lib/python2.7/dist-packages/cloudinit/CloudConfig/__init__.py", line 78, in handle     [name, self.cfg, self.cloud, cloudinit.log, args])   File "/usr/lib/python2.7/dist-packages/cloudinit/__init__.py", line 326, in sem_and_run     func(*args)   File "/usr/lib/python2.7/dist-packages/cloudinit/CloudConfig/cc_apt_update_upgrade.py", line 75, in handle     errors = add_sources(cloud, cfg['apt_sources'], params) TypeError: add_sources() takes at most 2 arguments (3 given) 2013-01-16 16:32:56,919 - __init__.py[ERROR]: config handling of apt-update-upgrade, None, [] failed 2013-01-16 16:32:56,998 - cloud-init-cfg[ERROR]: errors running cloud_config [config]: ['apt-update-upgrade'] errors running cloud_config [config]: ['apt-update-upgrade'] Attached are /var/log/cloud-init.log and the MAAS provided user-data from /var/lib/cloud/instance/user-data.txt [Impact] cloud-init can no longer process cloud-config that specify a 'packages' value. This affects any tools (Juju, MAAS) that rely on cloud-init to install various packages as part of its first-boot. This seems to trigger when specifying apt_sources in cloud-init. [Test Case] Create a userdata file: cat >>/tmp/ud.txt #cloud-config apt_sources: - {source: 'ppa:openstack-ubuntu-testing/folsom-trunk-testing'} apt_update: true apt_upgrade: false packages: [python-novaclient] <<END Boot a precise daily Ubuntu Cloud image, specifying the ud.txt as user-data: euca-run-instances -k adam -t m1.tiny -f /tmp/ud.txt ami-04d Allow instance to boot. Since MAAS provisiongs nodes on-the-fly, first boot will have the affected cloud-init version and the attached traceback in /var/log/cloud-init-output.log. For cloud instances, its most likely running an older version of cloud-init and the cloud-config was processed okay on first boot. To test the affected version there, ssh to the machine: # revert changes to system from initial cloud-init run. $ sudo dpkg -P python-novaclient ;\   sudo rm -rf /etc/apt/sources.list.d/openstack-ubuntu-testing-folsom-trunk-testing-precise.list ;\   sudo rm -rf /var/lib/cloud/* # Install cloud-init from precise-updates $ sudo apt-get install cloud-init=0.6.3-0ubuntu1.3 # re-run cloud-init $ sudo cloud-init start-local ; sudo cloud-init start; sudo cloud-init-cfg all config ; sudo cloud-init-cfg all final Observe traceback ending in "TypeError: add_sources() takes at most 2 arguments (3 given)", the expected package was not reinstalled. Apply fix to /usr/share/pyshared/cloudinit/CloudConfig/cc_apt_update_upgrade.py and re-run cloud-init. Works as expected, package is reinstalled. [Regression Potential] Minimal, simple one-line. [Original Report] Noticed juju deployed services were failing to come up using Juju+MAAS and the newly released cloud-init SRU for 12.04. The node provisioned thru MAAS okay, but after first boot did not have the required agents installed. On further investigation, it looks like the required juju packages were never installed. /var/log/cloud-init-output.log shows the following traceback: The key fingerprint is: b4:83:ef:99:6e:65:15:a3:a3:a2:83:0b:f1:3c:62:7f root@test-08 The key's randomart image is: +--[ECDSA 256]---+ | | | o | | . . o | | o .o . | |. . S. o | | + ....o | |o.+. . ..o | |.oo.oE ..o | | .o.. o= | +-----------------+ Generating locales...   en_US.UTF-8... up-to-date Generation complete. 2013-01-16 16:32:56,919 - __init__.py[WARNING]: Traceback (most recent call last):   File "/usr/lib/python2.7/dist-packages/cloudinit/CloudConfig/__init__.py", line 117, in run_cc_modules     cc.handle(name, run_args, freq=freq)   File "/usr/lib/python2.7/dist-packages/cloudinit/CloudConfig/__init__.py", line 78, in handle     [name, self.cfg, self.cloud, cloudinit.log, args])   File "/usr/lib/python2.7/dist-packages/cloudinit/__init__.py", line 326, in sem_and_run     func(*args)   File "/usr/lib/python2.7/dist-packages/cloudinit/CloudConfig/cc_apt_update_upgrade.py", line 75, in handle     errors = add_sources(cloud, cfg['apt_sources'], params) TypeError: add_sources() takes at most 2 arguments (3 given) 2013-01-16 16:32:56,919 - __init__.py[ERROR]: config handling of apt-update-upgrade, None, [] failed 2013-01-16 16:32:56,998 - cloud-init-cfg[ERROR]: errors running cloud_config [config]: ['apt-update-upgrade'] errors running cloud_config [config]: ['apt-update-upgrade'] Attached are /var/log/cloud-init.log and the MAAS provided user-data from /var/lib/cloud/instance/user-data.txt
2013-01-17 00:44:12 Chris Halse Rogers bug added subscriber Ubuntu Stable Release Updates Team
2013-01-17 00:44:16 Chris Halse Rogers bug added subscriber SRU Verification
2013-01-17 00:44:19 Chris Halse Rogers tags verification-needed
2013-01-17 06:06:14 Adam Gandelman attachment added lp1100491_verification.txt https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1100491/+attachment/3485805/+files/lp1100491_verification.txt
2013-01-17 06:06:27 Adam Gandelman tags verification-needed verification-done
2013-01-17 14:08:29 Scott Moser summary [SRU] cloud-init 0.6.3-0ubuntu1.3 failing to process juju-generated userdata [SRU] apt_sources broken in 0.6.3-0ubuntu1.3
2013-01-17 14:08:38 Scott Moser summary [SRU] apt_sources broken in 0.6.3-0ubuntu1.3 [SRU] apt_sources broken in 0.6.3-0ubuntu1.3 [regression]
2013-01-17 14:09:03 Scott Moser tags verification-done regression-update verification-done
2013-01-17 15:34:43 Scott Moser cloud-init (Ubuntu): status In Progress Fix Committed
2013-01-17 15:42:52 Scott Moser attachment added user-data showing and collecting output https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1100491/+attachment/3486284/+files/lp-1100491.txt
2013-01-17 16:51:52 Colin Watson removed subscriber Ubuntu Stable Release Updates Team
2013-01-17 16:52:11 Launchpad Janitor cloud-init (Ubuntu): status Fix Committed Fix Released
2013-01-18 15:17:35 Launchpad Janitor branch linked lp:~smoser/ubuntu/precise/cloud-init/sru
2013-08-28 11:32:39 Launchpad Janitor branch linked lp:~ubuntu-branches/ubuntu/precise/cloud-init/precise-proposed
2013-08-28 11:32:56 Launchpad Janitor branch linked lp:~ubuntu-branches/ubuntu/precise/cloud-init/precise-updates