cc_set_passwords fails to change passwords specified as chpasswd['list'] in cloud-config

Bug #1665694 reported by Serg Lystopad
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
cloud-init
Fix Released
Medium
Unassigned
cloud-init (Ubuntu)
Fix Released
Medium
Unassigned
Xenial
Fix Released
Medium
Unassigned
Yakkety
Fix Released
Medium
Unassigned
Zesty
Fix Released
Medium
Unassigned

Bug Description

=== Being SRU Template ===
[Impact]
Users of cloud-init can change passwords on a system by providing input
to chpasswd as a string:
  #cloud-config
  chpasswd:
    list: |
      user1:password1

Confusingly, the 'list' is actually not a list, but a multi-line string.
The change made in this bug supports either.

[Test Case]
There is an integration test in cloud-init that runs though this code.
To run that:

$ git clone https://git.launchpad.net/cloud-init
$ cd cloud-init

# download the appropriate deb for cloud-init from -proposed
$ rel=xenial
$ pver=$(rmadison --url=ubuntu --suite=$rel-proposed cloud-init | awk '{print $3}')
$ fname="cloud-init_${pver}_all.deb"
$ wget "http://archive.ubuntu.com/ubuntu/pool/main/c/cloud-init/$fname"
$ ln -sf $fname cloud-init_all.$rel.deb
$ tox -e citest -- run -v -n $rel --deb=cloud-init_all.$rel.deb \
   -t tests/cloud_tests/testcases/modules/set_password_list_string.py \
   -t tests/cloud_tests/testcases/modules/set_password_list.py

That will install the new cloud-init into a container and run
with user data to excercise this new feature.

[Regression Potential]
Very low regression potential. The test case shown provides both
the previously supported path (a string) and the new path (a list).

[Other Info]
Upstream commit:
 https://git.launchpad.net/cloud-init/commit/?id=7f2b51054a5defe

=== End SRU Template ===

If cloud-config contains list of user:password pairs as in example below

chpasswd:
  list:
    - user1:pwd001
    - user2:pwd002

cc_set_passwords module fails to change passwords with error:
Feb 17 15:52:48 si-man [CLOUDINIT] stages.py[DEBUG]: Running module set-passwords (<module 'cloudinit.config.cc_set_passwords' from '/usr/lib/python3/dist-packages/cloudinit/config/cc_set_passwords.py'>) with frequency once-per-instance
Feb 17 15:52:48 si-man [CLOUDINIT] handlers.py[DEBUG]: start: modules-config/config-set-passwords: running config-set-passwords with frequency once-per-instance
Feb 17 15:52:48 si-man [CLOUDINIT] util.py[DEBUG]: Writing to /var/lib/cloud/instances/6d822e81-98a1-4b43-bed2-db8d0cf045bb/sem/config_set_passwords - wb: [420] 25 bytes
Feb 17 15:52:48 si-man [CLOUDINIT] helpers.py[DEBUG]: Running config-set-passwords using lock (<FileLock using file '/var/lib/cloud/instances/6d822e81-98a1-4b43-bed2-db8d0cf045bb/sem/config_set_passwords'>)
Feb 17 15:52:48 si-man [CLOUDINIT] cc_set_passwords.py[DEBUG]: Changing password for ["['user1"]:
Feb 17 15:52:48 si-man [CLOUDINIT] util.py[DEBUG]: Running command ['chpasswd'] with allowed return codes [0] (shell=False, capture=True)
Feb 17 15:52:48 si-man [CLOUDINIT] util.py[WARNING]: Failed to set passwords with chpasswd for ["['user1"]
Feb 17 15:52:48 si-man [CLOUDINIT] util.py[DEBUG]: Failed to set passwords with chpasswd for ["['user1"]#012Traceback (most recent call last):#012 File "/usr/lib/python3/dist-packages/cloudinit/config/cc_set_passwords.py", line 121, in handle#012 util.subp(['chpasswd'], ch_in)#012 File "/usr/lib/python3/dist-packages/cloudinit/util.py", line 1836, in subp#012 cmd=args)#012cloudinit.util.ProcessExecutionError: Unexpected error while running command.#012Command: ['chpasswd']#012Exit code: 1#012Reason: -#012Stdout: ''#012Stderr: "chpasswd: (user ['user1) pam_chauthtok() failed, error:\nAuthentication token manipulation error\nchpasswd: (line 1, user ['user1) password not changed\n"
Feb 17 15:52:48 si-man [CLOUDINIT] util.py[DEBUG]: Running command ['passwd', '--expire', "['user1"] with allowed return codes [0] (shell=False, capture=True)
Feb 17 15:52:48 si-man [CLOUDINIT] util.py[WARNING]: Failed to set 'expire' for ['user1
Feb 17 15:52:48 si-man [CLOUDINIT] util.py[DEBUG]: Failed to set 'expire' for ['user1#012Traceback (most recent call last):#012 File "/usr/lib/python3/dist-packages/cloudinit/config/cc_set_passwords.py", line 136, in handle#012 util.subp(['passwd', '--expire', u])#012 File "/usr/lib/python3/dist-packages/cloudinit/util.py", line 1836, in subp#012 cmd=args)#012cloudinit.util.ProcessExecutionError: Unexpected error while running command.#012Command: ['passwd', '--expire', "['user1"]#012Exit code: 1#012Reason: -#012Stdout: ''#012Stderr: "passwd: user '['user1' does not exist\n"
Feb 17 15:52:48 si-man [CLOUDINIT] cc_set_passwords.py[DEBUG]: 2 errors occured, re-raising the last one

The issue affects cloud-init installed in xenial-server-cloudimg-amd64-disk1.img
# apt-cache policy cloud-init
cloud-init:
  Installed: 0.7.8-49-g9e904bb-0ubuntu1~16.04.4
  Candidate: 0.7.8-49-g9e904bb-0ubuntu1~16.04.4
  Version table:
 *** 0.7.8-49-g9e904bb-0ubuntu1~16.04.4 500
        500 http://zone-1.clouds.archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages
        100 /var/lib/dpkg/status
     0.7.7~bzr1212-0ubuntu1 500
        500 http://zone-1.clouds.archive.ubuntu.com/ubuntu xenial/main amd64 Packages

cc_set_passwords converts list of user:password lists to str and as result user names get corrupted.

Related branches

Revision history for this message
Serg Lystopad (slystopad) wrote :
Scott Moser (smoser)
description: updated
Changed in cloud-init:
status: New → Confirmed
importance: Undecided → Medium
status: Confirmed → Fix Committed
Scott Moser (smoser)
Changed in cloud-init (Ubuntu Xenial):
status: New → Confirmed
Changed in cloud-init (Ubuntu Yakkety):
status: New → Confirmed
Changed in cloud-init (Ubuntu Zesty):
status: New → Fix Released
Changed in cloud-init (Ubuntu Xenial):
importance: Undecided → Medium
Changed in cloud-init (Ubuntu Yakkety):
importance: Undecided → Medium
Changed in cloud-init (Ubuntu Zesty):
importance: Undecided → Medium
Scott Moser (smoser)
description: updated
Revision history for this message
Brian Murray (brian-murray) wrote : Please test proposed package

Hello Serg, or anyone else affected,

Accepted cloud-init into yakkety-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/0.7.9-90-g61eb03fe-0ubuntu1~16.10.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in cloud-init (Ubuntu Yakkety):
status: Confirmed → Fix Committed
tags: added: verification-needed
Revision history for this message
Brian Murray (brian-murray) wrote :

Hello Serg, or anyone else affected,

Accepted cloud-init into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/0.7.9-90-g61eb03fe-0ubuntu1~16.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in cloud-init (Ubuntu Xenial):
status: Confirmed → Fix Committed
Revision history for this message
Chad Smith (chad.smith) wrote :
Download full text (18.0 KiB)

# Ran integration tests which cover both use cases

######### xenial
$ rel=xenial
$ pver=$(rmadison --url=ubuntu --suite=$rel-proposed cloud-init | awk '{print $3}')
$ echo $pver
0.7.9-90-g61eb03fe-0ubuntu1~16.04.1
$ fname="cloud-init_${pver}_all.deb"
$ wget "http://archive.ubuntu.com/ubuntu/pool/main/c/cloud-init/$fname"
...
Saving to: ‘cloud-init_0.7.9-90-g61eb03fe-0ubuntu1~16.04.1_all.deb’
$ ln -sf $fname cloud-init_all.$rel.deb
$ tox -e citest -- run -v -n $rel --deb=cloud-init_all.$rel.deb \
> -t tests/cloud_tests/testcases/modules/set_password_list_string.py \
> -t tests/cloud_tests/testcases/modules/set_password_list.py
GLOB sdist-make: /home/csmith/cloud-init/setup.py
citest inst-nodeps: /home/csmith/cloud-init/.tox/dist/cloud-init-0.7.9.zip
citest installed: appdirs==1.4.3,asn1crypto==0.22.0,cffi==1.10.0,cloud-init==0.7.9,configobj==5.0.6,cryptography==1.8.1,idna==2.5,Jinja2==2.9.6,jsonpatch==1.15,jsonpointer==1.10,MarkupSafe==1.0,oauthlib==2.0.2,packaging==16.8,pbr==2.1.0,pkg-resources==0.0.0,prettytable==0.7.2,pycparser==2.17,pylxd==2.1.3,python-dateutil==2.6.0,PyYAML==3.12,requests==2.11.1,requests-unixsocket==0.1.5,six==1.10.0,urllib3==1.20,ws4py==0.4.2
citest runtests: PYTHONHASHSEED='3017348944'
citest runtests: commands[0] | /home/csmith/cloud-init/.tox/citest/bin/python -m tests.cloud_tests run -v -n xenial --deb=cloud-init_all.xenial.deb -t tests/cloud_tests/testcases/modules/set_password_list_string.py -t tests/cloud_tests/testcases/modules/set_password_list.py
2017-04-13 15:28:11,619 - tests.cloud_tests - DEBUG - running with args: Namespace(deb='cloud-init_all.xenial.deb', os_name=['xenial'], platform=['lxd'], ppa=None, quiet=False, repo=None, result=None, rpm=None, script=None, subcmd='run', test_config=['tests/cloud_tests/testcases/modules/set_password_list.py', 'tests/cloud_tests/testcases/modules/set_password_list_string.py'], upgrade=False, verbose=True)

2017-04-13 15:28:11,620 - tests.cloud_tests - DEBUG - using tmpdir /tmp/cloud_test_data_zk8wv7mv
2017-04-13 15:28:11,623 - tests.cloud_tests - INFO - setting up platform: lxd
2017-04-13 15:28:11,694 - tests.cloud_tests - INFO - acquiring image for os: xenial
 2017-04-13 15:28:29,747 - tests.cloud_tests - INFO - setting up image: distro=ubuntu, release=xenial
2017-04-13 15:28:29,754 - tests.cloud_tests - DEBUG - installing deb: cloud-init_all.xenial.deb into target
/home/csmith/cloud-init/.tox/citest/lib/python3.5/site-packages/pylxd/deprecation.py:24: DeprecationWarning: execute will return a ContainerExecuteResult in pylxd 2.2
  warnings.warn(self.message, DeprecationWarning)
2017-04-13 15:28:55,117 - tests.cloud_tests - DEBUG - successfully installed: cloud-init_all.xenial.deb, version: '0.7.9-90-g61eb03fe-0ubuntu1~16.04.1'
2017-04-13 15:28:55,117 - tests.cloud_tests - DEBUG - creating snapshot for xenial
2017-04-13 15:29:11,396 - tests.cloud_tests - INFO - collecting test data for os: xenial
2017-04-13 15:29:11,406 - tests.cloud_tests - INFO - collecting test data for test: modules/set_password_list
...
2017-04-13 15:29:29,958 - tests.cloud_tests - DEBUG - running collect script: instance-id
2017-04-13 15:29:30,212 - tests.cloud_tests - DEBUG - running coll...

Chad Smith (chad.smith)
tags: added: verification-done-xenial verification-done-yakkety
removed: verification-needed
Revision history for this message
Launchpad Janitor (janitor) wrote :
Download full text (3.4 KiB)

This bug was fixed in the package cloud-init - 0.7.9-90-g61eb03fe-0ubuntu1~16.10.1

---------------
cloud-init (0.7.9-90-g61eb03fe-0ubuntu1~16.10.1) yakkety; urgency=medium

  * debian/cloud-init.templates: add Bigstep to list of sources. (LP: #1676460)
  * New upstream snapshot.
    - OpenStack: add 'dvs' to the list of physical link types. (LP: #1674946)
    - Fix bug that resulted in an attempt to rename bonds or vlans.
      (LP: #1669860)
    - tests: update OpenNebula and Digital Ocean to not rely on host
      interfaces.
    - net: in netplan renderer delete known image-builtin content.
      (LP: #1675576)
    - doc: correct grammar in capabilities.rst [David Tagatac]
    - ds-identify: fix detecting of maas datasource. (LP: #1677710)
    - netplan: remove debugging prints, add debug logging [Ryan Harper]
    - ds-identify: do not write None twice to datasource_list.
    - support resizing partition and rootfs on system booted without
      initramfs. [Steve Langasek] (LP: #1677376)
    - apt_configure: run only when needed. (LP: #1675185)
    - OpenStack: identify OpenStack by product 'OpenStack Compute'.
      (LP: #1675349)
    - GCE: Search GCE in ds-identify, consider serial number in check.
      (LP: #1674861)
    - Add support for setting hashed passwords [Tore S. Lonoy] (LP: #1570325)
    - Fix filesystem creation when using "partition: auto"
      [Jonathan Ballet] (LP: #1634678)
    - ConfigDrive: support reading config drive data from /config-drive.
      (LP: #1673411)
    - ds-identify: fix detection of Bigstep datasource. (LP: #1674766)
    - test: add running of pylint [Joshua Powers]
    - ds-identify: fix bug where filename expansion was left on.
    - advertise network config v2 support (NETWORK_CONFIG_V2) in features.
    - Bigstep: fix bug when executing in python3. [root]
    - Fix unit test when running in a system deployed with cloud-init.
    - Bounce network interface for Azure when using the built-in path.
      [Brent Baude] (LP: #1674685)
    - cloudinit.net: add network config v2 parsing and rendering [Ryan Harper]
    - net: Fix incorrect call to isfile [Joshua Powers] (LP: #1674317)
    - net: add renderers for automatically selecting the renderer.
    - doc: fix config drive doc with regard to unpartitioned disks.
      (LP: #1673818)
    - test: Adding integratiron test for password as list [Joshua Powers]
    - render_network_state: switch arguments around, do not require target
    - support 'loopback' as a device type.
    - Integration Testing: improve testcase subclassing [Wesley Wiedenmeier]
    - gitignore: adding doc/rtd_html [Joshua Powers]
    - doc: add instructions for running integration tests via tox.
      [Joshua Powers]
    - test: avoid differences in 'date' output due to daylight savings.
    - Fix chef config module in omnibus install. [Jeremy Melvin] (LP: #1583837)
    - Add feature flags to cloudinit.version. [Wesley Wiedenmeier]
    - tox: add a citest environment
    - Support chpasswd/list being a list in addition to a string.
      [Sergio Lystopad] (LP: #1665694)
    - doc: Fix configuration example for cc_set_passwords module.
      [Sergio Lystopad] (LP: #1665773)
    - ...

Read more...

Changed in cloud-init (Ubuntu Yakkety):
status: Fix Committed → Fix Released
Revision history for this message
Steve Langasek (vorlon) wrote : Update Released

The verification of the Stable Release Update for cloud-init has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Launchpad Janitor (janitor) wrote :
Download full text (3.4 KiB)

This bug was fixed in the package cloud-init - 0.7.9-90-g61eb03fe-0ubuntu1~16.04.1

---------------
cloud-init (0.7.9-90-g61eb03fe-0ubuntu1~16.04.1) xenial-proposed; urgency=medium

  * debian/cloud-init.templates: add Bigstep to list of sources. (LP: #1676460)
  * New upstream snapshot.
    - OpenStack: add 'dvs' to the list of physical link types. (LP: #1674946)
    - Fix bug that resulted in an attempt to rename bonds or vlans.
      (LP: #1669860)
    - tests: update OpenNebula and Digital Ocean to not rely on host
      interfaces.
    - net: in netplan renderer delete known image-builtin content.
      (LP: #1675576)
    - doc: correct grammar in capabilities.rst [David Tagatac]
    - ds-identify: fix detecting of maas datasource. (LP: #1677710)
    - netplan: remove debugging prints, add debug logging [Ryan Harper]
    - ds-identify: do not write None twice to datasource_list.
    - support resizing partition and rootfs on system booted without
      initramfs. [Steve Langasek] (LP: #1677376)
    - apt_configure: run only when needed. (LP: #1675185)
    - OpenStack: identify OpenStack by product 'OpenStack Compute'.
      (LP: #1675349)
    - GCE: Search GCE in ds-identify, consider serial number in check.
      (LP: #1674861)
    - Add support for setting hashed passwords [Tore S. Lonoy] (LP: #1570325)
    - Fix filesystem creation when using "partition: auto"
      [Jonathan Ballet] (LP: #1634678)
    - ConfigDrive: support reading config drive data from /config-drive.
      (LP: #1673411)
    - ds-identify: fix detection of Bigstep datasource. (LP: #1674766)
    - test: add running of pylint [Joshua Powers]
    - ds-identify: fix bug where filename expansion was left on.
    - advertise network config v2 support (NETWORK_CONFIG_V2) in features.
    - Bigstep: fix bug when executing in python3. [root]
    - Fix unit test when running in a system deployed with cloud-init.
    - Bounce network interface for Azure when using the built-in path.
      [Brent Baude] (LP: #1674685)
    - cloudinit.net: add network config v2 parsing and rendering [Ryan Harper]
    - net: Fix incorrect call to isfile [Joshua Powers] (LP: #1674317)
    - net: add renderers for automatically selecting the renderer.
    - doc: fix config drive doc with regard to unpartitioned disks.
      (LP: #1673818)
    - test: Adding integratiron test for password as list [Joshua Powers]
    - render_network_state: switch arguments around, do not require target
    - support 'loopback' as a device type.
    - Integration Testing: improve testcase subclassing [Wesley Wiedenmeier]
    - gitignore: adding doc/rtd_html [Joshua Powers]
    - doc: add instructions for running integration tests via tox.
      [Joshua Powers]
    - test: avoid differences in 'date' output due to daylight savings.
    - Fix chef config module in omnibus install. [Jeremy Melvin] (LP: #1583837)
    - Add feature flags to cloudinit.version. [Wesley Wiedenmeier]
    - tox: add a citest environment
    - Support chpasswd/list being a list in addition to a string.
      [Sergio Lystopad] (LP: #1665694)
    - doc: Fix configuration example for cc_set_passwords module.
      [Sergio Lystopad] (LP: #1665773...

Read more...

Changed in cloud-init (Ubuntu Xenial):
status: Fix Committed → Fix Released
Revision history for this message
Scott Moser (smoser) wrote : Fixed in Cloud-init 17.1

This bug is believed to be fixed in cloud-init in 17.1. If this is still a problem for you, please make a comment and set the state back to New

Thank you.

Changed in cloud-init:
status: Fix Committed → Fix Released
Revision history for this message
James Falcon (falcojr) wrote :
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.