cc_set_passwords fails to change passwords specified as chpasswd['list'] in cloud-config

Bug #1665694 reported by Serg Lystopad on 2017-02-17
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
cloud-init
Medium
Unassigned
cloud-init (Ubuntu)
Medium
Unassigned
Xenial
Medium
Unassigned
Yakkety
Medium
Unassigned
Zesty
Medium
Unassigned

Bug Description

=== Being SRU Template ===
[Impact]
Users of cloud-init can change passwords on a system by providing input
to chpasswd as a string:
  #cloud-config
  chpasswd:
    list: |
      user1:password1

Confusingly, the 'list' is actually not a list, but a multi-line string.
The change made in this bug supports either.

[Test Case]
There is an integration test in cloud-init that runs though this code.
To run that:

$ git clone https://git.launchpad.net/cloud-init
$ cd cloud-init

# download the appropriate deb for cloud-init from -proposed
$ rel=xenial
$ pver=$(rmadison --url=ubuntu --suite=$rel-proposed cloud-init | awk '{print $3}')
$ fname="cloud-init_${pver}_all.deb"
$ wget "http://archive.ubuntu.com/ubuntu/pool/main/c/cloud-init/$fname"
$ ln -sf $fname cloud-init_all.$rel.deb
$ tox -e citest -- run -v -n $rel --deb=cloud-init_all.$rel.deb \
   -t tests/cloud_tests/testcases/modules/set_password_list_string.py \
   -t tests/cloud_tests/testcases/modules/set_password_list.py

That will install the new cloud-init into a container and run
with user data to excercise this new feature.

[Regression Potential]
Very low regression potential. The test case shown provides both
the previously supported path (a string) and the new path (a list).

[Other Info]
Upstream commit:
 https://git.launchpad.net/cloud-init/commit/?id=7f2b51054a5defe

=== End SRU Template ===

If cloud-config contains list of user:password pairs as in example below

chpasswd:
  list:
    - user1:pwd001
    - user2:pwd002

cc_set_passwords module fails to change passwords with error:
Feb 17 15:52:48 si-man [CLOUDINIT] stages.py[DEBUG]: Running module set-passwords (<module 'cloudinit.config.cc_set_passwords' from '/usr/lib/python3/dist-packages/cloudinit/config/cc_set_passwords.py'>) with frequency once-per-instance
Feb 17 15:52:48 si-man [CLOUDINIT] handlers.py[DEBUG]: start: modules-config/config-set-passwords: running config-set-passwords with frequency once-per-instance
Feb 17 15:52:48 si-man [CLOUDINIT] util.py[DEBUG]: Writing to /var/lib/cloud/instances/6d822e81-98a1-4b43-bed2-db8d0cf045bb/sem/config_set_passwords - wb: [420] 25 bytes
Feb 17 15:52:48 si-man [CLOUDINIT] helpers.py[DEBUG]: Running config-set-passwords using lock (<FileLock using file '/var/lib/cloud/instances/6d822e81-98a1-4b43-bed2-db8d0cf045bb/sem/config_set_passwords'>)
Feb 17 15:52:48 si-man [CLOUDINIT] cc_set_passwords.py[DEBUG]: Changing password for ["['user1"]:
Feb 17 15:52:48 si-man [CLOUDINIT] util.py[DEBUG]: Running command ['chpasswd'] with allowed return codes [0] (shell=False, capture=True)
Feb 17 15:52:48 si-man [CLOUDINIT] util.py[WARNING]: Failed to set passwords with chpasswd for ["['user1"]
Feb 17 15:52:48 si-man [CLOUDINIT] util.py[DEBUG]: Failed to set passwords with chpasswd for ["['user1"]#012Traceback (most recent call last):#012 File "/usr/lib/python3/dist-packages/cloudinit/config/cc_set_passwords.py", line 121, in handle#012 util.subp(['chpasswd'], ch_in)#012 File "/usr/lib/python3/dist-packages/cloudinit/util.py", line 1836, in subp#012 cmd=args)#012cloudinit.util.ProcessExecutionError: Unexpected error while running command.#012Command: ['chpasswd']#012Exit code: 1#012Reason: -#012Stdout: ''#012Stderr: "chpasswd: (user ['user1) pam_chauthtok() failed, error:\nAuthentication token manipulation error\nchpasswd: (line 1, user ['user1) password not changed\n"
Feb 17 15:52:48 si-man [CLOUDINIT] util.py[DEBUG]: Running command ['passwd', '--expire', "['user1"] with allowed return codes [0] (shell=False, capture=True)
Feb 17 15:52:48 si-man [CLOUDINIT] util.py[WARNING]: Failed to set 'expire' for ['user1
Feb 17 15:52:48 si-man [CLOUDINIT] util.py[DEBUG]: Failed to set 'expire' for ['user1#012Traceback (most recent call last):#012 File "/usr/lib/python3/dist-packages/cloudinit/config/cc_set_passwords.py", line 136, in handle#012 util.subp(['passwd', '--expire', u])#012 File "/usr/lib/python3/dist-packages/cloudinit/util.py", line 1836, in subp#012 cmd=args)#012cloudinit.util.ProcessExecutionError: Unexpected error while running command.#012Command: ['passwd', '--expire', "['user1"]#012Exit code: 1#012Reason: -#012Stdout: ''#012Stderr: "passwd: user '['user1' does not exist\n"
Feb 17 15:52:48 si-man [CLOUDINIT] cc_set_passwords.py[DEBUG]: 2 errors occured, re-raising the last one

The issue affects cloud-init installed in xenial-server-cloudimg-amd64-disk1.img
# apt-cache policy cloud-init
cloud-init:
  Installed: 0.7.8-49-g9e904bb-0ubuntu1~16.04.4
  Candidate: 0.7.8-49-g9e904bb-0ubuntu1~16.04.4
  Version table:
 *** 0.7.8-49-g9e904bb-0ubuntu1~16.04.4 500
        500 http://zone-1.clouds.archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages
        100 /var/lib/dpkg/status
     0.7.7~bzr1212-0ubuntu1 500
        500 http://zone-1.clouds.archive.ubuntu.com/ubuntu xenial/main amd64 Packages

cc_set_passwords converts list of user:password lists to str and as result user names get corrupted.

Related branches

Scott Moser (smoser) on 2017-03-09
description: updated
Changed in cloud-init:
status: New → Confirmed
importance: Undecided → Medium
status: Confirmed → Fix Committed
Scott Moser (smoser) on 2017-04-04
Changed in cloud-init (Ubuntu Xenial):
status: New → Confirmed
Changed in cloud-init (Ubuntu Yakkety):
status: New → Confirmed
Changed in cloud-init (Ubuntu Zesty):
status: New → Fix Released
Changed in cloud-init (Ubuntu Xenial):
importance: Undecided → Medium
Changed in cloud-init (Ubuntu Yakkety):
importance: Undecided → Medium
Changed in cloud-init (Ubuntu Zesty):
importance: Undecided → Medium
Scott Moser (smoser) on 2017-04-04
description: updated

Hello Serg, or anyone else affected,

Accepted cloud-init into yakkety-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/0.7.9-90-g61eb03fe-0ubuntu1~16.10.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in cloud-init (Ubuntu Yakkety):
status: Confirmed → Fix Committed
tags: added: verification-needed
Brian Murray (brian-murray) wrote :

Hello Serg, or anyone else affected,

Accepted cloud-init into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/0.7.9-90-g61eb03fe-0ubuntu1~16.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in cloud-init (Ubuntu Xenial):
status: Confirmed → Fix Committed
Chad Smith (chad.smith) wrote :
Download full text (18.0 KiB)

# Ran integration tests which cover both use cases

######### xenial
$ rel=xenial
$ pver=$(rmadison --url=ubuntu --suite=$rel-proposed cloud-init | awk '{print $3}')
$ echo $pver
0.7.9-90-g61eb03fe-0ubuntu1~16.04.1
$ fname="cloud-init_${pver}_all.deb"
$ wget "http://archive.ubuntu.com/ubuntu/pool/main/c/cloud-init/$fname"
...
Saving to: ‘cloud-init_0.7.9-90-g61eb03fe-0ubuntu1~16.04.1_all.deb’
$ ln -sf $fname cloud-init_all.$rel.deb
$ tox -e citest -- run -v -n $rel --deb=cloud-init_all.$rel.deb \
> -t tests/cloud_tests/testcases/modules/set_password_list_string.py \
> -t tests/cloud_tests/testcases/modules/set_password_list.py
GLOB sdist-make: /home/csmith/cloud-init/setup.py
citest inst-nodeps: /home/csmith/cloud-init/.tox/dist/cloud-init-0.7.9.zip
citest installed: appdirs==1.4.3,asn1crypto==0.22.0,cffi==1.10.0,cloud-init==0.7.9,configobj==5.0.6,cryptography==1.8.1,idna==2.5,Jinja2==2.9.6,jsonpatch==1.15,jsonpointer==1.10,MarkupSafe==1.0,oauthlib==2.0.2,packaging==16.8,pbr==2.1.0,pkg-resources==0.0.0,prettytable==0.7.2,pycparser==2.17,pylxd==2.1.3,python-dateutil==2.6.0,PyYAML==3.12,requests==2.11.1,requests-unixsocket==0.1.5,six==1.10.0,urllib3==1.20,ws4py==0.4.2
citest runtests: PYTHONHASHSEED='3017348944'
citest runtests: commands[0] | /home/csmith/cloud-init/.tox/citest/bin/python -m tests.cloud_tests run -v -n xenial --deb=cloud-init_all.xenial.deb -t tests/cloud_tests/testcases/modules/set_password_list_string.py -t tests/cloud_tests/testcases/modules/set_password_list.py
2017-04-13 15:28:11,619 - tests.cloud_tests - DEBUG - running with args: Namespace(deb='cloud-init_all.xenial.deb', os_name=['xenial'], platform=['lxd'], ppa=None, quiet=False, repo=None, result=None, rpm=None, script=None, subcmd='run', test_config=['tests/cloud_tests/testcases/modules/set_password_list.py', 'tests/cloud_tests/testcases/modules/set_password_list_string.py'], upgrade=False, verbose=True)

2017-04-13 15:28:11,620 - tests.cloud_tests - DEBUG - using tmpdir /tmp/cloud_test_data_zk8wv7mv
2017-04-13 15:28:11,623 - tests.cloud_tests - INFO - setting up platform: lxd
2017-04-13 15:28:11,694 - tests.cloud_tests - INFO - acquiring image for os: xenial
 2017-04-13 15:28:29,747 - tests.cloud_tests - INFO - setting up image: distro=ubuntu, release=xenial
2017-04-13 15:28:29,754 - tests.cloud_tests - DEBUG - installing deb: cloud-init_all.xenial.deb into target
/home/csmith/cloud-init/.tox/citest/lib/python3.5/site-packages/pylxd/deprecation.py:24: DeprecationWarning: execute will return a ContainerExecuteResult in pylxd 2.2
  warnings.warn(self.message, DeprecationWarning)
2017-04-13 15:28:55,117 - tests.cloud_tests - DEBUG - successfully installed: cloud-init_all.xenial.deb, version: '0.7.9-90-g61eb03fe-0ubuntu1~16.04.1'
2017-04-13 15:28:55,117 - tests.cloud_tests - DEBUG - creating snapshot for xenial
2017-04-13 15:29:11,396 - tests.cloud_tests - INFO - collecting test data for os: xenial
2017-04-13 15:29:11,406 - tests.cloud_tests - INFO - collecting test data for test: modules/set_password_list
...
2017-04-13 15:29:29,958 - tests.cloud_tests - DEBUG - running collect script: instance-id
2017-04-13 15:29:30,212 - tests.cloud_tests - DEBUG - running coll...

Chad Smith (chad.smith) on 2017-04-14
tags: added: verification-done-xenial verification-done-yakkety
removed: verification-needed
Launchpad Janitor (janitor) wrote :
Download full text (3.4 KiB)

This bug was fixed in the package cloud-init - 0.7.9-90-g61eb03fe-0ubuntu1~16.10.1

---------------
cloud-init (0.7.9-90-g61eb03fe-0ubuntu1~16.10.1) yakkety; urgency=medium

  * debian/cloud-init.templates: add Bigstep to list of sources. (LP: #1676460)
  * New upstream snapshot.
    - OpenStack: add 'dvs' to the list of physical link types. (LP: #1674946)
    - Fix bug that resulted in an attempt to rename bonds or vlans.
      (LP: #1669860)
    - tests: update OpenNebula and Digital Ocean to not rely on host
      interfaces.
    - net: in netplan renderer delete known image-builtin content.
      (LP: #1675576)
    - doc: correct grammar in capabilities.rst [David Tagatac]
    - ds-identify: fix detecting of maas datasource. (LP: #1677710)
    - netplan: remove debugging prints, add debug logging [Ryan Harper]
    - ds-identify: do not write None twice to datasource_list.
    - support resizing partition and rootfs on system booted without
      initramfs. [Steve Langasek] (LP: #1677376)
    - apt_configure: run only when needed. (LP: #1675185)
    - OpenStack: identify OpenStack by product 'OpenStack Compute'.
      (LP: #1675349)
    - GCE: Search GCE in ds-identify, consider serial number in check.
      (LP: #1674861)
    - Add support for setting hashed passwords [Tore S. Lonoy] (LP: #1570325)
    - Fix filesystem creation when using "partition: auto"
      [Jonathan Ballet] (LP: #1634678)
    - ConfigDrive: support reading config drive data from /config-drive.
      (LP: #1673411)
    - ds-identify: fix detection of Bigstep datasource. (LP: #1674766)
    - test: add running of pylint [Joshua Powers]
    - ds-identify: fix bug where filename expansion was left on.
    - advertise network config v2 support (NETWORK_CONFIG_V2) in features.
    - Bigstep: fix bug when executing in python3. [root]
    - Fix unit test when running in a system deployed with cloud-init.
    - Bounce network interface for Azure when using the built-in path.
      [Brent Baude] (LP: #1674685)
    - cloudinit.net: add network config v2 parsing and rendering [Ryan Harper]
    - net: Fix incorrect call to isfile [Joshua Powers] (LP: #1674317)
    - net: add renderers for automatically selecting the renderer.
    - doc: fix config drive doc with regard to unpartitioned disks.
      (LP: #1673818)
    - test: Adding integratiron test for password as list [Joshua Powers]
    - render_network_state: switch arguments around, do not require target
    - support 'loopback' as a device type.
    - Integration Testing: improve testcase subclassing [Wesley Wiedenmeier]
    - gitignore: adding doc/rtd_html [Joshua Powers]
    - doc: add instructions for running integration tests via tox.
      [Joshua Powers]
    - test: avoid differences in 'date' output due to daylight savings.
    - Fix chef config module in omnibus install. [Jeremy Melvin] (LP: #1583837)
    - Add feature flags to cloudinit.version. [Wesley Wiedenmeier]
    - tox: add a citest environment
    - Support chpasswd/list being a list in addition to a string.
      [Sergio Lystopad] (LP: #1665694)
    - doc: Fix configuration example for cc_set_passwords module.
      [Sergio Lystopad] (LP: #1665773)
    - ...

Read more...

Changed in cloud-init (Ubuntu Yakkety):
status: Fix Committed → Fix Released

The verification of the Stable Release Update for cloud-init has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Launchpad Janitor (janitor) wrote :
Download full text (3.4 KiB)

This bug was fixed in the package cloud-init - 0.7.9-90-g61eb03fe-0ubuntu1~16.04.1

---------------
cloud-init (0.7.9-90-g61eb03fe-0ubuntu1~16.04.1) xenial-proposed; urgency=medium

  * debian/cloud-init.templates: add Bigstep to list of sources. (LP: #1676460)
  * New upstream snapshot.
    - OpenStack: add 'dvs' to the list of physical link types. (LP: #1674946)
    - Fix bug that resulted in an attempt to rename bonds or vlans.
      (LP: #1669860)
    - tests: update OpenNebula and Digital Ocean to not rely on host
      interfaces.
    - net: in netplan renderer delete known image-builtin content.
      (LP: #1675576)
    - doc: correct grammar in capabilities.rst [David Tagatac]
    - ds-identify: fix detecting of maas datasource. (LP: #1677710)
    - netplan: remove debugging prints, add debug logging [Ryan Harper]
    - ds-identify: do not write None twice to datasource_list.
    - support resizing partition and rootfs on system booted without
      initramfs. [Steve Langasek] (LP: #1677376)
    - apt_configure: run only when needed. (LP: #1675185)
    - OpenStack: identify OpenStack by product 'OpenStack Compute'.
      (LP: #1675349)
    - GCE: Search GCE in ds-identify, consider serial number in check.
      (LP: #1674861)
    - Add support for setting hashed passwords [Tore S. Lonoy] (LP: #1570325)
    - Fix filesystem creation when using "partition: auto"
      [Jonathan Ballet] (LP: #1634678)
    - ConfigDrive: support reading config drive data from /config-drive.
      (LP: #1673411)
    - ds-identify: fix detection of Bigstep datasource. (LP: #1674766)
    - test: add running of pylint [Joshua Powers]
    - ds-identify: fix bug where filename expansion was left on.
    - advertise network config v2 support (NETWORK_CONFIG_V2) in features.
    - Bigstep: fix bug when executing in python3. [root]
    - Fix unit test when running in a system deployed with cloud-init.
    - Bounce network interface for Azure when using the built-in path.
      [Brent Baude] (LP: #1674685)
    - cloudinit.net: add network config v2 parsing and rendering [Ryan Harper]
    - net: Fix incorrect call to isfile [Joshua Powers] (LP: #1674317)
    - net: add renderers for automatically selecting the renderer.
    - doc: fix config drive doc with regard to unpartitioned disks.
      (LP: #1673818)
    - test: Adding integratiron test for password as list [Joshua Powers]
    - render_network_state: switch arguments around, do not require target
    - support 'loopback' as a device type.
    - Integration Testing: improve testcase subclassing [Wesley Wiedenmeier]
    - gitignore: adding doc/rtd_html [Joshua Powers]
    - doc: add instructions for running integration tests via tox.
      [Joshua Powers]
    - test: avoid differences in 'date' output due to daylight savings.
    - Fix chef config module in omnibus install. [Jeremy Melvin] (LP: #1583837)
    - Add feature flags to cloudinit.version. [Wesley Wiedenmeier]
    - tox: add a citest environment
    - Support chpasswd/list being a list in addition to a string.
      [Sergio Lystopad] (LP: #1665694)
    - doc: Fix configuration example for cc_set_passwords module.
      [Sergio Lystopad] (LP: #1665773...

Read more...

Changed in cloud-init (Ubuntu Xenial):
status: Fix Committed → Fix Released

This bug is believed to be fixed in cloud-init in 17.1. If this is still a problem for you, please make a comment and set the state back to New

Thank you.

Changed in cloud-init:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers