unit tests fail as non-root on maas deployed system

Bug #1635350 reported by Joshua Powers on 2016-10-20
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
cloud-init
Medium
Unassigned
cloud-init (Ubuntu)
Medium
Unassigned
Xenial
Low
Unassigned
Yakkety
Low
Unassigned

Bug Description

=== Begin SRU Template ===
[Impact]
Running cloud-init's unit test cases on a system deployed by MAAS would
fail. The reason is that the non-root user would not be able to read
files with MAAS node credentials in /etc/cloud/cloud.cfg.d

We want this change SRU so that an attempt to build and run tests on a
system deployed by maas will work rather than fail due to unit test failure.

[Test Case]
Run unit tests on a system deployed by maas, or even just with:
  f=/etc/cloud/cloud.cfg.d/90_dpkg_maas.cfg
  sh -c 'mkdir -p "${1%/*}" && touch "$1" && chmod ugo-r "$1"' -- "$f"
  tox -e py3

[Regression Potential]
This was just to fix a build break or unit tests being run.
Changes are only to unit tests.
=== End SRU Template ===

Observed Behavior:

On a system deployed by MAAS I checked out master and then tried to immediately build it:
> git clone https://git.launchpad.net/cloud-init
> cd cloud-init
> ./packages/bddeb

I get a number of errors around permission issues around this file:
PermissionError: [Errno 13] Permission denied: \'/etc/cloud/cloud.cfg.d/90_dpkg_maas.cfg\'

See: https://paste.ubuntu.com/23354559/
 or formatted better: http://paste.ubuntu.com/23374383/

If I run as root however, it build as expected.

Expected Behavior:
Running bddeb works as a non-root user.

Related branches

Revision history for this message
Scott Moser (smoser) wrote :

just as a work around, you can actually build with:
 DEB_BUILD_OPTIONS=nocheck

and then it wont run the tests.

obviously good reason to run the tests, but you may be able to assume it ran already.

description: updated
Revision history for this message
Ryan Harper (raharper) wrote :

<rharper> nTraceback (most recent call last):\n File "/tmp/tmpvkejmell/cloud-init-0.7.8-19-g7ae2011/tests/unittests/test_data.py", line 375, in test_unhandl

Scott Moser (smoser) on 2016-10-24
Changed in cloud-init:
status: New → Confirmed
importance: Undecided → Medium
Scott Moser (smoser) on 2016-10-24
Changed in cloud-init:
status: Confirmed → Fix Committed
summary: - bddeb does not work on a system deployed by maas
+ unit tests fail on maas deployed system
summary: - unit tests fail on maas deployed system
+ unit tests fail as non-root on maas deployed system
Scott Moser (smoser) on 2016-11-07
Changed in cloud-init (Ubuntu):
status: New → Fix Released
importance: Undecided → Medium
Changed in cloud-init (Ubuntu Xenial):
status: New → Confirmed
Changed in cloud-init (Ubuntu Yakkety):
status: New → Confirmed
Changed in cloud-init (Ubuntu Xenial):
importance: Undecided → Low
Changed in cloud-init (Ubuntu Yakkety):
importance: Undecided → Low
description: updated
Scott Moser (smoser) on 2016-11-07
description: updated
Revision history for this message
Steve Langasek (vorlon) wrote : Please test proposed package

Hello Joshua, or anyone else affected,

Accepted cloud-init into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/0.7.8-47-gb6561a1-0ubuntu1~16.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in cloud-init (Ubuntu Xenial):
status: Confirmed → Fix Committed
tags: added: verification-needed
Revision history for this message
Steve Langasek (vorlon) wrote :

Hello Joshua, or anyone else affected,

Accepted cloud-init into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/0.7.8-49-g9e904bb-0ubuntu1~16.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Scott Moser (smoser) on 2016-11-21
tags: added: verification-done
removed: verification-needed
Revision history for this message
Scott Moser (smoser) wrote :

I've marked this as verification done.
It really was only a unit test issue, so not really related to SRU.
That said, I verified that, on a maas deployed system.

$ ls -l /etc/cloud/cloud.cfg.d/90_dpkg_maas.cfg
-rw------- 1 root root 309 Apr 11 2016 /etc/cloud/cloud.cfg.d/90_dpkg_maas.cfg
$ id
uid=1003(smoser) gid=1003(smoser) groups=1003(smoser),110(lxd),117(sbuild),1004(libvirt),1005(kvm),1008(curtin)

## So the file /etc/cloud/cloud.cfg.d/90_dpkg_maas.cfg is not readable
## by my user.

$ git clone git://git.launchpad.net/cloud-init cloud-init
$ cd cloud-init
$ git checkout ubuntu/0.7.8-49-g9e904bb-0ubuntu1
$ tox -e py3
...
Ran 750 tests in 22.480s

OK (SKIP=6)
____________________ summary ____________________
  py3: commands succeeded
  congratulations :)

Revision history for this message
Launchpad Janitor (janitor) wrote :
Download full text (4.0 KiB)

This bug was fixed in the package cloud-init - 0.7.8-49-g9e904bb-0ubuntu1~16.04.1

---------------
cloud-init (0.7.8-49-g9e904bb-0ubuntu1~16.04.1) xenial-proposed; urgency=medium

  * debian/cloud-init.postinst: update /etc/fstab on Azure to fix
    future resize operations. (LP: #1611074)
  * New upstream snapshot.
    - Add activate_datasource, for datasource specific code paths.
      (LP: #1611074)
    - systemd: cloud-init-local use RequiresMountsFor=/var/lib/cloud
      (LP: #1642062)

cloud-init (0.7.8-47-gb6561a1-0ubuntu1~16.04.1) xenial-proposed; urgency=medium

  * debian/cloud-init.templates: enable DigitalOcean by default [Ben Howard]
  * New upstream snapshot.
    - systemd/cloud-init-local.service:
      + replace 'Wants' and 'After' on local-fs.target with more granular
        After=systemd-remount-fs.service and RequiresMountsFor=/var/lib
        and Before=sysinit.target.
        This is done run sufficiently early enough to update /etc/fstab.
        (LP: #1611074)
      + add Before=NetworkManager.service so that cloud-init can render
        NetworkManager network config before it would apply them.
    - systemd/cloud-init.service:
      + add Before=sysinit.target and DefaultDependencies=no (LP: #1611074)
      + drop Requires=networking.service to work where networking.service is
        not needed.
      + add Conflicts=shutdown.target
      + drop unnecessary Wants=local-fs.target
    - net: support reading ipv6 dhcp config from initramfs [LaMont Jones]
      (LP: #1621615)
    - dmidecode: Allow dmidecode to be used on aarch64, and only attempt
      usage on x86, x86_64, and aarch64. [Robert Schweikert]
    - disk-config: udev settle after partitioning in gpt format.
      (LP: #1626243)
    - Add support for snap create-user on Ubuntu Core images. [Ryan Harper]
      (LP: #1619393)
    - Fix sshd restarts for rhel distros. [Jim Gorz]
    - Move user/group functions to new ug_util file [Joshua Harlow]
    - update Gentoo initscripts to run in the correct order [Matthew Thode]
    - MAAS: improve the debugging tool in datasource to consider
      config provided on kernel cmdline.
    - lxd: Update network config for LXD 2.3 [Stéphane Graber] (LP: #1640556)
    - Decode unicode types in decode_binary [Robert Schweikert]
    - Allow ephemeral drive to be unpartitioned [Paul Meyer]
    - subp: add 'update_env' argument which allows for more easily adding
      environment variables to a subprocess call.
    - Adjust mounts and disk configuration for systemd. (LP: #1611074)
    - DataSources:
      + Ec2: protect against non-dictionary in block-device-mapping.
      + AliYun: Add new datasource for Ali-Cloud ECS, that is
        available but not enabled by default [kaihuan.pkh]
      + DigitalOcean: use meta-data for network configuration and
        enable data source by default. [Ben Howard]
      + OpenNebula: replace parsing of 'ip' command with similar function
        available in cloudinit.net. This fixed unit tests when running
        in environment with no networking.
    - doc changes:
      + Add documentation on stages of boot.
      + make the RST files consistently formated and other improvements.
     ...

Read more...

Changed in cloud-init (Ubuntu Xenial):
status: Fix Committed → Fix Released
Revision history for this message
Adam Conrad (adconrad) wrote : Update Released

The verification of the Stable Release Update for cloud-init has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Chris Halse Rogers (raof) wrote : Please test proposed package

Hello Joshua, or anyone else affected,

Accepted cloud-init into yakkety-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/0.7.8-49-g9e904bb-0ubuntu1~16.10.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in cloud-init (Ubuntu Yakkety):
status: Confirmed → Fix Committed
tags: removed: verification-done
tags: added: verification-needed
Revision history for this message
Joshua Powers (powersj) wrote :

Tested using integration tests [1] and manually downloaded and installed deb locally.

[1] https://jenkins.ubuntu.com/server/job/cloud-init-integration-y/16/consoleText

tags: added: verification-done
removed: verification-needed
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package cloud-init - 0.7.8-49-g9e904bb-0ubuntu1~16.10.1

---------------
cloud-init (0.7.8-49-g9e904bb-0ubuntu1~16.10.1) yakkety; urgency=medium

  * debian/cloud-init.templates: enable DigitalOcean by default [Ben Howard]
  * debian/cloud-init.postinst: update /etc/fstab on Azure to fix
    future resize operations. (LP: #1611074)
  * New upstream snapshot.
    - systemd/cloud-init-local.service:
      + replace 'Wants' and 'After' on local-fs.target with more granular
        After=systemd-remount-fs.service and RequiresMountsFor=/var/lib
        and Before=sysinit.target.
        This is done run sufficiently early enough to update /etc/fstab.
        (LP: #1611074)
    - systemd/cloud-init.service:
      + add Before=sysinit.target and DefaultDependencies=no (LP: #1611074)
      + drop Requires=networking.service to work where networking.service is
        not needed.
      + add Conflicts=shutdown.target
      + drop unnecessary Wants=local-fs.target
    - net: support reading ipv6 dhcp config from initramfs [LaMont Jones]
      (LP: #1621615)
    - dmidecode: Allow dmidecode to be used on aarch64, and only attempt
      usage on x86, x86_64, and aarch64. [Robert Schweikert]
    - disk-config: udev settle after partitioning in gpt format.
      (LP: #1626243)
    - Add support for snap create-user on Ubuntu Core images. [Ryan Harper]
      (LP: #1619393)
    - Fix sshd restarts for rhel distros. [Jim Gorz]
    - Move user/group functions to new ug_util file [Joshua Harlow]
    - update Gentoo initscripts to run in the correct order [Matthew Thode]
    - MAAS: improve the debugging tool in datasource to consider
      config provided on kernel cmdline.
    - DataSources:
      + Ec2: protect against non-dictionary in block-device-mapping.
      + AliYun: Add new datasource for Ali-Cloud ECS, that is
        available but not enabled by default [kaihuan.pkh]
      + OpenNebula: replace parsing of 'ip' command with similar function
        available in cloudinit.net. This fixed unit tests when running
        in environment with no networking.
    - doc changes:
      + Add documentation on stages of boot.
      + make the RST files consistently formated and other improvements.
      + fixed example to not overwrite /etc/hosts [Chris Glass]
      + fix spelling / typos in ca_certs and scripts_vendor.
      + improve HACKING.rst file
      + Add documentation for logging features. [Wesley Wiedenmeier]
    - code style and unit test changes:
      + pep8: fix style errors reported by pycodestyle 2.1.0
      + pyflakes: fix issue with pyflakes 1.3 found in ubuntu zesty-proposed.
      + Add coverage dependency to bddeb to fix package build.
      + Add coverage collection to tox unit tests. [Joshua Powers]
      + do not read system /etc/cloud/cloud.cfg.d (LP: #1635350)
      + tests: silence the Cheetah UserWarning about NameMapper C version.
      + Fix python2.6 things found running in centos 6.

 -- Scott Moser <email address hidden> Tue, 22 Nov 2016 17:04:36 -0500

Changed in cloud-init (Ubuntu Yakkety):
status: Fix Committed → Fix Released
Revision history for this message
Scott Moser (smoser) wrote :

This is fixed in cloud-init 0.7.9.

Changed in cloud-init:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers