Set systemd as default service provider

[Ivan Suzdal]

[Impact]

 * Puppet incorrectly detects the service manager on Ubuntu 16.04 and 16.10 as upstart, rather than systemd.

 * Various puppet-driven commands fail as they do not use the systemd interfaces.

[Test Case]

 * # upstart is required to confuse Puppet
apt install -y puppet upstart tftpd-hpa

# fails without fix
puppet apply -e "service { 'tftpd-hpa': ensure => stopped }"

# install fixed package

# works
puppet apply -e "service { 'tftpd-hpa': ensure => stopped }"

[Regression Potential]

 * Currently, puppet is unable to apply configurations to 16.04, 16.10 and 17.04 (fix is in -proposed for 17.04) when upstart is installed. This change will make puppet correctly use systemd for service manipulation, and I believe the likelihood of regression is low.

 * The primary source of regressions would be users who have modified their puppet installations to work around this issue. I believe they will be generally unaffected by fixing the underlying issue.

---

AFAIK, Ubuntu has systemd as default init system since 15.04 release. Although, puppet thinks it's still upstart.
This behavior is already fixed in upstream puppet code. Please, add this patch to Xenial puppet package.

[Ubuntu Foundations Team Bug Bot (crichton)]

The attachment "patch for puppet (v3.8.5-2)" seems to be a patch. If it isn't, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are a member of the ~ubuntu-reviewers, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issues please contact him.]

[Nish Aravamudan (nacc)]

@Ivan, have you ever produced a debdiff before? Would you be comfortable updating your diff to be a proper debdiff against the current puppet package?

[Simon Déziel (sdeziel)]

I am also interested in this so here's a debdiff.

[Nish Aravamudan (nacc)]

@Simon, @Ivan, looking upstream, this seems to be a combination of:

https://github.com/puppetlabs/puppet/commit/7fe61647f23650fc4c93cd6b54c654a20ff7c9f9
https://github.com/puppetlabs/puppet/commit/920d58df681bf02e5293686c704d89d71baa0f28
https://github.com/puppetlabs/puppet/commit/9000a5da8048742939e5fb010b5aae69da84e5b5

(one of which implies the inclusion of: https://github.com/puppetlabs/puppet/commit/c36f1e9e6c61dad042cd5827f816a570bc71cc6b)

Just wanted to check that the other related changes (at least those affecting Ubuntu) weren't necessary as well?

[Ivan Suzdal (isuzdal)]

In my case these changes were enough. Not sure what other changes weren't necessary - I checked on the 16.04 and unfortunately I've no opportunity to check on an early releases.

[Ivan Suzdal (isuzdal)]

I've tested simple manifest which includes 'ensure service running' on 15.04 and 15.10 and have no faced with any issues.

[Dominic Cleal (domcleal)]

I've tested the diff in #4 on Ubuntu 16.04, with puppet-common 3.8.5-2, it seems to fix the issue correctly.

Prior to the patch, Puppet would try and manage services using upstart:

/Stage[main]/Tftp::Service/Service[tftpd-hpa]: Could not evaluate: Execution of '/sbin/status tftpd-hpa' returned 1: status: Unable to connect to Upstart: Failed to connect to socket /com/ubuntu/upstart: Connection refused

[Ivan Suzdal (isuzdal)]

@Nish, is any ETA here? Could I help somehow yet?

[Diamond (giahung)]

Hi, please add this patch to next release.

[Robie Basak (racb)]

Simon, thank you for the debdiff. Sorry we didn't manage to get to this in the week before release. I think this probably does qualify for an SRU.

Nish, would you mind driving this please? I'll be happy to sponsor following your review. We need to make sure this is fixed in the development release first, has SRU paperwork for Xenial, the fix minimises future maintenance burden, etc.

[Dmitry Burmistrov (dmburmistrov)]

Is this patch confirmed? Is it going to be included 16.04.1 update?

[Nish Aravamudan (nacc)]

@Dmitry, sorry I have been busy with a few other tasks, I will look at this today or tomorrow and respond.

[Nish Aravamudan (nacc)]

@Simon, would you be able to help me determine if the other upstream patches are necessary or not? I worry that while it may work for some testers in some environments, that unless we backport the complete set of upstream-related changes, there will be some broken situations still.

[Simon Déziel (sdeziel)]

@nacc, I aggregated all the commits you mentioned in update 5. I needed to apply some parts manually, especially for the spec/ tests portions.

# upstart is required to confuse Puppet
apt-get install -y puppet-common upstart tftpd-hpa

# fails
puppet apply -e "service { 'tftpd-hpa': ensure => stopped }"

# install fixed package
dpkg -i puppet-common_3.8.5-2ubuntu1_all.deb

# works
puppet apply -e "service { 'tftpd-hpa': ensure => stopped }" # works

[Simon Déziel (sdeziel)]

Same debdiff but with a proper address in the changelog.

[Nish Aravamudan (nacc)]

@Simon and others affected, I noticed that puppet in Yakkety is at 4.5.0-4 in proposed. Can that be tested as well, that way we can confirm the fix is already released in the development release and then work on the SRU/backport.

[Nish Aravamudan (nacc)]

@Simon, I think the new debdiff is an improvement, but I have the following questions/requests:

1) Can you add more DEP3 headers? Author and Source are missing, I think. I think this is especially important since we're folding multiple upstream commits into one.
2) Can you use dep3changelog to make the changelog? It makes for a more consistent formatting in debian/changelog.
3) I noticed the lib/puppet/provider/service/debian.rb changes from https://github.com/puppetlabs/puppet/commit/7fe61647f23650fc4c93cd6b54c654a20ff7c9f9 do not seem present in your debdiff.
4) From the same commit, I see the change to introduce debian_enabled? has been backported, but no callers have. The caller would be in enabled? which looks quite different in 3.8.5, unfortunately, and might imply more history needs to be imported to properly support systemd? Looking through the commits that came before that one in the enabled? function, one specifically mentions lxc and containers, which feels like a relevant case too.

My primary worry is introduction of regression, potentially, for the SRU. My preference, to avoid that, is for us to match upstream patches exactly, if we are going to fold them together. If we are not going to match exactly, then it seems like (to me) it's better to make individual patches and explain why hunks are dropped in the patch message. This (given an ideal upstream project) also makes it easier to know when something was fixed upstream, and if that upstream fix turns out to be a regression-inducer or have other bugs, it might be so tagged upstream too.

@Robie do you have an opinion here or guidance on the typical process?

[Simon Déziel (sdeziel)]

@nacc, the proposed version in Yakkety (4.5.0-4) is also affected. This probably makes sense because upstream only enables systemd handling based on known and hardcoded version numbers. Since 16.10 is not one of those, upstart is implied by default.

[Simon Déziel (sdeziel)]

When I fool puppet into thinking that Yakkety is version 16.04, it works. Here's what I did in a Yakkety container:

# upstart is required to confuse Puppet
apt-get install -y puppet upstart tftpd-hpa # note: puppet-common is gone

# fails
puppet apply -e "service { 'tftpd-hpa': ensure => stopped }"

# fool puppet into thinking that Yakkety is version 16.04
sed -i 's/DISTRIB_RELEASE=.*/DISTRIB_RELEASE=16.04/' /etc/lsb-release

# works
puppet apply -e "service { 'tftpd-hpa': ensure => stopped }"

[Nish Aravamudan (nacc)]

Just an FYI to document what Simon and I discussed on IRC. We did an upstream fix (https://github.com/puppetlabs/puppet/pull/5009) which has been merged to use systemd on 16.10. We can then backport that to the Ubuntu package (Simon is planning on this), which will fix Yakkety. We then can properly backport the pile of patches (all documented plus this new one) and SRU.

[Nish Aravamudan (nacc)]

This latest debdiff has been tested (by me) to fix the testsuite regressions in yakkety-proposed with 4.5.0-4.

I have just uploaded the same to: https://launchpad.net/~nacc/+archive/ubuntu/lp1570472.

@Simon, once the above is finished building, if you could test it in Yakkety, then Robie should be able to sponsor, and we work on the SRU. I will prepare the appropriate debdiff to test for 16.04 shortly.

Thanks!
-Nish

[Nish Aravamudan (nacc)]

@Simon, actually the above build build only fixes the test failure for puppet-module-asciiduck-sssd, the puppet-module-puppetlabs-ntp failure still occurs. However, I noticed that if I drop to a shell in a local LXD based test-run, the run-tests script passes. Do you have any idea what might be going on? I don't know enough about puppet to know how to fix that. It also seems to be an issue with Debian's autopkgtests.

[Simon Déziel (sdeziel)]

@Nish, I used the simple test case from comment #20 on Yakkety:

3.8.5-2: fails
4.5.0-4: fails
4.5.0-4ubuntu1: works

As for the autopkgtest failures, I don't know what's going on either as it works for me too in a LXD container.

[Robie Basak (racb)]

+- defaultfor :operatingsystem => :ubuntu, :operatingsystemmajrelease => ["15.04","15.10","16.04"]
++ defaultfor :operatingsystem => :ubuntu, :operatingsystemmajrelease => ["15.04","15.10","16.04","16.10"]

This will regress as soon as Yakkety+1 opens. Can we fix it properly? The logic should be "systemd for >= 15.04". That may need something better than a set of string comparisons. But doing anything different is clearly broken.

[Robie Basak (racb)]

Or, if you like, reverse it: systemd by default, upstart only for the known set of releases that shipped with upstart. Presumably pre-upstart is before any currently supported release, so we don't need to worry about that.

[Nish Aravamudan (nacc)]

On 27.06.2016 [13:12:17 -0000], Robie Basak wrote:
> +- defaultfor :operatingsystem => :ubuntu, :operatingsystemmajrelease => ["15.04","15.10","16.04"]
> ++ defaultfor :operatingsystem => :ubuntu, :operatingsystemmajrelease => ["15.04","15.10","16.04","16.10"]
>
> This will regress as soon as Yakkety+1 opens. Can we fix it properly?
> The logic should be "systemd for >= 15.04". That may need something
> better than a set of string comparisons. But doing anything different is
> clearly broken.

This logic is "easy" to modify for that, but the other iteration (also
a version list) is not, as it's not using a comparison, but extracting
the value itself.

I have asked upstream in https://github.com/puppetlabs/puppet/pull/5009
about this and your other suggestion.

[Nish Aravamudan (nacc)]

@Robie,

So upstream discussions around this are ongoing. I am working with the puppet community on the "right" fix, which I will pursue into 16.10 via upstream (or via a fresh merge with Debian, depending). Would you be willing to sponsor the fixes here so we can make some progress on this bug in the meanwhile?

-Nish

[Dmitry Burmistrov (dmburmistrov)]

Hello!
This patch/issue doesn't seem really complicated. Do you need some help maybe? I'm hoping proper code would appear in this SRU - proper default provider is "must have" point.

[Nish Aravamudan (nacc)]

@Ivan, @Simon, @Dmitry, or anyone else affected:

I'm trying to understand exactly what is wrong/expected here (not overly familiar with puppet myself).

So Simon's test case was `puppet apply -e "service { 'tftpd-hpa': ensure => stopped }"`.

On 16.04 and 16.10, without 'upstart' installed (but puppet and tftpd-hpa), I get:

puppet apply -e "service { 'tftpd-hpa': ensure => stopped }"
Notice: Compiled catalog for nonfermentable-maribeth.lxd in environment production in 0.14 seconds
Notice: /Stage[main]/Main/Service[tftpd-hpa]/ensure: ensure changed 'running' to 'stopped'
Notice: Finished catalog run in 0.19 seconds

and `service tftpd-hpa status` indicates it is properly stopped.

If I install the 'upstart' package, things break, but that shouldn't be installed by default in 16.04 anymore?

What am I missing? What command reproduces the exact issue you are seeing Ivan?

FWIW, the 16.10 version of puppet has the versioned code already present (although not the bit for 16.10). But again, in 16.10 the same behavior is seen. So I think I need a clearer statement of the problem so I can verify the fix better.

[Ivan Suzdal (isuzdal)]

@Nish, from my point of view it's absolutely expected behavior. In this request, we trying to change default puppet service provider from 'upstart' to 'systemd'. So when you change your init system from 'systemd' to 'upstart' - puppet shall fall. Probably we can add another init system checks, but it's looks like overcomplicating of existing code.

I guess 'upstart' shouldn't be installed by default in 16.04. But I'm not familiar enough with "replacing init system" policies in Ubuntu. It'll be fine if someone else can confirm.

[Robie Basak (racb)]

On Wed, Jul 20, 2016 at 11:13:13AM -0000, Ivan Suzdal wrote:
> I guess 'upstart' shouldn't be installed by default in 16.04. But I'm
> not familiar enough with "replacing init system" policies in Ubuntu.
> It'll be fine if someone else can confirm.

Right now, I don't think the Ubuntu Server Team expects to support
anything other than systemd in releases since 15.04. If you think we
should, please take it up on the mailing list.

But since Debian supports multiple init systems and we inherit that
capability, I have no objection to patches to fix behaviour in the
development release, provided that we also have someone step up to
maintain any Ubuntu delta for this as required.

For fixing server and upstart related bugs in stable releases that ship
with systemd, I guess it probably depends on the individual regression
risk.

I originally thought that this bug affected puppet when running under
systemd as is the default. If this is not the case and only affects
upstart in a pretty unusual and unsupported (by the server team) system
configuration, then though I still welcome patches from users affected,
I would expect the Ubuntu Server Team to focus on other bugs instead.

[Nish Aravamudan (nacc)]

On 20.07.2016 [11:13:13 -0000], Ivan Suzdal wrote:
> @Nish, from my point of view it's absolutely expected behavior. In this
> request, we trying to change default puppet service provider from
> 'upstart' to 'systemd'. So when you change your init system from
> 'systemd' to 'upstart' - puppet shall fall. Probably we can add another
> init system checks, but it's looks like overcomplicating of existing
> code.

Yes, I understand that. But "change default puppet service provider from
'upstart' to 'systemd'" is what I'm referring to -- where do you see
this? I need an explicit test-case so I can figure out where things are
wrong (esp. in the development release) and why.

As I mentioned in my last comment, *if* the upstart package was not
installed to begin with, I see no issues with puppet in 16.04. Was
this a 14.04 -> 16.04 issue?

More clearly, what is the basis (test case) for the OP statement of
"puppet thinks it's still upstart"?

[Ivan Suzdal (isuzdal)]

@Nish, after a superficial RCA I can say here is probably some mistake from my side. Perhaps, in previous release (sadly I can't find particular ISO where I faced with issue) were some parts of upstart ('/sbin/start' for example) which causes issue like in comment #8.
For now I can't reproduce on a clean installation from latest available iso without explicit upstart installation.
So, you must be right about "a 14.04 -> 16.04 issue".

> what is the basis (test case) for the OP statement
Seems like you need to install upstart first.
I didn't make check on a latest ISO's, sorry for that.

[Nish Aravamudan (nacc)]

Alright, so this is a bug, but there is at least one clear workaround if you're on 15.04 or later, remove the 'upstart' package.
However, that's ugly, and I think there is a better fix, as in the attached debdiffs.

Please test https://launchpad.net/~nacc/+archive/ubuntu/lp1570472

Basically, my "fix" is to modify the provider logic in puppet so that systemd is used (regardless of ubuntu's version) if systemd is running (which is the case if /run/systemd/system exists, AIUI).

For 16.10, that is a small change to just remove the version checks for the systemd provider.

For 16.04, that is a larger change to add the confine for /run/systemd/system detection (similar to what is in 16.10, but without the conditional, since this is an Ubuntu package) and add the ubuntu default in the systemd provider.

In my testing, this fixes the issue in 16.04 and 16.10, with or without upstart installed.

I've filed https://tickets.puppetlabs.com/browse/PUP-6531 so hopefully upstream can remove the version checking logic, but in the meanwhile, we should probably simply carry this change in puppet in Ubuntu as a delta.

-Nish

[Nish Aravamudan (nacc)]

To clarify one statement. The workaround only works because the base 'service' provider works regardless of upstart or systemd for tftpd-hpa. Not sure it works generally. So we do want to fix this properly so systemd is leveraged where it is running.

-Nish

[Nish Aravamudan (nacc)]

# apt-cache policy puppet
puppet:
  Installed: 4.5.2-1ubuntu1
  Candidate: 4.5.2-1ubuntu1
  Version table:
 *** 4.5.2-1ubuntu1 500
        500 http://archive.ubuntu.com/ubuntu yakkety/universe amd64 Packages
        100 /var/lib/dpkg/status
# apt-cache policy upstart
upstart:
  Installed: 1.13.2-0ubuntu26
  Candidate: 1.13.2-0ubuntu26
  Version table:
 *** 1.13.2-0ubuntu26 500
        500 http://archive.ubuntu.com/ubuntu yakkety/main amd64 Packages
        100 /var/lib/dpkg/status
# puppet apply -e "service { 'tftpd-hpa': ensure => stopped }"
Notice: Compiled catalog for unputrefied-marleigh.lxd in environment production in 0.22 seconds
Error: /Stage[main]/Main/Service[tftpd-hpa]: Could not evaluate: Execution of '/sbin/status tftpd-hpa' returned 1: status: Unable to connect to Upstart: Failed to connect to socket /com/ubuntu/upstart: Connection refused
Notice: Applied catalog in 0.03 seconds
...
# apt-cache policy puppet
puppet:
  Installed: 4.5.2-1ubuntu2~ppa3
  Candidate: 4.5.2-1ubuntu2~ppa3
  Version table:
 *** 4.5.2-1ubuntu2~ppa3 500
        500 http://ppa.launchpad.net/nacc/lp1570472/ubuntu yakkety/main amd64 Packages
        100 /var/lib/dpkg/status
     4.5.2-1ubuntu1 500
        500 http://archive.ubuntu.com/ubuntu yakkety/universe amd64 Packages
# puppet apply -e "service { 'tftpd-hpa': ensure => stopped }"
Notice: Compiled catalog for unputrefied-marleigh.lxd in environment production in 0.23 seconds
Notice: /Stage[main]/Main/Service[tftpd-hpa]/ensure: ensure changed 'running' to 'stopped'
Notice: Applied catalog in 0.10 seconds

[Nish Aravamudan (nacc)]

# apt-cache policy puppet
puppet:
  Installed: 3.8.5-2
  Candidate: 3.8.5-2
  Version table:
 *** 3.8.5-2 500
        500 http://archive.ubuntu.com/ubuntu xenial/universe amd64 Packages
        100 /var/lib/dpkg/status
# apt-cache policy upstart
upstart:
  Installed: 1.13.2-0ubuntu21.1
  Candidate: 1.13.2-0ubuntu21.1
  Version table:
 *** 1.13.2-0ubuntu21.1 500
        500 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages
        100 /var/lib/dpkg/status
     1.13.2-0ubuntu21 500
        500 http://archive.ubuntu.com/ubuntu xenial/main amd64 Packages
root@ubuntu:~# puppet apply -e "service { 'tftpd-hpa': ensure => stopped }"
Notice: Compiled catalog for untransubstantiated-vallie.lxd in environment production in 0.13 seconds
Error: /Stage[main]/Main/Service[tftpd-hpa]: Could not evaluate: Execution of '/sbin/status tftpd-hpa' returned 1: status: Unable to connect to Upstart: Failed to connect to socket /com/ubuntu/upstart: Connection refused
Notice: Finished catalog run in 0.02 seconds
...
[ ignore the bad version string this matches the debdiff just attached]
# apt-cache policy puppet
puppet:
  Installed: 3.8.5-2ubuntu1~ppa5
  Candidate: 3.8.5-2ubuntu1~ppa5
  Version table:
 *** 3.8.5-2ubuntu1~ppa5 500
        500 http://ppa.launchpad.net/nacc/lp1570472/ubuntu xenial/main amd64 Packages
        100 /var/lib/dpkg/status
     3.8.5-2 500
        500 http://archive.ubuntu.com/ubuntu xenial/universe amd64 Packages
# puppet apply -e "service { 'tftpd-hpa': ensure => stopped }"
Notice: Compiled catalog for untransubstantiated-vallie.lxd in environment production in 0.14 seconds
Notice: /Stage[main]/Main/Service[tftpd-hpa]/ensure: ensure changed 'running' to 'stopped'
Notice: Finished catalog run in 0.19 seconds

[Nish Aravamudan (nacc)]

I'm also working on changing upstream so this is not a delta in the future with: https://github.com/puppetlabs/puppet/pull/5133

[Nish Aravamudan (nacc)]

Updated the patches to match the version upstream that passes CI.

[Dmitry Teselkin (teselkin-d)]

@nacc, I've tested puppet 3.8 from your ppa, and found one issue with it. It looks like 'defaultfor' works well only when it compares values for facts.
:exists doesn't look like a fact, and defaultfor works incorrectly. I fixed it by adding fact :runs_on_systemd and testing its value instead, see attached patch.

Don't know if it applicable for puppet 4.5 though.

Without the patch

# puppet apply -dv --execute 'service{cron:}'
...
Info: Applying configuration version '1469565569'
Debug: Executing '/usr/sbin/service cron status'
Debug: Finishing transaction 30515820
Debug: Storing state
Debug: Stored state in 0.51 seconds
Notice: Finished catalog run in 1.09 seconds
...

With patch applied

# puppet apply -dv --execute 'service{cron:}'
...
Info: Applying configuration version '1469565628'
Debug: Executing '/bin/systemctl is-active cron'
Debug: Finishing transaction 33627360
Debug: Storing state
Debug: Stored state in 0.45 seconds
Notice: Finished catalog run in 1.11 seconds
...

I use Ubuntu 16.04.1, no upstart installed.

[Nish Aravamudan (nacc)]

@Dmitry, thanks for the update! I'll test some other cases and see if we can get it fixed up both upstream and in 16.04!

[Nish Aravamudan (nacc)]

@Dmitry, I've pushed your changes to my tree on GH (https://github.com/puppetlabs/puppet/pull/5133). Thanks!

I reworded it slightly to match what I think makes more sense (running_on_systemd, rather than runs_on_systemd).

I'll submit some new testbuilds to the same PPA with the updates.

Should be:

 4.5.2-1ubuntu2~ppa5 for 16.10
        3.8.5-2ubuntu1~ppa6 for 16.04 (which should actually be 3.8.5-2ubuntu0.1~ppa6, sorry!)

-Nish

[Nish Aravamudan (nacc)]

I apologize, I had a syntax error on my GH commit and thus the backports. Fixed in GH just now (135ecbbcc98d) and uploaded to the PPA:

  4.5.2-1ubuntu2~ppa6 for 16.10
  3.8.5-2ubuntu1~ppa7 for 16.04

Thanks to Martin Stofko for pointing the issue out to me.

[Nish Aravamudan (nacc)]

Apologies for the churn, but the above versions didn't pass GH's auto-checks. Uploaded

4.5.2-1ubuntu2~ppa7
3.8.5-2ubuntu1~ppa8

which should match the current GH tree.

[Neil Hoggarth (neil-hoggarth)]

Upstart is still installed on desktop Xenial systems, even on a fresh installation; the unity-greeter package depends on it. Attempting to "apt-get remove upstart" also offers to remove the "ubuntu-desktop", "unity", and "unity-greeter" packages, along with the "upstart" package.

So this bug prevents the use of Puppet from correctly managing any services on a Xenial desktop system.

[ghomem (gustavo)]

As the previous commenter said, upstart is still installed on default Xenial desktops (with Unity). And those system need to be managed via puppet as well. If you remove upstart the whole desktop breaks.

[ghomem (gustavo)]

I confirm that

puppet-3.8.5-2

from this PPA

https://launchpad.net/~nacc/+archive/ubuntu/lp1570472

resolves the problem, at least on Ubuntu 16.04 x64.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Walter Hop (lifeforms)]

For those who are running into this problem and stumble upon this bug: adding the following to your "site.pp" file is a workaround. It sets 'systemd' as default provider for any service resource.

if $operatingsystem == 'Ubuntu' and versioncmp($operatingsystemrelease, '16.04') >= 0 {
 # Sets the new systemd as default service provider on Ubuntu 16.04 and higher.
 # Works around upstart being used by Puppet.
 # See also: https://bugs.launchpad.net/ubuntu/+source/puppet/+bug/1570472
 Service {
  provider => systemd
 }
}

[Saverio Proto (zioproto)]

I tested the PPA from Nish Aravamudan https://launchpad.net/~nacc/+archive/ubuntu/lp1570472

It fixes the problem for me. Any chance to have this merged to the official packages ?

Saverio

[Nish Aravamudan (nacc)]

Debian has integrated some changes in this area, so I'm refreshing my changes for Zesty, so I can pursue the SRU.

[Nish Aravamudan (nacc)]

Each release is getting a slightly different fix, unfortunately.

Zesty (4.8.2-3ubuntu1) is getting an adaptation of the Debian change which simply makes systemd the default on all Ubuntu and then switches back to upstart if it doesn't see /run/systemd/system (I will coordinate with the Debian maintainers to see if this is going upstream).

Yakkety has a version of my upstreamed change, so we just add 16.10 to the list.

Xenial is a much older codebase, but the minimal fix tested by a few folks via my PPA still applies and is what I have uploaded. It resembles the Debian and upstream fixes and is minimally invasive.

[Saverio Proto (zioproto)]

Hello Nish,
what is the workflow for Xenial ? are we going to see new puppet packages in Xenial Proposed repo soon ?
thank you

[Christian Ehrhardt  (paelzer)]

Hi zioproto,
documenting here as just discussed on LP.

Step 5 of https://wiki.ubuntu.com/StableReleaseUpdates#Procedure is what applies here.
That means Nish has uploaded it and you can see it (at the moment) in e.g.:
https://launchpad.net/ubuntu/yakkety/+queue?queue_state=1
https://launchpad.net/ubuntu/xenial/+queue?queue_state=1

The SRU Team will give it an extra evaluation, the bug will go to Fix Committed and gets an extra comment asking people to test -proposed.

[Robie Basak (racb)]

puppet 4.8.2-3ubuntu1 is stuck in zesty-proposed due to an autopkgtest failure that appears to be related to systemd - so related to this fix? Please check.

[Nish Aravamudan (nacc)]

Unrelated to this fix. It's a networking issue with the tests that I'm trying to resolve (https proxy in a sub-VM).

[Brian Murray (brian-murray)]

Hello Ivan, or anyone else affected,

Accepted puppet into yakkety-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/puppet/4.5.2-1ubuntu1.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Brian Murray (brian-murray)]

Hello Ivan, or anyone else affected,

Accepted puppet into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/puppet/3.8.5-2ubuntu0.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Nish Aravamudan (nacc)]

Tested on both xenial and yakkety with the provided testcase. Without -proposed, the testcase fails with:

Error: /Stage[main]/Main/Service[tftpd-hpa]: Could not evaluate: Execution of '/sbin/status tftpd-hpa' returned 1: status: Unable to connect to Upstart: Failed to connect to socket /com/ubuntu/upstart: Connection refused

With the fix from -proposed:

Notice: /Stage[main]/Main/Service[tftpd-hpa]/ensure: ensure changed 'running' to 'stopped'

Marking verification-done.

[Launchpad Janitor (janitor)]

This bug was fixed in the package puppet - 4.8.2-3ubuntu2

---------------
puppet (4.8.2-3ubuntu2) zesty; urgency=medium

  * d/t/spec/puppet-master-passenger/zz_puppet3_compat_spec.rb: specify
    to curl that 'puppet' should not be proxied.
  * d/t/control: generate a certificate for puppet master if it does not
    already exist, as it is needed by the tests.
  * Drop:
    - d/t/spec/puppet-master/install_spec.rb: generate host certificate
      before attempting to use it. There is no guarantee the certificate
      exist in the autopkgtest environment.
    [ By generating the certificate if not present in d/t/control, we
      don't need to in the spec ]

 -- Nishanth Aravamudan <email address hidden> Thu, 23 Feb 2017 17:18:09 -0800

[Ivan Suzdal (isuzdal)]

@nacc, hello. It's great news!
I've tested package from "-proposed" and can confirm it works. But as an always, it would be nice if someone else could confirm.

[Launchpad Janitor (janitor)]

This bug was fixed in the package puppet - 4.5.2-1ubuntu1.1

---------------
puppet (4.5.2-1ubuntu1.1) yakkety; urgency=medium

  * debian/patches/0004-update_systemd_default_16.10.patch: Indicate
    systemd is also the default on 16.10. Closes LP: #1570472.

 -- Nishanth Aravamudan <email address hidden> Fri, 17 Feb 2017 15:57:21 -0800

[Brian Murray (brian-murray)]

The verification of the Stable Release Update for puppet has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package puppet - 3.8.5-2ubuntu0.1

---------------
puppet (3.8.5-2ubuntu0.1) xenial; urgency=medium

  * debian/patches/0008-Default-to-systemd-if-systemd.patch: systemd is
    the default service provider if it is running. Closes LP: #1570472.

 -- Nishanth Aravamudan <email address hidden> Fri, 17 Feb 2017 16:28:47 -0800

[Anil Shashikumar Belur (askb23)]

I am hitting the same issue even with latest version of puppet4 of 4.9.4 on ubuntu 16.04:

cat file.pp
--------------------------
if $operatingsystem == 'Ubuntu' and versioncmp($operatingsystemrelease, '16.04') >= 0 {
 # Sets the new systemd as default service provider on Ubuntu 16.04 and higher.
 # Works around upstart being used by Puppet.
 # See also: https://bugs.launchpad.net/ubuntu/+source/puppet/+bug/1570472
 Service {
  provider => systemd
 }
}

#include docker
class { 'docker':
  tcp_bind => 'tcp://0.0.0.0:5555',
  extra_parameters => '--bip=10.250.0.254/24',
}
--------------------------

# puppet --version
4.9.4

Error: /Stage[main]/Docker::Service/Service[docker]: Provider upstart is not functional on this host
Notice: Applied catalog in 34.83 seconds

Also tried workaround from #57, which does not seem to have any use. What am I missing ?

[Nish Aravamudan (nacc)]

@askb23: There is no Ubuntu-provided puppet 4.9.4 (not even on 17.04). Please contact upstream puppet for support.