ntp postinst user/group add commands are not idempotent

[Niklas Ögren]

At do release upgrade from 12.04.05 to 14.04.01, the ntp package failed:
Errors were encountered while processing:
 ntp
Error in function:

A fatal error occurred

Please report this as a bug and include the files
/var/log/dist-upgrade/main.log and /var/log/dist-upgrade/apt.log in
your report. The upgrade has aborted.
Your original sources.list was saved in
/etc/apt/sources.list.distUpgrade.

SystemError: E:Sub-process /usr/bin/dpkg returned an error code (1)

Could not install the upgrades

The upgrade has aborted. Your system could be in an unusable state. A
recovery will run now (dpkg --configure -a).

Please report this bug in a browser at
http://bugs.launchpad.net/ubuntu/+source/ubuntu-release-upgrader/+filebug
and attach the files in /var/log/dist-upgrade/ to the bug report.
installArchives() failed

Setting up ntp (1:4.2.6.p5+dfsg-3ubuntu2) ...
dpkg: error processing package ntp (--configure):
 subprocess installed post-installation script returned error exit status 1
Errors were encountered while processing:
 ntp

Upgrade complete

The upgrade has completed but there were errors during the upgrade
process.

In /var/lib/dpkg/info/ntp.postinst the add group / adduser commands fail due to already existing group / user.
Appended “ || /bin/true” to those lines, and “dpkg —configure ntp” succeeded.

[Niklas Ögren (no-o)]

Same error happened now when "apt-get upgrade" tried to upgrade the ntp package to 4.2.6.p5+dfsg-3ubuntu2.

[Niklas Ögren (no-o)]

Same error happened now again when upgrading to 4.2.6.p5+dfsg-3ubuntu2.14.04.3

[Robie Basak (racb)]

Importance -> Low as this appears to only affect one user and so I think probably applies to only unusual system configurations. If we find otherwise we can always bump it.

[Ryan Harper (raharper)]

I booted a precise server image:

ubuntu@ubuntu:~$ cat /etc/issue
Ubuntu 12.04.5 LTS \n \l

ubuntu@ubuntu:~$ uname -a
Linux ubuntu 3.2.0-88-virtual #126-Ubuntu SMP Mon Jul 6 21:50:59 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

Installed ntp:

% sudo apt-get install ntp
...
http://archive.ubuntu.com/ubuntu/ precise-updates/main ntp amd64 1:4.2.6.p3+dfsg-1ubuntu3.4

ubuntu@ubuntu:~$ grep ntp /etc/group
ntp:x:112:

Then do-release-upgrade to move to 14.04 with no issues.

Examining the postinst scripts in current precise ( ), and various trusty versions of the ntp package, the script is always idempotent w.r.t adding the ntp group. It's not clear to me which version of the ntp package had a post-inst script that wasn't idempotent. Getting a copy of the ntp.postinst script that is failing would be most useful.

One workaround to the dist-upgrade issue is to apt-get remove --purge ntp (which will run the postrm hook to remove the ntp group) prior to upgrading.

[Niklas Ögren (no-o)]

Ryan,

Thanks for looking into this.
Yes, the machine in question has been running for many years, might even have started with a non LTS ubuntu before 12.04.
Attached you can find my current copy. It’s a bit weird that it’s not replaced when upgrading to the 12.04 version of ntp package ..

/n

> On 04 Aug 2015, at 21:01, Ryan Harper <email address hidden> wrote:
>
> I booted a precise server image:
>
> ubuntu@ubuntu:~$ cat /etc/issue
> Ubuntu 12.04.5 LTS \n \l
>
> ubuntu@ubuntu:~$ uname -a
> Linux ubuntu 3.2.0-88-virtual #126-Ubuntu SMP Mon Jul 6 21:50:59 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
>
> Installed ntp:
>
> % sudo apt-get install ntp
> ...
> http://archive.ubuntu.com/ubuntu/ precise-updates/main ntp amd64 1:4.2.6.p3+dfsg-1ubuntu3.4
>
> ubuntu@ubuntu:~$ grep ntp /etc/group
> ntp:x:112:
>
> Then do-release-upgrade to move to 14.04 with no issues.
>
> Examining the postinst scripts in current precise ( ), and various
> trusty versions of the ntp package, the script is always idempotent
> w.r.t adding the ntp group. It's not clear to me which version of the
> ntp package had a post-inst script that wasn't idempotent. Getting a
> copy of the ntp.postinst script that is failing would be most useful.
>
> One workaround to the dist-upgrade issue is to apt-get remove --purge
> ntp (which will run the postrm hook to remove the ntp group) prior to
> upgrading.
>
> ** Changed in: ntp (Ubuntu)
> Status: New => Incomplete
>
> ** Changed in: ntp (Ubuntu)
> Assignee: Ryan Harper (raharper) => (unassigned)
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1372051
>
> Title:
> ntp postinst user/group add commands are not idempotent
>
> Status in ntp package in Ubuntu:
> Incomplete
>
> Bug description:
> At do release upgrade from 12.04.05 to 14.04.01, the ntp package failed:
> Errors were encountered while processing:
> ntp
> Error in function:
>
>
> A fatal error occurred
>
> Please report this as a bug and include the files
> /var/log/dist-upgrade/main.log and /var/log/dist-upgrade/apt.log in
> your report. The upgrade has aborted.
> Your original sources.list was saved in
> /etc/apt/sources.list.distUpgrade.
>
> SystemError: E:Sub-process /usr/bin/dpkg returned an error code (1)
>
>
> Could not install the upgrades
>
> The upgrade has aborted. Your system could be in an unusable state. A
> recovery will run now (dpkg --configure -a).
>
> Please report this bug in a browser at
> http://bugs.launchpad.net/ubuntu/+source/ubuntu-release-upgrader/+filebug
> and attach the files in /var/log/dist-upgrade/ to the bug report.
> installArchives() failed
>
> Setting up ntp (1:4.2.6.p5+dfsg-3ubuntu2) ...
> dpkg: error processing package ntp (--configure):
> subprocess installed post-installation script returned error exit status 1
> Errors were encountered while processing:
> ntp
>
> Upgrade complete
>
> The upgrade has completed but there were errors during the upgrade
> process.
>
>
> In /var/lib/dpkg/info/ntp.postinst the add group / adduser commands fail due to already existing grou...

[Ryan Harper (raharper)]

This command:

addgroup --system --quiet ntp

is idempotent on default precise systems, so it's not quite clear why it fails on your system. Maybe it's possible to do some debugging on running that command.

ubuntu@ubuntu:~$ cat /etc/issue
Ubuntu 12.04.5 LTS \n \l

ubuntu@ubuntu:~$ dpkg --list | grep ntp
ii ntp 1:4.2.6.p3+dfsg-1ubuntu3.4 Network Time Protocol daemon and utility programs
ii ntpdate 1:4.2.6.p3+dfsg-1ubuntu3.4 client for setting system time from NTP servers
ubuntu@ubuntu:~$ grep ntp /etc/group
ntp:x:112:
ubuntu@ubuntu:~$ sudo addgroup --system --quiet ntp; echo $?
0

[Niklas Ögren (no-o)]

Ryan,

no@trillian:~$ cat /etc/issue
Ubuntu 14.04.2 LTS \n \l

no@trillian:~$ dpkg --list | grep ntp
ii ntp 1:4.2.6.p5+dfsg-3ubuntu2.14.04.3 amd64 Network Time Protocol daemon and utility programs
ii ntpdate 1:4.2.6.p5+dfsg-3ubuntu2.14.04.3 amd64 client for setting system time from NTP servers
no@trillian:~$ grep ntp /etc/group
ntp:x:1004:
no@trillian:~$ sudo addgroup --system --quiet ntp; echo $?
1

I have run this on the 12.04 system as well, most likely, during that upgrade. But I also saw this issue when updating my 14.04 system, when there was a security patch on ntp. (Using apt-get upgrade)

no@trillian:~$ which addgroup
/usr/sbin/addgroup
no@trillian:~$ addgroup --version
adduser version 3.113+nmu3ubuntu3

Adds a user or group to the system.

Copyright (C) 1997, 1998, 1999 Guy Maor <email address hidden>
Copyright (C) 1995 Ian Murdock <email address hidden>,
                   Ted Hajek <email address hidden>

This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or (at
your option) any later version.

This program is distributed in the hope that it will be useful, but
WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
General Public License, /usr/share/common-licenses/GPL, for more details.
no@trillian:~$ dpkg -S /usr/sbin/addgroup
adduser: /usr/sbin/addgroup
no@trillian:~$ dpkg -l adduser
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-====================================-=======================-=======================-==============================================================================
ii adduser 3.113+nmu3ubuntu3 all add and remove users and groups

/n

> On 04 Aug 2015, at 23:09, Ryan Harper <email address hidden> wrote:
>
> This command:
>
> addgroup --system --quiet ntp
>
> is idempotent on default precise systems, so it's not quite clear why it
> fails on your system. Maybe it's possible to do some debugging on
> running that command.
>
> ubuntu@ubuntu:~$ cat /etc/issue
> Ubuntu 12.04.5 LTS \n \l
>
> ubuntu@ubuntu:~$ dpkg --list | grep ntp
> ii ntp 1:4.2.6.p3+dfsg-1ubuntu3.4 Network Time Protocol daemon and utility programs
> ii ntpdate 1:4.2.6.p3+dfsg-1ubuntu3.4 client for setting system time from NTP servers
> ubuntu@ubuntu:~$ grep ntp /etc/group
> ntp:x:112:
> ubuntu@ubuntu:~$ sudo addgroup --system --quiet ntp; echo $?
> 0
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1372051
>
> Title:
> ntp postinst user/group add commands are not idempotent...

[Ryan Harper (raharper)]

addgroup is a perl script, so you can inspect it to see what's going on. You can get additional information about the error running with:

sudo VERBOSE=1 addgroup --system ntp

I think what we're seeing is that your ntp group is a user group (gid >= 1000) instead of a system group (gid < 1000).

From your output above, it looks like you're hitting this logic:

if ($action eq "addsysgroup") {

    # Check if requested group already exists and we can exit safely
    my $ret = existing_group_ok($new_name, $new_gid);

    if ($ret == 3) {
        print STDERR "$0: " if $verbose;
        printf STDERR (gtx("The group `%s' already exists as a system group. Exiting.\n"), $new_name) if $verbose;
        exit RET_OK;
    }

    if ($ret == 1) {
        print STDERR "$0: " if $verbose;
        printf STDERR (gtx("The group `%s' already exists and is not a system group. Exiting.\n"), $new_name) if $verbose;
        exit RET_OBJECT_ALREADY_EXISTS;
    }

So, you're exiting 1 due to the gid of group ntp being > 1000. The fix here would be to apt-get remove --purge ntp ; the --purge flag will trigger the ntp.postrm script to remove the group first. Then when you re-install, it will install the ntp group with --system flag and allocate a GID under <1000.

See if that helps you moving forward.

[Niklas Ögren (no-o)]

Ryan,

Yes you are right. My passwd/group files has been alive for a long time ..

Case can be closed.

/n

Quoting Ryan Harper <email address hidden>:

> addgroup is a perl script, so you can inspect it to see what's going on.
> You can get additional information about the error running with:
>
> sudo VERBOSE=1 addgroup --system ntp
>
> I think what we're seeing is that your ntp group is a user group (gid >=
> 1000) instead of a system group (gid < 1000).
>
>> From your output above, it looks like you're hitting this logic:
>
>
> if ($action eq "addsysgroup") {
>
> # Check if requested group already exists and we can exit safely
> my $ret = existing_group_ok($new_name, $new_gid);
>
> if ($ret == 3) {
> print STDERR "$0: " if $verbose;
> printf STDERR (gtx("The group `%s' already exists as a
> system group. Exiting.\n"), $new_name) if $verbose;
> exit RET_OK;
> }
>
> if ($ret == 1) {
> print STDERR "$0: " if $verbose;
> printf STDERR (gtx("The group `%s' already exists and is not
> a system group. Exiting.\n"), $new_name) if $verbose;
> exit RET_OBJECT_ALREADY_EXISTS;
> }
>
>
> So, you're exiting 1 due to the gid of group ntp being > 1000. The
> fix here would be to apt-get remove --purge ntp ; the --purge flag
> will trigger the ntp.postrm script to remove the group first. Then
> when you re-install, it will install the ntp group with --system
> flag and allocate a GID under <1000.
>
> See if that helps you moving forward.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1372051
>
> Title:
> ntp postinst user/group add commands are not idempotent
>
> Status in ntp package in Ubuntu:
> Incomplete
>
> Bug description:
> At do release upgrade from 12.04.05 to 14.04.01, the ntp package failed:
> Errors were encountered while processing:
> ntp
> Error in function:
>
>
> A fatal error occurred
>
> Please report this as a bug and include the files
> /var/log/dist-upgrade/main.log and /var/log/dist-upgrade/apt.log in
> your report. The upgrade has aborted.
> Your original sources.list was saved in
> /etc/apt/sources.list.distUpgrade.
>
> SystemError: E:Sub-process /usr/bin/dpkg returned an error code (1)
>
>
> Could not install the upgrades
>
> The upgrade has aborted. Your system could be in an unusable state. A
> recovery will run now (dpkg --configure -a).
>
> Please report this bug in a browser at
> http://bugs.launchpad.net/ubuntu/+source/ubuntu-release-upgrader/+filebug
> and attach the files in /var/log/dist-upgrade/ to the bug report.
> installArchives() failed
>
> Setting up ntp (1:4.2.6.p5+dfsg-3ubuntu2) ...
> dpkg: error processing package ntp (--configure):
> subprocess installed post-installation script returned error exit status 1
> Errors were encountered while processing:
> ntp
>
> Upgrade complete
>
> The upgrade has completed but there were errors during the upgrade
> process.
>
>
> In /var/lib/dpkg/info/ntp.postinst the add group / adduser
> commands fail due to already existing group / user.
> Append...

[Robie Basak (racb)]

Thanks Niklas, marking Invalid as advised.