Request to backport sosreport v3.5

Bug #1734983 reported by Eric Desrochers on 2017-11-28
20
This bug affects 1 person
Affects Status Importance Assigned to Milestone
sosreport
New
Unknown
sosreport (Debian)
New
Unknown
sosreport (Ubuntu)
Status tracked in Bionic
Trusty
Medium
Eric Desrochers
Xenial
Medium
Eric Desrochers
Zesty
Medium
Eric Desrochers
Artful
Medium
Eric Desrochers
Bionic
Medium
Eric Desrochers

Bug Description

[Impact]

Canonical support team (AKA STS) largely depend on sosreport package to troubleshoot system.

sosreport is always in constant development including new bugfixes, new features such as new plugins[1], .... that can be interesting to have in a support context.

v3.5 is already found in devel release (bionic). We create this LP in the attempt to justify the SRU of v3.5 backport into current supported stable releases.

[1] - New plugins for :
        * perl
        * boom
        * vdo
        * os_net_config
        * conntrackd
        * ovirt_imageio
        * nss
        * sas3ircu
        * openstack_aodh
        * docker_distribution
        * gluster_block
        * snappy

    Plugin API enhancements
        * Plugin triggers by executable name
        * Improved log size limit handling
        * Better handling of compressed log files
        * Per-plugin package verification lists

    Updates to 227 plugins

This will also allow us to close some sosreport LP bugs:
*Docker plugin uses the wrong command for Ubuntu (LP: #1693574)

[Test Case]

 * Install sosreport
   $ apt-get install sosreport

 * Run sosreport
   $ sosreport -a
   $ sosreport -o <PLUGIN>
   $ ...

 * Make sure it generate a tar.xz file in /tmp in the form of
   "/tmp/sosreport-<HOSTNAME>-<DATE>.tar.xz"

 * Extract files from archive
   $ tar Jxvf /tmp/sosreport-<HOSTNAME>-<DATE>.tar.xz

 * Look the content of sosreport, make sure the information is accurate and valid, ....

 * Make sure all files that should to be collected by sosreport aren't 0 size. (Files that should be collected may varies from one system to another, it depends on what packages are installed, ...)
   $ find /tmp/sosreport-<HOSTNAME>-<DATE> -size 0

# Note :
It is normal to see some 0 size files here and there, again it depend on how the plugin is run (package detection or not inside the plugins, ...) and if the package is installed or not on the system where you run sosreport. But in most cases, if the package is installed and if sosreport has a plugin for it then it should gather information, if not then it might be a problem with the plugins itself that need adjustments.

[Regression Potential]

 * autopkgtest[2] didn't reveal any
[2] - See Comment #1

 * We,STS, have intensively tested the package.

During our testing, we found a severe regression that we already took action to fix it via :

- Upstream issue - https://github.com/sosreport/sos/issues/1155
- Debian bug - https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=883537
- Fix Released in Ubuntu Devel release (bionic) via debdiff : lp1734983_bionic.debdiff

[Other Info]

 * Sosreport upstream :
   https://github.com/sosreport/sos

 * rmadison -u debian,ubuntu sosreport
debian:
sosreport | 3.2-2 | oldstable | source, amd64, arm64, armel, armhf, i386, mips, mipsel, powerpc, ppc64el, s390x
sosreport | 3.2-2 | oldstable-kfreebsd | source, kfreebsd-amd64, kfreebsd-i386
sosreport | 3.3+git50-g3c0349b-2 | stable | source, amd64, arm64, armel, armhf, i386, mips, mips64el, mipsel, ppc64el, s390x
sosreport | 3.5-1 | testing | source, amd64, arm64, armel, armhf, i386, mips, mips64el, mipsel, ppc64el, s390x
sosreport | 3.5-1 | unstable | source, amd64, arm64, armel, armhf, hurd-i386, i386, kfreebsd-amd64, kfreebsd-i386, mips, mips64el, mipsel, powerpc, ppc64el, s390x
ubuntu:
 sosreport | 3.0-1~ubuntu12.04.1 | precise-backports/universe | source, amd64, armel, armhf, i386, powerpc
 sosreport | 3.1-1ubuntu2 | trusty | source, amd64, arm64, armhf, i386, powerpc, ppc64el
 sosreport | 3.1-1ubuntu2.2 | trusty-security | source, amd64, arm64, armhf, i386, powerpc, ppc64el
 sosreport | 3.2-2~ubuntu14.04.1 | trusty-backports | source, amd64, arm64, armhf, i386, powerpc, ppc64el
 sosreport | 3.2+git276-g7da50d6-3ubuntu1 | xenial | source, amd64, arm64, armhf, i386, powerpc, ppc64el, s390x
 sosreport | 3.3+git50-g3c0349b-2 | zesty | source, amd64, arm64, armhf, i386, ppc64el, s390x
 sosreport | 3.4-1~ubuntu14.04.1 | trusty-updates | source, amd64, arm64, armhf, i386, powerpc, ppc64el
 sosreport | 3.4-1~ubuntu16.04.1 | xenial-updates | source, amd64, arm64, armhf, i386, powerpc, ppc64el, s390x
 sosreport | 3.4-1~ubuntu17.04.1 | zesty-updates | source, amd64, arm64, armhf, i386, ppc64el, s390x
 sosreport | 3.4-1 | artful | source, amd64, arm64, armhf, i386, ppc64el, s390x
 sosreport | 3.5-1 | bionic | source, amd64, arm64, armhf, i386, ppc64el, s390x

Eric Desrochers (slashd) on 2017-11-28
Changed in sosreport (Ubuntu Bionic):
status: New → Fix Released
tags: added: sts
Eric Desrochers (slashd) on 2017-11-28
summary: - Request to backport upstream release v3.5
+ Request to backport sosreport v3.5
Eric Desrochers (slashd) wrote :

* autopkgtest result (All manual autopkgtest runs (pre-SRU )passed)

[ARTFUL]
$ /usr/bin/adt-run 'sosreport_3.5-1~ubuntu17.10.1.dsc' --- lxc adt-artful-amd64
adt-run [14:16:01]: @@@@@@@@@@@@@@@@@@@@ summary
sos-run-nonroot.py PASS
sos-run.py PASS

[ZESTY]
$ /usr/bin/adt-run 'sosreport_3.5-1~ubuntu17.04.1.dsc' --- lxc adt-zesty-amd64
adt-run [14:38:58]: @@@@@@@@@@@@@@@@@@@@ summary
sos-run-nonroot.py PASS
sos-run.py PASS

[XENIAL]
$ /usr/bin/adt-run 'sosreport_3.5-1~ubuntu16.04.1.dsc' --- lxc adt-xenial-amd64
adt-run [14:46:21]: @@@@@@@@@@@@@@@@@@@@ summary
sos-run-nonroot.py PASS
sos-run.py PASS

[TRUSTY]
$ /usr/bin/adt-run 'sosreport_3.5-1~ubuntu14.04.1.dsc' --- lxc adt-trusty-amd64
adt-run [14:52:19]: @@@@@@@@@@@@@@@@@@@@ summary
sos-run-nonroot.py PASS
sos-run.py PASS

description: updated
Eric Desrochers (slashd) on 2017-11-28
description: updated
description: updated
Eric Desrochers (slashd) on 2017-11-28
description: updated
description: updated
Bryan Quigley (bryanquigley) wrote :

Found a severe regression in that it appears many packages = lines no longer work which prevent automatic package detection. Bionic is also affected.

This in an upstream issue and investigating is ongoing.

Eric Desrochers (slashd) wrote :

I can reproduce the situation as well with sosreport upstream and v3.5 debian package that I have built for pre-SRU testing.

Example:
Plugin such as [acpid], [boot], ... are no longer running with 'sosreport -a' even if "packages=" are installed.

Let's put the SRU on hold until we or upstream figure this out.

- Eric

Eric Desrochers (slashd) wrote :

Seems like something is broken in the "package=" mechanism behaviour.
The no longer running plugins that we have identified coincide with recent package add commit :

18ab068 [dbus] add missing packages list
3d0ea25 [cron] add missing packages list
0348833 [acpid] add missing packages list
....

After a bisect the upstream offending commit is :
82e4ee2 [sosreport] add per-plugin package verification lists

- Eric

Eric Desrochers (slashd) on 2017-11-28
description: updated
Changed in sosreport:
status: Unknown → New
Eric Desrochers (slashd) on 2017-12-04
description: updated
Changed in sosreport (Debian):
status: Unknown → New
Eric Desrochers (slashd) wrote :

Here's the debdiff fixing the regression found by Bryan Quigley.

I talked with the SRU team and it is okay to proceed with the upload of this fix in Devel and Stable release if waiting too long for upstream to merge.

In this case, the fix will likely be merge in April 2018 for v3.6, and by then we might be in a freeze period for 18.04. So let's make sure we have an healthy sosreport (v3.5) package before 18.04 get release.

- Eric

description: updated
Eric Desrochers (slashd) on 2017-12-07
Changed in sosreport (Ubuntu Bionic):
status: Fix Released → In Progress
assignee: nobody → Eric Desrochers (slashd)
importance: Undecided → Medium
tags: added: patch
Eric Desrochers (slashd) on 2017-12-08
Changed in sosreport (Ubuntu Bionic):
status: In Progress → Fix Released
tags: removed: patch
Eric Desrochers (slashd) on 2017-12-08
Changed in sosreport (Ubuntu Artful):
status: New → In Progress
Changed in sosreport (Ubuntu Zesty):
status: New → In Progress
Changed in sosreport (Ubuntu Xenial):
status: New → In Progress
Changed in sosreport (Ubuntu Trusty):
status: New → In Progress
assignee: nobody → Eric Desrochers (slashd)
Changed in sosreport (Ubuntu Xenial):
assignee: nobody → Eric Desrochers (slashd)
Changed in sosreport (Ubuntu Zesty):
assignee: nobody → Eric Desrochers (slashd)
Changed in sosreport (Ubuntu Artful):
assignee: nobody → Eric Desrochers (slashd)
importance: Undecided → Medium
Changed in sosreport (Ubuntu Zesty):
importance: Undecided → Medium
Changed in sosreport (Ubuntu Xenial):
importance: Undecided → Critical
importance: Critical → Medium
Changed in sosreport (Ubuntu Trusty):
importance: Undecided → Medium
Eric Desrochers (slashd) wrote :

v3.5 sponsored for AZXT, waiting for SRU verification team to approve to start building in -proposed.

- Eric

Eric Desrochers (slashd) wrote :

For SRU,

I have already contacted sil2100 that should take care of it, but just in case of someone else have a look at the upload and wonder why there is 2 uploads for the same thing.

I have uploaded the sosreport package twice for each release because I forgot to add the (LP: #XXXX) syntax in the d/changelog.

For this reason, I have to upload the package a second time (keeping the same version).

Feel free to reject the less recent upload and proceed with the most recent ones for AZXT.

Regards,
Eric

description: updated
Eric Desrochers (slashd) on 2018-01-03
description: updated
Eric Desrochers (slashd) wrote :

sil2100 has rejected the first upload and kept the second which include the (LP: #XXXXX) syntax in d/changelog.

Things are a bit blocked everywhere. Accepting the uploads now will do nothing as we have no builders. sil2100 will review the uploads later when there will be some builders available.

- Eric

Hello Eric, or anyone else affected,

Accepted sosreport into artful-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/sosreport/3.5-1~ubuntu17.10.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-artful to verification-done-artful. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-artful. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in sosreport (Ubuntu Artful):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-artful
Changed in sosreport (Ubuntu Zesty):
status: In Progress → Fix Committed
tags: added: verification-needed-zesty
Łukasz Zemczak (sil2100) wrote :

Hello Eric, or anyone else affected,

Accepted sosreport into zesty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/sosreport/3.5-1~ubuntu17.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-zesty to verification-done-zesty. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-zesty. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in sosreport (Ubuntu Xenial):
status: In Progress → Fix Committed
tags: added: verification-needed-xenial
Łukasz Zemczak (sil2100) wrote :

Hello Eric, or anyone else affected,

Accepted sosreport into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/sosreport/3.5-1~ubuntu16.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in sosreport (Ubuntu Trusty):
status: In Progress → Fix Committed
tags: added: verification-needed-trusty
Łukasz Zemczak (sil2100) wrote :

Hello Eric, or anyone else affected,

Accepted sosreport into trusty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/sosreport/3.5-1~ubuntu14.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-trusty to verification-done-trusty. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-trusty. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

David Coronel (davecore) wrote :

I tested installing the new package in trusty, xenial, zesty and artful in LXD containers. I ran sosreport with the -a option and I didn't notice any errors. Made a few spot checks. LGTM.

===================
Trusty

# dpkg -l | grep -i sosreport
ii sosreport 3.5-1~ubuntu14.04.1 amd64 Set of tools to gather troubleshooting data from a system

# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 14.04.5 LTS
Release: 14.04
Codename: trusty

# cat /tmp/sosreport-trustysosreport-20180115221214/sos_logs/ui.log
2018-01-15 22:12:13,702 INFO:
sosreport (version 3.5)

2018-01-15 22:12:14,802 INFO: Setting up archive ...
2018-01-15 22:12:14,804 INFO: Setting up plugins ...
2018-01-15 22:12:15,087 INFO: Running plugins. Please wait ...
2018-01-15 22:12:15,087 INFO:
2018-01-15 22:12:35,831 INFO:

===================
Xenial

# dpkg -l | grep -i sosreport
ii sosreport 3.5-1~ubuntu16.04.1 amd64 Set of tools to gather troubleshooting data from a system

# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04.3 LTS
Release: 16.04
Codename: xenial

# cat /tmp/sosreport-xenialsosreport-20180115221212/sos_logs/ui.log
2018-01-15 22:12:10,954 INFO:
sosreport (version 3.5)

2018-01-15 22:12:12,008 INFO: Setting up archive ...
2018-01-15 22:12:12,009 INFO: Setting up plugins ...
2018-01-15 22:12:12,221 INFO: Running plugins. Please wait ...
2018-01-15 22:12:12,221 INFO:
2018-01-15 22:12:20,600 INFO:

===================
Zesty

# dpkg -l | grep -i sosreport
ii sosreport 3.5-1~ubuntu17.04.1 amd64 Set of tools to gather troubleshooting data from a system

# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 17.04
Release: 17.04
Codename: zesty

# cat /tmp/sosreport-zestysosreport-20180115221209/sos_logs/ui.log
2018-01-15 22:12:08,035 INFO:
sosreport (version 3.5)

2018-01-15 22:12:09,512 INFO: Setting up archive ...
2018-01-15 22:12:09,514 INFO: Setting up plugins ...
2018-01-15 22:12:09,676 INFO: Running plugins. Please wait ...
2018-01-15 22:12:09,676 INFO:
2018-01-15 22:12:19,021 INFO:

===================
Artful

# dpkg -l | grep -i sosreport
ii sosreport 3.5-1~ubuntu17.10.1 amd64 Set of tools to gather troubleshooting data from a system

# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 17.10
Release: 17.10
Codename: artful

# more /tmp/sosreport-artfulsosreport-20180115221206/sos_logs/ui.log
2018-01-15 22:12:01,146 INFO:
sosreport (version 3.5)

2018-01-15 22:12:06,482 INFO: Setting up archive ...
2018-01-15 22:12:06,488 INFO: Setting up plugins ...
2018-01-15 22:12:06,640 INFO: Running plugins. Please wait ...
2018-01-15 22:12:06,640 INFO:
2018-01-15 22:12:14,803 INFO:

Alex Moldovan (alexmoldovan) wrote :

Marked Zesty as invalid as it reached end of like on Jan 13 2018.

Changed in sosreport (Ubuntu Zesty):
status: Fix Committed → Invalid
Alex Moldovan (alexmoldovan) wrote :

Testing done for Trusty, Xenial and Arful with no errors noticed.

Doug Parrish (dparrish) wrote :

Ran 3.5 version for Trusty, Xenial, and Artful. No problems appear to have been introduced by this version.

Chris Newcomer (chris-8k) wrote :

Ran 3.5 version for Trusty, Xenial, and Artful on a VM that was MAAS-deployed. I have not been able to discover any issues. I also installed it on a bare-metal server running Xenial with no problems as well.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.