[sru] sos upstream 4.2

Bug #1941745 reported by Eric Desrochers
16
This bug affects 1 person
Affects Status Importance Assigned to Milestone
sosreport (Debian)
Fix Released
Unknown
sosreport (Ubuntu)
Fix Released
Medium
Eric Desrochers
Bionic
Confirmed
Low
Unassigned
Focal
Fix Released
Medium
Eric Desrochers
Hirsute
Fix Released
Medium
Eric Desrochers
Impish
Fix Released
Medium
Eric Desrochers
Jammy
Fix Released
Medium
Eric Desrochers

Bug Description

[IMPACT]

The sos team is pleased to announce the release of sos-4.2. This release contains numerous improvements and bug fixes to several components within sos, including an overhaul to the project's test suite and infrastructure.

[TEST PLAN]

Documentation for Special Cases:
https://wiki.ubuntu.com/SosreportUpdates

[WHERE PROBLEM COULD OCCUR]

Regression could occur at core functionality, which may prevent sos (or its subcommand to work. I consider this regression type as 'low'. That is generally well tested, and we would find a problem at an early stage during the verification phase if it is the case.

On the other end, regression could happen and are some kind of expected at plugins levels. As of today, sos has more than 300 plugins. It is nearly impossible to test them all.

If a regression is found in a plugin, it is rarely affecting sos core functionalities nor other plugins. So mainly the impact would be limited to that plugin. The impact being that the plugin can't or partially can collect the information that it is instructed to gather.

A 3rd party vendor would then ask user/customer to collect the information manually for that particular plugins.

Plugins are segmented by services and/or applications (e.g. openstack_keystone, bcache, system, logs, ...) in order to collect things accordingly to the plugin detected or intentionally requested for.

Sosreport plugins philosophy is to (as much as possible) maintain backward compatibility when updating a plugin. The risk that an ancient version of a software has been dropped, is unlikely, unless it was intended to be that way for particular reasons. Certain plugin also support the DEB installation way and the snap one (MAAS, LXD, ...) so all Ubuntu standard installation types are covered.

[OTHER INFORMATION]

Release note:
https://github.com/sosreport/sos/releases/tag/4.2

Debian RFS:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=994564

Revision history for this message
Eric Desrochers (slashd) wrote :

# Note

Currently facing a problem with dh_auto_test: while running test w/ avocado-framework.

Avocado is not packaged in Ubuntu/Debian, and requires to be installed via 'pip3 install avocado-framework' which doesn't seem to be accepted by the builder.

Tried severals scenarios such as:

# d/rules:
override_dh_auto_test:
        pip3 install avocado-framework
 export PYTHONPATH=tests/
 avocado run -t stageone tests/{cleaner,collect,report,vendor}_tests

but got the following error at build time from my PPA:

debian/rules override_dh_auto_test
make[1]: Entering directory '/<<PKGBUILDDIR>>'
pip3 install avocado-framework
WARNING: The directory '/sbuild-nonexistent/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f5a4cd153d0>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/avocado-framework/
WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f5a4cc68cd0>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/avocado-framework/
WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f5a4cc680a0>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/avocado-framework/
WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f5a4cc685b0>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/avocado-framework/
WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f5a4cc68760>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/avocado-framework/
ERROR: Could not find a version that satisfies the requirement avocado-framework
ERROR: No matching distribution found for avocado-framework

Still under investigation.

I might have no other option but to package avocado-framework and add it as a Build-Depends, as I'm not a fan of disabling dh_auto_test: .

- Eric

tags: added: seg sts
Revision history for this message
Eric Desrochers (slashd) wrote :

While I figure out the 'avocado' portion on the test build.

I have a package ready, produces by skipping the tests at build time that can be found here:

sudo add-apt-repository ppa:slashd/sos42test
sudo apt-get update

Revision history for this message
Eric Desrochers (slashd) wrote :

Other possible option is to revert the avocado change that has been made between 4.1 and 4.2 and continue the former way by reverting the necessary commits....

Example:
e325f38e [tests] use Avocado "recursive" behavior by default
f568fb21 [tests] Start using avocado for test suite

Revision history for this message
Eric Desrochers (slashd) wrote :

I'll evaluate how feasible avocado-framework can be "debianized" for impish (AKA active devel release), and test.

So that sos4.2 will have the necessary bits for its testing exactly like upstream does.
Then, I'll review what can be done for stable release. I'm afraid I won't be able to introduce a new package in stable release like I can do in devel.

Stay tuned !!

Changed in sosreport (Ubuntu Impish):
assignee: nobody → Eric Desrochers (slashd)
Revision history for this message
Eric Desrochers (slashd) wrote :

Still working on making the 'testsuite' working with avocado-framework.
impish reaches a few freezes at this point.

I'll continue the work, but will likely wait until 'k' become official devel release.

As I'll have to push avocado-framework as well as a new package.

Changed in sosreport (Ubuntu Impish):
assignee: Eric Desrochers (slashd) → nobody
Revision history for this message
Eric Desrochers (slashd) wrote :

I'll probably drop 'avocado-framework' testing for now and keep the nosetest.

override_dh_auto_test:
    nosetests3 -v --with-cover --cover-package=sos tests/unittests

Eric Desrochers (slashd)
description: updated
Changed in sosreport (Debian):
status: Unknown → New
Eric Desrochers (slashd)
description: updated
description: updated
Eric Desrochers (slashd)
Changed in sosreport (Ubuntu Jammy):
status: New → In Progress
assignee: nobody → Eric Desrochers (slashd)
importance: Undecided → Medium
Eric Desrochers (slashd)
summary: - [FFe][sync][sru] sos upstream 4.2
+ [sru] sos upstream 4.2
Eric Desrochers (slashd)
Changed in sosreport (Ubuntu Impish):
status: New → In Progress
assignee: nobody → Eric Desrochers (slashd)
importance: Undecided → Medium
Revision history for this message
Steve Langasek (vorlon) wrote : Please test proposed package

Hello Eric, or anyone else affected,

Accepted sosreport into impish-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/sosreport/4.2-1ubuntu0.21.10.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-impish to verification-done-impish. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-impish. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in sosreport (Ubuntu Impish):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-impish
Eric Desrochers (slashd)
Changed in sosreport (Ubuntu Hirsute):
status: New → In Progress
importance: Undecided → Medium
assignee: nobody → Eric Desrochers (slashd)
description: updated
Eric Desrochers (slashd)
Changed in sosreport (Ubuntu Focal):
status: New → Confirmed
Changed in sosreport (Ubuntu Bionic):
status: New → Incomplete
status: Incomplete → Confirmed
Revision history for this message
Brian Murray (brian-murray) wrote :

Hello Eric, or anyone else affected,

Accepted sosreport into hirsute-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/sosreport/4.2-1ubuntu0.21.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-hirsute to verification-done-hirsute. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-hirsute. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in sosreport (Ubuntu Hirsute):
status: In Progress → Fix Committed
tags: added: verification-needed-hirsute
Revision history for this message
Ubuntu SRU Bot (ubuntu-sru-bot) wrote : Autopkgtest regression report (sosreport/4.2-1ubuntu0.21.04.1)

All autopkgtests for the newly accepted sosreport (4.2-1ubuntu0.21.04.1) for hirsute have finished running.
The following regressions have been reported in tests triggered by the package:

sosreport/4.2-1ubuntu0.21.04.1 (s390x, amd64, ppc64el)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/hirsute/update_excuses.html#sosreport

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

Mathew Hodson (mhodson)
tags: added: upgrade-software-version
Changed in sosreport (Ubuntu Bionic):
importance: Undecided → Low
Changed in sosreport (Ubuntu Focal):
importance: Undecided → Low
Eric Desrochers (slashd)
Changed in sosreport (Ubuntu Focal):
status: Confirmed → Triaged
status: Triaged → In Progress
assignee: nobody → Eric Desrochers (slashd)
importance: Low → Medium
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package sosreport - 4.2-1ubuntu2

---------------
sosreport (4.2-1ubuntu2) jammy; urgency=medium

  * d/p/0006-report-check-for-symlink-before-rmtree.patch:
    - Fixing --estimate-only option by checking if the dirs
      are also symlink before performing rmtree() method so
      that unlink() method can be used instead.

 -- Eric Desrochers <email address hidden> Tue, 19 Oct 2021 14:20:44 -0400

Changed in sosreport (Ubuntu Jammy):
status: In Progress → Fix Released
Revision history for this message
Łukasz Zemczak (sil2100) wrote : Please test proposed package

Hello Eric, or anyone else affected,

Accepted sosreport into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/sosreport/4.2-1ubuntu0.20.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in sosreport (Ubuntu Focal):
status: In Progress → Fix Committed
tags: added: verification-needed-focal
Revision history for this message
Ubuntu SRU Bot (ubuntu-sru-bot) wrote : Autopkgtest regression report (sosreport/4.2-1ubuntu0.20.04.1)

All autopkgtests for the newly accepted sosreport (4.2-1ubuntu0.20.04.1) for focal have finished running.
The following regressions have been reported in tests triggered by the package:

sosreport/4.2-1ubuntu0.20.04.1 (ppc64el, amd64)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/focal/update_excuses.html#sosreport

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

Revision history for this message
Eric Desrochers (slashd) wrote :

[VERIFICATION HIRSUTE/IMPISH]

Since theses 2 releases are non-LTS. They are less exposed to various context & environment types such as: Openstack, Ceph, k8s, JuJu, MAAS, Landscape, ...

It's safe to say that non-LTS should be limited to general sosreport run, most likely in the Desktop area.

For that reasons, I don't judge we need to spend time testing non-LTS for such context/environments that typically belongs to LTS version.

The autopkgtest (simple.sh) was run successfully, and it already does a lot of general sosreport runs testing. I also tested sosreport myself on H/I and didn't see any problem.

With that being said, if someone is keen to do more test, please feel free to do so, but as the maintainer, I consider the verification done for H/I.

- Eric

tags: added: verification-done-hirsute verification-done-impish
removed: verification-needed-hirsute verification-needed-impish
Revision history for this message
Eric Desrochers (slashd) wrote :

Let's focus our time testing Focal/20.04LTS, which can cover various areas:
Openstack, Ceph, k8s, MAAS, juju, Landscape, ... (to only name those)

- Eric

Revision history for this message
Łukasz Zemczak (sil2100) wrote :

Ok, I think the rationale seems sane. Let's proceed.

Revision history for this message
Łukasz Zemczak (sil2100) wrote : Update Released

The verification of the Stable Release Update for sosreport has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package sosreport - 4.2-1ubuntu0.21.04.1

---------------
sosreport (4.2-1ubuntu0.21.04.1) hirsute; urgency=medium

  * New 4.2 upstream release. (LP: #1941745)
    - This release contains numerous improvements
      and bug fixes to several components within sos,
      including an overhaul to the project's test suite
      and infrastructure.

  * For more details, full release note is available here:
    - https://github.com/sosreport/sos/releases/tag/4.2

  * Remaining patches:
   - d/p/0001-debian-change-tmp-dir-location.patch

  * New patches:
    - d/p/0002-report-implement_estimate-only.patch
    - d/p/0003-ceph-add-support-for-containerized-ceph-setup.patch
    - d/p/0004-ceph-split-plugin-by-components.patch
    - d/p/0005-openvswitch-get-userspace-datapath-implementations.patch
    - d/p/0006-report-check-for-symlink-before-rmtree.patch

  * Former patches, now fixed:
    - d/p/0002-clean-prevent-parsing-ubuntu-user.patch
    - d/p/0003-ubuntu-policy-fix-upload.patch
    - d/p/0004-chrony-configuration-can-now-be-fragmented.patch
    - d/p/0005-global-drop-plugin-version.patch
    - d/p/0006-networking-check-presence-of-devlink.patch
    - d/p/0007-sosnode-avoid-checksum-cleanup-if-no-archive.patch

  * d/control:
   - Add 'python3-coverage' as part of the build depends.

  * d/rules:
   - Fix misplaced and duplicated sos.conf file in /usr/config.

 -- Eric Desrochers <email address hidden> Sun, 24 Oct 2021 13:16:22 -0400

Changed in sosreport (Ubuntu Hirsute):
status: Fix Committed → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package sosreport - 4.2-1ubuntu0.21.10.1

---------------
sosreport (4.2-1ubuntu0.21.10.1) impish; urgency=medium

  * New 4.2 upstream release. (LP: #1941745)
    - This release contains numerous improvements
      and bug fixes to several components within sos,
      including an overhaul to the project's test suite
      and infrastructure.

  * For more details, full release note is available here:
    - https://github.com/sosreport/sos/releases/tag/4.2

  * Remaining patches:
   - d/p/0001-debian-change-tmp-dir-location.patch

  * New patches:
    - d/p/0002-report-implement_estimate-only.patch
    - d/p/0003-ceph-add-support-for-containerized-ceph-setup.patch
    - d/p/0004-ceph-split-plugin-by-components.patch
    - d/p/0005-openvswitch-get-userspace-datapath-implementations.patch
    - d/p/0006-report-check-for-symlink-before-rmtree.patch

  * Former patches, now fixed:
    - d/p/0002-clean-prevent-parsing-ubuntu-user.patch
    - d/p/0003-ubuntu-policy-fix-upload.patch
    - d/p/0004-chrony-configuration-can-now-be-fragmented.patch
    - d/p/0005-global-drop-plugin-version.patch
    - d/p/0006-networking-check-presence-of-devlink.patch
    - d/p/0007-sosnode-avoid-checksum-cleanup-if-no-archive.patch

  * d/control:
   - Add 'python3-coverage' as part of the build depends.

  * d/rules:
   - Fix misplaced and duplicated sos.conf file in /usr/config.

 -- Eric Desrochers <email address hidden> Wed, 20 Oct 2021 12:38:22 -0400

Changed in sosreport (Ubuntu Impish):
status: Fix Committed → Fix Released
Eric Desrochers (slashd)
description: updated
Revision history for this message
Edward Hope-Morley (hopem) wrote :

I've verified version 4.2-1ubuntu0.20.04.1 from focal-proposed and it looks good. I generated a sos with 4.1-1ubuntu0.20.04.3 and the newer version and diffed between them and the contents look sane.

tags: added: verification-done verification-done-focal
removed: verification-needed verification-needed-focal
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package sosreport - 4.2-1ubuntu0.20.04.1

---------------
sosreport (4.2-1ubuntu0.20.04.1) focal; urgency=medium

  * New 4.2 upstream release. (LP: #1941745)
    - This release contains numerous improvements
      and bug fixes to several components within sos,
      including an overhaul to the project's test suite
      and infrastructure.

  * For more details, full release note is available here:
    - https://github.com/sosreport/sos/releases/tag/4.2

  * Remaining patches:
   - d/p/0001-debian-change-tmp-dir-location.patch

  * New patches:
    - d/p/0002-report-implement_estimate-only.patch
    - d/p/0003-ceph-add-support-for-containerized-ceph-setup.patch
    - d/p/0004-ceph-split-plugin-by-components.patch
    - d/p/0005-openvswitch-get-userspace-datapath-implementations.patch
    - d/p/0006-report-check-for-symlink-before-rmtree.patch

  * Former patches, now fixed:
    - d/p/0002-clean-prevent-parsing-ubuntu-user.patch
    - d/p/0003-ubuntu-policy-fix-upload.patch
    - d/p/0004-networking-check-presence-of-devlink.patch

  * d/control:
   - Add 'python3-coverage' as part of the build depends.

  * d/rules:
   - Fix misplaced and duplicated sos.conf file in /usr/config.

 -- Eric Desrochers <email address hidden> Mon, 01 Nov 2021 09:02:20 -0400

Changed in sosreport (Ubuntu Focal):
status: Fix Committed → Fix Released
Changed in sosreport (Debian):
status: New → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.