[plugin][apt] Move unattended-upgrades log collection

[Eric Desrochers]

[Impact]

Having u-u logs collection inside the apt plugin make more sense (instead of logs plugin) as it is using apt functionalities and python modules such as apt, apt_inst, apt_pkg.

u-u is a mechanism to automatically install security upgrades on Debian/Ubuntu. It is enabled by default at installation.

'sos' philosophy is that each plugin should be taken care of its own logs.

[Test case]

* Deploy Ubuntu (A container will do)
* Run sosreport and make sure the systemd plugin is executed
  ** sosreport -o apt
* Look the content of the generated tarball under 'path_to_sosreport/var/log/unattended-upgrades/'

* And under 'path_to_sosreport/sos_logs/sos.log' for the execution log of the 'apt' plugin:

sos.log:2021-02-06 11:23:55,257 INFO: [plugin:apt] collecting path '/var/log/unattended-upgrades/.....

[Where problem could occur]

If a problem have to occur, it will only be impacting the apt plugin itself, not the core functionalities of 'sos' nor its other plugins.

A third party vendor might have to break the habit to assume that the 'logs' plugin will take care of the 'u-u' logs, and that 'apt' is the way to move forward now.

[Other information]
https://github.com/sosreport/sos/commit/9f057fb43a949cabb496d06506a59e411ef17812

[Launchpad Janitor (janitor)]

This bug was fixed in the package sosreport - 4.0-1ubuntu6

---------------
sosreport (4.0-1ubuntu6) hirsute; urgency=medium

  [Eric Desrochers]
  * d/p/0012-apt-move-unattended-upgrades-log-collection.patch
  (LP: #1906302)

  [Ponnuvel Palaniyappan]
  * d/p/0013-bcache-add-a-new-plugin-for-bcache.patch
  (LP: #1913284)

  [Felipe Reyes]
  * d/p/0014-k8s-add-cdk-master-auth-webhook-to-journal.patch
  * d/p/0015-k8s-fix-cdk-related-file-paths.patch
  (LP: #1913583)

  [Michael Biebl]
  * d/p/0016-systemd-prefer-resolvectl-over-systemd-resolve.patch
    (LP: #1913581)

 -- Eric Desrochers <email address hidden> Fri, 29 Jan 2021 12:20:50 -0500

[Łukasz Zemczak (sil2100)]

Hello Eric, or anyone else affected,

Accepted sosreport into groovy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/sosreport/4.0-1ubuntu2.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-groovy to verification-done-groovy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-groovy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Łukasz Zemczak (sil2100)]

Hello Eric, or anyone else affected,

Accepted sosreport into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/sosreport/4.0-1~ubuntu0.20.04.4 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Eric Desrochers (slashd)]

I ran 'sos report -o apt' on each release and look for the logs and if the content was part of the sosreport tarball. Running it with this particular CLI prevent limit the run to 'apt' and prevent other plugin to be exercise. I also ran it with 'sos report -a' where all plugin that meet conditions runs. In either case, all went well.

I confirm the /var/log/unattended-upgrades logs are no longer collected by the 'logs' plugin anymore but by the 'apt' plugin now as it should.

[VERIFICATION HIRSUTE]

# sos.log
2021-02-18 14:09:28,786 INFO: [plugin:apt] collecting path '/var/log/unattended-upgrades/unattended-upgrades-shutdown.log'

[VERIFICATION GROOVY]

# sos.log
2021-02-18 14:14:22,544 INFO: [plugin:apt] collecting path '/var/log/unattended-upgrades/unattended-upgrades-shutdown.log'

ls <PATH_TO_SOSREPORT>/var/log/unattended-upgrades/
unattended-upgrades-shutdown.log

[VERIFICATION FOCAL]

# sos.log
2021-02-18 14:17:41,217 INFO: [plugin:apt] collecting path '/var/log/unattended-upgrades/unattended-upgrades-shutdown.log'

ls <PATH_TO_SOSREPORT>/var/log/unattended-upgrades/
unattended-upgrades-shutdown.log

[Łukasz Zemczak (sil2100)]

Hello Eric, or anyone else affected,

Accepted sosreport into groovy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/sosreport/4.1-1ubuntu0.20.10.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-groovy to verification-done-groovy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-groovy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Łukasz Zemczak (sil2100)]

Hello Eric, or anyone else affected,

Accepted sosreport into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/sosreport/4.1-1ubuntu0.20.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Łukasz Zemczak (sil2100)]

Hello Eric, or anyone else affected,

Accepted sosreport into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/sosreport/4.1-1ubuntu0.18.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted sosreport (4.1-1ubuntu0.20.04.1) for focal have finished running.
The following regressions have been reported in tests triggered by the package:

sosreport/4.1-1ubuntu0.20.04.1 (amd64)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/focal/update_excuses.html#sosreport

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted sosreport (4.1-1ubuntu0.18.04.1) for bionic have finished running.
The following regressions have been reported in tests triggered by the package:

sosreport/4.1-1ubuntu0.18.04.1 (s390x, arm64, i386, ppc64el, amd64)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/bionic/update_excuses.html#sosreport

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Łukasz Zemczak (sil2100)]

Hey Eric! The provided verification doesn't seem to fit the test case in the bug description. The bug (and test case) mention unattended-upgrades log collection, while your verification is related to ubuntu-advantage. Is this related? Could you clarify the testing done to verify this bug?

[Eric Desrochers (slashd)]

Probably bad cut/paste. Sorry.

I ran 'sos report -o apt' on each release and look for the logs and if the content was part of the sosreport tarball. Running it with this particular CLI limit the run to 'apt' only and prevent other plugin to be exercise. I also ran it with 'sos report -a' where all plugin that meet conditions runs. In either case, all went well.

I confirm the /var/log/unattended-upgrades logs are no longer collected by the 'logs' plugin anymore but by the 'apt' plugin now as it should.

[VERIFICATION FOCAL][BIONIC][FOCAL]

# sos.log
INFO: [plugin:apt] collecting path '/var/log/unattended-upgrades/unattended-upgrades-shutdown.log'

ls <PATH_TO_SOSREPORT>/var/log/unattended-upgrades/
unattended-upgrades-shutdown.log

[Launchpad Janitor (janitor)]

This bug was fixed in the package sosreport - 4.1-1ubuntu0.20.10.1

---------------
sosreport (4.1-1ubuntu0.20.10.1) groovy; urgency=medium

  * New 4.1 upstream minor release. (LP: #1917894)
    - https://github.com/sosreport/sos/releases/tag/4.1

  * d/tests/simple.sh:
    - Update the script from upstream
    - Modify the script to use /tmp as a target, instead
      of /var/tmp.

  * d/tests/control:
    - Adding isolation-machine as simple.sh wants to
      interact with the kernel.

  * Former patches, now fixed:
    - d/p/0002-fix-dict-order-py38-incompatibility.patch
    - d/p/0003-sosclean-fix-handling-of-filepath-with-archive-name.patch
    - d/p/0004-sosclean-fix-tarball-skipping-regex.patch
    - d/p/0005-ceph-collect-balancer-and-pg-autoscale-status.patch
    - d/p/0006-rabbitmq-add-info-on-maybe-stuck-processes.patch
    - d/p/0007-rabbitmq-add-10sec-timeout-to-call-to-maybestuck.patch
    - d/p/0008-networking-include-ip-neigh-and-rule-info.patch
    - d/p/0009-conntrack-add-conntrack-info.patch
    - d/p/0010-conntrack-gather-per-namespace-data.patch
    - d/p/0011-ceph-include-time-sync-status-for-ceph-mon.patch
    - d/p/0012-apt-move-unattended-upgrades-log-collection.patch
    - d/p/0013-bcache-add-a-new-plugin-for-bcache.patch
    - d/p/0014-k8s-add-cdk-master-auth-webhook-to-journal.patch
    - d/p/0015-k8s-fix-cdk-related-file-paths.patch
    - d/p/0016-systemd-prefer-resolvectl-over-systemd-resolve.patch
    - d/p/0017-ovn-extend-information.patch
    - d/p/0018-ua-prefer-new-ua-cmd-over-the-deprecated-one.patch
    - d/p/0019-ovn-fix-sbctl-cmd-execution.patch

  * Remaining patches:
   - d/p/0001-debian-change-tmp-dir-location.patch

  * New patches:
   - d/p/0002-clean-prevent-parsing-ubuntu-user.patch

  * Fixing the following LP bugs:
   - (LP: #1910264)
   - (LP: #1906302)
   - (LP: #1913284)
   - (LP: #1913583)
   - (LP: #1913581)
   - (LP: #1915072)

 -- Eric Desrochers <email address hidden> Tue, 23 Mar 2021 19:56:28 +0000

[Łukasz Zemczak (sil2100)]

The verification of the Stable Release Update for sosreport has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package sosreport - 4.1-1ubuntu0.20.04.1

---------------
sosreport (4.1-1ubuntu0.20.04.1) focal; urgency=medium

  * New 4.1 upstream minor release. (LP: #1917894)
    - https://github.com/sosreport/sos/releases/tag/4.1

  * d/tests/simple.sh:
    - Update the script from upstream
    - Modify the script to use /tmp as a target, instead
      of /var/tmp.

  * d/tests/control:
    - Adding isolation-machine as simple.sh wants to
      interact with the kernel.

  * Former patches, now fixed:
    - d/p/0002-fix-dict-order-py38-incompatibility.patch
    - d/p/0003-sosclean-fix-handling-of-filepath-with-archive-name.patch
    - d/p/0004-sosclean-fix-tarball-skipping-regex.patch
    - d/p/0005-ceph-collect-balancer-and-pg-autoscale-status.patch
    - d/p/0006-rabbitmq-add-info-on-maybe-stuck-processes.patch
    - d/p/0007-rabbitmq-add-10sec-timeout-to-call-to-maybestuck.patch
    - d/p/0008-networking-include-ip-neigh-and-rule-info.patch
    - d/p/0009-conntrack-add-conntrack-info.patch
    - d/p/0010-conntrack-gather-per-namespace-data.patch
    - d/p/0011-ceph-include-time-sync-status-for-ceph-mon.patch
    - d/p/0012-apt-move-unattended-upgrades-log-collection.patch
    - d/p/0013-bcache-add-a-new-plugin-for-bcache.patch
    - d/p/0014-k8s-add-cdk-master-auth-webhook-to-journal.patch
    - d/p/0015-k8s-fix-cdk-related-file-paths.patch
    - d/p/0016-systemd-prefer-resolvectl-over-systemd-resolve.patch
    - d/p/0017-ovn-extend-information.patch
    - d/p/0018-ua-prefer-new-ua-cmd-over-the-deprecated-one.patch
    - d/p/0019-ovn-fix-sbctl-cmd-execution.patch

  * Remaining patches:
   - d/p/0001-debian-change-tmp-dir-location.patch

  * New patches:
   - d/p/0002-clean-prevent-parsing-ubuntu-user.patch

  * Fixing the following LP bugs:
   - (LP: #1910264)
   - (LP: #1906302)
   - (LP: #1913284)
   - (LP: #1913583)
   - (LP: #1913581)
   - (LP: #1915072)

 -- Eric Desrochers <email address hidden> Thu, 25 Mar 2021 19:02:37 +0000

[Launchpad Janitor (janitor)]

This bug was fixed in the package sosreport - 4.1-1ubuntu0.18.04.1

---------------
sosreport (4.1-1ubuntu0.18.04.1) bionic; urgency=medium

  * New 4.1 upstream minor release. (LP: #1917894)
    - https://github.com/sosreport/sos/releases/tag/4.1

  * d/tests/*:
    - Remove obsolete scripts

  * d/tests/simple.sh:
    - Update the script from upstream to match sos-4.1
    - Modify the script to use /tmp as a target, instead
      of sos default /var/tmp.

  * d/tests/control:
    - Adding isolation-machine as simple.sh wants to
      interact with the kernel.

  * New config file location now under /etc/sos/sos.conf
    - The old config (/etc/sos.conf) contents will not be
      removed nor carried over after update. Users will
      have to modify the new file instead (as needed).

  * Former patches, now fixed:
    - d/p/0001-lshw-command.patch
    - d/p/0002-lds-substitute-oidc-conf.patch
    - d/p/0003-kvm-change-trigger-to-dev-kvm.patch
    - d/p/0004-maas-add-snap-support.patch
    - d/p/0005-conntrack-add-conntrack-info.patch
    - d/p/0006-conntrack-gather-per-namespace-data.patch
    - d/p/0007-networking-include-ns-ip-neigh-and-ip-rule-info.patch

  * New patches:
   - d/p/0001-debian-change-tmp-dir-location.patch
   - d/p/0002-clean-prevent-parsing-ubuntu-user.patch

  * Fixing the following LP bugs:
   - (LP: #1910264)
   - (LP: #1906302)
   - (LP: #1913284)
   - (LP: #1913583)
   - (LP: #1913581)
   - (LP: #1915072)

 -- Eric Desrochers <email address hidden> Fri, 26 Mar 2021 13:47:44 +0000