[plugin][ceph] include time-sync-status for ceph mon

Bug #1910264 reported by Eric Desrochers
12
This bug affects 1 person
Affects Status Importance Assigned to Milestone
sosreport (Ubuntu)
Fix Released
Low
Eric Desrochers
Bionic
Fix Released
Medium
Ponnuvel Palaniyappan
Focal
Fix Released
Medium
Ponnuvel Palaniyappan
Groovy
Fix Released
Medium
Ponnuvel Palaniyappan
Hirsute
Fix Released
Low
Eric Desrochers

Bug Description

[Impact]

This will be useful to know if there is a clock skew.

The option has been first introduced in Luminous (v12) which is part of Ubuntu since Bionic/18.04LTS as follows:

# rmadison ceph
 ceph | 12.2.4-0ubuntu1 | bionic
 ceph | 12.2.13-0ubuntu0.18.04.4 | bionic-security
 ceph | 12.2.13-0ubuntu0.18.04.5 | bionic-updates
 ceph | 15.2.1-0ubuntu1 | focal
 ceph | 15.2.5-0ubuntu1 | groovy
 ceph | 15.2.5-0ubuntu1.1 | groovy-security
 ceph | 15.2.7-0ubuntu0.20.04.1 | focal-updates
 ceph | 15.2.7-0ubuntu0.20.10.1 | groovy-updates
 ceph | 15.2.7-0ubuntu1 | hirsute
 ceph | 15.2.7-0ubuntu3 | hirsute-proposed

Ceph mons might get into time sync problems if ntp/chrony
isn't installed or configured correctly. Since Luminous
release, upstream support 'time-sync-status' to detect this
more easily.

This information used to be part of ceph health structured output prior to Luminous but was removed in favor of the separate ceph time-sync-status command.

Upstream documentation about the option:
https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-mon/

[Test Case]

* Connect to a ceph mon node via ssh/juju ssh/...
* Run sosreport (this can be run in various ways but for the sake of this documenation let's use the most common):
 ** sosreport -a
 ** sosreport -o ceph

[Where problems could occur]

If something arise after this change. It will only affect the data collection of the ceph plugin. It won't affect other plugins nor core functionalities.

Worse case scenario would be that user will have to collect ceph debugging data manually (outside a usual sosreport run) in order to provide it to a third-party support vendor.

[Other information]

Issues:
https://github.com/sosreport/sos/issues/2356

PR:
https://github.com/sosreport/sos/pull/2357/commits/7859968b6e5e26c11d4bf4d8349646bd74723e52

[Original Description]

Ceph mons might get into time sync problems if ntp/chrony
isn't installed or configured correctly. Since Luminous
release, upstream support 'time-sync-status' to detect this
more easily.

This information used to be part of ceph health structured output prior to Luminous but was removed in favor of the separate ceph time-sync-status command.

Eric Desrochers (slashd)
tags: added: seg sts
Eric Desrochers (slashd)
description: updated
description: updated
description: updated
Changed in sosreport (Ubuntu Hirsute):
assignee: nobody → Eric Desrochers (slashd)
status: New → In Progress
importance: Undecided → Low
Changed in sosreport (Ubuntu Groovy):
assignee: nobody → Ponnuvel Palaniyappan (pponnuvel)
Changed in sosreport (Ubuntu Focal):
assignee: nobody → Ponnuvel Palaniyappan (pponnuvel)
Changed in sosreport (Ubuntu Bionic):
assignee: nobody → Ponnuvel Palaniyappan (pponnuvel)
Revision history for this message
Eric Desrochers (slashd) wrote :

Uploaded in Hirsute (current active devel release)

description: updated
description: updated
description: updated
Changed in sosreport (Ubuntu Focal):
importance: Undecided → Medium
Changed in sosreport (Ubuntu Groovy):
importance: Undecided → Medium
Changed in sosreport (Ubuntu Bionic):
importance: Undecided → Medium
Eric Desrochers (slashd)
description: updated
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package sosreport - 4.0-1ubuntu5

---------------
sosreport (4.0-1ubuntu5) hirsute; urgency=medium

  [Ponnuvel Palaniyappan]
  * d/p/0011-ceph-include-time-sync-status-for-ceph-mon.patch:
    - Ceph mons might get into time sync problems if ntp/chrony
      isn't installed or configured correctly. Since Luminous
      release, upstream support 'time-sync-status' to detect this
      more easily. (LP: #1910264)

 -- Eric Desrochers <email address hidden> Tue, 05 Jan 2021 11:18:38 -0500

Changed in sosreport (Ubuntu Hirsute):
status: In Progress → Fix Released
Revision history for this message
Ponnuvel Palaniyappan (pponnuvel) wrote :

bionic debdiff

Changed in sosreport (Ubuntu Bionic):
status: New → In Progress
Changed in sosreport (Ubuntu Focal):
status: New → In Progress
Changed in sosreport (Ubuntu Groovy):
status: New → Incomplete
status: Incomplete → In Progress
Revision history for this message
Ponnuvel Palaniyappan (pponnuvel) wrote :

focal debdiff

Revision history for this message
Ponnuvel Palaniyappan (pponnuvel) wrote :

groovy debdiff

Revision history for this message
Ponnuvel Palaniyappan (pponnuvel) wrote :

Hi Eric,

Attached debdiff's for bionic, focal, and groovy. Please let me know if they're OK.

Revision history for this message
Eric Desrochers (slashd) wrote :

There is currently an SRU for sosreport started by mfo.
I'll have to wait for it to end.

I'll take a note to revisit this bug sponsoring in 1 week.

- Eric

tags: added: sts-sponsors-slashd
Revision history for this message
Eric Desrochers (slashd) wrote :

[sts-sponsors][groovy][focal]

Sponsored in G/F.

Thanks for your contributions.

- Eric

Revision history for this message
Łukasz Zemczak (sil2100) wrote : Please test proposed package

Hello Eric, or anyone else affected,

Accepted sosreport into groovy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/sosreport/4.0-1ubuntu2.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-groovy to verification-done-groovy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-groovy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in sosreport (Ubuntu Groovy):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-groovy
Changed in sosreport (Ubuntu Focal):
status: In Progress → Fix Committed
tags: added: verification-needed-focal
Revision history for this message
Łukasz Zemczak (sil2100) wrote :

Hello Eric, or anyone else affected,

Accepted sosreport into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/sosreport/4.0-1~ubuntu0.20.04.4 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Revision history for this message
Ponnuvel Palaniyappan (pponnuvel) wrote :

Verified that time-sync-status is generated.

Attached file shows sosreport and ceph time-sync-status output from a test run.

tags: added: verification-done-focal
removed: verification-needed-focal
Revision history for this message
Ponnuvel Palaniyappan (pponnuvel) wrote :

Verified that time-sync-status is generated.

Attached file shows sosreport and ceph time-sync-status output from a test run on Groovy.

tags: added: verification-done-groovy
removed: verification-needed-groovy
Eric Desrochers (slashd)
Changed in sosreport (Ubuntu Groovy):
status: Fix Committed → In Progress
Revision history for this message
Łukasz Zemczak (sil2100) wrote :

Hello Eric, or anyone else affected,

Accepted sosreport into groovy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/sosreport/4.1-1ubuntu0.20.10.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-groovy to verification-done-groovy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-groovy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in sosreport (Ubuntu Groovy):
status: In Progress → Fix Committed
tags: added: verification-needed-groovy
removed: verification-done-groovy
Revision history for this message
Łukasz Zemczak (sil2100) wrote :

Hello Eric, or anyone else affected,

Accepted sosreport into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/sosreport/4.1-1ubuntu0.20.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

tags: added: verification-needed-focal
removed: verification-done-focal
Revision history for this message
Łukasz Zemczak (sil2100) wrote :

Hello Eric, or anyone else affected,

Accepted sosreport into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/sosreport/4.1-1ubuntu0.18.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in sosreport (Ubuntu Bionic):
status: In Progress → Fix Committed
tags: added: verification-needed-bionic
Revision history for this message
Ubuntu SRU Bot (ubuntu-sru-bot) wrote : Autopkgtest regression report (sosreport/4.1-1ubuntu0.20.04.1)

All autopkgtests for the newly accepted sosreport (4.1-1ubuntu0.20.04.1) for focal have finished running.
The following regressions have been reported in tests triggered by the package:

sosreport/4.1-1ubuntu0.20.04.1 (amd64)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/focal/update_excuses.html#sosreport

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

Revision history for this message
Ubuntu SRU Bot (ubuntu-sru-bot) wrote : Autopkgtest regression report (sosreport/4.1-1ubuntu0.18.04.1)

All autopkgtests for the newly accepted sosreport (4.1-1ubuntu0.18.04.1) for bionic have finished running.
The following regressions have been reported in tests triggered by the package:

sosreport/4.1-1ubuntu0.18.04.1 (s390x, arm64, i386, ppc64el, amd64)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/bionic/update_excuses.html#sosreport

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

Revision history for this message
Ponnuvel Palaniyappan (pponnuvel) wrote :

Verified on Bionic with 4.1-1ubuntu0.18.04.1.

Time sync status is collected as expected. Attached a reference.

tags: added: verification-done-bionic
removed: verification-needed-bionic
Revision history for this message
Ponnuvel Palaniyappan (pponnuvel) wrote :

Verified on Focal with 4.1-1ubuntu0.20.04.1.

Time sync status is collected as expected. Attached a reference.

tags: added: verification-done-focal
removed: verification-needed-focal
Revision history for this message
Ponnuvel Palaniyappan (pponnuvel) wrote :

Verified on Groovy with 4.1-1ubuntu0.20.10.1.

Time sync status is collected as expected. Attached a reference.

tags: added: verification-done verification-done-groovy
removed: verification-needed verification-needed-groovy
Revision history for this message
Ponnuvel Palaniyappan (pponnuvel) wrote :

SRU verification:

Verification done on Bionic, Focal, and Groovy. Please refer comments above and let me know if there are questions.

Thanks,
Pon

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package sosreport - 4.1-1ubuntu0.20.10.1

---------------
sosreport (4.1-1ubuntu0.20.10.1) groovy; urgency=medium

  * New 4.1 upstream minor release. (LP: #1917894)
    - https://github.com/sosreport/sos/releases/tag/4.1

  * d/tests/simple.sh:
    - Update the script from upstream
    - Modify the script to use /tmp as a target, instead
      of /var/tmp.

  * d/tests/control:
    - Adding isolation-machine as simple.sh wants to
      interact with the kernel.

  * Former patches, now fixed:
    - d/p/0002-fix-dict-order-py38-incompatibility.patch
    - d/p/0003-sosclean-fix-handling-of-filepath-with-archive-name.patch
    - d/p/0004-sosclean-fix-tarball-skipping-regex.patch
    - d/p/0005-ceph-collect-balancer-and-pg-autoscale-status.patch
    - d/p/0006-rabbitmq-add-info-on-maybe-stuck-processes.patch
    - d/p/0007-rabbitmq-add-10sec-timeout-to-call-to-maybestuck.patch
    - d/p/0008-networking-include-ip-neigh-and-rule-info.patch
    - d/p/0009-conntrack-add-conntrack-info.patch
    - d/p/0010-conntrack-gather-per-namespace-data.patch
    - d/p/0011-ceph-include-time-sync-status-for-ceph-mon.patch
    - d/p/0012-apt-move-unattended-upgrades-log-collection.patch
    - d/p/0013-bcache-add-a-new-plugin-for-bcache.patch
    - d/p/0014-k8s-add-cdk-master-auth-webhook-to-journal.patch
    - d/p/0015-k8s-fix-cdk-related-file-paths.patch
    - d/p/0016-systemd-prefer-resolvectl-over-systemd-resolve.patch
    - d/p/0017-ovn-extend-information.patch
    - d/p/0018-ua-prefer-new-ua-cmd-over-the-deprecated-one.patch
    - d/p/0019-ovn-fix-sbctl-cmd-execution.patch

  * Remaining patches:
   - d/p/0001-debian-change-tmp-dir-location.patch

  * New patches:
   - d/p/0002-clean-prevent-parsing-ubuntu-user.patch

  * Fixing the following LP bugs:
   - (LP: #1910264)
   - (LP: #1906302)
   - (LP: #1913284)
   - (LP: #1913583)
   - (LP: #1913581)
   - (LP: #1915072)

 -- Eric Desrochers <email address hidden> Tue, 23 Mar 2021 19:56:28 +0000

Changed in sosreport (Ubuntu Groovy):
status: Fix Committed → Fix Released
Revision history for this message
Łukasz Zemczak (sil2100) wrote : Update Released

The verification of the Stable Release Update for sosreport has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package sosreport - 4.1-1ubuntu0.20.04.1

---------------
sosreport (4.1-1ubuntu0.20.04.1) focal; urgency=medium

  * New 4.1 upstream minor release. (LP: #1917894)
    - https://github.com/sosreport/sos/releases/tag/4.1

  * d/tests/simple.sh:
    - Update the script from upstream
    - Modify the script to use /tmp as a target, instead
      of /var/tmp.

  * d/tests/control:
    - Adding isolation-machine as simple.sh wants to
      interact with the kernel.

  * Former patches, now fixed:
    - d/p/0002-fix-dict-order-py38-incompatibility.patch
    - d/p/0003-sosclean-fix-handling-of-filepath-with-archive-name.patch
    - d/p/0004-sosclean-fix-tarball-skipping-regex.patch
    - d/p/0005-ceph-collect-balancer-and-pg-autoscale-status.patch
    - d/p/0006-rabbitmq-add-info-on-maybe-stuck-processes.patch
    - d/p/0007-rabbitmq-add-10sec-timeout-to-call-to-maybestuck.patch
    - d/p/0008-networking-include-ip-neigh-and-rule-info.patch
    - d/p/0009-conntrack-add-conntrack-info.patch
    - d/p/0010-conntrack-gather-per-namespace-data.patch
    - d/p/0011-ceph-include-time-sync-status-for-ceph-mon.patch
    - d/p/0012-apt-move-unattended-upgrades-log-collection.patch
    - d/p/0013-bcache-add-a-new-plugin-for-bcache.patch
    - d/p/0014-k8s-add-cdk-master-auth-webhook-to-journal.patch
    - d/p/0015-k8s-fix-cdk-related-file-paths.patch
    - d/p/0016-systemd-prefer-resolvectl-over-systemd-resolve.patch
    - d/p/0017-ovn-extend-information.patch
    - d/p/0018-ua-prefer-new-ua-cmd-over-the-deprecated-one.patch
    - d/p/0019-ovn-fix-sbctl-cmd-execution.patch

  * Remaining patches:
   - d/p/0001-debian-change-tmp-dir-location.patch

  * New patches:
   - d/p/0002-clean-prevent-parsing-ubuntu-user.patch

  * Fixing the following LP bugs:
   - (LP: #1910264)
   - (LP: #1906302)
   - (LP: #1913284)
   - (LP: #1913583)
   - (LP: #1913581)
   - (LP: #1915072)

 -- Eric Desrochers <email address hidden> Thu, 25 Mar 2021 19:02:37 +0000

Changed in sosreport (Ubuntu Focal):
status: Fix Committed → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package sosreport - 4.1-1ubuntu0.18.04.1

---------------
sosreport (4.1-1ubuntu0.18.04.1) bionic; urgency=medium

  * New 4.1 upstream minor release. (LP: #1917894)
    - https://github.com/sosreport/sos/releases/tag/4.1

  * d/tests/*:
    - Remove obsolete scripts

  * d/tests/simple.sh:
    - Update the script from upstream to match sos-4.1
    - Modify the script to use /tmp as a target, instead
      of sos default /var/tmp.

  * d/tests/control:
    - Adding isolation-machine as simple.sh wants to
      interact with the kernel.

  * New config file location now under /etc/sos/sos.conf
    - The old config (/etc/sos.conf) contents will not be
      removed nor carried over after update. Users will
      have to modify the new file instead (as needed).

  * Former patches, now fixed:
    - d/p/0001-lshw-command.patch
    - d/p/0002-lds-substitute-oidc-conf.patch
    - d/p/0003-kvm-change-trigger-to-dev-kvm.patch
    - d/p/0004-maas-add-snap-support.patch
    - d/p/0005-conntrack-add-conntrack-info.patch
    - d/p/0006-conntrack-gather-per-namespace-data.patch
    - d/p/0007-networking-include-ns-ip-neigh-and-ip-rule-info.patch

  * New patches:
   - d/p/0001-debian-change-tmp-dir-location.patch
   - d/p/0002-clean-prevent-parsing-ubuntu-user.patch

  * Fixing the following LP bugs:
   - (LP: #1910264)
   - (LP: #1906302)
   - (LP: #1913284)
   - (LP: #1913583)
   - (LP: #1913581)
   - (LP: #1915072)

 -- Eric Desrochers <email address hidden> Fri, 26 Mar 2021 13:47:44 +0000

Changed in sosreport (Ubuntu Bionic):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.