[ffe][sru]: Update sosreport [last] plugin

Bug #1842489 reported by Eric Desrochers on 2019-09-03
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
sosreport (Ubuntu)
Medium
Eric Desrochers
Xenial
Medium
Eric Desrochers
Bionic
Medium
Eric Desrochers
Disco
Medium
Eric Desrochers

Bug Description

[IMPACT]

Canonical support is about to offer SOSCleaner project for UA customer who wants to obfuscate sensible data from an existing sosreport tarball such as user, hostname, network information obfuscation.

For user obfuscation, currently SOSCleaner relies on what sosreport offers which is "lastlog", reporting all user regardless of their UID.

Unfortunately, this generate a lot of false positives (and require a significant ignored_users list) inside SOSCleaner especially for user in the UID range 0-999 (e.g.'sys', 'bin', 'syslog', and much more)

I have submitted a PR in sosreport upstream in order to split lastlog output by UID ranges, in order to help SOSCleaner to obfuscate what we really want to obfuscate and ignore the rest.

https://github.com/sosreport/sos/issues/1743
https://github.com/sosreport/sos/pull/1770

Once approved upstream, I'd like to push that change in E/D/B/X in order to unblock the development of the tool "SOSCleaner" which unfortunately won't land in the archive for now, will be offered in a Private PPA. Note that the plan is for the package to land in the Ubuntu archive in the near future (few months still) and this change will highly impact the delay if this fix doesn't land in sosreport (Ubuntu archive).

[TEST CASE]

* Install sosreport
 ** Run sosreport:
 - sosreport -a
* Install soscleaner
 ** Run soscleaner:
 - soscleaner /tmp/sosreport-<TARBALL>
* Make sure only user in the 1000-60000 UID range are obfuscated.

[REGRESSION POTENTIAL]
- None, sosreport will still collect the entire lastlog (current behavior), we are not removing that behaviour, we simply add an alternative which is not costly for typical systems.

[OTHER INFO]

* Upstream fix not merged yet, but +1 by maintainers:
https://github.com/sosreport/sos/issues/1743
https://github.com/sosreport/sos/pull/1770

* Build logs:
https://launchpad.net/~slashd/+archive/ubuntu/lp1842489/+build/17517716
https://launchpad.net/~slashd/+archive/ubuntu/lp1842489/+build/17517717

Eric Desrochers (slashd) on 2019-09-03
summary: - Update sosrepor [last] plugin
+ Update sosreport [last] plugin
tags: added: sts
Changed in sosreport (Ubuntu):
status: New → In Progress
assignee: nobody → Eric Desrochers (slashd)
description: updated
description: updated
Eric Desrochers (slashd) on 2019-09-03
description: updated
Eric Desrochers (slashd) on 2019-09-03
description: updated
Eric Desrochers (slashd) on 2019-09-03
Changed in sosreport (Ubuntu):
importance: Undecided → Low
importance: Low → Medium
Łukasz Zemczak (sil2100) wrote :

Ok, this seems fine. Be sure to test-build the changes before uploading to eoan to protect against it FTBFS in the archive. +1 on the FFe from eoan's POV.

Changed in sosreport (Ubuntu):
status: In Progress → Triaged
Eric Desrochers (slashd) on 2019-09-04
description: updated
Eric Desrochers (slashd) wrote :
Download full text (3.4 KiB)

[EOAN VERIFICATION PRE-UPLOAD]

$ ls -altr sosreport-eoansponsor-20190904170850/sos_commands/last/
total 40
-rw-r--r-- 1 root root 113 Sep 4 17:08 lastlog_-u_65537-4294967295
-rw-r--r-- 1 root root 113 Sep 4 17:08 lastlog_-u_60001-65536
-rw-r--r-- 1 root root 113 Sep 4 17:08 lastlog_-u_1000-60000
-rw-r--r-- 1 root root 1877 Sep 4 17:08 lastlog_-u_0-999
-rw-r--r-- 1 root root 2066 Sep 4 17:08 lastlog
-rw-r--r-- 1 root root 38 Sep 4 17:08 last_shutdown
-rw-r--r-- 1 root root 183 Sep 4 17:08 last_reboot
-rw-r--r-- 1 root root 183 Sep 4 17:08 last
drwx------ 2 root root 4096 Sep 4 17:08 .
drwx------ 47 root root 4096 Sep 4 17:08 ..

$ more sosreport-eoansponsor-20190904170850/sos_commands/last/lastlog_-u_*
sosreport-eoansponsor-20190904170850/sos_commands/last/lastlog_-u_0-999
::::::::::::::
Username Port From Latest
root **Never logged in**
daemon **Never logged in**
bin **Never logged in**
sys **Never logged in**
sync **Never logged in**
games **Never logged in**
man **Never logged in**
lp **Never logged in**
mail **Never logged in**
news **Never logged in**
uucp **Never logged in**
proxy **Never logged in**
www-data **Never logged in**
backup **Never logged in**
list **Never logged in**
irc **Never logged in**
gnats **Never logged in**
systemd-timesync **Never logged in**
systemd-network **Never logged in**
systemd-resolve **Never logged in**
messagebus **Never logged in**
syslog **Never logged in**
_apt **Never logged in**
uuidd **Never logged in**
sshd **Never logged in**
landscape **Never logged in**
pollinate **Never logged in**
systemd-coredump **Never logged in**
lxd **Never logged in**
::::::::::::::
sosreport-eoansponsor-20190904170850/sos_commands/last/lastlog_-u_1000-60000
::::::::::::::
Username Port From Latest
ubuntu **Never logged in**
::::::::::::::
sosreport-eoansponsor-20190904170850/sos_commands/last/lastlog_-u_60001-65536
::::::::::::::
Username Port From Latest
nobody **Never logged in**
::::::::::::::
sosreport-eoansponsor-20190904170850/sos_c...

Read more...

description: updated
Eric Desrochers (slashd) on 2019-09-04
Changed in sosreport (Ubuntu):
status: Triaged → In Progress
summary: - Update sosreport [last] plugin
+ Ffe: Update sosreport [last] plugin

This bug was fixed in the package sosreport - 3.6-1ubuntu3

---------------
sosreport (3.6-1ubuntu3) eoan; urgency=medium

  * d/p/split-lastlog-per-uid-ranges.patch: (LP: #1842489)
    - Split lastlog per UID ranges.

 -- Eric Desrochers <email address hidden> Wed, 04 Sep 2019 16:47:09 +0000

Changed in sosreport (Ubuntu):
status: In Progress → Fix Released
Eric Desrochers (slashd) on 2019-09-04
Changed in sosreport (Ubuntu Xenial):
assignee: nobody → Eric Desrochers (slashd)
Changed in sosreport (Ubuntu Bionic):
assignee: nobody → Eric Desrochers (slashd)
Changed in sosreport (Ubuntu Disco):
assignee: nobody → Eric Desrochers (slashd)
status: New → In Progress
Changed in sosreport (Ubuntu Bionic):
status: New → In Progress
Changed in sosreport (Ubuntu Xenial):
status: New → In Progress
importance: Undecided → Medium
Changed in sosreport (Ubuntu Bionic):
importance: Undecided → Medium
Changed in sosreport (Ubuntu Disco):
importance: Undecided → Medium
summary: - Ffe: Update sosreport [last] plugin
+ [ffe][sru]: Update sosreport [last] plugin
Łukasz Zemczak (sil2100) wrote :

Even though this is more of a new feature than an SRU-typical bugfix, I do see merits in getting this backported to the stable series.

Changed in sosreport (Ubuntu Disco):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-disco

Hello Eric, or anyone else affected,

Accepted sosreport into disco-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/sosreport/3.6-1ubuntu2.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-disco to verification-done-disco. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-disco. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in sosreport (Ubuntu Bionic):
status: In Progress → Fix Committed
tags: added: verification-needed-bionic
Łukasz Zemczak (sil2100) wrote :

Hello Eric, or anyone else affected,

Accepted sosreport into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/sosreport/3.6-1ubuntu0.18.04.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in sosreport (Ubuntu Xenial):
status: In Progress → Fix Committed
tags: added: verification-needed-xenial
Łukasz Zemczak (sil2100) wrote :

Hello Eric, or anyone else affected,

Accepted sosreport into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/sosreport/3.6-1ubuntu0.16.04.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Eric Desrochers (slashd) wrote :

[VERIFICATION DISCO]

proposed package version: 3.6-1ubuntu2.1

# sosreport -a
# tar Jxvf /tmp/sreport-<REDACTED_HOST>-20190910212551.tar.xz

# ls -altr sosreport-<REDACTED_HOST>-20190910212551/sos_commands/last/
-rw-r--r-- 1 root root 0 Sep 10 21:25 lastlog_-u_65537-4294967295 # New behaviour to split by UID Range work as expected.
-rw-r--r-- 1 root root 113 Sep 10 21:25 lastlog_-u_60001-65536 # ...
-rw-r--r-- 1 root root 113 Sep 10 21:25 lastlog_-u_1000-60000 # ...
-rw-r--r-- 1 root root 1877 Sep 10 21:25 lastlog_-u_0-999 # End
-rw-r--r-- 1 root root 2003 Sep 10 21:25 lastlog
-rw-r--r-- 1 root root 38 Sep 10 21:25 last_shutdown
-rw-r--r-- 1 root root 549 Sep 10 21:25 last_reboot
-rw-r--r-- 1 root root 549 Sep 10 21:25 last # Old behaviour remain intact

I also looked at the content of each file, and everything look good.

tags: added: verification-done-disco
removed: verification-needed-disco
Eric Desrochers (slashd) wrote :

[VERIFICATION BIONIC]

-proposed package version: 3.6-1ubuntu0.18.04.3

# sosreport -a

# tar Jxvf /tmp/sosreport-<REDACTED_HOST>-20190911113800.tar.xz

# ls -altr sosreport-<REDACTED_HOST>-20190911113800/sos_commands/last/
-rw-r--r-- 1 root root 0 Sep 10 21:25 lastlog_-u_65537-4294967295 # New behaviour to split by UID Range work as expected.
-rw-r--r-- 1 root root 113 Sep 10 21:25 lastlog_-u_60001-65536 # ...
-rw-r--r-- 1 root root 113 Sep 10 21:25 lastlog_-u_1000-60000 # ...
-rw-r--r-- 1 root root 1877 Sep 10 21:25 lastlog_-u_0-999 # End
-rw-r--r-- 1 root root 2003 Sep 10 21:25 lastlog
-rw-r--r-- 1 root root 38 Sep 10 21:25 last_shutdown
-rw-r--r-- 1 root root 549 Sep 10 21:25 last_reboot
-rw-r--r-- 1 root root 549 Sep 10 21:25 last # Old behaviour remain intact

I also looked at the content of each file, and everything look good.

tags: added: verification-done-bionic
removed: verification-needed-bionic
Eric Desrochers (slashd) wrote :

[VERIFICATION XENIAL]

-proposed package version: 3.6-1ubuntu0.16.04.3

# sosreport -a

# tar Jxvf /tmp/sosreport-<REDACTED_HOST>-20190911114236.tar.xz

# ls -altr sosreport-<REDACTED_HOST>-20190911114236/sos_commands/last/
-rw-r--r-- 1 root root 0 Sep 10 21:25 lastlog_-u_65537-4294967295 # New behaviour to split by UID Range work as expected.
-rw-r--r-- 1 root root 113 Sep 10 21:25 lastlog_-u_60001-65536 # ...
-rw-r--r-- 1 root root 113 Sep 10 21:25 lastlog_-u_1000-60000 # ...
-rw-r--r-- 1 root root 1877 Sep 10 21:25 lastlog_-u_0-999 # End
-rw-r--r-- 1 root root 2003 Sep 10 21:25 lastlog
-rw-r--r-- 1 root root 38 Sep 10 21:25 last_shutdown
-rw-r--r-- 1 root root 549 Sep 10 21:25 last_reboot
-rw-r--r-- 1 root root 549 Sep 10 21:25 last # Old behaviour remain intact

I also looked at the content of each file, and everything look good.

tags: added: verification-done-xenial
removed: verification-needed-xenial
Łukasz Zemczak (sil2100) wrote :

And I see upstream actually approved the PR 5 hours ago, so we're good to go. Thanks for validating!

Launchpad Janitor (janitor) wrote :

This bug was fixed in the package sosreport - 3.6-1ubuntu2.1

---------------
sosreport (3.6-1ubuntu2.1) disco; urgency=medium

  * d/p/split-lastlog-per-uid-ranges.patch: (LP: #1842489)
    - Split lastlog per UID ranges.

 -- Eric Desrochers <email address hidden> Wed, 04 Sep 2019 20:38:52 +0000

Changed in sosreport (Ubuntu Disco):
status: Fix Committed → Fix Released

The verification of the Stable Release Update for sosreport has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Launchpad Janitor (janitor) wrote :

This bug was fixed in the package sosreport - 3.6-1ubuntu0.18.04.3

---------------
sosreport (3.6-1ubuntu0.18.04.3) bionic; urgency=medium

  * d/p/split-lastlog-per-uid-ranges.patch: (LP: #1842489)
    - Split lastlog per UID ranges.

 -- Eric Desrochers <email address hidden> Wed, 04 Sep 2019 20:51:59 +0000

Changed in sosreport (Ubuntu Bionic):
status: Fix Committed → Fix Released
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package sosreport - 3.6-1ubuntu0.16.04.3

---------------
sosreport (3.6-1ubuntu0.16.04.3) xenial; urgency=medium

  * d/p/split-lastlog-per-uid-ranges.patch: (LP: #1842489)
    - Split lastlog per UID ranges.

 -- Eric Desrochers <email address hidden> Wed, 04 Sep 2019 21:32:28 +0000

Changed in sosreport (Ubuntu Xenial):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers