sysstat service is not enabled by default

[Robie Basak]

[Impact]

In 24.04, we shipped sysstat by default as part of a wider performance engineering effort. The idea is that relevant performance engineering tooling is already present and available when a user finds themselves needing to solve a performance engineering problem.

sysstat gathers performance data in the background so as to provide a baseline when a performance issue needs to be investigated. With packaging shipping sysstat as disabled by default, this cannot occur. To effectively investigate a performance issue with sysstat, users would need to enable sysstat first, wait to gather baseline data, then reproduce the original performance issue again. This is contrary to the idea that the tool is instantly available.

[Other Information]

There is an interaction with bug 2073284 here. In the case of a fresh installation of 24.04, due to that bug, sysstat services _are actually_ already enabled by default. However, upgrades from previous releases are affected. To resolve both bugs together, we need to change the packaging to make the default to enable the service. This will make behaviour consistent for users.

[Where problems could occur]

This section is written considering the changes to resolve this bug and bug 2073284 together, since they are quite intertwined.

TBC

[Robie Basak (racb)]

# Performance Impact Analysis

Summary: CPU, memory, storage and power management behaviour changes are negligible.

All analysis was done on Noble, but there's no reason to believe that the behaviour will be different on Oracular.

## CPU and direct storage

Robie did the following:

* `time -p sh -c 'for i in $(seq 1 1000); do /usr/lib/sysstat/sa1 1 1;done'`

Time taken was real 6.29 user 1.87 sys 4.40. /var/log/sysstat/sa21 grew from 848920 to 7420920\. This equates to 6.41 kiB per run. At one run every 10 minutes, we can therefore expect 923.04 kiB per day from the 10 minute runs, and normal CPU time consumption of 6.29ms once every 10 minutes.

Inside a lxd container where sysstat was left running in its default configuration for a week, the /var/log/sysstat/sar\* files are exactly 843923 bytes (824 kiB) each.

Daily consumption on a typical system is therefore 1747 kiB, or 52.9 MiB in total, as these files are rotated monthly by default (they are named by day of month number).

This machine had 36 block devices according to lsblk (including nbd0 through nbd15) and 16 block devices according to blkid (it didn’t list nbd).

## Memory

As root, valgrind \--tool=massif /usr/lib/sysstat/sa1 1 1 does nothing apparently because that file is a shell script that calls the real binary. Using \--trace-children=yes doesn’t seem to help either. What did produce results was valgrind \--tool=massif \--trace-children=yes \--pages-as-heap=yes /usr/lib/sysstat/sadc \-F \-L \-S DISK 1 1 /var/log/sysstat (pages-as-heap is needed because we’re most interested in the total consumption of the entire process). This suggests that usage peaks at 9.781 MiB, but Robie would like review to confirm that he’s interpreting the output of ms\_print correctly. Most of this appears to be in mmaps, presumably of shared libraries. Without \--pages-as-heap, we see a peak of 135.6 kiB of actual heap use.

Note that POSIX sh memory use is excluded. Since use of POSIX sh is ubiquitous, we can assume this is negligible, and its mmaps are likely to already be loaded and require few page cache misses.

/usr/lib/sysstat/sadc only dynamically links libc, libm and libsensors. libsensors.so.5.0.0 as shipped in Noble is 59 kiB. sadc itself is only 83200 bytes.

I think it’s fair to say that memory use is negligible, the majority are likely already be mmapped in other processes, while page cache misses may occur on the 10 minute schedule, they are unlikely to be significant in size.

## Feedback

* Andreas asked: will sysstat wake a sleeping disk?

I booted ubuntu-24.04-desktop-amd64.iso (SHA256 81fae9cc21e2b1e3a9a4526c7dad3131b668e346c580702235ad4d02645d9455) over USB on my spare laptop, a Lenovo Thinkpad T510i. This laptop has a spinning disk. I found I needed to boot it into safe graphics mode to get it to boot at all, but I don’t think that will make a difference to this test.

After booting, lsblk listed the internal disk as sda. hdparm -y /dev/sda spins down the disk as expected, and I heard it spin down. hdparm -C /dev/sda then lists the disk as being in standby mode.

I ran systemctl start sysstat-collect.service and systemctl start sysstat-summary.service. Listening while I did...

[Launchpad Janitor (janitor)]

This bug was fixed in the package sysstat - 12.7.5-2ubuntu1

---------------
sysstat (12.7.5-2ubuntu1) oracular; urgency=medium

  * d/sysstat.templates: enable services by default (LP: #2073285).
  * d/p/service-conditional-on-setting, d/sysstat.postinst: replace
    maintainer script management of systemd service enablement state
    with ExecCondition= systemd directives instead to avoid state
    conflict with systemd presets (LP: #2073284).
  * d/t/control, d/t/enablement: add dep8 tests to cover expected
    behaviour following the above changes.

 -- Robie Basak <email address hidden> Thu, 15 Aug 2024 11:32:23 +0000