collectd missing in 22.04/jammy

Bug #1971093 reported by Marian Rainer-Harbach
356
This bug affects 63 people
Affects Status Importance Assigned to Milestone
collectd (Ubuntu)
Invalid
Undecided
Unassigned
Jammy
Fix Committed
Undecided
Christian Ehrhardt 

Bug Description

[ Impact ]

 * This was removed [2] in Jammy-proposed on 2023-02-24 due to
   being an rdep for liboping in bug 1960612

 * That bug in liboping was resolved later and closed, but this
   being an reverse dependency, was not added back.

 * Due to that the collectd is in all releases prior and post
   this release, but missing in Jammy

 * We can take the latest version that is in pre-kinetic which
   is 5.12.0-10 and build it for Jammy to add it back to Jammy.

[ Test Plan ]

 * Simplified the test could be described as "apt install collectd"
   which as of now fails in Jammy, but would work after this being
   completed.
   - to make this a bit more non-superficial one should check if
     the service works on install and if it can interact.
     After install run:

systemc root@l:~# systemctl status collectd
● collectd.service - Statistics collection and monitoring daemon
     Loaded: loaded (/lib/systemd/system/collectd.service; enabled; preset: enabled)
     Active: active (running) since Mon 2023-01-16 11:30:35 UTC; 12s ago

   - then install the utils and make sure it is healthy

   $ apt install collectd-utils rrdtool

    The rrd tool is rather complex and auto-enabled.
    Have a look at what it collects for CPU0 ...

   $ rrdtool info /var/lib/collectd/rrd/*/cpu-1/cpu-interrupt.rrd

   You can look at other data points, but that should already list
   you some values that collectd provided.

 * We have many users here that could also give it a shot to
   test it if they want. So far many use builds from other Ubuntu
   releases which is a band-aid at best.

[ Where problems could occur ]

 * There is not much "regression-risk" as it just doesn't
   exists in jammy before accepting this.
   I've double checked reverse dependencies that might change
   behavior, but there are no explicit as they would have been
   removed as well back then.

[ Other Info ]

 * I have prepared a test build for jammy [1] in a PPA.
 * I think this is really fixing a regression for users of anything
   that depends on it (see all the pings on this bug) as well
   upgrade perspective of users that had it e.g. on Focal - it
   should be better to add it back.
   I see no conflict of this in regard to the SRU policy, so let
   us get this started. If rejecting int on SRU review for policy
   reasons please suggest an alternative way to pursue.
 * There are also a list of other packages with do not depend, but are
   meant to work with collectd. They will benefit from having it back in
   the archive.
   - golang-collectd-dev - Utilities for using collectd together with Golang
   - golang-github-kimor79-gollectd-dev - go parser for the collectd binary protocol
   - mtail - Extract monitoring data from logs for collection in a timeseries database
   - pgcluu - PostgreSQL performance monitoring and auditing tool
   - puppet-module-voxpupuli-collectd - Puppet module for collectd
   - kcollectd - simple collectd graphing front-end for KDE

[1]: https://launchpad.net/~paelzer/+archive/ubuntu/jammy-get-collectd-back/+packages
[2]: https://launchpad.net/ubuntu/+source/collectd/+publishinghistory

--- original report ---

collectd is missing in the 22.04/jammy repositories. collectd was available in the proposed repository at first, but has been removed on April 29. Why was it removed and when will collectd be available in 22.04?

Related branches

Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in collectd (Ubuntu):
status: New → Confirmed
Revision history for this message
Marian Rainer-Harbach (marianrh) wrote (last edit ):

As a workaround, the packages that were in the jammy proposed repository can be downloaded from https://launchpad.net/ubuntu/+source/collectd/5.12.0-9/+build/23189375. I only needed collectd_5.12.0-9_amd64.deb and collectd-core_5.12.0-9_amd64.deb.
Installing them via
apt install ./collectd_5.12.0-9_amd64.deb ./collectd-core_5.12.0-9_amd64.deb
results in a working collectd on 22.04.

Revision history for this message
Girish (Cloudron) (gramakri) wrote :

Can confirm that suggestion in #2 works . Still would be good to know why this is not part of universe. It was there for a brief period of time.

Revision history for this message
Max Rower (maxrower) wrote :

I'm not sure, the packages from the proposed repository really do work.
Installed them in an LXC container running 22.04 on a 20.04 host. Statistics are not stored locally, but sent to the 20.04 host, where rrdcached runs.
Currently only the uptime plugin (and exec as well for zigbee2mqtt) is active, and it sends total garbage to the 20.04 host.
The uptime plugin running in other LXC containers using 20.04 instead of 22.04 is fine.

Revision history for this message
Philipp Wendler (philw85) wrote :

I have tried out the packages from Jammy proposed, and they work fine here (Ubuntu 22.04, many collectd plugins active, forwarding to a Graphite server). Debian sid also has collectd 5.12.0-9 with the exact same changelog as the Ubuntu package (https://metadata.ftp-master.debian.org/changelogs//main/c/collectd/collectd_5.12.0-9_changelog) with no recent bugs in their tracker. So there does not seem to be a reason why Ubuntu should not ship that package.

Anway, if a package is shipped and later removed from the package repository for whatever reason, I would expect to be able to find the reason for this somewhere, e.g., here in the bug tracker. But I could not find any mention of this.

Revision history for this message
Steffy (fe80) wrote :

Hello,

Probably related with #1960612

Revision history for this message
Mike Battersby (mib-8) wrote :

Since liboping in 1960612 is now "Fix released" can we get collectd back?

Revision history for this message
Imri Paloja (blade19899) wrote :

CollectD is critical to my monitoring stack.

Is there anyway to speed up the recovery process of CollectD to the Ubuntu Repos?!

Revision history for this message
Matthias Runge (mrunge) wrote :

collectd was never written with upper-case D. Where does that come from?

liboping is only used for the ping plugin, it should not block the inclusion of collectd anyways. The rest of the roughly 170 plugins would be still available and useful.

Revision history for this message
Roman Ovchinnikov (coolcold) wrote :

+1 person who is affected by missing package and nasty surprise I'd say

Revision history for this message
Malte Schmidt (maltris) wrote :

For the point release I think this should have been fixed prior.

Revision history for this message
Martin Letáček (letynsoft) wrote :

As i can see libopng in my aptitude, it would be nice to re-add the collectd packages back to the repository too... (and definitely before the .1 release)

Revision history for this message
Ron Bühler (rbuehler) wrote :

I also would like to see collectd to be readded.

Revision history for this message
Constantin (cwildfoerster) wrote :

+1 why was collectd even removed? it is still unter active development and well maintained.

Revision history for this message
aurynn (aurynn) wrote :

+1 are there any updates on this package being re-included in Ubuntu? I'd like to be able to deploy 22.04, but this is a blocker for me.

Revision history for this message
Sandeep Shandilya (skshandilya69) wrote :

+1 on this. I don't know why this was removed, collectd is in active development and very widely used.

Revision history for this message
brad miller (madbriller) wrote (last edit ):

+1

Revision history for this message
Tommy (tommysitehost) wrote :

+1 collectd should be added back to Jammy.

Revision history for this message
Jack Peterson (jack-peterson) wrote :

+1 The absence of this package breaks my simple upgrade path to 22.04.

Revision history for this message
Jack Peterson (jack-peterson) wrote :

### fetch the deb files for your respective build. In my case I'm installing collectd for the aws cloudwatch metrics agent. Below is what I needed to do to get my puppet manifest to not error out on the dependency for collectd in ubuntu 22.04.

## 1. Get the .deb's https://launchpad.net/ubuntu/+source/collectd
## 1.a. wget https://launchpad.net/ubuntu/+archive/primary/+files/collectd-core_5.12.0-11_amd64.deb
### 1.b. https://launchpad.net/ubuntu/+archive/primary/+files/collectd_5.12.0-11_amd64.deb

```ruby
if $facts['os']['name'] == 'Ubuntu' and versioncmp($facts['os']['release']['full'], '22.04') >= 0 {
    package { "librrd8":
      ensure => present
    }

    file { "/tmp/collectd-core.deb":
      source => "puppet:///modules/pb_profile/home/ubuntu/collectd-core_5.12.0-11_amd64.deb",
      ensure => present
    }
    package { "collectd-core":
      source => "/tmp/collectd-core.deb",
      require => [Package["librrd8"], File["/tmp/collectd-core.deb"]],
      provider => dpkg,
      ensure => latest,
    }

    file { "/tmp/collectd.deb":
      source => "puppet:///modules/pb_profile/home/ubuntu/collectd_5.12.0-11_amd64.deb",
      ensure => present
    }
    package { "collectd":
      source => "/tmp/collectd.deb",
      require => [Package["collectd-core"], File["/tmp/collectd.deb"]],
      provider => dpkg,
      ensure => latest,
    }
  }
  else {
    package { "collectd":
      ensure => present
    }
  }
```

Revision history for this message
Ian (ian-buffington) wrote :

+1, I also ran into this while trying to install aws cloudwatch metrics agent on an AWS EC2 instance. Collectd is required.

The workaround I used:
wget https://launchpad.net/ubuntu/+archive/primary/+files/collectd-core_5.12.0-11_amd64.deb
sudo apt install ./collectd-core_5.12.0-11_amd64.deb

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

I was made aware of this case today by lathiat - thanks.
I agree that it must be a pain to miss this and yes, IMHO it should be added back.

I have prepared a test build for jammy [1] in a PPA.

Which works just fine AFAICS, feel free to try that and have a look.

It really is - as assumed above and seen in [2] - a removal because the dependency liboping was FTBFS in bug 1960612 however this was fixed and got published in jammy pretty late, but collectd wasn't restored.
Collectd exists in all previous and future versions so makes sense to restore it to jammy I think.

So from a users of anything that depends on it (see all the pings on this bug) as well upgrade perspective of users that had it e.g. on Focal - it should be better to add it back.
I see no conflict of this in regard to the SRU policy, so let us get this started.

[1]: https://launchpad.net/~paelzer/+archive/ubuntu/jammy-get-collectd-back/+packages
[2]: https://launchpad.net/ubuntu/+source/collectd/+publishinghistory

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

@lathiat could you give me a review on [1] please? And if you can a test of [2].
Once ok I'd upload that for review by the SRU team.

[1]: https://code.launchpad.net/~paelzer/ubuntu/+source/collectd/+git/collectd/+merge/435821
[2]: https://launchpad.net/~paelzer/+archive/ubuntu/jammy-get-collectd-back/+packages

description: updated
Changed in collectd (Ubuntu):
assignee: nobody → Christian Ehrhardt  (paelzer)
tags: added: server-todo
Revision history for this message
Trent Lloyd (lathiat) wrote :

+1 on this from me.

I've tested the PPA built package. I used the standard plugin set plus also enabled prometheus. I scraped the prometheus endpoint and also confirmed data being recorded into "rrdtool dump" for a few different RRDs.

I also installed collectd on focal, had it collect some data and then upgraded the entire instance to jammy and the PPA version and verified it continued to work as expected.

The review was already merged and the package was uploaded, currently in the jammy NEW queue.

Revision history for this message
Jason-Morries Adam (jasonmadam) wrote :

+1 from me.

I've tested it on 22.04 without any previous packages installed, everything worked as expected.

Revision history for this message
Henri Nougayrede (hnoug) wrote :

Hi
Tested on 22.04 and it works correctly.

Revision history for this message
Filyph (filyph) wrote :

+1 this package is widely used

Revision history for this message
Thomas Alexander Frederiksen (thomasaf) wrote :

+1 - it's keeping back a rolling upgrade of a large number of hosts here for the time being.

Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in collectd (Ubuntu Jammy):
status: New → Confirmed
Revision history for this message
Łukasz Zemczak (sil2100) wrote : Please test proposed package

Hello Marian, or anyone else affected,

Accepted collectd into jammy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/collectd/5.12.0-10ubuntu0.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-jammy to verification-done-jammy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-jammy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in collectd (Ubuntu Jammy):
status: Confirmed → Fix Committed
tags: added: verification-needed verification-needed-jammy
Revision history for this message
Jussi Mäkinen (jmakinen) wrote :

apt show collectd-core
Package: collectd-core
Version: 5.12.0-10ubuntu0.1

/var/lib/collectd/rrd/ has data. Seems working to me.

(Big thanks)

Revision history for this message
Christian Ehrhardt  (paelzer) wrote (last edit ):
Download full text (53.9 KiB)

Starting on a system without collectd

root@j:~# apt-cache policy collectd
collectd:
  Installed: (none)
  Candidate: 5.12.0-10ubuntu0.1
  Version table:
     5.12.0-10ubuntu0.1 500
        500 http://archive.ubuntu.com/ubuntu jammy-proposed/universe amd64 Packages

Installing from proposed, that pulls many dependencies as it can monitor so much.

root@j:~# apt install collectd collectd-utils rrdtool
...
worked fine, full output at the end as it is so long.

Collecting Data as expected ...
root@j:~# rrdtool info /var/lib/collectd/rrd/*/cpu-1/cpu-interrupt.rrd | head
filename = "/var/lib/collectd/rrd/j.lxd/cpu-1/cpu-interrupt.rrd"
rrd_version = "0003"
step = 10
last_update = 1674542785
header_size = 3496
ds[value].index = 0
ds[value].type = "DERIVE"
ds[value].minimal_heartbeat = 20
ds[value].min = 0.0000000000e+00
ds[value].max = NaN

Marking this as verified

# Full
root@j:~# apt install collectd collectd-utils rrdtool
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following additional packages will be installed:
  alsa-topology-conf alsa-ucm-conf ca-certificates-java collectd-core default-jre-headless fontconfig fontconfig-config fonts-dejavu-core intel-cmt-cat java-common libabsl20210324
  libasound2 libasound2-data libavahi-client3 libavahi-common-data libavahi-common3 libbson-1.0-0 libc-ares2 libcairo2 libcollectdclient1 libcups2 libdatrie1 libdbi1 libdeflate0 libesmtp6
  libfdt1 libfontconfig1 libgdk-pixbuf-2.0-0 libgdk-pixbuf2.0-bin libgdk-pixbuf2.0-common libgps28 libgraphite2-3 libgrpc++1 libgrpc10 libharfbuzz0b libhiredis0.14 libi2c0 libjbig0
  libjpeg-turbo8 libjpeg8 liblcms2-2 liblua5.3-0 libmemcached11 libmicrohttpd12 libmodbus5 libmongoc-1.0-0 libmongocrypt0 libmosquitto1 libmysqlclient21 libnotify4 libopenipmi0 liboping0
  libpango-1.0-0 libpangocairo-1.0-0 libpangoft2-1.0-0 libpcsclite1 libpixman-1-0 libpq5 libprotobuf-c1 libprotobuf23 libqpid-proton11 librabbitmq4 librdkafka1 libriemann-client0 librrd8
  librte-eal22 librte-ethdev22 librte-kvargs22 librte-mbuf22 librte-mempool22 librte-meter22 librte-net22 librte-ring22 librte-telemetry22 libsensors-config libsensors5 libsnappy1v5
  libsnmp-base libsnmp40 libthai-data libthai0 libtiff5 libvarnishapi2 libvirt0 libwebp7 libxcb-render0 libxcb-shm0 libxencall1 libxendevicemodel1 libxenevtchn1 libxenforeignmemory1
  libxengnttab1 libxenhypfs1 libxenmisc4.16 libxenstore4 libxentoolcore1 libxentoollog1 libxrender1 libyajl2 mysql-common openjdk-11-jre-headless
Suggested packages:
  collectd-dev librrds-perl liburi-perl libhtml-parser-perl libregexp-common-perl libconfig-general-perl httpd-cgi apache2 apcupsd bind9 ceph chrony default-mysql-server gpsd ipvsadm
  lm-sensors mbmon memcached nginx notification-daemon openvpn olsrd pdns-server postgresql redis-server slapd varnish zookeeper default-jre libasound2-plugins alsa-utils cups-common
  liblcms2-utils gnome-shell | notification-daemon pcscd snmp-mibs-downloader libnss-mdns fonts-dejavu-extra fonts-ipafont-gothic fonts-ipafont-mincho fonts-wqy-microhei | fonts-wqy-zenhei
  fonts-indic
The following NEW packages will be installed:
  alsa-topology-conf alsa-ucm-conf ca-certifica...

tags: added: verification-done verification-done-jammy
removed: verification-needed verification-needed-jammy
Revision history for this message
Jussi Mäkinen (jmakinen) wrote :

Wait, a lot of this kind of messages in syslog (for every plugin for every 10+ seconds)

Jan 24 09:22:08 jammyjellyfish collectd[99041]: rrdtool plugin: rrd_update_r (/var/lib/collectd/rrd/jammyjellyfish/cpu-0/cpu-user.rrd) failed: /var/lib/collectd/rrd/jammyjellyfish/cpu-0/cpu-user.rrd: illegal attempt to update using time 1674544919 when last update time is 1674544926 (minimum one second step)

The configuration is the same as in our focal fossa machines.

Yes, I have:
System clock synchronized: yes
              NTP service: active

If I comment CacheTimeout on collectd.conf, no error messages then…
<Plugin rrdtool>
        DataDir "/var/lib/collectd/rrd"
        # Keep incoming data for x seconds in cache before writing it to disk
# CacheTimeout 10

        # Add some random to above value to avoid multiple files being written same time
        RandomTimeout 5

        # When no data is received write existing data in cache to disk after x seconds
        CacheFlush 900

        # Write only x rrd files per second.
        # At the time writing we have around 122 rrd files per server.
        WritesPerSecond 50
</Plugin>

Revision history for this message
Rick Frey (gribnut) wrote (last edit ):

Upgraded collectd and collectd-core from -proposed repo (prior version was PPA 5.12.0-9 which had been upgraded from 20.04-LTS 5.9.2.g-1ubuntu5 version).

> apt show collectd
Package: collectd
Version: 5.12.0-10ubuntu0.1

I do not use rrdtool output plugin, but have tested writing to network output using number of input plugins with no error.
Jan 24 11:14:17 scorpion collectd[548381]: plugin_load: plugin "syslog" successfully loaded.
Jan 24 11:14:17 scorpion collectd[548381]: plugin_load: plugin "bind" successfully loaded.
Jan 24 11:14:17 scorpion collectd[548381]: plugin_load: plugin "cpu" successfully loaded.
Jan 24 11:14:17 scorpion collectd[548381]: plugin_load: plugin "df" successfully loaded.
Jan 24 11:14:17 scorpion collectd[548381]: plugin_load: plugin "disk" successfully loaded.
Jan 24 11:14:17 scorpion collectd[548381]: plugin_load: plugin "interface" successfully loaded.

Efforts in getting collectd packages back in jammy are much appreciated.

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Thanks Rick and Jussi for testing as well.

@Jussi:

For a start, at least the times are not totally off, the stamps convert to:
 - Tuesday, January 24, 2023 7:22:06 AM
 - Tuesday, January 24, 2023 7:21:59 AM

Also NTP enabled and syncing, and it would (hopefully) drift slowly if you would be off.
But the values indicate you jump back in time by ~7 seconds in the example.
Since you say this happens about every ~10 seconds you seem to go back in time rather quick.

The default conf has
  # CacheTimeout 120

which obviously does not show that behavior

But I've changed a test system to use what you have

<Plugin rrdtool>
 DataDir "/var/lib/collectd/rrd"
 CacheTimeout 10
 CacheFlush 900
 RandomTimeout 5

And that also does not expose the problem.

I found in other places [1] that one can also - after the config change - delete the files in the rrd path and then restart the service.

$ rm -rf /var/lib/collectd/rrd/*
$ systemctl restart collectd

That makes the issue show up for me as well.
But only on service start, not in a regular pattern.

I'm unsure - it is an issue, but it is no regression to having no collectd at all - so I'm unsure if we should make this hold up the SRU. I'd prefer to spawn a new bug to continue on this for a potential follow on fix. Especially as we have to wait and follow upstream issue which fell silent so far.
=> Please chime in on bug 2003839 for that aspect

But as I said IMHO this should not hold up the update here. Maybe we extend the time in -proposed to give us a chance to generate insights on this other case?

Revision history for this message
Jussi Mäkinen (jmakinen) wrote :

My opinion is that do not hold up because of me/our configuration. But after saying previously that collectd is working marvellously, I thought I should bring this up.

Any collectd is better that no collectd for those who use it.

(Thanks again, this has been unfortunate before this new development.)

Changed in collectd (Ubuntu):
status: Confirmed → Invalid
Changed in collectd (Ubuntu Jammy):
assignee: nobody → Christian Ehrhardt  (paelzer)
Changed in collectd (Ubuntu):
assignee: Christian Ehrhardt  (paelzer) → nobody
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Related questions

Remote bug watches

Bug watches keep track of this bug in other bug trackers.