[caraca|2024.1] the 22.0 version of python3-ceilometer is missing files thus breaking the notification service

[Alex Kavanagh]

[Impact]
Meter definition files are not included in the binary package resulting in a non-functional ceilometer after install.

[Test Case - Basic]
sudo apt install python3-ceilometer
find /usr/lib/python3 -name meters.yaml -> file is not found.

[Test Case - Advanced]
Deploy OpenStack Caracal using Charmed OpenStack Tester
Ceilometer agents will fail to start due to missing data files.

[Where problems could occur]
Limited - we're picking a patch that's been in Debian for sometime (2016) so its had good testing exposure already; as to why we need this in Noble whereas in Mantic this problem did not exist - probably something todo with installation behaviour changes between Python versions.

[Original Bug Report]
The issue I discovered is that (in an installed system) the:

/usr/lib/python3/dist-packages/ceilometer/pipeline/data/...

directory is missing. This is supplied by the python3-ceilometer .deb file.

For bobcat (mantic):

$ dpkg -c python3-ceilometer_21.0.0-0ubuntu1~cloud0_all.deb | grep data
drwxr-xr-x root/root 0 2023-10-05 19:34 ./usr/lib/python3/dist-packages/ceilometer/data/
drwxr-xr-x root/root 0 2023-10-05 19:34 ./usr/lib/python3/dist-packages/ceilometer/data/meters.d/
-rw-r--r-- root/root 13538 2023-10-04 12:09 ./usr/lib/python3/dist-packages/ceilometer/data/meters.d/meters.yaml
drwxr-xr-x root/root 0 2023-10-05 19:34 ./usr/lib/python3/dist-packages/ceilometer/pipeline/data/
-rw-r--r-- root/root 17573 2023-10-04 12:09 ./usr/lib/python3/dist-packages/ceilometer/pipeline/data/event_definitions.yaml
-rw-r--r-- root/root 176 2023-10-04 12:09 ./usr/lib/python3/dist-packages/ceilometer/pipeline/data/event_pipeline.yaml
-rw-r--r-- root/root 175 2023-10-04 12:09 ./usr/lib/python3/dist-packages/ceilometer/pipeline/data/pipeline.yaml
drwxr-xr-x root/root 0 2023-10-05 19:34 ./usr/lib/python3/dist-
...

For caracal (noble):

dpkg -c python3-ceilometer_22.0.0-0ubuntu1~cloud0_all.deb | grep data
-rw-r--r-- root/root 5113 2024-04-03 12:08 ./usr/lib/python3/dist-packages/ceilometer/tests/unit/compute/pollsters/test_location_metadata.py
-rw-r--r-- root/root 32485 2024-04-03 12:08 ./usr/lib/python3/dist-packages/ceilometer/tests/unit/ipmi/notifications/ipmi_test_data.py
-rw-r--r-- root/root 14151 2024-04-03 12:08 ./usr/lib/python3/dist-packages/ceilometer/tests/unit/ipmi/platform/ipmitool_test_data.py

i.e. the data directory is completely missing. It is available in the upstream ceilometer repo: https://opendev.org/openstack/ceilometer/src/branch/stable/2024.1/ceilometer/pipeline/data

I suspect something is amiss with the package src files.

[Launchpad Janitor (janitor)]

This bug was fixed in the package ceilometer - 2:22.0.0-0ubuntu2

---------------
ceilometer (2:22.0.0-0ubuntu2) oracular; urgency=medium

  * d/patches/install-missing-files.patch: Ensure that all data files
    in the Python module directories are included in the binary package
    (LP: #2071939).

 -- James Page <email address hidden> Mon, 08 Jul 2024 09:24:13 +0100

[James Page (james-page)]

Uploaded for SRU team review.

[Andreas Hasenack (ahasenack)]

Can we also do a diff of the package listing with this patch, to be sure we are not including unwanted files all of a sudden?

[Andreas Hasenack (ahasenack)]

Hello Alex, or anyone else affected,

Accepted ceilometer into noble-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ceilometer/2:22.0.0-0ubuntu1.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-noble to verification-done-noble. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-noble. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[James Page (james-page)]

Hello Alex, or anyone else affected,

Accepted ceilometer into caracal-proposed. The package will build now and be available in the Ubuntu Cloud Archive in a few hours, and then in the -proposed repository.

Please help us by testing this new package. To enable the -proposed repository:

  sudo add-apt-repository cloud-archive:caracal-proposed
  sudo apt-get update

Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-caracal-needed to verification-caracal-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-caracal-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Guillaume Boutry (gboutry)]

I built a ceilometer Rock from the UCA caracal-proposed and was able to confirm this fixes the issue.

Built the rock by applying the following diff to https://github.com/canonical/ubuntu-openstack-rocks/blob/6707266619e67abe4e1a48ab0fe91e81b6aa5b36/rocks/ceilometer-consolidated/rockcraft.yaml#L13

diff --git a/rocks/ceilometer-consolidated/rockcraft.yaml b/rocks/ceilometer-consolidated/rockcraft.yaml
index e9679b1..bcc49b5 100644
--- a/rocks/ceilometer-consolidated/rockcraft.yaml
+++ b/rocks/ceilometer-consolidated/rockcraft.yaml
@@ -13,6 +13,7 @@ platforms:
 package-repositories:
   - type: apt
     cloud: caracal
+ pocket: proposed
     priority: always

From inside the Rock
apt policy ceilometer-agent-notification
ceilometer-agent-notification:
  Installed: 2:22.0.0-0ubuntu1.1~cloud0
  Candidate: 2:22.0.0-0ubuntu1.1~cloud0
  Version table:
 *** 2:22.0.0-0ubuntu1.1~cloud0 100
        100 /var/lib/dpkg/status

I can confirm the notification agent is now starting with the following logs

2024-07-15 17:23:28
2024-07-15 15:23:27.065 22 WARNING ceilometer.publisher.gnocchi [-] Filtered project [service] not found in keystone, ignoring the filter_project option: keystoneauth1.exceptions.http.NotFound: 404 (HTTP 404)

2024-07-15 17:23:26
2024-07-15 15:23:25.925 22 WARNING ceilometer.pipeline.sample [-] metering data image.size for 983818c1-1c70-488d-9a7f-02ac94f43aa1 @ 2024-07-11T15:33:30.023650+00:00 has no volume (volume: None), the sample will be dropped

2024-07-15 17:23:26
2024-07-15 15:23:25.915 22 WARNING ceilometer.pipeline.sample [-] metering data image.size for cbcc29a7-adb7-4272-a4b6-ac577c06e9fe @ 2024-07-11T15:32:49.683748+00:00 has no volume (volume: None), the sample will be dropped

2024-07-15 17:23:26
2024-07-15 15:23:25.897 22 WARNING ceilometer.pipeline.sample [-] metering data image.size for 841ec872-05e2-4347-999d-26420e5a8194 @ 2024-07-11T15:32:10.519858+00:00 has no volume (volume: None), the sample will be dropped

2024-07-15 17:23:26
2024-07-15 15:23:25.887 22 WARNING oslo.messaging._drivers.pool [-] Connection pool limit exceeded: current size 30 surpasses max configured rpc_conn_pool_size 30

/* removed duplicated spam logs */

2024-07-15 17:23:26
2024-07-15 15:23:25.683 22 WARNING oslo.messaging._drivers.pool [-] Connection pool limit exceeded: current size 30 surpasses max configured rpc_conn_pool_size 30

2024-07-15 17:23:26
Sub('sub(/.*/, )')

2024-07-15 17:23:21
2024-07-15 15:23:15.781 22 INFO ceilometer.publisher.messaging [-] Publishing policy set to default

2024-07-15 17:23:20
2024-07-15 15:23:15.781 22 INFO ceilometer.publisher.messaging [-] Publishing policy set to default

Before the fix, the notification didn't start, failing on the reported exception.

[Guillaume Boutry (gboutry)]

I built a ceilometer Rock from Noble Proposed and was able to confirm this fixes the issue.

Built the rock by applying the following diff to https://github.com/canonical/ubuntu-openstack-rocks/blob/6707266619e67abe4e1a48ab0fe91e81b6aa5b36/rocks/ceilometer-consolidated/rockcraft.yaml#L13

diff --git a/rocks/ceilometer-consolidated/rockcraft.yaml b/rocks/ceilometer-consolidated/rockcraft.yaml
index e9679b1..6f539bf 100644
--- a/rocks/ceilometer-consolidated/rockcraft.yaml
+++ b/rocks/ceilometer-consolidated/rockcraft.yaml
@@ -5,14 +5,16 @@ description: |
   Ubuntu distribution of OpenStack ceilometer-consolidated
 version: '2024.1'

-# renovate: base: ubuntu:22.04@sha256:1b8d8ff4777f36f19bfe73ee4df61e3a0b789caeff29caa019539ec7c9a57f95
-base: ubuntu@22.04
+base: ubuntu@24.04
 platforms:
   amd64:

 package-repositories:
   - type: apt
- cloud: caracal
+ url: http://archive.ubuntu.com/ubuntu/
+ suites: [noble-proposed]
+ components: [main, universe, restricted, multiverse]
+ key-id: F6ECB3762474EDA9D21B7022871920D1991BC93C
     priority: always

 parts:

From inside the Rock
apt policy ceilometer-agent-notification
ceilometer-agent-notification:
  Installed: 2:22.0.0-0ubuntu1.1
  Candidate: 2:22.0.0-0ubuntu1.1
  Version table:
 *** 2:22.0.0-0ubuntu1.1 100
        100 /var/lib/dpkg/status

I can confirm the notification agent is now starting with the following logs:

2024-07-15 18:05:50
2024-07-15 16:05:45.757 22 INFO ceilometer.publisher.messaging [-] Publishing policy set to default

2024-07-15 18:05:49
2024-07-15 16:05:45.757 22 INFO ceilometer.publisher.messaging [-] Publishing policy set to default

Before the fix, the notification didn't start, failing on the reported exception.