| 2019-10-16 05:24:55 |
Wouter van Bommel |
bug |
|
|
added bug |
| 2019-10-16 05:28:01 |
Wouter van Bommel |
bug |
|
|
added subscriber Canonical IS BootStack |
| 2019-10-16 05:28:10 |
Wouter van Bommel |
bug |
|
|
added subscriber The Canonical Sysadmins |
| 2019-10-23 23:30:55 |
Wouter van Bommel |
bug |
|
|
added subscriber Canonical Field Medium |
| 2019-10-24 14:05:15 |
Ryan Beisner |
charm-octavia: importance |
Undecided |
Wishlist |
|
| 2019-10-29 13:48:46 |
Alex Kavanagh |
charm-octavia: status |
New |
Triaged |
|
| 2019-11-01 00:06:02 |
Dominique Poulain |
bug |
|
|
added subscriber Dominique Poulain |
| 2019-11-06 21:04:10 |
Jorge Niedbalski |
bug task added |
|
ceilometer |
|
| 2019-11-07 19:33:39 |
Jorge Niedbalski |
description |
Hi,
It looks like octavia is not reporting the same set of metrics as e.g. lbaasv2 is doing (https://docs.openstack.org/ceilometer/stein/admin/telemetry-measurements.html#openstack-compute).
We have a customer asking for more insight in the 'performance' of the loadbalancer so they are keen on getting as much information as possible.
What would be the best way forward to get this realized? |
[Environment]
Stein OpenStack
Ubuntu Bionic
[Description]
From my understanding the current Octavia API should be backwards compatible with LBaaS v2, therefore, the current LBaaS v2 pollster [1] should suffice to gather the following meters:
meters:
- network.services.lb.outgoing.bytes
- network.services.lb.incoming.bytes
- network.services.lb.pool
- network.services.lb.listener
- network.services.lb.member
- network.services.lb.health_monitor
- network.services.lb.loadbalancer
- network.services.lb.total.connections
- network.services.lb.active.connections
However, the following warning is noticed when restarted the ceilometer services.
2999:2019-11-07 15:05:41.665 22487 WARNING ceilometer.publisher.gnocchi [-] metric network.services.lb.loadbalancer is not handled by Gnocchi
3467:2019-11-07 15:06:21.830 3916 WARNING ceilometer.publisher.gnocchi [-] metric network.services.lb.loadbalancer is not handled by Gnocchi
In fact, by checking at the gnocchi metric list (https://pastebin.canonical.com/p/zyMTFv8vww/) the network.services.lb
metrics/resources haven't been created.
The reason is that for a gnocchi resource to exist, ceilometer has to create the resource type explicitly.
This action is commanded by the ceilometer gnocchi client [2] and it uses the file /usr/lib/python3/dist-packages/ceilometer/publisher/data/gnocchi_resources.yaml
for the resource definitions, which doesn't includes the lbaasv2 directives.
[Proposed solution]
Enable the resources on /usr/lib/python3/dist packages/ceilometer/publisher/data/gnocchi_resources.yaml
and modify the gnocchi client provider. I am able to gather the metrics for the octavia load balancers.
ubuntu@niedbalski-bastion:~/stsstack-bundles/openstack$ gnocchi metric list|grep network.services
| 3947b249-b1ed-489d-9f51-b7549b5b78ce | ceilometer-low | network.services.lb.loadbalancer | loadbalancer | f97838f1-215d-4fcb-bd78-61e1f9859507 |
| 5527c5b6-d17e-4218-868b-c8d5326575a6 | ceilometer-low | network.services.lb.active.connections | connection | f97838f1-215d-4fcb-bd78-61e1f9859507 |
| 57d63bcb-48e1-47fb-825c-1109ad7d966d | ceilometer-low | network.services.lb.loadbalancer | loadbalancer | 82d129f3-64a5-40d6-a72b-7ae704046176 |
| 5d93c751-0645-4b7b-a315-26b0e96dda3d | ceilometer-low | network.services.lb.outgoing.bytes | B | 82d129f3-64a5-40d6-a72b-7ae704046176 |
| 71ace776-9216-40a4-848c-42e9cc6f93a0 | ceilometer-low | network.services.lb.listener | listener | e7a4dd96-73a1-4aca-946d-8e87ba8f7a37 |
| 738b1dd0-0930-4bfa-b358-1a27fb63121e | ceilometer-low | network.services.lb.total.connections | connection | f97838f1-215d-4fcb-bd78-61e1f9859507 |
| 7b4726f6-9490-4d6c-a586-7e12b738b001 | ceilometer-low | network.services.lb.health_monitor | health_monitor | 3ebc3ac0-8e1c-47d9-8789-bb51e6d29eba |
| 8689e872-c6d0-450f-bb73-d19197c62418 | ceilometer-low | network.services.lb.incoming.bytes | B | 82d129f3-64a5-40d6-a72b-7ae704046176 |
| ae72c5f3-4d8b-400b-ba86-2aaf068c20fc | ceilometer-low | network.services.lb.incoming.bytes | B | f97838f1-215d-4fcb-bd78-61e1f9859507 |
| b59ef5ae-9d71-4884-bb7c-66e0d46f5ecb | ceilometer-low | network.services.lb.outgoing.bytes | B | f97838f1-215d-4fcb-bd78-61e1f9859507 |
| bdf259a1-7471-4c23-ab2b-cac11f522f45 | ceilometer-low | network.services.lb.active.connections | connection | 82d129f3-64a5-40d6-a72b-7ae704046176 |
| e5d2e433-9482-4e2a-9485-e73d7a13271a | ceilometer-low | network.services.lb.listener | listener | c87e0f88-26bf-496f-a882-71b1c67956b4 |
| eb634e4e-8e94-4ee4-8fac-4f4db12d56aa | ceilometer-low | network.services.lb.total.connections | connection | 82d129f3-64a5-40d6-a72b-7ae704046176 |
| ef04f494-884a-4097-ba58-74845f5fbf19 | ceilometer-low | network.services.lb.pool | pool | 6c4a5a4f-0455-42e6-8bf4-ed9c24e2c06c |
ubuntu@niedbalski-bastion:~/stsstack-bundles/openstack$ gnocchi measures show ef04f494-884a-4097-ba58-74845f5fbf19
+---------------------------+-------------+-------+
| timestamp | granularity | value |
+---------------------------+-------------+-------+
| 2019-11-07T18:30:00+00:00 | 300.0 | 1.0 |
| 2019-11-07T18:35:00+00:00 | 300.0 | 1.0 |
| 2019-11-07T18:40:00+00:00 | 300.0 | 1.0 |
+---------------------------+-------------+-------+ |
|
| 2019-11-07 21:38:40 |
OpenStack Infra |
ceilometer: status |
New |
In Progress |
|
| 2019-11-07 21:38:40 |
OpenStack Infra |
ceilometer: assignee |
|
Jorge Niedbalski (niedbalski) |
|
| 2019-11-11 14:38:12 |
OpenStack Infra |
ceilometer: status |
In Progress |
Fix Released |
|
| 2019-11-12 20:26:48 |
Corey Bryant |
bug task added |
|
ceilometer (Ubuntu) |
|
| 2019-11-12 20:26:57 |
Corey Bryant |
ceilometer (Ubuntu): status |
New |
Triaged |
|
| 2019-11-12 20:27:08 |
Corey Bryant |
ceilometer (Ubuntu): importance |
Undecided |
Medium |
|
| 2019-11-12 20:28:37 |
Corey Bryant |
nominated for series |
|
Ubuntu Focal |
|
| 2019-11-12 20:28:37 |
Corey Bryant |
bug task added |
|
ceilometer (Ubuntu Focal) |
|
| 2019-11-12 20:28:37 |
Corey Bryant |
nominated for series |
|
Ubuntu Bionic |
|
| 2019-11-12 20:28:37 |
Corey Bryant |
bug task added |
|
ceilometer (Ubuntu Bionic) |
|
| 2019-11-12 20:28:37 |
Corey Bryant |
nominated for series |
|
Ubuntu Eoan |
|
| 2019-11-12 20:28:37 |
Corey Bryant |
bug task added |
|
ceilometer (Ubuntu Eoan) |
|
| 2019-11-12 20:28:37 |
Corey Bryant |
nominated for series |
|
Ubuntu Disco |
|
| 2019-11-12 20:28:37 |
Corey Bryant |
bug task added |
|
ceilometer (Ubuntu Disco) |
|
| 2019-11-12 20:28:52 |
Corey Bryant |
bug task added |
|
cloud-archive |
|
| 2019-11-12 20:29:26 |
Corey Bryant |
nominated for series |
|
cloud-archive/queens |
|
| 2019-11-12 20:29:26 |
Corey Bryant |
bug task added |
|
cloud-archive/queens |
|
| 2019-11-12 20:29:26 |
Corey Bryant |
nominated for series |
|
cloud-archive/train |
|
| 2019-11-12 20:29:26 |
Corey Bryant |
bug task added |
|
cloud-archive/train |
|
| 2019-11-12 20:29:26 |
Corey Bryant |
nominated for series |
|
cloud-archive/stein |
|
| 2019-11-12 20:29:26 |
Corey Bryant |
bug task added |
|
cloud-archive/stein |
|
| 2019-11-12 20:29:26 |
Corey Bryant |
nominated for series |
|
cloud-archive/ussuri |
|
| 2019-11-12 20:29:26 |
Corey Bryant |
bug task added |
|
cloud-archive/ussuri |
|
| 2019-11-12 20:29:26 |
Corey Bryant |
nominated for series |
|
cloud-archive/rocky |
|
| 2019-11-12 20:29:26 |
Corey Bryant |
bug task added |
|
cloud-archive/rocky |
|
| 2019-11-12 20:29:41 |
Corey Bryant |
ceilometer (Ubuntu Eoan): importance |
Undecided |
Medium |
|
| 2019-11-12 20:29:41 |
Corey Bryant |
ceilometer (Ubuntu Eoan): status |
New |
Triaged |
|
| 2019-11-12 20:29:53 |
Corey Bryant |
ceilometer (Ubuntu Disco): importance |
Undecided |
Medium |
|
| 2019-11-12 20:29:53 |
Corey Bryant |
ceilometer (Ubuntu Disco): status |
New |
Triaged |
|
| 2019-11-12 20:30:19 |
Corey Bryant |
ceilometer (Ubuntu Bionic): importance |
Undecided |
Medium |
|
| 2019-11-12 20:30:19 |
Corey Bryant |
ceilometer (Ubuntu Bionic): status |
New |
Triaged |
|
| 2019-11-12 20:30:47 |
Corey Bryant |
cloud-archive/ussuri: importance |
Undecided |
Medium |
|
| 2019-11-12 20:30:47 |
Corey Bryant |
cloud-archive/ussuri: status |
New |
Triaged |
|
| 2019-11-12 20:30:59 |
Corey Bryant |
cloud-archive/train: importance |
Undecided |
Medium |
|
| 2019-11-12 20:30:59 |
Corey Bryant |
cloud-archive/train: status |
New |
Triaged |
|
| 2019-11-12 20:31:10 |
Corey Bryant |
cloud-archive/stein: importance |
Undecided |
Medium |
|
| 2019-11-12 20:31:10 |
Corey Bryant |
cloud-archive/stein: status |
New |
Triaged |
|
| 2019-11-12 20:31:22 |
Corey Bryant |
cloud-archive/rocky: importance |
Undecided |
Medium |
|
| 2019-11-12 20:31:22 |
Corey Bryant |
cloud-archive/rocky: status |
New |
Triaged |
|
| 2019-11-12 20:31:33 |
Corey Bryant |
cloud-archive/queens: importance |
Undecided |
Medium |
|
| 2019-11-12 20:31:33 |
Corey Bryant |
cloud-archive/queens: status |
New |
Triaged |
|
| 2019-11-21 17:31:09 |
Jorge Niedbalski |
attachment added |
|
lp1848286-stein.debdiff https://bugs.launchpad.net/charm-octavia/+bug/1848286/+attachment/5306922/+files/lp1848286-stein.debdiff |
|
| 2019-11-21 17:33:43 |
Jorge Niedbalski |
bug |
|
|
added subscriber Ubuntu Sponsors Team |
| 2019-11-27 02:40:53 |
OpenStack Infra |
cloud-archive/stein: status |
Triaged |
Fix Committed |
|
| 2019-11-27 09:35:16 |
OpenStack Infra |
cloud-archive/rocky: status |
Triaged |
Fix Committed |
|
| 2019-11-27 14:19:04 |
Jorge Niedbalski |
description |
[Environment]
Stein OpenStack
Ubuntu Bionic
[Description]
From my understanding the current Octavia API should be backwards compatible with LBaaS v2, therefore, the current LBaaS v2 pollster [1] should suffice to gather the following meters:
meters:
- network.services.lb.outgoing.bytes
- network.services.lb.incoming.bytes
- network.services.lb.pool
- network.services.lb.listener
- network.services.lb.member
- network.services.lb.health_monitor
- network.services.lb.loadbalancer
- network.services.lb.total.connections
- network.services.lb.active.connections
However, the following warning is noticed when restarted the ceilometer services.
2999:2019-11-07 15:05:41.665 22487 WARNING ceilometer.publisher.gnocchi [-] metric network.services.lb.loadbalancer is not handled by Gnocchi
3467:2019-11-07 15:06:21.830 3916 WARNING ceilometer.publisher.gnocchi [-] metric network.services.lb.loadbalancer is not handled by Gnocchi
In fact, by checking at the gnocchi metric list (https://pastebin.canonical.com/p/zyMTFv8vww/) the network.services.lb
metrics/resources haven't been created.
The reason is that for a gnocchi resource to exist, ceilometer has to create the resource type explicitly.
This action is commanded by the ceilometer gnocchi client [2] and it uses the file /usr/lib/python3/dist-packages/ceilometer/publisher/data/gnocchi_resources.yaml
for the resource definitions, which doesn't includes the lbaasv2 directives.
[Proposed solution]
Enable the resources on /usr/lib/python3/dist packages/ceilometer/publisher/data/gnocchi_resources.yaml
and modify the gnocchi client provider. I am able to gather the metrics for the octavia load balancers.
ubuntu@niedbalski-bastion:~/stsstack-bundles/openstack$ gnocchi metric list|grep network.services
| 3947b249-b1ed-489d-9f51-b7549b5b78ce | ceilometer-low | network.services.lb.loadbalancer | loadbalancer | f97838f1-215d-4fcb-bd78-61e1f9859507 |
| 5527c5b6-d17e-4218-868b-c8d5326575a6 | ceilometer-low | network.services.lb.active.connections | connection | f97838f1-215d-4fcb-bd78-61e1f9859507 |
| 57d63bcb-48e1-47fb-825c-1109ad7d966d | ceilometer-low | network.services.lb.loadbalancer | loadbalancer | 82d129f3-64a5-40d6-a72b-7ae704046176 |
| 5d93c751-0645-4b7b-a315-26b0e96dda3d | ceilometer-low | network.services.lb.outgoing.bytes | B | 82d129f3-64a5-40d6-a72b-7ae704046176 |
| 71ace776-9216-40a4-848c-42e9cc6f93a0 | ceilometer-low | network.services.lb.listener | listener | e7a4dd96-73a1-4aca-946d-8e87ba8f7a37 |
| 738b1dd0-0930-4bfa-b358-1a27fb63121e | ceilometer-low | network.services.lb.total.connections | connection | f97838f1-215d-4fcb-bd78-61e1f9859507 |
| 7b4726f6-9490-4d6c-a586-7e12b738b001 | ceilometer-low | network.services.lb.health_monitor | health_monitor | 3ebc3ac0-8e1c-47d9-8789-bb51e6d29eba |
| 8689e872-c6d0-450f-bb73-d19197c62418 | ceilometer-low | network.services.lb.incoming.bytes | B | 82d129f3-64a5-40d6-a72b-7ae704046176 |
| ae72c5f3-4d8b-400b-ba86-2aaf068c20fc | ceilometer-low | network.services.lb.incoming.bytes | B | f97838f1-215d-4fcb-bd78-61e1f9859507 |
| b59ef5ae-9d71-4884-bb7c-66e0d46f5ecb | ceilometer-low | network.services.lb.outgoing.bytes | B | f97838f1-215d-4fcb-bd78-61e1f9859507 |
| bdf259a1-7471-4c23-ab2b-cac11f522f45 | ceilometer-low | network.services.lb.active.connections | connection | 82d129f3-64a5-40d6-a72b-7ae704046176 |
| e5d2e433-9482-4e2a-9485-e73d7a13271a | ceilometer-low | network.services.lb.listener | listener | c87e0f88-26bf-496f-a882-71b1c67956b4 |
| eb634e4e-8e94-4ee4-8fac-4f4db12d56aa | ceilometer-low | network.services.lb.total.connections | connection | 82d129f3-64a5-40d6-a72b-7ae704046176 |
| ef04f494-884a-4097-ba58-74845f5fbf19 | ceilometer-low | network.services.lb.pool | pool | 6c4a5a4f-0455-42e6-8bf4-ed9c24e2c06c |
ubuntu@niedbalski-bastion:~/stsstack-bundles/openstack$ gnocchi measures show ef04f494-884a-4097-ba58-74845f5fbf19
+---------------------------+-------------+-------+
| timestamp | granularity | value |
+---------------------------+-------------+-------+
| 2019-11-07T18:30:00+00:00 | 300.0 | 1.0 |
| 2019-11-07T18:35:00+00:00 | 300.0 | 1.0 |
| 2019-11-07T18:40:00+00:00 | 300.0 | 1.0 |
+---------------------------+-------------+-------+ |
[Environment]
Stein OpenStack
Ubuntu Bionic
[Description]
From my understanding the current Octavia API should be backwards compatible with LBaaS v2, therefore, the current LBaaS v2 pollster [1] should suffice to gather the following meters:
meters:
- network.services.lb.outgoing.bytes
- network.services.lb.incoming.bytes
- network.services.lb.pool
- network.services.lb.listener
- network.services.lb.member
- network.services.lb.health_monitor
- network.services.lb.loadbalancer
- network.services.lb.total.connections
- network.services.lb.active.connections
However, the following warning is noticed when restarted the ceilometer services.
2999:2019-11-07 15:05:41.665 22487 WARNING ceilometer.publisher.gnocchi [-] metric network.services.lb.loadbalancer is not handled by Gnocchi
3467:2019-11-07 15:06:21.830 3916 WARNING ceilometer.publisher.gnocchi [-] metric network.services.lb.loadbalancer is not handled by Gnocchi
In fact, by checking at the gnocchi metric list (https://pastebin.canonical.com/p/zyMTFv8vww/) the network.services.lb
metrics/resources haven't been created.
The reason is that for a gnocchi resource to exist, ceilometer has to create the resource type explicitly.
This action is commanded by the ceilometer gnocchi client [2] and it uses the file /usr/lib/python3/dist-packages/ceilometer/publisher/data/gnocchi_resources.yaml
for the resource definitions, which doesn't includes the lbaasv2 directives.
[Test case]
1) Deploy a gnocchi backed ceilometer unit with octavia deployed.
2) check that lb meters aren't being gathered
$ gnocchi metric list|grep loadbalancer | wc -l
0
3) install the patched version
4) configure all pollsters
$ juju config ceilometer enable-all-pollsters=true
5) sudo systemctl restart ceilometer*
6) sudo ceilometer-upgrade
7) Check that lb metrics are being gathered
ubuntu@niedbalski-bastion:~/stsstack-bundles/openstack$ gnocchi metric list|grep loadbalancer | wc -l
72
ubuntu@niedbalski-bastion:~/stsstack-bundles/openstack$ gnocchi metric list|grep network.services
| 3947b249-b1ed-489d-9f51-b7549b5b78ce | ceilometer-low | network.services.lb.loadbalancer | loadbalancer | f97838f1-215d-4fcb-bd78-61e1f9859507 |
| 5527c5b6-d17e-4218-868b-c8d5326575a6 | ceilometer-low | network.services.lb.active.connections | connection | f97838f1-215d-4fcb-bd78-61e1f9859507 |
| 57d63bcb-48e1-47fb-825c-1109ad7d966d | ceilometer-low | network.services.lb.loadbalancer | loadbalancer | 82d129f3-64a5-40d6-a72b-7ae704046176 |
| 5d93c751-0645-4b7b-a315-26b0e96dda3d | ceilometer-low | network.services.lb.outgoing.bytes | B | 82d129f3-64a5-40d6-a72b-7ae704046176 |
| 71ace776-9216-40a4-848c-42e9cc6f93a0 | ceilometer-low | network.services.lb.listener | listener | e7a4dd96-73a1-4aca-946d-8e87ba8f7a37 |
| 738b1dd0-0930-4bfa-b358-1a27fb63121e | ceilometer-low | network.services.lb.total.connections | connection | f97838f1-215d-4fcb-bd78-61e1f9859507 |
| 7b4726f6-9490-4d6c-a586-7e12b738b001 | ceilometer-low | network.services.lb.health_monitor | health_monitor | 3ebc3ac0-8e1c-47d9-8789-bb51e6d29eba |
| 8689e872-c6d0-450f-bb73-d19197c62418 | ceilometer-low | network.services.lb.incoming.bytes | B | 82d129f3-64a5-40d6-a72b-7ae704046176 |
| ae72c5f3-4d8b-400b-ba86-2aaf068c20fc | ceilometer-low | network.services.lb.incoming.bytes | B | f97838f1-215d-4fcb-bd78-61e1f9859507 |
| b59ef5ae-9d71-4884-bb7c-66e0d46f5ecb | ceilometer-low | network.services.lb.outgoing.bytes | B | f97838f1-215d-4fcb-bd78-61e1f9859507 |
| bdf259a1-7471-4c23-ab2b-cac11f522f45 | ceilometer-low | network.services.lb.active.connections | connection | 82d129f3-64a5-40d6-a72b-7ae704046176 |
| e5d2e433-9482-4e2a-9485-e73d7a13271a | ceilometer-low | network.services.lb.listener | listener | c87e0f88-26bf-496f-a882-71b1c67956b4 |
| eb634e4e-8e94-4ee4-8fac-4f4db12d56aa | ceilometer-low | network.services.lb.total.connections | connection | 82d129f3-64a5-40d6-a72b-7ae7040
[Regression Potential]
There is no regression potential identified on this patch as it uses
the new metrics upgrade and it doesn't
enables any new metrics it if the lbaasv2 pollster isn't configured.
[Proposed solution]
Enable the resources on /usr/lib/python3/dist packages/ceilometer/publisher/data/gnocchi_resources.yaml
and modify the gnocchi client provider. I am able to gather the metrics for the octavia load balancers.
ubuntu@niedbalski-bastion:~/stsstack-bundles/openstack$ gnocchi metric list|grep network.services
| 3947b249-b1ed-489d-9f51-b7549b5b78ce | ceilometer-low | network.services.lb.loadbalancer | loadbalancer | f97838f1-215d-4fcb-bd78-61e1f9859507 |
| 5527c5b6-d17e-4218-868b-c8d5326575a6 | ceilometer-low | network.services.lb.active.connections | connection | f97838f1-215d-4fcb-bd78-61e1f9859507 |
| 57d63bcb-48e1-47fb-825c-1109ad7d966d | ceilometer-low | network.services.lb.loadbalancer | loadbalancer | 82d129f3-64a5-40d6-a72b-7ae704046176 |
| 5d93c751-0645-4b7b-a315-26b0e96dda3d | ceilometer-low | network.services.lb.outgoing.bytes | B | 82d129f3-64a5-40d6-a72b-7ae704046176 |
| 71ace776-9216-40a4-848c-42e9cc6f93a0 | ceilometer-low | network.services.lb.listener | listener | e7a4dd96-73a1-4aca-946d-8e87ba8f7a37 |
| 738b1dd0-0930-4bfa-b358-1a27fb63121e | ceilometer-low | network.services.lb.total.connections | connection | f97838f1-215d-4fcb-bd78-61e1f9859507 |
| 7b4726f6-9490-4d6c-a586-7e12b738b001 | ceilometer-low | network.services.lb.health_monitor | health_monitor | 3ebc3ac0-8e1c-47d9-8789-bb51e6d29eba |
| 8689e872-c6d0-450f-bb73-d19197c62418 | ceilometer-low | network.services.lb.incoming.bytes | B | 82d129f3-64a5-40d6-a72b-7ae704046176 |
| ae72c5f3-4d8b-400b-ba86-2aaf068c20fc | ceilometer-low | network.services.lb.incoming.bytes | B | f97838f1-215d-4fcb-bd78-61e1f9859507 |
| b59ef5ae-9d71-4884-bb7c-66e0d46f5ecb | ceilometer-low | network.services.lb.outgoing.bytes | B | f97838f1-215d-4fcb-bd78-61e1f9859507 |
| bdf259a1-7471-4c23-ab2b-cac11f522f45 | ceilometer-low | network.services.lb.active.connections | connection | 82d129f3-64a5-40d6-a72b-7ae704046176 |
| e5d2e433-9482-4e2a-9485-e73d7a13271a | ceilometer-low | network.services.lb.listener | listener | c87e0f88-26bf-496f-a882-71b1c67956b4 |
| eb634e4e-8e94-4ee4-8fac-4f4db12d56aa | ceilometer-low | network.services.lb.total.connections | connection | 82d129f3-64a5-40d6-a72b-7ae704046176 |
| ef04f494-884a-4097-ba58-74845f5fbf19 | ceilometer-low | network.services.lb.pool | pool | 6c4a5a4f-0455-42e6-8bf4-ed9c24e2c06c |
ubuntu@niedbalski-bastion:~/stsstack-bundles/openstack$ gnocchi measures show ef04f494-884a-4097-ba58-74845f5fbf19
+---------------------------+-------------+-------+
| timestamp | granularity | value |
+---------------------------+-------------+-------+
| 2019-11-07T18:30:00+00:00 | 300.0 | 1.0 |
| 2019-11-07T18:35:00+00:00 | 300.0 | 1.0 |
| 2019-11-07T18:40:00+00:00 | 300.0 | 1.0 |
+---------------------------+-------------+-------+ |
|
| 2019-11-27 14:20:05 |
Jorge Niedbalski |
description |
[Environment]
Stein OpenStack
Ubuntu Bionic
[Description]
From my understanding the current Octavia API should be backwards compatible with LBaaS v2, therefore, the current LBaaS v2 pollster [1] should suffice to gather the following meters:
meters:
- network.services.lb.outgoing.bytes
- network.services.lb.incoming.bytes
- network.services.lb.pool
- network.services.lb.listener
- network.services.lb.member
- network.services.lb.health_monitor
- network.services.lb.loadbalancer
- network.services.lb.total.connections
- network.services.lb.active.connections
However, the following warning is noticed when restarted the ceilometer services.
2999:2019-11-07 15:05:41.665 22487 WARNING ceilometer.publisher.gnocchi [-] metric network.services.lb.loadbalancer is not handled by Gnocchi
3467:2019-11-07 15:06:21.830 3916 WARNING ceilometer.publisher.gnocchi [-] metric network.services.lb.loadbalancer is not handled by Gnocchi
In fact, by checking at the gnocchi metric list (https://pastebin.canonical.com/p/zyMTFv8vww/) the network.services.lb
metrics/resources haven't been created.
The reason is that for a gnocchi resource to exist, ceilometer has to create the resource type explicitly.
This action is commanded by the ceilometer gnocchi client [2] and it uses the file /usr/lib/python3/dist-packages/ceilometer/publisher/data/gnocchi_resources.yaml
for the resource definitions, which doesn't includes the lbaasv2 directives.
[Test case]
1) Deploy a gnocchi backed ceilometer unit with octavia deployed.
2) check that lb meters aren't being gathered
$ gnocchi metric list|grep loadbalancer | wc -l
0
3) install the patched version
4) configure all pollsters
$ juju config ceilometer enable-all-pollsters=true
5) sudo systemctl restart ceilometer*
6) sudo ceilometer-upgrade
7) Check that lb metrics are being gathered
ubuntu@niedbalski-bastion:~/stsstack-bundles/openstack$ gnocchi metric list|grep loadbalancer | wc -l
72
ubuntu@niedbalski-bastion:~/stsstack-bundles/openstack$ gnocchi metric list|grep network.services
| 3947b249-b1ed-489d-9f51-b7549b5b78ce | ceilometer-low | network.services.lb.loadbalancer | loadbalancer | f97838f1-215d-4fcb-bd78-61e1f9859507 |
| 5527c5b6-d17e-4218-868b-c8d5326575a6 | ceilometer-low | network.services.lb.active.connections | connection | f97838f1-215d-4fcb-bd78-61e1f9859507 |
| 57d63bcb-48e1-47fb-825c-1109ad7d966d | ceilometer-low | network.services.lb.loadbalancer | loadbalancer | 82d129f3-64a5-40d6-a72b-7ae704046176 |
| 5d93c751-0645-4b7b-a315-26b0e96dda3d | ceilometer-low | network.services.lb.outgoing.bytes | B | 82d129f3-64a5-40d6-a72b-7ae704046176 |
| 71ace776-9216-40a4-848c-42e9cc6f93a0 | ceilometer-low | network.services.lb.listener | listener | e7a4dd96-73a1-4aca-946d-8e87ba8f7a37 |
| 738b1dd0-0930-4bfa-b358-1a27fb63121e | ceilometer-low | network.services.lb.total.connections | connection | f97838f1-215d-4fcb-bd78-61e1f9859507 |
| 7b4726f6-9490-4d6c-a586-7e12b738b001 | ceilometer-low | network.services.lb.health_monitor | health_monitor | 3ebc3ac0-8e1c-47d9-8789-bb51e6d29eba |
| 8689e872-c6d0-450f-bb73-d19197c62418 | ceilometer-low | network.services.lb.incoming.bytes | B | 82d129f3-64a5-40d6-a72b-7ae704046176 |
| ae72c5f3-4d8b-400b-ba86-2aaf068c20fc | ceilometer-low | network.services.lb.incoming.bytes | B | f97838f1-215d-4fcb-bd78-61e1f9859507 |
| b59ef5ae-9d71-4884-bb7c-66e0d46f5ecb | ceilometer-low | network.services.lb.outgoing.bytes | B | f97838f1-215d-4fcb-bd78-61e1f9859507 |
| bdf259a1-7471-4c23-ab2b-cac11f522f45 | ceilometer-low | network.services.lb.active.connections | connection | 82d129f3-64a5-40d6-a72b-7ae704046176 |
| e5d2e433-9482-4e2a-9485-e73d7a13271a | ceilometer-low | network.services.lb.listener | listener | c87e0f88-26bf-496f-a882-71b1c67956b4 |
| eb634e4e-8e94-4ee4-8fac-4f4db12d56aa | ceilometer-low | network.services.lb.total.connections | connection | 82d129f3-64a5-40d6-a72b-7ae7040
[Regression Potential]
There is no regression potential identified on this patch as it uses
the new metrics upgrade and it doesn't
enables any new metrics it if the lbaasv2 pollster isn't configured.
[Proposed solution]
Enable the resources on /usr/lib/python3/dist packages/ceilometer/publisher/data/gnocchi_resources.yaml
and modify the gnocchi client provider. I am able to gather the metrics for the octavia load balancers.
ubuntu@niedbalski-bastion:~/stsstack-bundles/openstack$ gnocchi metric list|grep network.services
| 3947b249-b1ed-489d-9f51-b7549b5b78ce | ceilometer-low | network.services.lb.loadbalancer | loadbalancer | f97838f1-215d-4fcb-bd78-61e1f9859507 |
| 5527c5b6-d17e-4218-868b-c8d5326575a6 | ceilometer-low | network.services.lb.active.connections | connection | f97838f1-215d-4fcb-bd78-61e1f9859507 |
| 57d63bcb-48e1-47fb-825c-1109ad7d966d | ceilometer-low | network.services.lb.loadbalancer | loadbalancer | 82d129f3-64a5-40d6-a72b-7ae704046176 |
| 5d93c751-0645-4b7b-a315-26b0e96dda3d | ceilometer-low | network.services.lb.outgoing.bytes | B | 82d129f3-64a5-40d6-a72b-7ae704046176 |
| 71ace776-9216-40a4-848c-42e9cc6f93a0 | ceilometer-low | network.services.lb.listener | listener | e7a4dd96-73a1-4aca-946d-8e87ba8f7a37 |
| 738b1dd0-0930-4bfa-b358-1a27fb63121e | ceilometer-low | network.services.lb.total.connections | connection | f97838f1-215d-4fcb-bd78-61e1f9859507 |
| 7b4726f6-9490-4d6c-a586-7e12b738b001 | ceilometer-low | network.services.lb.health_monitor | health_monitor | 3ebc3ac0-8e1c-47d9-8789-bb51e6d29eba |
| 8689e872-c6d0-450f-bb73-d19197c62418 | ceilometer-low | network.services.lb.incoming.bytes | B | 82d129f3-64a5-40d6-a72b-7ae704046176 |
| ae72c5f3-4d8b-400b-ba86-2aaf068c20fc | ceilometer-low | network.services.lb.incoming.bytes | B | f97838f1-215d-4fcb-bd78-61e1f9859507 |
| b59ef5ae-9d71-4884-bb7c-66e0d46f5ecb | ceilometer-low | network.services.lb.outgoing.bytes | B | f97838f1-215d-4fcb-bd78-61e1f9859507 |
| bdf259a1-7471-4c23-ab2b-cac11f522f45 | ceilometer-low | network.services.lb.active.connections | connection | 82d129f3-64a5-40d6-a72b-7ae704046176 |
| e5d2e433-9482-4e2a-9485-e73d7a13271a | ceilometer-low | network.services.lb.listener | listener | c87e0f88-26bf-496f-a882-71b1c67956b4 |
| eb634e4e-8e94-4ee4-8fac-4f4db12d56aa | ceilometer-low | network.services.lb.total.connections | connection | 82d129f3-64a5-40d6-a72b-7ae704046176 |
| ef04f494-884a-4097-ba58-74845f5fbf19 | ceilometer-low | network.services.lb.pool | pool | 6c4a5a4f-0455-42e6-8bf4-ed9c24e2c06c |
ubuntu@niedbalski-bastion:~/stsstack-bundles/openstack$ gnocchi measures show ef04f494-884a-4097-ba58-74845f5fbf19
+---------------------------+-------------+-------+
| timestamp | granularity | value |
+---------------------------+-------------+-------+
| 2019-11-07T18:30:00+00:00 | 300.0 | 1.0 |
| 2019-11-07T18:35:00+00:00 | 300.0 | 1.0 |
| 2019-11-07T18:40:00+00:00 | 300.0 | 1.0 |
+---------------------------+-------------+-------+ |
[Environment]
Stein OpenStack
Ubuntu Bionic
[Description]
From my understanding the current Octavia API should be backwards compatible with LBaaS v2, therefore, the current LBaaS v2 pollster [1] should suffice to gather the following meters:
meters:
- network.services.lb.outgoing.bytes
- network.services.lb.incoming.bytes
- network.services.lb.pool
- network.services.lb.listener
- network.services.lb.member
- network.services.lb.health_monitor
- network.services.lb.loadbalancer
- network.services.lb.total.connections
- network.services.lb.active.connections
However, the following warning is noticed when restarted the ceilometer services.
2999:2019-11-07 15:05:41.665 22487 WARNING ceilometer.publisher.gnocchi [-] metric network.services.lb.loadbalancer is not handled by Gnocchi
3467:2019-11-07 15:06:21.830 3916 WARNING ceilometer.publisher.gnocchi [-] metric network.services.lb.loadbalancer is not handled by Gnocchi
In fact, by checking at the gnocchi metric list (https://pastebin.canonical.com/p/zyMTFv8vww/) the network.services.lb
metrics/resources haven't been created.
The reason is that for a gnocchi resource to exist, ceilometer has to create the resource type explicitly.
This action is commanded by the ceilometer gnocchi client [2] and it uses the file /usr/lib/python3/dist-packages/ceilometer/publisher/data/gnocchi_resources.yaml
for the resource definitions, which doesn't includes the lbaasv2 directives.
[Test case]
1) Deploy a gnocchi backed ceilometer unit with octavia deployed.
2) check that lb meters aren't being gathered
$ gnocchi metric list|grep loadbalancer | wc -l
0
3) install the patched version
4) configure all pollsters
$ juju config ceilometer enable-all-pollsters=true
5) sudo systemctl restart ceilometer*
6) sudo ceilometer-upgrade
7) Check that lb metrics are being gathered
ubuntu@niedbalski-bastion:~/stsstack-bundles/openstack$ gnocchi metric list|grep loadbalancer | wc -l
72
ubuntu@niedbalski-bastion:~/stsstack-bundles/openstack$ gnocchi metric list|grep network.services
| 3947b249-b1ed-489d-9f51-b7549b5b78ce | ceilometer-low | network.services.lb.loadbalancer | loadbalancer | f97838f1-215d-4fcb-bd78-61e1f9859507 |
| 5527c5b6-d17e-4218-868b-c8d5326575a6 | ceilometer-low | network.services.lb.active.connections | connection | f97838f1-215d-4fcb-bd78-61e1f9859507 |
| 57d63bcb-48e1-47fb-825c-1109ad7d966d | ceilometer-low | network.services.lb.loadbalancer | loadbalancer | 82d129f3-64a5-40d6-a72b-7ae704046176 |
| 5d93c751-0645-4b7b-a315-26b0e96dda3d | ceilometer-low | network.services.lb.outgoing.bytes | B | 82d129f3-64a5-40d6-a72b-7ae704046176 |
| 71ace776-9216-40a4-848c-42e9cc6f93a0 | ceilometer-low | network.services.lb.listener | listener | e7a4dd96-73a1-4aca-946d-8e87ba8f7a37 |
| 738b1dd0-0930-4bfa-b358-1a27fb63121e | ceilometer-low | network.services.lb.total.connections | connection | f97838f1-215d-4fcb-bd78-61e1f9859507 |
| 7b4726f6-9490-4d6c-a586-7e12b738b001 | ceilometer-low | network.services.lb.health_monitor | health_monitor | 3ebc3ac0-8e1c-47d9-8789-bb51e6d29eba |
| 8689e872-c6d0-450f-bb73-d19197c62418 | ceilometer-low | network.services.lb.incoming.bytes | B | 82d129f3-64a5-40d6-a72b-7ae704046176 |
| ae72c5f3-4d8b-400b-ba86-2aaf068c20fc | ceilometer-low | network.services.lb.incoming.bytes | B | f97838f1-215d-4fcb-bd78-61e1f9859507 |
| b59ef5ae-9d71-4884-bb7c-66e0d46f5ecb | ceilometer-low | network.services.lb.outgoing.bytes | B | f97838f1-215d-4fcb-bd78-61e1f9859507 |
| bdf259a1-7471-4c23-ab2b-cac11f522f45 | ceilometer-low | network.services.lb.active.connections | connection | 82d129f3-64a5-40d6-a72b-7ae704046176 |
| e5d2e433-9482-4e2a-9485-e73d7a13271a | ceilometer-low | network.services.lb.listener | listener | c87e0f88-26bf-496f-a882-71b1c67956b4 |
| eb634e4e-8e94-4ee4-8fac-4f4db12d56aa | ceilometer-low | network.services.lb.total.connections | connection | 82d129f3-64a5-40d6-a72b-7ae7040
[Regression Potential]
There is no regression potential identified on this patch as it uses
the new metrics upgrade path and it doesn't
enables any new metrics if the lbaasv2 pollster isn't configured.
[Proposed solution]
Enable the resources on /usr/lib/python3/dist packages/ceilometer/publisher/data/gnocchi_resources.yaml
and modify the gnocchi client provider. I am able to gather the metrics for the octavia load balancers.
ubuntu@niedbalski-bastion:~/stsstack-bundles/openstack$ gnocchi metric list|grep network.services
| 3947b249-b1ed-489d-9f51-b7549b5b78ce | ceilometer-low | network.services.lb.loadbalancer | loadbalancer | f97838f1-215d-4fcb-bd78-61e1f9859507 |
| 5527c5b6-d17e-4218-868b-c8d5326575a6 | ceilometer-low | network.services.lb.active.connections | connection | f97838f1-215d-4fcb-bd78-61e1f9859507 |
| 57d63bcb-48e1-47fb-825c-1109ad7d966d | ceilometer-low | network.services.lb.loadbalancer | loadbalancer | 82d129f3-64a5-40d6-a72b-7ae704046176 |
| 5d93c751-0645-4b7b-a315-26b0e96dda3d | ceilometer-low | network.services.lb.outgoing.bytes | B | 82d129f3-64a5-40d6-a72b-7ae704046176 |
| 71ace776-9216-40a4-848c-42e9cc6f93a0 | ceilometer-low | network.services.lb.listener | listener | e7a4dd96-73a1-4aca-946d-8e87ba8f7a37 |
| 738b1dd0-0930-4bfa-b358-1a27fb63121e | ceilometer-low | network.services.lb.total.connections | connection | f97838f1-215d-4fcb-bd78-61e1f9859507 |
| 7b4726f6-9490-4d6c-a586-7e12b738b001 | ceilometer-low | network.services.lb.health_monitor | health_monitor | 3ebc3ac0-8e1c-47d9-8789-bb51e6d29eba |
| 8689e872-c6d0-450f-bb73-d19197c62418 | ceilometer-low | network.services.lb.incoming.bytes | B | 82d129f3-64a5-40d6-a72b-7ae704046176 |
| ae72c5f3-4d8b-400b-ba86-2aaf068c20fc | ceilometer-low | network.services.lb.incoming.bytes | B | f97838f1-215d-4fcb-bd78-61e1f9859507 |
| b59ef5ae-9d71-4884-bb7c-66e0d46f5ecb | ceilometer-low | network.services.lb.outgoing.bytes | B | f97838f1-215d-4fcb-bd78-61e1f9859507 |
| bdf259a1-7471-4c23-ab2b-cac11f522f45 | ceilometer-low | network.services.lb.active.connections | connection | 82d129f3-64a5-40d6-a72b-7ae704046176 |
| e5d2e433-9482-4e2a-9485-e73d7a13271a | ceilometer-low | network.services.lb.listener | listener | c87e0f88-26bf-496f-a882-71b1c67956b4 |
| eb634e4e-8e94-4ee4-8fac-4f4db12d56aa | ceilometer-low | network.services.lb.total.connections | connection | 82d129f3-64a5-40d6-a72b-7ae704046176 |
| ef04f494-884a-4097-ba58-74845f5fbf19 | ceilometer-low | network.services.lb.pool | pool | 6c4a5a4f-0455-42e6-8bf4-ed9c24e2c06c |
ubuntu@niedbalski-bastion:~/stsstack-bundles/openstack$ gnocchi measures show ef04f494-884a-4097-ba58-74845f5fbf19
+---------------------------+-------------+-------+
| timestamp | granularity | value |
+---------------------------+-------------+-------+
| 2019-11-07T18:30:00+00:00 | 300.0 | 1.0 |
| 2019-11-07T18:35:00+00:00 | 300.0 | 1.0 |
| 2019-11-07T18:40:00+00:00 | 300.0 | 1.0 |
+---------------------------+-------------+-------+ |
|
| 2019-11-27 14:57:58 |
Corey Bryant |
cloud-archive/rocky: status |
Fix Committed |
Triaged |
|
| 2019-11-27 14:58:11 |
Corey Bryant |
cloud-archive/stein: status |
Fix Committed |
Triaged |
|
| 2019-11-27 15:08:24 |
Corey Bryant |
bug |
|
|
added subscriber Ubuntu Stable Release Updates Team |
| 2019-11-27 19:22:22 |
Launchpad Janitor |
ceilometer (Ubuntu Focal): status |
Triaged |
Fix Released |
|
| 2019-11-28 13:24:36 |
OpenStack Infra |
cloud-archive/train: status |
Triaged |
Fix Committed |
|
| 2019-11-29 11:34:38 |
Timo Aaltonen |
ceilometer (Ubuntu Eoan): status |
Triaged |
Fix Committed |
|
| 2019-11-29 11:34:44 |
Timo Aaltonen |
bug |
|
|
added subscriber SRU Verification |
| 2019-11-29 11:34:50 |
Timo Aaltonen |
tags |
|
verification-needed verification-needed-eoan |
|
| 2019-11-29 11:38:34 |
Timo Aaltonen |
ceilometer (Ubuntu Disco): status |
Triaged |
Fix Committed |
|
| 2019-11-29 11:38:42 |
Timo Aaltonen |
tags |
verification-needed verification-needed-eoan |
verification-needed verification-needed-disco verification-needed-eoan |
|
| 2019-12-02 14:41:30 |
Corey Bryant |
tags |
verification-needed verification-needed-disco verification-needed-eoan |
verification-needed verification-needed-disco verification-needed-eoan verification-train-needed |
|
| 2019-12-02 14:43:52 |
Corey Bryant |
cloud-archive/stein: status |
Triaged |
Fix Committed |
|
| 2019-12-02 14:43:57 |
Corey Bryant |
tags |
verification-needed verification-needed-disco verification-needed-eoan verification-train-needed |
verification-needed verification-needed-disco verification-needed-eoan verification-stein-needed verification-train-needed |
|
| 2019-12-02 14:44:42 |
Corey Bryant |
cloud-archive/rocky: status |
Triaged |
Fix Committed |
|
| 2019-12-02 14:44:45 |
Corey Bryant |
tags |
verification-needed verification-needed-disco verification-needed-eoan verification-stein-needed verification-train-needed |
verification-needed verification-needed-disco verification-needed-eoan verification-rocky-needed verification-stein-needed verification-train-needed |
|
| 2019-12-02 16:39:05 |
Jorge Niedbalski |
tags |
verification-needed verification-needed-disco verification-needed-eoan verification-rocky-needed verification-stein-needed verification-train-needed |
verification-done verification-done-disco verification-done-eoan verification-rocky-done verification-stein-done verification-train-done |
|
| 2019-12-09 08:28:26 |
Launchpad Janitor |
ceilometer (Ubuntu Eoan): status |
Fix Committed |
Fix Released |
|
| 2019-12-09 08:28:32 |
Łukasz Zemczak |
removed subscriber Ubuntu Stable Release Updates Team |
|
|
|
| 2019-12-09 08:28:49 |
Launchpad Janitor |
ceilometer (Ubuntu Disco): status |
Fix Committed |
Fix Released |
|
| 2019-12-09 13:39:05 |
Corey Bryant |
cloud-archive/train: status |
Fix Committed |
Fix Released |
|
| 2019-12-09 13:39:39 |
Corey Bryant |
cloud-archive/stein: status |
Fix Committed |
Fix Released |
|
| 2019-12-09 13:40:12 |
Corey Bryant |
bug task deleted |
cloud-archive/ussuri |
|
|
| 2019-12-09 13:41:01 |
Corey Bryant |
cloud-archive/rocky: status |
Fix Committed |
Fix Released |
|
| 2020-07-17 22:36:53 |
Simon Quigley |
removed subscriber Ubuntu Sponsors Team |
|
|
|
| 2020-08-18 08:00:50 |
Stefan Hoffmann |
bug |
|
|
added subscriber Stefan Hoffmann |
| 2020-12-23 08:33:10 |
Mathew Hodson |
affects |
charm-octavia |
ubuntu-translations |
|
| 2020-12-23 08:33:25 |
Mathew Hodson |
bug task deleted |
ubuntu-translations |
|
|
