nova.notification does not sending 'locked' attribute in compute.instance.* events

Bug #1948776 reported by Nikolay Nosenko
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Triaged
Wishlist
Unassigned

Bug Description

We need to receive events for action compute.instance.update with instance attribute 'locked', but don't received it. I've checked information, that nova sending to RabbitMQ after locking instance (ngrep trace):

........<.(...nova.notifications.info........+.<.............application/json.utf-8..............{"oslo.message": "{\"_context_domain\": null, \"_context_roles\": [\"_member_\", \"admin\"], \"_context_global_request_id\": null, \"_context_quota_class\": null, \"event_type\": \"compute.instance.update\", \"_context_request_id\": \"req-28b73ad5-09ea-461c-86df-8f79e330466e\", \"_context_service_catalog\": [{\"endpoints\": [{\"adminURL\": \"http://nova-api.p.nova-lab-2:28776/v2/341fff2ede3a45bd89fefc4707427e43\", \"region\": \"lab-2\", \"internalURL\": \"http://nova-api.p.nova-lab-2:28776/v2/341fff2ede3a45bd89fefc4707427e43\", \"publicURL\": \"https://volume.nova-lab-2:8776/v2/341fff2ede3a45bd89fefc4707427e43\"}], \"type\": \"volumev2\", \"name\": \"cinderv2\"}, {\"endpoints\": [{\"adminURL\": \"http://glance-api.p.nova-lab-2:29292\", \"region\": \"lab-2\", \"internalURL\": \"http://glance-api.p.nova-lab-2:29292\", \"publicURL\": \"https://images.nova-lab-2:9292\"}], \"type\": \"image\", \"name\": \"glance\"}, {\"endpoints\": [{\"adminURL\": \"http://nova-api.p.nova-lab-2:28778\", \"region\": \"lab-2\", \"internalURL\": \"http://nova-api.p.nova-lab-2:28778\", \"publicURL\": \"https://compute.nova-lab-2:8778\"}], \"type\": \"placement\", \"name\": \"placement\"}, {\"endpoints\": [{\"adminURL\": \"http://nova-api.p.nova-lab-2:28776/v3/341fff2ede3a45bd89fefc4707427e43\", \"region\": \"lab-2\", \"internalURL\": \"http://nova-api.p.nova-lab-2:28776/v3/341fff2ede3a45bd89fefc4707427e43\", \"publicURL\": \"https://volume.nova-lab-2:8776/v3/341fff2ede3a45bd89fefc4707427e43\"}], \"type\": \"volumev3\", \"name\": \"cinderv3\"}, {\"endpoints\": [{\"adminURL\": \"http://neutron-server.p.nova-lab-2:29696\", \"region\": \"lab-2\", \"internalURL\": \"http://neutron-server.p.nova-lab-2:29696\", \"publicURL\": \"https://network.nova-lab-2:9696\"}], \"type\": \"network\", \"name\": \"neutron\"}], \"timestamp\": \"2021-10-26 07:18:24.871121\", \"_context_user\": \"943c3b8ee03940cbb21ff364627119b3\", \"_unique_id\": \"799a3556705349fdb539ca1f7ca16bcc\", \"_context_resource_uuid\": null, \"_context_instance_lock_checked\": false, \"_context_is_admin_project\": true, \"_context_user_id\": \"943c3b8ee03940cbb21ff364627119b3\", \"payload\": {\"state_description\": \"\", \"availability_zone\": \"nova\", \"terminated_at\": \"\", \"ephemeral_gb\": 0, \"instance_type_id\": 6, \"bandwidth\": {}, \"deleted_at\": \"\", \"reservation_id\": \"r-f03eog7o\", \"instance_id\": \"e751a63a-27f2-4a01-b2ae-8398436f7f0b\", \"display_name\": \"ams_test_1\", \"hostname\": \"ams-test-1\", \"state\": \"active\", \"old_state\": \"active\", \"progress\": \"\", \"launched_at\": \"2021-09-14T12:47:46.000000\", \"metadata\": {}, \"node\": \"compute1.nova-lab-2\", \"ramdisk_id\": \"\", \"access_ip_v6\": null, \"disk_gb\": 181, \"access_ip_v4\": null, \"kernel_id\": \"\", \"host\": \"compute1\", \"user_id\": \"42ad97da2b194c42a48726115ed1b224\", \"image_ref_url\": \"http://glance-api.p.nova-lab-2:29292/images/7fd6c515-714f-4f70-a7ed-cf09a8372349\", \"cell_name\": \"\", \"audit_period_beginning\": \"2021-10-01T00:00:00.000000\", \"root_gb\": 181, \"tenant_id\": \"341fff2ede3a45bd89fefc4707427e43\", \"created_at\": \"2021-09-14 09:48:50+00:00\", \"old_task_state\": null, \"memory_mb\": 16384, \"instance_type\": \"SSD.180\", \"vcpus\": 6, \"image_meta\": {\"os_distro\": \"ubuntu\", \"display_priority\": \"250\", \"min_ram\": \"0\", \"disk_format\": \"qcow2\", \"obsolete\": \"True\", \"os_arch\": \"x86_64\", \"os_version\": \"20.04\", \"architecture\": \"x86_64\", \"container_format\": \"bare\", \"min_disk\": \"181\", \"base_image_ref\": \"7fd6c515-714f-4f70-a7ed-cf09a8372349\"}, \"architecture\": \"x86_64\", \"new_task_state\": null, \"audit_period_ending\": \"2021-10-26T07:18:24.864303\", \"os_type\": null, \"instance_flavor_id\": \"180\"}, \"_context_project_name\": \"demotenant\", \"_context_read_deleted\": \"no\", \"_context_user_identity\": \"943c3b8ee03940cbb21ff364627119b3 341fff2ede3a45bd89fefc4707427e43 - default default\", \"_context_auth_token\": \"gAAAAABhd6vAermQrNwlH_2RCXbyO726OO61W7V8OrCsZKU3W6tpwUy0ES1HAf-95twFF9SYeGOLEXtAElDgH7xN-0IhFHpoQsecNFJH-6FFyCif9W4xUKz9lsfh0-8AfEhgjrMVo1f-icfEjRP7E0bMZ-mq2-8thlhgg2HVF_YuDbd_Kk9olsY\", \"_context_show_deleted\": false, \"_context_tenant\": \"341fff2ede3a45bd89fefc4707427e43\", \"priority\": \"INFO\", \"_context_read_only\": false, \"_context_is_admin\": true, \"_context_project_id\": \"341fff2ede3a45bd89fefc4707427e43\", \"_context_project_domain\": \"default\", \"_context_timestamp\": \"2021-10-26T07:18:24.663849\", \"_context_user_domain\": \"default\", \"_context_user_name\": \"admin\", \"publisher_id\": \"compute.ops1\", \"message_id\": \"200a83cd-a5b9-4545-9a9c-2039237ed200\", \"_context_project\": \"341fff2ede3a45bd89fefc4707427e43\", \"_context_remote_address\": \"192.168.26.61\"}", "oslo.version": "2.0"}.

The payload does not have locked. We found, that payload of instance attributes collecting by function nova.notification.base.info_from_instance and after adding locked attribute to instance_info dictionary we starting to receive 'locked' attribute.

Diff:
diff --git a/nova/notifications/base.py b/nova/notifications/base.py
index 4b1f7beeb4..c68a5d76a2 100644
--- a/nova/notifications/base.py
+++ b/nova/notifications/base.py
@@ -491,6 +491,7 @@ def info_from_instance(context, instance, network_info,
         # accessIPs
         access_ip_v4=instance.access_ip_v4,
         access_ip_v6=instance.access_ip_v6,
+ locked=instance.locked
         )

     if network_info is not None:

RabbitMQ receive:
........<.(...nova.notifications.info........+.<.............application/json.utf-8..............{"oslo.message": "{\"_context_domain\": null, \"_context_roles\": [\"_member_\", \"admin\"], \"_context_global_request_id\": null, \"_context_quota_class\": null, \"event_type\": \"compute.instance.update\", \"_context_request_id\": \"req-ca3d18fa-ad65-4ed8-beda-0ba23eab92f9\", \"_context_service_catalog\": [{\"endpoints\": [{\"adminURL\": \"http://nova-api.p.nova-lab-2:28776/v2/341fff2ede3a45bd89fefc4707427e43\", \"region\": \"lab-2\", \"internalURL\": \"http://nova-api.p.nova-lab-2:28776/v2/341fff2ede3a45bd89fefc4707427e43\", \"publicURL\": \"https://volume.nova-lab-2:8776/v2/341fff2ede3a45bd89fefc4707427e43\"}], \"type\": \"volumev2\", \"name\": \"cinderv2\"}, {\"endpoints\": [{\"adminURL\": \"http://glance-api.p.nova-lab-2:29292\", \"region\": \"lab-2\", \"internalURL\": \"http://glance-api.p.nova-lab-2:29292\", \"publicURL\": \"https://images.nova-lab-2:9292\"}], \"type\": \"image\", \"name\": \"glance\"}, {\"endpoints\": [{\"adminURL\": \"http://nova-api.p.nova-lab-2:28778\", \"region\": \"lab-2\", \"internalURL\": \"http://nova-api.p.nova-lab-2:28778\", \"publicURL\": \"https://compute.nova-lab-2:8778\"}], \"type\": \"placement\", \"name\": \"placement\"}, {\"endpoints\": [{\"adminURL\": \"http://nova-api.p.nova-lab-2:28776/v3/341fff2ede3a45bd89fefc4707427e43\", \"region\": \"lab-2\", \"internalURL\": \"http://nova-api.p.nova-lab-2:28776/v3/341fff2ede3a45bd89fefc4707427e43\", \"publicURL\": \"https://volume.nova-lab-2:8776/v3/341fff2ede3a45bd89fefc4707427e43\"}], \"type\": \"volumev3\", \"name\": \"cinderv3\"}, {\"endpoints\": [{\"adminURL\": \"http://neutron-server.p.nova-lab-2:29696\", \"region\": \"lab-2\", \"internalURL\": \"http://neutron-server.p.nova-lab-2:29696\", \"publicURL\": \"https://network.nova-lab-2:9696\"}], \"type\": \"network\", \"name\": \"neutron\"}], \"timestamp\": \"2021-10-26 09:14:26.609089\", \"_context_user\": \"943c3b8ee03940cbb21ff364627119b3\", \"_unique_id\": \"0750bd9b83b044a68b25bd2c53357b7f\", \"_context_resource_uuid\": null, \"_context_instance_lock_checked\": false, \"_context_is_admin_project\": true, \"_context_user_id\": \"943c3b8ee03940cbb21ff364627119b3\", \"payload\": {\"state_description\": \"\", \"availability_zone\": \"nova\", \"terminated_at\": \"\", \"ephemeral_gb\": 0, \"instance_type_id\": 6, \"bandwidth\": {}, \"deleted_at\": \"\", \"reservation_id\": \"r-f03eog7o\", \"instance_id\": \"e751a63a-27f2-4a01-b2ae-8398436f7f0b\", \"display_name\": \"ams_test_1\", \"hostname\": \"ams-test-1\", \"state\": \"active\", \"old_state\": \"active\", \"progress\": \"\", \"launched_at\": \"2021-09-14T12:47:46.000000\", \"metadata\": {}, \"node\": \"compute1.nova-lab-2\", \"ramdisk_id\": \"\", \"access_ip_v6\": null, \"disk_gb\": 181, \"access_ip_v4\": null, \"kernel_id\": \"\", \"host\": \"compute1\", \"user_id\": \"42ad97da2b194c42a48726115ed1b224\", \"image_ref_url\": \"http://glance-api.p.nova-lab-2:29292/images/7fd6c515-714f-4f70-a7ed-cf09a8372349\", \"cell_name\": \"\", \"audit_period_beginning\": \"2021-10-01T00:00:00.000000\", \"root_gb\": 181, \"locked\": true, \"tenant_id\": \"341fff2ede3a45bd89fefc4707427e43\", \"created_at\": \"2021-09-14 09:48:50+00:00\", \"old_task_state\": null, \"memory_mb\": 16384, \"instance_type\": \"SSD.180\", \"vcpus\": 6, \"image_meta\": {\"os_distro\": \"ubuntu\", \"display_priority\": \"250\", \"min_ram\": \"0\", \"disk_format\": \"qcow2\", \"obsolete\": \"True\", \"os_arch\": \"x86_64\", \"os_version\": \"20.04\", \"architecture\": \"x86_64\", \"container_format\": \"bare\", \"min_disk\": \"181\", \"base_image_ref\": \"7fd6c515-714f-4f70-a7ed-cf09a8372349\"}, \"architecture\": \"x86_64\", \"new_task_state\": null, \"audit_period_ending\": \"2021-10-26T09:14:26.602357\", \"os_type\": null, \"instance_flavor_id\": \"180\"}, \"_context_project_name\": \"demotenant\", \"_context_read_deleted\": \"no\", \"_context_user_identity\": \"943c3b8ee03940cbb21ff364627119b3 341fff2ede3a45bd89fefc4707427e43 - default default\", \"_context_auth_token\": \"gAAAAABhd8bxhm2kjE_hX5PuNbDndvo_op5jjciIKsp4_EKTfRX8VbvJ43ZqpDBltvsKQQ8oYE-yPIer2yIi9JNNNw44-Po0nEabBLTe7d_I9zn6Yrc1j3yWS7yeUCiZlJoK68CkEH_E47ea66ypPv4oF3Yu1MiuzKO49q6vQIl8ALYGuks0zrA\", \"_context_show_deleted\": false, \"_context_tenant\": \"341fff2ede3a45bd89fefc4707427e43\", \"priority\": \"INFO\", \"_context_read_only\": false, \"_context_is_admin\": true, \"_context_project_id\": \"341fff2ede3a45bd89fefc4707427e43\", \"_context_project_domain\": \"default\", \"_context_timestamp\": \"2021-10-26T09:14:26.399880\", \"_context_user_domain\": \"default\", \"_context_user_name\": \"admin\", \"publisher_id\": \"compute.ops1\", \"message_id\": \"096a04ef-4cec-4497-8c93-27941a51f5e6\", \"_context_project\": \"341fff2ede3a45bd89fefc4707427e43\", \"_context_remote_address\": \"192.168.26.61\"}", "oslo.version": "2.0"}.

Tested with Queens versions, checked code in Ussuri - does not exist - https://github.com/openstack/nova/blob/stable/ussuri/nova/notifications/base.py#L391-L451

Please, add provided diff to Ussuri and newer releases.

Revision history for this message
Sylvain Bauza (sylvain-bauza) wrote :
Changed in nova:
status: New → Confirmed
importance: Undecided → Low
Revision history for this message
sean mooney (sean-k-mooney) wrote :

we cannot alter the payload of version notification so realisticlly this is a gap that we can only adress in yoga so i don think we shoudl be taggin thie with queens or ussuri since it is not backporatble.

but going forward we could extend the version notificaiton with this new filed.

Changed in nova:
importance: Low → Wishlist
status: Confirmed → Triaged
tags: removed: nova queens ussuri
tags: removed: events
Revision history for this message
Balazs Gibizer (balazs-gibizer) wrote :

I agree with Sean. It is feature request not a bug.

description: updated
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.