ironic hypervisor is still available when the node is in maintenance status.

Bug #1435153 reported by Haomeng,Wang
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Ironic
Won't Fix
High
Unassigned
OpenStack Compute (nova)
Invalid
Undecided
Unassigned

Bug Description

In my env, we have two ironic nodes, so in hypervisor-stats, show count=2, after I set one of them to maintenance state, the hypervisor-stats still show count=2, I understand for the node which is in maintenance status, it should not be counted into nova hypervisor stats.

[root@rhel7-osee ~]# nova hypervisor-stats
+----------------------+-------+
| Property | Value |
+----------------------+-------+
| count | 2 |
| current_workload | 0 |
| disk_available_least | 0 |
| free_disk_gb | 0 |
| free_ram_mb | 0 |
| local_gb | 40 |
| local_gb_used | 40 |
| memory_mb | 2048 |
| memory_mb_used | 2048 |
| running_vms | 2 |
| vcpus | 2 |
| vcpus_used | 2 |
+----------------------+-------+
[root@rhel7-osee ~]# ironic node-set-maintenance d4edf8c7-ae8d-40ed-b3a6-c5600ff09287 on
[root@rhel7-osee ~]# nova hypervisor-stats
+----------------------+-------+
| Property | Value |
+----------------------+-------+
| count | 2 |
| current_workload | 0 |
| disk_available_least | 0 |
| free_disk_gb | 0 |
| free_ram_mb | 0 |
| local_gb | 40 |
| local_gb_used | 40 |
| memory_mb | 2048 |
| memory_mb_used | 2048 |
| running_vms | 2 |
| vcpus | 2 |
| vcpus_used | 2 |
+----------------------+-------+

Tags: ironic
Haomeng,Wang (whaom)
summary: - ironic hyprevisor still available after it is set to [root@rhel7-osee
- ~]# nova hypervisor-stats +----------------------+-------+ | Property
- | Value | +----------------------+-------+ | count | 2
- | | current_workload | 0 | | disk_available_least | 0 | |
- free_disk_gb | 0 | | free_ram_mb | 0 | |
- local_gb | 40 | | local_gb_used | 40 | |
- memory_mb | 2048 | | memory_mb_used | 2048 | |
- running_vms | 2 | | vcpus | 2 | | vcpus
+ ironic hyprevisor still available when node is in maintenance status.
summary: - ironic hyprevisor still available when node is in maintenance status.
+ ironic hypervisor still available when node is in maintenance status.
Haomeng,Wang (whaom)
summary: - ironic hypervisor still available when node is in maintenance status.
+ ironic hypervisor is still available when the node is in maintenance
+ status.
Revision history for this message
Jim Rollenhagen (jim-rollenhagen) wrote :

Are we sure this isn't resource tracker lag?

Changed in ironic:
milestone: none → liberty-1
importance: Undecided → High
status: New → Triaged
Changed in ironic:
assignee: nobody → Zhenguo Niu (niu-zglinux)
Revision history for this message
Zhenguo Niu (niu-zglinux) wrote :

I'll upload a patch to address this issue on nova side.

Changed in nova:
assignee: nobody → Zhenguo Niu (niu-zglinux)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/177575

Changed in nova:
status: New → In Progress
Revision history for this message
Dmitry Tantsur (divius) wrote :

Thanks for the Nova fix. I guess we can't fix anything on Ironic side, can we?

Changed in ironic:
status: Triaged → Won't Fix
Revision history for this message
Zhenguo Niu (niu-zglinux) wrote :

@Dmitry yes, we don't need to fix on Ironic side.

Changed in ironic:
assignee: Zhenguo Niu (niu-zglinux) → nobody
Michael Still (mikal)
tags: added: ironic
Revision history for this message
Jim Rollenhagen (jim-rollenhagen) wrote :

I don't think this is a valid bug.

I think this is just resource tracker lag; the resource tracker only runs periodically. The Ironic driver certainly presents a node in maintenance as unavailable: https://github.com/openstack/nova/blob/8121b2397c531fc1479ad90eb296c58efa025944/nova/virt/ironic/driver.py#L283

stack@jim-devstack:~/devstack$ nova hypervisor-stats
+----------------------+-------+
| Property | Value |
+----------------------+-------+
| count | 3 |
| current_workload | 0 |
| disk_available_least | 30 |
| free_disk_gb | 30 |
| free_ram_mb | 3072 |
| local_gb | 30 |
| local_gb_used | 0 |
| memory_mb | 3072 |
| memory_mb_used | 0 |
| running_vms | 0 |
| vcpus | 3 |
| vcpus_used | 0 |
+----------------------+-------+
stack@jim-devstack:~/devstack$ ironic node-set-maintenance a0a07a43-2a62-40ce-abdd-13ad8af89fdc on
stack@jim-devstack:~/devstack$ nova hypervisor-stats
+----------------------+-------+
| Property | Value |
+----------------------+-------+
| count | 3 |
| current_workload | 0 |
| disk_available_least | 30 |
| free_disk_gb | 30 |
| free_ram_mb | 3072 |
| local_gb | 30 |
| local_gb_used | 0 |
| memory_mb | 3072 |
| memory_mb_used | 0 |
| running_vms | 0 |
| vcpus | 3 |
| vcpus_used | 0 |
+----------------------+-------+
stack@jim-devstack:~/devstack$ sleep 300
stack@jim-devstack:~/devstack$ nova hypervisor-stats
+----------------------+-------+
| Property | Value |
+----------------------+-------+
| count | 3 |
| current_workload | 0 |
| disk_available_least | 20 |
| free_disk_gb | 20 |
| free_ram_mb | 2048 |
| local_gb | 20 |
| local_gb_used | 0 |
| memory_mb | 2048 |
| memory_mb_used | 0 |
| running_vms | 0 |
| vcpus | 2 |
| vcpus_used | 0 |
+----------------------+-------+

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on nova (master)

Change abandoned by Zhenguo Niu (<email address hidden>) on branch: master
Review: https://review.openstack.org/177575

Revision history for this message
Ruby Loo (rloo) wrote :

Setting this to invalid because some of us don't think this is a bug. Ie, that nodes in maintenance should be counted/shown in nova. Their resources show up as unavailable, so it seems fine.

See comments in https://review.openstack.org/177575.

Changed in nova:
status: In Progress → Invalid
Changed in nova:
assignee: Zhenguo Niu (niu-zglinux) → nobody
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.