Comment 4 for bug 1902516

The failed tempest run with concurrency=4 so it might be multiple VMs launched in parallel.

Looking at the dstat log around the time when the VM failed there was around 110MB free memory on the hypervisor, the VM was booted with a flavor requesting 128MB of ram.

When the scheduler accepted the host it saw > 4G free ram.

Nov 02 09:13:20.149808 ubuntu-bionic-rax-ord-0021331595 nova-scheduler[20312]: DEBUG nova.scheduler.filter_scheduler [None req-9c065faf-96c5-42fd-833c-e0c015457188 tempest-TrunkTest-649105798 tempest-TrunkTest-649105798] [instance: 69366a0b-6d15-4e8b-9f90-e9bec06ca126] Selected host: (ubuntu-bionic-rax-ord-0021331595, ubuntu-bionic-rax-ord-0021331595) ram: 4258MB disk: 29696MB io_ops: 0 instances: 7

The last compute resource report before the VM boot failure reported the following:

_report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1039}}
Nov 02 09:13:19.335170 ubuntu-bionic-rax-ord-0021331595 nova-compute[23193]: DEBUG nova.compute.resource_tracker [None req-b4825f7c-63f9-412c-8579-126a178955a8 None None] Final resource view: name=ubuntu-bionic-rax-ord-0021331595 phys_ram=7970MB used_ram=3712MB phys_disk=76GB used_disk=31GB total_vcpus=8 used_vcpus=7 pci_stats=[] {{(pid=23193) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1061}}

This is consistent what the scheduler used for the decision making that the request fits on this host.

So I think the value of the reserverd_host_memory_mb in nova-cpu.conf (which is 512) are not reflecting the reality on the hypervisor.