Cannot launch an instance with local storage

Bug #1799312 reported by Juan Carlos Alonso on 2018-10-22
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
StarlingX
High
yong hu

Bug Description

Title
-----

Cannot launch an instance with local storage (aggregate_instance_extra_specs=local_lvm)

Brief Description
-----------------

After create a flavor with metadata: 'aggregate_instance_extra_specs=local_lvm', image, network, subnetwork, proceed to launch an instance but get an Error status.

Severity
--------

<Critical: System/Feature is not usable after the defect>

Steps to Reproduce
------------------

$ openstack flavor create --ram 2048 --disk 20 --vcpus 1 m1.small
$ openstack flavor set --property aggregate_instance_extra_specs=local_lvm m1.small
$ openstack image create --disk-format qcow2 --file cirros.img cirros
$ openstack network create net-1
$ openstack subnet create --network net-1 --subnet-range 192.168.0.0/24 --ip-version 4 --dhcp subnet-1
$ openstack server create --flavor m1.small --image cirros --nic net-id=<net-id> cirros-vm

Expected Behavior
------------------

Instance launched successfully

Actual Behavior
----------------

[wrsroot@controller-1 ~(keystone_admin)]$ openstack server list
+--------------------------------------+-----------+--------+----------+--------+----------+
| ID | Name | Status | Networks | Image | Flavor |
+--------------------------------------+-----------+--------+----------+--------+----------+
| ea98c771-2de5-4fdd-80ac-0fb648d79e1c | cirros-vm | ERROR | | cirros | m1.small |
+--------------------------------------+-----------+--------+----------+--------+----------+

Reproducibility
---------------

100%

System Configuration
--------------------

Multinode Local Storage Virtual Environment
Multinode Local Storage Bare Metal
Multinode External Storage Virtual Environment
Multinode External Storage Bare Metal

Bruce Jones (brucej) wrote :

Can work around this by not setting the local_lvm flag. Not release blocking.

tags: added: stx.2019.03
Changed in starlingx:
assignee: nobody → Cindy Xie (xxie1)
status: New → Triaged
importance: Undecided → High
Ghada Khalil (gkhalil) on 2018-10-23
tags: added: stx.distro.openstack
yong hu (yhu6) on 2018-10-28
Changed in starlingx:
assignee: Cindy Xie (xxie1) → yong hu (yhu6)
haitao wang (hwang85) on 2018-10-29
Changed in starlingx:
assignee: yong hu (yhu6) → haitao wang (hwang85)
yong hu (yhu6) on 2018-10-29
Changed in starlingx:
assignee: haitao wang (hwang85) → yong hu (yhu6)
yong hu (yhu6) wrote :

this cmd "openstack flavor set --property aggregate_instance_extra_specs=local_lvm m1.small" is not correct, and it should be "openstack flavor set --property aggregate_instance_extra_specs:storage=local_lvm m1.small"

even with correct cmd, such error messages were seen:

047700a00ef2424dae7d0e92170cdf03 e354a72f089241c5af98c52dccdefe62 - default default] AggregateInstanceExtraSpecsFilter: (compute-0) REJECT: extra_specs 'set([u'local_image'])' do not match 'local_lvm'

and accordingly the scheduling request was reject:

 'reject_map': {'compute-0': [u"(AggregateInstanceExtraSpecsFilter) extra_specs 'set([u'local_image'])' do not match 'local_lvm'"],
                'compute-ext': [u"(AggregateInstanceExtraSpecsFilter) extra_specs 'set([u'local_image'])' do not match 'local_lvm'"]

according to "nova/scheduler/filters/aggregate_instance_extra_specs.py",

 metadata = utils.aggregate_metadata_get_by_host(host_state)
...
aggregate_vals = metadata.get(key, None)

aggregate_vals only has "local_image" on host compute-0.

NEXT step is to figure out why.

yong hu (yhu6) wrote :

The root cause is figured out:
by default, compute node lvg (nova-local) is using COW-image. If we need to use "local_lvm" we need to config compute node first by following steps:
step 1: lock compute node
# system host-lock m-compute-0 //if any active VMs running on the compute node, we need to stop them first, otherwise compute-0 cannot be locked
step 2: change nova-local to use lvm
system host-lvg-modify -b lvm m-compute-0 nova-local
step3: unlock compute node

Once the compute node is unlocked successfully, we can use the following cmd to check lvg "nova-local" and will see "lvm" is one of parameters now:
[wrsroot@controller-0 ~(keystone_admin)]$ system host-lvg-show m-compute-0 nova-local
+-----------------+-----------------------------------------------------------------------------------------------+
| Property | Value |
+-----------------+-----------------------------------------------------------------------------------------------+
| lvm_vg_name | nova-local |
| vg_state | provisioned |
| uuid | 5f871e21-3edc-4519-a846-c250281bb40d |
| ihost_uuid | c056ddfa-e560-4fa3-a922-64790db72dff |
| lvm_vg_access | wz--n- |
| lvm_max_lv | 0 |
| lvm_cur_lv | 2 |
| lvm_max_pv | 0 |
| lvm_cur_pv | 1 |
| lvm_vg_size_gib | 197.65 |
| lvm_vg_total_pe | 50599 |
| lvm_vg_free_pe | 2425 |
| created_at | 2018-11-02T03:14:13.314837+00:00 |
| updated_at | 2018-11-02T03:53:42.362393+00:00 |
| parameters | {u'concurrent_disk_operations': 2, u'instance_backing': u'lvm', 'instances_lv_size_gib': 5.0}

by now, we can create a flavor with "local_lvm" and create VMs.
Special NOTE: we need to make this cmd right:
# openstack flavor set --property aggregate_instance_extra_specs:storage=local_lvm m1.small

yong hu (yhu6) wrote :

flavor with local_lvm

yong hu (yhu6) wrote :

VM with local_lvm

Changed in starlingx:
status: Triaged → Invalid
Ken Young (kenyis) on 2019-01-18
tags: added: stx.2019.05
removed: stx.2019.03
Ken Young (kenyis) on 2019-04-05
tags: added: stx.2.0
removed: stx.2019.05
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers