VM instance building forever when an RPC error occurs

Bug #1799949 reported by Wallace Cardoso
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Confirmed
Wishlist
Unassigned

Bug Description

Description
===============
For the cases where an error occur inside the box of a component (nova-conductor, nova-compute, nova-scheduler), the system seems to handle the error putting the VM instance in the error state. But, earlier errors in the exchange between each component are not well handled as inside. I found that when an error occurs in the RPC server libs, for instance, when deserializing an object, the flow of creating the VM is not changed and stay forever building.

Steps to reproduce
=====================
1) Create a valid flavor (e.g. RAM=64mb, DISK=0, VCPUS=1);
2) Create a valid image (e.g. cirros);
3) Create the VM (i.e. the server) from the client of your preference with cells enabled;
4) Intercept the RPC call from nova-api to nova-conductor for the method 'schedule_and_build_instances';
5) Change the 'id' of "schedule_and_build_instances.args.build_requests->'nova_object.data'.id" parameter by a non-parseable value (e.g. None, '');
6) Waits for some seconds until the errors propagate on the system (i.e. possible to see in the logs of nova-conductor);

Expected result
==================
Based on the behavior of errors occurring in other circunstances, the VM state should be set to 'error', not building forever once an error was faced between nova-api and nova-conductor;

Actual result
================
VM state is 'build' forever

Environment
==============
I used the devstack to initialize all OpenStack components. This considers a basic installation in a single node. The version I have used is the Queens.

Logs & Configs
=================
Logs attached to this BugReport.

Revision history for this message
Wallace Cardoso (wallacec) wrote :
Revision history for this message
Wallace Cardoso (wallacec) wrote :
summary: - VM instance building forever when a RPC error occurs
+ VM instance building forever when an RPC error occurs
Matt Riedemann (mriedem)
tags: added: fault-injection
Revision history for this message
Balazs Gibizer (balazs-gibizer) wrote :

I don't think this is a bug. It is an effect of a bug in the RPC lib. I think the good place to fix such bug is in the RPC lib itself. I consider this as a possible improvement on nova side hence the Whislist priority.

Changed in nova:
status: New → Confirmed
importance: Undecided → Wishlist
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.