nova.service does not handle attribute specific exceptions and client hangs

[Sateesh]

when we try to allocate a floating ip to tenant using dashboard or novaclient cli, it just hangs, doesn't return.
The failure is because the network driver is FlatManager which doesn't support allocation of floating ips.

nova-api has casted the message to network service correctly. But while returning the method in __getattr__() of nova.server.Service, it fails due to AttributeError as there is no such method as allocate_floating_ip in FlatManager. As a result the network service doesn't return anything and nova-api doesn't get any response and user sees his nova command or dashboard hung.

Apart from this specific case, it might happen that some drivers doesn't support some functionality. Service need to ensure return atleast a method that returns "NotImplementedError()"

[John Garbutt (johngarbutt)]

It looks like this now makes horizon fail to open the Access & Security tab, when you are using Flat networking.

A possible fix looks to be adding extra stubs into the Network manager base class, something like this:

@@ -557,6 +557,21 @@ class NetworkManager(manager.SchedulerDependentManager):
         self.compute_api.trigger_security_group_members_refresh(admin_context,
                                                                     group_ids)

+ def get_floating_ip(self, context, id):
+ # NOTE(johngarbutt) This is just a stub function. Managers supporting
+ # floating ips MUST override this or use the Mixin
+ return []
+
+ def get_floating_ip_by_address(self, context, address):
+ # NOTE(johngarbutt) This is just a stub function. Managers supporting
+ # floating ips MUST override this or use the Mixin
+ return []
+
+ def get_floating_ips_by_project(self, context):
+ # NOTE(johngarbutt) This is just a stub function. Managers supporting
+ # floating ips MUST override this or use the Mixin
+ return []
+
     def get_floating_ips_by_fixed_address(self, context, fixed_address):
         # NOTE(jkoelker) This is just a stub function. Managers supporting
         # floating ips MUST override this or use the Mixin

[Armando Migliaccio (armando-migliaccio)]

the underlying problem is to do with:

def _process_data(self, ctxt, method, args):
        """Thread that magically looks for a method on the proxy object and calls it."""

        node_func = getattr(self.proxy, str(method))
        node_args = dict((str(k), v) for k, v in args.iteritems())
        # NOTE(vish): magic is fun!
        try:
            rval = node_func(context=ctxt, **node_args)

in nova/rpc/impl_kombu.py. The getattr call should be in the try catch.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/2848

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/2871

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/2877

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/2877
Committed: http://github.com/openstack/nova/commit/421e5cb2b4a3f1b5a8db58543b9cd96df62e9599
Submitter: Jenkins
Branch: master

commit 421e5cb2b4a3f1b5a8db58543b9cd96df62e9599
Author: Armando Migliaccio <email address hidden>
Date: Fri Jan 6 18:40:49 2012 +0000

    Bug #894683: nova.service does not handle attribute specific exceptions and client hangs

    As Sateesh points out nova.service.Service.__getattr__ throws an AttributeError exception when the upcall method is not available.

    However nova.rpc should catch all exceptions. This does not happen in this specific case because the following statement was outside the try-catch-all:

    node_func = getattr(self.proxy, str(method))

    Change-Id: I437c88783bca037e4054078d2687ef41c8fc2b83

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/diablo
Review: https://review.openstack.org/3081

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/3081
Committed: http://github.com/openstack/nova/commit/3bcd2721765fd2d74c94095c6abf6d5cc6f9f32e
Submitter: Jenkins
Branch: stable/diablo

commit 3bcd2721765fd2d74c94095c6abf6d5cc6f9f32e
Author: Armando Migliaccio <email address hidden>
Date: Fri Jan 6 18:40:49 2012 +0000

    Bug #894683: nova.service does not handle attribute specific exceptions and client hangs

    As Sateesh points out nova.service.Service.__getattr__ throws an AttributeError exception when the upcall method is not available.

    However nova.rpc should catch all exceptions. This does not happen in this specific case because the following statement was outside the try-catch-all:

    node_func = getattr(self.proxy, str(method))

    (cherry picked from commit 421e5cb2b4a3f1b5a8db58543b9cd96df62e9599)

    Change-Id: I437c88783bca037e4054078d2687ef41c8fc2b83

[Matthew Piatkowski (progone)]

Zcz/ iv:-)

Matt Piatkowski email from Xoom Tablet
On Jan 25, 2012 5:13 AM, "Thierry Carrez" <email address hidden>
wrote:

> ** Changed in: nova
> Status: Fix Committed => Fix Released
>
> ** Changed in: nova
> Milestone: None => essex-3
>
> --
> You received this bug notification because you are subscribed to
> OpenStack.
> Matching subscriptions: my_openStack_bug_report
> https://bugs.launchpad.net/bugs/894683
>
> Title:
> nova.service does not handle attribute specific exceptions and client
> hangs
>
> Status in OpenStack Compute (Nova):
> Fix Released
> Status in OpenStack Compute (nova) diablo series:
> Fix Released
>
> Bug description:
> when we try to allocate a floating ip to tenant using dashboard or
> novaclient cli, it just hangs, doesn't return.
> The failure is because the network driver is FlatManager which doesn't
> support allocation of floating ips.
>
> nova-api has casted the message to network service correctly. But
> while returning the method in __getattr__() of nova.server.Service, it
> fails due to AttributeError as there is no such method as
> allocate_floating_ip in FlatManager. As a result the network service
> doesn't return anything and nova-api doesn't get any response and user
> sees his nova command or dashboard hung.
>
> Apart from this specific case, it might happen that some drivers
> doesn't support some functionality. Service need to ensure return
> atleast a method that returns "NotImplementedError()"
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/nova/+bug/894683/+subscriptions
>