API unittests coverage is unsufficient

[Sylvain Bauza]

Hi,

I just discovered when trying to add unittests for Host API that the coverage is not sufficient for all the controllers.

For example, this section :
    def test_make_app(self):
        self.app.make_app()
        self.fake_ff.assert_called_once_with(self.fake_app().config,
                                             admin_user='admin',
                                             admin_tenant_name='admin',
                                             auth_port='35357',
                                             auth_protocol='http',
                                             auth_version='v2.0',
                                             admin_password='climate',
                                             auth_host='127.0.0.1')

only checks that the remote method is called, but doesn't check the values, which can lead to a successful test while the method could raise an Exception.

Let me give a quick example on why only checking assert_called is not enough.
Provided this code snippet :
import A
def foo()
    myA = A()
    mydict = myA.to_dict()
    return mydict['id']

and provided this unittest method :
def test_foo(self):
    fake_Adict = self.patch(A,'to_dict')
    self.fake_Adict.assert_called_once()

it will work.
If we change foo() to return mydict['name'] instead of mydict['id'], the test will still say OK while it will break at runtime...

The coverage would be better saying :
fake_Adict.return_value = {'id': 'bar'}
self.assertEqual('bar', test_foo())

[Dina Belova (dbelova)]

Of course, that's a good idea only in case like 'something interesting is going on in this method, but we check only its calling'. We do not need to test every simple and doing nothing thing.

[Sylvain Bauza (sylvain-bauza)]

Dina : Agreed, there is need for testing values only if the tested method is *doing things* and not only wrapping an other classmethod

[Masahito Muroi (muroi-masahito)]

The API tests is resloved by gabbit. See the discussion in https://etherpad.openstack.org/p/blazar-ptg-rocky