Import of tempest.test has side-effect that config is parsed

Bug #1592345 reported by Jakub Libosvar
This bug affects 2 people
Affects Status Importance Assigned to Milestone

Bug Description

During implementation of tempest scenario for qos, it revealed that calling

 from tempest import test

has a side-effect. It replaces oslo config object with a proxy object. Calling __getattr__ on this object has a side-effect of parsing the config which makes config object no longer capable of registering new CLI opts.

In [1]: from oslo_config import cfg

In [2]: from tempest import test
2016-06-14 10:21:46.063 7637 INFO tempest [-] Using tempest config file /etc/tempest/tempest.conf
In [3]: cfg.CONF.register_cli_opts([cfg.StrOpt('foo')])
ArgsAlreadyParsedError Traceback (most recent call last)
<ipython-input-3-a1153935307d> in <module>()
----> 1 cfg.CONF.register_cli_opts([cfg.StrOpt('foo')])

/usr/lib/python2.7/site-packages/oslo_config/cfg.pyc in __inner(self, *args, **kwargs)
   2106 def __inner(self, *args, **kwargs):
   2107 if kwargs.pop('clear_cache', True):
-> 2108 result = f(self, *args, **kwargs)
   2109 self.__cache.clear()
   2110 return result

/usr/lib/python2.7/site-packages/oslo_config/cfg.pyc in register_cli_opts(self, opts, group)
   2290 """Register multiple CLI option schemas at once."""
   2291 for opt in opts:
-> 2292 self.register_cli_opt(opt, group, clear_cache=False)
   2294 def register_group(self, group):

/usr/lib/python2.7/site-packages/oslo_config/cfg.pyc in __inner(self, *args, **kwargs)
   2110 return result
   2111 else:
-> 2112 return f(self, *args, **kwargs)
   2114 return __inner

/usr/lib/python2.7/site-packages/oslo_config/cfg.pyc in register_cli_opt(self, opt, group)
   2282 """
   2283 if self._args is not None:
-> 2284 raise ArgsAlreadyParsedError("cannot register CLI option")
   2286 return self.register_opt(opt, group, cli=True, clear_cache=False)

ArgsAlreadyParsedError: arguments already parsed: cannot register CLI option

Itzik Brown (itzikb1)
Changed in tempest:
status: New → Confirmed
Changed in tempest:
assignee: nobody → Daniel Mellado (daniel-mellado)
Changed in tempest:
assignee: Daniel Mellado (daniel-mellado) → chandan kumar (chkumar246)
Revision history for this message
chandan kumar (chkumar246) wrote :

@jakub, I have tried the above steps, found the same traceback but i am not able to get where it is replacing the oslo config object. Please add some more information so that i can get into the issue?

Revision history for this message
Jakub Libosvar (libosvar) wrote :

I went through tempest code and this is what I meant -

It overrides __getattr__ which means first call to config object parses CLI args by calling

Revision history for this message
Jordan Pittier (jordan-pittier) wrote :

So the idea is that plugins shouldn"t import tempest.test. Because tempest.test is not a stable interface, we don"t make any guarantee on its stability. The way forward is to pick/move the parts of tempest.test you are interested in, and put them somewhere in tempest/lib.

Revision history for this message
Jakub Libosvar (libosvar) wrote :

@Jordan: Makes sense, I'm adding Neutron as affected project then as it seems we're doing wrong imports.

Revision history for this message
Masayuki Igawa (igawa) wrote :

yeah, I agree with Jordan. But I found that many projects do it..


Revision history for this message
Jakub Libosvar (libosvar) wrote :

For Neutron in particular the test module is mostly using decorators, so I think switch to decorators module from tempest-lib won't be a problem. We also do imports of other stuff from tempest instead of tempest-lib, I'll need to look deeper at this issue. I'm glad we got some movement here :) Thanks for help guys.

Revision history for this message
Ghanshyam Mann (ghanshyammann) wrote :

Yea, tempest.test is not stable interface but we do have lot of base framework which is needed by plugin and almost all plugin import tempest.test.

And all plugins register their config values and use them.

I did not get ArgsAlreadyParsedError if config options are not being already registered. Can you please points us to the patch where you are getting this error.

If config options are different then i do not think any problem even importing tempest.test

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on neutron (master)

Change abandoned by Kevin Benton (<email address hidden>) on branch: master
Reason: This review is > 4 weeks without comment, and failed Jenkins the last time it was checked. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and leaving a 'recheck' comment to get fresh test results.

Martin Kopec (mkopec)
Changed in tempest:
assignee: chandan kumar (chkumar246) → nobody
Revision history for this message
Lukas Piwowarski (lukas-piwowarski) wrote :

I believe this is no longer a valid bug. I've manually tested the import and it passed without any error. Also as pointed out by Masayuki many tempest plugins use this import without any issue [1]. Therefore I'm moving this bug to "Invalid". Feel free to change the status if you think this decision is not correct.


Changed in tempest:
status: Confirmed → Invalid
Changed in neutron:
status: New → Invalid
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.