vmware_nsx_tempest tempest plugin issues with tempest

[chandan kumar]

Upstream Tempest-plugin-sanity gate http://logs.openstack.org/76/457876/12/check/gate-tempest-plugin-sanity-ubuntu-xenial-nv/7be093e/console.html failed due to
 + /home/jenkins/workspace/gate-tempest-plugin-sanity-ubuntu-xenial-nv/tools/with_venv.sh tempest init /home/jenkins/workspace/gate-tempest-plugin-sanity-ubuntu-xenial-nv/tempest_sanity
2017-05-16 06:36:39.172355 | 2017-05-16 06:36:39.172 5501 INFO tempest [-] Using tempest config file /etc/tempest/tempest.conf
2017-05-16 06:36:39.238896 | 2017-05-16 06:36:39.238 5501 ERROR tempest.test_discover.plugins [-] Could not load 'vmware-nsx-tempest-plugin': No module named vmware_nsx_tempest.plugin
2017-05-16 06:36:39.239060 | 2017-05-16 06:36:39.238 5501 ERROR tempest [-] No module named vmware_nsx_tempest.plugin

As tempest fails to discover the vmware-nsx-tempest-plugin because on doing pip install the entry_point path defined for tempest.test_plugins is not found as it is not included under packages in setup.cfg.

After fixing the above issue, we encountered other issues:
* pbr.version.VersionInfo issue failed to execute to tempest cli.

(.venv) [chkumar246@assassin vmware-nsx]$ cd ..
(.venv) [chkumar246@assassin tmp]$ mkdir foobar
(.venv) [chkumar246@assassin tmp]$ cd foobar/
(.venv) [chkumar246@assassin foobar]$ tempest init
2017-05-16 15:23:33.145 13780 INFO tempest [-] Using tempest config file /etc/tempest/tempest.conf
2017-05-16 15:23:33.223 13780 ERROR tempest.test_discover.plugins [-] Could not load 'vmware-nsx-tempest-plugin': Versioning for this project requires either an sdist tarball, or access to an upstream git repository. It's also possible that there is a mismatch between the package name in setup.cfg and the argument given to pbr.version.VersionInfo. Project name vmware_nsx_tempest was given, but was not able to be found.
2017-05-16 15:23:33.223 13780 ERROR tempest [-] Versioning for this project requires either an sdist tarball, or access to an upstream git repository. It's also possible that there is a mismatch between the package name in setup.cfg and the argument given to pbr.version.VersionInfo. Project name vmware_nsx_tempest was given, but was not able to be found.

Removing version.info from the vmware_nsx_tempest/__init_.py fixes the issue.

* On doing tempest init, It started throwing TypeError: 'OptGroup' object is not iterable as in vmware_nsx_tempest/plugin.py under get_opt_lists function, we have not listed all the options and sections defined in config.py.

(.venv) [chkumar246@assassin playground]$ tempest list-plugins
+---------------------------+--------------------------------------------------+
| Name | EntryPoint |
+---------------------------+--------------------------------------------------+
| vmware-nsx-tempest-plugin | vmware_nsx_tempest.plugin:VMwareNsxTempestPlugin |
+---------------------------+--------------------------------------------------+
(.venv) [chkumar246@assassin playground]$ tempest init foobar
2017-05-16 17:36:52.733 19997 INFO tempest [-] Using tempest config file /etc/tempest/tempest.conf
2017-05-16 17:36:52.745 19997 ERROR tempest [-] 'OptGroup' object is not iterable
(.venv) [chkumar246@assassin playground]$ rm -fr foobar/ ~/.tempest/
(.venv) [chkumar246@assassin playground]$ tempest init foobar --debug
2017-05-16 17:37:14.677 20019 INFO tempest [-] Using tempest config file /etc/tempest/tempest.conf
2017-05-16 17:37:14.689 20019 ERROR tempest [-] 'OptGroup' object is not iterable
2017-05-16 17:37:14.689 20019 ERROR tempest Traceback (most recent call last):
2017-05-16 17:37:14.689 20019 ERROR tempest File "/home/chkumar246/playground/.venv/lib/python2.7/site-packages/cliff/app.py", line 400, in run_subcommand
2017-05-16 17:37:14.689 20019 ERROR tempest result = cmd.run(parsed_args)
2017-05-16 17:37:14.689 20019 ERROR tempest File "/home/chkumar246/playground/.venv/lib/python2.7/site-packages/cliff/command.py", line 97, in run
2017-05-16 17:37:14.689 20019 ERROR tempest return self.take_action(parsed_args) or 0
2017-05-16 17:37:14.689 20019 ERROR tempest File "/home/chkumar246/playground/.venv/lib/python2.7/site-packages/tempest/cmd/init.py", line 181, in take_action
2017-05-16 17:37:14.689 20019 ERROR tempest self.create_working_dir(parsed_args.dir, config_dir)
2017-05-16 17:37:14.689 20019 ERROR tempest File "/home/chkumar246/playground/.venv/lib/python2.7/site-packages/tempest/cmd/init.py", line 163, in create_working_dir
2017-05-16 17:37:14.689 20019 ERROR tempest self.generate_sample_config(local_dir)
2017-05-16 17:37:14.689 20019 ERROR tempest File "/home/chkumar246/playground/.venv/lib/python2.7/site-packages/tempest/cmd/init.py", line 131, in generate_sample_config
2017-05-16 17:37:14.689 20019 ERROR tempest output_file])
2017-05-16 17:37:14.689 20019 ERROR tempest File "/home/chkumar246/playground/.venv/lib/python2.7/site-packages/oslo_config/generator.py", line 530, in main
2017-05-16 17:37:14.689 20019 ERROR tempest generate(conf)
2017-05-16 17:37:14.689 20019 ERROR tempest File "/home/chkumar246/playground/.venv/lib/python2.7/site-packages/oslo_config/generator.py", line 510, in generate
2017-05-16 17:37:14.689 20019 ERROR tempest groups = _get_groups(_list_opts(conf.namespace))
2017-05-16 17:37:14.689 20019 ERROR tempest File "/home/chkumar246/playground/.venv/lib/python2.7/site-packages/oslo_config/generator.py", line 436, in _list_opts
2017-05-16 17:37:14.689 20019 ERROR tempest return _cleanup_opts(opts)
2017-05-16 17:37:14.689 20019 ERROR tempest File "/home/chkumar246/playground/.venv/lib/python2.7/site-packages/oslo_config/generator.py", line 373, in _cleanup_opts
2017-05-16 17:37:14.689 20019 ERROR tempest for opt in opts:
2017-05-16 17:37:14.689 20019 ERROR tempest TypeError: 'OptGroup' object is not iterable
2017-05-16 17:37:14.689 20019 ERROR tempest
2017-05-16 17:37:14.690 20019 CRITICAL tempest [-] TypeError: 'OptGroup' object is not iterable
2017-05-16 17:37:14.690 20019 ERROR tempest Traceback (most recent call last):
2017-05-16 17:37:14.690 20019 ERROR tempest File "/home/chkumar246/playground/.venv/bin/tempest", line 11, in <module>
2017-05-16 17:37:14.690 20019 ERROR tempest sys.exit(main())
2017-05-16 17:37:14.690 20019 ERROR tempest File "/home/chkumar246/playground/.venv/lib/python2.7/site-packages/tempest/cmd/main.py", line 48, in main
2017-05-16 17:37:14.690 20019 ERROR tempest return the_app.run(argv)
2017-05-16 17:37:14.690 20019 ERROR tempest File "/home/chkumar246/playground/.venv/lib/python2.7/site-packages/cliff/app.py", line 279, in run
2017-05-16 17:37:14.690 20019 ERROR tempest result = self.run_subcommand(remainder)
2017-05-16 17:37:14.690 20019 ERROR tempest File "/home/chkumar246/playground/.venv/lib/python2.7/site-packages/cliff/app.py", line 400, in run_subcommand
2017-05-16 17:37:14.690 20019 ERROR tempest result = cmd.run(parsed_args)
2017-05-16 17:37:14.690 20019 ERROR tempest File "/home/chkumar246/playground/.venv/lib/python2.7/site-packages/cliff/command.py", line 97, in run
2017-05-16 17:37:14.690 20019 ERROR tempest return self.take_action(parsed_args) or 0
2017-05-16 17:37:14.690 20019 ERROR tempest File "/home/chkumar246/playground/.venv/lib/python2.7/site-packages/tempest/cmd/init.py", line 181, in take_action
2017-05-16 17:37:14.690 20019 ERROR tempest self.create_working_dir(parsed_args.dir, config_dir)
2017-05-16 17:37:14.690 20019 ERROR tempest File "/home/chkumar246/playground/.venv/lib/python2.7/site-packages/tempest/cmd/init.py", line 163, in create_working_dir
2017-05-16 17:37:14.690 20019 ERROR tempest self.generate_sample_config(local_dir)
2017-05-16 17:37:14.690 20019 ERROR tempest File "/home/chkumar246/playground/.venv/lib/python2.7/site-packages/tempest/cmd/init.py", line 131, in generate_sample_config
2017-05-16 17:37:14.690 20019 ERROR tempest output_file])
2017-05-16 17:37:14.690 20019 ERROR tempest File "/home/chkumar246/playground/.venv/lib/python2.7/site-packages/oslo_config/generator.py", line 530, in main
2017-05-16 17:37:14.690 20019 ERROR tempest generate(conf)
2017-05-16 17:37:14.690 20019 ERROR tempest File "/home/chkumar246/playground/.venv/lib/python2.7/site-packages/oslo_config/generator.py", line 510, in generate
2017-05-16 17:37:14.690 20019 ERROR tempest groups = _get_groups(_list_opts(conf.namespace))
2017-05-16 17:37:14.690 20019 ERROR tempest File "/home/chkumar246/playground/.venv/lib/python2.7/site-packages/oslo_config/generator.py", line 436, in _list_opts
2017-05-16 17:37:14.690 20019 ERROR tempest return _cleanup_opts(opts)
2017-05-16 17:37:14.690 20019 ERROR tempest File "/home/chkumar246/playground/.venv/lib/python2.7/site-packages/oslo_config/generator.py", line 373, in _cleanup_opts
2017-05-16 17:37:14.690 20019 ERROR tempest for opt in opts:
2017-05-16 17:37:14.690 20019 ERROR tempest TypeError: 'OptGroup' object is not iterable

I am proposing the patch to fix all the above issues.

[chandan kumar (chkumar246)]

Few more issue on doing ostestr -l
sion 'Pike' and will be removed in a future version
  @test.attr(type='common')
--- import errors ---
Failed to import test module: vmware_nsx_tempest.tests.scenario.test_qos_ops
Traceback (most recent call last):
  File "/tmp/.venv/lib/python2.7/site-packages/unittest2/loader.py", line 456, in _find_test_path
    module = self._get_module_from_name(name)
  File "/tmp/.venv/lib/python2.7/site-packages/unittest2/loader.py", line 395, in _get_module_from_name
    __import__(name)
  File "/tmp/.venv/lib/python2.7/site-packages/vmware_nsx_tempest/tests/scenario/test_qos_ops.py", line 25, in <module>
    import pyshark
ImportError: No module named pyshark
Non-zero exit code (2) from test listing

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/465084
Committed: https://git.openstack.org/cgit/openstack/vmware-nsx/commit/?id=eb293bd418918fb43837b5d533dd062ba188ba1f
Submitter: Jenkins
Branch: master

commit eb293bd418918fb43837b5d533dd062ba188ba1f
Author: Chandan Kumar <email address hidden>
Date: Tue May 16 20:01:53 2017 +0530

    Fixes vmware_nsx_tempest tempest plugin issues with tempest

    * Added vmware_nsx_tempest under packages in setup.cfg so that
      tempest can discovers it.
    * Removed pbr version from vmware_nsx_tempest (in-tree tempest plugin)
      as it causes versioning issue with the main package vmware_nsx.
    * Added all sections and options defined in tempest plugin conf
      so that tempest can retrive all the tempest sample configurations.

    Depends-on: Iab0202a28bfa525c4cd91e776ac2bdba56a807f6

    Change-Id: I2f706b8cdb31c53d951b059f939fb0d6afc32958
    Closes-Bug: #1691122