IPA does not have functional testing

[Josh Gachnang]

IPA needs to finish support for functional testing independent of Ironic. There's some work to enable this, but coverage is pretty lacking. The entire API should be testable from a laptop, without standing up Ironic.

Use Nova's work as an example on how to proceed.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/224247

[Michael Davies (mrda)]

Mario - did you want to add some coverage metrics here to see the impact your patch has?

[Mario Villaplana (mario-villaplana-j)]

For some reason, tox -ecover isn't working:

```
# tox -ecover
cover create: /root/ironic-python-agent/.tox/cover
cover installdeps: -r/root/ironic-python-agent/requirements.txt, -r/root/ironic-python-agent/test-requirements.txt
cover develop-inst: /root/ironic-python-agent
cover installed: appdirs==1.4.0,Babel==2.1.1,beautifulsoup4==4.4.0,coverage==4.0,debtcollector==0.8.0,discover==0.4.0,docutils==0.12,eventlet==0.17.4,extras==0.0.3,fasteners==0.13.0,fixtures==1.3.1,flake8==2.2.4,funcsigs==0.4,greenlet==0.4.9,hacking==0.10.2,-e <email address hidden>:openstack/ironic-python-agent.git@47fccbc0f75562c8f02fd9fee584ccdc4082e715#egg=ironic_python_agent-review_mario_villaplana_bug_1492036,iso8601==0.1.10,Jinja2==2.8,keystoneauth1==1.1.0,linecache2==1.0.0,logutils==0.3.3,Mako==1.0.2,MarkupSafe==0.23,mccabe==0.2.1,mock==1.3.0,monotonic==0.3,mox3==0.10.0,msgpack-python==0.4.6,netaddr==0.7.18,netifaces==0.10.4,os-client-config==1.7.5,oslo.concurrency==2.6.0,oslo.config==2.4.0,oslo.context==0.6.0,oslo.i18n==2.6.0,oslo.log==1.11.0,oslo.serialization==1.9.0,oslo.service==0.9.0,oslo.utils==2.5.0,oslosphinx==3.2.0,oslotest==1.11.0,Paste==2.0.2,PasteDeploy==1.5.2,pbr==1.8.0,pecan==1.0.2,pep8==1.5.7,Pint==0.6,psutil==1.2.1,pyflakes==0.8.1,Pygments==2.0.2,python-mimeparse==0.1.4,python-subunit==1.1.0,pytz==2015.6,pyudev==0.17,PyYAML==3.11,repoze.lru==0.6,requests==2.7.0,retrying==1.3.3,Routes==2.2,simplegeneric==0.8.1,singledispatch==3.4.0.3,six==1.9.0,Sphinx==1.2.3,sphinxcontrib-httpdomain==1.4.0,sphinxcontrib-pecanwsme==0.8.0,stevedore==1.8.0,testrepository==0.0.20,testscenarios==0.5.0,testtools==1.8.0,traceback2==1.4.0,unittest2==1.1.0,waitress==0.8.10,WebOb==1.4.1,WebTest==2.0.18,wheel==0.24.0,wrapt==1.10.5,WSME==0.8.0
cover runtests: PYTHONHASHSEED='3062610904'
cover runtests: commands[0] | python setup.py testr --coverage ironic_python_agent
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
   or: setup.py --help [cmd1 cmd2 ...]
   or: setup.py --help-commands
   or: setup.py cmd --help

error: invalid command 'ironic_python_agent'
ERROR: InvocationError: '/root/ironic-python-agent/.tox/cover/bin/python setup.py testr --coverage ironic_python_agent'
```

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/224247
Committed: https://git.openstack.org/cgit/openstack/ironic-python-agent/commit/?id=4a7b954d148461128f57bd148d85fa8a6b9c0dfd
Submitter: Jenkins
Branch: master

commit 4a7b954d148461128f57bd148d85fa8a6b9c0dfd
Author: Mario Villaplana <email address hidden>
Date: Tue Sep 15 21:28:34 2015 +0000

    Adds more functional tests for commands

    This adds additional functional tests for the commands API.
    It also restructures the current functional test to run in
    sequence as a monolithic test to preserve IPA state and test
    order between test runs. The time to wait for IPA to start
    was increased due to an intermittent race condition that
    occurs with the larger test suite.

    Partial-Bug: 1492036
    Change-Id: Iff9b41fb531d34225d702b9bfd39826e3c7195ad

[Dmitry Tantsur (divius)]

Now that we have functional tests in place, let's file separate bugs for extending coverage.