[RFE] Ironic functional tests and hacking checks

[John L. Villalovos]

For now, there is nothing in ironic/tests/functional/ dir. That means there is no functional tests for Ironic. I think it is time to add some for Ironic.

The functional tests could come from these perspectives below:

1. API tests: Also known as API sample tests. A test case in this part is
   going to send JSON based API requests to Ironic server, and gets a
   response and check if the response is correct as we expects.

2. DB tests: Tests for DB migrations, models and so on.

3. Driver tests: Driver providers could add functional tests for their
   specific driver in some specific scenarios.

4. Regression tests: Tests for some fixed bugs.

And also hacking checks:

1. Hacking checks: These could be some additional checks for pep8 tests.
   For example, if we want to assert if a value is None, we should use
   assertIsNone(xxx) instead of assertEqual(None, xxx). pep8 cannot test out
   things like this, and we can add hacking checks to enhance it.

Please also take a look at this REF below:
https://bugs.launchpad.net/ironic/+bug/1607679

Now, Ironic does its unit tests more like functional, which is totally
unnecessary. I think if we want to do functional tests, we have to split
the functional-like part of code from the unit tests first.

And also refer to ironic client functional tests:
https://bugs.launchpad.net/python-ironicclient/+bug/1491671

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/228612
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=f350d87af82ba8e33c19fb03eeb67625027838db
Submitter: Jenkins
Branch: master

commit f350d87af82ba8e33c19fb03eeb67625027838db
Author: John L. Villalovos <email address hidden>
Date: Mon Sep 28 11:36:37 2015 -0700

    Prepare for functional testing

    Prepare for functional testing by creating two new directories:
      * ironic/tests/unit
      * ironic/tests/functional

    Move all files currently in ironic/tests/* to ironic/tests/unit/

    Update imports from 'ironic.tests.YYY' to 'ironic.tests.unit.YYY'

    Fix any PEP8 issues due to longer lines

    Partial-Bug: #1491670
    Change-Id: I1166f20adf8c84b2042b62e348926502a4851e08

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/236152

[John L. Villalovos (happycamp)]

Gerrit topic:

https://review.openstack.org/#/q/status:open+project:openstack/ironic+branch:master+topic:ironic-functional,n,z

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/236152
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=3ab18a882bbf3999bcbfe160882bcef386d36448
Submitter: Jenkins
Branch: master

commit 3ab18a882bbf3999bcbfe160882bcef386d36448
Author: John L. Villalovos <email address hidden>
Date: Fri Oct 16 15:17:51 2015 -0700

    Enable cmd/api & cmd/conductor to be launched directly

    This will be needed for functional testing, so we can execute them with:
      python -m ironic.cmd.api
      python -m ironic.cmd.conductor

    Change-Id: I2f201ee1f6517879fec6da4e6f40a6155763d979
    Partial-Bug: #1491670

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/237773

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/237773
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=d1cb7180d5e89cdaa6e9f2b81fef0186947f37b1
Submitter: Jenkins
Branch: master

commit d1cb7180d5e89cdaa6e9f2b81fef0186947f37b1
Author: John L. Villalovos <email address hidden>
Date: Tue Oct 20 13:43:26 2015 -0700

    Move ironic.tests.unit.base to ironic.tests.base

    We will use the base.py file for both unit and functional testing, so
    move it under ironic/tests/base.py

    Change-Id: Ifc36d61be60c820dd8e5ee0f9b9dadb52d8b480a
    Partial-Bug: #1491670

[Tang Chen (tangchen)]

Hi guys,

The last update of this RFE was in March, and I don't see any related patches on going in the community. So I'm taking it over and continue to work. :)

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/348965

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/349421

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/350365

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/356188

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/356287

[OpenStack Infra (hudson-openstack)]

Change abandoned by Tang Chen (<email address hidden>) on branch: master
Review: https://review.openstack.org/348965
Reason: Merged to https://review.openstack.org/#/c/349421/