tempest: very unstable

[Isaku Yamahata]

Recently(starting around March 6, 2017) openstack CI is quite unstable.
In order to unblock pending patch, some test cases need to be disabled and fixed later.
e.g. https://review.openstack.org/#/c/444376/

test_server_connectivity_reboot
test_network_basic_ops
test_in_tenant_traffic
test_volume_boot_pattern

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/445023

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/445023
Committed: https://git.openstack.org/cgit/openstack/networking-odl/commit/?id=29268f6f7b22a27abf0f24f8183cfee8e5573527
Submitter: Jenkins
Branch: master

commit 29268f6f7b22a27abf0f24f8183cfee8e5573527
Author: Isaku Yamahata <email address hidden>
Date: Mon Mar 13 10:23:24 2017 -0700

    bug work around: disable several test cases

    From around March 13, 2017, CI is quite unstable which is causing
    many pending patches that isn't related to it.
    Disable several testcases to unblock long pending patches.
    Revert this patch as soon as possible.

    Change-Id: I3fdac32a851b9b8f9d1934deb3978a3062ece5af
    Related-bug: #1672462

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/ocata
Review: https://review.openstack.org/458164

[OpenStack Infra (hudson-openstack)]

Change abandoned by Manjeet Singh Bhatia (<email address hidden>) on branch: stable/ocata
Review: https://review.openstack.org/458164
Reason: this was a experiment cherry-pick from set of patches to test gate.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/443361
Committed: https://git.openstack.org/cgit/openstack/networking-odl/commit/?id=e98909116641d2bc70c94352fd862a837f23ce84
Submitter: Jenkins
Branch: stable/ocata

commit e98909116641d2bc70c94352fd862a837f23ce84
Author: Isaku Yamahata <email address hidden>
Date: Thu Feb 23 23:28:21 2017 -0500

    devstack: use localrc_set and use local.conf

    change id of I111a6df3adc07ed426ac714cb9e64667a63e0e5a
    changeset of 19561a4cb44fe3a20a0eab50ce91d83790d0333d devstack
    changed to ignore localrc when local.conf exists.
    create local.conf and use localrc_set instead of directly
    creating the file.

    tox: remove sitepackage=True

    remove sitepackage=True to set sitepackage=False by default.
    sitepackage=True causes pip install error with stable/ocata branch.
    appdirs 1.4.3 is installed on system, but stable/ocata requires appdirs 1.4.0.
    pip find appdirs 1.4.3 of global package, but pip fails to uninstall appdirs
    in venv. sitepackage=False resolves it.

    bug work around: disable several test cases

    From around March 13, 2017, CI is quite unstable which is causing
    many pending patches that isn't related to it.
    Disable several testcases to unblock long pending patches.
    since the public network connectivity is broken with netvirt,
    which is used in ODL. Backports are piling up, so disabling
    these to expedite backports to avoid other breakages.

    devstack: odl_snapshot_full_version misargument

    the third argument is taken wrongly as $1.
    It should be $3. And fix typo.

    Change-Id: Ib0a2ad2adc55845d890f8f005da526fa100dd536
    (cherry picked from commit cf71c3b6c0d1900fb6e57e74c36c633fe0fd2dc8)
    (cherry picked from commit I40b60f53b18b60594f9adcf1e280b2b32d35d123)
    (cherry picked from commit 29268f6f7b22a27abf0f24f8183cfee8e5573527)
    (cherry picked from commit a77f62ed93a6ef779bfefea27a5b2dab6f668ce1)
    Related-bug: #1672462

[Isaku Yamahata (yamahata)]

This is a bit old and description is too wide.
we should have concrete report per testcase failures.