Health checks leak floating-ips with Neutron Networking

Bug #1288547 reported by Brad Durrow on 2014-03-06
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
High
Tatyanka

Bug Description

Fuel 4.0 HA with Neutron VLAN Networking

{"build_id": "2013-12-27_00-24-14", "ostf_sha": "83ada35fec2664089e07fdc0d34861ae2a4d948a", "build_number": "214", "nailgun_sha": "af1598bcc9faf468d4d9265cc5c51fa8cea53136", "fuelmain_sha": "17eed776b30886851ae0042fa7a30184f5cd8eb6", "astute_sha": "6ce36837882399e0d3bb1ffdb2c3b2d8dcb84b54", "release": "4.0", "fuellib_sha": "eebe07913ee09311c8e7c9231f6785081327dc0e"}

My health checks related to Floating IPs have started failing, here is the debugging I have done so far if anyone feels generous enough to help: http://paste.openstack.org/show/72743/

Looks like I found a health check bug. If I neutron floatingip-delete one of the floating ips; then run the functional health checks a few things happen...

 1) All of the health checks except "Check network connectivity from instance via floating IP" pass. Previously "Check that VM is accessible via floating IP address" would also fail.

 2) The floatingip I deleted is back, but with a new id

 3) Subsequent health checks show both of the checks involving floating IPs fail.

Vladimir Kuklin (vkuklin) wrote :

Brad, could you please post a diagnostic snapshot as it contains OSTF logs.

Changed in fuel:
assignee: nobody → Fuel QA Team (fuel-qa)
milestone: none → 4.1.1
Changed in fuel:
importance: Undecided → High
status: New → Confirmed
Brad Durrow (l-brad) wrote :

it was faster for me to patch the problem than to submit logs. If you want logs it is pretty easy to replicate the problem just make a neutron deployment that only has a n floating IPs and run the health checks n/2+1 times.

Fix proposed to branch: master
Review: https://review.openstack.org/79591

Changed in fuel:
assignee: Fuel QA Team (fuel-qa) → Tatyana (tatyana-leontovich)
status: Confirmed → In Progress

Reviewed: https://review.openstack.org/79591
Committed: https://git.openstack.org/cgit/stackforge/fuel-ostf/commit/?id=14e57ba5474442d00662b3455099cb21a6afa54f
Submitter: Jenkins
Branch: master

commit 14e57ba5474442d00662b3455099cb21a6afa54f
Author: Tatyana Leontovich <email address hidden>
Date: Tue Mar 11 14:23:20 2014 +0000

    Add deletion of floating ip after disassociation

    First problem - we have 2 very simmilar tests
    that differs only by 1 step. In this patch
    this 2 tests merge into 1.

    Second one - during the tests we do not release
    floating ip, we just disassociate it,
    and release only in teardown(
    when all tests from suite were executed).To fix this issue
    add floating ip deleteion as a separate step

    Change-Id: I5cc0a782cdc5914e250ae5176aa8a724b797a934
    Closes-bug: #1288547

Changed in fuel:
status: In Progress → Fix Committed
Andrew Woodward (xarses) on 2014-03-12
tags: added: backports-4.1.1
tags: added: in progress

Verified on iso
{"build_id": "2014-03-12_13-26-57", "mirantis": "yes", "build_number": "25", "nailgun_sha": "9c02b6aab96814a2311dbb7a7014a1901cddb8f9", "ostf_sha": "41b5c491b11bfad194e6bb17e3fec1ab5fa5a624", "fuelmain_sha": "b4fb821cceeb100241a99b36ff7a5523b19093a7", "astute_sha": "9f0a2a3135ace2e8d9bf7a4c5f661b8326ace285", "release": "5.0", "fuellib_sha": "48ced96753378e49883cdade8957160ef1b29899"}

Changed in fuel:
status: Fix Committed → Fix Released
tags: removed: in progress
Tatyanka (tatyana-leontovich) wrote :

set in progress status according fix should be back-ported in 4.1.1

Changed in fuel:
status: Fix Released → In Progress

Reviewed: https://review.openstack.org/83088
Committed: https://git.openstack.org/cgit/stackforge/fuel-ostf/commit/?id=cb71838f1cc572b3223e01a0c24bc694b0b72537
Submitter: Jenkins
Branch: stable/4.1

commit cb71838f1cc572b3223e01a0c24bc694b0b72537
Author: Tatyana Leontovich <email address hidden>
Date: Tue Mar 11 14:23:20 2014 +0000

    4.1.1 Add deletion of floating ip

    First problem - we have 2 very simmilar tests
    that differs only by 1 step. In this patch
    this 2 tests merge into 1.

    Second one - during the tests we do not release
    floating ip, we just disassociate it,
    and release only in teardown(
    when all tests from suite were executed).To fix this issue
    add floating ip deleteion as a separate step

    Change-Id: I5cc0a782cdc5914e250ae5176aa8a724b797a934
    Closes-bug: #1288547

Changed in fuel:
status: In Progress → Fix Committed
tags: added: in progress

Verified on iso 4.1 265

Changed in fuel:
status: Fix Committed → Fix Released
Andrew Woodward (xarses) on 2014-04-04
tags: removed: in progress
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers