test_GET_with_duplicate_but_sufficient_frag_indexes occasionally fails

Bug #1710951 reported by Tim Burke
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Object Storage (swift)
Fix Released
Medium
Alistair Coles

Bug Description

Happens ~0.2% of the time; failure looks like

======================================================================
FAIL: test_GET_with_duplicate_but_sufficient_frag_indexes (test.unit.proxy.controllers.test_obj.TestECDuplicationObjController)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/vagrant/swift/test/unit/proxy/controllers/test_obj.py", line 4241, in test_GET_with_duplicate_but_sufficient_frag_indexes
    len(log), self.policy.ec_n_unique_fragments + 1)
AssertionError: 13 not greater than or equal to 15
-------------------- >> begin captured stdout << ---------------------
proxy-server DEBUG: Loaded override config for (default): ProxyOverrideOptions({}, {'sorting_method': 'shuffle', 'read_affinity': '', 'write_affinity': '', 'write_affinity_node_count': '2 * replicas', 'write_affinity_handoff_delete_count': None}) (txn: txn1) (client_ip: 127.0.0.2)

--------------------- >> end captured stdout << ----------------------
    """Fail immediately, with the given message."""
>> raise self.failureException('13 not greater than or equal to 15')

----------------------------------------------------------------------

Tags: flakey-test
Changed in swift:
status: New → Confirmed
importance: Undecided → Medium
assignee: nobody → Alistair Coles (alistair-coles)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to swift (master)

Fix proposed to branch: master
Review: https://review.openstack.org/494172

Changed in swift:
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to swift (master)

Reviewed: https://review.openstack.org/494172
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=88252cdfd6c1e446f4d1cde6cc07a9c67b701e65
Submitter: Jenkins
Branch: master

commit 88252cdfd6c1e446f4d1cde6cc07a9c67b701e65
Author: Alistair Coles <email address hidden>
Date: Wed Aug 16 11:07:19 2017 +0100

    Fix flakey EC frag duplication policy test

    Fixes test_GET_with_duplicate_but_sufficient_frag_indexes
    failing intermittently.

    The original non-duplicated-frags version of the test in
    TestECObjController verified that the controller would search all
    primaries, ignoring repeated fragment indexes and finally find the
    last necessary unique fragment on the first handoff. The version of
    the test for a duplicated fragments policy in
    TestECDuplicationObjController had the final necessary fragment on the
    first of the 'duplicate' primaries (index ec_num_unique_fragments + 1)
    so there was a chance that it would be found before all of the first
    ec_num_unique_fragments had been tried, causing the test to fail as
    reported in the linked bug report.

    This patch moves the final necessary unique fragment to the first
    handoff (index 2 * index ec_num_unique_fragments + 1) and aligns the
    test with the intent of the equivalent test in TestECObjController,
    so that the two tests can be moved to a single implementation in an
    inherited mixin.

    test_GET_with_duplicate_but_insufficient_frag is also moved to the
    new mixin since the two implementations of that test were also very
    similar.

    Change-Id: I4d99dea8e941bccb6fa610bab39e38a6976a3721
    Closes-Bug: #1710951

Changed in swift:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/swift 2.15.1

This issue was fixed in the openstack/swift 2.15.1 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.