[systest] Tests for backup/restore fail by timeout

Bug #1543658 reported by Artem Panchenko
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Invalid
High
Alexander Kurenyshev
8.0.x
Fix Released
High
Alexander Kurenyshev

Bug Description

System tests for backup/restore fail by timeout while executing `dockerctl restore` command:

TimeoutException: 'dockerctl restore' run longer then 600 sec

Manually restore works fine, but takes more than 10 minutes:

http://paste.openstack.org/show/486450/

[root@nailgun ~]# time dockerctl restore /var/backup/fuel/backup_2016-02-09_0727/fuel_backup_2016-02-09_0727.tar.lrz
Shut down
Stopping containers...
...
Restore complete.

real 16m57.226s
user 10m47.404s
sys 1m36.753s

Looks like we need to increase timeout to 20 minutes in tests:

https://github.com/openstack/fuel-qa/blob/09ab2dfd874b026d528c3f9aeea74d62620f9b54/fuelweb_test/tests/test_backup_restore.py#L67
https://github.com/openstack/fuel-qa/blob/09ab2dfd874b026d528c3f9aeea74d62620f9b54/fuelweb_test/tests/test_backup_restore.py#L146
https://github.com/openstack/fuel-qa/blob/09ab2dfd874b026d528c3f9aeea74d62620f9b54/fuelweb_test/tests/test_backup_restore.py#L236
https://github.com/openstack/fuel-qa/blob/09ab2dfd874b026d528c3f9aeea74d62620f9b54/fuelweb_test/tests/test_backup_restore.py#L308
https://github.com/openstack/fuel-qa/blob/09ab2dfd874b026d528c3f9aeea74d62620f9b54/fuelweb_test/tests/test_backup_restore.py#L422

Revision history for this message
Tatyanka (tatyana-leontovich) wrote :

we merge a docker removal for 9.0 - so need to check how backup/restore looks like there now

Changed in fuel:
status: New → Incomplete
Revision history for this message
Alexander Kurenyshev (akurenyshev) wrote :

As long as we have a bug [0] unfixed and backup/restore doesn't works well, we can't say that time is not enough for restoring. Let's wait for a resolving issue [0]

[0] https://bugs.launchpad.net/fuel/+bug/1541539

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-qa (master)

Fix proposed to branch: master
Review: https://review.openstack.org/281314

Changed in fuel:
assignee: Fuel QA Team (fuel-qa) → Alexander Kurenyshev (akurenyshev)
status: Incomplete → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-qa (stable/8.0)

Reviewed: https://review.openstack.org/281316
Committed: https://git.openstack.org/cgit/openstack/fuel-qa/commit/?id=d6e78db6227201d7166d65fc70998e367342afa3
Submitter: Jenkins
Branch: stable/8.0

commit d6e78db6227201d7166d65fc70998e367342afa3
Author: Alexander Kurenyshev <email address hidden>
Date: Wed Feb 17 17:41:03 2016 +0300

    Increase restore timeout to 20 mins

    A restore operation takes more than 10 minutes.
    In a manual reproducing it takes about 16 mins.
    An increasing timeouts to 20 minutes should solve
    a problem.

    Change-Id: Ida4e9b38e323f369668ce95859a1ccfdb9dfbfae
    Closes-Bug:1543658

Changed in fuel:
status: In Progress → Invalid
Revision history for this message
Alexander Kurenyshev (akurenyshev) wrote :

Technically the timeout was increased to 20 mins. Tests failed because of lzip failure.
So I moved this bug to the Fix Released status.

tags: removed: non-release
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on fuel-qa (master)

Change abandoned by Fuel DevOps Robot (<email address hidden>) on branch: master
Review: https://review.openstack.org/281314
Reason: This review is > 4 weeks without comment, and failed Jenkins the last time it was checked. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and leaving a 'recheck' comment to get fresh test results.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.