Juniper/contrail reset_db*.sh scripts can hang

Bug #1765506 reported by Krzysztof Klimonda
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Juniper Openstack
Status tracked in Trunk
R5.0
Fix Committed
High
Michał Błotniak
Trunk
Fix Committed
Undecided
Michał Błotniak
OpenContrail
Fix Committed
Undecided
Unassigned

Bug Description

tools/reset_db.sh and tools/reset_db_psql.sh have a potential for hanging on the command that redirects stdin into `docker exec` call.

When that happens, quick look through the affected VM shows that `docker exec` is waiting for the spawned process to finish, and mysql/psql calls wait on readv(0, indefinitely (fd 0 is stdin). This could be docker version specific, or could be caused by piping large .sql files.

Tags: provisioning
Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] master

Review in progress for https://review.opencontrail.org/42182
Submitter: Krzysztof Klimonda (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/42260
Submitter: Michal Blotniak (<email address hidden>)

Jeba Paulaiyan (jebap)
Changed in opencontrail:
importance: Undecided → High
assignee: nobody → Michał Błotniak (buoto)
assignee: Michał Błotniak (buoto) → nobody
importance: High → Undecided
tags: added: provisioning
Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/42260
Submitter: Krzysztof Klimonda (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : A change has been merged

Reviewed: https://review.opencontrail.org/42182
Committed: http://github.com/Juniper/contrail/commit/2292ba68fbeaadf3944d44a7f69b360440832c7e
Submitter: Zuul v3 CI (<email address hidden>)
Branch: master

commit 2292ba68fbeaadf3944d44a7f69b360440832c7e
Author: Krzysztof Klimonda <email address hidden>
Date: Thu Apr 19 15:57:10 2018 +0200

Lower job timeout to 90 minutes

We're hitting some random deadlocks on redirecting stdin to docker exec
calls. This review lowers a timeout for the job to 90 minutes, so that
reviews stuck on that fail faster.

Partial-Bug: #1765506
Change-Id: Ie4b31bf29fe1652f13ee458919ae2014afb796d1

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] master

Review in progress for https://review.opencontrail.org/42260
Submitter: Michal Blotniak (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : A change has been merged

Reviewed: https://review.opencontrail.org/42260
Committed: http://github.com/Juniper/contrail/commit/27f029ffea0906941544bb9661891b1c119ce4ef
Submitter: Zuul v3 CI (<email address hidden>)
Branch: master

commit 27f029ffea0906941544bb9661891b1c119ce4ef
Author: Michał Błotniak <email address hidden>
Date: Fri Apr 20 10:33:14 2018 +0200

Mount sourcedir in DB containers

Passing huge init SQL files via stdin to docker exec caused random
deadlocks. To avoid this files are mounted and read by DB clients inside
containers.

Note that containers in dev environments should be rebuilt
with `make testenv`.

Change-Id: Ia7688c7048da88cb016c28682754145dc37c547b
Partial-Bug: #1765506

Changed in opencontrail:
status: New → Fix Committed
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.