Service down and errors in log during image related operations

[Gorka Eguileor]

In some systems, when doing image related operations we start having RabbitMQ, and DB errors. Which in turns makes the volume service appear as down to the scheduler.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/615934

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/615934
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=e4de4fb33d3c54ee48fa2e3689e45787557ee805
Submitter: Zuul
Branch: master

commit e4de4fb33d3c54ee48fa2e3689e45787557ee805
Author: Gorka Eguileor <email address hidden>
Date: Tue Nov 6 17:19:43 2018 +0100

    Ensure image utils don't block greenthreads

    When doing image operations in Cinder we may start getting errors on
    RabbitMQ and DB connections, which result in the volume service
    appearing as down to the scheduler.

    This is caused by file I/O operations, that in some cases block
    greenthreads, preventing switching to another greenthread on I/O as they
    should.

    This results in many different errors, so this patch makes sure that
    image operations (fetch, put, verify image, ...) that could prevent
    greenthread switching are executed in native threads.

    Closes-Bug: #1801958
    Change-Id: I8a8fbf96875319a7e5ca167fcd2bca45f57649c8

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/rocky
Review: https://review.openstack.org/624497

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/624497
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=f30f6debdc07f03b494ab136e1afe553b9fe071c
Submitter: Zuul
Branch: stable/rocky

commit f30f6debdc07f03b494ab136e1afe553b9fe071c
Author: Gorka Eguileor <email address hidden>
Date: Tue Nov 6 17:19:43 2018 +0100

    Ensure image utils don't block greenthreads

    When doing image operations in Cinder we may start getting errors on
    RabbitMQ and DB connections, which result in the volume service
    appearing as down to the scheduler.

    This is caused by file I/O operations, that in some cases block
    greenthreads, preventing switching to another greenthread on I/O as they
    should.

    This results in many different errors, so this patch makes sure that
    image operations (fetch, put, verify image, ...) that could prevent
    greenthread switching are executed in native threads.

    Signed-off-by: Sofia Enriquez <email address hidden>

    Closes-Bug: #1801958
    Change-Id: I8a8fbf96875319a7e5ca167fcd2bca45f57649c8
    (cherry-picked from I8a8fbf96875319a7e5ca167fcd2bca45f57649c8)

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/queens
Review: https://review.openstack.org/625994

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/625994
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=da016c47a90d85fac1059d4abb41997696a593ea
Submitter: Zuul
Branch: stable/queens

commit da016c47a90d85fac1059d4abb41997696a593ea
Author: Gorka Eguileor <email address hidden>
Date: Tue Nov 6 17:19:43 2018 +0100

    Ensure image utils don't block greenthreads

    When doing image operations in Cinder we may start getting errors on
    RabbitMQ and DB connections, which result in the volume service
    appearing as down to the scheduler.

    This is caused by file I/O operations, that in some cases block
    greenthreads, preventing switching to another greenthread on I/O as they
    should.

    This results in many different errors, so this patch makes sure that
    image operations (fetch, put, verify image, ...) that could prevent
    greenthread switching are executed in native threads.

    NOTE(s10): Part of the change related to the image verification has not been
    backported because image signature verification is only in Rocky.

    Signed-off-by: Sofia Enriquez <email address hidden>

    Closes-Bug: #1801958
    Change-Id: I8a8fbf96875319a7e5ca167fcd2bca45f57649c8
    (cherry picked from commit f30f6debdc07f03b494ab136e1afe553b9fe071c)
    (cherry-picked from commit e4de4fb33d3c54ee48fa2e3689e45787557ee805)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/cinder 12.0.5 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/cinder 13.0.3 release.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/pike
Review: https://review.openstack.org/644950

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/cinder 14.0.0.0rc1 release candidate.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/644950
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=4db9d829e13369220886443d611f37ce228ea234
Submitter: Zuul
Branch: stable/pike

commit 4db9d829e13369220886443d611f37ce228ea234
Author: Gorka Eguileor <email address hidden>
Date: Tue Nov 6 17:19:43 2018 +0100

    Ensure image utils don't block greenthreads

    When doing image operations in Cinder we may start getting errors on
    RabbitMQ and DB connections, which result in the volume service
    appearing as down to the scheduler.

    This is caused by file I/O operations, that in some cases block
    greenthreads, preventing switching to another greenthread on I/O as they
    should.

    This results in many different errors, so this patch makes sure that
    image operations (fetch, put, verify image, ...) that could prevent
    greenthread switching are executed in native threads.

    NOTE(s10): Part of the change related to the image verification has not been
    backported because image signature verification is only in Rocky.

    Signed-off-by: Sofia Enriquez <email address hidden>

    Closes-Bug: #1801958
    Change-Id: I8a8fbf96875319a7e5ca167fcd2bca45f57649c8
    (cherry picked from commit f30f6debdc07f03b494ab136e1afe553b9fe071c)
    (cherry-picked from commit e4de4fb33d3c54ee48fa2e3689e45787557ee805)
    (cherry picked from commit da016c47a90d85fac1059d4abb41997696a593ea)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/cinder 11.2.2 release.