XenAPI Glance Plugin logs incorrect message when retrying download

[Dan Emmons]

When the XenAPI Glance plugin encounters a retryable error, it logs a message stating that it is retrying the action. This was added when retries were only implemented on uploads, but the retry logic was later moved up a level to be usable for all glance plugin functions. I've noticed that the messages logged indicate they will retry vhd_upload regardless of the Glance plugin function actually being used, most obvious during a vhd_download:

Dec 30 19:22:42 DEBUG nova.virt.xenapi.client.session Got exception: ['XENAPI_PLUGIN_FAILURE', 'signal: SIGTERM', '', ''] _unwrap_plugin_exceptions /nova/virt/xenapi/client/session.py:249

Dec 30 19:22:42 DEBUG nova.virt.xenapi.client.session Error due to a signal, retrying upload_vhd _is_retryable_exception /nova/virt/xenapi/client/session.py:232

Dec 30 19:22:42 WARNING nova.virt.xenapi.client.session glance.download_vhd failed. Retrying call.

Dec 30 19:22:42 INFO nova.virt.xenapi.client.session glance.download_vhd attempt 2/6

This example was in a Rackspace environment, so I have redacted a few things from the logs, but the issue is clear and easily reproducible. For this test I built a new instance, logged into the host that the build started on, and killed the glance process with a SIGTERM so it would retry and log the message. I've confirmed that the issue in the code is still present in the current trunk in GitHub.

This will be trivial to fix and I plan to assign this bug to myself and submit the code patch. Essentially, my proposed solution is to add an optional argument to _is_retryable_exception() to specify the function, and then reference that if it is set or use a generic message otherwise.

[Dan Emmons (dan-emmons)]

Assigning to myself and a fix should be submitted shortly

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/64473

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/64546

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/64558

[Dan Emmons (dan-emmons)]

I've abandoned the first two of the proposed fixes, having learned quite a bit about the Gerrit review process and Git in the process :\ and the third looks good so far but it will need another look from the other reviewers (was initially marked Do Not Merge due to duplicates but that should be sorted out now)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/64558
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=75935ab810fa6b4c183159bae425d3285504cc91
Submitter: Jenkins
Branch: master

commit 75935ab810fa6b4c183159bae425d3285504cc91
Author: Dan Emmons <email address hidden>
Date: Tue Dec 31 15:53:23 2013 +0000

    Fixes messages logged on Glance plugin retries

    Retry logic is now being used for both upload_vhd and download_vhd,
    but nova/virt/xenapi/client/session.py still had 'upload_vhd'
    hardcoded in its logged messages. This change causes it to accept the
    function being used as an argument and display that.

    Fixes-Bug: #1265054
    Change-Id: I6203aef81960078f92fa7839e47ee1c84df95745