RetryDecorator shouldn't be logging warnings and errors for expected exceptions

[Matt Riedemann]

The loopingcall.RetryDecorator is constructed with a list of expected exceptions and when the method being decorated raises an exception in that list, the decorator sleeps and retries (either until the function passes or we timeout).

For each retry, the expected exception is logged at warning level, along with it's stacktrace:

https://github.com/openstack/oslo.service/blob/0.7.0/oslo_service/loopingcall.py#L235

And if we actually timeout, the final exception is logged at error level:

https://github.com/openstack/oslo.service/blob/0.7.0/oslo_service/loopingcall.py#L242

This is bad practice because only the caller of the method being retried has context as to what's going on and knows what level things should be logged at (and if tracebacks should be logged at all). I'd guess that in most cases where a retry decorator is needed anyway is when waiting for something to show up or be gone, and those aren't really reasons to be logging warnings/errors.

It'd be fine to log debug messages to show progress, but if we fail the retry and timeout, that's when the caller should handle the exception and do the appropriate logging.

[Matt Riedemann (mriedem)]

Also:

http://specs.openstack.org/openstack/openstack-specs/specs/log-guidelines.html#definition-of-log-levels

"Error: Serious issue with cloud, administrator should be notified immediately via email/pager. On call people expected to respond."

[Matt Riedemann (mriedem)]

https://review.openstack.org/#/c/231162/