Manual provider jujud can fail to stop on clean-up

[Mick Gregg]

On occasion, the manual provider bootstrap node clean-up script can fail to stop the jujud process.

http://qa.jujucharms.com/releases/issue/57b1c1f9749a567693457040

This has been observed (@sinzui) on machines under load, and may be caused by an uninterruptible sleep state effectively blocking the abort and kill signals.

The fixes for bug 1642295 included some logging that may help diagnose this.

[Mick Gregg (macgreagoir)]

This was originally tracked in bug 1642295, but that bug was hijacked to track the clean-up of non-bootstrap nodes in the controller model.

[Mick Gregg (macgreagoir)]

I've just seen an example and am noting it here:
http://reports.vapour.ws/releases/4683/job/manual-deploy-xenial-arm64/attempt/772

The process is in S, not D, state but may still be waiting on IO... maybe.

[Anastasia (anastasia-macmood)]

It's a pity that the links with the full stack are no longer available.

There is not enough information here to proceed. We have done some re-working on manual provider that changed the behavior of cleanup and could have affected/fixed how manual jujud stops. Without a reference to a more recent failure, we cannot proceed and can only assume that the issue has been addressed.

I'll mark as Fix Released. If we'd see similar failure, we'd need new logs with stack trace extract and a new report.