more verbose error logging around errors from workers

Bug #1799476 reported by Junien F
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Canonical Juju
Fix Released
High
Tim Penhey

Bug Description

Hi,

On a 2.4.3 controller, we have the following error logs :

2018-10-23 14:17:08 ERROR juju.worker.dependency engine.go:587 "compute-provisioner" manifold worker returned unexpected error:
The provided credentials could not be validated and
may not be authorized to carry out the request.
Ensure that your account is authorized to use the Amazon EC2 service and
that you are using the correct access keys.
These keys are obtained via the "Security Credentials"
page in the AWS console.
: AWS was not able to validate the provided access credentials (AuthFailure)

and also

2018-10-23 14:17:08 ERROR juju.worker.dependency engine.go:587 "firewaller" manifold worker returned unexpected error: cannot respond to units changes for "machine-2":
The provided credentials could not be validated and
may not be authorized to carry out the request.
Ensure that your account is authorized to use the Amazon EC2 service and
that you are using the correct access keys.
These keys are obtained via the "Security Credentials"
page in the AWS console.
: AWS was not able to validate the provided access credentials (AuthFailure)

Could we please get more details about the model generating these errors ?

Thanks !

Changed in juju:
status: New → Triaged
importance: Undecided → High
milestone: none → 2.5.1
Changed in juju:
assignee: nobody → Anastasia (anastasia-macmood)
Revision history for this message
Anastasia (anastasia-macmood) wrote :

This errors come from the cloud calls made by a dependency engine.

By the time, we get this error, we are already in the cloud and it's not possible to determine what model made a call. What we can do is to ensure that any error reported via dependency agent *knows* the model. We can do that by keeping track of model uuids in worker manifest.

I have briefly mentioned the idea to the team and will start investigating solution implementation. Thank you for suggestion!

Changed in juju:
assignee: Anastasia (anastasia-macmood) → nobody
Ian Booth (wallyworld)
Changed in juju:
milestone: 2.5.1 → 2.5.2
Changed in juju:
milestone: 2.5.2 → 2.5.3
Changed in juju:
milestone: 2.5.3 → 2.5.4
Changed in juju:
milestone: 2.5.4 → 2.5.5
Changed in juju:
milestone: 2.5.6 → 2.5.8
summary: - more verbose error logging around credential errors
+ more verbose error logging around errors from workers
Changed in juju:
milestone: 2.5.8 → 2.5.9
Revision history for this message
Tim Penhey (thumper) wrote :

While the messages aren't prefixed, they are now split out into different files and log collections.

See https://discourse.jujucharms.com/t/upcoming-2-7-model-logs-for-controller-workers/2220

Changed in juju:
milestone: 2.5.9 → 2.7-beta1
assignee: nobody → Tim Penhey (thumper)
status: Triaged → Fix Committed
Revision history for this message
Junien F (axino) wrote :

This looks great, thanks !

Changed in juju:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.