Comment 4 for bug 1669834
Revision history for this message
| John A Meinel (jameinel) wrote Re: [Bug 1669834] Re: Config command failed because of bogus upgrade in progress | #4 |
© 2004
Canonical Ltd.
•
Terms of use
•
Data privacy
•
Contact Launchpad Support
•
Blog
•
Careers
•
System status
•
67d34a1
(Get the code!)
Until juju has finished initialization and determined that there isn't an
upgrade, we reject connections with "upgrade in progress". So that isn't an
actual upgrade vs just taking a while to initialize. Looking at the
timestamps, I'm a little surprised for it to take a full minute for it to
be available, but otherwise it isn't too surprising.
John
=:->
On Tue, Oct 24, 2017 at 2:07 AM, Casey Marshall <
<email address hidden>> wrote:
> Today I received a nagios alert which seems to have been caused by nrpe
> getting restarted while it got polled (that's the best explanation I can
> rationalize it with).
>
> The nrpe unit agent log showed:
>
> 2017-10-23 12:05:46 ERROR juju.api monitor.go:59 health ping timed out
> after 30s
> 2017-10-23 12:05:46 ERROR juju.worker.
> "api-caller" manifold worker returned unexpected error: api connection
> broken unexpectedly
> 2017-10-23 12:06:01 ERROR juju.worker.
> "api-caller" manifold worker returned unexpected error: cannot open api:
> try again (try again)
> 2017-10-23 12:06:15 ERROR juju.worker.
> "api-caller" manifold worker returned unexpected error: cannot open api:
> try again (try again)
> 2017-10-23 12:06:47 ERROR juju.worker.
> "api-caller" manifold worker returned unexpected error: cannot open api:
> try again (try again)
> 2017-10-23 12:07:13 ERROR juju.worker.
> "api-caller" manifold worker returned unexpected error: cannot open api:
> unknown model: "49b0ef05-
> 2017-10-23 12:07:25 ERROR juju.worker.
> "api-caller" manifold worker returned unexpected error: cannot open api:
> login for "unit-nrpe-103" blocked because upgrade in progress
> 2017-10-23 12:07:30 ERROR juju.worker.
> "api-caller" manifold worker returned unexpected error: cannot open api:
> login for "unit-nrpe-103" blocked because upgrade in progress
> 2017-10-23 12:07:36 ERROR juju.worker.
> "api-caller" manifold worker returned unexpected error: cannot open api:
> login for "unit-nrpe-103" blocked because upgrade in progress
> 2017-10-23 12:07:41 ERROR juju.worker.
> "api-caller" manifold worker returned unexpected error: cannot open api:
> login for "unit-nrpe-103" blocked because upgrade in progress
> 2017-10-23 12:07:53 ERROR juju.worker.
> "api-caller" manifold worker returned unexpected error: cannot open api:
> try again (try again)
> 2017-10-23 12:08:02 ERROR juju.worker.
> "api-caller" manifold worker returned unexpected error: cannot open api:
> try again (try again)
> 2017-10-23 12:08:11 ERROR juju.worker.
> "api-caller" manifold worker returned unexpected error: cannot open api:
> try again (try again)
> 2017-10-23 12:08:20 ERROR juju.worker.
> "api-caller" manifold worker returned unexpected error: cannot open api:
> try again (try again)
> 2017-10-23 12:08:29 ERROR juju.worker.
> "api-caller" manifold worker returned unexpected error: cannot open api:
> try again (try again)
> 2017-10-23 12:08:37 ERROR juju.worker.
> "api-caller" manifold worker returned unexpected error: cannot open api:
> try again (try again)
> 2017-10-23 12:09:05 DEBUG config-changed Hit:1 http://
> clouds.
> 2017-10-23 12:09:05 DEBUG config-changed Get:2 http://
> clouds.
> 2017-10-23 12:09:05 DEBUG config-changed Get:3 http://
> ubuntu xenial-security InRelease [102 kB]
> 2017-10-23 12:09:05 DEBUG config-changed Ign:4 http://
> canonical.
> 2017-10-23 12:09:05 DEBUG config-changed Hit:5 http://
> telegraf-
> 2017-10-23 12:09:05 DEBUG config-changed Get:6 http://
> clouds.
> 2017-10-23 12:09:05 DEBUG config-changed Hit:7 http://
> canonical.
> 2017-10-23 12:09:07 DEBUG config-changed Fetched 306 kB in 0s (725 kB/s)
> 2017-10-23 12:09:08 DEBUG config-changed Reading package lists...
> 2017-10-23 12:09:08 INFO juju-log Installing ['nagios-
> 'nagios-
> ['--option=
> 2017-10-23 12:09:08 DEBUG config-changed Reading package lists...
> 2017-10-23 12:09:08 DEBUG config-changed Building dependency tree...
> 2017-10-23 12:09:08 DEBUG config-changed Reading state information...
> 2017-10-23 12:09:08 DEBUG config-changed nagios-
> the newest version (2.1.2-2ubuntu2).
> 2017-10-23 12:09:08 DEBUG config-changed nagios-
> already the newest version (2.1.2-2ubuntu2).
> 2017-10-23 12:09:08 DEBUG config-changed rsync is already the newest
> version (3.1.1-3ubuntu1).
> 2017-10-23 12:09:08 DEBUG config-changed nagios-nrpe-server is already the
> newest version (2.15-1ubuntu1.1).
> 2017-10-23 12:09:08 DEBUG config-changed 0 upgraded, 0 newly installed, 0
> to remove and 38 not upgraded.
> 2017-10-23 12:09:08 INFO juju-log /usr/bin/rsync -r --executability
> /var/lib/
> /usr/local/
> 2017-10-23 12:09:08 DEBUG config-changed inactive
> 2017-10-23 12:09:08 DEBUG config-changed Failed to start
> nrpe-install.
> 2017-10-23 12:09:08 DEBUG config-changed inactive
> 2017-10-23 12:09:08 DEBUG config-changed inactive
>
> The machine agent log showed:
>
> 2017-10-23 12:06:05 ERROR juju.api monitor.go:59 health ping timed out
> after 30s
> 2017-10-23 12:06:05 ERROR juju.worker.
> "api-caller" manifold worker returned unexpected error: api connection
> broken unexpectedly
> 2017-10-23 12:06:05 ERROR juju.worker runner.go:381 fatal
> "12-container-
> shut down
> 2017-10-23 12:06:05 ERROR juju.worker runner.go:381 fatal
> "stateconverter": connection is shut down
> 2017-10-23 12:06:16 ERROR juju.worker.
> "api-caller" manifold worker returned unexpected error: cannot open api:
> try again (try again)
> 2017-10-23 12:06:47 ERROR juju.worker.
> "api-caller" manifold worker returned unexpected error: cannot open api:
> try again (try again)
> 2017-10-23 12:07:12 ERROR juju.worker.
> "api-caller" manifold worker returned unexpected error: cannot open api:
> unknown model: "49b0ef05-
> 2017-10-23 12:07:18 ERROR juju.worker.
> "api-caller" manifold worker returned unexpected error: cannot open api:
> login for "machine-12" blocked because upgrade in progress
> 2017-10-23 12:07:21 ERROR juju.worker.
> "api-caller" manifold worker returned unexpected error: cannot open api:
> login for "machine-12" blocked because upgrade in progress
> 2017-10-23 12:07:24 ERROR juju.worker.
> "api-caller" manifold worker returned unexpected error: cannot open api:
> login for "machine-12" blocked because upgrade in progress
> 2017-10-23 12:07:28 ERROR juju.worker.
> "api-caller" manifold worker returned unexpected error: cannot open api:
> login for "machine-12" blocked because upgrade in progress
> 2017-10-23 12:07:33 ERROR juju.worker.
> "api-caller" manifold worker returned unexpected error: cannot open api:
> login for "machine-12" blocked because upgrade in progress
> 2017-10-23 12:07:38 ERROR juju.worker.
> "api-caller" manifold worker returned unexpected error: cannot open api:
> login for "machine-12" blocked because upgrade in progress
> 2017-10-23 12:07:42 ERROR juju.worker.
> "api-caller" manifold worker returned unexpected error: cannot open api:
> login for "machine-12" blocked because upgrade in progress
>
> Discussed w/webops, they confirm that controller machines were restarted
> around the time of the health timeout & API connection errors. They've
> confirmed though that there was no upgrade in progress, Juju was already
> at 2.2.4 on the controllers & agents.
>
> --
> You received this bug notification because you are subscribed to juju.
> Matching subscriptions: juju bugs
> https:/
>
> Title:
> Config command failed because of bogus upgrade in progress
>
> To manage notifications about this bug go to:
> https:/
>