hook failures - nil pointer dereference

[Tim Van Steenburgh]

I deployed the following on wolfe-01 (ppc64el) using the local provider:

The charms used:

  parent branch: bzr+ssh://bazaar.launchpad.net/~charmers/charms/precise/haproxy/trunk/
  parent branch: bzr+ssh://bazaar.launchpad.net/~charmers/charms/precise/memcached/trunk/
  parent branch: bzr+ssh://bazaar.launchpad.net/~mbruzek/charms/trusty/mysql/apache2fix/
  parent branch: bzr+ssh://bazaar.launchpad.net/~cabs-team/charms/trusty/sugarcrm/trunk/

The deploy script:

juju deploy local:trusty/sugarcrm
juju deploy local:trusty/mysql
juju deploy local:trusty/memcached
juju deploy local:trusty/haproxy
juju set mysql dataset-size="1G"
juju add-relation sugarcrm mysql
juju add-relation sugarcrm memcached
juju add-relation sugarcrm haproxy
juju expose haproxy

I waited for all services to start up successfully, and they did.

Fast-forward 7 hours, I log in to wolfe-01 and this is the juju status:
http://paste.ubuntu.com/7216852/

I've attached the all-machines.log, which contains nil pointer errors.

Unfortunately, I can't be sure what transpired on the machine during those 7 hours - only that I didn't touch anything.

[Kapil Thangavelu (hazmat)]

this is not local provider.. this is client panic/failure on ppc.

[Andrew Wilkins (axwalk)]

Looks like the local provider to me, from the log.

Tim: any obvious issues with OOM or disk space?

[Kapil Thangavelu (hazmat)]

it is the local provider, but its totally unrelated to this being local provider.. this is just a flat out ppc issue with the websocket code afaics.

[Tim Van Steenburgh (tvansteenburgh)]

There's lots of free disk (94G). I can't be sure about RAM since I didn't check that at the time, but I think OOM would have been unlikely since there was 24G allocated to wolfe-01 at the time.

[Dave Cheney (dave-cheney)]

Hi Tim,

all-machines.log has been corrupted by rsync, can you please, and in the future, include the machine-0.log file which represents the provision er and the API server.

I can see you are building from source and using --upload-tools

Can you please tell me the revision you are building from and the version of gccgo-4.9 and gccgo-go you are using.

Thanks

Dave

[Dave Cheney (dave-cheney)]

not rsync, rsyslog.

On Tue, Apr 8, 2014 at 9:22 AM, Dave Cheney <email address hidden> wrote:
> Hi Tim,
>
> all-machines.log has been corrupted by rsync, can you please, and in the
> future, include the machine-0.log file which represents the provision er
> and the API server.
>
> I can see you are building from source and using --upload-tools
>
> Can you please tell me the revision you are building from and the
> version of gccgo-4.9 and gccgo-go you are using.
>
> Thanks
>
> Dave
>
> --
> You received this bug notification because you are subscribed to juju-
> core.
> Matching subscriptions: MOAR JUJU SPAM!
> https://bugs.launchpad.net/bugs/1303787
>
> Title:
> hook failures - nil pointer dereference
>
> Status in juju-core:
> Triaged
>
> Bug description:
> I deployed the following on wolfe-01 (ppc64el) using the local
> provider:
>
> The charms used:
>
> parent branch: bzr+ssh://bazaar.launchpad.net/~charmers/charms/precise/haproxy/trunk/
> parent branch: bzr+ssh://bazaar.launchpad.net/~charmers/charms/precise/memcached/trunk/
> parent branch: bzr+ssh://bazaar.launchpad.net/~mbruzek/charms/trusty/mysql/apache2fix/
> parent branch: bzr+ssh://bazaar.launchpad.net/~cabs-team/charms/trusty/sugarcrm/trunk/
>
> The deploy script:
>
> juju deploy local:trusty/sugarcrm
> juju deploy local:trusty/mysql
> juju deploy local:trusty/memcached
> juju deploy local:trusty/haproxy
> juju set mysql dataset-size="1G"
> juju add-relation sugarcrm mysql
> juju add-relation sugarcrm memcached
> juju add-relation sugarcrm haproxy
> juju expose haproxy
>
> I waited for all services to start up successfully, and they did.
>
> Fast-forward 7 hours, I log in to wolfe-01 and this is the juju status:
> http://paste.ubuntu.com/7216852/
>
> I've attached the all-machines.log, which contains nil pointer errors.
>
> Unfortunately, I can't be sure what transpired on the machine during
> those 7 hours - only that I didn't touch anything.
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/juju-core/+bug/1303787/+subscriptions

[Tim Van Steenburgh (tvansteenburgh)]

Ugh, sorry Dave, but since the original bug report, juju has been upgraded on wolfe-01 to 1.18. I don't know what the original versions were, except that juju was 1.17.7.

[Dave Cheney (dave-cheney)]

Marking as incomplete, need more information.

[Dave Cheney (dave-cheney)]

Sure, can you please provide the rest of the information. I'd be
surprised if 1.17 -> 1.18 had any impact on the frequency of this bug.

On Tue, Apr 8, 2014 at 10:00 AM, Tim Van Steenburgh
<email address hidden> wrote:
> Ugh, sorry Dave, but since the original bug report, juju has been
> upgraded on wolfe-01 to 1.18. I don't know what the original versions
> were, except that juju was 1.17.7.
>
> --
> You received this bug notification because you are subscribed to juju-
> core.
> Matching subscriptions: MOAR JUJU SPAM!
> https://bugs.launchpad.net/bugs/1303787
>
> Title:
> hook failures - nil pointer dereference
>
> Status in juju-core:
> Incomplete
>
> Bug description:
> I deployed the following on wolfe-01 (ppc64el) using the local
> provider:
>
> The charms used:
>
> parent branch: bzr+ssh://bazaar.launchpad.net/~charmers/charms/precise/haproxy/trunk/
> parent branch: bzr+ssh://bazaar.launchpad.net/~charmers/charms/precise/memcached/trunk/
> parent branch: bzr+ssh://bazaar.launchpad.net/~mbruzek/charms/trusty/mysql/apache2fix/
> parent branch: bzr+ssh://bazaar.launchpad.net/~cabs-team/charms/trusty/sugarcrm/trunk/
>
> The deploy script:
>
> juju deploy local:trusty/sugarcrm
> juju deploy local:trusty/mysql
> juju deploy local:trusty/memcached
> juju deploy local:trusty/haproxy
> juju set mysql dataset-size="1G"
> juju add-relation sugarcrm mysql
> juju add-relation sugarcrm memcached
> juju add-relation sugarcrm haproxy
> juju expose haproxy
>
> I waited for all services to start up successfully, and they did.
>
> Fast-forward 7 hours, I log in to wolfe-01 and this is the juju status:
> http://paste.ubuntu.com/7216852/
>
> I've attached the all-machines.log, which contains nil pointer errors.
>
> Unfortunately, I can't be sure what transpired on the machine during
> those 7 hours - only that I didn't touch anything.
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/juju-core/+bug/1303787/+subscriptions

[Dave Cheney (dave-cheney)]

Here is my script to create the test environment.

http://paste.ubuntu.com/7219530/

[Dave Cheney (dave-cheney)]

Yup. I can reproduce this error. I'm spinning up an amd64 trusty image to see if the problem is isolated to ppc64l