CI: jobs failing with Error: /Stage[main]/Rabbitmq::Config/Rabbitmq_erlang_cookie[/var/lib/rabbitmq/.erlang.cookie]/content: <snip> `encode': "\xC3" on US-ASCII (Encoding::InvalidByteSequenceError)

Bug #1643059 reported by James Slagle on 2016-11-18
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Critical
James Slagle

Bug Description

Seeing a lot (if not all) job failing with this error on the overcloud-controller-0:

Nov 18 19:37:25 localhost os-collect-config: #033[1;31mError: /Stage[main]/Rabbitmq::Config/Rabbitmq_erlang_cookie[/var/lib/rabbitmq/.erlang.cookie]/content: change from NFNKSOGWBJTZCZDNLQEN to weDcaWyG7Obatc48K8T6 failed: Execution of '/usr/bin/puppet resource service rabbitmq-server ensure=stopped' returned 1: /usr/share/gems/gems/json-1.7.7/lib/json/common.rb:155:in `encode': "\xC3" on US-ASCII (Encoding::InvalidByteSequenceError)

Example failed job:

http://logs.openstack.org/62/385162/6/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha/3a08ed4/

Tags: ci Edit Tag help
James Slagle (james-slagle) wrote :

Comparing package manifests from a failed job to a recently passed one, the differences are:

[jslagle@localhost tmp.L5KeHNvp42]$ diff -u work2 broke2
--- work2 2016-11-18 15:40:08.286175901 -0500
+++ broke2 2016-11-18 15:39:23.380459895 -0500
@@ -617,7 +617,7 @@
 openvswitch-2.5.0-2.el7.x86_64
 os-apply-config-5.0.1-0.20161116200506.d72c4d9.el7.centos.noarch
 os-collect-config-6.0.0-0.20161117222156.e8e2f3d.el7.centos.noarch
-os-net-config-6.0.0-0.20161117225400.9971eb3.el7.centos.noarch
+os-net-config-6.0.0-0.20161118182225.433a0f3.el7.centos.noarch
 os-prober-1.58-5.el7.x86_64
 os-refresh-config-5.0.1-0.20161116194316.796f4e8.el7.centos.noarch
 p11-kit-0.20.7-3.el7.x86_64
@@ -770,7 +770,7 @@
 puppet-pacemaker-0.4.0-0.20161108164007.1e2560f.el7.centos.noarch
 puppet-rabbitmq-5.6.0-0.20161111010126.12aa7cc.el7.centos.noarch
 puppet-redis-1.2.3-0.20161015002837.9711564.el7.centos.noarch
-puppet-remote-0.0.1-0.20161003203714.35cc557.el7.centos.noarch
+puppet-remote-0.0.1-0.20161118173454.b370d28.el7.centos.noarch
 puppet-rsync-0.4.0-0.20161003183908.8cc3c6f.el7.centos.noarch
 puppet-sahara-9.4.0-0.20161117203415.99639c8.el7.centos.noarch
 puppet-sensu-2.1.0-0.20161107104621.6d10bae.el7.centos.noarch
@@ -783,7 +783,7 @@
 puppet-tempest-9.4.0-0.20161117195904.185f467.el7.centos.noarch
 puppet-timezone-3.3.0-0.20161003185332.cf62f1b.el7.centos.noarch
 puppet-tomcat-1.6.0-0.20161111192530.f53d54b.el7.centos.noarch
-puppet-tripleo-6.0.0-0.20161118172008.e1b7ad8.el7.centos.noarch
+puppet-tripleo-6.0.0-0.20161118162559.92f9c95.el7.centos.noarch
 puppet-trove-9.4.0-0.20161117203846.47e28a5.el7.centos.noarch
 puppet-uchiwa-1.0.1-0.20161003185621.64ce619.el7.centos.noarch
 puppet-vcsrepo-1.4.0-0.20161118170152.f012768.el7.centos.noarch

The new puppet-remote might be the problem. I noticed this new commit:

https://github.com/paramite/puppet-remote/commit/b370d28ca4eff9af64e176e83ae0cbccb7ccb144

The added file metadata.json is not valid json (should not be a comma at end of line 25).

I'm not sure if that would cause this issue or not.

tags: added: alert ci
Changed in tripleo:
status: New → Triaged
importance: Undecided → Critical
milestone: none → ocata-2
Gabriele Cerami (gcerami) wrote :

I think https://github.com/paramite/puppet-remote/issues/2 is the issue here. The á in Mágr encoding is \xC3\xA1 in iso-8859-1.

Gabriele Cerami (gcerami) wrote :

Parsing was blocking at á because it comes first, but fixed that, parsing would have blocked at the incorrect comma after, so both pull requests are needed.

They are both merged.

Changed in tripleo:
status: Triaged → Fix Released
tags: removed: alert
Changed in tripleo:
assignee: nobody → James Slagle (james-slagle)
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.