juju-core

worker/peergrouper: data race in package

Bug #1467715 reported by Dave Cheney on 2015-06-23
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
​ juju-core
Fix Released
Critical
Menno Finlay-Smits
Also affects project (?) Also affects distribution/package Nominate for series
See original description
Curtis Hovey (sinzui) on 2015-06-23
Changed in juju-core:
status: New → Triaged
importance: Undecided → Medium
tags: added: intermittent-failure race-condition
Curtis Hovey (sinzui) on 2016-04-19
description: updated
Curtis Hovey (sinzui) wrote :
Download full text (4.4 KiB)

This iss is not critical because it is consistently contributing to curses of master. The current race looks like

==================
WARNING: DATA RACE
Write by goroutine 139:
  reflect.storeIword()
      /usr/lib/go/src/pkg/reflect/value.go:208 +0x395
  reflect.Value.Set()
      /usr/lib/go/src/pkg/reflect/value.go:1389 +0x13c
  github.com/juju/testing.PatchValue()
      /home/ubuntu/juju-core_2.0-beta5/src/github.com/juju/testing/patch.go:46 +0x2f4
  github.com/juju/testing.(*CleanupSuite).PatchValue()
      /home/ubuntu/juju-core_2.0-beta5/src/github.com/juju/testing/cleanup.go:123 +0x69
  github.com/juju/juju/worker/peergrouper.func·025()
      /home/ubuntu/juju-core_2.0-beta5/src/github.com/juju/juju/worker/peergrouper/worker_test.go:370 +0x3df
  github.com/juju/juju/worker/peergrouper.DoTestForIPv4AndIPv6()
      /home/ubuntu/juju-core_2.0-beta5/src/github.com/juju/juju/worker/peergrouper/worker_test.go:59 +0x6f
  github.com/juju/juju/worker/peergrouper.(*workerSuite).TestSetMembersErrorIsNotFatal()
      /home/ubuntu/juju-core_2.0-beta5/src/github.com/juju/juju/worker/peergrouper/worker_test.go:384 +0xc4
  runtime.call16()
      /usr/lib/go/src/pkg/runtime/asm_amd64.s:338 +0x31
  reflect.Value.Call()
      /usr/lib/go/src/pkg/reflect/value.go:345 +0xaa
  gopkg.in/check%2ev1.func·003()
      /home/ubuntu/juju-core_2.0-beta5/src/gopkg.in/check.v1/check.go:772 +0x504
  gopkg.in/check%2ev1.func·001()
      /home/ubuntu/juju-core_2.0-beta5/src/gopkg.in/check.v1/check.go:666 +0xf3

Previous read by goroutine 10:
  github.com/juju/juju/worker/peergrouper.(*pgWorker).loop()
      /home/ubuntu/juju-core_2.0-beta5/src/github.com/juju/juju/worker/peergrouper/worker.go:205 +0x76e
  github.com/juju/juju/worker/peergrouper.*pgWorker.(github.com/juju/juju/worker/peergrouper.loop)·fm()
      /home/ubuntu/juju-core_2.0-beta5/src/github.com/juju/juju/worker/peergrouper/worker.go:135 +0x45
  github.com/juju/juju/worker/catacomb.func·003()
      /home/ubuntu/juju-core_2.0-beta5/src/github.com/juju/juju/worker/catacomb/catacomb.go:115 +0xcd

Goroutine 139 (running) created at:
  gopkg.in/check%2ev1.(*suiteRunner).forkCall()
      /home/ubuntu/juju-core_2.0-beta5/src/gopkg.in/check.v1/check.go:667 +0x529
  gopkg.in/check%2ev1.(*suiteRunner).forkTest()
      /home/ubuntu/juju-core_2.0-beta5/src/gopkg.in/check.v1/check.go:804 +0x168
  gopkg.in/check%2ev1.(*suiteRunner).runTest()
      /home/ubuntu/juju-core_2.0-beta5/src/gopkg.in/check.v1/check.go:809 +0x3e
  gopkg.in/check%2ev1.(*suiteRunner).run()
      /home/ubuntu/juju-core_2.0-beta5/src/gopkg.in/check.v1/check.go:615 +0x44a
  gopkg.in/check%2ev1.Run()
      /home/ubuntu/juju-core_2.0-beta5/src/gopkg.in/check.v1/run.go:92 +0x56
  gopkg.in/check%2ev1.RunAll()
      /home/ubuntu/juju-core_2.0-beta5/src/gopkg.in/check.v1/run.go:84 +0x111
  gopkg.in/check%2ev1.TestingT()
      /home/ubuntu/juju-core_2.0-beta5/src/gopkg.in/check.v1/run.go:72 +0x4d6
  github.com/juju/juju/worker/peergrouper.TestPackage()
      /home/ubuntu/juju-core_2.0-beta5/src/github.com/juju/juju/worker/peergrouper/suite_test.go:13 +0x34
  testing.tRunner()
      /usr/lib/go/src/pkg/testing/testing.go:391 +0x10f

Goroutine 10 (running) cre...

Read more...

Changed in juju-core:
importance: Medium → Critical
Menno Finlay-Smits (menno.smits) on 2016-04-20
Changed in juju-core:
assignee: nobody → Menno Smits (menno.smits)
Curtis Hovey (sinzui) wrote :

The race tests are run in lxc, which seems to expose raciness.

Curtis Hovey (sinzui) on 2016-04-20
tags: added: regression
tags: added: ci
Menno Finlay-Smits (menno.smits) on 2016-04-20
Changed in juju-core:
status: Triaged → In Progress
Menno Finlay-Smits (menno.smits) wrote :

Fix is here: https://github.com/juju/juju/pull/5225

Menno Finlay-Smits (menno.smits) on 2016-04-20
Changed in juju-core:
status: In Progress → Fix Committed
Menno Finlay-Smits (menno.smits) wrote :

The runs of HA related CI jobs have been clean with respect to the peergrouper issues since the fix was merged. I'll leave the bug open for a while longer until I have more confidence that the issue has been resolved but I'm cautiously optimistic.

Menno Finlay-Smits (menno.smits) wrote :

The HA related CI runs of master have been clean so far, but the add-models feature branch just had a voting enable failure and that branch has the most recent peergrouper improvments.

http://juju-ci.vapour.ws/job/functional-ha-backup-restore/3649/

I will keep looking at this.

Changed in juju-core:
status: Fix Committed → In Progress
Menno Finlay-Smits (menno.smits) wrote :

Gah! The last 2 comments should have been on bug 1453805.

This one is actually fixed.

Changed in juju-core:
status: In Progress → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.
Subscribing...

Other bug subscribers