provisioning agent gets confused when machines are terminated

[Clint Byrum]

Have noticed this that sometimes when I terminate a machine the provisioning agent starts repeatedly logging this in debug-log:

2011-09-28 19:53:38,420 provision:ec2: juju.agents.provision ERROR: Cannot process machine 6
Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.7/juju/agents/provision.py", line 192, in process_machines
    machine_state_id, provider_machines)
  File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1018, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/usr/lib/python2.7/dist-packages/twisted/python/failure.py", line 350, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/usr/lib/pymodules/python2.7/juju/agents/provision.py", line 227, in process_machine
    machine_state_id)
  File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1020, in _inlineCallbacks
    result = g.send(result)
  File "/usr/lib/pymodules/python2.7/juju/state/machine.py", line 83, in get_machine_state
    raise MachineStateNotFound(machine_id)
MachineStateNotFound: Machine 6 was not found

This is on the latest packages from PPA

$ apt-cache policy juju
juju:
  Installed: 0.5+bzr363-1juju1~oneiric1
  Candidate: 0.5+bzr363-1juju1~oneiric1
  Version table:
 *** 0.5+bzr363-1juju1~oneiric1 0
        500 http://ppa.launchpad.net/juju/pkgs/ubuntu/ oneiric/main amd64 Packages
        100 /var/lib/dpkg/status

[Kapil Thangavelu (hazmat)]

Terminate a machine how?.. with the juju remove-machine api.. or ec2
destroy-instance api ?

Excerpts from Clint Byrum's message of Wed Sep 28 23:59:41 UTC 2011:
> Public bug reported:
>
> Have noticed this that sometimes when I terminate a machine the
> provisioning agent starts repeatedly logging this in debug-log:
>

[Clint Byrum (clint-fewbar)]

Excerpts from Kapil Thangavelu's message of Thu Sep 29 01:14:02 UTC 2011:
> Terminate a machine how?.. with the juju remove-machine api.. or ec2
> destroy-instance api ?

Terminated with

juju terminate-machine 6

After having destroyed the services that were deployed on it.

[Clint Byrum (clint-fewbar)]

Note that this seems to be triggered by bug 862595 , but I'm not convinced that the two are the same bug.

[Kapil Thangavelu (hazmat)]

If anyone is able to reproduce this, it would be helpful to debugging and fixing this to have the entire provisioning agent log. I've tried a few times to reproduce without success, and i suspect it may have some timing dependency to reproduce.

[Clint Byrum (clint-fewbar)]

http://www.outflux.net/blog/archives/2011/12/05/ec2-instances-in-support-of-a-bsp/

I'm going to move this as affecting juju in oneiric only, since Kees was unable to reproduce once he started using juju from the PPA.

http://www.outflux.net/blog/archives/2011/12/07/juju-bug-fixing/

[Clint Byrum (clint-fewbar)]

Removing Jim. I don't think this is reproducible, and may be a duplicate of some other bug we already fixed since r398.

[Rolf Leggewie (r0lf)]

oneiric has seen the end of its life and is no longer receiving any updates. Marking the oneiric task for this ticket as "Won't Fix".