[2.0a2] Deploying machine with primary rack dead fails

Bug #1558755 reported by Blake Rouse
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
MAAS
Fix Released
Critical
Blake Rouse

Bug Description

Now PXE booting works but the preseed generation fails when the primary_rack controller is down.

Looking at the stack trace below it is trying to perform RPC to maas-2 (4y3h7p), when the rack controller it actually booted from was the secondary rack controller maas-1 (4y3h7n).

Mar 17 19:24:04 localhost sh[13986]: 2016-03-17 19:24:04 [-] 127.0.0.1 - - [17/Mar/2016:19:24:03 +0000] "GET /MAAS/metadata/latest/by-id/4y3h7t/?op=get_preseed HTTP/1.1" 500 70 "-" "Cloud-Init/0.7.7"
Mar 17 19:24:05 localhost sh[13986]: 2016-03-17 19:24:05 [maasserver] ERROR: ################################ Exception: Unable to connect to rack controller 4y3h7p; no connections available. ################################
Mar 17 19:24:05 localhost sh[13986]: 2016-03-17 19:24:05 [maasserver] ERROR: Traceback (most recent call last):
Mar 17 19:24:05 localhost sh[13986]: File "/usr/lib/python3/dist-packages/django/core/handlers/base.py", line 132, in get_response
Mar 17 19:24:05 localhost sh[13986]: response = wrapped_callback(request, *callback_args, **callback_kwargs)
Mar 17 19:24:05 localhost sh[13986]: File "/usr/lib/python3/dist-packages/maasserver/utils/views.py", line 180, in view_atomic_with_post_commit_savepoint
Mar 17 19:24:05 localhost sh[13986]: return view_atomic(*args, **kwargs)
Mar 17 19:24:05 localhost sh[13986]: File "/usr/lib/python3.5/contextlib.py", line 30, in inner
Mar 17 19:24:05 localhost sh[13986]: return func(*args, **kwds)
Mar 17 19:24:05 localhost sh[13986]: File "/usr/lib/python3/dist-packages/maasserver/api/support.py", line 54, in __call__
Mar 17 19:24:05 localhost sh[13986]: response = upcall(request, *args, **kwargs)
Mar 17 19:24:05 localhost sh[13986]: File "/usr/lib/python3/dist-packages/django/views/decorators/vary.py", line 21, in inner_func
Mar 17 19:24:05 localhost sh[13986]: response = func(*args, **kwargs)
Mar 17 19:24:05 localhost sh[13986]: File "/usr/lib/python3/dist-packages/piston3/resource.py", line 190, in __call__
Mar 17 19:24:05 localhost sh[13986]: result = self.error_handler(e, request, meth, em_format)
Mar 17 19:24:05 localhost sh[13986]: File "/usr/lib/python3/dist-packages/piston3/resource.py", line 188, in __call__
Mar 17 19:24:05 localhost sh[13986]: result = meth(request, *args, **kwargs)
Mar 17 19:24:05 localhost sh[13986]: File "/usr/lib/python3/dist-packages/maasserver/api/support.py", line 210, in dispatch
Mar 17 19:24:05 localhost sh[13986]: return function(self, request, *args, **kwargs)
Mar 17 19:24:05 localhost sh[13986]: File "/usr/lib/python3/dist-packages/metadataserver/api.py", line 768, in get_preseed
Mar 17 19:24:05 localhost sh[13986]: return HttpResponse(get_preseed(node), content_type="text/plain")
Mar 17 19:24:05 localhost sh[13986]: File "/usr/lib/python3/dist-packages/maasserver/preseed.py", line 397, in get_preseed
Mar 17 19:24:05 localhost sh[13986]: osystem=node.get_osystem(), release=node.get_distro_series())
Mar 17 19:24:05 localhost sh[13986]: File "/usr/lib/python3/dist-packages/maasserver/preseed.py", line 680, in render_preseed
Mar 17 19:24:05 localhost sh[13986]: node, osystem, release, rack_controller=rack_controller))
Mar 17 19:24:05 localhost sh[13986]: File "/usr/lib/python3/dist-packages/maasserver/preseed.py", line 637, in get_node_preseed_context
Mar 17 19:24:05 localhost sh[13986]: 'preseed_data': compose_preseed(get_preseed_type_for(node), node),
Mar 17 19:24:05 localhost sh[13986]: File "/usr/lib/python3/dist-packages/maasserver/compose_preseed.py", line 236, in compose_preseed
Mar 17 19:24:05 localhost sh[13986]: return get_preseed_data(preseed_type, node, token, metadata_url)
Mar 17 19:24:05 localhost sh[13986]: File "/usr/lib/python3/dist-packages/provisioningserver/utils/twisted.py", line 200, in wrapper
Mar 17 19:24:05 localhost sh[13986]: return func(*args, **kwargs)
Mar 17 19:24:05 localhost sh[13986]: File "/usr/lib/python3/dist-packages/maasserver/clusterrpc/osystems.py", line 118, in get_preseed_data
Mar 17 19:24:05 localhost sh[13986]: client = getClientFor(node.get_boot_primary_rack_controller().system_id)
Mar 17 19:24:05 localhost sh[13986]: File "/usr/lib/python3/dist-packages/provisioningserver/utils/twisted.py", line 144, in wrapper
Mar 17 19:24:05 localhost sh[13986]: return func_in_reactor(*args, **kwargs).wait(LONGTIME)
Mar 17 19:24:05 localhost sh[13986]: File "/usr/lib/python3/dist-packages/crochet/_eventloop.py", line 231, in wait
Mar 17 19:24:05 localhost sh[13986]: result.raiseException()
Mar 17 19:24:05 localhost sh[13986]: File "/usr/lib/python3/dist-packages/twisted/python/failure.py", line 368, in raiseException
Mar 17 19:24:05 localhost sh[13986]: raise self.value.with_traceback(self.tb)
Mar 17 19:24:05 localhost sh[13986]: File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 588, in _runCallbacks
Mar 17 19:24:05 localhost sh[13986]: current.result = callback(current.result, *args, **kw)
Mar 17 19:24:05 localhost sh[13986]: File "/usr/lib/python3/dist-packages/maasserver/rpc/regionservice.py", line 843, in cancelled
Mar 17 19:24:05 localhost sh[13986]: "available." % system_id, uuid=system_id)
Mar 17 19:24:05 localhost sh[13986]: provisioningserver.rpc.exceptions.NoConnectionsAvailable: Unable to connect to rack controller 4y3h7p; no connections available.

Related branches

description: updated
Changed in maas:
status: In Progress → Fix Committed
Changed in maas:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.