Unable to create Node: timed out

[Daniel]

Running

Description: Ubuntu 12.04 LTS
Release: 12.04

maas:
  Installed: 0.1+bzr482+dfsg-0ubuntu1
  Candidate: 0.1+bzr482+dfsg-0ubuntu1
  Version table:
 *** 0.1+bzr482+dfsg-0ubuntu1 0
        500 http://us.archive.ubuntu.com/ubuntu/ precise/main amd64 Packages
        100 /var/lib/dpkg/status

When I trying to add a node (the first one) on the maas web interface. The error: Unable to create Node: timed out shows up.

But, when I use the command cobbler system list the node was created.

/var/log/cobbler/cobbler.log

Thu Apr 26 21:41:10 2012 - DEBUG | authorize; ['maas', 'xedit_system', 'SCpkM/PnUKq5QwY3XqXmJX8FuEHKbZhQtQ==', None, True]
Thu Apr 26 21:41:10 2012 - DEBUG | REMOTE maas authorization result: True; user(?)
Thu Apr 26 21:41:10 2012 - DEBUG | get_item; ['system', 'node-10a9da2a-900a-11e1-8ed2-0050568b4c67']
Thu Apr 26 21:41:10 2012 - DEBUG | done with get_item; ['system', 'node-10a9da2a-900a-11e1-8ed2-0050568b4c67']
Thu Apr 26 21:41:10 2012 - INFO | Exception occured: <class 'cobbler.cexceptions.CX'>
Thu Apr 26 21:41:10 2012 - INFO | Exception value: 'internal error, unknown system name node-10a9da2a-900a-11e1-8ed2-0050568b4c67'
Thu Apr 26 21:41:10 2012 - INFO | Exception Info:
  File "/usr/lib/python2.7/dist-packages/cobbler/remote.py", line 1944, in _dispatch
    return method_handle(*params)
   File "/usr/lib/python2.7/dist-packages/cobbler/remote.py", line 873, in xapi_object_edit
    handle = self.get_item_handle(object_type, object_name)
   File "/usr/lib/python2.7/dist-packages/cobbler/remote.py", line 655, in get_item_handle
    raise CX("internal error, unknown %s name %s" % (what,name))

Thu Apr 26 21:41:10 2012 - DEBUG | authorize; ['maas', 'xedit_system', 'SCpkM/PnUKq5QwY3XqXmJX8FuEHKbZhQtQ==', None, True]
Thu Apr 26 21:41:10 2012 - DEBUG | REMOTE maas authorization result: True; user(?)
Thu Apr 26 21:41:10 2012 - DEBUG | get_item; ['system', 'node-10a9da2a-900a-11e1-8ed2-0050568b4c67']
Thu Apr 26 21:41:10 2012 - DEBUG | done with get_item; ['system', 'node-10a9da2a-900a-11e1-8ed2-0050568b4c67']
Thu Apr 26 21:41:10 2012 - INFO | Exception occured: <class 'cobbler.cexceptions.CX'>
Thu Apr 26 21:41:10 2012 - INFO | Exception value: 'internal error, unknown system name node-10a9da2a-900a-11e1-8ed2-0050568b4c67'
Thu Apr 26 21:41:10 2012 - INFO | Exception Info:
  File "/usr/lib/python2.7/dist-packages/cobbler/remote.py", line 862, in xapi_object_edit
    handle = self.get_item_handle(object_type, object_name)
   File "/usr/lib/python2.7/dist-packages/cobbler/remote.py", line 655, in get_item_handle
    raise CX("internal error, unknown %s name %s" % (what,name))

Thu Apr 26 21:41:10 2012 - INFO | REMOTE new_item(system); user(maas)
Thu Apr 26 21:41:10 2012 - DEBUG | authorize; ['maas', 'new_system', None, None, True]
Thu Apr 26 21:41:10 2012 - DEBUG | REMOTE maas authorization result: True; user(?)
Thu Apr 26 21:41:10 2012 - INFO | REMOTE modify_item(system); user(maas); object_id(___NEW___system::NjdLyADncRBJoEWN9bo9nrUYc6V12zkzyg==); attribute(profile)
Thu Apr 26 21:41:10 2012 - DEBUG | authorize; ['maas', 'modify_system', <cobbler.item_system.System instance at 0x18b7ea8>, 'profile', True]
Thu Apr 26 21:41:10 2012 - DEBUG | REMOTE maas authorization result: True; user(?)
Thu Apr 26 21:41:10 2012 - INFO | REMOTE modify_item(system); user(maas); object_id(___NEW___system::NjdLyADncRBJoEWN9bo9nrUYc6V12zkzyg==); attribute(hostname)
Thu Apr 26 21:41:10 2012 - DEBUG | authorize; ['maas', 'modify_system', <cobbler.item_system.System instance at 0x18b7ea8>, 'hostname', True]
Thu Apr 26 21:41:10 2012 - DEBUG | REMOTE maas authorization result: True; user(?)
Thu Apr 26 21:41:10 2012 - INFO | REMOTE modify_item(system); user(maas); object_id(___NEW___system::NjdLyADncRBJoEWN9bo9nrUYc6V12zkzyg==); attribute(power_type)
Thu Apr 26 21:41:10 2012 - DEBUG | authorize; ['maas', 'modify_system', <cobbler.item_system.System instance at 0x18b7ea8>, 'power_type', True]
Thu Apr 26 21:41:10 2012 - DEBUG | REMOTE maas authorization result: True; user(?)
Thu Apr 26 21:41:10 2012 - INFO | REMOTE modify_item(system); user(maas); object_id(___NEW___system::NjdLyADncRBJoEWN9bo9nrUYc6V12zkzyg==); attribute(ks_meta)
Thu Apr 26 21:41:10 2012 - DEBUG | authorize; ['maas', 'modify_system', <cobbler.item_system.System instance at 0x18b7ea8>, 'ks_meta', True]
Thu Apr 26 21:41:10 2012 - DEBUG | REMOTE maas authorization result: True; user(?)
Thu Apr 26 21:41:10 2012 - INFO | REMOTE modify_item(system); user(maas); object_id(___NEW___system::NjdLyADncRBJoEWN9bo9nrUYc6V12zkzyg==); attribute(name)
Thu Apr 26 21:41:10 2012 - DEBUG | authorize; ['maas', 'modify_system', <cobbler.item_system.System instance at 0x18b7ea8>, 'name', True]
Thu Apr 26 21:41:10 2012 - DEBUG | REMOTE maas authorization result: True; user(?)
Thu Apr 26 21:41:10 2012 - INFO | REMOTE modify_item(system); user(maas); object_id(___NEW___system::NjdLyADncRBJoEWN9bo9nrUYc6V12zkzyg==); attribute(modify_interface)
Thu Apr 26 21:41:10 2012 - DEBUG | authorize; ['maas', 'modify_system', <cobbler.item_system.System instance at 0x18b7ea8>, 'modify_interface', True]
Thu Apr 26 21:41:10 2012 - DEBUG | REMOTE maas authorization result: True; user(?)
Thu Apr 26 21:41:10 2012 - INFO | REMOTE save_item(system); user(maas); object_id(___NEW___system::NjdLyADncRBJoEWN9bo9nrUYc6V12zkzyg==)
Thu Apr 26 21:41:10 2012 - DEBUG | authorize; ['maas', 'save_system', <cobbler.item_system.System instance at 0x18b7ea8>, None, True]
Thu Apr 26 21:41:10 2012 - DEBUG | REMOTE maas authorization result: True; user(?)
Thu Apr 26 21:41:10 2012 - INFO | add_item(system); ['node-10a9da2a-900a-11e1-8ed2-0050568b4c67']
Thu Apr 26 21:41:10 2012 - DEBUG | get_items; ['system']
Thu Apr 26 21:41:10 2012 - DEBUG | done with get_items; ['system']
Thu Apr 26 21:41:10 2012 - INFO | REMOTE sync; user(maas)
Thu Apr 26 21:41:10 2012 - DEBUG | authorize; ['maas', 'sync', None, None, True]
Thu Apr 26 21:41:10 2012 - DEBUG | REMOTE maas authorization result: True; user(?)
Thu Apr 26 21:41:10 2012 - INFO | sync
Thu Apr 26 21:41:10 2012 - INFO | running pre-sync triggers
Thu Apr 26 21:41:10 2012 - INFO | cleaning trees
Thu Apr 26 21:41:10 2012 - INFO | removing: /var/www/cobbler/images/precise-x86_64
Thu Apr 26 21:41:10 2012 - INFO | removing: /var/www/cobbler/images/precise-i386
Thu Apr 26 21:41:10 2012 - INFO | removing: /var/www/cobbler/images/precise-i386-maas-ephemeral
Thu Apr 26 21:41:10 2012 - INFO | removing: /var/www/cobbler/images/precise-x86_64-maas-ephemeral
Thu Apr 26 21:41:10 2012 - INFO | removing: /var/lib/tftpboot/pxelinux.cfg/default
Thu Apr 26 21:41:10 2012 - INFO | removing: /var/lib/tftpboot/grub/images
Thu Apr 26 21:41:10 2012 - INFO | removing: /var/lib/tftpboot/grub/efidefault
Thu Apr 26 21:41:10 2012 - INFO | removing: /var/lib/tftpboot/images/precise-x86_64
Thu Apr 26 21:41:10 2012 - INFO | removing: /var/lib/tftpboot/images/precise-i386
Thu Apr 26 21:41:10 2012 - INFO | removing: /var/lib/tftpboot/images/memtest86+_multiboot.bin
Thu Apr 26 21:41:10 2012 - INFO | removing: /var/lib/tftpboot/images/precise-i386-maas-ephemeral
Thu Apr 26 21:41:10 2012 - INFO | removing: /var/lib/tftpboot/images/precise-x86_64-maas-ephemeral
Thu Apr 26 21:41:10 2012 - INFO | removing: /var/lib/tftpboot/images/memtest86+.bin
Thu Apr 26 21:41:10 2012 - INFO | removing: /var/lib/tftpboot/s390x/profile_list
Thu Apr 26 21:41:10 2012 - INFO | copying bootloaders
Thu Apr 26 21:41:10 2012 - INFO | copying: /boot/memtest86+_multiboot.bin -> /var/lib/tftpboot/images/memtest86+_multiboot.bin
Thu Apr 26 21:41:10 2012 - INFO | copying: /boot/memtest86+.bin -> /var/lib/tftpboot/images/memtest86+.bin
Thu Apr 26 21:41:10 2012 - INFO | copying distros to tftpboot
Thu Apr 26 21:41:10 2012 - INFO | copying files for distro: precise-x86_64
Thu Apr 26 21:41:10 2012 - INFO | trying hardlink /var/www/cobbler/ks_mirror/precise-x86_64/linux -> /var/lib/tftpboot/images/precise-x86_64/linux
Thu Apr 26 21:41:10 2012 - INFO | trying hardlink /var/www/cobbler/ks_mirror/precise-x86_64/initrd.gz -> /var/lib/tftpboot/images/precise-x86_64/initrd.gz
Thu Apr 26 21:41:10 2012 - INFO | copying files for distro: precise-i386
Thu Apr 26 21:41:10 2012 - INFO | trying hardlink /var/www/cobbler/ks_mirror/precise-i386/linux -> /var/lib/tftpboot/images/precise-i386/linux
Thu Apr 26 21:41:10 2012 - INFO | trying hardlink /var/www/cobbler/ks_mirror/precise-i386/initrd.gz -> /var/lib/tftpboot/images/precise-i386/initrd.gz
Thu Apr 26 21:41:10 2012 - INFO | copying files for distro: precise-i386-maas-ephemeral
Thu Apr 26 21:41:10 2012 - INFO | trying hardlink /var/lib/maas/ephemeral/precise/ephemeral/i386/20120418/kernel -> /var/lib/tftpboot/images/precise-i386-maas-ephemeral/kernel
Thu Apr 26 21:41:10 2012 - INFO | trying hardlink /var/lib/maas/ephemeral/precise/ephemeral/i386/20120418/initrd -> /var/lib/tftpboot/images/precise-i386-maas-ephemeral/initrd
Thu Apr 26 21:41:10 2012 - INFO | copying files for distro: precise-x86_64-maas-ephemeral
Thu Apr 26 21:41:10 2012 - INFO | trying hardlink /var/lib/maas/ephemeral/precise/ephemeral/amd64/20120418/kernel -> /var/lib/tftpboot/images/precise-x86_64-maas-ephemeral/kernel
Thu Apr 26 21:41:10 2012 - INFO | trying hardlink /var/lib/maas/ephemeral/precise/ephemeral/amd64/20120418/initrd -> /var/lib/tftpboot/images/precise-x86_64-maas-ephemeral/initrd
Thu Apr 26 21:41:10 2012 - INFO | copying images
Thu Apr 26 21:41:10 2012 - INFO | generating GPXE/PXE configuration files
Thu Apr 26 21:41:10 2012 - INFO | generating PXE menu structure
Thu Apr 26 21:41:10 2012 - WARNING | warning: kernel option length exceeds 255
Thu Apr 26 21:41:10 2012 - WARNING | warning: kernel option length exceeds 255
Thu Apr 26 21:41:10 2012 - WARNING | warning: kernel option length exceeds 255
Thu Apr 26 21:41:10 2012 - WARNING | warning: kernel option length exceeds 255
Thu Apr 26 21:41:10 2012 - INFO | copying files for distro: precise-x86_64
Thu Apr 26 21:41:10 2012 - INFO | trying hardlink /var/www/cobbler/ks_mirror/precise-x86_64/linux -> /var/www/cobbler/images/precise-x86_64/linux
Thu Apr 26 21:41:10 2012 - INFO | trying hardlink /var/www/cobbler/ks_mirror/precise-x86_64/initrd.gz -> /var/www/cobbler/images/precise-x86_64/initrd.gz
Thu Apr 26 21:41:10 2012 - INFO | copying files for distro: precise-i386
Thu Apr 26 21:41:10 2012 - INFO | trying hardlink /var/www/cobbler/ks_mirror/precise-i386/linux -> /var/www/cobbler/images/precise-i386/linux
Thu Apr 26 21:41:10 2012 - INFO | trying hardlink /var/www/cobbler/ks_mirror/precise-i386/initrd.gz -> /var/www/cobbler/images/precise-i386/initrd.gz
Thu Apr 26 21:41:10 2012 - INFO | copying files for distro: precise-i386-maas-ephemeral
Thu Apr 26 21:41:10 2012 - INFO | trying hardlink /var/lib/maas/ephemeral/precise/ephemeral/i386/20120418/kernel -> /var/www/cobbler/images/precise-i386-maas-ephemeral/kernel
Thu Apr 26 21:41:10 2012 - INFO | trying hardlink /var/lib/maas/ephemeral/precise/ephemeral/i386/20120418/initrd -> /var/www/cobbler/images/precise-i386-maas-ephemeral/initrd
Thu Apr 26 21:41:10 2012 - INFO | copying files for distro: precise-x86_64-maas-ephemeral
Thu Apr 26 21:41:10 2012 - INFO | trying hardlink /var/lib/maas/ephemeral/precise/ephemeral/amd64/20120418/kernel -> /var/www/cobbler/images/precise-x86_64-maas-ephemeral/kernel
Thu Apr 26 21:41:10 2012 - INFO | trying hardlink /var/lib/maas/ephemeral/precise/ephemeral/amd64/20120418/initrd -> /var/www/cobbler/images/precise-x86_64-maas-ephemeral/initrd
Thu Apr 26 21:41:10 2012 - WARNING | warning: kernel option length exceeds 255
Thu Apr 26 21:41:10 2012 - WARNING | warning: kernel option length exceeds 255
Thu Apr 26 21:41:10 2012 - WARNING | warning: kernel option length exceeds 255
Thu Apr 26 21:41:10 2012 - WARNING | warning: kernel option length exceeds 255
Thu Apr 26 21:41:10 2012 - INFO | rendering DHCP files
Thu Apr 26 21:41:10 2012 - INFO | rendering DNS files
Thu Apr 26 21:41:10 2012 - INFO | rendering TFTPD files
Thu Apr 26 21:41:10 2012 - INFO | generating /etc/xinetd.d/tftp
Thu Apr 26 21:41:10 2012 - INFO | cleaning link caches
Thu Apr 26 21:41:10 2012 - INFO | running post-sync triggers
Thu Apr 26 21:41:10 2012 - DEBUG | running python triggers from /var/lib/cobbler/triggers/sync/post/*
Thu Apr 26 21:41:10 2012 - DEBUG | running python trigger cobbler.modules.sync_post_restart_services
Thu Apr 26 21:41:10 2012 - INFO | running: service dnsmasq restart
Thu Apr 26 21:41:20 2012 - INFO | received on stdout: * Restarting DNS forwarder and DHCP server dnsmasq
   ...done.

Thu Apr 26 21:41:20 2012 - DEBUG | received on stderr:
Thu Apr 26 21:41:20 2012 - DEBUG | running shell triggers from /var/lib/cobbler/triggers/sync/post/*
Thu Apr 26 21:41:20 2012 - DEBUG | running python triggers from /var/lib/cobbler/triggers/change/*
Thu Apr 26 21:41:20 2012 - DEBUG | running python trigger cobbler.modules.scm_track
Thu Apr 26 21:41:20 2012 - DEBUG | running shell triggers from /var/lib/cobbler/triggers/change/*

/var/log/maas/maas.log

ERROR 2012-04-26 21:41:17,663 maas.maasserver Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/django/core/handlers/base.py", line 111, in get_response
    response = callback(request, *callback_args, **callback_kwargs)
  File "/usr/lib/python2.7/dist-packages/django/views/decorators/vary.py", line 22, in inner_func
    response = func(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/piston/resource.py", line 166, in __call__
    result = self.error_handler(e, request, meth, em_format)
  File "/usr/lib/python2.7/dist-packages/piston/resource.py", line 164, in __call__
    result = meth(request, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/maasserver/api.py", line 251, in dispatcher
    self, request, request.method, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/maasserver/api.py", line 193, in perform_api_operation
    return method(handler, request, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/maasserver/api.py", line 493, in new
    node = create_node(request)
  File "/usr/lib/python2.7/dist-packages/maasserver/api.py", line 418, in create_node
    return form.save()
  File "/usr/lib/python2.7/dist-packages/maasserver/forms.py", line 234, in save
    node = super(NodeWithMACAddressesForm, self).save()
  File "/usr/lib/python2.7/dist-packages/django/forms/models.py", line 363, in save
    fail_message, commit, construct=False)
  File "/usr/lib/python2.7/dist-packages/django/forms/models.py", line 85, in save_instance
    instance.save()
  File "/usr/lib/python2.7/dist-packages/maasserver/models.py", line 114, in save
    return super(CommonInfo, self).save(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/django/db/models/base.py", line 460, in save
    self.save_base(using=using, force_insert=force_insert, force_update=force_update)
  File "/usr/lib/python2.7/dist-packages/django/db/models/base.py", line 570, in save_base
    created=(not record_exists), raw=raw, using=using)
  File "/usr/lib/python2.7/dist-packages/django/dispatch/dispatcher.py", line 172, in send
    response = receiver(signal=self, sender=sender, **named)
  File "/usr/lib/python2.7/dist-packages/maasserver/provisioning.py", line 485, in provision_post_save_Node
    profile, power_type, preseed_data)
  File "/usr/lib/python2.7/dist-packages/maasserver/provisioning.py", line 245, in __call__
    result = self.method(*args)
  File "/usr/lib/python2.7/xmlrpclib.py", line 1224, in __call__
    return self.__send(self.__name, args)
  File "/usr/lib/python2.7/xmlrpclib.py", line 1578, in __request
    verbose=self.__verbose
  File "/usr/lib/python2.7/xmlrpclib.py", line 1264, in request
    return self.single_request(host, handler, request_body, verbose)
  File "/usr/lib/python2.7/xmlrpclib.py", line 1294, in single_request
    response = h.getresponse(buffering=True)
  File "/usr/lib/python2.7/httplib.py", line 1030, in getresponse
    response.begin()
  File "/usr/lib/python2.7/httplib.py", line 407, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python2.7/httplib.py", line 365, in _read_status
    line = self.fp.readline()
  File "/usr/lib/python2.7/socket.py", line 447, in readline
    data = self._sock.recv(self._rbufsize)
timeout: timed out

[Julian Edwards (julian-edwards)]

The problem is that cobbler took 10 seconds to restart dnsmasq:
Thu Apr 26 21:41:10 2012 - INFO | running: service dnsmasq restart
Thu Apr 26 21:41:20 2012 - INFO | received on stdout: * Restarting DNS forwarder and DHCP server dnsmasq

But the MAAS server has a 7 second timeout waiting for pserv/cobbler operations.

Does re-adding the node work quicker the second time?

[Daniel (danielfm)]

Same behaviour occurs when I tried to re-adding the node.
How can I speed things up? The dnsmasq is working well by doing dhcp and resolving names.

[Julian Edwards (julian-edwards)]

I'm stumped as to why it takes so long, I've never seen that before. I'll get someone else to take a look at this.

[Mark Furneaux (markfurneaux)]

Same problem here. I still cannot add my first node.

[Julian Edwards (julian-edwards)]

Mark, do you see the same delay in the cobbler log when it's restarting dnsmasq?

Everyone: is anything else running on your machine that would slow it all down? maas-import-isos?

On the bright side, we are going to re-architect this to make it asynchronous so the web app doesn't need to wait.

[Daniel (danielfm)]

Julian, this is a fresh install maas-servers only. I reinstalled the server and I succesfully added the first node, but when tried the next one, again the time out error. I do this *after* the maas-import-isos.

[Mark Furneaux (markfurneaux)]

Julian, I see the same delay in the cobbler log, however it is 27 seconds in my case. There is nothing else running on the system and I have already run maas-import-isos. I am running this in a VM at the moment to experiment before I do a real hardware install. The VM has access to 8 cores and the host OS is idling. According to the cobbler command, the node is still added though despite the error.

[Julian Edwards (julian-edwards)]

So cobbler is doing something weird, I will have to punt to someone who knows it better than I.

[Julian Edwards (julian-edwards)]

I thought of something you can try - if you change the cobbler settings to not manage dhcp and set your external dhcp up manually, it might work. Please let me know.

[Sebastian Malcolm (smalcolm)]

Thanks Julian for giving us some insight into the cause of this issue! If you have some insight into the command sequence MAAS is running then I hope you can help identify a workaround to enable adding nodes or create a mini-Howto manually remove a partially added node (from Cobbler, dnsmasq, etc managed by MAAS).

> But the MAAS server has a 7 second timeout waiting for pserv/cobbler operations.
Can we change this timeout as a workaround for this issue?

> Does re-adding the node work quicker the second time?
How to do this if when the MAAS "Add Node" process is incomplete but did define a system in Cobbler so the same node cannot be re-added in MAAS web UI?

In the couple of "timed out" failed adding of nodes that I've seen, the MAAS web UI doesn't allow re-adding or appear to have an option to re-try. Just naively re-entering the MAC address fails and I recall Cobbler logging "duplicate MAC address" or similar error after defining a duplicate incomplete system record (missing a mac-address field value).

I've tried re-adding after finding the newly added node's name from the output of "cobbler system list" then remove it with "cobbler system remove --name=<NewNodeName>".

[Julian Edwards (julian-edwards)]

I'm sorry this is troublesome for you. I don't think we ever tested it with having cobbler manage DHCP or dnsmasq, we just expected people to set next-server on the existing DHCP.

For timeouts, there's a setting called PSERV_TIMEOUT which is currently set to 7, try changing that and let me know.

[Sebastian Malcolm (smalcolm)]

To find some way to get MAAS to wait longer, I I initially looked in "/usr/share/pyshared/cobbler/modules/sync_post_restart_services.py" and found code that looks like it uses subprocess_sp() from "/usr/share/pyshared/cobbler/utils.py" to run "service dnsmasq restart". Perhaps the code for restarting services could print out a progress bar or message like "will wait PSERV_TIMEOUT seconds for service to restart".

I found PSERV_TIMEOUT in /usr/share/maas/maas/settings.py then added the 2 lines below in /etc/maas/maas_local_settings.py that should override the 7.0 seconds default value:

# Time-out for socket operations against the Provisioning API.
PSERV_TIMEOUT = 60.0 # seconds.

== 5 step mini-HOWTO for fixing an incomplete "Add Node" operation ==
(1) Remove node added to Cobbler as a System, but not visible in MAAS Web UI
# cobbler system remove --name=node-e59f0a74-a0a5-11e1-86c0-00187184baf2

(2) Removed any existing DHCP leases for this node that didn't finish getting added to MAAS
[The node I was getting "timed out" error on was booted into an existing Ubuntu install and getting a DHCP lease, so i unplugged it's network cable before attempting to re-add it to MAAS]
# service dnsmasq stop
# vi /var/lib/misc/dnsmasq.leases
# service dnsmasq start

(3) Check node removed from cobbler
[The two node-.... entries below are two offline machines I had successfully added before PXE booting them]
# cobbler system list
   default
   node-9c4d6890-9fe5-11e1-8148-00187184baf2
   node-dd577d68-a098-11e1-86c0-00187184baf2

(4) Manually restart dnsmasq & cobbler
# service dnsmasq restart
# service cobbler restart

(5) Test MAAS / Retry adding the MAC for the node
I had two nodes added (in Offline/Commissioning state), so to check things were working I powered one on (manually) and it successfully PXE booted from the MAAS server then powered itself off and entered the Queued/Ready state. I retried again to add the problematic node... and this time it worked!

One or more of these things I did may have "fixed" the Add Node timed out problem:
(a) increase PSERV_TIMEOUT in /etc/maas/maas_local_settings.py,; or
(b) unplug network cable of this machine that was already powered on & running Ubuntu desktop & getting DHCP lease from dnsmasq; or
(c) also removing the existing lease line for that machine from "/var/lib/misc/dnsmasq.leases" file (after stopping dnsmasq).

TODO: Test if this Add Node timed out problem only occurs when attempting to add a machine that has obtained a DHCP lease from dnsmasq (because it powered online with an existing OS installed). Could also try enlisting with the boot menu option from Ubuntu Server 12.04 from CDROM/USB instead of manually adding the MAC address.

[Julian Edwards (julian-edwards)]

This is fixed in trunk with the removal of Cobbler

[Andres Rodriguez (andreserl)]

Thank you for taking the time to report this bug.

MAAS in precise (-updates) no longer uses cobbler, for this reason I'm marking this bug as won't fix. If you feel this bug is still present, please re-open the bug.