Comment 25 for bug 1639202

Revision history for this message
Spyderdyne (spyderdyne) wrote :

This is happening to me on an Intel NUC 5i5MYHE blade on PXE. The error is an iSCSI destination error apparently. Same scenario...

Maas version: MAAS Version 2.1.3+bzr5573-0ubuntu1 (16.04.1)

On node PXE boot console output (sorry, but some of this is cut off on this screen, typing what I can see. will switch to another monitor and reproduce if this is not enough of the error):

lvmetad is not active yet, using sysinit

disk/by-path/ip-192.168.199.2:3260-iscsi-iqn.2004-05.com.ubuntu:maas:ephemeral...
...ubuntu-amd64-generic-xenial-daily-lun-1"

tgtd on the MaaS node seems to be running fine but netstat output does not show the initiator contacting it when I search for the IP that MaaS assigned on PXE init.

tgtd MaaS node is Raspberry Pi 3B, so not a Dell issue specifically:

root@juju-rack2:~# cat /etc/issue
Ubuntu 16.04.1 LTS \n \l

root@juju-rack2:~# uname -r
4.4.43-v7+

MAAS Version 2.1.3+bzr5573-0ubuntu1 (16.04.1)

Intel blade is also running PCIe solid state drive, not a USB device.

Fails about every 60 seconds according to maas.log:

"Feb 8 21:47:17 juju-rack2 maas.service_monitor: [error] While monitoring service 'tgt' an error was encountered: Unable to parse the active state from systemd for service 'tgt', active state reported as 'deactivating'.
Feb 8 21:48:18 juju-rack2 maas.service_monitor: [error] While monitoring service 'tgt' an error was encountered: Unable to parse the active state from systemd for service 'tgt', active state reported as 'deactivating'.
Feb 8 21:49:17 juju-rack2 maas.service_monitor: [error] While monitoring service 'tgt' an error was encountered: Unable to parse the active state from systemd for service 'tgt', active state reported as 'deactivating'."

/var/log/rackd/log output:

2017-02-08 21:32:23 sstreams: [info] maas:v2:download/maas:boot:ubuntu:amd64:hwe-16.04:xenial: to_add=['20170207'] to_remove=[]
2017-02-08 21:32:23 sstreams: [info] maas:v2:download/maas:boot:ubuntu:armhf:hwe-16.04:xenial: to_add=['20170207'] to_remove=[]
2017-02-08 21:32:23 sstreams: [info] maas:v2:download/maas:boot:ubuntu:amd64:ga-16.04-lowlatency:xenial: to_add=['20170207'] to_remove=[]
2017-02-08 21:32:23 sstreams: [info] maas:v2:download/maas:boot:grub-efi:arm64:generic:uefi: to_add=['20170125.0'] to_remove=[]
2017-02-08 21:32:23 sstreams: [info] maas:v2:download/maas:boot:grub-efi-signed:amd64:generic:uefi: to_add=['20170125.0'] to_remove=[]
2017-02-08 21:32:23 sstreams: [info] maas:v2:download/maas:boot:ubuntu:armhf:ga-16.04:xenial: to_add=['20170207'] to_remove=[]
2017-02-08 21:32:23 sstreams: [info] maas:v2:download/maas:boot:ubuntu:armhf:generic-lpae:xenial: to_add=['20170207'] to_remove=[]
2017-02-08 21:32:23 sstreams: [info] maas:v2:download/maas:boot:ubuntu:armhf:hwe-16.04-edge:xenial: to_add=['20170207'] to_remove=[]
2017-02-08 21:32:23 sstreams: [info] maas:v2:download/maas:boot:ubuntu:amd64:ga-16.04:xenial: to_add=['20170207'] to_remove=[]
2017-02-08 21:32:23 sstreams: [info] maas:v2:download/maas:boot:centos:amd64:generic:centos70: to_add=['20161201_01'] to_remove=[]
2017-02-08 21:32:23 sstreams: [info] maas:v2:download/maas:boot:grub-ieee1275:ppc64el:generic:open-firmware: to_add=['20170125.0'] to_remove=[]
2017-02-08 21:32:23 sstreams: [info] maas:v2:download/maas:boot:ubuntu:amd64:hwe-16.04-edge:xenial: to_add=['20170207'] to_remove=[]
2017-02-08 21:32:23 sstreams: [info] maas:v2:download/maas:boot:pxelinux:i386:generic:pxe: to_add=['20160930.0'] to_remove=[]
2017-02-08 21:32:23 sstreams: [info] maas:v2:download/maas:boot:ubuntu:amd64:hwe-16.04-lowlatency:xenial: to_add=['20170207'] to_remove=[]
2017-02-08 21:32:23 sstreams: [info] maas:v2:download/maas:boot:ubuntu:amd64:hwe-16.04-lowlatency-edge:xenial: to_add=['20170207'] to_remove=[]
2017-02-08 21:32:23 provisioningserver.rackdservices.image_download_service: [critical] Downloading images failed.

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 434, in errback
    self._startRunCallbacks(fail)
  File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 501, in _startRunCallbacks
    self._runCallbacks()
  File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 588, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1184, in gotResult
    _inlineCallbacks(r, g, deferred)
--- <exception caught here> ---
  File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1126, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/usr/lib/python3/dist-packages/twisted/python/failure.py", line 389, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/usr/lib/python3/dist-packages/provisioningserver/rackdservices/image_download_service.py", line 128, in maybe_start_download
    yield self._start_download()
  File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1126, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/usr/lib/python3/dist-packages/twisted/python/failure.py", line 389, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/usr/lib/python3/dist-packages/provisioningserver/rackdservices/image_download_service.py", line 115, in _start_download
    get_proxy_url("https"))
  File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1126, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/usr/lib/python3/dist-packages/twisted/python/failure.py", line 389, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/usr/lib/python3/dist-packages/provisioningserver/rpc/boot_images.py", line 145, in _import_boot_images
    imported = yield deferToThread(_run_import, sources, **proxies)
  File "/usr/lib/python3/dist-packages/twisted/python/threadpool.py", line 246, in inContext
    result = inContext.theWork()
  File "/usr/lib/python3/dist-packages/twisted/python/threadpool.py", line 262, in <lambda>
    inContext.theWork = lambda: context.call(ctx, func, *args, **kw)
  File "/usr/lib/python3/dist-packages/twisted/python/context.py", line 118, in callWithContext
    return self.currentContext().callWithContext(ctx, func, *args, **kw)
  File "/usr/lib/python3/dist-packages/twisted/python/context.py", line 81, in callWithContext
    return func(*args,**kw)
  File "/usr/lib/python3/dist-packages/provisioningserver/utils/twisted.py", line 225, in wrapper
    result = func(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/provisioningserver/rpc/boot_images.py", line 119, in _run_import
    imported = boot_resources.import_images(sources)
  File "/usr/lib/python3/dist-packages/provisioningserver/import_images/boot_resources.py", line 278, in import_images
    update_iscsi_targets(os.path.join(storage, 'current'))
  File "/usr/lib/python3/dist-packages/provisioningserver/import_images/boot_resources.py", line 238, in update_iscsi_targets
    update_targets_conf(snapshot_path)
  File "/usr/lib/python3/dist-packages/provisioningserver/import_images/boot_resources.py", line 198, in update_targets_conf
    service_monitor.ensureService("tgt").wait(30)
  File "/usr/lib/python3/dist-packages/crochet/_eventloop.py", line 231, in wait
    result.raiseException()
  File "/usr/lib/python3/dist-packages/twisted/python/failure.py", line 368, in raiseException
    raise self.value.with_traceback(self.tb)
  File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1126, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/usr/lib/python3/dist-packages/twisted/python/failure.py", line 389, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/usr/lib/python3/dist-packages/provisioningserver/utils/service_monitor.py", line 535, in _ensureService
    state = yield self.getServiceState(service.name, now=True)
  File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1126, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/usr/lib/python3/dist-packages/twisted/python/failure.py", line 389, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/usr/lib/python3/dist-packages/provisioningserver/utils/service_monitor.py", line 251, in getServiceState
    active_state, process_state = yield self._loadServiceState(service)
  File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1128, in _inlineCallbacks
    result = g.send(result)
  File "/usr/lib/python3/dist-packages/provisioningserver/utils/service_monitor.py", line 478, in _loadSystemDServiceState
    service.service_name, active_state))
provisioningserver.utils.service_monitor.ServiceParsingError: Unable to parse the active state from systemd for service 'tgt', active state reported as 'deactivating'.