[2.x, pod] maas cli fails to compose pod after, and doesn't show why

Bug #1712406 reported by Jeff Hillman on 2017-08-22
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
MAAS
Medium
Unassigned

Bug Description

Testing in CPE lab, failing to compose pod after pod creation

$ maas admin pod compose 1 hostname=infra-1 cores=1 memory=4096 storage=root:50 zone=1
Unable to compose machine because: Failed talking to pod: sequence item 3: expected str instance, NoneType found

$ maas admin pod compose 1
Unable to compose machine because: Failed talking to pod: sequence item 3: expected str instance, NoneType found

This is 100% reproducible. happens with MAAS 2.2.2

The pod is there...

$ maas admin pods read
Success.
Machine-readable output follows:
[
    {
        "used": {
            "memory": 0,
            "cores": 0,
            "local_storage": 0
        },
        "name": "infra-1",
        "type": "virsh",
        "id": 1,
        "available": {
            "memory": 64313,
            "local_storage": 404381908336,
            "cores": 16
        },
        "capabilities": [
            "composable",
            "dynamic_local_storage",
            "over_commit"
        ],
        "resource_uri": "/MAAS/api/2.0/pods/1/",
        "total": {
            "memory": 64313,
            "cores": 16,
            "local_storage": 404381908336
        },
        "architectures": [
            "amd64/generic"
        ]
    }
]

Trying to specify a pod that doesn't exist gives a different output, a more expected one...

$ maas admin pod compose 5 hostname=infra-1 cores=1 memory=4096 storage=root:50 zone=1
Not Found

$ maas admin pod compose 5
Not Found

$ dpkg --list | grep maas
ii maas-cli 2.2.2-6099-g8751f91-0ubuntu1~16.04.1 all MAAS client and command-line interface
ii maas-common 2.2.2-6099-g8751f91-0ubuntu1~16.04.1 all MAAS server common files
ii maas-dhcp 2.2.2-6099-g8751f91-0ubuntu1~16.04.1 all MAAS DHCP server
ii maas-dns 2.2.2-6099-g8751f91-0ubuntu1~16.04.1 all MAAS DNS server
ii maas-proxy 2.2.2-6099-g8751f91-0ubuntu1~16.04.1 all MAAS Caching Proxy
ii maas-rack-controller 2.2.2-6099-g8751f91-0ubuntu1~16.04.1 all Rack Controller for MAAS
ii maas-region-api 2.2.2-6099-g8751f91-0ubuntu1~16.04.1 all Region controller API service for MAAS
ii maas-region-controller 2.2.2-6099-g8751f91-0ubuntu1~16.04.1 all Region Controller for MAAS
ii python3-django-maas 2.2.2-6099-g8751f91-0ubuntu1~16.04.1 all MAAS server Django web framework (Python 3)
ii python3-maas-client 2.2.2-6099-g8751f91-0ubuntu1~16.04.1 all MAAS python API client (Python 3)
ii python3-maas-provisioningserver 2.2.2-6099-g8751f91-0ubuntu1~16.04.1 all MAAS server provisioning libraries (Python 3)

Andres Rodriguez (andreserl) wrote :

Hi Jeff,

I imagine you have a KVM pod. Can you provide more information on how have you installed and configured your libvirt/KVM? Have you created a storage pool, changed the networks, etc?

virsh net-list
virsh pool-list

Also, please provide logs:

/var/log/maas/*.log

Changed in maas:
status: New → Incomplete
milestone: none → 2.3.0
tags: added: pod
summary: - maas cli fails to compose pod after pod create
+ [2.x] maas cli fails to compose pod after pod create

libvirt-bin was installed as part of the hand installation by selecting "Virtualization Host" as part of the package selection.

the default network was deleted ( so that it didn't show up as a fabric in MAAS ).

$ virsh net-list
 Name State Autostart Persistent
----------------------------------------------------------

$ virsh pool-list
 Name State Autostart
-------------------------------------------
 pool_virt_images active yes

Blake Rouse (blake-rouse) wrote :

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 434, in errback
    self._startRunCallbacks(fail)
  File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 501, in _startRunCallbacks
    self._runCallbacks()
  File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 588, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1184, in gotResult
    _inlineCallbacks(r, g, deferred)
--- <exception caught here> ---
  File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1126, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/usr/lib/python3/dist-packages/twisted/python/failure.py", line 389, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/usr/lib/python3/dist-packages/provisioningserver/drivers/pod/virsh.py", line 906, in compose
    created_machine = yield deferToThread(conn.create_domain, request)
  File "/usr/lib/python3/dist-packages/twisted/python/threadpool.py", line 246, in inContext
    result = inContext.theWork()
  File "/usr/lib/python3/dist-packages/twisted/python/threadpool.py", line 262, in <lambda>
    inContext.theWork = lambda: context.call(ctx, func, *args, **kw)
  File "/usr/lib/python3/dist-packages/twisted/python/context.py", line 118, in callWithContext
    return self.currentContext().callWithContext(ctx, func, *args, **kw)
  File "/usr/lib/python3/dist-packages/twisted/python/context.py", line 81, in callWithContext
    return func(*args,**kw)
  File "/usr/lib/python3/dist-packages/provisioningserver/utils/twisted.py", line 232, in wrapper
    result = func(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/provisioningserver/drivers/pod/virsh.py", line 750, in create_domain
    self.attach_interface(request.hostname, best_network)
  File "/usr/lib/python3/dist-packages/provisioningserver/drivers/pod/virsh.py", line 638, in attach_interface
    '--model', 'virtio', '--config'])
  File "/usr/lib/python3/dist-packages/provisioningserver/drivers/pod/virsh.py", line 242, in run
    cmd = ' '.join(args)
builtins.TypeError: sequence item 3: expected str instance, NoneType found

Changed in maas:
status: Incomplete → Triaged
importance: Undecided → High
Ante Karamatić (ivoks) on 2017-08-24
tags: added: cpec
Andres Rodriguez (andreserl) wrote :

Hi Jeff,

You need a network defined to make it work w/ libvirt. That's the network where you should be providing DHCP on.

MAAS doesn't support creating machines against bridge networking, only machines against an specific network defined.

The error here is error surface.

Ante Karamatić (ivoks) wrote :

Right, we can confirm that with networking in place it all works fine.

Ante Karamatić (ivoks) wrote :

@MAAS you might reconsider and invalidate this bug, since it was a configuration error. However, a cleaner error/feedback to the user would be helpful.

tags: added: cpe-onsite
removed: cpec

Hi Ante,

I'll keep this open to fix the error surfacing.

summary: - [2.x] maas cli fails to compose pod after pod create
+ [2.x] maas cli fails to compose pod after, but error surface is bad
Changed in maas:
importance: High → Medium
summary: - [2.x] maas cli fails to compose pod after, but error surface is bad
+ [2.x] maas cli fails to compose pod after, and doesn't show why
tags: added: internal
tags: added: error-surface
summary: - [2.x] maas cli fails to compose pod after, and doesn't show why
+ [2.x, pod] maas cli fails to compose pod after, and doesn't show why
Changed in maas:
milestone: 2.3.0 → 2.3.x
Changed in maas:
milestone: 2.3.x → 2.4.x
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Bug attachments