Failed Deployment: No Arrays Found

[Ryan Kather]

MaaS 1.9 will not deploy Ubuntu images to commissioned physical nodes. The nodes are HP Proliant DL 380 G5 servers with HP Smart Array P400 Controllers.

    mdadm: No arrays found in config file or automatically
    An error occured handling 'cciss!c0d0': FileNotFoundError - [Errno 2] No such file or directory: '/sys/block/c0d0/holders'
    [Errno 2] No such file or directory: '/sys/block/c0d0/holders'
    Installation failed with exception: Unexpected error while running command.
    Command: ['curtin', 'block-meta', 'custom']
    Exit code: 3
    Reason: -
    Stdout: b"mdadm: No arrays found in config file or automatically\nAn error occured handling 'cciss!c0d0': FileNotFoundError - [Errno 2] No such file or directory: '/sys/block/c0d0/holders'\n[Errno 2] No such file or directory: '/sys/block/c0d0/holders'\n"
Stderr: ''

Verified behavior on 14.04 and 15.10 Ubuntu releases. With flat and LVM volume layouts. Deployment worked on these servers in earlier MaaS version.

[Andres Rodriguez (andreserl)]

Hi Ryan,

Can you please get the output of:

 maas <user> node get-curtin-config <node's system id>

while the machine is failed deployment?

Also, can you try to re-commission the machjine and try the dpeloyment again? if it fails, could you please obtain the data as above?

Thanks!

[Ryan Kather (rkather)]

Thank you for the quick response. I've tried recommission and redeployment with same behavior. I should mention I have 8 servers all identical, all exhibiting this behavior.

I've attached the output of maas <user> node get-curtin-config <node's system id>

[Blake Rouse (blake-rouse)]

This looks to be an issue on how curtin is handling your specific hardware. Not a MAAS related problem.

[Ryan Kather (rkather)]

Interesting, it seems this might be related to this bug (though the discussion makes the mistake of not including HP Proliant G5s in their scenario discussion). I think the CCISS disappeared in G6 and onward;

https://bugs.launchpad.net/curtin/+bug/1263181

From the discussion, it should have been fixed in an earlier package of curtin that was pushed to udpates. However, it seems I have a later package installed and the issue persists (at least on the cluster controller and region controller). I'm curious if there was a regression at some point, or does maas use an older version on the image (not really sure where curtin is called from to be honest.

Installed Curtin on MaaS cluster:
ii curtin-common 0.1.0~bzr314-0ubuntu1 all Library and tools for curtin installer
ii python-curtin 0.1.0~bzr314-0ubuntu1 all Library and tools for curtin installer

[FelixD (generalkalbasa)]

i have a set of DL 380 G5's, and can confirm curtin has stoped working with them since maas 1.9 and curtin 314, though i have checked the block_meta.py, __init__.py and common and they contain the fixes from curtin 221, so the problem seems to be elsewhere

[iulianpojar (iulianpojar)]

i have the same problem with HP DL 380 G5, can not deploy any node.

[iulianpojar (iulianpojar)]

same problems, bug not fixed in xenial.

[iulianpojar (iulianpojar)]

An error occured handling 'cciss!c0d0': FileNotFoundError - [Errno 2] No such file or directory: '/sys/block/c0d0/holders'
[Errno 2] No such file or directory: '/sys/block/c0d0/holders'
Installation failed with exception: Unexpected error while running command.
Command: ['curtin', 'block-meta', 'custom']
Exit code: 3
Reason: -
Stdout: b"An error occured handling 'cciss!c0d0': FileNotFoundError - [Errno 2] No such file or directory: '/sys/block/c0d0/holders'\n[Errno 2] No such file or directory: '/sys/block/c0d0/holders'\n"
Stderr: ''

[Andy S (andy-speed)]

I have the same issue. Is there any workaround or fix in the pipeline? I am also running xenial.

[Andy S (andy-speed)]

Have found a workaround for this that has allowed me to commission and deploy some of my older HP servers. This was to go into the Settings of MAAS and add the following to the Global Kernel Parameters section:

cciss.blacklist=yes modprobe.blacklist=cciss hpsa.hpsa_allow_any=1

By doing this it was able to see the storage as a regular sda device which at least allowed me to continue with my test lab setup.

This was based on the following post:

https://askubuntu.com/questions/728317/maas-node-deployment-failing-on-hp-proliant-dl380-g5