[AMD] I just installed microstack on a fresh ubuntu and cannot launch instances of cirrus

[rudy casper]

dashboard works, when I run microstack.openstack catalog list i get the same results fond in the tutorial https://tutorials.ubuntu.com/tutorial/microstack-get-started#3

when I use the CLI to get an instance I get

rudy@bigboy:~$ microstack.launch cirros --name test
Launching server ...
Allocating floating ip ...
Server test launched! (status is BUILD)

Access it with `ssh -i $HOME/.ssh/id_microstack cirros@10.20.20.248`
You can also visit the OpenStack dashboard at http://10.20.20.1:80

but when I check the instance its status is Error, when I click the instance hyperlink I get

Name
    test
ID
    715d99a4-c196-4c54-b808-b07f1e21f2bd
Description
    test
Project ID
    598414f090a740ccad54c6e22ed2f76a
Status
    Error
Locked
    False
Availability Zone
    -
Created
    Jan. 5, 2020, 2:43 a.m.
Age
    1 minute

Fault

Message
    Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 715d99a4-c196-4c54-b808-b07f1e21f2bd.
Code
    500
Details
    Traceback (most recent call last): File "/snap/microstack/198/lib/python3.6/site-packages/nova/conductor/manager.py", line 645, in build_instances raise exception.MaxRetriesExceeded(reason=msg) nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 715d99a4-c196-4c54-b808-b07f1e21f2bd.
Created
    Jan. 5, 2020, 2:43 a.m.

Specs

Flavor Name
    m1.tiny
Flavor ID
    1
RAM
    512MB
VCPUs
    1 VCPU
Disk
    1GB

IP Addresses

Security Groups

Not available

Metadata

Key Name
    microstack
Image Name
    cirros
Image ID
    c37397aa-a3b8-4c53-b855-e71bb539f5a5

Volumes Attached

I am running and AMD RYZEN 9 3900k
32Gram
1T SSD HD

uname -a
Linux bigboy 5.0.0-37-generic #40~18.04.1-Ubuntu SMP Thu Nov 14 12:06:39 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

[Pen Gale (pengale)]

Thank you for the bug report! I have a few questions and requests, when you have some time:

First, would you please run the following command and drop a link to a pastebin of the output here?

    sudo systemctl status snap.microstack.* --no-pager -l

Also, do you see any interesting error messages scroll by if you run `journalctl -xef` while launching an instance? Alternately, are there any logs in /var/snap/microstack/common/log/libvirt/qemu? How much free space do you have on the hard drive?

I have a hunch that the code that should set MicroStack up to play nicely with AMD virtualization extensions is breaking somewhere -- most of our test machines are Intel boxen, so it's quite possible that an error slipped through. The logs and info above should help diagnose and troubleshoot.

Alternately, OpenStack may just think that it doesn't have enough disk space available, and is reporting that in a somewhat obfuscated way (I'd like to do some work to make MicroStack smarter about this in the future).

[Jose Pirlot (jpissy)]

Hello Pete
I encounters exactly the same problem as Rudy while trying to launch a Cirros instance:

Message
    Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance f414c8a7-7a4f-4d15-aab4-2367bb010509.
Code 500
Details
    Traceback (most recent call last): File "/snap/microstack/196/lib/python3.6/site-packages/nova/conductor/manager.py", line 645, in build_instances raise exception.MaxRetriesExceeded(reason=msg) nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance f414c8a7-7a4f-4d15-aab4-2367bb010509.
Created 4 mars 2020 18:47

Output of sudo systemctl status snap.microstack.* --no-pager -l (extracts with errors only) is available on https://pastebin.com/3YyR3Y9P

Note that the error occurs on a fresh install with 200 GB free on SSD partition.
Harware: M/B ASUS Z170-A / 32GB DDR4 2400MHz / Intel i7-6700 Quad Core 3,4GHz 8Mb cache / SSD Crucial MX300 525 GB SATA
OS: Ubuntu 18.04.4

Thanks for your help.
Jose

[Pen Gale (pengale)]

@jpissy: Thank you for the log output!

It looks like libvirtd is failing to probe the kernel for capabilities, leading to the higher level errors "exhausting available hosts". (Basically, there are no hosts available, because libvirtd is crashing.)

I suspect that we're doing the right thing on Intel processors, and the wrong thing on AMD boxen. The fix might involve dropping some compile time flags into the libvirt part in the snapcraft.yaml, and it might involve appropriate config in /var/snap/microstack/common/etc.

Suggestions for said config welcome! (In the meantime, it might behoove us to get some amd machines in our lab.)

[Jose Pirlot (jpissy)]

Please note that the processor I use is an Intel one, not an AMD: i7-6700 Quad Core 3,4GHz.

[Jose Pirlot (jpissy)]

I can now launch instances on MicroStack beta. Error was on my side - I didn't specify the --devmode option while installing the edge version (205) of MicroStack.

[Billy Olsen (billy-olsen)]

I'm not sure if this is still an issue or not. Last comment from Jose indicates he was able to resolve his issue using --devmode.

@rudy.casper - please reopen this issue if you are still running into problems on the latest --beta.

[Launchpad Janitor (janitor)]

[Expired for MicroStack because there has been no activity for 60 days.]