Install failures: port in use and a timeout

[Adam Israel]

Hey all,

Excited to try microstack and tried to spin it up while on the road tonight. It didn't quite work, but I'm excited about it nonetheless.

tl;dr, If one of the services being install hits a port conflict because an existing service is using that port (http and mysql, particularly) the snap install fails. Resolving that leads to another failure, where a file is downloading but times out.

OS: Ubuntu 18.04.1

What I did:

$ snap install microstack --classic --candidate
 error: cannot perform the following tasks:
- Start snap "microstack" (65) services ([start snap.microstack.nginx.service] failed with exit status 1: Job for snap.microstack.nginx.service failed because the control process exited with error code.
See "systemctl status snap.microstack.nginx.service" and "journalctl -xe" for details.

journalctl -xe reports:

microstack.nginx[17632]: nginx: [emerg] bind() to 0.0.0.0:80 failed (98: Address already in use)

As it turns out, I had another service listening on port 80, so I removed that and re-ran the install but ran into this:

Creating OpenStack Databases
Using conf file: /var/snap/microstack/common/etc/mysql/my.cnf
ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/snap/microstack/common/run/mysql/mysqld.sock' (2)

After stopping my existing mysql server, the install got farther but stopped again:

Run configure hook of "microstack" snap if present [68/1332]
un configure hook of "microstack" snap if present |
error: cannot perform the following tasks:
- Run configure hook of "microstack" snap if present (run hook "configure":
-----
  4400K .......... .......... .......... .......... .......... 34% 24.4K 5m2s
  4450K .......... .......... .......... .......... .......... 34% 35.1K 5m0s
  4500K .......... .......... .......... .......... .......... 35% 52.4K 4m57s
  4550K .......... .......... .......... .......... .......... 35% 72.8K 4m53s
  4600K .......... .......... .......... .......... .......... 35% 60.6K 4m49s
  4650K .......... .......... .......... .......... .......... 36% 124K 4m45s
  4700K .......... .......... .......... .......... .......... 36% 104K 4m41s
  [...]
  9050K .......... .......... .......... .......... .......... 70% 98.9K 1m49s
  9100K .......... .......... .......... .......... .......... 70% 88.5K 1m47s
  9150K .......... .......... .......... .......... .......... 71% 90.1K 1m46s
  9200K .......... .......... .......... .......... .......... 71% 98.1K 1m44s
  9250K .......... .......... .......... .......... .......... 71% 78.4K 1m42s
  9300K .......... ..........
<exceeded maximum runtime of 5m0s>

and dropped me back to the terminal, with no other error message.

I do see this in journalctl several times:

Feb 05 22:58:06 ronin libvirtd[18360]: ethtool ioctl error: No such device

I assume it was related to the snap, but the timeout seems to be with downloading something (and over hotel wifi, not surprising).

[Pen Gale (pengale)]

Right now, microstack is pretty rude about port selection, hogging the default http and mysql ports. This makes it easy to find stuff, but not easy to run it alongside other projects on a laptop. I think that we should give this some thought, as possibly switch up the ports that it uses.

The second issue is probably a timeout while downloading the cirros image over hotel wifi. A workaround is to download the cirros cloud image to /images:

sudo curl -o /images/cirros-0.3.5-x86_64-disk.img http://download.cirros-cloud.net/0.3.5/cirros-0.3.5-x86_64-disk.img

(Note: it might be better to just bundle the cirros image, or, at the very least, put it in a place that gets cleaned up when the snap is uninstalled.)

[Pen Gale (pengale)]

An update on the workaround for the download timeout issue: the cirros image now gets downloaded to /var/snap/microstack/common/images, so you'll want to sudo mkdir -p that path, and put the cirros image there if you want to download in advance.

Work on resolving the port issue automagically is still in the planning stages.

[Pen Gale (pengale)]

I've got a PR in for using non default ports for the dashboard (as well as MySQL and Rabbit, though both of those are hidden from operators).

Do others feel that this is the right approach?

[Pen Gale (pengale)]

PR: https://review.opendev.org/#/c/693450/2

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/693450
Committed: https://git.openstack.org/cgit/x/microstack/commit/?id=1a25e50a172db7331edf2f836f3c2005222bb4c5
Submitter: Zuul
Branch: master

commit 1a25e50a172db7331edf2f836f3c2005222bb4c5
Author: Pete Vander Giessen <email address hidden>
Date: Tue Nov 5 19:54:35 2019 +0000

    Made horizon, rabbit and mysql ports configurable

    Addresses requests to make it easier to avoid conflicts between the
    Horizon dashboard and http services that might already be running on
    the machine.

    Configurable via snap config. Exposing via arguments to .init and
    testing post init configuration is left for a separate PR.

    Eventually, these may move to non standard ports by default. This PR
    sets the stage for that, but further discussion is needed before we
    decide whether to implement.

    (This commit also contains a sneaky fix for the username display at the
    end of the launch script.)

    Closes-Bug: 1814829
    Change-Id: If728d6ec8024bca4d3e809637fbdcc03ed4e6934