stx.2018.10: Magnum Service Issue: ERROR: Invalid input for field 'identity/password/user/password': None is not of type 'string'

[Yatindra]

Title
-----
Magnum Service Issue: ERROR: Invalid input for field 'identity/password/user/password': None is not of type 'string'

Brief Description
-----------------
I am trying to setup kubernetes clusters using Magnum 5.0.1 available in the StarlingX 2018.10. I have downloaded latest 2018.10 ISO image from the StarlingX Mirror repositories.
But I am facing multiple issues while enabling the Magnum service and running the commands. Is it supported and tested fully?

Severity
--------
Major:

Steps to Reproduce
------------------
Create Endpoint for the Magnum service and run the magnum service-list command.
Issue occurs with any magnum command

Expected Behavior
------------------
It should show the list

Actual Behavior
----------------
Giving error like below:

[wrsroot@controller-0 ~(keystone_admin)]$ magnum service-list
ERROR: Invalid input for field 'identity/password/user/password': None is not of type 'string'

Failed validating 'type' in schema['properties']['identity']['properties']['password']['properties']['user']['properties']['password']:
    {'type': 'string'}

On instance['identity']['password']['user']['password']:
    None (HTTP 400) (Request-ID: req-e1fd9612-a616-421f-92a3-2b5ee1f4e5ec) (HTTP 500) (Request-ID: req-38d34547-9fb0-423b-bd33-1dcfa26ada68)
[wrsroot@controller-0 ~(keystone_admin)]$

Reproducibility
---------------
It can be reporduced multiple times.

System Configuration
--------------------
Intel Purely Hardware with 4 NIC cards, 56 cores and 98 GB or RAM

Branch/Pull Time/Commit
-----------------------
Downloaded ISO image from the StarlingX remote mirror repositories

[Ghada Khalil (gkhalil)]

There hasn't been any magnum testing done as part of the verification of stx.2018.10.
stx master now provides support for k8s containers. Suggest you try using master.

[Ghada Khalil (gkhalil)]

Requesting that Bruce follows up with the reporter regarding the importance of this issue. My recommendation is to close as "Won't Fix" with a suggestion to use stx master for any container related activities.

[Ghada Khalil (gkhalil)]

Another data point on this: I had a brief discussion with Jerry Sun who has some previous experience with magnum. He believes that magnum is not functional in stx.2018.10.

[Yatindra (yatindra)]

I have enable as per the Openstack documentation as per link https://docs.openstack.org/magnum/latest/install/install-guide-from-source.html except downloading magnum, as it was already there. I had enabled endpoints accordingly.
I believe that StarlingX is open and support Openstack components without any problem, so should be some only configuration problem.

[Ghada Khalil (gkhalil)]

Thanks for the info. I'll leave it to Cindy to assign someone to investigate further if that's what she wishes to do.

[Mao Jiaming (jiamingm)]

Hi,
    When I was trying to reproduce this bugs, I found some difference between the starlingx and the magnum installation guides. Would you kindly show me how you solve these problems.
     1. The database in starlingx is postgresql, Since the guides requires mysql.
     2. How do you start the magnum service Since I cannot find the file magnum-api.service and the magnum-conductor.service in the starlinx without downloading them.

[Ghada Khalil (gkhalil)]

Marking as Incomplete until the reporter provides the info requested by the developer investigating the issue.

[Yatindra (yatindra)]

@mao Jiaming: Magnum was already in the build (you can check the rpm), I don't have to download it but yes, it's true that I have to enable those services with the commands as given through the openstack cli.

Yes I used the postgresql database which starlingX already has, I have created a user with the access to mgnum db and gave same user detail in the magnum.conf.

[wanghejun (wanghejun)]

Hello:
    I did the following in the 2019-5 and 2018-10 versions.
    In the 2019-5 version, after creating the endpoint, send the magnum service-list command to indicate that the connection is not on the service (log1). Directly starting the magnum service failed, indicating that there is no such service (log2). It is currently inferred that the 2019-5 version does not have the magnum service or other related components installed, causing the magnum service to fail to start.
    In the 2018-10 version, after the above-mentioned hand-operating operation, the connection timeout (log3) is prompted. During the investigation, it was found that there was a problem with the rabbitmq installed by default, and the rabbitmq service started to report an error (log4), inferring that rabbitmq caused magnum to fail to work properly. Attempts to reinstall erlang and rabbitmq, and then based on further results to determine whether I need to manually install magnum.
    Can you provide the steps to make the magnum service start successfully.

error log1:
[wrsroot@controller-0 ~(keystone_admin)]$ magnum service-list
ERROR: Unable to establish connection to http://192.168.204.2:2401/mservices: HTTPConnectionPool(host='192.168.204.2', port=2401): Max retries exceeded with url: /mservices (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa6c764ec50>: Failed to establish a new connection: [Errno 111] Connection refused',))

error log2:
[wrsroot@controller-0 ~(keystone_admin)]$ systemctl start magnum
Failed to start magnum.service: Interactive authentication required.
See system logs and 'systemctl status magnum.service' for details.

[wrsroot@controller-0 ~(keystone_admin)]$ systemctl status magnum
Unit magnum.service could not be found.

error log3:
[wrsroot@controller-0 /(keystone_admin)]$ magnum service-list
ERROR: Unable to establish connection to http://192.168.204.2:53/mservices: HTTPConnectionPool(host='192.168.204.2', port=53): Max retries exceeded with url: /mservices (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7f319cc24350>: Failed to establish a new connection: [Errno 110] Connection timed out',))

error log4:
controller-0:~$ systemctl status rabbitmq-server
● rabbitmq-server.service - RabbitMQ broker
   Loaded: loaded (/usr/lib/systemd/system/rabbitmq-server.service; disabled; vendor preset: disabled)
  Drop-In: /etc/systemd/system/rabbitmq-server.service.d
           └─limits.conf
   Active: inactive (dead)
controller-0:~$ systemctl start rabbitmq-server
Failed to start rabbitmq-server.service: Interactive authentication required.
See system logs and 'systemctl status rabbitmq-server.service' for details.
controller-0:~$ sudo systemctl start rabbitmq-server
Password:
Job for rabbitmq-server.service failed because the control process exited with error code. See "systemctl status rabbitmq-server.service" and "journalctl -xe" for details.

controller-0:~$ sudo systemctl status rabbitmq-server
● rabbitmq-server.service - RabbitMQ broker
   Loaded: loaded (/usr/lib/systemd/system/rabbitmq-server.service; disabled; vendor preset: disabled)
  Drop-In: /etc/syst...

[wanghejun (wanghejun)]

Hello,
    Sorry, please ignore the last reply.
    According to the following steps, the "magnum-service-list" command is successfully sent.
   1.StarlingX 2018.10 virtual machine deployment,
    See https://docs.starlingx.io/deployment_guides/current/simplex.html
   2.Install MySQL5.6
   3.magnum related configuration,
    See https://docs.openstack.org/magnum/latest/install/install-rdo.html#top
   4.Execute the command "magnum service-list"

log:
[wrsroot@controller-0 magnum(keystone_admin)]$ vim /var/log/magnum/
[wrsroot@controller-0 magnum(keystone_admin)]$ vim /var/log/magnum/magnum-api.log
[wrsroot@controller-0 magnum(keystone_admin)]$ sudo systemctl restart openstack-magnum-api.service openstack-magnum-conductor.service
[wrsroot@controller-0 magnum(keystone_admin)]$ magnum service-list
+----+------+------------------+-------+----------+-----------------+---------------------------+---------------------------+
| id | host | binary | state | disabled | disabled_reason | created_at | updated_at |
+----+------+------------------+-------+----------+-----------------+---------------------------+---------------------------+
| 1 | - | magnum-conductor | up | False | - | 2019-06-20T07:48:06+00:00 | 2019-06-20T07:51:50+00:00 |
+----+------+------------------+-------+----------+-----------------+---------------------------+---------------------------+

   There is no error (ERROR: Invalid input for field 'identity/password/user/password': None is not of type 'string') mentioned in the bug during debugging. If further confirmation is required, please provide detailed steps.

[wanghejun (wanghejun)]

Hello,
    I tried to configure magnum in the starlingx (2019/06/26) environment, according to the official website(https://docs.openstack.org/magnum/latest/install/install-rdo.html#top), after installation When the related service is prompted, there is no corresponding download resource. The log is as follows:
---------------------
controller-0:~$ sudo yum install openstack-magnum-api openstack-magnum-conductor python-magnumclient
Password:
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
base | 3.6 kB 00:00:00
extras | 3.4 kB 00:00:00
updates | 3.4 kB 00:00:00
No package openstack-magnum-api available.
No package openstack-magnum-conductor available.
Nothing to do
---------------------
    Trying to manually install the above 3 services did not succeed.
    It is currently judged that the new version of starlingx does not support the magnum service and may be replaced by other services.
    According to the current situation, if magnum is no longer maintained in the new version of starlingx, it is recommended to close the bug; if you need to further confirm other matters, then analyze it according to the actual situation.

[Al Bailey (albailey1974)]

In the current codebase:

magnum service-list
ERROR: internal endpoint for container service in RegionOne region not found

openstack coe service list
internal endpoint for container service in RegionOne region not found

This is because there are no magnum endpoints setup.

[Cindy Xie (xxie1)]

Close this ticket as Magnum is not a supported feature for stx.2.0. If anybody require Magnum support, we shall open a feature request from Storyboard.

[wanghejun (wanghejun)]

@Al Bailey (albailey1974):
  Thank you very much for your reply.
  The information you mentioned has been encountered before and has been resolved. Please refer to my information in step_3 in my reply to "#11".
  It has been confirmed that the latest version of starlingx does not support the magnum service. It is recommended to turn off the bug.