[ Build 4.0-10 ]:K8S Cluster: Analytics auth mode incorrect, Webui login failed

[chhandak]

With Build 10 in contrail kubernetes cluster I am unable to login to contrail-webui. Though service is running fine and url is accessible, login attempt always fails.

Setup
-------
Node 10.87.121.34 (root/c0ntrail123)
Ui Link 10.87.121.34:8085 (admin/contrail123) . In K8s cluster we change the webui port to 8085 as kume-api listed to same port number.

Log from: journalctl -u contrail-webui | grep error
-----------------------------------------------------
May 22 19:21:05 5b7s18 contrailWebServer.sh[6793]: 05/22/2017 07:21:05 PM - error: URL [http://10.87.121.34:8081/analytics/uves/bgp-peers] returned error ["Authentication required"]
root@5b7s18(controller):/# contrail-status
== Contrail Control ==
contrail-control: active
contrail-named: active
contrail-dns: active
contrail-control-nodemgr: active
== Contrail Config ==
contrail-api: active
contrail-schema: active
contrail-svc-monitor: active
contrail-device-manager: active
contrail-config-nodemgr: active
== Contrail Web UI ==
contrail-webui: active
contrail-webui-middleware: active
== Contrail Support Services ==
rabbitmq-server: active (disabled on boot)
zookeeper: active
root@5b7s18(controller):/# contrail-version | grep ui
Package Version Build-ID | Repo | Package Name
contrail-openstack-webui 4.0.0.0-10 10
root@5b7s18(controller):/#

journalctl -u contrail-webui | grep error
May 22 19:16:50 5b7s18 contrailWebServer.sh[6793]: 05/22/2017 07:16:50 PM - error: URL [http://10.87.121.34:8081/analytics/uves/virtual-networks?cfilt=UveVirtualNetworkAgent] returned error ["Authentication required"]
May 22 19:16:50 5b7s18 contrailWebServer.sh[6793]: 05/22/2017 07:16:50 PM - error: REST Server Error: Authentication required
May 22 19:16:50 5b7s18 contrailWebServer.sh[6793]: 05/22/2017 07:16:50 PM - error: URL [http://10.87.121.34:8081/analytics/uves/control-node] returned error ["Authentication required"]
May 22 19:16:50 5b7s18 contrailWebServer.sh[6793]: 05/22/2017 07:16:50 PM - error: Encountered broken link: /analytics/uves/control-node
May 22 19:16:50 5b7s18 contrailWebServer.sh[6793]: 05/22/2017 07:16:50 PM - error: REST Server Error: Authentication required
May 22 19:16:50 5b7s18 contrailWebServer.sh[6793]: 05/22/2017 07:16:50 PM - error: Control Node UVE Data Parse error :TypeError: Cannot read property 'bgp_router_ip_list' of undefined
May 22 19:16:50 5b7s18 contrailWebServer.sh[6793]: 05/22/2017 07:16:50 PM - error: URL [http://10.87.121.34:8081/analytics/query] returned error ["Authentication required"]
May 22 19:16:50 5b7s18 contrailWebServer.sh[6793]: 05/22/2017 07:16:50 PM - error: Error Run Query: REST Server Error: Authentication required
May 22 19:16:50 5b7s18 contrailWebServer.sh[6793]: 05/22/2017 07:16:50 PM - error: REST Server Error: Authentication required
May 22 19:16:50 5b7s18 contrailWebServer.sh[6793]: 05/22/2017 07:16:50 PM - error: URL [http://10.87.121.34:8081/analytics/query] returned error ["Authentication required"]
May 22 19:16:50 5b7s18 contrailWebServer.sh[6793]: 05/22/2017 07:16:50 PM - error: Error Run Query: REST Server Error: Authentication required
May 22 19:16:50 5b7s18 contrailWebServer.sh[6793]: 05/22/2017 07:16:50 PM - error: REST Server Error: Authentication required
May 22 19:16:50 5b7s18 contrailWebServer.sh[6793]: 05/22/2017 07:16:50 PM - error: URL [http://10.87.121.34:8081/analytics/uves/analytics-node] returned error ["Authentication required"]
May 22 19:16:50 5b7s18 contrailWebServer.sh[6793]: 05/22/2017 07:16:50 PM - error: REST Server Error: Authentication required
May 22 19:16:50 5b7s18 contrailWebServer.sh[6793]: 05/22/2017 07:16:50 PM - error: URL [http://10.87.121.34:8081/analytics/query] returned error ["Authentication required"]
May 22 19:16:50 5b7s18 contrailWebServer.sh[6793]: 05/22/2017 07:16:50 PM - error: Error Run Query: REST Server Error: Authentication required
May 22 19:16:50 5b7s18 contrailWebServer.sh[6793]: 05/22/2017 07:16:50 PM - error: REST Server Error: Authentication required
May 22 19:16:50 5b7s18 contrailWebServer.sh[6793]: 05/22/2017 07:16:50 PM - error: URL [http://10.87.121.34:8081/analytics/query] returned error ["Authentication required"]
May 22 19:16:50 5b7s18 contrailWebServer.sh[6793]: 05/22/2017 07:16:50 PM - error: Error Run Query: REST Server Error: Authentication required
May 22 19:16:50 5b7s18 contrailWebServer.sh[6793]: 05/22/2017 07:16:50 PM - error: REST Server Error: Authentication required
May 22 19:16:50 5b7s18 contrailWebServer.sh[6793]: 05/22/2017 07:16:50 PM - error: URL [http://10.87.121.34:8081/analytics/query] returned error ["Authentication required"]
May 22 19:16:50 5b7s18 contrailWebServer.sh[6793]: 05/22/2017 07:16:50 PM - error: Error Run Query: REST Server Error: Authentication required
May 22 19:16:50 5b7s18 contrailWebServer.sh[6793]: 05/22/2017 07:16:50 PM - error: REST Server Error: Authentication required
May 22 19:16:50 5b7s18 contrailWebServer.sh[6793]: 05/22/2017 07:16:50 PM - error: URL [http://10.87.121.34:8081/analytics/query] returned error ["Authentication required"]
May 22 19:16:50 5b7s18 contrailWebServer.sh[6793]: 05/22/2017 07:16:50 PM - error: Error Run Query: REST Server Error: Authentication required
May 22 19:16:50 5b7s18 contrailWebServer.sh[6793]: 05/22/2017 07:16:50 PM - error: REST Server Error: Authentication required
May 22 19:16:52 5b7s18 contrailWebServer.sh[6793]: 05/22/2017 07:16:52 PM - error: URL [http://10.87.121.34:8081/analytics/query] returned error ["Authentication required"]
May 22 19:16:52 5b7s18 contrailWebServer.sh[6793]: 05/22/2017 07:16:52 PM - error: Error Run Query: REST Server Error: Authentication required
May 22 19:16:52 5b7s18 contrailWebServer.sh[6793]: 05/22/2017 07:16:52 PM - error: REST Server Error: Authentication required
May 22 19:21:05 5b7s18 contrailWebServer.sh[6793]: 05/22/2017 07:21:05 PM - error: URL [http://127.0.0.1:5000/v2.0/tokens] returned error [{"code":"ECONNREFUSED","errno":"ECONNREFUSED","syscall":"connect"}]
May 22 19:21:05 5b7s18 contrailWebServer.sh[6793]: 05/22/2017 07:21:05 PM - error: authPostV2Req() error:REST Server Error: { [Error: connect ECONNREFUSED]
May 22 19:21:05 5b7s18 contrailWebServer.sh[6793]: 05/22/2017 07:21:05 PM - error: URL [http://127.0.0.1:5000/v2.0/tokens] returned error [{"code":"ECONNREFUSED","errno":"ECONNREFUSED","syscall":"connect"}]
May 22 19:21:05 5b7s18 contrailWebServer.sh[6793]: 05/22/2017 07:21:05 PM - error: authPostV2Req() error:REST Server Error: { [Error: connect ECONNREFUSED]
May 22 19:21:05 5b7s18 contrailWebServer.sh[6793]: 05/22/2017 07:21:05 PM - error: URL [http://10.87.121.34:8081/analytics/uves/bgp-peers] returned error ["Authentication required"]
May 22 19:21:05 5b7s18 contrailWebServer.sh[6793]: 05/22/2017 07:21:05 PM - error: URL [http://127.0.0.1:5000/v2.0/tokens] returned error [{"code":"ECONNREFUSED","errno":"ECONNREFUSED","syscall":"connect"}]
May 22 19:21:05 5b7s18 contrailWebServer.sh[6793]: 05/22/2017 07:21:05 PM - error: authPostV2Req() error:REST Server Error: { [Error: connect ECONNREFUSED]
May 22 19:21:05 5b7s18 contrailWebServer.sh[6793]: 05/22/2017 07:21:05 PM - error: URL [http://10.87.121.34:8081/analytics/uves/bgp-peers] returned error ["Authentication required"]

[chhandak (chhandak)]

Logs copied to /auto/cores/1692713

[Jeba Paulaiyan (jebap)]

As per chhandak: this workaround worked.

Hi Chhandak,
The issue is in contrail-analytics-api.conf file, we have aaa_mode set as cloud-admin, where as contrail-aapi.conf, it was no-auth.
Setting no-auth in contrail-analytics.api.conf should solve the issue.

With Regards,
Biswajit

[Rahul (rahuls)]

Vedu also raised the same issue and his provisioning was setting analytics auth mode to aaa_mode instead of no-auth.

Please confirm that the provisioning params are incorrect and this isn't a default setting.

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/32003
Submitter: Nikhil Bansal (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/32034
Submitter: Santosh Gupta (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/32035
Submitter: Santosh Gupta (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/32035
Committed: http://github.com/Juniper/contrail-ansible-internal/commit/b91a0238e5c811db68d4fd4baec267f0fd4ee9c5
Submitter: Zuul (<email address hidden>)
Branch: R4.0

commit b91a0238e5c811db68d4fd4baec267f0fd4ee9c5
Author: Santosh Gupta <email address hidden>
Date: Tue May 23 15:24:15 2017 -0700

Remove default definition of aaa_mode

During evaluation of analytics_api_aaa_mode_orig aaa_mode should be defined
only if value is passed from inventory or cluster.json file.

Closes-Bug: #1692713

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/32034
Committed: http://github.com/Juniper/contrail-ansible-internal/commit/31a7e68a351738772a922226be48eb16c49d8777
Submitter: Zuul (<email address hidden>)
Branch: master

commit 31a7e68a351738772a922226be48eb16c49d8777
Author: Santosh Gupta <email address hidden>
Date: Tue May 23 15:25:48 2017 -0700

Remove default definition of aaa_mode

During evaluation of analytics_api_aaa_mode_orig aaa_mode should be defined
only if value is passed from inventory or cluster.json file.

Change-Id: I818fd0da0b7ef049070ed40d28403bb2718ef7b3
Closes-Bug: #1692713