Horizon VIP is inaccessible ns haproxy

[Malgorzata Rembas]

Detailed bug description:
Horizon VIP is inaccessible from external networks. Missing routing definition in one of controller for haproxy namespace. For HAProxy all controllers are offline.

 Steps to reproduce:
  3 controllers, 4 compute nodes 3 storage nodes. Deploy openstack with 3 controllers in HA configuration

 Expected results:
 Accessible Horizon VIP after openstack deployment

 Actual result:
  working by manualy fixing

 Reproducibility:

  ip netns exec haproxy ip r
  default via 100.64.0.1 dev b_public metric 10
  default via 240.0.0.1 dev hapr-ns metric 10000
MISSING ENTRY: 10.10.0.0/24 via 240.0.0.1 dev hapr-ns metric 10000
  100.64.0.0/20 dev b_public proto kernel scope link src 100.64.0.12
MISSING ENTRY: 192.168.0.0/24 via 240.0.0.1 dev hapr-ns metric 10000
MISSING ENTRY: 192.168.1.0/24 via 240.0.0.1 dev hapr-ns metric 10000
  240.0.0.0/30 dev hapr-ns proto kernel scope link src 240.0.0.2

After add missing entry check pcsd status should change to online in all nodes and Horizon VIP should be accesible

 Workaround:
  check routing table in ns haproxy in all controllers and add manually missing routing entries

 Impact:
  Horizon VIP is inaccessible from external networks. Missing routing definition in haproxy namespaces in one of controller. In haproxy status all controllers are offline

 Description of the environment:
  feature_groups: []
  openstack_version: mitaka-9.0 (Mitaka on Ubuntu 14.04)
  release: '9.0'
  Release : 1.mos6349
  Operation system: Master Fuel Node CentOS Linux release 7.2.1511 (Core)
  Network model: Tagged VLANs for external and internal network except pxe

[Oleksiy Molchanov (omolchanov)]

What entry is missing? Please attach diagnostic snapshot. Marking as incomplete.

[Malgorzata Rembas (mrembas)]

Please find diagnostics snapshot:
https://1drv.ms/f/s!AjakciGUCNIegZ57T-DsrkYO0DTdvw

MISSING ENTRY: 10.10.0.0/24 via 240.0.0.1 dev hapr-ns metric 10000
MISSING ENTRY: 192.168.0.0/24 via 240.0.0.1 dev hapr-ns metric 10000
MISSING ENTRY: 192.168.1.0/24 via 240.0.0.1 dev hapr-ns metric 10000

Public - 10.10.x.x
MGM - 192.168.0.x
Storage - 192.168.1.x

[Oleksiy Molchanov (omolchanov)]

Hi, Malgorzata

I was trying to debug your issue using logs you provided, but they do not contain information we need.

Would you be so kind to:

1. Change 1-st line in /usr/lib/ocf/resource.d/fuel/ns_haproxy to "#!/bin/bash -x"
2. Run 'pcs resource disable p_haproxy' on controller that is affected
3. Wait few seconds and provide me with output of 'pcs resource debug-start p_haproxy'

[Malgorzata Rembas (mrembas)]

Hi Unfortunately the setup was reuse for other deployment so I can't provide any more logs.

[Alexander Kurenyshev (akurenyshev)]

Moved to Invalid, please feel free to reopen this bug with an attached info Oleksiy has requested.