RHOSP: upgrade from 3.2.3 to 4.1.1 failed : 'vnc_cfg_api_server.gen.resource_xsd.BgpSessionAttributes'> does not have field route_origin_override

[shajuvk]

[root@contrailcontroller03 heat-admin]# python /opt/contrail/utils/provision_control.py --host_name contrailcontroller03.vimbox.com --host_ip 192.168.215.20 --router_asn 64512 --ibgp_auto_mesh --api_server_ip 192.168.215.12 --api_server_port 8082 --api_server_use_ssl false --admin_user admin --admin_password n23m79Yp8C3UrYuKRv2cqMYVb --admin_tenant admin --oper add
Traceback (most recent call last):
  File "/opt/contrail/utils/provision_control.py", line 230, in <module>
    main()
  File "/opt/contrail/utils/provision_control.py", line 226, in main
    ControlProvisioner(args_str)
  File "/opt/contrail/utils/provision_control.py", line 64, in __init__
    self._args.address_families, self._args.md5, self._args.local_autonomous_system)
  File "/opt/contrail/utils/provision_bgp.py", line 96, in add_bgp_router
    vnc_lib.bgp_router_create(bgp_router_obj)
  File "/usr/lib/python2.7/site-packages/vnc_api/vnc_api.py", line 42, in wrapper
    return func(self, *args, **kwargs)
  File "/usr/lib/python2.7/site-packages/vnc_api/vnc_api.py", line 500, in _object_create
    rest.OP_POST, obj_cls.create_uri, data=json_body)
  File "/usr/lib/python2.7/site-packages/vnc_api/vnc_api.py", line 889, in _request_server
    retry_after_authn=retry_after_authn, retry_count=retry_count)
  File "/usr/lib/python2.7/site-packages/vnc_api/vnc_api.py", line 996, in _request
    raise BadRequest(status, content)
cfgm_common.exceptions.BadRequest: Bad reference in create: Error validating reference bgp_router_refs value {u'to': [u'default-domain', u'default-project', u'ip-fabric', u'__default__', u'contrailcontroller01.vimbox.com'], u'attr': {u'session': [{u'attributes': [{u'auth_data': None, u'route_origin_override': None, u'admin_down': False, u'private_as_action': None, u'as_override': False, u'bgp_router': None, u'local_autonomous_system': 0, u'passive': False, u'family_attributes': [], u'hold_time': 0, u'loop_count': 0, u'address_families': {u'family': [u'route-target', u'inet-vpn', u'e-vpn', u'erm-vpn', u'inet6-vpn']}}], u'uuid': None}]}} class <class 'vnc_cfg_api_server.gen.resource_xsd.BgpSessionAttributes'> does not have field route_origin_override

===
Attached rpms list from contrail-controller node.

[shajuvk (shajuvk)]

rpm list from contrail-config node

[Shivayogi Ugaji (shivayogi123)]

From: Anantharamu Suryanarayana <email address hidden>
Date: Tuesday, July 3, 2018 at 5:54 PM
To: Shivayogi Ugaji <email address hidden>, Nagendra Maynattamai <email address hidden>, "Shaju V.K" <email address hidden>, Sanae El Yousfi <email address hidden>
Cc: Jeba Paulaiyan <email address hidden>, Abhay Joshi <email address hidden>, Nikhil Bansal <email address hidden>
Subject: Re: upgrade error

+Nikhil

Hi Nikhil, can you please take a look ? I am off for the rest of this week.

Thanks

Regards,
Ananth

From: Shivayogi Ugaji
Sent: Tuesday, July 3, 2018 5:08:26 PM
To: Nagendra Maynattamai; Shaju V.K; Sanae El Yousfi
Cc: Jeba Paulaiyan; Anantharamu Suryanarayana; Abhay Joshi
Subject: Re: upgrade error

+Ananth.

Hi Ananth,

Can you please take a look.

Thanks
Shivayogi

From: Nagendra Maynattamai <email address hidden>
Date: Tuesday, July 3, 2018 at 5:02 PM
To: "Shaju V.K" <email address hidden>, Sanae El Yousfi <email address hidden>, Shivayogi Ugaji <email address hidden>
Cc: Jeba Paulaiyan <email address hidden>
Subject: Re: upgrade error

Shivayogi,
Looks like introduced by https://github.com/Juniper/contrail-controller/commit/7bbf59bfce0214b79629965738908074340ee43e Could anyone from control team take a look?

I don’t see RouteOriginOverride in R3.2 and I think the upgrade is properly handled.
Regards,
Nagendra Prasath

From: "Shaju V.K" <email address hidden>
Date: Tuesday, July 3, 2018 at 3:05 PM
To: Sanae El Yousfi <email address hidden>, Nagendra Maynattamai <email address hidden>, Shivayogi Ugaji <email address hidden>
Cc: Jeba Paulaiyan <email address hidden>
Subject: Re: upgrade error

Hi Nagendra,

Openstack provided the steps for upgrade, the command used is “openstack overcloud update stack -i overcloud”

https://www.juniper.net/documentation/en_US/contrail4.1/topics/concept/upgrade-326-to-41-rhoso.html
Please check the session upgrade overcloud.

Thanks,
Shaju

From: Sanae El Yousfi <email address hidden>
Date: Tuesday, July 3, 2018 at 3:00 PM
To: Nagendra Maynattamai <email address hidden>, Shivayogi Ugaji <email address hidden>
Cc: "Shaju V.K" <email address hidden>, Jeba Paulaiyan <email address hidden>
Subject: Re: upgrade error

Hello,
the upgrade is done using openstack stack update overcloud it is a yum update, thanks.
regards

Envoyé depuis mon smartphone Samsung Galaxy.

-------- Message d'origine --------
De : Nagendra Maynattamai <email address hidden>
Date : 03/07/2018 23:56 (GMT+01:00)
À : Shivayogi Ugaji <email address hidden>
Cc : "Shaju V.K" <email address hidden>, Sanae El Yousfi <email address hidden>, Jeba Paulaiyan <email address hidden>
Objet : Re: upgrade error

Quick question: how was the upgrade done? Yum update/upgrade?
Regards,
Nagendra Maynattamai

On Jul 3, 2018, at 2:51 PM, Shivayogi Ugaji <email address hidden> wrote:
+Nagendra

Nagendra, can you please take a look.

Regards
Shivayogi

From: "Shaju V.K" <email address hidden>
Date: Tuesday, July 3, 2018 at 2:18 PM
To: Shivayogi Ugaji <email address hidden>
Cc: Sanae El Yousfi <email address hidden>, Jeba Paulaiyan <email address hidden>
Subject: Re: upgrade error

Hi Shivayogi,

We are hitting an issue at Orange environment during upgrade. Looks l...

[Nikhil Bansal (nikhilb-u)]

Need access to setup to debug it further. It was resolved by restarting config services so looks like some timing issue during upgrade

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/46090
Submitter: alexey-mr (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/46090
Committed: http://github.com/Juniper/puppet-contrail/commit/b5ec14743d6c13cf847f277f1d28a404931c7b0e
Submitter: Zuul (<email address hidden>)
Branch: R4.1

commit b5ec14743d6c13cf847f277f1d28a404931c7b0e
Author: alexey-mr <email address hidden>
Date: Mon Sep 10 15:31:14 2018 +0300

Do config-api restart on each puppet run.

This avoid issues with upgrade, config change, etc.

Change-Id: Ie729d8d0173636dc5d54eb5a69f6f33e69c6ef16
Closes-Bug: #1779943

[Jeba Paulaiyan (jebap)]

Hi, Jeba,
https://bugs.launchpad.net/juniperopenstack/+bug/1779943
it looks that it is applicable only for 4.1 ..
maybe it is worth to remove others