Vcenter only-provisioning : Need to have consistent naming for the contrail-vm interface

Bug #1781121 reported by Sandip Dey on 2018-07-11
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Juniper Openstack
Status tracked in Trunk
R5.0
Fix Committed
Medium
amudhar
Trunk
Fix Committed
Medium
amudhar

Bug Description

Build R5.0 build 133

We are naming the ctrl-data interfaces differently for different setups

For example

1.SRIOV setup ctrl-data interface names : eth20,eth21
2.single ctrl-data setup ctrl-data interface name : ens192
3.pcie pass through ctrl-data interface names ens160f0,ens160f1 (sometimes eth0 in when re-ran the provisining)

We need to have a consistent naming for the ctrl-data interface.

Same is the case for the vmware physical interface

1.pcie enabled vm - vmware physical interface ens192
2.sriov enabled vm - vmware physical interface ens192
3.single ctrl-data interface - vmware physical interface ens224 and ens192 connected to ctrl-data-dvs

In fab provisioning,vmware physical interface was always 'eth1'
ctrl-data interfaces used to start with eth20.It was a better design as interfaces were predictable,did not depend on which kind of setup was that.Also it was easier for automation code to assign ip addresses (since it was predictable), and create bond interfaces.

amudhar (amudha) wrote :
Download full text (11.1 KiB)

From: Amudha R <email address hidden>
Date: Wednesday, July 11, 2018 at 1:05 AM
To: Sandip Dey <email address hidden>, Sachchidanand Vaidya <email address hidden>, Ram Yadav <email address hidden>
Cc: Sudheendra Rao <email address hidden>
Subject: Re: R5.0.1 vcenter provisioning

Please read inline.

Thanks,
Amudha

From: Sandip Dey <email address hidden>
Date: Tuesday, July 10, 2018 at 11:36 AM
To: Amudha R <email address hidden>, Sachchidanand Vaidya <email address hidden>, Ram Yadav <email address hidden>
Cc: Sudheendra Rao <email address hidden>
Subject: Re: R5.0.1 vcenter provisioning

My setup is up with sriov/pcie/ctrl-data single interface and eam.

You can use if you want to check things

10.204.216.61 - controller
All the files are here /root/contrail-ansible-deployer

Amudha, the pcie interface came up with name eth0 this time.
[Amudha] - Did you mean eth20?

We need to fix the interface name something consistent for all kind of configuration of the ctrl-data interface.
[Amudha] - ctrl-data interface is a vmxnet3 interface and comes up as ens*. In case of PCI and SR-IOV interfaces,
they are ixgbe interfaces and will come up as eth* interfaces.

I have a script to give ip to the ctrl-data interface.If the the name is not same for all the compute-vms,script will fail.
[Amudha] Can your script check for ens and eth* as well? And may be use the driver type for additional check if needed?..

Also will write a script to create bond .I would assume the interfaces names to be eth20 and eth21 for bond.
[Amudha] Should work with the current vmdk.

The vmwarephysicalinterface should also ideally be same for any kind of configuration for ctrl-data.That was the case for fab.It used to be always ‘eth1’
[Amudha] I am not sure about this.. Would think it to be eth2 in case of ctrl-data interface defined.

We have to get rid of the situation where vmwarephysicalinterface=ens192 for some case and vmwarephysicalinterface=ens224 for some other.
[Amudha] Each scenario creates different interfaces, and this is hidden from user. Would not prefer changing this in 5.0.1 beta.
We can have a discussion once all are available, and see if it really needs changes.

Regards
Sandip

From: Sandip Dey <email address hidden>
Date: Tuesday, July 10, 2018 at 7:51 PM
To: Amudha R <email address hidden>, Sachchidanand Vaidya <email address hidden>, Ram Yadav <email address hidden>
Cc: Sudheendra Rao <email address hidden>
Subject: Re: R5.0.1 vcenter provisioning

Hi Amudha

Please see inline….

Ram,

What is the behavior if we create a pg in the vm dvs from vcenter ui for which vn is not there in api server?Does the vcenter-plugin delete it since, its not there in the api server?

I wanted to delete the contrail-vm/vm dvs from vcenter and rerun the vcenter provisioning when rest of the contrail components are up and running.I see the dvs pg on vm dvs getting deleted as soon as it was created.
I am not sure if the vcenter plugin deleting it,because there is not explicit log for deletion.I see only this.

Any port group created on that vm dvs is getting deleted.

[2018-07-10 12:50:01,070] [INFO ] [Thread-3:VCenterDB@794] [Found <dpg c4k4u14_dvpg, dvs c4k4_dvs, datacenter c4_da...

amudhar (amudha) wrote :

3.pcie pass through ctrl-data interface names ens160f0,ens160f1 (sometimes eth0 in when re-ran the provisioning)

This shouldn't be the case, and does not happen on my ContrailVMs deployed on two different
ESXi hosts.

Sandip Dey (sandipd) wrote :

today I brought up again with build 139.
Pcie setup got interface 'eth0'

[sandipd@nodem4 ~]$ ssh root@10.204.216.183
root@10.204.216.183's password:
Last login: Wed Jul 4 22:30:28 2018 from 172.29.108.219
[root@nodek6-compute-vm ~]# ifconfig
ens160: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
        inet 10.204.216.183 netmask 255.255.255.0 broadcast 10.204.216.255
        inet6 fe80::9288:aba4:d2e0:e6e prefixlen 64 scopeid 0x20<link>
        inet6 fe80::56da:6476:c3b1:4d9 prefixlen 64 scopeid 0x20<link>
        inet6 fe80::8afe:b1fc:ae8c:3231 prefixlen 64 scopeid 0x20<link>
        ether 00:50:56:aa:aa:05 txqueuelen 1000 (Ethernet)
        RX packets 3988 bytes 265441 (259.2 KiB)
        RX errors 0 dropped 0 overruns 0 frame 0
        TX packets 128 bytes 14155 (13.8 KiB)
        TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

ens224: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
        ether 00:50:56:bb:bb:04 txqueuelen 1000 (Ethernet)
        RX packets 2 bytes 120 (120.0 B)
        RX errors 0 dropped 0 overruns 0 frame 0
        TX packets 48 bytes 7968 (7.7 KiB)
        TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
        ether 00:25:90:c7:7f:f4 txqueuelen 1000 (Ethernet)
        RX packets 49 bytes 13656 (13.3 KiB)
        RX errors 0 dropped 1 overruns 0 frame 0
        TX packets 46 bytes 7284 (7.1 KiB)
        TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
        device memory 0xfd400000-fd4fffff

lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
        inet 127.0.0.1 netmask 255.0.0.0
        inet6 ::1 prefixlen 128 scopeid 0x10<host>
        loop txqueuelen 1000 (Local Loopback)
        RX packets 0 bytes 0 (0.0 B)
        RX errors 0 dropped 0 overruns 0 frame 0
        TX packets 0 bytes 0 (0.0 B)
        TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

Sandip Dey (sandipd) wrote :
Download full text (3.4 KiB)

My setup is up with sriov/pcie/ctrl-data single interface and eam.

You can use if you want to check things

10.204.216.61 - controller
All the files are here /root/contrail-ansible-deployer

Amudha, the pcie interface came up with name eth0 this time.
[Amudha] - Did you mean eth20?
+++sandipd: It was eth0.When I brought up 2 pcie interfaces, they were ens160f0,ens160f1

ens160: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
        inet 10.204.216.183 netmask 255.255.255.0 broadcast 10.204.216.255
        inet6 fe80::250:56ff:feaa:aa05 prefixlen 64 scopeid 0x20<link>
        ether 00:50:56:aa:aa:05 txqueuelen 1000 (Ethernet)
        RX packets 1690602 bytes 579191961 (552.3 MiB)
        RX errors 0 dropped 2 overruns 0 frame 0
        TX packets 171108 bytes 232670877 (221.8 MiB)
        TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

ens192: flags=4419<UP,BROADCAST,RUNNING,PROMISC,MULTICAST> mtu 1500
        inet6 fe80::250:56ff:febb:bb04 prefixlen 64 scopeid 0x20<link>
        ether 00:50:56:bb:bb:04 txqueuelen 1000 (Ethernet)
        RX packets 207 bytes 66846 (65.2 KiB)
        RX errors 0 dropped 0 overruns 0 frame 0
        TX packets 8 bytes 648 (648.0 B)
        TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
        inet6 fe80::225:90ff:fec7:7ff4 prefixlen 64 scopeid 0x20<link>
        ether 00:25:90:c7:7f:f4 txqueuelen 1000 (Ethernet)
        RX packets 127288 bytes 8523996 (8.1 MiB)
        RX errors 0 dropped 17 overruns 0 frame 0
        TX packets 140608 bytes 8934212 (8.5 MiB)
        TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
        device memory 0xfd400000-fd4fffff

We need to fix the interface name something consistent for all kind of configuration of the ctrl-data interface.
[Amudha] - ctrl-data interface is a vmxnet3 interface and comes up as ens*. In case of PCI and SR-IOV interfaces,
they are ixgbe interfaces and will come up as eth* interfaces.

I have a script to give ip to the ctrl-data interface.If the the name is not same for all the compute-vms,script will fail.
[Amudha] Can your script check for ens and eth* as well? And may be use the driver type for additional check if needed?..

Also will write a script to create bond .I would assume the interfaces names to be eth20 and eth21 for bond.
[Amudha] Should work with the current vmdk.

The vmwarephysicalinterface should also ideally be same for any kind of configuration for ctrl-data.That was the case for fab.It used to be always ‘eth1’
[Amudha] I am not sure about this.. Would think it to be eth2 in case of ctrl-data interface defined.

+++sandipd:It was always ‘eth1’.Please check the fab code and document attached

We have to get rid of the situation where vmwarephysicalinterface=ens192 for some case and vmwarephysicalinterface=ens224 for some other.
[Amudha] Each scenario creates different interfaces, and this is hidden from user. Would not prefer changing this in 5.0.1 beta.
We can have a discussion once all are available, and see if it really needs changes.

+++sandipd:'Each scenario creates different interfaces’.This is th...

Read more...

information type: Proprietary → Public

Review in progress for https://review.opencontrail.org/45469
Submitter: amudha ramachandran (<email address hidden>)

Review in progress for https://review.opencontrail.org/45470
Submitter: amudha ramachandran (<email address hidden>)

amudhar (amudha) wrote :

- Fix involves - not executing /firstboot at ContrailVM creation,
  this will result in the following names for the ContrailVM

                mgmt ctrl-data vmware_physical_intf

single-intf ens160 -- ens192

multi-intf ens160 ens192 ens224
(ctrl-data)

multi-intf ens160 ens192 ens224
(pci/sriov)

  This change is done and saved in the ContrailVM vmdk at:
  /cs-shared/contrail-vcenter/vmdk/centos-7.5/LATEST

- When finding vmware_physical_interface, exclude PCI/SRIOV
  interfaces (driver_type should be vmxnet3)
  https://review.opencontrail.org/#/c/45469/

Sandip Dey (sandipd) wrote :
Download full text (6.5 KiB)

Hi Amudha

I took the latest ovf from the specified location.

Ran ansible-playbook playbooks/vcenter.yml . There is still something missing.Please fix it.

[root@nodek4-compute-vm ~]#
[root@nodek4-compute-vm ~]# ifconfig
ens160: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
        inet6 fe80::9288:aba4:d2e0:e6e prefixlen 64 scopeid 0x20<link>
        ether 00:50:56:a4:92:2a txqueuelen 1000 (Ethernet)
        RX packets 38 bytes 9120 (8.9 KiB)
        RX errors 0 dropped 1 overruns 0 frame 0
        TX packets 54 bytes 8996 (8.7 KiB)
        TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

ens192: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
        inet 10.204.216.181 netmask 255.255.255.0 broadcast 10.204.216.255
        inet6 fe80::a0af:9125:f772:13ad prefixlen 64 scopeid 0x20<link>
        ether 00:50:56:aa:aa:03 txqueuelen 1000 (Ethernet)
        RX packets 476 bytes 44433 (43.3 KiB)
        RX errors 0 dropped 1 overruns 0 frame 0
        TX packets 73 bytes 9173 (8.9 KiB)
        TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

ens224: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
        inet6 fe80::9ee2:b14c:219:186b prefixlen 64 scopeid 0x20<link>
        ether 00:50:56:a4:20:56 txqueuelen 1000 (Ethernet)
        RX packets 0 bytes 0 (0.0 B)
        RX errors 0 dropped 0 overruns 0 frame 0
        TX packets 51 bytes 8994 (8.7 KiB)
        TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
        inet 127.0.0.1 netmask 255.0.0.0
        inet6 ::1 prefixlen 128 scopeid 0x10<host>
        loop txqueuelen 1000 (Local Loopback)
        RX packets 32 bytes 2592 (2.5 KiB)
        RX errors 0 dropped 0 overruns 0 frame 0
        TX packets 32 bytes 2592 (2.5 KiB)
        TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

[root@nodek4-compute-vm ~]#

[root@nodek6-compute-vm ~]#
[root@nodek6-compute-vm ~]#
[root@nodek6-compute-vm ~]# ifconfig
ens160: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
        ether 00:25:90:c7:7f:f4 txqueuelen 1000 (Ethernet)
        RX packets 44 bytes 11540 (11.2 KiB)
        RX errors 0 dropped 0 overruns 0 frame 0
        TX packets 59 bytes 9290 (9.0 KiB)
        TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
        device memory 0xfd400000-fd4fffff

ens192: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
        inet 10.204.216.183 netmask 255.255.255.0 broadcast 10.204.216.255
        inet6 fe80::3a64:a693:b05:b908 prefixlen 64 scopeid 0x20<link>
        ether 00:50:56:aa:aa:05 txqueuelen 1000 (Ethernet)
        RX packets 654 bytes 57651 (56.2 KiB)
        RX errors 0 dropped 1 overruns 0 frame 0
        TX packets 76 bytes 9519 (9.2 KiB)
        TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

ens224: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
        ether 00:50:56:a4:f6:c6 txqueuelen 1000 (Ethernet)
        RX packets 0 bytes 0 (0.0 B)
        RX errors 0 dropped 0 overruns 0 frame 0
        TX packets 50 bytes 8652 (8.4 KiB)
        TX errors 0 dropped 0 overruns 0 carrier 0 col...

Read more...

Sandip Dey (sandipd) on 2018-08-10
tags: added: blocker

Reviewed: https://review.opencontrail.org/45469
Committed: http://github.com/Juniper/contrail-container-builder/commit/123d46112cce27bdf3ff467589a02cda3dcf191f
Submitter: Zuul v3 CI (<email address hidden>)
Branch: master

commit 123d46112cce27bdf3ff467589a02cda3dcf191f
Author: Amudha <email address hidden>
Date: Thu Aug 9 20:05:32 2018 -0700

vCenter: Fix ContrailVM interface names

- Fix involves - not executing /firstboot at ContrailVM creation,
this will result in the following names for the ContrailVM

mgmt ctrl-data vmware_physical_intf

single-intf ens160 -- ens192

multi-intf ens160 ens192 ens224
(ctrl-data)

multi-intf ens160 ens192 ens224
(pci/sriov)

This change is done and saved in the ContrailVM vmdk.

- When finding vmware_physical_interface, exclude PCI/SRIOV
interfaces (driver_type should be vmxnet3)

Change-Id: Iffed0cd9125cd98e5b232ce30428f3ece6a19986
Closes-Bug: #1781121

tags: added: sanity-blocker
removed: blocker
tags: added: sanityblocker
removed: sanity-blocker

Reviewed: https://review.opencontrail.org/45470
Committed: http://github.com/Juniper/contrail-container-builder/commit/22259f0a38eb77b29351533dd956891025336daa
Submitter: Zuul v3 CI (<email address hidden>)
Branch: R5.0

commit 22259f0a38eb77b29351533dd956891025336daa
Author: Amudha <email address hidden>
Date: Thu Aug 9 20:05:32 2018 -0700

vCenter: Fix ContrailVM interface names

- Fix involves - not executing /firstboot at ContrailVM creation,
this will result in the following names for the ContrailVM

mgmt ctrl-data vmware_physical_intf

single-intf ens160 -- ens192

multi-intf ens160 ens192 ens224
(ctrl-data)

multi-intf ens160 ens192 ens224
(pci/sriov)

This change is done and saved in the ContrailVM vmdk.

- When finding vmware_physical_interface, exclude PCI/SRIOV
interfaces (driver_type should be vmxnet3)

Change-Id: Iffed0cd9125cd98e5b232ce30428f3ece6a19986
Closes-Bug: #1781121

amudhar (amudha) wrote :

mgmt
----
- The mgmt interface will be ens160.

ctrl/data
---------
- The ctrl-data interface will be the next added interface - ens192, and add on as ens224, etc.
- In case of PCI/SRIOV, interfaces will start at ens320, and add on as ens352 and so on.

vmware_physical_interface
-------------------------
- In any setup, the last available vmxnet3 interface will be the vmware_physical_interface.

                   mgmt ctrl-data vmware_phy_int
single-intf ens160 --- ens192
multi-intf ens160 ens192 ens224
multi-intf bond ens160 ens192,ens224 ens256
pci/sriov ens160 ens320 ens192
pci-sriov bond ens160 ens320,ens352 ens192

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers