Refactor CVM

Bug #1777404 reported by Adrian Szczepański on 2018-06-18
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Juniper Openstack
Status tracked in Trunk
R5.0
Fix Committed
Medium
Adrian Szczepański
Trunk
Fix Committed
Medium
Adrian Szczepański
OpenContrail
New
Undecided
Unassigned

Bug Description

Contrail-VCenter-Manager is growing fast and adding new functionalities/fixing bugs becomes more and more difficult. We need to put some effort into making the code more readable and maintainable.

Please, submit any changes involving refactoring with Partial-Bug tag.

Review in progress for https://review.opencontrail.org/43907
Submitter: Adrian Szczepanski (<email address hidden>)

Review in progress for https://review.opencontrail.org/43908
Submitter: Adrian Szczepanski (<email address hidden>)

Reviewed: https://review.opencontrail.org/43908
Committed: http://github.com/Juniper/contrail-vcenter-manager/commit/1cf1aa642d0a283c0f92ae3e0ca823c69be5a019
Submitter: Zuul v3 CI (<email address hidden>)
Branch: master

commit 1cf1aa642d0a283c0f92ae3e0ca823c69be5a019
Author: Adrian Szczepański <email address hidden>
Date: Thu Jun 14 14:06:00 2018 +0200

Move port managing to a separate service

Add read_port to VRouterAPIClient
Add _port_needs_an_update method in VMI Service
Add VRouterPortService to manage ports on vRouter

Partial-Bug: #1777404
Change-Id: I5515d46bb12e575f024184252ada244759264331

Jeba Paulaiyan (jebap) on 2018-06-19
tags: added: vmware

Review in progress for https://review.opencontrail.org/43990
Submitter: Adrian Szczepanski (<email address hidden>)

Reviewed: https://review.opencontrail.org/43907
Committed: http://github.com/Juniper/contrail-vcenter-manager/commit/48d0407833b0e2a453516d546bdcca1edf674364
Submitter: Zuul v3 CI (<email address hidden>)
Branch: R5.0

commit 48d0407833b0e2a453516d546bdcca1edf674364
Author: Adrian Szczepański <email address hidden>
Date: Thu Jun 14 14:06:00 2018 +0200

Move port managing to a separate service

Add read_port to VRouterAPIClient
Add _port_needs_an_update method in VMI Service
Add VRouterPortService to manage ports on vRouter

Partial-Bug: #1777404
Change-Id: I5515d46bb12e575f024184252ada244759264331

OpenContrail Admin (ci-admin-f) wrote :

Reviewed: https://review.opencontrail.org/43990
Committed: http://github.com/Juniper/contrail-vcenter-manager/commit/7f3b6588ace16c7ad24412239c14852e5a4b9afc
Submitter: Zuul v3 CI (<email address hidden>)
Branch: master

commit 7f3b6588ace16c7ad24412239c14852e5a4b9afc
Author: Adrian Szczepański <email address hidden>
Date: Wed Jun 20 11:14:55 2018 +0200

VCenter Port management refactor

Use Criteria to fetch DVPorts
Add method for retrieving DVS by UUID
Change VCenterPort constructor to consume whole device object
Change set_vlan_id signature to consume whole VCenterPort object
Change get_vlan_id signature to consume only VCenterPort object
Change restore_vlan_id signature to consume only VCenterPort object
Extract fetching port from dvs to a function
Extract creating inherited port config spec to a function
Remove unused get_ip_pool methods from VCenterAPIClient
Catch exceptions for vrouter-uuid annotation reading
Remove NoneType check from create_and_read_instance_ip

Partial-Bug: #1777404
Change-Id: I271ff574509fbbb070acb0ddaa0a35f5a76ea5ce

Review in progress for https://review.opencontrail.org/44019
Submitter: Adrian Szczepanski (<email address hidden>)

Reviewed: https://review.opencontrail.org/44019
Committed: http://github.com/Juniper/contrail-vcenter-manager/commit/f528dba3968d69b5dd97345757561ec614f9ae51
Submitter: Zuul v3 CI (<email address hidden>)
Branch: R5.0

commit f528dba3968d69b5dd97345757561ec614f9ae51
Author: Adrian Szczepański <email address hidden>
Date: Wed Jun 20 11:14:55 2018 +0200

VCenter Port management refactor

Use Criteria to fetch DVPorts
Add method for retrieving DVS by UUID
Change VCenterPort constructor to consume whole device object
Change set_vlan_id signature to consume whole VCenterPort object
Change get_vlan_id signature to consume only VCenterPort object
Change restore_vlan_id signature to consume only VCenterPort object
Extract fetching port from dvs to a function
Extract creating inherited port config spec to a function
Remove unused get_ip_pool methods from VCenterAPIClient
Catch exceptions for vrouter-uuid annotation reading
Remove NoneType check from create_and_read_instance_ip

Partial-Bug: #1777404
Change-Id: I271ff574509fbbb070acb0ddaa0a35f5a76ea5ce
(cherry picked from commit 7f3b6588ace16c7ad24412239c14852e5a4b9afc)

Review in progress for https://review.opencontrail.org/44052
Submitter: Adrian Szczepanski (<email address hidden>)

Review in progress for https://review.opencontrail.org/44088
Submitter: Adrian Szczepanski (<email address hidden>)

Reviewed: https://review.opencontrail.org/44052
Committed: http://github.com/Juniper/contrail-vcenter-manager/commit/193f076b57daf931977650cc9ed75565803d6f07
Submitter: Zuul v3 CI (<email address hidden>)
Branch: master

commit 193f076b57daf931977650cc9ed75565803d6f07
Author: Adrian Szczepański <email address hidden>
Date: Fri Jun 22 11:19:30 2018 +0200

VMI sync refactor

Add a list of VMIs to VM Model
Update VMIs based on vmis_to_update list
Delete VMIs based on vmis_to_delete list

Partial-Bug: #1777404
Change-Id: I39e2d38a0a5f4aa1ca185b9ab990742955fe292c

Reviewed: https://review.opencontrail.org/44088
Committed: http://github.com/Juniper/contrail-vcenter-manager/commit/d13cef4c94791043ed2aa6d6a5bc8803c36f2880
Submitter: Zuul v3 CI (<email address hidden>)
Branch: R5.0

commit d13cef4c94791043ed2aa6d6a5bc8803c36f2880
Author: Adrian Szczepański <email address hidden>
Date: Fri Jun 22 11:19:30 2018 +0200

VMI sync refactor

Add a list of VMIs to VM Model
Update VMIs based on vmis_to_update list
Delete VMIs based on vmis_to_delete list

Partial-Bug: #1777404
Change-Id: I39e2d38a0a5f4aa1ca185b9ab990742955fe292c

Review in progress for https://review.opencontrail.org/44273
Submitter: Adrian Szczepanski (<email address hidden>)

Reviewed: https://review.opencontrail.org/44273
Committed: http://github.com/Juniper/contrail-vcenter-manager/commit/d966bab78e8aa9d7a082aa2791b5de75ea5a0d38
Submitter: Zuul v3 CI (<email address hidden>)
Branch: master

commit d966bab78e8aa9d7a082aa2791b5de75ea5a0d38
Author: Adrian Szczepański <email address hidden>
Date: Mon Jul 2 09:39:39 2018 +0200

Controller refactor

Create Update Handler for storing all event handlers
Move all update event handling to a new handler
Add AbstractChangeHandler
Add GuestNetHandler for guest.net updates
Add VmwareToolsStatusHandler

Partial-Bug: #1777404
Change-Id: Ie879f48007b8b64c2e6f20a872781022bd2e9622

Review in progress for https://review.opencontrail.org/44363
Submitter: Krzysztof Gromadzki (<email address hidden>)

Reviewed: https://review.opencontrail.org/44363
Committed: http://github.com/Juniper/contrail-vcenter-manager/commit/8721db41884e2d6e681e3bdb530c77b29ca5f1a7
Submitter: Zuul v3 CI (<email address hidden>)
Branch: R5.0

commit 8721db41884e2d6e681e3bdb530c77b29ca5f1a7
Author: Adrian Szczepański <email address hidden>
Date: Fri Jun 29 15:21:41 2018 +0200

Fix VMI delete

Remove VMIs from to_delete queue

Closes-Bug: #1779321

Use one global VLAN ID pool

Store DVSwitch as an attribute of VCenterAPIClient
Add VlanIdPool to VMI Service
Modify assign_vlan_id method to use global id pool
Modify restore_vlan_id method to use global id pool
Remove unused methods and tests
Add DVSwitch name as a config parameter

Closes-Bug: #1779100

More informative logs for API Clients
Changed logging level for database queries
Logging for CVM Models

Partial-Bug: #1779323

Controller refactor

Create Update Handler for storing all event handlers
Move all update event handling to a new handler
Add AbstractChangeHandler
Add GuestNetHandler for guest.net updates
Add VmwareToolsStatusHandler

Partial-Bug: #1777404

CVM checks ownership of VNC entities before modification

Closes-Bug: #1779885

Handle External IPAM cases

Create Instance IP for external IPAM
Remove unused functions, tests and imports

Closes-Bug: #1779896

React to powerState property changes

Add PowerStateHandler
Add runtime.powerState to observed properties
Add method for updating powerState in VM Service
Update update_vmware_tools_status method
Update update_nic method
Remove excessive logs

Change-Id: I16210e5c7ae53a111e284ea1329945d245c7b96f
Closes-Bug: #1778758

Review in progress for https://review.opencontrail.org/44383
Submitter: Adrian Szczepanski (<email address hidden>)

Review in progress for https://review.opencontrail.org/44384
Submitter: Adrian Szczepanski (<email address hidden>)

Reviewed: https://review.opencontrail.org/44383
Committed: http://github.com/Juniper/contrail-vcenter-manager/commit/0a12e4f2374480f8ca0405de93bcdbedca26c8ed
Submitter: Zuul v3 CI (<email address hidden>)
Branch: master

commit 0a12e4f2374480f8ca0405de93bcdbedca26c8ed
Author: Adrian Szczepański <email address hidden>
Date: Thu Jul 5 11:48:54 2018 +0200

Models refactoring

Remove unused functions and methods
Remove vmware_vm from VM Model
Move ID_PERMS to constants.py

Partial-Bug: #1777404
Change-Id: Iaf6de737e8898d32098c8de7db0ebdc115ac32ba

Reviewed: https://review.opencontrail.org/44384
Committed: http://github.com/Juniper/contrail-vcenter-manager/commit/165c2996ff6fd5f30988c8a10c75755ed8dfb17e
Submitter: Zuul v3 CI (<email address hidden>)
Branch: R5.0

commit 165c2996ff6fd5f30988c8a10c75755ed8dfb17e
Author: Adrian Szczepański <email address hidden>
Date: Thu Jul 5 11:48:54 2018 +0200

Models refactoring

Remove unused functions and methods
Remove vmware_vm from VM Model
Move ID_PERMS to constants.py

Partial-Bug: #1777404
Change-Id: Iaf6de737e8898d32098c8de7db0ebdc115ac32ba

Download full text (19.5 KiB)

Marcin/Adrian,
Can you follow the instruction in the email from VMware and try out the python script.
Also modify the CVM cord accordingly.
Share the patch for the same by tomorrow sunnyvale time.

Thanks,
Ram

On 8/27/18, 9:03 PM, "VMware Technical Support" <email address hidden> wrote:

>** Please do not change the subject line of this e-mail if you wish to respond. **
>
>Hi Ram,
>
>Sure, we will connect with them to get the requested details. In the meantime kindly test the script attached with this email and let us know the test results. Please see the below list of parameters and example:
>
>Parameters always needed to run the script:
>--host --user --pwd --dvs --obj
>
>1. Create a portgroup and enable vlan override on the dvportgroup. --obj set to portgroup, and --portgroup is specified.
>
># python3 port_reconfigure.py --host=10.192.229.18 --<email address hidden>' --pwd='Admin!23' --dvs=DSwitch --obj=portgroup --portgroup='cute111'
>successfully added portgroup:cute111 on dvs:DSwitch
>successfully enabled vlan override on portgroup:cute111
>
>2. Modify the vlan. --obj set to port, and --key and --vlan is needed.
>
>python3 port_reconfigure.py --host=10.192.229.18 --<email address hidden>' --pwd='Admin!23' --dvs=DSwitch --obj=port --key=21 --vlan=1024
>successfully modified port 21 's vlan to:1024 on dvs:DSwitch
>
>3. For more details, you can use #python3 port_reconfigure.py --help
>
>Please let us know if you find any difficulties in executing the script. We will wait for your further response.
>
>Thank you.
>
>Case Reported: Issue regarding VLAN override with overlapping VLAN’s for VM’s belonging to same Port Group but on different ESXi
>Status: Waiting on Customer
>Next Touch Date: 30th August 2018
>
>Regards,
>Chetan Patidar
>VMware SDK Developer Support
>Global Support Services, VMware Inc.
>Global Support Phone Numbers: https://urldefense.proofpoint.com/v2/url?u=https-3A__www.vmware.com_support_contacts_us-5Fsupport&d=DwIFaQ&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=5cGBwnyUCiMK2Tm6UX_iIiml04qYJeDz4xQzoG2Hi8E&m=HJPbXgvwaHljGcPH2s36sLkauOImQiky90mMrpVweVI&s=leZ5XIpageOxaytxnLquhj6jDc16OjOJ3u3wGVFMVmA&e=
>
>Office Hours: Monday - Friday 06:00 AM – 3:00 PM IST
>
>If you cannot reach me, kindly contact my direct manager, Nalin Kumar Gupta, at <email address hidden>. Satisfied customers are our top priority.
>
>VMware Skyline: VMware’s new proactive support technology; sign up here - https://urldefense.proofpoint.com/v2/url?u=https-3A__app.smartsheet.com_b_form_f44fcd08d8a04234b6360e1b60051840&d=DwIFaQ&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=5cGBwnyUCiMK2Tm6UX_iIiml04qYJeDz4xQzoG2Hi8E&m=HJPbXgvwaHljGcPH2s36sLkauOImQiky90mMrpVweVI&s=MdUTZq2nMRHSi3Umh0i9Vx8ZAO6hsZmmzr1eGP83U4Y&e=
>
>--------------- Original Message ---------------
>From: Ram Yadav [<email address hidden>]
>Sent: 8/28/2018 9:09 AM
>To: <email address hidden>; <email address hidden>; <email address hidden>; <email address hidden>; <email address hidden>
>Cc: <email address hidden>
>Subject: Re: VMware Support Request 18890121108 [ ]
>
>Hi Chetan and Vmware t...

Download full text (20.0 KiB)

Also test the CVM after applying the changes, before providing the patch.

On 8/27/18, 9:14 PM, "Ram Yadav" <email address hidden> wrote:

>Marcin/Adrian,
>Can you follow the instruction in the email from VMware and try out the python script.
>Also modify the CVM cord accordingly.
>Share the patch for the same by tomorrow sunnyvale time.
>
>Thanks,
>Ram
>
>
>
>On 8/27/18, 9:03 PM, "VMware Technical Support" <email address hidden> wrote:
>
>>** Please do not change the subject line of this e-mail if you wish to respond. **
>>
>>Hi Ram,
>>
>>Sure, we will connect with them to get the requested details. In the meantime kindly test the script attached with this email and let us know the test results. Please see the below list of parameters and example:
>>
>>Parameters always needed to run the script:
>>--host --user --pwd --dvs --obj
>>
>>1. Create a portgroup and enable vlan override on the dvportgroup. --obj set to portgroup, and --portgroup is specified.
>>
>># python3 port_reconfigure.py --host=10.192.229.18 --<email address hidden>' --pwd='Admin!23' --dvs=DSwitch --obj=portgroup --portgroup='cute111'
>>successfully added portgroup:cute111 on dvs:DSwitch
>>successfully enabled vlan override on portgroup:cute111
>>
>>2. Modify the vlan. --obj set to port, and --key and --vlan is needed.
>>
>>python3 port_reconfigure.py --host=10.192.229.18 --<email address hidden>' --pwd='Admin!23' --dvs=DSwitch --obj=port --key=21 --vlan=1024
>>successfully modified port 21 's vlan to:1024 on dvs:DSwitch
>>
>>3. For more details, you can use #python3 port_reconfigure.py --help
>>
>>Please let us know if you find any difficulties in executing the script. We will wait for your further response.
>>
>>Thank you.
>>
>>Case Reported: Issue regarding VLAN override with overlapping VLAN’s for VM’s belonging to same Port Group but on different ESXi
>>Status: Waiting on Customer
>>Next Touch Date: 30th August 2018
>>
>>Regards,
>>Chetan Patidar
>>VMware SDK Developer Support
>>Global Support Services, VMware Inc.
>>Global Support Phone Numbers: https://urldefense.proofpoint.com/v2/url?u=https-3A__www.vmware.com_support_contacts_us-5Fsupport&d=DwIFaQ&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=5cGBwnyUCiMK2Tm6UX_iIiml04qYJeDz4xQzoG2Hi8E&m=HJPbXgvwaHljGcPH2s36sLkauOImQiky90mMrpVweVI&s=leZ5XIpageOxaytxnLquhj6jDc16OjOJ3u3wGVFMVmA&e=
>>
>>Office Hours: Monday - Friday 06:00 AM – 3:00 PM IST
>>
>>If you cannot reach me, kindly contact my direct manager, Nalin Kumar Gupta, at <email address hidden>. Satisfied customers are our top priority.
>>
>>VMware Skyline: VMware’s new proactive support technology; sign up here - https://urldefense.proofpoint.com/v2/url?u=https-3A__app.smartsheet.com_b_form_f44fcd08d8a04234b6360e1b60051840&d=DwIFaQ&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=5cGBwnyUCiMK2Tm6UX_iIiml04qYJeDz4xQzoG2Hi8E&m=HJPbXgvwaHljGcPH2s36sLkauOImQiky90mMrpVweVI&s=MdUTZq2nMRHSi3Umh0i9Vx8ZAO6hsZmmzr1eGP83U4Y&e=
>>
>>--------------- Original Message ---------------
>>From: Ram Yadav [<email address hidden>]
>>Sent: 8/28/2018 9:09 AM
>>To: <email address hidden>; cscharmann@vm...

Download full text (21.9 KiB)

Ram, do join 7am meeting tomorrow?
Also did you try the patches one by one ?

Sent from my iPhone

> On Aug 27, 2018, at 9:31 PM, ram yadav <email address hidden> wrote:
>
> Marcin/Adrian,
> Can you follow the instruction in the email from VMware and try out the python script.
> Also modify the CVM cord accordingly.
> Share the patch for the same by tomorrow sunnyvale time.
>
> Thanks,
> Ram
>
>
> On 8/27/18, 9:03 PM, "VMware Technical Support" <email address hidden>
> wrote:
>
>> ** Please do not change the subject line of this e-mail if you wish to respond. **
>>
>> Hi Ram,
>>
>> Sure, we will connect with them to get the requested details. In the meantime kindly test the script attached with this email and let us know the test results. Please see the below list of parameters and example:
>>
>> Parameters always needed to run the script:
>> --host --user --pwd --dvs --obj
>>
>> 1. Create a portgroup and enable vlan override on the dvportgroup. --obj set to portgroup, and --portgroup is specified.
>>
>> # python3 port_reconfigure.py --host=10.192.229.18 --<email address hidden>' --pwd='Admin!23' --dvs=DSwitch --obj=portgroup --portgroup='cute111'
>> successfully added portgroup:cute111 on dvs:DSwitch
>> successfully enabled vlan override on portgroup:cute111
>>
>> 2. Modify the vlan. --obj set to port, and --key and --vlan is needed.
>>
>> python3 port_reconfigure.py --host=10.192.229.18 --<email address hidden>' --pwd='Admin!23' --dvs=DSwitch --obj=port --key=21 --vlan=1024
>> successfully modified port 21 's vlan to:1024 on dvs:DSwitch
>>
>> 3. For more details, you can use #python3 port_reconfigure.py --help
>>
>> Please let us know if you find any difficulties in executing the script. We will wait for your further response.
>>
>> Thank you.
>>
>> Case Reported: Issue regarding VLAN override with overlapping VLAN’s for VM’s belonging to same Port Group but on different ESXi
>> Status: Waiting on Customer
>> Next Touch Date: 30th August 2018
>>
>> Regards,
>> Chetan Patidar
>> VMware SDK Developer Support
>> Global Support Services, VMware Inc.
>> Global Support Phone Numbers: https://urldefense.proofpoint.com/v2/url?u=https-3A__www.vmware.com_support_contacts_us-5Fsupport&d=DwIFaQ&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=5cGBwnyUCiMK2Tm6UX_iIiml04qYJeDz4xQzoG2Hi8E&m=HJPbXgvwaHljGcPH2s36sLkauOImQiky90mMrpVweVI&s=leZ5XIpageOxaytxnLquhj6jDc16OjOJ3u3wGVFMVmA&e=
>>
>> Office Hours: Monday - Friday 06:00 AM – 3:00 PM IST
>>
>> If you cannot reach me, kindly contact my direct manager, Nalin Kumar Gupta, at <email address hidden>. Satisfied customers are our top priority.
>>
>> VMware Skyline: VMware’s new proactive support technology; sign up here - https://urldefense.proofpoint.com/v2/url?u=https-3A__app.smartsheet.com_b_form_f44fcd08d8a04234b6360e1b60051840&d=DwIFaQ&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=5cGBwnyUCiMK2Tm6UX_iIiml04qYJeDz4xQzoG2Hi8E&m=HJPbXgvwaHljGcPH2s36sLkauOImQiky90mMrpVweVI&s=MdUTZq2nMRHSi3Umh0i9Vx8ZAO6hsZmmzr1eGP83U4Y&e=
>>
>> --------------- Original Message ---------------
>> From: Ram Yadav [ryada...

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers