StarlingX R2 duplex: VM not getting IP assigned to the vlan network when re-spawned on controller-0 after shutting down controller-1

Bug #1849655 reported by Akshay on 2019-10-24
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
StarlingX
High
marvin Yu

Bug Description

Brief Description
-----------------
Setup: I have deployed Bare Metal StarlingX R2 duplex mode. While testing HA, I tested a case in which I have spawned 2 VMs from horizon together on IPv6 flat and IPv6 vlan networks. One gets spawned on controller-0 and other on controller-1. Both VM gets IP assigned on each network and they are able to ping each other.

Test Case: Now I switched off the controller-1 and the VM on controller-1 gets rebuilt on controller-0 showing same IPs as of earlier on horizon.

Issue: But when I tried to check inside the VM, it gets IP on flat network but it did NOT get any IP assigned on the vlan network.

I tried this case many times with same result.
Please guide me to solve this issue.

Severity
--------

Critical

Steps to Reproduce
------------------
1. Deploy Bare Metal StarlingX R2 duplex mode.
2. Spawn 2 VMs together on IPv6 flat and IPv6 vlan from horizon.
3. Switch off the controller-1.
4. Check the rebuilt VM whether it got IPs assigned or not.

Expected Behavior
------------------
Rebuild VM should get the IP assigned inside the VM similar to as visible on horizon.

Actual Behavior
----------------
Rebuilt VM not getting IP assigned on vlan network.

Reproducibility
---------------
Reproducible

System Configuration
--------------------
Two node system

Last Pass
---------
NO

Ghada Khalil (gkhalil) wrote :

Assigning to the networking team to triage/investigate. This issue is not reported for any stx.2.0 testing done by the verification teams. It may be a procedural issue.

tags: added: stx.2.0 stx.networking
Changed in starlingx:
importance: Undecided → High
status: New → Triaged
assignee: nobody → Forrest Zhao (forrest.zhao)
Changed in starlingx:
assignee: Forrest Zhao (forrest.zhao) → marvin Yu (marvin-yu)
marvin Yu (marvin-yu) wrote :

Hi Akshay,

Could you please provide the log when VM gets rebulid on controller-0? This bug is similar to the previous one. I suspect that the vlan dhcp-agent hasn`t been reconstructed on controller-0.

marvin Yu (marvin-yu) on 2019-11-01
Changed in starlingx:
status: Triaged → In Progress
Akshay (yadavakshay58) wrote :

Hi Marvin,

Which logs ?
1. Are you asking about the "nova-conductor.log" or nova components related logs ?
   -If yes, I can't find any logs in any nova containers. I can't even enable it in configuration files as it
    states "Changing a read-only file" and "Can't open file for writing".
2. or you are asking for >collect all ?
   -If yes, then for which controller node?

marvin Yu (marvin-yu) wrote :

Hi Akshay,

It`s ok, I just want to see instance console log. I have tried to reproduce this bug, but I've run into some problems now. could you please provide more detailed steps? e.g. steps to create ipv6 network, steps to create vm.

I also have several questions as below:
1. When the VM is created, does the VM get IPv6 address inside ?
2. Which controller node does the flat VM run on?

Akshay (yadavakshay58) wrote :

Sure Marvin,

Answers to your queries:
1. Yes, it gets IPv6 address assigned inside the VM initially before HA.
2. Flat IPv6 and vlan IPv6 both networks are attached on VMs, Initially both VMs have IPs assigned on both the networks ON EACH CONTROLLER NODE.

Steps tp create IPv6 networks:
1. I have created both the networks from horizon with abcd:abcd:abcd::0/64 as subnet and IPv6 as network type.
2. For flat_IPv6, physical network: physnet0
3. For vlan_IPv6, physical network: physnet1

Steps to create VM:
1. Image:centos
2. Networks: flat_IPv6, vlan_IPv6
3. No Security group added.
4. Cloud Config used only to change login credentials as username:centos and passwd:XXXXXX.

Akshay (yadavakshay58) wrote :

HI Marvin,

One new finding for your information !

When I tried the above same test case WITH VOLUMES ATTACHED to VMs, then I didn't face this VLAN IP issue. Respawned VM gets IPs assigned as expected.

marvin Yu (marvin-yu) wrote :

Hi Akshay,
Thanks for the info. This seems to be an occasional bug, I'll keep an eye on it. If you have any new discoveries, please let me know. thanks a lot~

Akshay (yadavakshay58) wrote :

Hi Marvin,

I have tested both the cases (with volume and without volume) many times and it is reproducible every time for me. So I don't think it is an occasional bug.

marvin Yu (marvin-yu) wrote :

Hi Akshay,

which ipv6-address-mode did you specify?

marvin Yu (marvin-yu) wrote :

and could you please provide the openstack image file?
Thanks a lot.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers