After stopping vswitch service no alarm is generated

Bug #1838432 reported by Raviteja naidu Jagalmarri
26
This bug affects 2 people
Affects Status Importance Assigned to Milestone
StarlingX
Fix Released
Medium
marvin Yu

Bug Description

Brief Description
-----------------
When stopping vswitch physical port state, It should generate alarm No alarm is generated  >>systemctl stop ovs-vswitchd.service
Please suggest, how to trigger alarm when stopping systemctl stop ovs-vswitchd.service.

Severity
--------
<Minor: System/Feature is not usable>

Steps to Reproduce
------------------
1. On Controller-0, Pre check the alarms
     >> fm alarm-list
2. Stop the vswitch service
     >> systemctl stop ovs-vswitchd.service
    Expected: Relevant alarm has not generated(Like vswitch service is stopped or related alarm should generate)
3. Check the status of vswitch service
    >> systemctl status ovs-vswitchd.service
    >> vswitch service should be inactive
4. Restart the vswitch service
    >> systemctl restart ovs-vswitchd.service
    >> vswitch servie should be active

Expected Behavior
------------------
1. Alarm should generate whenever vswitch service is stopped

Reproducibility
---------------
Reproducible

System Configuration
--------------------
- Duplex
- Bare metal

Branch/Pull Time/Commit
-----------------------
BUILD_DATE="2019-07-24 01:30:00 +0000"

Last Pass
--------------
Not passed

Ghada Khalil (gkhalil)
tags: added: stx.networking
Revision history for this message
Ghada Khalil (gkhalil) wrote :

I believe this should be covered to some degree by:
https://storyboard.openstack.org/#!/story/2002947

I'm not sure about the details of triggering the alarms, so assigning to Chenjie for his input since he implemented the above story.

Revision history for this message
Ghada Khalil (gkhalil) wrote :

In my opinion, this is a medium priority item and can potentially be deferred to stx.3.0.

Changed in starlingx:
importance: Undecided → Medium
assignee: nobody → ChenjieXu (midone)
Frank Miller (sensfan22)
tags: added: stx.3.0
Revision history for this message
ChenjieXu (midone) wrote :

This bug has been reproduced on StarlingX AIO Simplex baremetal with 0709 iso image. Will continue investigating.

Changed in starlingx:
status: New → Triaged
Revision history for this message
Raviteja naidu Jagalmarri (raviteja0218) wrote :

observing the issue with (BUILD_DATE="2019-08-09 01:30:00 +0000") also.

Revision history for this message
marvin Yu (marvin-yu) wrote :

I have reproduced this bug on AIO-SX and AIO-DX with 0902 iso.
After investigating the cause, the bug`s detail as below.

now, the ovs-vswitchd.conf(at /etc/pmon.d/ovs-vswitchd.conf) subfunction is set to worker. as the process Monitor`s code logic show.

>>>>>>>>>>>>>>>>>>metal/mtce/src/pmon/pmonFsm.cpp line.858>>>>>>>>>>>>>>
...
                    else if ( !strcmp (ptr->subfunction, "worker" ) )
                    {
                        if ( daemon_is_file_present ( DISABLE_WORKER_SERVICES ) == true )
                        {
                            /* Compute services are disabled - do not start monitoring */
                            start_monitoring = false;
                            waiting_for = DISABLE_WORKER_SERVICES;
                        }
                    }
...

If the file DISABLE_WORKER_SERVICES(at /var/run/.disable_worker_services) exists. these process that the subfunction type is worker will be ignored until the file DISABLE_WORKER_SERVICES is removed.

the file DISABLE_WORKER_SERVICES is created at config/puppet-manifests/src/modules/platform/manifests/kubernetes.pp line550, branch r/stx.2.0.

>>>config/puppet-manifests/src/modules/platform/manifests/kubernetes.pp line.550>>>>>
...
  file { '/var/run/.disable_worker_services':
    ensure => file,
    replace => no,
  }
...
After stx.2.0, the puppet-manifests has been removed from project config. the kubernetes.pp at project stx-puppet with the same path now.

Are there any conditions that the file will be removed at /var/run/.disable_worker_services? if not, the file will always be there and the process monitor is useless if the subfunction type is worker on AIO environment.

Revision history for this message
marvin Yu (marvin-yu) wrote :

Hi Ghada,

This bug not with relation to networking, It caused by Legacy of previous design. as Bart said:
"...This situation no longer exists because the bare metal nova-compute services do not exist after our transition to containers...I think the proper solution here is to remove all references to the disable_worker_services file from the codebase - this flag is no longer needed. This would also include removing the worker_services init script, which is also unnecessary". detail as below.
http://lists.starlingx.io/pipermail/starlingx-discuss/2019-September/006134.html

I think we need a better candidate to solve this, maybe Bart can give some advice.

marvin Yu (marvin-yu)
Changed in starlingx:
assignee: ChenjieXu (midone) → marvin Yu (marvin-yu)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to stx-puppet (master)

Fix proposed to branch: master
Review: https://review.opendev.org/684175

Changed in starlingx:
status: Triaged → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to ha (master)

Fix proposed to branch: master
Review: https://review.opendev.org/684179

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to config (master)

Fix proposed to branch: master
Review: https://review.opendev.org/684205

Ghada Khalil (gkhalil)
tags: added: stx.config
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to ha (master)

Reviewed: https://review.opendev.org/684179
Committed: https://git.openstack.org/cgit/starlingx/ha/commit/?id=0d5e7e54091ecc13a109d0071f696379438c39fc
Submitter: Zuul
Branch: master

commit 0d5e7e54091ecc13a109d0071f696379438c39fc
Author: marvin <email address hidden>
Date: Tue Sep 3 05:02:34 2019 +0000

    Removing unused flag disable_worker_services

    The disable_worker_services file was originally created
    to prevent the (bare metal) nova-compute services from
    running on a newly upgraded controller in an AIO-DX
    configuration. This situation no longer exists because
    the bare metal nova-compute services do not exist after
    transiting to containers. this flag is no longer needed.
    Removing all references to the disable_worker_services file.

    Change-Id: Ic9555a36890f613f440e97f9090b22ff5ec8fd82
    Partial-Bug: #1838432
    Signed-off-by: marvin <email address hidden>

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to config (master)

Reviewed: https://review.opendev.org/684205
Committed: https://git.openstack.org/cgit/starlingx/config/commit/?id=2407f451c953d209bbbc4843b9aaf3027324af0a
Submitter: Zuul
Branch: master

commit 2407f451c953d209bbbc4843b9aaf3027324af0a
Author: marvin <email address hidden>
Date: Tue Sep 3 09:09:34 2019 +0000

    Removing unused flag disable_worker_services

    The disable_worker_services file was originally created
    to prevent the (bare metal) nova-compute services from
    running on a newly upgraded controller in an AIO-DX
    configuration. This situation no longer exists because
    the bare metal nova-compute services do not exist after
    transiting to containers. this flag is no longer needed.
    Removing all references to the disable_worker_services file.

    Change-Id: I551122d0383eb7f7d6e53defa4010e1d62c1c899
    Partial-Bug: #1838432
    Signed-off-by: marvin <email address hidden>

Changed in starlingx:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to stx-puppet (master)

Reviewed: https://review.opendev.org/684175
Committed: https://git.openstack.org/cgit/starlingx/stx-puppet/commit/?id=a2c247cd2df64b32168daa91d65162435d90c5db
Submitter: Zuul
Branch: master

commit a2c247cd2df64b32168daa91d65162435d90c5db
Author: marvin <email address hidden>
Date: Tue Sep 3 04:52:24 2019 +0000

    Removing unused flag disable_worker_services

    The disable_worker_services file was originally created
    to prevent the (bare metal) nova-compute services from
    running on a newly upgraded controller in an AIO-DX
    configuration. This situation no longer exists because
    the bare metal nova-compute services do not exist after
    transiting to containers. this flag is no longer needed.
    Removing all references to the disable_worker_services file.

    Change-Id: I4f11674956f9aae1e4e805db2eb0696c3023e71e
    Closes-Bug: #1838432
    Signed-off-by: marvin <email address hidden>

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to metal (master)

Reviewed: https://review.opendev.org/684174
Committed: https://git.openstack.org/cgit/starlingx/metal/commit/?id=5f743f1402cdda253978fbae264bc457f87d1847
Submitter: Zuul
Branch: master

commit 5f743f1402cdda253978fbae264bc457f87d1847
Author: marvin <email address hidden>
Date: Tue Sep 3 04:23:06 2019 +0000

    Removing unused flag disable_worker_services

    The disable_worker_services file was originally created
    to prevent the (bare metal) nova-compute services from
    running on a newly upgraded controller in an AIO-DX
    configuration. This situation no longer exists because
    the bare metal nova-compute services do not exist after
    transiting to containers. this flag is no longer needed.
    Removing all references to the disable_worker_services file.

    Change-Id: I20e08db737bb0df6ba34c071e2435f1a18f7c3ed
    Partial-Bug: #1838432
    Signed-off-by: marvin <email address hidden>

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers