Juniper Openstack

centos71 - neutron server fails to start on R3.0 build42

Bug #1586023 reported by Sudheendra Rao
16
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Juniper Openstack
Status tracked in Trunk
R3.0
Fix Committed
High
Ignatious Johnson Christopher
Juniper Openstack r3.0.3.0
R3.0.2.x
Fix Committed
High
Nagendra Prasath
Juniper Openstack r3.0.2.1
R3.1
Fix Committed
High
Nagendra Prasath
Juniper Openstack r3.1.0.0-fcs
Trunk
Fix Committed
High
Nagendra Prasath
Juniper Openstack r3.2.0.0-fcs "r3.2.0.0"

Bug Description

neutron server fails to start on centos71 on R3.0 build42, most of the sanity TCs are failing due to this error.
restarting neutron server resolves the problem.

I couldn't get any logs for this!!!

[root@nodeb2 ~]# neutron net-list
<html><body><h1>503 Service Unavailable</h1>
No server is available to handle this request.
</body></html>

[root@nodeb2 ~]#

[root@nodeb2 neutron]# service neutron-server status
Redirecting to /bin/systemctl status neutron-server.service
● neutron-server.service - OpenStack Neutron Server
   Loaded: loaded (/usr/lib/systemd/system/neutron-server.service; enabled; vendor preset: disabled)
   Active: failed (Result: timeout) since Thu 2016-05-26 14:32:02 IST; 1h 1min ago
 Main PID: 1418 (code=killed, signal=TERM)

May 26 14:32:02 nodeb2.englab.juniper.net systemd[1]: neutron-server.service start operation timed out. Term...ing.
May 26 14:32:02 nodeb2.englab.juniper.net systemd[1]: Failed to start OpenStack Neutron Server.
May 26 14:32:02 nodeb2.englab.juniper.net systemd[1]: Unit neutron-server.service entered failed state.
May 26 14:32:02 nodeb2.englab.juniper.net systemd[1]: neutron-server.service failed.
Hint: Some lines were ellipsized, use -l to show in full.
[root@nodeb2 neutron]#

Jeba Paulaiyan (jebap)
tags: added: sanity
Revision history for this message
Sudheendra Rao (sudheendra-k) wrote :

The problem is seen on build43 also, testbed nodeb2.englab.juniper.net is in the problem state.

Revision history for this message
Sudheendra Rao (sudheendra-k) wrote :

Workaround is to restart neutron-server.

tags: added: releasenote
Nagabhushana R (bhushana)
Changed in juniperopenstack:
assignee: nobody → Nagendra Prasath (npchandran)
Ashish Ranjan (aranjan-n)
tags: removed: releasenote
Revision history for this message
Vedamurthy Joshi (vedujoshi) wrote :

it seems to be timing out due to something.

messages:Jun 7 00:57:07 nodeb5 systemd: neutron-server.service start operation timed out. Terminating.
messages:Jun 7 00:57:07 nodeb5 systemd: Unit neutron-server.service entered failed state.
messages:Jun 7 00:57:07 nodeb5 systemd: neutron-server.service failed.

Revision history for this message
Sudheendra Rao (sudheendra-k) wrote :

This problem is seen in R3.1 also.
Earlier it was observed that the neutron server was running at the end of provisioning and failed while running the sanity (couldn't root cause the problem).
But this time on R3.1 build2, neutron server is down after provision is completed.
Job didn't run sanity, it was stopped after provision completed.

Again there is not much info from the log, setup is in the failed state (nodei31)

[root@nodei31 ~]# service neutron-server status
Redirecting to /bin/systemctl status neutron-server.service
● neutron-server.service - OpenStack Neutron Server
   Loaded: loaded (/usr/lib/systemd/system/neutron-server.service; enabled; vendor preset: disabled)
   Active: failed (Result: timeout) since Mon 2016-07-18 13:05:32 IST; 9min ago
 Main PID: 1552 (code=killed, signal=TERM)

Jul 18 13:02:23 nodei31.englab.juniper.net systemd[1]: Starting OpenStack Neutron Server...
Jul 18 13:05:32 nodei31.englab.juniper.net systemd[1]: neutron-server.service start operation timed out. Terminating.
Jul 18 13:05:32 nodei31.englab.juniper.net systemd[1]: Failed to start OpenStack Neutron Server.
Jul 18 13:05:32 nodei31.englab.juniper.net systemd[1]: Unit neutron-server.service entered failed state.
Jul 18 13:05:32 nodei31.englab.juniper.net systemd[1]: neutron-server.service failed.
[root@nodei31 ~]#

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] master

Review in progress for https://review.opencontrail.org/22503
Submitter: Ignatious Johnson Christopher (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R3.1

Review in progress for https://review.opencontrail.org/22588
Submitter: Ignatious Johnson Christopher (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R3.0

Review in progress for https://review.opencontrail.org/22589
Submitter: Ignatious Johnson Christopher (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R3.1

Review in progress for https://review.opencontrail.org/22588
Submitter: Ignatious Johnson Christopher (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] master

Review in progress for https://review.opencontrail.org/22503
Submitter: Ignatious Johnson Christopher (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R3.0.2.x

Review in progress for https://review.opencontrail.org/22637
Submitter: Ignatious Johnson Christopher (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] master

Review in progress for https://review.opencontrail.org/22638
Submitter: Ignatious Johnson Christopher (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R3.1

Review in progress for https://review.opencontrail.org/22639
Submitter: Ignatious Johnson Christopher (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] master

Review in progress for https://review.opencontrail.org/22638
Submitter: Ignatious Johnson Christopher (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R3.0.2.x

Review in progress for https://review.opencontrail.org/22637
Submitter: Ignatious Johnson Christopher (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : A change has been merged

Reviewed: https://review.opencontrail.org/22639
Committed: http://github.org/Juniper/contrail-neutron-plugin/commit/63ff7ea0007abbb6c561c9137ccb2f673f01e9b0
Submitter: Zuul
Branch: R3.1

commit 63ff7ea0007abbb6c561c9137ccb2f673f01e9b0
Author: Ignatious Johnson Christopher <email address hidden>
Date: Fri Jul 29 13:23:05 2016 -0700

Decouple neutron-server and api-server services.

plugin specified in neutron.conf should not
try to connect to api-server, until it is
used/plugged-in when creating Loadbalancer
pool/vip/member/health monitors etc, thus
allowing neutron-server to start independently
without depending on api-server to be up and running.

Change-Id: I476b81b56dd5f42039a422e940349eed64eb6ad4
Closes-Bug: 1586023

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Reviewed: https://review.opencontrail.org/22637
Committed: http://github.org/Juniper/contrail-neutron-plugin/commit/bbde8790a437d0cc35d1e89753e59d68fb7a54ec
Submitter: Zuul
Branch: R3.0.2.x

commit bbde8790a437d0cc35d1e89753e59d68fb7a54ec
Author: Ignatious Johnson Christopher <email address hidden>
Date: Wed Jul 27 11:17:16 2016 -0700

Decouple neutron-server and api-server services.

plugin specified in neutron.conf should not
try to connect to api-server, until it is
used/plugged-in when creating Loadbalancer
pool/vip/member/health monitors etc, thus
allowing neutron-server to start independently
without depending on api-server to be up and running.

Change-Id: I6a3c145789d2851e9799ba698337508cd36a1207
Closes-Bug: 1586023

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Reviewed: https://review.opencontrail.org/22638
Committed: http://github.org/Juniper/contrail-neutron-plugin/commit/7b319bf17cbe1abf27ef671f428870253b83e5bd
Submitter: Zuul
Branch: master

commit 7b319bf17cbe1abf27ef671f428870253b83e5bd
Author: Ignatious Johnson Christopher <email address hidden>
Date: Fri Jul 29 13:19:40 2016 -0700

Decouple neutron-server and api-server services.

plugin specified in neutron.conf should not
try to connect to api-server, until it is
used/plugged-in when creating Loadbalancer
pool/vip/member/health monitors etc, thus
allowing neutron-server to start independently
without depending on api-server to be up and running.

Change-Id: I9a2a47b29ae2f1ba7e77f9f0574071760399ff02
Closes-Bug: 1586023

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R3.0

Review in progress for https://review.opencontrail.org/24311
Submitter: Ignatious Johnson Christopher (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : A change has been merged

Reviewed: https://review.opencontrail.org/24311
Committed: http://github.org/Juniper/contrail-neutron-plugin/commit/57834ead2e567be3ef95e71cd60904667ae1d403
Submitter: Zuul
Branch: R3.0

commit 57834ead2e567be3ef95e71cd60904667ae1d403
Author: Ignatious Johnson Christopher <email address hidden>
Date: Wed Jul 27 11:17:16 2016 -0700

Decouple neutron-server and api-server services.

plugin specified in neutron.conf should not
try to connect to api-server, until it is
used/plugged-in when creating Loadbalancer
pool/vip/member/health monitors etc, thus
allowing neutron-server to start independently
without depending on api-server to be up and running.

Closes-Bug: 1586023
(cherry picked from commit bbde8790a437d0cc35d1e89753e59d68fb7a54ec)

Change-Id: I377a7459811c96cb725449e08d5ace0565738c1d

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.