Juniper Openstack

R3.2 setup_all of centos74 cluster failed unable to start supervisor-database

  1. Series r3.2
  2. Bug #1773412
Bug #1773412 reported by wenqing liang
12
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Juniper Openstack
Invalid
High
Unassigned
R3.2
Fix Committed
High
Santosh Gupta
Juniper Openstack r3.2.11.0
R4.0
Fix Committed
High
Santosh Gupta
R4.1
Fix Committed
High
Santosh Gupta
Juniper Openstack r4.1.2.0

Bug Description

For R3.2 of centos74 cluster, setup_all fails unable to start supervisor-database:

[root@server3 utils]# service supervisor-database status
● supervisor-database.service - SYSV: Supervisord instance for Contrail Database Package
   Loaded: loaded (/etc/rc.d/init.d/supervisor-database; bad; vendor preset: disabled)
   Active: active (exited) since Thu 2018-05-17 20:09:20 UTC; 1h 30min ago
     Docs: man:systemd-sysv-generator(8)
  Process: 12135 ExecStart=/etc/rc.d/init.d/supervisor-database start (code=exited, status=0/SUCCESS)

May 17 20:08:59 server3 systemd[1]: Starting SYSV: Supervisord instance for Contrail Database Package...
May 17 20:08:59 server3 supervisor-database[12135]: Starting Supervisor Daemon for Contrail Database PackageError: Invalid user name kafka in section 'program:kafka' (file: '/etc...abase.conf')
May 17 20:08:59 server3 supervisor-database[12135]: For help, use /usr/bin/supervisord -h
May 17 20:08:59 server3 supervisor-database[12135]: [58B blob data]
May 17 20:09:20 server3 systemd[1]: Started SYSV: Supervisord instance for Contrail Database Package.
Hint: Some lines were ellipsized, use -l to show in full.
[root@server3 utils]# echo $?
3
[root@server3 utils]#

Logs uploaded to /cs-shared/bugs/.

Arvind V. debugged the issue and found in 3.2, database packages for rpm don’t create user kafka as part of the post install.

For recovery, we can issue the following:
1) “adduser kafka”
2) chown -R kafka /tmp/kafka-logs
3) chown -R kafka /var/log/kafka
4) restart supervisor

See original description
wenqing liang (wliang)
description: updated
Revision history for this message
Arvind (arvindv) wrote :

Had to add Kafka user manually in the testbed because the script that adds the user failed mid way, while provisioning.

Here is the relevant snippet,
2018-05-17 19:41:16:883163: [root@10.0.0.5] out: Installing : contrail-openstack-database-3.2.10.0-75.el7.centos.noa 110/110
2018-05-17 19:41:16:883221: [root@10.0.0.5] out: chown: cannot access ‘/usr/share/contrail-utils/contrail-cassandra-status.py’: No such file or directory
2018-05-17 19:41:16:915448: [root@10.0.0.5] out: warning: %post(contrail-openstack-database-3.2.10.0-75.el7.centos.noarch) scriptlet failed, exit status 1
2018-05-17 19:41:16:915541: [root@10.0.0.5] out: Non-fatal POSTIN scriptlet failure in rpm package contrail-openstack-database-3.2.10.0-75.el7.centos.noarch

[root@server3 ~]# ls -lh /usr/share/contrail-utils/contrail-cassandra-status.py
ls: cannot access /usr/share/contrail-utils/contrail-cassandra-status.py: No such file or directory

however we tested on build 147 of 3.2.10.0 and we found the file, probably Wenquing tested against an older build.

root@sangupta-u14:~# ls -lrt /usr/bin/contrail-cassandra-*
lrwxrwxrwx 1 contrail contrail 52 May 14 01:40 /usr/bin/contrail-cassandra-status -> ../share/contrail-utils/contrail-cassandra-status.py
lrwxrwxrwx 1 contrail contrail 52 May 14 01:40 /usr/bin/contrail-cassandra-repair -> ../share/contrail-utils/contrail-cassandra-repair.py

so closing this.

Changed in juniperopenstack:
status: New → Invalid
Revision history for this message
Santosh Gupta (sangupta) wrote :

Ubuntu has contrail-cassandra-{status|repair}.py in /usr/share/contrail-utils directory.
root@sangupta-u14:/usr/share/contrail-utils# pwd
/usr/share/contrail-utils
root@sangupta-u14:/usr/share/contrail-utils# ls contrail-cassandra-*py
contrail-cassandra-repair.py contrail-cassandra-status.py
root@sangupta-u14:/usr/share/contrail-utils#

Redhat/centos have it in /usr/share/contrail directory.
[root@server3-de9594da-30ca-49d9-9ae6-eb3706009203 contrail]# pwd
/usr/share/contrail
[root@server3-de9594da-30ca-49d9-9ae6-eb3706009203 contrail]# ls contrail-cassandra-*py
contrail-cassandra-repair.py contrail-cassandra-status.py

This mismatch is causing the script to fail. Need to be fixed as listed below.
https://review.opencontrail.org/#/c/42857/3/common/rpm/contrail-openstack-database.spec line# 60
https://review.opencontrail.org/#/c/42856/3/rpm/contrail/contrail.spec line#551, 553

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R3.2

Review in progress for https://review.opencontrail.org/43586
Submitter: Santosh Gupta (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/43587
Submitter: Santosh Gupta (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R4.0

Review in progress for https://review.opencontrail.org/43588
Submitter: Santosh Gupta (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/43589
Submitter: Santosh Gupta (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R4.1

Review in progress for https://review.opencontrail.org/43590
Submitter: Santosh Gupta (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/43591
Submitter: Santosh Gupta (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : A change has been merged

Reviewed: https://review.opencontrail.org/43591
Committed: http://github.com/Juniper/contrail-packaging/commit/70ebf60751833a33cbb1d8131d2603eec6316470
Submitter: Zuul (<email address hidden>)
Branch: R4.1

commit 70ebf60751833a33cbb1d8131d2603eec6316470
Author: Santosh Gupta <email address hidden>
Date: Wed Jun 6 13:39:52 2018 -0700

Use correct directory for contrail utils on centos (2/2)

Centos uses /usr/share/contrail/ for contrail utils.
Changed script to use correct dir.

Change-Id: Ib46bfd19165e2e57a543f11f4e89279da71484d3
Partial-Bug: #1773412

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Reviewed: https://review.opencontrail.org/43590
Committed: http://github.com/Juniper/contrail-packages/commit/1c34ad09ec94494153f87464c981ceca1615abd5
Submitter: Zuul (<email address hidden>)
Branch: R4.1

commit 1c34ad09ec94494153f87464c981ceca1615abd5
Author: Santosh Gupta <email address hidden>
Date: Wed Jun 6 13:39:27 2018 -0700

Use correct directory for contrail utils on centos (1/2)

Centos uses /usr/share/contrail/ for contrail utils.
Changed script to use correct dir.

Change-Id: I3da2bbb4abf8db65990522747651f14c2e8817b4
Partial-Bug: #1773412

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Reviewed: https://review.opencontrail.org/43586
Committed: http://github.com/Juniper/contrail-packages/commit/96f3ecf2af69eaf1069544d63e4b0528d80ab551
Submitter: Zuul (<email address hidden>)
Branch: R3.2

commit 96f3ecf2af69eaf1069544d63e4b0528d80ab551
Author: Santosh Gupta <email address hidden>
Date: Wed Jun 6 13:21:02 2018 -0700

Use correct directory for contrail utils on centos (1/2)

Centos uses /usr/share/contrail/ for contrail utils.
Changed script to use correct dir.

Change-Id: I004e50a21863eda9b0cef139e511b3aa3e9d1e41
Partial-Bug: #1773412

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Reviewed: https://review.opencontrail.org/43587
Committed: http://github.com/Juniper/contrail-packaging/commit/1d38d96b97286c61ebee9f51b12ee7a663eda2df
Submitter: Zuul (<email address hidden>)
Branch: R3.2

commit 1d38d96b97286c61ebee9f51b12ee7a663eda2df
Author: Santosh Gupta <email address hidden>
Date: Wed Jun 6 13:21:37 2018 -0700

Use correct directory for contrail utils on centos (2/2)

Centos uses /usr/share/contrail/ for contrail utils.
Changed script to use correct dir.

Change-Id: I1a91383597710d4217861de249d327de79090e4d
Partial-Bug: #1773412

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R3.2

Review in progress for https://review.opencontrail.org/43705
Submitter: Santosh Gupta (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/43706
Submitter: Santosh Gupta (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : A change has been merged

Reviewed: https://review.opencontrail.org/43588
Committed: http://github.com/Juniper/contrail-packages/commit/23d946cffd9c73147530670aa7345bdafa7ce190
Submitter: Zuul (<email address hidden>)
Branch: R4.0

commit 23d946cffd9c73147530670aa7345bdafa7ce190
Author: Santosh Gupta <email address hidden>
Date: Wed Jun 6 13:38:27 2018 -0700

Use correct directory for contrail utils on centos (1/2)

Centos uses /usr/share/contrail/ for contrail utils.
Changed script to use correct dir.

Change-Id: Ia939ff774ea615a342107f3bda8eb87030a3b555
Partial-Bug: #1773412

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Reviewed: https://review.opencontrail.org/43589
Committed: http://github.com/Juniper/contrail-packaging/commit/21d68158e7143b26fd2eb51f39848bf92d8a8db6
Submitter: Zuul (<email address hidden>)
Branch: R4.0

commit 21d68158e7143b26fd2eb51f39848bf92d8a8db6
Author: Santosh Gupta <email address hidden>
Date: Wed Jun 6 13:38:56 2018 -0700

Use correct directory for contrail utils on centos (2/2)

Centos uses /usr/share/contrail/ for contrail utils.
Changed script to use correct dir.

Change-Id: I81bc5e618f4fabe0b457d15970deea80b1731250
Partial-Bug: #1773412

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R4.0

Review in progress for https://review.opencontrail.org/43708
Submitter: Santosh Gupta (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/43709
Submitter: Santosh Gupta (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R4.1

Review in progress for https://review.opencontrail.org/43710
Submitter: Santosh Gupta (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/43711
Submitter: Santosh Gupta (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : A change has been merged

Reviewed: https://review.opencontrail.org/43705
Committed: http://github.com/Juniper/contrail-packages/commit/47c0ef19f452c99365bcab69e62adef74c2808be
Submitter: Zuul (<email address hidden>)
Branch: R3.2

commit 47c0ef19f452c99365bcab69e62adef74c2808be
Author: Santosh Gupta <email address hidden>
Date: Mon Jun 11 14:37:15 2018 -0700

Remove chown for non-existent script on centos (1/2)

Centos doesnt create symlink to contrail-cassandra-{status|repair}.py files.
Instead files are copied in /usr/bin/. So chown statements not needed for .py files.

Change-Id: Ia45d7693e8a29dab3e7ea3655ccf936076e2c34e
Partial-Bug: #1773412

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Reviewed: https://review.opencontrail.org/43706
Committed: http://github.com/Juniper/contrail-packaging/commit/5a5c1bf0235360f9d8aa41b2c492ed910e20e61b
Submitter: Zuul (<email address hidden>)
Branch: R3.2

commit 5a5c1bf0235360f9d8aa41b2c492ed910e20e61b
Author: Santosh Gupta <email address hidden>
Date: Mon Jun 11 14:38:17 2018 -0700

Remove chown for non-existent script on centos (2/2)

Centos doesnt create symlink to contrail-cassandra-{status|repair}.py files.
Instead files are copied in /usr/bin/. So chown statements not needed for .py files.

Change-Id: I88e337270fbb25c2f82084c4c8b2b0562edf1d71
Partial-Bug: #1773412

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Reviewed: https://review.opencontrail.org/43711
Committed: http://github.com/Juniper/contrail-packaging/commit/af5e97565e763b93191e63e45135d30bd424ecf7
Submitter: Zuul (<email address hidden>)
Branch: R4.1

commit af5e97565e763b93191e63e45135d30bd424ecf7
Author: Santosh Gupta <email address hidden>
Date: Mon Jun 11 15:53:51 2018 -0700

Remove chown for non-existent script on centos (2/2)

Centos doesnt create symlink to contrail-cassandra-{status|repair}.py files.
Instead files are copied in /usr/bin/. So chown statements not needed for .py files.

Change-Id: I56ee8d9dd8978b9f93f8779da26c64fa3632eb07
Partial-Bug: #1773412

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Reviewed: https://review.opencontrail.org/43710
Committed: http://github.com/Juniper/contrail-packages/commit/90e203841e3b47fc85d76244164290ff71a86d17
Submitter: Zuul (<email address hidden>)
Branch: R4.1

commit 90e203841e3b47fc85d76244164290ff71a86d17
Author: Santosh Gupta <email address hidden>
Date: Mon Jun 11 15:53:22 2018 -0700

Remove chown for non-existent script on centos (1/2)

Centos doesnt create symlink to contrail-cassandra-{status|repair}.py files.
Instead files are copied in /usr/bin/. So chown statements not needed for .py files.

Change-Id: I77f5f6e44e9f73dfb625570ca851bd3ee327fd28
Partial-Bug: #1773412

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Reviewed: https://review.opencontrail.org/43708
Committed: http://github.com/Juniper/contrail-packages/commit/cca0b7a133c202c31182442906fbfc5aa87cf646
Submitter: Zuul (<email address hidden>)
Branch: R4.0

commit cca0b7a133c202c31182442906fbfc5aa87cf646
Author: Santosh Gupta <email address hidden>
Date: Mon Jun 11 15:53:22 2018 -0700

Remove chown for non-existent script on centos (1/2)

Centos doesnt create symlink to contrail-cassandra-{status|repair}.py files.
Instead files are copied in /usr/bin/. So chown statements not needed for .py files.

Change-Id: I77f5f6e44e9f73dfb625570ca851bd3ee327fd28
Partial-Bug: #1773412

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Reviewed: https://review.opencontrail.org/43709
Committed: http://github.com/Juniper/contrail-packaging/commit/6f0286aefb427c56447e8b52e2acfc3bf42d655f
Submitter: Zuul (<email address hidden>)
Branch: R4.0

commit 6f0286aefb427c56447e8b52e2acfc3bf42d655f
Author: Santosh Gupta <email address hidden>
Date: Mon Jun 11 15:53:51 2018 -0700

Remove chown for non-existent script on centos (2/2)

Centos doesnt create symlink to contrail-cassandra-{status|repair}.py files.
Instead files are copied in /usr/bin/. So chown statements not needed for .py files.

Change-Id: I56ee8d9dd8978b9f93f8779da26c64fa3632eb07
Partial-Bug: #1773412

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.