Activity log for bug #1557798

Date Who What changed Old value New value Message
2016-03-15 23:37:13 Vinoth Kannan Ganapathy bug added bug
2016-03-15 23:58:50 Jeba Paulaiyan nominated for series juniperopenstack/r2.22.x
2016-03-15 23:58:50 Jeba Paulaiyan bug task added juniperopenstack/r2.22.x
2016-03-15 23:58:50 Jeba Paulaiyan nominated for series juniperopenstack/trunk
2016-03-15 23:58:50 Jeba Paulaiyan bug task added juniperopenstack/trunk
2016-03-15 23:58:50 Jeba Paulaiyan nominated for series juniperopenstack/r3.0
2016-03-15 23:58:50 Jeba Paulaiyan bug task added juniperopenstack/r3.0
2016-03-15 23:58:56 Jeba Paulaiyan juniperopenstack/r2.22.x: importance Undecided High
2016-03-15 23:58:58 Jeba Paulaiyan juniperopenstack/r3.0: importance Undecided High
2016-03-15 23:59:00 Jeba Paulaiyan juniperopenstack/trunk: importance Undecided High
2016-03-15 23:59:07 Jeba Paulaiyan juniperopenstack/trunk: milestone r3.0.1.0 r3.1.0.0-fcs
2016-03-15 23:59:15 Jeba Paulaiyan juniperopenstack/r3.0: milestone r3.0.1.0
2016-03-15 23:59:23 Jeba Paulaiyan juniperopenstack/r2.22.x: milestone r2.22.2
2016-03-15 23:59:35 Jeba Paulaiyan juniperopenstack/r2.22.x: assignee aswani kumar (aswanikumar90)
2016-03-15 23:59:43 Jeba Paulaiyan juniperopenstack/r3.0: assignee aswani kumar (aswanikumar90)
2016-03-15 23:59:54 Jeba Paulaiyan juniperopenstack/trunk: assignee aswani kumar (aswanikumar90)
2016-03-16 00:08:48 Vinoth Kannan Ganapathy description After restoring the databases, zookeeper went down and other services failed to connect to zookeeper. later found that the ownership of the /var/lib/zookeeper set to ntp.. changing ownership brings back all the service up and running root@contrail-node7:/opt/contrail/utils# ls -altr /var/lib/ | grep zookeeper drwxr-xr-x 3 ntp ntp 4096 Feb 2 22:29 zookeeper root@contrail-node7:/opt/contrail/utils# ls -altr /var/lib/zookeeper total 12 lrwxrwxrwx 1 root root 24 Jun 10 2013 myid -> /etc/zookeeper/conf/myid drwxr-xr-x 3 ntp ntp 4096 Feb 2 22:29 . drwxr-xr-x 2 ntp ntp 4096 Mar 5 04:42 version-2 drwxr-xr-x 61 root root 4096 Mar 15 14:40 .. root@contrail-node7:/opt/contrail/utils# ls -altr /var/lib/ | grep cass drwxr-xr-x 5 cassandra cassandra 4096 Mar 15 11:45 cassandra root@contrail-node7:/opt/contrail/utils# ls -altr /var/lib/ | grep my drwxr-xr-x 10 mysql mysql 4096 Mar 15 11:57 mysql root@contrail-node7:/opt/contrail/utils# root@contrail-node7:/opt/contrail/utils# root@contrail-node7:/opt/contrail/utils# root@contrail-node7:/opt/contrail/utils# root@contrail-node7:/opt/contrail/utils# fab -R database -- "chown -R zookeeper:zookeeper /var/lib/zookeeper/" [root@10.102.44.41] Executing task '<remainder>' 2016-03-15 16:07:16:463172: [root@10.102.44.41] run: chown -R zookeeper:zookeeper /var/lib/zookeeper/ 2016-03-15 16:07:16:463317: [root@10.102.44.42] Executing task '<remainder>' 2016-03-15 16:07:16:707813: [root@10.102.44.42] run: chown -R zookeeper:zookeeper /var/lib/zookeeper/ 2016-03-15 16:07:16:707939: [root@10.102.44.43] Executing task '<remainder>' 2016-03-15 16:07:16:951495: [root@10.102.44.43] run: chown -R zookeeper:zookeeper /var/lib/zookeeper/ 2016-03-15 16:07:16:951745: 2016-03-15 16:07:17:206137: Done. 2016-03-15 16:07:17:206190: Disconnecting from 10.102.44.43... done. 2016-03-15 16:07:17:319913: Disconnecting from 10.102.44.41... done. 2016-03-15 16:07:17:351463: Disconnecting from 10.102.44.42... done. 2016-03-15 16:07:17:465336: root@contrail-node7:/opt/contrail/utils# root@contrail-node7:/opt/contrail/utils# root@contrail-node7:/opt/contrail/utils# root@contrail-node7:/opt/contrail/utils# ls -altr /var/lib/ | grep zookeeper drwxr-xr-x 3 zookeeper zookeeper 4096 Feb 2 22:29 zookeeper root@contrail-node7:/opt/contrail/utils# root@contrail-node7:/opt/contrail/utils# root@contrail-node7:/opt/contrail/utils# root@contrail-node7:/opt/contrail/utils# ps -ef | grep Zoo root 12815 30324 0 16:07 pts/3 00:00:00 grep --color=auto Zoo root@contrail-node7:/opt/contrail/utils# ps -ef | grep zoo zookeep+ 12888 1 0 16:07 ? 00:00:00 /usr/bin/java -cp /etc/zookeeper/conf:/usr/share/java/jline.jar:/usr/share/java/log4j-1.2.jar:/usr/share/java/xercesImpl.jar:/usr/share/java/xmlParserAPIs.jar:/usr/share/java/netty.jar:/usr/share/java/slf4j-api.jar:/usr/share/java/slf4j-log4j12.jar:/usr/share/java/zookeeper.jar -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.local.only=false -Dzookeeper.log.dir=/var/log/zookeeper -Dzookeeper.root.logger=INFO,CONSOLE,ROLLINGFILE org.apache.zookeeper.server.quorum.QuorumPeerMain /etc/zookeeper/conf/zoo.cfg root 12907 30324 0 16:07 pts/3 00:00:00 grep --color=auto zoo root@contrail-node7:/opt/contrail/utils# contrail-status == Contrail vRouter == supervisor-vrouter: active After restoring the databases, zookeeper went down and other services failed to connect to zookeeper. later found that the ownership of the /var/lib/zookeeper set to ntp.. changing ownership brings back all the service up and running also the nova instances directory.. root@contrail-node7:/opt/contrail/utils# ls -altr /var/lib/ | grep zookeeper drwxr-xr-x 3 ntp ntp 4096 Feb 2 22:29 zookeeper root@contrail-node7:/opt/contrail/utils# ls -altr /var/lib/zookeeper total 12 lrwxrwxrwx 1 root root 24 Jun 10 2013 myid -> /etc/zookeeper/conf/myid drwxr-xr-x 3 ntp ntp 4096 Feb 2 22:29 . drwxr-xr-x 2 ntp ntp 4096 Mar 5 04:42 version-2 drwxr-xr-x 61 root root 4096 Mar 15 14:40 .. root@contrail-node7:/opt/contrail/utils# ls -altr /var/lib/ | grep cass drwxr-xr-x 5 cassandra cassandra 4096 Mar 15 11:45 cassandra root@contrail-node7:/opt/contrail/utils# ls -altr /var/lib/ | grep my drwxr-xr-x 10 mysql mysql 4096 Mar 15 11:57 mysql root@contrail-node7:/opt/contrail/utils# root@contrail-node7:/opt/contrail/utils# root@contrail-node7:/opt/contrail/utils# root@contrail-node7:/opt/contrail/utils# root@contrail-node7:/opt/contrail/utils# fab -R database -- "chown -R zookeeper:zookeeper /var/lib/zookeeper/" [root@10.102.44.41] Executing task '<remainder>' 2016-03-15 16:07:16:463172: [root@10.102.44.41] run: chown -R zookeeper:zookeeper /var/lib/zookeeper/ 2016-03-15 16:07:16:463317: [root@10.102.44.42] Executing task '<remainder>' 2016-03-15 16:07:16:707813: [root@10.102.44.42] run: chown -R zookeeper:zookeeper /var/lib/zookeeper/ 2016-03-15 16:07:16:707939: [root@10.102.44.43] Executing task '<remainder>' 2016-03-15 16:07:16:951495: [root@10.102.44.43] run: chown -R zookeeper:zookeeper /var/lib/zookeeper/ 2016-03-15 16:07:16:951745: 2016-03-15 16:07:17:206137: Done. 2016-03-15 16:07:17:206190: Disconnecting from 10.102.44.43... done. 2016-03-15 16:07:17:319913: Disconnecting from 10.102.44.41... done. 2016-03-15 16:07:17:351463: Disconnecting from 10.102.44.42... done. 2016-03-15 16:07:17:465336: root@contrail-node7:/opt/contrail/utils# root@contrail-node7:/opt/contrail/utils# root@contrail-node7:/opt/contrail/utils# root@contrail-node7:/opt/contrail/utils# ls -altr /var/lib/ | grep zookeeper drwxr-xr-x 3 zookeeper zookeeper 4096 Feb 2 22:29 zookeeper root@contrail-node7:/opt/contrail/utils# root@contrail-node7:/opt/contrail/utils# root@contrail-node7:/opt/contrail/utils# root@contrail-node7:/opt/contrail/utils# ps -ef | grep Zoo root 12815 30324 0 16:07 pts/3 00:00:00 grep --color=auto Zoo root@contrail-node7:/opt/contrail/utils# ps -ef | grep zoo zookeep+ 12888 1 0 16:07 ? 00:00:00 /usr/bin/java -cp /etc/zookeeper/conf:/usr/share/java/jline.jar:/usr/share/java/log4j-1.2.jar:/usr/share/java/xercesImpl.jar:/usr/share/java/xmlParserAPIs.jar:/usr/share/java/netty.jar:/usr/share/java/slf4j-api.jar:/usr/share/java/slf4j-log4j12.jar:/usr/share/java/zookeeper.jar -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.local.only=false -Dzookeeper.log.dir=/var/log/zookeeper -Dzookeeper.root.logger=INFO,CONSOLE,ROLLINGFILE org.apache.zookeeper.server.quorum.QuorumPeerMain /etc/zookeeper/conf/zoo.cfg root 12907 30324 0 16:07 pts/3 00:00:00 grep --color=auto zoo root@contrail-node7:/opt/contrail/utils# contrail-status == Contrail vRouter == supervisor-vrouter: active
2016-03-16 11:01:54 aswani kumar juniperopenstack/r2.22.x: status New In Progress
2016-03-16 11:01:58 aswani kumar juniperopenstack/r3.0: status New In Progress
2016-03-16 11:02:11 aswani kumar juniperopenstack/trunk: status New In Progress
2016-03-19 07:45:40 OpenContrail Admin nominated for series juniperopenstack/r2.20
2016-03-19 07:45:40 OpenContrail Admin bug task added juniperopenstack/r2.20
2016-03-19 07:45:40 OpenContrail Admin bug task added juniperopenstack/r2.20
2016-03-20 03:49:19 OpenContrail Admin juniperopenstack/r3.0: status In Progress Fix Committed
2016-03-20 03:52:33 OpenContrail Admin juniperopenstack/r2.20: milestone r2.23
2016-03-20 03:52:34 OpenContrail Admin juniperopenstack/r2.20: status In Progress Fix Committed
2016-03-20 03:53:01 OpenContrail Admin juniperopenstack/trunk: status In Progress Fix Committed
2016-03-23 06:08:18 OpenContrail Admin juniperopenstack/r2.22.x: milestone r2.22.2 r2.22.1
2016-03-23 06:08:20 OpenContrail Admin juniperopenstack/r2.22.x: status In Progress Fix Committed
2016-03-31 12:46:27 Vinay Mahuli juniperopenstack/r2.22.x: milestone r2.22.1 r2.22.2
2016-05-04 17:34:59 Nagabhushana R information type Proprietary Public