DCSMaster web showing incorrect MXOSRVR process info

[Jieping Zhang]

In the attached screen shot, column "Value" of row "Metrics" showed the totalRegistered MXOSRVR is 114, totalAvailable MXOSRVR is 189. But actually total MXOSRVR process on the Trafodion server is 120, among the 120 MXOSRVR processes, 1 is in connected state, so the totalAvaiable MXOSRVR should be 119.

Also, on the website, it shows 2 MXOSRVR are in CONNECTED state. Actually these 2 MXOSRVR processes are the same process because they have the same MXOSRVR process id: 19017. But 1 showed with the wrong "Start Time" -- "Tue May 05 23:30:45 UTC 2015". I checked on the cluster, process 19107 was started on June 07, not May 05. So the 2nd one is correct.

Below is the MXOSRVR process info on the Trafodion server side:

[trafodion@n015 scripts]$ sqps | grep mxosrvr|wc
    120 960 9720

[trafodion@n015 scripts]$ pdsh $MY_NODES ps -ef|grep 19017
n015: 501 19017 18727 2 Jun07 ? 04:33:34 mxosrvr -ZKHOST n014.cm.cluster:2181,n015.cm.cluster:2181,n013.cm.cluster:2181 -RZ g4q0091.houston.hp.com:3:1 -ZKPNODE /trafodion -CNGTO 60 -ZKSTO 180 -EADSCO 0 -TCPADD 16.235.158.115 -MAXHEAPPCT 0 -STATISTICSINTERVAL 60 -STATISTICSLIMIT 60 -STATISTICSTYPE aggregated -STATISTICSENABLE true -SQLPLAN true -PORTMAPTOSECS -1 -PORTBINDTOSECS -1
n015: 501 32642 21783 1 22:28 pts/7 00:00:00 grep 19017
[trafodion@n015 scripts]$

[Matt Brown (mattbrown-2)]

Hi, DCS doesn't use Trafodion for it's metrics. It looks directly in Zookeeper to find out what's registered, connected, available, etc. Need the zookeeper znode listings from /<user>/dcs/running & registered to determine what is root problem on this. This can be done with zkutil commands e.g., get on the znode will retrieve the data from a particular znode.

[Matt Brown (mattbrown-2)]

If this can be recreated or still exists I'd like to examine zookeeper.