Performance regression in git150326 due to hbase-trx

[Atanu Mishra]

What we were seeing is that on a 1024 stream OE 30 minute test, the TPM rate would level out. At around half way through or a little after, the TPM rate started continually dropping every minute. By the 30 minute mark we were down ~20%.

The git032415 and git032515 builds did not display this rapid degradation in performance.

For these two builds, we did see performance degrade over time, but the TPM rate at the 30 minute mark was consistent with the numbers seen previously.

[Oliver Bucaojit (oliver-bucaojit)]

Discussed with Joanie and ShangSheng the change and it was a 'static' stoppable variable that was also being used to synchronize a block in the TrxRegionEndpoing.

The fix was to make the stoppable non-static and have a separate static Object to use for the synchronization.

Joanie has the fix and Guy tested the UTT to make sure the performance did not degrade.

[Atanu Mishra (atanu-mishra)]

Fix was checked in and confirmed by performance.