VSBB update causes TM heap leak

Bug #1463179 reported by Buddy Wilbanks
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Trafodion
Fix Committed
Critical
Joanie Cooper

Bug Description

Since the May 1 executor checkin that turned on VSBB update, the TM will leak memory to the point of an OOM condition after only 26 hours of the longevity test. Pretty sure it will be much quicker if we limit longevity to only perform the delivery transaction, which is the only VSBB invoker.

We've collected jmaps for some of the TMs on zircon4, and core files for analysis. Located on /home/squser4/fqw/TMdumps. You can see the corefile size increasing as well as the old space usage. Here's the first and last corefiles taken for pid 9915

-rw------- 1 squser4 seaquest 113142116 Jun 8 18:51 jmapdump-9915.0
-rw------- 1 squser4 seaquest 341615426 Jun 8 19:58 jmapdump-9915.13

and here is the initial jmap-9915.0 old space:

PS Old Generation
   capacity = 1407713280 (1342.5MB)
   used = 100979864 (96.3019027709961MB)
   free = 1306733416 (1246.198097229004MB)
   7.17332609094943% used

and then jmap-9915.13

PS Old Generation
   capacity = 571998208 (545.5MB)
   used = 250169256 (238.57999420166016MB)
   free = 321828952 (306.92000579833984MB)

it grows about 12Meg every 5 minutes.

Tags: dtm heap leak vsbb
Changed in trafodion:
status: New → In Progress
Revision history for this message
Joanie Cooper (joanie-cooper) wrote :

We have identified a problem in the start/end key values used in the coprocessor service call for a putMultiple or deleteMultiple invocation. We will now use the first row in the list of puts or deletes to determine an accurate start/end key pair for the service call. The changes have been made and SQL regressions pass. A UTT was provided to the performance team. They reported no problems or performance regressions. The fix is being delivered to the "core" branch.

Revision history for this message
Joanie Cooper (joanie-cooper) wrote :

The fix was committed on 06/10/15.

Changed in trafodion:
status: In Progress → Fix Committed
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers