Trafodion

Bug #1380734
Comment #4

Comment 4 for bug 1380734

Revision history for this message

Oliver Bucaojit (oliver-bucaojit) wrote on 2014-12-17:

Checked-in fix: https://review.trafodion.org/#/c/838/ Merged 12/16

-- Email of the fix, details --
I took a look at the issue and used the upsert query to reproduce the problem.

While looking at the tracing I saw that there were 3 put’s being sent to 1 region, yet 2 regions were being registered to. In the client-side code I saw the work was being done in the multi-put method so that made sense to me as to how there was a chance that the row was being sent to the wrong region. In the multi-put, you send a list of puts over to the server and so that is how the rows made its way over to the incorrect region, the startKey that was being used in the coprocessor call came from a different row in the list.

Looking deeper into the multi-put I saw that it puts the HRegionLocations in a HashMap with the list of puts as the value. And that was where the problem was. There is an issue we found before (issue for us, but doesn’t seem to be one for the greater HBase) where the HRegionLocations are equal as long as their server name is equal. That lead us to extend that class and use TransactionRegionLocation instead so we can also compare the HRegionInfo for more granularity.

So by switching the HRegionLocation to TransactionRegionLocation in the multi-put, the HashMap was able to have separate region location keys instead of lumping them all into one key, which fixes the issue. I tested with the below upsert command and with the setup/load scripts from Mike Hanlon.

I’ll look for other locations where the change would be needed and run some tests then check-in the fix.

---

SQL output of successful run.

--- SQL operation complete.
>>CREATE TABLE TCHAR_T4
       (
              A CHAR(5) not null
              , B CHAR(5) not null
              , PRIMARY KEY (A)
       ) salt using 8 partitions
;
upsert into TCHAR_T4 values ('A', 'B'), ('C', 'D'), ('E', 'F');
+>+>+>+>+>+>

--- SQL operation complete.
>>
--- 3 row(s) inserted.
>>>>select * from TCHAR_T4;

A B
----- -----

A B
C D
E F

--- 3 row(s) selected.
>>exit

Thanks, Oliver

Checked-in fix: https://review.trafodion.org/#/c/838/ Merged 12/16

-- Email of the fix, details --
I took a look at the issue and used the upsert query to reproduce the problem.

While looking at the tracing I saw that there were 3 put’s being sent to 1 region, yet 2 regions were being registered to.  In the client-side code I saw the work was being done in the multi-put method so that made sense to me as to how there was a chance that the row was being sent to the wrong region.  In the multi-put, you send a list of puts over to the server and so that is how the rows made its way over to the incorrect region, the startKey that was being used in the coprocessor call came from a different row in the list.

Looking deeper into the multi-put I saw that it puts the HRegionLocations in a HashMap with the list of puts as the value.  And that was where the problem was.  There is an issue we found before (issue for us, but doesn’t seem to be one for the greater HBase) where the HRegionLocations are equal as long as their server name is equal.  That lead us to extend that class and use TransactionRegionLocation instead so we can also compare the HRegionInfo for more granularity.

So by switching the HRegionLocation to TransactionRegionLocation in the multi-put, the HashMap was able to have separate region location keys instead of lumping them all into one key, which fixes the issue.  I tested with the below upsert command and with the setup/load scripts from Mike Hanlon.

I’ll look for other locations where the change would be needed and run some tests then check-in the fix.

---

SQL output of successful run.

--- SQL operation complete.
>>CREATE TABLE TCHAR_T4
       (
              A             CHAR(5) not null
              , B        CHAR(5) not null
              , PRIMARY KEY (A)
       ) salt using 8 partitions
;
upsert into TCHAR_T4 values ('A', 'B'), ('C', 'D'), ('E', 'F');
+>+>+>+>+>+>

--- SQL operation complete.
>>
--- 3 row(s) inserted.
>>>>select * from TCHAR_T4;

A      B
-----  -----

A      B
C      D
E      F

--- 3 row(s) selected.
>>exit

Thanks, Oliver