openjdk6 regression causes finalizers never to be called

Bug #1295987 reported by Alex Bligh on 2014-03-22
42
This bug affects 6 people
Affects Status Importance Assigned to Milestone
openjdk-6 (Ubuntu)
Undecided
Jamie Strandboge
Lucid
Undecided
Jamie Strandboge
Precise
Undecided
Jamie Strandboge
Quantal
Undecided
Jamie Strandboge
Saucy
Undecided
Jamie Strandboge
Trusty
Undecided
Jamie Strandboge

Bug Description

Upgrading openjdk6 from 6b27-1.12.6-1ubuntu0.12.04.4 to 6b30-1.13.1-1ubuntu2~0.12.04.1 (Precise update in Jan / Feb 2014) causes finalizers not to run reliably. Whilst it is good practice to avoid use of finalizers, JDBC uses them to close connections, sockets use them to close sockets etc., and after this upgrade both of them leak apparently randomly under moderate load.

This is difficult to replicate but we believe it is due to: https://java.net/jira/browse/OPENJDK6-29 which was introduced in b28. The critical sentences here are: "This can cause mayhem in the JVM because the flag fields at the end of the struct end up taking values determined by the contents of the stack frame where info is allocated. In particular, it can cause the post-jdk6-b28 GC to enqueue discovered References using the wrong link field which means that no reference processing (including finalization) occurs."

This is fixed in b31, and the fix is a one line obviously correct change:
  http://hg.openjdk.java.net/jdk6/jdk6/jdk/rev/04e4c3ec6516

which correctly memset's a structure to zero, as opposed to only setting the first 8 bytes (64 bit) or 4 bytes (32 bit) to zero.

As this is a clear regression caused by an update with a substantial impact (unresolvable memory leaks of objects with finalizers, other resource leaks including FD leaks), I think this is an SRU candidate for Precise (which is where we are seeing it).

Alex Bligh (ubuntu-alex-org) wrote :

https://launchpad.net/ubuntu/+source/openjdk-6/+publishinghistory

confirms that this would have been introduced into Precise on 2014-02-27

Alex Bligh (ubuntu-alex-org) wrote :

Mailing list discussion:
  http://comments.gmane.org/gmane.comp.java.openjdk.jdk6.devel/1114

cleaned up in openjdk7 amongst a much larger commit

Alex Bligh (ubuntu-alex-org) wrote :

I'm currently running with the following patch (not entirely sure how the packaging works and whether I've done this the right wy). Passes all tests.

Alex Bligh (ubuntu-alex-org) wrote :

The above patch has a second instance of the same issue fixed. Mailing list thread from upstream here:
http://mail.openjdk.java.net/pipermail/jdk6-dev/2014-March/003253.html

The attachment "Patch to ubuntu packaging" seems to be a patch. If it isn't, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are a member of the ~ubuntu-reviewers, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issues please contact him.]

tags: added: patch
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in openjdk-6 (Ubuntu):
status: New → Confirmed
tags: added: precise regression-update
Changed in openjdk-6 (Ubuntu):
status: Confirmed → Triaged
assignee: nobody → Jamie Strandboge (jdstrand)
Changed in openjdk-6 (Ubuntu Lucid):
status: New → Triaged
Changed in openjdk-6 (Ubuntu Precise):
status: New → Triaged
Changed in openjdk-6 (Ubuntu Quantal):
status: New → Triaged
Changed in openjdk-6 (Ubuntu Saucy):
status: New → Triaged
Jamie Strandboge (jdstrand) wrote :

I have uploaded packages for Ubuntu 12.04 and 14.04 to https://launchpad.net/~ubuntu-security-proposed/+archive/ppa and they should be done building in a couple hours.

Can someone who is affected by this bug comment on either to confirm that they fix this bug? If the issue is fixed, I will prepare packages for the other releases and a USN.

Thanks!

Changed in openjdk-6 (Ubuntu Trusty):
status: Triaged → Fix Committed
Changed in openjdk-6 (Ubuntu Precise):
status: Triaged → Fix Committed
Changed in openjdk-6 (Ubuntu Saucy):
status: Triaged → In Progress
Changed in openjdk-6 (Ubuntu Quantal):
status: Triaged → In Progress
Changed in openjdk-6 (Ubuntu Precise):
assignee: nobody → Jamie Strandboge (jdstrand)
Changed in openjdk-6 (Ubuntu Quantal):
assignee: nobody → Jamie Strandboge (jdstrand)
Changed in openjdk-6 (Ubuntu Lucid):
assignee: nobody → Jamie Strandboge (jdstrand)
Changed in openjdk-6 (Ubuntu Saucy):
assignee: nobody → Jamie Strandboge (jdstrand)
Alex Bligh (ubuntu-alex-org) wrote :

Jamie: we've verified that the patch I applied *appears* to fix the bug we are seeing. I say *appears* as the bug manifesting itself seems to be a function of uninitialized values on the stack; on some runs it explodes with leaking stuff straight away, other times it runs fine for many many hours, and only later explodes. I haven't verified your .deb files but could you tell me what patch(es) they are using? I can then get QA here to play with them.

Jamie Strandboge (jdstrand) wrote :

Hrm, seems I missed the patch to hotspot: https://bugs.openjdk.java.net/browse/JDK-8035893.

Alex Bligh (ubuntu-alex-org) wrote :

Yes , the version I tested (with my patch, as attached to this bug) has both of those fixed.

Changed in openjdk-6 (Ubuntu Trusty):
status: Fix Committed → In Progress
Changed in openjdk-6 (Ubuntu Precise):
status: Fix Committed → In Progress
Jamie Strandboge (jdstrand) wrote :

Updated packages have been uploaded to the security-proposed PPA (6b30-1.13.1-1ubuntu2~0.12.04.3 for precise and 6b30-1.13.1-1ubuntu4 for trusty). These incorporate both fixes. Also, I verified that these patches are applied in upstream IcedTea as well so this should not regress in the next update.

Changed in openjdk-6 (Ubuntu Lucid):
status: Triaged → In Progress
Changed in openjdk-6 (Ubuntu Precise):
status: In Progress → Fix Committed
Changed in openjdk-6 (Ubuntu Trusty):
status: In Progress → Fix Committed
Jamie Strandboge (jdstrand) wrote :

Can someone who is affected by this bug comment on either the new 14.04 or the 12.04 packages to confirm that they fix this bug? If the issue is fixed, I will prepare packages for the other releases and issue a USN.

Thanks!

Alex Bligh (ubuntu-alex-org) wrote :

Will do once they have finished building. Is there a link to the diff in the mean time so I can check it against the one we've been using?

Jamie Strandboge (jdstrand) wrote :

Here is the debdiff used for precise.

Alex Bligh (ubuntu-alex-org) wrote :

Jamie: thanks. They look good to me (from inspection of the diffs). Will get these checked tomorrow once they have built. Normally our tests take a couple of days to run.

Michael Kosteva (mkosteva) wrote :

A half-day's worth of before/after testing in our QA environment, and we have not seen a recurrence of the bugged behaviour with the builds from PPA (12.04 amd64), while consistently being able to reproduce it on pre-patched debs on the same instances.

thanks

Alex Bligh (ubuntu-alex-org) wrote :

48 hours of testing show no problems with these packages, whereas the previously released packages show leaks. We will continue to run these tests over the weekend, but it's looking good for us so far.

Jamie Strandboge (jdstrand) wrote :

Thanks for the feedback! I uploaded updates for 10.04, 12.10 and 13.10 to the security-proposed ppa. Unless a problem comes up over the weekend, I plan to push these out on Monday.

Changed in openjdk-6 (Ubuntu Lucid):
status: In Progress → Fix Committed
Changed in openjdk-6 (Ubuntu Quantal):
status: In Progress → Fix Committed
Changed in openjdk-6 (Ubuntu Saucy):
status: In Progress → Fix Committed
Alex Bligh (ubuntu-alex-org) wrote :

I can confirm the Precise packages passed our torture tests over the weekend. These are good to go as far as I am concerned.

Alex Bligh (ubuntu-alex-org) wrote :

Jamie: any news on this one?

Jamie:
I can also confirm that those packages do fix the problem for us.

I've added your proposed precise package 6b30-1.13.1-1ubuntu2~0.12.04.3 0 to two of our servers which were crashing with the original 1.13.1 packages frequently (twice a day).

Both servers have been running now around 25hour without any problems.

pionic (o-marius) wrote :

Any update on this ticket? Thanks

Jamie Strandboge (jdstrand) wrote :

It will be pushed today.

Changed in openjdk-6 (Ubuntu Trusty):
status: Fix Committed → Fix Released
Alex Bligh (ubuntu-alex-org) wrote :

Any news on the fix being released for Lucid?

Launchpad Janitor (janitor) wrote :

This bug was fixed in the package openjdk-6 - 6b30-1.13.1-1ubuntu2~0.10.04.2

---------------
openjdk-6 (6b30-1.13.1-1ubuntu2~0.10.04.2) lucid-security; urgency=medium

  * Update to properly zero version info (LP: #1295987)
    - 6989972.diff: JDK fails to zero jdk_version_info. Apply in
      DISTRIBUTION_PATCHES
    - debian/openjdk/8035893.diff: JVM_GetVersionInfo fails to zero structure.
      Apply in DISTRIBUTION_PATCHES
 -- Jamie Strandboge <email address hidden> Fri, 28 Mar 2014 12:58:47 -0500

Changed in openjdk-6 (Ubuntu Lucid):
status: Fix Committed → Fix Released
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package openjdk-6 - 6b30-1.13.1-1ubuntu2~0.12.10.2

---------------
openjdk-6 (6b30-1.13.1-1ubuntu2~0.12.10.2) quantal-security; urgency=medium

  * Update to properly zero version info (LP: #1295987)
    - 6989972.diff: JDK fails to zero jdk_version_info. Apply in
      DISTRIBUTION_PATCHES
    - debian/openjdk/8035893.diff: JVM_GetVersionInfo fails to zero structure.
      Apply in DISTRIBUTION_PATCHES
 -- Jamie Strandboge <email address hidden> Fri, 28 Mar 2014 13:00:04 -0500

Changed in openjdk-6 (Ubuntu Quantal):
status: Fix Committed → Fix Released
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package openjdk-6 - 6b30-1.13.1-1ubuntu2~0.13.10.2

---------------
openjdk-6 (6b30-1.13.1-1ubuntu2~0.13.10.2) saucy-security; urgency=medium

  * Update to properly zero version info (LP: #1295987)
    - 6989972.diff: JDK fails to zero jdk_version_info. Apply in
      DISTRIBUTION_PATCHES
    - debian/openjdk/8035893.diff: JVM_GetVersionInfo fails to zero structure.
      Apply in DISTRIBUTION_PATCHES
 -- Jamie Strandboge <email address hidden> Fri, 28 Mar 2014 13:01:05 -0500

Changed in openjdk-6 (Ubuntu Saucy):
status: Fix Committed → Fix Released
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package openjdk-6 - 6b30-1.13.1-1ubuntu2~0.12.04.3

---------------
openjdk-6 (6b30-1.13.1-1ubuntu2~0.12.04.3) precise-security; urgency=medium

  * Update to properly zero version info (LP: #1295987)
    - 6989972.diff: JDK fails to zero jdk_version_info. Apply in
      DISTRIBUTION_PATCHES
    - debian/openjdk/8035893.diff: JVM_GetVersionInfo fails to zero structure.
      Apply in DISTRIBUTION_PATCHES
 -- Jamie Strandboge <email address hidden> Wed, 26 Mar 2014 08:43:13 -0500

Changed in openjdk-6 (Ubuntu Precise):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers