gnome-shell assert failure: double free or corruption (fasttop) in g_free() from g_error_free() from cogl_error_free() from cogl_texture_new_with_size() from clutter_offscreen_effect_real_create_texture()

Bug #1790525 reported by Gert van de Kraats on 2018-09-03
14
This bug affects 1 person
Affects Status Importance Assigned to Milestone
mutter (Ubuntu)
Medium
Marco Trevisan (Treviño)
Bionic
Undecided
Marco Trevisan (Treviño)
Cosmic
Undecided
Marco Trevisan (Treviño)

Bug Description

[ Impact ]

Gnome shell crashes on double free

Problem is occurring if dual monitor is used. Second monitor is repeatingly blank and activated again. At the bottom it probably contains the upper part of the first monitor. I had the same problem at 18.04 when using gdm3 without wayland during logon.

[ Test case ]

- Run gnome-shell with multimonitor
- No flashing should happen on gdm initialization

Also we should monitor crashes in e.u.c

[ Regression potential ]

Low, the proposed fix is part of the current stable branch upstream and not changed in further revisions.

--

ProblemType: CrashDistroRelease: Ubuntu 18.10
Package: gnome-shell 3.29.90-2ubuntu1
ProcVersionSignature: Ubuntu 4.17.0-9.10-generic 4.17.17
Uname: Linux 4.17.0-9-generic i686
ApportVersion: 2.20.10-0ubuntu9
Architecture: i386
AssertionMessage: double free or corruption (fasttop)
CurrentDesktop: ubuntu:GNOME
Date: Mon Sep 3 21:02:58 2018
DisplayManager: gdm3
ExecutablePath: /usr/bin/gnome-shell
GsettingsChanges: b'org.gnome.desktop.interface' b'gtk-im-module' b"'gtk-im-context-simple'"
ProcCmdline: /usr/bin/gnome-shell
ProcEnviron:
 LANGUAGE=en_US:en
 PATH=(custom, no user)
 XDG_RUNTIME_DIR=<set>
 LANG=en_US.UTF-8
 SHELL=/bin/bash
Signal: 6SourcePackage: gnome-shell
StacktraceTop:
 __libc_signal_restore_set (set=0xbfcd5e9c) at ../sysdeps/unix/sysv/linux/nptl-signals.h:80
 __GI_raise (sig=6) at ../sysdeps/unix/sysv/linux/raise.c:48
 __GI_abort () at abort.c:79
 __libc_message (action=do_abort, fmt=<optimized out>) at ../sysdeps/posix/libc_fatal.c:181
 malloc_printerr (str=str@entry=0xb6b437e8 "double free or corruption (fasttop)") at malloc.c:5350
Title: gnome-shell assert failure: double free or corruption (fasttop)
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups: adm cdrom dip lpadmin plugdev sambashare sudo

Related branches

Gert van de Kraats (gkraats) wrote :

StacktraceTop:
 __libc_signal_restore_set (set=0xbfcd5e9c) at ../sysdeps/unix/sysv/linux/nptl-signals.h:80
 __GI_raise (sig=6) at ../sysdeps/unix/sysv/linux/raise.c:48
 __GI_abort () at abort.c:79
 __libc_message (action=do_abort, fmt=<optimized out>) at ../sysdeps/posix/libc_fatal.c:181
 malloc_printerr (str=str@entry=0xb6b437e8 "double free or corruption (fasttop)") at malloc.c:5350

Changed in gnome-shell (Ubuntu):
importance: Undecided → Medium
tags: removed: need-i386-retrace

Problem disappeared as soon as manually at $HOME/.config the missing monitors.xml file with the correct dual monitor configuration is written. Initial configuration of this file using the graphical interface is not possible, because the second monitor is switching on and off, because of some looping

summary: - gnome-shell assert failure: double free or corruption (fasttop)
+ gnome-shell assert failure: double free or corruption (fasttop) in
+ g_free() from g_error_free() from cogl_error_free() from
+ cogl_texture_new_with_size() from
+ clutter_offscreen_effect_real_create_texture()
information type: Private → Public
Gert van de Kraats (gkraats) wrote :
Download full text (4.2 KiB)

Extra info:

1.The syslog at a crash shows next error. Probably memory management at an errormemssage is not correct. Also look at the size 2560 x 1024
Sep 08 00:18:35 Gert2 gnome-shell[2141]: CoglError set over the top of a previous CoglError or uninitialized memory.
                                          This indicates a bug in someone's code. You must ensure an error is NULL before it's set.
                                          The overwriting error message was: Sliced texture size of 2560 x 1024 not possible with max waste set to -1
 Sep 08 00:18:35 Gert2 org.gnome.Shell.desktop[2141]: double free or corruption (fasttop)
 Sep 08 00:18:35 Gert2 org.gnome.Shell.desktop[2141]: GNOME Shell crashed with signal 6

2.It looks like the problem in an older problem. At least at Ubuntu 18.04 with display manager lightdm the same crash occurs if no monitors.xml is present and dual monitor is present.

3.It looks like the problem is caused by the different sizes of the monitors.
At startup gdm3 detects next sizes for laptop-monitor and external monitor:

Sep 9 20:22:54 Gert2 /usr/lib/gdm3/gdm-x-session[1915]: (--) intel(0): Output LVDS1 using initial mode 1280x800 on pipe 1
Sep 9 20:22:54 Gert2 /usr/lib/gdm3/gdm-x-session[1915]: (--) intel(0): Output VGA1 using initial mode 1280x1024 on pipe 0

If at logon there is no monitors.xml file for the user, gdm3 by default tries to configure the external VGA1-monitor at the right of the laptop LVDS1-monitor. This fails, probably because the vertical size 800 of LVDS1 is smaller than 1024 for VGA1. gdm3 retries, which is causing a loop until VDA1-monitor is disconnected (see next log):

Sep 9 20:23:05 Gert2 /usr/lib/gdm3/gdm-x-session[1915]: (II) intel(0): resizing framebuffer to 1280x800
Sep 9 20:23:13 Gert2 /usr/lib/gdm3/gdm-x-session[1915]: (II) intel(0): resizing framebuffer to 2560x1024
Sep 9 20:23:13 Gert2 /usr/lib/gdm3/gdm-x-session[1915]: (II) intel(0): switch to mode 1280x1024@60.0 on VGA1 using pipe 0, position (1280, 0), rotation normal, reflection none

Sep 9 20:23:18 Gert2 /usr/lib/gdm3/gdm-x-session[1915]: (II) intel(0): resizing framebuffer to 1280x800
Sep 9 20:23:27 Gert2 /usr/lib/gdm3/gdm-x-session[1915]: (II) intel(0): resizing framebuffer to 2560x1024
Sep 9 20:23:27 Gert2 /usr/lib/gdm3/gdm-x-session[1915]: (II) intel(0): switch to mode 1280x1024@60.0 on VGA1 using pipe 0, position (1280, 0), rotation normal, reflection none

4.If monitors.xml is present with configuration of VGA1 above LVDS1, dual monitors can be used. Also in this case it is not possible to graphically configure via the displays arrangement the configuration with above sizes and VGA1 right of LVDS1.

5. A correct monitors.xml-file does not totally solve the problem.
Frequently the start if Ubuntu hangs completely before showing the logon-screen. It is showing a character screen with next last line.

[ OK ] Started GNOME Display Manager.

Syslog gives next lines then:

Sep 9 00:40:47 Gert2 gnome-shell[1309]: JS WARNING: [resource:///org/gnome/shell/ui/windowManager.js 1573]: reference to undefined property "MetaWindowXwayland"
Sep 9 00:40:47 Gert2 gnome-shell[1309]: Failed to allocate texture: Faile...

Read more...

Gert van de Kraats (gkraats) wrote :
Download full text (8.7 KiB)

Proposed solution:

As the stacktrace below shows the problem is caused by module cogl_texture_new_with_size at cogl/cogl/deprecated/cogl/auto-texture.c .
This module is trying to create a texture with size 2560 x 1024, which is by default is configured by gdm3 with wayland for 2 monitors.
This is not supported by graphics-card, causing error "Failed to create texture 2d due to size/format constraints".
The error is freed by cogl_texture_new_with_size, but the variable skip_error is not set to NULL by the call to cogl_error_free (because the parameter is a copy).

Then module tries to allocate a slice with max_waste -1. This also fails, causing error "Sliced texture size of 2560 x 1024 not possible with max waste set to -1".
Module cogl_set_error complains about the skip_error not being NULL with message "CoglError set...", but does not set a new value to skip_error.
Also the second error is programmed to be freed by cogl_error_free at cogl_texture_new_with_size, that in this way tries to free the same memory twice. This causes the crash "double free or corruption (fasttop)", unless by accident the same memory-address is allocated again.

To solve, the statement skip_error = NULL; should be added:
cogl_texture_new_with_size (unsigned int width,
       unsigned int height,
                            CoglTextureFlags flags,
       CoglPixelFormat internal_format)
{
  CoglTexture *tex;
  CoglError *skip_error = NULL;

  _COGL_GET_CONTEXT (ctx, NULL);

  if ((_cogl_util_is_pot (width) && _cogl_util_is_pot (height)) ||
      (cogl_has_feature (ctx, COGL_FEATURE_ID_TEXTURE_NPOT_BASIC) &&
       cogl_has_feature (ctx, COGL_FEATURE_ID_TEXTURE_NPOT_MIPMAP)))
    {
      /* First try creating a fast-path non-sliced texture */
      tex = COGL_TEXTURE (cogl_texture_2d_new_with_size (ctx, width, height));

      _cogl_texture_set_internal_format (tex, internal_format);

      if (!cogl_texture_allocate (tex, &skip_error))
        {
          cogl_error_free (skip_error);
          skip_error = NULL;

This solution is tested and solves the problem during startup and for right corner click.

Stacktrace with modulenames/linenumbers:
Package: gnome-shell 3.30.0-1ubuntu2
Stacktrace:
 #0 0xb7ef9d41 in __kernel_vsyscall ()
 #1 0xb6a4e512 in __libc_signal_restore_set (set=0xbfcb8a0c) at ../sysdeps/unix/sysv/linux/internal-signals.h:84
         set = {__val = {0, 0, 1482184750, 5789784, 2237142784, 273, 273, 3080022123, 3080956952, 16, 3080959932, 16, 3080023695, 16, 3217787612, 3066097232, 3080956952, 3080959932, 19876880, 3217787612, 3080285386, 11, 0, 3080023577, 3080285340, 3080956952, 16, 3080959932, 3080016471, 3217788168, 0, 3217787712}}
         pid = <optimized out>
         tid = <optimized out>
         ret = 0
 #2 0xb6a4e512 in __GI_raise (sig=6) at ../sysdeps/unix/sysv/linux/raise.c:48
         set = {__val = {0, 0, 1482184750, 5789784, 2237142784, 273, 273, 3080022123, 3080956952, 16, 3080959932, 16, 3080023695, 16, 3217787612, 3066097232, 3080956952, 3080959932, 19876880, 3217787612, 3080285386, 11, 0, 3080023577, 3080285340, 3080956952, 16, 3080959932, 3080016471, 3217788168, 0, 3217787712}}
         pid = <optimized out>
         tid = <...

Read more...

Changed in mutter (Ubuntu):
importance: Undecided → Medium
status: New → Triaged
assignee: nobody → Daniel van Vugt (vanvugt)
Changed in mutter (Ubuntu):
status: Triaged → In Progress
Changed in mutter (Ubuntu):
assignee: Daniel van Vugt (vanvugt) → Marco Trevisan (Treviño) (3v1n0)
Changed in gnome-shell (Ubuntu):
status: New → In Progress
no longer affects: gnome-shell (Ubuntu)
description: updated
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package mutter - 3.30.2-5

---------------
mutter (3.30.2-5) unstable; urgency=medium

  * d/p/clutter-Avoid-rounding-compensation-when-invalidating-2D-.patch,
    d/p/clutter-Fix-offscreen-effect-painting-of-clones.patch:
    - Fix offscreen-effect painting of clones in zoom mode (LP: #1767648,
      LP: #1779615)
  * d/p/cogl-auto-texture-Avoid-a-double-free-crash.patch,
    d/p/clutter-offscreen-effect-Disable-if-no-texture.patch:
    - Fix crash in dual monitor setup and gdm activation (LP: #1790525,
      LP: #1795774)

 -- Marco Trevisan (Treviño) <email address hidden> Thu, 24 Jan 2019 18:00:14 +0000

Changed in mutter (Ubuntu):
status: In Progress → Fix Released
Iain Lane (laney) on 2019-01-25
Changed in mutter (Ubuntu Bionic):
status: New → Incomplete
status: Incomplete → In Progress
Changed in mutter (Ubuntu Cosmic):
status: New → In Progress
Changed in mutter (Ubuntu Bionic):
assignee: nobody → Marco Trevisan (Treviño) (3v1n0)
Changed in mutter (Ubuntu Cosmic):
assignee: nobody → Marco Trevisan (Treviño) (3v1n0)

Hello Gert, or anyone else affected,

Accepted mutter into cosmic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/mutter/3.30.2-1~ubuntu18.10.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-cosmic to verification-done-cosmic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-cosmic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in mutter (Ubuntu Cosmic):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-cosmic
Gert van de Kraats (gkraats) wrote :

Reinstalled current libmutter.
With dual monitor horizontal aside this gives
Feb 1 22:26:58 Gert2 gnome-shell[1038]: Failed to allocate texture: Failed to create texture 2d due to size/format constraints
Feb 1 22:26:58 Gert2 gnome-shell[1038]: CoglError set over the top of a previous CoglError or uninitialized memory.#012This indicates a bug in someone's code. You must ensure an error is NULL before it's set.#012The overwriting error message was: Sliced texture size of 2560 x 1024 not possible with max waste set to -1

Logon-session crashes before showing logon-screen.

Installed proposed version:

apt list libmutter-3-0
Listing... Done
libmutter-3-0/cosmic-proposed,now 3.30.2-1~ubuntu18.10.3 i386 [installed]

Problem is solved by proposed package. No new other problems detected.

tags: added: verification-done-cosmic
removed: cosmic verification-needed-cosmic
tags: added: cosmic
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers