[regression] ubiquity crashed in debconf.py:104 with ValueError: invalid literal for int() with base 10: ''

[Jean-Baptiste Lallement]

* Impact
The Ubuntu installer crashes on some machines (seems more often on hiDPI machines (QHD/UHD etc)).

* Test case
Try installing Ubuntu on an hidpi machine, it should complete installation instead of crashing.

* Regression potential
The fix in ubiquity touches the code handle uid drop/privilege, that can have unexpected side effects so we need proper/complete testing of the installer

----------

Update: Actually the crash occurs on slow-ish systems due to a race condition. It's not strictly only hi-DPI machines - that's just the most common place it is experienced.

---

https://errors.ubuntu.com/problem/82f7f7e7923663c7b2123c7f1f49af29f6ff4d77
https://errors.ubuntu.com/problem/735a2b847e0eeab6c8a7b954de5110e43889be15
https://errors.ubuntu.com/problem/dcd4c9da5ee0cc6d36324446e0e49d39705c90b7
https://errors.ubuntu.com/problem/cb82f70f9ede07369e8104da9ddf87e28b42257d
https://errors.ubuntu.com/problem/84a5563af3d2b85f098da832ece4cb8450bfd524

---

WORKAROUND:

1. Boot into the live session.
2. Settings > Devices > Displays > Scale = 100%
3. Click Apply.
4. Proceed with installation: Click "Install Ubuntu 18.04 LTS".

---

Crashed in a VM in the middle of installation. The host is Bionic up to date.

From the journal
Feb 23 12:52:27 ubuntu kernel: traps: ubiquity[2646] trap int3 ip:7f5a76936961 sp:7ffde5090c50 error:0 in libglib-2.0.so.0.5400.1[7f5a768e6000+111000]
Feb 23 12:52:41 ubuntu /install.py[6858]: Exception during installation:
Feb 23 12:52:41 ubuntu /install.py[6858]: Traceback (most recent call last):
Feb 23 12:52:41 ubuntu /install.py[6858]: File "/usr/share/ubiquity/install.py", line 757, in <module>
Feb 23 12:52:41 ubuntu /install.py[6858]: install.run()
Feb 23 12:52:41 ubuntu /install.py[6858]: File "/usr/share/ubiquity/install.py", line 135, in run
Feb 23 12:52:41 ubuntu /install.py[6858]: self.copy_all()
Feb 23 12:52:41 ubuntu /install.py[6858]: File "/usr/share/ubiquity/install.py", line 505, in copy_all
Feb 23 12:52:41 ubuntu /install.py[6858]: self.db.progress('SET', 10 + copy_progress)
Feb 23 12:52:41 ubuntu /install.py[6858]: File "/usr/lib/python3/dist-packages/debconf.py", line 83, in <lambda>
Feb 23 12:52:41 ubuntu /install.py[6858]: lambda *args, **kw: self.command(command, *args, **kw))
Feb 23 12:52:41 ubuntu /install.py[6858]: File "/usr/lib/python3/dist-packages/debconf.py", line 104, in command
Feb 23 12:52:41 ubuntu /install.py[6858]: status = int(status)
Feb 23 12:52:41 ubuntu /install.py[6858]: ValueError: invalid literal for int() with base 10: ''

ProblemType: Crash
DistroRelease: Ubuntu 18.04
Package: ubiquity 18.04.1
ProcVersionSignature: Ubuntu 4.13.0-32.35-generic 4.13.13
Uname: Linux 4.13.0-32-generic x86_64
ApportVersion: 2.20.8-0ubuntu10
Architecture: amd64
CasperVersion: 1.388
Date: Fri Feb 23 12:52:28 2018
ExecutablePath: /usr/lib/ubiquity/bin/ubiquity
InstallCmdLine: file=/cdrom/preseed/ubuntu.seed boot=casper initrd=/casper/initrd.lz quiet splash --- keyboard-configuration/layoutcode=fr keyboard-configuration/variantcode=oss
InterpreterPath: /usr/bin/python3.6
LiveMediaBuild: Ubuntu 18.04 LTS "Bionic Beaver" - Alpha amd64 (20180222)
ProcCmdline: /usr/bin/python3 /usr/lib/ubiquity/bin/ubiquity -d
ProcEnviron:
 TERM=xterm-256color
 PATH=(custom, no user)
 LANG=C.UTF-8
 SHELL=/bin/bash
Python3Details: /usr/bin/python3.6, Python 3.6.4+, python3-minimal, 3.6.4-1
PythonDetails: /usr/bin/python2.7, Python 2.7.14+, python-minimal, 2.7.14-4
Signal: 5
SourcePackage: ubiquity
StacktraceTop:
 ?? () from /usr/lib/x86_64-linux-gnu/libX11.so.6
 ?? () from /usr/lib/x86_64-linux-gnu/libX11.so.6
 _XEventsQueued () from /usr/lib/x86_64-linux-gnu/libX11.so.6
 XPending () from /usr/lib/x86_64-linux-gnu/libX11.so.6
 ?? () from /usr/lib/x86_64-linux-gnu/libgdk-3.so.0
Title: ubiquity crashed with signal 5 in _XEventsQueued()
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups:

[Bugs-freedesktop (bugs-freedesktop)]

_cairo_xlib_display_fini_shm sets pool->attached to XNextRequest() assuming
the approaching XShmAttach() will be the next request.
https://cgit.freedesktop.org/cairo/tree/src/cairo-xlib-surface-shm.c?id=3f1a6f7225e31057a8af9313f051a1d311df0c69#n602

This assumption can be invalid when another request is performed on another
thread before the XShmAttach() reads |request| from the display.

An |attached| sequence number that is too old means that
_cairo_xlib_shm_pool_cleanup() can call _cairo_xlib_display_shm_pool_destroy()
and so shmdt() before the server processes the ShmAttach request, resulting in
BadAccess errors.

Similarly _cairo_xlib_shm_surface_mark_active() is called and uses
XNextRequest() before the corresponding request, leading to similar races
affecting _cairo_xlib_shm_surface_flush() and get_compositor() and
_cairo_xlib_shm_info_cleanup(). I assume _cairo_xlib_shm_surface_get_obdata()
has similar issues.

[Freedesktop-treblig (freedesktop-treblig)]

I wonder whether the fix of setting the pool->attached = XNextRequest(dpy)
after the XShmAttach would work. That's effectively delaying the shmdt until after whatever the next X request is after the attach has completed; who knows what that is, but as long as there is one (ahem, that might be a bit of an assumption; could we just add a dummy?) then it should complete. Not got a clue what the perf would be.

[Bugs-freedesktop (bugs-freedesktop)]

pool->attached = XNextRequest(dpy) - 1;
after the XShmAttach would work, yes.
It would then have a sequence number either equal to that of the XShmAttach request or to that of a subsequent request, if there was one.

[Psychon-d (psychon-d)]

Correct me if I'm wrong, but XLockDisplay() says that there is no other thread that could "steal the next request", right?
Also, feel free to correct me, but cairo should be calling XLockDisplay() before doing "SHM stuff".

[Spitzak-k (spitzak-k)]

XLockDisplay only works if XInitThreads was done, and that his highly not recommended as it slows Xlib down enormously (because it was very badly implemented with fine-grained locking).

[Bugs-freedesktop (bugs-freedesktop)]

XLockDisplay() would work, and there is no bug if there is only one thread (i.e. when XInitThreads() is not called), but XLockDisplay() is not necessary.

[Apport retracing service (apport)]

StacktraceTop:
 XPending (dpy=0x1384000) at ../../src/Pending.c:55
 client_message_to_wire (event=0x7ffde5091650, ev=0x0) at ../../../../../gdk/x11/gdkasync.c:199
 _gdk_x11_send_client_message_async (display=<optimized out>, window=140728446031488, propagate=1935767139, event_mask=140728446031480, event_send=0x0, callback=0x1291c00, data=0x2939b9c) at ../../../../../gdk/x11/gdkasync.c:263
 ?? ()
 ?? ()

[Sebastien Bacher (seb128)]

Steve, what makes you think it might be a glib issue?

[Iain Lane (laney)]

Marking Incomplete for glib per the previous comment.

[Ubuntu QA Website (ubuntuqa)]

This bug has been reported on the Ubuntu ISO testing tracker.

A list of all reports related to this bug can be found here:
http://iso.qa.ubuntu.com/qatracker/reports/bugs/1751252

[Casey Matson-deKay (caseymdk)]

I am getting the same issue, same error. Crashes once I reach the progress screen of 18.04 installation.

Syslog report:

......

Apr 28 20:26:54 ubuntu localechooser: info: Set debian-installer/locale = 'en_CA.UTF-8'
Apr 28 20:26:54 ubuntu localechooser: info: System locale (debian-installer/locale) = 'en_CA.UTF-8'
Apr 28 20:26:54 ubuntu ubiquity: /usr/lib/ubiquity/localechooser/localechooser: 910: [: C.UTF-8: unexpected operator
Apr 28 20:26:54 ubuntu ubiquity[3103]: debconffilter_done: ubi-timezone (current: ubi-timezone)
Apr 28 20:26:54 ubuntu ubiquity[3103]: Step_before = stepLocation
Apr 28 20:26:54 ubuntu partman: mke2fs 1.44.1 (24-Mar-2018)
Apr 28 20:26:54 ubuntu ubiquity[3103]: switched to page usersetup
Apr 28 20:26:54 ubuntu clear_partitions: Considering /home,/dev/nvme0n1p6.
Apr 28 20:26:54 ubuntu kernel: [ 100.161039] EXT4-fs (nvme0n1p6): mounted filesystem with ordered data mode. Opts: (null)
Apr 28 20:26:54 ubuntu kernel: [ 100.278116] EXT4-fs (nvme0n1p7): mounted filesystem with ordered data mode. Opts: errors=remount-ro
Apr 28 20:26:54 ubuntu kernel: [ 100.296127] EXT4-fs (nvme0n1p6): mounted filesystem with ordered data mode. Opts: (null)
Apr 28 20:26:55 ubuntu ubiquity: File descriptor 3 (pipe:[67210]) leaked on pvs invocation. Parent PID 18458: /bin/sh
Apr 28 20:26:55 ubuntu ubiquity[3103]: debconffilter_done: ubiquity.components.partman_commit (current: ubi-usersetup)
Apr 28 20:26:55 ubuntu /install.py: keeping packages due to preseeding: icedtea6-plugin openoffice.org
Apr 28 20:26:55 ubuntu /install.py: keeping language packs for: en_US.UTF-8
Apr 28 20:26:56 ubuntu ubiquity: W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/bionic/InRelease Could not resolve 'archive.ubuntu.com'
Apr 28 20:26:56 ubuntu ubiquity: W: Failed to fetch http://security.ubuntu.com/ubuntu/dists/bionic-security/InRelease Could not resolve 'security.ubuntu.com'
Apr 28 20:26:56 ubuntu ubiquity: W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/bionic-updates/InRelease Could not resolve 'archive.ubuntu.com'
Apr 28 20:26:56 ubuntu ubiquity: W: Some index files failed to download. They have been ignored, or old ones used instead.
Apr 28 20:26:56 ubuntu ubiquity: W: --force-yes is deprecated, use one of the options starting with --allow instead.
Apr 28 20:27:13 ubuntu ubiquity[3103]: debconffilter_done: ubi-usersetup (current: ubi-usersetup)
Apr 28 20:27:13 ubuntu ubiquity[3103]: Step_before = stepUserInfo
Apr 28 20:27:14 ubuntu kernel: [ 119.884166] do_trap: 26 callbacks suppressed
Apr 28 20:27:14 ubuntu kernel: [ 119.884168] traps: ubiquity[3103] trap int3 ip:7ff0a1502c41 sp:7ffe0681d810 error:0 in libglib-2.0.so.0.5600.1[7ff0a14b1000+113000]
Apr 28 20:27:25 ubuntu /install.py: Exception during installation:
Apr 28 20:27:25 ubuntu /install.py: Traceback (most recent call last):
Apr 28 20:27:25 ubuntu /install.py: File "/usr/share/ubiquity/install.py", line 765, in <module>
Apr 28 20:27:25 ubuntu /install.py: install.run()
Apr 28 20:27:25 ubuntu /install.py: File "/usr/share/ubiquity/install.py", line 135, in run
Apr 28 20:27:25 ubuntu /install.py: self.copy_all()
Apr 28 20:27:25 ubuntu /install.py: File "/usr/sha...

[Daniel van Vugt (vanvugt)]

Also tracking in:
https://errors.ubuntu.com/problem/82f7f7e7923663c7b2123c7f1f49af29f6ff4d77
https://errors.ubuntu.com/problem/735a2b847e0eeab6c8a7b954de5110e43889be15

[Daniel van Vugt (vanvugt)]

And my own crash from 18.04 final:

Apr 29 22:54:52 ubuntu ubiquity[3581]: debconffilter_done: ubi-usersetup (current: ubi-usersetup)
Apr 29 22:54:52 ubuntu ubiquity[3581]: Step_before = stepUserInfo
Apr 29 22:54:52 ubuntu kernel: do_trap: 28 callbacks suppressed
Apr 29 22:54:52 ubuntu kernel: traps: ubiquity[3581] trap int3 ip:7fd2ca14ec41 sp:7fffd3ffdfa0 error:0 in libglib-2.0.so.0.5600.1[7fd2ca0fd000+113000]
Apr 29 22:55:04 ubuntu /install.py[8718]: Exception during installation:
Apr 29 22:55:04 ubuntu /install.py[8718]: Traceback (most recent call last):
Apr 29 22:55:04 ubuntu /install.py[8718]: File "/usr/share/ubiquity/install.py", line 765, in <module>
Apr 29 22:55:04 ubuntu /install.py[8718]: install.run()
Apr 29 22:55:04 ubuntu /install.py[8718]: File "/usr/share/ubiquity/install.py", line 135, in run
Apr 29 22:55:04 ubuntu /install.py[8718]: self.copy_all()
Apr 29 22:55:04 ubuntu /install.py[8718]: File "/usr/share/ubiquity/install.py", line 513, in copy_all
Apr 29 22:55:04 ubuntu /install.py[8718]: self.db.progress('SET', 10 + copy_progress)
Apr 29 22:55:04 ubuntu /install.py[8718]: File "/usr/lib/python3/dist-packages/debconf.py", line 83, in <lambda>
Apr 29 22:55:04 ubuntu /install.py[8718]: lambda *args, **kw: self.command(command, *args, **kw))
Apr 29 22:55:04 ubuntu /install.py[8718]: File "/usr/lib/python3/dist-packages/debconf.py", line 104, in command
Apr 29 22:55:04 ubuntu /install.py[8718]: status = int(status)
Apr 29 22:55:04 ubuntu /install.py[8718]: ValueError: invalid literal for int() with base 10: ''
Apr 29 22:55:04 ubuntu /install.py[8718]:

[Daniel van Vugt (vanvugt)]

The X error (from comment #3) is:

#3 0x00007f5a769377f7 in g_log_structured (log_domain=0x7f5a7365304e <_gdk_wayland_display_utf8_to_string_target+270> "f\220I\213U\bH\215J\001I;M\020s\"I\213u", log_level=G_LOG_LEVEL_ERROR) at ../../../../glib/gmessages.c:1768
        args = {{gp_offset = 48, fp_offset = 48, overflow_arg_area = 0x7ffde50913e8, reg_save_area = 0x7ffde50912d0}}
        buffer = "`\t\223\002\000\000\000\000\300\r\t\345\375\177\000\000\020\v\223\002\000\000\000\000\220\r\t\345\375\177\000\000\240\v\223\002\000\000\000\000`\r\t\345\375\177\000\000\360\t\223\002\000\000\000\000\060\r\t\345\375\177\000\000\200\n\223\002\000\000\000\000\000\r\t\345\375\177\000\000\000\000\000\000\000\000\000\000D\366\221vZ\177\000\000@\310\222\002\000\000\000\000\300\243R\002\000\000\000\000\200\254:\001\000\000\000\000\231\001\000\000\000\000\000\000\360\020\t\345\375\177\000\000P\022\t\345\375\177\000\000 z\222\002\000\000\000\000\000\224\221\002\000\000\000\000@7\351EZ\177\000\000\222T\302vZ\177\000\000\300\223fsZ\177\000\000\000\351\063o\366\363[%\000\000\000\000\000\000\000\000"...
        message_allocated = 0x118ebd0 "The program 'ubiquity' received an X Window System error.\nThis probably reflects a bug in the program.\nThe error was 'BadAccess (attempt to access private resource denied)'.\n (Details: serial 14686 e"...
        format = 0x1 <error: Cannot access memory at address 0x1>
        message = <optimized out>
        p = <optimized out>
        n_fields = 6
        i = 6
        stack_fields = {{key = 0x7f5a76982af1 "MESSAGE", value = 0x118ebd0, length = -1}, {key = 0x7f5a76982b04 "PRIORITY", value = 0x7f5a7697cc9a, length = -1}, {key = 0x7f5a76982b5e "GLIB_DOMAIN", value = 0x7f5a7365304e <_gdk_wayland_display_utf8_to_string_target+270>, length = -1}, {key = 0x7f5a736532a1 <data_source_dnd_finished+81> "\261%", value = 0x7f5a73671828 <__func__.67903+8>, length = -1}, {key = 0x7f5a73653281 <data_source_dnd_finished+49> "1\300\350\250\f\371\377H\213\065a\261%", value = 0x7f5a736712af, length = -1}, {key = 0x7f5a736532b3 <data_source_dnd_finished+99> "\017\037D", value = 0x7f5a73671f70 <__func__.66747+16>, length = -1}, {key = 0x0, value = 0x7ffde5090eb0, length = 140026516315808}, {key = 0x7f5a76c11192 <g_object_new_valist+258> "H\205\300H\211\301\017\204\312\003", value = 0x0, length = 0}, {key = 0x0, value = 0x7ffde5090eb0, length = 140026516315808}, {key = 0x7f5a76c11192 <g_object_new_valist+258> "H\205\300H\211\301\017\204\312\003", value = 0x7ffde5090eb0, length = 43252528}, {key = 0x293fb30 "\200\222\221\002", value = 0x7ffde5091000, length = 0}, {key = 0x0, value = 0x7ffde5090d00, length = 43190912}, {key = 0x2919280 "\002", value = 0x602c55ce0, length = 0}, {key = 0x0, value = 0x0, length = 0}, {key = 0x0, value = 0x0, length = 0}, {key = 0x0, value = 0x2932c50, length = 140728446029296}}
        fields = 0x7ffde5090d40
        fields_allocated = 0x0
        array = <optimized out>

[Daniel van Vugt (vanvugt)]

Here's the crash with GDK_SYNCHRONIZE=1. It appears to be graphical. And I have a theory this maybe only happens on machines that default to hi-DPI (scale 200%) in the live session...

#6 0x00007fffef656c8d in _XError () from /usr/lib/x86_64-linux-gnu/libX11.so.6
#7 0x00007fffef653bb7 in ?? () from /usr/lib/x86_64-linux-gnu/libX11.so.6
#8 0x00007fffef653c75 in ?? () from /usr/lib/x86_64-linux-gnu/libX11.so.6
#9 0x00007fffef654b88 in _XReply () from /usr/lib/x86_64-linux-gnu/libX11.so.6
#10 0x00007fffef65057d in XSync () from /usr/lib/x86_64-linux-gnu/libX11.so.6
#11 0x00007fffef65061b in ?? () from /usr/lib/x86_64-linux-gnu/libX11.so.6
#12 0x00007fffef65760f in ?? () from /usr/lib/x86_64-linux-gnu/libX11.so.6
#13 0x00007fffef40d7b7 in XShmAttach ()
   from /usr/lib/x86_64-linux-gnu/libXext.so.6
#14 0x00007fffef18b82b in ?? () from /usr/lib/x86_64-linux-gnu/libcairo.so.2
#15 0x00007fffef18c05e in ?? () from /usr/lib/x86_64-linux-gnu/libcairo.so.2
#16 0x00007fffef189c81 in ?? () from /usr/lib/x86_64-linux-gnu/libcairo.so.2
#17 0x00007fffef1868e8 in ?? () from /usr/lib/x86_64-linux-gnu/libcairo.so.2
#18 0x00007fffef16a585 in ?? () from /usr/lib/x86_64-linux-gnu/libcairo.so.2
---Type <return> to continue, or q <return> to quit---
#19 0x00007fffef16a78c in ?? () from /usr/lib/x86_64-linux-gnu/libcairo.so.2
#20 0x00007fffef16ae28 in ?? () from /usr/lib/x86_64-linux-gnu/libcairo.so.2
#21 0x00007fffef10e021 in ?? () from /usr/lib/x86_64-linux-gnu/libcairo.so.2
#22 0x00007fffef157efc in ?? () from /usr/lib/x86_64-linux-gnu/libcairo.so.2
#23 0x00007fffef11657e in ?? () from /usr/lib/x86_64-linux-gnu/libcairo.so.2
#24 0x00007fffef10fb39 in ?? () from /usr/lib/x86_64-linux-gnu/libcairo.so.2
#25 0x00007fffef108a55 in cairo_fill ()
   from /usr/lib/x86_64-linux-gnu/libcairo.so.2
#26 0x00007fffc2e66240 in ?? ()
   from /usr/lib/x86_64-linux-gnu/libwebkit2gtk-4.0.so.37
#27 0x00007fffc1fd1e61 in ?? ()
   from /usr/lib/x86_64-linux-gnu/libwebkit2gtk-4.0.so.37
#28 0x00007fffc2051a3b in ?? ()
   from /usr/lib/x86_64-linux-gnu/libwebkit2gtk-4.0.so.37
#29 0x00007fffc1fdbbd1 in ?? ()
   from /usr/lib/x86_64-linux-gnu/libwebkit2gtk-4.0.so.37
#30 0x00007fffc1fe1897 in ?? ()
   from /usr/lib/x86_64-linux-gnu/libwebkit2gtk-4.0.so.37
#31 0x00007fffc1f4879f in ?? ()
   from /usr/lib/x86_64-linux-gnu/libwebkit2gtk-4.0.so.37
#32 0x00007fffc1ccabde in ?? ()
   from /usr/lib/x86_64-linux-gnu/libwebkit2gtk-4.0.so.37
#33 0x00007fffc1daeed2 in ?? ()
---Type <return> to continue, or q <return> to quit---
   from /usr/lib/x86_64-linux-gnu/libwebkit2gtk-4.0.so.37
#34 0x00007fffc1cc67ab in ?? ()
   from /usr/lib/x86_64-linux-gnu/libwebkit2gtk-4.0.so.37
#35 0x00007fffc1cc7095 in ?? ()
   from /usr/lib/x86_64-linux-gnu/libwebkit2gtk-4.0.so.37
#36 0x00007fffc1005903 in WTF::RunLoop::performWork() ()
   from /usr/lib/x86_64-linux-gnu/libjavascriptcoregtk-4.0.so.18
#37 0x00007fffc102e959 in ?? ()
   from /usr/lib/x86_64-linux-gnu/libjavascriptcoregtk-4.0.so.18
#38 0x00007ffff4b9e0f5 in g_main_context_dispatch ()
   from /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0
#39 0x00007ffff4b9e4c0 in ?? () from /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0
#40 0x00007ffff4b9e7d2 in g_main_loop_run ...

[Daniel van Vugt (vanvugt)]

WORKAROUND:

1. Boot into the live session.
2. Settings > Devices > Displays > Scale = 100%
3. Click Apply.
4. Proceed with installation: Click "Install Ubuntu 18.04 LTS".

[Daniel van Vugt (vanvugt)]

Stack trace with debug symbols (mostly).

[Daniel van Vugt (vanvugt)]

This looks odd. We start with a 16x16 icon upscaled to 32x32 (scale=200%) but the pixbuf_x/y offset seems to be negative (memory underrun?). Is that a faulty attempt at rendering 32x32 from 16x16 input?

#19 0x00007fffef158e83 in INT_cairo_surface_create_similar_image (other=<optimized out>, format=CAIRO_FORMAT_ARGB32, width=32, height=32) at ../../../../src/cairo-surface.c:595
#20 0x00007ffff18884bc in gdk_cairo_set_source_pixbuf (cr=cr@entry=0x1336800, pixbuf=pixbuf@entry=0x25fba40, pixbuf_x=-16, pixbuf_y=-16) at ../../../../gdk/gdkcairo.c:339
#21 0x00007fffec8bb784 in gtk_css_image_icon_theme_draw (image=<optimized out>, cr=0x1336800, width=<optimized out>, height=16)
    at ../../../../gtk/gtkcssimageicontheme.c:84
#22 0x00007fffec8b6db1 in _gtk_css_image_draw (image=0x11990c0, cr=0x1336800, width=16, height=16)
    at ../../../../gtk/gtkcssimage.c:239

[Daniel van Vugt (vanvugt)]

Ignore comment #16. Negative offsets appear to be valid: https://developer.gnome.org/gdk3/stable/gdk3-Cairo-Interaction.html#gdk-cairo-set-source-pixbuf

[Daniel van Vugt (vanvugt)]

Here's the offending source code up to the failing X call. I'm not so sure that the math is right (or how getting it wrong might trigger this bug)...

static cairo_xlib_shm_t *
_cairo_xlib_shm_pool_create(cairo_xlib_display_t *display,
                            size_t size, void **ptr)
{
    Display *dpy = display->display;
    cairo_xlib_shm_t *pool;
    size_t bytes, maxbits = 16, minbits = MIN_BITS;
    Status success;

    pool = malloc (sizeof (cairo_xlib_shm_t));
    if (pool == NULL)
        return NULL;

    bytes = 1 << maxbits;
    while (bytes <= size)
        bytes <<= 1, maxbits++;
    bytes <<= 3;

    minbits += (maxbits - 16) / 2;

    pool->shm.shmid = shmget (IPC_PRIVATE, bytes, IPC_CREAT | 0600);
    while (pool->shm.shmid == -1 && bytes >= 2*size) {
        bytes >>= 1;
        pool->shm.shmid = shmget (IPC_PRIVATE, bytes, IPC_CREAT | 0600);
    }
    if (pool->shm.shmid == -1)
        goto cleanup;

    pool->shm.readOnly = FALSE;
    pool->shm.shmaddr = shmat (pool->shm.shmid, NULL, 0);
    if (pool->shm.shmaddr == (char *) -1) {
        shmctl (pool->shm.shmid, IPC_RMID, NULL);
        goto cleanup;
    }

    pool->attached = XNextRequest (dpy);
    success = XShmAttach (dpy, &pool->shm);

Possibly worth noting is that size==4096 at scale 200%, and size==256 at scale 100%.

[Daniel van Vugt (vanvugt)]

Correction:
Possibly worth noting is that size==4096 at scale 200%, and size==1024 at scale 100%.

[Eric Desrochers (slashd)]

It has been brought to my attention by a community user, that the workaround (change scale from 200% to 100%) found in comment#14 works.

[Mario Limonciello (superm1)]

On an Dell XPS 9360 with a 3200x1800 display it also defaults to 200% and I reliably reproduce this bug with the 18.04 final image.

I can confirm setting the scaling to 100% in the live session resolved the crash.

[PJSingh5000 (pjsingh5000)]

Here is the backtrace (bt f) with debug symbols.

I still have gdb open, if anyone needs me to do anything else?

[Daniel van Vugt (vanvugt)]

According to this page, the problem possibly started in ubiquity version 18.04.11:
https://errors.ubuntu.com/problem/84a5563af3d2b85f098da832ece4cb8450bfd524

[Daniel van Vugt (vanvugt)]

This one also started in version 18.04.11:
https://errors.ubuntu.com/problem/735a2b847e0eeab6c8a7b954de5110e43889be15

But this one started in 18.04.3:
https://errors.ubuntu.com/problem/82f7f7e7923663c7b2123c7f1f49af29f6ff4d77

[Daniel van Vugt (vanvugt)]

Thanks @pjsingh5000. Your attachment shows the python stack ends at:

        # There's still work to do (postinstall). Let's keep the user
        # entertained.
        self.start_slideshow()
        Gtk.main()

Which agrees with what I see on screen when the bug occurs. It's the slideshow that never starts (or does, as a blank black/white image).

After that there is no debug info from comment #22. Although my attachment from comment #15 probably fills in the blanks: https://launchpadlibrarian.net/368099559/dbg.txt

So it looks like the bug here is entirely in the C code.

[Daniel van Vugt (vanvugt)]

The BadAccess from ShmAttach is coming from the Xorg process somewhere in here:

static int
ProcShmAttach(ClientPtr client)
{
    SHMSTAT_TYPE buf;
    ShmDescPtr shmdesc;

    REQUEST(xShmAttachReq);

    REQUEST_SIZE_MATCH(xShmAttachReq);
    LEGAL_NEW_RESOURCE(stuff->shmseg, client);
    if ((stuff->readOnly != xTrue) && (stuff->readOnly != xFalse)) {
        client->errorValue = stuff->readOnly;
        return BadValue;
    }
    for (shmdesc = Shmsegs; shmdesc; shmdesc = shmdesc->next) {
        if (!SHMDESC_IS_FD(shmdesc) && shmdesc->shmid == stuff->shmid)
            break;
    }
    if (shmdesc) {
        if (!stuff->readOnly && !shmdesc->writable)
            return BadAccess;
        shmdesc->refcnt++;
    }
    else {
        shmdesc = malloc(sizeof(ShmDescRec));
        if (!shmdesc)
            return BadAlloc;
#ifdef SHM_FD_PASSING
        shmdesc->is_fd = FALSE;
#endif
        shmdesc->addr = shmat(stuff->shmid, 0,
                              stuff->readOnly ? SHM_RDONLY : 0);
        if ((shmdesc->addr == ((char *) -1)) || SHMSTAT(stuff->shmid, &buf)) {
            free(shmdesc);
            return BadAccess;
        }

        /* The attach was performed with root privs. We must
         * do manual checking of access rights for the credentials
         * of the client */

        if (shm_access(client, &(SHM_PERM(buf)), stuff->readOnly) == -1) {
            shmdt(shmdesc->addr);
            free(shmdesc);
            return BadAccess;
        }

        shmdesc->shmid = stuff->shmid;
        shmdesc->refcnt = 1;
        shmdesc->writable = !stuff->readOnly;
        shmdesc->size = SHM_SEGSZ(buf);
        shmdesc->next = Shmsegs;
        Shmsegs = shmdesc;
    }
    if (!AddResource(stuff->shmseg, ShmSegType, (void *) shmdesc))
        return BadAlloc;
    return Success;
}

[Daniel van Vugt (vanvugt)]

Possibly relevant to permissions: The uid of this process changed from 'root' in 17.10 (where it works), to 'ubuntu' in 18.04 (where it doesn't work).

ubuntu 4468 17.8 1.2 502716 97352 tty1 Sl+ 08:17 0:01 /usr/bin/python3 /usr/lib/ubiquity/bin/ubiquity gtk_ui

In both cases the Xorg process is running as 'ubuntu'.

[Daniel van Vugt (vanvugt)]

Getting closer. It's this failing in the Xorg process:

        /* The attach was performed with root privs. We must
         * do manual checking of access rights for the credentials
         * of the client */

        if (shm_access(client, &(SHM_PERM(buf)), stuff->readOnly) == -1) {
            shmdt(shmdesc->addr);
            free(shmdesc);
            return BadAccess;
        }

[Daniel van Vugt (vanvugt)]

Unfortunately I can't tell what in shm_access is failing exactly because gdb skips straight from shm.c:314 back to the caller. So that suggests this code path, but I can't verify it:

static int
shm_access(ClientPtr client, SHMPERM_TYPE * perm, int readonly)
{
    int uid, gid;
    mode_t mask;
    int uidset = 0, gidset = 0;
    LocalClientCredRec *lcc;

    if (GetLocalClientCreds(client, &lcc) != -1) { // FALSE
    ...
    }
    /* Otherwise, check everyone else */
    mask = S_IROTH;
    if (!readonly) {
        mask |= S_IWOTH;
    }
    return (SHMPERM_MODE(perm) & mask) == mask ? 0 : -1;
}

So Xorg _should_ be returning an error, and it is. The problem seems to be that the client (python3 gtk_ui process) has a real uid=0 and effective uid=999. So that allows it to connect to Xorg (normally Xorg rejects root clients), but doesn't allow Xorg to check the client shm permissions. So Xorg rejects the client's attempt at XShmAttach.

How any of this relates to scale=200% and why it works at 100%, I'm not sure. We might need to review comment #15 in more detail.

[Daniel van Vugt (vanvugt)]

Given that XShmAttach can fail (you can google the reasons), I think maybe we need to handle that in ubiquity/cairo and fall back to an alternative rendering method.

I'm not so sure we should be treating XShmAttach failure as a bug.

[Daniel van Vugt (vanvugt)]

OK, I can now reproduce the problem at scale=100%. The trick is to slow down the installer with GDK_SYNCHRONIZE=1. So this is just a race condition, usually triggered by slower rendering of scale 200%, but can be triggered other ways.

[Daniel van Vugt (vanvugt)]

So it sounds like a race between Xorg and the ubiquity perms dropping which then breaks Xorg's ability to authenticate XShmAttach used in cairo.

[PJSingh5000 (pjsingh5000)]

Ubiquity also crashes inside a virtual machine (VirtualBox) running on a non-HiDPI host machine.

I have observed corrupted graphics in the "main area" of Ubiquity when this bug occurs. By "main area", I am referring to the rendered rectangular area in Ubiquity below the title bar and above the "Copying files..." progress bar.

The corruption looks like jagged diagonal lines or "snow". I have seen this on both actual hardware (HiDPI machine scaled at 200%) and in a virtual machine (non-HiDPI hardware scaled at 100%).

[Jerry Kao (jerry.kao)]

I can reproduce this issue on xps13 9350. crash report is attached.

[Daniel van Vugt (vanvugt)]

Please report crash files by running this command:

  ubuntu-bug YOURFILE.crash

Attaching them to bugs is not helpful to us, and a security risk to you.

[Daniel van Vugt (vanvugt)]

Created attachment 139297
Handle-XShmAttach-failures-v1.patch

I found myself amongst this code in the process of trying to fix a crash in the Ubuntu 18.04 graphical installer (https://launchpad.net/bugs/1751252).

Here's a patch to fix that, and hopefully this bug too.

[Daniel van Vugt (vanvugt)]

A patch to fix the crash is here:
https://bugs.freedesktop.org/show_bug.cgi?id=98883#c6

If we fail to get cairo fixed for any reason, it's probably possible to do a workaround/fix in ubiquity. But I think there are now multiple reasons to fix cairo as first preference.

[Andrea Azzarone (azzar1)]

Fix for ubiquity proposed here: https://code.launchpad.net/~azzar1/ubiquity/+git/ubiquity/+merge/345056

[PJSingh5000 (pjsingh5000)]

I tried the patch for /usr/lib/ubiquity/ubiquity/misc.py in a virtual machine.

Instead of corrupted graphics in the Ubiquity window, I saw a blank white area above the "Copying files..." progress bar.

After a few seconds, the Ubiquity slide show appeared as usual, and I did *not* get a crash.

I will also try this on actual hardware.

[Daniel van Vugt (vanvugt)]

You will get the same behaviour using the cairo fix too:
https://bugs.freedesktop.org/show_bug.cgi?id=98883#c6

[Andrey Arapov (andrey-arapov)]

The patch https://code.launchpad.net/~azzar1/ubiquity/+git/ubiquity/+merge/345056 did work for me on my mid-2017 MacBookPro (HW, not VM).

Thank you, Andrea!

[Łukasz Zemczak (sil2100)]

An upload of ubiquity to bionic-proposed has been rejected from the upload queue for the following reason: "All looks good with the package itself, but sadly there seems to be some leftover .gladep file included with the changes. Could you reupload without it? Or, if it's needed, just mention that in the bug? Thank you!".

[Iain Lane (laney)]

Ah, I'm sorry, I didn't see that there had been a bionic upload for this since the vcs was untouched and there was nothing in the unapproved queue... I just uploaded ubiquity for this to *cosmic* - not bionic yet. If it gets approved it should be in the next day's dailies and hopefully someone can test those to verify it's good.

[Iain Lane (laney)]

Removed the .gladep file; it seems to have crept in by error during the git migration (also in my cosmic upload) and uploaded bionic. Thanks and sorry cyphermox if this treads on your toes.

[Launchpad Janitor (janitor)]

This bug was fixed in the package ubiquity - 18.10.1

---------------
ubiquity (18.10.1) cosmic; urgency=medium

  [ Iain Lane ]
  * Bump sources to cosmic
  * Automatic update of included source packages: bterm-unifont 1.5,
    choose-mirror 2.78ubuntu4, preseed 1.71ubuntu8.

  [ Andrea Azzarone ]
  * misc.py: Restore the corrent euid in regain_privileges_save. Calling
    regain_privileges_save should restore the effective user-id to the one
    before the call to drop_privileges_save. We need to call os.setresuid and
    os.setresgid twice to avoid permission issues when calling os.setgroups.
    (LP: #1751252)

 -- Iain Lane <email address hidden> Tue, 08 May 2018 12:07:44 +0100

[Łukasz Zemczak (sil2100)]

Hello Jean-Baptiste, or anyone else affected,

Accepted ubiquity into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ubiquity/18.04.14.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Dmitrii Shcherbakov (dmitriis)]

I can confirm that the package version from bionic-proposed solves the problem.

During an installation I used Dell 2715Q with settings as in the screenshot attached (single monitor mode, 200% scaling).

# the updated package
apt policy ubiquity
ubiquity:
  Installed: 18.04.14.1
  Candidate: 18.04.14.1
  Version table:
 *** 18.04.14.1 500
        500 http://archive.ubuntu.com/ubuntu bionic-proposed/main amd64 Packages
        100 /var/lib/dpkg/status
     18.04.14 500
        500 http://archive.ubuntu.com/ubuntu bionic/main amd64 Packages

Before adding "deb http://archive.ubuntu.com/ubuntu/ bionic-proposed main restricted" to sources.list and doing `apt install ubiquity` I had this in syslog:

ay 13 12:39:12 ubuntu ubiquity: W: --force-yes is deprecated, use one of the options starting with --allow instead.
May 13 12:39:12 ubuntu ubiquity[4617]: debconffilter_done: ubi-usersetup (current: ubi-usersetup)
May 13 12:39:12 ubuntu ubiquity[4617]: Step_before = stepUserInfo
May 13 12:39:12 ubuntu kernel: [ 249.861407] do_trap: 40 callbacks suppressed
May 13 12:39:12 ubuntu kernel: [ 249.861410] traps: ubiquity[4617] trap int3 ip:7fc1a36cdc41 sp:7ffe6a90a760 error:0 in libglib-2.0.so.0.5600.1[7fc1a367c000+113000]
May 13 12:39:21 ubuntu update-notifier[4223]: GtkDialog mapped without a transient parent. This is discouraged.
May 13 12:39:22 ubuntu /install.py: Exception during installation:
May 13 12:39:22 ubuntu /install.py: Traceback (most recent call last):
May 13 12:39:22 ubuntu /install.py: File "/usr/share/ubiquity/install.py", line 765, in <module>
May 13 12:39:22 ubuntu /install.py: install.run()
May 13 12:39:22 ubuntu /install.py: File "/usr/share/ubiquity/install.py", line 135, in run
May 13 12:39:22 ubuntu /install.py: self.copy_all()
May 13 12:39:22 ubuntu /install.py: File "/usr/share/ubiquity/install.py", line 513, in copy_all
May 13 12:39:22 ubuntu /install.py: self.db.progress('SET', 10 + copy_progress)
May 13 12:39:22 ubuntu /install.py: File "/usr/lib/python3/dist-packages/debconf.py", line 83, in <lambda>
May 13 12:39:22 ubuntu /install.py: lambda *args, **kw: self.command(command, *args, **kw))
May 13 12:39:22 ubuntu /install.py: File "/usr/lib/python3/dist-packages/debconf.py", line 104, in command
May 13 12:39:22 ubuntu /install.py: status = int(status)
May 13 12:39:22 ubuntu /install.py: ValueError: invalid literal for int() with base 10: ''

After the update a newly started installation went just fine.

[Samuel Bancal (samuel-bancal)]

Hi,

I do confirm proposed worked also for me on a Dell XPS 15 9530.

[Launchpad Janitor (janitor)]

This bug was fixed in the package ubiquity - 18.04.14.1

---------------
ubiquity (18.04.14.1) bionic; urgency=medium

  [ Iain Lane ]
  * Update Vcs-* for git migration

  [ Andrea Azzarone ]
  * misc.py: Restore the corrent euid in regain_privileges_save. Calling
    regain_privileges_save should restore the effective user-id to the one
    before the call to drop_privileges_save. We need to call os.setresuid and
    os.setresgid twice to avoid permission issues when calling os.setgroups.
    (LP: #1751252)

 -- Iain Lane <email address hidden> Tue, 08 May 2018 15:22:49 +0100

[Łukasz Zemczak (sil2100)]

The verification of the Stable Release Update for ubiquity has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[gene_wood (gene.wood)]

For anyone looking for how to use this fix with the current Ubuntu 18.04 installation ISO, this page ( https://askubuntu.com/a/1040322/14601 ) has a good description how to use the patch produced above in Comment 37.

[Sebastien Bacher (seb128)]

Could a maintainer review the patch there?

[Psychon-d (psychon-d)]

@Sebastian: I don't know who you count as a maintainer, but I'll take a look...

So... this bug is about _cairo_xlib_shm_pool_create() getting the sequence number for pool->attached wrong, resulting in cairo possibly destroying the shared memory segment to early. The discussion concluded that setting pool->attached = XNextRequest(dpy)-1 after the request is the right fix.
In comment #3 I wondered about XLockDisplay(), but that seems unrelated. I was possibly confused since there are two places calling XShmAttach() and one of them locks the display while the other one does not.

The patch in comment #6 says it is about https://bugs.launchpad.net/ubuntu/+source/ubiquity/+bug/1751252 instead of this bug. https://bugs.launchpad.net/ubuntu/+source/ubiquity/+bug/1751252 must be a different bug since it also happens with GDK_SYNCHRONIZE=1, which means that the race that this bug is about cannot happen.

What the patch does is to make XShmAttach synchronous (aka "slow"). I do not know why, but nothing in this bug report suggests that this is necessary. The description of the patch does not say anything about this either. XShmAttach() should not just randomly fail, so why does the patch try to handle random failures?

[Spitzak-k (spitzak-k)]

The XLockDisplay is needed so that an error in another thread does not get reported as a failure of XShmAttach.

[Bugs-freedesktop (bugs-freedesktop)]

XSetErrorHandler() is not appropriate for use with multi-thread code.

[Daniel van Vugt (vanvugt)]

It's been a few months so my memory is not completely clear. I do however think Uli's statement is incorrect in the final two sentences:

> What the patch does is to make XShmAttach synchronous (aka "slow"). I do not
> know why, but nothing in this bug report suggests that this is necessary.
> The description of the patch does not say anything about this either.
> XShmAttach() should not just randomly fail, so why does the patch try to
> handle random failures?

attachment 139297 mentions multiple times "why" it is necessary to make it synchronous. Because it's impossible to detect errors in XShmAttach otherwise.

It's possible we're using to the wrong bug here, but I believe the patch is still correct and we have no choice but to make XShmAttach slow. You can only choose between maximum performance or reliable error handling, and not have both. The reasons are detailed in attachment 139297.

[Daniel van Vugt (vanvugt)]

So I think reliability should be the first priority.

Also, I don't *think* the affected function would usually be used in any performance-sensitive location so the slowdown required to catch errors is probably never noticeable. And it's less undesirable than the crashes we get from not catching them at all.

[Bugs-freedesktop (bugs-freedesktop)]

(In reply to Daniel van Vugt from comment #11)
> The reasons are detailed in attachment 139297.

I don't see an explanation why XShmAttach is failing.

[Daniel van Vugt (vanvugt)]

The patch says:

"XShmAttach returns True, even on error."

and

"XShmAttach is hard-coded in libXext to return True, even on failure."

You can dig through the related source code to verify that.

[Bugs-freedesktop (bugs-freedesktop)]

(In reply to Daniel van Vugt from comment #11)
> attachment 139297 [details] [review] mentions multiple times "why" it is
> necessary to make it synchronous. Because it's impossible to detect errors
> in XShmAttach otherwise.

It is not impossible to detect errors without XSync.

Display::async_handlers can be used. The result would be async and
the patch comment claims that _cairo_xlib_shm_pool_create must return
a reliable status synchronously. I haven't check that claim, but if so, I
wonder whether it would be possible to take a fallback path until the pool is
successfully created.

[Daniel van Vugt (vanvugt)]

Great. I would be happy to see alternative solutions but also don't need to be involved at all...

In Ubuntu we are not dependent on this patch because we have instead modified other components to indirectly avoid triggering the failure in Cairo. Effectively we have a permanent workaround already.

[Bugs-freedesktop (bugs-freedesktop)]

(In reply to Daniel van Vugt from comment #14)
> The patch says:
>
> "XShmAttach returns True, even on error."
>
> and
>
> "XShmAttach is hard-coded in libXext to return True, even on failure."

The question was what error or failure, and why is that happening.

XShmAttach returns True to indicate it successfully queued the request.
It does not fail to queue the request.

If the server fails to process the request successfully, then it sends an
error response.

It would be helpful to understand why you are expecting an error response.

The best clue appears to be the patch at the end of
https://code.launchpad.net/~azzar1/ubiquity/+git/ubiquity/+merge/345056

[Daniel van Vugt (vanvugt)]

> XShmAttach returns True to indicate it successfully queued the request.
> It does not fail to queue the request.
>
> If the server fails to process the request successfully, then it sends an
> error response.
>
> It would be helpful to understand why you are expecting an error response.

Essentially we have the Xorg server process having its privileges demoted and re-promoted. If the XShmAttach request comes in while it is demoted then it can't complete the request and returns BadAccess to the client. The client initiated the request from _cairo_xlib_shm_pool_create but has since (asynchronously) moved on to other code. So it crashes in an unpredictable location when the BadAccess eventually comes in from the server.

I am happy with your explanation that "XShmAttach returns True to indicate it successfully queued the request". However I don't believe the same applies to _cairo_xlib_shm_pool_create, so I think any final fix would have to happen in that function still.

[Gitlab-migration (gitlab-migration)]

-- GitLab Migration Automatic Message --

This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity.

You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/cairo/cairo/issues/49.

[Sebastien Bacher (seb128)]

Untargetting from cairo/bionic, the ubiquity fix is enough there

[Seyed Mustafa Afzouni (afzouni)]

In the Live Version on MacBook Pro 2015 (13 inches, Retina) I changed the 'Screen Scale' from 200% to 100%, settings > Device > Display, and solved!

[Amsalu Abegaz (zelekea19)]

an able to upgrade from 17.04 to 18.04

[Daniel van Vugt (vanvugt)]

I think the fix is in Ubuntu 18.04.1 (not 18.04) so please use 18.04.1:

http://releases.ubuntu.com/18.04/