qemu-system-x86 crashes when VNC connection is established

[halfdog]

[Impact]

 * some more uncommon vnc configurations (e.g. very long names, but also
   potentially various other cases that make
   vnc_init_basic_info_from_server_addr fail) will lead to random data
   (after alloc) in a struct that will then be used on calls (e.g to free)

 * The fix would avoid hard crashes (due to freeing random or null
   pointers) in qemu of xenial

[Test Case]

 * To trigger the issue you can use e.g. a very long vnc string.
 Console 1
   $ mkdir /tmp/service
   $ qemu-system-x86_64 -enable-kvm -vnc unix:/tmp/service/../service/../service/../service/vnc-sock
 Console 2
   $ socat - UNIX:/tmp/service/vnc-sock

[Regression Potential]

 * I'd consider the regression potential very low for the following
   reasons:
   - small change (easier to review)
   - changing alloc to zeroing alloc (to avoid random data in struct)
   - the change is from upstream and quite old without being reverted or
     post-fixed
 * What could happen?
   Overall due to the change now just initializing memory the only
   regression I could think of would be something that required !=0
   content and worked all the time by accident (since random has so many
   changes to be !=0 but only one to be =0, but TBH I can't think
   of such an issue in that area of the code

[Other Info]

 * pre testable in ppa https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/3245

Following minimal test case crashes qemu-system-i386 on amd64 host:

qemu-system-i386 -name test -nodefconfig -no-user-config -nodefaults -sandbox off -machine none -m 256 -balloon none -no-acpi -parallel none -vga virtio -display "vnc=unix:vnc.socket" -boot menu=on

and open the connection (not even real VNC client needed):

socat - UNIX:vnc.socket

Result:

*** Error in `qemu-system-i386': free(): invalid pointer: 0x00007fbad024eb78 ***
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x777e5)[0x7fbacff017e5]
/lib/x86_64-linux-gnu/libc.so.6(+0x8037a)[0x7fbacff0a37a]
/lib/x86_64-linux-gnu/libc.so.6(cfree+0x4c)[0x7fbacff0e53c]
qemu-system-i386(+0x4a630d)[0x56145bd6930d]
qemu-system-i386(visit_type_VncServerInfo+0xa2)[0x56145bd7b342]
qemu-system-i386(qapi_free_VncServerInfo+0x30)[0x56145bd68910]
qemu-system-i386(+0x4358fa)[0x56145bcf88fa]
qemu-system-i386(+0x43aa03)[0x56145bcfda03]
qemu-system-i386(+0x43abe5)[0x56145bcfdbe5]
qemu-system-i386(aio_dispatch+0x68)[0x56145bd1f9e8]
qemu-system-i386(+0x44fcce)[0x56145bd12cce]
/lib/x86_64-linux-gnu/libglib-2.0.so.0(g_main_context_dispatch+0x2a7)[0x7fbad0be2197]
...

$ lsb_release -rd
Description: Ubuntu 16.04.2 LTS
Release: 16.04

$ apt-cache policy qemu-system-x86
qemu-system-x86:
  Installed: 1:2.5+dfsg-5ubuntu10.14
  Candidate: 1:2.5+dfsg-5ubuntu10.14
  Version table:
 *** 1:2.5+dfsg-5ubuntu10.14 500
        500 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages
        500 http://archive.ubuntu.com/ubuntu xenial-security/main amd64 Packages
        100 /var/lib/dpkg/status
     1:2.5+dfsg-5ubuntu10 500
        500 http://archive.ubuntu.com/ubuntu xenial/main amd64 Packages

[Joshua Powers (powersj)]

Hi, thanks for taking the time to report the issue!

Can you verify that you have a fully updated system? Also, is this running on bare-metal, in a container, or in a VM?

I tried reproducing in a lxc container, a uvt image, and on baremetal. I launched the qemu cli you have, then in another terminal ran the socat command, however I saw no crashes.

[Launchpad Janitor (janitor)]

[Expired for qemu (Ubuntu) because there has been no activity for 60 days.]

[Sergeev Alexander (sergeev917)]

I just ran into the same bug (Ubuntu 16.04.4 LTS, fully updated at time of the comment).

I've played with cmd arguments a little and it seems that the bug is triggered at some length of the vnc socket path.

This is running okay:
$ qemu-system-x86_64 -enable-kvm -vnc unix:./service/vnc-sock

Also okay:
$ qemu-system-x86_64 -enable-kvm -vnc unix:../test-vm/service/vnc-sock

But this fails:
$ qemu-system-x86_64 -enable-kvm -vnc unix:../test-vm/../test-vm/service/vnc-sock

*** Error in `qemu-system-x86_64': free(): invalid pointer: 0x00007f1cfc939b78 ***
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x777e5)[0x7f1cfc5ec7e5]
/lib/x86_64-linux-gnu/libc.so.6(+0x8037a)[0x7f1cfc5f537a]
/lib/x86_64-linux-gnu/libc.so.6(cfree+0x4c)[0x7f1cfc5f953c]
qemu-system-x86_64(+0x4b228d)[0x55fcafa3a28d]
qemu-system-x86_64(visit_type_VncServerInfo+0xa2)[0x55fcafa4c2c2]
qemu-system-x86_64(qapi_free_VncServerInfo+0x30)[0x55fcafa39890]
qemu-system-x86_64(+0x44184a)[0x55fcaf9c984a]
qemu-system-x86_64(+0x446953)[0x55fcaf9ce953]
qemu-system-x86_64(+0x446b35)[0x55fcaf9ceb35]
qemu-system-x86_64(aio_dispatch+0x68)[0x55fcaf9f0988]
qemu-system-x86_64(+0x45bc1e)[0x55fcaf9e3c1e]
/lib/x86_64-linux-gnu/libglib-2.0.so.0(g_main_context_dispatch+0x2a7)[0x7f1cfd2cd197]
qemu-system-x86_64(main_loop_wait+0x18b)[0x55fcaf9ef28b]
qemu-system-x86_64(main+0x17b4)[0x55fcaf769714]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x7f1cfc595830]
qemu-system-x86_64(_start+0x29)[0x55fcaf76eda9]

Versions:
ii ipxe-qemu 1.0.0+git-20150424.a25a16d-1ubuntu1.2 all PXE boot firmware - ROM images for qemu
ii qemu-block-extra:amd64 1:2.5+dfsg-5ubuntu10.25 amd64 extra block backend modules for qemu-system and qemu-utils
ii qemu-kvm 1:2.5+dfsg-5ubuntu10.25 amd64 QEMU Full virtualization
ii qemu-system-common 1:2.5+dfsg-5ubuntu10.25 amd64 QEMU full system emulation binaries (common files)
ii qemu-system-x86 1:2.5+dfsg-5ubuntu10.25 amd64 QEMU full system emulation binaries (x86)
ii qemu-utils 1:2.5+dfsg-5ubuntu10.25 amd64 QEMU utilities

[Christian Ehrhardt  (paelzer)]

Thanks Sergeev, that is an interesting insight.

$ mkdir /tmp/service
# works
$ qemu-system-x86_64 -enable-kvm -vnc unix:/tmp/service/vnc-sock
# fails
$ qemu-system-x86_64 -enable-kvm -vnc unix:/tmp/service/../service/../service/../service/vnc-sock

And while these are running in another shell run:
$ socat - UNIX:/tmp/service/vnc-sock

Checked a few versions:
Qemu 2.10.1 works
Qemu 2.8.0 fails.
Qemu 2.5.0 fails.

A few potential fixes that come to mind, but unfortunately in Summer 2016 there were quite some rewrites int the area. So for the regression risk we should really only identify those needed and carefully review that (instead of picking more fixes along).

Iused gdb on a crashing qemu and found it really is exactly what this fixes:
  3e7f136d vnc: fix crash when vnc_server_info_get has an error

This change is nice and clean and should address the issue.
I build a test ppa at [1] which we can check later if it fixes the issue.

[1]: https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/3245

[Christian Ehrhardt  (paelzer)]

Ok, I can confirm the ppa fixing this case.
And OTOH it is fixed in qemu >=2.7.

Furthermore the change is very small and easily reviewable (essentially only changing a malloc to a malloc0 to initialize properly).

I'm marking the tasks accordingly and prep this as an SRU.

[Christian Ehrhardt  (paelzer)]

Note just passed regression test run from the ppa fine as expected.

SRU template prepared.

Making it available to the SRU team for evaluation now.
Uploaded as qemu_2.5+dfsg-5ubuntu10.26_source.changes

[Robie Basak (racb)]

Hello halfdog, or anyone else affected,

Accepted qemu into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/qemu/1:2.5+dfsg-5ubuntu10.26 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Christian Ehrhardt  (paelzer)]

Using the repro steps above:

1. as-is in Xenial:

ii qemu-system-x86 1:2.5+dfsg-5ubuntu10.25 amd64 QEMU full system emulation binaries (x86)

$ qemu-system-x86_64 -enable-kvm -vnc unix:/tmp/service/../service/../service/../service/vnc-sock
# due to socat in other shell:
*** Error in `qemu-system-x86_64': free(): invalid pointer: 0x00007f1e3c2bbbd8 ***

Once again, from proposed
qemu-block-extra/xenial-proposed 1:2.5+dfsg-5ubuntu10.26 amd64 [upgradable from: 1:2.5+dfsg-5ubuntu10.25]
qemu-kvm/xenial-proposed 1:2.5+dfsg-5ubuntu10.26 amd64 [upgradable from: 1:2.5+dfsg-5ubuntu10.25]
qemu-system-common/xenial-proposed 1:2.5+dfsg-5ubuntu10.26 amd64 [upgradable from: 1:2.5+dfsg-5ubuntu10.25]
qemu-system-x86/xenial-proposed 1:2.5+dfsg-5ubuntu10.26 amd64 [upgradable from: 1:2.5+dfsg-5ubuntu10.25]
qemu-utils/xenial-proposed 1:2.5+dfsg-5ubuntu10.26 amd64 [upgradable from: 1:2.5+dfsg-5ubuntu10.25]

$ qemu-system-x86_64 -enable-kvm -vnc unix:/tmp/service/../service/../service/../service/vnc-sock
# Staying alive on socat and the socat gets the Ref as it should:
$ socat - UNIX:/tmp/service/vnc-sock
RFB 003.008

Per the above, verified.

[Brian Murray (brian-murray)]

The verification of the Stable Release Update for qemu has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package qemu - 1:2.5+dfsg-5ubuntu10.26

---------------
qemu (1:2.5+dfsg-5ubuntu10.26) xenial; urgency=medium

  * d/p/ubuntu/lp-1705743-fix-vnc-crash.patch: fix crash when using long or
    invalid vnc connection setups (LP: #1705743)

 -- Christian Ehrhardt <email address hidden> Mon, 23 Apr 2018 10:18:51 +0200