https websockets not working in 2.5 or 2.6

[Ben Aitchison]

% gdb --args ./x86_64-softmmu/qemu-system-x86_64 -vnc 0.0.0.0:1,tls,x509=/etc/pki/libvirt-le,websocket=5701

GNU gdb (GDB) 7.11
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-pc-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./x86_64-softmmu/qemu-system-x86_64...done.
(gdb) run
Starting program: /home/ben/qemu/qemu-2.6.0/x86_64-softmmu/qemu-system-x86_64 -vnc 0.0.0.0:1,tls,x509=/etc/pki/libvirt-le,websocket=5701
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib/libthread_db.so.1".
[New Thread 0x7fffe16f6700 (LWP 12767)]
[New Thread 0x7fffde2d4700 (LWP 12768)]
[New Thread 0x7fffd3fff700 (LWP 12769)]
Initializing VNC server with x509 no auth
Client sioc=0x55555874d6b0 ws=1 auth=1 subauth=0
New client on socket 0x55555874d6b0
vnc_set_share_mode/0x55555874d6b0: undefined -> connecting
TLS Websocket connection required
Start TLS WS handshake process
Handshake failed TLS handshake failed: The TLS connection was non-properly terminated.
Closing down client sock: protocol error
vnc_set_share_mode/0x55555779f510: connecting -> disconnected
Client sioc=0x55555873c6a0 ws=1 auth=1 subauth=0
New client on socket 0x55555873c6a0
vnc_set_share_mode/0x55555873c6a0: undefined -> connecting
TLS Websocket connection required
Start TLS WS handshake process
TLS handshake complete, starting websocket handshake
Websocket negotiate starting
Websock handshake complete, starting VNC protocol
Write Plain: Pending output 0x555557b91c60 size 4096 offset 12. Wait SSF 0
Wrote wire 0x555557b91c60 12 -> 12

Thread 1 "qemu-system-x86" received signal SIGSEGV, Segmentation fault.
0x0000000000000001 in ?? ()
(gdb) thread apply all bt

Thread 4 (Thread 0x7fffd3fff700 (LWP 12769)):
#0 0x00007fffef35a09f in pthread_cond_wait@@GLIBC_2.3.2 () from /usr/lib/libpthread.so.0
#1 0x0000555555a20bd9 in qemu_cond_wait (cond=cond@entry=0x5555587267e0,
    mutex=mutex@entry=0x555558726810) at util/qemu-thread-posix.c:123
#2 0x00005555559770ab in vnc_worker_thread_loop (queue=queue@entry=0x5555587267e0)
    at ui/vnc-jobs.c:228
#3 0x00005555559775e8 in vnc_worker_thread (arg=0x5555587267e0) at ui/vnc-jobs.c:335
#4 0x00007fffef354474 in start_thread () from /usr/lib/libpthread.so.0
#5 0x00007fffea43c69d in clone () from /usr/lib/libc.so.6

Thread 3 (Thread 0x7fffde2d4700 (LWP 12768)):
#0 0x00007fffef35a09f in pthread_cond_wait@@GLIBC_2.3.2 () from /usr/lib/libpthread.so.0
#1 0x0000555555a20bd9 in qemu_cond_wait (cond=<optimized out>,
---Type <return> to continue, or q <return> to quit---
    emu_global_mutex>) at util/qemu-thread-posix.c:123
#2 0x0000555555715edf in qemu_tcg_wait_io_event (cpu=0x5555564ee840) at /home/ben/qemu/qemu-2.6.0/cpus.c:1015
#3 qemu_tcg_cpu_thread_fn (arg=<optimized out>) at /home/ben/qemu/qemu-2.6.0/cpus.c:1161
#4 0x00007fffef354474 in start_thread () from /usr/lib/libpthread.so.0
#5 0x00007fffea43c69d in clone () from /usr/lib/libc.so.6

Thread 2 (Thread 0x7fffe16f6700 (LWP 12767)):
#0 0x00007fffea438229 in syscall () from /usr/lib/libc.so.6
#1 0x0000555555a20ee8 in futex_wait (val=<optimized out>, ev=<optimized out>) at util/qemu-thread-posix.c:292
#2 qemu_event_wait (ev=ev@entry=0x55555641ece4 <rcu_call_ready_event>) at util/qemu-thread-posix.c:399
#3 0x0000555555a2f2ae in call_rcu_thread (opaque=<optimized out>) at util/rcu.c:250
#4 0x00007fffef354474 in start_thread () from /usr/lib/libpthread.so.0
#5 0x00007fffea43c69d in clone () from /usr/lib/libc.so.6

Thread 1 (Thread 0x7ffff7f5bb00 (LWP 12763)):
#0 0x0000000000000001 in ?? ()
#1 0x00005555559efb53 in qio_task_free (task=0x555558212140) at io/task.c:58
#2 0x00005555559efd89 in qio_task_complete (task=task@entry=0x555558212140) at io/task.c:145
#3 0x00005555559ef5aa in qio_channel_websock_handshake_send (ioc=0x55555873c6a0, condition=<optimized out>,
    user_data=0x555558212140) at io/channel-websock.c:289
#4 0x00007fffecf59c8a in g_main_context_dispatch () from /usr/lib/libglib-2.0.so.0
#5 0x000055555598a6b3 in glib_pollfds_poll () at main-loop.c:213
#6 os_host_main_loop_wait (timeout=<optimized out>) at main-loop.c:258
#7 main_loop_wait (nonblocking=<optimized out>) at main-loop.c:506
#8 0x00005555556e1fbd in main_loop () at vl.c:1934
#9 main (argc=<optimized out>, argv=<optimized out>, envp=<optimized out>) at vl.c:4656

[Daniel Berrange (berrange)]

The crash in 2.6 is fixed by

  https://lists.gnu.org/archive/html/qemu-devel/2016-06/msg01885.html

The dropped connection in 2.5 is fixed by

  https://lists.gnu.org/archive/html/qemu-devel/2016-06/msg01884.html

[Thomas Huth (th-huth)]

Fix has been included in QEMU v2.7.0:
http://git.qemu.org/?p=qemu.git;a=commitdiff;h=bc35d51077b33e68a0

[Daniel Berrange (berrange)]

Fixed in stable-2.6 branch in

commit 510531ea442a02048b1837fcf574d03559b38c9e
Author: Daniel P. Berrange <email address hidden>
Date: Tue Jun 7 12:27:51 2016 +0100

    io: remove mistaken call to object_ref on QTask

    The QTask struct is just a standalone struct, not a QOM Object,
    so calling object_ref() on it is not appropriate. This results
    in mangling the 'destroy' field in the QTask struct, causing
    the later call to qtask_free() to try to call the function
    at address 0x1, with predictably segfault happy results.

    There is in fact no need for ref counting with QTask, as the
    call to qtask_abort() or qtask_complete() will automatically
    free associated memory.

    This fixes the crash shown in

      https://bugs.launchpad.net/qemu/+bug/1589923

    Reviewed-by: Eric Blake <email address hidden>
    Signed-off-by: Daniel P. Berrange <email address hidden>
    (cherry picked from commit bc35d51077b33e68a0ab10a057f352747214223f)
    Signed-off-by: Michael Roth <email address hidden>

[Christian Ehrhardt  (paelzer)]

Fixed in >=2.7 and thereby >=Zesty.
Needs to be considered for SRUs now.

[Christian Ehrhardt  (paelzer)]

I added this to my list a while ago but now closed out the immediate tasks for artful, so I'm coming back here.
Writing repro steps which will be needed for the SRU.

# Brute force create dummy keys for this
openssl req -out ca.pem -new -x509
openssl genrsa -out server.key 1024
openssl req -key server.key -new -out server.req
echo 00 > file.srl
openssl x509 -req -in server.req -CA ca.pem -CAkey privkey.pem -CAserial file.srl -out server-cert.pem
openssl genrsa -out client.key 1024
openssl req -key client.key -new -out client.req
openssl x509 -req -in client.req -CA ca.pem -CAkey privkey.pem -CAserial file.srl -out client-cert.pem
ln -s ca.pem ca-cert.pem
ln -s server.key server-key.pem

# run qemu with x509 websocket
/usr/bin/qemu-system-x86_64 -vnc 0.0.0.0:1,tls,x509=$(pwd),websocket=5707

It is not crashing and I can even connect with krdc (likely also other clients) against it without breaking.

But then I'd think your command above for the crash was the one for the crash in 2.6 which was yakkety.

But what is left to fix is the dropped connection [1] in 2.5 (Xenial).

I tried to read a better testcase out of that mail thread but failed, if you'd have a better setup description to still trigger this blocked handshake that eventually fails once the signal sets data - that would be great.
The actual report I found [2] is actually this bug bridges onto the ML so no more info there for me.

[1]: https://lists.gnu.org/archive/html/qemu-devel/2016-06/msg01884.html
[2]: https://lists.gnu.org/archive/html/qemu-devel/2016-06/msg01917.html