VNC heap corruption at 1400x1050 (with % 16 != 0)
is a multiple of 16. If it's not, then it accesses beyond the end of the row
by a few bytes. On all but the last row, this is mostly harmless (it can
result in unnecessarily marking the end of the row dirty), but on the last row,
it copies over heap metadata. This triggers a crash when changing resolutions or disconnecting and reconnecting a client.
I can trigger this reliably with a Windows 7 guest at 1400x1050 with -vga std.
The attached patch (rather ugly, with debugging code for good measure) partially fixes the issue. There's still a black stripe on the right side of the screen, presumably because there are other bugs in vnc.c (or I messed up the patch).
I'm marking this as a security vulnerability because it allows the guest to overwrite host memory.
The same issue is tracked in Red Hat's bugzilla here: