[vmware] mir 0.26 failing with nested server in protobuf when bringing up NestedDisplay [CHECK failed: (index) >= (0)]

[Gerry Boland]

Setup:
1. VMWare 12, with the following line added to ~/.vmware/preferences:
      mks.gl.allowBlacklistedDrivers = "TRUE"
2. Created a VM with Xenial + Overlay PPA
3. Installed unity8-desktop-session-mir, rebooted
4. Tried to log in to a Unity8 session

Problem: Unity8 fails to start, but is exiting cleanly as opposed to crashing. The following is all that is printed of the log:

[2017-02-02:07:18:21.137] qtmir.screens: ScreensModel::ScreensModel
[2017-02-02 07:18:21.143261] mirplatform: Found graphics driver: mir:mesa-kms (version 0.26.0)
[2017-02-02 07:18:21.143413] mirplatform: Found graphics driver: mir:mesa-x11 (version 0.26.0)
[2017-02-02 07:18:21.144063] mirserver: Starting
[libprotobuf FATAL /usr/include/google/protobuf/repeated_field.h:885] CHECK failed: (index) >= (0):
[2017-02-02:07:18:21.165] CHECK failed: (index) >= (0):
initctl: No such variable: UNITY_MIR_SOCKET

USC is coming up correctly however.

If I try to launch miral-shell manually, I get similar error:

MIR_SOCKET=$XDG_RUNTIME_DIR/mir_socket MIR_SERVER_PROMPT_FILE=1 MIR_SERVER_HOST_SOCKET=/run/mir_socket MIR_SERVER_FILE=$XDG_RUNTIME_DIR/mir_socket MIR_SERVER_NAME=session-0 miral-shell
[2017-02-02 07:55:56.686669] mirplatform: Found graphics driver: mir:mesa-kms (version 0.26.0)
[2017-02-02 07:55:56.687395] mirplatform: Found graphics driver: mir:mesa-x11 (version 0.26.0)
[2017-02-02 07:55:56.689036] mirserver: Starting
[libprotobuf FATAL /usr/include/google/protobuf/repeated_field.h:885] CHECK failed: (index) >= (0):
ERROR: Dynamic exception type: google::protobuf::FatalException
std::exception::what: CHECK failed: (index) >= (0):

I tried to grab a backtrace at the location of the last call to /usr/include/google/protobuf/repeated_field.h:885 which might print the error:

http://pastebin.ubuntu.com/23911595/

Thread 4 "MirServerThread" hit Breakpoint 1, mir::protobuf::DisplayOutput::SerializeWithCachedSizes(google::protobuf::io::CodedOutputStream*) const (this=0x7fffd804f7a0, output=0x7fffe0848280) at /build/mir-Hd65rv/mir-0.26.0+16.04.20170126.3/obj-x86_64-linux-gnu/src/protobuf/mir_protobuf.pb.cc:8417
8417 in /build/mir-Hd65rv/mir-0.26.0+16.04.20170126.3/obj-x86_64-linux-gnu/src/protobuf/mir_protobuf.pb.cc
#0 mir::protobuf::DisplayOutput::SerializeWithCachedSizes(google::protobuf::io::CodedOutputStream*) const (this=0x7fffd804f7a0, output=0x7fffe0848280) at /build/mir-Hd65rv/mir-0.26.0+16.04.20170126.3/obj-x86_64-linux-gnu/src/protobuf/mir_protobuf.pb.cc:8417
#1 0x00007fffeba5c3e8 in mir::protobuf::DisplayConfiguration::SerializeWithCachedSizes(google::protobuf::io::CodedOutputStream*) const (this=0x7fffd804b830, output=0x7fffe0848280) at /build/mir-Hd65rv/mir-0.26.0+16.04.20170126.3/obj-x86_64-linux-gnu/src/protobuf/mir_protobuf.pb.cc:11436
#2 0x00007fffeb38a615 in google::protobuf::MessageLite::SerializeWithCachedSizesToArray(unsigned char*) const () from /usr/lib/x86_64-linux-gnu/libprotobuf-lite.so.9
#3 0x00007fffed696798 in mir_blob_from_display_config () from /usr/lib/x86_64-linux-gnu/libmirclient.so.9
#4 0x00007fffed1d7ec6 in copy_config (conf=<optimized out>) at /build/mir-Hd65rv/mir-0.26.0+16.04.20170126.3/src/server/graphics/nested/nested_display_configuration.cpp:44
#5 __base_ctor (other=warning: RTTI symbol not found for class 'mir::graphics::nested::NestedDisplayConfiguration'
..., this=0x7fffd8047300) at /build/mir-Hd65rv/mir-0.26.0+16.04.20170126.3/src/server/graphics/nested/nested_display_configuration.cpp:61

[Gerry Boland (gerboland)]

http://pastebin.ubuntu.com/23911675/ my packages

[Lukáš Tinkl (lukas-kde)]

Likely a dupe of https://bugs.launchpad.net/mir/+bug/1661163 ?

[Andreas Pokorny (andreas-pokorny)]

bug #1661163 is about an incomplete MirDisplayConfig without any cards..

This one is different - the index passed to Get(index) turns out negative:
This is the code causing this.
  // repeated .mir.protobuf.DisplayMode mode = 3;
  for (int i = 0; i < this->mode_size(); i++) {
    ::google::protobuf::internal::WireFormatLite::WriteMessage(
      3, this->mode(i), output);
  }

[Daniel van Vugt (vanvugt)]

The display config API got some major work done in Mir 0.26. This and the other bug look like fairly simple array bounds errors. And yes, negative mode indices are expected; it's clearly documented in the client headers. We just need to handle it better...

[Andreas Pokorny (andreas-pokorny)]

This is inside generated code .. looping from zero to size() .. failing in a index < 0 assertion..

[Brandon Schaefer (brandontschaefer)]

Are we not able to support vmware through jenkins as a platform?

[Michał Sawicz (saviq)]

Syncing task from Mir.

[Gerry Boland (gerboland)]

Problem is that the "real" KMS platform is unable to determine the current mode when running on VMware, leaving the current_mode_index at an invalid value of UINTMAX, which is being passed through to the nested platform.

[Gerry Boland (gerboland)]

Ok, VMWare is setting the current crtc mode at a generic 800x600, which isn't in the list of modes for the connected display. Only approach I see is to compare the mode name with "preferred", for which I see prior art:
https://github.com/wayland-project/weston/blob/master/compositor/main.c#L1068

[Launchpad Janitor (janitor)]

This bug was fixed in the package mir - 0.31.0.1-0ubuntu1

---------------
mir (0.31.0.1-0ubuntu1) bionic; urgency=medium

  * New upstream release 0.31.0 (LP: #1757952)
    - ABI summary:
      + mirclient ABI unchanged at 9
      + miral ABI bumped to 3
      + mirserver ABI bumped to 46
      + mircommon ABI unchanged at 7
      + mirplatform ABI unchanged at 16
      + mirprotobuf ABI unchanged at 3
      + mirplatformgraphics ABI unchanged at 13
      + mirclientplatform ABI unchanged at 5
      + mirinputplatform ABI unchanged at 7
      + mircore ABI unchanged at 1
    - Enhancements:
      + Miral version 2.0 - API cleanup and support for CSD initiated "resize"
      + Reorganized examples and reworked miral server examples
      + Better organization of, and fixes to, Wayland server code
      + Initial support for Wayland xdg-shell
    - Bugs fixed:
      + [miral] CommandLineOption should accept lambdas (Fixes #215)
      + [miral] Implement the policy addenda in WindowManagementTrace
        (Fixes #192)
      + [Wayland] Send repeat info (Fixes #180)
      + Explicitly initialise glm matricies, (Fixes #200)
      + KMS platform - fallback if crtc mode invalid (LP: #1661295)
      + Many fixes to Wayland support
  * debian/patches/:
    - Drop all patches; in new upstream release.
  * debian/rules:
    - Support DEB_BUILD_OPTIONS="nocheck"

 -- Christopher James Halse Rogers <email address hidden> Thu, 22 Mar 2018 16:36:52 +1100