Activity log for bug #1818473

Date Who What changed Old value New value Message
2019-03-04 06:34:23 Christian Ehrhardt  bug added bug
2019-03-04 06:34:29 Christian Ehrhardt  open-vm-tools (Ubuntu): status New Triaged
2019-03-04 06:34:38 Christian Ehrhardt  bug watch added https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=915031
2019-03-04 06:34:38 Christian Ehrhardt  bug task added open-vm-tools (Debian)
2019-03-04 06:34:56 Christian Ehrhardt  bug watch added https://github.com/vmware/open-vm-tools/issues/214
2019-03-04 06:34:56 Christian Ehrhardt  bug task added open-vm-tools
2019-03-04 06:51:51 Christian Ehrhardt  description This is about the tracking of a bug that was reported and fixed in Debian (and thereby also Ubuntu 19.04 already) to also SRU that back as part of our open-vm-tools backports to latest LTS. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=915031 https://github.com/vmware/open-vm-tools/issues/214 https://github.com/bzed/pkg-open-vm-tools/pull/13 [Impact] * The service for vmware base kernel mode setting fails sometimes (racy) to work after boot, but suceeds when restarted. The reason is that loading the required module is not always done in time. * This is fixed by a drop-in snippet extending open-vm-tools.service. (Not modifying the base .service file as the service and the drop in are owned by different packages). This drop in makes the service ensuring the load of the module in the ExecStartPre phase * Backport the change [1] as part of the regular backport we do for the latest Ubuntu LTS [Test Case] * Due to the racy nature of this issue there isn't a great 100% reliable test. But we can do something to at least try based on retries. * install a VMware guest with e.g. Ubuntu Desktop and install open-vm-tools-desktop * Put that guest into a reboot loop for a while, but wait until Desktop is fully up before that * Checking resulution sucks (manual on each reboot) but without the fix chances are (due to the race only chances) that the log contains the error triggering, see /var/log/vmware-vmsvc.log It will have: [ warning] [resolutionCommon] resolutionCheckForKMS: No system support for resolutionKMS. While with the fix it should always work which looks like: [ message] [resolutionCommon] resolutionCheckForKMS: System support available for resolutionKMS. [ message] [vmtoolsd] Plugin 'resolutionKMS' initialized. [Regression Potential] * I wondered first if a regression could be that for some users this always failed and due to that now after the change they get e.g. different guest resolution. But the race seems unable to be reliable either way, so those users would today be flaky with/without KMS working and the fix would make that reliable. Therefore that is no regression but an actual fix for those users. * Also failing to load the module is not a (regressing) problem. In our kernel packaging that module is only part of the -oem, -lowlatency and all modules-extra-... packages. That said there can be cases where e.g. running the virt kernels the modules isn't available. But that will not make the service fail as it is using the prefix "-" which means that if failing it still goes on to start the service itself [2]. [Other Info] * n/a [1]: https://github.com/bzed/pkg-open-vm-tools/commit/db2a3642d45 [2]: https://www.freedesktop.org/software/systemd/man/systemd.service.html --- This is about the tracking of a bug that was reported and fixed in Debian (and thereby also Ubuntu 19.04 already) to also SRU that back as part of our open-vm-tools backports to latest LTS. Related: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=915031 https://github.com/vmware/open-vm-tools/issues/214 https://github.com/bzed/pkg-open-vm-tools/pull/13
2019-03-04 06:52:51 Christian Ehrhardt  nominated for series Ubuntu Cosmic
2019-03-04 06:52:51 Christian Ehrhardt  bug task added open-vm-tools (Ubuntu Cosmic)
2019-03-04 06:52:51 Christian Ehrhardt  nominated for series Ubuntu Bionic
2019-03-04 06:52:51 Christian Ehrhardt  bug task added open-vm-tools (Ubuntu Bionic)
2019-03-04 06:52:58 Christian Ehrhardt  open-vm-tools (Ubuntu): status Triaged Fix Released
2019-03-04 06:53:00 Christian Ehrhardt  open-vm-tools (Ubuntu Bionic): status New Triaged
2019-03-04 06:53:02 Christian Ehrhardt  open-vm-tools (Ubuntu Cosmic): status New Triaged
2019-03-04 07:03:24 Bug Watch Updater open-vm-tools (Debian): status Unknown Fix Released
2019-03-04 07:41:48 Christian Ehrhardt  description [Impact] * The service for vmware base kernel mode setting fails sometimes (racy) to work after boot, but suceeds when restarted. The reason is that loading the required module is not always done in time. * This is fixed by a drop-in snippet extending open-vm-tools.service. (Not modifying the base .service file as the service and the drop in are owned by different packages). This drop in makes the service ensuring the load of the module in the ExecStartPre phase * Backport the change [1] as part of the regular backport we do for the latest Ubuntu LTS [Test Case] * Due to the racy nature of this issue there isn't a great 100% reliable test. But we can do something to at least try based on retries. * install a VMware guest with e.g. Ubuntu Desktop and install open-vm-tools-desktop * Put that guest into a reboot loop for a while, but wait until Desktop is fully up before that * Checking resulution sucks (manual on each reboot) but without the fix chances are (due to the race only chances) that the log contains the error triggering, see /var/log/vmware-vmsvc.log It will have: [ warning] [resolutionCommon] resolutionCheckForKMS: No system support for resolutionKMS. While with the fix it should always work which looks like: [ message] [resolutionCommon] resolutionCheckForKMS: System support available for resolutionKMS. [ message] [vmtoolsd] Plugin 'resolutionKMS' initialized. [Regression Potential] * I wondered first if a regression could be that for some users this always failed and due to that now after the change they get e.g. different guest resolution. But the race seems unable to be reliable either way, so those users would today be flaky with/without KMS working and the fix would make that reliable. Therefore that is no regression but an actual fix for those users. * Also failing to load the module is not a (regressing) problem. In our kernel packaging that module is only part of the -oem, -lowlatency and all modules-extra-... packages. That said there can be cases where e.g. running the virt kernels the modules isn't available. But that will not make the service fail as it is using the prefix "-" which means that if failing it still goes on to start the service itself [2]. [Other Info] * n/a [1]: https://github.com/bzed/pkg-open-vm-tools/commit/db2a3642d45 [2]: https://www.freedesktop.org/software/systemd/man/systemd.service.html --- This is about the tracking of a bug that was reported and fixed in Debian (and thereby also Ubuntu 19.04 already) to also SRU that back as part of our open-vm-tools backports to latest LTS. Related: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=915031 https://github.com/vmware/open-vm-tools/issues/214 https://github.com/bzed/pkg-open-vm-tools/pull/13 [Impact]  * The service for vmware base kernel mode setting fails sometimes (racy)    to work after boot, but suceeds when restarted. The reason is that    loading the required module is not always done in time.  * This is fixed by a drop-in snippet extending open-vm-tools.service.    (Not modifying the base .service file as the service and the    drop in are owned by different packages).    This drop in makes the service ensuring the load of the module in the    ExecStartPre phase  * Backport the change [1] as part of the regular backport we do    for the latest Ubuntu LTS [Test Case]  * Due to the racy nature of this issue there isn't a great 100%    reliable test. But we can do something to at least try based on    retries.  * install a VMware guest with e.g. Ubuntu Desktop and install    open-vm-tools-desktop  * Put that guest into a reboot loop for a while, but wait until Desktop    is fully up before that  * Checking resulution sucks (manual on each reboot) but without the fix    chances are (due to the race only chances) that the log contains the    error triggering, see /var/log/vmware-vmsvc.log    It will have:        [ warning] [resolutionCommon] resolutionCheckForKMS: No system        support for resolutionKMS.    While with the fix it should always work which looks like:        [ message] [resolutionCommon] resolutionCheckForKMS: System        support available for resolutionKMS.        [ message] [vmtoolsd] Plugin 'resolutionKMS' initialized. * For a somewhat faked test you could drop a file that prevents the module from loading e.g. $ echo "blacklist vmwgfx" | sudo tee /etc/modprobe.d/blacklist-vmware.conf $ echo "install vmwgfx /bin/false" | sudo tee -a /etc/modprobe.d/blacklist-vmware.conf $ sudo update-initramfs -u -k all Then reboot which will make it start without the module and trigger the error condition as if it would have raced (service up but the module not loaded). On a service restart you'll see the error: [2019-03-04T08:17:34.112Z] [ message] [resolutionCommon] resolutionCheckForKMS: dlopen succeeded. [2019-03-04T08:17:34.268Z] [ warning] [resolutionCommon] resolutionCheckForKMS: No system support for resolutionKMS. Even if you now remove the blacklist it will still fail that way. If you modprobe vmwgfx it will switch to [2019-03-04T08:18:42.983Z] [ message] [resolutionCommon] resolutionCheckForKMS: dlopen succeeded. [2019-03-04T08:18:42.986Z] [ message] [resolutionCommon] resolutionCheckForKMS: System support available for resolutionKMS. The latter should work without manual modprobe and with the fix it does. [Regression Potential]  * I wondered first if a regression could be that for some users    this always failed and due to that now after the change they get    e.g. different guest resolution. But the race seems unable to be    reliable either way, so those users would today be flaky with/without    KMS working and the fix would make that reliable. Therefore that is    no regression but an actual fix for those users.  * Also failing to load the module is not a (regressing) problem.    In our kernel packaging that module is only part of the -oem,    -lowlatency and all modules-extra-... packages. That said there    can be cases where e.g. running the virt kernels the modules isn't    available. But that will not make the service fail as it is using    the prefix "-" which means that if failing it still goes on to    start the service itself [2]. [Other Info]  * n/a [1]: https://github.com/bzed/pkg-open-vm-tools/commit/db2a3642d45 [2]: https://www.freedesktop.org/software/systemd/man/systemd.service.html --- This is about the tracking of a bug that was reported and fixed in Debian (and thereby also Ubuntu 19.04 already) to also SRU that back as part of our open-vm-tools backports to latest LTS. Related: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=915031 https://github.com/vmware/open-vm-tools/issues/214 https://github.com/bzed/pkg-open-vm-tools/pull/13
2019-03-04 08:21:06 Bug Watch Updater open-vm-tools: status Unknown New
2019-03-08 21:22:59 Steve Langasek open-vm-tools (Ubuntu Cosmic): status Triaged Fix Committed
2019-03-08 21:23:01 Steve Langasek bug added subscriber Ubuntu Stable Release Updates Team
2019-03-08 21:23:04 Steve Langasek bug added subscriber SRU Verification
2019-03-08 21:23:06 Steve Langasek tags verification-needed verification-needed-cosmic
2019-03-08 21:29:44 Steve Langasek open-vm-tools (Ubuntu Bionic): status Triaged Fix Committed
2019-03-08 21:29:50 Steve Langasek tags verification-needed verification-needed-cosmic verification-needed verification-needed-bionic verification-needed-cosmic
2019-03-11 08:00:11 Christian Ehrhardt  tags verification-needed verification-needed-bionic verification-needed-cosmic verification-done-bionic verification-needed verification-needed-cosmic
2019-03-11 08:07:42 Christian Ehrhardt  tags verification-done-bionic verification-needed verification-needed-cosmic verification-done verification-done-bionic verification-done-cosmic
2019-04-16 19:31:13 Launchpad Janitor open-vm-tools (Ubuntu Cosmic): status Fix Committed Fix Released
2019-04-16 19:31:38 Brian Murray removed subscriber Ubuntu Stable Release Updates Team
2019-04-16 19:32:42 Launchpad Janitor open-vm-tools (Ubuntu Bionic): status Fix Committed Fix Released