Backport recent release 10.3.5 to latest LTS

Bug #1813944 reported by Christian Ehrhardt  on 2019-01-30
18
This bug affects 1 person
Affects Status Importance Assigned to Milestone
open-vm-tools (Ubuntu)
Undecided
Unassigned
Bionic
Undecided
vmware-gos-Yuhua
Cosmic
Undecided
vmware-gos-Yuhua

Bug Description

[Impact]

 * Without SRUing the newer version users get issues running on more
   recent hypervisors.

 * This is not backporting a single fix, but the version of a latter
   Ubuntu release

[Test Case]

 * TL;DR is "use open-vm-tools" but that can be quite complex for the
   variety of potential Host versions.
 * VMWare itself took ownership of verifying these backports and will test the same bits from a PPA and the SRU for the official "ack"
 * We tried upgrading and the setup that I had available, everybody that
   has different setups is invited to test theirs.
 * In general I recommend to give this some extra time in -proposed to see if
   anybody comes up with issues on this.

[Regression Potential]

 * It is a new version which might contain new issues, and other than in
   most MRE cases this isn't a just a stable-release (Of course we are
   not switching major releases, but also no pure fix release).

 * As agreed back when processing bug 1741390 the real verification of
   open-vm-tools for having the test matrix and project ownership is on
   VMWare which verified this from [1] already.

[Other Info]

 * After bug 1741390 was sort of first of its kind we do this on a
   once-per-cycle schedule with bug 1784638 beign the last one.
   The intention is to get regular updates not only into new -dev releases
   but also as SRU to the latest LTS under term [2] "Long Term Support
   releases we regularly want to enable new hardware" being "virtual
   hardware" in this case.
 * Associated bugs being part of the backport - while overall this is
   a MRE we can/want to discuss and verify these sub-cases individually
   to ensure the SRU is sane and to help the SRU team to understand the
   scope.
   Those bugs have individual SRU templates even thou this overall
   is an MRE.
   - bug 1807441
   - bug 1814832
   - bug 1818473
   - bug 1804287

[1]: https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/3617
[2]: https://wiki.ubuntu.com/StableReleaseUpdates#Other_safe_cases

---

There is a new Upstream version available.
  https://github.com/vmware/open-vm-tools/releases/tag/stable-10.3.5

We left 10.3.5 some time in 19.04 to see if any issues come up, but none appeared.
So lets do the backport to the LTSes.

Related branches

Changed in open-vm-tools (Ubuntu):
status: New → Triaged

I subscribed the open-vm-tools related VMWare users.
Also I opened packaging MPs to get a second pair of eyes checking for potential drawbacks and mistakes.

The builds in the PPA [1] are complete and work in my trivial Tests.
@VMWare as usually please verify these builds across your supported matrix of hipervisors and confirm here if things are ok or not.

TL;DR of the next steps:
1. VMWare to confirm validity of the PPA for Bionic and Cosmic
2. I'll put this onto the SRU queue
3. SRU team accepts this into Bionic/Cosmic-proposed
4. VMWare to confirm the validity in Bionic/Cosmic-proposed

I'll also write a mail, just to be sure.

[1]: https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/3617

Changed in open-vm-tools (Ubuntu):
status: Triaged → Fix Released
Changed in open-vm-tools (Ubuntu Bionic):
status: New → Triaged
Changed in open-vm-tools (Ubuntu Cosmic):
status: New → Triaged
description: updated
description: updated

We would most likely want to make bug 1814832 part of all of this.

I learned that testing is stalled due to Chinese new year anyway, which actually is good for that.
That provides the time to get it resolved in Disco and then bundled here.

So do -not- test this PPA until I ping again after having bug 1814832 bundled.

The backports for Bionic and Cosmic are prepared and bundled the fixes for bug 1814832 as planned.
So I re-call for your usual full regression verification on the the PPA [1] before we upload this to the SRU queue.

Setting incomplete as we wait for feedback on this.

[1]: https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/3617

Changed in open-vm-tools (Ubuntu Bionic):
status: Triaged → Incomplete
Changed in open-vm-tools (Ubuntu Cosmic):
status: Triaged → Incomplete
Oliver Kurth (okurth-1) wrote :

We did a quick sanity check for the quiescing related changes, looks good. I also asked our test team to test this (all of 10.3.5) too.

Thanks a lot Oliver already - I've heard that testing those could take a few days due to the breaks all around Asia. But I guess next week most people will be back and once this is pre-checked from the PPA we can upload to the SRU to have the final verification then.

vmware-gos-Yuhua (yhzou) wrote :

It works well when do sanity check for updated open-vm-tools 10.3.5 from ppa:
   https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/3617

Check items:
  * install / upgrade open-vm-tools and open-vm-tools-desktop(for desktop environment) from
    ppa: ci-train-ppa-service/3617
  * tools service and VGAuth service are running when install / upgrade open-vm-tools
  * tools service and VGAuth service are running after install / upgrade and reboot guestOS
  * uninstall open-vm-tools
  * sanity check for snapshot

in following VMs:
  * ubuntu-18.10-Desktop-amd64
  * ubuntu-18.10-live-server-amd64
  * ubuntu-18.04-Desktop-amd64
  * ubuntu-18.04-server-amd64

Thanks.

Thanks for the pre-check!
Ok, then all is ready to go for the real SRU where you then (once accepted) can do the real final verification.

Uploaded to Bionic/Cosmic SRU queue now.
Once the SRU Team accepts it they will call for the next (last) round of testing.

Changed in open-vm-tools (Ubuntu Bionic):
status: Incomplete → In Progress
Changed in open-vm-tools (Ubuntu Cosmic):
status: Incomplete → In Progress
Steve Langasek (vorlon) wrote :

This falls under the general "hardware enablement" SRU exception for VM substrates.

An upload of open-vm-tools to cosmic-proposed has been rejected from the upload queue for the following reason: "needs a separate SRU bug for the debian/desktop.conf addition. Also, upload fails to comply with https://wiki.ubuntu.com/DebianMaintainerField".

Steve Langasek (vorlon) wrote :

An upload of open-vm-tools to bionic-proposed has been rejected from the upload queue for the following reason: "needs a separate SRU bug for the debian/desktop.conf addition. Also, upload fails to comply with https://wiki.ubuntu.com/DebianMaintainerField".

I had more assumed it would be #2 of the list that was polled - and due to that did it sometimes more strictly and sometimes not. I had so far never read the full official rules on the field (just got told what to do), thanks for the hint Steve.
But that is easy to prepare as a fixed update.

I'll also take an extra look at the modprobe in the desktop.conf

Changed in open-vm-tools (Ubuntu Bionic):
status: In Progress → Triaged
Changed in open-vm-tools (Ubuntu Cosmic):
status: In Progress → Triaged

After improving on what we have found triggered by the first SRU review this is ready again.
Changes:
- Despite being an MRE in general all associated individual bugs have full SRU templates now.
- maintainers correctly updated
- The change to the vgauth Dependencies is now safer

That said, this is uploaded to Bionic/Cosmic unapproved again and waiting for SRU Team re-review.

description: updated

Hello Christian, or anyone else affected,

Accepted open-vm-tools into cosmic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/open-vm-tools/2:10.3.5-7~ubuntu0.18.10.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-cosmic to verification-done-cosmic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-cosmic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in open-vm-tools (Ubuntu Cosmic):
status: Triaged → Fix Committed
tags: added: verification-needed verification-needed-cosmic
Steve Langasek (vorlon) wrote :

Hello Christian, or anyone else affected,

Accepted open-vm-tools into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/open-vm-tools/2:10.3.5-7~ubuntu0.18.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in open-vm-tools (Ubuntu Bionic):
status: Triaged → Fix Committed
tags: added: verification-needed-bionic

Thanks Steve for accepting.

@Oliver/John - as usual verifying overall backport (in Bionic and Cosmic this time) is up to you, I'll also send a mail.

Changed in open-vm-tools (Ubuntu Bionic):
assignee: nobody → vmware-gos-Yuhua (yhzou)
Changed in open-vm-tools (Ubuntu Cosmic):
assignee: nobody → vmware-gos-Yuhua (yhzou)
vmware-gos-Yuhua (yhzou) wrote :

 It works well when do sanity check for 10.3.5 with repo “proposed“

 Check items:
 1. install / uninstall / upgrade open-vm-tools, check service
 2. reboot VM, check open-vm-tools/ VGAuth service
 3. check VGAuth basic function
      1) VGAuth service
      2) basic operation via VC (Guest OS —> View Guest User Mappings)
 4. snapshot with quiesce and revert

 in following VMs:
 1. ubuntu-18.10-Desktop-amd64
 2. ubuntu-18.10-live-server-amd64
 3. ubuntu-18.04-Desktop-amd64
 4. ubuntu-18.04-server-amd64
 5. ubuntu-18.04-live-server-amd64

Great, thanks Yuhua,
I'm marking this verified then - only the two special cases (SRM/Snapshots in extra bugs) are left to be confirmed then.

tags: added: verification-done verification-done-bionic verification-done-cosmic
removed: verification-needed verification-needed-bionic verification-needed-cosmic
Launchpad Janitor (janitor) wrote :
Download full text (5.1 KiB)

This bug was fixed in the package open-vm-tools - 2:10.3.5-7~ubuntu0.18.10.1

---------------
open-vm-tools (2:10.3.5-7~ubuntu0.18.10.1) cosmic; urgency=medium

  * Backport recent open-vm-tools (LP: #1813944)
    - also adresses handling of quiesced snapshot failures (LP: #1814832)
    - also adresses issues with resolutionKMS plugins sometimes fails to
      load at boot (LP: #1818473)

open-vm-tools (2:10.3.5-7) unstable; urgency=medium

  [ Christian Ehrhardt ]
  * [71b468f] make vgauth service execution more reliable.
    Since d3d47039 "Start vgauth before vmtoolsd" there is a potential race
    of starting vgauth so early that it might have issues. This was
    discussed back in the day in [1] to [2], but confirmed to be ok by
    VMWare.
    We were all somewhat convinced by this, but a bad feeling remained not
    only with me but also with Bernd [4].
    A recent SRU review denial made me rethink all of it and I think we can
    make it safer without thwarting the purpose of the original change.
    Note: Disambiguation of service names used below:
    vgauth - open-vm-tools.vgauth.service
    vmtoolsd - open-vm-tools.service
    fs - systemd-remount-fs.service
    tmp - systemd-tmpfiles-setup.service
    cloud-init - cloud-init-local.service
    Currently we have these dependency requirements:
    - vgauth should be before vmtoolsd
    - cloud init should be before vmtoolsd
    - cloud init has to be really early in general
      - therefore this is using DefaultDependencies=No
    That lead to this graph:
     fs / tmp -> vmtoolsd -> cloud-init
    And d3d47039 added it to be like:
     fs / tmp -> vmtoolsd -> cloud-init
                   ^
     vgauth --|
    But there is no need to have vgauth without any pre-dependencies at all.
    It is only needed to be "before" vmtoolsd, therefore we can make it:
     fs / tmp -> vgauth -> vmtoolsd -> cloud-init
    That will make execution of vgauth much less error-prone (even though I
    have no hard issue to report) while at the same time holding up all
    known required ordering constraints.
    [1]: https://bugs.launchpad.net/ubuntu/+source/open-vm-tools/+bug/1804287/comments/3
    [2]: https://bugs.launchpad.net/ubuntu/+source/open-vm-tools/+bug/1804287/comments/12
    [3]: https://bugs.launchpad.net/ubuntu/+source/open-vm-tools/+bug/1804287/comments/25
    [4]: https://github.com/bzed/pkg-open-vm-tools/pull/15#issuecomment-447237910
    Signed-off-by: Christian Ehrhardt <email address hidden>

open-vm-tools (2:10.3.5-6) unstable; urgency=medium

  * [43ec618] Correct and/or improve handling of certain quiesced
    snapshot failures.
    Thanks to Oliver Kurth (Closes: #921470)

open-vm-tools (2:10.3.5-5) unstable; urgency=medium

  * [54cce3e] Start vmtoolsd after apparmor.service.
    Github issue #17

open-vm-tools (2:10.3.5-4) unstable; urgency=medium

  [ Alf Gaida ]
  * [e13792d] udevadm trigger should not fail (Closes: #917642)

open-vm-tools (2:10.3.5-3) unstable; urgency=medium

  [ Christian Ehrhardt ]
  * [d3d4703] Start vgauth before vmtoolsd.
    VGAuthService needs to be ready when vmtoolsd runs. Certain cases - e.g.
    Site Reco...

Read more...

Changed in open-vm-tools (Ubuntu Cosmic):
status: Fix Committed → Fix Released

The verification of the Stable Release Update for open-vm-tools has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Launchpad Janitor (janitor) wrote :
Download full text (5.1 KiB)

This bug was fixed in the package open-vm-tools - 2:10.3.5-7~ubuntu0.18.04.1

---------------
open-vm-tools (2:10.3.5-7~ubuntu0.18.04.1) bionic; urgency=medium

  * Backport recent open-vm-tools (LP: #1813944)
    - also adresses handling of quiesced snapshot failures (LP: #1814832)
    - also adresses issues with resolutionKMS plugins sometimes fails to
      load at boot (LP: #1818473)

open-vm-tools (2:10.3.5-7) unstable; urgency=medium

  [ Christian Ehrhardt ]
  * [71b468f] make vgauth service execution more reliable.
    Since d3d47039 "Start vgauth before vmtoolsd" there is a potential race
    of starting vgauth so early that it might have issues. This was
    discussed back in the day in [1] to [2], but confirmed to be ok by
    VMWare.
    We were all somewhat convinced by this, but a bad feeling remained not
    only with me but also with Bernd [4].
    A recent SRU review denial made me rethink all of it and I think we can
    make it safer without thwarting the purpose of the original change.
    Note: Disambiguation of service names used below:
    vgauth - open-vm-tools.vgauth.service
    vmtoolsd - open-vm-tools.service
    fs - systemd-remount-fs.service
    tmp - systemd-tmpfiles-setup.service
    cloud-init - cloud-init-local.service
    Currently we have these dependency requirements:
    - vgauth should be before vmtoolsd
    - cloud init should be before vmtoolsd
    - cloud init has to be really early in general
      - therefore this is using DefaultDependencies=No
    That lead to this graph:
     fs / tmp -> vmtoolsd -> cloud-init
    And d3d47039 added it to be like:
     fs / tmp -> vmtoolsd -> cloud-init
                   ^
     vgauth --|
    But there is no need to have vgauth without any pre-dependencies at all.
    It is only needed to be "before" vmtoolsd, therefore we can make it:
     fs / tmp -> vgauth -> vmtoolsd -> cloud-init
    That will make execution of vgauth much less error-prone (even though I
    have no hard issue to report) while at the same time holding up all
    known required ordering constraints.
    [1]: https://bugs.launchpad.net/ubuntu/+source/open-vm-tools/+bug/1804287/comments/3
    [2]: https://bugs.launchpad.net/ubuntu/+source/open-vm-tools/+bug/1804287/comments/12
    [3]: https://bugs.launchpad.net/ubuntu/+source/open-vm-tools/+bug/1804287/comments/25
    [4]: https://github.com/bzed/pkg-open-vm-tools/pull/15#issuecomment-447237910
    Signed-off-by: Christian Ehrhardt <email address hidden>

open-vm-tools (2:10.3.5-6) unstable; urgency=medium

  * [43ec618] Correct and/or improve handling of certain quiesced
    snapshot failures.
    Thanks to Oliver Kurth (Closes: #921470)

open-vm-tools (2:10.3.5-5) unstable; urgency=medium

  * [54cce3e] Start vmtoolsd after apparmor.service.
    Github issue #17

open-vm-tools (2:10.3.5-4) unstable; urgency=medium

  [ Alf Gaida ]
  * [e13792d] udevadm trigger should not fail (Closes: #917642)

open-vm-tools (2:10.3.5-3) unstable; urgency=medium

  [ Christian Ehrhardt ]
  * [d3d4703] Start vgauth before vmtoolsd.
    VGAuthService needs to be ready when vmtoolsd runs. Certain cases - e.g.
    Site Reco...

Read more...

Changed in open-vm-tools (Ubuntu Bionic):
status: Fix Committed → Fix Released

open-vm-tools 11.0 will be too late this cycle (mid September), therefore in our "one MRE per cycle" approach to ensure thr latest LTS stays up to match hypervisors we will backport 10.3.10 now.

sorry - comment on wrong bug :-)

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers