add sdmp plugin as a separate package

[Oliver Kurth]

[Impact]

 * In the past a lack of updated open-vm-tools versions was found fatal
   and therefore we agreed to regularly backport those to the latest LTS.
   This is backporting the recent version 11.1 to Focal.

 * The main bug for 11.0 is bug 1877672. But this time other than the
   fixes/improvements this will also bring a new
   plugin "sdmp" which I'd like to:
    a) discuss separately fro the SRU process
    b) have VMWare test explicitly as part of the verification

[Test Case]

 * Test open-vm-tools done by VMWare QA Team as agreed for these
   backports. In this case please explicitly test the sdmp functionality,
   not the general regression check.

[Regression Potential]

 * As I said this bug is especially for the sdmp plugin. I doubt it will
   regress anyone since it is not installed by default (only a suggests).
   The only (unlikely) case I can tihnk of is if someone does like
     apt install open-vm-*
   or something like that. But even in that case the plugin does not do
   much until actually probed from the host.

[Other Info]

 * This is a practice for or quite a while (almost two years now), see:
   - https://bugs.launchpad.net/ubuntu/+source/open-vm-tools/+bug/1741390
   - https://bugs.launchpad.net/ubuntu/+source/open-vm-tools/+bug/1784638
   - https://bugs.launchpad.net/ubuntu/+source/open-vm-tools/+bug/1813944
   - https://bugs.launchpad.net/ubuntu/+source/open-vm-tools/+bug/1822204
   - https://bugs.launchpad.net/ubuntu/+source/open-vm-tools/+bug/1844834
   - https://bugs.launchpad.net/ubuntu/+source/open-vm-tools/+bug/1868012

The request back then came from the community and the decision back then was between VMWare and Ubuntu. VMWare QA Team since then does the qualification of these uploads as we (I) don't have a matrix of Host versions for that around.

Under the accepted special SRU cases [1] this qualifies for #2 and #3 of [2].

[1]: https://wiki.ubuntu.com/StableReleaseUpdates#Documentation_for_Special_Cases
[2]: https://wiki.ubuntu.com/StableReleaseUpdates#Other_safe_cases

---

Version 11.1.0 adds the sdmp plugin, see https://marketplace.vmware.com/vsx/solutions/vrealize-operations-service-discovery-management-pack?ref=search

This should be in an optional separate package. Attached is a patch for the Debian scripts that would create this package. This has been used in our internal testing. Note that there is an additional runtime dependency on net-tools for netstat.

Also at Debian: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=960065

Thanks,
Oliver

[Lucas Kanashiro (lucaskanashiro)]

Thank you for taking the time to file a bug report.

The version 11.1.0 is still not available either in Ubuntu nor in Debian. Since the Debian maintainer is considering/reviewing your patch and currently we have no delta on top of the package coming from Debian, I'd prefer to get it accepted in Debian and then sync it to Ubuntu. Let's see how responsive the Debian maintainer will be.

[Christian Ehrhardt  (paelzer)]

Hi Lucas "Let's see how responsive the Debian maintainer will be" - he is very responsive and I already pinged him on IRC if he wants me to help. Consider this bug under control for now (assigning myself).

[Christian Ehrhardt  (paelzer)]

I had to fix some missing dependencies.
As Bernd already mentioned netstat is deprecated, mid term you should consider using more modern tools. I recommend to read e.g. https://dougvitale.wordpress.com/2011/12/21/deprecated-linux-networking-commands-and-their-replacements/

[Christian Ehrhardt  (paelzer)]

FYI covered in
https://salsa.debian.org/vmware-packaging-team/pkg-open-vm-tools/-/merge_requests/3
https://salsa.debian.org/vmware-packaging-team/pkg-open-vm-tools/-/merge_requests/4
https://salsa.debian.org/vmware-packaging-team/pkg-open-vm-tools/-/merge_requests/5

[Christian Ehrhardt  (paelzer)]

FYI - my MPs were accepted and currently wait in Debian NEW-queue

[Oliver Kurth (okurth-1)]

As noted on the Debian bug:

Hi,

because of "netstat" being widely deprecated now, we are going to change the sdmp plugin to use "ss" instead. There is no point in continuing with the current version of sdmp in 11.1.0, so please put this on hold. We are going to have a fixed version in our next release, and may provide a patch for 11.1.0.

This does not affect the rest of open-vm-tools 11.1.0 , so the update can go ahead without the plugin. Note that it's not built by default.

Thanks,
Oliver

[Oliver Kurth (okurth-1)]

We now have a patch that replaces 'netstat' with 'ss' because the former is deprecated. We just created an extra branch with the changes, plus two minor sdmp related fixes: https://github.com/vmware/open-vm-tools/commits/stable-11.1.0-SDMP-fixes

[Christian Ehrhardt  (paelzer)]

Thanks Oliver!

Abandoned
https://salsa.debian.org/vmware-packaging-team/pkg-open-vm-tools/-/merge_requests/6
and added
https://salsa.debian.org/vmware-packaging-team/pkg-open-vm-tools/-/merge_requests/7

[Launchpad Janitor (janitor)]

This bug was fixed in the package open-vm-tools - 2:11.1.0-2

---------------
open-vm-tools (2:11.1.0-2) unstable; urgency=medium

  [ Christian Ehrhardt ]
  * [4d69c6a] d/p/lp-1877678-: fixes for the sdmp plugin that is new in 11.1.0.
    Signed-off-by: Christian Ehrhardt <email address hidden>
  * [38bd11e] d/control: change net-tools dependency to iproute2.
    Signed-off-by: Christian Ehrhardt <email address hidden>

  [ Bernd Zeimetz ]
  * [c15c08d] Add net-tools as dependency again.
    Various scripts still use ifconfig.

 -- Bernd Zeimetz <email address hidden> Fri, 19 Jun 2020 14:05:44 +0200

[Christian Ehrhardt  (paelzer)]

As I expected installing but not configuring/triggering sdmp does nothing than loading the .so.

ubuntu@node-1:~$ sudo lsof +fg | grep vmsvc
vmtoolsd 14567 root mem REG 8,1 44048 1444886 /usr/lib/open-vm-tools/plugins/vmsvc/libvmbackup.so
vmtoolsd 14567 root mem REG 8,1 23112 1444885 /usr/lib/open-vm-tools/plugins/vmsvc/libtimeSync.so
vmtoolsd 14567 root mem REG 8,1 23152 1441835 /usr/lib/open-vm-tools/plugins/vmsvc/libserviceDiscovery.so
vmtoolsd 14567 root mem REG 8,1 57800 1444882 /usr/lib/open-vm-tools/plugins/vmsvc/libguestInfo.so
vmtoolsd 14567 root mem REG 8,1 15024 1444883 /usr/lib/open-vm-tools/plugins/vmsvc/libpowerOps.so
vmtoolsd 14567 root mem REG 8,1 23272 1444881 /usr/lib/open-vm-tools/plugins/vmsvc/libdeployPkgPlugin.so
vmtoolsd 14567 root mem REG 8,1 19272 1444880 /usr/lib/open-vm-tools/plugins/vmsvc/libappInfo.so
vmtoolsd 14567 root 3w REG W,LG 8,1 2913 524311 /var/log/vmware-vmsvc-root.log
vmtoolsd 14567 14581 HangDetec root mem REG 8,1 44048 1444886 /usr/lib/open-vm-tools/plugins/vmsvc/libvmbackup.so
vmtoolsd 14567 14581 HangDetec root mem REG 8,1 23112 1444885 /usr/lib/open-vm-tools/plugins/vmsvc/libtimeSync.so
vmtoolsd 14567 14581 HangDetec root mem REG 8,1 23152 1441835 /usr/lib/open-vm-tools/plugins/vmsvc/libserviceDiscovery.so
vmtoolsd 14567 14581 HangDetec root mem REG 8,1 57800 1444882 /usr/lib/open-vm-tools/plugins/vmsvc/libguestInfo.so
vmtoolsd 14567 14581 HangDetec root mem REG 8,1 15024 1444883 /usr/lib/open-vm-tools/plugins/vmsvc/libpowerOps.so
vmtoolsd 14567 14581 HangDetec root mem REG 8,1 23272 1444881 /usr/lib/open-vm-tools/plugins/vmsvc/libdeployPkgPlugin.so
vmtoolsd 14567 14581 HangDetec root mem REG 8,1 19272 1444880 /usr/lib/open-vm-tools/plugins/vmsvc/libappInfo.so
vmtoolsd 14567 14581 HangDetec root 3w REG W,LG 8,1 2913 524311 /var/log/vmware-vmsvc-root.log
ubuntu@node-1:~$ systemctl status open-vm-tools
● open-vm-tools.service - Service for virtual machines hosted on VMware
     Loaded: loaded (/lib/systemd/system/open-vm-tools.service; enabled; vendor preset: enabled)
     Active: active (r...

[Christian Ehrhardt  (paelzer)]

Uploaded to focal-unapproved for the SRU Team to take a look at it.

[Robie Basak (racb)]

Hello Oliver, or anyone else affected,

Accepted open-vm-tools into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/open-vm-tools/2:11.1.0-2~ubuntu20.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[vmware-gos-Yuhua (yhzou)]

By default, the package "open-vm-tools-sdmp" won't be installed when we install open-vm-tools with command "apt-get install open-vm-tools"

Sanity check for install/uninstall this package:
  After type command "apt-get install open-vm-tools-sdmp", we can find the newly added files:
   /usr/lib/open-vm-tools/plugins/vmsvc/libserviceDiscovery.so
   /usr/lib/open-vm-tools/serviceDiscovery/scripts/get-connection-info.sh
   /usr/lib/open-vm-tools/serviceDiscovery/scripts/get-listening-process-perf-metrics.sh
   /usr/lib/open-vm-tools/serviceDiscovery/scripts/get-listenioong-process-info.sh
   /usr/lib/open-vm-tools/serviceDiscovery/scripts/get-versions.sh

  After type command "apt-get remove --purge open-vm-tools-sdmp", delete the follows file and directory:
   /usr/lib/open-vm-tools/plugins/vmsvc/libserviceDiscovery.so
   /usr/lib/open-vm-tools/serviceDiscovery

VMware will conduct more testing about it.

[vmware-gos-Yuhua (yhzou)]

>vmware-gos-Yuhua (yhzou) wrote on 2020-07-08: #13

 From vware dev's confirmation:
 The four shell script files and the main library file were properly installed (as expected). Also, after the purge/remove, the files were properly deleted. This is working from a packaging perspective.

I am waiting the confirmation about core functionality validation

[Christian Ehrhardt  (paelzer)]

Thanks for the pre-checks Yuhua, looking forward to your ping here once the core functionality of the sdmp plugin was tested as well.

[vmware-gos-Yuhua (yhzou)]

Hi All,
The core functionality of the sdmp plugin was tested.
We can ship the SDMP plugin in open-vm-tools 11.1.0.

Best regards
Yuhua Zou

[Christian Ehrhardt  (paelzer)]

Thank you Yuhua!
Marking this as verified as well.

[Launchpad Janitor (janitor)]

This bug was fixed in the package open-vm-tools - 2:11.1.0-2~ubuntu20.04.1

---------------
open-vm-tools (2:11.1.0-2~ubuntu20.04.1) focal; urgency=medium

  * Update to latest release v11.1.0 (LP: #1877672)
    - among many fixes this also brings the sdmp plugin (LP: #1877678)
    - fixes upgrade issues due to bad package versions (LP: #1884483)
    - Revert "Add net-tools as dependency again." as we don't want to
      modify the focal seed/ISO content without a real issue behind it.

open-vm-tools (2:11.1.0-2) unstable; urgency=medium

  [ Christian Ehrhardt ]
  * [4d69c6a] d/p/lp-1877678-: fixes for the sdmp plugin that is new in 11.1.0.
    Signed-off-by: Christian Ehrhardt <email address hidden>
  * [38bd11e] d/control: change net-tools dependency to iproute2.
    Signed-off-by: Christian Ehrhardt <email address hidden>

  [ Bernd Zeimetz ]
  * [c15c08d] Add net-tools as dependency again.
    Various scripts still use ifconfig.

open-vm-tools (2:11.1.0-1) unstable; urgency=medium

  [ Christian Ehrhardt ]
  * [6b7d31d] New upstream version 11.1.0
    (Closes: #960061) (LP: #1877672)
  * [3ece93a14] d/control, d/rules, d//*sdmp*: add service discovery plugin (sdmp)
    (Closes: #960065) (LP: #1877678)
    Thanks to Oliver Kurth for the initial contribution, changes in addition:
    - d/control: improve description
    - rules fix whitespace damage
    - maintscripts: fixed some whihtespace damage
    - maintscripts: fixed maintainer scripts per skeletons from dh_make
    - maintscripts: added the service-active-before-restart check to postinst
      as well (was only in rm)
    - maintscripts: use deb-systemd-invoke
    - d/control: add further dependencies used in sdmp
  * [e0c9fbc14] remove patches applied upstream in 11.1.0
    - d/p/4ee0bd3c8_Rectify-a-log-spew-in-vmsvc-logging-vmware-vmsvc-root.log
    - d/p/89c0d4445_GitHub-Issue-367.-Remove-references-to-deprecated-G_INLINE_FUNC
    - d/p/f1f0b812e_add-appinfo-plugin
  * [f4cf14931] d/rules: drop perm fixup of vm-support as it is properly
    in /usr/bin/ now
  * [d71e99e33] lintian: add overrides for intentional cases
  * [ba27a73eb] d/p/debian/vmxnet_fix_kernel_4.7.patch: drop unused patch
  * [7488e6e2f] d/copyright: fix tab in text

 -- Christian Ehrhardt <email address hidden> Mon, 22 Jun 2020 08:40:58 +0200

[Łukasz Zemczak (sil2100)]

The verification of the Stable Release Update for open-vm-tools has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.