Guestagent prepare fails when trying to install the mysql package if package is not installed in image

[Steve Leon]

When using an image with no mysql installed, the guest agent prepare() installs it. However, just shortly after installing it, it fails executing a external command with root privilege.

Repro:
1- Create an image with no mysql server package installed.
2- Create a trove instance

Expected:
Prepare call to install the mysql package and succeed. Instance in ACTIVE state.

Actual:
Prepare call fails. The package is installed but it fails to execute commands with root privilege.

Here is the exception:
2014-11-14 01:57:59.553 ERROR trove.openstack.common.rpc.amqp [req-2125e2f0-3372-40e5-9de8-995c1dbc26db 036cb257b6ab4a3193afc2732e4c9df5 883e86ed20664c53b3
91829e3b89ef9d] Exception during message handling
2014-11-14 01:57:59.553 TRACE trove.openstack.common.rpc.amqp Traceback (most recent call last):
2014-11-14 01:57:59.553 TRACE trove.openstack.common.rpc.amqp File "/home/ubuntu/trove/trove/openstack/common/rpc/amqp.py", line 440, in _process_data
2014-11-14 01:57:59.553 TRACE trove.openstack.common.rpc.amqp **args)
2014-11-14 01:57:59.553 TRACE trove.openstack.common.rpc.amqp File "/home/ubuntu/trove/trove/openstack/common/rpc/dispatcher.py", line 172, in dispatch
2014-11-14 01:57:59.553 TRACE trove.openstack.common.rpc.amqp result = getattr(proxyobj, method)(ctxt, **kwargs)
2014-11-14 01:57:59.553 TRACE trove.openstack.common.rpc.amqp File "/home/ubuntu/trove/trove/guestagent/datastore/mysql/manager.py", line 120, in prepare
2014-11-14 01:57:59.553 TRACE trove.openstack.common.rpc.amqp app.install_if_needed(packages)
2014-11-14 01:57:59.553 TRACE trove.openstack.common.rpc.amqp File "/home/ubuntu/trove/trove/guestagent/datastore/mysql/service.py", line 608, in install_if
_needed
2014-11-14 01:57:59.553 TRACE trove.openstack.common.rpc.amqp packager.pkg_install(packages, pkg_opts, self.TIME_OUT)
2014-11-14 01:57:59.553 TRACE trove.openstack.common.rpc.amqp File "/home/ubuntu/trove/trove/guestagent/pkg.py", line 359, in pkg_install
2014-11-14 01:57:59.553 TRACE trove.openstack.common.rpc.amqp self._fix_package_selections(packages, config_opts)
2014-11-14 01:57:59.553 TRACE trove.openstack.common.rpc.amqp File "/home/ubuntu/trove/trove/guestagent/pkg.py", line 268, in _fix_package_selections
2014-11-14 01:57:59.553 TRACE trove.openstack.common.rpc.amqp shell=True)
2014-11-14 01:57:59.553 TRACE trove.openstack.common.rpc.amqp File "/home/ubuntu/trove/trove/openstack/common/processutils.py", line 200, in execute
2014-11-14 01:57:59.553 TRACE trove.openstack.common.rpc.amqp cmd=sanitized_cmd)
2014-11-14 01:57:59.553 TRACE trove.openstack.common.rpc.amqp ProcessExecutionError: Unexpected error while running command.
2014-11-14 01:57:59.553 TRACE trove.openstack.common.rpc.amqp Command: sudo debconf-set-selections /tmp/tmpWvZslb && dpkg --configure -a
2014-11-14 01:57:59.553 TRACE trove.openstack.common.rpc.amqp Exit code: 1
2014-11-14 01:57:59.553 TRACE trove.openstack.common.rpc.amqp Stdout: u''
2014-11-14 01:57:59.553 TRACE trove.openstack.common.rpc.amqp Stderr: u'usage: sudo [-D level] -h | -K | -k | -V\nusage: sudo -v [-AknS] [-D level] [-g groupn
ame|#gid] [-p prompt] [-u user\n name|#uid]\nusage: sudo -l[l] [-AknS] [-D level] [-g groupname|#gid] [-p prompt] [-U user\n name] [-u user name|#uid] [-g
 groupname|#gid] [command]\nusage: sudo [-AbEHknPS] [-C fd] [-D level] [-g groupname|#gid] [-p prompt] [-u\n user name|#uid] [-g groupname|#gid] [VAR=value] [-i|-s]
[<command>]\nusage: sudo -e [-AknS] [-C fd] [-D level] [-g groupname|#gid] [-p prompt] [-u\n user name|#uid] file ...\n'
2014-11-14 01:57:59.553 TRACE trove.openstack.common.rpc.amqp

[Denis M. (dmakogon)]

It seems like package selection is broken.
https://github.com/openstack/trove/blob/master/trove/guestagent/pkg.py#L266

Not quite sure that given exec is correct. I'd say that currect exec is a set of two commands and each of them should be executed with sudo privileges.

I assume that next fix would work:

sudo debconf-set-selections /tmp/tmpWvZslb && sudo dpkg --configure -a

Code should look like:
utils.execute("debconf-set-selections %s"
                          % fname, run_as_root=True, root_helper="sudo",
                          shell=True)

utils.execute("dpkg --configure -a", run_as_root=True, root_helper="sudo",
                          shell=True)

Also taking into account such problems i strongly suggest to wrap this command into try:except block to give to guest properly way to define instance status (even if it'll be failed).

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/141767

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/141767
Committed: https://git.openstack.org/cgit/openstack/trove/commit/?id=3ffbbe15342f4b9c7315f6ff0a8fbc763200c696
Submitter: Jenkins
Branch: master

commit 3ffbbe15342f4b9c7315f6ff0a8fbc763200c696
Author: Sushil Kumar <email address hidden>
Date: Mon Dec 15 10:47:12 2014 +0000

    Fixes package configuration method in pkg.py

    Reasons:
    - Package configuration method _fix_package_selections is failing.
    - Because of pkg._fix_package_selections failing, the code fails to configure
      the mysql guest in case it does not have the package installed,
      which in-turn causes guest to fail.

    Changes:
    - Divided the command into two separate executions
      in pkg._fix_package_selections
    - Also added the exception handling block around utils.execute in
      pkg._fix_package_selections so that correct exception could be visible.

    Change-Id: I8f7bff9dccb7521189bd0eb08bb4b9e67d514ff8
    Closes-Bug: #1392870