job runner is broken for jobs using plugin: user-interact

[Chris Gregan]

It seems the testing applet we launch from the vertical and horizontal scroll testing is not installed or not functioning.

[Brendan Donegan (brendan-donegan)]

This appears to be a general issue with launching GUI applications from within c-d-t-s. It seems to 'swallow' the window of that application.

[Zygmunt Krynicki (zyga)]

We need to check if we get user session environment inside 'checkbox service' when auto-started on demand by dbus.

[Daniel Manrique (roadmr)]

Setting up testing for this turned out to be a bit cumbersome, in the end the easiest way was to install all the packages from the c-d-t-s tarball and modify /usr/share/plainbox-providers-1/checkbox/jobs/touchpad.txt directly to experiment. i mostly changed the touchpad/horizontal job.

What I found is:

commands in user-verify (and possibly user-interact) jobs don't launch when clicking the test button. This happens whether you launch checkbox service manually, or it auto starts via dbus. I confirmed (by setting the command to "touch /tmp/control" and then checking that the file is NOT touched when clicking test) that this is regardless of the GUIness of the command itself.

If I change the plugin to "shell", then the command does run fine, even graphical ones as touch_test or zenity.

Changing the user doesn't alter the behavior (i.e. shell commands launch gui programs just fine, hooray for the trusted launcher) but user-* ones don't work :/

[Zygmunt Krynicki (zyga)]

This is, most likely, not a plainbox bug at all. The gui never really asks us to run that job. It *doesn't* even try to start it.

[Zygmunt Krynicki (zyga)]

The bug that I found so far is that the GUI is clueless about unsupported jobs:

DEBUG checkbox.ng.service: _result_changed(<MemoryJobResult>, <MemoryJobResult comments:'job cannot be started: resource expression "\'Button Horiz Wheel Left\' in xinput.button_labels and \'Button Horiz Wheel Right\' in xinput.button_labels" evaluates to false' outcome:'not-supported'>)

This has *no* sane feedback in the application. I'll modify the job not to require anything and try again. Curiously, I can still run that same test from command line with "plainbox -c src run -i touchpad/horizontal".

[Zygmunt Krynicki (zyga)]

I suspect that over-eager filtering causes the GUI to ignore the run list. It seems that the app doesn't start xinput resource.

[Zygmunt Krynicki (zyga)]

It *seems* that we crash somewhere in a very odd way, that is *job output* not plainbox though:

/plainbox/DiskJobResult/140086603008336
 [org.freedesktop.DBus.Properties]
 [com.canonical.certification.PlainBox.Result1]
  comments =
  return_code = -6
  io_log = dbus.Array([dbus.Struct((dbus.Double(0.012067), dbus.String('stderr'), dbus.ByteArray(b"Fatal Python error: Py_Initialize: can't initialize sys standard streams\n")), signature=None), dbus.Struct((dbus.Double(0.000224), dbus.String('stderr'), dbus.ByteArray(b'Traceback (most recent call last):\n')), signature=None), dbus.Struct((dbus.Double(4.7e-05), dbus.String('stderr'), dbus.ByteArray(b' File "/usr/lib/python3.3/io.py", line 52, in <module>\n')), signature=None), dbus.Struct((dbus.Double(0.000107), dbus.String('stderr'), dbus.ByteArray(b' File "./abc.py", line 38, in <module>\n')), signature=None), dbus.Struct((dbus.Double(0.000445), dbus.String('stderr'), dbus.ByteArray(b'ImportError: cannot import name ABCMeta\n')), signature=None)], signature=dbus.Signature('(dsay)'), variant_level=1)
  execution_duration = 0.12436127662658691
  outcome = fail
 [org.freedesktop.DBus.ObjectManager]
   (asking about managed object)
 [org.freedesktop.DBus.Introspectable]

[Zygmunt Krynicki (zyga)]

xinput resource job crashes:

Fatal Python error: Py_Initialize: can't initialize sys standard streams
Traceback (most recent call last):
  File "/usr/lib/python3.3/io.py", line 52, in <module>
  File "./abc.py", line 38, in <module>
ImportError: cannot import name ABCMeta

[Zygmunt Krynicki (zyga)]

DEBUG plainbox.ctrl: Symlink nest for executables: /tmp/nest-kz0215.e281a595d249fffcd5b608a85157ad910b423e4a4d23438e295c291f252d4724
DEBUG plainbox.ctrl: job[xinput] executing ['bash', '-c', 'xinput_resource'] with env {'GTK_MODULES': 'overlay-scrollbar:unity-gtk-module', 'GDMSESSION': 'ubuntu', 'WINDOWID': '54525959', 'XDG_SEAT_PATH': '/org/freedesktop/DisplayManager/Seat0', 'COMPIZ_CONFIG_PROFILE': 'ubuntu', 'SSH_AGENT_PID': '1944', 'PWD': '/home/zyga/checkbox/plainbox/plainbox', 'SESSION': 'ubuntu', 'XDG_RUNTIME_DIR': '/run/user/1000', 'CLUTTER_IM_MODULE': 'xim', 'SESSIONTYPE': 'gnome-session', 'USER': 'zyga', 'XDG_CONFIG_DIRS': '/etc/xdg/xdg-ubuntu:/usr/share/upstart/xdg:/etc/xdg', 'GNOME_KEYRING_CONTROL': '/run/user/1000/keyring-WYtU5D', 'PIP_DOWNLOAD_CACHE': '/home/zyga/.cache/pip/downloads/', 'MANDATORY_PATH': '/usr/share/gconf/ubuntu.mandatory.path', 'DESKTOP_SESSION': 'ubuntu', 'XDG_CURRENT_DESKTOP': 'Unity', 'LANG': 'C.UTF-8', 'VIRTUAL_ENV': '/tmp/venv', 'PYTHONPATH': '/home/zyga/checkbox-testing/local-packages/checkbox/checkbox-old:', 'SESSION_MANAGER': 'local/fx:@/tmp/.ICE-unix/2029,unix/fx:/tmp/.ICE-unix/2029', 'GPG_AGENT_INFO': '/run/user/1000/keyring-WYtU5D/gpg:0:1', 'VAGRANT_APT_CACHE': 'http://192.168.0.3:3142', 'SSH_AUTH_SOCK': '/run/user/1000/keyring-WYtU5D/ssh', 'XMODIFIERS': '@im=ibus', 'LESSCLOSE': '/usr/bin/lesspipe %s %s', 'SHELL': '/bin/bash', 'QT4_IM_MODULE': 'xim', 'DISPLAY': ':0', 'COMPIZ_BIN_PATH': '/usr/bin/', 'OLDPWD': '/home/zyga/checkbox/plainbox', 'IM_CONFIG_PHASE': '1', 'CHECKBOX_SHARE': '/home/zyga/checkbox-testing/local-packages/checkbox/checkbox-old', 'DEBEMAIL': '<email address hidden>', 'SHLVL': '1', 'HOME': '/home/zyga', 'QUILT_PATCHES': 'debian/patches', 'ALIOTH_USER': 'zyga-guest', 'EDITOR': 'vim', 'XAUTHORITY': '/home/zyga/.Xauthority', 'XDG_SESSION_ID': 'c2', 'GDM_LANG': 'en_US', 'DBUS_SESSION_BUS_ADDRESS': 'unix:abstract=/tmp/dbus-7sGzlimM0F', '_': '/tmp/venv/bin/checkbox', 'DEFAULTS_PATH': '/usr/share/gconf/ubuntu.default.path', 'GNOME_KEYRING_PID': '1875', 'PS1': '(venv)${debian_chroot:+($debian_chroot)}\\[\\033[01;32m\\]\\u@\\h\\[\\033[00m\\]:\\[\\033[01;34m\\]\\w\\[\\033[00m\\]\\$ ', 'GNOME_DESKTOP_SESSION_ID': 'this-is-deprecated', 'XDG_SEAT': 'seat0', 'UBUNTU_MENUPROXY': '1', 'XDG_VTNR': '7', 'UPSTART_SESSION': 'unix:abstract=/com/ubuntu/upstart-session/1000/1877', 'XDG_DATA_DIRS': '/usr/share/ubuntu:/usr/share/gnome:/usr/local/share/:/usr/share/', 'PATH': '/tmp/nest-kz0215.e281a595d249fffcd5b608a85157ad910b423e4a4d23438e295c291f252d4724:/tmp/venv/bin:/usr/local/heroku/bin:/usr/local/heroku/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games', 'VTE_VERSION': '3409', 'TEXTDOMAIN': 'im-config', 'TEXTDOMAINDIR': '/usr/share/locale/', 'DEBFULLNAME': 'Zygmunt Krynicki', 'GTK_IM_MODULE': 'ibus', 'JOB': 'dbus', 'SELINUX_INIT': 'YES', 'GOPATH': '/home/zyga/.local', 'QUILT_REFRESH_ARGS': '-p ab --no-timestamps --no-index', 'LOGNAME': 'zyga', 'XDG_MENU_PREFIX': 'gnome-', 'XDG_SESSION_PATH': '/org/freedesktop/DisplayManager/Session0', 'TERM': 'xterm-256color', 'CHECKBOX_DATA': '/home/zyga/.cache/plainbox/sessions/pbox-utgxqs.session/CHECKBOX_DATA', 'apt_...

[Zygmunt Krynicki (zyga)]

Theory: due to the peculiar PATH arrangement python is picking up 'plainbox.abc' as 'abc'.

Fatal Python error: Py_Initialize: can't initialize sys standard streams
Traceback (most recent call last):
  File "/usr/lib/python3.3/io.py", line 52, in <module>
  File "./abc.py", line 38, in <module>
ImportError: cannot import name ABCMeta

The abc.py, line 38 mentioned above is actually plainbox/plainbox/abc.py, at that line it tries to import 'abc' to get ABCMeta.

[Zygmunt Krynicki (zyga)]

Running plainbox or checkbox from lp:checkbox/plainbox fails. This bug *goes away* when I do that.

I then get to a point where we run the user-interact plugin which... does nothing!

[Zygmunt Krynicki (zyga)]

This is broken by:

commit 4614cd7948b24786ea46dd21468d5d45e00d5593
Author: Sylvain Pineau <email address hidden>
Date: Thu Oct 10 18:30:05 2013 +0200

    plainbox:runner: Non automated jobs return OUTCOME_UNDECIDED

    To let the client set the outcome, non automated jobs use this special
    value to trigger the corresponding interaction.

    The manual and user-interact plugins now return a MemoryJobResult
    containing {'outcome': IJobResult.OUTCOME_UNDECIDED}).

    The user-verify and user-interact-verify only alter the outcome of the
    prerun command result, this way we preserve all logs and associated state.

diff --git a/plainbox/plainbox/impl/runner.py b/plainbox/plainbox/impl/runner.py
index 91e1246..21e96dd 100644
--- a/plainbox/plainbox/impl/runner.py
+++ b/plainbox/plainbox/impl/runner.py
@@ -376,7 +376,7 @@ class JobRunner(IJobRunner):
         """
         if job.plugin != "manual":
             raise ValueError("bad job plugin value")
- return self._call_interaction_callback(job, config)
+ return MemoryJobResult({'outcome': IJobResult.OUTCOME_UNDECIDED})

     def run_user_interact_job(self, job, config):
         """
@@ -416,7 +416,7 @@ class JobRunner(IJobRunner):
         """
         if job.plugin != "user-interact":
             raise ValueError("bad job plugin value")
- return self._just_run_command(job, config)
+ return MemoryJobResult({'outcome': IJobResult.OUTCOME_UNDECIDED})

     def run_user_verify_job(self, job, config):
         """
@@ -464,14 +464,7 @@ class JobRunner(IJobRunner):
         # Run the command
         result_cmd = self._just_run_command(job, config)
         # Maybe ask the user
- result_user = self._call_interaction_callback(job, config)
- # FIXME: result_user must never be None -- it is currently broken in
- # the existing service/highlevel code as emitAskForOutcomeSignal()
- # doesn't return anything.
- if result_user is not None:
- # Copy whatever user has decided over to the command result. This
- # way we preserve all logs and associated state.
- result_cmd.outcome = result_user.outcome
+ result_cmd.outcome = IJobResult.OUTCOME_UNDECIDED
         return result_cmd

     def run_user_interact_verify_job(self, job, config):
@@ -521,14 +514,7 @@ class JobRunner(IJobRunner):
         # Run the command
         result_cmd = self._just_run_command(job, config)
         # Maybe ask the user
- result_user = self._call_interaction_callback(job, config)
- # FIXME: result_user must never be None -- it is currently broken in
- # the existing service/highlevel code as emitAskForOutcomeSignal()
- # doesn't return anything.
- if result_user is not None:
- # Copy whatever user has decided over to the command result. This
- # way we preserve all logs and associated state.
- result_cmd.outcome = result_user.outcome
+ result_cmd.outcome = IJobResult.OUTCOME_UNDECIDED
         return result_cmd

     def _call_interaction_callback(self, job, config):

[Zygmunt Krynicki (zyga)]

It seems that we've accidentally changed the semantics of the user-interact job type:

 - return self._just_run_command(job, config)
+ return MemoryJobResult({'outcome': IJobResult.OUTCOME_UNDECIDED})

Here it's never ever going to run the command. It wasn't noticed because the gui was using undocumented methods to directly bypass the job runner and run the command manually. This broke when we dropped that code.

For reference, this is how the user-interact plugin is documented to work:

    def run_user_interact_job(self, job, config):
        """
        Method called to run a job with plugin field equal to 'user-interact'

        The 'user-interact' job implements the following scenario:

        * Display the description to the user
        * Ask the user to perform some operation
        * Wait for the user to confirm this is done
        * The API states that :meth:`JobRunner.run_job()` should only be
          called at this time.
        * Run the command and wait for it to finish
        * Decide on the outcome based on the return code
        * The method ends here

        .. note::
            User interaction jobs are candidates for further automation as the
            outcome can be already determined automatically but some
            interaction, yet, cannot.

        .. note::
            User interaction jobs are a hybrid between shell jobs and manual
            jobs. They finish automatically, once triggered but still require a
            human to understand and follow test instructions and prepare the
            process. Instructions may range to getting a particular hardware
            setup, physical manipulation (pressing a key, closing the lid,
            plugging in a removable device) or talking to a microphone to get
            some sound recorded.

        .. note::
            The user may want to re-run the test a number of times, perhaps
            because there is some infrequent glitch or simply because he or she
            was distracted the first time it ran. Users should be given that
            option but it must always produce a separate result (simply re-run
            the same API again).
        """
        if job.plugin != "user-interact":
            raise ValueError("bad job plugin value")
        return MemoryJobResult({'outcome': IJobResult.OUTCOME_UNDECIDED})