Provide script to set up a temporary user session

Bug #1376423 reported by Leo Arias on 2014-10-01
18
This bug affects 3 people
Affects Status Importance Assigned to Milestone
Autopilot
Undecided
Unassigned
Ubuntu Touch session manager
New
Undecided
Unassigned

Bug Description

For manual and automated testing we generally need to be able to start the applications in a clean environment, in a way that the tests are not affected by existing files and configurations. And as the tests should be easily runnable on the devs development machines, the changes made while running the tests should not affect the real environment.

In the past we were able to get a reasonably clean environment setting the value of $HOME to a temporary directory.

Now this is not enough by a variety of reasons, like a hierarchy of dirs in HOME needed by app armor, apps being launched by upstart, and services that keep listening to the previous value of HOME.

Every test suite has solved this on it's own why, some use the real environment but backup the existing files (which is really bad), some patch the HOME and keep adding details every time something changes, and some others just pollute the real environment and can't be run on a dev machine. This is what has been causing failures on the majority of autopilot tests for the last month.

Instead of keep doing these multiple solutions, and keep adjusting them every time they stop working; I think we need to find one single way that works for all the apps. And we need to add automated tests for this single way to make sure that no new releases will break it. I'm currently thinking of a testability helper in ubuntu-app-launch that we call to launch an app and it will make sure that it will start all the needed XDG vars pointing to a clean temporary environment. We might still need to restart some services, and we might hit some other problems, but this is just an idea. Feel free to propose alternatives.

Leo Arias (elopio) wrote :
Download full text (16.2 KiB)

<elopio> ping ted.
<elopio> do you have some time to talk again about a temp home dir?
<elopio> https://bugs.launchpad.net/ubuntu-app-launch/+bug/1376423
<ubot5> Ubuntu bug 1376423 in Ubuntu Application Launcher "There is no easy and future-proof way of starting an app in a clean environment" [Undecided,New]
<ted> pong elopio
<elopio> ubuntu-qa, thomi and balloons: ^ feel free to also comment that bug.
<ted> I kinda disagree with your statement there.
<ted> "In the past we were able to get a reasonably clean environment setting the value of $HOME to a temporary directory."
<ted> That was always a bad idea.
<ted> elopio, Why can't you just wipe the device for a clean environment?
<elopio> ted: that's clear for me now.
<ted> That's really the only way it's really "clean"
<elopio> ted: we are not always testing on phones, we also need to test on dev machines.
<dobey> hmm
<elopio> ted: and some of the devices we are using are also the ones we dogfood.
<ted> elopio, Okay, in a container.
<elopio> it's bad to have to reset them every time.
<ted> elopio, dog fooded devices can't ever be clean.
<elopio> ted: a container can be a good solution.
<ted> I'm hoping people don't use that for real testing.
<elopio> ted: not for something like running the whole suite to decide if a version is promoted or not.
<elopio> but we need to use the dogfooding devices to check that a new test we added works.
<elopio> or to debug a test that started failing.
<elopio> otherwise we would need two devices. Of every model.
<ted> But, you'll need to wipe it to land the silo with the new test, no?
-*- balloons settles in
<ted> So you kinda need a device you can wipe of every model you care about.
<elopio> ted: that's for the silo testers.
<ted> Don't we silo MRs for tests?
<elopio> they are not necessarily the same as the dogfooders or automation devs.
<balloons> ted, we do need a way to run tests sanely as a developer or test writer too
<ted> Sure, but those folks don't need a clean test environment either.
<elopio> ted: I didn't get that last question. But I think you would agree that any solution we find, needs to be easily runnable from a dev machine. Do we agree there?
<balloons> but you can argue about whether the test or the test harness should prep the environment
<ted> I'm just saying that we're not going to get "clean" on a device that isn't wiped.
<ted> It's always going to be sullied in some way.
<elopio> ted: maybe we could get a container on a device that is not wiped
<ted> Or there's enough of a chance that it could be that it's not a "result" just a checkup.
<balloons> at the very least the test shouldn't be polluting, while still allow us to setup at least the parts we are responsible for correctly
<elopio> and generally we will run on an unwiped device just a small subset of tests.
<elopio> maybe for that case we don't need a perfectly clean environment. Just something that doesn't delete all the photos I took yesterday.
<ted> Sure, the testing shouldn't be destructive.
<dobey> elopio: can you not run the tests in a schroot?
<elopio> ted: but we need to test the behavior of the gallery app without existing tests. So we need to have an environment that temp...

Leo Arias (elopio) wrote :

That is a long log of the discussion we started on oct 1st, 2014.

Ted Gould (ted) wrote :

So I don't think there's any UAL work required here. So going to mark as invalid. Feel free to add other bug tasks as needed.

Changed in ubuntu-app-launch:
status: New → Invalid
Martin Pitt (pitti) wrote :

Closing this as invalid seems a bit premature to me. If possible, a test should never mess with a production $HOME, but always use a temporary one. It's safer, allows running the test on a production machine like a developer's workstation, and also provides better isolation. If your answer is "well, then always run it on a throwaway phone", I really doubt that you run your tests very often, as it's just way too inconvenient to do so.

It's not that hard to set up a temporary session d-bus, upstart, env variables, and a temp $HOME to run tests in. autopkgtest has a script to do just that (http://anonscm.debian.org/cgit/autopkgtest/autopkgtest.git/tree/setup-commands/ubuntu-touch-session) and it's working quite well. But it would be nicer to move that to a place where upstream tests can actually use it, instead of reinventing the wheel every time (and while the resulting script looks quite simple, it really took me some two hours to figure this out, as it's not really well documented).

I'm reopening for these reasons. If you still disagree, mark it as "wontfix" for U-A-L and we'll find another place, but I heavily disagree on "invalid". It's a very valid concern.

Changed in ubuntu-app-launch:
status: Invalid → New
summary: - There is no easy and future-proof way of starting an app in a clean
- environment
+ Provide script to set up a temporary user session
Martin Pitt (pitti) wrote :

I forgot to comment about creating a new user: This is the difference between "the test needs root privileges and is even more invasive to the system" and "the test can run with normal user privileges and does not leave permanent clutter behind". I. e. creating a new user introduces the obligation to ensure that it always gets removed again, even if the test or test runner fails in the middle of the run.

Also, doing so doesn't actually help at all with the main issue here to centralize the knowledge how to start an UAL session: We need to do that regardless of whether this happens for a "real" user with a temp $HOME or for a temp user.

So in summary, creating temp users only makes the test bed setup more complicated, less convenient, and potentially harmful (due to requiring root privs) without gaining much.

Changed in ubuntu-app-launch:
assignee: nobody → Martin Pitt (pitti)
Nicholas Skaggs (nskaggs) wrote :

Meeting with Martin, Leo, Ted et la, we arrived at the following conclusions:

- Migrate test setup complexity to a common fixture within autopilot. Tests should use this fixture.
- Create script that will setup environment and make it available in a common component (ubuntu-touch-session, ubuntu-app-launch, autopilot)
- Add autopkgtest for setup script to ensure any changes to setup assumptions are caught before release

Martin has agreed to begin research on what the script needs to provide. Once it is written, we need a test with it's assumptions to prevent future breakage, and a fixture in autopilot we can import inside testcases. Autopilot launcher methods might potentially need tweaking as well.

Nicholas Skaggs (nskaggs) wrote :

As part of the work in closing this bug, the click.rules file should be modified and cleaned up to provide only the basic introspection needs. See https://bugs.launchpad.net/autopilot/+bug/1379488

Ted Gould (ted) wrote :

I think that this should probably go in the touch-session package as that's where the other parts of the session are.

affects: ubuntu-app-launch → ubuntu-touch-session
Martin Pitt (pitti) on 2015-09-01
Changed in ubuntu-touch-session:
assignee: Martin Pitt (pitti) → nobody
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers