FFE Request for Upstart in raring

[James Hunt]

= Overview =

This bug documents a number of new features we'd like to introduce to the upstart package for raring.

= New Features =

== Enabling of User Sessions ==

https://wiki.ubuntu.com/FoundationsTeam/Specs/RaringUpstartUserSessions

=== Rationale ===

Upstart 1.7, which is already in the archive, provides enhanced user sessions. These offer a wealth of benefits as listed on:

http://upstart.ubuntu.com/cookbook/#session-init

Further, they are planned for inclusion in raring at the request of the Desktop Team (seb128 is a prime contact).

=== Status ===

The upstart package has now been modified fully to support user sessions, but there are a few remaining desktop packages that still need work before sessions can be enabled:

 D-Bus: stgraber has a branch in proposed with the required /usr/share/upstart/sessions/dbus.conf job file.
- ssh-agent: TODO.
- im-switch: TODO.

Once the final elements are in place, the default 'ubuntu' desktop will migrate from being managed by gnome-session to being manged by Upstart. This will require final testing by both the Foundations and Desktop teams.

The actual change necessary to enable the User sessions is to uncomment 'ubuntu' in file /etc/upstart-xsessions (which is already in the upstart package).

=== Risk ===

LOW/MEDIUM - although well tested by the Foundations Team who have been running desktop sessions managed by upstart for some weeks now, once the final elements are ready, another round of testing will be required by both the Foundations and Desktop teams.

=== Contingency ===

Revert to the gnome-session-managed desktop. However, this is potentially hampered by bug 1131968.

== New Upstart Bridge: upstart-file-bridge ==

The upstart-file-bridge is a now process that will be run both as a system job (as root), and as a user session job (1 per Upstart Session).

=== Rationale ===

The upstart-file-bridge provides the ability for Upstart jobs to 'start on' actual files in the filesystem being created, modified or deleted. This allows some of what are currently long-running daemons to just run 'on demand'. A good example of this is whoopsie: currently it runs all the time, but with the file bridge its Upstart job configuration file can be modified to be:

    start on file FPATH=/var/crash/*.crash FEVENT=create
    exec whoopsie

This will have the effect of only starting whoopsie iff and when crash files are created. As such, this is a valuable tool in our ongoing efforts to reduce memory consumption and speed up the boot.

=== Status ===

Branch lp:~jamesodhunt/upstart/file-bridge-MP is currently awaiting review to be merged into upstream Upstart. When it passes review and is merge into upstream, the intention is to cherry-pick it into Ubuntu.

=== Risk ===

LOW - this is a new facility. If it does not work, any application that makes use of it may not start, but can still be started manually should the need arise. Additionally, this bridge is not running within init itself so any instability will not affect system reliability.

=== Contingency ===

Disable the bridge if it proves problematic and revert applications such as whoopsie to running all the time.

== Upstart GUI: upstart-monitor ==

The upstart-monitor is a simple python application that shows upstart events (both system-level and session-level) as they occur.

=== Rationale ===

Now that Upstart is able to manage a user sessions, non-privileged users will potentially be able to explore Upstarts benefits for themselves without having to worry about destablising their systems. This is a huge opportunity for Upstart and any tooling that makes it easier to use should be considered IMHO.

This tool should be extremely useful to newcomers to Upstart and even to experienced admins who wish to observe event flows. It also makes it significantly easier to write custom session jobs.

=== Status ===

Branch lp:~jamesodhunt/upstart/upstart-monitor contains the latest code. It is 98% (still requires install logic for the icon/.desktop file). The intention is to raise an MP on this for lp:upstart within the next few days. Once approved, it will be merged into lp:upstart then cherry-picked into Ubuntu.

=== Risk ===

LOW - this tool runs as a non-privileged user and does not provide any more information than can currently be ascertained using existing tooling - it just puts it into a familiar form.

=== Contingency ===

Disable/remove the tool.

== Apport hook ==

=== Rationale ===

- Upstart currently has no apport hook.
- We intend to enable User Sessions which expands the scope for upstart so although we have tested the code as thoroughly as possible, if problems occur, an apport hook will be invaluable to simplify the job of both reporting the issue and diagnosing them.

=== Status ===

The hook is in branch lp:~jamesodhunt/ubuntu/raring/upstart/add-apport-hook. It is in the process of being tested.

=== Risk ===

LOW - this hook is not part of the init daemon itself and is only invoked when Upstart crashes.

=== Contingency ===

Even if this apport hook crashes, apport still allows a problem to be reported; it just won't have all the extra attachments the hook is designed to capture.

== DEP-8 tests: libtestupstart ==

=== Rationale ===

The current autopkgtest tests simply run the unit tests. The intention is to add a python module that will allow us to write runtime tests easily and have them run prior to an upload hitting the archive.

=== Status ===

I'm currently working on a libtestupstart.py module with is working in a very basic fashion.

=== Risk ===

LOW - if libtestupstart.py itself fails, any DEP-8 tests using it can be disabled to return us to the status quo.

=== Contingency ===

Disable any DEP-8 tests using libtestupstart.py

[Scott Kitterman (kitterman)]

Do user sessions then get used everywhere or only on the Ubuntu desktop?

[Iain Lane (laney)]

I thought we agreed to have the upstart sessions on but disabled by default, as they are currently.

If they are to be enabled, here's an issue that I know of.

The current implementation shadows all Xsession.d files - that means gpg-agent, ssh-agent and im-config on my system. They're shortcircuited out and therefore don't run in the session. Ideally they would be managed by jobs instead, but that is not the case currently.

I believe a simple fix to that would be to move Upstart to 55upstart instead of 99upstart - that's after the needed 55gnome-session_gnomerc but before the others.

Then we'd also need to modify dbus to be a noop if $UPSTART is set.

The rest of the changes sound alright.

[Iain Lane (laney)]

s/dbus/dbus's Xsession.d script/

[Stéphane Graber (stgraber)]

I think this proposal covers user sessions only for the "ubuntu" session (unity).
All the work so far was done only with Unity and minimal testing was also done with gnome-fallback and gnome-shell. In theory any session based on gnome-session should work too.

Now as to the FFe, I think we should first make sure user sessions are completely regression free before discussing about enabling them by default. At the moment we're aware of at least two problems, one is the input method integration, the other is ssh-agent.

I suspect there are a few more of those and I think it's reasonable to grant a freeze exception to get those resolved (by updating their Xsession script and adding a matching user job). This should be checked against the whole archive (archive scan for /etc/X11/Xsession.d/*) and only when we know they're all fixed, should we start discussing making user sessions the default.

The current implementation offers the advantage of making possible to do all this preparation work directly in the archive without any real risk, make it very easy for people interested in it to test and report bugs against the current implementation so when we finally consider enabling the feature by default, we'll know it'll be almost risk free.

[James Hunt (jamesodhunt)]

As Stéphane says, this proposal indeed only relates to the 'ubuntu' Unity session.

[Scott Kitterman (kitterman)]

OK. It does seem rather late to be doing this by default.

[Steve Langasek (vorlon)]

Note that bug #1131968 is not an issue, as the implementation that landed in raring reuses the existing 'Ubuntu' session rather than registering a separate 'Ubuntu (upstart)' session. So turning this on or rolling it back both requires a simple change to /etc/upstart-xsessions.

[Iain Lane (laney)]

I think on balance it's probably better to keep it off by default for Raring. It's easy for people to turn it on if they want it to test and/or develop user jobs. Seems like an ideal thing to enable right away in S-cycle.

The rest is fine IMO. I'm happy to approve that as long as we keep an eye on stability and execute the reversion plans swiftly if necessary.

[Stéphane Graber (stgraber)]

Quick update on this one. I uploaded an updated dbus and im-config this morning which will make the input method daemon (ibus) work properly under use sessions.

I also opened a Debian bug against openssh to include a user job for the ssh-agent. Colin has now landed this in the Debian bzr branch and we expect this to be synced in Ubuntu by Thursday.

With those bits in and with upstart 1.8 being almost ready for upload (assuming the paperwork is in order, I haven't checked), that should be all we need to have optional well-working user sessions for 13.04.

I'll then start looking into all the other (non-default) Xsession scripts for anything else that needs porting, but as far as the default Ubuntu desktop is concerned, we should be all good now.

[James Hunt (jamesodhunt)]

Although the features listed on this bug are correct, the focus has changed slightly now that they are part of a new upstream release (upstart 1.8).

[James Hunt (jamesodhunt)]

The unit tests run as part of the build. I've also install tested and downgrade/upgrade tested the package.

[Stéphane Graber (stgraber)]

Upstart 1.8-0ubuntu1 has been uploaded to raring and will be available in a few minutes.
Additionally my ssh-agent job has been uploaded to Debian and synced to Ubuntu yesterday, so we're now all good for 13.04 with all features of a standard Ubuntu Desktop install being covered by user sessions.