Doesn't launch in <5 seconds on benchmark hardware

[Matthew Paul Thomas]

1. Start Ubuntu.
2. Launch Ubuntu Software Center.

During user testing of Unity in Ubuntu 10.10, Ubuntu Software Center took between 8 and 20 seconds to launch on a Dell Inspiron Mini 10. (It took 52 seconds to appear for one participant, but that was because of a bug in Unity's Workspaces view.) <https://blueprints.launchpad.net/ubuntu/+spec/unity-usability-testing>

Twenty seconds is a long time. So, let's set a target: On benchmark hardware, on a cold launch, the Ubuntu Software Center window should appear within five seconds.

This does not mean the software catalogue needs to have loaded by that time. The main pane might show a spinner (or, even better, a progress bar) until the catalogue is ready.

And it does not mean all your software channels need to be known by that time. The navigation pane might show just the top-level items (with disclosure triangles disabled) until it knows what the children are.

All it means is that the window appears within five seconds. Like it does for Calculator, or Character Map, or Disk Usage Analyzer, or Gnome Terminal.

This bug will be fixed when:
1. we know, and publish, what "benchmark hardware" means, exactly; (DONE - Dell Mini 10)
2. every day, or every revision of trunk, an automated test measures the time from cold launch to the window opening on that benchmark hardware; (DONE)
3. the results are publicly accessible, preferably as a graph; (DONE - <http://people.canonical.com/~mvo/software-center/mini10-startup/startup-times.png>)
4. the graph shows launch to open window taking less than five seconds (DONE).

[Michael Vogt (mvo)]

What hardware was this ? We selected a dell mini 10 as the reference hardware and we are working towards improving the startup time (and general performance improvements). Initial graphs are here:
http://people.canonical.com/~mvo/software-center/mini10-startup/startup-times.png
(note that this is with a warm hdd cache).

A 60s startup sounds like something is seriously wrong.

[Matthew Paul Thomas (mpt)]

Seeing the improvement on the current graph is awesome. However, most USC launches will be cold rather than warm, so I think we really should start measuring cold launch time instead. How can we do that in a non-tedious way?

[Martin Pitt (pitti)]

Suggestion for measuring with cold cache:

# first, drop all caches
echo 3 | sudo tee /proc/sys/vm/drop_caches
# restart gnome-panel and nautilus to reload most of GNOME/gvfs/GTK/libc/etc bits back into cache:
nautilus -q
setsid gnome-panel --replace

[Michael Vogt (mvo)]

A cold cold cache is going to be a problem on the mini.

$ echo 3 | sudo tee /proc/sys/vm/drop_caches
$ nautilus -q ; setsid gnome-panel --replace; time python -c 'import gtk'

real 0m4.541s
user 0m0.328s
sys 0m0.108s

Just importing "gtk" from python may take 4.5s already.

Loading python is more realistic as system-config-printer is python and reloaded.
$ python -c 'import sys';killall gnome-panel;nautilus -q ; setsid gnome-panel --replace; time python -c 'import gtk'

real 0m2.759s
user 0m0.288s
sys 0m0.048s

[Gary Lasker (gary-lasker)]

It seems like we may at this point be under the 5 second target for benchmark hardware (software-center trunk at revision 1416 on mvo's Dell Mini 10). Once we get some reasonable measurements on the Mini under cold-cache conditions (per comments above) we can make the determination.

A graph of the current state of startup times can be viewed at the usual place:

  http://people.canonical.com/~mvo/software-center/mini10-startup/startup-times.png

On my own Dell Mini 9 (with SSD), I am seeing the following:

  Warm caches: 1.75s
  Using pitti's script (comment #3): 3.2s
  drop_caches and nothing more: 3.7s

Note that on my Dell laptop (2.1Ghz Core 2 Duo and SSD) I regularly get startup times of less than .5s with warm caches. Before we started this effort it was just over 3 seconds. Needless to say it feels quite speedy now. ;)

[Gary Lasker (gary-lasker)]

The startup time has not increased appreciably since my most recent comment, even with the many additional new features that have been added to Software Center since that time. As always, startup time results for the Dell Mini 10 reference machine can be viewed here:

  http://people.canonical.com/~mvo/software-center/mini10-startup/startup-times.png

The goal of obtaining consistent measurements in a testing environment that reflect actual startup times as experienced by users under real-world conditions has proven to be a bit elusive as it depends on many factors. The fact is, the perception and impact of startup time in a user's experience is a subjective thing and what is actually important is how it feels for the user. Is startup so slow that a person tries to click to launch again? Is it just plain frustrating waiting for the damned app to finally launch? Both of these were clearly the case in Maverick's Software Center.

However, when I launch Software Center on my Dell Mini running Maverick vs. my Dell Mini running current Natty, the extent of the improvement that has been made over the course of this cycle becomes quite dramatically apparent to me. It starts up fast now, faster in fact than a lot of other apps that I use all the time.

The improvement on my everyday machine (Dell laptop with 2.1Ghz Core 2 Duo and SSD) has been equally dramatic, with startup reduced from over 3 seconds to a current .5 second and holding. At half a second to launch, it feels instant.

So, based on the significant speedup as displayed in the graph over the course of the cycle so far, and based on my observations here and in comment #5, I will close this bug as fixed at this point. At the start of the Oneiric cycle, let's take another baseline and hopefully we can do another round of user testing to see we need to make further improvements at that time.

Meanwhile, we will continue to monitor startup time through the rest of Natty and into the future, using the tools and processes we developed for this bug. If there are regressions in startup time, we will see them immediately, and we can address them immediately. We will also continue to make further reductions in startup time as we identify additional strategies to do so. Keeping Software Center starting fast is unavoidably an ongoing process.

A sincere thank you and kudos to all the folks who were involved in this effort for your help and your good work!

[Gary Lasker (gary-lasker)]

Assign to me for spec tracking.

[peterzay (peterzay)]

On 11.10 Oneiric, software-center takes way too long to load.

Note that it was less bad on 11.04 Natty.

[Matthew Paul Thomas (mpt)]

(The next phase of the launch performance campaign is bug 1021779.)