Upgrading from UNE 10.04 to 10.10 (Unity) took about 6 hours - kernel problem with ahci mode?

[Ara Pulido]

Binary package hint: update-manager

I installed UNE 10.04.1 in a Dell Mini 9 (SSD drive) and just after that I upgraded to 10.10 (Unity)

The upgrade time took about 6 hours, most of the time was dpkg running (logs attached)

ProblemType: Bug
DistroRelease: Ubuntu 10.10
Package: update-manager 1:0.142.14
ProcVersionSignature: User Name 2.6.35-22.33-generic 2.6.35.4
Uname: Linux 2.6.35-22-generic i686
Architecture: i386
Date: Fri Sep 24 09:51:34 2010
InstallationMedia: Ubuntu-Netbook 10.04 "Lucid Lynx" - Release i386 (20100429.4)
PackageArchitecture: all
ProcEnviron:
 LANG=es_ES.UTF-8
 SHELL=/bin/bash
SourcePackage: update-manager

[Jean-Baptiste Lallement (jibel)]

Thanks for your report.

I'm unable to reproduce the problem in a VM (no ssd) where the upgrade runs in less than 1 hour, so this could be related to the hardware.
Could you please attach the output of dmesg just after the boot of your system to this report.
Thanks.

[Michael Vogt (mvo)]

It looks like this is releated to the hardware of the netbooks, I did a test upgrade yesterday with a atom based netbook and got a similar result, the upgrade took hours. While looking at the upgrade top reported a lot of idle and io-wait (20-40%).

[Michael Vogt (mvo)]

This was with the GUI upgrade (gnome). I will try a text-only upgrade next to see if this makes a difference (I doubt that).

[Michael Vogt (mvo)]

Note that this does not appear to be memory size releated, my netbook has a 2GB configuration and still is much slower upgrading than a VM with just 768MB.

[Jean-Baptiste Lallement (jibel)]

Can you give some metrics with the following commands:
$ echo 3 | sudo tee /proc/sys/vm/drop_caches; sudo dpkg -i /var/cache/apt/archives/ubuntu-docs_10.04.3_all.deb |ts
$ echo 3 | sudo tee /proc/sys/vm/drop_caches; sudo dpkg -i /var/cache/apt/archives/openoffice.org-common_1%3a3.2.1-6ubuntu1~10.04.1_all.deb |ts

(ts is in moreutils)

Can you try with ext3 instead of ext4 ?

[Michael Vogt (mvo)]

Here is the output on a atom based netbook with a regular 320GB WD hdd:

10.04, ext4, dpkg-1.15.5.6ubuntu4:
# echo 3 | sudo tee /proc/sys/vm/drop_caches; sudo dpkg -i /var/cache/apt/archives/ubuntu-docs_10.04.3_all.deb |ts
3
Sep 27 11:27:42 (Lese Datenbank ... 123948 Dateien und Verzeichnisse sind derzeit installiert.)
Sep 27 11:27:42 Vorbereiten zum Ersetzen von ubuntu-docs 10.04.3 (durch .../ubuntu-docs_10.04.3_all.deb) ...
Sep 27 11:27:42 Entpacke Ersatz für ubuntu-docs ...
Sep 27 11:28:22 Richte ubuntu-docs ein (10.04.3) ...
Sep 27 11:28:24

10.04, ext4, dpkg-1.15.5.6ubuntu4.3:
~# echo 3 | sudo tee /proc/sys/vm/drop_caches; sudo dpkg -i /var/cache/apt/archives/ubuntu-docs_10.04.3_all.deb |ts
3
Sep 27 11:35:36 (Lese Datenbank ... 123948 Dateien und Verzeichnisse sind derzeit installiert.)
Sep 27 11:35:36 Vorbereiten zum Ersetzen von ubuntu-docs 10.04.3 (durch .../ubuntu-docs_10.04.3_all.deb) ...
Sep 27 11:35:36 Entpacke Ersatz für ubuntu-docs ...
Sep 27 11:36:08 Richte ubuntu-docs ein (10.04.3) ...
Sep 27 11:36:10

[Michael Vogt (mvo)]

10.04, ext4, dpkg-1.15.5.6ubuntu4:
# echo 3 | sudo tee /proc/sys/vm/drop_caches | sudo dpkg -i /var/cache/apt/archives/openoffice.org-common_1%3a3.2.0-7ubuntu4.1_all.deb |ts
Sep 27 12:02:26 (Lese Datenbank ... 123948 Dateien und Verzeichnisse sind derzeit installiert.)
Sep 27 12:02:26 Vorbereiten zum Ersetzen von openoffice.org-common 1:3.2.0-7ubuntu4.1 (durch .../openoffice.org-common_1%3a3.2.0-7ubuntu4.1_all.deb) ...
Sep 27 12:02:27 Entpacke Ersatz für openoffice.org-common ...
Sep 27 12:03:39 Richte openoffice.org-common ein (1:3.2.0-7ubuntu4.1) ...
Sep 27 12:03:40
Sep 27 12:03:40 Verarbeite Trigger für man-db ...
Sep 27 12:03:44 Verarbeite Trigger für hicolor-icon-theme ...
Sep 27 12:03:48 Verarbeite Trigger für shared-mime-info ...
Sep 27 12:03:50 Verarbeite Trigger für desktop-file-utils ...
Sep 27 12:03:51 Verarbeite Trigger für python-gmenu ...
Sep 27 12:03:51 Rebuilding /usr/share/applications/desktop.de_DE.utf8.cache...
Sep 27 12:03:55 Verarbeite Trigger für python-support ...

10.04, ext4, dpkg-1.15.5.6ubuntu4.3:
# echo 3 | sudo tee /proc/sys/vm/drop_caches | sudo dpkg -i /var/cache/apt/archives/openoffice.org-common_1%3a3.2.0-7ubuntu4.1_all.deb |ts
Sep 27 11:57:19 (Lese Datenbank ... 123948 Dateien und Verzeichnisse sind derzeit installiert.)
Sep 27 11:57:19 Vorbereiten zum Ersetzen von openoffice.org-common 1:3.2.0-7ubuntu4 (durch .../openoffice.org-common_1%3a3.2.0-7ubuntu4.1_all.deb) ...
Sep 27 11:57:20 Entpacke Ersatz für openoffice.org-common ...
Sep 27 11:57:33 Richte openoffice.org-common ein (1:3.2.0-7ubuntu4.1) ...
Sep 27 11:57:34
Sep 27 11:57:34 Verarbeite Trigger für man-db ...
Sep 27 11:57:39 Verarbeite Trigger für hicolor-icon-theme ...
Sep 27 12:00:22 Verarbeite Trigger für shared-mime-info ...
Sep 27 12:00:24 Verarbeite Trigger für desktop-file-utils ...
Sep 27 12:00:25 Verarbeite Trigger für python-gmenu ...
Sep 27 12:00:25 Rebuilding /usr/share/applications/desktop.de_DE.utf8.cache...
Sep 27 12:00:30 Verarbeite Trigger für python-support ...

[Michael Vogt (mvo)]

Now with ext3:

10.04, ext3, dpkg-1.15.5.6ubuntu4:

# echo 3 | sudo tee /proc/sys/vm/drop_caches; sudo dpkg -i /var/cache/apt/archives/ubuntu-docs_10.04.3_all.deb |ts
3
Sep 27 15:35:11 (Lese Datenbank ... 125884 Dateien und Verzeichnisse sind derzeit installiert.)
Sep 27 15:35:11 Vorbereiten zum Ersetzen von ubuntu-docs 10.04.3 (durch .../ubuntu-docs_10.04.3_all.deb) ...
Sep 27 15:35:11 Entpacke Ersatz für ubuntu-docs ...
Sep 27 15:35:30 Richte ubuntu-docs ein (10.04.3) ...
Sep 27 15:35:32

10.04, ext3, dpkg-1.15.5.6ubuntu4.3:

# echo 3 | sudo tee /proc/sys/vm/drop_caches; sudo dpkg -i /var/cache/apt/archives/ubuntu-docs_10.04.3_all.deb |ts
3
Sep 27 15:42:21 (Lese Datenbank ... 125884 Dateien und Verzeichnisse sind derzeit installiert.)
Sep 27 15:42:21 Vorbereiten zum Ersetzen von ubuntu-docs 10.04.3 (durch .../ubuntu-docs_10.04.3_all.deb) ...
Sep 27 15:42:22 Entpacke Ersatz für ubuntu-docs ...
Sep 27 15:42:59 Richte ubuntu-docs ein (10.04.3) ...

10.04, ext3, dpkg-1.15.5.6ubuntu4:

# echo 3 | sudo tee /proc/sys/vm/drop_caches; sudo dpkg -i /var/cache/apt/archives/openoffice.org-common_1%3a3.2.0-7ubuntu4.1_all.deb |ts
3
Sep 27 15:36:51 (Lese Datenbank ... 125884 Dateien und Verzeichnisse sind derzeit installiert.)
Sep 27 15:36:51 Vorbereiten zum Ersetzen von openoffice.org-common 1:3.2.0-7ubuntu4 (durch .../openoffice.org-common_1%3a3.2.0-7ubuntu4.1_all.deb) ...
Sep 27 15:36:51 Entpacke Ersatz für openoffice.org-common ...
Sep 27 15:41:12 Richte openoffice.org-common ein (1:3.2.0-7ubuntu4.1) ...
Sep 27 15:41:13
Sep 27 15:41:13 Verarbeite Trigger für man-db ...
Sep 27 15:41:17 Verarbeite Trigger für hicolor-icon-theme ...
Sep 27 15:41:22 Verarbeite Trigger für shared-mime-info ...
Sep 27 15:41:24 Verarbeite Trigger für desktop-file-utils ...
Sep 27 15:41:24 Verarbeite Trigger für python-gmenu ...
Sep 27 15:41:24 Rebuilding /usr/share/applications/desktop.de_DE.utf8.cache...
Sep 27 15:41:29 Verarbeite Trigger für python-support ...

10.04, ext3, dpkg-1.15.5.6ubuntu4.3:
# echo 3 | sudo tee /proc/sys/vm/drop_caches; sudo dpkg -i /var/cache/apt/archives/openoffice.org-common_1%3a3.2.0-7ubuntu4.1_all.deb |ts
3
Sep 27 15:48:26 (Lese Datenbank ... 125884 Dateien und Verzeichnisse sind derzeit installiert.)
Sep 27 15:48:26 Vorbereiten zum Ersetzen von openoffice.org-common 1:3.2.0-7ubuntu4.1 (durch .../openoffice.org-common_1%3a3.2.0-7ubuntu4.1_all.deb) ...
Sep 27 15:48:26 Entpacke Ersatz für openoffice.org-common ...
Sep 27 15:48:45 Richte openoffice.org-common ein (1:3.2.0-7ubuntu4.1) ...
Sep 27 15:48:45
Sep 27 15:48:45 Verarbeite Trigger für man-db ...
Sep 27 15:48:51 Verarbeite Trigger für hicolor-icon-theme ...
Sep 27 15:48:56 Verarbeite Trigger für shared-mime-info ...
Sep 27 15:48:58 Verarbeite Trigger für desktop-file-utils ...
Sep 27 15:48:59 Verarbeite Trigger für python-gmenu ...
Sep 27 15:48:59 Rebuilding /usr/share/applications/desktop.de_DE.utf8.cache...
Sep 27 15:49:05 Verarbeite Trigger für python-support ...

[Michael Vogt (mvo)]

Raw performance of the hdd seems to be ok:
# hdparm -t /dev/sda

/dev/sda:
 Timing buffered disk reads: 156 MB in 3.02 seconds = 51.57 MB/sec

[UndiFineD (k.dejong)]

attached my logs as requested

[UndiFineD (k.dejong)]

apt-term.log

[UndiFineD (k.dejong)]

# uname -a
Linux ubuntu 2.6.31-14-generic-pae #48-Ubuntu SMP Fri Oct 16 15:22:42 UTC 2009 i686 GNU/Linux

# lsb_release -d
Description: Ubuntu maverick (development branch)

[Michael Vogt (mvo)]

The data is pretty interessting, when comparing ext4 vs ext3 with the dpkg from lucid-proposed its a difference of "3:11min" vs "00:39sec" for openoffice.org-common. That does explain the slowness in general, but it seems to be disproportional on the atom based systems.

[Michael Vogt (mvo)]

On my system at least something diferent is going on (its a lenovo s12). I can upgrade openoffice.org-common on ext4 with dpkg from lucid-proposed in ~40s (stopwatch) *if* I constantly press a key. If I don't do that it takes about 6:30min(!)

[Michael Vogt (mvo)]

In my VM (kvm, 768mb) with new-dpkg openoffice.org upgrades in 42s.

[Michael Vogt (mvo)]

I had to use a stopwatch, the systemclock is not reliable on the s12 when io is happening. It considerable slowed down.

[Michael Vogt (mvo)]

The effect (keys vs no-keys) is 100% reliable for me. When I run strace it goes away too.

[Michael Vogt (mvo)]

The puzzle seems to be solved (at least for the s12). When I switch the S12 from "AHCI" mode to "compat mode " (in the bios) I get 30s OOo upgrade times and no keypressing needed anymore.

I wonder if the mini9 (who presumably has the same chipset) has also a AHCI set in the bios as sata mode?

[Jean-Baptiste Lallement (jibel)]

That really seems hardware specific.
On an Asus 1015PE (Atom N450), I get 11s for the unpack stage of OO.o-common on ext4 in AHCI mode.

[Michael Vogt (mvo)]

Thanks for confirming Jean-Baptiste - the S12 has a Intel Atom N270 processor, a N10 chipset and a ICH7 IDE controller. So my test data is moot on most system (unless e.g. the mini9 shows similar symptoms).

[Ara Pulido (ara)]

The mini9 uses an Intel i945 chipset, and the AHCI option is not in the BIOS (dell disabled it??).
Anyway, my times (using a stopwatch) for upgrading openoffice.org-common takes 1min28s

[Jean-Baptiste Lallement (jibel)]

Switching to AHCI on a SSD doesn't makes sense because one of the feature of ahci is NCQ (Native Command Queuing) which optimize the way the heads are moving on the hd plates. It (should) greatly increases the performance of standard rotational drives but it has no bearing on SSDs.

I think that michael's and ara's issues are different here and we need advice from a kernel expert.

[sam tygier (samtygier)]

looks like bug #643822

using nolapic_timer as a boot argument makes it better