eBox 2300 boots VERY slow with Ubuntu/LTSP-5

[Jim McQuillan]

The eBox e2300 thin client is a very cheap small thin client. Around $100 in large quantities. This unit is becoming very popular. The problem is, it boots VERY slowly in Ubuntu's LTSP on feisty.

I'm talking SLOW. With LTSP-4.2 installed on feisty, it boots in 41 seconds, all the way from hitting the power button, to getting a login prompt. That's pretty respectable. But, in feisty, it takes 4 minutes and 43 seconds to boot !!! That's 41 seconds vs 281 seconds.

A faster client, the T-1220 from DisklessWorkstations.com takes 32 seconds to boot in LTSP-4.2 and 94 seconds to boot with Ubuntu/LTSP-5. 94 seconds is pretty slow, but LTSP-5 is doing alot more work, so it's understandable.

281 seconds isn't understandable, and people will be angry if they buy a classroom full of e2300's and it takes that long to boot. This will cause people to recommend k12ltsp over Edubuntu, because k12ltsp is still using LTSP-4.2.

LDM is a big part of the speed problem, but it's not the ONLY problem. If I set SCREEN_07 = startx, to use xdmcp, instead of LDM, it reduces the boot time considerably. From 281 seconds down to 171 seconds. That's still too slow.

Another speed issue is the amount of time it takes to log out, and get a login box to appear on the screen. when using XDMCP, with either LTSP-4.2 or LTSP-5, it takes only about 8-10 seconds. when using LDM, it takes 106 seconds. That's approaching 2 minutes to get a login screen after logging out. Even the faster workstation takes 28 seconds to get the login screen after logging out. Back on LTSP-4.2, we could boot a whole workstation in the same amount of time it takes just to re-launch LDM on ltsp-5.

I've tried not to complain too much about boot times between LTSP-4.2 and Ubuntu's LTSP, but I really believe that now it's a serious issue that will turn people away if we don't come up with a solution.

[Oliver Grawert (ogra)]

could you also attach an xorg.conf from a running client and the snippets of teh servers syslog that are written by the client ?

[Jim McQuillan (jam-mcquil)]

I used the ubuntu kernel source for 2.6.20, and built a kernel for LTSP-4.2. I modified the .config extensively, turning off an awful lot of stuff that a thin client doesn't need. I probably turned off too much stuff, but it's enough to prove that the 2.6.20 will boot with ltsp-4.2 AND, it boots fast.

The interesting thing to look at is around line 96 (13.942148 secs) where the initramfs is unpacked. In the kernel supplied with Ubuntu/LTSP-5, that event doesn't happen until 49 seconds into the boot. You can see that in the syslog that I attached earlier.

So, why does my ltsp-4.2 kernel get to that point 36 seconds faster?

Is there some settings in the ubuntu kernel config that the SiS CPU doesn't like?

Next thing I want to try is building a kernel for Ubuntu/LTSP-5 using my .config file, to see how well it performs.

And certainly, more comparison between the 2 syslog files is required.

[Jim McQuillan (jam-mcquil)]

This is the .config file that I used to build a 2.6.20 kernel for LTSP-4.2

I'd like to build an Ubuntu/LTSP-5 kernel using this config file.

[Jim McQuillan (jam-mcquil)]

Looking at the 2 syslog files (ltsp-5 vs ltsp-4.2), I found the 36 second difference at line 41. the LTSP-4.2 kernel that I compiled jumps from 0.000000 to 12.826513, while the LTSP-5 Ubuntu supplied kernel jumps from 0.000000 to 46.854847.

/me wonders why my kernel pauses only 13 seconds, while the Ubuntu/LTSP-5 kernel pauses 46 seconds.

[Oliver Grawert (ogra)]

i went through the initscripts and moved all slowing down parts we dont need to bring up X after the screenscript startup (localdev, printing, sound) so that should speed up minimally to present the screen earlier to teh user. additionally i found a stray sleep(5) command in ldm which i dropped, this should gain another slim speedup.

for the kernel issues and other general profiling we urgently need to schedule a profiling session for ltsp at UDSSevilla

please run the test again with a freshly built chroot from the following version of ltsp:

 ltsp (5.0.5) feisty; urgency=low
 .
   * move printserver startup, localdev helperscripts and soundserver
     startup after start of X to speed up booting (should partially
     help with LP #97456)
   * add pulseaudio-esound-compat to ltsp-server-standalone recommends
     (for people feeling the need to still use esddsp with apps)
   * drop duplicated module-detect option from pulseaudio startup to
     avoid warning in client bootlogs
   * drop duplicated touching of /etc/nbd-client from ltsp-build-client,
     we start nbd from the ltsp-client-setup initscript, no need for a
     second run.
   * remove stray sleep(5) call in ldm for speedup (LP #97456)
   * add --copy-sourceslist option to ltsp-build-client to reuse the servers
     sources.list with ltsp-buiuld-client (fixes LP #48601)
   * move loopback device entry creation in /etc/network/interfaces to
     ltsp-build-client, fixes a race condition while bringing up lo

[Jim McQuillan (jam-mcquil)]

I rebuild the chroot based on ogra's latest updates, and it managed to shave a whopping 10 seconds off the boot time. So now, instead of 4mins 45secs, it boots in 4mins 35secs. I'd say we still have some shaving to do.

[Oliver Grawert (ogra)]

now on to the real ldm speedup:

sudo chroot /opt/ltsp/i386 apt-get remove xfonts-75dpi xfonts-100dpi xfonts-scalable

try that one and look if that speeds up ldm

[Oliver Grawert (ogra)]

as confirmed on IRC that speeds up a bit, added to the package:

 ltsp (5.0.6) feisty; urgency=low
 .
   * move printing back, doesnt gain us anything
   * make sure the localdev scritps dont start new instaces with every ldm
     restart on logout
   * fix quoting in sources.list plugin
   * change package selection to not install *all* fonts, we only need the
     ones ldm uses (should help massively with bug #97456)
   * fix yaboot linking in ltsp-update-kernels (closes LP #101927)

[Gavin McCullagh (gmccullagh)]

would there be any benefit in generating bootcharts of the e2300 and T-1220 to try and pinpoint where the slowdowns are? I put together a short howto here:

http://www.hamilton.ie/gavinmc/edubuntu/bootchart.html

[Jim McQuillan (jam-mcquil)]

Gavin,

it's all been done already and Ollie and Scott have made great progress
in getting the boot speed fixed.

I think they've got the boot down to about 90 seconds now, which is
quite a bit better than it was. They've still got a few tricks up their
sleeves, so I expect it to get better yet.

Thanks,
Jim.

gavinmc wrote:
> would there be any benefit in generating bootcharts of the e2300 and
> T-1220 to try and pinpoint where the slowdowns are? I put together a
> short howto here:
>
> http://www.hamilton.ie/gavinmc/edubuntu/bootchart.html
>

[Gavin McCullagh (gmccullagh)]

That's great news. I look forward to a slicker ltsp5 in gutsy then :-)

[erthourcery (gregg-opensourcery)]

On Ubuntu 8.04 w/ LTS 5.0.40, servicing series 2300 eBox's, default-install boot times are still upwards of three full minutes. Comment 17 hints that improvements have been made, but I'm not seeing them - is there something non-default which needs to be enabled and/or installed?

The most profound slowdowns appear to be:
Oct 12 18:17:44 nano dhcpd: DHCPOFFER on 192.168.1.102 to 44:4d:50:e2:6a:5c via eth1
Oct 12 18:17:45 nano dhcpd: DHCPREQUEST for 192.168.1.102 (192.168.1.1) from 44:4d:50:e2:6a:5c via eth1
Oct 12 18:17:45 nano dhcpd: DHCPACK on 192.168.1.102 to 44:4d:50:e2:6a:5c via eth1
Oct 12 18:18:25 nano dhcpd: DHCPOFFER on 192.168.1.102 to 44:4d:50:e2:6a:5c via eth1
Oct 12 18:18:25 nano dhcpd: DHCPREQUEST for 192.168.1.102 (192.168.1.1) from 44:4d:50:e2:6a:5c via eth1
Oct 12 18:18:25 nano dhcpd: DHCPACK on 192.168.1.102 to 44:4d:50:e2:6a:5c via eth1
Oct 12 18:18:26 nano dhcpd: DHCPREQUEST for 192.168.1.102 (192.168.1.1) from 44:4d:50:e2:6a:5c via eth1
Oct 12 18:18:26 nano dhcpd: DHCPACK on 192.168.1.102 to 44:4d:50:e2:6a:5c via eth1
Oct 12 18:18:26 nano nbdrootd[6698]: connect from 192.168.1.102 (192.168.1.102)
Oct 12 18:18:26 nano nbd_server[6699]: connect from 192.168.1.102, assigned file is /opt/ltsp/images/i386.img
Oct 12 18:20:06 192.168.1.102 syslogd 1.5.0#1ubuntu1: restart.
Oct 12 18:20:21 nano ldminfod[6776]: connect from 192.168.1.102 (192.168.1.102)
Oct 12 18:22:52 192.168.1.102 kernel: Inspecting /boot/System.map-2.6.24-19-generic
Oct 12 18:38:04 192.168.1.102 kernel: Loaded 27884 symbols fro3m /boot/System.map-2.6.24-19-generic.
Oct 12 18:38:09 192.168.1.102 kernel: Symbols match kernel version 2.6.24.
Oct 12 18:38:14 192.168.1.102 kernel: Loaded 10529 symbols from 49 modules.
Oct 12 18:38:19 192.168.1.102 kernel: [ 0.000000] Initializing cgroup subsys cpuset
Oct 12 18:38:24 192.168.1.102 kernel: [ 0.000000] Initializing cgroup subsys cpu
Oct 12 18:38:29 192.168.1.102 kernel: [ 0.000000] Linux version 2.6.24-19-generic (buildd@terranova) (gcc version 4.2.3 (Ubuntu 4.2.3-2ubuntu7)) #1 SMP Wed Aug 20 22:56:21 UTC 2008 (Ubuntu 2.6.24-19.41-generic)
Oct 12 18:38:34 192.168.1.102 kernel: [ 0.000000] CPU: Vendor unknown, using generic init.
Oct 12 18:38:39 192.168.1.102 kernel: [ 0.000000] CPU: Your system may be unstable.
Oct 12 18:38:44 192.168.1.102 kernel: [ 0.000000] BIOS-provided physical RAM map:
Oct 12 18:38:49 192.168.1.102 kernel: [ 0.000000] BIOS-e820: 0000000000000000 - 000000000009fc00 (usable)
Oct 12 18:38:54 192.168.1.102 kernel: [ 0.000000] BIOS-e820: 000000000009fc00 - 00000000000a0000 (reserved)
Oct 12 18:38:59 192.168.1.102 kernel: [ 0.000000] BIOS-e820: 00000000000ec000 - 0000000000100000 (reserved)
Oct 12 18:39:04 192.168.1.102 kernel: [ 0.000000] BIOS-e820: 0000000000100000 - 0000000007800000 (usable)
Oct 12 18:39:09 192.168.1.102 kernel: [ 0.000000] BIOS-e820: 00000000ffe80000 - 00000000ffef0000 (reserved)
Oct 12 18:39:14 192.168.1.102 kernel: [ 0.000000] BIOS-e820: 00000000fffc0000 - 0000000100000000 (reserved)
Oct 12 18:39:19 192.168.1.102 kernel: [ 0.000000] 0MB HIGHMEM available.
Oct 12 18:39:24 192.168.1.102 kernel: [ 0.000000] 120MB LOWMEM available.
Oct 1...

[Oliver Grawert (ogra)]

i think 3 minutes vs 5-8 minutes is an improvement ... you wont get out much more of that hardware with a standard 2.6 kernel

[erthourcery (gregg-opensourcery)]

I'm sorry, comment #17 left me with the impression that boot-times could be reduced to as low as ~90seconds...my mistake.

[LaserJock (laserjock)]

Oli, is the status on this bug right (In Progress)?

[Stéphane Graber (stgraber)]

What's the status of this bug ?
I see mention of Feisty and Gutsy which are both non-supported release.

Is there an actual bug with that thin client running on recent LTSP that we could try to fix in LTSP ?

[Jonathan Carter (jonathan)]

As far as I understand from Oli's comment #20, it's already booting the system as fast as it possibly can with a 2.6 kernel. Since Ubuntu will never support a 2.4 kernel, this bug can probably be closed now?

[Jim McQuillan (jam-mcquil)]

I'm fine with that.

Jim.

Jonathan Carter wrote:
> As far as I understand from Oli's comment #20, it's already booting the
> system as fast as it possibly can with a 2.6 kernel. Since Ubuntu will
> never support a 2.4 kernel, this bug can probably be closed now?
>