Comment 15 for bug 394350

Revision history for this message
Neil Van Dyke (neil-neilvandyke) wrote :

I have just reproduced the problem with a fresh install with latest updates, and kept a detailed log as I did so. Perhaps someone can spot something I am doing wrong? I would assume that I'm doing something wrong, except comparable things work on Debian and with upstream, and my users also experienced the problem independently before I did. A fresh pair of eyes on this would be appreciated!

* Download Ubuntu Server 8.04.2 for i386, and burn it to a CD-R. I used
  the following, as fetched on 2009-07-01:
  http://ftp-mirror.internap.com/pub/ubuntu-releases/hardy/ubuntu-8.04.2-server-i386.iso

* Use machine IBM/Lenovo ThinkPad T60 with Intel Core Duo (32-bit X86).
  (Note that we have also observed the same behavior on 64-bit server
  hardware.)

* Plug machine into Ethernet.

* Boot the CD-R and do the following:

* Language: English

* From CD boot menu, select: Check CD for defects. Result: "The CD-ROM
  integrity test was successful. The CD-ROM is valid." Reboot system.

* Language: English

* From CD boot menu, select: Test memory. Reboot after a lot of
  successful testing passing.

* Language: English

* From CD boot menu, select: Install Ubuntu Server.

* Choose language: English

* Country: United States

* Detect keyboard layout: No.

* Keyboard origin: USA

* Keyboard layout: USA

* Wait for some device scanning and installing and DHCP.

* Hostname: myserver

* Time zone: Eastern

* Partition disks: Guided - use entire disk. Write changes to disk.

* Wait for partitioning and fs creation.

* Full name of new user: John Smith

* Username: john

* Supply password.

* No HTTP proxy.

* Additional software selection: OpenSSH server

* Wait, then remove CD when ejected, then let reboot.

* SSH into "myserver" as "john" from another machine. Subsequent
  commands are in this shell unless otherwise specified.

* sudo su -

* apt-get update

* apt-get upgrade

* Note that the following packages are upgraded (at approx. 10pm EDT
  2009-07-10):

  apparmor apparmor-utils apt apt-utils base-files cpp-4.2 cron dash
  file gcc-4.2-base initscripts installation-report libcurl3-gnutls
  libgcc1 libgnutls13 libkrb53 libldap-2.4-2 libmagic1 libsasl2-2
  libsasl2-modules libssl0.9.8 libstdc++6 libvolume-id0
  linux-image-2.6.24-23-server linux-ubuntu-modules-2.6.24-23-server
  lsb-base lsb-release ntpdate python-apt sudo sysv-rc sysvutils tasksel
  tasksel-data tzdata udev update-manager-core

* shutdown -r now

* Wait for reboot.

* SSH into "myserver" as "john" from another machine. Subsequent
  commands are in this shell unless otherwise specified.

* sudo su -

* apt-get install apache2

* Create file "/usr/lib/cgi-bin/forever" with the contents:

---- CUT HERE ----
#!/bin/sh
echo "Content-Type: text/html"
echo ""
echo "BEFORE"
while true ; do
    expr 1 + 1 > /dev/null
done
echo "AFTER"
---- CUT HERE ----

* chmod 0755 /usr/lib/cgi-bin/forever

* /etc/init.d/apache2 stop

* Edit "/etc/apache2/sites-available/default" to add an erroneous
  directive to the top of the file: "XXX"

* /etc/init.d/apache2 start

* Verify that the start failed because of the erroneous directive, which
  confirms this config file really is being used.

* Edit "/etc/apache2/sites-available/default" to remove the erroneous
  "XXX" directive. Below the "ServerAdmin" directive line, add the
  directive: RLimitCPU 2 2

* /etc/init.d/apache2 start

* From a Web browser, access "forever" CGI (using the appropriate address):
  http://192.168.1.79/cgi-bin/forever

* Observe that "BEFORE" appears, and the HTTP connection remains open
  and loading.

* Determine PID of the "forever" process (in this case 5179).
  ps auxwf | grep forever

* cat /proc/5179/limits

---- CUT HERE ----
cat /proc/5179/limits
Limit Soft Limit Hard Limit Units
Max cpu time unlimited unlimited ms
Max file size unlimited unlimited bytes
Max data size unlimited unlimited bytes
Max stack size 8388608 unlimited bytes
Max core file size 0 unlimited bytes
Max resident set unlimited unlimited bytes
Max processes 24566 24566 processes
Max open files 1024 1024 files
Max locked memory 32768 32768 bytes
Max address space unlimited unlimited bytes
Max file locks unlimited unlimited locks
Max pending signals 24566 24566 signals
Max msgqueue size 819200 819200 bytes
Max nice priority 0 0
Max realtime priority 0 0
---- CUT HERE ----

* Observe that CPU time is "unlimited".

* dpkg -l | grep apache

---- CUT HERE ----
ii apache2 2.2.8-1ubuntu0.9 Next generation, scalable, extendable web se
ii apache2-mpm-worker 2.2.8-1ubuntu0.9 High speed threaded model for Apache HTTPD
ii apache2-utils 2.2.8-1ubuntu0.9 utility programs for webservers
ii apache2.2-common 2.2.8-1ubuntu0.9 Next generation, scalable, extendable web se
---- CUT HERE ----

* Observe that "forever" process is still running even after 20 CPU
  seconds (and several wall clock minutes) have accrued.

* Observe that HTTP connection and loading is browser is still happening.

* dpkg -l | grep linux-image

---- CUT HERE ----
ii linux-image-2.6.24-23-server 2.6.24-23.52 Linux kernel image for version 2.6.24 on x86
ii linux-image-server 2.6.24.23.25 Linux kernel image on Server Equipment.
---- CUT HERE ----

* cat /proc/5179/status

---- CUT HERE ----
Name: forever
State: R (running)
Tgid: 5179
Pid: 5179
PPid: 5120
TracerPid: 0
Uid: 33 33 33 33
Gid: 33 33 33 33
FDSize: 32
Groups: 33
VmPeak: 1772 kB
VmSize: 1772 kB
VmLck: 0 kB
VmHWM: 500 kB
VmRSS: 500 kB
VmData: 168 kB
VmStk: 84 kB
VmExe: 76 kB
VmLib: 1420 kB
VmPTE: 12 kB
Threads: 1
SigQ: 0/24566
SigPnd: 0000000000000000
ShdPnd: 0000000000000000
SigBlk: 0000000000000000
SigIgn: 0000000000000000
SigCgt: 0000000000000002
CapInh: 0000000000000000
CapPrm: 0000000000000000
CapEff: 0000000000000000
Cpus_allowed: 03
Mems_allowed: 1
voluntary_ctxt_switches: 852381
nonvoluntary_ctxt_switches: 1233444
---- CUT HERE ----

* ps auxwf

The following is a subset of the output.
---- CUT HERE ----
root 5119 0.0 0.0 10472 2596 ? Ss 22:47 0:00 /usr/sbin/apache2 -k start
www-data 5120 0.0 0.0 10244 1848 ? S 22:47 0:00 \_ /usr/sbin/apache2 -k start
www-data 5179 6.3 0.0 1772 500 ? R 22:48 0:55 | \_ /bin/sh /usr/lib/cgi-bin/forever
www-data 23160 0.0 0.0 0 0 ? Z 23:03 0:00 | \_ [expr] <defunct>
www-data 5122 0.0 0.0 231808 2400 ? Sl 22:47 0:00 \_ /usr/sbin/apache2 -k start
www-data 5126 0.0 0.0 231808 2840 ? Sl 22:47 0:00 \_ /usr/sbin/apache2 -k start
---- CUT HERE ----

* Try rebooting, to see if "forever" is correctly terminated by Apache
  if Apache is started at boot time, rather than from within the "sudo
  su -":
  shutdown -r now

* Wait for reboot.

* After comes back up, SSH in again as "john". Following commands are
  in SSH unless otherwise specified.

* sudo su -

* Request "forever" CGI in Web browser again:
  http://192.168.1.79/cgi-bin/forever

* Observe that "forever" is not terminated by the "RLimitCPU", that the
  max cpu time in "/proc/<pid>/limits" is "unlimited", and that "strace"
  of the "forever" process continues running and showing heavy activity
  of creating its child processes.

* Temporarily edit "forever" script to change the "expr" command to
  simply setting a variable so that no subprocesses are created, verify
  that nevertheless this version of the CGI also is not terminated by
  the "RLimitCPU".