Missing console output

[Scott Moser]

I saw this strange behavior today when running maverick RC tests.
*Some* console output would be missing, when other surrounding evidence indicates that it was written.

From previous discussion with kernel team, I believe that this might be related to bug 554172. However, I understood that bug to be exposed when all file descriptors for /dev/console were closed. Here, I see some output of a single shell script to the console and other data lost.

I saw missing data 3 times:
* ud-multipart-01 test failure (ami-42164607 i386 m1.small)
  Test case reports "FAIL: not found in console: HELLO RC.LOCAL LIKE WORLD"
  The investigating the remote test logs, the file writen by this
  script did exist, so the job *did* run. The job writes the file
  by tee'ing output to it, so output goes to stdout (/dev/console) and file
  but the stdout content did not get to the console.

  attachment file is console-ud-multipart.txt

* "BEGIN SSH" but no "END SSH" (ami-cc6e9ba5 x86_64 m1.large)
  This console had no 'BEGIN SSH' in its console output, but *did*
  have "END SSH". Both are written by the same /bin/sh process and
  should find their way to the console through a pipe to 'logger -s' [1].
  For example:
  { echo 1; echo 2; ssh-keygen; echo 3; } | logger -s -t ec2:

  attachment file is console-begin-no-end.txt

* "END SSH" but no "BEGIN SSH"
  Similar to above, but this console only had 'END' with no 'BEGIN'

  attachment file is console-end-no-begin.txt

--
[1] http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/annotate/head%3A/cloudinit/CloudConfig/cc_ssh.py

ProblemType: Bug
DistroRelease: Ubuntu 10.10
Package: linux-image-2.6.35-22-virtual 2.6.35-22.33
Regression: No
Reproducible: No
ProcVersionSignature: User Name 2.6.35-22.33-virtual 2.6.35.4
Uname: Linux 2.6.35-22-virtual i686
AlsaDevices: Error: command ['ls', '-l', '/dev/snd/'] failed with exit code 2: ls: cannot access /dev/snd/: No such file or directory
AplayDevices: Error: [Errno 2] No such file or directory
Architecture: i386
ArecordDevices: Error: [Errno 2] No such file or directory
CurrentDmesg: [ 12.796323] eth0: no IPv6 routers present
Date: Wed Sep 29 18:24:38 2010
Ec2AMI: ami-42164607
Ec2AMIManifest: ubuntu-images-testing-us-west-1/ubuntu-maverick-daily-i386-server-20100928.4.manifest.xml
Ec2AvailabilityZone: us-west-1b
Ec2InstanceType: m1.small
Ec2Kernel: aki-99a0f1dc
Ec2Ramdisk: unavailable
Frequency: Once every few days.
Lspci:

Lsusb: Error: command ['lsusb'] failed with exit code 1:
ProcCmdLine: root=LABEL=uec-rootfs ro console=hvc0
ProcEnviron:
 LANG=en_US.UTF-8
 SHELL=/bin/bash
ProcModules: acpiphp 17239 0 - Live 0xedc11000
SourcePackage: linux

[Jeremy Foshee (jeremyfoshee)]

Hi Scott,

If you could also please test the latest upstream kernel available that would be great. It will allow additional upstream developers to examine the issue. Refer to https://wiki.ubuntu.com/KernelMainlineBuilds . Once you've tested the upstream kernel, please remove the 'needs-upstream-testing' tag. This can be done by clicking on the yellow pencil icon next to the tag located at the bottom of the bug description and deleting the 'needs-upstream-testing' text. Please let us know your results.

Thanks in advance.

    [This is an automated message. Apologies if it has reached you inappropriately; please just reply to this message indicating so.]

[Scott Moser (smoser)]

It is not possible to use the -generic kernels on ec2.
An attempt to results in:
root (hd0)
 Filesystem type is ext2fs, using whole disk
kernel /boot/vmlinuz-2.6.36-020636rc6-generic root=LABEL=uec-rootfs ro console
=hvc0
initrd /boot/initrd.img-2.6.36-020636rc6-generic

ERROR Invalid kernel: elf_xen_note_check: ERROR: Will only load images built for the generic loader or Linux images
xc_dom_parse_image returned -1

Error 9: Unknown boot failure

[Joseph Salisbury (jsalisbury)]

@Scott

Do you happen to know if this issue is still happening with Oneiric and/or Natty?

[Scott Moser (smoser)]

I'm closing this as a dupe of bug 682831.
It really seems like the same symptoms.