linux-image-3.13.0-65-generic breaks serial communication

[Chris L8]

Today after updating to the linux-image-3.13.0-65-generic 3.13.0-65 Kernel on Lubuntu 14.04.3 LTS two different programs that rely on PySerial quit working. miniterm.py and my ROS (Robot Operating System) based Python node quit working.

When rebooted and selected 3.13.0-63-generic via GRUB everything worked again.

Here are the steps to create the issue with miniterm.py, note that you need something connected via a USB to Serial adapter for this to work. I am working with a Parallax Activity Board, but an Arduino should also exhibit the same behavior.

sudo apt install linux-image-3.13.0-65-generic
sudo apt install linux-image-extra-3.13.0-65-generic
reboot

 $ miniterm.py /dev/ttyUSB0 115200
--- Miniterm on /dev/ttyUSB0: 115200,8,N,1 ---
--- Quit: Ctrl+] | Menu: Ctrl+T | Help: Ctrl+T followed by Ctrl+H ---

--- exit ---
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 763, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/usr/bin/miniterm.py", line 220, in reader
    data = character(self.serial.read(1))
  File "/usr/lib/python2.7/dist-packages/serial/serialposix.py", line 460, in read
    raise SerialException('device reports readiness to read but returned no data (device disconnected?)')
SerialException: device reports readiness to read but returned no data (device disconnected?)

Reverting to the 3.13.0-63 kernel allows miniterm.py and other Python Serial based programs to work normally.

[Chris L8 (ekpyroticfrood)]

I think that this bug report may also be the same issue:
https://bugs.launchpad.net/ubuntu/+source/linux-lts-trusty/+bug/1500860

I also found this which seems like it may be relevant, but I am not a kernel developer so I'm not certain:
https://<email address hidden>/msg136898.html

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Flo (spam01)]

Similar issue here - i own a RFXTRX device which internally uses a FTDI USB/serial converter. After update to kernel 3.13.0-65, the software which uses the RFXTRX device ( FHEM ) stops working and hangs while reading from it.

[Chris L8 (ekpyroticfrood)]

Based on this thread, it looks like the fix is on the way:
https://<email address hidden>/msg137245.html
but I don't know how long these things take to go "live".

[pEEf (p4l)]

Broke Python serial with RFCOMM bluetooth binding.

[Henk van der Laak (henklaak-r)]

Problem occured also when using Qt's QSerialPort.

QSerialPort::waitforReadyRead() returns true, bute QSerialPort::readAll() returns no data.

Downgrading kernel version to 3.13.0-63 'fixed' the problem.

Gr. Henk
-----------------
Qt 5.5.0
Linux henklaak-desktop 3.13.0-65-generic #105-Ubuntu SMP Mon Sep 21 18:50:58 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

[Rob G. (rob-gaddi)]

I also found it first in PySerial, but it affects everything down to the underlying poll() implementation. Demo C code is attached (compile with -Wall -std=gnu99). Under -63 the poll() call blocks for the 1 second it's supposed to, a ping is sent once a second, and if you short pins 2-3 of the serial port the ping loops back and the code starts running I/O bound; pinging as fast as it can until you remove the screwdriver.

Under -65 poll() returns immediately even with nothing to read and the console I/O is the only thing restraining the process.

In short, this is a driver bug, not a Python specific thing, and most likely breaks any code that relies on select, poll, or epoll for serial I/O.

[psylem (subnetjet)]

I can confirm that 3.13.0-63 kernel works.

If you rely on this functionality, editing GRUB_DEFAULT to something like "Advanced options for Ubuntu>Ubuntu, with Linux 3.13.0-63-generic" is a graceful way to downgrade kernel in the interim. You can list available kernels with "cat /boot/grub/grub.cfg | grep menuentry". Just run update-grub to apply, then reboot. I'd then recommend subscribing to this bug for updates so you don't forget to revert grub settings once it's fixed.

I didn't have 3.13.0-63 available on my system until I manually installed the packages (linux-headers-3.13.0-63 linux-headers-3.13.0-63-generic linux-image-3.13.0-63-generic linux-image-extra-3.13.0-63-generic linux-signed-image-3.13.0-63-generic).

[Rob Ward (rl-ward)]

I have found the "serial" library no longer works in Python.2.7. I discovered this when using Python to read and parse a serial stream from my Arduino Uno based weather station. The Arduino listens in on 433MHz signals from my Oregon Scientific weather station sensors and sends a stream of weather data every minute to the web server via a USB/Serial inerface. The web server has worked well with this system 24/7 for the last 5-6 years and suddenly it is broken on the upgrade to 3.13.0-65-generic kernel. Downgrading the Kernel to 3.13.0-63-generic has worked for me and also I am using Ubuntu 14.04.3 LTS as XUbuntu.

Whether it is a problem with Ubuntu or Python or Arduino, I don't know as the latter two may have problems with how the serial interface is initiated or maintained, and the Ubuntu may just have a higher specification now that they can't deal with.

However my Arduino IDE downloads programs and 'listens' to the serial stream from the Uno with 3.13.0-65-generic Ok, So my suspicions is the broken bit is between the kernel and Python???

Here is the Python error message-
Traceback (most recent call last):
  File "SerialDemo.py", line 19, in <module>
    dataIn = ser.readline()
  File "/usr/lib/python2.7/dist-packages/serial/serialposix.py", line 460, in read
    raise SerialException('device reports readiness to read but returned no data (device disconnected?)')
serial.serialutil.SerialException: device reports readiness to read but returned no data (device disconnected?)

[Michael (muman613)]

I realize my issue is not related to Python but I find the same problem when I use the serial port from Perl using the Device::SerialPort class. My scripts have run for almost 10 years without a problem till the last system upgrade to kernel ending in -65. I find that reverting to kernel ending in -63 the problem does not exist, but it clearly exists under kernel -65...

I concur with the poster above who considers this a bug in the serial port driver or possibly in the kernel's poll() or select() function. For the time being our organization has a policy to not upgrade to any new kernels till this issue is resolved.

We consider this a very very bad bug in the OS... We have been loyal Ubuntu users since 2008 (Ubuntu 8.04) and have been very happy with the distro... I hope that the kernel developers consider this a HIGH PRIORITY issue and repair it by the next kernel release.

Thank you,
Michael Uman

[kerlyn (kerlyn)]

I agree with @rob-gaddi, this appears to be a driver bug. I am running a C application called tunslip, which is provided as part of the Contiki OS distro. select() indicates there are bytes available on &rset even when this is not the case. A subsequent fread() on the serial device returns zero bytes, resulting in an exception.

[aradhya (aradhya-mathur95)]

if you use try and except serial.serialutil.SerialException
 then u dont encounter the error
raise SerialException('device reports readiness to read but returned no data (device disconnected?)')

however the output does not synchronize properly

[bouke (bouke-krom)]

Seems to be fixed in 3.13.0-66.107, which is available for Ubuntu 14.04 now.

[Sebastian Unger (sebunger44)]

Same here. Could not reproduce the issue on 3.13.0-66.108 any more. Cheers

[aradhya (aradhya-mathur95)]

cheers the bug is no more present after latest kernel update

On Wed, Oct 21, 2015 at 8:50 AM, Sebastian Unger <email address hidden>
wrote:

> Same here. Could not reproduce the issue on 3.13.0-66.108 any more.
> Cheers
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1501345
>
> Title:
> linux-image-3.13.0-65-generic breaks serial communication
>
> Status in linux-lts-trusty package in Ubuntu:
> Confirmed
>
> Bug description:
> Today after updating to the linux-image-3.13.0-65-generic 3.13.0-65
> Kernel on Lubuntu 14.04.3 LTS two different programs that rely on
> PySerial quit working. miniterm.py and my ROS (Robot Operating System)
> based Python node quit working.
>
> When rebooted and selected 3.13.0-63-generic via GRUB everything
> worked again.
>
> Here are the steps to create the issue with miniterm.py, note that you
> need something connected via a USB to Serial adapter for this to work.
> I am working with a Parallax Activity Board, but an Arduino should
> also exhibit the same behavior.
>
> sudo apt install linux-image-3.13.0-65-generic
> sudo apt install linux-image-extra-3.13.0-65-generic
> reboot
>
> $ miniterm.py /dev/ttyUSB0 115200
> --- Miniterm on /dev/ttyUSB0: 115200,8,N,1 ---
> --- Quit: Ctrl+] | Menu: Ctrl+T | Help: Ctrl+T followed by Ctrl+H ---
>
> --- exit ---
> Exception in thread Thread-1:
> Traceback (most recent call last):
> File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner
> self.run()
> File "/usr/lib/python2.7/threading.py", line 763, in run
> self.__target(*self.__args, **self.__kwargs)
> File "/usr/bin/miniterm.py", line 220, in reader
> data = character(self.serial.read(1))
> File "/usr/lib/python2.7/dist-packages/serial/serialposix.py", line
> 460, in read
> raise SerialException('device reports readiness to read but returned
> no data (device disconnected?)')
> SerialException: device reports readiness to read but returned no data
> (device disconnected?)
>
> Reverting to the 3.13.0-63 kernel allows miniterm.py and other Python
> Serial based programs to work normally.
>
> To manage notifications about this bug go to:
>
> https://bugs.launchpad.net/ubuntu/+source/linux-lts-trusty/+bug/1501345/+subscriptions
>

[aradhya (aradhya-mathur95)]

thank u the bug seems to be fixed

On Tue, Oct 20, 2015 at 1:53 PM, bouke <email address hidden> wrote:

> Seems to be fixed in 3.13.0-66.107, which is available for Ubuntu 14.04
> now.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1501345
>
> Title:
> linux-image-3.13.0-65-generic breaks serial communication
>
> Status in linux-lts-trusty package in Ubuntu:
> Confirmed
>
> Bug description:
> Today after updating to the linux-image-3.13.0-65-generic 3.13.0-65
> Kernel on Lubuntu 14.04.3 LTS two different programs that rely on
> PySerial quit working. miniterm.py and my ROS (Robot Operating System)
> based Python node quit working.
>
> When rebooted and selected 3.13.0-63-generic via GRUB everything
> worked again.
>
> Here are the steps to create the issue with miniterm.py, note that you
> need something connected via a USB to Serial adapter for this to work.
> I am working with a Parallax Activity Board, but an Arduino should
> also exhibit the same behavior.
>
> sudo apt install linux-image-3.13.0-65-generic
> sudo apt install linux-image-extra-3.13.0-65-generic
> reboot
>
> $ miniterm.py /dev/ttyUSB0 115200
> --- Miniterm on /dev/ttyUSB0: 115200,8,N,1 ---
> --- Quit: Ctrl+] | Menu: Ctrl+T | Help: Ctrl+T followed by Ctrl+H ---
>
> --- exit ---
> Exception in thread Thread-1:
> Traceback (most recent call last):
> File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner
> self.run()
> File "/usr/lib/python2.7/threading.py", line 763, in run
> self.__target(*self.__args, **self.__kwargs)
> File "/usr/bin/miniterm.py", line 220, in reader
> data = character(self.serial.read(1))
> File "/usr/lib/python2.7/dist-packages/serial/serialposix.py", line
> 460, in read
> raise SerialException('device reports readiness to read but returned
> no data (device disconnected?)')
> SerialException: device reports readiness to read but returned no data
> (device disconnected?)
>
> Reverting to the 3.13.0-63 kernel allows miniterm.py and other Python
> Serial based programs to work normally.
>
> To manage notifications about this bug go to:
>
> https://bugs.launchpad.net/ubuntu/+source/linux-lts-trusty/+bug/1501345/+subscriptions
>

[Gawarmel (gawarmel)]

Confirmed. Bug's gone with kernel upgrade to 3.13.0-66.108. Thanks! Can we close this?

[Rob Ward (rl-ward)]

I can confirm that my Weather sensors->Arduino->USB->(Python+Apache+XUbuntu) serial communication is back to reporting my weather 24/7 as normal. Your assistance in fixing this quickly for us is greatly appreciated.

[aradhya (aradhya-mathur95)]

I initially encountered the same error on kernel 3.13.0.66 so I m still
investigating if this error comes back to the surface and trying to
recreate what caused so I wud not want the thread to be closed as yet.
Although since the last few trials I haven't encountered the bug again
On 24 Oct 2015 02:35, "Gawarmel" <email address hidden> wrote:

> Confirmed. Bug's gone with kernel upgrade to 3.13.0-66.108. Thanks! Can
> we close this?
>
> ** Changed in: linux-lts-trusty (Ubuntu)
> Status: Confirmed => Fix Committed
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1501345
>
> Title:
> linux-image-3.13.0-65-generic breaks serial communication
>
> Status in linux-lts-trusty package in Ubuntu:
> Fix Committed
>
> Bug description:
> Today after updating to the linux-image-3.13.0-65-generic 3.13.0-65
> Kernel on Lubuntu 14.04.3 LTS two different programs that rely on
> PySerial quit working. miniterm.py and my ROS (Robot Operating System)
> based Python node quit working.
>
> When rebooted and selected 3.13.0-63-generic via GRUB everything
> worked again.
>
> Here are the steps to create the issue with miniterm.py, note that you
> need something connected via a USB to Serial adapter for this to work.
> I am working with a Parallax Activity Board, but an Arduino should
> also exhibit the same behavior.
>
> sudo apt install linux-image-3.13.0-65-generic
> sudo apt install linux-image-extra-3.13.0-65-generic
> reboot
>
> $ miniterm.py /dev/ttyUSB0 115200
> --- Miniterm on /dev/ttyUSB0: 115200,8,N,1 ---
> --- Quit: Ctrl+] | Menu: Ctrl+T | Help: Ctrl+T followed by Ctrl+H ---
>
> --- exit ---
> Exception in thread Thread-1:
> Traceback (most recent call last):
> File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner
> self.run()
> File "/usr/lib/python2.7/threading.py", line 763, in run
> self.__target(*self.__args, **self.__kwargs)
> File "/usr/bin/miniterm.py", line 220, in reader
> data = character(self.serial.read(1))
> File "/usr/lib/python2.7/dist-packages/serial/serialposix.py", line
> 460, in read
> raise SerialException('device reports readiness to read but returned
> no data (device disconnected?)')
> SerialException: device reports readiness to read but returned no data
> (device disconnected?)
>
> Reverting to the 3.13.0-63 kernel allows miniterm.py and other Python
> Serial based programs to work normally.
>
> To manage notifications about this bug go to:
>
> https://bugs.launchpad.net/ubuntu/+source/linux-lts-trusty/+bug/1501345/+subscriptions
>

[aradhya (aradhya-mathur95)]

yes the serial functioning is working properly thnx for ur cooperation

On Tue, Oct 20, 2015 at 1:53 PM, bouke <email address hidden> wrote:

> Seems to be fixed in 3.13.0-66.107, which is available for Ubuntu 14.04
> now.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1501345
>
> Title:
> linux-image-3.13.0-65-generic breaks serial communication
>
> Status in linux-lts-trusty package in Ubuntu:
> Confirmed
>
> Bug description:
> Today after updating to the linux-image-3.13.0-65-generic 3.13.0-65
> Kernel on Lubuntu 14.04.3 LTS two different programs that rely on
> PySerial quit working. miniterm.py and my ROS (Robot Operating System)
> based Python node quit working.
>
> When rebooted and selected 3.13.0-63-generic via GRUB everything
> worked again.
>
> Here are the steps to create the issue with miniterm.py, note that you
> need something connected via a USB to Serial adapter for this to work.
> I am working with a Parallax Activity Board, but an Arduino should
> also exhibit the same behavior.
>
> sudo apt install linux-image-3.13.0-65-generic
> sudo apt install linux-image-extra-3.13.0-65-generic
> reboot
>
> $ miniterm.py /dev/ttyUSB0 115200
> --- Miniterm on /dev/ttyUSB0: 115200,8,N,1 ---
> --- Quit: Ctrl+] | Menu: Ctrl+T | Help: Ctrl+T followed by Ctrl+H ---
>
> --- exit ---
> Exception in thread Thread-1:
> Traceback (most recent call last):
> File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner
> self.run()
> File "/usr/lib/python2.7/threading.py", line 763, in run
> self.__target(*self.__args, **self.__kwargs)
> File "/usr/bin/miniterm.py", line 220, in reader
> data = character(self.serial.read(1))
> File "/usr/lib/python2.7/dist-packages/serial/serialposix.py", line
> 460, in read
> raise SerialException('device reports readiness to read but returned
> no data (device disconnected?)')
> SerialException: device reports readiness to read but returned no data
> (device disconnected?)
>
> Reverting to the 3.13.0-63 kernel allows miniterm.py and other Python
> Serial based programs to work normally.
>
> To manage notifications about this bug go to:
>
> https://bugs.launchpad.net/ubuntu/+source/linux-lts-trusty/+bug/1501345/+subscriptions
>

[Jon Watte (jwatte)]

This bug is back in kernel 4.8 series. I'm seeing it in pyserial, using the Cura 2.4 application, on current Ubuntu 16.04 LTS install.