vcgencmd get stuck and become zombie if killed

[Stefan]

I regularly check PI core temp using vcgencmd and have been doing so successfully from August 2013.

A few days into 2014 I discovered that "vcgencmd" may become permanently blocked and its no longer possible to read the core temp, in fact, no action is possible with vcgencmd, equally when trying to read core temp from the system file the "cat" command also becomes blocked.

I regularly perform firmware and OS updates; I suspect problem was introduced with firmware released around new year.

A reboot is necessary, but problem will return in 0.5 to 14 days requiring a new reboot and so on...

Example of hung process:
pi@RPI1 /var/www $ ps alx | grep Z | grep -v grep
F UID PID PPID PRI NI VSZ RSS WCHAN STAT TTY TIME COMMAND
0 1000 25111 1 20 0 0 0 exit Zl ? 0:00 [vcgencmd] <defunct>

I have a mod B PI purchased in August 2013 and use a NOOBS image from http://www.raspberrypi.org/downloads to install the OS:
Linux RPI1 3.10.32+ #648 PREEMPT Fri Feb 28 16:29:56 GMT 2014 armv6l GNU/Linux

pi@RPI1 /var/www $ dpkg -s libc6 | grep ^Version
Version: 2.13-38+rpi2

I use this PI to read data off a 1-wire network via an I2C interface (https://www.m.nu/rpi-i2c-1wire-expansion-module-v11-p-677.html). I use Apache, MySQL, PHP and some simple JavaScript to do the job.

dmesg output after detecting a hung vcgencmd:
[728060.523899] INFO: task VCHIQ completio:25113 blocked for more than 120 seconds.
[728060.523925] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[728060.523940] VCHIQ completio D c040118c 0 25113 1 0x00000005
[728060.524019] [<c040118c>] (__schedule+0x278/0x558) from [<c03fff08>] (schedule_timeout+0x170/0x200)
[728060.524085] [<c03fff08>] (schedule_timeout+0x170/0x200) from [<c0400c08>] (__down+0x88/0xc0)
[728060.524133] [<c0400c08>] (__down+0x88/0xc0) from [<c0041a4c>] (down+0x60/0x64)
[728060.524208] [<c0041a4c>] (down+0x60/0x64) from [<c029826c>] (vchiq_release+0xf4/0x2d4)
[728060.524249] [<c029826c>] (vchiq_release+0xf4/0x2d4) from [<c00d7768>] (__fput+0x8c/0x228)
[728060.524293] [<c00d7768>] (__fput+0x8c/0x228) from [<c0039d30>] (task_work_run+0x84/0xb4)
[728060.524334] [<c0039d30>] (task_work_run+0x84/0xb4) from [<c0023f90>] (do_exit+0x6b8/0x91c)
[728060.524363] [<c0023f90>] (do_exit+0x6b8/0x91c) from [<c002425c>] (do_group_exit+0x3c/0xc4)
[728060.524406] [<c002425c>] (do_group_exit+0x3c/0xc4) from [<c002eb54>] (get_signal_to_deliver+0x160/0x5ac)
[728060.524439] [<c002eb54>] (get_signal_to_deliver+0x160/0x5ac) from [<c03fc1ec>] (do_signal+0xa8/0x3cc)
[728060.524481] [<c03fc1ec>] (do_signal+0xa8/0x3cc) from [<c0010850>] (do_work_pending+0x9c/0xb0)
[728060.524512] [<c0010850>] (do_work_pending+0x9c/0xb0) from [<c000db00>] (work_pending+0xc/0x20)

Cheers
Stefan

[Diederik (didi-debian)]

Is this still an issue?

If so, you should report it at https://github.com/raspberrypi/firmware/issues

Note that the use of `rpi-update` has been deprecated for a while and you should update the kernel/firmware from the `raspberrypi-bootloader` package, which you can update via the regular `apt-get update` and `apt-get upgrade`.

[Stefan (stefan-stroemqvist)]

I guess, but due to the consequences (script hang in 5-10 days with reboot
required to read core temp again) I have abandoned using vccmdgen for this
purpose.

On another RPi I had a hang of "Domoticz" (a home automation software, easy
to Google for...).
I found a child process of Domoticz hung on, guess what... vcgencmd...
Domoticz is automatically logging the core temperature and disk utilization
- I guess they have the same issue as me.

But personally, I have stopped logging the core temp and is now reading
temperature from a 1wire sensor located in the same enclosure as the RPi.

Please let me know if you want me to reproduce a hang and take some
actions, e.g. memory dumps. With some simple instructions on how to use
gdb, or whatever debugger that is available I can create process memory
dumps, maybe even kernel dumps for you to analyze.

/Stefan

On Wed, Jan 13, 2016 at 2:15 PM, Diederik <email address hidden> wrote:

> Is this still an issue?
>
> If so, you should report it at
> https://github.com/raspberrypi/firmware/issues
>
> Note that the use of `rpi-update` has been deprecated for a while and
> you should update the kernel/firmware from the `raspberrypi-bootloader`
> package, which you can update via the regular `apt-get update` and `apt-
> get upgrade`.
>
> ** Changed in: raspbian
> Status: New => Incomplete
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1294374
>
> Title:
> vcgencmd get stuck and become zombie if killed
>
> Status in Raspbian:
> Incomplete
>
> Bug description:
> I regularly check PI core temp using vcgencmd and have been doing so
> successfully from August 2013.
>
> A few days into 2014 I discovered that "vcgencmd" may become
> permanently blocked and its no longer possible to read the core temp,
> in fact, no action is possible with vcgencmd, equally when trying to
> read core temp from the system file the "cat" command also becomes
> blocked.
>
> I regularly perform firmware and OS updates; I suspect problem was
> introduced with firmware released around new year.
>
> A reboot is necessary, but problem will return in 0.5 to 14 days
> requiring a new reboot and so on...
>
> Example of hung process:
> pi@RPI1 /var/www $ ps alx | grep Z | grep -v grep
> F UID PID PPID PRI NI VSZ RSS WCHAN STAT TTY TIME
> COMMAND
> 0 1000 25111 1 20 0 0 0 exit Zl ? 0:00
> [vcgencmd] <defunct>
>
> I have a mod B PI purchased in August 2013 and use a NOOBS image from
> http://www.raspberrypi.org/downloads to install the OS:
> Linux RPI1 3.10.32+ #648 PREEMPT Fri Feb 28 16:29:56 GMT 2014 armv6l
> GNU/Linux
>
> pi@RPI1 /var/www $ dpkg -s libc6 | grep ^Version
> Version: 2.13-38+rpi2
>
> I use this PI to read data off a 1-wire network via an I2C interface
> (https://www.m.nu/rpi-i2c-1wire-expansion-module-v11-p-677.html). I
> use Apache, MySQL, PHP and some simple JavaScript to do the job.
>
> dmesg output after detecting a hung vcgencmd:
> [728060.523899] INFO: task VCHIQ completio:25113 blocked for more than
> 120 seconds.
> [728060.523925] "echo ...

[Diederik (didi-debian)]

Me personally won't be able to help you with this, but the people at https://github.com/raspberrypi/firmware/ really would as they are the creators/maintainer of that program and have MUCH more technical know-how in that area then I have.

What you have reported is of a good quality and if the issue is still present in recent kernel/firmware (3.10.32+ #648 is really old), I guess they'd love to hear about it.

(I still don't understand how to use gdb and when I report a useful stack trace that's because of various -dbg packages installed and a program which does the hard/difficult stuff for me)

[Launchpad Janitor (janitor)]

[Expired for Raspbian because there has been no activity for 60 days.]