Broken pipe due to grep -q
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
linux-purge |
Fix Released
|
Undecided
|
Unassigned |
Bug Description
I kept getting a writing to broken pipe error from dpkg-query.
Eventually tracked this down to grep quitting on first match when the -q option was used.
Fixed it by removing the -q and instead redirecting any grep output to null, i.e. "&> /dev/null", wherever grep is used.
Jarno Suni (jarnos) wrote : | #1 |
Jarno Suni (jarnos) wrote : | #2 |
I think it would be better just to redirect stderr of dpkg-query like this: "2>/dev/null"
Jarno Suni (jarnos) wrote : | #3 |
Which version of dpkg are you using?
Changed in linux-purge: | |
status: | New → In Progress |
status: | In Progress → Fix Committed |
Changed in linux-purge: | |
status: | Fix Committed → Incomplete |
Jarno Suni (jarnos) wrote : | #4 |
Broken pipe error is not dangerous in this case, because no more information is needed from dpkg-query and the error does not affect to the exit code of pipeline (because pipefail option is not set in shell). So I suppress the error messages by redirecting in the following commit. The side effect is that no other possible error messages of dpkg-query are shown either, but such error is very rare in this case, I think.
Here is the commit
https:/
I still wonder why you get the error message and I do not. Maybe I still change the fix later...
Jarno Suni (jarnos) wrote : | #5 |
Now broken pipe error message may also happen, if you press q quickly after launching
linux-purge --info
(in a terminal window that does have few rows) but it is harmless.
Jarno Suni (jarnos) wrote : | #6 |
I reverted the commit mentioned in #4 because it will suppress possible other error messages as well.
I dealt with the broken pipe errors referred in #5 in another new commit.
I am still waiting Chris White to respond.
Jarno Suni (jarnos) wrote : | #7 |
Which Ubuntu release you used when this happened?
Jarno Suni (jarnos) wrote : | #8 |
I have made some changes. Does this bug still occur to you? I know that your fix works, but it does some extra computing, and I have still not been able to reproduce the issue.
What does 'trap -p PIPE' output in the Bash shell you call linux-purge from? Or do you have some other environment?
Launchpad Janitor (janitor) wrote : | #9 |
[Expired for linux-purge because there has been no activity for 60 days.]
Changed in linux-purge: | |
status: | Incomplete → Expired |
Chris White (whitecf) wrote : | #10 |
Apologies as I missed any notice for the fix having been made. Just updated my installed version,
> linux-purge --version
linux-purge (Linux Purge for Ubuntu) 1.0.110-d5962da
Copyright (C) 2017-2020 Jarno Ilari Suni
License GPLv3+: GNU GPL version 3 or later <http://
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Written by Jarno Ilari Suni.
Still seem to be getting the error but also have some more stuff now,
> linux-purge --auto-only --keep=2 --yes
dpkg-query: error: error writing to '<standard output>': Broken pipe
[linux-purge] ERROR: There are no meta-kernels so can not decide which kernels
to keep.
Chris White (whitecf) wrote : | #11 |
FYI,
> bash -x linux-purge --auto-only --keep=2 --yes
+ set -u -e
+ export LC_ALL=C
+ LC_ALL=C
+ export TMPDIR=/dev/shm
+ TMPDIR=/dev/shm
+ readonly program_
+ program_
+ program_
+ program_
+ copyright_
+ MB=1000000
+ width=72
++ uname -r
++ stat --format=%m /boot
+ readonly 'w_error=Terminal screen not wide enough. Minimum is 72 columns.' 'h_error=Not enough lines in terminal screen.' lockfile=
+ w_error='Terminal screen not wide enough. Minimum is 72 columns.'
+ h_error='Not enough lines in terminal screen.'
+ lockfile=
+ current=
+ boot=/boot
+ readonly 'cs=;'
+ cs=';'
++ mktemp /dev/shm/
+ readonly error_file=
+ error_file=
+ keep=
+ auto_only=
+ clear_boot=
+ choose=
+ debug=
+ debug_file=
+ show_debug_
+ fix=
+ info=
+ simulate=
+ yes=
+ manual=
+ legend=1
+ list=
+ optimize=
+ interface=d
+ operation=purge
+ update_
+ update_grub=
+ unset -v LatestReleases ManualReleases HoldReleases
+ trap finish EXIT
+ trap 'finish INT; trap - INT; kill -s INT -- $$' INT
+ env -u GETOPT_COMPATIBLE getopt --test
+ [[ 4 -eq 4 ]]
++ env -u GETOPT_COMPATIBLE getopt -o abcd::fhi:
+ params=' --auto-only --keep '\''2'\'' --yes --'
+ eval set -- ' --auto-only --keep '\''2'\'' --yes --'
++ set -- --auto-only --keep 2 --yes --
+ :
+ case ${1-} in
+ auto_only=1
+ shift
+ :
+ case ${1-} in
+ keep=1
+ n=2
+ shift
+ shift
+ :
+ case ${1-} in
+ yes=-y
+ shift
+ :
+ case ${1-} in
+ shift
+ break
+ [[ 0 -eq 0 ]]
+ [[ -n '' ]]
+ [[ -n '' ]]
+ [[ -n 1 ]]
+ [[ 2 =~ ^[0-9]+$ ]]
+ [[ -n 1 ]]
+ [[ ! -n 1 ]]
+ [[ -n 1 ]]
+ [[ -n '' ]]
+ [[ -n '' ]]
+ [[ -n '' ]]
+ [[ -z '' ]]
+ [[ 0 -ne 0 ]]
+ [[ -n '' ]]
++ command -v update-grub
+ update_
+ [[ -n /usr/sbin/
+ chmod +x /usr/sbin/
+ [[ d = d ]]
+ hash dialog
+ [[ -n '' ]]
+ operate
+ [[ -n '' ]]
+ purge_normally
++ dpkg-query -W '-f=${Package} ${db:Status-
+++ literalize 4.15.0-115-generic
+++ printf %s 4.15.0-115-generic
+++ sed -e 's/[^^]/[&]/g' -e 's/\^/\\^/g'
++ grep -qE '^linux-
dpkg-query: error: error writing to '<standard output>': Broken pipe
++ dpkg-query -W '-f=${Package} ${Status}\n' 'linux-*'
++ awk '$3!="ok" || ($4!="installed" && $4!="not-installed" && $4!="config-files")
'
+ Pkgs=
+ [[ -n '' ]]
++ dpkg-query -W '-f=${db:
++ sed -nr 's/^[ih]i linux-image-
++ linux-version sort
+ Releases=
4.15.0-106-generic
4.15.0-107-generic
4.15.0-112-generic
4.15.0-115-generic'
+ [[ -n '' ]]
+ [[ ...
Chris White (whitecf) wrote : | #12 |
From the online man page for grep,
"... Portable shell scripts should avoid both -q and -s and should redirect standard and error output to /dev/null instead. ..."
Changed in linux-purge: | |
status: | Expired → In Progress |
Jarno Suni (jarnos) wrote : | #13 |
As for portability, linux-purge is meant for Ubuntu, so I think I can assume you are using GNU grep. (You can run "grep --version" to verify that.)
Please update linux-purge and run
linux-purge --auto-only --keep=2 --simulate --debug
Can you confirm that the broken pipe error is gone?
Please attach the file whose path is show finally.
Chris White (whitecf) wrote : | #14 |
I've made two attempts to post a comment with the file attached but as best I can tell this just isn't working so I'll paste the text into this comment,
> linux-purge --auto-only --keep=2 --simulate --debug
dpkg-query: error: error writing to '<standard output>': Broken pipe
[linux-purge] ERROR: There are no meta-kernels so can not decide which kernels
to keep.
[linux-purge] NOTE: The debug information is saved in
/dev/shm/
> cat /dev/shm/
## start time: Mon Sep 7 11:33:33 BST 2020
## linux-purge version: 1.0.110-d5962da
## options: --auto-only --keep '2' --simulate --debug '' --
## OS: Linux Mint 18.3 Sylvia
## awk executable: /usr/bin/gawk
## awk version:
GNU Awk 4.1.3, API: 1.1 (GNU MPFR 3.1.4, GNU MP 6.1.0)
Copyright (C) 1989, 1991-2015 Free Software Foundation.
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see http://
## linux packages:
ii linux-base
ii linux-firmware
un linux-firmware-
un linux-headers
un linux-headers-
un linux-headers-
un linux-headers-3.0
ii linux-headers-
ii linux-headers-
ii linux-headers-
ii linux-headers-
ii linux-headers-
ii linux-headers-
un linux-headers-
un linux-headers-amd64
un linux-headers-
un linux-headers-
un linux-hwe-tools
un linux-image
ii linux-image-
ii linux-image-
un linux-image-
un linux-image-
un linux-image-
ii linux-image-
un linux-initramfs
un linux-kernel-
un linux-kernel-
ii linux-libc-dev
ii linux-modules-
ii linux-modules-
ii linux-modules-
ii linux-modules-
ii linux-modules-
un linux-restricte
ii linux-sound-base
## trace and output:
+ debug() 147: operate
++ debug() 147: tee -ia -- /dev/shm/
+ operate() 909: [[ -n '' ]]
+ operate() 912: purge_normally
++ debug() 147: tee -ia -- /dev/shm/
++ purge_normally() 748: dpkg-query -W '-f=${Package} ${db:Status-
+++ purge_normally() 748: literalize 4.15.0-115-generic
+++ literalize() 247: printf %s 4.15.0-115-generic
+++ literalize() 247: sed -e 's/[^^]/[&]/g' -e 's/\^/\\^/g'
++ purge_normally() 748: grep -qE '^linux-
dpkg-query: error: error writing to '<standard output>': Broken pipe
++ purge_normal...
Jarno Suni (jarnos) wrote : | #15 |
You used the old version for the debug information. Please update by reinstalling using the instructions in https:/
I do not understand how you get kernel updates as there seems not to be any depending meta kernel installed. Please also show the output of:
apt-cache rdepends --important linux-image-
(or whichever is the latest kernel)
You seem to have some unsigned kernel installed, too, for some reason. Have you used some other tool to remove kernels besides linux-purge?
Chris White (whitecf) wrote : | #16 |
> sh -c 'cd /tmp && wget -N https:/
--2020-09-07 12:46:38-- https:/
Resolving git.launchpad.net (git.launchpad.
Connecting to git.launchpad.net (git.launchpad.
HTTP request sent, awaiting response... 200 OK
Length: 959 [text/plain]
Saving to: ‘install-
install-
2020-09-07 12:46:39 (155 MB/s) - ‘install-
--2020-09-07 12:46:39-- https:/
Resolving git.launchpad.net (git.launchpad.
Connecting to git.launchpad.net (git.launchpad.
HTTP request sent, awaiting response... 200 OK
Length: 35346 (35K) [text/plain]
Saving to: ‘linux-purge’
linux-purge 100%[==
2020-09-07 12:46:39 (3.71 MB/s) - ‘linux-purge’ saved [35346/35346]
--2020-09-07 12:46:39-- https:/
Resolving git.launchpad.net (git.launchpad.
Connecting to git.launchpad.net (git.launchpad.
HTTP request sent, awaiting response... 200 OK
Length: 4493 (4.4K) [text/plain]
Saving to: ‘linux-purge.8’
linux-purge.8 100%[==
2020-09-07 12:46:39 (1.00 GB/s) - ‘linux-purge.8’ saved [4493/4493]
--2020-09-07 12:46:43-- https:/
Resolving git.launchpad.net (git.launchpad.
Connecting to git.launchpad.net (git.launchpad.
HTTP request sent, awaiting response... 200 OK
Length: 1517 (1.5K) [text/plain]
Saving to: ‘linux-
linux-purge.
2020-09-07 12:46:43 (445 MB/s) - ‘linux-
Replaced current shell in order to make the Bash completion work.
Chris White (whitecf) wrote : | #17 |
> linux-purge --version
linux-purge (Linux Purge for Ubuntu) 1.0.113-c15c34a
Copyright (C) 2017-2020 Jarno Ilari Suni
License GPLv3+: GNU GPL version 3 or later <http://
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Written by Jarno Ilari Suni.
Chris White (whitecf) wrote : | #18 |
> linux-purge --auto-only --keep=2 --simulate --debug[
to keep.
[linux-purge] NOTE: The debug information is saved in
/dev/shm/
> cat /dev/shm/
## start time: Mon Sep 7 12:47:59 BST 2020
## linux-purge version: 1.0.113-c15c34a
## options: --auto-only --keep '2' --simulate --debug '' --
## OS: Linux Mint 18.3 Sylvia
## awk executable: /usr/bin/gawk
## awk version:
GNU Awk 4.1.3, API: 1.1 (GNU MPFR 3.1.4, GNU MP 6.1.0)
Copyright (C) 1989, 1991-2015 Free Software Foundation.
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see http://
## linux packages:
ii linux-base
ii linux-firmware
un linux-firmware-
un linux-headers
un linux-headers-
un linux-headers-
un linux-headers-3.0
ii linux-headers-
ii linux-headers-
ii linux-headers-
ii linux-headers-
ii linux-headers-
ii linux-headers-
un linux-headers-
un linux-headers-amd64
un linux-headers-
un linux-headers-
un linux-hwe-tools
un linux-image
ii linux-image-
ii linux-image-
un linux-image-
un linux-image-
un linux-image-
ii linux-image-
un linux-initramfs
un linux-kernel-
un linux-kernel-
ii linux-libc-dev
ii linux-modules-
ii linux-modules-
ii linux-modules-
ii linux-modules-
ii linux-modules-
un linux-restricte
ii linux-sound-base
## trace and output:
+ debug() 147: operate
++ debug() 147: tee -ia -- /dev/shm/
+ operate() 909: [[ -n '' ]]
+ operate() 912: purge_normally
++ debug() 147: tee -ia -- /dev/shm/
++ purge_normally() 748: dpkg-query -W '-f=${Package} ${db:Status-
+++ purge_normally() 748: literalize 4.15.0-115-generic
+++ literalize() 247: printf %s 4.15.0-115-generic
+++ literalize() 247: sed -e 's/[^^]/[&]/g' -e 's/\^/\\^/g'
++ purge_normally() 748: grep -qE '^linux-
++ purge_normally() 758: dpkg-query -W '-f=${Package} ${Status}\n' 'linux-*'
++ purge_normally() 760: awk '$3!="ok" || ($4!="installed" && $4!="not-installed" && $4!="config-files")
'
+ purge_normally() 758: Pkgs=
+ purge_normally() 759: [[ -n '' ]]
++ purge_normally() 773: dpkg-query -W '-f=${db:
Chris White (whitecf) wrote : | #19 |
> apt-cache rdepends --important linux-image-
linux-image-
Reverse Depends:
linux-
|linux-
linux-
linux-
linux-
Chris White (whitecf) wrote : | #20 |
> hostnamectl
Static hostname: Blodwyn
Icon name: computer-desktop
Chassis: desktop
Machine ID: 6a4e92f40a6e4d7
Boot ID: 926b1423adde41c
Operating System: Linux Mint 18.3
Kernel: Linux 4.15.0-115-generic
Architecture: x86-64
Chris White (whitecf) wrote : | #21 |
"... Have you used some other tool to remove kernels besides linux-purge?"
Just the Linux Mint "Update Manager", mintUpdate 5.3.2.5, as linux-purge wasn't removing anything.
Jarno Suni (jarnos) wrote : | #22 |
I think you should install package linux-generic-
The broken pipe issue seems to have disappeared.
Changed in linux-purge: | |
status: | In Progress → Fix Committed |
Jarno Suni (jarnos) wrote : | #23 |
You should now update linux-purge. See: https:/
In your case you want to have linux-generic-
Jarno Suni (jarnos) wrote : | #24 |
Is it working now?
Chris White (whitecf) wrote : | #25 |
I'm still looking into if it it's ok for me to install linux-generic-
Jarno Suni (jarnos) wrote : | #26 |
You could run the following scripts in shell to find out, if/when linux-generic-
grep -FH 'purge linux-generic-
for i in /var/log/
If it prints some match from time when you were using version 1.0.110-d5962da or 1.0.113-c15c34a of linux-purge, the package was probably purged by linux-purge due to a bug that is fixed in later version, see https:/
Can you show the output?
P.S. As for easy updating linux-purge, see https:/
Chris White (whitecf) wrote : | #27 |
> grep -FH 'purge linux-generic-
> for i in /var/log/
No matches on either search I'm afraid.
Jarno Suni (jarnos) wrote : | #28 |
That is odd. It seems like it was never installed then.
The --keep functionality of linux-purge relies on meta kernels, but you should be able to use linux-purge without --keep even if you do not have a meta kernel installed.
Jarno Suni (jarnos) wrote : | #29 |
Did you find out, if you should install (or have installed) linux-generic-
Do you still have an issue with 'grep -q' with the latest version of linux-purge? If there is, please tell what grep --version reports.
Chris White (whitecf) wrote : | #30 |
I've recently upgraded to Mint 20.1 and don't know what changes this might have made but linux-purge seems to be working ok now.
System: Kernel: 5.4.0-67-generic x86_64 bits: 64 compiler: gcc v: 9.3.0 Desktop: Cinnamon 4.8.6
wm: muffin dm: LightDM Distro: Linux Mint 20.1 Ulyssa base: Ubuntu 20.04 focal
> linux-purge --version
linux-purge (Linux Purge for Ubuntu) 1.0.119-c64222c
Copyright (C) 2017-2020 Jarno Ilari Suni
License GPLv3+: GNU GPL version 3 or later <http://
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Written by Jarno Ilari Suni.
> grep -FH 'purge linux-generic-hwe' /var/log/dpkg.log /var/log/dpkg.log.*
> for i in /var/log/
> grep --version
grep (GNU grep) 3.4
Copyright © 2020 Free Software Foundation, Inc.
Licence GPLv3+: GNU GPL version 3 or later <https:/
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Written by Mike Haertel and others; see
<https:/
>
Jarno Suni (jarnos) wrote : | #31 |
I suppose you have linux-generic installed.
I recommend that you upgrade to the latest version of linux-purge.
Jarno Suni (jarnos) wrote : | #32 |
I am marking this as fixed. Please create another bug report if you still can reproduce the bug in some system with the latest version of linux-purge.
Changed in linux-purge: | |
status: | Fix Committed → Fix Released |
Oh, I can not reproduce the issue. I guess it is due to race condition. Does the error make script somehow fail or is it just an annoying error message?