[SRU] powerpc-utils commands exhibits "command substitution: ignored null byte in input" warning message

Bug #1692420 reported by bugproxy on 2017-05-22
12
This bug affects 1 person
Affects Status Importance Assigned to Milestone
The Ubuntu-power-systems project
Medium
Canonical Foundations Team
powerpc-utils (Ubuntu)
Medium
Adam Conrad
Zesty
Medium
Unassigned

Bug Description

[Impact]

The current version of powerpc-utils in Ubuntu Zesty and Artful carries a bug exposed by bash 4.4+ which causes a warning to be logged to stderr in 4 different commands: ls-veth, ls-vscsi, lsdevinfo, and ofpathname. Bash is unable to store the NUL bytes in the result of a command substitution, prior to 4.4 it silently dropped the NUL bytes, but since 4.4 it drops the NUL bytes with a warning: "command substitution: ignored null byte in input".

Additionally to that the ofpathname command also had a bug that caused a "cd: too many arguments" warning.

Upstream has fixed both [1] and they now need to be backport to Zesty.

[Test Case 1]

Call any of the affected commands without arguments, they should output "command substitution: ignored null byte in input" to stderr.

The same warning can easily be reproduced in bash 4.4+ by executing:
$ x=$(cat < /proc/$$/cmdline)
bash: warning: command substitution: ignored null byte in input

And to fix it:
$ x=$(tr -d '\0' < /proc/$$/cmdline)
<no output, as expected>

[Test Case 2]
Call ofpathname with no arguments:
$ ofpathname
cd: too many arguments

[Regression Potential]

The replacement of 'cat' with 'tr -d "\0"' causes no regression. Also, both commands are part of coreutils.

The 'cd' fix replaces it with a drop_dir call. In case the requested path is not found then the previous behavior would be kept (ie. no directory change).

[Other Info]

Upstream fix link:
[1] https://github.com/ibm-power-utilities/powerpc-utils/commit/4fcc611a09caea70e6bca6a1275211c72ac4bb45

[Old description]
Please cherry pick...

https://github.com/ibm-power-utilities/powerpc-utils/commit/4fcc611a09caea70e6bca6a1275211c72ac4bb45

Default Comment by Bridge

tags: added: architecture-ppc64le bugnameltc-150352 severity-medium targetmilestone-inin1704
Changed in ubuntu:
assignee: nobody → Ubuntu on IBM Power Systems Bug Triage (ubuntu-power-triage)
affects: ubuntu → powerpc-utils (Ubuntu)
Manoj Iyer (manjo) on 2017-06-01
tags: added: ubuntu-17.04

------- Comment From <email address hidden> 2017-06-20 07:27 EDT-------
let me know if this patch get merged into Ubuntu build.

Manoj Iyer (manjo) on 2017-07-19
Changed in powerpc-utils (Ubuntu):
assignee: Ubuntu on IBM Power Systems Bug Triage (ubuntu-power-triage) → Canonical Kernel Team (canonical-kernel-team)
importance: Undecided → Medium
Changed in ubuntu-power-systems:
importance: Undecided → Medium
Changed in powerpc-utils (Ubuntu):
assignee: Canonical Kernel Team (canonical-kernel-team) → Canonical Foundations Team (canonical-foundations)

Could someone from foundations team please cherry-pick this commit?

tags: added: triage-a
Manoj Iyer (manjo) on 2017-08-21
tags: added: triage-r
removed: triage-a
Steve Langasek (vorlon) on 2017-08-22
Changed in powerpc-utils (Ubuntu):
status: New → Triaged
description: updated
summary: - Ubuntu 17.04: "command substitution: ignored null byte in input" warning
- message is displayed while running few RAS commands.
+ [SRU] powerpc-utils commans exhibits "command substitution: ignored null
+ byte in input" warning message
Tiago Stürmer Daitx (tdaitx) wrote :
description: updated
Changed in ubuntu-power-systems:
assignee: nobody → Canonical Foundations Team (canonical-foundations)

The attachment "powerpc-utils_1.3.2-1ubuntu2_1.3.2-1ubuntu3.debdiff" seems to be a debdiff. The ubuntu-sponsors team has been subscribed to the bug report so that they can review and hopefully sponsor the debdiff. If the attachment isn't a patch, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are member of the ~ubuntu-sponsors, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issue please contact him.]

Tiago Stürmer Daitx (tdaitx) wrote :

Manoj, could you please confirm that the testcases I wrote in the description are the right testcases for this issue? If any of them are wrong, please update accordingly with a working one.

summary: - [SRU] powerpc-utils commans exhibits "command substitution: ignored null
- byte in input" warning message
+ [SRU] powerpc-utils commands exhibits "command substitution: ignored
+ null byte in input" warning message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package powerpc-utils - 1.3.2-1ubuntu3

---------------
powerpc-utils (1.3.2-1ubuntu3) artful; urgency=medium

  * d/p/scripts-Command-substituion-warning.patch: fix bash warning
    "command substitution: ignored null byte in input" in ls-veth,
    ls-vscsi, lsdevinfo, and ofpathname; fix "cd: too many arguments"
    in ofpathname. LP: #1692420.

 -- Tiago Stürmer Daitx <email address hidden> Thu, 24 Aug 2017 09:48:27 -0700

Changed in powerpc-utils (Ubuntu):
status: Triaged → Fix Released
Changed in powerpc-utils (Ubuntu Zesty):
status: New → In Progress
importance: Undecided → Medium
Manoj Iyer (manjo) on 2017-08-28
tags: added: triage-g
removed: triage-r

Hello bugproxy, or anyone else affected,

Accepted powerpc-utils into zesty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/powerpc-utils/1.3.2-1ubuntu3~17.04 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-zesty to verification-done-zesty. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-zesty. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in powerpc-utils (Ubuntu Zesty):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-zesty

------- Comment From <email address hidden> 2017-09-04 07:43 EDT-------
(In reply to comment #40)
> Hello bugproxy, or anyone else affected,
>
> Accepted powerpc-utils into zesty-proposed. The package will build now and
> be available at
> https://launchpad.net/ubuntu/+source/powerpc-utils/1.3.2-1ubuntu3~17.04 in a
> few hours, and then in the -proposed repository.
>
> Please help us by testing this new package. See
> https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to
> enable and use -proposed.Your feedback will aid us getting this update out
> to other Ubuntu users.
>
> If this package fixes the bug for you, please add a comment to this bug,
> mentioning the version of the package you tested and change the tag from
> verification-needed-zesty to verification-done-zesty. If it does not fix the
> bug for you, please add a comment stating that, and change the tag to
> verification-failed-zesty. In either case, details of your testing will help
> us make a better decision.
>
> Further information regarding the verification process can be found at
> https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in
> advance!

Tried to install package from proposed, getting below error.

root@alp10:~# apt-get install powerpc-utils/zesty-proposed
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Release 'zesty-proposed' for 'powerpc-utils' was not found

Thanks,
Pavithra

bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2017-09-05 00:27 EDT-------
(In reply to comment #41)
> (In reply to comment #40)
> > Hello bugproxy, or anyone else affected,
> >
> > Accepted powerpc-utils into zesty-proposed. The package will build now and
> > be available at
> > https://launchpad.net/ubuntu/+source/powerpc-utils/1.3.2-1ubuntu3~17.04 in a
> > few hours, and then in the -proposed repository.
> >
> > Please help us by testing this new package. See
> > https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to
> > enable and use -proposed.Your feedback will aid us getting this update out
> > to other Ubuntu users.
> >
> > If this package fixes the bug for you, please add a comment to this bug,
> > mentioning the version of the package you tested and change the tag from
> > verification-needed-zesty to verification-done-zesty. If it does not fix the
> > bug for you, please add a comment stating that, and change the tag to
> > verification-failed-zesty. In either case, details of your testing will help
> > us make a better decision.
> >
> > Further information regarding the verification process can be found at
> > https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in
> > advance!
>
>
> Tried to install package from proposed, getting below error.
>
> root@alp10:~# apt-get install powerpc-utils/zesty-proposed
> Reading package lists... Done
> Building dependency tree
> Reading state information... Done
> E: Release 'zesty-proposed' for 'powerpc-utils' was not found
>
> Thanks,
> Pavithra

Canonical, Can you check ?

On Mon, Sep 04, 2017 at 11:50:09AM -0000, bugproxy wrote:
> > Further information regarding the verification process can be found at
> > https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in
> > advance!

> Tried to install package from proposed, getting below error.

> root@alp10:~# apt-get install powerpc-utils/zesty-proposed
> Reading package lists... Done
> Building dependency tree
> Reading state information... Done
> E: Release 'zesty-proposed' for 'powerpc-utils' was not found

That is not valid apt syntax. Please
see https://wiki.ubuntu.com/QATeam/PerformingSRUVerification for guidance on
how to run tests for SRUs.

Manoj Iyer (manjo) on 2017-09-11
Changed in ubuntu-power-systems:
status: New → Fix Committed

------- Comment From <email address hidden> 2017-09-27 03:23 EDT-------
(In reply to comment #43)
> On Mon, Sep 04, 2017 at 11:50:09AM -0000, bugproxy wrote:
> > > Further information regarding the verification process can be found at
> > > https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in
> > > advance!
>
> > Tried to install package from proposed, getting below error.
>
> > root@alp10:~# apt-get install powerpc-utils/zesty-proposed
> > Reading package lists... Done
> > Building dependency tree
> > Reading state information... Done
> > E: Release 'zesty-proposed' for 'powerpc-utils' was not found
>
> That is not valid apt syntax. Please
> see https://wiki.ubuntu.com/QATeam/PerformingSRUVerification for guidance on
> how to run tests for SRUs.

Tried the steps given in above link, still could not install the new package.

1. Added below in /etc/apt/sources.list

deb http://archive.ubuntu.com/ubuntu/ zesty-proposed main restricted universe

2. root@alp10:~# sudo apt-get update
Hit:1 http://us.ports.ubuntu.com/ubuntu-ports zesty InRelease
Get:2 http://us.ports.ubuntu.com/ubuntu-ports zesty-updates InRelease [89.2 kB]
Get:3 http://us.ports.ubuntu.com/ubuntu-ports zesty-backports InRelease [89.2 kB]
Get:4 http://ports.ubuntu.com/ubuntu-ports zesty-security InRelease [89.2 kB]
Get:5 http://archive.ubuntu.com/ubuntu zesty-proposed InRelease [240 kB]
Get:6 http://us.ports.ubuntu.com/ubuntu-ports zesty-updates/main ppc64el Packages [205 kB]
Ign:7 http://archive.ubuntu.com/ubuntu zesty-proposed/main ppc64el Packages
Get:8 http://archive.ubuntu.com/ubuntu zesty-proposed/main Translation-en [17.7 kB]
Ign:9 http://archive.ubuntu.com/ubuntu zesty-proposed/universe ppc64el Packages
Get:10 http://archive.ubuntu.com/ubuntu zesty-proposed/universe Translation-en [12.9 kB]
Ign:7 http://archive.ubuntu.com/ubuntu zesty-proposed/main ppc64el Packages
Ign:9 http://archive.ubuntu.com/ubuntu zesty-proposed/universe ppc64el Packages
Ign:7 http://archive.ubuntu.com/ubuntu zesty-proposed/main ppc64el Packages
Ign:9 http://archive.ubuntu.com/ubuntu zesty-proposed/universe ppc64el Packages
Err:7 http://archive.ubuntu.com/ubuntu zesty-proposed/main ppc64el Packages
404 Not Found [IP: 91.189.88.152 80]
Ign:9 http://archive.ubuntu.com/ubuntu zesty-proposed/universe ppc64el Packages
Fetched 743 kB in 1s (578 kB/s)
Reading package lists... Done
E: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/zesty-proposed/main/binary-ppc64el/Packages 404 Not Found [IP: 91.189.88.152 80]
E: Some index files failed to download. They have been ignored, or old ones used instead.
root@alp10:~# apt-get dist-upgrade^C
root@alp10:~# sudo apt-get install powerpc-utils
Reading package lists... Done
Building dependency tree
Reading state information... Done
powerpc-utils is already the newest version (1.3.2-1ubuntu2~17.04).
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

Thanks,
Pavithra

On Wed, Sep 27, 2017 at 07:29:56AM -0000, bugproxy wrote:
> > > root@alp10:~# apt-get install powerpc-utils/zesty-proposed
> > > Reading package lists... Done
> > > Building dependency tree
> > > Reading state information... Done
> > > E: Release 'zesty-proposed' for 'powerpc-utils' was not found

> > That is not valid apt syntax. Please
> > see https://wiki.ubuntu.com/QATeam/PerformingSRUVerification for guidance on
> > how to run tests for SRUs.

> Tried the steps given in above link, still could not install the new
> package.

> 1. Added below in /etc/apt/sources.list

> deb http://archive.ubuntu.com/ubuntu/ zesty-proposed main restricted
> universe

Ah - sorry, for ppc64el this needs to be:

 deb http://us.ports.ubuntu.com/ubuntu-ports/ zesty-proposed main restricted universe

I've updated the wiki documentation to match.

tags: added: id-5971d05a2dbce21bf929e27b
Brian Murray (brian-murray) wrote :

We are still awaiting verification of the package in -proposed.

------- Comment From <email address hidden> 2017-12-20 04:18 EDT-------
Installed powerpc-utils from - proposed. Issue is still observed.

~# sudo apt-get install powerpc-utils
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages will be upgraded:
powerpc-utils
1 upgraded, 0 newly installed, 0 to remove and 27 not upgraded.
Need to get 1,706 B of archives.
After this operation, 0 B of additional disk space will be used.
Get:1 http://us.ports.ubuntu.com/ubuntu-ports zesty-proposed/main ppc64el powerpc-utils ppc64el 1.3.2-1ubuntu3~17.04 [1,706 B]
Fetched 1,706 B in 0s (8,144 B/s)
(Reading database ... 51522 files and directories currently installed.)
Preparing to unpack .../powerpc-utils_1.3.2-1ubuntu3~17.04_ppc64el.deb ...
Unpacking powerpc-utils (1.3.2-1ubuntu3~17.04) over (1.3.2-1ubuntu2~17.04) ...
Setting up powerpc-utils (1.3.2-1ubuntu3~17.04) ...
:~# lsdevinfo
/usr/sbin/lsdevinfo: line 233: warning: command substitution: ignored null byte in input
device:
name="ibmveth2"
uniquetype="adapter/vdevice/IBM,l-lan"
class="adapter"
subclass="vdevice"
type="IBM,l-lan"
prefix="eth"
driver="ibmveth"
status="1"

Brian Murray (brian-murray) wrote :

I think the upstream patch for lsdevinfo is incomplete as there is another call to cat. From the code:

254 # get the physical location
255 physloc=$($CAT $dev/ibm,loc-code)
256 type="$($OD -t x2 $dev/vendor-id $dev/device-id |
257 $CUT -f3,5 -d ' ' -s --output-delimiter='')"

Do the other commands that were problematic (ls-vscsi, ls-veth, and ofpathname) work?

tags: added: verification-failed verification-failed-zesty
removed: verification-needed verification-needed-zesty
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2017-12-21 00:16 EDT-------
(In reply to comment #51)
> I think the upstream patch for lsdevinfo is incomplete as there is another
> call to cat. From the code:
>
> 254 # get the physical location
> 255 physloc=$($CAT $dev/ibm,loc-code)
> 256 type="$($OD -t x2 $dev/vendor-id $dev/device-id |
> 257 $CUT -f3,5 -d ' ' -s --output-delimiter='')"
>
> Do the other commands that were problematic (ls-vscsi, ls-veth, and
> ofpathname) work?

Issue is observed even with other commands.

:~# ls-vscsi
/usr/sbin/ls-vscsi: line 72: warning: command substitution: ignored null byte in input
host0 U8286.42A.1069B3T-V8-C17-T1
~# ls-veth
/usr/sbin/ls-veth: line 74: warning: command substitution: ignored null byte in input
ibmveth2 U8286.42A.1069B3T-V8-C2-T1
:~# ofpathname /dev/sda
/usr/sbin/ofpathname: line 812: warning: command substitution: ignored null byte in input
/usr/sbin/ofpathname: line 865: cd: too many arguments
/vdevice/v-scsi@30000011/disk@8100000000000000

Brian Murray (brian-murray) wrote :

The command substitution errors are happening in the same line number where the change from cat to tr was made. Could you confirm that the files installed in /usr/sbin/ are using the tr instead of cat?

bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2017-12-22 00:13 EDT-------
(In reply to comment #53)
> The command substitution errors are happening in the same line number where
> the change from cat to tr was made. Could you confirm that the files
> installed in /usr/sbin/ are using the tr instead of cat?

files installed in /usr/sbin/ still uses cat. but package installation was successful.

Thanks,
Pavithra

Brian Murray (brian-murray) wrote :

With regards to comment #17 I've opened the following issue upstream.

https://github.com/ibm-power-utilities/powerpc-utils/issues/24

bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2018-01-05 04:10 EDT-------
(In reply to comment #55)
> With regards to comment #17 I've opened the following issue upstream.
>
> https://github.com/ibm-power-utilities/powerpc-utils/issues/24

My commit already had those changes

https://github.com/ibm-power-utilities/powerpc-utils/commit/4fcc611a09caea70e6bca6a1275211c72ac4bb45

(In reply to comment #55)
> With regards to comment #17 I've opened the following issue upstream.
>
> https://github.com/ibm-power-utilities/powerpc-utils/issues/24

root@tuleta4u-lp7:/usr/sbin# dpkg -l | grep powerpc
ii powerpc-ibm-utils 1.3.2-1ubuntu3~17.04 ppc64el utilities for maintenance of IBM PowerPC platforms
ii powerpc-utils 1.3.2-1ubuntu3~17.04 ppc64el utilities for maintenance of PowerPC platforms (metapackage)
root@tuleta4u-lp7:/usr/sbin#

.
I am not able to see the issue in powerpc package 1.3.2-1ubuntu3~17.04.

Also could see files installed in /usr/sbin/ still uses tr.

.
Attaching the output of the commands.

------- Comment (attachment only) From <email address hidden> 2018-01-05 04:11 EDT-------

Brian Murray (brian-murray) wrote :

While I see that the commit did have the change to lsdevinfo around line 276 it looks like it somehow got reverted as trunk shows it still using cat.

https://github.com/ibm-power-utilities/powerpc-utils/blob/master/scripts/lsdevinfo#L276

------- Comment From <email address hidden> 2018-01-05 15:55 EDT-------
(In reply to comment #58)
> While I see that the commit did have the change to lsdevinfo around line 276
> it looks like it somehow got reverted as trunk shows it still using cat.
>
> https://github.com/ibm-power-utilities/powerpc-utils/blob/master/scripts/
> lsdevinfo#L276

Actually, Seeteena's patch only touched two places in the lsdevinfo script. Another patch hunk went in several months later that shifted the original line numbers, and is exposing to us that their are two other places in the script that use cat that weren't fixed. Namely, for PCI ethernet devices, and VFC devices.

[tyreld@oc6857751186 powerpc-utils]$ git blame scripts/lsdevinfo | grep "$dev/ibm,loc-code"
74715cc0 (Nathan Fontenot 2017-05-22 16:40:58 -0400 233) physloc=$(tr -d '\0' < $dev/ibm,loc-code)
4fcc611a (Seeteena Thoufeek 2017-01-11 15:36:33 +0530 254) physloc=$(tr -d '\0' < $dev/ibm,loc-code)
c600f291 (Santiago Leon 2012-04-16 08:49:18 -0500 276) physloc=$($CAT $dev/ibm,loc-code)
4fcc611a (Seeteena Thoufeek 2017-01-11 15:36:33 +0530 296) physloc=$(tr -d '\0' < $dev/ibm,loc-code)
8a8ff90e (Santiago Leon 2009-09-21 14:16:35 -0500 441) physloc=$(cat $dev/ibm,loc-code)

bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2018-01-05 16:19 EDT-------
I've sent a secondary patch out to the mailing list to fixup the two remaining problem spots in lsdevinfo.

https://groups.google.com/forum/#!topic/powerpc-utils-devel/QrB58JnT3Wk

If nobody has any issues I can pull it into the "next" branch immediately.

bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2018-01-10 03:27 EDT-------
(In reply to comment #60)
> I've sent a secondary patch out to the mailing list to fixup the two
> remaining problem spots in lsdevinfo.
>
> https://groups.google.com/forum/#!topic/powerpc-utils-devel/QrB58JnT3Wk
>
> If nobody has any issues I can pull it into the "next" branch immediately.

Thanks Tyrel.

Changed in powerpc-utils (Ubuntu):
status: Fix Released → Triaged
Changed in powerpc-utils (Ubuntu Zesty):
status: Fix Committed → Won't Fix
Changed in powerpc-utils (Ubuntu):
assignee: Canonical Foundations Team (canonical-foundations) → Adam Conrad (adconrad)
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.