rename.ul refuses to rename links that don't resolve (regression)

[gregrwm]

[Impact]

rename.ul refuses to rename links that don't resolve.

bionic rename.ul renames links whether or not they resolve (util-linux 2.31.1-0.4ubuntu3.6).

focal rename.ul refuses to rename links that don't resolve (regression) (util-linux 2.34-0.1ubuntu9).

Eoan is EOL, so removed.

[Test case]

Commit has good test case. I paste it here

Before:

  $ touch file-found
  $ ln -s file-found symlink-1
  $ ./rename sym symbolic- symlink-1 # XPASS.
  $ echo $?
  0

  $ ln -s file-not-found symlink-2
  $ ./rename sym symbolic- symlink-2 # FAIL! REGRESSION.
  rename: symlink-2: not accessible: No such file or directory
  $ echo $?
  1

  $ ./rename sym symbolic- symlink-3 # XFAIL.
  rename: symlink-3: not accessible: No such file or directory
  $ echo $?
  1

  $ touch file-found
  $ ./rename found existing file-found # XPASS.
  $ echo $?
  0

  $ ./rename found existing file-not-found # XFAIL.
  rename: file-not-found: not accessible: No such file or directory
  $ echo $?
  1

After:

  $ touch file-found
  $ ln -s file-found symlink-1
  $ ./rename sym symbolic- symlink-1 # XPASS.
  $ echo $?
  0

  $ ln -s file-not-found symlink-2
  $ ./rename sym symbolic- symlink-2 # PASS! REGRESSION FIXED.
  $ echo $?
  0

  $ ./rename sym symbolic- symlink-3 # XFAIL.
  rename: symlink-3: not accessible: No such file or directory
  $ echo $?
  1

  $ touch file-found
  $ ./rename found existing file-found # XPASS.
  $ echo $?
  0

  $ ./rename found existing file-not-found # XFAIL.
  rename: file-not-found: not accessible: No such file or directory
  $ echo $?
  1

[Regression Potential]
This commit changes code that checking accessibility to file or sym link.
If this fails, rename command will not work properly.it could be crashed or causes failure of rename.

[Original Description]

rename.ul refuses to rename links that don't resolve.

bionic rename.ul renames links whether or not they resolve (util-linux 2.31.1-0.4ubuntu3.6).

focal rename.ul refuses to rename links that don't resolve (regression) (util-linux 2.34-0.1ubuntu9).

ProblemType: Bug
DistroRelease: Ubuntu 20.04
Package: util-linux 2.34-0.1ubuntu9
ProcVersionSignature: Ubuntu 5.4.0-21.25-generic 5.4.27
Uname: Linux 5.4.0-21-generic x86_64
ApportVersion: 2.20.11-0ubuntu27.3
Architecture: amd64
CasperMD5CheckResult: skip
Date: Sat Jul 4 20:38:06 2020
InstallationDate: Installed on 2020-04-05 (90 days ago)
InstallationMedia: Ubuntu 20.04 LTS "Focal Fossa" - Beta amd64 (20200331)
ProcEnviron:
 LC_TIME=en_DK.utf8
 TERM=screen
 PATH=(custom, no user)
 LANG=en_US.utf8
 SHELL=/bin/bash
SourcePackage: util-linux
UpgradeStatus: No upgrade log present (probably fresh install)

[Mauricio Faria de Oliveira (mfo)]

Hi @gregrwm,

Thanks for reporting this bug/regression.

Patch just submitted upstream to fix it [1].
Let's see how it goes.

cheers,
Mauricio

[1] https://github.com/karelzak/util-linux/pull/1095

[Mauricio Faria de Oliveira (mfo)]

Patch applied upstream [1].
Next steps are to apply it downstream in Debian and Ubuntu.

[1] https://git.kernel.org/pub/scm/utils/util-linux/util-linux.git/commit/?id=477239ce0d60384642b51c950688136fdefec815

[Mauricio Faria de Oliveira (mfo)]

The offending commit has been introduced in util-linux v2.33,
so the fix should go into Eoan (v2.34) and later.

$ git describe --contains 5454df9
v2.33-rc1~309^2~9

$ rmadison -a source util-linux
 util-linux | 2.20.1-1ubuntu3 | precise | source
 util-linux | 2.20.1-1ubuntu3.1 | precise-updates | source
 util-linux | 2.20.1-5.1ubuntu20 | trusty | source
 util-linux | 2.20.1-5.1ubuntu20.9 | trusty-updates | source
 util-linux | 2.27.1-6ubuntu3 | xenial | source
 util-linux | 2.27.1-6ubuntu3.10 | xenial-updates | source
 util-linux | 2.31.1-0.4ubuntu3 | bionic | source
 util-linux | 2.31.1-0.4ubuntu3.6 | bionic-updates | source
 util-linux | 2.34-0.1ubuntu2 | eoan | source
 util-linux | 2.34-0.1ubuntu2.4 | eoan-updates | source
 util-linux | 2.34-0.1ubuntu9 | focal | source
 util-linux | 2.35.2-5ubuntu3 | groovy | source

[Mauricio Faria de Oliveira (mfo)]

Hi Seyeong!

Thank you for taking this LP bug.
I'm setting myself as sponsor for once you have the debdiffs ready.

Please note that Eoan is EOL now, so please feel free to adjust it.

And it would be great to have this reported to Debian as well, since it's an upstream bug/fix.

Please let me know if you'd like pointers about the procedure to report bugs to the Debian Bug Tracking System (BTS).

cheers,
Mauricio

[Ubuntu Foundations Team Bug Bot (crichton)]

The attachment "lp1886300_groovy.debdiff" seems to be a debdiff. The ubuntu-sponsors team has been subscribed to the bug report so that they can review and hopefully sponsor the debdiff. If the attachment isn't a patch, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are member of the ~ubuntu-sponsors, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issue please contact him.]

[Mauricio Faria de Oliveira (mfo)]

Hi Seyeong,

Thanks for the debdiffs!
They look good in general but need a few fixes:

1) groovy-proposed has util-linux 2.35.2-9ubuntu1 (debdiff used groovy-updates / 2.35.2-7ubuntu3)

2) the DEP3 headers should be before a triple-dash "---" line per [1].
   I'd suggest moving it to before/after the git format-patch headers.

3) Could you please report a bug to Debian and add the Bug-Debian DEP3 header?
   Debian sid/unstable is still on 2.35.2-8 (fix landed in v2.36-rc2).
   No need to wait on them, but just have the bug reported there too.

Did you build this on Launchpad PPAs?
Sometimes there are differences to local build environments that lead
to "interesting" issues to debug when it builds on PPAs. :-)

[1] https://dep-team.pages.debian.net/deps/dep3/
"A line containing 3 dashes (---) should stop the parsing: lines after it are not relevant part of the meta-information."

Thanks,
Mauricio

[Seyeong Kim (seyeongkim)]

@mfo

Thanks for your advice, re-uploaded patches

[Mauricio Faria de Oliveira (mfo)]

Hi Seyeong,

Thanks for the updated debdiffs and reporting Debian bug 966106
(Please feel free to comment in the LP bug when you report a Debian bug.)

Per the Debian maintainer, they recently stopped shipping/installing rename.ul, on 2.35.2-5.
Fortunately this only affects Groovy and later (non LTS.)

So, Debian moved to 2.36 (which contains the fix) and Groovy followed, with a 2.36-based package in -proposed right now.
This is good as we don't need the debdiff/core-developer review/upload to Groovy/devel release.

I'm watching the -proposed migration on Groovy (had to check re-run some failed autopkgtests [1] which are blocking it.)

And after Groovy has 2.36 released, I'll upload to Focal.
(This week Focal is in release freeze, so not a problem.)

cheers,
Mauricio

[1] https://people.canonical.com/~ubuntu-archive/proposed-migration/update_excuses.html

[Mauricio Faria de Oliveira (mfo)]

Hi Seyeong,

The proposed-migration of util-linux in groovy is blocked on one last autopkgtest of gfs2-utils failing on ppc64el (the 'mount' test times out; reproducible even w/ previous versions) [1, 2]

This is new and unrelated to these changes (not uploaded to groovy), but prevents uploading these changes to Focal as they have to be on Groovy first.

So, if you have some time and would like to learn/practice/understand more about how autopkgtests work, and try to repro the bug on the ppc64el architecture, that would be great. :)

Otherwise I can try to take a look at it when back from time off.

cheers,
Mauricio

[1] https://people.canonical.com/~ubuntu-archive/proposed-migration/update_excuses.html
(search for "util-linux (2.35.2-9ubuntu1 to 2.36-2ubuntu1)")
[2] https://autopkgtest.ubuntu.com/packages/g/gfs2-utils/groovy/ppc64el

[Mauricio Faria de Oliveira (mfo)]

Groovy now has the fix released in util-linux 2.36-2ubuntu1. Focal can now move on.

I've changed the debdiff slightly on the DEP3 headers and version number, small nitpicks.

The package builds fine on all architectures in a PPA, and testing is successful:

focal-release / fail:

$ dpkg -s util-linux | grep -i version:
Version: 2.34-0.1ubuntu9

$ ln -sf file-not-found symlink-2

$ rename.ul sym symbolic- symlink-2
rename.ul: symlink-2: not accessible: No such file or directory

ppa build / pass:

$ sudo add-apt-repository ppa:mfo/lp1886300
$ sudo apt install util-linux

$ dpkg -s util-linux | grep -i version:
Version: 2.34-0.1ubuntu9.1

$ ln -sf file-not-found symlink-2
$ rename.ul sym symbolic- symlink-2
$

$ ls -l sym*
<...> symbolic-link-2 -> file-not-found

[Mauricio Faria de Oliveira (mfo)]

Uploaded to Focal.

Seyeong, could you please update the Test Case section of the SRU template
to reflect the usage of Ubuntu commands and not upstream/built commands?

Thanks,
Mauricio

[Brian Murray (brian-murray)]

Hello gregrwm, or anyone else affected,

Accepted util-linux into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/util-linux/2.34-0.1ubuntu9.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted util-linux (2.34-0.1ubuntu9.1) for focal have finished running.
The following regressions have been reported in tests triggered by the package:

gfs2-utils/3.2.0-3 (ppc64el)
systemd/245.4-4ubuntu3.2 (s390x, amd64, ppc64el)
tracker/2.3.4-1 (armhf)
kopanocore/8.7.0-7ubuntu1 (ppc64el, armhf, amd64)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/focal/update_excuses.html#util-linux

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Mauricio Faria de Oliveira (mfo)]

Autopkgstests failures:
- gfs2-utils/ppc64el: unrelated; I'll take a look (it held groovy's proposed migration weeks ago.)
- systemd: likely unrelated, see bug 1892358 (success rate dropped)
- tracker: flaky; now passing.
- kopanocore: flaky; now passing.

[gregrwm (gregrwm)]

works. test cases pasted in description all pass or fail correctly.

$ dpkg-query -W util-linux
util-linux 2.34-0.1ubuntu9.1

didn't see a 'verification-needed-focal' tag, but added a 'verification-done-focal' tag anyway.

[gregrwm (gregrwm)]

details of verification test. note return status is first digit in subsequent prompt.

  0$00uf$Aug27Thu22:22$greg@e540:1:/tmp/c$ touch file-found
  0$00uf$Aug27Thu22:23$greg@e540:1:/tmp/c$ ln -s file-found symlink-1
  0$00uf$Aug27Thu22:23$greg@e540:1:/tmp/c$ rename.ul sym symbolic- symlink-1 # XPASS.
  0$00uf$Aug27Thu22:23$greg@e540:1:/tmp/c$ ln -s file-not-found symlink-2
  0$00uf$Aug27Thu22:24$greg@e540:1:/tmp/c$ rename.ul sym symbolic- symlink-2 # PASS! REGRESSION FIXED.
  0$00uf$Aug27Thu22:24$greg@e540:1:/tmp/c$ rename.ul sym symbolic- symlink-3 # XFAIL.
rename.ul: symlink-3: not accessible: No such file or directory
  1$00uf$Aug27Thu22:24$greg@e540:1:/tmp/c$ touch file-found
  0$00uf$Aug27Thu22:24$greg@e540:1:/tmp/c$ rename.ul found existing file-found # XPASS.
  0$00uf$Aug27Thu22:24$greg@e540:1:/tmp/c$ rename.ul found existing file-not-found # XFAIL.
rename.ul: file-not-found: not accessible: No such file or directory
  1$00uf$Aug27Thu22:25$greg@e540:1:/tmp/c$

thanks!

[Mauricio Faria de Oliveira (mfo)]

Thanks for verifying focal-proposed (and adding the tag.)

Oh, I forgot to mark Groovy as Fix Released on comment #15; doing it now.

[Launchpad Janitor (janitor)]

This bug was fixed in the package util-linux - 2.34-0.1ubuntu9.1

---------------
util-linux (2.34-0.1ubuntu9.1) focal; urgency=medium

  * d/p/rename_fix_regression_for_symlink_with_non-existing_target.patch
    - rename: fix regression for symlink with non-existing target
      (LP: #1886300)

 -- Seyeong Kim <email address hidden> Tue, 21 Jul 2020 16:49:28 +0900

[Brian Murray (brian-murray)]

The verification of the Stable Release Update for util-linux has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.