various packages failed to install/upgrade: ErrorMessage: unable to make backup link of <some file> before installing new version: Input/output error

[Michael Martin-Smucker]

Fresh installation of Beta 2, and I got this error while updating for the first time. Hopefully any other necessary information will be automatically included, but if not, I'll gladly help debug however I can.

ProblemType: Package
DistroRelease: Ubuntu 10.04
Package: libglibmm-2.4-1c2a 2.24.0-0ubuntu1
ProcVersionSignature: Ubuntu 2.6.32-19.28-generic 2.6.32.10+drm33.1
Uname: Linux 2.6.32-19-generic x86_64
Architecture: amd64
Date: Fri Apr 9 11:15:14 2010
ErrorMessage: ErrorMessage: unable to make backup link of `./usr/share/doc/libglibmm-2.4-1c2a/README' before installing new version: Input/output error
LiveMediaBuild: Ubuntu 10.04 "Lucid Lynx" - Beta amd64 (20100406.1)
SourcePackage: glibmm2.4
Title: package libglibmm-2.4-1c2a 2.24.0-0ubuntu1 failed to install/upgrade: ErrorMessage: unable to make backup link of `./usr/share/doc/libglibmm-2.4-1c2a/README' before installing new version: Input/output error

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Martin Pitt (pitti)]

There's a ton of similar errors (https://bugs.launchpad.net/bugs/+bugs?field.searchtext=unable+to+make+backup+link), I'll duplicate them together and assign to dpkg for now. This could also be a kernel issue, see bug 1498162, although that particular issue was for a different case.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[TJ (tj)]

I've just encountered this with 16.04 whilst developing a bash script that cross-builds (on amd64) a complete custom armhf RasPi 2/3 installation image.

It triggered when linux-image packages were being unpacked by 'dpkg' to the chroot target /boot/ directory.

The amd64 system has qemu-user-static and binfmt-support installed so that commands can be executed in the armhf target chroot.

The command:

$ sudo chroot /mnt/raspi dpkg -i /var/local/cache/apt/archives/linux-image-4.5.0-v7+_4.5.0-v7+-2_armhf.deb
(Reading database ... 30501 files and directories currently installed.)
Preparing to unpack .../linux-image-4.5.0-v7+_4.5.0-v7+-2_armhf.deb ...
Unpacking linux-image-4.5.0-v7+ (4.5.0-v7+-2) over (4.5.0-v7+-2) ...
dpkg: error processing archive /var/local/cache/apt/archives/linux-image-4.5.0-v7+_4.5.0-v7+-2_armhf.deb (--install):
 unable to make backup link of './boot/config-4.5.0-v7+' before installing new version: Operation not permitted
dpkg-deb: error: subprocess paste was killed by signal (Broken pipe)
Errors were encountered while processing:
 /var/local/cache/apt/archives/linux-image-4.5.0-v7+_4.5.0-v7+-2_armhf.deb

and strace reveals:

13605 utimes("/boot/config-4.5.0-v7+.dpkg-new", {{1462489111, 0}, {1458064430, 0}}) = 0
13605 link("/boot/config-4.5.0-v7+", "/boot/config-4.5.0-v7+.dpkg-tmp") = -1 EPERM (Operation not permitted)

RasPi images require the boot file-system to be VFAT:

$ mount | grep /mnt/raspi/boot
/dev/mapper/VG_DATA-raspi1 on /mnt/raspi/boot type vfat (rw,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=iso8859-1,shortname=mixed,errors=remount-ro)

$ sudo blkid /dev/mapper/VG_DATA-raspi1
/dev/mapper/VG_DATA-raspi1: SEC_TYPE="msdos" LABEL="RASPI_4084" UUID="C5B8-3907" TYPE="vfat" PARTLABEL="primary" PARTUUID="009a2020-4eec-451a-9bb7-b18241729fd1"

As far as I recall it isn't possible to hard link on VFAT.

[TJ (tj)]

Additional debug info using dpkg -D7777 ...

...
D000100: setupvnamevbs main='/boot' tmp='/boot.dpkg-tmp' new='/boot.dpkg-new'
D000100: tarobject already exists
D000100: tarobject directory exists
D000010: tarobject ti->name='./boot/config-4.5.0-v7+' mode=100644 owner=0:0 type=48(-) ti->linkname='' namenode='/boot/config-4.5.0-v7+' flags=2 instead='<non
e>'
D000100: setupvnamevbs main='/boot/config-4.5.0-v7+' tmp='/boot/config-4.5.0-v7+.dpkg-tmp' new='/boot/config-4.5.0-v7+.dpkg-new'
D000100: tarobject already exists
D000010: path_remove_tree '/boot/config-4.5.0-v7+.dpkg-new'
D000010: path_remove_tree '/boot/config-4.5.0-v7+.dpkg-tmp'
D000100: tarobject file open size=133889
D000100: tarobject file hash=2666c2e844c3e9ce3a7121842743ed95
D000100: tarobject nondirectory, 'link' backup
dpkg: error processing archive /var/local/cache/apt/archives/linux-image-4.5.0-v7+_4.5.0-v7+-2_armhf.deb (--install):
 unable to make backup link of './boot/config-4.5.0-v7+' before installing new version: Operation not permitted
D000010: cu_installnew '/boot/config-4.5.0-v7+' flags=202
D000100: setupvnamevbs main='/boot/config-4.5.0-v7+' tmp='/boot/config-4.5.0-v7+.dpkg-tmp' new='/boot/config-4.5.0-v7+.dpkg-new'
D000100: cu_installnew not restoring
D000100: secure_remove '/boot/config-4.5.0-v7+.dpkg-new' unlink OK
...

[TJ (tj)]

Just realised this bug is specific to the Input/Output error so I've filed mine as bug 1578863 "unable to make backup link of <some-file> before installing new version: Operation not permitted". Sorry for the noise.

[Dan Kegel (dank)]

So... what was the fix?