tar: futimens() with a bad file descriptor (AT_FDCWD) causes bootstrapping failure with kernels < 2.6.22

Bug #539814 reported by Andrew Pollock
106
This bug affects 16 people
Affects Status Importance Assigned to Milestone
Release Notes for Ubuntu
Fix Released
Undecided
Colin Watson
debootstrap (Ubuntu)
Triaged
Wishlist
Unassigned
Lucid
Won't Fix
Wishlist
Unassigned
tar (Debian)
Fix Released
Unknown
tar (Fedora)
Fix Released
High
tar (Ubuntu)
Fix Released
Medium
Unassigned
Lucid
Fix Released
Medium
Matthias Klose

Bug Description

Binary package hint: tar

Bootstrapping Ubuntu Lucid with debootstrap is currently impossible with any kernel < 2.6.22, due to a bug in tar - specifically it fails because of a change to eglibc.

According to the debian bug report It seems that before eglibc 2.10.2-3, calls to futimens failed silently, however due to a patch made to eglibc 2.10.2-3, if futimens is called without a valid file descriptor, it now fails with an "Bad File Descriptor" error, because the corresponding utimensat() syscall was not added until kernel 2.6.22

Test Case:
1) Boot a kernel older than 2.6.22
2) Attempt to Bootstrap Ubuntu Lucid to any target.
The bootstrap will fail once it attempts to unpack packages with: tar: ./postinst: Cannot utime: Bad file descriptor

A patch is available (attached in launchpad) originating from the associated debian bug, that ensures that futimens is only called with a valid file descriptor.
A debdiff for 1.22-2 ( changing to 1.22-2ubuntu1 ) will be attached shortly.

Related branches

Revision history for this message
In , Michal (michal-redhat-bugs) wrote :

Description of problem:
When I create tar archive with new tar (tar-1.22-10.fc12.x86_64) old tar is not able to open this archive.

Version-Release number of selected component (if applicable):
tar-1.22-10.fc12.x86_64

How reproducible:
always

Steps to Reproduce:
1.create archive with new tar
2.try to extract it with old tar
3.

Actual results:
/bin/tar: squirrelmail-1.4.20-RC2/images/folder.png: Cannot utime: Bad file descriptor

I've got this error in koji when updating package:

http://koji.fedoraproject.org/koji/getfile?taskID=1901164&name=build.log

Expected results:
extracted archive

Additional info:
this breaks package updates where maintainer has to recreate tar archive. Also upstream tarballs where upstream uses Fedora system may produce broken tarballs which is even worse, so I consider this to be high severity.

Revision history for this message
In , Kamil (kamil-redhat-bugs) wrote :

We figured out off-list the "old tar" from original report is 1.22-11.fc13, which is more likely a newer one :-)

Revision history for this message
In , Ondrej (ondrej-redhat-bugs) wrote :

As a sidenote - there were more such reports off-the-bugzlla - most likely caused by recent change in the new glibc - anyway - as the glibc maintainers are quite sure the problem is in tar, we should probably fix it.

Revision history for this message
In , Ondrej (ondrej-redhat-bugs) wrote :

> > That looks like a bug in tar, trying to call futimens with a bad file
> > descriptor.
> > Andreas Schwab
> Probably, possibly exposed by this recent change in glibc-2.11.90-5:
> - Handle AT_FDCWD in futimens (BZ#10992).
> Tomas Bzatek

Adding relevant part of the off-the-bugzilla conversation.

Revision history for this message
In , Ondrej (ondrej-redhat-bugs) wrote :

Here is duplicate report in debian (they have same problems with eglibc updated to POSIX2008) - patch fixing the issue is available there - http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=563726 - However - I tend to use latest gnulib lib/utimens.c - though - as they are other utimens/utimensat call related fixes.

Revision history for this message
In , Ondrej (ondrej-redhat-bugs) wrote :

Ah - even the latest gnulib doesn't handle this new recent change - so touch from coreutils might be affected as well as mentioned in debian bugzilla from comment #4. Therefore adding coreutils/gnulib upstream maintainer into CC.

Jim, what do you think? Is the patch from http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=563726#66 good way to go and should be applied to gnulib/coreutils-8.3 to prevent touch (or better said futimens() with AT_FDCWD file descriptor) failures after glibc update?

Revision history for this message
In , Ondrej (ondrej-redhat-bugs) wrote :

Fixed in tar-1.22-12.fc13, closing RAWHIDE.

Revision history for this message
In , Jim (jim-redhat-bugs) wrote :

Hi Ondrej,

Thanks again for the heads up.
I've replied on bug-gnulib and Cc'd the debian bug address.
(too bad bugzilla doesn't have a reply-to-via-email option):

http://bugs.debian.org/563726#88

Revision history for this message
In , Ondrej (ondrej-redhat-bugs) wrote :

Thanks Jim for replies/confirmation of bugfix correctness. Yep - it would be good bugzilla improvement to have such thing.

Revision history for this message
Andrew Pollock (apollock) wrote :

I think this issue is fixed in tar 1.23, which is currently in Debian unstable.

Revision history for this message
Andrew Pollock (apollock) wrote :

I've confirmed that 1.23-1 from Debian unstable fixes the problem.

Victor Vargas (kamus)
Changed in tar (Ubuntu):
importance: Undecided → Medium
Revision history for this message
Nathan Stratton Treadway (nathanst) wrote :

I just ran into this while testing the upgrade from hardy to lucid, on a server machine that was running linux 2.6.18. The do-release-update worked fine and successfully unpacked and installed all the new packages... but when I tried to run "aptitude install" immediately after the upgrade script finished, the aptitude/dpkg/tar combo failed (with the same error messages as shown in the initial bug report).

Extracting the "tar" binary from the Debian 1.23 tar package (using a different machine) and copying it into /bin/tar on the test machine allowed aptitude/dpkg to start working again.

I know I will need to upgrade the kernel on this machine for other reasons, but given that having package installs suddenly start failing in the middle of an upgrade session is both surprising and inconvenient, it seems like it might be worth pushing to get tar 1.23 into lucid....

Revision history for this message
Andrew Pollock (apollock) wrote : Re: FFe: Sync tar 1.23-1 (main) from Debian squeeze (main)
summary: - Impossible to bootstrap Lucid on Dapper
+ FFe: Sync tar 1.23-1 (main) from Debian squeeze (main)
Revision history for this message
Andrew Pollock (apollock) wrote :

The build log for when this was autobuilt in Debian for AMD64 is at https://buildd.debian.org/fetch.cgi?pkg=tar;ver=1.23-1;arch=amd64;stamp=1268242055

Revision history for this message
Andrew Pollock (apollock) wrote :

Here's the upgrade of tar from 1.22-2 to 1.23-1:

(Reading database ... 231026 files and directories currently installed.)
Preparing to replace tar 1.22-2 (using /tmp/tar_1.23-1_amd64.deb) ...
Unpacking replacement tar ...
Setting up tar (1.23-1) ...

Processing triggers for man-db ...

Revision history for this message
Andrew Pollock (apollock) wrote :
Download full text (113.8 KiB)

Here's some evidence of tar continuing to work:

$ tar -tvzf partman-crypto_40ubuntu1.tar.gz

drwxr-xr-x 0/0 0 2009-12-05 14:28 partman-crypto/
drwxr-xr-x 0/0 0 2007-11-29 06:23 partman-crypto/choose_method/
drwxr-xr-x 0/0 0 2009-08-31 18:39 partman-crypto/choose_method/crypto/
-rwxr-xr-x 0/0 311 2007-12-12 02:18 partman-crypto/choose_method/crypto/do_option
-rwxr-xr-x 0/0 274 2007-12-12 02:18 partman-crypto/choose_method/crypto/choices
-rw-r--r-- 0/0 10 2007-11-29 06:23 partman-crypto/choose_method/_numbers
drwxr-xr-x 0/0 0 2007-11-29 06:23 partman-crypto/choose_partition/
drwxr-xr-x 0/0 0 2009-11-10 06:14 partman-crypto/choose_partition/crypto/
-rwxr-xr-x 0/0 2531 2009-11-10 06:04 partman-crypto/choose_partition/crypto/do_option
-rwxr-xr-x 0/0 301 2009-07-20 03:47 partman-crypto/choose_partition/crypto/choices
-rw-r--r-- 0/0 10 2007-11-29 06:23 partman-crypto/choose_partition/_numbers
drwxr-xr-x 0/0 0 2009-12-05 14:30 partman-crypto/debian/
drwxr-xr-x 0/0 0 2009-12-05 14:26 partman-crypto/debian/po/
-rw-r--r-- 0/0 25321 2009-11-10 05:15 partman-crypto/debian/po/hu.po
-rw-r--r-- 0/0 25770 2009-11-10 04:49 partman-crypto/debian/po/cs.po
-rw-r--r-- 0/0 25074 2009-12-05 14:26 partman-crypto/debian/po/et.po
-rw-r--r-- 0/0 26352 2009-11-10 05:36 partman-crypto/debian/po/pt.po
-rw-r--r-- 0/0 25917 2009-11-10 05:16 partman-crypto/debian/po/id.po
-rw-r--r-- 0/0 25716 2009-12-05 14:26 partman-crypto/debian/po/fi.po
-rw-r--r-- 0/0 36093 2009-11-10 05:32 partman-crypto/debian/po/ne.po
-rw-r--r-- 0/0 19335 2009-11-10 05:11 partman-crypto/debian/po/ga.po
-rw-r--r-- 0/0 26615 2009-12-05 14:26 partman-crypto/debian/po/tr.po
-rw-r--r-- 0/0 35656 2009-11-10 05:21 partman-crypto/debian/po/ka.po
-rw-r--r-- 0/0 25478 2009-11-10 06:01 partman-crypto/debian/po/wo.po
-rw-r--r-- 0/0 26020 2009-11-10 05:02 partman-crypto/debian/po/eu.po
-rw-r--r-- 0/0 28050 2009-12-05 14:26 partman-crypto/debian/po/ar.po
-rw-r--r-- 0/0 7 2007-11-29 06:23 partman-crypto/debian/po/output
-rw-r--r-- 0/0 18728 2009-11-10 06:02 partman-crypto/debian/po/templates.pot
-rw-r--r-- 0/0 24646 2009-12-05 14:26 partman-crypto/debian/po/am.po
-rw-r--r-- 0/0 26029 2009-11-10 05:32 partman-crypto/debian/po/nn.po
-rw-r--r-- 0/0 34054 2009-12-05 14:26 partman-crypto/debian/po/el.po
-rw-r--r-- 0/0 30469 2009-11-10 04:45 partman-crypto/debian/po/be.po
-rw-r--r-- 0/0 27701 2009-11-10 05:12 partman-crypto/debian/po/he.po
-rw-r--r-- 0/0 26436 2009-11-10 05:59 partman-crypto/debian/po/tl.po
-rw-r--r-- 0/0 38672 2009-11-10 05:22 partman-crypto/debian/po/km.po
-rw-r--r-- 0/0 29409 2009-11-10 05:21 partman-crypto/debian/po/kk.po
-rw-r--r-- 0/0 19272 2009-11-10 04:49 partman-crypto/debian/po/cy.po
-rw-r--r-- 0/0 27707 2009-12-05 14:26 partman-crypto/debian/po/de.po
-rw-r--r-- 0/0 311...

Revision history for this message
Nathan Stratton Treadway (nathanst) wrote :

For what it's worth, a couple of related notes:

I found that Bug #503109 is actually about this same problem, but on the powerpc platform. In that case, the solution was to roll back, on the powerpc plotform only, the eglibc patch that triggered the behavior.

Also, here is the direct link to the git commit contianing the specific fix for this problem:
  http://git.savannah.gnu.org/cgit/gnulib.git/commit/?id=41c44a66101c6c14b4e7f151247d517579760ba9

Changed in tar (Debian):
status: Unknown → Fix Released
Revision history for this message
Steve Langasek (vorlon) wrote :

The SIGPIPE changes concern me; I know SIGPIPE handling in and around tar has been a fiddly thing in the past, and a quick web search tells me that tar 1.23's SIGPIPE handling is thought to have a regression vs. 1.22. So this looks pretty risky to me; I would prefer a cherry pick of the high-priority fix here, as I don't think we can count on the upstream discussion of the regression to settle in time for release.

Revision history for this message
Dmitry Nedospasov (dmitry-nedos) wrote :

Is there any chance that this will get fixed before the lucid release. I find this quite annoying as i have server machines running 2.6.18 (mostly rhel/centos) with which i want to use debootstrap to install lucid for chroot or for virtual machines. I really don't want to have to "hack" debootstrap to make it compatible with my 2.6.18 kernel. Would really appreciate any feedback.

Revision history for this message
Andrew Pollock (apollock) wrote :

I personally don't have the bandwidth to cherry-pick just the parts of 1.23 that are needed to fix this bug. If anyone else is able to come up with a diff between 1.22 and 1.23 that has just the fixes necessary for this bug, that's going to be the way to get it fixed.

Revision history for this message
Nathan Stratton Treadway (nathanst) wrote :

I am no expert on tar/gnulib, but as far as I can tell from reading the Debian bug and related discussions, the gnulib commit that I mentioned earlier (i.e.
http://git.savannah.gnu.org/cgit/gnulib.git/commit/?id=41c44a66101c6c14b4e7f151247d517579760ba9
) is all that is needed to fix this problem.

Revision history for this message
Steve Langasek (vorlon) wrote :

At this point there's no room to get this in for the 10.04.0 release, but a cherry picked patch should be a shoe-in for an SRU immediately afterwards.

Revision history for this message
Dmitry Nedospasov (dmitry-nedos) wrote :

Like i said, i'm just sure that I'm not the only planning to use debootstrap quite extensively, i would go as far as to say that this is my main install method. If this makes it in shortly after release, then thats fine as well.

Revision history for this message
Nathan Stratton Treadway (nathanst) wrote :

Given that this won't be fixed in time for the release, any chance that an item could be added to the
http://www.ubuntu.com/getubuntu/releasenotes/1004
page warning people that installing or upgrading on older kernels will fail because of this issue (along with bug #552816)?

(As mentioned in this bug report, it definitely fails on 2.6.18; based on the discussionin Debian BTS #563726 it sounds like this may apply to kernels uip to 2.6.21, though I don't have any way to test anything other than 2.6.18 myself.)

Nathan

Revision history for this message
Steve Langasek (vorlon) wrote :

It's a struggle to keep the release notes down to a length that users might even consider reading them; since this issue only affects upgrades on systems with unsupported kernels (i.e. chroots) and the bug reports should be reasonably discoverable after the fact, I don't think it makes sense to include this in the release notes.

Revision history for this message
Scott Kitterman (kitterman) wrote :

Sorry. Missed the bit about not putting it in the release notes until after I'd added the project.

Changed in ubuntu-release-notes:
status: New → Won't Fix
Revision history for this message
Nathan Stratton Treadway (nathanst) wrote :

Can you point me to a place that documents which kernels are considered "supported"?

Revision history for this message
Steve Langasek (vorlon) wrote :

The supported kernels are the ones included in the Ubuntu release in question, or that shipped in the previous Ubuntu release for which an upgrade path is supported.

Revision history for this message
Dmitry Nedospasov (dmitry-nedos) wrote :

Steve,

If this is the case, one could make the claim, that Dapper servers will also have this problem as they should be supported until 2011 iirc? The stable kernel in Dapper is 2.6.15.

On a side note this will affect most Xen and XenServer Server's (which almost all run 2.6.18) which imho is more than you think. These are also the systems where i first noticed this issue. I consider Xen to be a very popular platform for Ubuntu to run on. A lot of VPS hosters would be able to offer Lucid if this were fixed. Also a lot of VPS hosters do not allow the user to load a newer kernel in which case it is even more important that this is fixed.

But I didn't quite understand, will this get rolled out into updates pretty quickly, so that debootstrap does work? AFAIK debootstrap will then function as soon as the tar package lands in updates, as it will use the tar package from updates for the installation.

Revision history for this message
Andrew Pollock (apollock) wrote : Re: Impossible to bootstrap Lucid on Dapper
summary: - FFe: Sync tar 1.23-1 (main) from Debian squeeze (main)
+ Impossible to bootstrap Lucid on Dapper
Revision history for this message
Andrew Pollock (apollock) wrote :
Revision history for this message
Andrew Pollock (apollock) wrote :

It'd be really awesome if one of the other people affected by this bug could rebuild Lucid's tar using the debdiff I've attached, and provide some feedback on how it goes with a kernel < 2.6.22

summary: - Impossible to bootstrap Lucid on Dapper
+ tar/gnulib bug prevents bootstraping Lucid on Dapper, causes Lucid
+ upgrade to abort on older linux kernels (e.g. in a chroot or Xen VM)
Revision history for this message
Dmitry Nedospasov (dmitry-nedos) wrote : Re: tar/gnulib bug prevents bootstraping Lucid on Dapper, causes Lucid upgrade to abort on older linux kernels (e.g. in a chroot or Xen VM)

Snippet from 2.6.18 kernel ouput with old tar and with the patched version.

Revision history for this message
Dmitry Nedospasov (dmitry-nedos) wrote :

Andrew,

Your patched worked great. I'm running 2.6.18-164.15.1.el5xen

Revision history for this message
Nathan Stratton Treadway (nathanst) wrote :

@Steve,

Ah! I see, there is currently no attempt to "support" any case where one's kernel comes from "outside" the Ubuntu installation being upgraded.

As Dmitry mentions (and as indicated by the general discussion here), I think there are a lot of people who are using Ubuntu one way or another where that does happen. I wonder if there is any chance the Ubuntu project could come up with some "policy" regarding support for those situations (for Maverick, obviously).

 (I would be happy to open new bug for that issue if you thought there would be any interest in pursuing the idea further, especially if you could give me a hint as to what "package" such a bug should be be filed against.)

Anyway, on the topic of the Lucid release notes, I certainly understand the desire to keep them as short as possible. On the other hand, it really seems that even a user "advanced" enough to be running e.g. a Xen server would prefer to have *some* warning before initiating an upgrade to Lucid that is doomed to abort mid-upgrade this way (leaving dpkg/apt/aptitude in a "broken" configuration)....

Perhaps a compromise would be to add a one-line paragraph saying something like "Users who would like to run Lucid using a kernel older than 2.6.24 should see <ThisPage>", pointing to a page somewhere under help.ubuntu.com or wiki.ubuntu.com. [Using "2.6.24" because that was the version in Hardy.]

I would be willing to create the initial version of such a page, provided that you were interested in linking to it (and especially if someone what was more familiar than I am with the existing wiki/help pages could recommend a good location for creating it).

Revision history for this message
Steve Langasek (vorlon) wrote : Re: [Bug 539814] Re: tar/gnulib bug prevents bootstraping Lucid on Dapper, causes Lucid upgrade to abort on older linux kernels (e.g. in a chroot or Xen VM)

On Thu, Apr 22, 2010 at 10:35:49PM -0000, Nathan Stratton Treadway wrote:

> Ah! I see, there is currently no attempt to "support" any case where
> one's kernel comes from "outside" the Ubuntu installation being
> upgraded.

> As Dmitry mentions (and as indicated by the general discussion here), I
> think there are a lot of people who are using Ubuntu one way or another
> where that does happen. I wonder if there is any chance the Ubuntu
> project could come up with some "policy" regarding support for those
> situations (for Maverick, obviously).

I don't believe this is likely to happen, fwiw; we can certainly have the
discussion, but supporting two-year-old kernels for LTS upgrade purposes
already requires ongoing work, and I don't believe we're ever going to be
able to support running the Ubuntu userspace on an arbitrary non-Ubuntu
kernel except on a best-effort basis (like in this case).

> Anyway, on the topic of the Lucid release notes, I certainly understand
> the desire to keep them as short as possible. On the other hand, it
> really seems that even a user "advanced" enough to be running e.g. a Xen
> server would prefer to have *some* warning before initiating an upgrade to
> Lucid that is doomed to abort mid-upgrade this way (leaving
> dpkg/apt/aptitude in a "broken" configuration)....

When declining the release notes task I was thinking in terms of chroots
which can easily be recovered after the fact. Xen is obviously a different
matter; given that this also affects running on Xen, I agree we should
document this in the release notes.

> Perhaps a compromise would be to add a one-line paragraph saying
> something like "Users who would like to run Lucid using a kernel older
> than 2.6.24 should see <ThisPage>", pointing to a page somewhere under
> help.ubuntu.com or wiki.ubuntu.com. [Using "2.6.24" because that was
> the version in Hardy.]

> I would be willing to create the initial version of such a page,
> provided that you were interested in linking to it (and especially if
> someone what was more familiar than I am with the existing wiki/help
> pages could recommend a good location for creating it).

Under the circumstances, perhaps it's better to simply use the release notes
to warn users off of upgrading to 10.04 until these issues are fixed, and
focus on getting these SRUs done?

--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer http://www.debian.org/
<email address hidden> <email address hidden>

Revision history for this message
Dmitry Nedospasov (dmitry-nedos) wrote : Re: tar/gnulib bug prevents bootstraping Lucid on Dapper, causes Lucid upgrade to abort on older linux kernels (e.g. in a chroot or Xen VM)

@Steve: I agree, i think this is the best solution, given how little time we have before release.

How do distributions upgrades work? I just downloaded upgrade-manager-core and was greeted by a bunch of python scripts... which means I'm not gonna look through them. Does anyone know how this works? What is the backend in the upgrade? Will tar get pulled from SRU? If it tar gets pulled from the stable release, we have an issue, if it gets pulled from SRU, then we're in the clear.

Revision history for this message
Nathan Stratton Treadway (nathanst) wrote : Re: [Bug 539814] Re: tar/gnulib bug prevents bootstraping Lucid on Dapper, causes Lucid upgrade to abort on older linux kernels (e.g. in a chroot or Xen VM)

On Sat, Apr 24, 2010 at 21:57:51 -0000, Steve Langasek wrote:
> On Thu, Apr 22, 2010 at 10:35:49PM -0000, Nathan Stratton Treadway
> > where that does happen. I wonder if there is any chance the Ubuntu
> > project could come up with some "policy" regarding support for those
> > situations (for Maverick, obviously).
>
> I don't believe this is likely to happen, fwiw; we can certainly have the
> discussion, but supporting two-year-old kernels for LTS upgrade purposes
> already requires ongoing work, and I don't believe we're ever going to be
> able to support running the Ubuntu userspace on an arbitrary non-Ubuntu
> kernel except on a best-effort basis (like in this case).

I don't think anyone would expect support for arbitrary kernel
versions, but perhaps the outcome of such a discussion would be
to pick specific "outside of the current release" versions for
some level of support.

(For example, supporting 2.6.18 because so many Xen hosts are
currently tied to that version, or supporting chroot installs
under old Ubuntu releases going back some number of releases,
etc.)

Also, there is some middle ground between "supported" as in "we
guarantee everything will work on this kernel version" and
"unsupported". Something like "we'll try to keep things working
on this Linux kernel unless there is a good reason not to, and
if we can't we'll be sure to warn you" might be perfectly
appropriate for this type of usage.

> > Perhaps a compromise would be to add a one-line paragraph saying
> > something like "Users who would like to run Lucid using a kernel older
> > than 2.6.24 should see <ThisPage>", pointing to a page somewhere under
> > help.ubuntu.com or wiki.ubuntu.com. [Using "2.6.24" because that was
> > the version in Hardy.]
[...]
> Under the circumstances, perhaps it's better to simply use the release notes
> to warn users off of upgrading to 10.04 until these issues are fixed, and
> focus on getting these SRUs done?

I'll leave it to you to decide if it's easier to just write a
release note that specifically mentions these bugs (e.g. LP:
#539814 and LP: #552816), or to do a generic note that points to
a Wiki page. The former is a little more streamlined, though
the latter would allow users who discover other
running-on-older-kernel issues to add them to the list later.

Either way, the point is just to get some warning in front of
people before they start an upgrade and then discover that their
system/installation is broken....

      Nathan

Revision history for this message
Dmitry Nedospasov (dmitry-nedos) wrote : Re: tar/gnulib bug prevents bootstraping Lucid on Dapper, causes Lucid upgrade to abort on older linux kernels (e.g. in a chroot or Xen VM)

Any status updates? How quickly might this get into SRU?

Thanks,

D.

Steve Langasek (vorlon)
Changed in ubuntu-release-notes:
status: Won't Fix → Triaged
Revision history for this message
Keith Ward (kward) wrote :

Debian has released a patch, that fixes just this problem, in January?

See http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=563726 for more details. - Specifically http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=563726#66.

Patch attached.

Keith

Changed in tar (Ubuntu Lucid):
status: New → Fix Released
Changed in tar (Ubuntu):
status: New → Fix Released
Revision history for this message
Keith Ward (kward) wrote :

Apologies - set these to the wrong status.

-Keith

Changed in tar (Ubuntu):
status: Fix Released → In Progress
Changed in tar (Ubuntu Lucid):
status: Fix Released → In Progress
Changed in tar (Ubuntu):
status: In Progress → Fix Released
Revision history for this message
Paul van Genderen (paulvg) wrote :

Is it possible to upgrade an existing vm or container?

Revision history for this message
Adrian Bridgett (adrian-bridgett) wrote :

Please add something about this to the release notes. I've just upgraded a production Linode (Xen based) guest and it's broken. "Abiword freezes when accessing help" is surely of lesser importance than "your computer is now broken".

Revision history for this message
Dmitry Nedospasov (dmitry-nedos) wrote : Re: [Bug 539814] Re: tar/gnulib bug prevents bootstraping Lucid on Dapper, causes Lucid upgrade to abort on older linux kernels (e.g. in a chroot or Xen VM)

On 6/9/10 12:50 PM, Paul van Genderen wrote:
> Is it possible to upgrade an existing vm or container?

If the vm is running a 2.6.18, then the upgrade will fail as it will on
any non-vms. The reason xen was mentioned is that many dom0's run 2.6.18
and many domU's run the dom0's kernel, so this issue is relevant to both
dom0 as well as domU upgrades.

Revision history for this message
Patrick Welche (prlw1) wrote : Re: tar/gnulib bug prevents bootstraping Lucid on Dapper, causes Lucid upgrade to abort on older linux kernels (e.g. in a chroot or Xen VM)

I have just hit this problem upgrading from hardy 8.04 to lucid 10.04, so you don't need to go back as far as dapper to see it. I now have a partially upgraded system, and a

aptitude safe-upgrade

gives

Preconfiguring packages ...
tar: ./postinst: Cannot utime: Bad file descriptor
tar: ./preinst: Cannot utime: Bad file descriptor
tar: ./prerm: Cannot utime: Bad file descriptor
tar: ./postrm: Cannot utime: Bad file descriptor
tar: ./conffiles: Cannot utime: Bad file descriptor
tar: ./md5sums: Cannot utime: Bad file descriptor
tar: ./control: Cannot utime: Bad file descriptor
tar: .: Cannot utime: Bad file descriptor
tar: Exiting with failure status due to previous errors
dpkg-deb: subprocess tar returned error exit status 2
dpkg: error processing /var/cache/apt/archives/bash_4.1-2ubuntu3_i386.deb (--unpack):
 subprocess dpkg-deb --control returned error exit status 2
Errors were encountered while processing:
 /var/cache/apt/archives/bash_4.1-2ubuntu3_i386.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)

Tar version is 1.22-2, and /bin/tar is the same as the binary on another working 10.04.1 system
gnulib is not installed - same as on another working 10.04.1 system

One difference is that the working system has:

> ldd `which tar`
        linux-gate.so.1 => (0x00f66000)
        librt.so.1 => /lib/tls/i686/cmov/librt.so.1 (0x00574000)
        libc.so.6 => /lib/tls/i686/cmov/libc.so.6 (0x00ad0000)
        libpthread.so.0 => /lib/tls/i686/cmov/libpthread.so.0 (0x00665000)
        /lib/ld-linux.so.2 (0x007b8000)

whereas the "broken" one has:

# ldd `which tar`
        linux-gate.so.1 => (0xffffe000)
        librt.so.1 => /lib/librt.so.1 (0xb7fcc000)
        libc.so.6 => /lib/libc.so.6 (0xb7e83000)
        libpthread.so.0 => /lib/libpthread.so.0 (0xb7e6b000)
        /lib/ld-linux.so.2 (0x80000000)

Revision history for this message
Patrick Welche (prlw1) wrote :

P.S. both have libc version 2.11.1-0ubuntu7.2
Given the bug, it is hard for me to install libc-i686 on the "broken" one...

Ah: the "broken" one has a 2.6.17-12-386 kernel! not 2.6.32 - "broken" one was upgraded by
boot into recovery mode with netroot
aptitude update
aptitude safe-upgrade
-> utime trouble...

Revision history for this message
Evan Carroll (evancarroll) wrote :

It would have been nice if someone could have at least published a .dev on this.

Revision history for this message
Evan Carroll (evancarroll) wrote :

I can't even `apt-get source tar` to patch tar, because of a non-working tar. This is a pain in the ass and should have been in the changelogs.

Revision history for this message
Keith Ward (kward) wrote :

Firstly, apologies to anyone if the following comment comes across as confrontational, but this bug has gone on far too long.

I've gone ahead Subscribed SRU to this bug, as it is urgent, due to the fact it prevents any system with a kernel older than 2.6.22 from bootstrapping lucid - (which means it applies to dapper), as tar simply fails due to the previously mentioned utime errors.

I'm surprised this hasn't been patched already especially as it was previously mentioned that debian patched it a long long time ago. I'm even more surprised that Lucid was released without this being fixed - especially as the patch has been available since at least march!

Just to clarify this bug prevents any type of bootstrapping from Dapper or running of tar on any kernel < 2.6.22, (this means you can't debootstrap lucid on the majority of Xen Hosts, as they run kernel 2.6.18!

While I agree that it is not possible to support every kernel known to man, due to the fact this this applies to more than just dapper - there are a lot of hosts running .18 kernels, for various reasons: mostly xen hosts, and the fact a patch has been available since the beginning half of the year, and has been fixed in *both* Debian and fedora, just makes the whole situation we are now in, completely unbelievable.

The resolution to this problem is to either use the patch I attached a few months ago from the debian bug, and push that for an SRU (It fixes this bug, and only this bug). Or to push the package from maverick into lucid as an SRU (this bug according to the debian information at debbugs is fixed in version 1.23-1).

As to which option is preferred i shall leave up to the developers, personally if we're updating the package again, it may be an idea to push over the version from maverick as opposed to just creating a new -rev for 1.22.

Regards,

Keith Ward (kward)
summary: - tar/gnulib bug prevents bootstraping Lucid on Dapper, causes Lucid
- upgrade to abort on older linux kernels (e.g. in a chroot or Xen VM)
+ tar: futimens() with a bad file descriptor (AT_FDCWD) causes
+ bootstrapping failure with kernels < 2.6.22
Revision history for this message
Keith Ward (kward) wrote :

Attached debfiff for tar-1.22 that fixes this.

description: updated
Iain Lane (laney)
Changed in tar (Ubuntu Lucid):
status: In Progress → Triaged
Revision history for this message
Tim Gehpunkt (rollercoaster) wrote :

This also affects cp and touch from coreutils, maybe others.

root@monster:~# mkdir bla
root@monster:~# cp -a bla blubb
cp: preserving times for `blubb': Bad file descriptor
root@monster:~# touch bla
touch: setting times of `bla': Bad file descriptor

Why ist there still no fix after six months? Had to use the files from debian :(

Revision history for this message
Brian J. Murrell (brian-interlinx) wrote :

So, yet another bug that gets raised and "denied" because we are "too close to release" and then just gets forgotten about even when subsequent point releases are made. Why does this seem to happen a lot around here?

Is nobody actually tracking bugs and assessing them for inclusion in upcoming (point) releases? As soon as somebody denied this bug for 10.04.0 because it was too late, that same person should have added it the tracker for 10.04.1 to make sure that it made it out on that release. Why does this sort of basic release tracking not happen around here?

Changed in tar (Ubuntu Lucid):
milestone: none → lucid-updates
assignee: nobody → Canonical Foundations Team (canonical-foundations)
Revision history for this message
Matthias Klose (doko) wrote :

uploaded to lucid-proposed, waiting for approval

Changed in tar (Ubuntu Lucid):
assignee: Canonical Foundations Team (canonical-foundations) → Matthias Klose (doko)
status: Triaged → In Progress
Revision history for this message
Nathan Stratton Treadway (nathanst) wrote : Re: [Bug 539814] Re: tar: futimens() with a bad file descriptor (AT_FDCWD) causes bootstrapping failure with kernels < 2.6.22

On Fri, Sep 10, 2010 at 13:13:44 -0000, rollercoaster wrote:
> This also affects cp and touch from coreutils, maybe others.
>
> root@monster:~# mkdir bla
> root@monster:~# cp -a bla blubb
> cp: preserving times for `blubb': Bad file descriptor
> root@monster:~# touch bla
> touch: setting times of `bla': Bad file descriptor

(For what it's worth, LP: #552816 covers this gnulib/futimens()/old
kernel bug as it applies to coreutils .)

      Nathan

Revision history for this message
Brian J. Murrell (brian-interlinx) wrote :

On Mon, 2010-09-20 at 12:27 +0000, Matthias Klose wrote:
> uploaded to lucid-proposed, waiting for approval

How long until it's in the repos? archive.ubuntu.com isn't seeing it
yet.

Sorry for being so impatient but I am just waiting for this to get some
work depending on it done.

Revision history for this message
Iain Lane (laney) wrote :

Hiya,

On Mon, Sep 20, 2010 at 03:23:17PM -0000, Brian J. Murrell wrote:
>On Mon, 2010-09-20 at 12:27 +0000, Matthias Klose wrote:
>> uploaded to lucid-proposed, waiting for approval
>
>How long until it's in the repos? archive.ubuntu.com isn't seeing it
>yet.
>
>Sorry for being so impatient but I am just waiting for this to get some
>work depending on it done.

It needs someone from the Stable Release Update (SRU) team to review
the diff and then approve the package into lucid-proposed. Usually
takes a few days. You'll then have to have the -proposed repository
enabled to have the package available.

Keep watching this bug and you'll be updated when this happens.

Cheers,
Iain

Revision history for this message
Colin Watson (cjwatson) wrote : Please test proposed package

Accepted tar into lucid-proposed, the package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

Changed in tar (Ubuntu Lucid):
status: In Progress → Fix Committed
tags: added: verification-needed
Revision history for this message
Michael K. Edwards (m-k-edwards) wrote :

I hacked a bit on debootstrap to teach it about lucid-proposed, and can confirm that this updated tar resolves the inability to debootstrap lucid on an old kernel (in this case, the 2.6.18 variant shipped with RHEL5.4). Thanks for following up and driving this to closure; I was not looking forward to backporting enough current packages to lenny to run the LAMP stack app I need. (RHEL was of course not even an option, userland-wise; but as long as I leave it on there as the boot environment, I don't have to deal with the hardware.)

The debootstrap hack I did was pretty gross, and doesn't know about lucid-updates or lucid-security. Assuming this version of tar makes it out of lucid-proposed, will it go to lucid or lucid-updates? If the latter, it won't do people a lot of good unless there's a better way than my hack to get debootstrap to use -updates right out the chute. Is there another debootstrap floating around that does know how to compress the "debootstrap, update sources.list, apt-get update, apt-get dist-upgrade" sequence down to a single pass?

Revision history for this message
Jean-Baptiste Lallement (jibel) wrote :

SRU verification for Lucid:
I have reproduced the problem with tar 1.22-2 in lucid and have verified that the version of tar 1.22-2ubuntu1 in -proposed fixes the issue. I've reproduced the original issue with a lucid chroot on a dapper host. I've run the regression tests and found no regression.

Note that in this configuration dpkg is not usable and fail with the error:
rm: cannot remove `/var/lib/dpkg/tmp.ci/control': Function not implemented

Marking as verification-done

tags: added: verification-done
removed: verification-needed
Revision history for this message
Nathan Stratton Treadway (nathanst) wrote : Re: [Bug 539814] Re: tar: futimens() with a bad file descriptor (AT_FDCWD) causes bootstrapping failure with kernels < 2.6.22

On Thu, Sep 30, 2010 at 16:02:58 -0000, Jean-Baptiste Lallement wrote:
> SRU verification for Lucid:
> I have reproduced the problem with tar 1.22-2 in lucid
> and have verified that the version of tar 1.22-2ubuntu1
> in -proposed fixes the issue. I've reproduced the
> original issue with a lucid chroot on a dapper host.
> I've run the regression tests and found no regression.
>
> Note that in this configuration dpkg is not usable and
> fail with the error: rm: cannot remove `/var/lib/dpkg/tmp.ci/control': Function not implemented

Did you have the current -proposed version of coreutils
(7.4-2ubuntu3) installed in this lucid chroot at the point
you got this error from "rm"?

Nathan

Revision history for this message
Jean-Baptiste Lallement (jibel) wrote :

yes, the coreutils from -proposed was installed and it's working fine.
The problem was that /proc was not mounted in the chroot. So my comment about dpkg is not relevant anymore.
Thank you.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package tar - 1.22-2ubuntu1

---------------
tar (1.22-2ubuntu1) lucid-proposed; urgency=low

  * lib/utimens.c: Patch from Debian bug #563726 to ensure that futimens
    is only called with a valid file descriptor. Fixes bootstrapping Lucid
    from Dapper (LP: #539814)
 -- Keith Ward <email address hidden> Tue, 24 Aug 2010 18:50:56 +0200

Changed in tar (Ubuntu Lucid):
status: Fix Committed → Fix Released
Revision history for this message
Michael K. Edwards (m-k-edwards) wrote :

The tar bug is fixed; the inability to debootstrap lucid on an old kernel remains. Is Canonical likely to provide a version of debootstrap that knows how to use packages from lucid-updates during the initial bootstrapping pass? I adjusted my hacked debootstrap to get supplemental packages from lucid-updates rather than lucid-proposed, and that seems to work OK. Is there somewhere that I should post that patch for consideration -- with the understanding that it's half-baked (for instance, it breaks debootstrap for lenny, which doesn't have a corresponding lenny-updates)?

Revision history for this message
Brian J. Murrell (brian-interlinx) wrote :

On Thu, 2010-10-07 at 02:03 +0000, Michael K. Edwards wrote:
> I adjusted my hacked debootstrap to get
> supplemental packages from lucid-updates rather than lucid-proposed, and
> that seems to work OK.

It seems entirely reasonable to me that debootstrap should include
<release>-updates in it's initial fetch.

> Is there somewhere that I should post that patch
> for consideration

To this ticket would be nice. Or open a new ticket regarding
debootstrap using -updates and please post the new bug number here if
you do.

Revision history for this message
Colin Watson (cjwatson) wrote :

I've added a debootstrap task to this bug.

Changed in debootstrap (Ubuntu):
status: New → Triaged
importance: Undecided → Wishlist
Changed in debootstrap (Ubuntu Lucid):
status: New → Triaged
importance: Undecided → Wishlist
Revision history for this message
Colin Watson (cjwatson) wrote :

I've added a release note to https://wiki.ubuntu.com/LucidLynx/ReleaseNotes (since I don't know when the debootstrap bug might be fixed):

== Bootstrapping on old kernels ==

Ubuntu 10.04 LTS cannot currently be bootstrapped (using `debootstrap`) on a system running a kernel prior to 2.6.22, due to a bug in tar and the inability of `debootstrap` to use the fix in `lucid-updates`. (Bug:539814)

Changed in ubuntu-release-notes:
assignee: nobody → Colin Watson (cjwatson)
status: Triaged → Fix Released
Revision history for this message
Rolf Leggewie (r0lf) wrote :

lucid has seen the end of its life and is no longer receiving any updates. Marking the lucid task for this ticket as "Won't Fix".

Changed in debootstrap (Ubuntu Lucid):
status: Triaged → Won't Fix
Changed in tar (Fedora):
importance: Unknown → High
status: Unknown → Fix Released
Revision history for this message
Ken Sharp (kennybobs) wrote :

Can the whole task for Ubuntu be marked wontfix? (I can't do it)

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Related questions

Remote bug watches

Bug watches keep track of this bug in other bug trackers.