Fixed fields are not fixed, leading to buffer overflows and tar explosions.

[rlpowell]

The problem is in /usr/lib64/python2.4/site-packages/duplicity/tarfile.py , in this code:

        for value, fieldsize in (
                (self.name, 100),
                ("%07o" % self.mode, 8),
                ("%07o" % self.uid, 8),
                ("%07o" % self.gid, 8),
                ("%011o" % self.size, 12),
                ("%011o" % self.mtime, 12),
                (" ", 8),
                (self.type, 1),
                (self.linkname, 100),
                (MAGIC, 6),
                (VERSION, 2),
                (self.uname, 32),
                (self.gname, 32),
                ("%07o" % self.devmajor, 8),
                ("%07o" % self.devminor, 8),
                (self.prefix, 155)
                ):
            l = len(value)
            parts.append(value + (fieldsize - l) * "\0")

and possibly elsewhere; I haven't exactly done a full code audit here.

The problem is that %011o, or whatever, doesn't stop the field from being *bigger* than 11 characters, so that for example if you have a file like this:

# stat [customer file]
  File: `[customer file]'
  Size: 20366 Blocks: 48 IO Block: 4096 regular file
Device: fd00h/64768d Inode: 2755310 Links: 1
Access: (0777/-rwxrwxrwx) Uid: ( 1006/ UNKNOWN) Gid: ( 1000/articulatedman)
Access: 8330-03-21 02:25:15.000000000 -0800
Modify: 8330-03-21 02:25:15.000000000 -0800
Change: 2009-07-25 18:18:49.000000000 -0700

(note the mtime) which is an mtime of:

# date -r [customer file] +%s
200709051915

Which converts to:

>>> ("%011o" % 200709051915 )
'2727314213013'

13 characters, which drives the next field into the field after it, leaving a blank space in the "type" field, but *no error on creation*, which means that a full works just fine, but if you try to do an incremental or restore you get:

Traceback (most recent call last):
  File "/usr/bin/duplicity", line 1236, in ?
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1229, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1211, in main
    incremental_backup(sig_chain)
  File "/usr/bin/duplicity", line 487, in incremental_backup
    globals.backend)
  File "/usr/bin/duplicity", line 294, in write_multivol
    globals.gpg_profile, globals.volsize)
  File "/usr/lib64/python2.4/site-packages/duplicity/gpg.py", line 282, in GPGWriteFile
    data = block_iter.next(min(block_size, bytes_to_go)).data
  File "/usr/lib64/python2.4/site-packages/duplicity/diffdir.py", line 509, in next
    result = self.process(self.input_iter.next(), size)
  File "/usr/lib64/python2.4/site-packages/duplicity/diffdir.py", line 191, in get_delta_iter
    for new_path, sig_path in collated:
  File "/usr/lib64/python2.4/site-packages/duplicity/diffdir.py", line 279, in collate2iters
    relem2 = riter2.next()
  File "/usr/lib64/python2.4/site-packages/duplicity/diffdir.py", line 348, in combine_path_iters
    refresh_triple_list(triple_list)
  File "/usr/lib64/python2.4/site-packages/duplicity/diffdir.py", line 334, in refresh_triple_list
    new_triple = get_triple(old_triple[1])
  File "/usr/lib64/python2.4/site-packages/duplicity/diffdir.py", line 320, in get_triple
    path = path_iter_list[iter_index].next()
  File "/usr/lib64/python2.4/site-packages/duplicity/diffdir.py", line 249, in sigtar2path_iter
    ropath.init_from_tarinfo(tarinfo)
  File "/usr/lib64/python2.4/site-packages/duplicity/path.py", line 200, in init_from_tarinfo
    raise PathException("Unknown tarinfo type %s" % (type,))
PathException: Unknown tarinfo type

because not only are the field widths not fixed, but the expansion isn't checked, and the type error isn't printed in a useful fashion, which means that there's no indication *which* file is causing the problem, or even that a file is what caused it.

Since this is a pure overflow bug, I imagine it could be used to cause security problems, but I haven't investigated.

-Robin

[rlpowell (rlpowell)]

Allow me to suggest, by the way, that the easy way to fix this is: every time you create a header, immediately dissect it, and make sure all the values match. If not, error and clearly state the problem file.

-Robin

[rlpowell (rlpowell)]

A friend points out that "tarfile.py is now (and has been for a long time now) part of the python standard library. And the version they're using is ancient. and all of that code has been basically replaced now.".

So, umm, *yeah*.

-Robin

[Kenneth Loafman (kenneth-loafman)]

Access: 8330-03-21 02:25:15.000000000 -0800
Modify: 8330-03-21 02:25:15.000000000 -0800
Change: 2009-07-25 18:18:49.000000000 -0700

If you take a look at the atime and mtime you'll find that they are way above 2038 when the current timekeeping system will go into meldown (ok, just confusion). The question should be: "How was your clock set to such a ridiculous time?".

Yes, the current tarfile.py is ancient, but it is intertwined with duplicity. I need volunteers to step up to help replace it. It looks like you need a new clock on your computer. In an ideal world, your computer would give the correct time and tarfile would have just rejected the atime and mtime settings as invalid and stored zeroes. That I can fix.

[rlpowell (rlpowell)]

On Thu, Nov 26, 2009 at 04:48:05PM -0000, Kenneth Loafman wrote:
> Access: 8330-03-21 02:25:15.000000000 -0800
> Modify: 8330-03-21 02:25:15.000000000 -0800
> Change: 2009-07-25 18:18:49.000000000 -0700
>
> If you take a look at the atime and mtime you'll find that they
> are way above 2038 when the current timekeeping system will go
> into meldown (ok, just confusion). The question should be: "How
> was your clock set to such a ridiculous time?".

Yes, but what about the "size" field? Same problem.

-Robin

--
They say: "The first AIs will be built by the military as weapons."
And I'm thinking: "Does it even occur to you to try for something
other than the default outcome?" See http://shrunklink.com/cdiz
http://www.digitalkingdom.org/~rlpowell/ *** http://www.lojban.org/

[Kenneth Loafman (kenneth-loafman)]

rlpowell wrote:
> On Thu, Nov 26, 2009 at 04:48:05PM -0000, Kenneth Loafman wrote:
>> Access: 8330-03-21 02:25:15.000000000 -0800
>> Modify: 8330-03-21 02:25:15.000000000 -0800
>> Change: 2009-07-25 18:18:49.000000000 -0700
>>
>> If you take a look at the atime and mtime you'll find that they
>> are way above 2038 when the current timekeeping system will go
>> into meldown (ok, just confusion). The question should be: "How
>> was your clock set to such a ridiculous time?".
>
> Yes, but what about the "size" field? Same problem.

No argument about getting it fixed. It's just a matter of time.

...Ken

[Michael Terry (mterry)]

I'm looking into replacing duplicity's tarfile module with Python's standard tarfile module. Less code for us to maintain, and we get all the bug fixes in the meantime! :)