rdiff-backup fails on file with unknown unicode character

Bug #223280 reported by Hanno Stock (hefe_bia) on 2008-04-27
4
Affects Status Importance Assigned to Milestone
rdiff-backup
Unknown
Unknown
rdiff-backup (Ubuntu)
Undecided
Unassigned
Nominated for Hardy by Hanno Stock (hefe_bia)

Bug Description

Binary package hint: rdiff-backup

The problem occured while backing up from an ext3 volume to a NTFS (ntfs3g) volume with rdiff-backup 1.1.15. (prevu backport from Hardy)

Operating System: Ubuntu 7.10.

Trace:

Processing changed file hanno/.caff/keys/2008-03-30/XXXXXXXXXXXXX.key.1.XXXXXXXei�_x9f <xxxxxxxxxxxxxxxxxxxxxx>.asc
Exception '[Errno 84] Invalid or incomplete multibyte or wide character: '/media/BIG_BACKUP/home_backup/hanno/.caff/keys/2008-03-30/XXXXXXXXXXXXX.key.1.XXXXXXXei\xc3_x9f <xxxxxxxxxxxxxxxxxxxxxx>.asc'' raised of class '<type 'exceptions.OSError'>':
File "/var/lib/python-support/python2.5/rdiff_backup/Main.py", line 302, in error_check_Main
try: Main(arglist)
File "/var/lib/python-support/python2.5/rdiff_backup/Main.py", line 322, in Main
take_action(rps)
File "/var/lib/python-support/python2.5/rdiff_backup/Main.py", line 278, in take_action
elif action == "backup": Backup(rps[0], rps[1])
File "/var/lib/python-support/python2.5/rdiff_backup/Main.py", line 344, in Backup
backup.Mirror(rpin, rpout)
File "/var/lib/python-support/python2.5/rdiff_backup/backup.py", line 38, in Mirror
DestS.patch(dest_rpath, source_diffiter)
File "/var/lib/python-support/python2.5/rdiff_backup/backup.py", line 225, in patch
ITR(diff.index, diff)
File "/var/lib/python-support/python2.5/rdiff_backup/rorpiter.py", line 281, in _call_
last_branch.fast_process(*args)
File "/var/lib/python-support/python2.5/rdiff_backup/backup.py", line 519, in fast_process
self.CCPP.get_rorps(index), self.basis_root_rp)
File "/var/lib/python-support/python2.5/rdiff_backup/longname.py", line 200, in get_mirror_inc_rps
mirror_rp, alt_mirror, alt_inc = mir_triple_new(new_rorp)
File "/var/lib/python-support/python2.5/rdiff_backup/longname.py", line 171, in mir_triple_new
mirror_rp = check_new_index(mirror_root, new_rorp.index)
File "/var/lib/python-support/python2.5/rdiff_backup/longname.py", line 125, in check_new_index
rp = wrap_call(base.new_index, index)
File "/var/lib/python-support/python2.5/rdiff_backup/longname.py", line 111, in wrap_call
try: result = func(*args)
File "/var/lib/python-support/python2.5/rdiff_backup/rpath.py", line 1040, in new_index
return self.__class__(self.conn, self.base, index)
File "/var/lib/python-support/python2.5/rdiff_backup/rpath.py", line 757, in _init_
else: self.setdata()
File "/var/lib/python-support/python2.5/rdiff_backup/rpath.py", line 781, in setdata
self.data = self.conn.C.make_file_dict(self.path)

Traceback (most recent call last):
File "/usr/bin/rdiff-backup", line 23, in <module>
rdiff_backup.Main.error_check_Main(sys.argv[1:])
File "/var/lib/python-support/python2.5/rdiff_backup/Main.py", line 302, in error_check_Main
try: Main(arglist)
File "/var/lib/python-support/python2.5/rdiff_backup/Main.py", line 322, in Main
take_action(rps)
File "/var/lib/python-support/python2.5/rdiff_backup/Main.py", line 278, in take_action
elif action == "backup": Backup(rps[0], rps[1])
File "/var/lib/python-support/python2.5/rdiff_backup/Main.py", line 344, in Backup
backup.Mirror(rpin, rpout)
File "/var/lib/python-support/python2.5/rdiff_backup/backup.py", line 38, in Mirror
DestS.patch(dest_rpath, source_diffiter)
File "/var/lib/python-support/python2.5/rdiff_backup/backup.py", line 225, in patch
ITR(diff.index, diff)
File "/var/lib/python-support/python2.5/rdiff_backup/rorpiter.py", line 281, in _call_
last_branch.fast_process(*args)
File "/var/lib/python-support/python2.5/rdiff_backup/backup.py", line 519, in fast_process
self.CCPP.get_rorps(index), self.basis_root_rp)
File "/var/lib/python-support/python2.5/rdiff_backup/longname.py", line 200, in get_mirror_inc_rps
mirror_rp, alt_mirror, alt_inc = mir_triple_new(new_rorp)
File "/var/lib/python-support/python2.5/rdiff_backup/longname.py", line 171, in mir_triple_new
mirror_rp = check_new_index(mirror_root, new_rorp.index)
File "/var/lib/python-support/python2.5/rdiff_backup/longname.py", line 125, in check_new_index
rp = wrap_call(base.new_index, index)
File "/var/lib/python-support/python2.5/rdiff_backup/longname.py", line 111, in wrap_call
try: result = func(*args)
File "/var/lib/python-support/python2.5/rdiff_backup/rpath.py", line 1040, in new_index
return self.__class__(self.conn, self.base, index)
File "/var/lib/python-support/python2.5/rdiff_backup/rpath.py", line 757, in _init_
else: self.setdata()
File "/var/lib/python-support/python2.5/rdiff_backup/rpath.py", line 781, in setdata
self.data = self.conn.C.make_file_dict(self.path)
OSError: [Errno 84] Invalid or incomplete multibyte or wide character: '/media/BIG_BACKUP/home_backup/hanno/.caff/keys/2008-03-30/XXXXXXXXXXXXX.key.1.XXXXXXXei\xc3_x9f <xxxxxxxxxxxxxxxxxxxxxx>.asc'

I guess you are referring to "The OS configures the setting in a too late stage during the boot process, only after the NTFS volume was already mounted.", however the volume in question was mounted after booting via Nautilus.

In any case I think rdiff-backup should handle this more gracefully, since it is supposed to store backups even on filesystems that do not support all filenames that the source filesystem supports.

Andrew Ferguson (adferguson) wrote :

Hi,

Thank you for reporting this error.

I have corrected the problem in CVS -- rdiff-backup will no longer have a fatal error if it cannot handle an unknown Unicode character. The fix will be present in rdiff-backup versions 1.1.17 and above.

Python 3 / 2.6 will introduce much better support for Unicode in Python. The plan is for the next edition of rdiff-backup development (1.3.x and beyond) to include complete support for Unicode (and never encounter an 'unknown' character).

thanks,
Andrew

Andrew Ferguson (adferguson) wrote :

Fix in rdiff-backup CVS -- will be included in rdiff-backup 1.1.17 and above.

Changed in rdiff-backup:
status: New → Fix Committed

Nominated for fixing in Hardy.
I'll look into cherrypicking the fix for Hardy.

I think Intrepid will just get the new upstream version.

Changed in rdiff-backup:
assignee: nobody → hanno-stock
Hew (hew) wrote :

Marking Fix Released as this was fixed by rdiff-backup 1.2.0.

Changed in rdiff-backup:
assignee: hanno-stock → nobody
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.