| 2014-10-22 23:49:30 |
Joe Harrington |
bug |
|
|
added bug |
| 2014-11-07 18:52:33 |
Launchpad Janitor |
rsync (Ubuntu): status |
New |
Confirmed |
|
| 2014-12-15 08:06:15 |
Christoph Lukas |
bug |
|
|
added subscriber Christoph Lukas |
| 2015-02-02 00:57:38 |
Robie Basak |
bug |
|
|
added subscriber Ubuntu Server Team |
| 2015-02-11 16:28:54 |
Ryan Harper |
bug |
|
|
added subscriber Ryan Harper |
| 2015-06-15 17:55:54 |
Kevin_Traas |
bug |
|
|
added subscriber Kevin_Traas |
| 2015-06-23 17:05:29 |
Robie Basak |
rsync (Ubuntu): assignee |
|
Ryan Harper (raharper) |
|
| 2015-06-23 17:08:55 |
Robie Basak |
nominated for series |
|
Ubuntu Trusty |
|
| 2015-06-23 17:08:55 |
Robie Basak |
bug task added |
|
rsync (Ubuntu Trusty) |
|
| 2015-06-23 17:09:03 |
Robie Basak |
rsync (Ubuntu): status |
Confirmed |
Fix Released |
|
| 2015-06-23 17:09:06 |
Robie Basak |
rsync (Ubuntu Trusty): status |
New |
Confirmed |
|
| 2015-06-23 17:09:15 |
Robie Basak |
rsync (Ubuntu Trusty): assignee |
|
Ryan Harper (raharper) |
|
| 2015-08-18 00:31:19 |
Andrew |
bug |
|
|
added subscriber Andrew |
| 2015-12-23 14:43:56 |
Viktor Szathmáry |
bug |
|
|
added subscriber Viktor Szathmáry |
| 2016-01-11 14:58:28 |
Robie Basak |
rsync (Ubuntu Trusty): status |
Confirmed |
Invalid |
|
| 2016-01-11 14:59:40 |
Robie Basak |
rsync (Ubuntu Trusty): status |
Invalid |
Incomplete |
|
| 2016-01-11 14:59:53 |
Robie Basak |
bug |
|
|
added subscriber Robie Basak |
| 2016-01-15 10:08:54 |
André Freitas |
bug |
|
|
added subscriber André Freitas |
| 2016-03-04 13:53:09 |
Luca Lesinigo |
bug |
|
|
added subscriber Luca Lesinigo |
| 2016-06-07 20:03:03 |
Ryan Harper |
description |
Copying large (>10GB) files with rsync -z (compression) leads to a long hang and eventual error after transferring part of the file. The error is consistent. The file copies at normal speed until it reaches its maximum size (1.4 GB out of 20 GB for one, 6.9 GB out of 29 GB for another). Then nothing happens for a while (many minutes). Finally, there is an error:
[....]
jh/.VirtualBox/win7/win7.vbox
jh/.VirtualBox/win7/win7.vbox-prev
jh/.VirtualBox/win7/win7.vdi
rsync: [sender] write error: Broken pipe (32)
rsync error: error in rsync protocol data stream (code 12) at io.c(837) [sender=3.1.0]
In this case, 6.9 GB of 29 GB transferred. Without -z, it works.
See the following upstream report, with a comment at the end from the rsync maintainer:
https://bugzilla.samba.org/show_bug.cgi?id=10372
According to this report, version 3.1.0 (included in 14.04) uses a different compression package from prior versions. Prior versions did not have this problem for me using the same command on the same systems. Both hosts ran Ubuntu 11.10 at the time, and all run 14.04 now, in each case with all updates applied, Intel hardware. Network connection between them is gigabit ethernet through one switch. A shell ssh between them in a terminal works and stays up during the failure, so it is not a network issue. There are no relevant entries in syslog on either machine. There is sufficient capacity on the receiving disk. All filesystems are ext4.
rsync command:
/usr/bin/rsync -aHSxvz --delete --stats --exclude=lost+found --exclude=.gvfs --exclude=/nonlaptop /home/ backup.host.edu:/bu/host/home/
(yes, I changed the machine names)
Current release (both hosts):
Description: Ubuntu 14.04.1 LTS
Release: 14.04
Current package (both hosts):
rsync:
Installed: 3.1.0-2ubuntu0.1
Candidate: 3.1.0-2ubuntu0.1
Version table:
*** 3.1.0-2ubuntu0.1 0
500 http://us.archive.ubuntu.com/ubuntu/ trusty-updates/main amd64 Packages
500 http://security.ubuntu.com/ubuntu/ trusty-security/main amd64 Packages
100 /var/lib/dpkg/status
3.1.0-2 0
500 http://us.archive.ubuntu.com/ubuntu/ trusty/main amd64 Packages
Thanks,
--jh-- |
[Status]
This issue is currently Incomplete.
We're currently blocked on obtaining access to a reliable, and publicly
available dataset that can be shared to reproduce the issue. The test
is critical to being able to evaluate the impact of the change on other
users when assessing if the fix can be SRU'ed to trusty.
If you are affected by this bug and can reliably reproduce the issue
with a specific dataset that you are willing and permitted to share please
comment and specify the dataset location, and how you reproduce the issue.
[Original Description]
Copying large (>10GB) files with rsync -z (compression) leads to a long hang and eventual error after transferring part of the file. The error is consistent. The file copies at normal speed until it reaches its maximum size (1.4 GB out of 20 GB for one, 6.9 GB out of 29 GB for another). Then nothing happens for a while (many minutes). Finally, there is an error:
[....]
jh/.VirtualBox/win7/win7.vbox
jh/.VirtualBox/win7/win7.vbox-prev
jh/.VirtualBox/win7/win7.vdi
rsync: [sender] write error: Broken pipe (32)
rsync error: error in rsync protocol data stream (code 12) at io.c(837) [sender=3.1.0]
In this case, 6.9 GB of 29 GB transferred. Without -z, it works.
See the following upstream report, with a comment at the end from the rsync maintainer:
https://bugzilla.samba.org/show_bug.cgi?id=10372
According to this report, version 3.1.0 (included in 14.04) uses a different compression package from prior versions. Prior versions did not have this problem for me using the same command on the same systems. Both hosts ran Ubuntu 11.10 at the time, and all run 14.04 now, in each case with all updates applied, Intel hardware. Network connection between them is gigabit ethernet through one switch. A shell ssh between them in a terminal works and stays up during the failure, so it is not a network issue. There are no relevant entries in syslog on either machine. There is sufficient capacity on the receiving disk. All filesystems are ext4.
rsync command:
/usr/bin/rsync -aHSxvz --delete --stats --exclude=lost+found --exclude=.gvfs --exclude=/nonlaptop /home/ backup.host.edu:/bu/host/home/
(yes, I changed the machine names)
Current release (both hosts):
Description: Ubuntu 14.04.1 LTS
Release: 14.04
Current package (both hosts):
rsync:
Installed: 3.1.0-2ubuntu0.1
Candidate: 3.1.0-2ubuntu0.1
Version table:
*** 3.1.0-2ubuntu0.1 0
500 http://us.archive.ubuntu.com/ubuntu/ trusty-updates/main amd64 Packages
500 http://security.ubuntu.com/ubuntu/ trusty-security/main amd64 Packages
100 /var/lib/dpkg/status
3.1.0-2 0
500 http://us.archive.ubuntu.com/ubuntu/ trusty/main amd64 Packages
Thanks,
--jh-- |
|
| 2016-11-25 08:41:59 |
Michael Foley |
attachment added |
|
rsync compression error test data set https://bugs.launchpad.net/ubuntu/+source/rsync/+bug/1384503/+attachment/4782830/+files/lp-bug-1384503-rsync-bad-data.tgz |
|
| 2016-11-25 10:27:24 |
Tom Haddon |
bug |
|
|
added subscriber The Canonical Sysadmins |
| 2016-11-25 10:27:43 |
Tom Haddon |
rsync (Ubuntu Trusty): status |
Incomplete |
New |
|
| 2016-11-25 10:28:33 |
Tom Haddon |
rsync (Ubuntu Trusty): status |
New |
Confirmed |
|
| 2016-12-01 05:33:38 |
Haw Loeung |
bug |
|
|
added subscriber Haw Loeung |
| 2016-12-01 05:39:12 |
William Grant |
bug |
|
|
added subscriber William Grant |
| 2016-12-07 15:26:36 |
Jon Grimm |
rsync (Ubuntu Trusty): status |
Confirmed |
Triaged |
|
| 2016-12-07 15:26:45 |
Jon Grimm |
rsync (Ubuntu Trusty): importance |
Undecided |
Medium |
|
| 2017-01-03 04:45:22 |
Robie Basak |
rsync (Ubuntu Trusty): assignee |
Ryan Harper (raharper) |
|
|
| 2017-01-03 13:43:46 |
Christoph Lukas |
removed subscriber Christoph Lukas |
|
|
|
| 2017-06-08 17:59:12 |
David Britton |
rsync (Ubuntu Trusty): status |
Triaged |
Won't Fix |
|
