Progress not working with many backends

[Artur Bodera]

The --progress flag and progress bar seem to be broken with a lot of backends.
So far I've tested CF+Hubic and onedrive, both behave the same way. With --verbosity debug I can see that subsequent volumes are being sent to backend for transfer, backend keeps transferring them, but the progress bar after 8 hours of work looks the same:

0.0KB 08:00:00 [0.0B/s] [> ] 0% ETA Stalled!
0.0KB 08:00:03 [0.0B/s] [> ] 0% ETA Stalled!
0.0KB 08:00:06 [0.0B/s] [> ] 0% ETA Stalled!
0.0KB 08:00:09 [0.0B/s] [> ] 0% ETA Stalled!

It never moves.

I understand that some backends might not be good at reporting speed or file transfer progress, but I don't see why the progress bar would not be updated at least after each volume has been transferred ( no. MB transferred / no MB total to transfer = progress).

[Artur Bodera (abodera)]

Same happens with onedrive.

[Cysioland (cysioland)]

Same happens with plain, old file:///. So I guess it counts as confirmed, amirite?

[dmtroyer (dmtroyer)]

happening with b2 on 0.7.06.

[Courtney Rosenthal (courtneyr)]

I can confirm this problem with rsync backend on 0.7.06.

[Samic (i-samic)]

Same thing with rsync backend on 0.6.23

[Xytime (xytime)]

Happens with 0.7.07.1 (ArchLinux) with the sftp:// backend

[Guy Rutenberg (guyrutenberg)]

Also affects the pydrive backend (Google Drive).

[Paweł Krawczyk (pawel-krawczyk)]

Also with S3.

[Paweł Krawczyk (pawel-krawczyk)]

In my case this was caused by iptables blocking packets with "invalid" state (-m state --state INVALID) which seems to block some packets from AWS. Removing that rule restored the full connection speed.

[Artur Puzio (cytadela8-g)]

For me also happens with 0.7.07.1(Gentoo) with the sftp:// backend.

[cortocopy (cortocopy)]

It happens with Swift too. S3 works fine however

using duplicity 0.7.10

[orest (piahoo-u)]

same issue on fedora26, duplicity 0.7.15

[Kevin (wittyman37)]

Same issue with duplicity 0.7.06, Ubuntu GNOME 16.04.4, and webdav server

[Ignacy (ignacy-sawa)]

PARTIAL SOLUTION:

Progress can be monitored using `pv` function. It's especially useful when backing up large files.
First grab the process id, and then put it as parameter to pv.

Example:
`ps -ef | grep duplicity`
We read the PID (first number after the username).
pv -d PID

Or a one-liner*:
*assuming we are copying a file named "disk". The second grep narrows the search results.

pv -d $(ps -ef | grep duplicity | grep disk | tr -s ' ' | cut -d ' ' -f 2)

[Kevin (wittyman37)]

Here's a slightly shorter version

pv -d "$(pgrep -u $USER duplicity)"

[iBart (bart-)]

Still present with rsync version 0.8.09

[Kenneth Loafman (kenneth-loafman)]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

 status confirmed
 importance medium
 done
-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEExwf+p6/3mDvUSsaGH1G0ZBEHM/EFAmBmFhgACgkQH1G0ZBEH
M/EelQgArjMyCrUPB6SDdzRm0HaIRaCWp868f0bnCAulW1CYoQBCPe3lYPdYjmMH
WU62hzntxnQxKOYvZ2tNV7uzmaZtIvyliCFcu++PTvIKpZhDTf+bi3lDbW6h6vq2
CRx2zTkE865ev8AwbFmwD2yXntZ43BDL4x6Jp9tEw52e9nSW8qIsS/oGsWarWNcx
W5E8hi99ObGcv7/KFOxtosa9/meH3NyjmIzXokEseOvwUPux4qSJpufzDiaVCds9
djHmZbJpffQKbqKihZhdpfO7iHQECTMIzrrFiViJclbGVs3F1FXU8AKXDSnN2z1q
6i6K1N9gpd4ddcTc/YiIgy4NOf0EYw==
=/Q/F
-----END PGP SIGNATURE-----

[ChieftainY2k (chieftainy2k)]

Hello, I am also affected by this bug.

But since this issue is alive and well for so long :) I wonder if there is any pay-per-bug plan for this project ?

I would gladly pay someone to fix this bug. I'm serious.