a progress bar for 'cp' and 'mv'

Bug #64067 reported by Hari Seldon on 2006-10-04
70
This bug affects 12 people
Affects Status Importance Assigned to Milestone
coreutils (Debian)
Won't Fix
Unknown
coreutils (Fedora)
Won't Fix
Medium
coreutils (Ubuntu)
Wishlist
Unassigned

Bug Description

Binary package hint: coreutils

Hello.
First sorry for my english, but it is not my native langage.

I'm a shelluser, and in my every day utilisation I copy and move some files, and some of them are big (>500Mo for example). So i was searching for progress bar with indication of the pourcentage and something like 34Mo/400Mo, like wget. On linuxfr.org (http://linuxfr.org/~pascalscl/13847.html) i found a discution about that, and i discover a secret option for 'cp' : the -g (for --gauge). I discover too a bug report about that on the archive of the Bug-coreutils mailing list (http://<email address hidden>/msg00610.html).
But it appear that this patch isn't present in the various release of coreutils on Ubuntu.

I know that isn't a bug, and I know that isn't a vital part of the coreutils. But I'm persuaded that the gauge is an interesant evolution for the tools 'cp' and 'mv'. For all the shelluser around the world i think it will be a good comfort to facilitate their utilisation of the shell.

So my question is : what about a future integration of a progress bar in some tools in the coreutils ?

Thank you for your patience :)

Regards.

Hari Seldon (illovae)

tags 185152 +patch

And progress bars in mv, rm and, most important, dd would also be nice!

Best regards,

Onno

# Automatically generated email from bts, devscripts version 2.8.10
merge 284514 185152

tags 68603 wontfix
tags 185152 wontfix
tags 17025 wontfix
retitle 120650 coreutils: chmod: Please add R, W, S and T options
retitle 210475 coreutils: stty: Please add option to force stty to act immediately, not to wait for output to be empty
retitle 21750 coreutils: install: Please add option to compress files
tags 21750 wontfix
tags 48038 upstream
retitle 55218 coreutils: ls: Please add option to sort by type (i.e., all directories before all files)
tags 56844 wontfix
tags 89335 wontfix
stop

Hello.

I wonder if there is an updated version of that patch which works for the
current version of the coreutils in Debian Sid.

I found a HowTo but it’s for Sarge’s version and won’t get applied successfully.

Regards, Mathias

--
debian/rules

merge 185152 389326

Binary package hint: coreutils

Hello.
First sorry for my english, but it is not my native langage.

I'm a shelluser, and in my every day utilisation I copy and move some files, and some of them are big (>500Mo for example). So i was searching for progress bar with indication of the pourcentage and something like 34Mo/400Mo, like wget. On linuxfr.org (http://linuxfr.org/~pascalscl/13847.html) i found a discution about that, and i discover a secret option for 'cp' : the -g (for --gauge). I discover too a bug report about that on the archive of the Bug-coreutils mailing list (http://<email address hidden>/msg00610.html).
But it appear that this patch isn't present in the various release of coreutils on Ubuntu.

I know that isn't a bug, and I know that isn't a vital part of the coreutils. But I'm persuaded that the gauge is an interesant evolution for the tools 'cp' and 'mv'. For all the shelluser around the world i think it will be a good comfort to facilitate their utilisation of the shell.

So my question is : what about a future integration of a progress bar in some tools in the coreutils ?

Thank you for your patience :)

Regards.

Hari Seldon (illovae)

effraie (effraie) wrote :

i comfirm ;)

Scott Bronson (bronson) wrote :

So long as it isn't on by default (so shell scripts don't get messed up), I'm all for it. I'd love to see this.

Daniel Robitaille (robitaille) wrote :

By the way, this was asked in the past in Debian (for "cp") and rejected.

Changed in coreutils:
importance: Undecided → Wishlist
status: Unconfirmed → Confirmed
effraie (effraie) wrote :

why was it rejected?

Daniel Robitaille (robitaille) wrote :

No reasons were given in the Debian bug report about the rejection.

Laurent Bigonville (bigon) wrote :

I think this should be done upstream...

Changed in coreutils:
status: Unknown → Confirmed

It would be bad if Ubuntu introduces some -g option and then later upstream uses -g for something else. Having different options in coreutils in various distros is not good. This should be done upstream.

Yes it's true, the best will be to integrate the "option -g" in all coreutils for all the distribution of GNU/Linux and derivated. But how can we contact the developers of coreutils ? And can an end-shell-user can be allowed to do this without appear insulting opposite to their labor ?

Hello.

For anyone interested I’ll attach the necessary patch, working at least for
version 5.97-5. I made sure it builds using the acl and selinux patches.

Regards, Mathias

--
debian/rules

Micah Cowan (micahcowan) wrote :

I think such a feature should possibly be enabled via a long-option only. This should drastically decrease the likelihood of conflicting with cp/mv from other OSses. (My 2¢.)

There are a lot of cases when you're copying something locally that being able
to have a progress indicator like you get with scp or rsync would be nice. And
rather than writing Yet Another Utility, it seems like the right thing to do is
for cp to provide the functionality.

Hi Jeremy,
Many people have requested this, over the years, and there are probably a few
complete patch sets implementing it. Why do you think it's best to add this to
cp, when you can already use rsync as a cp-replacement?

(In reply to comment #1)
> Many people have requested this, over the years, and there are probably a few
> complete patch sets implementing it. Why do you think it's best to add this to
> cp, when you can already use rsync as a cp-replacement?

Because then you have a dependency on rsync instead of cp... cp is pretty much
guaranteed to exist, rsync not as much. Also, rsync is in /usr/bin whereas cp
is in /bin. And rsync is 5 times larger.

Given that a lot of the time you're going to care about text mode cp and
progress is in early boot, you want to avoid all of the above.

Well, consider that some folks want a GUI-oriented progress bar, and some want
text-only. Some are happy to incur the cost of a curses library, others want
the minimal footprint and are happy with less eye candy. We've hashed out all
of this, over the years, and came to the conclusion that if there is to be any
such thing associated with GNU cp (and mv, and dd, and maybe others, like
install), then the mechanism must be sufficiently generic to work with an
independent program that would be in charge of displaying the progress meter.
Then, the core copying code needn't be polluted with code that some will never
want or use. Here's a proposed implementation from 2004:
http://thread.gmane.org/gmane.comp.gnu.coreutils.bugs/1663

(In reply to comment #3)
> Well, consider that some folks want a GUI-oriented progress bar, and some want
> text-only. Some are happy to incur the cost of a curses library, others want
> the minimal footprint and are happy with less eye candy. We've hashed out all
> of this, over the years, and came to the conclusion that if there is to be any
> such thing associated with GNU cp (and mv, and dd, and maybe others, like
> install), then the mechanism must be sufficiently generic to work with an
> independent program that would be in charge of displaying the progress meter.
> Then, the core copying code needn't be polluted with code that some will never
> want or use. Here's a proposed implementation from 2004:
> http://thread.gmane.org/gmane.comp.gnu.coreutils.bugs/1663

And at this point, you have entered well into the realm of over-engineering a
simple problem ;-)

If you're using curses, etc and wanting to show the output of cp, you're already
writing an app at which point the part of cp that are going to be cared about
aren't that hard to implement. The value here is in being able to have a shell
script give some form of progress feedback as it copies a large file.

From a distance, most problems look simpler than they are :-)

If you're interested in copying a single file, then maybe dd already provides
what you want? From dd --help:

Sending a USR1 signal to a running `dd' process makes it
print I/O statistics to standard error and then resume copying.

  $ dd if=/dev/zero of=/dev/null& pid=$!
  $ kill -USR1 $pid; sleep 1; kill $pid
  18335302+0 records in
  18335302+0 records out
  9387674624 bytes (9.4 GB) copied, 34.6279 seconds, 271 MB/s

With that basis, a little wrapper script could do what you want.

It seems your patch have an error with coreutils 5.97-5.3:

if
colorgcc -std=gnu99 -DHAVE_CONFIG_H -DLIBDIR=\"/usr/lib\" -I. -I. -I.. -I.. -I. -g -DSYSLOG_SUCCESS -DSYSLOG_FAILURE -DSYSLOG_NON_ROOT -O2 -MT
utimecmp.o -MD -MP -MF ".deps/utimecmp.Tpo" -c -o utimecmp.o utimecmp.c; \
        then mv -f ".deps/utimecmp.Tpo" ".deps/utimecmp.Po"; else
rm -f ".deps/utimecmp.Tpo"; exit 1; fi
In file included from utimecmp.c:41:
utimens.h:2: error: conflicting types for 'futimens'
/usr/include/sys/stat.h:370: error: previous declaration of 'futimens' was
here

I realize that this is just a wishlist level item but I wanted to
register my objection to the additional code to implement progress
bars to cp, mv, rm, etc. Those utilities are already too bloated.

Also the functionality of progress bars exists in rsync. If someone
wants a tool with everything including the kitchen sink then using
rsync works well and already exists.

  rsync --progress -av SOURCE DESTINATION

Bob

Hi Victor-Philipp.

Victor-Philipp Busch, 13.07.2007 11:31:
> It seems your patch have an error with coreutils 5.97-5.3:
>
> if
> colorgcc -std=gnu99 -DHAVE_CONFIG_H -DLIBDIR=\"/usr/lib\" -I. -I. -I.. -I.. -I. -g -DSYSLOG_SUCCESS -DSYSLOG_FAILURE -DSYSLOG_NON_ROOT -O2 -MT
> utimecmp.o -MD -MP -MF ".deps/utimecmp.Tpo" -c -o utimecmp.o utimecmp.c; \
> then mv -f ".deps/utimecmp.Tpo" ".deps/utimecmp.Po"; else
> rm -f ".deps/utimecmp.Tpo"; exit 1; fi
> In file included from utimecmp.c:41:
> utimens.h:2: error: conflicting types for 'futimens'
> /usr/include/sys/stat.h:370: error: previous declaration of 'futimens' was
> here

No, that’s a bug in coreutils. However, I don’t know why it doesn’t work for you
where it did for me. See #433394 for a fix. (Might still require backporting
since the patch over there is for version 6.0.)

I updated the patch to work with coreutils 6.0-1 from Experimental. As ususal,
use at your own risk; it works fine here, though.

Regards, Mathias

--
debian/rules

Changed in coreutils:
status: Confirmed → Won't Fix

I agree that a progress bar in cp & mv would be very nice at times. rsync can
be very overkill (or unusable in certain situations (like writing to an FS
that doesn't support dot files - vfat, for example - and I would prefer not to
use rsync's -T option to write to yet another fs).

This bug exists on other forums as well:
https://bugs.launchpad.net/ubuntu/+source/coreutils/+bug/64067

# Automatically generated email from bts, devscripts version 2.10.11
tags 185152 - patch

Based on the date this bug was created, it appears to have been reported
against rawhide during the development of a Fedora release that is no
longer maintained. In order to refocus our efforts as a project we are
flagging all of the open bugs for releases which are no longer
maintained. If this bug remains in NEEDINFO thirty (30) days from now,
we will automatically close it.

If you can reproduce this bug in a maintained Fedora version (7, 8, or
rawhide), please change this bug to the respective version and change
the status to ASSIGNED. (If you're unable to change the bug's version
or status, add a comment to the bug and someone will change it for you.)

Thanks for your help, and we apologize again that we haven't handled
these issues to this point.

The process we're following is outlined here:
http://fedoraproject.org/wiki/BugZappers/F9CleanUp

We will be following the process here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping to ensure this
doesn't happen again.

Adding words future feature and moving back to ASSIGNED. As Jim said, there are
already some complete patches implementing the progress bar feature to cp, mv
and install - usually via long option or via -g option. So far it seems that
upstream is not interested in accepting progress bar feature (even via long
option) and making fork will mean one more patch which will be not oneliner and
will need some maintainance as cp/mv/install source codes are changing quiet a
lot between releases.

C de-Avillez (hggdh2) wrote :

This option(s) have been suggested upstream for quite some time now; last one was at about end of 2007. If something is to be done for a progress bar on cp, rm, mv, etc, it should, indeed, be upstream.

You can contact upstream at <email address hidden>.

Kamil Dudka (kdudka) wrote :

It is suggested to use rsync as cp replacement in such cases - consider this bug: https://bugzilla.redhat.com/show_bug.cgi?id=237553

Changed in coreutils:
status: Unknown → In Progress

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi there,

Updated patch for coreutils at least version 7.1.

Regards, Mathias

- --
debian/rules
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAknG0O0ACgkQYfUFJ3ewsJiBlQCfQejSCG42ptuvrHhKL4vGByXU
6t8AmwS7e/h1XU+mWkXbsjJvfLXrapJa
=pKDe
-----END PGP SIGNATURE-----

hackeron (hackeron) wrote :

I can't find an example that will show the overall progress of recursively copying a directory? - Anyone?

Rsync shows progress per file and an overall filecount, but is there anything that will show overall progress for size?

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi.

Here’s the patch refreshed against the latest coreutils 8.1.

Regards, Mathias

- --
debian/rules
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAksL2vYACgkQYfUFJ3ewsJj0wACeLK/CzGP4pu7RXxuEdtib3q3d
epQAn3ruREoL6Zv/jJfrD39JundZHlNR
=5VqH
-----END PGP SIGNATURE-----

Time to close that bugzilla...
Discussed many times on upstream list (see archives),
in bugzillas of other linux distributions (e.g.
https://bugs.launchpad.net/ubuntu/+source/coreutils/+bug/64067
http://www.linux-archive.org/archlinux-development/167447-add-g-option-cp-mv.html
)

If you really want to have this behaviour, you could use some shell script alias to get this. One of such scripts is available at http://chris-lamb.co.uk/2008/01/24/can-you-get-cp-to-give-a-progress-bar-like-wget/ . Closing WONTFIX.

hackeron (hackeron) wrote :

Anything? :)

C de-Avillez (hggdh2) wrote :

For the record, the RH bug has been closed WONTFIX, here you have Ondrej's comment:

"Time to close that bugzilla...
Discussed many times on upstream list (see archives),
in bugzillas of other linux distributions (e.g.
https://bugs.launchpad.net/ubuntu/+source/coreutils/+bug/64067
http://www.linux-archive.org/archlinux-development/167447-add-g-option-cp-mv.html
)

If you really want to have this behaviour, you could use some shell script
alias to get this. One of such scripts is available at
http://chris-lamb.co.uk/2008/01/24/can-you-get-cp-to-give-a-progress-bar-like-wget/
 . Closing WONTFIX."

I will keep this bug open only because upstream will look at a *generic*, usable-everywhere solution. But -- unless you intend to code it, please do not expect any coreutils-implemented solution anytime soon (if at all). There are other ways of getting it (see Ondrej's comment above.

Changed in coreutils (Ubuntu):
status: Confirmed → Triaged

As of F19, the cp command has the -v (--verbose) option that shows what files are being copied:

$ man cp
...
       -v, --verbose
              explain what is being done
...

$ tar -xf python-bugzilla-0.9.0.tar.bz2
$ ls
python-bugzilla-0.9.0/ python-bugzilla-0.9.0.tar.bz2
$
$ cp -v -a python-bugzilla-0.9.0 foo-1
‘python-bugzilla-0.9.0’ -> ‘foo-1’
‘python-bugzilla-0.9.0/.mailmap’ -> ‘foo-1/.mailmap’
‘python-bugzilla-0.9.0/bz-api-notes.txt’ -> ‘foo-1/bz-api-notes.txt’
... [snipped] ...
‘python-bugzilla-0.9.0/bin/bugzilla’ -> ‘foo-1/bin/bugzilla’
‘python-bugzilla-0.9.0/bugzilla.1’ -> ‘foo-1/bugzilla.1’
‘python-bugzilla-0.9.0/setup.py’ -> ‘foo-1/setup.py’
$ ls
foo-1/ python-bugzilla-0.9.0/ python-bugzilla-0.9.0.tar.bz2
$

Tested with:
$ rpm -qf `which cp`
coreutils-8.21-11.fc19.x86_64

(In reply to Steve Tyler from comment #12)
> As of F19, the cp command has the -v (--verbose) option that shows what
> files are being copied:

The --verbose option has been there at least since fileutils-3.8.3b (released in 1993), I did not dig deeper in the history...

Thanks for your digging, Kamil. None of the earlier comments mentioned the --verbose option, so I assumed that it must have been added more recently than 2010 (Comment 11).

Høst (helvete) wrote :

Hello. There are a coreutils package with "advanced" cp and mv that can show the progress bar on file operations like copy/move.
I installed it on my system and extracted the cp and mv binaries.

hoest@hoest:~$ which mv
/usr/local/bin/mv
hoest@hoest:~$ which cp
/usr/local/bin/cp

You need to place the 2 binaries in that locations. After, put this 2 aliases in your .bashrc:

alias cp='/usr/local/bin/cp -g'
alias mv='/usr/local/bin/mv -g'

'-g' is the argument that tells to them command to show the progress bar. Hope it works

The attachment "Compiled binaries with the progress bar included" seems to be a patch. If it isn't, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are a member of the ~ubuntu-reviewers, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issues please contact him.]

tags: added: patch
Changed in coreutils (Fedora):
importance: Unknown → Medium
status: In Progress → Won't Fix
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.