use gzip in a subprocess on unix

[Robert Collins]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Rather than using gzip file and doing a seek(0), fast-import could:
 - use gzip in a subprocess
 - reopen the process to start over

I suspect this would be faster on mp machines, due to freeing up an
entire CPU to do decompression. !citation needed of course.

 affects bzr-fastimport

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkqkg3UACgkQ42zgmrPGrq7l3gCgkUJRdRljl78v/51VtmX1m+I0
/GQAnR2I2EIxCw7lIVtdmMsiGuYP/9N/
=M31S
-----END PGP SIGNATURE-----

[John A Meinel (jameinel)]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Robert Collins wrote:
> Public bug reported:
>
> Rather than using gzip file and doing a seek(0), fast-import could:
> - use gzip in a subprocess
> - reopen the process to start over
>
> I suspect this would be faster on mp machines, due to freeing up an
> entire CPU to do decompression. !citation needed of course.
>
> affects bzr-fastimport
>

I should also mention that gzip.GzipFile(open('foo.gz')).read() is often
slower than Popen(['gzip foo.gz']).stdout.read()

I don't remember all the details, but I do remember benchmarking that
sort of thing. (I believe this was even on a single processor machine.)

John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkqmZocACgkQJdeBCYSNAAOE1QCgvXnjCO+YoDPUgh360Amnpw2B
kzgAoIO6FKNIz7aRzkUYeCGdFZ/vWmDG
=4h8j
-----END PGP SIGNATURE-----