Comment 5 for bug 174481

Based on all the testing and feedback (thank you especially Dan and Marcus), I've checked in the changes. Here's what I've done, along with some explanatory notes.

First, the results of the changes:

- pdfs are viewable error-free on a wider range of applications
- color pdfs are much smaller (roughly half their former size)
- grayscale pdfs are, in most cases, slightly smaller
- when viewed in most applications, text clarity is detectably better (and dramatically so in the case of Preview under Mac 10.4 - I'm attaching a pair of before/after screen shots of that)

The call to pdf_comp, for both color and grayscale pdfs, now specifies the following overrides to default values:

-layers 0 -quality 4 -fsample 1

We allow the other options to default; two of those default values of particular interest are:

-image jp2 -bsample 3

Now the explanation:

Color pdfs had been using '-image jpg' (meaning that foreground and background images were compressed using jpg compression). Switching to the default of jp2 is what shrunk them in half.

'-layers' controls whether the user can separately turn the layers on or off for viewing. (The different layers are created in any case; the option just controls whether they're individually switchable.) The default of 'layers 1' causes viewing errors in some applications, including odd "inverse video" effects in Mac Preview (under both 10.4 and 10.5). So we now use 0.

'-quality' defaults to 6 (max is 10). I could find no visible difference among quality values ranging from 4 to 8, but they do affect file size.

'-fsample' and '-bsample' control the foreground and background image downsampling rate. The default varies with the quality setting, but would be 3 in our case. Bumping '-fsample' to 1 (i.e., no downsampling at all) is what increased the text clarity (most dramatically in Preview; perhaps it does worse anti-aliasing than Acrobat Reader). It also, however, increases file size by about as much as the size was reduced by shifting '-quality' from 6 to 4. Different compromises are, of course, possible - we could, for instance, use '-fsample 2', which would shrink file size while looking indistinguishable from '-fsample 1' in some applications but somewhat worse in others.

I left '-bsample' at the default of 3, as 2 wasn't visibly better, and 4 didn't save much on file size.

Finally, here's the combined list of platforms/apps on which one of us compared the new/old pdfs and checked the new ones for error-free display:

on Ubuntu 6.06:
  Adobe Reader 7.0.1
  Document Viewer (evince 0.5.2)
  xpdf 3.01

on Ubuntu 7.10:
   Document Viewer (evince 2.20.1)

on Windows XP Professional Version 2002:
   Adobe Reader 8
   FoxIT Reader 2.2
   SumatraPDF-0.7
   cutePDF
   eXPertPDF 2.0.50

on Mac OS X 10.4.11
  Adobe Reader 8.1.1
  Preview 3.0.9

on Mac OS X 10.5
  Preview (error-checking only)

also, Marcus printed new/old grayscale pdfs of one short book on the Espresso and saw no difference