[nc] Mono font isn't discoverable by formatting objects processor

[Jeremy Kerr]

I'm using the formatting object processor ("fop") to produce PDFs. I can use the proportional Ubuntu just fine, but fop can't seem to use the Ubuntu mono font.

I'm using fop version 1.0 from http://xmlgraphics.apache.org/fop/. Version 0.95 is packaged in Ubuntu, but that does not have functionality to autodiscover fonts.

My testcase:

[jk@pororo fop]$ cat fopconfig.xml
<?xml version="1.0"?>
<fop version="1.0">
 <renderers>
  <renderer mime="application/pdf">
   <fonts>
    <auto-detect/>
   </fonts>
  </renderer>
 </renderers>
</fop>

[jk@pororo fop]$ cat test.fo
<?xml version="1.0"?>
<fo:root xmlns:fo="http://www.w3.org/1999/XSL/Format"
 font-size="10pt">
  <fo:layout-master-set>
    <fo:simple-page-master master-name="master"
     page-width="60mm" page-height="60mm"
     margin-top="10mm" margin-bottom="10mm"
     margin-left="10mm" margin-right="10mm">
      <fo:region-body display-align="center" region-name="body"/>
    </fo:simple-page-master>
  </fo:layout-master-set>

  <fo:page-sequence master-reference="master">
    <fo:flow flow-name="body">
      <fo:block font-family="Ubuntu">hello, world</fo:block>
      <fo:block font-family="Ubuntu Mono">hello, world</fo:block>
    </fo:flow>
  </fo:page-sequence>

</fo:root>

[jk@pororo fop]$ /opt/fop-1.0/fop -pdf test.pdf -fo test.fo -c fopconfig.xml
19/12/2011 4:28:37 PM org.apache.fop.events.LoggingEventListener processEvent
WARNING: Font "Ubuntu Mono,normal,400" not found. Substituting with "any,normal,400".

[jk@pororo fop]$ /opt/fop-1.0/fop -version
FOP Version 1.0

- note that the Ubuntu font doesn't generate any warning, and appears correctly in the generated PDF. The second "hello, world" (which should be rendered in Ubuntu Mono) appears in a fallback font.

I have version 0.80-0ubuntu1~medium of the ttf-ubuntu-font-family package installed.

[Jeremy Kerr (jk-ozlabs)]

When trying to generate the metrics for the Ubuntu Mono font manually, the metrics processor throws an ArrayIndexOutOfBoundException:

[jk@pororo fop]$ java -cp /opt/fop-1.0/build/fop.jar:/opt/fop-1.0/lib/commons-logging-1.0.4.jar:/opt/fop-1.0/lib/commons-io-1.3.1.jar:/opt/fop-1.0/lib/xmlgraphics-commons-1.4.jar org.apache.fop.fonts.apps.TTFReader /usr/share/fonts/truetype/ubuntu-font-family/UbuntuMono-R.ttf UbuntuMono.metrics.xml
TTF Reader for Apache FOP 1.0

Parsing font...
Reading /usr/share/fonts/truetype/ubuntu-font-family/UbuntuMono-R.ttf...
Error while building XML font metrics file.
java.lang.ArrayIndexOutOfBoundsException: 1047

[jk@pororo fop]$

[Jeremy Kerr (jk-ozlabs)]

So it looks like this is cause by an invalid "post" table in the font file. The post table declares that there are 1047 glyph names (above the standard macintosh set), but the glyph name indices in the post table go well above 1046.

[Paul Sladen (sladen)]

After patching the line-drawing characters back in (see, LP: #788757), I don't think the index was being rebuilt, so this is probably the same as the other cases (bug #864202 and dupes) where the meta-data is being checked consistency.

In the 'lp:ubuntu-font-family' build source, there is the script that us used to rebuild the mono fonts from the Dalton Maag binaries there. This is 'midstream/patch-monospaced-mono.py' and there should be a function 'map_line_drawing_alternates()' based on Cody Boisclair's solution, that currently has two lines commented out involving a getGlyphOrder()/setGlyphOrder() call pair on the belief that they were superfluous.

Does uncommenting those and rebuilding the mono fonts create a difference (try comparing with 'sfddiff' in the 'fontforge-nox' No-X Window utility package). Do re-rebuilt UFF mono fonts then load? If so we should have a the cause and it can be duped against the other bugs about the mono mapping discrepancy (bug #864202) .

Either way, it should be cured in a couple of months with a new release from Dalton Maag that maps these and the other CP437/ASCII characters by default (bug #812396).

[Jeremy Kerr (jk-ozlabs)]

Hi Paul,

Thanks for the quick response.

I've got the build source, but can't seem to work out what to do with it; it doesn't contain any of the font sources, just a Makefile and some python scripts. The Makefile (midstream/rules) is able to download some metadata, but doesn't have any reference to upstream font sources.

Could you let me know which source file you want me to run the patch-monospaced-mono.py file on? Or how I'm supposed to use the lp:ubuntu-font-family repo?

[Paul Sladen (sladen)]

Jeremy: it should, I think, be okay just to re-run the file on the 'UbuntuMono-{R,B}{I,}.ttf' files. If that doesn't work, I'll try and find you privately the raw unprocessed tarball from Dalton Maag which 'midstream' transforms to make the 'upstream' release packages.

[Jeremy Kerr (jk-ozlabs)]

Paul - thanks for the details. Looks like the modified python script does not fix the post table:

[jk@pororo ubuntu-font-family]$ bzr diff
=== modified file 'midstream/patch-monospaced-mono.py'
--- midstream/patch-monospaced-mono.py 2011-09-26 15:11:51 +0000
+++ midstream/patch-monospaced-mono.py 2011-12-20 00:35:11 +0000
@@ -117,14 +117,14 @@
         # Microsoft Unicode: platform 3, encoding 1
         msMap = f['cmap'].getcmap(3,1).cmap

- #order = f.getGlyphOrder()
+ order = f.getGlyphOrder()
         for name in renGlyphs:
             mapping = agl.AGL2UV[name]
             uniMap[mapping] = name
             msMap[mapping] = name
             print '%s mapped to Unicode code point: %d' % (name, mapping)

- #f.setGlyphOrder(order)
+ f.setGlyphOrder(order)

     # Thanks to Cody Boisclair (codeman38) for 'fixboxdraw-ft.py'
     def force_widths(f, advance=500):

[jk@pororo ubuntu-font-family]$ ./midstream/patch-monospaced-mono.py --rewrite-mono --map-box-drawing /usr/share/fonts/truetype/ubuntu-font-family/UbuntuMono-R.ttf ./UbuntuMono-R.new.ttf
0 1 0 1 1 0 2 1 0 3 1 0 4 1 0 5 1 0 6 1 0 7 1 0 8 1 0 9 1 0 11 1 0 12 1 0 16 1 0 17 1 0 18 1 0 0 3 1 1 3 1 2 3 1 3 3 1 4 3 1 5 3 1 6 3 1 7 3 1 8 3 1 9 3 1 11 3 1 12 3 1 16 3 1 17 3 1 18 3 1

[jk@pororo ubuntu-font-family]$ java -classpath /opt/fop-1.0/build/fop.jar:/opt/fop-1.0/lib/commons-logging-1.0.4.jar:/opt/fop-1.0/lib/commons-io-1.3.1.jar:/opt/fop-1.0/lib/xmlgraphics-commons-1.4.jar org.apache.fop.fonts.apps.TTFReader ./UbuntuMono-R.new.ttf UbuntuMono.metrics.xml
TTF Reader for Apache FOP 1.0

Parsing font...
Reading ./UbuntuMono-R.new.ttf...
Error while building XML font metrics file.
java.lang.ArrayIndexOutOfBoundsException: 1047

[Jeremy Kerr (jk-ozlabs)]

Reproduceable on precise, using the (newly-package) distro version of fop.