Unicode variation selectors not supported

[Aleksandr]

I'm trying to typeset some texts in ancient Mongolian and Phags-pa, both of which have been included in Unicode. For Mongolian, I've installed the ttf-manchufont package. For Phags-pa, I've used the font available here: http://babelstone.blogspot.com/2006/12/phags-pa-shaping-behaviour.html

Both languages use the Unicode Variation Selectors. In a nutshell, the combination character + variation selector should change the representation of the character to a different variant. See the table of Standardised Unicode variants here: http://www.unicode.org/Public/UNIDATA/StandardizedVariants.html

The expected behavior does not work in any of the packages I've tried (gedit, gimp and OpenOffice). Instead, the variation selector remains as it's own character and the preceding character is unmodified (see attachment).

This is NOT an issue with the fonts because:
a. The fonts have the appropriate substitutions defined, as can be observed by editing the fonts in e.g. fontforge.
b. The expected behavior happens in WIndows 7 using these fonts.

This is an issue with whatever package handles Unicode glyph substitution.

ProblemType: Bug
DistroRelease: Ubuntu 10.10
Package: xorg 1:7.5+6ubuntu3
ProcVersionSignature: Ubuntu 2.6.35-27.48-generic 2.6.35.11
Uname: Linux 2.6.35-27-generic x86_64
NonfreeKernelModules: fglrx
Architecture: amd64
Date: Wed Mar 16 10:30:20 2011
InstallationMedia: Ubuntu 10.04.1 LTS "Lucid Lynx" - Release amd64 (20100816.1)
MachineType: MICRO-STAR INTERNATIONAL CO.,LTD MS-7576
ProcCmdLine: BOOT_IMAGE=/boot/vmlinuz-2.6.35-27-generic root=UUID=136a2918-79f6-4777-a8b5-b9d86515a24b ro quiet splash
ProcEnviron:
 LANGUAGE=en_US:en
 LANG=en_US.UTF-8
 SHELL=/bin/bash
SourcePackage: xorg
Symptom: display
dmi.bios.date: 12/17/2009
dmi.bios.vendor: American Megatrends Inc.
dmi.bios.version: V4.0
dmi.board.asset.tag: To Be Filled By O.E.M.
dmi.board.name: 790GX-G65 (MS-7576)
dmi.board.vendor: MICRO-STAR INTERNATIONAL CO.,LTD
dmi.board.version: 1.0
dmi.chassis.asset.tag: To Be Filled By O.E.M.
dmi.chassis.type: 3
dmi.chassis.vendor: MICRO-STAR INTERNATIONAL CO.,LTD
dmi.chassis.version: 1.0
dmi.modalias: dmi:bvnAmericanMegatrendsInc.:bvrV4.0:bd12/17/2009:svnMICRO-STARINTERNATIONALCO.,LTD:pnMS-7576:pvr1.0:rvnMICRO-STARINTERNATIONALCO.,LTD:rn790GX-G65(MS-7576):rvr1.0:cvnMICRO-STARINTERNATIONALCO.,LTD:ct3:cvr1.0:
dmi.product.name: MS-7576
dmi.product.version: 1.0
dmi.sys.vendor: MICRO-STAR INTERNATIONAL CO.,LTD
glxinfo: Error: [Errno 2] No such file or directory
system:
 distro: Ubuntu
 codename: maverick
 architecture: x86_64
 kernel: 2.6.35-27-generic

[Fabio Marconi (fabiomarconi)]

assign to scim because i don't know the right package.

[Aleksandr (aleksandr-andreev)]

Update in Ubuntu 11.04: Variation Selectors are fully supported in XeTeX. However, they are not supported in LibreOffice.

[Rolf Leggewie (r0lf)]

I'd be surprise if this was a scim issue. Please attach or paste in a comment the output of "dpkg -l *scim*"

[Aleksandr (aleksandr-andreev)]

Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Description
+++-==============-==============-============================================
un scim <none> (no description available)

[Aleksandr (aleksandr-andreev)]

Result of dpkg -l *scim*

Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Description
+++-==============-==============-============================================
un scim <none> (no description available)

[Aleksandr (aleksandr-andreev)]

Result of dpkg -l *scim*

[Rolf Leggewie (r0lf)]

Thank you for the quick reply which confirmed my suspicion that this has nothing to do with scim. Unfortunately, I have no idea which package deals with input in that language. Reassigning to Ubuntu for further triage.

[Rolf Leggewie (r0lf)]

Aleksandr, try running gnome-keyboard-properties from the command line and see if you can get the right configuration from there. I've got a hunch that it's either a configuration problem or bug in that program.

Assigning to gnome-control-center for further triage.

[Rolf Leggewie (r0lf)]

I'm sure not many people have experience with Mongolian, so it's very well possible this is a bug in the upstream code. You might want to discuss your issue at https://bugzilla.gnome.org/enter_bug.cgi?product=gnome-control-center to see if you can find more information. If you do, please make sure to let us know of the URL for the upstream ticket here.

[Aleksandr (aleksandr-andreev)]

Mongolian is just an example. Var selectors do not appear to work in
any script in gedit or LibreOffice.

They do work in Pango and Firefox (which uses Pango).

On Thu, Sep 29, 2011 at 8:55 PM, Rolf Leggewie
<email address hidden> wrote:
> I'm sure not many people have experience with Mongolian, so it's very
> well possible this is a bug in the upstream code.  You might want to
> discuss your issue at https://bugzilla.gnome.org/enter_bug.cgi?product
> =gnome-control-center to see if you can find more information.  If you
> do, please make sure to let us know of the URL for the upstream ticket
> here.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/736153
>
> Title:
>  Unicode variation selectors not supported
>
> Status in LibreOffice:
>  New
> Status in “gnome-control-center” package in Ubuntu:
>  New
>
> Bug description:
>  I'm trying to typeset some texts in ancient Mongolian and Phags-pa,
>  both of which have been included in Unicode. For Mongolian, I've
>  installed the ttf-manchufont package. For Phags-pa, I've used the font
>  available here: http://babelstone.blogspot.com/2006/12/phags-pa-
>  shaping-behaviour.html
>
>  Both languages use the Unicode Variation Selectors. In a nutshell, the
>  combination character + variation selector should change the
>  representation of the character to a different variant. See the table
>  of Standardised Unicode variants here:
>  http://www.unicode.org/Public/UNIDATA/StandardizedVariants.html
>
>  The expected behavior does not work in any of the packages I've tried
>  (gedit, gimp and OpenOffice). Instead, the variation selector remains
>  as it's own character and the preceding character is unmodified (see
>  attachment).
>
>  This is NOT an issue with the fonts because:
>  a. The fonts have the appropriate substitutions defined, as can be observed by editing the fonts in e.g. fontforge.
>  b. The expected behavior happens in WIndows 7 using these fonts.
>
>  This is an issue with whatever package handles Unicode glyph
>  substitution.
>
>  ProblemType: Bug
>  DistroRelease: Ubuntu 10.10
>  Package: xorg 1:7.5+6ubuntu3
>  ProcVersionSignature: Ubuntu 2.6.35-27.48-generic 2.6.35.11
>  Uname: Linux 2.6.35-27-generic x86_64
>  NonfreeKernelModules: fglrx
>  Architecture: amd64
>  Date: Wed Mar 16 10:30:20 2011
>  InstallationMedia: Ubuntu 10.04.1 LTS "Lucid Lynx" - Release amd64 (20100816.1)
>  MachineType: MICRO-STAR INTERNATIONAL CO.,LTD MS-7576
>  ProcCmdLine: BOOT_IMAGE=/boot/vmlinuz-2.6.35-27-generic root=UUID=136a2918-79f6-4777-a8b5-b9d86515a24b ro quiet splash
>  ProcEnviron:
>   LANGUAGE=en_US:en
>   LANG=en_US.UTF-8
>   SHELL=/bin/bash
>  SourcePackage: xorg
>  Symptom: display
>  dmi.bios.date: 12/17/2009
>  dmi.bios.vendor: American Megatrends Inc.
>  dmi.bios.version: V4.0
>  dmi.board.asset.tag: To Be Filled By O.E.M.
>  dmi.board.name: 790GX-G65 (MS-7576)
>  dmi.board.vendor: MICRO-STAR INTERNATIONAL CO.,LTD
>  dmi.board.version: 1.0
>  dmi.chassis.asset.tag: To Be Filled By O.E.M.
>  dmi.chassis.type: 3
>  dmi.chassis.vendor: MICRO-STAR INTERNATIONAL CO.,LTD
>  dmi.chassis.version: 1.0
>  dmi.modali...

[Aleksandr (aleksandr-andreev)]

> Aleksandr, try running gnome-keyboard-properties from the command line

This pulls up the Keyboard configuration window. And then?

FWIW, I think it's an issue with handling OpenType features like GSUB.
But I am not an expert at this stuff.

A

On Thu, Sep 29, 2011 at 8:48 PM, Rolf Leggewie
<email address hidden> wrote:
> Aleksandr, try running gnome-keyboard-properties from the command line
> and see if you can get the right configuration from there.  I've got a
> hunch that it's either a configuration problem or bug in that program.
>
> Assigning to gnome-control-center for further triage.
>
> ** Package changed: ubuntu => gnome-control-center (Ubuntu)
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/736153
>
> Title:
>  Unicode variation selectors not supported
>
> Status in LibreOffice:
>  New
> Status in “gnome-control-center” package in Ubuntu:
>  New
>
> Bug description:
>  I'm trying to typeset some texts in ancient Mongolian and Phags-pa,
>  both of which have been included in Unicode. For Mongolian, I've
>  installed the ttf-manchufont package. For Phags-pa, I've used the font
>  available here: http://babelstone.blogspot.com/2006/12/phags-pa-
>  shaping-behaviour.html
>
>  Both languages use the Unicode Variation Selectors. In a nutshell, the
>  combination character + variation selector should change the
>  representation of the character to a different variant. See the table
>  of Standardised Unicode variants here:
>  http://www.unicode.org/Public/UNIDATA/StandardizedVariants.html
>
>  The expected behavior does not work in any of the packages I've tried
>  (gedit, gimp and OpenOffice). Instead, the variation selector remains
>  as it's own character and the preceding character is unmodified (see
>  attachment).
>
>  This is NOT an issue with the fonts because:
>  a. The fonts have the appropriate substitutions defined, as can be observed by editing the fonts in e.g. fontforge.
>  b. The expected behavior happens in WIndows 7 using these fonts.
>
>  This is an issue with whatever package handles Unicode glyph
>  substitution.
>
>  ProblemType: Bug
>  DistroRelease: Ubuntu 10.10
>  Package: xorg 1:7.5+6ubuntu3
>  ProcVersionSignature: Ubuntu 2.6.35-27.48-generic 2.6.35.11
>  Uname: Linux 2.6.35-27-generic x86_64
>  NonfreeKernelModules: fglrx
>  Architecture: amd64
>  Date: Wed Mar 16 10:30:20 2011
>  InstallationMedia: Ubuntu 10.04.1 LTS "Lucid Lynx" - Release amd64 (20100816.1)
>  MachineType: MICRO-STAR INTERNATIONAL CO.,LTD MS-7576
>  ProcCmdLine: BOOT_IMAGE=/boot/vmlinuz-2.6.35-27-generic root=UUID=136a2918-79f6-4777-a8b5-b9d86515a24b ro quiet splash
>  ProcEnviron:
>   LANGUAGE=en_US:en
>   LANG=en_US.UTF-8
>   SHELL=/bin/bash
>  SourcePackage: xorg
>  Symptom: display
>  dmi.bios.date: 12/17/2009
>  dmi.bios.vendor: American Megatrends Inc.
>  dmi.bios.version: V4.0
>  dmi.board.asset.tag: To Be Filled By O.E.M.
>  dmi.board.name: 790GX-G65 (MS-7576)
>  dmi.board.vendor: MICRO-STAR INTERNATIONAL CO.,LTD
>  dmi.board.version: 1.0
>  dmi.chassis.asset.tag: To Be Filled By O.E.M.
>  dmi.chassis.type: 3
>  dmi.chassis.vendor: MICRO-STAR INTERNA...

[Arne Goetje (arnegoetje)]

This is a rendering issue, not an input issue. My guess is that pango/QT/LibreOffice are of fault here. Freetype should be able to support variation selectors, but this needs to be confirmed.
Pango is used for GTK+ applications, Libre Office comes with its own rendering engine.
Let me do some research here, I'll post the results.

[Arne Goetje (arnegoetje)]

This is a rendering issue, not an input issue. My guess is that pango/QT/LibreOffice are of fault here. Freetype should be able to support variation selectors, but this needs to be confirmed.
Pango is used for GTK+ applications, LibreOffice comes with its own rendering engine.
Let me do some research here, I'll post the results.

[Arne Goetje (arnegoetje)]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

This is a rendering issue, not an input issue. My guess is that pango /
QT / LibreOffice are of fault here. Freetype should be able to support
variation selectors, but this needs to be confirmed.
Pango is used for GTK+ applications, LibreOffice comes with its own
rendering engine.
Let me do some research here, I'll post the results.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQIcBAEBCAAGBQJOhRfzAAoJENiwmJtstTzsrlUP/ibQfwtfDnCvn5w01599ii7z
kkdtJmO5G3D3hY0em9nLbiTxJS/+gle8lsmZrpTuDiKAEuLm5KkJ6K7+ywc5yk6x
aOXapr9NGtYp11j4sr0xOhmaIxe7Xt7ruhMSxToRMO3BBp0pFlHqcioA6w0xUjxz
MS7zWwgcjh2UAfMN/TlWEO0sykaDXoBfsE16nCOqboTS8Pw65BjubtmFB7UzkFML
yoS8beyVvh9SOTsdFBlXuhQ9tX3JQWCkL0lQvS56jFu7nD1pKAdKHGwLJsfGrlLH
M+Q/S1QCSPIyt8ulneLZw9pwfHaozrQb5IP24v9PgmOOPdpVnjhDJfbdFSera88x
3Zh796DVLy4sqKMP5CIr0BVhIrvcl05m0oQpdbZw/oW4w+3oz+JRemHycnj+S+PN
uPjbplpbZ54HnyT/XRxRRZk+V67G47hLOQglGdURoDnUMsB+8IF/TyJa+LdyYhIZ
6gZ1Jp0x0ToPJZq647fv5ujzWjOceR8rmya4VVmISBl+cbKGeHfk5klZe2IIYnz4
RSFUZ70eDAI66tUWcAErPrvBfUf7oVmUAyDx/S+HSYPp+98GmjRvsFivK2dNBk9M
zHjfD8cNtjpsayXOx6PoInkSaVPmNWHkaQy+FF4Ky9RmyH6oHt/ObYH4mfBEFsWr
fh3DvTcwjiwwYV6eN+er
=0WfC
-----END PGP SIGNATURE-----

[Arne Goetje (arnegoetje)]

This is a rendering issue, not an input issue. My guess is that pango / QT / LibreOffice are of fault here. Freetype should be able to support variation selectors, but this needs to be confirmed.
Pango is used for GTK+ applications, LibreOffice comes with its own rendering engine.
Let me do some research here, I'll post the results.

[Steve Langasek (vorlon)]

> This is an issue with whatever package handles Unicode glyph
> substitution.

That package is pango, not freetype.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.