Ligatures problem in Arabic keyboard layout

[Me!]

Hi
There is in the Arabic keyboard a One key for two chr ; and that is normal

But the problem is this two chr typed as one only !!. so can not connect it with another chr.

This chr is "ﻻ" , "ﻵ" , "ﻷ" , "ﻹ"

for exmple :
"ﻻ" = "ل" + "ا" = "لا"

there is a diffrence between "ﻻ" & "لا" in editing

we must do it as two chr

thank you
شكراً

[Freedesktop (freedesktop)]

The analysis is completely correct. The presentation forms should die. A
single keystroke may generate the entire sequence for the ligature. I don't
remember how this is done with Xkb though.

[Daniel Stone (daniels)]

it's not currently possible to generate multiple keysyms with one key.

[MSameer (msameer)]

What does it take to enable multiple keysyms with 1 key ?

[Daniel Stone (daniels)]

see bug #4575. it would require a change to xkbcomp, the keymap compiler, to
deal with multiple keysyms; a protocol change for when you send the map over the
wire; removing xkm; dealing with multiple keysyms in the server.

it's non-trivial.

[Jg-freedesktop (jg-freedesktop)]

(In reply to comment #4)
> see bug #4575. it would require a change to xkbcomp, the keymap compiler, to
> deal with multiple keysyms; a protocol change for when you send the map over the
> wire; removing xkm; dealing with multiple keysyms in the server.
>
> it's non-trivial.

Do you have a guess as to the size of the project? man-days, weeks, months?
                                  Thanks,
                                           - Jim

[Daniel Stone (daniels)]

(In reply to comment #5)
> (In reply to comment #4)
> > see bug #4575. it would require a change to xkbcomp, the keymap compiler, to
> > deal with multiple keysyms; a protocol change for when you send the map over the
> > wire; removing xkm; dealing with multiple keysyms in the server.
> >
> > it's non-trivial.
>
> Do you have a guess as to the size of the project? man-days, weeks, months?

working full time, I'd guess a month or two? who knows.

[Freedesktop (freedesktop)]

What if we just remove those four from the keyboard? They are not really needed
afterall and are just leftovers from the typewriter era. Just remove them and
use the precious keyboard real estate for more important keys. What about
forming a working group and devising a better keyboard layout with several
per-territory layouts all merged in.? We did that for Persian and we are
really happy about the outcome.

[youss44 (chahibi-deactivatedaccount)]

(In reply to comment #7)
> What if we just remove those four from the keyboard? They are not really
needed
> afterall and are just leftovers from the typewriter era. Just remove them
and
> use the precious keyboard real estate for more important keys. What about
> forming a working group and devising a better keyboard layout with several
> per-territory layouts all merged in.? We did that for Persian and we are
> really happy about the outcome.

I agree. I really the Persian keyboard layout for the broad range of Unicode
Bidi control characters and the various keys it contains. The Arabic keyboard
layout should be improved again (New bug report?) based on the Persian layout.
I don't think those keys are even used, may be in the past when ligatures were
hard to input in typewriter. At least, we can make a new variant
named "typewriter" for Arabic along the other variants (Azerty, Querty, Indic
digits, Arabic digits).

[Behnam Esfahbod "ZWNJ" (behnam)]

Youssef, I can help on the Arabic keyboard layout. Just let's start it on the
wiki, our group mail, when it finished, you can open a bug and ask to commit it.

[Jg-freedesktop (jg-freedesktop)]

Very interesting suggestion to solve this by keyboard layout: we have the luxury
of building our own keyboards; we are in the middle of layout right now.

This won't help other existing keyboards of course.

Behnam, IIRC we plan to see you here at OLPC when the Thai delegation visits us
very soon. In the meanwhile, I expect we can get interested people proposed
keyboard layouts for review.

[Freedesktop (freedesktop)]

(In reply to comment #10)
> Very interesting suggestion to solve this by keyboard layout: we have the luxury
> of building our own keyboards; we are in the middle of layout right now.
>
> This won't help other existing keyboards of course.

Right. I didn't have that in mind. For Persian, most people don't use a
labeled keyboard anyway, so the layout is not hardcoded by the hardware.

> Behnam, IIRC we plan to see you here at OLPC when the Thai delegation visits us
> very soon. In the meanwhile, I expect we can get interested people proposed
> keyboard layouts for review.

Err... I'm "Behdad", going to be in Boston Oct 7th..22th. "Behnam" is my
brother in Iran :-).

Behnam, Youssef, can you two send out a message to the Arabeyes list to gather a
list of characters, then review current layouts and come up with a proposed one.
 CC me in all communications and I'll try to help time permitting. Thanks.

[Me! (news-swwwk)]

Hi
There is in the Arabic keyboard a One key for tow chr ; and that is normal

But the problem is this tow chr typed as one only !!. so can not connect it with another chr.

This chr is "ﻻ" , "ﻵ" , "ﻷ" , "ﻹ"

for exmple :
"ﻻ" = "ل" + "ا" = "لا"

there is adefrent between "ﻻ" & "لا" in editing

we must do it as tow chr

thank you
شكراً

[towsonu2003 (towsonu2003)]

thanks for your bug report. which ubuntu version are you using?

[Zied ALAYA (zied.alaya)]

in your service,
I m using Ubuntu 6.06 LTS which is always up to date

2006/9/30, towsonu2003 <email address hidden>:
>
> thanks for your bug report. which ubuntu version are you using?
>
> ** Changed in: Ubuntu
> Sourcepackagename: None => xkeyboard-config
>
> ** Changed in: xkeyboard-config (Ubuntu)
> Status: Unconfirmed => Needs Info
>
> --
> The Arabic Write error !
> https://launchpad.net/bugs/62587
>

[Me! (news-swwwk)]

Dapper

Thank you

[Isam Bayazidi (isam)]

This is a known issue in Xorg/XFree86 .. It is indeed a major bug, but I think that the problem is that it is not possible to map a keyboard key to 2 characters.. last I checked (2 years ago) it was not possible to do so.. Now the Big problem is, the keyaboard definition of Arabic in X (KDE/Gnome,etc) map the LAM_ALEF glyph (presentation form) to the key.. which is a major mistake because it breaks iconv and recode conversion of text, as the presentation form should NEVER be stored in a file, and having it mapped in the keybaord layout break that.

[Munzir Taha (منذر طه) (munzirtaha)]

+1
Well said Isam.

[Zied ALAYA (zied.alaya)]

Thank you very mach for the explanation
i hope this problem will be solved as soon as possible

2006/9/30, Isam Bayazidi <email address hidden>:
>
> This is a known issue in Xorg/XFree86 .. It is indeed a major bug, but I
> think that the problem is that it is not possible to map a keyboard key
> to 2 characters.. last I checked (2 years ago) it was not possible to do
> so.. Now the Big problem is, the keyaboard definition of Arabic in X
> (KDE/Gnome,etc) map the LAM_ALEF glyph (presentation form) to the key..
> which is a major mistake because it breaks iconv and recode conversion
> of text, as the presentation form should NEVER be stored in a file, and
> having it mapped in the keybaord layout break that.
>
> --
> The Arabic Write error !
> https://launchpad.net/bugs/62587
>

[sam (sa418)]

can u please not send me any emails?? how can I do that? why am I
receiving these emails now??? how do I stop them??

thanks

On 22 Sep 2006, at 14:28, ALAYA wrote:

> Thank you very mach for the explanation
> i hope this problem will be solved as soon as possible
>
> 2006/9/30, Isam Bayazidi <email address hidden>:
>>
>> This is a known issue in Xorg/XFree86 .. It is indeed a major bug,
>> but I
>> think that the problem is that it is not possible to map a
>> keyboard key
>> to 2 characters.. last I checked (2 years ago) it was not possible
>> to do
>> so.. Now the Big problem is, the keyaboard definition of Arabic in X
>> (KDE/Gnome,etc) map the LAM_ALEF glyph (presentation form) to the
>> key..
>> which is a major mistake because it breaks iconv and recode
>> conversion
>> of text, as the presentation form should NEVER be stored in a
>> file, and
>> having it mapped in the keybaord layout break that.
>>
>> --
>> The Arabic Write error !
>> https://launchpad.net/bugs/62587
>>
>
> --
> xkeyboard Arabic problem
> https://launchpad.net/bugs/62587

[towsonu2003 (towsonu2003)]

sam wrote:
> can u please not send me any emails?? how can I do that? why am I
> receiving these emails now??? how do I stop them??

I checked around, and it seems I accidentally subscribed the Arabic
Translators Team, of which you are a member, to this bug, while I was
looking to find a localization team [which, I couldn't find].

I couldn't find a way to unsubscribe the team, but the team leader /
maintainer may have an idea on how to do that.

I sincerely apologize for the mistake (though it seems that it attracted
some love ;) and attention to the bug in question).

[Me! (news-swwwk)]

I hope to fixing this problem as soon as possible

Thanks all

[sam (sa418)]

REMOVE ME FROM THE LIST PLLLLLLEASE!!!!!!

On 22 Oct 2006, at 14:28, Me! wrote:

> I hope to fixing this problem as soon as possible
>
> Thanks all
>
> --
> xkeyboard Arabic problem
> https://launchpad.net/bugs/62587

[Wearabnet (wearabnet)]

(In reply to comment #11)
> (In reply to comment #10)
> > Very interesting suggestion to solve this by keyboard layout: we have the luxury
> > of building our own keyboards; we are in the middle of layout right now.
> >
> > This won't help other existing keyboards of course.
>
> Right. I didn't have that in mind. For Persian, most people don't use a
> labeled keyboard anyway, so the layout is not hardcoded by the hardware.
>
>
> > Behnam, IIRC we plan to see you here at OLPC when the Thai delegation visits us
> > very soon. In the meanwhile, I expect we can get interested people proposed
> > keyboard layouts for review.
>
> Err... I'm "Behdad", going to be in Boston Oct 7th..22th. "Behnam" is my
> brother in Iran :-).
>
> Behnam, Youssef, can you two send out a message to the Arabeyes list to gather a
> list of characters, then review current layouts and come up with a proposed one.
> CC me in all communications and I'll try to help time permitting. Thanks.

Hello,

Is this the samae bug?
https://bugs.freedesktop.org/show_bug.cgi?id=9100

[youss44 (chahibi-deactivatedaccount)]

(In reply to comment #11)
> (In reply to comment #10)
> > Very interesting suggestion to solve this by keyboard layout: we have the luxury
> > of building our own keyboards; we are in the middle of layout right now.
> >
> > This won't help other existing keyboards of course.
>
> Right. I didn't have that in mind. For Persian, most people don't use a
> labeled keyboard anyway, so the layout is not hardcoded by the hardware.
>
>
> > Behnam, IIRC we plan to see you here at OLPC when the Thai delegation visits us
> > very soon. In the meanwhile, I expect we can get interested people proposed
> > keyboard layouts for review.
>
> Err... I'm "Behdad", going to be in Boston Oct 7th..22th. "Behnam" is my
> brother in Iran :-).
>
> Behnam, Youssef, can you two send out a message to the Arabeyes list to gather a
> list of characters, then review current layouts and come up with a proposed one.
> CC me in all communications and I'll try to help time permitting. Thanks.

Sorry, Behdad for the delay. I don't think suggesting a new layout is a solution. Many typers are used to the current one and those ligatures are printed in all available keyboards. Apparently, ligatures on the keyboard inherit from old typewriters that don't support shaping ل + ا to a ligature.
I think the abilility to type a combination of characters should be enabled since it is needed by many languages other than Arabic. The most serious case is the Kurdish Arabic script layout: To type the phonem /e/ they need to type Ha' (ه) + ZWNJ which is not a solution for a so frequently typed letter. See http://lists.arabeyes.org/archives/general/2006/August/msg00010.html .

[Freedesktop (freedesktop)]

(In reply to comment #13)

> Sorry, Behdad for the delay. I don't think suggesting a new layout is a
> solution. Many typers are used to the current one and those ligatures are
> printed in all available keyboards. Apparently, ligatures on the keyboard
> inherit from old typewriters that don't support shaping ل + ا to a ligature.
> I think the abilility to type a combination of characters should be enabled
> since it is needed by many languages other than Arabic. The most serious case
> is the Kurdish Arabic script layout: To type the phonem /e/ they need to type
> Ha' (ه) + ZWNJ which is not a solution for a so frequently typed letter. See
> http://lists.arabeyes.org/archives/general/2006/August/msg00010.html .

I agree that the feature is generally useful in some non-major languages (Kurdish and Uighar come to mind). What I don't agree is that we should be limited by the limitations of old typewriters these days. You've got to change *some* day. We've done that in Persian and are really happy so far. Apple has adopted our new layout. Linux has had it for long. And there are multiple drivers for Windows. No hardware still has it printed, but that will take some time. The positive side is that the new layout is a lot better and everyone who has tried it agrees.

And last but not least, the olpc machine doesn't have to use the legacy layout.

[Mahmoud Kassem (mmkassem)]

(In reply to comment #14)
This is about adding support for a layout and not limiting new layouts. Adding this feature does not affect the Persian layout. Its all about removing limitations here.

[Freedesktop (freedesktop)]

Fine. Still, for the OLPC, I recommend a new, simplified, layout.

[sam (sa418)]

how come I am on this list again?? I thought I asked to be removed?

On 23 Feb 2007, at 08:09, Mahmoud Kassem wrote:

> ** Bug watch added: Freedesktop Bugzilla #8195
> https://bugs.freedesktop.org/show_bug.cgi?id=8195
>
> ** Also affects: xorg-server (upstream) via
> https://bugs.freedesktop.org/show_bug.cgi?id=8195
> Importance: Unknown
> Status: Unknown
>
> --
> xkeyboard Arabic problem
> https://launchpad.net/bugs/62587

[Mahmoud Kassem (mmkassem)]

Anyone who does not want to be notified has to unsubscribe himself or herself from the the bug if listed and/or from Ubuntu Arabic Translators group and not by replying to this bug report.

[sam (sa418)]

I ALREADY DID! AND STILL RECEIVING THESE EMAILS!! STOPPED FOR A
WHILE AND NOW COMING BACK! WHAT CAN I DO??

On 23 Feb 2007, at 08:30, Mahmoud Kassem wrote:

> Anyone who does not want to be notified has to unsubscribe himself or
> herself from the the bug if listed and/or from Ubuntu Arabic
> Translators
> group and not by replying to this bug report.
>
> --
> Ligatures problem in Arabic keyboard layout
> https://launchpad.net/bugs/62587

[Mahmoud Kassem (mmkassem)]

Sure, I agree about that. As much as I know it is already taken care of in the new OLPC Arabic keyboard: http://dev.laptop.org/ticket/420

(In reply to comment #16)
> Fine. Still, for the OLPC, I recommend a new, simplified, layout.
>

[Timo Aaltonen (tjaalton)]

sam: you are in "Ubuntu Arabic Translators" -team, leave the team and all is well.

[Sewar (sewar)]

Please unsubscribe "Ubuntu Arabic Translators" from this bug,
"Translators" is not mean "Developers".

On 2/23/07, Timo Aaltonen <email address hidden> wrote:
> sam: you are in "Ubuntu Arabic Translators" -team, leave the team and
> all is well.
>
> --
> Ligatures problem in Arabic keyboard layout
> https://launchpad.net/bugs/62587
>

[Daniel Stone (daniels)]

Sorry about the phenomenal bug spam, guys. Adding xorg-team@ to the QA contact so bugs don't get lost in future.

[dark-templer (hedayaty)]

There are many languages which may get use of having multiple charecters for a key. Most people change their keyboard layout to to current condition, if this feature is added many people may use it.

By the way, having this feature is not too optimistic. Lack of this feature and complexity of X keyboard section is realy a bug. I think, there is a need to clean up this part of X server and get rid of all its bugs.

[Islam Alshaikh إسلام الشيخ (islam-alshaikh)]

Basicly make the one chr tow as:
1 replace the chr "ﻻ" in b -> "ل"+"ا" in g+h
2 replace the chr "ﻵ" in B -> "ل"+"آ" in g+N
3 replace the chr "ﻷ" in G -> "ل"+"أ" in g+H
4 replace the chr "ﻹ" in T -> "ل"+"إ" in g+Y

It is a small bug , But a hug problem for the typers!
Thank you

See also
https://bugs.launchpad.net/xorg-server/+bug/63310

[Khaled Hosny (khaledhosny)]

This is an X bug, it is already reported https://bugs.freedesktop.org/show_bug.cgi?id=8195

[Saleh Odeh (odeh)]

why did you send me this email?

On 01/01/2008, Khaled Hosny <email address hidden> wrote:
>
> This is an X bug, it is already reported
> https://bugs.freedesktop.org/show_bug.cgi?id=8195
>
> --
> Ligatures problem in Arabic keyboard layout
> https://bugs.launchpad.net/bugs/62587
> You received this bug notification because you are a member of Ubuntu
> Arabic Translators, which is a direct subscriber.
>

--
Saleh M. Odeh

[Khaled Hosny (khaledhosny)]

We managed to get around this limitation by using Composite keys that map the presentation forms lam-alef ligatures to their "decomposed" codes. See #17228.

[Simos Xenitellis  (simosx)]

Arabic Lam-Alef ligatures have been added to the Compose file.
See at the end of the page at

http://gitweb.freedesktop.org/?p=xorg/lib/libX11.git;a=blob;f=nls/en_US.UTF-8/Compose.pre

What's missing now is to update GTK+ with these compose sequences, and make sure the Arabic keyboard layout is updated for the compose sequences.

[Simos Xenitellis  (simosx)]

(In reply to comment #20)
> We managed to get around this limitation by using Composite keys that map the
> presentation forms lam-alef ligatures to their "decomposed" codes. See #17228.
>

I believe you refer to the compose sequences at the end of the file,
http://cgit.freedesktop.org/xorg/lib/libX11/tree/nls/en_US.UTF-8/Compose.pre
(warning, big HTML file), which are

6290 XCOMM
6291 XCOMM Arabic Lam-Alef ligatures
6292 XCOMM
6293
6294 <UFEFB> : "لا" # ARABIC LIGATURE LAM WITH ALEF
6295 <UFEF7> : "لأ" # ARABIC LIGATURE LAM WITH ALEF WITH HAMZA ABOVE
6296 <UFEF9> : "لإ" # ARABIC LIGATURE LAM WITH ALEF WITH HAMZA BELOW
6297 <UFEF5> : "لآ" # ARABIC LIGATURE LAM WITH ALEF WITH MADDA ABOVE

I am interested in adding these sequences to GTK+ IM.
Could you please tell me how I can test that these work?

Which Arabic layout/variant shall I activate,
what keys I should press and
what I am expected to get?

[James H. Cloos Jr. (cloos-jhcloos)]

> 6294 <UFEFB> : "لا" # ARABIC LIGATURE LAM WITH ALEF
> 6295 <UFEF7> : "لأ" # ARABIC LIGATURE LAM WITH ALEF WITH HAMZA ABOVE
> 6296 <UFEF9> : "لإ" # ARABIC LIGATURE LAM WITH ALEF WITH HAMZA BELOW
> 6297 <UFEF5> : "لآ" # ARABIC LIGATURE LAM WITH ALEF WITH MADDA ABOVE

First, in x11proto/keysymdef.h (typically installed as
/usr/include/X11/keysymdef.h) you will find the formula for converting
between a UXXXX keysym and its integer value: 0x01000000 + the hex value
from the sym.

As such, UFEFB is 0x0100FEFB, etc.

> Which Arabic layout/variant shall I activate,

A grep through the xkeyboard-config repo shows that nothing there uses them.

However, a grep though the output of »git log -p -M -C symbols/ara«
shows that prior to commit b772edc289a844539ee096b2bb2a37bc74e1ef06
they were at:

 key <AD05> { [ Arabic_feh, 0x100fef9 ] };
 key <AC05> { [ Arabic_lam, 0x100fef7 ] };
 key <AB05> { [ 0x100fefb, 0x100fef5 ] };

> what keys I should press and

if your keyboard has US labels and you select the ara layout of an
old-enough version of xkeyboard-config, then:

 UFEF9 would be on SHIFT T
 UFEF7 would be on SHIFT G
 UFEFB would be on B
 UFEF5 would be on SHIFT B

If you do not have US labels (printed on the physical keyboard),
you can see from the key names above that each is the fifth alphabetic
key from the left in the fourth through second rows. (The spacebar row
is of course the first row.)

> what I am expected to get?

When pressing a key which is mapped to eg UFEFB you should get a string of the
two characters »ل« U+0644 ARABIC LETTER LAM and »ا« U+0627 ARABIC LETTER ALEF.

The other three also give two character strings, but will U+0623, U+0625
or U+0622 instead of U+0627.

[Simos Xenitellis  (simosx)]

Thanks James.

I put together an initial patch, available at
http://bugzilla.gnome.org/show_bug.cgi?id=537457

[Khaled Hosny (khaledhosny)]

(In reply to comment #21)
> Which Arabic layout/variant shall I activate,
> what keys I should press and
> what I am expected to get?
>

You'll need to apply the second patch in #13894 to restore the removed keys, pressing the keys corresponding to b, shift+b, shift+g and shift+t with the Arabic layout should give you لا and لآ and لأ and لإ respectively as two characters not a single character (presentation form).

[Simos Xenitellis  (simosx)]

(In reply to comment #24)
> (In reply to comment #21)
> > Which Arabic layout/variant shall I activate,
> > what keys I should press and
> > what I am expected to get?
> >
>
> You'll need to apply the second patch in #13894 to restore the removed keys,
> pressing the keys corresponding to b, shift+b, shift+g and shift+t with the
> Arabic layout should give you لا and لآ and لأ and لإ respectively as
> two characters not a single character (presentation form).
>

Thanks, I managed to figure out my way with James's explanation.

The status we are now at, is this

1. GTK+ applications (Openoffice, Firefox, all of GNOME, Gimp, inkscape, etc) require to have their own copy of the compose sequences so that it works. Otherwise, the default installation will not be able to work for these compose sequences.

2. I produced a patch for this at
http://bugzilla.gnome.org/show_bug.cgi?id=537457
which covers Khmer and Arabic, the only two scripts with such compose sequences.
Of course, I trid it out and it works fine.

3. If you want to go for it and use these compose sequences, the situation is is like this: the next GTK+ stable release comes in a week or so, so it's tough to get this patch included now. It looks feasible to get the patch in six months time, when a new stable release comes out.

4. I am not sure what input method you use for the OLPC. If you use GTK+ IM, I am happy to work on a suitable patch so that it works for you, and you can use now.

[Bryce Harrington (bryce)]

If I understand correctly, this has been resolved upstream. If I'm wrong, please clarify and reopen the bug.

[Islam Alshaikh إسلام الشيخ (islam-alshaikh)]

Not fixed yet

They solve it by canceling the keys !!

[Simos Xenitellis  (simosx)]

The situation is that this works if you make some changes in your system.

Specifically,
1. Type
sudo gedit /etc/environment
2. Add a line
GTK_IM_MODULE=xim
3. Save and restart your computer.

Now you should be able to type proper Arabic for those characters.

The proper solution would be not to have to perform the above process. :-(

[Simos Xenitellis  (simosx)]

I have linked upstream to https://bugzilla.gnome.org/show_bug.cgi?id=537457

[Nathan Wells (sungkhum)]

This is also a problem for the Khmer language (Cambodia) but Simos's suggestion fixed the problem: Specifically,
1. Type
sudo gedit /etc/environment
2. Add a line
GTK_IM_MODULE=xim
3. Save and restart your computer.
And making sure the following exists in:
sudo gedit /usr/share/X11/locale/en_US.UTF-8/Compose

#
# Khmer digraphs
#
<U17ff> : "ាំ"
<U17fe> : "ោះ"
<U17fd> : "េះ"
<U17fc> : "ុំ"
<U17fb> : "ុះ"

I hope this will be resolved so no manual changes have to be made.

[Simos Xenitellis  (simosx)]

The proper solution is to work on the patch at
https://bugzilla.gnome.org/show_bug.cgi?id=537457
Once this is done, both Arabic and Khmer will work out of the box.

I am not motivated to work on this at this time ;-'(,
so what some people can do is use the nasty workaround that enables 'XIM'.

To use 'XIM' is a workaround, it we should not become part of Ubuntu.
The proper solution is to work on https://bugzilla.gnome.org/show_bug.cgi?id=537457
and I think there is time to make it for gtk+3.2.

[مصعب الزعبي (moceap)]

2006 ------------> 2013

Any News ????