Need way to insert arbitrary unicode characters in Kubuntu

[dotancohen]

KDE, Qt, and Xorg are in disagreement about who should implement ISO 14755, which would facilitate the input of arbitrary unicode characters. This is the original KDE bug:
https://bugs.kde.org/show_bug.cgi?id=103788

It was pushed upstream to Qt:
http://bugreports.qt.nokia.com/browse/QTBUG-8

From there it was pushed further upstream to Xorg:
https://bugs.freedesktop.org/show_bug.cgi?id=26747

Xorg pushes the bug downstream, back to either Qt or KDE to implement, just as Gnome implemented the fix themselves. An Xorg developer who does not want to be named says that if KDE won't implement it then my distro should. So here I file that request.

Note that in KDE 3 one could use the Kcharselect applet to enter arbitrary unicode characters, however that has been disabled in KDE 4:
https://bugs.kde.org/show_bug.cgi?id=190776

[Freedesktop-org (freedesktop-org)]

Here is the downstream Qt bug:
http://bugreports.qt.nokia.com/browse/QTBUG-8

Here is the downstream KDE bug:
https://bugs.kde.org/show_bug.cgi?id=103788

[Julien Cristau (jcristau)]

Moving to xlib, this is an input method issue.

[dotancohen (dotancohen)]

KDE, Qt, and Xorg are in disagreement about who should implement ISO 14755, which would facilitate the input of arbitrary unicode characters. This is the original KDE bug:
https://bugs.kde.org/show_bug.cgi?id=103788

It was pushed upstream to Qt:
http://bugreports.qt.nokia.com/browse/QTBUG-8

From there it was pushed further upstream to Xorg:
https://bugs.freedesktop.org/show_bug.cgi?id=26747

Xorg pushes the bug downstream, back to either Qt or KDE to implement, just as Gnome implemented the fix themselves. An Xorg developer who does not want to be named says that if KDE won't implement it then my distro should. So here I file that request.

Note that in KDE 3 one could use the Kcharselect applet to enter arbitrary unicode characters, however that has been disabled in KDE 4:
https://bugs.kde.org/show_bug.cgi?id=190776

[dotancohen (dotancohen)]

Using gucharmap is not a viable workaround as one cannot bookmark common characters, and the characters are furthermore accessed in a cumbersome way with copying then pasting, not to mention that the user has to switch applications and even have Gnome dependencies.

[Jonathan Thomas (echidnaman)]

I don't really see anything in the Xorg report pushing this back down to the distro. They seem to acknowledge that it is an Xlib issue...

[dotancohen (dotancohen)]

> I don't really see anything in the Xorg report
> pushing this back down to the distro.

In private mail, an Xorg developer let me know that it won't be fixed in Xorg. Just as Gnome managed to implement a fix, so must either Qt or KDE. He suggested that I file the issue at my distro if KDE / Qt won't take it. So here I am :)

[Bryce Harrington (bryce)]

Hi dotancohen,

Please attach the output of `lspci -vvnn` and `dmesg`, and attach your /var/log/Xorg.0.log (and maybe Xorg.0.log.old) file from after reproducing this issue. If you're using a custom /etc/X11/xorg.conf please attach that as well.

[This is an automated message. Apologies if it has reached you inappropriately; please just reply to this message indicating so.]

[dotancohen (dotancohen)]

Thank you Bryce, this is a feature request, not a bug. No need for the bot!

[Daniel Stone (daniels)]

er, any 'native x solution' would be in the standard xlib input method, which both qt and gtk bypass completely ...

[Freedesktop-org (freedesktop-org)]

Daniel, please mention that on the downstream Qt and KDE bugs. Thanks!
http://bugreports.qt.nokia.com/browse/QTBUG-8
https://bugs.kde.org/show_bug.cgi?id=103788

[Freedesktop-org (freedesktop-org)]

The Qt dev says that Qt does use XIM by default, and that this is really something that should be implemented by the input method. The bug is thus remaining closed at Qt, and closed at KDE.

Please fix this.

[Bryce Harrington (bryce)]

While it sounds like a legitimate issue, it seems to be kubuntu specific so not something we're likely to fix as part of the Ubuntu xorg work. Perhaps you should speak to the kubuntu folks if it needs implemented at the distro level.

Beyond that, patches would be entertained.

[dotancohen (dotancohen)]

> While it sounds like a legitimate issue, it seems to be kubuntu specific so
> not something we're likely to fix as part of the Ubuntu xorg work. Perhaps
> you should speak to the kubuntu folks if it needs implemented at the
> distro level.
>

That is quite the reason for this bug report. Please change the "xorg (Ubuntu)" assignment to "xorg (Kubuntu)" as I do not seem to have permissions for that.

[Nick Coghlan (ncoghlan)]

I'd like to add my support for this issue. Trying to do unicode testing for Python 3 core development on Kubuntu is currently very annoying because it is so inconvenient to enter arbitrary unicode characters.

Canonical are fond of saying that Kubuntu is just as valued as Ubuntu, it would be nice if that extended to providing proper support for entry of arbitrary Unicode characters :)

[Barry Warsaw (barry)]

Scott K is the Kubuntu guru I know, so I've subscribed him.

[Paul Sladen (sladen)]

I've just been trying to document how to access various special codepoints in the Ubuntu Font Family, for example the Indian Rupee Sign ('₹') U+20B9. For Firefox and Gtk+-based software there can be entered with the sequence:

  Ctrl-Shift-u 2 0 b 9 <enter>

and in the process of looking for the equivalent KDE/Kubuntu method I ended up here.

[Navid-zamani (navid-zamani)]

Lol, it’s 2011, and X still can’t do this. *facepalm*
Even Windows 2000 could do this, over, you guessed it, 11 years ago.

[Shriramana Sharma (jamadagni)]

...*and* there is an ISO standard for this: ISO 14755: http://www.cl.cam.ac.uk/~mgk25/volatile/ISO-14755.pdf

[Alan Coopersmith (alan-coopersmith)]

Yep, it's 2011 and X still has many fewer developers than it would take
to implement every feature someone wants to see. Patches welcome if
this is important to you.

[Daniel Stone (daniels)]

On Thu, Jan 06, 2011 at 11:50:39PM -0800, <email address hidden> wrote:
> Lol, it’s 2011, and X still can’t do this. *facepalm*
> Even Windows 2000 could do this, over, you guessed it, 11 years ago.

Thanks so much for your valuable contribution to this bug report.

[Navid-zamani (navid-zamani)]

(In reply to comment #8)
> Yep, it's 2011 and X still has many fewer developers than it would take
> to implement every feature someone wants to see. Patches welcome if
> this is important to you.

(In reply to comment #9)
> Thanks so much for your valuable contribution to this bug report.

Guys, since I get X for free, I’m not expecting anything, and think it’s really nice of you to do this work. But I’m not talking about implementing some colorful clickables or display bling. I’m talking about one of the most basic features. So basic it has its own ISO standard. And also about 50 lines of code. Max.

What I meant to say is, that if it were my code, and yes I am a professional programmer for 20 years now, I would be ashamed of overlooking something like this. It’s a thing of pride. Maybe because I see it as more fundamental than you.

I would also love to submit a patch. If the thing weren’t written in that horrible horrible excuse of a programming language called C. I know lots of languages, including Haskell (my favorite and main languages), but I have no intention in torturing myself with C-like languages.
Maybe if the thing was written in Haskell or OCaml, there would be more free time…

Whatever. Maybe I’ll save 50€ and send them to you to implement it. 1€ a line should be enough, no? ;)

[Daniel Stone (daniels)]

On Fri, Jan 07, 2011 at 09:55:19AM -0800, <email address hidden> wrote:
> Guys, since I get X for free, I’m not expecting anything, and think it’s really
> nice of you to do this work. But I’m not talking about implementing some
> colorful clickables or display bling. I’m talking about one of the most basic
> features. So basic it has its own ISO standard. And also about 50 lines of
> code. Max.

I look forward to your 50-line patch then.

> What I meant to say is, that if it were my code, and yes I am a professional
> programmer for 20 years now, I would be ashamed of overlooking something like
> this. It’s a thing of pride. Maybe because I see it as more fundamental than
> you.

Well, either we are rubbish programmers with no pride, or are simply too
busy with the other 2.532 million (I just checked) lines of code that X
has.

> I would also love to submit a patch. If the thing weren’t written in that
> horrible horrible excuse of a programming language called C. I know lots of
> languages, including Haskell (my favorite and main languages), but I have no
> intention in torturing myself with C-like languages.

So what you're saying is that you're not going to help, but instead sit
here banging on about pride and how Windows 2000 did it and it's really
easy. Helpful!

(Oh, and Windows 2000 may have had Ctrl+Shift+[codepoint] for generating
 Unicode for ten years now, but people have been unconstructive assholes
 on the internet for at least 20 years now. So you're even further
 behind than we are.)

Sorry to everyone else Cc'ed ...

[Jeremy Sequoia (jeremyhu)]

Hey Navid, have you learned C yet? I think we're all still looking forward to
your 50line patch...

In all honesty, I don't expect this to be 50 lines, and I don't expect it to be
popular enough to demand attention over higher priority issues.

[Freedesktop-org (freedesktop-org)]

Daniel, Jeremy, please don't feed a possible troll. Just take his badly-worded comment as another ++ vote for the feature and be done with it. It would be a shame if this feature were ignored due to a single troll's heckling.

In fact, let's look at it another way. I am willing to learn C and fix this issue if a current X dev will mentor me. I'm serious. Of course it would take _more_ time to fix the bug this way, but then X gains another enthusiastic developer. I've done a simple C course in university and I know a bit of PHP and Java. Are there any X devs who would mentor me through this bug?

Thanks.

[Navid-zamani (navid-zamani)]

(In reply to comment #11)

> I look forward to your 50-line patch then.
As I said, I have no intention in learning a language as bad as C. I’d rather rewrite the whole thing in a modern language. And I’d like even more to have someone else do it, while I do some other contribution in return. How about that? :)

> Well, either we are rubbish programmers with no pride, or are simply too
> busy with the other 2.532 million (I just checked) lines of code that X
> has.
So you’re essentially suggesting that it’s just extremely bloated? ^^
But hey, there doesn’t have to be a “or”. Usually the latter comes from the former. ;)

> So what you're saying is that you're not going to help, but instead sit
> here banging on about pride and how Windows 2000 did it and it's really
> easy. Helpful!
No, (Windows * still sucked). I was saying:

> Maybe I’ll save 50€ and send them to you to implement it. 1€ a line
> should be enough, no? ;)

But I already see where this is going: Straw man arguments: First you deliberately misinterpret my statements, and then you call me an idiot because of them. Sorry mate, that won’t work. ^^

------------------------------------------------------------

(In reply to comment #12)

> Hey Navid, have you learned C yet?
Maybe the Navid in your delusional parallel world where I didn’t say I despised C did.
Why don’t you check?

> In all honesty, I don't expect this to be 50 lines, and I don't expect it to > be popular enough to demand attention over higher priority issues.

Well, if it doesn’t take 50 lines, something is deeply wrong with the code. Then again it’s C and it’s 2.532 million lines of it, so that kinda answers that. ^^

PROTIP: Quick sort in Haskell:

  qsort (x:xs) = qsort [ i | i <- xs, i < x ] ++ [x] ++ qsort [ i | i <- xs, i >= x ]
  qsort [] = []

50 lines? Hell yeah! ^^

------------------------------------------------------------

I have no interest in the typical Internet dicks calling everyone who isn’t fitting their mind a “troll”. So you can fail with your archaic C, bloat, weird priorities and distorted perception until the end of time.
Actually it’s a good thing, because the more you’re ignorant of the unfolding reality, the more those who don’t gain an edge over you. So, hey… in a way you’re still helping me. Thanks!
And good bye. ^^

[Shriramana Sharma (jamadagni)]

Navid, please cease and desist from making unhelpful comments. I myself would like to see this bug fixed soon, and I also do not have enough C-knowledge to contribute a patch, and I do occasionally post "hello can anybody please fix this bug" kind of comments to bug reports, but at least I try to stay polite and *request* something of the developers who obviously have a particular skill which I don't. It is my experience that more often than not, developers are quite polite and at least explain why the bug can't be fixed soon or something like that.

Now my polite request: can any of the kind developers fix this bug please? I'm sure with your skills it would be a simple job if someone just allocated a small amount of time to it?

After upgrading my Kubuntu system to Oneiric, the ibus stopped working (https://bugs.launchpad.net/ubuntu/+bug/854333) so I'm falling back to XIM which IMHO is much cleaner even if not context-sensitive. So I would really like to see this soon. Thanks! :)

[Freedesktop-org (freedesktop-org)]

I think that it is about time that we had this Navid character banned. I'll see who to contact about that.

I reiterate my pledge. I am willing to learn C and fix this
issue if a current X dev will mentor me. I'm serious. Of course it would take
_more_ time to fix the bug this way, but then X gains another enthusiastic
developer. I've done a simple C course in university and I know a bit of PHP
and Java. Are there any X devs who would mentor me through this bug?

Thanks.

[Jeremy Sequoia (jeremyhu)]

Dotan,

I'd love to help you learn C. My advise to you is to check out this course at
UC Berkeley designed to teach C to students who already know how to program in
another language:
http://inst.eecs.berkeley.edu/~selfpace/class/cs9c

Most of the materials are available online. You just need to buy or borrow the
book, and you don't have access to the SPC tutors and obviously don't get a
grade or course credit. I used to tutor the course back in the day, so I'd
love to help you out if that means we'd have another X11 contributor.

But let's take this offline for now... just send me an email when you want to
get started and need some help/questions.

[Jehan-7 (jehan-7)]

Hello everyone,

If none steps in, I am willing to implement such a feature. But first I would need more information before diving into any code, because I want it to be in the right place for the feature to matter.
In particular that would mean that it would be usable in most (if not all) X environments.

Daniel Stone wrote above:
« any 'native x solution' would be in the standard xlib input method, which
both qt and gtk bypass completely ... »

Is it true? If that is true, probably X is not the right place, is it? Also aren't the Xlib Input Method and the X Input Method (XIM) the same thing? Because others claim that KDE uses XIM, so it seems contradictory to me.
Or is there something I missed?
I am not especially a KDE or Gnome user, but I do use them from time to time, depending on circumstances. And in any case, generic and standard is better.

Secondly there is also the set of features which needs to be discussed. Because the code point input method is probably necessary but really user-unfriendly.

Unicode Code Point
===================

ISO-14755 proposes to enter Unicode code point (Gnome supports with ctrl-shift-u, and vim with ctrl-v-u). That's the most extensive solution (whole Unicode supported), but also the more annoying in real life (how many code point do you actually know out of ASCII?).

Function Key Symbols
====================

ISO-14755 again proposes to associate symbols to usual "generic" keyboard keys, like tab, insert, pause, etc. The full list is in ISO-9995-7: http://www.msg.gouv.qc.ca/documents/standards/ISO_9995-7.pdf
That's nice to input easily stuff like arrows, because mnemonics are pretty good. But this is a very limited set of characters.
If we look Unicode charts, they also plan these association (written under a character when there is one, for instance ⇱ is the "home" key):
http://www.unicode.org/charts/PDF/U2190.pdf
I don't know of any system actually supporting this.

Defined Character Mnemonics
===========================

RFC-1345 (http://tools.ietf.org/html/rfc1345) defines a list of mnemonics for a whole bunch of characters. They are good mnemonics and the range of character is big but still I will never remember them all and will always end up checking up the internet list at some point for a character I don't use often.
vim supports these with ctrl-k.

Keywords Input?
================

A kind of input I would really like to see (or to develop) is a word base input. Let's say I want to write a quarter note. I obviously don't remember its code point (2669). I don't know either the mnemonics 'Md' if that's the first time I input it (or it has been a long time since). I'd like to be able to write 'note' and be proposed a significant list of Unicode characters related to notes.
Similar to what anthy (Japanese input method) does when it proposes you Kanjis after you enter a "sound description" in romanji (which then becomes hiragana, and from there kanjis). Except that here we could enter a keyword (searched from the name of the characters probably).

I don't know if there is any RFC/ISO about such a feature, but I could not find one. The closer is the "screen selection entry method" (section 5.3 in ISO...

[James H. Cloos Jr. (cloos-jhcloos)]

> If we look Unicode charts, they also plan these association (written
> under a character when there is one, for instance ⇱ is the "home" key)
> I don't know of any system actually supporting this.

rxvt-unicode does, using ctrl-shift as the introducer.

It also supports C-S + code point entry and C-S + left click to get the
code point of the character under the mouse cursor.

[Ruslan (b7-10110111)]

There's a project (created by me) which tries to work around this limitation of X by providing a daemon-like app intercepting Ctrl+Shift+U and allowing to enter UTF-32 character codes:

https://gitorious.org/ucode/ucode/

[Libreoffice-org (libreoffice-org)]

Thank you Ruslan. I do stress to the developers that this is a workaround, not a solution, due to issues such as the user of "sticky keys".

[Ivo Anjo (knuckles)]

+1 thank you Ruslan.

I'm guessing that at this point we should be asking for this to be implemented on Wayland, as it seems that X will be deprecated soon enough and its input code is complex and strange enough (no screensaver when a pop-up menu is open, for instance) that this won't make it.

[Samuel Suther (s-suther)]

+ 10 for the implementation of this UNICODE-FEATURE.

At the moment I'm coding on a Keyboard-Firmware (ErgoDox). Problem is that I can send UNICODE-Chars easy and nice with the GNOME-Method : CTRL+Shift+<digits, but there is no way to send them to QT-Based Programms.

So keep calm and implement the Shortcut which is still implemented in Gnome (and please, please, please don't choose another one like CTRL+ META or such different Stuff...) Keep things easy and implement like it still works in Gnome.

BTW: Thanks a lot for all your work. I love KDE.

[Gitlab-migration (gitlab-migration)]

-- GitLab Migration Automatic Message --

This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity.

You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/xorg/lib/libx11/issues/9.