non ASCII chars in hashtags dont work

[benni]

hashtags with non-ascii-chars are not correct searchable. they link only to the part before the non-ascii-char. example: #verdächtig (means "suspect" in german) links to search of #verd.

[Colin Dean (colindean)]

I encountered this with the post "#HUHN #FASAN #GRÄUS #chicken #pheasant #grouse".

Seems like the regex used to handle the highlighting needs to be expanded to include all letters and numbers.

[Miguel Vieira (miguelsvieiracadastros)]

It also happens with tilde-accented letters: "õ", "ã".

[nanake (nanake)]

I encountered with Thai language hashtags #ภาษาไทย

[Omer Akram (om26er)]

Can any one please check if this is still an issue with gwibber 2.29.94?

[Tim 'avatar' Bartel (avataryt)]

Problem still exists on 2.31.1 - http://twitter.com/avatar/status/11820340965

[Omer Akram (om26er)]

Yes I can reproduce this too

[Alexander Karlstad (alexander.karlstad)]

Same problem here. Norwegian letters like æ, ø and å in hashtags doesn't work, yet it does on Twitters website.

[Omer Akram (om26er)]

Ake K, your patch does not work with the latest version things have changed a bit can you please make a new patch?

[Nigel Babu (nigelbabu)]

This patch does not apply to the lucid gwibber source. Please see if the patch can be redone for the current sources. I'm marking this 'patch-needswork', once you have another patch for review, feel free to change the tag to 'patch'

[Adam Pogany (adam-pogany)]

this one should do the trick against 2.30.0.1 (and afaik current revision too)

[Sigve Indregard (sigve-indregard)]

You should probably copy Twitter's own twitter-text library. Their regex is a bit more complicated, and in Ruby, an I am not proficient enough in Python to push a real patch:

LATIN_ACCENTS = [(0xc0..0xd6).to_a, (0xd8..0xf6).to_a, (0xf8..0xff).to_a].flatten.pack('U*').freeze
HASHTAG_CHARACTERS = /[a-z0-9_#{LATIN_ACCENTS}]/io

However, HASHTAG_CHARACTERS are only allowed from position 2 and on:

REGEXEN[:auto_link_hashtags] = /(^|[^0-9A-Z&\/]+)(#|＃)([0-9A-Z_]*[A-Z_]+#{HASHTAG_CHARACTERS}*)/io

[Omer Akram (om26er)]

the patch in comment #11 works fine. Ken could you please review and merge.

[Ken VanDine (ken-vandine)]

I tested the patch in comment #11, but it didn't seem to work using the original test case. I tried searching for #verdächtig and got no results.

[Frédéric Grosshans (fgrosshans)]

Why is this bug "incomplete" ? What information is missing ?

[Nils Dabrock (nils-nilsdabrock)]

The given patch fixes the bug. Works for me with revision 974.

[Xavier Guillot (valeryan-24)]

Problem still present in Gwibber 3.1.4.1 and also with french accents : é è à...

Only letters before accents are taken in the hashtag link.

[Bilal Shahid (s9iper1)]

"Thanks for your patch, unfortunately our busy developers haven't been
able to review your patch in a timely manor. The gwibber codebase has
seen significant change and it is likely this patch no longer applies.
Please review it again and if it is still applicable, update it to work
with the latest gwibber trunk. We will be doing a patch review day in
the next few weeks and would like to review your patch. Thanks again for
your contribution!"

[Launchpad Janitor (janitor)]

[Expired for gwibber (Ubuntu) because there has been no activity for 60 days.]

[Frédéric Grosshans (fgrosshans)]

The bug seems corrected (Gwibber 3.4.1 on Precise )