Twitter uses wrong encoding when remove urls is ticked

[Dan Garner]

Something goes wrong with Twitter in the parsing of special characters like apostrophes, ' turns into â as seen in this example:

"Good students of @KaiminNews & @UMJSchool, enjoy winter break. I’m going to make your brain hurt starting next week. http://www.ronreason.com/pollner/ "

turns into:

"Good students of @KaiminNews & @UMJSchool, enjoy winter break. I’m going to make your brain hurt starting next week."

I'm not sure if this an issue in the character encoding used or what might be causing this, but it definitely looks bad.

[Dan Garner (dangarner)]

can you try replacing the following line in modules/twitter.module.php ?

Line 829:
$tweetText = preg_replace("@(https?://([-\w\.]+[-\w])+(:\d+)?(/([\w/_\.#-]*(\?\S+)?[^\.\s])?)?)@", '', $tweet->text);

becomes

$tweetText = preg_replace("@(https?://([-\w\.]+[-\w])+(:\d+)?(/([\w/_\.#-]*(\?\S+)?[^\.\s])?)?)@u", '', $tweet->text);

[markuhde (mark-markuhde)]

That seems to fix the apostrophes but there is still some odd behaviour:

RT @mtpublicradio: Find #Montana Legislature coverage from MTPR, YPR and @UMJSchool here http://t.co/m5t62B1gBE #mtleg http://t.co/Tj5IzwTm…

Becomes:

RT @mtpublicradio: Find #Montana Legislature coverage from MTPR, YPR and @UMJSchool here #mtleg €¦

It seems to me that the "¦" isn't necessarily ideal to add.