countrychooser: db_go fails when choosing Arabic as language

[Debian Bug Importer]

Automatically imported from Debian bug report #272136 http://bugs.debian.org/272136

[Debian Bug Importer (debzilla)]

Message-Id: <email address hidden>
Date: Fri, 17 Sep 2004 19:10:03 +0200
From: Christian Perrier <email address hidden>
To: Debian Bug Tracking System <email address hidden>
Subject: countrychooser: db_go fails when choosing Arabic as language

Package: countrychooser
Version: 1.03
Severity: important

This bug may be reproduced on any sid_d-i ISO image.

When choosing Arabic as language, the screen flickers instead of displaying
countrychooser first screen. Then this screen is displayed, however it is
displayed in English.

Tracking down this bug, I found that countrychooser postinst fails at the db_go call
in the state machine:

   if db_go; then
      STATE=$(($STATE + 1))

This makes the debian-installer script fail...It is then restarted by init,
but the environment then lacks the LANG variable, so countrychooser postinst
is displayed in English and the whole installer then continues in English.

This seems related to the display of the Arabic screen, with country names
in Arabic.

This does not happen in RC1....in which the country names were not
translated to Arabic because of incomplete iso-codes translation.

I suspect some nasty thing related to Arabic display in these conditions but
I have no idea how to track this down.

-- System Information:
Debian Release: 3.1
  APT prefers unstable
  APT policy: (500, 'unstable'), (500, 'testing')
Architecture: i386 (i686)
Kernel: Linux 2.6.8-1-686
Locale: LANG=fr_FR.UTF-8, LC_CTYPE=fr_FR.UTF-8 (ignored: LC_ALL set to fr_FR.UTF-8)

[Debian Bug Importer (debzilla)]

Message-ID: <email address hidden>
Date: Fri, 17 Sep 2004 15:26:03 -0400
From: Joey Hess <email address hidden>
To: Christian Perrier <email address hidden>, <email address hidden>
Subject: Re: Bug#272136: countrychooser: db_go fails when choosing Arabic as language

--RIYY1s2vRbPFwWeW
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

Christian Perrier wrote:
> This bug may be reproduced on any sid_d-i ISO image.
>=20
> When choosing Arabic as language, the screen flickers instead of displayi=
ng
> countrychooser first screen. Then this screen is displayed, however it is
> displayed in English.
>=20
> Tracking down this bug, I found that countrychooser postinst fails at the=
 db_go call
> in the state machine:
>=20
> if db_go; then
> STATE=3D$(($STATE + 1))
> =20
> This makes the debian-installer script fail...It is then restarted by ini=
t,
> but the environment then lacks the LANG variable, so countrychooser posti=
nst
> is displayed in English and the whole installer then continues in English.
>=20
> This seems related to the display of the Arabic screen, with country names
> in Arabic.
>=20
> This does not happen in RC1....in which the country names were not
> translated to Arabic because of incomplete iso-codes translation.
>=20
> I suspect some nasty thing related to Arabic display in these conditions =
but
> I have no idea how to track this down.

This sounds similar to the also unexplained #256405.

--=20
see shy jo

--RIYY1s2vRbPFwWeW
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Digital signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.5 (GNU/Linux)

iD8DBQFBSzpKd8HHehbQuO8RAvyBAJ9n4Xd5GLGOvYoJ2inBGIQ/AV8yTACeNAiG
lKnxKmXBZO5i8UAFJsb8BQc=
=MDJd
-----END PGP SIGNATURE-----

--RIYY1s2vRbPFwWeW--

[Debian Bug Importer (debzilla)]

Message-ID: <email address hidden>
Date: Sat, 18 Sep 2004 07:07:13 +0200
From: Christian Perrier <email address hidden>
To: <email address hidden>
Subject: Re: Bug#272136: countrychooser: db_go fails when choosing Arabic as language

> > I suspect some nasty thing related to Arabic display in these conditions but
> > I have no idea how to track this down.
>
> This sounds similar to the also unexplained #256405.

(Arabic hanging on countrychooser)

I will try to narrow down this to either the short country list
template or the list of countries.....

[Debian Bug Importer (debzilla)]

Message-ID: <email address hidden>
Date: Sat, 18 Sep 2004 07:14:32 +0200
From: Christian Perrier <email address hidden>
To: Joey Hess <email address hidden>, <email address hidden>,
 <email address hidden>
Subject: Re: Bug#272136: countrychooser: db_go fails when choosing Arabic as language

> This sounds similar to the also unexplained #256405.

By the way, 256405 does not happen anymore....

Should we close it even if it not explained?

[Debian Bug Importer (debzilla)]

Message-Id: <email address hidden>
Date: Sat, 18 Sep 2004 13:07:27 +0200
From: Frans Pop <email address hidden>
To: <email address hidden>
Subject: countrychooser: db_go fails when choosing Arabic as language

Hmmm. The behavior I am seeing is somewhat different.
I can reproduce this both during normal installation and in qemu.
I have also made some screenshots of this.

1. Start d-i with linux26 debconf/priority=medium
2. Choose ar in languagechooser
3. Choose countrychooser
4. Countrylist is displayed correctly (translated countrynames) [1]
5. Choose "Go back"
6. Choose countrychooser again [2]
7. Countrychooser fails, I am returned to main menu and screen is garbled
   The log (/var/log/syslog) shows:
   - Process '/sbin/debian-installer' (pid ###) exited. Scheduling it for
     restart.
   - Starting pid ###, console /dev/vc/1: '/sbin/debian-installer'
   Looking at output form ps, I see the following processes have new pid's:
   - /bin/sh /sbin/debian-installer
   - /usr/bin/bterm -f /unifont.bgf -l C.UTF-8 /lib/debian...
   - debconf -o d-i /usr/bin/main-menu
   - /usr/bin/main-menu
   I think the restart of bterm could explain the garbled screen.
8. Repair display by switching to vt2 and back (main menu is still in ar)
9. Choose countrychooser and everything repeats as from step 3.

I see something else that may be relevant if I scroll down in the country
list.
1. I scroll using page down/page up, everything is OK.
2. I scroll all the way down using down arrow (there are two or three lines
   'below the screen'), looks OK [3]
3. I scroll all the way up again using up arrow, looks OK
4. I press down arrow, last position of selection bar stays on top line [4]
5. I press down arrow, selection bar is again full length [5]

This might indicate there is something wrong with either the first, second,
last but one or last ('other') country name (my guess would be the second).

[1] http://home.tiscali.nl/isildur/d-i/ar1.jpg
[2] http://home.tiscali.nl/isildur/d-i/ar2.jpg
[3] http://home.tiscali.nl/isildur/d-i/ar3.jpg
[4] http://home.tiscali.nl/isildur/d-i/ar4.jpg
[5] http://home.tiscali.nl/isildur/d-i/ar5.jpg

[Debian Bug Importer (debzilla)]

Message-Id: <email address hidden>
Date: Sat, 18 Sep 2004 15:28:55 +0200
From: Frans Pop <email address hidden>
To: <email address hidden>
Cc: Christian Perrier <email address hidden>
Subject: Re: Bug#272136: countrychooser: db_go fails when choosing Arabic as language

On Saturday 18 September 2004 13:07, Frans Pop wrote:
> This might indicate there is something wrong with either the first,
> second, last but one or last ('other') country name (my guess would be
> the second).

Hmmm again.
I tested some more and the behavior is very inconsistent. Anyway, I decided
to add countries to the shortlist in a structured way (starting from the
bottom) and the following shortlist displays correctly:
SD IQ KW MA IN YE TN OM QA LB EG
Adding SY or SA to this list results in problems.

A shortlist of JO AE BH DZ LY also causes problems, but only JO BH is ok.
AE definitely causes problems.

So the problem is somewhere in the translations for AE DZ LY SY and/or SA.
I'll leave it to the arabists to sort out which character(s) is/are the
culprit.

As the translations for these 5 countries are also the longest, it might
also have to do something with length.

[Debian Bug Importer (debzilla)]

Message-ID: <email address hidden>
Date: Sat, 18 Sep 2004 15:05:13 -0400
From: Joey Hess <email address hidden>
To: Christian Perrier <email address hidden>
Cc: <email address hidden>, <email address hidden>
Subject: Re: Bug#272136: countrychooser: db_go fails when choosing Arabic as language

--xo44VMWPx7vlQ2+2
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

Christian Perrier wrote:
> > This sounds similar to the also unexplained #256405.
>=20
>=20
> By the way, 256405 does not happen anymore....

Yes it does, I just reproduced it.

--=20
see shy jo

--xo44VMWPx7vlQ2+2
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Digital signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.5 (GNU/Linux)

iD8DBQFBTIbpd8HHehbQuO8RAuPaAKCj8ePQzE2tmWgXSAH4Y77k4t0y1gCfXLNh
g0M+6vPl1AtLG3rL1w22vgc=
=ijml
-----END PGP SIGNATURE-----

--xo44VMWPx7vlQ2+2--

[Debian Bug Importer (debzilla)]

Message-ID: <email address hidden>
Date: Sun, 19 Sep 2004 09:27:56 +0200
From: Christian Perrier <email address hidden>
To: <email address hidden>
Cc: <email address hidden>
Subject: [URGENT] Debian Installer: Bug#272136: countrychooser: db_go fails when choosing Arabic as
 language

--r7U+bLA8boMOj+mD
Content-Type: text/plain; charset=iso-8859-15
Content-Disposition: inline
Content-Transfer-Encoding: 8bit

Salam,

We have some very tricky problems with the Debian Installer and Arabic
translations.

I a few cases, Arabic causes D-I parts to crash. This is described in
the following bug reports:

http://bugs.debian.org/272136
http://bugs.debian.org/256405

In both cases, the cause seems to be a missing character in the font
we use.

The font used in the Installer is bogl-bterm unifont font.

This font is "reduced" when the installer is built. This means that
all characters which are used in all languages are kept....and, as a
consequence, all other characters are removed.

Each language provides, in a special file, all Unicode characters
needed for a proper display of this language. I have attached this
file for Arabic.

It seems that Arbaic is causing a specific problem�: the shaping code
may cause a character which is not originally in an Arabic string to
be used for display after shaping occurred. Thus, if this characters
was removed by the font reduction routine, we have the problem.

Below, you will find an investigation by Frans Pop about bug
#272136. Frans seems to indicate that the problem for instance occurs
with the translation in Arabic of the country names for SY (Syria) and
SA (Saudi Arabia).

Do you by chance have an idea of what is causing this problem. I
suspect a missing character in the attached file, this character being
indeed used after shaping of the translations for Syria and Saudi
Arabia.

The same probably happens for #256145 with the translation of iso-scan
menu item. See attached ar.po file for the current Arabic translation.

This is a very important problem as currently, Arabic translations are
unusable in Debian Installer

Please keep <email address hidden> to your answers.

----- Forwarded message from Frans Pop <email address hidden> -----

From: Frans Pop <email address hidden>
To: <email address hidden>
Subject: Re: Bug#272136: countrychooser: db_go fails when choosing Arabic as language
Date: Sat, 18 Sep 2004 15:28:55 +0200
Cc: Christian Perrier <email address hidden>

On Saturday 18 September 2004 13:07, Frans Pop wrote:
> This might indicate there is something wrong with either the first,
> second, last but one or last ('other') country name (my guess would be
> the second).

Hmmm again.
I tested some more and the behavior is very inconsistent. Anyway, I decided
to add countries to the shortlist in a structured way (starting from the
bottom) and the following shortlist displays correctly:
SD IQ KW MA IN YE TN OM QA LB EG
Adding SY or SA to this list results in problems.

A shortlist of JO AE BH DZ LY also causes problems, but only JO BH is ok.
AE definitely causes problems.

So the problem is somewhere in the translations for AE DZ LY SY and/or SA.
I'll leave it to the arabists to sort out w...

[Debian Bug Importer (debzilla)]

Message-Id: <1095579917.1339.1.camel@localhost>
Date: Sun, 19 Sep 2004 11:45:17 +0400
From: Mohammed Elzubeir <email address hidden>
To: Documentation and Translation <email address hidden>
Cc: <email address hidden>
Subject: Re: [URGENT] Debian Installer: Bug#272136: countrychooser: db_go
 fails when choosing Arabic as language

--=-4ERgtTkMTkfrU8rukVPm
Content-Type: text/plain
Content-Transfer-Encoding: quoted-printable

On Sun, 2004-09-19 at 11:27, Christian Perrier wrote:
> Salam,
>=20
> We have some very tricky problems with the Debian Installer and Arabic
> translations.
>=20
> I a few cases, Arabic causes D-I parts to crash. This is described in
> the following bug reports:
>=20
> http://bugs.debian.org/272136
> http://bugs.debian.org/256405
>=20
> In both cases, the cause seems to be a missing character in the font
> we use.
>=20

Looks like the problem is coming from diacritics (harakat). I don't see
stand-alone harakat in your first attachment. Also, the only thing that
is common between the country names mentioned and the po file you have
attached, it seems, is that they include the 'shadda' diacritic.

Others, feel free to correct me if I am wrong.

Regards,
Mohammed Elzubeir

--=-4ERgtTkMTkfrU8rukVPm
Content-Type: application/pgp-signature; name=signature.asc
Content-Description: This is a digitally signed message part

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)

iD8DBQBBTTkN1zah8P9Q4KQRAg5sAKCDzEhT+/OXjDNZoSfyd609RnurIQCfeW/6
9NQUwvvEyiTJNGPbU4TTMBM=
=eu8h
-----END PGP SIGNATURE-----

--=-4ERgtTkMTkfrU8rukVPm--

[Debian Bug Importer (debzilla)]

Message-ID: <email address hidden>
Date: Sun, 19 Sep 2004 09:50:47 +0200
From: Christian Perrier <email address hidden>
To: Documentation and Translation <email address hidden>
Cc: <email address hidden>
Subject: Re: [URGENT] Debian Installer: Bug#272136: countrychooser: db_go fails when choosing Arabic
 as language

> Looks like the problem is coming from diacritics (harakat). I don't see
> stand-alone harakat in your first attachment. Also, the only thing that
> is common between the country names mentioned and the po file you have
> attached, it seems, is that they include the 'shadda' diacritic.
>
> Others, feel free to correct me if I am wrong.

Hmmm, diacritics indeed modify another character, right?

This means that combining a character and a diacritic would give a
different character. The combination is done by the shaping code and
thus the final character is indeed not present in the original file,
right?

If that is true, this means that the needed characters (the rsult of
the combination of harakat and the original character) should be in
the needed-characters/ar file as well probably as the standalone
harakat.

Could you complete the "ar" file I originally sent so that it includes
harakat as well as combined characters (if I'm not wrong above)�?

[Debian Bug Importer (debzilla)]

Message-Id: <1095581753.1345.16.camel@localhost>
Date: Sun, 19 Sep 2004 12:15:53 +0400
From: Mohammed Elzubeir <email address hidden>
To: <email address hidden>
Subject: [Fwd: Re: [URGENT] Debian Installer: Bug#272136: countrychooser:
 db_go fails when choosing Arabic as language]

--=-C1T9+N+Jyh6jGc+45M5X
Content-Type: text/plain
Content-Transfer-Encoding: quoted-printable

-----Forwarded Message-----
From: Mohammed Elzubeir <email address hidden>
To: Documentation and Translation <email address hidden>
Subject: Re: [URGENT] Debian Installer: Bug#272136: countrychooser: db_go f=
ails when choosing Arabic as language
Date: Sun, 19 Sep 2004 12:11:41 +0400

On Sun, 2004-09-19 at 11:50, Christian Perrier wrote:

> Hmmm, diacritics indeed modify another character, right?
>=20
Not really. A diacritic like a 'SHADDA' for example, would be displayed
on top of a character's glyph. The result would "look" as if it's a
different character but it's not. In reality, what the shadda means is
the equivalent of repeating the character (e.g. alarabiYYa -- in Arabic,
instead of having two Y's you would have a Y and a shadda on top of it).

> This means that combining a character and a diacritic would give a
> different character. The combination is done by the shaping code and
> thus the final character is indeed not present in the original file,
> right?
>=20

No, the shaping code is not involved in this process. Shaping only
concerns itself with what glyph to use to represent a character,
depending on its position -- diacritics is out of its scope.

> If that is true, this means that the needed characters (the rsult of
> the combination of harakat and the original character) should be in
> the needed-characters/ar file as well probably as the standalone
> harakat.

Harakat are those guys U+064B - U+0652

>=20
> Could you complete the "ar" file I originally sent so that it includes
> harakat as well as combined characters (if I'm not wrong above) ?
>=20

Sure -- but first, I can't seem to figure out what that last character
is in that file. Can anyone tell me what it is? (looks very strange
LAM+ALEF with 'something' else there..)

Regards,
Mohammed Elzubeir

________________________________________________________________________
_______________________________________________
Doc mailing list
<email address hidden>
http://lists.arabeyes.org/mailman/listinfo/doc

--=-C1T9+N+Jyh6jGc+45M5X
Content-Type: application/pgp-signature; name=signature.asc
Content-Description: This is a digitally signed message part

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)

iD8DBQBBTUA51zah8P9Q4KQRAjBvAJ9/zeGv/N71VCxLq2NR3rsVAlyzSACggXL4
hoVtRiwyLBOXDJk2Vgfj7yM=
=MDUi
-----END PGP SIGNATURE-----

--=-C1T9+N+Jyh6jGc+45M5X--

[Debian Bug Importer (debzilla)]

Message-Id: <email address hidden>
Date: Sun, 19 Sep 2004 11:42:55 +0200
From: Frans Pop <email address hidden>
To: <email address hidden>
Cc: <email address hidden>
Subject: Re: Bug#272136: [URGENT] Debian Installer: Bug#272136: countrychooser: db_go fails when
 choosing Arabic as language

On Sunday 19 September 2004 09:27, Christian Perrier wrote:
> Below, you will find an investigation by Frans Pop about bug
> #272136. Frans seems to indicate that the problem for instance occurs
> with the translation in Arabic of the country names for SY (Syria) and
> SA (Saudi Arabia).

Note that there are _five_ countrynames which may cause the problem, not
only SY and SA (see below).

> ----- Forwarded message from Frans Pop <email address hidden> -----

> So the problem is somewhere in the translations for AE DZ LY SY and/or
> SA. I'll leave it to the arabists to sort out which character(s) is/are
> the culprit.

[Debian Bug Importer (debzilla)]

Message-ID: <email address hidden>
Date: Sun, 19 Sep 2004 17:45:16 +0300
From: Eugeniy Meshcheryakov <email address hidden>
To: Christian Perrier <email address hidden>, <email address hidden>
Subject: Re: Bug#272136: [URGENT] Debian Installer: Bug#272136: countrychooser:
 db_go fails when choosing Arabic as language

--------------enig07DD76747EA4ADE4A030F38A
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Christian Perrier написав:
>>Looks like the problem is coming from diacritics (harakat). I don't see
>>stand-alone harakat in your first attachment. Also, the only thing that
>>is common between the country names mentioned and the po file you have
>>attached, it seems, is that they include the 'shadda' diacritic.
>>
>>Others, feel free to correct me if I am wrong.
>
>
>
> Hmmm, diacritics indeed modify another character, right?
>
> This means that combining a character and a diacritic would give a
> different character. The combination is done by the shaping code and
> thus the final character is indeed not present in the original file,
> right?
>
It is not correct. For Arabic diacrits console should render two
characters in one position. As I see from bterm sources it supports such
type of combining characters.

> If that is true, this means that the needed characters (the rsult of
> the combination of harakat and the original character) should be in
> the needed-characters/ar file as well probably as the standalone
> harakat.
>
harakats should be in needed-characters.

> Could you complete the "ar" file I originally sent so that it includes
> harakat as well as combined characters (if I'm not wrong above) ?
>
>
>

--
Eugeniy Meshcheryakov

Kyiv National Taras Shevchenko University
Information and Computing Centre
http://icc.univ.kiev.ua

--------------enig07DD76747EA4ADE4A030F38A
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFBTZuEKaC6+zmozOIRAsqQAJ9LBSd9hFCkqqH1S3KacvjgHckR3ACghakF
lybHcinZ8URWBVOBk1TW7Sc=
=kwsA
-----END PGP SIGNATURE-----

--------------enig07DD76747EA4ADE4A030F38A--

[Debian Bug Importer (debzilla)]

Message-ID: <email address hidden>
Date: Sun, 19 Sep 2004 17:59:43 +0200
From: Christian Perrier <email address hidden>
To: <email address hidden>
Cc: <email address hidden>
Subject: Re: [URGENT] Debian Installer: Bug#272136: countrychooser: db_go fails when choosing Arabic
 as language

Quoting Mohammed Elzubeir (<email address hidden>):

> Harakat are those guys U+064B - U+0652

I have tried adding all Harakat to the needed-characters file.

Indeed I found some of these in country names such as Saudi Arabia and
Syria. I'm not sure that all countries mentioned by Frans Pop as
"problematic" have Harakat, but anyway, this should not harm

[Debian Bug Importer (debzilla)]

Message-ID: <email address hidden>
Date: Sun, 19 Sep 2004 22:27:48 +0300
From: Eugeniy Meshcheryakov <email address hidden>
To: Christian Perrier <email address hidden>, <email address hidden>
Subject: Re: Bug#272136: [URGENT] Debian Installer: Bug#272136: countrychooser:
 db_go fails when choosing Arabic as language

--------------enigB8FA60C3C1FC608659AFFD60
Content-Type: multipart/mixed;
 boundary="------------030105040409030606030106"

This is a multi-part message in MIME format.
--------------030105040409030606030106
Content-Type: text/plain; charset=KOI8-U
Content-Transfer-Encoding: 8bit

Christian Perrier ����

> Do you by chance have an idea of what is causing this problem. I
> suspect a missing character in the attached file, this character being
> indeed used after shaping of the translations for Syria and Saudi
> Arabia.
>
As I can see this is not a bug in debian-installer, I can reproduce it
with attached script (UTF8-encoded). Probably, it is not a bug in
shaping/bidi code, I can reproduce it without /usr/lib/libfribidi.so.0.
So this is bug in newt or slang.

--
Eugeniy Meshcheryakov

Kyiv National Taras Shevchenko University
Information and Computing Centre
http://icc.univ.kiev.ua

--------------030105040409030606030106
Content-Type: text/plain;
 name="out"
Content-Transfer-Encoding: 8bit
Content-Disposition: inline;
 filename="out"

#! /bin/sh
/usr/bin/whiptail "--menu" "menu" "20" "33" "12" "--" "ألمانيّة سويسريّة" ""

--------------030105040409030606030106--

--------------enigB8FA60C3C1FC608659AFFD60
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFBTd24KaC6+zmozOIRAgURAJ917z5nydhUdKGZgD/ZLBygsu6+wACgh6Bd
m8kgXcMOlcQtHA4EMQyldVE=
=XnGw
-----END PGP SIGNATURE-----

--------------enigB8FA60C3C1FC608659AFFD60--

[Debian Bug Importer (debzilla)]

Message-ID: <email address hidden>
Date: Sun, 19 Sep 2004 21:18:00 -0700
From: Steve Langasek <email address hidden>
To: Mohammed Elzubeir <email address hidden>
Cc: <email address hidden>
Subject: Re: Bug#272136: [Fwd: Re: [URGENT] Debian Installer: Bug#272136: countrychooser: db_go
 fails when choosing Arabic as language]

--LiQwW4YX+w4axhAx
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Sun, Sep 19, 2004 at 12:15:53PM +0400, Mohammed Elzubeir wrote:
> -----Forwarded Message-----
> From: Mohammed Elzubeir <email address hidden>
> To: Documentation and Translation <email address hidden>
> Subject: Re: [URGENT] Debian Installer: Bug#272136: countrychooser: db_go=
 fails when choosing Arabic as language
> Date: Sun, 19 Sep 2004 12:11:41 +0400

> On Sun, 2004-09-19 at 11:50, Christian Perrier wrote:

> > Hmmm, diacritics indeed modify another character, right?

> Not really. A diacritic like a 'SHADDA' for example, would be displayed
> on top of a character's glyph. The result would "look" as if it's a
> different character but it's not. In reality, what the shadda means is
> the equivalent of repeating the character (e.g. alarabiYYa -- in Arabic,
> instead of having two Y's you would have a Y and a shadda on top of it).

> > This means that combining a character and a diacritic would give a
> > different character. The combination is done by the shaping code and
> > thus the final character is indeed not present in the original file,
> > right?

> No, the shaping code is not involved in this process. Shaping only
> concerns itself with what glyph to use to represent a character,
> depending on its position -- diacritics is out of its scope.

> > If that is true, this means that the needed characters (the rsult of
> > the combination of harakat and the original character) should be in
> > the needed-characters/ar file as well probably as the standalone
> > harakat.

> Harakat are those guys U+064B - U+0652

> > Could you complete the "ar" file I originally sent so that it includes
> > harakat as well as combined characters (if I'm not wrong above) ?

> Sure -- but first, I can't seem to figure out what that last character
> is in that file. Can anyone tell me what it is? (looks very strange
> LAM+ALEF with 'something' else there..)

The /last/ character in the file is U+0655. The file already does
contain all of U+064B - U+0652, as well as U+0653, U+0654, and U+0655.

--=20
Steve Langasek
postmodern programmer

--LiQwW4YX+w4axhAx
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Digital signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)

iD8DBQFBTln3KN6ufymYLloRAtnQAKCRrZ4RgHScVPD8sEkLDIfDa5SfpACeNPiD
OHaxhIJUb3bAeo+djPzypNA=
=1VB+
-----END PGP SIGNATURE-----

--LiQwW4YX+w4axhAx--

[Debian Bug Importer (debzilla)]

Message-ID: <email address hidden>
Date: Mon, 20 Sep 2004 06:58:43 +0200
From: Christian Perrier <email address hidden>
To: Eugeniy Meshcheryakov <email address hidden>, <email address hidden>,
 <email address hidden>, <email address hidden>
Subject: Re: Bug#272136: [URGENT] Debian Installer: Bug#272136: countrychooser: db_go fails when
 choosing Arabic as language

--uTRFFR9qmiCqR05s
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
X-MIME-Autoconverted: from 8bit to quoted-printable by onera.onera.fr id i8K67aOn004195

Steve, Alastair,

Could you have a look at 272136=C2=A0?

This bug has a very long story. It started by "Debian installer
crashes on Arabic menu for countrychooser". Now, after a lot of
investigation, Eugeniy reduced it to "something crashes when trying to
display Arabic with whiptail".

If I understand properly what is mentioned by Evgueniy, this even
happens without libfribidi installed.

This is somewhat important as it currently makes the Arabic
translation of D-I completely useless....

I re-attach the test script for you to test. I have been able to
reproduce the crash.

I want to have your advice before reassigning this to newt.

Quoting Eugeniy Meshcheryakov (<email address hidden>):
> Christian Perrier =D0=BD=D0=B0=D0=BF=D0=B8=D1=81=D0=B0=D0=B2:
>=20
> > Do you by chance have an idea of what is causing this problem. I
> > suspect a missing character in the attached file, this character bein=
g
> > indeed used after shaping of the translations for Syria and Saudi
> > Arabia.
> >=20
> As I can see this is not a bug in debian-installer, I can reproduce it
> with attached script (UTF8-encoded). Probably, it is not a bug in
> shaping/bidi code, I can reproduce it without /usr/lib/libfribidi.so.0.
> So this is bug in newt or slang.
>=20
> --=20
> Eugeniy Meshcheryakov
>=20
> Kyiv National Taras Shevchenko University
> Information and Computing Centre
> http://icc.univ.kiev.ua

> #! /bin/sh
> /usr/bin/whiptail "--menu" "menu" "20" "33" "12" "--" "???????????????=
? ????????????????" ""=20

--=20

--uTRFFR9qmiCqR05s
Content-Type: text/plain; charset=utf-8
Content-Disposition: attachment; filename=out
Content-Transfer-Encoding: base64
X-MIME-Autoconverted: from 8bit to base64 by onera.onera.fr id i8K67aOn004195

IyEgL2Jpbi9zaA0KL3Vzci9iaW4vd2hpcHRhaWwgIi0tbWVudSIgIm1lbnUiICIyMCIgIjMz
IiAiMTIiICItLSIgICLYo9mE2YXYp9mG2YrZkdipINiz2YjZitiz2LHZitmR2KkiICIiIA0K
--uTRFFR9qmiCqR05s--

[Debian Bug Importer (debzilla)]

Message-ID: <email address hidden>
Date: Mon, 20 Sep 2004 07:40:13 +0200
From: Christian Perrier <email address hidden>
To: <email address hidden>
Cc: <email address hidden>, <email address hidden>
Subject: More progress about the "Arabic crash" in Debian Installer

After some investigation, Eugeniy Meshcheryakov narrowed the problem
of Debian Installer crashing to what could be a bug in whiptail indeed
(whiptail is the backend used by Debian Installer for building menus).

The attached script segfaults while trying to display a menu
containing some Arabic phrase. No idea where Eugeniy found these two
words and how he managed to narrow stuff to them, but the problem is
reproducible...

What is so special in these words=A0? If I'm correct, there is a shadda
in them (at least this is what I find in looking in an Unicode table=A0:
U0651=3D=3D"Arabic shadda") but trying with it alone did not trigger the
crash.

/me, damn puzzled....

(please keep the CC list in your answers)

--=20

[Debian Bug Importer (debzilla)]

Message-ID: <email address hidden>
Date: Mon, 20 Sep 2004 07:55:02 +0200
From: Christian Perrier <email address hidden>
To: <email address hidden>
Cc: <email address hidden>, <email address hidden>
Subject: Re: More progress about the "Arabic crash" in Debian Installer

--wayzTnRSUXKNfBqd
Content-Type: text/plain; charset=iso-8859-15
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
X-MIME-Autoconverted: from 8bit to quoted-printable by onera.onera.fr id i8K67dOn004201

> What is so special in these words=A0? If I'm correct, there is a shadda
> in them (at least this is what I find in looking in an Unicode table=A0=
:
> U0651=3D=3D"Arabic shadda") but trying with it alone did not trigger th=
e
> crash.

Yet more progress.....

There are indeed two shadda characters in this string

Removing any of the two shaddas removes the case for the crash.

Keeping the two shaddas with the character before and the character
after for each make the script crashing (see out.crash).

Removing ANY character in out.crash (including the space character!)=20
prevents it from crashing.

--wayzTnRSUXKNfBqd
Content-Type: text/plain; charset=utf-8
Content-Disposition: attachment; filename="out.crash"
Content-Transfer-Encoding: base64
X-MIME-Autoconverted: from 8bit to base64 by onera.onera.fr id i8K67dOn004201

IyEgL2Jpbi9zaA0KL3Vzci9iaW4vd2hpcHRhaWwgIi0tbWVudSIgIm1lbnUiICIyMCIgIjMz
IiAiMTIiICItLSIgICLZitmR2Kkg2YrZkdipIiAiIiANCg==
--wayzTnRSUXKNfBqd--

[Debian Bug Importer (debzilla)]

Message-ID: <email address hidden>
Date: Mon, 20 Sep 2004 07:41:04 +0200
From: Christian Perrier <email address hidden>
To: <email address hidden>, <email address hidden>
Cc: <email address hidden>
Subject: Re: More progress about the "Arabic crash" in Debian Installer

--uc35eWnScqDcQrv5
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline

> The attached script segfaults while trying to display a menu

Now really attached

--uc35eWnScqDcQrv5
Content-Type: text/plain; charset=utf-8
Content-Disposition: attachment; filename="out.save"
Content-Transfer-Encoding: base64
X-MIME-Autoconverted: from 8bit to base64 by onera.onera.fr id i8K67eOn004213

IyEgL2Jpbi9zaA0KL3Vzci9iaW4vd2hpcHRhaWwgIi0tbWVudSIgIm1lbnUiICIyMCIgIjMz
IiAiMTIiICItLSIgICLYo9mE2YXYp9mG2YrZkdipINiz2YjZitiz2LHZitmR2KkiICIiIA0K
--uc35eWnScqDcQrv5--

[Debian Bug Importer (debzilla)]

Message-ID: <email address hidden>
Date: Sun, 19 Sep 2004 23:24:02 -0700
From: Steve Langasek <email address hidden>
To: Christian Perrier <email address hidden>
Cc: Eugeniy Meshcheryakov <email address hidden>,
 <email address hidden>, <email address hidden>
Subject: Re: Bug#272136: [URGENT] Debian Installer: Bug#272136: countrychooser: db_go fails when
 choosing Arabic as language

--VaKJWhUROU/xPxjb
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Mon, Sep 20, 2004 at 06:58:43AM +0200, Christian Perrier wrote:
> Steve, Alastair,

> Could you have a look at 272136=A0?

> This bug has a very long story. It started by "Debian installer
> crashes on Arabic menu for countrychooser". Now, after a lot of
> investigation, Eugeniy reduced it to "something crashes when trying to
> display Arabic with whiptail".

> If I understand properly what is mentioned by Evgueniy, this even
> happens without libfribidi installed.

libfribidi is only responsible for the bi-directional text, not the
shaping; shaping was patched into slang directly.

> This is somewhat important as it currently makes the Arabic
> translation of D-I completely useless....

> I re-attach the test script for you to test. I have been able to
> reproduce the crash.

> I want to have your advice before reassigning this to newt.

I'm pretty sure it's a slang bug, not a newt bug. I.e., my fault. ;)
I'll see what I can come up with given that test case, thanks.

--=20
Steve Langasek
postmodern programmer

--VaKJWhUROU/xPxjb
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Digital signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)

iD8DBQFBTnd4KN6ufymYLloRAi0oAJ4nmqi5scr18UhXR0JS9WvhjN048wCguF7L
x4E5rfj2pHzdPO0K+tW7VF0=
=nAmw
-----END PGP SIGNATURE-----

--VaKJWhUROU/xPxjb--

[Debian Bug Importer (debzilla)]

Message-ID: <email address hidden>
Date: Mon, 20 Sep 2004 08:50:44 +0200
From: Christian Perrier <email address hidden>
To: <email address hidden>, <email address hidden>, <email address hidden>
Subject: Re: More progress about the "Arabic crash" in Debian Installer

Quoting Christian Perrier (<email address hidden>):
> > What is so special in these words=A0? If I'm correct, there is a shad=
da
> > in them (at least this is what I find in looking in an Unicode table=A0=
:
> > U0651=3D=3D"Arabic shadda") but trying with it alone did not trigger =
the
> > crash.
>=20
> Yet more progress.....
>=20
> There are indeed two shadda characters in this string
>=20
> Removing any of the two shaddas removes the case for the crash.
>=20
> Keeping the two shaddas with the character before and the character
> after for each make the script crashing (see out.crash).
>=20
> Removing ANY character in out.crash (including the space character!)=20
> prevents it from crashing.
>=20
>=20
>=20

> #! /bin/sh
> /usr/bin/whiptail "--menu" "menu" "20" "33" "12" "--" "??? ???" ""=20

Another info.

The crash only happens when the current locale is a UTF-8 locale

export LC_ALL=3DC=A0; ./out --> no crash
export LC_ALL=3Dfr_FR.UTF-8=A0; ./out --> crash

(Steve tried with another UTF-8 locale and had the crash)

IF the locale is not UTF-8, the characters are not interpreted as
Arabic and thus the crash does not happen

Another info=A0: Steve tried without libfribidi0 and the crash didn't hap=
pen

--=20

[Debian Bug Importer (debzilla)]

Message-ID: <email address hidden>
Date: Mon, 20 Sep 2004 09:57:41 +0300
From: Eugeniy Meshcheryakov <email address hidden>
To: Steve Langasek <email address hidden>, <email address hidden>
CC: Christian Perrier <email address hidden>, <email address hidden>
Subject: Re: Bug#272136: [URGENT] Debian Installer: Bug#272136: countrychooser:
 db_go fails when choosing Arabic as language

--------------enig1C75EE46CCBB192EC6D9E330
Content-Type: text/plain; charset=KOI8-U
Content-Transfer-Encoding: 8bit

Steve Langasek ����
> On Mon, Sep 20, 2004 at 06:58:43AM +0200, Christian Perrier wrote:
>
>>Steve, Alastair,
>
>
>>Could you have a look at 272136 ?
>
>
>>This bug has a very long story. It started by "Debian installer
>>crashes on Arabic menu for countrychooser". Now, after a lot of
>>investigation, Eugeniy reduced it to "something crashes when trying to
>>display Arabic with whiptail".
>
>
>>If I understand properly what is mentioned by Evgueniy, this even
>>happens without libfribidi installed.
>
>
> libfribidi is only responsible for the bi-directional text, not the
> shaping; shaping was patched into slang directly.
>
But slang does not execute shaping code when libfribidi is not loaded...

--
Eugeniy Meshcheryakov

Kyiv National Taras Shevchenko University
Information and Computing Centre
http://icc.univ.kiev.ua

--------------enig1C75EE46CCBB192EC6D9E330
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFBTn9pKaC6+zmozOIRArhSAJ4xbUNTibOPXBRARV9jzccQRhtt+wCcCvsl
uu5usomXVMbwjAawfUj6BtU=
=Mf/6
-----END PGP SIGNATURE-----

--------------enig1C75EE46CCBB192EC6D9E330--

[Debian Bug Importer (debzilla)]

Message-ID: <email address hidden>
Date: Mon, 20 Sep 2004 08:58:51 +0200
From: Petter Reinholdtsen <email address hidden>
To: <email address hidden>
Cc: <email address hidden>, <email address hidden>
Subject: Re: Bug#272136: More progress about the "Arabic crash" in Debian Installer

[Christian Perrier]
> Now really attached

Running the program in valgrind gave this error report. Most of them
look like bugs in glibc to me. Not sure if these bugs are the ones
trigging the crash.

==9047== Memcheck, a memory error detector for x86-linux.
==9047== Copyright (C) 2002-2004, and GNU GPL'd, by Julian Seward et al.
==9047== Using valgrind-2.2.0, a program supervision framework for x86-linux.
==9047== Copyright (C) 2000-2004, and GNU GPL'd, by Julian Seward et al.
==9047== For more details, rerun with: -v
==9047==
==9047== Invalid write of size 4
==9047== at 0x1B9E7BD9: __gconv_transform_utf8_internal (loop.c:311)
==9047== by 0x1BA488CD: mbrtowc (mbrtowc.c:72)
==9047== by 0x1B9533C7: SLsmg_write_nstring (in /lib/libslang.so.1-UTF8.4.9) ==9047== by 0x1B91FFE9: (within /usr/lib/libnewt.so.0.51.6)
==9047== Address 0x1BCB3878 is 0 bytes after a block of size 64 alloc'd
==9047== at 0x1B904EDD: malloc (vg_replace_malloc.c:131)
==9047== by 0x1B953361: SLsmg_write_nstring (in /lib/libslang.so.1-UTF8.4.9) ==9047== by 0x1B91FFE9: (within /usr/lib/libnewt.so.0.51.6)
==9047== by 0x1B91CC75: newtDrawForm (in /usr/lib/libnewt.so.0.51.6)
==9047==
==9047== Invalid read of size 4
==9047== at 0x1BA488DA: mbrtowc (mbrtowc.c:88)
==9047== by 0x1B9533C7: SLsmg_write_nstring (in /lib/libslang.so.1-UTF8.4.9) ==9047== by 0x1B91FFE9: (within /usr/lib/libnewt.so.0.51.6)
==9047== by 0x1B91CC75: newtDrawForm (in /usr/lib/libnewt.so.0.51.6)
==9047== Address 0x1BCB3878 is 0 bytes after a block of size 64 alloc'd
==9047== at 0x1B904EDD: malloc (vg_replace_malloc.c:131)
==9047== by 0x1B953361: SLsmg_write_nstring (in /lib/libslang.so.1-UTF8.4.9) ==9047== by 0x1B91FFE9: (within /usr/lib/libnewt.so.0.51.6)
==9047== by 0x1B91CC75: newtDrawForm (in /usr/lib/libnewt.so.0.51.6)
==9047==
==9047== Invalid read of size 4
==9047== at 0x1B9533DE: SLsmg_write_nstring (in /lib/libslang.so.1-UTF8.4.9) ==9047== by 0x1B91FFE9: (within /usr/lib/libnewt.so.0.51.6)
==9047== by 0x1B91CC75: newtDrawForm (in /usr/lib/libnewt.so.0.51.6)
==9047== by 0x1B91D53E: newtFormRun (in /usr/lib/libnewt.so.0.51.6)
==9047== Address 0x1BCB3878 is 0 bytes after a block of size 64 alloc'd
==9047== at 0x1B904EDD: malloc (vg_replace_malloc.c:131)
==9047== by 0x1B953361: SLsmg_write_nstring (in /lib/libslang.so.1-UTF8.4.9) ==9047== by 0x1B91FFE9: (within /usr/lib/libnewt.so.0.51.6)
==9047== by 0x1B91CC75: newtDrawForm (in /usr/lib/libnewt.so.0.51.6)
==9047==
==9047== Invalid read of size 4
==9047== at 0x1B9535B1: (within /lib/libslang.so.1-UTF8.4.9)
==9047== by 0x1B95387D: (within /lib/libslang.so.1-UTF8.4.9)
==9047== by 0x1B953999: SLsmg_write_nwchars (in /lib/libslang.so.1-UTF8.4.9) ==9047== by 0x1B95340C: SLsmg_write_nstring (in /lib/libslang.so.1-UTF8.4.9) ==9047== Address 0x1BCB3878 is 0 bytes after a block of size 64 alloc'd
=...

[Debian Bug Importer (debzilla)]

Message-ID: <email address hidden>
Date: Mon, 20 Sep 2004 10:10:36 +0300
From: Eugeniy Meshcheryakov <email address hidden>
To: Petter Reinholdtsen <email address hidden>
CC: <email address hidden>, <email address hidden>
Subject: Re: Bug#272136: More progress about the "Arabic crash" in Debian
 Installer

--------------enigD03989A680191AF7F6021512
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Petter Reinholdtsen написав:
> [Christian Perrier]
>
>>Now really attached
>
>
> Running the program in valgrind gave this error report. Most of them
> look like bugs in glibc to me. Not sure if these bugs are the ones
> trigging the crash.
>
I have different output on my system (from from gdb, without fribidi):

Program received signal SIGSEGV, Segmentation fault.

0x401344dd in malloc_consolidate (av=0x401f1fe0) at malloc.c:4382

4382 malloc.c: No such file or directory.

        in malloc.c

(gdb) bt

#0 0x401344dd in malloc_consolidate (av=0x401f1fe0) at malloc.c:4382

#1 0x40133f7a in _int_malloc (av=0x401f1fe0, bytes=1604) at
malloc.c:4103
#2 0x40132ef3 in __libc_malloc (bytes=52) at malloc.c:3296

#3 0x40054cf9 in SLsmg_write_nchars () from /lib/libslang.so.1-UTF8

#4 0x4005431e in SLsmg_write_string () from /lib/libslang.so.1-UTF8

#5 0x4001dc1a in buttonDrawIt (co=0x80597b8, active=0, pushed=16) at
button.c:113
#6 0x4001db6d in buttonDraw (co=0x644) at button.c:96

#7 0x4001ec76 in newtDrawForm (co=0x80586c0) at form.c:609

#8 0x4001f53f in newtFormRun (co=0x80586c0, es=0xbffff5a0) at
form.c:911
#9 0x4001f29c in newtRunForm (co=0x80586c0) at form.c:816

#10 0x0804bafb in listBox (text=0x644 <Address 0x644 out of bounds>,
height=18, width=31,
    optCon=0x804e770, flags=0, default_item=0x0, result=0x644) at
dialogboxes.c:330
#11 0x0804ab13 in main (argc=1604, argv=0x644) at whiptail.c:540

--
Eugeniy Meshcheryakov

Kyiv National Taras Shevchenko University
Information and Computing Centre
http://icc.univ.kiev.ua

--------------enigD03989A680191AF7F6021512
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFBToJwKaC6+zmozOIRAhmaAJ9hXPcC/02Z8AWRI8+O6p6FewQ/EQCfVQ/C
ouXqcOoKluOdv25XgmbSaTU=
=rv31
-----END PGP SIGNATURE-----

--------------enigD03989A680191AF7F6021512--

[Debian Bug Importer (debzilla)]

Message-ID: <email address hidden>
Date: Mon, 20 Sep 2004 10:14:02 +0300
From: Eugeniy Meshcheryakov <email address hidden>
To: Christian Perrier <email address hidden>, <email address hidden>
CC: <email address hidden>
Subject: Re: Bug#272136: More progress about the "Arabic crash" in Debian
 Installer

--------------enig4A55B6A30AF8C48243140014
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Christian Perrier написав:

>
> Another info : Steve tried without libfribidi0 and the crash didn't happen
>
Strange, when I tried without fribidi it crashed too...
>
>

--
Eugeniy Meshcheryakov

Kyiv National Taras Shevchenko University
Information and Computing Centre
http://icc.univ.kiev.ua

--------------enig4A55B6A30AF8C48243140014
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFBToM6KaC6+zmozOIRAoojAJ9UVpGQV4CvtI3k8X4T6fqFQYFuTQCeLs8a
52U+J7s+xzB7o5zdjoHLiBY=
=SADB
-----END PGP SIGNATURE-----

--------------enig4A55B6A30AF8C48243140014--

[Debian Bug Importer (debzilla)]

Message-ID: <email address hidden>
Date: Mon, 20 Sep 2004 00:49:57 -0700
From: Steve Langasek <email address hidden>
To: <email address hidden>
Subject: Re: More progress about the "Arabic crash" in Debian Installer

--Ms5iOKSBOB9YS8zC
Content-Type: multipart/mixed; boundary="fLj60tP2PZ34xyqD"
Content-Disposition: inline

--fLj60tP2PZ34xyqD
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

Please try the attached patch to slang.

Thanks,
--=20
Steve Langasek
postmodern programmer

--fLj60tP2PZ34xyqD
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment; filename="slang-272136.diff"
Content-Transfer-Encoding: quoted-printable

diff -uNr slang-1.4.9/src/slsmg.c ../build-tree.new/slang-1.4.9-utf8/src/sl=
smg.c
--- slang-1.4.9/src/slsmg.c 2004-09-20 00:45:46.000000000 -0700
+++ ../build-tree.new/slang-1.4.9-utf8/src/slsmg.c 2004-09-20 00:41:14.0000=
00000 -0700
@@ -304,12 +304,13 @@
    char blank =3D ' ';
    mbstate_t mbstate;
    wchar_t *wptr;
+ int cellsleft =3D n;
=20
    /* Avoid a problem if a user accidently passes a negative value */
    if ((int) n < 0)
      return;
=20
- wptr =3D (wchar_t *)malloc(sizeof(wchar_t)*(n+1));
+ wptr =3D (wchar_t *)malloc(sizeof(wchar_t)*n);
=20
    if (str !=3D NULL && wptr !=3D NULL)
    {
@@ -323,15 +324,18 @@
         k !=3D (size_t)(-2))
  {
       w =3D wcwidth(wptr[i]);
- if (w < 0 || w > n)
+ if (w < 0 || w > cellsleft)
         break;
       str +=3D k;
- n -=3D w;
+ cellsleft -=3D w;
       i++;
+ if (i >=3D n) {
+ wptr =3D realloc(wptr,sizeof(wchar_t)*(i+1));
+ }
  }
  SLsmg_write_nwchars (wptr, i);
    }
- while (n-- > 0) SLsmg_write_nchars (&blank, 1);
+ while (cellsleft-- > 0) SLsmg_write_nchars (&blank, 1);
    if (wptr) free(wptr);
 }
 #else

--fLj60tP2PZ34xyqD--

--Ms5iOKSBOB9YS8zC
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Digital signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)

iD8DBQFBToujKN6ufymYLloRAkDbAKDQpblZ3Q0r0cCN2tG2jgL4RRKbgwCgq2Sz
sFHQufpodTUksB6OCoycs7Q=
=5mnG
-----END PGP SIGNATURE-----

--Ms5iOKSBOB9YS8zC--

[Debian Bug Importer (debzilla)]

Message-ID: <email address hidden>
Date: Mon, 20 Sep 2004 10:22:14 +0200
From: Bastian Blank <email address hidden>
To: Steve Langasek <email address hidden>, <email address hidden>
Subject: Re: Bug#272136: More progress about the "Arabic crash" in Debian Installer

--X3gaHHMYHkYqP6yf
Content-Type: multipart/mixed; boundary="osDK9TLjxFScVI/L"
Content-Disposition: inline

--osDK9TLjxFScVI/L
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Mon, Sep 20, 2004 at 12:49:57AM -0700, Steve Langasek wrote:
> Please try the attached patch to slang.

I did some cleanups on the patch.

Bastian

--=20
Insufficient facts always invite danger.
  -- Spock, "Space Seed", stardate 3141.9

--osDK9TLjxFScVI/L
Content-Type: text/plain; charset=utf-8
Content-Disposition: attachment; filename=diff
Content-Transfer-Encoding: quoted-printable

diff -ur slang-1.4.9.orig/src/slsmg.c slang-1.4.9-utf8/src/slsmg.c
--- slang-1.4.9.orig/src/slsmg.c 2004-09-20 09:17:58.000000000 +0200
+++ slang-1.4.9-utf8/src/slsmg.c 2004-09-20 10:10:44.000000000 +0200
@@ -309,7 +309,8 @@
    if ((int) n < 0)
      return;
=20
- wptr =3D (wchar_t *)malloc(sizeof(wchar_t)*(n+1));
+ size_t wptr_len =3D n;
+ wptr =3D malloc(sizeof(wchar_t) * wptr_len);
=20
    if (str !=3D NULL && wptr !=3D NULL)
    {
@@ -328,6 +329,14 @@
       str +=3D k;
       n -=3D w;
       i++;
+ /* We have some null-width chars in the string and need more =
space */
+ if (i >=3D wptr_len - 1) {
+ wchar_t *new_wptr =3D realloc(wptr, sizeof(wchar_t) * wpt=
r_len * 2);
+ if (!new_wptr)
+ break;
+ wptr_len *=3D 2;
+ wptr =3D new_wptr;
+ }
  }
  SLsmg_write_nwchars (wptr, i);
    }

--osDK9TLjxFScVI/L--

--X3gaHHMYHkYqP6yf
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Digital signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.5 (GNU/Linux)

iEYEARECAAYFAkFOkzUACgkQnw66O/MvCNHMXACeJ5NH77GHxkASpXtwD8pmubCE
VeQAniYyVjTnSopeuXax20Xdnht9NHw5
=v/3b
-----END PGP SIGNATURE-----

--X3gaHHMYHkYqP6yf--

[Debian Bug Importer (debzilla)]

Message-ID: <email address hidden>
Date: Mon, 20 Sep 2004 11:38:31 +0200
From: Christian Perrier <email address hidden>
To: Bastian Blank <email address hidden>, <email address hidden>
Cc: Steve Langasek <email address hidden>, <email address hidden>
Subject: Re: Bug#272136: More progress about the "Arabic crash" in Debian Installer

severity 256405 serious
severity 272136 serious
reassign 256405 slang1-utf8
reassign 272136 slang1-utf8
merge 256405 272136
merge

Quoting Bastian Blank (<email address hidden>):
> On Mon, Sep 20, 2004 at 12:49:57AM -0700, Steve Langasek wrote:
> > Please try the attached patch to slang.
>
> I did some cleanups on the patch.

So, properly assigning the bug....

[Debian Bug Importer (debzilla)]

Message-ID: <email address hidden>
Date: Mon, 20 Sep 2004 11:44:37 +0200
From: Bastian Blank <email address hidden>
To: Steve Langasek <email address hidden>, <email address hidden>
Subject: Re: Bug#272136: More progress about the "Arabic crash" in Debian Installer

--zS7rBR6csb6tI2e1
Content-Type: multipart/mixed; boundary="0vzXIDBeUiKkjNJl"
Content-Disposition: inline

--0vzXIDBeUiKkjNJl
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Mon, Sep 20, 2004 at 10:22:14AM +0200, Bastian Blank wrote:
> On Mon, Sep 20, 2004 at 12:49:57AM -0700, Steve Langasek wrote:
> > Please try the attached patch to slang.
> I did some cleanups on the patch.

Some further changes.

Bastian

--=20
One does not thank logic.
  -- Sarek, "Journey to Babel", stardate 3842.4

--0vzXIDBeUiKkjNJl
Content-Type: text/plain; charset=utf-8
Content-Disposition: attachment; filename=diff
Content-Transfer-Encoding: quoted-printable

diff -ur slang-1.4.9.orig/src/slsmg.c slang-1.4.9-utf8/src/slsmg.c
--- slang-1.4.9.orig/src/slsmg.c 2004-09-20 09:17:58.000000000 +0200
+++ slang-1.4.9-utf8/src/slsmg.c 2004-09-20 10:53:52.000000000 +0200
@@ -309,7 +309,8 @@
    if ((int) n < 0)
      return;
=20
- wptr =3D (wchar_t *)malloc(sizeof(wchar_t)*(n+1));
+ size_t wptr_len =3D n + 1;
+ wptr =3D malloc(sizeof(wchar_t) * wptr_len);
=20
    if (str !=3D NULL && wptr !=3D NULL)
    {
@@ -328,6 +329,14 @@
       str +=3D k;
       n -=3D w;
       i++;
+ /* We have some null-width chars in the string and need more space */
+ if (i >=3D wptr_len) {
+ wchar_t *new_wptr =3D realloc(wptr, sizeof(wchar_t) * wptr_len * 2);
+ if (!new_wptr)
+ break;
+ wptr_len *=3D 2;
+ wptr =3D new_wptr;
+ }
  }
  SLsmg_write_nwchars (wptr, i);
    }

--0vzXIDBeUiKkjNJl--

--zS7rBR6csb6tI2e1
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Digital signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.5 (GNU/Linux)

iEYEARECAAYFAkFOpoUACgkQnw66O/MvCNHWrgCfSLTv08f1G01pzaD3LcjgdSRX
dRAAn0RofHYmsbgqWz5OpXS7/hLt/xHj
=GtVp
-----END PGP SIGNATURE-----

--zS7rBR6csb6tI2e1--

[Debian Bug Importer (debzilla)]

Message-ID: <email address hidden>
Date: Mon, 20 Sep 2004 10:49:55 +0100
From: Martin Michlmayr <email address hidden>
To: <email address hidden>
Subject: Re: Processed: Re: Bug#272136: More progress about the "Arabic crash" in Debian Installer

reassign 256405 slang1a-utf8

--
Martin Michlmayr
<email address hidden>

[Debian Bug Importer (debzilla)]

Message-ID: <email address hidden>
Date: Mon, 20 Sep 2004 12:02:02 +0200
From: Bastian Blank <email address hidden>
To: <email address hidden>
Subject: Re: Processed: Re: Bug#272136: More progress about the "Arabic crash" in Debian Installer

reassign 256405 slang1a-utf8
reassign 272136 slang1a-utf8
--
You! What PLANET is this!
  -- McCoy, "The City on the Edge of Forever", stardate 3134.0

[Debian Bug Importer (debzilla)]

Marking as duplicate based on debbugs merge (256405,272136)

This bug has been marked as a duplicate of bug 8253.

[Debian Bug Importer (debzilla)]

Message-ID: <email address hidden>
Date: Mon, 20 Sep 2004 12:11:43 +0200
From: Christian Perrier <email address hidden>
To: Christian Perrier <email address hidden>, <email address hidden>
Cc: Bastian Blank <email address hidden>, Steve Langasek <email address hidden>, <email address hidden>
Subject: Re: Bug#272136: More progress about the "Arabic crash" in Debian Installer

reassign 256405 slang1a-utf8
reassign 272136 slang1a-utf8
merge 256405 272136
thanks

Assigning to he correct package...:-)

[Debian Bug Importer (debzilla)]

Message-Id: <email address hidden>
Date: Tue, 21 Sep 2004 16:17:13 -0400
From: Alastair McKinstry <email address hidden>
To: <email address hidden>
Subject: Bug#256405: fixed in slang 1.4.9dbs-8

Source: slang
Source-Version: 1.4.9dbs-8

We believe that the bug you reported is fixed in the latest version of
slang, which is due to be installed in the Debian FTP archive:

slang1-dev_1.4.9dbs-8_i386.deb
  to pool/main/s/slang/slang1-dev_1.4.9dbs-8_i386.deb
slang1-pic_1.4.9dbs-8_i386.deb
  to pool/main/s/slang/slang1-pic_1.4.9dbs-8_i386.deb
slang1-utf8-dev_1.4.9dbs-8_i386.deb
  to pool/main/s/slang/slang1-utf8-dev_1.4.9dbs-8_i386.deb
slang1-utf8-pic_1.4.9dbs-8_i386.deb
  to pool/main/s/slang/slang1-utf8-pic_1.4.9dbs-8_i386.deb
slang1_1.4.9dbs-8_i386.deb
  to pool/main/s/slang/slang1_1.4.9dbs-8_i386.deb
slang1a-utf8-udeb_1.4.9dbs-8_i386.udeb
  to pool/main/s/slang/slang1a-utf8-udeb_1.4.9dbs-8_i386.udeb
slang1a-utf8_1.4.9dbs-8_i386.deb
  to pool/main/s/slang/slang1a-utf8_1.4.9dbs-8_i386.deb
slang_1.4.9dbs-8.diff.gz
  to pool/main/s/slang/slang_1.4.9dbs-8.diff.gz
slang_1.4.9dbs-8.dsc
  to pool/main/s/slang/slang_1.4.9dbs-8.dsc

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed. If you
have further comments please address them to <email address hidden>,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Alastair McKinstry <email address hidden> (supplier of updated slang package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing <email address hidden>)

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Format: 1.7
Date: Tue, 21 Sep 2004 20:53:30 +0100
Source: slang
Binary: slang1-utf8-dev slang1-dev slang1a-utf8 slang1-utf8-pic slang1 slang1a-utf8-udeb slang1-pic
Architecture: source i386
Version: 1.4.9dbs-8
Distribution: unstable
Urgency: low
Maintainer: Jim Mintha <email address hidden>
Changed-By: Alastair McKinstry <email address hidden>
Description:
 slang1 - The S-Lang programming library - runtime version
 slang1-dev - The S-Lang programming library, development version
 slang1-pic - The S-Lang programming library, shared library subset kit
 slang1-utf8-dev - The S-Lang programming library, development version with utf8 sup
 slang1-utf8-pic - The S-Lang programming library, shared library subset with utf8 s
 slang1a-utf8 - The S-Lang programming library with utf8 support
 slang1a-utf8-udeb - S-Lang library with utf8 support (udeb)
Closes: 256405 272136
Changes:
 slang (1.4.9dbs-8) unstable; urgency=low
 .
   * Fix segfault on zero-width chars chars in Arabic. Closes: #256405, #272136.
Files:
 b579f71d6b51eeebc36171eb4045020b 727 devel optional slang_1.4.9dbs-8.dsc
 c5b9840181842204f4d3ba4e60ef8b24 88853 devel optional slang_1.4.9dbs-8.diff.gz
 f67df0d13d4b60410df30e0e6c1cb195 335036 devel optional slang1-dev_1.4.9dbs-8_i386.deb
 5cc72dc647267adbbe69de7a4f9d81be 295708 base standard slang1_1.4.9dbs-8_i386.deb
 953a2f9ea61c5822e9144559f81f8145 270966 libdevel optiona...

[Debian Bug Importer (debzilla)]

Message-Id: <email address hidden>
Date: Tue, 21 Sep 2004 16:17:13 -0400
From: Alastair McKinstry <email address hidden>
To: <email address hidden>
Subject: Bug#272136: fixed in slang 1.4.9dbs-8

Source: slang
Source-Version: 1.4.9dbs-8

We believe that the bug you reported is fixed in the latest version of
slang, which is due to be installed in the Debian FTP archive:

slang1-dev_1.4.9dbs-8_i386.deb
  to pool/main/s/slang/slang1-dev_1.4.9dbs-8_i386.deb
slang1-pic_1.4.9dbs-8_i386.deb
  to pool/main/s/slang/slang1-pic_1.4.9dbs-8_i386.deb
slang1-utf8-dev_1.4.9dbs-8_i386.deb
  to pool/main/s/slang/slang1-utf8-dev_1.4.9dbs-8_i386.deb
slang1-utf8-pic_1.4.9dbs-8_i386.deb
  to pool/main/s/slang/slang1-utf8-pic_1.4.9dbs-8_i386.deb
slang1_1.4.9dbs-8_i386.deb
  to pool/main/s/slang/slang1_1.4.9dbs-8_i386.deb
slang1a-utf8-udeb_1.4.9dbs-8_i386.udeb
  to pool/main/s/slang/slang1a-utf8-udeb_1.4.9dbs-8_i386.udeb
slang1a-utf8_1.4.9dbs-8_i386.deb
  to pool/main/s/slang/slang1a-utf8_1.4.9dbs-8_i386.deb
slang_1.4.9dbs-8.diff.gz
  to pool/main/s/slang/slang_1.4.9dbs-8.diff.gz
slang_1.4.9dbs-8.dsc
  to pool/main/s/slang/slang_1.4.9dbs-8.dsc

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed. If you
have further comments please address them to <email address hidden>,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Alastair McKinstry <email address hidden> (supplier of updated slang package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing <email address hidden>)

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Format: 1.7
Date: Tue, 21 Sep 2004 20:53:30 +0100
Source: slang
Binary: slang1-utf8-dev slang1-dev slang1a-utf8 slang1-utf8-pic slang1 slang1a-utf8-udeb slang1-pic
Architecture: source i386
Version: 1.4.9dbs-8
Distribution: unstable
Urgency: low
Maintainer: Jim Mintha <email address hidden>
Changed-By: Alastair McKinstry <email address hidden>
Description:
 slang1 - The S-Lang programming library - runtime version
 slang1-dev - The S-Lang programming library, development version
 slang1-pic - The S-Lang programming library, shared library subset kit
 slang1-utf8-dev - The S-Lang programming library, development version with utf8 sup
 slang1-utf8-pic - The S-Lang programming library, shared library subset with utf8 s
 slang1a-utf8 - The S-Lang programming library with utf8 support
 slang1a-utf8-udeb - S-Lang library with utf8 support (udeb)
Closes: 256405 272136
Changes:
 slang (1.4.9dbs-8) unstable; urgency=low
 .
   * Fix segfault on zero-width chars chars in Arabic. Closes: #256405, #272136.
Files:
 b579f71d6b51eeebc36171eb4045020b 727 devel optional slang_1.4.9dbs-8.dsc
 c5b9840181842204f4d3ba4e60ef8b24 88853 devel optional slang_1.4.9dbs-8.diff.gz
 f67df0d13d4b60410df30e0e6c1cb195 335036 devel optional slang1-dev_1.4.9dbs-8_i386.deb
 5cc72dc647267adbbe69de7a4f9d81be 295708 base standard slang1_1.4.9dbs-8_i386.deb
 953a2f9ea61c5822e9144559f81f8145 270966 libdevel optiona...