mblen() failing in Perl / Perl core dumping core on UBUNTU 19.04 by executing perl script, multiple architectures

[bugproxy]

== Comment: #0 - NAGENDRA P. DONTAMSETTY <email address hidden> - 2019-02-28 00:14:49 ==
---Problem Description---
Perl core dumping core on UBUNTU 19.04 by executing perl script

---uname output---
root@p8ct1p13:/tmp# uname -a Linux p8ct1p13.in.ibm.com 4.19.0-13-generic #14-Ubuntu SMP Thu Feb 7 21:50:00 UTC 2019 ppc64le ppc64le ppc64le GNU/Linux

Machine Type = ppc64le and power8

---Debugger Data---
root@p8ct1p13:/tmp# file core
core: ELF 64-bit LSB core file, 64-bit PowerPC or cisco 7500, version 1 (SYSV), SVR4-style, from '/usr/bin/perl /usr/bin/mkrsrc IBM.Ray Name=fvt1 NodeNameList={p8ct1p09.in.ibm.c', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/usr/bin/mkrsrc', platform: 'power8'

---Steps to Reproduce---
 Description: Perl core dumpinmg core on UBUNTU 19.04 by exec cmd "mkrsrc"

root@p8ct1p13:/tmp# uname -a
Linux p8ct1p13.in.ibm.com 4.19.0-13-generic #14-Ubuntu SMP Thu Feb 7 21:50:00 UTC 2019 ppc64le ppc64le ppc64le GNU/Linux

root@p8ct1p13:~# cat /etc/os-release
NAME="Ubuntu"
VERSION="19.04 (Disco Dingo)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu Disco Dingo (development branch)"
VERSION_ID="19.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=disco
UBUNTU_CODENAME=disco

root@p8ct1p13:~# ctversion -bv
RSCT_Build_Name=rsholxs002a 3.2.4.2 RSCT_Build_Time=19043.16:11:49 RSCT_Build_Context=ppc64le_linux_2

root@p8ct1p13:/tmp# mkrsrc IBM.Ray Name="fvt1" NodeNameList={"p8ct1p09.in.ibm.com"} ManualMode=0 Int32=00 String="Initial Test String 2"
perl: mbrtowc.c:105: __mbrtowc: Assertion `__mbsinit (data.__statep)' failed.
Aborted (core dumped)

root@p8ct1p13:/tmp# file core
core: ELF 64-bit LSB core file, 64-bit PowerPC or cisco 7500, version 1 (SYSV), SVR4-style, from '/usr/bin/perl /usr/bin/mkrsrc IBM.Ray Name=fvt1 NodeNameList={p8ct1p09.in.ibm.c', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/usr/bin/mkrsrc', platform: 'power8'

root@p8ct1p13:/tmp# which mkrsrc
/usr/bin/mkrsrc

root@p8ct1p13:/tmp# file /usr/bin/mkrsrc

/usr/bin/mkrsrc: symbolic link to /opt/rsct/bin/mkrsrc

root@p8ct1p13:/tmp# file /opt/rsct/bin/mkrsrc
/opt/rsct/bin/mkrsrc: Perl script text executable

Contact Information = Nagendra <email address hidden>, <email address hidden>

Userspace tool common name: perl 5

The userspace tool has the following bit modes: 64bit

Userspace rpm: ii perl 5.28.1-4 ppc64el

Userspace tool obtained from project website: na

*Additional Instructions for Nagendra <email address hidden>, <email address hidden>:
-Post a private note with access information to the machine that is currently in the debugger.
-Attach ltrace and strace of userspace application.

[Dimitri John Ledkov (xnox)]

/usr/bin/mkrsrc: symbolic link to /opt/rsct/bin/mkrsrc

but that's not provided by ubuntu, is it?

In general, perl does work on ppc64le on ubuntu, as we execute thousands of perl autopkgtests with every perl update, or any reverse dependency update. You can see that perl was proven to be working normally on ppc64le as recently as yesterday http://autopkgtest.ubuntu.com/packages/perl/disco/ppc64el

Is the third party tool incompatible with perl 5.28.1-4 in some ways? and/or needs to be rebuild against newer perl ABI?

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2019-03-07 07:10 EDT-------
>Is the third party tool incompatible with perl 5.28.1-4 in some ways? and/or needs to be rebuild against newer perl ABI?

It is just a standard normal perl script. (Not any compiled version or anything like that).
You can see the content of mkrsrc on the machine.

If it is incompatible, the question is why any perl scripts may become incompatible with new perl?
Could you explain what is causing it?

If there is any syntax error, perl should generate an error not failing.

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2019-03-09 10:53 EDT-------
In fact, this can be easily reproduced using a simple perl script.

test.pl
----------
#!//usr/bin/perl

use POSIX qw(mblen);

my $str = "data";

print "Calling mblen with str=$str\n";
my $len = mblen($str, MB_CUR_MAX);
print "mblen returned with len=$len\n";

[bugproxy (bugproxy)]

------- Comment on attachment From <email address hidden> 2019-03-09 10:52 EDT-------

This perl script will cause an error on Ubuntu 19.04:

# perl test.pl
Calling mblen with str=data
perl: mbrtowc.c:105: __mbrtowc: Assertion `__mbsinit (data.__statep)' failed.
Aborted

[Dimitri John Ledkov (xnox)]

ubuntu@baltar:~$ lscpu
Architecture: ppc64le
Byte Order: Little Endian
CPU(s): 160
On-line CPU(s) list: 0-159
Thread(s) per core: 4
Core(s) per socket: 20
Socket(s): 2
NUMA node(s): 2
Model: 2.2 (pvr 004e 1202)
Model name: POWER9, altivec supported
CPU max MHz: 3800.0000
CPU min MHz: 2166.0000
L1d cache: 32K
L1i cache: 32K
L2 cache: 512K
L3 cache: 10240K
NUMA node0 CPU(s): 0-79
NUMA node8 CPU(s): 80-159
ubuntu@baltar:~$ cat test.pl
#!//usr/bin/perl

use POSIX qw(mblen);

my $str = "data";

print "Calling mblen with str=$str\n";
my $len = mblen($str, MB_CUR_MAX);
print "mblen returned with len=$len\n";

ubuntu@baltar:~$ ./test.pl
Calling mblen with str=data
mblen returned with len=-2

Not sure if the result of "-2" is expected, or not. From my expectations of what mblen() is supposed to do, that's a wrong answer....
And that was POWER9, will redo on POWER8 now.

[Dimitri John Ledkov (xnox)]

On an amd64 machine, I do get a core dump:

xnox@ottawa:~$ lscpu
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
Address sizes: 39 bits physical, 48 bits virtual
CPU(s): 8
On-line CPU(s) list: 0-7
Thread(s) per core: 2
Core(s) per socket: 4
Socket(s): 1
NUMA node(s): 1
Vendor ID: GenuineIntel
CPU family: 6
Model: 94
Model name: Intel(R) Core(TM) i7-6700HQ CPU @ 2.60GHz
Stepping: 3
CPU MHz: 800.003
CPU max MHz: 3500.0000
CPU min MHz: 800.0000
BogoMIPS: 5184.00
Virtualization: VT-x
L1d cache: 32K
L1i cache: 32K
L2 cache: 256K
L3 cache: 6144K
NUMA node0 CPU(s): 0-7
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb invpcid_single pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx rdseed adx smap clflushopt intel_pt xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp flush_l1d
xnox@ottawa:~$ cat test.pl
#!//usr/bin/perl

use POSIX qw(mblen);

my $str = "data";

print "Calling mblen with str=$str\n";
my $len = mblen($str, MB_CUR_MAX);
print "mblen returned with len=$len\n";

xnox@ottawa:~$ ./test.pl
Calling mblen with str=data
perl: mbrtowc.c:105: __mbrtowc: Assertion `__mbsinit (data.__statep)' failed.
Aborted (core dumped)

[Dimitri John Ledkov (xnox)]

Sample C usage of mblen() appears to be fine, so libc is probably fine.

[Dimitri John Ledkov (xnox)]

bah, mbrtowc claims to fail, tested that with C too, that works too.
And I cannot understand how perl XS stuff works. But it is fairly recent that mbrtowc was started to be used.

See: 6de6aebdf89cb5abd8296cf686184e4b9461d11b

[Steve Langasek (vorlon)]

Dimitri didn't post results from a power8 system, but here's what I see in a Power8 VM, with up-to-date 19.04:

$ lscpu
Architecture: ppc64le
Byte Order: Little Endian
CPU(s): 1
On-line CPU(s) list: 0
Thread(s) per core: 1
Core(s) per socket: 1
Socket(s): 1
NUMA node(s): 1
Model: 2.1 (pvr 004b 0201)
Model name: POWER8E (raw), altivec supported
Hypervisor vendor: KVM
Virtualization type: para
L1d cache: 64K
L1i cache: 32K
NUMA node0 CPU(s): 0
$ ./test.pl
Calling mblen with str=data
mblen returned with len=-2
$

Same behavior with any of C, C.UTF-8, en_US.UTF-8 as the configured locale.

[Niko Tyni (ntyni)]

On Sat, Mar 09, 2019 at 03:59:22PM -0000, bugproxy wrote:

> This perl script will cause an error on Ubuntu 19.04:
>
> # perl test.pl
> Calling mblen with str=data
> perl: mbrtowc.c:105: __mbrtowc: Assertion `__mbsinit (data.__statep)' failed.
> Aborted

I've reported this bug upstream at

  https://rt.perl.org/Ticket/Display.html?id=133928

with a proposed patch.

It's also https://bugs.debian.org/924517

--
Niko Tyni <email address hidden>

[Dimitri John Ledkov (xnox)]

@ntyni excellent analysis! i like it a lot, and the patch is clean and easy to understand.

Are you going to make a debian upload soon, or should I cherrypick that for Ubuntu now?

[Niko Tyni (ntyni)]

On Thu, Mar 14, 2019 at 12:25:22AM -0000, Dimitri John Ledkov wrote:
> @ntyni excellent analysis! i like it a lot, and the patch is clean and
> easy to understand.
>
> Are you going to make a debian upload soon, or should I cherrypick that
> for Ubuntu now?

Fixed in Debian 5.28.1-5, which I've just uploaded.
--
Niko

[Steve Langasek (vorlon)]

On Sat, Mar 16, 2019 at 04:53:35PM -0000, Niko Tyni wrote:
> On Thu, Mar 14, 2019 at 12:25:22AM -0000, Dimitri John Ledkov wrote:
> > @ntyni excellent analysis! i like it a lot, and the patch is clean and
> > easy to understand.

> > Are you going to make a debian upload soon, or should I cherrypick that
> > for Ubuntu now?

> Fixed in Debian 5.28.1-5, which I've just uploaded.

Thanks, I've synced this to Ubuntu but unfortunately it's failing on the
newly-added arch-specific tests for NDBM:

autopkgtest [11:12:06]: test ndbm: [-----------------------
testing NDBM reading...
buster-ndbm OK
opening NDBM file failed: No such file or directory at -e line 1.
autopkgtest [11:12:06]: test ndbm: -----------------------]
ndbm FAIL non-zero exit status 1
autopkgtest [11:12:07]: test ndbm: - - - - - - - - - - results - - - - - - - - - -
autopkgtest [11:12:07]: test ndbm: - - - - - - - - - - stderr - - - - - - - - - -
opening NDBM file failed: No such file or directory at -e line 1.

So this is currently blocked in disco-proposed.

--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer https://www.debian.org/
<email address hidden> <email address hidden>

[Niko Tyni (ntyni)]

On Mon, Mar 18, 2019 at 05:48:51PM -0000, Steve Langasek wrote:
> On Sat, Mar 16, 2019 at 04:53:35PM -0000, Niko Tyni wrote:
> > On Thu, Mar 14, 2019 at 12:25:22AM -0000, Dimitri John Ledkov wrote:
> > > @ntyni excellent analysis! i like it a lot, and the patch is clean and
> > > easy to understand.
>
> > > Are you going to make a debian upload soon, or should I cherrypick that
> > > for Ubuntu now?
>
> > Fixed in Debian 5.28.1-5, which I've just uploaded.
>
> Thanks, I've synced this to Ubuntu but unfortunately it's failing on the
> newly-added arch-specific tests for NDBM:

Sorry about the inconvenience and thanks for noting. I've filed
https://bugs.debian.org/925179 about this.
--
Niko Tyni <email address hidden>

[Manoj Iyer (manjo)]

@steve, look like its now fixed in Debian 5.28.1-6 and we are close to beta freeze, are we going to cherry-pick this one?

[Dimitri John Ledkov (xnox)]

@manjo as you can see from launchpad and proposed-updates report it has been synced 17h ago, built and stuck in disco-proposed with autopkgtest regressions, and awaiting results for many more.

[Andrew Cloke (andrew-cloke)]

Great. Thanks Dimitri. We'd missed that upload.

[Adam Conrad (adconrad)]

This is fixed in perl 5.28.1-6, now in the disco release pocket.