has issues with tying read only arrays (DB_File,O_RDONLY)

[Debian Bug Importer]

Automatically imported from Debian bug report #256314
http://bugs.debian.org/256314

[Debian Bug Importer (debzilla)]

Message-Id: <E1Be3QU-0007Cs-9e@riker>
Date: Fri, 25 Jun 2004 23:00:21 -0400
From: Jayen Ashar <email address hidden>
To: Debian Bug Tracking System <email address hidden>
Subject: has issues with tying read only arrays (DB_File,O_RDONLY)

--===============1405697569==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

Package: perl
Version: 5.8.3-3
Severity: grave

marked grave because it caused me data loss, though someone else may not
suffer the same fate.

If I try to tie an array read only, and the file does not exist, the
file is created (wrong), but the tie fails (right).

If I try to tie an array read only (allowing creation), and the file
does not exist, the file is created (right), but the tie is not read
only (wrong).

If I try to tie an array read only, and the file does exist, the tie
fails with no error (wrong).

If I try to tie an array read only (allowing creation), and the file
does exist, the tie succeeds (right), but the tie is not read only
(wrong).

So, when trying to tie an array read only, I can't...

-- System Information:
Debian Release: testing/unstable
  APT prefers testing
  APT policy: (500, 'testing')
Architecture: i386 (i686)
Kernel: Linux 2.6.6-1-k7
Locale: LANG=C, LC_CTYPE=C

Versions of packages perl depends on:
ii libc6 2.3.2.ds1-13 GNU C Library: Shared libraries an
ii libdb4.0 4.0.14-2 Berkeley v4.0 Database Libraries [
ii libgdbm3 1.8.3-2 GNU dbm database routines (runtime
ii perl-base 5.8.3-3 The Pathologically Eclectic Rubbis
ii perl-modules 5.8.3-3 Core Perl modules.

-- no debconf information

--===============1405697569==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename="perlbug.sh"

rm -f a
./perlnocreat.pl
ls --full-time a
sleep 1
rm -f a
/perlcreat.pl
ls --full-time a
sleep 1
rm -f a
touch a
/perlnocreat.pl
ls --full-time a
sleep 1
rm -f a
touch a
/perlcreat.pl
ls --full-time a

--===============1405697569==
Content-Type: application/x-perl
MIME-Version: 1.0
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="perlcreat.pl"

IyEvdXNyL2Jpbi9wZXJsIC13CnVzZSBEQl9GaWxlOwp0aWUgQGEsJ0RCX0ZpbGUnLCdhJyxPX1JE
T05MWXxPX0NSRUFULDA2MDAsJERCX1JFQ05PIG9yIHByaW50ICJjb3VsZCBub3Qgb3BlbiBhOiAk
IVxuIjsKJGFbMF09J2InOwp1bnRpZSBAYTsK

--===============1405697569==
Content-Type: application/x-perl
MIME-Version: 1.0
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="perlnocreat.pl"

IyEvdXNyL2Jpbi9wZXJsIC13CnVzZSBEQl9GaWxlOwp0aWUgQGEsJ0RCX0ZpbGUnLCdhJyxPX1JE
T05MWSwwNjAwLCREQl9SRUNOTyBvciBwcmludCAiY291bGQgbm90IG9wZW4gYTogJCFcbiI7CiRh
WzBdPSdiJzsKdW50aWUgQGE7Cg==

--===============1405697569==--

[Matt Zimmerman (mdz)]

Remove myself from all these CCs now that we have the warty-bugs mailing list

[Debian Bug Importer (debzilla)]

Message-ID: <email address hidden>
Date: Sat, 10 Jul 2004 16:57:39 +0200
From: Frank Lichtenheld <email address hidden>
To: <email address hidden>, <email address hidden>
Subject: Found the cause

>If I try to tie an array read only, and the file does not exist, the
>file is created (wrong), but the tie fails (right).

This behaviour is caused by the following code in DB_File.pm:

    # make recno in Berkeley DB version 2 (or better) work like
    # recno in version 1.
    if ($db_version > 1 and defined $arg[4] and $arg[4] =~ /RECNO/ and
        $arg[1] and ! -e $arg[1]) {
        open(FH, ">$arg[1]") or return undef ;
        close FH ;
        chmod $arg[3] ? $arg[3] : 0666 , $arg[1] ;
    }

So, it's a feature, not a bug ;)

>If I try to tie an array read only (allowing creation), and the file
>does not exist, the file is created (right), but the tie is not read
>only (wrong).

It seems that O_RDONLY == 0

Therefor it is impossible to specify O_RDONLY together with any other
flags. So O_CREAT overwrites O_RDONLY.

So this is a documentation bug, but the expected behaviour.

>If I try to tie an array read only, and the file does exist, the tie
>fails with no error (wrong).

Hmm, writing to a RDONLY array should probably cause an error, but it
doesn't. Seems like a bug, but I don't know enough about the Perl XS
interaction to spot the exact place where a change would be needed.

>If I try to tie an array read only (allowing creation), and the file
>does exist, the tie succeeds (right), but the tie is not read only
>(wrong).

Same cause as for case two.

Gruesse,
--
Frank Lichtenheld <email address hidden>
www: http://www.djpig.de/

[Debian Bug Importer (debzilla)]

Message-ID: <email address hidden>
Date: Sat, 10 Jul 2004 12:09:21 -0400 (EDT)
From: Jayen Ashar <email address hidden>
To: Frank Lichtenheld <email address hidden>
cc: <email address hidden>
Subject: Re: Bug#256314: Found the cause

> >If I try to tie an array read only, and the file does not exist, the
> >file is created (wrong), but the tie fails (right).
>
> This behaviour is caused by the following code in DB_File.pm:
>
> # make recno in Berkeley DB version 2 (or better) work like
> # recno in version 1.
> if ($db_version > 1 and defined $arg[4] and $arg[4] =~ /RECNO/ and
> $arg[1] and ! -e $arg[1]) {
> open(FH, ">$arg[1]") or return undef ;
> close FH ;
> chmod $arg[3] ? $arg[3] : 0666 , $arg[1] ;
> }
>
> So, it's a feature, not a bug ;)

Okay, so it creates the file on purpose. But why does the tie fail?

> >If I try to tie an array read only, and the file does exist, the tie
> >fails with no error (wrong).
>
> Hmm, writing to a RDONLY array should probably cause an error, but it
> doesn't. Seems like a bug, but I don't know enough about the Perl XS
> interaction to spot the exact place where a change would be needed.
>

I didn't get to the point where I write yet. I'm just talking about
tying. Why does the tie fail? And why isn't there an error?

--Jayen

-----------
Jayen Ashar
Verification Engineer, IBM
ECE+CS+robotics+languages, CMU '02

[Debian Bug Importer (debzilla)]

Message-ID: <email address hidden>
Date: Sat, 10 Jul 2004 19:15:12 +0200
From: Frank Lichtenheld <email address hidden>
To: Jayen Ashar <email address hidden>
Cc: <email address hidden>
Subject: Re: Bug#256314: Found the cause

On Sat, Jul 10, 2004 at 12:09:21PM -0400, Jayen Ashar wrote:
> > >If I try to tie an array read only, and the file does not exist, the
> > >file is created (wrong), but the tie fails (right).
> >
> > This behaviour is caused by the following code in DB_File.pm:
> >
> > # make recno in Berkeley DB version 2 (or better) work like
> > # recno in version 1.
> > if ($db_version > 1 and defined $arg[4] and $arg[4] =~ /RECNO/ and
> > $arg[1] and ! -e $arg[1]) {
> > open(FH, ">$arg[1]") or return undef ;
> > close FH ;
> > chmod $arg[3] ? $arg[3] : 0666 , $arg[1] ;
> > }
> >
> > So, it's a feature, not a bug ;)
>
> Okay, so it creates the file on purpose. But why does the tie fail?
>
> > >If I try to tie an array read only, and the file does exist, the tie
> > >fails with no error (wrong).
> >
> > Hmm, writing to a RDONLY array should probably cause an error, but it
> > doesn't. Seems like a bug, but I don't know enough about the Perl XS
> > interaction to spot the exact place where a change would be needed.
> >
>
> I didn't get to the point where I write yet. I'm just talking about
> tying. Why does the tie fail? And why isn't there an error?

For me none of the ties failed... Sometimes it could write to the file,
sometimes not, but the tie never failed.

Gruesse,
--
Frank Lichtenheld <email address hidden>
www: http://www.djpig.de/

[Debian Bug Importer (debzilla)]

Message-Id: <email address hidden>
Date: Sun, 11 Jul 2004 12:08:45 +0200
From: Frank Lichtenheld <email address hidden>
To: <email address hidden>
Subject: tagging 256314

# Automatically generated email from bts, devscripts version 2.7.95.1
 # There may be some minor bugs to find in this bug report but the tie failures the submitter reports are unreproducible for me
tags 256314 moreinfo unreproducible

[Debian Bug Importer (debzilla)]

Message-ID: <email address hidden>
Date: Sun, 11 Jul 2004 12:54:23 -0400 (EDT)
From: Jayen Ashar <email address hidden>
To: Frank Lichtenheld <email address hidden>
cc: <email address hidden>
Subject: Re: Bug#256314: Found the cause

Okay, I got it. Though perl depends on libdb4.0, other packages (apache
apache-common apache-ssl apache-utils apt-build apt-utils bittorrent
libdb4.2 libdb4.2-dev libwxgtk2.4-python python python2.3
reportbug) depend on libdb4.2, so even though perl was built with the 4.0
headers & libs, it's linking against the 4.2 libs at runtime, causing the
open command to fail since an argument was removed from the db->open
function call.

Now my question is, how do I fix this?

--Jayen

On Sun, 11 Jul 2004, Jayen Ashar wrote:

> > Yeah, but set_re_source already specifies the filename for the later
> > open call.
> > See http://www.sleepycat.com/docs/api_c/db_set_re_source.html for more
> > information.
>
> just for kicks, i tried adding these lines around the set_re_source call:
> --> char*nameP;
> status = dbp->set_re_source(dbp, name) ;
> --> status = dbp->get_re_source(dbp, &nameP) ;
>
> and i got this error:
> DB_File.xs: In function `ParseOpenInfo':
> DB_File.xs:1393: error: structure has no member named `get_re_source'
>
> perhaps I have some weird version of some package? i just checked what
> libdb packages i have, and i think there's prolly something wrong here:
> ii libdb1-compat 2.1.3-7 The Berkeley database routines [glibc 2.0/2.
> ii libdb2 2.7.7.0-9 The Berkeley database routines (run-time fil
> ii libdb3 3.2.9-20 Berkeley v3 Database Libraries [runtime] ii libdb3-util 3.2.9-20 Berkeley v3 Database Utilities
> ii libdb4.0 4.0.14-2 Berkeley v4.0 Database Libraries [runtime]
> ii libdb4.0-dev 4.0.14-2 Berkeley v4.0 Database Libraries [developmen
> ii libdb4.1 4.1.25-17 Berkeley v4.1 Database Libraries [runtime]
> ii libdb4.2 4.2.52-16 Berkeley v4.2 Database Libraries [runtime]
> ii libdbd-mysql-p 2.9003-2 A Perl5 database interface to the MySQL data
> ii libdbi-perl 1.41-1 The Perl5 Database Interface by Tim Bunce
>
> > I still can't produce any error messages like this. You are using the
> > same scripts for testing you send to the bug report?
> > Please send me the full output of the scripts after enabling all printf
> > calls in DB_File.xs
>
> that's odd. i had this problem on one machine, and was able to repro it
> without a problem on another machine. i am using only the perlnocreat.pl
> script now, since i understand why the perlcreat.pl script overrides the
> readonly flag.
>
> the printf's don't provide much info, but here they are:
> In ParseOpenInfo name=[a] flags=[0] mode = [384]
> db_create returned 0 Successful return: 0
> open returned 22 Invalid argument
>
> I added a bunch more debugging statements (for my own info), so i am
> attaching my db_file.xs and here is my output from running perlnocreat.pl:
> DB_File.xs: 1508: isHASH: 0x0 00 0
> DB_File.xs: 1509: dbtype: DB_File
> DB_File.xs: 1510: flags: 0x...

[Debian Bug Importer (debzilla)]

Message-ID: <email address hidden>
Date: Sun, 11 Jul 2004 19:35:45 +0200
From: Frank Lichtenheld <email address hidden>
To: Jayen Ashar <email address hidden>
Cc: <email address hidden>
Subject: Re: Bug#256314: Found the cause

On Sun, Jul 11, 2004 at 12:54:23PM -0400, Jayen Ashar wrote:
> Okay, I got it. Though perl depends on libdb4.0, other packages (apache
> apache-common apache-ssl apache-utils apt-build apt-utils bittorrent
> libdb4.2 libdb4.2-dev libwxgtk2.4-python python python2.3
> reportbug) depend on libdb4.2, so even though perl was built with the 4.0
> headers & libs, it's linking against the 4.2 libs at runtime, causing the
> open command to fail since an argument was removed from the db->open
> function call.
>
> Now my question is, how do I fix this?

Perl in sid is compiled with 4.2. Sit and wait...

Gruesse,
--
Frank Lichtenheld <email address hidden>
www: http://www.djpig.de/

[Debian Bug Importer (debzilla)]

Message-Id: <email address hidden>
Date: Sun, 11 Jul 2004 19:43:10 +0200
From: Frank Lichtenheld <email address hidden>
To: <email address hidden>
Subject: tagging 256314, tagging 256314

tag 256314 - unreproducible
 # getting nearer, the bug only seems to affect perl in sarge and is probably related to the libdb4.0 vs. 4.2
tag 256314 + sarge

[Matt Zimmerman (mdz)]

Increase severity of RC bugs to major, now that we have other, non-RC bugs in
the list

[Fabio Massimo Di Nitto (fabbione)]

User error.

[Debian Bug Importer (debzilla)]

Message-ID: <email address hidden>
Date: Mon, 26 Jul 2004 08:56:11 -0400 (EDT)
From: Jayen Ashar <email address hidden>
To: Frank Lichtenheld <email address hidden>
cc: <email address hidden>
Subject: Re: Bug#256314: Found the cause

Is there anything (like a debian policy or something) to prevent/check
this problem in other packages? Like maybe multiple versions of libdb
shouldn't be allowed on a system? Or libdb<4.1 should conflict with
libdb>=4.1? Or perl/DB_File should be linked against a specific
version of libdb? Can this be generalized to other packages/libraries?

Just curious.

--Jayen

On Sun, 11 Jul 2004, Frank Lichtenheld wrote:

> On Sun, Jul 11, 2004 at 12:54:23PM -0400, Jayen Ashar wrote:
> > Okay, I got it. Though perl depends on libdb4.0, other packages (apache
> > apache-common apache-ssl apache-utils apt-build apt-utils bittorrent
> > libdb4.2 libdb4.2-dev libwxgtk2.4-python python python2.3
> > reportbug) depend on libdb4.2, so even though perl was built with the 4.0
> > headers & libs, it's linking against the 4.2 libs at runtime, causing the
> > open command to fail since an argument was removed from the db->open
> > function call.
> >
> > Now my question is, how do I fix this?
>
> Perl in sid is compiled with 4.2. Sit and wait...
>
> Gruesse,
> --
> Frank Lichtenheld <email address hidden>
> www: http://www.djpig.de/
>

[Debian Bug Importer (debzilla)]

Message-ID: <email address hidden>
Date: Mon, 26 Jul 2004 14:11:22 +0100
From: Andrew Suffield <email address hidden>
To: Jayen Ashar <email address hidden>, <email address hidden>
Cc: Frank Lichtenheld <email address hidden>
Subject: Re: Bug#256314: Found the cause

--17pEHd4RhPHOinZp
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Mon, Jul 26, 2004 at 08:56:11AM -0400, Jayen Ashar wrote:
> Is there anything (like a debian policy or something) to prevent/check
> this problem in other packages?

No. It's supposed to work, not break. Nobody's figured out why it
didn't work in this case, AFAIK.

> > On Sun, Jul 11, 2004 at 12:54:23PM -0400, Jayen Ashar wrote:
> > > Okay, I got it. Though perl depends on libdb4.0, other packages (apa=
che
> > > apache-common apache-ssl apache-utils apt-build apt-utils bittorrent
> > > libdb4.2 libdb4.2-dev libwxgtk2.4-python python python2.3
> > > reportbug) depend on libdb4.2, so even though perl was built with the=
 4.0
> > > headers & libs, it's linking against the 4.2 libs at runtime, causing=
 the
> > > open command to fail since an argument was removed from the db->open
> > > function call.

--=20
  .''`. ** Debian GNU/Linux ** | Andrew Suffield
 : :' : http://www.debian.org/ |
 `. `' |
   `- -><- |

--17pEHd4RhPHOinZp
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Digital signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)

iD8DBQFBBQL6lpK98RSteX8RAoNcAJ4xtY9tIHiNWrUHyyq6MlEIlZRNnACfZV6X
SMaE41Zhg1ny0OKCZ4jNDg4=
=Zg17
-----END PGP SIGNATURE-----

--17pEHd4RhPHOinZp--

[Debian Bug Importer (debzilla)]

Message-ID: <email address hidden>
Date: Tue, 27 Jul 2004 08:26:45 -0400 (EDT)
From: Jayen Ashar <email address hidden>
To: Andrew Suffield <email address hidden>
cc: <email address hidden>, Frank Lichtenheld <email address hidden>
Subject: Re: Bug#256314: Found the cause

Has anyone else been able to repro this on testing? Or is it just me?

--Jayen

On Mon, 26 Jul 2004, Andrew Suffield wrote:

> On Mon, Jul 26, 2004 at 08:56:11AM -0400, Jayen Ashar wrote:
> > Is there anything (like a debian policy or something) to prevent/check
> > this problem in other packages?
>
> No. It's supposed to work, not break. Nobody's figured out why it
> didn't work in this case, AFAIK.
>
> > > On Sun, Jul 11, 2004 at 12:54:23PM -0400, Jayen Ashar wrote:
> > > > Okay, I got it. Though perl depends on libdb4.0, other packages (apache
> > > > apache-common apache-ssl apache-utils apt-build apt-utils bittorrent
> > > > libdb4.2 libdb4.2-dev libwxgtk2.4-python python python2.3
> > > > reportbug) depend on libdb4.2, so even though perl was built with the 4.0
> > > > headers & libs, it's linking against the 4.2 libs at runtime, causing the
> > > > open command to fail since an argument was removed from the db->open
> > > > function call.
>
> --
> .''`. ** Debian GNU/Linux ** | Andrew Suffield
> : :' : http://www.debian.org/ |
> `. `' |
> `- -><- |
>

[Debian Bug Importer (debzilla)]

Message-Id: <email address hidden>
Date: Fri, 30 Jul 2004 13:38:10 +0200
From: Kjetil Kjernsmo <email address hidden>
To: <email address hidden>
CC: <email address hidden>
Subject: Re: Bug#256314: Reproduction attempt on Testing

> Has anyone else been able to repro this on testing? Or is it just me?

Well, I don't really grok this bug, but since I have a testing box,
Perl is an important package to me, and you have made it so easy to
try out, I figured I could try to make myself useful.

So, I downloaded your scripts, made some tiny modifications (put
'perl' ahead of the perl script calls), and this is the output:

test@owl:~$ sh perlbug.sh
could not open a: Inappropriate ioctl for device
-rw------- 1 test test 0 2004-07-30 13:28:05.832858152 +0200 a
-rw------- 1 test test 2 2004-07-30 13:28:06.924692168 +0200 a
could not open a:
-rw-r--r-- 1 test test 0 2004-07-30 13:28:07.943537280 +0200 a
-rw-r--r-- 1 test test 2 2004-07-30 13:28:09.026372664 +0200 a

Cheers,

Kjetil

-- System Information:
Debian Release: testing/unstable
  APT prefers testing
  APT policy: (990, 'testing')
Architecture: i386 (i686)
Kernel: Linux 2.6.7-ruby.2004-07-25.owl.1.oss
Locale: LANG=C, LC_CTYPE=C

Versions of packages perl depends on:
ii libc6 2.3.2.ds1-13 GNU C Library: Shared libraries an
ii libdb4.0 4.0.14-2 Berkeley v4.0 Database Libraries [
ii libgdbm3 1.8.3-2 GNU dbm database routines (runtime
ii perl-base 5.8.3-3 The Pathologically Eclectic Rubbis
ii perl-modules 5.8.3-3 Core Perl modules.

-- no debconf information

[Debian Bug Importer (debzilla)]

Message-Id: <email address hidden>
Date: Mon, 2 Aug 2004 14:03:24 +0200
From: Frank Lichtenheld <email address hidden>
To: <email address hidden>
Subject: tagging 256314, severity of 256314 is important

tag 256314 - sarge
 # isn't reproducible anymore as perl 5.8.4 is in sarge, but it is still not clear where the bug was so not closing for now
severity 256314 important

[Debian Bug Importer (debzilla)]

Message-Id: <E1BwLut-0000Op-At@riker>
Date: Sun, 15 Aug 2004 10:23:23 -0400
From: Jayen Ashar <email address hidden>
To: Debian Bug Tracking System <email address hidden>
Subject: 256314 with moreinfo tag? should be closed?

Package: perl
Version: 5.8.4-2
Followup-For: Bug #256314

since the version of perl that this bug was filed against is no longer in testing, should this bug be closed? what is the moreinfo tag for? i'm not sure what information you are looking for me to provide?

-- System Information:
Debian Release: 3.1
Architecture: i386 (i686)
Kernel: Linux 2.6.7-1-k7
Locale: LANG=C, LC_CTYPE=C

Versions of packages perl depends on:
ii libc6 2.3.2.ds1-13 GNU C Library: Shared libraries an
ii libdb4.2 4.2.52-16 Berkeley v4.2 Database Libraries [
ii libgdbm3 1.8.3-2 GNU dbm database routines (runtime
ii perl-base 5.8.4-2 The Pathologically Eclectic Rubbis
ii perl-modules 5.8.4-2 Core Perl modules.

-- no debconf information