Bug #613352 “wbDo is broken” : Bugs : Woda

Malcolm Fitzgerald (malcolm-notyourhomework) on 2010-08-04

Changed in woda:
importance:	Undecided → High

Revision history for this message

Xavier Brochard (xavier) wrote on 2010-08-06: Re: [Bug 613352] [NEW] wbDo is broken

#1

known bug, already fixed in trunk and in 4.620-xavier branch

see revision 75 (2009-10-09):
"Add require instead of nothing (was do) in sub wbDo. It works without but it
shouldn't, according to perl doc"

Xavier
<email address hidden> - 09 54 06 16 26

Revision history for this message

Xavier Brochard (xavier) wrote on 2010-08-07:

#2

already fix in trunk revision 75 (2009-10-09):
"Add require instead of nothing (was do) in sub wbDo. It works without but it shouldn't, according to perl doc"

Changed in woda:
assignee:	nobody → Xavier Brochard (xavier)
status:	New → Fix Released

Revision history for this message

Malcolm Fitzgerald (malcolm-notyourhomework) wrote on 2010-08-07:

#3

Hi Xavier,

We should push out a new version for download that has this bug and the security hole fixed. These two are very important. The security bug is most serious but the wbDo bug affects QRY, ROWS, FETCH because Xsearch is affected, and all the subroutines that I listed in the email.

p.s. I finally discovered the problem. It had a number of causes but in the end it was my fault for not taking more care at setup.

# 1 I used v4_610 from launchpad - so it had the wbDo bug.
# 2 The cPanel was not displaying errors in the error log and the web host had compiled grep into an unusual location
# 3 I didn't read the status report closely - even though I looked at it dozens of times. It said clearly could not execute grep.

It was only after I knew that there should be errors being generated by the apache logs that I requested the host provide me with full error logs for the previous week. I opened them up and saw the log was full of "cannot execute program /usr/local/bin/grep". I ran 'which grep' to obtain the location and corrected the path in woda.conf.

To make it harder to trouble shoot I had three definition files all pointing at the same data source. This is where I was stupid. I copied and pasted the entire definition, including WBF. After doing that I had requests for changes to the fields. I did that but as Richard noticed, I had duplicated one of the fields in one definition file. This meant that the sort order for the fields was different in each of the definition files! That was bad. I moved all the fields out into their own file named WBF.pl and used require to call them in. This cleared up a number of unusual issues. The following day I found the problem with grep and the databases began working properly.

This reminds me that I improved AdmStatus. I have to put more effort into my updates on launchpad. I have a version that runs "which" if the program location given in woda.conf fails. I changed the error to display clearly in bold, red text and then to display the correct location.

Malcolm

Hi Xavier,

We should push out a new version for download that has this bug and the security hole fixed. These two are very important. The security bug is most serious but the wbDo bug affects QRY, ROWS, FETCH because Xsearch is affected, and all the subroutines that I listed in the email.

p.s. I finally discovered the problem. It had a number of causes but in the end it was my fault for not taking more care at setup.

# 1 I used v4_610 from launchpad - so it had the wbDo bug.
# 2 The cPanel was not displaying errors in the error log and the web host had compiled grep into an unusual location
# 3 I didn't read the status report closely - even though I looked at it dozens of times. It said clearly could not execute grep.

It was only after I knew that there should be errors being generated by the apache logs that I requested the host provide me with full error logs for the previous week. I opened them up and saw the log was full of  "cannot execute program /usr/local/bin/grep". I ran 'which grep' to obtain the location and corrected the path in woda.conf.

To make it harder to trouble shoot I had three definition files all pointing at the same data source. This is where I was stupid. I copied and pasted the entire definition, including WBF. After doing that I had requests for changes to the fields. I did that but as Richard noticed, I had duplicated one of the fields in one definition file. This meant that  the sort order for the fields was different in each of the definition files! That was bad. I moved all the fields out into their own file named WBF.pl and used require to call them in.  This cleared up a number of unusual issues. The following day I found the problem with grep and the databases began working properly.

This reminds me that I improved AdmStatus. I have to put more effort into my updates on launchpad. I have a version that runs "which" if the program location given in woda.conf fails. I changed the error to display clearly in bold, red text and then to display the correct location.

Malcolm

Revision history for this message

Xavier Brochard (xavier) wrote on 2010-08-14:

#4

Le samedi 7 août 2010 15:37:08, vous avez écrit :
> We should push out a new version for download that has this bug and the
> security hole fixed. These two are very important. The security bug is
> most serious but the wbDo bug affects QRY, ROWS, FETCH because Xsearch
> is affected, and all the subroutines that I listed in the email.

agree, could you do that? I'm very busy

> p.s. I finally discovered the problem. It had a number of causes but in
> the end it was my fault for not taking more care at setup.
>
> # 1 I used v4_610 from launchpad - so it had the wbDo bug.
> # 2 The cPanel was not displaying errors in the error log and the web host
> had compiled grep into an unusual location # 3 I didn't read the status
> report closely - even though I looked at it dozens of times. It said
> clearly could not execute grep.

The status and debug page are not very readable and sometimes hard to find.
We should work on that, because it is the first information one should see
after a fresh install.

> It was only after I knew that there should be errors being generated by
> the apache logs that I requested the host provide me with full error
> logs for the previous week. I opened them up and saw the log was full of
> "cannot execute program /usr/local/bin/grep". I ran 'which grep' to
> obtain the location and corrected the path in woda.conf.
>
> To make it harder to trouble shoot I had three definition files all
> pointing at the same data source. This is where I was stupid. I copied
> and pasted the entire definition, including WBF.

I allways use a "require" to load one or two common setup file: common setup
for tables and common fields (as it was done in Scix/Sops).

> After doing that I had
> requests for changes to the fields. I did that but as Richard noticed, I
> had duplicated one of the fields in one definition file. This meant that
> the sort order for the fields was different in each of the definition
> files! That was bad. I moved all the fields out into their own file
> named WBF.pl and used require to call them in. This cleared up a number
> of unusual issues. The following day I found the problem with grep and
> the databases began working properly.

>
> This reminds me that I improved AdmStatus. I have to put more effort
> into my updates on launchpad. I have a version that runs "which" if the
> program location given in woda.conf fails. I changed the error to
> display clearly in bold, red text and then to display the correct
> location.

can you send it to me?
I would like to test. By putting it in the contrib folder it should replace
the current cgi.

We should use this feature (the contrib folder) to test improved or new
functions with the help of the list.

Xavier
<email address hidden> - 09 54 06 16 26

Le samedi 7 août 2010 15:37:08, vous avez écrit :
> We should push out a new version for download that has this bug and the
> security hole fixed. These two are very important. The security bug is
> most serious but the wbDo bug affects QRY, ROWS, FETCH because Xsearch
> is affected, and all the subroutines that I listed in the email.

agree, could you do that? I'm very busy

> p.s. I finally discovered the problem. It had a number of causes but in
> the end it was my fault for not taking more care at setup.
> 
> # 1 I used v4_610 from launchpad - so it had the wbDo bug.
> # 2 The cPanel was not displaying errors in the error log and the web host
> had compiled grep into an unusual location # 3 I didn't read the status
> report closely - even though I looked at it dozens of times. It said
> clearly could not execute grep.

The status and debug page are not very readable and sometimes hard to find. 
We should work on that, because it is the first information one should see 
after a fresh install.

> It was only after I knew that there should be errors being generated by
> the apache logs that I requested the host provide me with full error
> logs for the previous week. I opened them up and saw the log was full of
> "cannot execute program /usr/local/bin/grep". I ran 'which grep' to
> obtain the location and corrected the path in woda.conf.
> 
> To make it harder to trouble shoot I had three definition files all
> pointing at the same data source. This is where I was stupid. I copied
> and pasted the entire definition, including WBF.

I allways use a "require" to load one or two common setup file: common setup 
for tables and common fields (as it was done in Scix/Sops).

> After doing that I had
> requests for changes to the fields. I did that but as Richard noticed, I
> had duplicated one of the fields in one definition file. This meant that
> the sort order for the fields was different in each of the definition
> files! That was bad. I moved all the fields out into their own file
> named WBF.pl and used require to call them in.  This cleared up a number
> of unusual issues. The following day I found the problem with grep and
> the databases began working properly.

> 
> This reminds me that I improved AdmStatus. I have to put more effort
> into my updates on launchpad. I have a version that runs "which" if the
> program location given in woda.conf fails. I changed the error to
> display clearly in bold, red text and then to display the correct
> location.

can you send it to me? 
I would like to test. By putting it in the contrib folder it should replace 
the current cgi.

We should use this feature (the contrib folder) to test improved or new 
functions with the help of the list.

Xavier
xavier@alternatif.org - 09 54 06 16 26

Revision history for this message

Malcolm Fitzgerald (malcolm-notyourhomework) wrote on 2010-08-15:

#5

Download full text (8.0 KiB)

On 14/08/2010, at 8:11 PM, Xavier Brochard wrote:

> Le samedi 7 août 2010 15:37:08, vous avez écrit :
>> We should push out a new version for download that has this bug and the
>> security hole fixed. These two are very important. The security bug is
>> most serious but the wbDo bug affects QRY, ROWS, FETCH because Xsearch
>> is affected, and all the subroutines that I listed in the email.
>
> agree, could you do that? I'm very busy

yes, I'll look at it today.

>
>> p.s. I finally discovered the problem. It had a number of causes but in
>> the end it was my fault for not taking more care at setup.
>>
>> # 1 I used v4_610 from launchpad - so it had the wbDo bug.
>> # 2 The cPanel was not displaying errors in the error log and the web host
>> had compiled grep into an unusual location # 3 I didn't read the status
>> report closely - even though I looked at it dozens of times. It said
>> clearly could not execute grep.
>
> The status and debug page are not very readable and sometimes hard to find.
> We should work on that, because it is the first information one should see
> after a fresh install.

That should be fairly easy. The information is there, it just needs to be laid out nicely and explained.

>
>> It was only after I knew that there should be errors being generated by
>> the apache logs that I requested the host provide me with full error
>> logs for the previous week. I opened them up and saw the log was full of
>> "cannot execute program /usr/local/bin/grep". I ran 'which grep' to
>> obtain the location and corrected the path in woda.conf.
>>
>> To make it harder to trouble shoot I had three definition files all
>> pointing at the same data source. This is where I was stupid. I copied
>> and pasted the entire definition, including WBF.
>
> I allways use a "require" to load one or two common setup file: common setup
> for tables and common fields (as it was done in Scix/Sops).

That is normally what I would do too. I did it differently because I had seen some problems which I thought (my mistake) were caused by the package namespace. The real problem was the wbDo issue.

>
>> After doing that I had
>> requests for changes to the fields. I did that but as Richard noticed, I
>> had duplicated one of the fields in one definition file. This meant that
>> the sort order for the fields was different in each of the definition
>> files! That was bad. I moved all the fields out into their own file
>> named WBF.pl and used require to call them in. This cleared up a number
>> of unusual issues. The following day I found the problem with grep and
>> the databases began working properly.
>
>
>>
>> This reminds me that I improved AdmStatus. I have to put more effort
>> into my updates on launchpad. I have a version that runs "which" if the
>> program location given in woda.conf fails. I changed the error to
>> display clearly in bold, red text and then to display the correct
>> location.
>
> can you send it to me?
> I would like to test. By putting it in the contrib folder it should replace
> the current cgi.

sub myAdmWodaStatus {
# print WODA status
# try to locate program paths when programs defined in woda.conf a...

On 14/08/2010, at 8:11 PM, Xavier Brochard wrote:

> Le samedi 7 août 2010 15:37:08, vous avez écrit :
>> We should push out a new version for download that has this bug and the
>> security hole fixed. These two are very important. The security bug is
>> most serious but the wbDo bug affects QRY, ROWS, FETCH because Xsearch
>> is affected, and all the subroutines that I listed in the email.
> 
> agree, could you do that? I'm very busy

yes, I'll look at it today.

> 
>> p.s. I finally discovered the problem. It had a number of causes but in
>> the end it was my fault for not taking more care at setup.
>> 
>> # 1 I used v4_610 from launchpad - so it had the wbDo bug.
>> # 2 The cPanel was not displaying errors in the error log and the web host
>> had compiled grep into an unusual location # 3 I didn't read the status
>> report closely - even though I looked at it dozens of times. It said
>> clearly could not execute grep.
> 
> The status and debug page are not very readable and sometimes hard to find. 
> We should work on that, because it is the first information one should see 
> after a fresh install.

That should be fairly easy. The information is there, it just needs to be laid out nicely and explained.

> 
>> It was only after I knew that there should be errors being generated by
>> the apache logs that I requested the host provide me with full error
>> logs for the previous week. I opened them up and saw the log was full of
>> "cannot execute program /usr/local/bin/grep". I ran 'which grep' to
>> obtain the location and corrected the path in woda.conf.
>> 
>> To make it harder to trouble shoot I had three definition files all
>> pointing at the same data source. This is where I was stupid. I copied
>> and pasted the entire definition, including WBF. 
> 
> I allways use a "require" to load one or two common setup file: common setup 
> for tables and common fields (as it was done in Scix/Sops).

That is normally what I would do too. I did it differently because I had seen some problems which I thought (my mistake) were caused by the package namespace. The real problem was the wbDo issue.

> 
>> After doing that I had
>> requests for changes to the fields. I did that but as Richard noticed, I
>> had duplicated one of the fields in one definition file. This meant that
>> the sort order for the fields was different in each of the definition
>> files! That was bad. I moved all the fields out into their own file
>> named WBF.pl and used require to call them in.  This cleared up a number
>> of unusual issues. The following day I found the problem with grep and
>> the databases began working properly.
> 
> 
>> 
>> This reminds me that I improved AdmStatus. I have to put more effort
>> into my updates on launchpad. I have a version that runs "which" if the
>> program location given in woda.conf fails. I changed the error to
>> display clearly in bold, red text and then to display the correct
>> location.
> 
> can you send it to me? 
> I would like to test. By putting it in the contrib folder it should replace 
> the current cgi.

sub myAdmWodaStatus {     
# print WODA status 
# try to locate program paths when programs defined in woda.conf are not found

printHead ('WODA Status');

print "<h3>Can we execute another Perl program?</h3>";

$!='';
    $perl = "$PERL -v";

$out = `$perl`;

if ($out eq '') {
        print <<EOM;
<p class="ERROR">
Looks like it will not be possible to run external Perl programs.
Relational features such as &QRY function will not work!
Maype just <br>\$PERL=$PERL<br>was not set up right!
This is the error reported: <b>$!</b>
</p>
EOM
    } else {
        print <<EOM;
<p>
Command <b>$perl</b> produced:
<pre>
$out
</pre>
<p>
EOM
    }

print <<EOM;
<h3>Directories related to this application</h3>
<p>Note that only <b>$TMPDIR</b> and <b>$DATADIR</b> must exist.
<p>
EOM
    @x = &wbCheckDirectories();
    push (@x,$TESTCGIDIR);

$testFile = "_test.xxx";
    $textText = "This is some test text";
    $close = "</em></td></tr>\n";
    $et = qq~<em class="ERROR">~;
    print '<table border="1">';

foreach $x (@x) {
        $x = $DISK . $x unless $x =~ m/^[A-Z]\:/;
        print qq~<tr align="left" valign="top"><th>$x</th><td><em class="ERROR">\n~;

unless (-e "$x") {
            print "Does not exist!\n";
        print $close; next;
            }

unless (chdir ($x)) {
        print "Cannot chdir to $x!\n";
        }

unless ( open (H, ">$x/$testFile") ) {
        print "Cannot open file $testFile for writing. Check permissions!\n";
        print $close; next;
        } else {
        print H $testText;
        close (H);
        }

unless ( open (H, "$x/$testFile") ) {
        print "Cannot open file $testFile for reading!\n";
        } else {
        $xx = <H>;
        if ($xx ne $testText) {
            print "Read text not equal to written text!\n";
        }
        close (H);
        }

if ($MV) {
        `$MV $testFile $testFile.zzz`;
        unless (-e "$testFile.zzz") {
            print "Could not MV $testFile\n";
        } else {
            `$MV $testFile.zzz $testFile`;
        }
        }

unless ( unlink ("$x/$testFile") ) {
        print "Cannot delete file $testFile\n";
        print $close; next;
        }

print "<em>OK</em>\n";
        print $close;
    }
    print "</table>\n";

print "<h3>External programs</h3> Essential functions work without them! <table>";

@progs = ('$SENDMAIL','$AT','$GREP','$CRONTAB','$UPTIME');

# Ignoring dump and undump.
#The undump program was an old attempt to speed up your Perl program by 
# storing the already-compiled form to disk. This is no longer a viable 
# option, as it only worked on a few architectures

foreach my $prog (@progs) {
        $xprog = eval ($prog);
        print "<tr><td>$prog</td><td>$xprog</td><td>\n";

if (!$xprog) {
            print '<em class="WARN">not defined</em>';
        } elsif (-x $xprog) {
        print "executable";
        next;
        } elsif (-e $xprog && !UNIX) {
        print "exists";
        next;
        } else {
        m#.*/(.+)#;
        $prog = $1;
        my $test = "which $prog";
        my $out = `$test`;
        if ($out) {print "try $out"} else {print '<em class="WARN">does not exist</em>'}
        }

}

print "</table>";

if ($SCRIPTPACKAGE ne $WodaPackage) {
        print "<h3>Subroutines defined in package '$SCRIPTPACKAGE'</h3>\n";

@subs = @MYSubs;
        foreach $sub (@MYSubs) {
            print "$SCRIPTPACKAGE'$sub()<br>\n";
        }
    }

print <<EOM;
<h3>Can we use FETCH to get a Webpage into application?</h3>
EOM

$pg = 'http://www.ddatabase.com/index.htm';
    $back = &FETCH($pg);

print <<EOM;
<p>Tried &FETCH(<a href="$pg">$pg</a>). This is what I got back:
<table border="1"><tr><td>
$back
</td></tr></table>
EOM
}

> 
> We should use this feature (the contrib folder) to test improved or new 
> functions with the help of the list.
> 
> 
> Xavier
> xavier@alternatif.org - 09 54 06 16 26
> 
> -- 
> wbDo is broken
> https://bugs.launchpad.net/bugs/613352
> You received this bug notification because you are a member of Woda,
> which is the registrant for Woda.
> 
> Status in Woda, the Web Oriented DAtabase: Fix Released
> 
> Bug description:
> I've tracked the problems I had recently to a bug in the wbDo subroutine. The subroutine is missing the keyword REQUIRE. 
> 
> If you are using v4.6.10 you should replace wbDo with the version below. 
> 
> The following files are affected:
> 
> /usr/local/woda/en/woda-max.pl
> /usr/local/woda/en/include/cgiAdmAgentProcess.pl
> /usr/local/woda/en/include/cgiAdmAgentProcessForm.pl
> /usr/local/woda/en/include/cgiAdmDaily.pl
> /usr/local/woda/en/include/cgiAdmIndexPDF.pl
> /usr/local/woda/en/include/cgiAdmStatic.pl
> /usr/local/woda/en/include/cgiGo.pl
> /usr/local/woda/en/include/cgiQry.pl
> /usr/local/woda/en/include/cgiXsearch.pl
> 
> 
> sub wbDo { # do a cgi action ... load appropriate file first
> 
>    $_=shift (@_);;
>    require "$WodaLibDir/$_.pl" unless defined &$_;    # load  
>    &$_(@_);                # execute
> }
> 
>

Revision history for this message

Malcolm Fitzgerald (malcolm-notyourhomework) wrote on 2010-08-16:

#6

woda_4-611.zip Edit (801.0 KiB, application/zip; name="woda_4-611.zip")

The modified version is attached. I'm not sure how to make it available on launchpad.

malcolm

On 14/08/2010, at 8:11 PM, Xavier Brochard wrote:

> Le samedi 7 août 2010 15:37:08, vous avez écrit :
>> We should push out a new version for download that has this bug and the
>> security hole fixed. These two are very important. The security bug is
>> most serious but the wbDo bug affects QRY, ROWS, FETCH because Xsearch
>> is affected, and all the subroutines that I listed in the email.
>
> agree, could you do that? I'm very busy

Revision history for this message

Xavier Brochard (xavier) wrote on 2010-08-17:

#7

Hi Malcolm

Le lundi 16 août 2010 02:58:51, vous avez écrit :
> The modified version is attached. I'm not sure how to make it available
> on launchpad.

There is some problems in the directory tree: it contains a __MACOSX folder
and the "include" folder is in the wrong place. But more, it looks like you
have released from the 4.620 branch :-)

Well, I have some spare time now. I will patch the 4.610 branch and create the
release. I will document the process of releasing, as launchpad make it a bit
complicated (but it help to track bugs, release notes, etc.).
To display all registered brnch in launchpad, use the "Branches with status"
pulldown
or click on this: https://code.launchpad.net/woda?field.lifecycle=ALL

Xavier
<email address hidden> - 09 54 06 16 26

Revision history for this message

Malcolm Fitzgerald (malcolm-notyourhomework) wrote on 2010-08-17:

#8

On 18/08/2010, at 4:19 AM, Xavier Brochard wrote:

> Hi Malcolm
>
> Le lundi 16 août 2010 02:58:51, vous avez écrit :
>> The modified version is attached. I'm not sure how to make it available
>> on launchpad.
>
> There is some problems in the directory tree: it contains a __MACOSX folder
> and the "include" folder is in the wrong place. But more, it looks like you
> have released from the 4.620 branch :-)

Shit. I did the job twice. The first time I worked from 4.620 then realised my mistake and when back to 4.610 and did it over again. I must have got the files mixed up.

I forgot about the __macosx folder. The system generates them automatically. I can do the packaging directly from the command line using tar and bz instead.

Malcolm

Woda

wbDo is broken

Bug Description

Other bug subscribers

Bug attachments

Remote bug watches