popularity-contest does not work out of the box

[Gökdeniz Karadağ]

In Dapper, popularity-contest cronjob has a usage error.
popcon-upload-ubuntu reads the package list from stdin, but the cronjobs try to give it using "-f" parameter, then nothing gets sent and manual running of cronjob makes it wait indefinitely.

Cronjob should use input redirection. This small fix can be seen below.

--- popularity-contest.old 2006-06-28 00:37:18.000000000 +0300
+++ popularity-contest 2006-06-28 00:37:58.000000000 +0300
@@ -53,7 +53,7 @@
 if [ "$SUBMITURLS" ] && [ "yes" = "$USEHTTP" ]; then
     for URL in $SUBMITURLS ; do
        if setsid /usr/share/popularity-contest/popcon-upload-ubuntu \
- -u $URL -f $POPCON 2>/dev/null ; then
+ -u $URL < $POPCON 2>/dev/null ; then
                SUBMITTED=yes
        else
                logger -t popularity-contest "unable to submit report to $URL."

[x (xk2c-deactivatedaccount)]

I also came across this "manual running of cronjob makes it wait indefinitely" today as i wanted to test it.

With the fix above it seems to work (as i don´t now on which webpage to look for my data i can´t tell if they are correctly send out and received).

[Michel D'HOOGE (michel-dhooge)]

I am no Python expert, but it seems to me that the '-u $URL' part is ignored. Indeed, everything is hardcoded in the python script. I tried the 'simple' following line:

# /usr/share/popularity-contest/popcon-upload-ubuntu < /var/log/popularity-contest

which resulted in 69kb sent (according to tcpdump). This bug could surely also explains part of bug #35291.

[Doug Harple (dharple)]

Before today, this package hasn't seemed to work at all for me. HTTP submission is broken out of the box, and the e-mail address for submission is broken as well.

I tried a few other fixes (manual, patches submitted with other bugs), but this is the only fix that seemed to actually transmit something.

Thilo Six, to test the cron script from the command line, use nohup:
$ nohup /etc/cron.weekly/popularity-contest

[x (xk2c-deactivatedaccount)]

# ls -l nohup.out
-rw------- 1 root root 0 03.12.2006 21:07 nohup.out

it is emtpy after running it. Is this good or bad?

[Doug Harple (dharple)]

Thilo Six: that's fine. The cron job doesn't output anything.

As a follow-up to my previous post, it seems as if this worked.
Packages that I have on my system that weren't in the popcon results
yesterday are there today.

[Michael Vogt (mvo)]

Thanks for your bugreport.

Attached is a fix that I plan to upload to edgy-proposed. Please test and let me know if it works for you.

Thanks,
 Michael

[Gökdeniz Karadağ (gokdeniz)]

Yes this fix seems to work. The script ended normally and netstat showed an open connection and traffic to popcon.ubuntu...

[Dave Love (fx-gnu)]

Michael Vogt <email address hidden> writes:

> Attached is a fix that I plan to upload to edgy-proposed. Please test
> and let me know if it works for you.

I assumed there was some reason for not using the Perl version in
Ubuntu. If you go with that, presumably you should either remove the
-ubuntu version (and get Debian to) or apply my patch to fix it.

It also needs mention of mail submission removed from the doc and the
.conf file. (The doc also talks about Debian and debian.org in some
places where it should say Ubuntu.)

[Wolf Canis (wolf-canis)]

Hello,
I did a support request:

https://launchpad.net/ubuntu/+ticket/3249

And by this way I came to this bug report.
The fix as here suggest, doesn't work for me, details see at the support request as
described above.
I think I found a other solution but I'm not really sure. Below a description what I
did:

The python script
/usr/share/popularity-contest/popcon-upload-ubuntu
want to read from stdin, but the shell script /etc/cron.weekly/popularity-contest
gives the python script the parameter -u and -f. This parameter I have ignored, because
the server and filename are hard coded:

ret= post_multipart('popcon.ubuntu.com', '/popcon-submit.cgi', '' , data)

This should be changed. If I learn enough about python, perhaps I do so. ~:)

I did following changes:

 #if setsid /usr/share/popularity-contest/popcon-upload-ubuntu \
 # -u $URL -f $POPCON 2>/dev/null ; then
 if setsid /usr/share/popularity-contest/popcon-upload-ubuntu \
            < $POPCON 2>/dev/null ; then

Now the python script reads from stdin, so how it expected. The script runs with out any
errors. I don't know whether the data comes correctly at the Ubuntu server or not. Is
there a way to check this?

Thanks in advance,
                                 Mr Canis

[Colin Watson (cjwatson)]

http://librarian.launchpad.net/5581450/popularity-contest_1.33ubuntu2.1.debdiff approved for edgy-proposed. (Other cleanups should be done in feisty.)

[Michael Vogt (mvo)]

The patch I posted here is used in feisty already. The reason we used the -ubuntu version before was that the ubuntu version is older than the debian version.

[Wolf Canis (wolf-canis)]

Hello,
what patch is the right, that suggested from Colin Watson or from Michael Vogt? Or they
are the same?
The changes from the fix by Michael Vogt, doesn't fix the error on my system ( see
my support request https://launchpad.net/ubuntu/+ticket/3249 ). And please see my
explanations from 2007-01-21 13:07:43 CET. With this changes no errors appear.
What shall I do?
I don't want make my system incompatible to the Ubuntu update mechanism.

For any suggestions, thanks in advance,
                                                                      Mr Canis

[Tollef Fog Heen (tfheen)]

Accepted into edgy-proposed.

[Colin Watson (cjwatson)]

Mr Canis: my message was administrative, to do with the stable update process. Ignore it.

[Michael Vogt (mvo)]

Hello Mr. Canis,

thanks for your comment. I looked at the support ticket and it does not mention that you tried my patch. Could you please elaborate what did not work for you?

Thanks,
 Michael

[Michael Vogt (mvo)]

I also just noticed that this bug (or a different bug) seems to affect dapper as well. We certainly want to check that.

[Michael Vogt (mvo)]

Uploaded to edgy-proposed

[Wolf Canis (wolf-canis)]

Hello Michael Vogt,
I have ubuntu 6.10 edgy installed. What I did was following. I changed the two lines ( 55 and 56 ):

        #if setsid /usr/share/popularity-contest/popcon-upload-ubuntu \
 # -u $URL -f $POPCON 2>/dev/null ; then
 if setsid /usr/share/popularity-contest/popcon-upload \
     -u $URL -f $POPCON -C 2>/dev/null ; then

but I the same error appears, after start, at the command prompt, /etc/cron.weekly/popularity-contest:

Jan 19 10:41:35 laptop popularity-contest: unable to submit report to http://popcon.ubuntu.com/popcon-submit.cgi.
Jan 19 10:41:35 laptop popularity-contest: unable to submit report using sendmail.
Jan 19 10:41:35 laptop popularity-contest: unable to submit report.

Then I examined the python script. For the result see my message from 2007-01-21 13:07:43 CET. And with that changes no errors appear.
Perhaps I did something wrong. I questioned that in the message from 2007-01-22 16:45:43 CET.
What do you think, what happens?
How I mentioned earlier, I don't want make my system incompatible to the Ubuntu update mechanism.

Thanks in advance,
                                  Mr Canis

[Michael Vogt (mvo)]

Thanks Mr Canis for testing this fix!

we discovered that there was indeed another problem that made popcon-upload return the a "unable to submit report..." even when it did succseed. The latest version in feisty does fix this issue. And we are also preparing backports to fix this problem in dapper and feisty. I will post debdiffs to the bug #35291 to ensure that we also fix the problem that popcon tries to send mail for some people.

Cheers,
 Michael

[Martin Pitt (pitti)]

Setting edgy task back to unconfirmed since this has not been fixed in Feisty yet and I'd like to get the SRU bug list to a trackable state.

[Martin Pitt (pitti)]

SRU is already handled in bug 35291, closing backports tasks.

[Brian Murray (brian-murray)]

I tested with 1.33ubuntu2 and had the same failure with the cronjob in /etc/cron.weekly/popularity-contest. I then upgraded to 1.33ubuntu2.2 from edgy-proposed and the cronjob completed and there was also a log file in /var/log/popularity-contest so it looks good to me.

[Michael Vogt (mvo)]

This is fixed in feisty/gutsy, closing here too.