popcon tries to send mail when http submission does not succseed (was: <email address hidden> does not exist)

[Stuart Bishop]

My popularity-contest installation (originally installed with warty, now with dapper) is attempting to send results to <email address hidden>. Unfortunately, this email address does not exist:

 <email address hidden>: host fiordland.ubuntu.com[82.211.81.145] said: 550
    <email address hidden>: Recipient address rejected: User unknown in virtual
    alias table (in reply to RCPT TO command)

[Matt Zimmerman (mdz)]

Er, popcon in Ubuntu submits using HTTP, not mail? Check /etc/popularity-contest.conf

[Stuart Bishop (stub)]

It appears to still want to support either HTTP or email. /etc/popularity-contest.conf tells me to ue 'dpkg-reconfigure popularity-contest' to change stuff, and running this I get asked if I want to submit via HTTP or email. The email address listed in /usr/share/popularity-contest/default.conf is <email address hidden>.

(However, this wasn't the original problem as when I just checked /etc/popularity-contest.conf, participate was set to 'no' so dapper upgrades have since changed things)

[omni-vi (catcher)]

I encounterd the same bug. I used 'dpkg-reconfigure popularity-contest' to tell popularity-contest to send its results via mail (I just prefer it that way), but <email address hidden> does not exist. Which mail-address shall I use ?
I'm using dapper.

[Juan Jose Amor Iglesias (jjamor)]

My /etc/popularity-contest.conf is configured for using HTTP:

MY_HOSTID="d69b97a6824e4728b163576ade8f111c"
PARTICIPATE="yes"
USEHTTP="yes"

However, today I've received the following error e-mail:

<email address hidden>: host fiordland.ubuntu.com[82.211.81.145] said: 550
    <email address hidden>: Recipient address rejected: User unknown in virtual
    alias table (in reply to RCPT TO command)

So, here I'm seeing two errors: first, the package still tries to send the report using e-mail, although http method is selected. Second, e-mail method doesn't work because of problems in ubuntu.com mailservers.

Note that I've installed manually this package yesterday, that is, in Dapper, it's not upgraded from previous releases.

My 0,02 euro cents ;)

[Andrew Pam (xanni)]

I have the exact same problem on Dapper. See also bug #39777 "Receiving bounce mail messages, even though it should send via HTTP"

The original message was received at Sun, 23 Jul 2006 09:58:31 +0930
from [10.100.1.84]

   ----- The following addresses had permanent fatal errors -----
<email address hidden>
    (reason: 550 <email address hidden>: Recipient address rejected: User unknown in virtual alias table)

   ----- Transcript of session follows -----
... while talking to fiordland.ubuntu.com.:
>>> RCPT To:<email address hidden>
<<< 550 <email address hidden>: Recipient address rejected: User unknown in virtual alias table
550 5.1.1 <email address hidden>... User unknown

[Daniel Pittman (daniel-rimspace)]

I have encountered the same problem, after setting a number of system to send reports by email. This was done through the debconf interface to the package, which explicitly offers email as a submission method.

Since several of the machines in question can't directly submit HTTP results, email seemed a good method to communicate these results to your systems.

If email isn't supported then debconf for the package should be updated to stop offering it as an option, I think.

A sample bounced message is attached.

[x (xk2c-deactivatedaccount)]

> It appears to still want to support either HTTP or email.

You may also have a look at:
https://launchpad.net/distros/ubuntu/+source/popularity-contest/+bug/51149/

[Michel D'HOOGE (michel-dhooge)]

Matt Zimmerman at 2006-04-06 21:08:53 CEST
> Er, popcon in Ubuntu submits using HTTP, not mail? Check /etc/popularity-contest.conf

If you look at /etc/cron.weekly/popularity-contest, you can read:

SUBMITTED=no
# try to post the report through http POST
if [ "$SUBMITURLS" ] && [ "yes" = "$USEHTTP" ]; then
...
fi
# try to email the popularity contest data
if [ yes != "$SUBMITTED" ]; then

So, if HTTP fails, it always fallback to mail...

[Juan Jose Amor Iglesias (jjamor)]

Hmm ...

I've (permanently) connected my workstation to Internet. However, popcon still tries to send reports by email.

BTW, I think that virtual "<email address hidden>" must be defined, mainly if ubuntu guys are interested in getting (more) real statistics about package usage.

[Michel D'HOOGE (michel-dhooge)]

As I wrote in bug #51149, I modified the popcon script to correct a bug.
And after manually running the script again (sudo /etc/cron.weekly/popularity-contest), I observed with 'sudo ngrep host popcon.ubuntu.com' that the content was correctly sent and the http server response was "Thanks!". I also checked with 'postqueue -p' that the postfix queue was empty (since my postfix is configured to only deliver local mail, previous attempts kept the fallback mail delivery in the queue).

[Andrew Pam (xanni)]

Why is the status "Needs Info"? What info is required? I'm still getting an email bounce from ubuntu.com every month.

[Andrew Pam (xanni)]

See bug #39777

[dave58 (dave58)]

As of 15 Oct 2006, mail still bounces:
A message that you sent could not be delivered to one or more of its recipients. This is a permanent error. The following address(es) failed:
  <email address hidden>
    SMTP error from remote mailer after RCPT TO:<email address hidden>:
    host fiordland.ubuntu.com [82.211.81.145]: 550 <email address hidden>:
    Recipient address rejected: User unknown in virtual alias table

[Sebastian Breier (tomcat42)]

Still happens. Shouldn't this be quite easy to fix?

[Dave Love (fx-gnu)]

It looks to me as though this is required, in addition
to fixing the upload script, for when the HTTP upload
fails (e.g. when offline).

It would be helpful if the popcon mailbox could be
created and at least just ignore submissions.

The patch is for the edgy version.

[Matt LaPlante (cybrmatt)]

Can't believe this has been a bug since March... how hard is it to fix? It's pretty sad that it's broken out of the box, even if it's a less than vital app.

[hackel (hackel)]

I still have this problem on edgy. I tried dpkg-reconfigure popularity-contest, however it is *already* set to use HTTP. Yet every week I get a nice bounced email saying that <email address hidden> doesn't exist. This is really annoying, and I think the priority of this bug should be higher.

[Michael Vogt (mvo)]

Thanks for your bugreport.

I'm preparing a fixed package for feisty right now, and if it works well, I will issue SRUs for dapper and edgy too.

Thanks,
 Michael

[Sebastian Breier (tomcat42)]

Very nice. Thanks Michael.

[Michael Vogt (mvo)]

I uploaded a fixed package to feisty, I would appreciate feedback if it fixes the problem for you (it does for me).

[Michael Vogt (mvo)]

Here is the changelog for the patch to make the dapper popularity-contest fully working:

  * debian/cron.weekly:
    - Use popcon-upload -C instead of popcon-upload-ubuntu. This
      fixes popcon data uploading and adds proxy support
      (lp: #51149, #59348)
    - do not depend on MAILTO if USEHTTP is set
  * default.conf:
    - comment out MAILTO because we do not support sending in
        popcon data over mail (lp: #35291)
  * generate a new hostid for older versions to make sure that we
    don't have duplicated HOSTIDs in our database (lp: #82104)

[Michael Vogt (mvo)]

Edgy will need a update for this as well. This update should check if the version is in the range of 1.31ubuntu2.X and not generate a new HOSTID in this case.

[Martin Pitt (pitti)]

Closing Feisty task, since it is allegedly fixed in Feisty:

popularity-contest (1.39ubuntu3) feisty; urgency=low

  * default.conf:
    - comment out MAILTO because we do not support sending in
      popcon data over mail (lp: #35291)
  * popcon-upload:
    - fix the expected string from popcon-submit

 -- Michael Vogt <email address hidden> Thu, 25 Jan 2007 16:09:26 +0100

[Michael Vogt (mvo)]

Here is a patch for edgy-proposed:

popularity-contest (1.33ubuntu2.2) edgy-proposed; urgency=low

  * default.conf:
    - comment out MAILTO because we do not support sending in
      popcon data over mail (lp: #35291)
  * debian/postinst:
    - when re-generating the HOSTID for older versions of
      popularity-contest, exclude the version range 1.31ubuntu2.X
      to ensure that we do not regenerate the HOSTID for the fixed
      packages in dapper-proposed
  * popcon-upload:
    - fix expected anwer string from server to match ubuntus version

[Martin Pitt (pitti)]

http://librarian.launchpad.net/5933999/popularity-contest_1.33ubuntu2.2.debdiff approved for dapper-proposed.

[Martin Pitt (pitti)]

Sorry, s/dapper/edgy/ in my previous comment.

[Martin Pitt (pitti)]

http://librarian.launchpad.net/5916213/popularity-contest_1.31ubuntu2.1.debdiff approved for edgy-proposed if you fix

+if [ yes != "$SUBMITTED" ] && [ "$MAILTO" ]; then

to

+if [ yes != "$SUBMITTED" ] && [ -n "$MAILTO" ]; then

[Michael Vogt (mvo)]

Thanks for the review!

I modified the patches accordingly and uploaded both dapper-proposed and edgy-proposed packages.

[Martin Pitt (pitti)]

Accepted into dapper-proposed.

[Martin Pitt (pitti)]

Accepted into edgy-proposed, please go ahead with QA testing.

[John (klfhjvflpmef)]

http://popcon.ubuntu.com/all-popcon-results.txt.gz says:

Release: 1.33ubuntu2.2 12073

which ought to mean that it has been tested on more than 12000 computers. It has been over a month. It should be ready for edgy-updates.

[Michael Vogt (mvo)]

This one is uploaded now to edgy-updates and waits for the archive-admin.

[Martin Pitt (pitti)]

Accepted into edgy-updates.

[Michael Vogt (mvo)]

Here are instructions to verify on a fresh dapper install:

[verify that the new HOSTID is generated]
1. cat /etc/popularity-contest.conf and store MY_HOSTID
2. install new package
3. the HOSTID should change
4. reinstall the new package
5. the HOSTID should stay the same

[verify the MAILTO fix]
1. install postfix and configure it for local delivery
2. run sudo /etc/cron.weekly/popularity-contest
3. verify that a bounce mail was generated
4. install updated package
5. run sudo /etc/cron.weekly/popularity-contest
6. verify that no new bounce mail is generated

[Brian Murray (brian-murray)]

The new HOSTID part of the verification passed.

[verify that the new HOSTID is generated]
1. cat /etc/popularity-contest.conf and store MY_HOSTID
2. install new package
3. the HOSTID should change
4. reinstall the new package
5. the HOSTID should stay the same

[Brian Murray (brian-murray)]

This part of the verification passed also, however my '/etc/popularity-contest.conf' file did cause some problems for me in that USEHTTP was set initially.

[verify the MAILTO fix]
1. install postfix and configure it for local delivery
2. run sudo /etc/cron.weekly/popularity-contest
3. verify that a bounce mail was generated
4. install updated package
5. run sudo /etc/cron.weekly/popularity-contest
6. verify that no new bounce mail is generated

[Martin Pitt (pitti)]

Copied to dapper-updates.