Software Center doesn't detect changes in sources until update-apt-xapian-index is ran by cron

[Joaquin]

[Test case]
The test case for bug #1075537 should be used for this bug.

software-center 4.0.1, Ubuntu 11.04

For example, I add "ppa:hotot-team/ppa" to sources, and when I close the sources window, Software Center automatically starts updating repositories, but after that, doesn't appear on the left panel, I have to close Software Center, open it again, and now the new PPA is appearing.

The equivalent for removing a PPA is bug 574155.

[Alkis Georgopoulos (alkisg)]

It doesn't affect only PPAs, it affects all sources changes.
Also closing and reopening software-center isn't enough to make it aware of new sources.

The following allows me to immediately see the new source:

<add any source, not just a PPA one>
apt-get update
update-apt-xapian-index
software-center

If I omit update-apt-xapian-index, then software-center is not informed about the changes until much later on, maybe until update-apt-xapian-index is launched from /etc/cron.daily/apt.

[Simon McNair (simonmcnair)]

Can I add a me too on this ? I tried to install Clonezilla without using Synaptic and although the sources updated I could not see the software until I restarted software-center. Whilst this may be a medium in the grand scheme of software centre, the omission of synaptic in the default install makes it a high imho (this is in 12.04).

Simon

[Jonas (jonas-f-henriksen)]

Hi, I added the opera - repository through the software senter, but nothing happend. So I closed the center and ran sudo apt-get update, then restarted the software senter, but still nothing. I installed opera on the commandline instead.

[Michele Giacomoli (michele-giacomoli)]

Could we make apt-reposity and software-properties -gtk run update-apt-xapian-index after they have added/removed a repository?

IMHO It's very confusing adding adding a repo from Edit -> software properies in Software center and after that, don't see any change in it. A person trying for the first time could think he made something wrong. Otherwise we could add a dialog box (or something similar) asking us to update the database for checking for new programs, and pressing "check" button we should see something telling us it is really doing what it has to

[melrokz (melrockz)]

Yes, I second Michele. Please make add-apt-repository and software-properties-gtk run whatever is needed for software-center to detect the new PPA / sources.

[Frederik Elwert (frederik-elwert)]

I think a related, but slightly different issue in 11.10 and 12.04 is Bug #863463. It seems that in newer versions, the Software Center does not actually update the package information (apt-get update), but if one does that manually, it finds new packages without manually calling update-apt-xapian-index.

[psypher (psypher246)]

This bug is STILL happening in 13.04 Raring Ringtail.

This is not a difficult thing to solve why is this bug still around it hasn;t even been assigned yet?

[Matthew Paul Thomas (mpt)]

Thanks, psypher! When you're done, link your branch to this bug report and propose it for merging to trunk.

[Dylan Coakley (dylan-coakley)]

Removed assignee as people shouldn't be given false hope that someone is actually working on this issue.

[Vadim Peretokin (vperetokin)]

Well, psypher did mention that it is "not a difficult issue", maybe he did
mean that he has a solution - I'd love to hear it!

[psypher (psypher246)]

Unfortunately I do not have a fix, I am just a power user not a developer. But the issue seems simple to me as I don't get the complexity of adding a condition: if one of the sources boxes has been ticked then gksudo apt-get update gets run when the close button is pressed.

or just as simple, add a button next to revert:"Update Changes" which also runs gksudo apt-get update.

The reason why I called this out is that this bug was logged 2 years ago, with multiple duplicates associated to it and not assigned to anyone.

This issue is trivial enough if you are comfortable with the command line. But for an OS thats supposed to easy and have the average user in mind, having them choose a software source and then trying to install an app from that source and not finding that app just because the sources were not downloaded is an extremely bad user experience.

Minor user issues like this (of which there are many) detract from the rest of the polish being done on Ubuntu. But the attitude of just letting them slip by and languish for years does not help the user's experience or Ubuntu/Canonical's image.

I take the time to log issues and chase them up and that's the level of help I can provide. But from this interaction I don't even see the point of logging user bugs anymore.

[Dylan Coakley (dylan-coakley)]

Yeah im amazed that this has not been sorted yet. Its one of those bugs I experience all the time while using Ubuntu and its the same for anyone else who uses PPA's. The fact that Synaptic works better with PPA's than the new modern USC is pretty embarrassing.

In Synaptic I can add a PPA, reload the packages and it even puts the PPA packages under a section called "New in repository". How neat! Now, if only USC was half as good and could just reload the packages for us.

[Mark Eichin (eichin-gmail)]

If the index is updated from cron.daily, is it *ever* going to usefully trigger on a LiveCD (USB stick)?

[Bruno Nova (brunonova)]

Yes, this needs to be sorted out.

I was looking at the source code of the software-properties package and I may have found the problem.
In the file softwareproperties/gtk/SoftwarePropertiesGtk.py there is this method around line no. 991 (sorry if I'm not supposed to put code here):

    def on_close_button(self, widget):
        """Show a dialog that a reload of the channel information is required
           only if there is no parent defined"""
        if (self.modified_sourceslist == True and
                self.options.no_update == False):
            d = DialogCacheOutdated(self.window_main,
                                    self.datadir)
            d.run()
        self.quit()

This is where the program should ask the user if he wants to update the cache (the Kde version should have something similar), but that is not happening.
If I remove the "self.modified_sourceslist == True" part of the condition, it works, so I believe that variable is not set correctly (I suppose this variable should "tell" if the sources were changed).
I'm not an expert in Python, so it would be better if someone else checked this.

Now, for the Software Center, I haven't tested if it's still necessary to re-open it.

[Michele Giacomoli (michele-giacomoli)]

We should take a look on what DialogCacheOutdated() does. If it shows the
dialog I remember, it should just update apt's cache, not xapian one, but
we also could try to add the update-apt-xapian-index command inside of
DialogCacheOutdated() after "apt-get update" is run.

2013/10/10 Bruno Nova <email address hidden>

> Yes, this needs to be sorted out.
>
> I was looking at the source code of the software-properties package and I
> may have found the problem.
> In the file softwareproperties/gtk/SoftwarePropertiesGtk.py there is this
> method around line no. 991 (sorry if I'm not supposed to put code here):
>
> def on_close_button(self, widget):
> """Show a dialog that a reload of the channel information is
> required
> only if there is no parent defined"""
> if (self.modified_sourceslist == True and
> self.options.no_update == False):
> d = DialogCacheOutdated(self.window_main,
> self.datadir)
> d.run()
> self.quit()
>
> This is where the program should ask the user if he wants to update the
> cache (the Kde version should have something similar), but that is not
> happening.
> If I remove the "self.modified_sourceslist == True" part of the condition,
> it works, so I believe that variable is not set correctly (I suppose this
> variable should "tell" if the sources were changed).
> I'm not an expert in Python, so it would be better if someone else checked
> this.
>
> Now, for the Software Center, I haven't tested if it's still necessary
> to re-open it.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/782953
>
> Title:
> Software Center doesn't detect changes in sources until update-apt-
> xapian-index is ran by cron
>
> To manage notifications about this bug go to:
>
> https://bugs.launchpad.net/ubuntu/+source/software-center/+bug/782953/+subscriptions
>

[Bruno Nova (brunonova)]

Here's a quick (and temporary) workaround to make the DialogCacheOutdated appear on close if the options were changed, for the GTK version, as a patch for the software-properties source package.

Basically, instead of checking "self.modified_sourceslist" (that prevents the dialog from appearing), with the patch it checks if the revert button is enabled. (dirty workaround, but it seems to work).

Still, "self.modified_sourceslist" needs to be fixed, and this doesn't "refresh" the Software Center (I think).
I hope the patch is correct.

[Bruno Nova (brunonova)]

Here is the patch for the KDE version.
I haven't tested if this one works!!

[Ubuntu Foundations Team Bug Bot (crichton)]

The attachment "workaround-no-reload-on-close-gtk.patch" seems to be a patch. If it isn't, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are a member of the ~ubuntu-reviewers, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issues please contact him.]

[Bruno Nova (brunonova)]

Good news (I hope).

Disregard the patches.
I looked at the code a bit more and added a "real" fix. (a pretty simple one, actually)
It was all because of that "modified_sourceslist" not being set to True.
I've commited the fix to the linked branch and proposed a merge.
With the fix, editing the sources in software-properties-gtk and then exiting will correctly display the update cache dialog.
This is a fix for Bug #1075537.

I tested this with the Software Center and, curiously, this also seems to fix this bug.
When editing the software sources from the Software Center, the Software Center "grabs" and hides the dialog and the update progress is shown in the Software Center.
After the cache update is complete, it executes update-apt-xapian-index correctly (though it takes a long time to finish) and, when done, it refreshes the Sofware Center.

[Bruno Nova (brunonova)]

Ok, the branch was merged.

By the way Ubuntu Software Center developers, when the software sources are edited from the USC, the caches of both apt and xapian are updated correctly, but USC doesn't report the progress of 'update-apt-xapian-index' anywhere. It doesn't even indicate that it is running (at least in Raring).

[Michele Giacomoli (michele-giacomoli)]

Thank you Bruno for your patch, Hope to see this fix in my Ubuntu updates
soon ;)

2013/10/16 Bruno Nova <email address hidden>

> Ok, the branch was merged.
>
> By the way Ubuntu Software Center developers, when the software sources
> are edited from the USC, the caches of both apt and xapian are updated
> correctly, but USC doesn't report the progress of 'update-apt-xapian-
> index' anywhere. It doesn't even indicate that it is running (at least
> in Raring).
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/782953
>
> Title:
> Software Center doesn't detect changes in sources until update-apt-
> xapian-index is ran by cron
>
> To manage notifications about this bug go to:
>
> https://bugs.launchpad.net/ubuntu/+source/software-center/+bug/782953/+subscriptions
>

[Steve Langasek (vorlon)]

Hello Joaquin, or anyone else affected,

Accepted software-properties into saucy-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/software-properties/0.92.27.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Adam Conrad (adconrad)]

Hello Joaquin, or anyone else affected,

Accepted software-properties into saucy-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/software-properties/0.92.28 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Mateusz Stachowski (stachowski-mateusz)]

Software Center detects now changes properly in Saucy with software-properties 0.92.28.

Although after the arrows in USC stop spinning it takes a very long time until the list of PPA's gets the new entry. Removing PPA's also results in updated list and this also takes a very long time. On my PC it's about two minutes after the arrows stopped spinning.

[Vadim Peretokin (vperetokin)]

I think that is due to PPAs being very slow to respond and thus updating
taking a while.

On Sat, Oct 26, 2013 at 5:18 AM, Mateusz Stachowski <
<email address hidden>> wrote:

> Software Center detects now changes properly in Saucy with software-
> properties 0.92.28.
>
> Although after the arrows in USC stop spinning it takes a very long time
> until the list of PPA's gets the new entry. Removing PPA's also results
> in updated list and this also takes a very long time. On my PC it's
> about two minutes after the arrows stopped spinning.
>
> --
> You received this bug notification because you are subscribed to a
> duplicate bug report (1057822).
> https://bugs.launchpad.net/bugs/782953
>
> Title:
> Software Center doesn't detect changes in sources until update-apt-
> xapian-index is ran by cron
>
> To manage notifications about this bug go to:
>
> https://bugs.launchpad.net/ubuntu/+source/software-center/+bug/782953/+subscriptions
>

[Bruno Nova (brunonova)]

No, it's because, after the arrows stop spinning, USC runs "update-apt-xapian-index", as I said above.
That command takes a while to complete, and USC doesn't "say" that it is running.

[Vadim Peretokin (vperetokin)]

Ah.

On Sat, Oct 26, 2013 at 6:55 AM, Bruno Nova <email address hidden>wrote:

> No, it's because, after the arrows stop spinning, USC runs
> "update-apt-xapian-index", as I said above.
> That command takes a while to complete, and USC doesn't "say" that it is
> running.
>
> --
> You received this bug notification because you are subscribed to a
> duplicate bug report (1057822).
> https://bugs.launchpad.net/bugs/782953
>
> Title:
> Software Center doesn't detect changes in sources until update-apt-
> xapian-index is ran by cron
>
> To manage notifications about this bug go to:
>
> https://bugs.launchpad.net/ubuntu/+source/software-center/+bug/782953/+subscriptions
>

[Matthew Paul Thomas (mpt)]

Thanks Bruno! Please report a separate bug for USC about the lack of visible progress. Bug 1090791 might be a special case of this general problem.

[Scott Kitterman (kitterman)]

The verification of this Stable Release Update has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regresssions.

[Bruno Nova (brunonova)]

I have reported a bug for the lack of visible progress when update-apt-xapian-index is executed (#574155).

[psypher (psypher246)]

Someone has finally added thes simple dialog box when closing to reload the sources. I think this can be closed

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Britta Wieland (wieland-britt)]

<email address hidden>

[Rolf Leggewie (r0lf)]

saucy has seen the end of its life and is no longer receiving any updates. Marking the saucy task for this ticket as "Won't Fix".