USC relies on NM to be online, can't force online if not using NM

Bug #911706 reported by Alan Pope 🍺🐧🐱 🦄
210
This bug affects 39 people
Affects Status Importance Assigned to Milestone
software-center (Ubuntu)
Confirmed
High
Unassigned

Bug Description

Much like Firefox and Evolution, USC relies on knowing the system is online via Network Manager. If NM is 'offline' (e.g. disconnected from wired and not associated with any wifi access points) then USC will not show reviews and will disable the Install button.

This is fine if you use Network Manager to get online. If you have to use something else (for example to use pppoe) then you will not be able to install apps in USC, or read reviews.

Other applications work around this by providing a 'Go online' option somewhere in a menu. For example Firefox has the 'Work Offline' menu item in the "File" menu which becomes ticked when network manger is offline.

Perhaps USC could implement a similar feature?

ProblemType: Bug
DistroRelease: Ubuntu 12.04
Package: software-center 5.1.4
ProcVersionSignature: Ubuntu 3.2.0-7.13-generic 3.2.0-rc7
Uname: Linux 3.2.0-7-generic x86_64
NonfreeKernelModules: nvidia wl
ApportVersion: 1.90-0ubuntu1
Architecture: amd64
Date: Wed Jan 4 11:25:25 2012
InstallationMedia: Ubuntu 11.10 "Oneiric Ocelot" - Beta amd64 (20110901)
PackageArchitecture: all
SourcePackage: software-center
UpgradeStatus: Upgraded to precise on 2011-12-22 (12 days ago)

Revision history for this message
Alan Pope 🍺🐧🐱 🦄 (popey) wrote :
Revision history for this message
Rick Spencer (rick-rickspencer3) wrote :

assigned to software-center hackers to see if they want to take a look at enabling this scenario

Changed in software-center (Ubuntu):
assignee: nobody → software-center-developers (software-center-developers)
importance: Undecided → Low
Revision history for this message
Michael Vogt (mvo) wrote :

Thanks for your bugreport.

I asked mpt for design review of this, in the meantime I pushed:
 lp:~mvo/software-center/trivial-nm-state-override

that allows the SOFTWARE_CENTER_NET_CONNECTED environment to force a connection.

Changed in software-center (Ubuntu):
status: New → Triaged
Revision history for this message
Matthew Paul Thomas (mpt) wrote :

"You know lots about computers ... can you fix our computer?" It was the sentence that many geeks dread. In this case, it was from neighbors of my parents, Christmas 2010. Their Internet hadn't worked for weeks, and they didn't know why.

They had called their ISP, and the ISP had gone through a list of troubleshooting steps with them, without success. The ISP had even gone to the expense of sending them a new router, and its lights all flashed like they were supposed to -- but still no Internet.

Internet Explorer was the only browser installed on the computer. I fired it up to see what would happen. "Internet Explorer cannot display the webpage. Most likely causes: You are not connected to the Internet. The website is encountering problems. There might be a typing error in the address." So, that wasn't useful.

But because I'm the sort of person who pays attention to tiny details, I happened to notice a tiny red X in the status bar at the bottom of the window. Yes, Internet Explorer was in offline mode. And it had been in offline mode for weeks. As soon as I turned offline mode off, the Internet worked again.

Now, maybe Internet Explorer shouldn't have made it so easy to go into offline mode by mistake. Or maybe it should have had a clearer explanation that it was in offline mode, and how to get out of it. But the most certain way for the neighbors to have avoided this problem would have been to use Chrome or Safari, because those browsers do not have manual offline modes at all.

For any setting that exists, some people will twiddle it by mistake. Others will twiddle it, see no obvious effect, and leave it twiddled. That risk is worth running for many settings -- but not for this one. It is unreasonable to expect humans to tell Ubuntu Software Center whether the computer is online, when that's something the computer can jolly well figure out for itself. (The logical conclusion would be that *every* program, that wishes to behave intelligently when offline, should have a UI control for the human to say "well, no, actually the computer is online". That would be hugely redundant and unlikely to be implemented -- as you can tell from how seldom developers make Ubuntu applications work through proxies.)

So if the code USC uses to tell whether the computer is online is unreliable, then make it reliable. For example, if the problem is that network-manager sometimes isn't installed, then make software-center depend on network-manager. After all, no-one reasonably expects applications on Windows or OS X to work with multiple swappable networking systems! Or even if you do think that's a reasonable expectation on Ubuntu, then check whether Network Manager is running at all before asking it whether the computer is online. If it isn't, then ping ubuntu.com or check connectivity some other way. Just don't bother the human with it, please. :-)

Revision history for this message
Gary Lasker (gary-lasker) wrote :

Very, very nicely said, mpt! Thank you very much for your comment. You put everything into perspective so perfectly.

mvo, should we consider making Software Center depend on network-manager? That would be the easiest fix, but possibly not the most desirable one for some folks. Shall we try a connectivity check, also as suggested by mpt? Any other ideas?

Changed in software-center (Ubuntu):
status: Triaged → Incomplete
assignee: software-center-developers (software-center-developers) → Michael Vogt (mvo)
importance: Low → Medium
Revision history for this message
Gary Lasker (gary-lasker) wrote :

Please see bug 856504 (now set as a dupe of this one) for an example of a person who chooses not to use network-manager.

Changed in software-center (Ubuntu):
status: Incomplete → Confirmed
Revision history for this message
Selene ToyKeeper (toykeeper) wrote :

I ran into this today. NM hasn't been working so I switched to wicd and was surprised when it broke USC.

To resolve it, my first choice would be to simply attempt to use the network as usual and handle the exceptions if it fails.

However, if the UI needs to be configured differently depending on network state, I'd do a connectivity check like mpt suggested. Simply pinging or tcp connect()ing to ubuntu.com (or maybe the ratings and reviews server) would be sufficient in most cases. If you want to get a little more sophisticated and verify there's no content filter or captive portal in the way, one could instead HTTP GET a known page and verify the contents. But networking in Linux is so wildly variable on different systems that I wouldn't attempt to check any local config.

Michael Vogt (mvo)
Changed in software-center (Ubuntu):
milestone: none → ubuntu-12.10
assignee: Michael Vogt (mvo) → nobody
Revision history for this message
Gary Lasker (gary-lasker) wrote :

I have set this bug to triaged as I believe that we have enough information to begin work on the fix.

Many thanks!

Changed in software-center (Ubuntu):
status: Confirmed → Triaged
Revision history for this message
Mohammad Anwar Shah (mohammadanwarshah) wrote :

Thanks Gary, As given instruction and comment in bug 1014277, you are correct. It was a duplicate.

I am giving another screenshot now. Note that, I have a local repository from which i can install downloaded packages with apt-get. Only not with USC. And i also not doing install by double clicking on deb file. Though the install button is greyed out in that case also.

Revision history for this message
Gary Lasker (gary-lasker) wrote :

I set this one to high based on the number of people this is affecting (and annoying).

Changed in software-center (Ubuntu):
importance: Medium → High
Revision history for this message
Gary Lasker (gary-lasker) wrote :

Hi Mohammad, indeed, what you have observed is the bug itself. When NM is not online, the "Install" buttons in Software Center are disabled, even for the case when installing from a local deb file (or archive, as in your case). As you said, apt-get will work fine for installing from your local archive.

Thank you again for your help!

Michael Vogt (mvo)
Changed in software-center (Ubuntu):
milestone: ubuntu-12.10 → none
Revision history for this message
Marius B. Kotsbak (mariusko) wrote :

I just installed a package using Software center in Quantal version with an internet connection outside NM (using ModemManager from master branch).

To have this working, make sure Network Manager is not running (run "sudo stop network-manager" first). In this way, e.g. Firefox and Pidgin) also are working.

Changed in software-center (Ubuntu):
status: Triaged → Fix Released
Revision history for this message
Marius B. Kotsbak (mariusko) wrote :

Seems like USC checks the internet connection if NM is not available:

012-09-14 11:11:54,434 - root - INFO - Attempting one time ping of no.archive.ubuntu.com to test if internet connectivity exists.
2012-09-14 11:11:54,590 - softwarecenter.fixme - WARNING - logs to the root logger: '('/usr/share/software-center/softwarecenter/netstatus.py', 168, 'test_ping')'
2012-09-14 11:11:54,589 - root - INFO - ping output: 'Internet connection available!
PING ftp.acc.umu.se (130.239.18.138) 56(84) bytes of data.
64 bytes from saimei.acc.umu.se (130.239.18.138): icmp_req=1 ttl=50 time=26.5 ms

--- ftp.acc.umu.se ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 26.574/26.574/26.574/0.000 ms

Revision history for this message
Marius B. Kotsbak (mariusko) wrote :

When it has this check, maybe it could have been run if NM is not running as well?

Revision history for this message
bas india (baskarans) wrote :

Can anyone confirm if this bug fixed in 12.10 beta?

Revision history for this message
Adolfo Jayme Barrientos (fitojb) wrote :

I don't think it's fixed. When I use my computer in guest mode, sometimes NM fails to start from terminal and i get no internet indicator in the panel (but the Internet works). When this happens, installations and reviews are disabled in USC.

Revision history for this message
Adolfo Jayme Barrientos (fitojb) wrote :

Ah, just saw the related mere proposal and workarounded the problem by entering in a terminal: SOFTWARE_CENTER_NET_CONNECTED=true software-center

Revision history for this message
Marius B. Kotsbak (mariusko) wrote :

It is not supposed to work if network manager is running and it thinks its not connected. If you are not using network manager, please stop it:

"sudo stop network-manager"

and it should work.

Revision history for this message
Mateusz Stachowski (stachowski-mateusz) wrote :

Why this bug has status Fix Released when it's clearly not resolved.
Obviously Ubuntu developers think that users should search for half-assed workarounds
on forums and bugtrackers.

I'm using wvdial for my connection because it's a better option. My internet disconnects
once an hour passed and then I have to make new connection (it's a free internet and this is one
of it's limitations the other is 60 KB/s max).

NetworkManager in this case requiers baby sitting. While wvdial connects automatically.
With NM I have to put out the modem than plug it in and make a connection.

http://dug.net.pl/tekst/197/aero2_z_modemem_huawei_e3131/

http://translate.google.pl/translate?sl=pl&tl=en&js=n&prev=_t&hl=pl&ie=UTF-8&eotf=1&u=http%3A%2F%2Fdug.net.pl%2Ftekst%2F197%2Faero2_z_modemem_huawei_e3131%2F
(the same site as above translated with Google)

Ubuntu Software Center isn't the only program with this bug. Ubuntu One Client also doesn't work
with wvdial until I run "sudo stop network-manager" (it doesn't sync files).

I'm writing that from Ubuntu 12.10 fully updated with proposed actualisations enabled.

Revision history for this message
Adolfo Jayme Barrientos (fitojb) wrote :

> "Obviously Ubuntu developers think that users should search for half-assed workarounds on forums and bugtrackers."

PLEASE don't mix your assumptions and frustrations here.

I'm reopening this bug report because USC should at least allow me to install a .deb when it's in offline mode.

Changed in software-center (Ubuntu):
status: Fix Released → Triaged
Revision history for this message
Marius B. Kotsbak (mariusko) wrote :

Mateusz, if you don't use network manager, please stop it and stop it starting automatically at boot. You can do that with the "rcconf" console application. Also, you should be able to use network/modem manager. Are your issues covered by bug #848164? If not, please open a new bug report.

Adolfo, you are mixing this fixed bug with bug #926763.

Changed in software-center (Ubuntu):
status: Triaged → Fix Released
Revision history for this message
Mateusz Stachowski (stachowski-mateusz) wrote :

USC and UbuntuOne are the only programs that rely solely on Network Manager and that's wrong. I don't have to do anything for other programs to use them online.

WvDial connection works for:

Chrome / Chromium
Opera and Opera Next
Konqueror and rekonq
Transmission and qBittorrent
gPodder (Podcast client)
Steam native and Wine
Desura / Desurium native and Wine
Gnome System Monitor (displays all internet related information)
WiMP service like Spotify (it's a program made in Adobe AIR)
Nuvola Player
Thunderbird
JDownloader

and this list could go on and on.

I'm also sure that all mentioned programs wouldn't have problems if I would connect through Wicd or PPPoE but of course Ubuntu Software Center and Ubuntu One are better because they use the one and only Network Manager.

Revision history for this message
Selene ToyKeeper (toykeeper) wrote :

I must agree that this is a bug if USC acts like it's offline when there is a valid network connection of any type.

I've noticed lately that USC runs a quick ping test when I start it, and it uses that result to set its online status... but if erroneous status from Network Manager can override the ping test, it's still broken.

I haven't personally run into that issue since the ping test was added, but I don't have Network Manager running any more. If I allow it to run, it will actually knock me offline at random, actively interfering with whatever is managing the connection.

So, could you detail the exact steps needed to reproduce this bug?

Changed in software-center (Ubuntu):
status: Fix Released → Incomplete
Revision history for this message
Mateusz Stachowski (stachowski-mateusz) wrote :

USC runs that ping test only when Network Manager service is disabled.

I can disable upstart job for NM (it's upstart so rcconf won't help here) and then it doesn't interfere with USC and Ubuntu One online status (the connection is managed by WvDial).

End users shouldn't be requiered to do all of that just to get working two default Ubuntu programs. Most of them won't even know why this things don't work. I myself use Synaptic but when I launched USC and the install buttons were dimmed I was very confused and I didn't know why it's like that.

The exact steps are as follows (connected through WvDial):

- Network Manager service is enabled and running
- open terminal and use this command software-center 2>&1 | tee USC_network_manager.log
- USC install buttons are dimmed and there is information about no internet connection
- there was no ping test

USC_network_manager.log

- Network Manager service is stopped
- open terminal and use this command software-center 2>&1 | tee USC_only_wvdial.log
- USC is online and all the functions work
- there was a ping test

USC_only_wvdial.log

Revision history for this message
Mateusz Stachowski (stachowski-mateusz) wrote :
Revision history for this message
Mateusz Stachowski (stachowski-mateusz) wrote :
Revision history for this message
Nuriyasov (nuriyasov) wrote : Re: [Bug 911706] Re: USC relies on NM to be online, can't force online if not using NM

Seems like that "dimming install button" functionality is not needed, why
even implement such thing?
Perhaps dev should simply remove or disable it? Less code means less bugs,
doesn't it?

2013/1/9 Mateusz Stachowski <email address hidden>

> ** Attachment added: "log file with NM service stopped"
>
> https://bugs.launchpad.net/ubuntu/+source/software-center/+bug/911706/+attachment/3477331/+files/USC_only_wvdial.log
>
> --
> You received this bug notification because you are subscribed to a
> duplicate bug report (916928).
> https://bugs.launchpad.net/bugs/911706
>
> Title:
> USC relies on NM to be online, can't force online if not using NM
>
> Status in “software-center” package in Ubuntu:
> Incomplete
>
> Bug description:
> Much like Firefox and Evolution, USC relies on knowing the system is
> online via Network Manager. If NM is 'offline' (e.g. disconnected from
> wired and not associated with any wifi access points) then USC will
> not show reviews and will disable the Install button.
>
> This is fine if you use Network Manager to get online. If you have to
> use something else (for example to use pppoe) then you will not be
> able to install apps in USC, or read reviews.
>
> Other applications work around this by providing a 'Go online' option
> somewhere in a menu. For example Firefox has the 'Work Offline' menu
> item in the "File" menu which becomes ticked when network manger is
> offline.
>
> Perhaps USC could implement a similar feature?
>
> ProblemType: Bug
> DistroRelease: Ubuntu 12.04
> Package: software-center 5.1.4
> ProcVersionSignature: Ubuntu 3.2.0-7.13-generic 3.2.0-rc7
> Uname: Linux 3.2.0-7-generic x86_64
> NonfreeKernelModules: nvidia wl
> ApportVersion: 1.90-0ubuntu1
> Architecture: amd64
> Date: Wed Jan 4 11:25:25 2012
> InstallationMedia: Ubuntu 11.10 "Oneiric Ocelot" - Beta amd64 (20110901)
> PackageArchitecture: all
> SourcePackage: software-center
> UpgradeStatus: Upgraded to precise on 2011-12-22 (12 days ago)
>
> To manage notifications about this bug go to:
>
> https://bugs.launchpad.net/ubuntu/+source/software-center/+bug/911706/+subscriptions
>

Revision history for this message
Matthew Paul Thomas (mpt) wrote :

"End users shouldn't be requiered to do all of that just to get working two default Ubuntu programs." That's true, but they aren't: end users aren't installing then disabling Network Manager to begin with. If you're going to alter your system in that way, it's your responsibility to do it properly, so that NM isn't running for USC to ask it about connection status. (I trust that you've also checked that a bug is reported for whatever is missing from Network Manager that makes you use wvdial instead.)

Revision history for this message
Mateusz Stachowski (stachowski-mateusz) wrote :

The problem is that no other programs require to disable Network Manager service to function properly.

I've mentioned it before all of the browsers (besides default Firefox) work when connected through WvDial and NM service running. Thunderbird works and downloads mails without interference in it's offline settings. Transmission have no problems with downloads nor do qBittorrent or wget, gPodder, Steam (native and Wine), Desura / Desurium (native and Wine).

This means that the only programs that don't work when internet connection is managed through something other than Network Manager (and the service is running, which is default setting) are the ones installed with every Ubuntu. USC has the ping status checker but it's of no use because it only runs with NM disabled. Ubuntu One and Firefox don't have any kind of online status checker and they simply assume that if there is nothing in NM then user doesn't have internet connection. How can anybody think that this things are right is beyond my imagination.

Ubuntu Software Center and Ubuntu One have serious design flaws if they stupidly rely on one program for providing online status information.

Revision history for this message
Matthew Paul Thomas (mpt) wrote :

On the contrary, it is those other applications you list that behave stupidly when there is no Internet connection. For example, Firefox says "Firefox can't find the server at www.example.com", giving several possible explanations, when Ubuntu knows that only one of them is correct. Thunderbird gives you a stream of "Failed to connect to server" notification bubbles for however many mail accounts you have. And Transmission doesn't even suggest there's an error, merely saying "Remaining time unknown". None of them can tell you definitively what the problem is.

The reason those applications behave so stupidly is that they aren't asking Network Manager what the connection status is. The reason they aren't asking Network Manager what the connection status is is that they aren't assuming it will be present. And the reason they aren't assuming it will be present is that none of them are written specifically for Ubuntu.

Ubuntu Software Center and the Ubuntu One client are written specifically for Ubuntu. So they can assume Network Manager will be present, which means they can use it to find out what the connection status is, which means they can behave appropriately when there's no connection. That isn't a "serious design flaw", it's one of the benefits of running on a modern operating system.

Nobody would seriously suggest that you should be able to swap out the networking system in Windows or in OS X, and that applications should still work if you try. But *purely as a courtesy* to tinkerers who do that in Ubuntu, USC does the ping test if Network Manager is not running. (An alternative we seriously considered was just to make USC depend on Network Manager.) If you do swap out Network Manager, however, then as I said, it is your responsibility to do it completely.

Revision history for this message
Selene ToyKeeper (toykeeper) wrote :

If Network Manager were a rock-solid piece of software which always worked and was always installed and running, it might be reasonable to require it for default apps to function. But in the 7 years I've been trying to use it, NM hasn't worked correctly on *any* of the computers I've used it on. It's full of corner-case bugs, weird environmental incompatibilities, and is even a few missing basic functions (like letting the network be up when nobody is logged in to X).

Nobody would expect a different network stack in Windows or OSX because each only has one reasonable option... but Linux is a jungle, with dozens of valid systems for managing the network. We can mitigate that somewhat by setting sane and robust defaults, but we still have to respond gracefully when the defaults aren't being used.

If a user is on dial-up, it seems reasonable for them to install something like wvdial, but then they will have no clue why USC and U1 stopped working. Or if they're in a public place behind a captive portal, NM will report a valid network even when the system can't talk to anything other than the gateway. Or if they've already disabled NM completely, it tends to get re-enabled on each upgrade and sometimes on normal package updates, meaning the user must fix it over and over again.

As a general rule, in software I try to always use whatever method is simplest and least fragile... and for checking network status, that means sending a packet to an outside service and testing for the expected response. Network Manager is an awfully large and complex tool to use for something as simple as checking network status, and it's an unnecessary complication which gives us false negatives and weird failure modes.

I still don't see what's wrong with omitting all the look-before-you-leap checks entirely, and simply attempting to load content when requested. It'll fail sometimes even with a good network connection, so online error handlers need to be written anyway... and then the buggy pre-checks can simply be dropped as redundant. If you want to provide a smarter error message, make the error handler check the connection to narrow down the list of possible explanations. This logic could even be exported into its own network diagnostic package, to allow other apps to take advantage of an intelligent error handler for common issues.

Anyway, just my thoughts on the issue. It seems broken as-is, and there's a robust, straightforward way to fix it. It would be less error-prone if we removed the NM logic entirely.

Revision history for this message
Selene ToyKeeper (toykeeper) wrote :

Also, we seem to have a test description for reproducing the issue, though it might be easier to test with wicd than with wvdial, so I'm changing this from incomplete to confirmed.

Changed in software-center (Ubuntu):
status: Incomplete → Confirmed
tags: added: u1-support
Revision history for this message
Mateusz Stachowski (stachowski-mateusz) wrote :

Funny thing that there are multiple programs that can be used to connect the type of internet dial-up I'm using for Windows. There are also problems for Windows users to get auto-reconnect working the tutorials for configuring it look more complicated than WvDial configuration (atleast from my perspective because Windows users think differently).

http://translate.google.pl/translate?sl=pl&tl=en&js=n&prev=_t&hl=pl&ie=UTF-8&eotf=1&u=http%3A%2F%2Fjdtech.pl%2F2011%2F08%2Faero2-%E2%80%93-automatyczne-wznawianie-polaczen.html (Aero2 - Automatic resume calls)

Programs mentioned in the article (including the comments section):

HUAWEI Mobile Partner (which also has a Linux native version)
Top_NetInfo
AutoConnect
aero2auto-reconnect

I assume that people using those programs on Windows should have the same kind of problems that Ubuntu gives when not using the system integrated solution.

Revision history for this message
Igor Zubarev (igor.zubarev) wrote :

confirm bug for ubuntu 12.10.

Revision history for this message
Nuriyasov (nuriyasov) wrote :

You don't say

2013/1/22 Igor Zubarev <email address hidden>

> confirm bug for ubuntu 12.10.
>
> --
> You received this bug notification because you are subscribed to a
> duplicate bug report (916928).
> https://bugs.launchpad.net/bugs/911706
>
> Title:
> USC relies on NM to be online, can't force online if not using NM
>
> Status in “software-center” package in Ubuntu:
> Confirmed
>
> Bug description:
> Much like Firefox and Evolution, USC relies on knowing the system is
> online via Network Manager. If NM is 'offline' (e.g. disconnected from
> wired and not associated with any wifi access points) then USC will
> not show reviews and will disable the Install button.
>
> This is fine if you use Network Manager to get online. If you have to
> use something else (for example to use pppoe) then you will not be
> able to install apps in USC, or read reviews.
>
> Other applications work around this by providing a 'Go online' option
> somewhere in a menu. For example Firefox has the 'Work Offline' menu
> item in the "File" menu which becomes ticked when network manger is
> offline.
>
> Perhaps USC could implement a similar feature?
>
> ProblemType: Bug
> DistroRelease: Ubuntu 12.04
> Package: software-center 5.1.4
> ProcVersionSignature: Ubuntu 3.2.0-7.13-generic 3.2.0-rc7
> Uname: Linux 3.2.0-7-generic x86_64
> NonfreeKernelModules: nvidia wl
> ApportVersion: 1.90-0ubuntu1
> Architecture: amd64
> Date: Wed Jan 4 11:25:25 2012
> InstallationMedia: Ubuntu 11.10 "Oneiric Ocelot" - Beta amd64 (20110901)
> PackageArchitecture: all
> SourcePackage: software-center
> UpgradeStatus: Upgraded to precise on 2011-12-22 (12 days ago)
>
> To manage notifications about this bug go to:
>
> https://bugs.launchpad.net/ubuntu/+source/software-center/+bug/911706/+subscriptions
>

tags: added: u1-by-support
removed: u1-support
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Related questions

Remote bug watches

Bug watches keep track of this bug in other bug trackers.