QNetworkAccessManager hangs when in flight mode
| Affects | Status | Importance | Assigned to | Milestone | |
|---|---|---|---|---|---|
| | Canonical System Image |
High
|
Zoltan Balogh | ||
| | qtbase-opensource-src (Ubuntu) |
High
|
Unassigned | ||
| | qtbase-opensource-src (Ubuntu RTM) |
Undecided
|
Unassigned | ||
| | thumbnailer (Ubuntu) |
Undecided
|
Unassigned | ||
Bug Description
Testing the fix:
- Update rc-proposed (vivid) to citrain device-upgrade 32 0000
- Optional: If you want to test using the 'generic' bearer plugin instead of the Network Manager bearer, either do not upgrade ubuntu-
- Reboot
---
When my phone is in flight mode, HTTP requests made using QNetworkAccessM
From my tests, it looks like the HTTP request is blocked until the flight mode is disabled and a network connection is reestablished. This is contrary to the documentation, which states:
"""If the network is not accessible the network access manager will not process any new network requests, all such requests will fail with an error. Requests with URLs with the file:// scheme will still be processed."""
http://
By running strace on my test program, it doesn't even look like it attempts to open a TCP connection when blocked in flight mode.
| James Henstridge (jamesh) wrote : | #1 |
| Launchpad Janitor (janitor) wrote : | #2 |
Status changed to 'Confirmed' because the bug affects multiple users.
| Changed in qtbase-opensource-src (Ubuntu): | |
| status: | New → Confirmed |
| Michi Henning (michihenning) wrote : | #3 |
I also noticed that, no matter whether we have a network or not, accessible() always returns 0, even if we have just successfully downloaded something.
| Changed in qtbase-opensource-src (Ubuntu): | |
| importance: | Undecided → High |
| Lorn Potter (lorn-potter) wrote : | #4 |
As for NetworkAccessib
https:/
| Timo Jyrinki (timo-jyrinki) wrote : | #5 |
@Michi or anyone: Could you test landing-026 PPA on vivid-overlay (getting 5.4.1+dfsg-2ubuntu7 Qt versions from there), and report back if the accessible() problem would be fixed without seeming regressions?
| Michi Henning (michihenning) wrote : | #6 |
I can't really test this as is. The problem is that, to test it, I need the new thumbnailer from silo 10, but the ppa includes the old thumbnailer. I can't easily get the new thumbnailer onto the device because Jenkins builds for Wily, not Vivid.
Not sure what I can do for the moment. To test that networkAccessible() thing really is fixed, put the device into flight mode, and it should return zero. If the device isn't in flight mode, it should report that the network is accessible (even before sending the first request).
| Timo Jyrinki (timo-jyrinki) wrote : | #7 |
I'm gone now for 2 weeks, so if anyone is able to test that there are no regressions coming from the accessible patch, let sil2100 know about it. I've marked the (vivid) silo to be ready and AP tested in the landing info, so if the patch works / does not regress both wily + vivid silos can be landed.
| Michi Henning (michihenning) wrote : | #8 |
I just tested this with the packages from silo 21.
Unfortunately, this is still not working:
Successfully activated service 'com.canonical.
thumbnailer-
thumbnailer-
thumbnailer-
Note the ten-second delay is still there. Moreover, as the trace shows, networkAccessible() returns 1, even though the device is in flight mode.
It appears that the 1 return value happens for the first call we make. Thereafter, it looks like networkAccessible() correctly returns 0.
At any rate, this bug isn't fixed yet, by the looks of things. networkAcessible() is still lying the first time we call it, and the hang until we time out is still there.
| Lorn Potter (lorn-potter) wrote : | #9 |
I updated this patch today.
https:/
| Lorn Potter (lorn-potter) wrote : | #10 |
hmm. might take yet another.. momentarily
| Lorn Potter (lorn-potter) wrote : | #11 |
Moving this to upstream bug, so I can do the work there.
| Lorn Potter (lorn-potter) wrote : | #12 |
additional patch:
| Timo Jyrinki (timo-jyrinki) wrote : | #13 |
There's a qtbase build for vivid+overlay at https:/
- Make sure to report correct NetworkAccessib
- Make sure networkAccessib
- Fix hang in qnam when disconnecting
With the test program, instead of host not found + poll in strace, I get Network access is disabled.
However, at least the following Qt tests started failing because of those patches:
FAIL! : tst_QNetworkDis
FAIL! : tst_QXmlInputSo
(after skipping these two I simply disabled the tests for this test build)
The failing tests make the current build not suitable for landing. At minimum all the failing tests would need to individually checked out and disabled if they're deemed to be false positives, but probably there are some real test problems that the upstream tracker will also show when the commits are tried to be merged.
| Lorn Potter (lorn-potter) wrote : | #14 |
There is a bug I found which may effect this: :)
https:/
I have merged my fixes into https:/
This should make NetworkAccessib
| Timo Jyrinki (timo-jyrinki) wrote : | #15 |
Ok! I'm updating the build again, although I hit:
FAIL! : tst_QNetworkAcc
So I'll disable the tests again so that it'd build at least.
| Lorn Potter (lorn-potter) wrote : | #16 |
Looks like that test assumes it's always going to be either Accessible or NotAccessible, and does not account for UnknownAccessib
| Michi Henning (michihenning) wrote : | #17 |
I was going to test whether networkAccessible() returns the right thing now. But it looks like silo 32 won't build at the moment. I couldn't find another silo for this. Am I looking in the wrong place?
| Timo Jyrinki (timo-jyrinki) wrote : | #18 |
https:/
| Michi Henning (michihenning) wrote : | #19 |
I've tested with the PPA and a thumbnailer with some trace thrown in. Looks like it's OK now. I'm getting networkAccessible == 0 when in flight mode, and == 1 with the network up.
| Timo Jyrinki (timo-jyrinki) wrote : | #20 |
The networkAccessible test seems also fixed by the patch set 13 from Lorn.
I'm still getting tst_QXmlInputSo
I'm considering disabling that test but it would be useful understand whether it's also fallout from this bugfix making the accessible() working like it should. So, in the above build log, it had logged the test as "FAIL" but then the build was indefinitely hanged until I canceled it.
| Timo Jyrinki (timo-jyrinki) wrote : | #21 |
This is now on hold because of the test regressions. I've now also managed to test that the Weather App autopilot tests reliably regress for some reason if the "Make sure networkAccessib
However, without this last patch the bugs described in this bug report are also not fixed.
| Lorn Potter (lorn-potter) wrote : | #22 |
FAIL! : tst_QXmlInputSo
Loc: [tst_qxmlinputs
This test is http client/server test. I have tested this here and it hangs on previous version (5.4.2) as well. The request never gets the finished signal.
As for the attachment in #20, it also fails at:
FAIL! : tst_QUdpSocket:
Loc: [../tst_
In which case is a test that fails to check for start/stop capabilities before it tries to open a NetworkSession (which will fail using only the generic bearer plugin (like a system without networkmanager or connman), as it cannot start or stop interfaces/
Not knowing all that much about ubuntu tests, looking at that weather app log:
(but I do know app armor blocks QNAM/network manager dbus requests, and I believe they refuse to open app armor to allow clients to access QNAM/network manager)
QNetworkManager
QNetworkManager
| Timo Jyrinki (timo-jyrinki) wrote : | #23 |
Thanks for spotting the tst_QUdpSocket:
The problems might be explained by the more proper behavior with the patch, but the weather app would need to be tested at least so that it's functionally intact. No other test suites showed similar regression, including webbrowser. The apparmor denials are related to bug #1404188 - it's possible that with this patch the weather app happens to be able to send a fuller query which is then denied. If the weather app is functionally identical, then the autopilot tests just do a wrong assumption, but it will need to be looked into.
| Michi Henning (michihenning) wrote : | #24 |
Set to triaged for thumbnailer so we can track it.
| Changed in thumbnailer (Ubuntu): | |
| status: | New → Triaged |
| Timo Jyrinki (timo-jyrinki) wrote : | #25 |
Just to update on this, the new patch to use generic plugin by default (which many tests assume) and only disable it via environment variable made it possible to keep tests enabled. The last upstream commit related to fixing this bug was also merged to Qt 5.5. Silo 032 is updated for the backports to Qt 5.4.1.
I was not able to find out what to do about the weather app's newly gained network access problem, but if you want to help please try out https:/
| description: | updated |
| description: | updated |
| description: | updated |
| Michi Henning (michihenning) wrote : | #26 |
I just looked at the silo and it says that the build failed.
| Timo Jyrinki (timo-jyrinki) wrote : | #27 |
No, the bileto info should be just ignored. If you look at the actual silo (PPA), you see qtbase built: https:/
What failed to build is the ubuntu-
I'll just fix the bileto status too so that people are not confused.
| Michi Henning (michihenning) wrote : | #28 |
OK, I installed the PPA and built a thumbnailer that reverts the work-around we've been using. I print trace for the networkAccessible() and we unconditionally send a GET request, regardless of what networkAccessible() tells us.
I start out with flight mode enabled. networkAccessible() returns the correct value (0), and the GET request fails immediately with the correct error status.
I then disable flight mode. networkAccessible() now returns 1 as it should. However, the GET request has a problem. It doesn't return and we time it out after 10 seconds.
So, it looks like the networkAccessible() problem is fixed. However, the GET doesn't seem to realize when the status has changed from "down" to "up" and doesn't send the request as it should.
| Timo Jyrinki (timo-jyrinki) wrote : | #29 |
For reference, did you add the /etc/enviroment variable?
I've now made a vivid-overlay branch of the ubuntu-
| description: | updated |
| Michi Henning (michihenning) wrote : | #30 |
Yes, I did that edit and rebooted. I can double-check tomorrow with your new PPA.
| Michi Henning (michihenning) wrote : | #31 |
I just tried this again with the latest PPA. The behavior is still weird. Initially, I'm in flight mode and the status is correct, and the GET request fails immediately, as expected. Then I turn flight mode off and wait until the wifi connection is up, then request a few more thumbnails. At that point, the accessibility status is still "network down" for several GET requests in a row.
I'm also seeing a bunch of these in the log:
thumbnailer-
thumbnailer-
thumbnailer-
(The "QNAM" is the trace for the accessibility status.)
After a bunch more of these, in the end, I see this:
thumbnailer-
thumbnailer-
thumbnailer-
thumbnailer-
thumbnailer-
thumbnailer-
thumbnailer-
thumbnailer-
thumbnailer-
thumbnailer-
thumbnailer-
thumbnailer-
thumbnailer-
thumbnailer-
thumbnailer-
thumbnailer-
thumbnailer-
thumbnailer-
thumbnailer-
Note that the accessibility status each time is still -1 even though, eventually, a the requests succeed. So, something is definitely not right here still.
Now I kill off the music app and wait for the thumbnailer-service to exit, all while the wifi connection is up. I then try again and see this:
thumbnailer-
thumbn...
| Timo Jyrinki (timo-jyrinki) wrote : | #32 |
There should not be any difference before/after, now the environment variable is just set up automatically.
So the fixes do fix in principle the accessible() return value and definitely fix the original bug report of QNAM hangs, but with some downsides or at least it still not being perfect.
The same behaviour can be seen with Qt 5.5.1 so this will be the de facto future way the Qt network is going to work. If you want to continue comparisons, on xenial image (once they work) you can test https:/
Notably unresolved bug #1508945 is filed for one problem these patches or the Qt 5.5.1 will bring.
Thanks for trying out and reporting... all in all it's useful to have the PPAs but this isn't something that could be immediately landed to stable phone users at least because of that 1508945.
| description: | updated |
| Timo Jyrinki (timo-jyrinki) wrote : | #33 |
A quick update that with generic plugin there are no problems, except that QNAMs don't understand when connection is dropped from wifi to 3G or upgraded from 3G to wifi.
The new network problem when using NM bearer is because these new Qt patches fixing the bug backported from Qt 5.5 branch rely more than before on the blocked DBus traffic causing a failure. The same apparmor rule relaxing (https:/
| Lorn Potter (lorn-potter) wrote : | #34 |
I've done some work on QNAM recently in regards to accessibility property in QNAM, and see that the patch that (which was upstreamed to qt 5.6) makes this property follow the online state like it should, is missing from ubuntu's repo.
I will try to do some testing on phone and see if this can help.
I know of one bug even with the mentioned patch in regards to disconnect timing that needs to also be solved.
That said, I tried compiling net-test.cpp in the sdk for phone, which failed
I have a few of my own test apps and QNAM in flight mode returns something like:
void Netter:
So by going by the original description, QNAM no longer hangs in flight mode (using the default bearer configuration)
| Timo Jyrinki (timo-jyrinki) wrote : | #35 |
The silo 32 (https:/
https:/
https:/
https:/
https:/
With those this bug and accessibility property are fixed, but with the downside of NM bearer starting to somehow depend on more DBus calls that get blocked by apparmor (I meant "blocked by apparmor", not "blocking calls"), causing this same bug #1508945 as what happens with the PPA with Qt 5.5 packages (which have the one post-5.5.1 commit cherry-picked). If the apparmor rules are relaxed the same way as would be needed for fixing bug #1404188, the problems disappear, but alas we can't relax them for real.
So unfortunately the fixes can't be apparently landed if not also switching to connectivity-api bearer or if there's a solution to NM bearer that fixes the bugs fixed in those first three commits without causing the new problem in our current apparmor setup.
| Timo Jyrinki (timo-jyrinki) wrote : | #36 |
Here's the net-test compiled for the phone, which I've used to test the problem, and which shows the bug would be fixed with the current set of patches in 032.
| Timo Jyrinki (timo-jyrinki) wrote : | #37 |
The build 5.4.1+dfsg-
| Timo Jyrinki (timo-jyrinki) wrote : | #38 |
It seems like the fixes landed to bug #1480877 help here - the silo 032 is now rebased, and weather app again works even with NM backend and all these new patches.
There's a new patch pending related to that patch landing, but this silo 032 would now finally start to be a landing candidate after that is settled.
| Launchpad Janitor (janitor) wrote : | #39 |
This bug was fixed in the package qtbase-
---------------
qtbase-
* Update symbols for s390x.
-- Timo Jyrinki <email address hidden> Tue, 08 Dec 2015 13:35:46 +0000
| Changed in qtbase-opensource-src (Ubuntu): | |
| status: | Confirmed → Fix Released |
| Timo Jyrinki (timo-jyrinki) wrote : | #40 |
qtbase-
* debian/
- Replace the non-DFSG-free RFC files with public domain content.
- Adjust tests to the new files.
(LP: #1522442)
* debian/copyright: mention the replacement.
* Switch to opt-in for architectures to run tests on.
-- Timo Jyrinki <email address hidden> Mon, 07 Dec 2015 14:02:11 +0000
qtbase-
* Skip largefile test on s390x too, in addition to amd64.
* Make tests to keep going till the end with -k.
-- Dimitri John Ledkov <email address hidden> Fri, 04 Dec 2015 16:54:28 +0000
qtbase-
* New upstream release. (LP: #1437206) (LP: #1450137) (LP: #1474313)
(LP: #1470700) (LP: #1504631) (LP: #1423659) (LP: #1474775) (LP: #1508945)
* Replace load_testabilit
Add-
* Drop patches in upstream:
- Correct-
* Rebase enable-tests.patch. Disable one failing QtWidgets test.
* Build depend on GStreamer 1.0 and add a configure option for it.
* Update symbols.
* Mark/unmark private symbols.
* Replace two Ubuntu patches with upstreamed patches:
- Drop disable-
Add-
- Drop qopenglframebuf
Blacklist
* debian/
- Include a network fix from Qt 5.5 branch (merged after 5.5.1)
(LP: #1470700)
* debian/
- Fix a qdoc issue (LP: #1447182)
* Remove disable_
* debian/
- Backport. Prefer QT_PLUGIN_PATH over compiled-in paths (LP: #1519927)
* debian/
- Backport. Fix a crasher on exit (LP: #1436973)
* Replace our workaround for font rendering with new backported upstream
patches:
- Add debian/
- Add debian/
- Remove debian/
(LP: #1475205)
-- Timo Jyrinki <email address hidden> Tue, 01 Dec 2015 06:16:35 +0000
| Timo Jyrinki (timo-jyrinki) wrote : | #41 |
Silo 032 has long pending networking fixes that now have seen more recent work and the silo might be in shape for landing.
As an interested party in this networking related bug (among many), could you please give 032 a whirl on vivid-overlay (xenial also available) and test your networking in general in addition to this particular bug? If we see no regressions - and we do see fixes - we could try landing the silo.
Upgrade with 'citrain device-upgrade 32 0000'.
| Lorn Potter (lorn-potter) wrote : | #42 |
Tested this using 032 and with the attached code and no longer hangs:
phablet@
2016-02-04T04:23:50 (unknown:0) - Starting request
2016-02-04T04:23:50 (unknown:0) - QNAM finished: QNetworkReplyHt
2016-02-04T04:23:50 (unknown:0) - Error state: 0 "Unknown error"
<HTML><HEAD><meta http-equiv=
<TITLE>302 Moved</
<H1>302 Moved</H1>
The document has moved
<A HREF="http://
</BODY></HTML>
2016-02-04T04:23:50 (unknown:0) - Done. Sleeping 5 seconds
2016-02-04T04:23:55 (unknown:0) - Starting request
2016-02-04T04:23:55 (unknown:0) - "Method "GetAll" with signature "s" on interface "org.freedeskto
"
2016-02-04T04:23:55 (unknown:0) - QNAM finished: QNetworkReplyHt
2016-02-04T04:23:55 (unknown:0) - QNAM finished: QNetworkReplyHt
2016-02-04T04:23:55 (unknown:0) - Error state: 8 "Network session error."
2016-02-04T04:23:55 (unknown:0) - QNetworkReplyIm
2016-02-04T04:23:55 (unknown:0) - Done. Sleeping 5 seconds
2016-02-04T04:23:55 (unknown:0) - "Method "GetAll" with signature "s" on interface "org.freedeskto
"
2016-02-04T04:24:00 (unknown:0) - Starting request
2016-02-04T04:24:00 (unknown:0) - QNAM finished: QDisabledNetwor
2016-02-04T04:24:00 (unknown:0) - QNAM finished: QDisabledNetwor
2016-02-04T04:24:00 (unknown:0) - Error state: 99 "Network access is disabled."
2016-02-04T04:24:00 (unknown:0) - QIODevice::read: device not open
2016-02-04T04:24:00 (unknown:0) - Done. Sleeping 5 seconds
2016-02-04T04:24:05 (unknown:0) - Starting request
2016-02-04T04:24:05 (unknown:0) - QNAM finished: QDisabledNetwor
2016-02-04T04:24:05 (unknown:0) - QNAM finished: QDisabledNetwor
2016-02-04T04:24:05 (unknown:0) - Error state: 99 "Network access is disabled."
2016-02-04T04:24:05 (unknown:0) - QIODevice::read: device not open
2016-02-04T04:24:05 (unknown:0) - Done. Sleeping 5 seconds
2016-02-04T04:24:10 (unknown:0) - Starting request
2016-02-04T04:24:10 (unknown:0) - QNAM finished: QDisabledNetwor
2016-02-04T04:24:10 (unknown:0) - QNAM finished: QDisabledNetwor
2016-02-04T04:24:10 (unknown:0) - Error state: 99 "Network access is disabled."
2016-02-04T04:24:10 (unknown:0) - QIODevice::read: device not open
2016-02-04T04:24:10 (unknown:0) - Done. Sleeping 5 seconds
2016-02-04T04:24:15 (unknown:0) - Starting request
2016-02-04T04:24:15 (unknown:0) - QNAM finished: QDisabledNetwor
2016-02-04T04:24:15 (unknown:0) - QNAM finished: QDisabledNetwor
2016-02-04T04:24:15 (unknown:0) - Error state: 99 "Network access is disabled."
2016-02-04T04:24:15 (unknown:0) - QIODevice::read: device not open
2016-02-04T04:24:15 (unknown:0) - Done. Sleeping 5 seconds
2016-02-04T04:24:20 (unknown:0) - Starting request
2016-02-04T04:24:20 (unknown:0) - QNAM finished: QNetworkReplyHt
| Changed in canonical-devices-system-image: | |
| milestone: | none → ww08-2016 |
| Łukasz Zemczak (sil2100) wrote : | #43 |
This bug was fixed in the package qtbase-
---------------
qtbase-
* Lorn Potter's networking fixes:
- debian/
- debian/
- debian/
- debian/
- debian/
(LP: #1470700) (LP: #1506015) (LP: #1507769) (LP: #1528886) (LP: #1533508)
* debian/
- Backport to replace disable-
-- Timo Jyrinki <email address hidden> Mon, 14 Dec 2015 12:53:41 +0000
| Changed in qtbase-opensource-src (Ubuntu RTM): | |
| status: | New → Fix Released |
| Changed in canonical-devices-system-image: | |
| status: | New → In Progress |
| Changed in canonical-devices-system-image: | |
| status: | In Progress → Fix Committed |
| Changed in thumbnailer (Ubuntu): | |
| status: | Triaged → Invalid |
| Changed in canonical-devices-system-image: | |
| assignee: | nobody → Zoltan Balogh (bzoltan) |
| importance: | Undecided → High |
| Lorn Potter (lorn-potter) wrote : | #44 |
Just verified using the test app that this is fixed on:
phablet@
current build number: 377
device name: mako
channel: ubuntu-
last update: 2016-03-01 09:20:40
version version: 377
version ubuntu: 20160227
version device: 20160112
version custom: 20160227
| Changed in canonical-devices-system-image: | |
| status: | Fix Committed → Fix Released |
Attached is the test program. I compiled it within the click chroot using the following command:
arm-linux-
When run with a URL as an argument, it will attempt to retrieve the resource every 5 seconds. When in flight mode, I see the process blocked in poll():
poll([{fd=3, events=POLLIN}, {fd=7, events=POLLIN}], 2, -1)
Where the two file descriptors are:
eventfd2(0, O_NONBLOCK|
socket(PF_LOCAL, SOCK_STREAM|
connect(7, {sa_family=
(i.e. the second is a D-Bus connection to the system bus).