DNS cannot be resolved in Public / Hotel / Starbucks WiFi Hotspot

Bug #1766969 reported by Pete
104
This bug affects 20 people
Affects Status Importance Assigned to Milestone
systemd (Ubuntu)
Fix Released
Undecided
Unassigned
Artful
Won't Fix
Undecided
Unassigned
Bionic
Fix Released
Undecided
Unassigned
Cosmic
Fix Released
Undecided
Unassigned

Bug Description

[Impact]

 * More captive portals have been discovered that do not handle DNS requests with D0 set.
 * Thus extend previous aruba-networks fix from https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1727237 by retrying all NXDOMAIN results with lower feature levels just in case.

[Test Case]

 * systemd-resolved something-nonexistant.ubuntu.com
 * monitor the logs, and check that the transaction has been downgraded multiple times and tried at least once at a feature level below D0. There should be a downgrade message in the log.
 * Note, it will be cached, thus one may need to call $ systemd-resolved --flush-caches

[Regression Potential]

 * Legitimate NXDOMAIN queries will now take longer, as they will be retried multiple times with different features sets until an answer is found. At that point the query will be cached and will return quickly. This is needed to work-around bad captive portals that do not support dns queries with D0. Post-captive portal, the feature level can usually be cranked back up and it does after a grace period.

[Other Info]

 * Original bug report

I was asked to create a new bug for this in https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1727237 as it seems to be a different issue.

I have installed the nightly image of Kubuntu Bionic from 25th of April.

There systemd is in version 237-3ubuntu10.

When connecting to the wifi hotspot in my hotel (Quality Hotel Augsburg) I cannot open the hotspot landing page that should give me access to the WIFI. With Windows and on an Iphone it's working.

For the following distributions I can confirm it not working:
Kubuntu 17.10
Kubuntu 18.04 (nightly image 25th of April 2018)

The logs were taken on 18.04.

Workaround:
sudo systemctl disable systemd-resolved.service
sudo service systemd-resolved stop
sudo rm /etc/resolv.conf
sudo nano /etc/NetworkManager/NetworkManager.conf
  >> add "dns=default" under [main]
sudo service network-manager restart

Then I can connect to the WIFI and I see the login page in Firefox.

To capture some data I did the following:
 - connect to Hotspot
 - enter golem.de

Case 1: Fresh default Kubuntu install
With a default Kubuntu install it does not work. I can connect to the WIFI and get IP and DNS from DHCP but I cannot resolve any hostname. When trying to open the router ip directly in the browser it forwards to hotsplots.de which cannot be resolved.

Case 2: With aforementioned Workaround
I connect to the wifi, I open firefox and the login page shows up (if I havent been connected yet. In the capture I already was able to connect to the hotspot which allows immediately to connect to the webpage)

PS: I'll be in this hotel till Friday 27th if more information are required.

Revision history for this message
Pete (pt.pete.pt) wrote :

Wireshark logs for default conf of Kubuntu and logs with workaround

tags: added: dns systemd systemd-resolv
description: updated
Pete (pt.pete.pt)
description: updated
Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in systemd (Ubuntu):
status: New → Confirmed
Revision history for this message
Steve Langasek (vorlon) wrote :

Can confirm that the dns logs indicate that systemd-resolved is not falling back from UDP+EDNS0 to UDP in response to these NXDOMAIN answers.

The existing patch only implements this fallback when the portal name being looked up includes 'secure' as a substring:

+ if (DNS_PACKET_RCODE(p) == DNS_RCODE_NXDOMAIN && t->current_feature_level >= DNS_SERVER_FEATURE_LEVEL_EDNS0) {
+
+ char key_str[DNS_RESOURCE_KEY_STRING_MAX];
+ dns_resource_key_to_string(t->key, key_str, sizeof key_str);
+ if (strstr(key_str, "secure") != NULL) {
+ t->current_feature_level = t->current_feature_level - 1;
+
+ log_warning("Server returned error %s, suspecting DNS violation DVE-2018-0001, retrying transaction with reduced feature level %s.",

The packet capture shows a number of DNS lookups, but not containing the substring 'secure'; and none that appear to correspond to the captive portal itself. This may require a different sort of solution than the previous bug, I'm not sure.

Revision history for this message
Dimitri John Ledkov (xnox) wrote :

I'm now questioning if it is at all sensible to use EDNS by default.

It seems like the fallback should be widened for all NXDOMAIN lookups.

I've validated, that lookups fail, when DNSSEC enabled (however with a SERVFAIL, rather than NXDOMAIN). Note DNSSEC is not enabled by default.

Revision history for this message
Mark Daku (markdaku) wrote :

I have encountered this bug as well.

I have raised it with systemd-resolved.
The internal resolver of systemd does not properly search the local dns. Only fqdn's will resolve. This is easily mitigated and does not require a whole bunch of enabling or disabling of things.

The DHCP process will almost always provide a valid DNS locally and the DHCP client in systemd does actually create a valid resolve.conf. You just have to point /etc/resolv.conf to it.

At the time of writing the systemd people refuse to accept that this is a bug. They continue to refer to it as a documentation issue. I honestly don't believe they are testing it.

# Just making sure I'm out of the / dir
cd /etc
# resolv.conf is a link to ../run/systemd/resolve/stub-resolv.conf Which directs thing to use the local resolved. So I nuke it.
sudo rm resolve.conf
# Then I go and use the actual resolv.conf that actually works.
sudo ln -s ../run/systemd/resolve/resolv.conf resolv.conf

The above workaround is good. It will allow you to take your laptop onto all different types of networks. It will work correctly. This is for at least systemd version 237 and above. I haven't checked to see if 236 has the proper file generated or not.

Revision history for this message
Steve Langasek (vorlon) wrote :

> The internal resolver of systemd does not properly search the local dns.
> Only fqdn's will resolve. This is easily mitigated and does not require
> a whole bunch of enabling or disabling of things.

That is unrelated to this bug report.

Revision history for this message
Mark Schroeder (earth2mark-eeepc) wrote :

I have the latest systemd recommended by #1727237, and this issue still occurs at Starbucks as well. The Starbucks WIFI defect is #1767900 which appears to be a duplicate of this one.

ii systemd 237-3ubuntu10 amd64 system and service manager

I'm in a Starbucks now. #1727237 (the systemd above) does not fix my issue. aruba.odyssys.net is still not resolved using the stub-resolv.conf file (linked from /etc/resolv.conf).

cd /etc
sudo /bin/rm resolv.conf
sudo /bin/ln -s ../run/systemd/resolve/resolv.conf resolv.conf

Using the original resolv.conf however, as per the changes above, does work. I did not make any changes to /etc/hosts. The real resolv.conf above contains the google nameservers and is sufficient to resolve aruba.odyssys.net.

nameserver 8.8.8.8
nameserver 8.8.4.4
search home

Revision history for this message
Steve Langasek (vorlon) wrote :

I can reproduce @earth2mark-eeepc's findings at another Starbucks.

mike stewart (mdrmike)
summary: - DNS cannot be resolved in Hotel Hotspot
+ DNS cannot be resolved in Public / Hotel / Starbucks WiFi Hotspot
Revision history for this message
mike stewart (mdrmike) wrote :

This seems to affect many public Wifi Hotspots. A very common example is the new setup by Starbucks/Google (usually seen as **Google Starbucks Wifi**) in the US by (Tech behind it is https://globalreachtech.com/ with their http://odyssys.net/).

Under Ubuntu 16.04, Windows, Android, Mac OS, or iPhone, a WiFi guest user can connect without problem. However, since upgrading to Ubuntu 18.04, I've encountered this in at least three separate Starbucks stores in Southern California.

## WORKAROUND ##

An easier workaround, with fewer system modifications than the original bug report is to use `ip route` to determine the IP address of the host router, then add the hostname with that adress to `/etc/hosts`

Step by step details can be found here: https://askubuntu.com/a/1027605/139249

Changed in systemd (Ubuntu Cosmic):
status: Confirmed → Fix Committed
description: updated
Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in systemd (Ubuntu Artful):
status: New → Confirmed
Changed in systemd (Ubuntu Bionic):
status: New → Confirmed
Revision history for this message
Łukasz Zemczak (sil2100) wrote : Please test proposed package

Hello Pete, or anyone else affected,

Accepted systemd into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/systemd/237-3ubuntu10.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in systemd (Ubuntu Bionic):
status: Confirmed → Fix Committed
tags: added: verification-needed verification-needed-bionic
Revision history for this message
Pete (pt.pete.pt) wrote :

Hi,

I'd love to help but i won't be in the affected hotel anytime soon.

So i hope some of the others could test.

Best regards, Pete

Revision history for this message
Dimitri John Ledkov (xnox) wrote :

Using 237-3ubuntu10.1, with 8.8.8.8 as DNS server, the resolutions of nonexistant.ubuntu.com and secure.ubuntu.com both are retried with following messages in the journal:
systemd-resolved[1434]: Server returned error NXDOMAIN, mitigating potential DNS violation DVE-2018-0001, retrying transaction with reduced feature level UDP.

With 237-3ubuntu10, a similar message only appeared for "secure.ubuntu.com" NXDOMAIN query, but not for the nonexistant.ubuntu.com.

tags: added: verification-done verification-done-bionic
removed: verification-needed verification-needed-bionic
Revision history for this message
Steve Langasek (vorlon) wrote :

I confirm that this fixes the problem for the newly rolled out Starbucks aruba hotspots.

Revision history for this message
Tom McLaughlin (thomasjm42) wrote :

I confirm that 237-3ubuntu10.1 works also

Revision history for this message
Pete (pt.pete.pt) wrote :

@Tom: Than it's not the same issue. As this is the version the bug was reported with.

Revision history for this message
Pete (pt.pete.pt) wrote :

Correction: I did not see the .1 in your version number.

Jeff Nessen (jeff-jeffy)
Changed in systemd (Ubuntu Bionic):
status: Fix Committed → Fix Released
Revision history for this message
Jeff Nessen (jeff-jeffy) wrote :

Obviously I didn't just jump in and release a fix. I accidentally updated the status and can't set it back. Sorry folks. I didn't mean to get anyone excited about the fix being out.

Changed in systemd (Ubuntu Bionic):
status: Fix Released → Fix Committed
Revision history for this message
Steve Langasek (vorlon) wrote :

Hello Pete, or anyone else affected,

Accepted systemd into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/systemd/237-3ubuntu10.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

tags: added: verification-needed verification-needed-bionic
removed: verification-done verification-done-bionic
Revision history for this message
Renan DelValle (rdelvalle) wrote :

237-3ubuntu10.2 fixed the issue for me in Bionic. With this version installed I was able to successfully log into the wifi on the train in the Netherlands. Previously I could not load the accept page.

tags: added: verification-done-bionic
removed: verification-needed-bionic
tags: added: verification-done
removed: verification-needed
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package systemd - 237-3ubuntu10.2

---------------
systemd (237-3ubuntu10.2) bionic; urgency=medium

  * logind: backport v238/v239 fixes for handling DRM devices.
    These changes introduce all the fixes that correct handling of open fd's
    related to the DRM devices, as used by for example NVIDIA GPUs. This backport
    includes some refactoring, corrections, and comment updates. This to insure
    that correct history is preserved, code comments match reality, and to ease
    backporting logind fixes in the future SRUs. (LP: #1777099)
  * Disable dh_installinit generation of tmpfiles for the systemd package.
    Replace with a manual safe call to systemd-tmpfiles which will process any
    updates to the tmpfiles shipped by systemd package, taking into account any
    overrides shipped by other packages, sysadmin, or specified in the runtime
    directories. (LP: #1748147)

systemd (237-3ubuntu10.1) bionic; urgency=medium

  [ Dimitri John Ledkov ]
  * hwdb: Fix wlan/rfkill keycode on Dell systems. (LP: #1762385)
  * Cherrypick upstream fix for corrected detection of Virtualbox & Xen.
    (LP: #1768104)
  * Further improve captive portal workarounds.
    Retry any NXDOMAIN results with lower feature levels, instead of just those
    with 'secure' in the domain name. (LP: #1766969)

  [ Michael Biebl ]
  * Add dependencies of libsystemd-shared to Pre-Depends.
    This is necessary so systemctl is functional at all times during a
    dist-upgrade. (Closes: #897986) (LP: #1771791)

  [ Mario Limonciello ]
  * Fix hibernate disk offsets.
    Configure resume offset via sysfs, to enable resume from a swapfile.
    (LP: #1760106)

 -- Dimitri John Ledkov 🌈 <email address hidden> Fri, 22 Jun 2018 13:55:09 +0100

Changed in systemd (Ubuntu Bionic):
status: Fix Committed → Fix Released
Revision history for this message
Łukasz Zemczak (sil2100) wrote : Update Released

The verification of the Stable Release Update for systemd has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Pete (pt.pete.pt) wrote :

Hi,

for me it seems to work with latest bionic as well.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package systemd - 239-7ubuntu4

---------------
systemd (239-7ubuntu4) cosmic; urgency=medium

  * Workaround broken meson copying symlinked data files, as dangling symlinks.

 -- Dimitri John Ledkov <email address hidden> Wed, 22 Aug 2018 14:11:35 +0100

Changed in systemd (Ubuntu Cosmic):
status: Fix Committed → Fix Released
Revision history for this message
Stuart Gillies (gillies) wrote :

You may think you fixed it but I'm sitting in a hotel in The Netherlands with a connected phone and tablet but not the laptop. Grrr! I have 18.04 up to date yesterday. Wifi says it is connected but hotel logon page won't appear. Is there a usable workaround?

Revision history for this message
Ole (ole-ersoy) wrote :

Yes - Catch a flight to the US and go to a starbucks :). Perhaps you can use the Starbucks approach I documented here:
https://askubuntu.com/questions/1023429/cant-connect-to-google-starbucks-wifi-on-ubuntu-17-10

Changed in systemd (Ubuntu Artful):
status: Confirmed → Won't Fix
Revision history for this message
Michael Plourde (mplourde) wrote :

We are using lxc Ubuntu 18.04 on proxmox -5.3-7.
Network configuration obtain through DHCP.
We have desactivate /etc/resolv.conf modification over proxmox to get DHCP DNS use:
touch .pve-ignore.resolv.conf

Everything line in /etc/resolv.conf is comment:
# cat /etc/resolv.conf
# --- BEGIN PVE ---
#search digicom.ca
#nameserver 8.8.8.8
# --- END PVE ---

Network status shown DNS entry is OK:
#networkctl status
● State: routable
       Address: 172.16.8.23 on eth0
                fe80::a0c5:bbff:fe4f:bc on eth0
       Gateway: 172.16.8.1 (ICANN, IANA Department) on eth0
           DNS: 104.192.16.1
           NTP: 172.16.8.3
                104.192.20.4
                66.70.172.17

systemd-resolve works fine:

#systemd-resolve go.com
go.com: 23.236.60.174

-- Information acquired via protocol DNS in 1.3ms.
-- Data is authenticated: no

But resolve name using ping command or host failed:

# ping go.com
ping: go.com: Temporary failure in name resolution
# host go.com
;; connection timed out; no servers could be reached

The work around is to manually create a link to the systemd-resolve conf file:
mv /etc/resolv.conf /etc/resolv.conf.orig
ln -s /run/systemd/resolve/resolv.conf /etc/resolv.conf

Revision history for this message
Ubfan (ubfan1) wrote :

The fix that worked for me is to add the libnss-resolve package. This changes the /etc/nsswitch.conf file to actually attempt the dns lookup after the other conditions have started the UDB fallback mode.
Bug 1727237 had a comment requesting if libnss-resolve were installed, answered "no", but no further suggestions regarding the package. The package changes the hosts line in the nsswitch.conf to:
hosts: files mdns4_minimal [NOTFOUND=return] resolve [!UNAVAIL=return] dns myhostname

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.