Ubuntu 17.04/17.10: New feature in NetworkManager stops several WiFi adapters from working (MAC Address Randomization issue)

[Jesse]

According to
https://blogs.gnome.org/thaller/2016/08/26/mac-address-spoofing-in-networkmanager-1-4-0/
there is a new privacy feature in the new version of NetworkManager.
This privacy feature can cause some USB WiFi adapters to stop working
while they used to work with older versions of NetworkManager (Ubuntu 16.10 or older).

The purpose of this privacy feature is to get your computer to report a new random MAC address whenever you connect to a WiFi network.
This privacy feature is especially useful when you connect to public WiFi networks, so that the operators cannot identify you when you connect multiple times.

The downside of this privacy feature is that some USB WiFi adapters misbehave when NetworkManager tries to change their MAC address repeatedly.
The result is that those USB WiFi adapters cannot connect anymore to the WiFi network.

Original report follows:

My Panda USB wi-fi adapter works just fine on 16.10, but when I try to connect to my wi-fi router in 17.04, GNOME network manager reports "Connection failed." I did some tinkering, and noticed that my MAC address for my wifi adapter, according to GNOME, is DIFFERENT every time I make it forget my wifi settings and try to reconnect. Weird, right? Any leads on a possible fix or work-around?

I'm running the latest beta of Ubuntu GNOME 17.04, kernel 4.10.0-19-generic, GNOME 3.24.0.

https://blogs.gnome.org/thaller/2016/08/26/mac-address-spoofing-in-networkmanager-1-4-0/

[Jesse (jessedubord)]

Miraculously, a user from omgubuntu.co.uk had written a comment in one of their articles announcing the release of 17.04 that fixed this issue. The comment is as follows:

"# wifi and networkmanager

The new NetworkManager release 1.4.0 adds new features to change the current MAC address of your Ethernet or Wi-Fi card (spoofing/cloning). As a debian "sid" user i understand that being on bleeding edge, sometimes new features and configs can be Incompatible with my machine hardware!!! Now being a ubuntu user i can see that ubuntu is losing his "user friendly" name and no more care for desktop users! This was an unacceptable default config on Ubuntu's best days! Ubuntu haters and criticists must be happy now!

# fix it

edit the /etc/NetworkManager/NetworkManager.conf file

And add:

[device]
wifi.scan-rand-mac-address=no

Restart the networkmanager service and use your wireless connection :)"

I found that applying his fix by opening a Terminal and typing "sudo gedit /etc/NetworkManager/NetworkManager.conf", adding the text he described, then rebooting my machine fixed the issue.

That being said, this default behavior is still a major bug and breaks functionality of wi-fi hardware for users, and definitely needs to be looked at. Thanks for your time!

[Carnalbeast (carnalbeast)]

Congratulations ! You just released an Ubuntu version that will not allow most of the users to connect to wireless networks ! This is unacceptable !
So many users will trash their dvds after seeing this bug !
I'm so in to moving to another Linux distro !

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[woergi (woergi-ch)]

I can reproduce the bug on the Ubuntu Gnome Release version with a 'TP-Link TL-WN822N' and a 'USRobotics USR5423' as well.

[Christopher (soft-kristal)]

Comment #1 fixed my problem. Oddly, it wasn't an issue on my main desktop.

[Mike_IronFist (fistx)]

Hey friends, make sure you click "this bug affects me too" so that anyone who reads this report will know it's affecting a lot of users.

I can reproduce the bug on Ubuntu 17.04, Ubuntu Mate 17.04 and Ubuntu GNOME 17.04.
The workaround in the first comment here does fix the issue for me as well so it appears to just be a poor default configuration.

I'm using a Panda Wireless USB adapter, which has an Ralink RT5372 chipset in it.

[PioT (stephane-piotrowski-gmail)]

Thanks Jesse for the workaround !

[Dirk Schmidtke (dirkschmidtke)]

Same problem on SurfacePro 2 with Marvell chipset. Fixed it by editing /etc/NetworkManager/NetworkManager.conf. Ubuntu should fix this config glitch out of the box.

[Jesse (jessedubord)]

Happy to oblige, Piot! :) And thanks for the update and explanation of the bug, Simos.

I'm quite curious if a fix will be implemented, somehow, and re-packaged in an updated ISO for Ubuntu 17.04 or Ubuntu GNOME 17.04? Might be a big issue if WiFi doesn't work for clients out of the box, yeah?

[Simos Xenitellis  (simosx)]

Jesse, I do not have 17.04 so I cannot test these myself. You guys could help:

At https://blogs.gnome.org/thaller/2016/08/26/mac-address-spoofing-in-networkmanager-1-4-0/ it mentions that:

Update-2017-01-25: with 1.6 release and newer, the default value changed from “permanent” to “preserve” [commit],[bug].

The "preserve" default value means that NM will not attempt to change the MAC address upon activation.
I just checked the source of "network-manager" on 17.04 (using LXD with 17.04 container) and it goes not have this update.

Could you please check in NetworkManager to see what is the default value for
“ethernet.cloned-mac-address” and “wifi.cloned-mac-address”, on a Ubuntu 17.04 installation?

I think that "nmcli device show" might show it. If not, try somehow to figure out the default.

[Don Cady (doncady)]

I have this problem too. Adding wifi.scan-rand-mac-address=no to NetworkManager.conf does resolve it.

A relevant question: Why does this only affect usb wifi dongles? My internal iwl3945 chip was/is not affected. How does Network Manager treat pci and usb devices differently?

[wolfy1339 (wolfy1339)]

This affects Microsoft Surface Pro devices (internal WiFi adapter)

[Simos Xenitellis  (simosx)]

@Don Cady:

The source of the problem is the Linux kernel driver for the specific WiFi adapter or perhaps the firmware of the WiFi adapter.

In most of the drivers, they have implemented the facility to change (by software) the MAC address.
There are even tools to change the MAC address, like "macchanger". Or, you can even use "ifconfig wlan0 hw ether 00:11:22:33:44:55". Or, you can change it with NetworkManager, a facility available for many years already.

A WiFi card can support many other interesting features, including "AP Mode" (make your Ubuntu act like an Access Point), and "Monitor" mode (using those Aircrack tools). However, it takes effort to reverse-engineer AP Mode or Monitor mode, so many adapters do not support these features. It is much more common in USB adapters not to have full support for these features, and in some cases it is a deficiency in the firmware (firmware files: /lib/firmware/, and nothing can be done about it).

It is not clear how widespread is the problem. If you are reporting, please include which kernel driver is used for the WiFi and also what firmware is loaded.

@wolfy1339:
I think the MS Surface Pro has an (internal) USB WiFi card and uses the kernel driver "mwifiex_usb". Have a look in "lsmod".

[Jeremy Bícha (jbicha)]

I filed https://bugzilla.gnome.org/show_bug.cgi?id=781295 requesting that this setting be configurable in GNOME Settings>Network.

[Steve Kellett (omikl)]

I have tried the above work around, unfortunately without success.

Dell Inspiron 1012 running 17.04

Kernel 4.8.0-47

I have the same symptoms: WiFi fails to connect repeatedly.

[jose orlando (jorlando)]

Also affected by this bug on Ububtu 17.04. Using my computer as a wifi hotspot I couldn't connect from other devices and the log presented a lot of "inux wpa_supplicant[1385]: handle_probe_req: send failed
" messages.

The solution presented by Jesse (juderichbourne) on post #1 solved the issue.

[Jeremy (wa113y3s)]

Some users may need to disable wifi power management with

sudo sed -i 's/wifi.powersave = 3/wifi.powersave = 2/' /etc/NetworkManager/conf.d/default-wifi-powersave-on.conf

The command is one line and I know some Ralink USB devices work poorly with power management enabled

[Kubus Puchatek (kpuchatek)]

I am also affected by this bug. I was about to delete Ubuntu 17.04 and rollback to Mint 18.1 after many hours of fight. This one line makes magic, but I had to read thousands of lines before finding it.
My Wifi USB is Ralink (now Mediatek). I am using hidden ssid and mac filtering+WPA2 Personal on router.
On laptop integrated Intel Wifi no issue.

Ubuntu team - please do not make such mistakes ! User experience is completely broken.

[Jeremy Bícha (jbicha)]

This commit might help:

https://cgit.freedesktop.org/NetworkManager/NetworkManager/commit/?h=nm-1-4&id=cb18faf2d

[Russell Smith (qqrs)]

Comment #1 also fixed the problem for me. Ubuntu 17.04. Panda Wireless PAU06 Wi-Fi adapter.

[Marcos Nascimento (wstlmn)]

It also does not work in 17.10. He tries three times, but without success.

[fantonia (antoniazzifabio)]

My problem is a bit different. Eth0 connects at first but then disconnects. Then reconnect and work for a short time. stress! I solved it with wicd. I have a Huawei HG8346M router. I've tried other distro but it's the same thing. Neon 17.10.

[Adam Dingle (adam-yorba)]

The solution in comment #1 fixed this for me as well. I'm running 17.04 and am hosting a hotspot on a ThinkPenguin Wireless N USB Adapter. It didn't work until I applied this workaround.

[Marcos Nascimento (wstlmn)]

I installed the future ubuntu 17.10 and found the same problem. Searching I found the following solution:

1) To stabilize the device:

$ Sudo apt install dnsmasq-base
$ Sudo service network-manager restart

2) To correct authentication error:

$ Sudo nano /etc/NetworkManager/NetworkManager.conf

Add in the end:
[Device]
Wifi.scan-rand-mac-address = no

$ Sudo service network-manager restart

After this procedure, it worked perfectly.
I consider the solution provisional and I hope they will correct it soon.

Source:
http://ubuntuforum-br.org/index.php/topic,121686.0.html

[aleandro (aleandrodasilva)]

At 15th June 2017 under Ubuntu 17.04, upgraded to latest packages, I'm still struggling with wifi. All solutions proposed do not work. I have continuous disconnections.

It is a bad bug for a new release. You won't be able to surf without an Ethernet cable.

[Simos Xenitellis  (simosx)]

@Marcos:

Thanks for reporting that the same issue exists in current 17.10.
There has been a change in the default settings in NetworkManager in upstream
so that the MAC address randomization is not enabled by default.
Currently, Ubuntu 17.10 (daily) has NetworkManager 1.4.4.
I suppose it will get upgraded to 1.6.2 (https://wiki.gnome.org/Projects/NetworkManager)
which most probably has the new change.

@aleandro: Most probably something else is wrong with your case.
If you can connect for a bit and soon you get disconnected, then definitely something else is the problem.
Can you file another report for your case?
You can do that by running in a terminal the command:

ubuntu-bug network-manager

and then follow the instructions.

[PJSingh5000 (pjsingh5000)]

In my logs, I see the message:
"...aborting authentication ...by local choice (Reason: 3=DEAUTH_LEAVING)"
(I've listed below the relevant portion of the log file, since the logs above do not include this particular error).

The solution posted in comment #1 (https://bugs.launchpad.net/ubuntu/+source/network-manager/+bug/1681513/comments/1) worked for me...

    $ sudo nano /etc/NetworkManager/NetworkManager.conf
    # Append...
    - - - - - - - - - - - - - - - - - - - -
        [Device]
        wifi.scan-rand-mac-address=no
    - - - - - - - - - - - - - - - - - - - -
    $ sudo service network-manager restart

I did not need to install dnsmasq-base, as suggested in comment #24 (https://bugs.launchpad.net/ubuntu/+source/network-manager/+bug/1681513/comments/24).
What is the purpose of installing this?

- - - - - - - - - - - - - - - - - - - -
SYSTEM INFO
- - - - - - - - - - - - - - - - - - - -
Linux RS001 4.10.0-22-generic #24-Ubuntu SMP Mon May 22 17:43:20 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 17.04
Release: 17.04
Codename: zesty
- - - - - - - - - - - - - - - - - - - -

- - - - - - - - - - - - - - - - - - - -
LOG
- - - - - - - - - - - - - - - - - - - -
Jun 19 19:20:42 MY_COMPUTER NetworkManager[724]: <info> [1497914442.1301] device (wlx00227540404a): supplicant interface state: disconnected -> scanning
Jun 19 19:20:43 MY_COMPUTER wpa_supplicant[855]: wlx00227540404a: SME: Trying to authenticate with b4:75:0e:69:96:af (SSID='MY_WIFI' freq=2462 MHz)
Jun 19 19:20:43 MY_COMPUTER kernel: [ 3001.130115] wlx00227540404a: authenticate with b4:75:0e:69:96:af
Jun 19 19:20:43 MY_COMPUTER kernel: [ 3001.209885] wlx00227540404a: send auth to b4:75:0e:69:96:af (try 1/3)
Jun 19 19:20:43 MY_COMPUTER NetworkManager[724]: <info> [1497914443.7071] device (wlx00227540404a): supplicant interface state: scanning -> authenticating
Jun 19 19:20:43 MY_COMPUTER kernel: [ 3001.231002] wlx00227540404a: authenticated
Jun 19 19:20:45 MY_COMPUTER kernel: [ 3003.003601] wlx00227540404a: aborting authentication with b4:75:0e:69:96:af by local choice (Reason: 3=DEAUTH_LEAVING)
Jun 19 19:20:45 MY_COMPUTER NetworkManager[724]: <warn> [1497914445.4980] device (wlx00227540404a): Activation: (wifi) association took too long, failing activation
Jun 19 19:20:45 MY_COMPUTER NetworkManager[724]: <info> [1497914445.4980] device (wlx00227540404a): state change: config -> failed (reason 'ssid-not-found') [50 120 53]
Jun 19 19:20:45 MY_COMPUTER NetworkManager[724]: <info> [1497914445.4984] manager: NetworkManager state is now DISCONNECTED
Jun 19 19:20:45 MY_COMPUTER NetworkManager[724]: <warn> [1497914445.5053] device (wlx00227540404a): Activation: failed for connection 'MY_WIFI 1'
Jun 19 19:20:45 MY_COMPUTER NetworkManager[724]: <info> [1497914445.5111] device (wlx00227540404a): state change: failed -> disconnected (reason 'none') [120 30 0]
Jun 19 19:20:45 MY_COMPUTER wpa_supplicant[855]: wlx00227540404a: CTRL-EVENT-DISCONNECTED bssid=b4:75:0e:69:96:af reason=3 locally_generated=1
Jun 19 19:20:45 MY_COMPUTER kernel: [ 3003.063699] IPv6: ADDRCONF(NETDEV_UP): wlx00227540404a: link is not ready
Jun 19 1...

[PJSingh5000 (pjsingh5000)]

I forgot to add, my wi-fi adapter info is:

$ lsusb | grep Wireless
Bus 001 Device 002: ID 050d:825b Belkin Components F5D8055 N+ Wireless Adapter v2000 [Ralink RT3072]

And I believe it is using the following driver/module:

$ lsmod
rt2x00usb 20480 1 rt2800usb

Located at...
/lib/modules/4.10.0-22-generic/kernel/drivers/net/wireless/ralink/rt2x00/rt2800usb.ko

[PJSingh5000 (pjsingh5000)]

FYI, just found out the hard way that "Device" needs to be lower case!...

    $ sudo nano /etc/NetworkManager/NetworkManager.conf
    # Append...
        - - - - - - - - - - - - - - - - - - - -
        [device]
        wifi.scan-rand-mac-address=no
        - - - - - - - - - - - - - - - - - - - -
    $ sudo service network-manager restart

[Chris Moore (moore-a)]

I also have this problem running Ubuntu 17.04 with a TrendNet TEW-644UB USB WiFi stick (ID 148f:2770 Ralink Technology, Corp. RT2770 Wireless Adapter).

The symptoms are that it sees the networks but won't connect, even if unencrypted.

The workaround in #1 also works for me.
Many thanks, Jesse.

This bug shouldn't be in a final release and it is inexcusable that it still hasn't been corrected two months later.
I expect that, like me, many people have wasted a lot of time before finding this workaround.
(I even ordered a new USB WiFi stick.)

[Gregg Berkholtz (TOCICI) (tocici)]

Also confirming for Surface Pro 1:
WiFi would report successful association, and then immediately drop the connection giving "reason code 2" and "CMD_RESP: cmd 0x10f error, result=0x2" as the only hints.

Disabling MAC randomization fixed the issue.

[Robin King (robinhking)]

Many, many thanks, Jesse, for the fix! I have been using an ASUS N-13 adaptor, and up to Ubuntu 17.04 had no trouble. At first I suspected the adaptor and bought another one - quite unnecessarily as it turns out. Fortunately I had my little old friend, an ASUS Eee (also running Ubuntu 17.04), to help me, and so I was able to use it to ferret about, finally ending up at your posting. It worked! Evidently - and fortunately for me - the "improvement" to the network manager doesn't affect the Eee wifi card.

Couldn't agree more with the above comments about the irresponsibility of the development team in fielding such a vital piece of software without very careful and extensive testing. Shame on you lot! There are probably a lot of ordinary users like me out there who just don't have a clue what is happening now their network connection isn't working.

[Kerry Billingham (y-launchpad-avionicengineers-com)]

This issue I have is when I first boot my laptop (HP Envy Ubuntu 17.04) the wireless will not connect. If I cycle wireless button then I get a connection.

I found that if I use the work-around described above (don't use random mac address) then it appears that I do not have this issue.

One other point I also need to check. A long time ago I may have set my wireless router to assign specific IP addresses to specific MAC addresses so may have been the cause of my issue.

[Forage (forage)]

I have this same problem with an WiFi USB dongle:
Bus 005 Device 003: ID 13b1:0020 Linksys WUSB54GC v1 802.11g Adapter [Ralink RT73]

The provided work-around in comment #1 does the trick for me.

[Vince (collisv)]

Same problem on Asus EEEpc1001px with an Aetheros Ar2427 card

[NoName (rooster01)]

Device = BCM43142 802.11b/g/n
Manufacturer = Broadcom Limited
Driver = wl0
Driverversion = 6.30.223.271 (r587334)

I have this same problem with a build-in wifi

I had to add:

[Device]
wifi.scan-rand-mac-address=no

to NetworkManager.conf to make it work

[NoName (rooster01)]

Same problem with a build-in laptop wifi

Device = BCM43142 802.11b/g/n
Manufacturer = Broadcom Limited
Driver = wl0
Driverversion = 6.30.223.271 (r587334)

Solved by adding:

[Device]
wifi.scan-rand-mac-address=no

to NetworkManager.conf && restart network-manager service

[Launchpad Janitor (janitor)]

This bug was fixed in the package network-manager - 1.8.2-1ubuntu3

---------------
network-manager (1.8.2-1ubuntu3) artful; urgency=medium

  [ James Henstridge ]
  * Backport connectivity checking API from master. (LP: #1704162)

  [ Jeremy Bicha ]
  * NetworkManager.conf: disable MAC randomization feature. There is no
    easy way for desktop users to disable this feature yet. And there are
    reports that it doesn't work well with some systems. (LP: #1681513)

 -- James Henstridge <email address hidden> Tue, 22 Aug 2017 17:18:49 +0800

[Matthias (hering33)]

I've same problem under Ubuntu 17.04 since mainline kernel 4.13.2 is installed.

My Wifi dobngle is connecting to router but no internet connection will be established, the last working kernel is the mainline kernel 4.12.13.

[Matthias (hering33)]

oh shit i've forget to insert my Wifi adapter in my post.

is a rtl8188eus chpset

[Abhishek Kumar (kumar-abhishek0116)]

Thanks my problem has been solved from # fix it
edit the /etc/NetworkManager/NetworkManager.conf file
And add:
[device]
wifi.scan-rand-mac-address=no
Restart the networkmanager service and use your wireless connection :)"
I found that applying his fix by opening a Terminal and typing "sudo gedit /etc/NetworkManager/NetworkManager.conf", adding the text he described, then rebooting my machine fixed the issue.

[DougL (dlarue)]

I too ran up against this as I updated some custom bootable systems based on 17.04 and many of my test network dongles would not work. Took lots of time finding this solution, ie adding

[device]
wifi.scan-rand-mac-address=no

to the /etc/NetworkManager/NetworkManager.conf file.

[Walttheboss (walttheboss)]

Just installed 17.10.1 on 64 bit system.

Same issues as described above. The aformentioned line (wifi.scan . . . ) was already there. So someone is paying attention.

BUT WiFi will not connect unless you go through the KDE Wallet System. I tried for hours. Finally enabled the wallet and it connects just fine.

[Jeremy Bícha (jbicha)]

Ubuntu 17.04 reached End of Life in January 2018. Please upgrade to Ubuntu 17.10 which is not affected by this bug.

Ubuntu releases and their End of Life dates can be found at
https://wiki.ubuntu.com/Releases