n-m-openvpn shuts down VPN when openvpn soft-restarts

[Thierry Carrez]

Binary package hint: network-manager-openvpn

Using openvpn rc9-3ubuntu2 and network-manager-openvpn 0.7~~svn20080928t225540-0ubuntu1.

When openvpn sends itself a SIGUSR1, network-manager-openvpn calls nm-openvpn-service-openvpn-helper with the "restart" parameter but that fails, resulting in the tearing down of the openvpn connection :

nm-openvpn[5682]: [server] Inactivity timeout (--ping-restart), restarting
nm-openvpn[5682]: SIGUSR1[soft,ping-restart] received, process restarting
[...]
nm-openvpn[5682]: UDPv4 link remote: 192.168.122.1:1194
nm-openvpn[5682]: [server] Peer Connection Initiated with 192.168.122.1:1194
nm-openvpn[5682]: Preserving previous TUN/TAP instance: tun0
nm-openvpn[5682]: /usr/lib/network-manager-openvpn/nm-openvpn-service-openvpn-helper tun0 1500 1541 10.8.0.6 10.8.0.5 restart
NetworkManager: <info> VPN plugin failed: 2
nm-openvpn[5682]: script failed: external program exited with error status: 1
nm-openvpn[5682]: Exiting
NetworkManager: <info> VPN plugin failed: 1
NetworkManager: <info> VPN plugin state changed: 6
NetworkManager: <info> VPN plugin state change reason: 0
NetworkManager: <WARN> connection_state_changed(): Could not process the request because no VPN connection was active.
NetworkManager: nm_system_device_flush_ip4_routes_with_iface: assertion `iface_idx >= 0' failed
NetworkManager: nm_system_device_flush_ip4_addresses_with_iface: assertion `iface_idx >= 0' failed
NetworkManager: <info> Policy set (eth0) as default device for routing and DNS.
nm-dispatcher.action: Script '/etc/NetworkManager/dispatcher.d/01ifupdown' exited with error status 1.
NetworkManager: <debug> [1223468236.827482] ensure_killed(): waiting for vpn service pid 5674 to exit
NetworkManager: <debug> [1223468236.828433] ensure_killed(): vpn service pid 5674 cleaned up

That can occur by default just by inactivity : when connected using a udp-based openvpn, by default it uses "ping-restart 120" which will result in openvpn sending a SIGUSR1 to itself after 120 seconds of VPN inactivity.

[Thierry Carrez (ttx)]

Analysis:

network-manager-openvpn calls openvpn with the "--up /usr/lib/network-manager-openvpn/nm-openvpn-service-openvpn-helper" and "--up-restart" parameters. This means that nm-openvpn-service-openvpn-helper will be called when the VPN is initiated ("init" parameter) and restarted ("restart" parameter).

However the environment passed to the script is slightly different in the init and the restart cases. In particular, the "ifconfig_local" environment variable is apparently not set on the restart call.

nm-openvpn-service-openvpn-helper fails if that env var is not set (in file src/nm-openvpn-service-openvpn-helper.c):
    val = addr_to_gvalue (getenv ("ifconfig_local"));
    if (val)
        g_hash_table_insert (config, NM_VPN_PLUGIN_IP4_CONFIG_ADDRESS, val);
    else
        helper_failed (connection, "IP4 Address");

One way to avoid this is to drop the else helper_failed (like in the ifconfig_remote case that follows)...

[Thierry Carrez (ttx)]

Hm. Except that apparently it needs that information :

nm-openvpn[5945]: [server] Inactivity timeout (--ping-restart), restarting
nm-openvpn[5945]: SIGUSR1[soft,ping-restart] received, process restarting
[...]
nm-openvpn[5945]: /usr/lib/network-manager-openvpn/nm-openvpn-service-openvpn-helper tun0 1500 1541 10.8.0.6 10.8.0.5 restart
NetworkManager: <info> VPN connection 'VPN connection 1' (IP Config Get) reply received.
NetworkManager: nm_vpn_connection_ip4_config_get: invalid IP4 config received!
kernel: [ 480.167409] NetworkManager[4859]: segfault at 8 ip 000000000043fb1c sp 00007fff68955c80 error 4 in NetworkManager[400000+65000]

The proper way to do it is probably to get the information from the command-line rather than the environment. The local ip address is the 4th parameter in the nm-openvpn-service-openvpn-helper call.
<cmd> tun_dev tun_mtu link_mtu ifconfig_local_ip ifconfig_remote_ip [ init | restart ]

[Sergio Gómez (perseo)]

I'm the one from #270630 bug. Openvpn is not working yet with my server configuration. There are a lot of people using VPN with Ubuntu Hoary in my University and if they can't connect with Intrepid, probably they will use Windows client to connect.

I'd like to help, but I have no idea how I can do this.

In Hoary, I can configure link-mtu, cipher and keysize. In new version I can't and NM-0.7 is logging that parameters from client and server are inconsistent:

Oct 12 15:02:51 apolo nm-openvpn[15415]: WARNING: this configuration may cache passwords in memory -- use the auth-nocache option to prevent this
Oct 12 15:02:53 apolo nm-openvpn[15415]: WARNING: 'link-mtu' is used inconsistently, local='link-mtu 1544', remote='link-mtu 1560'
Oct 12 15:02:53 apolo nm-openvpn[15415]: WARNING: 'cipher' is used inconsistently, local='cipher BF-CBC', remote='cipher AES-256-CBC'
Oct 12 15:02:53 apolo nm-openvpn[15415]: WARNING: 'keysize' is used inconsistently, local='keysize 128', remote='keysize 256'

And then:

Oct 12 15:03:20 apolo nm-openvpn[15415]: [vpn1] Peer Connection Initiated with X.X.X.X:1194
Oct 12 15:03:21 apolo nm-openvpn[15415]: Preserving previous TUN/TAP instance: tun0
Oct 12 15:03:21 apolo nm-openvpn[15415]: /usr/lib/network-manager-openvpn/nm-openvpn-service-openvpn-helper tun0 1500 1544 192.168.132.18 192.168.132.17 restart
Oct 12 15:03:21 apolo NetworkManager: <info> VPN plugin failed: 2
Oct 12 15:03:21 apolo nm-openvpn[15415]: script failed: external program exited with error status: 1
Oct 12 15:03:21 apolo nm-openvpn[15415]: Exiting

I'll be pleased to help if someone says me how.

[Thierry Carrez (ttx)]

When the up script is called at restart, openvpn doesn't set ifconfig_remote, route_gateway_1, ifconfig_local, route_network_1, route_net_gateway, route_vpn_gateway and route_netmask_1, resulting in nm-openvpn-service-openvpn-helper failing.

I've submitted this upstream as :
http://bugzilla.gnome.org/show_bug.cgi?id=556134

One way for us around it is to disable the --up-restart parameter call in src/nm-openvpn-service.c, though the proper fix is probably to make n-m-openvpn use the old values if none are set by the --up-restart script call...

[Sergio Gómez (perseo)]

As I supposed, the problem is here:

Oct 12 15:02:53 apolo nm-openvpn[15415]: WARNING: 'cipher' is used inconsistently, local='cipher BF-CBC', remote='cipher AES-256-CBC'

I'm using a different cipher method. I modified the nm-openvpn-service.c and change this:

 /* Cipher */
 tmp = g_hash_table_lookup (properties, NM_OPENVPN_KEY_CIPHER);
 if (tmp && strlen (tmp)) {
  add_openvpn_arg (args, "--cipher");
  add_openvpn_arg (args, tmp);
 }

for this:

  add_openvpn_arg (args, "--cipher");
  add_openvpn_arg (args, "AES-256-CBC");

I have no idea where I can configure cipher parameter in openvpn-helper. But this is the point of the problem.

Now the connection don't hang up anymore, but there are still another problem. It's the routes are not configured quite well. I'm going to continue with my research.

Thanks.

[Sergio Gómez (perseo)]

Definitely, that's the cause. With gconf-editor I goes to /system/networking/connections/n/vpn and added a key called cipher with AES-256-CBC value. Now it works as well as with the patch I did.

But I'm not pretty sure that it's the cause of this bug. I guess I have to open a new one. What do you say?

[Thierry Carrez (ttx)]

Sergio:
In fact your bug was about two issues. One is reported here (the fact that soft-restarting the connection essentially kills the VPN) and the other is that network-manager-openvpn doesn't expose the possibility to change the Cipher from the default value (BF-CBC) in the GUI (but you can do so in gconf-editor).

Thanks for the follow-up ! Feel free to file a wishlist bug to network-manager-openvpn about this.

[Sergio Gómez (perseo)]

Thanks for your support Thierry,

Better than that, I'm going to try to implement it and sent a patch.

[Vladimir Elizarov (xengelpublicx)]

hello. i'm get this error (attachment whith error)
How do fix it?

[Thierry Carrez (ttx)]

Vladimir: there is no easy workaround. openvpn routinely restarts and network-manager-openvpn doesn't handle that situation properly.
The bug has been submitted upstream at: http://bugzilla.gnome.org/show_bug.cgi?id=556134
In the meantime if you need to use OpenVPN but are hit by this bug, I'd suggest not using network-manager-openvpn and use openvpn directly.

[Vladimir Elizarov (xengelpublicx)]

Thierry: I saw openvpn connection. 30 minutes openvpn not restart session. may problem is not it?

[Martin Emrich (emme)]

I also see the "Script '/etc/NetworkManager/dispatcher.d/01ifupdown' exited with error status 1." message when starting/stopping a vpnc connection. I am running current jaunty i386.

[Marcus Brinkmann (marcus-brinkmann)]

The "'/etc/NetworkManager/dispatcher.d/01ifupdown' exited with error status 1." message comes from the script being called with the "vpn-up"/"vpn-down" second parameter, which it doesn't understand. See https://bugs.launchpad.net/network-manager/+bug/269071/+viewstatus

[Brett Delle Grazie (bdellegrazie)]

Specifically for the "'/etc/NetworkManager/dispatcher.d/01ifupdown' exited with error status 1." message please see the following separate bug report as this is a network-manager issue not specific to the openvpn package: Bug #330056 . A trivial patch is included there.

[rklz (maxi-debian)]

Plase , add this , the bug is affecting Kubuntu 10.10 Maverick Merkat as well .

Address the patch .

Thank you

[Daniel Dumitrache (dandu)]

I can confirm this bug in 10.10.

But, in my opinion this is an openvpn bug, because openvpn is the one calling the script with some missing environment variables.

I submitted this bug to openvpn: https://community.openvpn.net/openvpn/ticket/93

In the meantime I am using as a workaround a wrapper script over nm-openvpn-service-openvpn-helper (renamed to .dist):

$ cat /usr/lib/network-manager-openvpn/nm-openvpn-service-openvpn-helper
#!/bin/bash

[ -z "$ifconfig_local" ] && export ifconfig_local=$4
[ -z "$ifconfig_remote" ] && export ifconfig_remote=$5

/usr/lib/network-manager-openvpn/nm-openvpn-service-openvpn-helper.dist $*

[Colan Schwartz (colan)]

Added the upstream OpenVPN bug. As not much has changed here, this is also a problem in 11.04 (Natty).

[Craig (candrews-integralblue)]

Confirmed that the same problem still occurs in Oneiric, and the same workaround from coment 16 still works too.

[Julian Alarcon (julian-alarcon)]

I confirm this same bug in Ubuntu 12.04 LTS Precise Pangolin. The same workaround works:

Steps to workaround:

1. Change filename of /usr/lib/NetworkManager/nm-openvpn-service-openvpn-helper
cd /usr/lib/NetworkManager/
mv nm-openvpn-service-openvpn-helper nm-openvpn-service-openvpn-helper.dist

2. Create new file with same old name:
nano nm-openvpn-service-openvpn-helper

3. Fill the file with variables to export and original executable. Copy and paste this:

#!/bin/bash

[ -z "$ifconfig_local" ] && export ifconfig_local=$4
[ -z "$ifconfig_remote" ] && export ifconfig_remote=$5

/usr/lib/NetworkManager/nm-openvpn-service-openvpn-helper.dist $*

4. Make the new file executable:
chmod +x nm-openvpn-service-openvpn-helper

Now it works!

[Aleksey Sukhonosov (aleyer)]

I modified the latter script to suit my needs. I have 2 problematic network which I can't connect to because of wrong ifconfig_local and ifconfig_remote variables. They have different network prefixes (192.168.5.0 and 192.168.1.0) and because $4 and $5 variables are empty, I have to choose them manually. Here is the script for this task. The local address is set in the script, x.x.x.201 in my case.

#!/bin/bash
if [ -z "$ifconfig_local" ]
then
a=$(su $USER -c 'export DISPLAY=:0.0; zenity --list --title="Network addresses" --column="Remote" --column="Local" "192.168.5.1" "192.168.5.201" "192.168.1.1" "192.168.1.201" ')
export ifconfig_remote=$a
export ifconfig_local=${ifconfig_remote/%.1/.201}
fi
/usr/lib/NetworkManager/nm-openvpn-service-openvpn-helper.dist $*

[Phil Weir (phil-weir)]

This seems to still be a problem in both 13.04 and 13.10 - I used the workaround in #19 for 13.04 but, on upgrade, the script got overwritten and the problem returned. Repeating the process seems to have sorted it again.

[Alexey Osipov (lion-simba)]

Can't this be ported to Ubuntu 14.04?

[laluz (laluz)]

Seems to be still present in 17.04