VPNaaS: ipsec addconn failed

[baojie]

When create an ipsec-connection

2015-05-05 14:06:41.875 4555 DEBUG neutron.agent.linux.utils [-] Running command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qrouter-a9e53c63-23fa-4544-9ad4-cdaa480eb5de', 'ipsec', 'addconn', '--ctlbase', '/var/lib/neutron/ipsec/a9e53c63-23fa-4544-9ad4-cdaa480eb5de/var/run/pluto.ctl', '--defaultroutenexthop', '10.62.72.1', '--config', '/var/lib/neutron/ipsec/a9e53c63-23fa-4544-9ad4-cdaa480eb5de/etc/ipsec.conf', '94a916ff-375f-46e8-8c58-8231ce0eea1c'] create_process /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:46
2015-05-05 14:06:41.973 4555 ERROR neutron.agent.linux.utils [-]
2015-05-05 14:06:41.974 4555 ERROR neutron.services.vpn.device_drivers.ipsec [-] Failed to enable vpn process on router a9e53c63-23fa-4544-9ad4-cdaa480eb5de
2015-05-05 14:06:41.974 4555 TRACE neutron.services.vpn.device_drivers.ipsec Traceback (most recent call last):
2015-05-05 14:06:41.974 4555 TRACE neutron.services.vpn.device_drivers.ipsec File "/usr/lib/python2.7/site-packages/neutron/services/vpn/device_drivers/ipsec.py", line 242, in enable
2015-05-05 14:06:41.974 4555 TRACE neutron.services.vpn.device_drivers.ipsec self.restart()
2015-05-05 14:06:41.974 4555 TRACE neutron.services.vpn.device_drivers.ipsec File "/usr/lib/python2.7/site-packages/neutron/services/vpn/device_drivers/ipsec.py", line 342, in restart
2015-05-05 14:06:41.974 4555 TRACE neutron.services.vpn.device_drivers.ipsec self.start()
2015-05-05 14:06:41.974 4555 TRACE neutron.services.vpn.device_drivers.ipsec File "/usr/lib/python2.7/site-packages/neutron/services/vpn/device_drivers/ipsec.py", line 395, in start
2015-05-05 14:06:41.974 4555 TRACE neutron.services.vpn.device_drivers.ipsec ipsec_site_conn['id']
2015-05-05 14:06:41.974 4555 TRACE neutron.services.vpn.device_drivers.ipsec File "/usr/lib/python2.7/site-packages/neutron/services/vpn/device_drivers/ipsec.py", line 314, in _execute
2015-05-05 14:06:41.974 4555 TRACE neutron.services.vpn.device_drivers.ipsec check_exit_code=check_exit_code)
2015-05-05 14:06:41.974 4555 TRACE neutron.services.vpn.device_drivers.ipsec File "/usr/lib/python2.7/site-packages/neutron/agent/linux/ip_lib.py", line 550, in execute
2015-05-05 14:06:41.974 4555 TRACE neutron.services.vpn.device_drivers.ipsec check_exit_code=check_exit_code, extra_ok_codes=extra_ok_codes)
2015-05-05 14:06:41.974 4555 TRACE neutron.services.vpn.device_drivers.ipsec File "/usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py", line 84, in execute
2015-05-05 14:06:41.974 4555 TRACE neutron.services.vpn.device_drivers.ipsec raise RuntimeError(m)
2015-05-05 14:06:41.974 4555 TRACE neutron.services.vpn.device_drivers.ipsec RuntimeError:
2015-05-05 14:06:41.974 4555 TRACE neutron.services.vpn.device_drivers.ipsec Command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qrouter-a9e53c63-23fa-4544-9ad4-cdaa480eb5de', 'ipsec', 'addconn', '--ctlbase', '/var/lib/neutron/ipsec/a9e53c63-23fa-4544-9ad4-cdaa480eb5de/var/run/pluto.ctl', '--defaultroutenexthop', '10.62.72.1', '--config', '/var/lib/neutron/ipsec/a9e53c63-23fa-4544-9ad4-cdaa480eb5de/etc/ipsec.conf', '94a916ff-375f-46e8-8c58-8231ce0eea1c']
2015-05-05 14:06:41.974 4555 TRACE neutron.services.vpn.device_drivers.ipsec Exit code: 255
2015-05-05 14:06:41.974 4555 TRACE neutron.services.vpn.device_drivers.ipsec Stdout: ''
2015-05-05 14:06:41.974 4555 TRACE neutron.services.vpn.device_drivers.ipsec Stderr: 'connect(pluto_ctl) failed: No such file or directory\n'
2015-05-05 14:06:41.974 4555 TRACE neutron.services.vpn.device_drivers.ipsec

[Paul Michali (pcm)]

Can you provide more information on your setup? Operating System and version? Which VPN driver are you using? OpenSwan?

Was this setup with OpenStack or DevStack? Is *Swan installed on the system?

You can try the command manually, and see if the expected files are there as well. For example:

sudo neutron-rootwrap /etc/neutron/rootwrap.conf ip netns exec qrouter-a9e53c63-23fa-4544-9ad4-cdaa480eb5de ipsec addconn --ctlbase /var/lib/neutron/ipsec/a9e53c63-23fa-4544-9ad4-cdaa480eb5de/var/run/pluto.ctl --defaultroutenexthop 10.62.72.1 --config /var/lib/neutron/ipsec/a9e53c63-23fa-4544-9ad4-cdaa480eb5de/etc/ipsec.conf 94a916ff-375f-46e8-8c58-8231ce0eea1c

[venkata anil (anil-venkata)]

looks like ipsec pluto is not running in your machine i.e

https://github.com/openstack/neutron-vpnaas/blob/master/neutron_vpnaas/services/vpn/device_drivers/ipsec.py#L417

        self._execute([self.binary,
                       'pluto',
                       '--ctlbase', self.pid_path,
                       '--ipsecdir', self.etc_dir,
                       '--use-netkey',
                       '--uniqueids',
                       '--nat_traversal',
                       '--secretsfile', self.secrets_file,
                       '--virtual_private', virtual_private
                       ])

might have failed

[baojie (baojie0627)]

Operating System: CentOS 7
VPN Driver: OpenSwan U2.6.43/K3.10.0-229.1.2.el7.x86_64

I have tried:
sudo neutron-rootwrap /etc/neutron/rootwrap.conf ip netns exec qrouter-a9e53c63-23fa-4544-9ad4-cdaa480eb5de ipsec addconn --ctlbase /var/lib/neutron/ipsec/a9e53c63-23fa-4544-9ad4-cdaa480eb5de/var/run/pluto.ctl --defaultroutenexthop 10.62.72.1 --config /var/lib/neutron/ipsec/a9e53c63-23fa-4544-9ad4-cdaa480eb5de/etc/ipsec.conf 94a916ff-375f-46e8-8c58-8231ce0eea1c

And it raised error:
connect(pluto_ctl) failed: No such file or directory

I think there is a bug around https://github.com/openstack/neutron-vpnaas/blob/master/neutron_vpnaas/services/vpn/device_drivers/ipsec.py#L431

self._execute([self.binary,
                           'addconn',
                           '--ctlbase', '%s.ctl' % self.pid_path,
                           '--defaultroutenexthop', nexthop,
                           '--config', self.config_file,
                           ipsec_site_conn['id']
                           ])

The '--ctlbase' config seems the error. When I replaced the above command with "... --ctlbase /var/lib/neutron/ipsec/a9e53c63-23fa-4544-9ad4-cdaa480eb5de/var/run/pluto ..." it worked well.

[Wei Hu (huwei-xtu)]

I agree #2. And don't think it is the wrong path.
I change openswan driver to libreswan can avoid this issue.

See this bug https://bugs.launchpad.net/neutron/+bug/1444017

[venkata anil (anil-venkata)]

As Wei Hu explained, please enable libreswan driver
https://github.com/openstack/neutron-vpnaas/blob/master/etc/vpn_agent.ini#L16
Please see this patch
https://review.openstack.org//#/c/174299

[baojie (baojie0627)]

I installed the Juno release in my lab environment. There is no libreswan driver in it. Finally I create ipsec connection successfully after I replaced
'--ctlbase', '%s.ctl' % self.pid_path
with
'--ctlbase', self.pid_path

I find it keeps the path: '%s.ctl' % self.pid_path in the Kilo release. But I have no environment to test it. However I still have a question about that. If it's the right path, why the '--ctlbase' value in https://github.com/openstack/neutron-vpnaas/blob/master/neutron_vpnaas/services/vpn/device_drivers/ipsec.py#L419 & L443 is self.pid_path. As I know these configs should be the same. Right?

[Wei Hu (huwei-xtu)]

I don't think these configs should be the same. When pluto is running. Pluto.pid file will be created.
In my environment (OpenStack: kilo applied libreswan patch. OS: rhel7) when using libreswan, all worked well.

[root@test100 run]# ps -ef | grep ipsec
root 18198 1 0 17:15 ? 00:00:00 /usr/libexec/ipsec/pluto --ctlbase /var/lib/neutron/ipsec/86243bc0-9bfc-401e-9371-43da6e5a1d6b/var/run/pluto --ipsecdir /var/lib/neutron/ipsec/86243bc0-9bfc-401e-9371-43da6e5a1d6b/etc --use-netkey --uniqueids --nat_traversal --secretsfile /var/lib/neutron/ipsec/86243bc0-9bfc-401e-9371-43da6e5a1d6b/etc/ipsec.secrets --virtual_private %v4:192.168.1.0/24,%v4:172.16.100.0/24
root 18774 23739 0 17:19 pts/1 00:00:00 grep --color=auto ipsec
[root@test100 run]# pwd
/var/lib/neutron/ipsec/86243bc0-9bfc-401e-9371-43da6e5a1d6b/var/run
[root@test100 run]# ls
pluto pluto.ctl pluto.pid

There's some doc about pluto.pid and pluto can be found in http://www.freeswan.org/freeswan_trees/CURRENT-TREE/doc/manpage.d/ipsec_pluto.8.html.

[baojie (baojie0627)]

In the link:
"""
--ctlbase path
basename for control files. path.ctl is the socket through which whack communicates with pluto. path.pid is the lockfile to prevent multiple pluto instances. The default is /var/run/pluto).
"""
I think the path should not include '.ctl'. Or the socket will be xxx.ctl.ctl.