Create IPsec connection error with VPN service use subnet has subnetpool

[Quynh Vuong]

Openstack: yoga
OS: Ubuntu
VPNaaS Driver: Strongswan

Subnet Pool:

+-------------------+--------------------------------------+
| Field | Value |
+-------------------+--------------------------------------+
| address_scope_id | None |
| created_at | 2023-02-17T02:29:36Z |
| default_prefixlen | 24 |
| default_quota | None |
| description | None |
| id | 4ae1490d-4670-40ca-b755-b5f1cd0df603 |
| ip_version | 4 |
| is_default | False |
| max_prefixlen | 32 |
| min_prefixlen | 8 |
| name | 4ae1490d-4670-40ca-b755-b5f1cd0df603 |
| prefixes | 10.0.0.0/8 |
| project_id | a2439087-60c0-4939-b381-209d7342ed37 |
| revision_number | 1 |
| shared | False |
| tags | |
| updated_at | 2023-02-17T02:29:36Z |
+-------------------+--------------------------------------+

Subnet:

+----------------------+--------------------------------------+
| Field | Value |
+----------------------+--------------------------------------+
| allocation_pools | 10.1.0.2-10.1.0.254 |
| cidr | 10.1.0.0/24 |
| created_at | 2023-02-17T02:30:19Z |
| description | None |
| dns_nameservers | |
| dns_publish_fixed_ip | None |
| enable_dhcp | True |
| gateway_ip | 10.1.0.1 |
| host_routes | |
| id | 00fa7e20-daac-4727-a997-860ff079f254 |
| ip_version | 4 |
| ipv6_address_mode | None |
| ipv6_ra_mode | None |
| name | 00fa7e20-daac-4727-a997-860ff079f254 |
| network_id | d0cd4d4a-4752-4c43-bf31-ada40eb27393 |
| project_id | a2439087-60c0-4939-b381-209d7342ed37 |
| revision_number | 1 |
| segment_id | None |
| service_types | |
| subnetpool_id | 4ae1490d-4670-40ca-b755-b5f1cd0df603 |
| tags | |
| updated_at | 2023-02-17T02:30:19Z |
+----------------------+--------------------------------------+

VPNservice:

+----------------+--------------------------------------+
| Field | Value |
+----------------+--------------------------------------+
| Description | |
| Flavor | None |
| ID | 273a2a07-23ff-41f1-a13e-453125399364 |
| Name | 4ccd88c5-f89d-4fb3-bb7c-7f4ceae34b0e |
| Project | a2439087-60c0-4939-b381-209d7342ed37 |
| Router | eca8dd96-74f2-40bf-9211-1527db9e7c1a |
| State | True |
| Status | PENDING_CREATE |
| Subnet | 00fa7e20-daac-4727-a997-860ff079f254 |
| external_v4_ip | 111.111.1.101 |
| external_v6_ip | None |
| project_id | a2439087-60c0-4939-b381-209d7342ed37 |
+----------------+--------------------------------------+

When I created IPSec site connection with VPNservice 273a2a07-23ff-41f1-a13e-453125399364, neutron server has log:

oslo_messaging.rpc.server [req-ebea8b3f-3ed7-413c-8fa7-48df474551b1 - - - - -] Can not send reply for message: Attribute 'SubnetPool' object has no attribute 'shared'
oslo_messaging.rpc.server Traceback (most recent call last):
oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.8/site-packages/oslo_messaging/rpc/server.py", line 184, in _process_incoming
oslo_messaging.rpc.server message.reply(res)
oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.8/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 150, in reply
oslo_messaging.rpc.server self._send_reply(conn, reply, failure)
oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.8/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 125, in _send_reply
oslo_messaging.rpc.server conn.direct_send(self.reply_q, rpc_common.serialize_msg(msg))
oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.8/site-packages/oslo_messaging/_drivers/common.py", line 292, in serialize_msg
oslo_messaging.rpc.server _MESSAGE_KEY: jsonutils.dumps(raw_msg)}
oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.8/site-packages/oslo_serialization/jsonutils.py", line 202, in dumps
oslo_messaging.rpc.server return json.dumps(obj, default=default, **kwargs)
oslo_messaging.rpc.server File "/usr/lib/python3.8/json/__init__.py", line 234, in dumps
oslo_messaging.rpc.server return cls(
oslo_messaging.rpc.server File "/usr/lib/python3.8/json/encoder.py", line 199, in encode
oslo_messaging.rpc.server chunks = self.iterencode(o, _one_shot=True)
oslo_messaging.rpc.server File "/usr/lib/python3.8/json/encoder.py", line 257, in iterencode
oslo_messaging.rpc.server return _iterencode(o, 0)
oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.8/site-packages/oslo_serialization/jsonutils.py", line 161, in to_primitive
oslo_messaging.rpc.server return recursive(dict(value.iteritems()), level=level + 1)
oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.8/site-packages/oslo_db/sqlalchemy/models.py", line 99, in iteritems
oslo_messaging.rpc.server return self._as_dict().items()
oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.8/site-packages/oslo_db/sqlalchemy/models.py", line 91, in _as_dict
oslo_messaging.rpc.server local = dict((key, value) for key, value in self)
oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.8/site-packages/oslo_db/sqlalchemy/models.py", line 91, in <genexpr>
oslo_messaging.rpc.server local = dict((key, value) for key, value in self)
oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.8/site-packages/neutron_lib/db/model_base.py", line 92, in next
oslo_messaging.rpc.server return n, getattr(self, n)
oslo_messaging.rpc.server Attribute 'SubnetPool' object has no attribute 'shared'

However, when I created VPNservice with subnet without subnetpool, IPSec site connection created with this VPNservice be OK but I didn't find any note or anything include code about "Shouldn't create VPNservice with subnet has subnetpool".

Thanks to read my report bug, I'm looking for receiving your help!

[Rodolfo Alonso (rodolfo-alonso-hernandez)]

Hello Quynh:

What is the agent that is raising this exception?
Can you describe what services you have enabled in your env (plugins, external projects libe neutron-vpnass, ML2 backend)?
Can you describe the steps to reproduce this issue?
Can you share the agent logs?

Regards.

[Quynh Vuong (quynhvuongg)]

Hi Rodolfo Alonso,

Thanks for the reply.

---

Openstack version: stable/yoga(Deploy by kolla-ansible)

Enable Neutron VPNaaS

Neutron server config:

- neutron.conf:

```sh
[DEFAULT]
debug = False
log_dir = /var/log/kolla/neutron
use_stderr = False
bind_host = 10.9.9.215
bind_port = 9696
api_paste_config = /etc/neutron/api-paste.ini
api_workers = 5
metadata_workers = 5
rpc_workers = 3
rpc_state_report_workers = 3
metadata_proxy_socket = /var/lib/neutron/kolla/metadata_proxy
interface_driver = openvswitch
allow_overlapping_ips = true
service_plugins = router,vpnaas
transport_url = rabbit://openstack:nRlOBOxPApJ7PJ7Az8sMt2Jlxnf0M4GPWdpnj6bE@10.9.9.215:5672,openstack:nRlOBOxPApJ7PJ7Az8sMt2Jlxnf0M4GPWdpnj6bE@10.9.9.216:5672,openstack:nRlOBOxPApJ7PJ7Az8sMt2Jlxnf0M4GPWdpnj6bE@10.9.9.218:5672//
ipam_driver = internal
rpc_response_timeout = 600
```

- neutron_vpnaas.conf:

```sh
[service_providers]
service_provider = VPN:strongswan:neutron_vpnaas.services.vpn.service_drivers.ipsec.IPsecVPNDriver:default
```

Neutron L3 Agent config:

- neutron-l3-agent/l3_agent.ini:

```sh
[DEFAULT]
agent_mode = legacy

[agent]
extensions = vpnaas

[ipsec]
enable_detailed_logging = False

[vpnagent]
vpn_device_driver = neutron_vpnaas.services.vpn.device_drivers.strongswan_ipsec.StrongSwanDriver

[ovs]
ovsdb_connection = tcp:127.0.0.1:6640
ovsdb_timeout = 10
```

Steps:

1. Create subnetpool:

+-------------------+--------------------------------------+
| Field | Value |
+-------------------+--------------------------------------+
| address_scope_id | None |
| created_at | 2023-02-17T11:51:19Z |
| default_prefixlen | 24 |
| default_quota | None |
| description | test |
| id | b059c009-7622-4d2e-b89c-b9e08c0f2298 |
| ip_version | 4 |
| is_default | False |
| max_prefixlen | 32 |
| min_prefixlen | 8 |
| name | b059c009-7622-4d2e-b89c-b9e08c0f2298 |
| prefixes | 10.123.0.0/16 |
| project_id | 973b5b62-c161-4d52-a79e-c68461debfa8 |
| revision_number | 1 |
| shared | False |
| tags | |
| tenant_id | 973b5b62-c161-4d52-a79e-c68461debfa8 |
| updated_at | 2023-02-17T11:51:19Z |
+-------------------+--------------------------------------+

2. Create subnet with subnetpool:

+----------------------+--------------------------------------+
| Field | Value |
+----------------------+--------------------------------------+
| allocation_pools | 10.123.1.2-10.123.1.254 |
| cidr | 10.123.1.0/24 |
| created_at | 2023-02-17T11:52:04Z |
| description | test ...

[Rodolfo Alonso (rodolfo-alonso-hernandez)]

Hello Quynh:

It is much more useful if you provide examples of the commands that need to be executed. There are also some missing steps, like for example the endpoint creation (subnet and cidr) and the ike and ipsec policy.

I'm trying to reproduce it but I'm stuck in the endpoint cidr creation:
$ openstack subnet pool create --pool-prefix 10.123.0.0/16 pool1
$ openstack network create net1
$ openstack subnet create --network net1 --subnet-pool pool1 --prefix-length 24 snet1
$ openstack router add subnet router1 snet1 # router1 is created by devstack
$ openstack vpn ipsec policy create ipsecpolicy
$ openstack vpn ike policy create ikepolicy
$ openstack vpn endpoint group create ep_subnet --type subnet --value snet1
$ openstack vpn endpoint group create ep_cidr --type cidr --value 192.168.1.0/24

This last command is failing:
stack@dev20:/opt/stack$ openstack vpn endpoint group create ep_cidr --type cdir --value 10.123.0.4/24
The python binding code in neutronclient will be deprecated in favor of OpenstackSDK, please use that!
Invalid input for type. Reason: cdir is not in valid_values.
Neutron server returns request_ids: ['req-828d2f20-ccc3-4262-a8f3-e210d91b50fb']

What value do I need to provide to create this endpoint?

Regards.

[Quynh Vuong (quynhvuongg)]

Hi Rodolfo Alonso,

Sorry for not listing all the steps, I followed https://docs.openstack.org/neutron/yoga/admin/vpnaas-scenario.html#configure-vpnaas-without-endpoint-group-the-legacy-way:

Step 1: Create Subnetpool
$ openstack subnet pool create --pool-prefix 10.157.0.0/16 --share subnetpool1

Step 2: Create network:
$ openstack network create --share network1

Step 3: Create subnet
$ openstack subnet create --network network1 --subnet-pool subnetpool1 --prefix-length 24 subnet1

Step 4: Create router
$ openstack router create router2

Step 5: Add subnet to router
$ openstack router add subnet router2 subnet1

Step 6: Add gateway to router (external network already created by Horizon)
$ openstack router set --external-gateway external_1 router2

Step 7: Create IKE Policy
$ openstack vpn ike policy create ikepolicy1

Step 8: Create IPSec Policy
$ openstack vpn ipsec policy create ipsecpolicy1

Step 9: Create VPN Service
$ openstack vpn service create --subnet subnet1 --router router2 vpnservice1

Step 10: Create IPSec site connection:
$ openstack vpn ipsec site connection create conn \
> --vpnservice vpnservice1 \
> --ikepolicy ikepolicy1 \
> --ipsecpolicy ipsecpolicy1 \
> --peer-address 192.168.20.11 \
> --peer-id 192.168.20.11 \
> --peer-cidr 192.168.1.0/24 \
> --psk secret

All to be created, however, the IPSec site connection always maintains pending create state and there are error logs as above that I described.

Best Regards.

[Rodolfo Alonso (rodolfo-alonso-hernandez)]

Hello:

While debugging this issue, I found other problems related to the creation or retrieval of the ike/ipsec policies and the vpn service. These are the bugs (with their corresponding patches) that I created:
* https://bugs.launchpad.net/neutron/+bug/2008001
* https://bugs.launchpad.net/neutron/+bug/2008767

Once solved these issues, the vpn connection creation is done without any problem. I'll mark this issue as duplicated of 2008001.

Regards.

[Quynh Vuong (quynhvuongg)]

Hi,

When I hotfix follow your solution: https://github.com/openstack/neutron-vpnaas/commit/792f2c65ec34b63ae8037a5328b7622320c4e1c4#diff-e5c13fe7ec0dfaa57d0d0477dbffd26b4b3878dfaefc358c1c81c91f3f55c1c2

But my bug isn't resolved, you can reproduce the error following the steps I described above.1

Best Regards.