Zun

zun-cni-daemon permission issue on ovs hybrid plug

Bug #1889440 reported by hongbin
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Zun
Fix Released
High
hongbin

Bug Description

User reported the following error on virtual-kubelet/openstack-zun [1]:

2020-07-28 07:13:08.837 23254 INFO oslo.privsep.daemon [-] Running privsep helper: ['sudo', 'zun-rootwrap', '/etc/zun/rootwrap.conf', 'privsep-helper', '--config-file', '/etc/zun/zun
.conf', '--privsep_context', 'vif_plug_ovs.privsep.vif_plug', '--privsep_sock_path', '/tmp/tmp_pnf0hwm/privsep.sock']
2020-07-28 07:13:09.623 23254 INFO oslo.privsep.daemon [-] Spawned new privsep daemon via rootwrap
2020-07-28 07:13:09.508 23272 INFO oslo.privsep.daemon [-] privsep daemon starting
2020-07-28 07:13:09.520 23272 INFO oslo.privsep.daemon [-] privsep process running with uid/gid: 0/0
2020-07-28 07:13:09.523 23272 INFO oslo.privsep.daemon [-] privsep process running with capabilities (eff/prm/inh): CAP_NET_ADMIN/CAP_NET_ADMIN/none
2020-07-28 07:13:09.524 23272 INFO oslo.privsep.daemon [-] privsep daemon running as pid 23272
2020-07-28 07:13:10.154 23254 INFO os_vif [-] Successfully plugged vif VIFBridge(active=False,address=fa:16:3e:02:1a:bb,bridge_name='qbr8cd87d70-5c',has_traffic_filtering=True,id=8cd
87d70-5cb7-4397-a5a1-7478a676de91,network=Network(a4322dbc-4ffa-4a38-895d-16a32ee1deb9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8cd8
7d70-5c')
2020-07-28 07:13:10.157 23254 INFO oslo.privsep.daemon [-] Running privsep helper: ['sudo', 'zun-rootwrap', '/etc/zun/rootwrap.conf', 'privsep-helper', '--config-file', '/etc/zun/zun
.conf', '--privsep_context', 'zun.common.privileged.cni', '--privsep_sock_path', '/tmp/tmpxuvn729u/privsep.sock']
2020-07-28 07:13:10.938 23254 INFO oslo.privsep.daemon [-] Spawned new privsep daemon via rootwrap
2020-07-28 07:13:10.820 23382 INFO oslo.privsep.daemon [-] privsep daemon starting
2020-07-28 07:13:10.823 23382 INFO oslo.privsep.daemon [-] privsep process running with uid/gid: 0/0
2020-07-28 07:13:10.826 23382 INFO oslo.privsep.daemon [-] privsep process running with capabilities (eff/prm/inh): CAP_NET_ADMIN|CAP_SYS_ADMIN|CAP_SYS_PTRACE/CAP_NET_ADMIN|CAP_SYS_$DMIN|CAP_SYS_PTRACE/none
2020-07-28 07:13:10.826 23382 INFO oslo.privsep.daemon [-] privsep daemon running as pid 23382
2020-07-28 07:13:12.330 23254 ERROR zun.cni.daemon.service [-] Error when processing addNetwork request. CNI Params: {'CNI_COMMAND': 'ADD', 'CNI_CONTAINERID': '995dfcab5c850280ddb01f
c407fefda4cf2116b37f130e9071df61de5eb6b5a4', 'CNI_NETNS': '/var/run/netns/cni-0e5ba6ca-74a9-0709-3973-db740eb0634e', 'CNI_ARGS': 'IgnoreUnknown=1;K8S_POD_NAMESPACE=default;K8S_POD_NA
ME=5973904c-be15-4c30-9df0-b0d48e8f98f2;K8S_POD_INFRA_CONTAINER_ID=995dfcab5c850280ddb01fc407fefda4cf2116b37f130e9071df61de5eb6b5a4', 'CNI_IFNAME': 'eth0', 'CNI_PATH': '/opt/cni/bin'
}: pyroute2.netlink.exceptions.NetlinkError: (1, 'Operation not permitted')
2020-07-28 07:13:12.330 23254 ERROR zun.cni.daemon.service Traceback (most recent call last):
2020-07-28 07:13:12.330 23254 ERROR zun.cni.daemon.service File "/usr/local/lib/python3.6/dist-packages/zun/cni/daemon/service.py", line 66, in add
2020-07-28 07:13:12.330 23254 ERROR zun.cni.daemon.service vif = self.plugin.add(params)
2020-07-28 07:13:12.330 23254 ERROR zun.cni.daemon.service File "/usr/local/lib/python3.6/dist-packages/zun/cni/plugins/zun_cni_registry.py", line 46, in add
2020-07-28 07:13:12.330 23254 ERROR zun.cni.daemon.service vifs = self._do_work(params, b_base.connect)
2020-07-28 07:13:12.330 23254 ERROR zun.cni.daemon.service File "/usr/local/lib/python3.6/dist-packages/zun/cni/plugins/zun_cni_registry.py", line 146, in _do_work
2020-07-28 07:13:12.330 23254 ERROR zun.cni.daemon.service container_id=params.CNI_CONTAINERID)
2020-07-28 07:13:12.330 23254 ERROR zun.cni.daemon.service File "/usr/local/lib/python3.6/dist-packages/zun/cni/binding/base.py", line 132, in connect
2020-07-28 07:13:12.330 23254 ERROR zun.cni.daemon.service driver.connect(vif, ifname, netns, container_id)
2020-07-28 07:13:12.330 23254 ERROR zun.cni.daemon.service File "/usr/local/lib/python3.6/dist-packages/zun/cni/binding/bridge.py", line 88, in connect
2020-07-28 07:13:12.330 23254 ERROR zun.cni.daemon.service h_br.add_port(host_ifname)
2020-07-28 07:13:12.330 23254 ERROR zun.cni.daemon.service File "/usr/lib/python3/dist-packages/pyroute2/ipdb/transactional.py", line 209, in __exit__
2020-07-28 07:13:12.330 23254 ERROR zun.cni.daemon.service self.commit()
2020-07-28 07:13:12.330 23254 ERROR zun.cni.daemon.service File "/usr/lib/python3/dist-packages/pyroute2/ipdb/interfaces.py", line 1078, in commit
2020-07-28 07:13:12.330 23254 ERROR zun.cni.daemon.service raise error
2020-07-28 07:13:12.330 23254 ERROR zun.cni.daemon.service File "/usr/lib/python3/dist-packages/pyroute2/ipdb/interfaces.py", line 769, in commit
2020-07-28 07:13:12.330 23254 ERROR zun.cni.daemon.service run(nl.link, 'update', index=i, master=self['index'])
2020-07-28 07:13:12.330 23254 ERROR zun.cni.daemon.service File "/usr/lib/python3/dist-packages/pyroute2/ipdb/interfaces.py", line 504, in _run
2020-07-28 07:13:12.330 23254 ERROR zun.cni.daemon.service raise error
2020-07-28 07:13:12.330 23254 ERROR zun.cni.daemon.service File "/usr/lib/python3/dist-packages/pyroute2/ipdb/interfaces.py", line 499, in _run
2020-07-28 07:13:12.330 23254 ERROR zun.cni.daemon.service return cmd(*argv, **kwarg)
2020-07-28 07:13:12.330 23254 ERROR zun.cni.daemon.service File "/usr/lib/python3/dist-packages/pyroute2/iproute/linux.py", line 1332, in link
2020-07-28 07:13:12.330 23254 ERROR zun.cni.daemon.service msg_flags=msg_flags)
2020-07-28 07:13:12.330 23254 ERROR zun.cni.daemon.service File "/usr/lib/python3/dist-packages/pyroute2/netlink/nlsocket.py", line 373, in nlm_request
2020-07-28 07:13:12.330 23254 ERROR zun.cni.daemon.service return tuple(self._genlm_request(*argv, **kwarg))
2020-07-28 07:13:12.330 23254 ERROR zun.cni.daemon.service File "/usr/lib/python3/dist-packages/pyroute2/netlink/nlsocket.py", line 864, in nlm_request
2020-07-28 07:13:12.330 23254 ERROR zun.cni.daemon.service callback=callback):
2020-07-28 07:13:12.330 23254 ERROR zun.cni.daemon.service File "/usr/lib/python3/dist-packages/pyroute2/netlink/nlsocket.py", line 376, in get
2020-07-28 07:13:12.330 23254 ERROR zun.cni.daemon.service return tuple(self._genlm_get(*argv, **kwarg))
2020-07-28 07:13:12.330 23254 ERROR zun.cni.daemon.service File "/usr/lib/python3/dist-packages/pyroute2/netlink/nlsocket.py", line 701, in get
2020-07-28 07:13:12.330 23254 ERROR zun.cni.daemon.service raise msg['header']['error']
2020-07-28 07:13:12.330 23254 ERROR zun.cni.daemon.service pyroute2.netlink.exceptions.NetlinkError: (1, 'Operation not permitted')
2020-07-28 07:13:12.330 23254 ERROR zun.cni.daemon.service
2020-07-28 07:13:12.333 23254 INFO werkzeug [-] 127.0.0.1 - - [28/Jul/2020 07:13:12] "POST /addNetwork HTTP/1.1" 500 -

[1] https://github.com/virtual-kubelet/openstack-zun/issues/14

Changed in zun:
assignee: nobody → hongbin (hongbin034)
status: New → In Progress
hongbin (hongbin034)
Changed in zun:
importance: Undecided → High
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to zun (master)

Reviewed: https://review.opendev.org/743664
Committed: https://git.openstack.org/cgit/openstack/zun/commit/?id=2905c488db46c6f7230dc5c1d04db1c4d6a7de67
Submitter: Zuul
Branch: master

commit 2905c488db46c6f7230dc5c1d04db1c4d6a7de67
Author: Hongbin Lu <email address hidden>
Date: Wed Jul 29 02:34:18 2020 +0000

    Fix ovs hybrid plug in zun cni

    Change-Id: I756f2ce5ed8ad5dd394e9862adcc84fdf4d227de
    Closes-Bug: #1889440

Changed in zun:
status: In Progress → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.