Activity log for bug #2051912

Date Who What changed Old value New value Message
2024-02-01 06:59:32 Seyeong Kim bug added bug
2024-02-01 07:36:07 Shunde Zhang tags sts
2024-02-01 23:13:14 Shunde Zhang bug added subscriber Shunde Zhang
2024-02-02 01:07:41 Seyeong Kim description After upgrading neutron from Wallaby to Xena(not ovn), the customer's neutron-l3-agent can't be restarted because of below error [1] This is because vrrp_pcount = linux_utils.get_process_count_by_name("keepalived") https://github.com/openstack/neutron/blob/16.4.1/neutron/agent/linux/utils.py#L205 We may put entry to apparmor profile for /proc but I also find below commit https://opendev.org/openstack/neutron/commit/f430cd00725f8303f5313cb7784c9aed4b982e62 which is not included in Xena, it is removing code using get_process_count_by_name() Also, if we backport it. it could be fixed as well. Could you please give any advice for this issue? Thanks a lot. [1] ERROR neutron Traceback (most recent call last): ERROR neutron File "/usr/bin/neutron-l3-agent", line 10, in <module> ERROR neutron sys.exit(main()) ERROR neutron File "/usr/lib/python3/dist-packages/neutron/cmd/eventlet/agents/l3.py", line 24, in main ERROR neutron l3_agent.main() ERROR neutron File "/usr/lib/python3/dist-packages/neutron/agent/l3_agent.py", line 52, in main ERROR neutron server = neutron_service.Service.create( ERROR neutron File "/usr/lib/python3/dist-packages/neutron/service.py", line 416, in create ERROR neutron service_obj = cls(host, binary, topic, manager, ERROR neutron File "/usr/lib/python3/dist-packages/neutron/service.py", line 351, in __init__ ERROR neutron self.manager = manager_class(host=host, *args, **kwargs) ERROR neutron File "/usr/lib/python3/dist-packages/neutron/agent/l3/agent.py", line 1006, in __init__ ERROR neutron super(L3NATAgentWithStateReport, self).__init__(host=host, conf=conf) ERROR neutron File "/usr/lib/python3/dist-packages/neutron/agent/l3/agent.py", line 348, in __init__ ERROR neutron self._check_ha_router_process_status() ERROR neutron File "/usr/lib/python3/dist-packages/neutron/agent/l3/agent.py", line 374, in _check_ha_router_process_status ERROR neutron vrrp_pcount = linux_utils.get_process_count_by_name("keepalived") ERROR neutron File "/usr/lib/python3/dist-packages/neutron/agent/linux/utils.py", line 198, in get_process_count_by_name ERROR neutron return len([p for p in psutil.process_iter(['name']) if ERROR neutron File "/usr/lib/python3/dist-packages/neutron/agent/linux/utils.py", line 198, in <listcomp> ERROR neutron return len([p for p in psutil.process_iter(['name']) if ERROR neutron File "/usr/lib/python3/dist-packages/psutil/__init__.py", line 1432, in process_iter ERROR neutron a = set(pids()) ERROR neutron File "/usr/lib/python3/dist-packages/psutil/__init__.py", line 1372, in pids ERROR neutron ret = sorted(_psplatform.pids()) ERROR neutron File "/usr/lib/python3/dist-packages/psutil/_pslinux.py", line 1597, in pids ERROR neutron return [int(x) for x in os.listdir(b(get_procfs_path())) if x.isdigit()] ERROR neutron PermissionError: [Errno 13] Permission denied: b'/proc' After upgrading neutron from Wallaby to Xena(not ovn), the customer's neutron-l3-agent can't be restarted because of below error [1] This is because vrrp_pcount = linux_utils.get_process_count_by_name("keepalived") https://github.com/openstack/neutron/blob/16.4.1/neutron/agent/linux/utils.py#L205 We may put entry to apparmor profile for /proc but I also find below commit https://opendev.org/openstack/neutron/commit/f430cd00725f8303f5313cb7784c9aed4b982e62 which is not included in Xena, it is removing code using get_process_count_by_name() get_process_count_by_name is changed from wallaby to xena https://github.com/openstack/neutron/blob/16.4.1/neutron/agent/linux/utils.py#L205 https://github.com/openstack/neutron/blob/19.7.0/neutron/agent/linux/utils.py#L198 Also, if we backport it. it could be fixed as well. Could you please give any advice for this issue? Thanks a lot. [1] ERROR neutron Traceback (most recent call last): ERROR neutron File "/usr/bin/neutron-l3-agent", line 10, in <module> ERROR neutron sys.exit(main()) ERROR neutron File "/usr/lib/python3/dist-packages/neutron/cmd/eventlet/agents/l3.py", line 24, in main ERROR neutron l3_agent.main() ERROR neutron File "/usr/lib/python3/dist-packages/neutron/agent/l3_agent.py", line 52, in main ERROR neutron server = neutron_service.Service.create( ERROR neutron File "/usr/lib/python3/dist-packages/neutron/service.py", line 416, in create ERROR neutron service_obj = cls(host, binary, topic, manager, ERROR neutron File "/usr/lib/python3/dist-packages/neutron/service.py", line 351, in __init__ ERROR neutron self.manager = manager_class(host=host, *args, **kwargs) ERROR neutron File "/usr/lib/python3/dist-packages/neutron/agent/l3/agent.py", line 1006, in __init__ ERROR neutron super(L3NATAgentWithStateReport, self).__init__(host=host, conf=conf) ERROR neutron File "/usr/lib/python3/dist-packages/neutron/agent/l3/agent.py", line 348, in __init__ ERROR neutron self._check_ha_router_process_status() ERROR neutron File "/usr/lib/python3/dist-packages/neutron/agent/l3/agent.py", line 374, in _check_ha_router_process_status ERROR neutron vrrp_pcount = linux_utils.get_process_count_by_name("keepalived") ERROR neutron File "/usr/lib/python3/dist-packages/neutron/agent/linux/utils.py", line 198, in get_process_count_by_name ERROR neutron return len([p for p in psutil.process_iter(['name']) if ERROR neutron File "/usr/lib/python3/dist-packages/neutron/agent/linux/utils.py", line 198, in <listcomp> ERROR neutron return len([p for p in psutil.process_iter(['name']) if ERROR neutron File "/usr/lib/python3/dist-packages/psutil/__init__.py", line 1432, in process_iter ERROR neutron a = set(pids()) ERROR neutron File "/usr/lib/python3/dist-packages/psutil/__init__.py", line 1372, in pids ERROR neutron ret = sorted(_psplatform.pids()) ERROR neutron File "/usr/lib/python3/dist-packages/psutil/_pslinux.py", line 1597, in pids ERROR neutron return [int(x) for x in os.listdir(b(get_procfs_path())) if x.isdigit()] ERROR neutron PermissionError: [Errno 13] Permission denied: b'/proc'
2024-02-02 01:29:40 Shunde Zhang description After upgrading neutron from Wallaby to Xena(not ovn), the customer's neutron-l3-agent can't be restarted because of below error [1] This is because vrrp_pcount = linux_utils.get_process_count_by_name("keepalived") https://github.com/openstack/neutron/blob/16.4.1/neutron/agent/linux/utils.py#L205 We may put entry to apparmor profile for /proc but I also find below commit https://opendev.org/openstack/neutron/commit/f430cd00725f8303f5313cb7784c9aed4b982e62 which is not included in Xena, it is removing code using get_process_count_by_name() get_process_count_by_name is changed from wallaby to xena https://github.com/openstack/neutron/blob/16.4.1/neutron/agent/linux/utils.py#L205 https://github.com/openstack/neutron/blob/19.7.0/neutron/agent/linux/utils.py#L198 Also, if we backport it. it could be fixed as well. Could you please give any advice for this issue? Thanks a lot. [1] ERROR neutron Traceback (most recent call last): ERROR neutron File "/usr/bin/neutron-l3-agent", line 10, in <module> ERROR neutron sys.exit(main()) ERROR neutron File "/usr/lib/python3/dist-packages/neutron/cmd/eventlet/agents/l3.py", line 24, in main ERROR neutron l3_agent.main() ERROR neutron File "/usr/lib/python3/dist-packages/neutron/agent/l3_agent.py", line 52, in main ERROR neutron server = neutron_service.Service.create( ERROR neutron File "/usr/lib/python3/dist-packages/neutron/service.py", line 416, in create ERROR neutron service_obj = cls(host, binary, topic, manager, ERROR neutron File "/usr/lib/python3/dist-packages/neutron/service.py", line 351, in __init__ ERROR neutron self.manager = manager_class(host=host, *args, **kwargs) ERROR neutron File "/usr/lib/python3/dist-packages/neutron/agent/l3/agent.py", line 1006, in __init__ ERROR neutron super(L3NATAgentWithStateReport, self).__init__(host=host, conf=conf) ERROR neutron File "/usr/lib/python3/dist-packages/neutron/agent/l3/agent.py", line 348, in __init__ ERROR neutron self._check_ha_router_process_status() ERROR neutron File "/usr/lib/python3/dist-packages/neutron/agent/l3/agent.py", line 374, in _check_ha_router_process_status ERROR neutron vrrp_pcount = linux_utils.get_process_count_by_name("keepalived") ERROR neutron File "/usr/lib/python3/dist-packages/neutron/agent/linux/utils.py", line 198, in get_process_count_by_name ERROR neutron return len([p for p in psutil.process_iter(['name']) if ERROR neutron File "/usr/lib/python3/dist-packages/neutron/agent/linux/utils.py", line 198, in <listcomp> ERROR neutron return len([p for p in psutil.process_iter(['name']) if ERROR neutron File "/usr/lib/python3/dist-packages/psutil/__init__.py", line 1432, in process_iter ERROR neutron a = set(pids()) ERROR neutron File "/usr/lib/python3/dist-packages/psutil/__init__.py", line 1372, in pids ERROR neutron ret = sorted(_psplatform.pids()) ERROR neutron File "/usr/lib/python3/dist-packages/psutil/_pslinux.py", line 1597, in pids ERROR neutron return [int(x) for x in os.listdir(b(get_procfs_path())) if x.isdigit()] ERROR neutron PermissionError: [Errno 13] Permission denied: b'/proc' After upgrading neutron from Wallaby to Xena(not ovn), the customer's neutron-l3-agent can't be restarted because of below error [1] This is because vrrp_pcount = linux_utils.get_process_count_by_name("keepalived") https://github.com/openstack/neutron/blob/16.4.1/neutron/agent/linux/utils.py#L205 We may put entry to apparmor profile for /proc but I also find below commit https://opendev.org/openstack/neutron/commit/f430cd00725f8303f5313cb7784c9aed4b982e62 which is not included in Xena, it is removing code using get_process_count_by_name() get_process_count_by_name is changed from ussuri to xena https://github.com/openstack/neutron/blob/16.4.1/neutron/agent/linux/utils.py#L205 https://github.com/openstack/neutron/blob/19.7.0/neutron/agent/linux/utils.py#L198 Also, if we backport it. it could be fixed as well. Could you please give any advice for this issue? Thanks a lot. [1] ERROR neutron Traceback (most recent call last): ERROR neutron File "/usr/bin/neutron-l3-agent", line 10, in <module> ERROR neutron sys.exit(main()) ERROR neutron File "/usr/lib/python3/dist-packages/neutron/cmd/eventlet/agents/l3.py", line 24, in main ERROR neutron l3_agent.main() ERROR neutron File "/usr/lib/python3/dist-packages/neutron/agent/l3_agent.py", line 52, in main ERROR neutron server = neutron_service.Service.create( ERROR neutron File "/usr/lib/python3/dist-packages/neutron/service.py", line 416, in create ERROR neutron service_obj = cls(host, binary, topic, manager, ERROR neutron File "/usr/lib/python3/dist-packages/neutron/service.py", line 351, in __init__ ERROR neutron self.manager = manager_class(host=host, *args, **kwargs) ERROR neutron File "/usr/lib/python3/dist-packages/neutron/agent/l3/agent.py", line 1006, in __init__ ERROR neutron super(L3NATAgentWithStateReport, self).__init__(host=host, conf=conf) ERROR neutron File "/usr/lib/python3/dist-packages/neutron/agent/l3/agent.py", line 348, in __init__ ERROR neutron self._check_ha_router_process_status() ERROR neutron File "/usr/lib/python3/dist-packages/neutron/agent/l3/agent.py", line 374, in _check_ha_router_process_status ERROR neutron vrrp_pcount = linux_utils.get_process_count_by_name("keepalived") ERROR neutron File "/usr/lib/python3/dist-packages/neutron/agent/linux/utils.py", line 198, in get_process_count_by_name ERROR neutron return len([p for p in psutil.process_iter(['name']) if ERROR neutron File "/usr/lib/python3/dist-packages/neutron/agent/linux/utils.py", line 198, in <listcomp> ERROR neutron return len([p for p in psutil.process_iter(['name']) if ERROR neutron File "/usr/lib/python3/dist-packages/psutil/__init__.py", line 1432, in process_iter ERROR neutron a = set(pids()) ERROR neutron File "/usr/lib/python3/dist-packages/psutil/__init__.py", line 1372, in pids ERROR neutron ret = sorted(_psplatform.pids()) ERROR neutron File "/usr/lib/python3/dist-packages/psutil/_pslinux.py", line 1597, in pids ERROR neutron return [int(x) for x in os.listdir(b(get_procfs_path())) if x.isdigit()] ERROR neutron PermissionError: [Errno 13] Permission denied: b'/proc'
2024-02-02 01:51:35 Seyeong Kim description After upgrading neutron from Wallaby to Xena(not ovn), the customer's neutron-l3-agent can't be restarted because of below error [1] This is because vrrp_pcount = linux_utils.get_process_count_by_name("keepalived") https://github.com/openstack/neutron/blob/16.4.1/neutron/agent/linux/utils.py#L205 We may put entry to apparmor profile for /proc but I also find below commit https://opendev.org/openstack/neutron/commit/f430cd00725f8303f5313cb7784c9aed4b982e62 which is not included in Xena, it is removing code using get_process_count_by_name() get_process_count_by_name is changed from ussuri to xena https://github.com/openstack/neutron/blob/16.4.1/neutron/agent/linux/utils.py#L205 https://github.com/openstack/neutron/blob/19.7.0/neutron/agent/linux/utils.py#L198 Also, if we backport it. it could be fixed as well. Could you please give any advice for this issue? Thanks a lot. [1] ERROR neutron Traceback (most recent call last): ERROR neutron File "/usr/bin/neutron-l3-agent", line 10, in <module> ERROR neutron sys.exit(main()) ERROR neutron File "/usr/lib/python3/dist-packages/neutron/cmd/eventlet/agents/l3.py", line 24, in main ERROR neutron l3_agent.main() ERROR neutron File "/usr/lib/python3/dist-packages/neutron/agent/l3_agent.py", line 52, in main ERROR neutron server = neutron_service.Service.create( ERROR neutron File "/usr/lib/python3/dist-packages/neutron/service.py", line 416, in create ERROR neutron service_obj = cls(host, binary, topic, manager, ERROR neutron File "/usr/lib/python3/dist-packages/neutron/service.py", line 351, in __init__ ERROR neutron self.manager = manager_class(host=host, *args, **kwargs) ERROR neutron File "/usr/lib/python3/dist-packages/neutron/agent/l3/agent.py", line 1006, in __init__ ERROR neutron super(L3NATAgentWithStateReport, self).__init__(host=host, conf=conf) ERROR neutron File "/usr/lib/python3/dist-packages/neutron/agent/l3/agent.py", line 348, in __init__ ERROR neutron self._check_ha_router_process_status() ERROR neutron File "/usr/lib/python3/dist-packages/neutron/agent/l3/agent.py", line 374, in _check_ha_router_process_status ERROR neutron vrrp_pcount = linux_utils.get_process_count_by_name("keepalived") ERROR neutron File "/usr/lib/python3/dist-packages/neutron/agent/linux/utils.py", line 198, in get_process_count_by_name ERROR neutron return len([p for p in psutil.process_iter(['name']) if ERROR neutron File "/usr/lib/python3/dist-packages/neutron/agent/linux/utils.py", line 198, in <listcomp> ERROR neutron return len([p for p in psutil.process_iter(['name']) if ERROR neutron File "/usr/lib/python3/dist-packages/psutil/__init__.py", line 1432, in process_iter ERROR neutron a = set(pids()) ERROR neutron File "/usr/lib/python3/dist-packages/psutil/__init__.py", line 1372, in pids ERROR neutron ret = sorted(_psplatform.pids()) ERROR neutron File "/usr/lib/python3/dist-packages/psutil/_pslinux.py", line 1597, in pids ERROR neutron return [int(x) for x in os.listdir(b(get_procfs_path())) if x.isdigit()] ERROR neutron PermissionError: [Errno 13] Permission denied: b'/proc' This commit introduced apparmor issue. https://opendev.org/openstack/neutron/commit/aa2f05870106d0402f4bd4c3f425c62ed11a2e7a But from Yoga, below commit is included and the issue is gone. https://opendev.org/openstack/neutron/commit/f430cd00725f8303f5313cb7784c9aed4b982e62 So the same issue will be happening in Wallaby and Xena. However, still we have potential issue with aa2f05870106d0402f4bd4c3f425c62ed11a2e7a If some function calls this, there must be the same issue. apparmor may include proper profile for this. ################### After upgrading neutron from Ussuri to Xena(not ovn), the customer's neutron-l3-agent can't be restarted because of below error [1] This is because vrrp_pcount = linux_utils.get_process_count_by_name("keepalived") https://github.com/openstack/neutron/blob/16.4.1/neutron/agent/linux/utils.py#L205 We may put entry to apparmor profile for /proc but I also find below commit https://opendev.org/openstack/neutron/commit/f430cd00725f8303f5313cb7784c9aed4b982e62 which is not included in Xena, it is removing code using get_process_count_by_name() get_process_count_by_name is changed from ussuri to xena https://github.com/openstack/neutron/blob/16.4.1/neutron/agent/linux/utils.py#L205 https://github.com/openstack/neutron/blob/19.7.0/neutron/agent/linux/utils.py#L198 Also, if we backport it. it could be fixed as well. Could you please give any advice for this issue? Thanks a lot. [1] ERROR neutron Traceback (most recent call last): ERROR neutron File "/usr/bin/neutron-l3-agent", line 10, in <module> ERROR neutron sys.exit(main()) ERROR neutron File "/usr/lib/python3/dist-packages/neutron/cmd/eventlet/agents/l3.py", line 24, in main ERROR neutron l3_agent.main() ERROR neutron File "/usr/lib/python3/dist-packages/neutron/agent/l3_agent.py", line 52, in main ERROR neutron server = neutron_service.Service.create( ERROR neutron File "/usr/lib/python3/dist-packages/neutron/service.py", line 416, in create ERROR neutron service_obj = cls(host, binary, topic, manager, ERROR neutron File "/usr/lib/python3/dist-packages/neutron/service.py", line 351, in __init__ ERROR neutron self.manager = manager_class(host=host, *args, **kwargs) ERROR neutron File "/usr/lib/python3/dist-packages/neutron/agent/l3/agent.py", line 1006, in __init__ ERROR neutron super(L3NATAgentWithStateReport, self).__init__(host=host, conf=conf) ERROR neutron File "/usr/lib/python3/dist-packages/neutron/agent/l3/agent.py", line 348, in __init__ ERROR neutron self._check_ha_router_process_status() ERROR neutron File "/usr/lib/python3/dist-packages/neutron/agent/l3/agent.py", line 374, in _check_ha_router_process_status ERROR neutron vrrp_pcount = linux_utils.get_process_count_by_name("keepalived") ERROR neutron File "/usr/lib/python3/dist-packages/neutron/agent/linux/utils.py", line 198, in get_process_count_by_name ERROR neutron return len([p for p in psutil.process_iter(['name']) if ERROR neutron File "/usr/lib/python3/dist-packages/neutron/agent/linux/utils.py", line 198, in <listcomp> ERROR neutron return len([p for p in psutil.process_iter(['name']) if ERROR neutron File "/usr/lib/python3/dist-packages/psutil/__init__.py", line 1432, in process_iter ERROR neutron a = set(pids()) ERROR neutron File "/usr/lib/python3/dist-packages/psutil/__init__.py", line 1372, in pids ERROR neutron ret = sorted(_psplatform.pids()) ERROR neutron File "/usr/lib/python3/dist-packages/psutil/_pslinux.py", line 1597, in pids ERROR neutron return [int(x) for x in os.listdir(b(get_procfs_path())) if x.isdigit()] ERROR neutron PermissionError: [Errno 13] Permission denied: b'/proc'