OSError: [Errno 98] Address already in use in pkg_mgr_mirror_error.log.

Bug #1956297 reported by chandan kumar
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Fix Released
High
Unassigned

Bug Description

While working on fs02 on cs9, we have a bunch of patches as depends on https://review.rdoproject.org/r/c/testproject/+/36713/18//COMMIT_MSG

```
Depends-On: https://review.rdoproject.org/r/c/rdo-jobs/+/36712

Depends-On: https://review.opendev.org/c/openstack/diskimage-builder/+/821772
Depends-On: https://review.opendev.org/c/openstack/tripleo-ci/+/822069
Depends-On: https://review.opendev.org/c/openstack/diskimage-builder/+/822382
Depends-On: https://review.opendev.org/c/openstack/tripleo-puppet-elements/+/822327
Depends-On: https://review.opendev.org/c/openstack/tripleo-puppet-elements/+/822528
Depends-On: https://review.opendev.org/c/openstack/tripleo-common/+/822603
Depends-On: https://review.opendev.org/c/openstack/puppet-pacemaker/+/822644
```

the puppet-pacemaker changes is built by DLRN and stored in gating repo and the gating repo needs to injected on overcloud images.

The above operation is done by install-built-repo: https://opendev.org/openstack/tripleo-quickstart-extras/src/branch/master/roles/install-built-repo/templates/install-built-repo.sh.j2

When the above got executed, it throws the following error:
https://logserver.rdoproject.org/13/36713/18/check/periodic-tripleo-ci-centos-9-ovb-1ctlr_1comp-featureset002-master/c86c21f/logs/undercloud/home/zuul/pkg_mgr_mirror_error.log.txt.gz

```
Traceback (most recent call last):
  File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/usr/lib64/python3.9/http/server.py", line 1290, in <module>
    test(
  File "/usr/lib64/python3.9/http/server.py", line 1245, in test
    with ServerClass(addr, HandlerClass) as httpd:
  File "/usr/lib64/python3.9/socketserver.py", line 452, in __init__
    self.server_bind()
  File "/usr/lib64/python3.9/http/server.py", line 1288, in server_bind
    return super().server_bind()
  File "/usr/lib64/python3.9/http/server.py", line 138, in server_bind
    socketserver.TCPServer.server_bind(self)
  File "/usr/lib64/python3.9/socketserver.py", line 466, in server_bind
    self.socket.bind(self.server_address)
OSError: [Errno 98] Address already in use
```

By taking a closer look at above script log:
https://logserver.rdoproject.org/13/36713/18/check/periodic-tripleo-ci-centos-9-ovb-1ctlr_1comp-featureset002-master/c86c21f/logs/undercloud/home/zuul/install_built_repo.log.txt.gz

```
2021-12-22 11:57:17 | + mkdir -p /opt
2021-12-22 11:57:17 | + pushd /opt
2021-12-22 11:57:17 | /opt /home/zuul
2021-12-22 11:57:17 | + tar xzf /tmp/gating_repo.tar.gz
2021-12-22 11:57:17 | + cat
2021-12-22 11:57:17 | + ps -ef
2021-12-22 11:57:17 | + grep -i python
2021-12-22 11:57:17 | + grep SimpleHTTPServer
2021-12-22 11:57:17 | + awk '{print $2}'
2021-12-22 11:57:17 | + xargs --no-run-if-empty kill -9
2021-12-22 11:57:17 | + pushd /opt/gating_repo
2021-12-22 11:57:17 | /opt/gating_repo /opt /home/zuul
2021-12-22 11:57:17 | + iptables -I INPUT -p tcp --dport 8766 -j ACCEPT
2021-12-22 11:57:17 | + command -v python3
2021-12-22 11:57:17 | /bin/python3
2021-12-22 11:57:17 | + popd
2021-12-22 11:57:17 | + nohup python3 -m http.server 8766
2021-12-22 11:57:17 | /opt /home/zuul
2021-12-22 11:57:17 | + cat
2021-12-22 11:57:17 | + dnf --enablerepo=gating-repo update -y
2021-12-22 11:57:18 | Gating repository 2.9 MB/s | 3.0 kB 00:00
2021-12-22 11:57:18 | Dependencies resolved.
2021-12-22 11:57:18 | Nothing to do.
2021-12-22 11:57:18 | Complete!
```

It might happen that ps -ef command does not executed properly https://opendev.org/openstack/tripleo-quickstart-extras/src/branch/master/roles/install-built-repo/templates/install-built-repo.sh.j2#L18

Leading to above error.

Tags: alert ci
Changed in tripleo:
status: Triaged → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-quickstart-extras (master)

Reviewed: https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/822783
Committed: https://opendev.org/openstack/tripleo-quickstart-extras/commit/5d3f1c2c9a337aeb219a90d282324e17562f6c66
Submitter: "Zuul (22348)"
Branch: master

commit 5d3f1c2c9a337aeb219a90d282324e17562f6c66
Author: Chandan Kumar (raukadah) <email address hidden>
Date: Thu Dec 23 18:09:33 2021 +0530

    Use Python3 based http.server command

    Current we are looking for python2 based http server and
    it fails to kill the running process which leads to port
    already in use error.

    Since Python3 is used everywhere so let's use that one.

    Closes-Bug: #1956297

    Signed-off-by: Chandan Kumar (raukadah) <email address hidden>
    Change-Id: I878c6adfa110fa28b9c83a7669d28af7ac0cd49f

Changed in tripleo:
status: In Progress → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.