autopkgtest is very flaky on s390x

[Balint Reczey]

https://autopkgtest.ubuntu.com/packages/m/mosquitto/impish/s390x

It fails most of the time wasting people's time in retrying.

[Gianfranco Costamagna (costamagnagianfranco)]

./11-persistent-subscription-no-local.py
FAIL: Received incorrect puback1b.
Received (not decoded, len=4): b':\x18\x00\x0c'
Expected: PUBACK, rl=2, mid=1
1624236576: The 'port' option is now deprecated and will be removed in a future version. Please use 'listener' instead.
1624236576: mosquitto version 2.0.11 starting
1624236576: Config loaded from 11-persistent-subscription-no-local.conf.
1624236576: Opening ipv4 listen socket on port 1888.
1624236576: Opening ipv6 listen socket on port 1888.
1624236576: mosquitto version 2.0.11 running
1624236576: New connection from 127.0.0.1:44198 on port 1888.
1624236576: Client <unknown> closed its connection.
1624236576: New connection from 127.0.0.1:44200 on port 1888.
1624236576: New client connected from 127.0.0.1:44200 as persistent-subscription-test (p5, c0, k60).
1624236576: No will message specified.
1624236576: Sending CONNACK to persistent-subscription-test (1, 0)
1624236576: Sending PUBLISH to persistent-subscription-test (d1, q1, r0, m1, 'subpub/local', ... (7 bytes))
1624236576: mosquitto version 2.0.11 terminating
1624236576: Saving in-memory database to /tmp/11-persistent-subscription-no-local.db.

[Simon Chopin (schopin)]

Note that the same failures can be spotted on amd64 as well, just not as often.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Olivier Gayot (ogayot)]

Another instance on amd64:

FAIL: Received incorrect puback1b.
Received (not decoded, len=4): b':\x18\x00\x0c'
Expected: PUBACK, rl=2, mid=1
1638935812: The 'port' option is now deprecated and will be removed in a future version. Please use 'listener' instead.
1638935812: mosquitto version 2.0.11 starting
1638935812: Config loaded from 11-persistent-subscription-no-local.conf.
1638935812: Opening ipv4 listen socket on port 1888.
1638935812: Opening ipv6 listen socket on port 1888.
1638935812: mosquitto version 2.0.11 running
1638935812: New connection from ::1:57542 on port 1888.
1638935812: Client <unknown> closed its connection.
1638935812: New connection from 127.0.0.1:36022 on port 1888.
1638935812: New client connected from 127.0.0.1:36022 as persistent-subscription-test (p5, c0, k60).
1638935812: No will message specified.
1638935812: Sending CONNACK to persistent-subscription-test (1, 0)
1638935812: Sending PUBLISH to persistent-subscription-test (d1, q1, r0, m1, 'subpub/local', ... (7 bytes))
1638935812: mosquitto version 2.0.11 terminating
1638935812: Saving in-memory database to /tmp/11-persistent-subscription-no-local.db.

I find it curious that sometimes mosquitto reports one connection over ipv4 and the other over ipv6.

1638935812: New connection from ::1:57542 on port 1888.
1638935812: New connection from 127.0.0.1:36022 on port 1888.

The tests have hard-coded use of AF_INET when connecting:

def do_client_connect(connect_packet, connack_packet, hostname="localhost", port=1888, timeout=10, connack_error="connack"):
    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    sock.settimeout(timeout)
    sock.connect((hostname, port))

Could it be caused by an external connection?