autopkg test timeouts on arm64

Bug #2080346 reported by Matthias Klose
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Auto Package Testing
Fix Released
Undecided
Skia
rust-reqwest (Ubuntu)
Fix Committed
Undecided
Skia

Bug Description

as seen on https://autopkgtest.ubuntu.com/packages/r/rust-reqwest/oracular/arm64
the autopkg tests fail on arm64, always on bos03, the only succeeding one succeeded on bos01.

How should we proceed with that?

Related branches

Revision history for this message
Skia (hyask) wrote :

iirc fron a previous investigation, here are some details on the issue:

The two failing tests are always
* connect_many_timeout
* connect_timeout

If I'm correct, those tests are trying to reach 10.255.255.1, which is supposed to be unavailable and raise a timeout but, maybe due to the proxy (maybe not, I'm not sure), that address is reachable in the infra, making the connection not to timeout, except when something (machine, network, etc...) is slow enough, and that explains why we sometimes see them pass.

One easy way to reproduce the problem locally is to have the 10.255.255.1 IP reachable.
In the `rust-reqwest` source package:
```
sudo ip addr add 10.255.255.1/32 dev enp5s0
sudo ip addr add 10.255.255.2/32 dev enp5s0 # only for connect_many_timeout, that also uses it
cargo test timeout
```

Maybe one possible way forward could be to blackhole those IPs on the testbed with our setup scripts. I'll try to experiment a bit with that solution and see how it goes.

Skia (hyask)
Changed in rust-reqwest (Ubuntu):
assignee: nobody → Skia (hyask)
status: New → Confirmed
Revision history for this message
Simon Chopin (schopin) wrote :

Note that while it's a bit curious that the infra suddenly started routing those IPs, here I think rust-reqwest is wrong in the end. If you want an IP that's not routed, you should use one from an IP range from RFC 5735, as those should be treated by reasonable admins as follow:

> Network operators SHOULD add these address blocks to the list of non-
> routeable address spaces, and if packet filters are deployed, then
> this address block SHOULD be added to packet filters.

https://www.rfc-editor.org/rfc/rfc5737.html

Duct-taping the infra to work around this is fine as a short-term solution but we really ought to work with upstream to fix this more durably.

Revision history for this message
Skia (hyask) wrote :

https://github.com/seanmonstar/reqwest/pull/2418
Here is an upstream pull request to fix this durably. The related MP can be reverted once this gets merged and released.

Paride Legovini (paride)
Changed in auto-package-testing:
importance: Undecided → Critical
importance: Critical → Undecided
Paride Legovini (paride)
Changed in rust-reqwest (Ubuntu):
status: Confirmed → Triaged
status: Triaged → Fix Committed
Changed in auto-package-testing:
status: New → Fix Released
assignee: nobody → Paride Legovini (paride)
assignee: Paride Legovini (paride) → Skia (hyask)
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.