Button in subiquity 'network connections" screen on s390x installation sometimes says "Continue without network" even with proper network in place

[Frank Heimes]

Today I used the daily-live image (time stamp April 20th) for s390x installation testing and I noticed that the "Network connections" screen sometimes shows "Continue" and sometimes "Continue without network", but the network is always active and okay.
I wasn't able to figure out under which circumstances I get one or the other named button at that installer screen.

Initially I thought that I get "Continue without network" only after installer updates, but that's not true - I faced it with 20.04.1+git44.670c9c83 as well as with 20.04.2.

However, if I click that button (regardeless of it's name) I have a working network and no issues, hence I assume there is a sporadic problem or race with the status of that button.

[Frank Heimes (fheimes)]

I've noticed that in the past (LP 1854965), but then didn't saw it for a couple of installations, hence the old ticket got closed. (Marked the old one now a dub of this one ...)

[Frank Heimes (fheimes)]

I could now see - on another installation - that when I moved to the 'network connections' screen that I first saw "Continue" or "Done" (it was for less than a sec, so not sure which of the two), and it then toggled to "Continue without network".
I was then interrupted came back to the step and found the button named "Done".
Looks like there are things happening asynchronously in the background ...

[Frank Heimes (fheimes)]

While working on a different installation (on a z/VM guest), I reached the "Network connections" screen with the button "Continue without network" at time index 19-14-45, and without using the system (not the UI nor the shell) it switched to "Done" at about time index 19-15-59.

Again, of course not super critical, since whenever I just proceed with the installation, the network is okay - it's just confusing.

And this doesn't happen all the time - I think it mainly happened on a (virtualized) system with limited resources where I hurried through the installer quite quickly...

We may decrease the priority to Low ...

[Michael Hudson-Doyle (mwhudson)]

I have a plan for this, I need to do it soon. Or at least write it down coherently.

[Frank Heimes (fheimes)]

The attached txt file shows the outcome of the following requested test:
"if you "continue without network" and then configure proxy, then come back to networking screen => does it say that you have networking now?"

The logs come attached with the following comment...

[Michael Hudson-Doyle (mwhudson)]

Something very strange is going on here.

Here's the default route going away (this is what triggers the "continue without network" button):

2020-07-22 08:51:09,691 DEBUG probert.network:730 route_change DEL {'family': 10, 'type': 1, 'table': 254, 'dst': b'default', 'ifindex': 7}

Here's the default route coming back 6 minutes later:

2020-07-22 08:57:01,170 DEBUG probert.network:730 route_change NEW {'family': 10, 'type': 1, 'table': 254, 'dst': b'default', 'ifindex': 7}

Why did it take so long? I don't know! There's nothing in syslog or the journal to explain this.

If you can try this again. can you check the output of 'ip route' while the "continue without network" button is being displayed? I wonder if subiquity is somehow the last to know...

[Frank Heimes (fheimes)]

I just completed a bunch of (interactive) test installation (~8) using jammy beta on different s390x systems (z/VM, LPAR and KVM) and with different hw and noticed that this issue hasn't occurred anymore - in none of the cases.

Looks to me that some work in the installer fixed this to be coincidence?!

[Dan Bungert (dbungert)]

Thank you Frank.

I suspect that some of the work to improve task cancellation has fixed this, or maybe we got lucky? I'm going to mark this as fix committed and I would like you to move it back if you see the issue again.

[Frank Heimes (fheimes)]

Okay, sounds good to me.
And if it will never come up again during the jammy dev. cycle, I guess we can close it at Fix Released at the jammy GA date.

[Frank Heimes (fheimes)]

I tried today several installations with the final jammy image (timestamp April 21st), and haven't seen this issue anymore on s390x (like before).
But I've seen a very quick 'sub-second' status update of that buttong while doing a ppc63el installation - which is fine, since it ends up with the correct state.
So I believe that this got probably fixed due to improvement in the event handling or so?!
And the good news on top is that this seems to have fixed LP#1874128, too!
So I'm closing both as Fixed Released.