Looking at periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset035-train:
  https://review.rdoproject.org/zuul/build/657be5515b6543488de53e032c03d5aa

The first attempt to introspect fails, on retry it succedes.

Initial attempt:

 DHCP is OK:
Oct 18 20:33:24 dnsmasq-dhcp[7]: DHCPDISCOVER(br-ctlplane) fa:16:3e:52:8f:ef 
Oct 18 20:33:24 dnsmasq-dhcp[7]: DHCPOFFER(br-ctlplane) 192.168.24.100 fa:16:3e:52:8f:ef 
Oct 18 20:33:24 dnsmasq-dhcp[7]: DHCPDISCOVER(br-ctlplane) fa:16:3e:52:8f:ef 
Oct 18 20:33:24 dnsmasq-dhcp[7]: DHCPOFFER(br-ctlplane) 192.168.24.100 fa:16:3e:52:8f:ef 
Oct 18 20:33:24 dnsmasq-dhcp[7]: DHCPREQUEST(br-ctlplane) 192.168.24.100 fa:16:3e:52:8f:ef 
Oct 18 20:33:24 dnsmasq-dhcp[7]: DHCPACK(br-ctlplane) 192.168.24.100 fa:16:3e:52:8f:ef
Oct 18 20:33:28 dnsmasq-dhcp[7]: DHCPDISCOVER(br-ctlplane) fa:16:3e:58:d7:73 
Oct 18 20:33:28 dnsmasq-dhcp[7]: DHCPOFFER(br-ctlplane) 192.168.24.101 fa:16:3e:58:d7:73 
Oct 18 20:33:28 dnsmasq-dhcp[7]: DHCPDISCOVER(br-ctlplane) fa:16:3e:58:d7:73 
Oct 18 20:33:28 dnsmasq-dhcp[7]: DHCPOFFER(br-ctlplane) 192.168.24.101 fa:16:3e:58:d7:73 
Oct 18 20:33:28 dnsmasq-dhcp[7]: DHCPREQUEST(br-ctlplane) 192.168.24.101 fa:16:3e:58:d7:73 
Oct 18 20:33:28 dnsmasq-dhcp[7]: DHCPACK(br-ctlplane) 192.168.24.101 fa:16:3e:58:d7:73 
Oct 18 20:33:46 dnsmasq-dhcp[7]: DHCPDISCOVER(br-ctlplane) fa:16:3e:d2:e2:f7 
Oct 18 20:33:46 dnsmasq-dhcp[7]: DHCPOFFER(br-ctlplane) 192.168.24.102 fa:16:3e:d2:e2:f7 
Oct 18 20:33:49 dnsmasq-dhcp[7]: DHCPDISCOVER(br-ctlplane) fa:16:3e:41:08:50 
Oct 18 20:33:49 dnsmasq-dhcp[7]: DHCPOFFER(br-ctlplane) 192.168.24.103 fa:16:3e:41:08:50 
Oct 18 20:33:49 dnsmasq-dhcp[7]: DHCPDISCOVER(br-ctlplane) fa:16:3e:d2:e2:f7 
Oct 18 20:33:49 dnsmasq-dhcp[7]: DHCPOFFER(br-ctlplane) 192.168.24.102 fa:16:3e:d2:e2:f7 
Oct 18 20:33:49 dnsmasq-dhcp[7]: DHCPDISCOVER(br-ctlplane) fa:16:3e:41:08:50 
Oct 18 20:33:49 dnsmasq-dhcp[7]: DHCPOFFER(br-ctlplane) 192.168.24.103 fa:16:3e:41:08:50 
Oct 18 20:33:49 dnsmasq-dhcp[7]: DHCPREQUEST(br-ctlplane) 192.168.24.102 fa:16:3e:d2:e2:f7 
Oct 18 20:33:49 dnsmasq-dhcp[7]: DHCPACK(br-ctlplane) 192.168.24.102 fa:16:3e:d2:e2:f7 
Oct 18 20:33:49 dnsmasq-dhcp[7]: DHCPDISCOVER(br-ctlplane) fa:16:3e:41:08:50 
Oct 18 20:33:49 dnsmasq-dhcp[7]: DHCPOFFER(br-ctlplane) 192.168.24.103 fa:16:3e:41:08:50 
Oct 18 20:33:49 dnsmasq-dhcp[7]: DHCPREQUEST(br-ctlplane) 192.168.24.102 fa:16:3e:d2:e2:f7 
Oct 18 20:33:49 dnsmasq-dhcp[7]: DHCPACK(br-ctlplane) 192.168.24.102 fa:16:3e:d2:e2:f7 
Oct 18 20:33:49 dnsmasq-dhcp[7]: DHCPREQUEST(br-ctlplane) 192.168.24.102 fa:16:3e:d2:e2:f7 
Oct 18 20:33:49 dnsmasq-dhcp[7]: DHCPACK(br-ctlplane) 192.168.24.102 fa:16:3e:d2:e2:f7 
Oct 18 20:33:49 dnsmasq-dhcp[7]: DHCPREQUEST(br-ctlplane) 192.168.24.102 fa:16:3e:d2:e2:f7 
Oct 18 20:33:49 dnsmasq-dhcp[7]: DHCPACK(br-ctlplane) 192.168.24.102 fa:16:3e:d2:e2:f7 
Oct 18 20:33:49 dnsmasq-dhcp[7]: DHCPREQUEST(br-ctlplane) 192.168.24.103 fa:16:3e:41:08:50 
Oct 18 20:33:49 dnsmasq-dhcp[7]: DHCPACK(br-ctlplane) 192.168.24.103 fa:16:3e:41:08:50 
Oct 18 20:33:51 dnsmasq-dhcp[7]: DHCPDISCOVER(br-ctlplane) fa:16:3e:52:8f:ef 
Oct 18 20:33:51 dnsmasq-dhcp[7]: DHCPOFFER(br-ctlplane) 192.168.24.100 fa:16:3e:52:8f:ef 
Oct 18 20:33:51 dnsmasq-dhcp[7]: DHCPREQUEST(br-ctlplane) 192.168.24.100 fa:16:3e:52:8f:ef 
Oct 18 20:33:51 dnsmasq-dhcp[7]: DHCPACK(br-ctlplane) 192.168.24.100 fa:16:3e:52:8f:ef 
Oct 18 20:33:54 dnsmasq-dhcp[7]: DHCPDISCOVER(br-ctlplane) fa:16:3e:58:d7:73 
Oct 18 20:33:54 dnsmasq-dhcp[7]: DHCPOFFER(br-ctlplane) 192.168.24.101 fa:16:3e:58:d7:73 
Oct 18 20:33:54 dnsmasq-dhcp[7]: DHCPREQUEST(br-ctlplane) 192.168.24.101 fa:16:3e:58:d7:73 
Oct 18 20:33:54 dnsmasq-dhcp[7]: DHCPACK(br-ctlplane) 192.168.24.101 fa:16:3e:58:d7:73 


Since we two DHCPACK's per node, I belive both PXE boot and ironic-python-agent did succesfully DHCP.


 HTTP download ironic-python-agent OK:
192.168.24.100 - - [18/Oct/2021:20:33:38 +0000] "GET /inspector.ipxe HTTP/1.1" 200 385 "-" "iPXE/1.0.0+ (c63ef)"
192.168.24.100 - - [18/Oct/2021:20:33:38 +0000] "GET /agent.kernel HTTP/1.1" 200 10218632 "-" "iPXE/1.0.0+ (c63ef)"
192.168.24.100 - - [18/Oct/2021:20:33:38 +0000] "GET /agent.ramdisk HTTP/1.1" 200 483681924 "-" "iPXE/1.0.0+ (c63ef)"
192.168.24.101 - - [18/Oct/2021:20:33:41 +0000] "GET /inspector.ipxe HTTP/1.1" 200 385 "-" "iPXE/1.0.0+ (c63ef)"
192.168.24.101 - - [18/Oct/2021:20:33:41 +0000] "GET /agent.kernel HTTP/1.1" 200 10218632 "-" "iPXE/1.0.0+ (c63ef)"
192.168.24.101 - - [18/Oct/2021:20:33:42 +0000] "GET /agent.ramdisk HTTP/1.1" 200 483681924 "-" "iPXE/1.0.0+ (c63ef)"
192.168.24.102 - - [18/Oct/2021:20:33:59 +0000] "GET /inspector.ipxe HTTP/1.1" 200 385 "-" "iPXE/1.0.0+ (c63ef)"
192.168.24.102 - - [18/Oct/2021:20:33:59 +0000] "GET /agent.kernel HTTP/1.1" 200 10218632 "-" "iPXE/1.0.0+ (c63ef)"
192.168.24.103 - - [18/Oct/2021:20:33:59 +0000] "GET /inspector.ipxe HTTP/1.1" 200 385 "-" "iPXE/1.0.0+ (c63ef)"
192.168.24.103 - - [18/Oct/2021:20:33:59 +0000] "GET /agent.kernel HTTP/1.1" 200 10218632 "-" "iPXE/1.0.0+ (c63ef)"
192.168.24.102 - - [18/Oct/2021:20:33:59 +0000] "GET /agent.ramdisk HTTP/1.1" 200 483681924 "-" "iPXE/1.0.0+ (c63ef)"
192.168.24.103 - - [18/Oct/2021:20:34:00 +0000] "GET /agent.ramdisk HTTP/1.1" 200 483681924 "-" "iPXE/1.0.0+ (c63ef)"


Unfortunately we have no log's to see what happened in the booted ramdisk. We just know it booted but never called back to ironic-inspector.

Would it be possible to start dumping 'openstack console log show' for the OVB nodes in the cloud tenant once inspection has started and have multiple timestamped logs collected?