zvmguestconfigure on SLES12 never completes

[Michael Friesenegger]

zvmguestconfigure is stopped which causes cloud-init to not start and no login prompt to be shown. The following is the output of the 3270 console of a SLES12SP4 guest provisioned by z/VM Cloud Connector.

zvmguestconfigureÝ821¨: modprobe: FATAL: Module vmcp not found in directory /lib
/modules/4.12.14-94.41-default
Ý OK ¨ Started wicked AutoIPv4 supplicant service.
Ý OK ¨ Started wicked DHCPv4 supplicant service.
Ý OK ¨ Started wicked DHCPv6 supplicant service.
         Starting wicked network management service daemon...
Ý OK ¨ Started wicked network management service daemon.
         Starting wicked network nanny service...
Ý OK ¨ Started System Logging Service.
2019-02-01T14:31:51.239676-05:00 linux-gpr9 kernel: ap.3677f7: The hardware syst
em does not support AP instructions
2019-02-01T14:31:51.239998-05:00 linux-gpr9 kernel: systemd: 13 output lines sup
pressed due to ratelimiting
Ý OK ¨ Started Rotate log files.
Ý OK ¨ Started wicked network nanny service.
zvmguestconfigureÝ821¨: Downloading record 0001: cfgdrive.iso
zvmguestconfigureÝ821¨: Downloading record 0002: iucvauth.sh
zvmguestconfigureÝ821¨: Downloading record 0003: network.doscript
zvmguestconfigureÝ821¨: QETH device 0.0.1000:0.0.1001:0.0.1002 already configure
d
zvmguestconfigureÝ821¨: Remove all network devices
zvmguestconfigureÝ821¨: Warning: this will affect network connectivity!
zvmguestconfigureÝ821¨: Successfully removed device 0.0.1000 (eth0)
zvmguestconfigureÝ821¨: Scanning for network devices...
zvmguestconfigureÝ821¨: znetconf: Error: Failed to configure layer2=1
zvmguestconfigureÝ821¨: Successfully configured device 0.0.1000 (eth1000)
Ý OK ¨ Stopped wicked managed network interfaces.

[Michael Friesenegger (mfriesenegger)]

https://review.gerrithub.io/c/mfcloud/python-zvm-sdk/+/443058

[jichenjc (jichenjc)]

Mike, really appreciate your findings
can you let us know what's the version of your SLES? I believe we did some test before on 12.1 and it works , will the new tools you suggested shipped by SLES 12.x and don't need to be installed ? thanks a lot ~

[jichenjc (jichenjc)]

forget above this, I learned this is SLES12SP4 but still curious on why wicked is ok while systemctl doesn't ...

[Michael Friesenegger (mfriesenegger)]

I am able to duplicate this issue with SLES12SP1. The problem is resolved with the proposed patch.

I am not able to answer your question why using wicked works while network does not. Here is the zvmguestconfigure output from /var/log/messages for sles12 sp1 without and with the patch.

sles12 sp1 without the patch

grep zvmguestconfigure /var/log/messages
2019-02-11T23:28:16.339651-05:00 linux-pyfd zvmguestconfigureÝ1108¨: Downloading
 record 0001: cfgdrive.iso
2019-02-11T23:28:16.401189-05:00 linux-pyfd zvmguestconfigureÝ1108¨: Downloading
 record 0002: iucvauth.sh
2019-02-11T23:28:16.448175-05:00 linux-pyfd zvmguestconfigureÝ1108¨: Downloading
 record 0003: network.doscript
2019-02-11T23:28:20.525992-05:00 linux-pyfd zvmguestconfigureÝ1108¨: Remove all
network devices
2019-02-11T23:28:20.526202-05:00 linux-pyfd zvmguestconfigureÝ1108¨: Warning: th
is will affect network connectivity!
2019-02-11T23:28:20.581358-05:00 linux-pyfd zvmguestconfigureÝ1108¨: Successfull
y removed device 0.0.1000 (eth0)
2019-02-11T23:28:24.637109-05:00 linux-pyfd zvmguestconfigureÝ1108¨: Scanning fo
r network devices...
2019-02-11T23:28:24.679149-05:00 linux-pyfd zvmguestconfigureÝ1108¨: znetconf: E
rror: Failed to configure layer2=1
2019-02-11T23:28:24.716295-05:00 linux-pyfd zvmguestconfigureÝ1108¨: Successfull
y configured device 0.0.1000 (eth1000)

sles12 sp1 with the patch

grep zvmguestconfigure /var/log/messages
2019-02-11T23:46:23.880869-05:00 linux-pyfd zvmguestconfigureÝ1092¨: Downloading
 record 0001: cfgdrive.iso
2019-02-11T23:46:23.957468-05:00 linux-pyfd zvmguestconfigureÝ1092¨: Downloading
 record 0002: iucvauth.sh
2019-02-11T23:46:24.005019-05:00 linux-pyfd zvmguestconfigureÝ1092¨: Downloading
 record 0003: network.doscript
2019-02-11T23:46:28.137240-05:00 linux-pyfd zvmguestconfigureÝ1092¨: Remove all
network devices
2019-02-11T23:46:28.137493-05:00 linux-pyfd zvmguestconfigureÝ1092¨: Warning: th
is will affect network connectivity!
2019-02-11T23:46:28.182154-05:00 linux-pyfd zvmguestconfigureÝ1092¨: Successfull
y removed device 0.0.1000 (eth0)
2019-02-11T23:46:32.243410-05:00 linux-pyfd zvmguestconfigureÝ1092¨: Scanning fo
r network devices...
2019-02-11T23:46:32.280486-05:00 linux-pyfd zvmguestconfigureÝ1092¨: znetconf: E
rror: Failed to configure layer2=1
2019-02-11T23:46:32.332017-05:00 linux-pyfd zvmguestconfigureÝ1092¨: Successfull
y configured device 0.0.1000 (eth1000)
2019-02-11T23:46:43.481918-05:00 linux-pyfd zvmguestconfigureÝ1092¨: lo
     up
2019-02-11T23:46:43.482166-05:00 linux-pyfd zvmguestconfigureÝ1092¨: eth1000
     up
2019-02-11T23:46:43.484633-05:00 linux-pyfd zvmguestconfigureÝ1092¨: zvmguestcon
figure has successfully processed the reader files.

As you can see, the output from sles12sp1 with the patch allows the network devices lo and eth1000 to come up as well as the message "zvmguestconfigure has successfully processed the reader files." which is the last message before setupIso is run.

Is there additional information that I can provide as you review the patch?

[jichenjc (jichenjc)]

Thanks for the detailed information, let me merge the code first and see why we havn't have the issue later , in the mean time need dig more on the command used.. thanks for the patch