Comment 8 for bug 1876087

Revision history for this message
yatin (yatinkarel) wrote :

I think this is happening due to corrupted cached cirros image in nodepool images, to confirm it i logged into a running undercloud and found:-

We had hit similar issues in past, i don't have bug handy currently for that though.

[zuul@undercloud ~]$ ls -lh /opt/cache/files/
total 32K
-rw-r--r--. 1 root root 273 May 7 09:52 cirros-0.3.4-x86_64-disk.img
-rw-r--r--. 1 root root 273 May 7 09:52 cirros-0.3.5-x86_64-disk.img
-rw-r--r--. 1 root root 273 May 7 09:52 cirros-0.4.0-x86_64-disk.img
-rw-r--r--. 1 root root 275 May 7 09:52 cirros-0.4.0-x86_64-uec.tar.gz
-rw-r--r--. 1 root root 274 May 7 09:52 cirros-0.5.1-aarch64-disk.img
-rw-r--r--. 1 root root 273 May 7 09:52 cirros-0.5.1-x86_64-disk.img
-rw-r--r--. 1 root root 275 May 7 09:52 cirros-0.5.1-x86_64-uec.tar.gz
-rw-r--r--. 1 root root 1.6K May 7 09:52 source-repository-images
(reverse-i-search)`ca': ls -lh /opt/^Cche/files/
[zuul@undercloud ~]$ cat /opt/cache/files/cirros-0.4.0-x86_64-disk.img
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>302 Found</title>
</head><body>
<h1>Found</h1>
<p>The document has moved <a href="https://github.com/cirros-dev/cirros/releases/download/0.4.0/cirros-0.4.0-x86_64-disk.img">here</a>.</p>
</body></html>

The reason it's not happening upstream is:- cirros images are cached differently, since cirros url is redirected, and without -L it download redirect text instead of image, https://softwarefactory-project.io/r/gitweb?p=config.git;a=blob;f=nodepool/elements/upstream-image-needs/post-install.d/89-get_cirros;h=5217aa06fa76788446d40b1715ddd4f6ba650b64;hb=HEAD#l27 is used to cache cirros images, will push a patch to fix this, and get nodepool images updated so we quickly get the results.

The reason it's not happening in train+:- os_tempest is used where cirros image is getting downloaded instead of using cached, https://opendev.org/openstack/tripleo-quickstart-extras/src/branch/master/playbooks/tempest.yml#L68