Unable to set proxy for training labs cluster

[Dakshina Ilangovan]

Hi,

I'm trying to install training labs as a cluster on Fedora 26, Kernel 4.13.5. The script requires proxy to download ubuntu ISO for VMs. I'm facing download error despite setting VM_PROXY="http://host:port" in labs/config/localrc.

Kindly help.

[Roger Luethi (rl-o)]

Can you explain what you mean by "The script requires proxy to download ubuntu ISO for VMs"? (The scripts should work fine without a proxy, unless your setup requires it. But of course VM_PROXY should work when used.)

What command are you using to build the cluster?

Can you show us the log leading up to error and the error message itself?

[Dakshina Ilangovan (dakshinai)]

Thank you for the follow up Roger.
My machine requires proxy to connect to external sources. Setting VM_PROXY does not help. Is this correct when set in labs/config/localrc.
VM_PROXY="http://host:port"
${PROVIDER:=kvm}

I'm using the command
./st.py -b cluster
INFO Using provider kvm.
[sudo] password for daks:
INFO stacktrain start at Fri Oct 27 15:58:19 2017
INFO Asked to delete VM base.
INFO not found
WARNING There is no file at given path:
        /home/daks/git/training-labs/labs/img/ubuntu-16.04.3-server-amd64.iso
INFO Downloading
        http://releases.ubuntu.com/16.04/ubuntu-16.04.3-server-amd64.iso
        to /home/daks/git/training-labs/labs/img/ubuntu-16.04.3-server-amd64.iso
INFO This may take a while.
WARNING Download failed.
WARNING Unable to get ISO image, trying to update URL.
ERROR Can't find newer ISO image. Aborting.

I also tried setting VM_PROXY without quotes since it is used to assign "export http_proxy" somewhere in the code. But I see the same error. The proxy is verified to be correct as local wget/git clone/dnf commands work fine. The URL is also accessible.

I also tried locally downloading and running the script. I see a new error. Any suggestions for this too?
./st.py -b cluster
INFO Using provider kvm.
INFO stacktrain start at Fri Oct 27 16:05:14 2017
INFO Asked to delete VM base.
INFO not found
INFO ISO image okay.
INFO Install ISO:
        /home/daks/git/training-labs/labs/img/ubuntu-16.04.3-server-amd64.iso
INFO base_fixups.sh -> 00_base_fixups.sh
INFO apt_init.sh -> 01_apt_init.sh
INFO apt_upgrade.sh -> 02_apt_upgrade.sh
INFO pre-download.sh -> 03_pre-download.sh
INFO apt_pre-download.sh -> 04_apt_pre-download.sh
INFO enable_osbash_ssh_keys.sh -> 05_enable_osbash_ssh_keys.sh
INFO zero_empty.sh -> 06_zero_empty.sh
INFO shutdown.sh -> 07_shutdown.sh
WARNING sudo virsh --connect=qemu:///system vol-create-as default base-ssh-pike-ubuntu-16.04-amd64 10000M --format qcow2
WARNING call_args: ['sudo', 'virsh', '--connect=qemu:///system', 'vol-create-as', 'default', 'base-ssh-pike-ubuntu-16.04-amd64', '10000M', '--format', 'qcow2']
WARNING rc: 1
WARNING output:
error: failed to get pool 'default'
error: Storage pool not found: no storage pool with matching name 'default'

ERROR virsh: Aborting.
Traceback (most recent call last):
  File "/home/daks/git/training-labs/labs/stacktrain/kvm/vm_create.py", line 65, in virsh
    output = subprocess.check_output(call_args, stderr=subprocess.STDOUT)
  File "/usr/lib64/python2.7/subprocess.py", line 219, in check_output
    raise CalledProcessError(retcode, cmd, output=output)
CalledProcessError: Command '['sudo', 'virsh', '--connect=qemu:///system', 'vol-create-as', 'default', 'base-ssh-pike-ubuntu-16.04-amd64', '10000M', '--format', 'qcow2']' returned non-zero exit status 1
WARNING -----------------------------------------------

[Roger Luethi (rl-o)]

Thanks. I am looking into it.

[Julen Larrucea (julenl)]

Does:
> echo $http_proxy
return something on your workstation?

As a quick fix, you could try downloading the iso manually.
With something like:

 wget http://releases.ubuntu.com/16.04/ubuntu-16.04.3-server-amd64.iso -O /home/daks/git/training-labs/labs/img/ubuntu-16.04.3-server-amd64.iso

If it does not work, you should configure properly your proxy settings (in your workstation/laptop) first.

[Dakshina Ilangovan (dakshinai)]

Yes julen, echo $http_proxy works. It has been set system wide, hence wget works too. I see a new error after locally downloading iso using wget. Please refer logs from previous my comment.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/516064

[Roger Luethi (rl-o)]

First of all, thank you for reporting this bug.

I posted a fix (see comment #6 above). If you can test the patch, that would be helpful. If not, let us know and we will push the patch into the repo and the archives quickly (I did test the patch myself, but it would be very helpful to know if it works for you as well).

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/516064
Committed: https://git.openstack.org/cgit/openstack/training-labs/commit/?id=b6d204184ed4e6c4485e2f30c6619a87824b5b33
Submitter: Zuul
Branch: master

commit b6d204184ed4e6c4485e2f30c6619a87824b5b33
Author: Roger Luethi <email address hidden>
Date: Sun Oct 29 08:20:08 2017 +0100

    Fix installation via proxy

    This changeset fixes a bug in the Python port of the server-side scripts
    that kept the installation via an HTTP proxy from working.

    backport: pike ocata

    Closes-Bug: #1727959
    Change-Id: If38dec034f768779a8d5d01238129a6bc348428c

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/pike
Review: https://review.openstack.org/516091

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/ocata
Review: https://review.openstack.org/516092

[Dakshina Ilangovan (dakshinai)]

Thank you so much Roger. I was able to successfully test master branch and can download the ISO via proxy.

[Roger Luethi (rl-o)]

Thanks for the confirmation. If you have any other issues, please let us know.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/516092
Committed: https://git.openstack.org/cgit/openstack/training-labs/commit/?id=de91b43548633dfb4c242877085c6d9e38af9828
Submitter: Zuul
Branch: stable/ocata

commit de91b43548633dfb4c242877085c6d9e38af9828
Author: Roger Luethi <email address hidden>
Date: Sun Oct 29 08:20:08 2017 +0100

    Fix installation via proxy

    This changeset fixes a bug in the Python port of the server-side scripts
    that kept the installation via an HTTP proxy from working.

    backport: pike ocata

    Closes-Bug: #1727959
    Change-Id: If38dec034f768779a8d5d01238129a6bc348428c
    (cherry picked from commit b6d204184ed4e6c4485e2f30c6619a87824b5b33)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/516091
Committed: https://git.openstack.org/cgit/openstack/training-labs/commit/?id=512f40e71fe804665c4585973bb50d187d1bbc9c
Submitter: Zuul
Branch: stable/pike

commit 512f40e71fe804665c4585973bb50d187d1bbc9c
Author: Roger Luethi <email address hidden>
Date: Sun Oct 29 08:20:08 2017 +0100

    Fix installation via proxy

    This changeset fixes a bug in the Python port of the server-side scripts
    that kept the installation via an HTTP proxy from working.

    backport: pike ocata

    Closes-Bug: #1727959
    Change-Id: If38dec034f768779a8d5d01238129a6bc348428c
    (cherry picked from commit b6d204184ed4e6c4485e2f30c6619a87824b5b33)