Proxy Options for TripleO QuickStart

[Venkatrangan Govindarajan]

Description
===========

Used TriploeO quickstack as per documentation in an environment where proxy was present and it was an authenticated proxy. It failed to install anything.

Steps to reproduce
==================

An environment with authenticated proxy, set proxy detaiuls to yum.conf and export http_proxy, https_proxy and no_proxy.

Try quiockstart.sh, It fails to proceed and work.

Expected result
===============

If proxy set in environment, the exepectaion is that the script musht pick the same and use. But it fails to do so.

Actual result
=============

Failed to isntall the tripleo-quickstack propject, had to manually edit the script to do so, But after that it fails in other steps due to proxy

[John Trowbridge (trown)]

hello, thanks for trying out tripleo-quickstart.

I do not have an authenticated proxy to test against. Could you provide more details on what you edited to make it work? Also, the log file you posted seemed to fail when defining the overcloud nodes. What was the exact quickstart command you used?

[Venkatrangan Govindarajan (venkatrangang)]

Hi,

 I modified the quickstart.sh to add http_proxy in differnt lines executed in virtual_env.

 Also modified the pip command to use --proxy option because they wer not using the environment variables that was set.

I will attach the modified script.

Regards,

Venkat G

________________________________
From: <email address hidden> <email address hidden> on behalf of John Trowbridge <email address hidden>
Sent: Thursday, April 27, 2017 11:53 AM
To: Venkatrangan G - ERS, HCL Tech
Subject: [Bug 1686353] Re: Proxy Options for TripleO QuickStart

hello, thanks for trying out tripleo-quickstart.

I do not have an authenticated proxy to test against. Could you provide
more details on what you edited to make it work? Also, the log file you
posted seemed to fail when defining the overcloud nodes. What was the
exact quickstart command you used?

--
You received this bug notification because you are subscribed to the bug
report.
https://bugs.launchpad.net/bugs/1686353

Title:
  Proxy Options for TripleO QuickStart

Status in tripleo:
  Triaged

Bug description:
  Description
  ===========

  Used TriploeO quickstack as per documentation in an environment where
  proxy was present and it was an authenticated proxy. It failed to
  install anything.

  Steps to reproduce
  ==================

  An environment with authenticated proxy, set proxy detaiuls to
  yum.conf and export http_proxy, https_proxy and no_proxy.

  Try quiockstart.sh, It fails to proceed and work.

  Expected result
  ===============

  If proxy set in environment, the exepectaion is that the script musht
  pick the same and use. But it fails to do so.

  Actual result
  =============

  Failed to isntall the tripleo-quickstack propject, had to manually
  edit the script to do so, But after that it fails in other steps due
  to proxy

To manage notifications about this bug go to:
https://bugs.launchpad.net/tripleo/+bug/1686353/+subscriptions

::DISCLAIMER::
----------------------------------------------------------------------------------------------------------------------------------------------------

The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only.
E-mail transmission is not guaranteed to be secure or error-free as information could be intercepted, corrupted,
lost, destroyed, arrive late or incomplete, or may contain viruses in transmission. The e mail and its contents
(with or without referred errors) shall therefore not attach any liability on the originator or HCL or its affiliates.
Views or opinions, if any, presented in this email are solely those of the author and may not necessarily reflect the
views or opinions of HCL or its affiliates. Any form of reproduction, dissemination, copying, disclosure, modification,
distribution and / or publication of this message without the prior written consent of authorized representative of
HCL is strictly prohibited. If you have received this email in error please delete it and notify the sender immediately.
Before opening any email and/or attachments, please check them for viruses and other defects.

--------------------------------------------------------------------...

[Venkatrangan Govindarajan (venkatrangang)]

Attached my modified script.

Also, You can configure a proxy using squid. But ensure that internet does not workj without the proxy. This will cause similar failures.

________________________________
From: Venkatrangan G - ERS, HCL Tech
Sent: Thursday, April 27, 2017 12:48 PM
To: Bug 1686353
Subject: Re: [Bug 1686353] Re: Proxy Options for TripleO QuickStart

Hi,

 I modified the quickstart.sh to add http_proxy in differnt lines executed in virtual_env.

 Also modified the pip command to use --proxy option because they wer not using the environment variables that was set.

I will attach the modified script.

Regards,

Venkat G

________________________________
From: <email address hidden> <email address hidden> on behalf of John Trowbridge <email address hidden>
Sent: Thursday, April 27, 2017 11:53 AM
To: Venkatrangan G - ERS, HCL Tech
Subject: [Bug 1686353] Re: Proxy Options for TripleO QuickStart

hello, thanks for trying out tripleo-quickstart.

I do not have an authenticated proxy to test against. Could you provide
more details on what you edited to make it work? Also, the log file you
posted seemed to fail when defining the overcloud nodes. What was the
exact quickstart command you used?

--
You received this bug notification because you are subscribed to the bug
report.
https://bugs.launchpad.net/bugs/1686353

Title:
  Proxy Options for TripleO QuickStart

Status in tripleo:
  Triaged

Bug description:
  Description
  ===========

  Used TriploeO quickstack as per documentation in an environment where
  proxy was present and it was an authenticated proxy. It failed to
  install anything.

  Steps to reproduce
  ==================

  An environment with authenticated proxy, set proxy detaiuls to
  yum.conf and export http_proxy, https_proxy and no_proxy.

  Try quiockstart.sh, It fails to proceed and work.

  Expected result
  ===============

  If proxy set in environment, the exepectaion is that the script musht
  pick the same and use. But it fails to do so.

  Actual result
  =============

  Failed to isntall the tripleo-quickstack propject, had to manually
  edit the script to do so, But after that it fails in other steps due
  to proxy

To manage notifications about this bug go to:
https://bugs.launchpad.net/tripleo/+bug/1686353/+subscriptions

::DISCLAIMER::
----------------------------------------------------------------------------------------------------------------------------------------------------

The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only.
E-mail transmission is not guaranteed to be secure or error-free as information could be intercepted, corrupted,
lost, destroyed, arrive late or incomplete, or may contain viruses in transmission. The e mail and its contents
(with or without referred errors) shall therefore not attach any liability on the originator or HCL or its affiliates.
Views or opinions, if any, presented in this email are solely those of the author and may not necessarily reflect the
views or opinions of HCL or its affiliates. Any form of reproduction, dissemination, copying, disclosure, modification,
distribution and / or publicati...

[wes hayutin (weshayutin)]

Ooof... apologies this bug got lost in shuffle.
Looking into this now.

[wes hayutin (weshayutin)]

Any chance using the option in L266 resolves the issue?

https://github.com/openstack/tripleo-quickstart/blob/master/quickstart.sh#L266

[Rafael Folco (rafaelfolco)]

Looking at this... attempt to reproduce the issue with proxy and venv mix.

[Rafael Folco (rafaelfolco)]

Please let me know if I am missing anything to reproduce the issue.

- proxy all web traffic
set http_proxy and https_proxy with auth
- proxy yum
set proxy user/password setting in yum.conf

Important:
- pip, wget, git, etc use https so you have to set env https_proxy with auth for all these tools to work in quickstart
- set the proxy config on deployer/virthost nodes (I used the same host deployer==virthost 127.0.0.2)

quickstart.sh works for me with the following config:

# install squid

[squid]
yum -y install squid
systemctl start squid
systemctl enable squid
systemctl status squid

# set auth proxy
yum -y install httpd-tools
touch /etc/squid/passwd && chown squid /etc/squid/passwd
htpasswd /etc/squid/passwd pxuser
/etc/squid/squid.conf

[/etc/squid/squid.conf]
#add auth settings
    auth_param basic program /usr/lib64/squid/basic_ncsa_auth /etc/squid/passwd
    auth_param basic children 5
    auth_param basic realm Squid Basic Authentication
    auth_param basic credentialsttl 2 hours
    acl auth_users proxy_auth REQUIRED
    http_access allow auth_users

systemctl restart squid

# all web traffic to auth proxy
export http_proxy=http://pxuser:pxpassword@127.0.0.2:3128
export https_proxy=http://pxuser:pxpassword@127.0.0.2:3128

# configure yum and clear cache
[/etc/yum.conf]
proxy=http://127.0.0.2:3128
proxy_username=pxuser
proxy_password=pxpassword

yum clean all

# run quickstart
bash quickstart.sh --tags all --clean 127.0.0.2

[Yossi Ovadia (jabadia)]

Hi, I have environment with proxy and "suffering" from same problem.

I have made several fixes and managed to overcome the issue. the idea I had is as follow -

bash quickstart.sh -R rocky -e overcloud_full_root_pwd=password -e proxy=http://123.245.78.90:8000

Later in the code ( not too many places ) something like -
For all curl commands :

- when: proxy is defined
  block:

  - name: Set Proxy
    shell: >
      echo "proxy = {{ proxy }}" > ~/.curlrc

For Yum -

{% if proxy is defined %}
if ! grep -qi ^proxy /etc/yum.conf; then
    echo "proxy = {{ proxy }}" > ~/.curlrc
    echo -e "\nproxy={{ proxy }}" >> /etc/yum.conf
else
    sed -i "s/proxy=.*/proxy={{ proxy }}/g" /etc/yum.conf
fi
{% endif %}

I have a private build that passed all proxy related. ( not have bug with hardcoded NTP server , but that's a different issue. )

@Rafael Folco - I would love to take ownership of this bug if ok with you.

[wes hayutin (weshayutin)]

@Yossi, go for it. Put up a patch and ping us in #tripleo
Thanks!

[Sorin Sbarnea (ssbarnea)]

I marked this as low even if it affects me personally as I do use a proxy locally. Still, this is low because I don't really see when the team will find time to address it.

Any CRs are welcomed, don't forget to add me to them as I can help with local proxy testing.

[Marios Andreou (marios-b)]

This is an automated action. Bug status has been set to 'Incomplete' and target milestone has been removed due to inactivity. If you disagree please re-set these values and reach out to us on freenode #tripleo