Exploitable services exposed on community test nodes

[Jeremy Stanley]

One of the donor service providers for the upstream OpenStack Infrastructure CI pool has notified us that their security team's periodic vulnerability scans have been identifying systems at random within our environment as running open SNMP servers with a read community of "public". Job correlation from these reports indicates each was running one of the following:

tripleo-ci-centos-7-3nodes-multinode
tripleo-ci-centos-7-containers-multinode
tripleo-ci-centos-7-scenario001-multinode-oooq-container
tripleo-ci-centos-7-scenario002-multinode-oooq-container
tripleo-ci-centos-7-scenario003-multinode-oooq-container
tripleo-ci-centos-7-scenario004-multinode-oooq-container

Please adjust the configuration of your job framework to prevent these services from being exposed to the Internet (through iptables ingress filters, service ACLs, configuring them to not listen on globally-routable interfaces, whatever works). Thanks!

[Emilien Macchi (emilienm)]

I have checked ocata, pike and master: the SNMP config is the same:
http://logs.openstack.org/11/541311/1/gate/tripleo-ci-centos-7-nonha-multinode-oooq/451f201/logs/subnode-2/etc/snmp/snmpd.conf.txt.gz

and we use:
rocommunity public 127.0.0.1

So the read community of public is not open to external networks.

[Jeremy Stanley (fungi)]

com2sec notConfigUser default public

com2sec6 notConfigUser default public

group notConfigGroup v1 notConfigUser

group notConfigGroup v2c notConfigUser

view systemview included .1.3.6.1.2.1.1

view systemview included .1.3.6.1.2.1.25.1.1

access notConfigGroup "" any noauth exact systemview none none

The above is, I believe, allowing any request authenticating via community "public" over the Internet to request OIDs in the defined systemview MIBs. Manual testing of a running job node confirms I'm able to `snmpwalk -v1 -c public 158.69.66.124 .1.3.6.1.2.1.1.3.0` and get back a response.

To be clear, the risk is that an attacker may find one of these nodes in a scan and use it in an amplified reflection DDoS by spoofing UDP snmpgets from a victim address (because this is UDP) of any OID returning a large amount of data.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/544530

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/544567

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/544567
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=cb90c8ce484d8e0328a0f2a8250e1c0fa81dd6cb
Submitter: Zuul
Branch: master

commit cb90c8ce484d8e0328a0f2a8250e1c0fa81dd6cb
Author: Emilien Macchi <email address hidden>
Date: Wed Feb 14 09:32:55 2018 -0800

    Disable SNMP service in all CI jobs

    Some work is being done in I46fce28926cb5a881f7384948480266712ae75e3
    to secure SNMP on a specific network but until then we need to stop
    opening the services so cloud providers won't report any security issue
    for TripleO jobs.

    Change-Id: Icd8a6ddda6152186d6be4a227f6449232fecba5e
    Related-Bug: #1749324

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/544530
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=43155ed1462a8e27c9efdbb345bfc5832c50bd2f
Submitter: Zuul
Branch: master

commit 43155ed1462a8e27c9efdbb345bfc5832c50bd2f
Author: Emilien Macchi <email address hidden>
Date: Wed Feb 14 08:35:10 2018 -0800

    Restrict SNMP to internal network

    Add a parameter, SnmpdIpSubnet, which can be an IP/MASK that will be
    used to secure with IPtables the source network authorized to reach
    SNMP service on the host.
    If SnmpdIpSubnet is left empty (default) the parameter will be set to
    SnmpdNetwork.

    Also change the IPtables id, 127 was used by Horizon, so let's switch
    SNMP to 124. No impact on users.

    Change-Id: I46fce28926cb5a881f7384948480266712ae75e3
    Closes-Bug: #1749324

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/pike
Review: https://review.openstack.org/546040

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/ocata
Review: https://review.openstack.org/546057

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/546058

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/546040
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=69565d743111914d8548aa0dd54a63abed9db605
Submitter: Zuul
Branch: stable/pike

commit 69565d743111914d8548aa0dd54a63abed9db605
Author: Emilien Macchi <email address hidden>
Date: Wed Feb 14 08:35:10 2018 -0800

    Restrict SNMP to internal network

    Add a parameter, SnmpdIpSubnet, which can be an IP/MASK that will be
    used to secure with IPtables the source network authorized to reach
    SNMP service on the host.
    If SnmpdIpSubnet is left empty (default) the parameter will be set to
    SnmpdNetwork.

    Also change the IPtables id, 127 was used by Horizon, so let's switch
    SNMP to 124. No impact on users.

    Change-Id: I46fce28926cb5a881f7384948480266712ae75e3
    Closes-Bug: #1749324
    (cherry picked from commit 43155ed1462a8e27c9efdbb345bfc5832c50bd2f)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/546057
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=a67b208476a023fefacff78ddfb1688de8f9cc20
Submitter: Zuul
Branch: stable/ocata

commit a67b208476a023fefacff78ddfb1688de8f9cc20
Author: Emilien Macchi <email address hidden>
Date: Wed Feb 14 08:35:10 2018 -0800

    Restrict SNMP to internal network

    Add a parameter, SnmpdIpSubnet, which can be an IP/MASK that will be
    used to secure with IPtables the source network authorized to reach
    SNMP service on the host.
    If SnmpdIpSubnet is left empty (default) the parameter will be set to
    SnmpdNetwork.

    Also change the IPtables id, 127 was used by Horizon, so let's switch
    SNMP to 124. No impact on users.

    Change-Id: I46fce28926cb5a881f7384948480266712ae75e3
    Depends-On: Ib203161b9676dcfaaf46eec2bddf767ec49282f7
    Closes-Bug: #1749324
    (cherry picked from commit 43155ed1462a8e27c9efdbb345bfc5832c50bd2f)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/546058
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=78b8c3b3f5016f62b63b09416e3f5a7be546f823
Submitter: Zuul
Branch: stable/newton

commit 78b8c3b3f5016f62b63b09416e3f5a7be546f823
Author: Emilien Macchi <email address hidden>
Date: Wed Feb 14 08:35:10 2018 -0800

    Restrict SNMP to internal network

    Add a parameter, SnmpdIpSubnet, which can be an IP/MASK that will be
    used to secure with IPtables the source network authorized to reach
    SNMP service on the host.
    If SnmpdIpSubnet is left empty (default) the parameter will be set to
    SnmpdNetwork.

    Also change the IPtables id, 127 was used by Horizon, so let's switch
    SNMP to 124. No impact on users.

    Note: in this backport we also change the heat_template_version for SNMP
    service so we support "conditions".

    Change-Id: I46fce28926cb5a881f7384948480266712ae75e3
    Depends-On: Ib203161b9676dcfaaf46eec2bddf767ec49282f7
    Closes-Bug: #1749324
    (cherry picked from commit 43155ed1462a8e27c9efdbb345bfc5832c50bd2f)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 8.0.0.0rc1 release candidate.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 7.0.10 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 6.2.11 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 5.3.10 release.