Exploitable services exposed on community test nodes

Bug #1749326 reported by Jeremy Stanley on 2018-02-14
This bug affects 1 person
Affects Status Importance Assigned to Milestone

Bug Description

One of the donor service providers for the upstream OpenStack Infrastructure CI pool has notified us that their security team's periodic vulnerability scans have been identifying systems at random within our environment as running open memcached servers. Job correlation from these reports indicates each was running one of the following:


Please adjust the configuration of your job framework to prevent these services from being exposed to the Internet (through iptables ingress filters, service ACLs, configuring them to not listen on globally-routable interfaces, whatever works). Thanks!

Jeremy Stanley (fungi) on 2018-02-14
tags: added: security
Changed in kolla-ansible:
importance: Undecided → High
status: New → Confirmed
Changed in kolla-ansible:
importance: High → Critical
Jeremy Stanley (fungi) wrote :

Any update on the state of this? It's really pretty urgent. An example of _why_ it's a problem: http://www.openwall.com/lists/oss-security/2018/03/02/1

Jeffrey Zhang (jeffrey4l) wrote :

we are trying to use iptables to prevent other IP to connect the memcached port.

I think in devstack, it has the same issue, how it avoid the issue?

Jeremy Stanley (fungi) wrote :

We preinstall restrictive iptables rulesets on our images when building them via http://git.openstack.org/cgit/openstack-infra/project-config/tree/nodepool/elements/nodepool-base/install.d/20-iptables and devstack configures keystone's memcached_servers setting to localhost:11211 so that it traverses the loopback interface rather than an externally-reachable address.

Change abandoned by Jeffrey Zhang (jeffrey.zhang@99cloud.net) on branch: master
Review: https://review.openstack.org/550325
Reason: check https://review.openstack.org/#/c/549715/1

Paul Bourke (pauldbourke) wrote :

Hi Jeremy, thanks for the link. I'm going to look at adding these rules to kolla tomorrow, will update here asap.

Jeremy Stanley (fungi) wrote :

It's worth noting that all the test nodes on which jobs run boot up with the rules I linked above already applied. If memcached or other services are being exposed on reachable interfaces of the node then it can only be because you're altering or tearing down the existing iptables ruleset.

Reviewed: https://review.openstack.org/550821
Committed: https://git.openstack.org/cgit/openstack/kolla-ansible/commit/?id=404d4d0a50f292b1fd6e916cf80813b260621840
Submitter: Zuul
Branch: master

commit 404d4d0a50f292b1fd6e916cf80813b260621840
Author: Paul Bourke <email address hidden>
Date: Thu Mar 8 12:55:05 2018 +0000

    Use zuul firewall rules in gate

    Till now we've been flusing iptables in the gates to allow cross node
    communication in the multi node ceph jobs. This raised security
    concerns, in particular it exposed memcached to the external net.

    This patch uses the infra provided role 'multi-node-firewall' in order
    to correctly configure iptables. Thanks to Jeremy Stanley and Jeffrey
    for help with this.

    Closes-Bug: #1749326
    Change-Id: Iafaf1cf1d9b0227b0f869969d0bd52fbde3791a0

Changed in kolla-ansible:
status: Confirmed → Fix Released

This issue was fixed in the openstack/kolla-ansible development milestone.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers