juju does not support charms firewalling within the model

[Jeff Hillman]

juju 2.6.10

On customer cloud, juju was bootstrapped using PCE, and subsequent model was deployed.

Upon customer security scans, it was found that the security groups being created by juju was open to ALL ingress.

On canonistack I recreated this issue. Bootstrapped juju 2.6.10 into openstack, enabled HA, and on the controller model the following ingress rules are allowed.

---

 created_at='2019-11-08T17:13:15Z', direction='ingress', ethertype='IPv6', id='271a56b2-5347-4f4f-b566-73e10423e999', port_range_max='65535', port_range_min='1', protocol='tcp', remote_group_id='5ce386dd-641d-4e7a-8b02-384f22aaa46d', updated_at='2019-11-08T17:13:15Z'
 created_at='2019-11-08T17:20:49Z', direction='ingress', ethertype='IPv4', id='2e5d5365-7faa-472a-b112-9dc4a1a51401', port_range_max='65535', port_range_min='1', protocol='udp', remote_group_id='5ce386dd-641d-4e7a-8b02-384f22aaa46d', updated_at='2019-11-08T17:20:49Z'
 created_at='2019-11-08T17:20:50Z', direction='ingress', ethertype='IPv4', id='6c0a9381-0f7d-48fd-84cf-3a7bf6d17808', port_range_max='65535', port_range_min='1', protocol='tcp', remote_group_id='5ce386dd-641d-4e7a-8b02-384f22aaa46d', updated_at='2019-11-08T17:20:50Z
 created_at='2019-11-08T17:13:14Z', direction='ingress', ethertype='IPv6', id='c091dcb6-01b6-4bb5-a949-3a6e48154be5', port_range_max='65535', port_range_min='1', protocol='udp', remote_group_id='5ce386dd-641d-4e7a-8b02-384f22aaa46d', updated_at='2019-11-08T17:13:14Z

---

this is just for the controller model. Still deploying in Canonistack.

The individual controller nodes also have a security group, but it appears to only have egress rules.

The full list of security rules in the controller model is as follows:

---

 created_at='2019-11-08T17:13:14Z', direction='ingress', ethertype='IPv6', id='1aed259b-82fc-4a29-8bc4-e51ef1d87fc6', port_range_max='17070', port_range_min='17070', protocol='tcp', remote_ip_prefix='::/0', updated_at='2019-11-08T17:13:14Z'
 created_at='2019-11-08T17:20:50Z', direction='ingress', ethertype='IPv4', id='2608e1dc-2519-4841-8316-ea63f63866a7', protocol='icmp', remote_group_id='5ce386dd-641d-4e7a-8b02-384f22aaa46d', updated_at='2019-11-08T17:20:50Z'
 created_at='2019-11-08T17:13:15Z', direction='ingress', ethertype='IPv6', id='271a56b2-5347-4f4f-b566-73e10423e999', port_range_max='65535', port_range_min='1', protocol='tcp', remote_group_id='5ce386dd-641d-4e7a-8b02-384f22aaa46d', updated_at='2019-11-08T17:13:15Z'
 created_at='2019-11-08T17:20:49Z', direction='ingress', ethertype='IPv4', id='2e5d5365-7faa-472a-b112-9dc4a1a51401', port_range_max='65535', port_range_min='1', protocol='udp', remote_group_id='5ce386dd-641d-4e7a-8b02-384f22aaa46d', updated_at='2019-11-08T17:20:49Z'
 created_at='2019-11-08T17:13:13Z', direction='egress', ethertype='IPv6', id='30706ca0-4def-4fba-9b24-b0ca19336759', updated_at='2019-11-08T17:13:13Z'
 created_at='2019-11-08T17:20:49Z', direction='ingress', ethertype='IPv4', id='3b65d404-b11f-4583-b741-5707fc7d6059', port_range_max='17070', port_range_min='17070', protocol='tcp', remote_ip_prefix='0.0.0.0/0', updated_at='2019-11-08T17:20:49Z'
 created_at='2019-11-08T17:13:15Z', direction='ingress', ethertype='IPv6', id='4d622d41-d475-4a56-810b-9c3be35bb801', protocol='icmp', remote_group_id='5ce386dd-641d-4e7a-8b02-384f22aaa46d', updated_at='2019-11-08T17:13:15Z'
 created_at='2019-11-08T17:13:13Z', direction='egress', ethertype='IPv4', id='6a990ca8-e71d-4ff0-92d1-97102907356a', updated_at='2019-11-08T17:13:13Z'
 created_at='2019-11-08T17:20:50Z', direction='ingress', ethertype='IPv4', id='6c0a9381-0f7d-48fd-84cf-3a7bf6d17808', port_range_max='65535', port_range_min='1', protocol='tcp', remote_group_id='5ce386dd-641d-4e7a-8b02-384f22aaa46d', updated_at='2019-11-08T17:20:50Z'
 created_at='2019-11-08T17:13:16Z', direction='ingress', ethertype='IPv6', id='aa8e23fc-5aef-4704-9ed2-d3112f6ca7f8', port_range_max='22', port_range_min='22', protocol='tcp', remote_ip_prefix='::/0', updated_at='2019-11-08T17:13:16Z'
 created_at='2019-11-08T17:13:14Z', direction='ingress', ethertype='IPv6', id='c091dcb6-01b6-4bb5-a949-3a6e48154be5', port_range_max='65535', port_range_min='1', protocol='udp', remote_group_id='5ce386dd-641d-4e7a-8b02-384f22aaa46d', updated_at='2019-11-08T17:13:14Z'
 created_at='2019-11-08T17:20:50Z', direction='ingress', ethertype='IPv4', id='eb3368d2-4dcf-4fad-b2c9-76640dd8ee4e', port_range_max='22', port_range_min='22', protocol='tcp', remote_ip_prefix='0.0.0.0/0', updated_at='2019-11-08T17:20:50Z'

---

As can be seen above, ports 17070 and 22 are already allowed. I don't see the need for there to be a wide open ingress rule for the controller model.

[Jeff Hillman (jhillman)]

I can also confirm that this same behavior happens on cluster (non controller) model security groups.

---

 created_at='2019-11-08T17:31:38Z', direction='ingress', ethertype='IPv4', id='1637538c-32c8-45e9-afa0-47b576e6f0c7', port_range_max='65535', port_range_min='1', protocol='udp', remote_group_id='41a101b5-be2a-4aee-9202-ecea262f279b', updated_at='2019-11-08T17:31:38Z'
 created_at='2019-11-08T17:31:39Z', direction='ingress', ethertype='IPv4', id='3fb36587-68e7-440a-87e5-6125083ea9ab', port_range_max='65535', port_range_min='1', protocol='tcp', remote_group_id='41a101b5-be2a-4aee-9202-ecea262f279b', updated_at='2019-11-08T17:31:39Z'
 created_at='2019-11-08T17:24:51Z', direction='ingress', ethertype='IPv6', id='7e0f3f4e-83b5-4aae-9e80-1fb96b41b017', port_range_max='65535', port_range_min='1', protocol='udp', remote_group_id='41a101b5-be2a-4aee-9202-ecea262f279b', updated_at='2019-11-08T17:24:51Z' |
 created_at='2019-11-08T17:24:52Z', direction='ingress', ethertype='IPv6', id='8a0d9843-ff61-4617-bae3-0297b6236112', port_range_max='65535', port_range_min='1', protocol='tcp', remote_group_id='41a101b5-be2a-4aee-9202-ecea262f279b', updated_at='2019-11-08T17:24:52Z'

---

This is opening all ports to all machines inside the same security group. This is still a violation of the customer's security policies.

[Narinder Gupta (narindergupta)]

I am also seeing same behavior in my cloud. Few other customers can complain about it.

[Vinod Kumar (vinodwa)]

This is blocking Verizon ASP Deployment because of a warning from the Verizon's security team. At this moment, we cannot continue deployment without fixing this bug.

[Richard Harding (rharding)]

Investigating these are all because Juju doesn't firewall between units in a model. Charms have no ability to expose internal to the model and so all units have open ports between each other. The firewall is only for external ingress.

We cannot change these groups because it would break working charms and bundles that have this expectation and if this is required it would need to go through a new field request for a feature in Juju to split expose/ports/firewalls into independent internal/external setups.

[Jeff Hillman (jhillman)]

Well, as opposed to forcing these rules, is it an option to have juju just use a pre-defined security group? Does that need to be a field request?

[John A Meinel (jameinel)]

To confirm, isn't this that we allow access to all machines in the same
security group, not external machines?

So it's not all the internet or even all instances in the cloud. Only the
instances within the same model.

John
=:->

On Fri, Nov 8, 2019, 09:35 Jeff Hillman <email address hidden> wrote:

> Public bug reported:
>
> juju 2.6.10
>
> On customer cloud, juju was bootstrapped using PCE, and subsequent model
> was deployed.
>
> Upon customer security scans, it was found that the security groups
> being created by juju was open to ALL ingress.
>
> On canonistack I recreated this issue. Bootstrapped juju 2.6.10 into
> openstack, enabled HA, and on the controller model the following ingress
> rules are allowed.
>
> ---
>
> created_at='2019-11-08T17:13:15Z', direction='ingress', ethertype='IPv6',
> id='271a56b2-5347-4f4f-b566-73e10423e999', port_range_max='65535',
> port_range_min='1', protocol='tcp',
> remote_group_id='5ce386dd-641d-4e7a-8b02-384f22aaa46d',
> updated_at='2019-11-08T17:13:15Z'
> created_at='2019-11-08T17:20:49Z', direction='ingress', ethertype='IPv4',
> id='2e5d5365-7faa-472a-b112-9dc4a1a51401', port_range_max='65535',
> port_range_min='1', protocol='udp',
> remote_group_id='5ce386dd-641d-4e7a-8b02-384f22aaa46d',
> updated_at='2019-11-08T17:20:49Z'
> created_at='2019-11-08T17:20:50Z', direction='ingress', ethertype='IPv4',
> id='6c0a9381-0f7d-48fd-84cf-3a7bf6d17808', port_range_max='65535',
> port_range_min='1', protocol='tcp',
> remote_group_id='5ce386dd-641d-4e7a-8b02-384f22aaa46d',
> updated_at='2019-11-08T17:20:50Z
> created_at='2019-11-08T17:13:14Z', direction='ingress', ethertype='IPv6',
> id='c091dcb6-01b6-4bb5-a949-3a6e48154be5', port_range_max='65535',
> port_range_min='1', protocol='udp',
> remote_group_id='5ce386dd-641d-4e7a-8b02-384f22aaa46d',
> updated_at='2019-11-08T17:13:14Z
>
> ---
>
> this is just for the controller model. Still deploying in Canonistack.
>
> The individual controller nodes also have a security group, but it
> appears to only have egress rules.
>
> The full list of security rules in the controller model is as follows:
>
> ---
>
> created_at='2019-11-08T17:13:14Z', direction='ingress', ethertype='IPv6',
> id='1aed259b-82fc-4a29-8bc4-e51ef1d87fc6', port_range_max='17070',
> port_range_min='17070', protocol='tcp', remote_ip_prefix='::/0',
> updated_at='2019-11-08T17:13:14Z'
> created_at='2019-11-08T17:20:50Z', direction='ingress', ethertype='IPv4',
> id='2608e1dc-2519-4841-8316-ea63f63866a7', protocol='icmp',
> remote_group_id='5ce386dd-641d-4e7a-8b02-384f22aaa46d',
> updated_at='2019-11-08T17:20:50Z'
> created_at='2019-11-08T17:13:15Z', direction='ingress', ethertype='IPv6',
> id='271a56b2-5347-4f4f-b566-73e10423e999', port_range_max='65535',
> port_range_min='1', protocol='tcp',
> remote_group_id='5ce386dd-641d-4e7a-8b02-384f22aaa46d',
> updated_at='2019-11-08T17:13:15Z'
> created_at='2019-11-08T17:20:49Z', direction='ingress', ethertype='IPv4',
> id='2e5d5365-7faa-472a-b112-9dc4a1a51401', port_range_max='65535',
> port_range_min='1', protocol='udp',
> remote_group_id='5ce386dd-641d-4e7a-8b02-384f22aaa46d',
> updated_at='2019-11-08T17:20:49Z'
> created_at='2019-11-08T17:13:13...

[Jeff Hillman (jhillman)]

@jam yes, that is what the issue is. Juju is basically doing an "any/any" for inside the models (controller or otherwise).

This is raising security alerts and causing the customer auditing team to go in and delete the offending rules, which is breaking juju communications.

[John A Meinel (jameinel)]

I have a hazy memory of adding support to have a defined 'default' security
group which could be included for all machines in a model. It would be
applied to each machine (so the rules would need to support everything that
needs to be exposed anywhere). I don't have the details at hand, but
someone should be able to dig up if you can force a security group.

John
=:->

On Tue, Nov 12, 2019, 09:05 Jeff Hillman <email address hidden> wrote:

> @jam yes, that is what the issue is. Juju is basically doing an
> "any/any" for inside the models (controller or otherwise).
>
> This is raising security alerts and causing the customer auditing team
> to go in and delete the offending rules, which is breaking juju
> communications.
>
> --
> You received this bug notification because you are a member of Canonical
> Field Critical, which is subscribed to the bug report.
> Matching subscriptions: juju bugs
> https://bugs.launchpad.net/bugs/1851866
>
> Title:
> juju does not support charms firewalling within the model
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/juju/+bug/1851866/+subscriptions
>

[John A Meinel (jameinel)]

John
=:->

On Tue, Nov 12, 2019, 09:05 Jeff Hillman <email address hidden> wrote:

> @jam yes, that is what the issue is. Juju is basically doing an
> "any/any" for inside the models (controller or otherwise).
>

Within the model is only if the apps are deployed in the controller model.
If you use separate models (possibly with cross model relations?) then you
still can have isolation between different groups of machines.

> This is raising security alerts and causing the customer auditing team
> to go in and delete the offending rules, which is breaking juju
> communications.
>
> --
> You received this bug notification because you are a member of Canonical
> Field Critical, which is subscribed to the bug report.
> Matching subscriptions: juju bugs
> https://bugs.launchpad.net/bugs/1851866
>
> Title:
> juju does not support charms firewalling within the model
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/juju/+bug/1851866/+subscriptions
>

[Canonical Juju QA Bot (juju-qa-bot)]

This bug has not been updated in 2 years, so we're marking it Low importance. If you believe this is incorrect, please update the importance.