[vbox] VBox scripts should use NAT-network for a Public vboxnet

[Andrey Danin]

https://github.com/stackforge/fuel-main/blob/cb36a45a6148f742665f4b0b426d69350a5f2243/virtualbox/actions/prepare-environment.sh#L78-L80
Now we create all vboxnets as host-only nets, and this blocks us to check an Internet connectivity from inside a deployed cluster. We should set a vboxnet used for a Public network as a NAT-network. And don't forget to check against Ubuntu, MacOS X and any modern Windows.

[Dmitry Pyzhov (dpyzhov)]

Right now we have instruction for nat setup on host machine: http://docs.mirantis.com/fuel/fuel-4.0/install-guide.html#access-to-public-net

We should update our docs after implementation of nat in vbox scripts

[Miroslav Anashkin (manashkin)]

Starting from version 4.3 VBox has new feature - NAT network.
This network is the same as host-only, but has gateway to physical NIC.
Starting from version 4.3.6 this feature even mo more crashes Linux networking.

[Miroslav Anashkin (manashkin)]

Will try to set NAT network for admin interface first.
Let us see, if it works.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/74650

[Miroslav Anashkin (manashkin)]

Looks like NAT network feature in VirtualBox is still very unstable.
So, will add iptables rule for Linux and pf rules for MacOS.

Not sure what is possible under Windows.
Adding Windows bridge to host-only network leads to immediate phisycal network port blocking on switch side due to network loop.
It happens because Windows adds bridge first, with default settings, and allows to configure it after addition only.

[Christopher Aedo (docaedo)]

A agree with Miroslav on this new approach - NAT was not working reliably for me under OSX 10.9.2. vbox 4.3.10 r93012. pf rules on host sound promising (But why not just route through fuel master node? Master node hasn't had any problems getting to outside networks for me so far - but I'm guessing that's been considered and there's a good reason not to take that approach.)

[Miroslav Anashkin (manashkin)]

Christopher,

Direct NAT for nodes is primary Openstack usage scenario.
We already had redirect through master node, but refused this approach for several reasons.

Kind regards,
Miroslav

[Mike Scherbakov (mihgen)]

I don't believe someone is working on it. Moving to fuel-library assignee.

[Miroslav Anashkin (manashkin)]

May we postpone this until NAT network get matured and move out from Experimental status in VirtualBox?
Currently (4.3.0-4.3.12) it is simply too buggy, that we probably should expect from a new experimental feature.

[Fabrizio Soppelsa (fsoppelsa)]

VirtualBox 4.3.16 has been released but despite NAT network seems grown a lot, it's still marked experimental http://www.virtualbox.org/manual/ch06.html#network_nat_service

Should we try an effort and see if switching to NAT network will work well with our scripts?

[Fabrizio Soppelsa (fsoppelsa)]

NAT network feature still experimental. We'll wait until a new release of Virtual Box will mark it stable.

[Sam Stoelinga (sammiestoel)]

I manually add the following iptables rule: iptables -t nat -A POSTROUTING -s 10.20.0.0/24 -j MASQUERADE

Guess this would be a more general approach instead: iptables -t nat -A POSTROUTING -i vboxnet0 -j MASQUERADE

Then each node deployed in vbox uses Fuel master node as gateway. You will need to ipv4 forwarding enabled also.

[Mike Scherbakov (mihgen)]

As now we have Ubuntu being installed from external repos, we need to improve UX for VirtualBox installations. If you try to use current version of scripts, your installation will fail, if you use default repos with Ubuntu packages. The reason for the failure is that slave nodes will lose network connectivity during deployment.

High level networking process is the following:
- Node loaded with bootstrap. DNS, default gateway point to Fuel Master node, and since Fuel Master has masquerading enabled, bootstrap has full access to Internet via Fuel Master
- Once slave is rebooted for OS provisioning, we still have access through Fuel master.
- When deployment is started, l23network puppet module configures all the interfaces, including routing table. As OpenStack requires default route to public network, it's being configured as so via corresponding interface.
- Once it's done using virtualbox scripts, default gateway for the slave node becomes 172.16.0.1. You can access this IP, but your traffic won't go over it.

In order to fix this, and allow traffic to go, you would need to do the following:
--- in Linux ---
sudo iptables -t nat -A POSTROUTING -s 172.16.1.0/24 \! -d 172.16.1.0/24 -j MASQUERADE
sudo echo 1 > /proc/sys/net/ipv4/ip_forward

--- in MacOS ---
Assuming that en0 is the external interface:
sudo /usr/sbin/natd -interface en0
sudo /sbin/ipfw -f flush
sudo /sbin/ipfw add divert natd all from any to any via en0
sudo /sbin/ipfw add pass all from any to any
sudo sysctl -w net.inet.ip.forwarding=1

In order to achieve better UX these steps have to be automated in VirtualBox scripts. Instructions for MacOS has to be modified in such a way that:
a) external interface identified automatically
b) configuration does not harm already existing configuration (i.e. we should avoid ipfw -f flush).
c) We have to let the user know, what we are doing with sudo, and why it is required.

This became Critical for 6.1

[Mike Scherbakov (mihgen)]

Nathan Trueblood's comment:
Unfortunately, both natd and ipfw are deprecated in the latest Mac OS X (Yosemite)

ipfw: https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man8/ipfw.8.html
natd: https://support.apple.com/en-us/HT202553

So for Mac OS X, we are going to have instructions using Internet Connection Sharing - probably.

See duplicate bug here: #1442988