Connecting Two Subnet inside VMs in Linux

This should be applicable to real world (physical devices) not just virtualized environment. The topology is simple:

                 +----------------------------+
                 |          The Host          |
                 |          --------          |
                 |           Router           |
                 +----------------------------+
                       |               |
                    +-----+         +-----+
        192.168.1.1 | br0 |         | br1 | 192.168.1.9
                    +-----+         +-----+
                       |               |
  Mask: 192.168.1.0/29 |               | Mask: 192.168.1.8/29
        +--------------+               +----------+
        |              |                          |
        |              |                          |
 +--------------+      |                   +---------------+
 |     VM 1     |      +----+              |    VM 3       |
 | 192.168.1.2  |           |              | 192.168.1.10  |
 +--------------+           |              +---------------+
                    +--------------+
                    |     VM 2     |
                    | 192.168.1.3  |
                    +--------------+

The Details

The goal is to connecting VM in different subnet so they can talk to each other. Btw, I'm using Virtualbox for this playground. Here's the breakdown from the schema above:

The Host had two interface br0 and br1 (Both are virtual)
The schema using two subnet 192.168.1.0/29 and 192.168.1.8/29
The Host had two bridge interfaces and three TAP interfaces (for each VMs)
tap0 for VM 1, tap1 for VM 2 and tap2 for VM 3
tap0 and tap1 attached to br0 and tap2 attached to br1

The Host: Creating the Interfaces

Let's create the tap interfaces for each VMs.

tunctl -t tap0 -u myuser
tunctl -t tap1 -u myuser
tunctl -t tap2 -u myuser
ip link set up dev tap0
ip link set up dev tap1
ip link set up dev tap2

Let's create the bridge interface for the host.

brctl addbr br0
brctl addbr br1
brctl addif br0 tap0
brctl addif br0 tap1
brctl addif br1 tap2
ip link set up br0
ip link set up br1

Give IP to the bridge which act as gateway for the VMs.

ip addr add 192.168.1.1 dev br0
ip route add 192.168.1.0/29 dev br0
ip addr add 192.168.1.9 dev br1
ip route add 192.168.1.8/29 dev br1

The VM 1

Log in to the VM 1 and set the IP. You could also using ifconfig tools instead of ip utilities since most of tiny distro such as TinyCore Linux does not include the ip utilities. In Virtualbox make sure you choose Bridge Adapter and select the tap0.

ifconfig eth0 192.168.1.2 netmask 255.255.255.248

Try to ping the br0 (192.168.1.1) address, it should be able to reply.

The VM 2

In Virtualbox make sure you choose Bridge Adapter and select the tap0.

ifconfig eth0 192.168.1.3 netmask 255.255.255.248

Try to ping the br0 (192.168.1.1) and VM 1 (192.168.1.2) address, it should be able to reply.

The VM 3

In Virtualbox make sure you choose Bridge Adapter and select the tap1.

ifconfig eth0 192.168.1.10 netmask 255.255.255.248

Try to ping the br1 (192.168.1.9) address, it should be able to reply.

Connecting the Subnet

At this point the VMs should be able to reach the gateway and others VM which in the same subnet. To make VM able reach each other even in different subnet, we just need one step to achieve it which is setting the gateway inside the VM.

Setting Gateway for VM 1 and VM 2

Point the gateway to the br0 address (192.168.1.1).

route add default gw 192.168.1.1

Setting Gateway for VM 3

Point the gateway to the br1 address (192.168.1.9).

route add default gw 192.168.1.9

To make sure it works try to ping from VM 1 to VM 3 or vice versa it should be able to reach each other.

References:

http://notes.rioastamal.net/2014/02/how-to-create-bridge-network-in-linux.html

It's a note not a blog