Re: Bridging eth0/br0 & NetworkManager - can they coexist?
On Monday 02 July 2012 12:30:32 Steve Dowe wrote:
> On 29/06/12 17:34, Neal Murphy wrote:
> > The bridge device (e.g. br0) is a network interface. The NIC is a network
> > interface. The tap device (e.g. tap0) appears as a network interface to
> > the VM. A bridge device doesn't need a real NIC to operate. It's
> > perfectly happy to bridge zero or more taps to itself. The host doesn't
> > need to actively use a brX device (with IP address, et al) for it to
> > bridge VMs together.
>
> I'm trying to get my head around this. I need to read more on this
> subject. :)
Think of a bridge (br0) as a length of wire that you create with the command
'brctl addbr br0'. The br0 IF is a wire from the host Linux to the bridge. Now
you start a KVM session with a tun/tap virtual NIC. That tap device is another
wire, this time between the bridge and the VM.
At this point, your host Linux and your VM can communicate across the bridge,
but nowhere else. Configure them with DHCP or staticly and you'll have them
gabbing TCP/IP.
"But," you lament, "I want to talk to the outside world!" So now add a real
NIC as a member device using 'brctl addif br0 eth0'. This appears as a wire
between the bridge and the NIC. Eight violas! Your host Linux's br0 and each
VM's virtual NIC can now also talk to the outside world. Configure them with
DHCP or staticly and they'll be talking TCP/IP with the planet.
Now, let's change the scenario a bit. Suppose you need a new network switch
but haven't the cash. But you *do* have a system with four dual-port NICs. So
install a basic GNU/Linux distro with brctl, create a bridge (brctl addbr br0)
and add all eight NICs to the bridge (for i in 0 1 2 3 4 5 6 7; do brctl addif
br0 eth$i; done). Plug the computers' net cables in (assuming you have
polarity-sensing NICs and/or crossover cables). You now have a functioning 8-
port switch. Not so good, it uses a lot more electrons that a real switch.
Also not so good, it won't do anything about loops. Better, your GNU/Linux
host is invisible. Best, you can monitor and/or log all traffic crossing your
switch and no one will be the wiser. Now add a spanning tree protocol (STP)
daemon. Your switch will become slightly translucent, but will help block
loops that could be used with bad intent.
To be sure, the wire metaphor isn't wholly true; it'd be more like an old-
timey hub where each node sees all traffic. A proper bridge device should
limit traffic to the two endpoints (and to the host's brX device). And, like
the bullets at the surreal OK Corral, all the 'wire' I mentioned is not real;
the bridge is entirely virtual, consisting of naught but bits.
I wrote my own KVM control script because I wanted to explicitly control
various aspects of the taps and bridges.
Each VM's virtual NIC MAC address encodes the VM instance and the attached tap
instance; I know which VM it is and which bridge the NIC is connected to (in
MAC addr byte 6, 0-3 is always RD/GN/PU/OR, respectively, regardless of how
many or how few virt NICs I assign to a VM):
----------
2: eth0: <BROADCAST,MULTICAST,NOTRAILERS,UP,LOWER_UP> mtu 1500 \
qdisc pfifo_fast state UP qlen 1000
link/ether 52:54:00:00:63:00 brd ff:ff:ff:ff:ff:ff
inet 10.20.30.194/24 brd 10.20.30.255 scope global eth0
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast \
state UP qlen 1000
link/ether 52:54:00:00:63:01 brd ff:ff:ff:ff:ff:ff
inet 10.99.1.1/24 brd 10.99.1.255 scope global eth1
4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast \
state UP qlen 1000
link/ether 52:54:00:00:63:02 brd ff:ff:ff:ff:ff:ff
inet 10.99.2.1/24 brd 10.99.2.255 scope global eth2
5: eth3: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
link/ether 52:54:00:00:63:03 brd ff:ff:ff:ff:ff:ff
----------
On the host, the tap name encodes the bridge name and the VM instance:
----------
13: tapRD0063: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 \
qdisc pfifo_fast state UNKNOWN qlen 500
link/ether 3e:61:17:12:c4:7a brd ff:ff:ff:ff:ff:ff
inet6 fe80::3c61:17ff:fe12:c47a/64 scope link
valid_lft forever preferred_lft forever
14: tapGN0063: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 \
qdisc pfifo_fast state UNKNOWN qlen 500
link/ether e2:80:25:09:24:89 brd ff:ff:ff:ff:ff:ff
inet6 fe80::e080:25ff:fe09:2489/64 scope link
valid_lft forever preferred_lft forever
15: tapOR0063: <BROADCAST,MULTICAST,PROMISC> mtu 1500 qdisc noop \
state DOWN qlen 500
link/ether e6:5c:11:b1:67:f9 brd ff:ff:ff:ff:ff:ff
16: tapPU0063: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 \
qdisc pfifo_fast state UNKNOWN qlen 500
link/ether 2e:b2:04:e0:98:98 brd ff:ff:ff:ff:ff:ff
inet6 fe80::2cb2:4ff:fee0:9898/64 scope link
valid_lft forever preferred_lft forever
----------
Also on the host, I gave the bridges meaningful names:
----------
5: RED: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN
link/ether 00:24:1d:c5:fb:eb brd ff:ff:ff:ff:ff:ff
inet 10.20.30.105/24 brd 10.20.30.255 scope global RED
inet6 fe80::224:1dff:fec5:fbeb/64 scope link
valid_lft forever preferred_lft forever
6: GREEN: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state
UNKNOWN
link/ether 00:24:1d:c5:fb:ed brd ff:ff:ff:ff:ff:ff
inet6 fe80::224:1dff:fec5:fbed/64 scope link
valid_lft forever preferred_lft forever
7: PURPLE: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state
UNKNOWN
link/ether 00:50:ba:4c:cd:2c brd ff:ff:ff:ff:ff:ff
inet6 fe80::250:baff:fe4c:cd2c/64 scope link
valid_lft forever preferred_lft forever
8: ORANGE: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state
UNKNOWN
link/ether e6:5c:11:b1:67:f9 brd ff:ff:ff:ff:ff:ff
inet6 fe80::b81c:3eff:fe76:4b05/64 scope link
valid_lft forever preferred_lft forever
----------
I no longer have any trouble firing up VMs and getting them networked. It just
works.
Granted, I mix decimal and hex; but some things are hard to avoid.
And I really don't know why Squeeze assigns IPv6 LL addresses to NIC/taps that
are members of a bridge, but dutifully erases IPv4 addresses from same.
Bridging isn't exactly rocket science, but the documentation can seem
frightfully similar at times.
Reply to: