Bridged firewall, port flapping
We run a small set of web infrastructure with the following configuration:
- Upstream routre
- Cisco Catalyst 2960G switch segmented into two vlans.
- Pair of Debian boxes with a transparent bridging firewall comprising
eth1 and eth2. These implement the physdev kernel module and
spanning tree to provide failover service in about 15 seconds should
the primary go down, or come up after having been down. eth0 is a
management interface connected to vlan2 and only accessible from
within the firewall or specified external addresses.
/etc/network/interfaces is included below.
The firewall hosts connect to both vlans on the Cisco. The addresses
on vlan2 are publically accessible, though access is restricted by
the firewalls.
The firewalls run iptables but NOT ebtables, which I've just learned
about (think iptables, but for MAC addresses).
- A set of internal systems (various Linux distros including Debian) on
vlan2 of the firewall.
The symptoms we've been seeing are:
1. Internal systems, on being restarted, show port flapping on the
2960G, with packet dumps showing IPV6 ICMP type 143 (multicast listener
discovery) from both the host's switch port, AND the active firewall's
port. Switching to a different host device may clear this condition.
Assigning a different MAC address to the problematic NIC does NOT clear
the issue. Disabling all IPV6 functionality on the firewalls and hosts
doesn't appear feasible -- some of our RHEL boxes want IPv6 for other
devices/modules.
Fully restarting the active firewall DOES clear the flapping issue.
Merely restarting networking, however, does not.
We'd prefer a more elegant solution. Initial web research suggests
ebtables might be same, but other suggestions are welcomed. It seems
that the problem is either caused by or would be remedied by NOT
propogating the IPv6 traffic to both vlans on the switch.
Up until a few months ago, our internal and external networks were
served off of separate switches, so the MAC forwarding wouldn't have
prompted flap detection and mitigation.
2. One of our internal boxes will lose network connectivity despite its
interface being configured, every few weeks. Restarting networking on
this box appears to resolve the issue, and we've deployed a watcher
script to check for this condition periodically, restarting networking
as needed. Since implementing this fix several weeks ago, we haven't
seen the problem again, of course. I'm not convinced that this is a
related problem, but I'm not convinced it's not, either.
3. We get a LOT of martian packets reported. Previously they showed up
on both hots and firewalls, now they're on the firewalls only, for the
most part.
That /etc/network/interfaces file (public IPs are obscured):
------------------------------------------------------------------------
# The loopback network interface
auto lo
iface lo inet loopback
# The primary network interface
auto eth0
iface eth0 inet static
address ***.***.65.68
netmask 255.255.255.192
gateway ***.***.65.65
dns-nameservers 127.0.0.1
auto eth1
iface eth1 inet static
address 0.0.0.0
netmask 0.0.0.0
gateway 0.0.0.0
auto eth2
iface eth2 inet static
address 0.0.0.0
netmask 0.0.0.0
gateway 0.0.0.0
auto br0
iface br0 inet static
address 0.0.0.0
netmask 0.0.0.0
gateway 0.0.0.0
bridge_ports eth1 eth2
bridge_bridgeprio 1
bridge_stp on
bridge_fd 4
bridge_hello 1
bridge_ageing 4
------------------------------------------------------------------------
--
Dr. Ed Morbius
Chief Scientist / Philologist / Robot Wrangler / Powerplant Operator
Krell Power Systems Unlimited
Reply to: