[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Bug#963826: cloud.debian.org: Policy based routing needed when using multiple network interfaces, subnets



Package: cloud.debian.org
Severity: normal

Debian's cloud images support multiple network interfaces. However, if a
secondary network interface is on a different subnet than the primary interface
it lacks routing rules required to respond to network traffic. This problem may
be unexpected for users accustomed to cloud platform images such as AWS' Amazon
Linux 2 which automatically adds policy based routing instructions when network
interfaces are attached.

Platform:      AWS
Image:         ami-00f5e9b43922addef
Instance Type: t2.micro

1. Create a VPC with two subnets (A, B)
2. Launch two Debian AMI EC2 instances into subnet A
3. Update the instance Security Group ingress rules, allow all
traffic/ports/protocols within the VPC address range
3. Create an ENI belonging to subnet B, using the modified SG, and attach it to
an instance
5. SSH into the other instance and attempt to speak to the other host using its
subnet B address

I talked briefly with rvandegrift and noahm in #debian-cloud to confirm that
the issue should be reported. In terms of a solution:

A customization can be made to the /etc/network/interfaces.d/* template that
images use, adding and removing policy based routing when devices are brought
up and down. I'm not familiar enough with non-AWS providers to determine
whether a platform agnostic solution is possible. If this problem is present on
other platforms the solution should be similar.

Scratchpad/napkin suggestions, needs ipv6 routing:

    auto $INTERFACE
    allow-hotplug $INTERFACE

    iface $INTERFACE inet dhcp
        post-up ip route add default via $GATEWAY dev $DEVICE table $TABLE
        post-up ip route add $CIDR dev $DEVICE proto kernel scope link src
$ADDR table $TABLE
        post-up ip rule add iif $DEVICE table $TABLE
        post-up ip route add default via $GATEWAY dev $DEVICE metric $TABLE

        pre-down ip route del default via $GATEWAY dev $DEVICE metric $TABLE
        pre-down ip rule del iif $DEVICE table $TABLE
        pre-down ip route del $CIDR dev $DEVICE proto kernel scope link src
$ADDR table $TABLE
        pre-down ip route del default via $GATEWAY dev $DEVICE table $TABLE

Another solution may be to modify and use AWS' ec2-net-utils directly.
https://github.com/aws/amazon-ec2-net-utils It's my opinion that debian-cloud-
images may have enough pieces already to make this more work than modifying the
existing scripts.

Kind Regards,

Bennett Goble



-- System Information:
Debian Release: 10.4
  APT prefers stable-updates
  APT policy: (500, 'stable-updates'), (500, 'stable')
Architecture: amd64 (x86_64)

Kernel: Linux 4.19.0-9-amd64 (SMP w/1 CPU core)
Kernel taint flags: TAINT_OOT_MODULE, TAINT_UNSIGNED_MODULE
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled


Reply to: