Re: network newbie seeks help combining routesets for VPN tunnel

[Tom Roche <Tom_Roche@pobox.com>]

Apologies for letting this thread[1] drop--I was forced to handle other interrupts for a few weeks. Basically, I need to make a networking configuration work, but am currently (apparently) blocked by inability to set a route. Details:

A brief summary of what I need to do to get back to work on a science project[2] is

1. I must `ssh` from a Debian laptop through a firewall to a compute cluster, where I can do the actual science.
2. The cluster's admins (aka "the agency") require use of an F5VPN to cross the firewall, and maintain the F5VPN server through which I must connect.
3. The F5VPN client is entirely {proprietary, blackbox to me, nonconfigurable by me}.
4. The agency (now) requires "security" features which I'm attempting to meet using a cloud node (on which I have root, and which also runs Debian and an OpenVPN server) as a jumpbox.
5. Both the F5VPN and OpenVPN clients run on my laptop (on which I am sole root).

(Full/gory details regarding networking requirements and history available here[3], toy diagram here[4].) My current problem is, I'm not able to set (i.e., `ip route add`) one of the routes I believe I need to make this VPN configuration work, and thus meet my networking requirements. I get to that point via the following sequence:

1. [initial routeset , start OpenVPN client] -> OpenVPN routeset
2. [OpenVPN routeset , start F5VPN client  ] -> F5VPN-imposed routeset
3. delete F5VPN-imposed routeset
4. set F5VPN-through-OpenVPN routeset

Sequence details:

My laptop's "initial routeset" (i.e., what `ip route show`s after restarting the laptop when connected to the modem) is

0: default via 192.168.1.1 dev eth0 proto static
1: 169.254.0.0/16 dev eth0 scope link metric 1000
2: 192.168.1.0/24 dev eth0 proto kernel scope link src LOCAL_IPN

(I'll parameterize some IP#s for clarity. E.g., in the above, LOCAL_IPN==192.168.1.142 , i.e., the laptop's LAN IP#.) At this point, DNS (and `ping`, etc) works, and if I browse to (e.g.) whatismyip.com I see my modem's IP#==INITIAL_PUBLIC_IPN . After I

1. start the OpenVPN server process on my cloud node
2. start the OpenVPN client process on my laptop

I get the following "OpenVPN routeset":

0: 0.0.0.0/1 via OPENVPN_ENDPT_IPN dev tun0
1: default via 192.168.1.1 dev eth0  proto static
2: OPENVPN_GATEWAY_IPN via OPENVPN_ENDPT_IPN dev tun0
3: OPENVPN_ENDPT_IPN dev tun0  proto kernel  scope link  src OPENVPN_SRCPT_IPN
4: 128.0.0.0/1 via OPENVPN_ENDPT_IPN dev tun0
5: 169.254.0.0/16 dev eth0  scope link  metric 1000
6: OPENVPN_PUBLIC_IPN via 192.168.1.1 dev eth0
7: 192.168.1.0/24 dev eth0  proto kernel  scope link  src LOCAL_IPN

(FWIW, I have never observed any change in the following IP#s, despite repeated use:

OPENVPN_GATEWAY_IPN='10.8.0.1'
OPENVPN_ENDPT_IPN=  '10.8.0.5'
OPENVPN_SRCPT_IPN=  '10.8.0.6'

) As suggested by the OpenVPN routeset, my OpenVPN client (which is quite "vanilla"[5]) adds a new link/interface=tun0, and subsequently other sites see a new public IP#==OPENVPN_PUBLIC_IPN . DNS still works, and networking life is good. But after I

1. login to the agency's remote-access website (RAW) using a special, F5-ed browser[6]
2. use the RAW's web UI to connect to the F5VPN server

I get the following "F5VPN-imposed routeset":

0: 0.0.0.0/1 via F5VPN_ENDPT_IPN dev ppp0  proto none  metric 1 
1: default via 192.168.1.1 dev eth0  proto static 
2: F5VPN_GATEWAY_IPN dev ppp0  proto kernel  scope link  src F5VPN_ENDPT_IPN 
3: 128.0.0.0/1 via F5VPN_ENDPT_IPN dev ppp0  proto none  metric 1 
4: F5VPN_PUBLIC_IPN via OPENVPN_ENDPT_IPN dev tun0  proto none  metric 1 

(F5VPN_GATEWAY_IPN, F5VPN_ENDPT_IPN, and F5VPN_PUBLIC_IPN all seem to change with each F5VPN use/connection.) At this point I also have a new link/interface=ppp0. However, at this point I cannot either `ping` or DNS: e.g.,

    $ ping -c 4 141.101.120.15 # == www.whatismyip.com
    PING 141.101.120.15 (141.101.120.15) 56(84) bytes of data.

    --- 141.101.120.15 ping statistics ---
    4 packets transmitted, 0 received, 100% packet loss, time 3022ms

    $ nslookup www.whatismyip.com
    ;; connection timed out; no servers could be reached

So it seems the F5VPN client cannot set routes that will work with the OpenVPN; to be fair, that is probably because the F5VPN knows nothing about the OpenVPN. Hence, after previous discussion (e.g., this post[7]), I first deleted (with this code[8]) the F5VPN-imposed routeset, then tried to set the following "F5VPN-through-OpenVPN routeset" (using this code[9]):

0: 192.168.1.0/24 dev eth0  proto kernel  scope link  src LOCAL_IPN
1: 0.0.0.0/1 via F5VPN_ENDPT_IPN dev ppp0  metric 1
2: default via 192.168.1.1 dev eth0  proto static
3: F5VPN_GATEWAY_IPN dev ppp0  proto kernel  scope link  src F5VPN_ENDPT_IPN
4: 128.0.0.0/1 via F5VPN_ENDPT_IPN dev ppp0  metric 1
5: OPENVPN_PUBLIC_IPN via 192.168.1.1 dev eth0
6: F5VPN_PUBLIC_IPN via OPENVPN_ENDPT_IPN dev tun0  metric 1

I can `ip route add` all of the above ... *except* the last route#=6! which gets the response

> RTNETLINK answers: Network is unreachable

This appears to be a real failure, in that if I subsequently (i.e., immediately after running the above script[9]) do

$ sudo ip route add ${F5VPN_PUBLIC_IPN} via ${OPENVPN_ENDPT_IPN} dev tun0  metric 1

from the commandline, I get the same failure. And, just to be clear, at this point my networking is just as broken as before: both `ping` and DNS fail until I disconnect from the F5VPN, stop the OpenVPN, and restore my initial routeset and linkset.

So ... how to fix this? What am I doing wrong? Any assistance you can provide is much appreciated! and will be paid-forward via the above code and wiki.

TIA, Tom Roche <Tom_Roche@pobox.com>

[1]: first post @ https://lists.debian.org/debian-user/2015/01/msg00732.html , last post before this one @ https://lists.debian.org/debian-user/2015/01/msg00905.html
[2]: https://bitbucket.org/tlroche/aqmeii-na_n2o/wiki/Home
[3]: https://bitbucket.org/tlroche/linode_jumpbox_config/wiki/Home
[4]: https://bitbucket.org/tlroche/linode_jumpbox_config/wiki/Home#rst-header-new-architecture-diagram
[5]: https://bitbucket.org/tlroche/linode_jumpbox_config/wiki/OpenVPN_install#rst-header-id5
[6]: https://bitbucket.org/tlroche/linode_jumpbox_config/wiki/Home#rst-header-f5nap
[7]: https://lists.debian.org/debian-user/2015/01/msg00905.html
[8]: https://bitbucket.org/tlroche/linode_jumpbox_config/raw/HEAD/scripts/delete_current_routes.sh
[9]: https://bitbucket.org/tlroche/linode_jumpbox_config/raw/HEAD/scripts/set_F5VPN_routes.sh