Keepalived LVS-TUN problems with return packet
Hello
This is a post I originally sent to lvs-users, but so far not much
luck getting replies. Maybe the debian community could help out
please?
I have set up an IPVS environment using keepalived. My IPVS machines
are in a DMZ, and my real servers are behind the firewall. I have
apache running on the real servers and I am providing a VIP with
HTTP/HTTPS service pointing to the RIP's.
I have created the tunl0 device with the VIP, and no-arp, on the real servers.
I can ping the VIP from a client, and health check on the IPVS shows
both realservers as healthy.
If I attempt to connect to the service from a client, I get a timeout.
I took a tcpdump in various places as I troubleshooted. My client is
receiving the return packet from the real server (as per the design)
but does not seem to accept it. I noticed in the dump that the
sequence numbers were not what I would expect: I send a SYN to the
VIP, it gets sent to a RIP over the IPIP tunnel, realserver responds
an ACK to the client. In the SYN if the sequence number is 1000 the
real server should ACK 1001... what is happening is that the
realserver is ACKing the tunnel packet, not the encapsulated packet. I
suspect this is where my problem is but I haven't found anything that
resembled this issue on Google.
All machines are Debian Lenny (2.6.26-2) Balancers are 32bit,
realservers are amd64.
The IPVS setup is through keepalived which is at version 1.1.15-1,
ipvsadm is at version 1:1.24-2.1
Can anyone suggest a fix?
I will paste some relevant tcpdump output. Notice my CLIENT SYN packet
is 4244383796, TUNNEL SYN packet is 1869554645. What the client
receives from the RIP is ACKing with 1869554646 and not 4244383797 as
I would have expected. If you look at the packet sent in the tunnel
(CIP to RIP Tunnel) the SYN number is the same as the IPIP packet, NOT
the same one my client IP sent initially.
CIP to VIP
18:01:29.521993 IP CIP.42852 > VIP.https: S 4244383796:4244383796(0)
win 5840 <mss 1460,sackOK,timestamp 99292997 0,nop,wscale 6>
IPVS to RIP (IPIP)
18:01:29.522040 IP IPVS > RIP: IP {CIP.42852 > VIP.https: S
1869554645:1869554645(0) win 5840 <mss 1380,sackOK,timestamp 99292997
0,nop,wscale 6>} (ipip-proto-4)
CIP to RIP (Tunnel)
18:01:29.522040 IP CIP.42852 > VIP.https: S 1869554645:1869554645(0)
win 5840 <mss 1380,sackOK,timestamp 99292997 0,nop,wscale 6>
RIP to CIP
18:01:29.522175 IP VIP.https > CIP.42852: S 2673651702:2673651702(0)
ack 1869554646 win 5792 <mss 1460,sackOK,timestamp 552990048
99292997,nop,wscale 7>
Am I missing something here? Is this behaviour by design?
Thanks in advance!
Pat
Reply to: