Re: IP performance question
On Tue, May 26, 2015 at 12:42:50PM +0200, Petter Adsen wrote:
> On Sun, 24 May 2015 16:01:41 +0300
> Reco <recoverym4n@gmail.com> wrote:
>
> > Hi.
> >
> > On Sun, 24 May 2015 13:47:48 +0200
> > Petter Adsen <petter@synth.no> wrote:
> >
> > > On Sun, 24 May 2015 13:26:52 +0200
> > > Petter Adsen <petter@synth.no> wrote:
> > > > Thanks to you, I now get ~880Mbps, which is a lot better. It seems
> > > > increasing the MTU was what had the most effect, so I won't bother
> > > > with TCP window size.
> > >
> > > Now, this is a little odd:
> > >
> > > petter@monster:/etc$ iperf -i 1 -c fenris -r
> > > ------------------------------------------------------------
> > > Server listening on TCP port 5001
> > > TCP window size: 85.3 KByte (default)
> > > ------------------------------------------------------------
> > > ------------------------------------------------------------
> > > Client connecting to fenris, TCP port 5001
> > > TCP window size: 280 KByte (default)
> > > ------------------------------------------------------------
> > > [ 5] local 192.168.0.105 port 49636 connected with 192.168.0.103
> > > port 5001 [ ID] Interval Transfer Bandwidth
> > > [ 5] 0.0- 1.0 sec 104 MBytes 875 Mbits/sec
> > > [ 5] 1.0- 2.0 sec 97.8 MBytes 820 Mbits/sec
> > > [ 5] 2.0- 3.0 sec 104 MBytes 868 Mbits/sec
> > > [ 5] 3.0- 4.0 sec 104 MBytes 876 Mbits/sec
> > > [ 5] 4.0- 5.0 sec 104 MBytes 876 Mbits/sec
> > > [ 5] 5.0- 6.0 sec 83.0 MBytes 696 Mbits/sec
> > > [ 5] 6.0- 7.0 sec 105 MBytes 879 Mbits/sec
> > > [ 5] 7.0- 8.0 sec 104 MBytes 875 Mbits/sec
> > > [ 5] 8.0- 9.0 sec 105 MBytes 884 Mbits/sec
> > > [ 5] 9.0-10.0 sec 104 MBytes 877 Mbits/sec
> > > [ 5] 0.0-10.0 sec 1016 MBytes 852 Mbits/sec
> > > [ 4] local 192.168.0.105 port 5001 connected with 192.168.0.103
> > > port 34815 [ 4] 0.0- 1.0 sec 98.5 MBytes 826 Mbits/sec
> > > [ 4] 1.0- 2.0 sec 98.5 MBytes 826 Mbits/sec
> > > [ 4] 2.0- 3.0 sec 97.4 MBytes 817 Mbits/sec
> > > [ 4] 3.0- 4.0 sec 98.0 MBytes 822 Mbits/sec
> > > [ 4] 4.0- 5.0 sec 98.5 MBytes 827 Mbits/sec
> > > [ 4] 5.0- 6.0 sec 98.1 MBytes 823 Mbits/sec
> > > [ 4] 6.0- 7.0 sec 98.6 MBytes 827 Mbits/sec
> > > [ 4] 7.0- 8.0 sec 98.5 MBytes 826 Mbits/sec
> > > [ 4] 8.0- 9.0 sec 98.5 MBytes 827 Mbits/sec
> > > [ 4] 9.0-10.0 sec 98.5 MBytes 826 Mbits/sec
> > > [ 4] 0.0-10.0 sec 984 MBytes 825 Mbits/sec
> > >
> > > I have run it many times, and the results are consistently ~50Mbps
> > > lower in the other direction. MTU is set to 7152 on both hosts, but
> > > the window size is back to the default values (212992).
> >
> > Hmm. A first thought is that you have a different TCP window size on
> > client and a server.
>
> Nope. Exactly the same.
>
> > And a second thought is that you probably should check interface
> > statistics with ifconfig or 'ip -s link show'. Every packet that is
> > not RX or TX means trouble.
>
> Clean. On both hosts.
I'm out of ideas then, sorry.
> And even worse, after starting to mess with this, browsing is
> _abysmal_. After taking a few speed tests online (speed.io etc),
> upload/download and ping times seem good, but the number of connections
> per minute are severely limited, hovering at ~700. A friend on the same
> network, just down the street and with the same connection gets over
> 1800. We are connected to the same node. Whether these tests are
> trustworthy, though, I have no idea.
And that means jumbo frames bit you. Don't worry, good old iproute comes
to the rescue.
A start state of non-router host (I'm assuming that eth0 has MTU > 1500):
# ip ro l
default via 192.168.32.1 dev eth0 metric 303
192.168.32.0/20 dev eth0 proto kernel scope link src 192.168.32.227 metric 303
Needed changes:
# ip ro d default
# ip ro a default via 192.168.32.1 dev eth0 mtu 1500
So, you keep non-standard MTU for your network, but set standard MTU for
outside world.
An example assumes that 192.168.32.0/20 is an internal network and
192.168.32.1 is a router.
The implementation I'd use is a post-up script in
/etc/network/interfaces. I'm not that familiar with DHCP so I cannot
comment if it's possible to advertise different MTUs on different
routes.
> I've tried to set everything back to the defaults, as I've documented
> every change I've made, but it doesn't seem to help. I'll try to reboot
> later today if I can, I have so much context up right now that I really
> don't want to lose, but I haven't made any permanent changes yet, so it
> should come up the way it was.
May I suggest using etckeeper for this? The tool is invaluable if one
needs to answer a question such as "what exactly did I changed a couple
of days ago?". The usual caveat is that using etckeeper requires at
least casual knowledge of any RCS that's supported by etckeeper (I
prefer git for this).
Reco
Reply to: