Gigabit ethernet low traffic and high latency

[Sylvain Le Gall <gildor@debian.org>]

Hello,

I have spend the two last day, trying to solve a strange problem. I have
2 hosts connected using Gigabit Ethernet link and a switch between:
* host giga: NFS server, Cat6 SFTP link, RTL8111/8168B NIC (r8169), Core 2 Duo
  T7600, 1GB RAM, SATA drive (xen dom0), linux 2.6.18, etch
* host grand: NFS client, Cat6 SFTP link, 3C940 NIC (skge), P4 SMT, 1GB
  RAM, linux 2.6.21.1, sid
* switch 3Com OfficeConnect 8 port Gigabit, all links negotiate
  1000BaseT/full

The directory /home/share/backup/old/gildor/ contains a lot of hidden
files (former home dir).

Here is my test :
grand$> umount /home/share && mount /home/share
grand$> time ls /home/share/backup/old/gildor/
...
real    0m59.553s
user    0m0.000s
sys     0m0.000s


giga$>ping -i 0.2 grand

grand$> umount /home/share && mount /home/share
grand$> time ls /home/share/backup/old/gildor/
...

real    0m0.470s
user    0m0.000s
sys     0m0.000s

When a ping is running hitting the host, the NFS returns almost
immediatly. It is possible to run the ping on both host, but you need to
ping the other end. This has no effect when you ping broadcast, or
another non existing host.

I try to :
* change the switch
* use other host as NFS client
* follow indication of http://dsd.lbl.gov/TCP-tuning/linux.html and
  http://datatag.web.cern.ch/datatag/howto/tcp.html
* change cable
* connect directly one host to another
* add hosts to /etc/hosts 
* disable/enable host caching in nscd

Does anyone already solve this problem ?

Regards,
Sylvain Le Gall