Re: Intel 82576 Gigabit on Debian 7 slow speed.
On 12/18/2015 09:44 AM, Mimiko wrote:
ifconfig
bond0 Link encap:Ethernet HWaddr
UP BROADCAST RUNNING MASTER MULTICAST MTU:9000 Metric:1
RX packets:654308767 errors:0 dropped:5238 overruns:0 frame:0
TX packets:761897714 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:343017135502 (319.4 GiB) TX bytes:877734258061
(817.4 GiB)
eth0 Link encap:Ethernet HWaddr
UP BROADCAST RUNNING SLAVE MULTICAST MTU:9000 Metric:1
RX packets:426388262 errors:0 dropped:0 overruns:0 frame:0
TX packets:504528230 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:201613380511 (187.7 GiB) TX bytes:593461966181
(552.7 GiB)
Memory:fafe0000-fb000000
eth1 Link encap:Ethernet HWaddr
UP BROADCAST RUNNING SLAVE MULTICAST MTU:9000 Metric:1
RX packets:227920505 errors:0 dropped:0 overruns:0 frame:0
TX packets:257369484 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:141403754991 (131.6 GiB) TX bytes:284272291880
(264.7 GiB)
Memory:faf60000-faf80000
What version of Debian? Which kernel? Any other customizations?
I think 5238 dropped packets for several days compared to received
packets is negligible.
Likely so.
ethtool -i bond0
driver: bonding
version: 3.7.1
firmware-version: 2
bus-info:
supports-statistics: no
supports-test: no
supports-eeprom-access: no
supports-register-dump: no
supports-priv-flags: no
ethtool eth0
Settings for eth0:
Supported ports: [ TP ]
Supported link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
Supported pause frame use: Symmetric
Supports auto-negotiation: Yes
Advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
Advertised pause frame use: Symmetric
Advertised auto-negotiation: Yes
Speed: 1000Mb/s
Duplex: Full
Port: Twisted Pair
PHYAD: 1
Transceiver: internal
Auto-negotiation: on
MDI-X: Unknown
Supports Wake-on: pumbg
Wake-on: g
Current message level: 0x00000007 (7)
drv probe link
Link detected: yes
ethtool eth1
Settings for eth1:
Supported ports: [ TP ]
Supported link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
Supported pause frame use: Symmetric
Supports auto-negotiation: Yes
Advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
Advertised pause frame use: Symmetric
Advertised auto-negotiation: Yes
Speed: 1000Mb/s
Duplex: Full
Port: Twisted Pair
PHYAD: 1
Transceiver: internal
Auto-negotiation: on
MDI-X: Unknown
Supports Wake-on: pumbg
Wake-on: g
Current message level: 0x00000007 (7)
drv probe link
Link detected: yes
I don't see any obvious problems.
> What does the (managed?) switch say?
It shows 1Gbit.
> Are the cables okay (cat6?)?
Yes. Cables are new cat6e.
Category 6E is an oddball:
https://en.wikipedia.org/wiki/Category_6_cable#Category_6e
But, if the cables ran at 100 MB/s under Windows you're probably okay.
> Which kind of transfer?nfs/cifs/iscsi or raw iperf-test?
I've tried send/receive cifs (samba). Also I've tried sendng file with
pscp using Kitty. Both give same low speed.
NFS or iSCSI isn't used.
Okay.
iperf -c ip
------------------------------------------------------------
Client connecting to ip, TCP port 5001
TCP window size: 23.5 KByte (default)
------------------------------------------------------------
[ 3] local ip port 36389 connected with ip port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-10.0 sec 835 MBytes 700 Mbits/sec
70% of 1Gbit. Is this seems a problem with samba?
iperf should be testing the hardware, device drivers, and kernel on both
ends. What make/ model Ethernet chips do you have on the two hosts?
# lspci | grep -i ethernet
What is your switch?
Is there anything else between the two computers?
dd if=/dev/zero of=test.bin bs=1M count=10000
10000+0 records in
10000+0 records out
10485760000 bytes (10 GB) copied, 24.5841 s, 427 MB/s
I assume you ran that on your Samba server (?).
A stream of zeros is very easy to compress and can give misleading
results. It's better to use random numbers. (A hardware RNG is useful
for this.)
dd if=test.bin of=/dev/null bs=1M
10000+0 records in
10000+0 records out
10485760000 bytes (10 GB) copied, 2.24662 s, 4.7 GB/s
The 4.7 GB/s came from the kernel page cache. You want to sync(1) and
drop all caches strategically when benchmarking:
http://www.linuxinsight.com/proc_sys_vm_drop_caches.html
Disk is a zfs raid:
zpool create -f -m none -o ashift=12 zfspool raidz2 .... (total 8 x 1TB
+ 8 x 2TB disk in SATA2 supermicro backplane)
zfs set atime=off zfspool
zfs set dedup=off zfspool
zfs create -V 4T zfspool/backup
zfs set compression=lz4 zfspool/backup
mkfs.ext4 -b 4096 -q /dev/zvol/zfspool/backup
Is this samba problem or zfs problem?
I assume you're running the latest ZFS on Linux? (ZFS-fuse is
considerably slower and is behind on features.)
Why did you have to -f (force) creation?
LUKS? On the raw drives, or on partitions? How many CPU's and cores?
HT? AES-NI?
Different-sized drives makes things more interesting. What arrangements
did you try? What did you settle on, and why?
Why raidz2?
Turning off atime is okay if none of the applications that use the file
systems need it.
General de-duplication can be risky, but for many backup solutions it is
a killer feature (along with replication).
A ZFS volume with an ext4 file system? Why?
For backups? I keep mine on dedicated disks in another machine.
Compression is generally regarded as good.
Lots of RAM is recommended in all cases. How much RAM does you server
have? Also note that some people strongly recommend ECC RAM. Is your
RAM ECC?
David
Reply to: