Re: NetXtreme BCM5722 strangeness on Proliant ML115

[Berni Elbourn <berni421@gmail.com>]

On 07/07/12 15:18, Camaleón wrote:
> On Fri, 06 Jul 2012 09:46:33 +0100, Berni Elbourn wrote:
>
> > 11:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5722
> > Gigabit Ethernet PCI Express
> >
> > With standard squeeze kernel seemingly once a day this nic stops
> > working. Looking at the switch port it seems the nic is transmitting (or
> > retransmitting) something as fast as possible. A simple ifdown/ifup
> > seems to cure for another day.
> >
> > On latest backport kernel 3.2.20-1~bpo60+1 (installed today) the nic
> > does seems to be stable. iperf reports 1gb performance. However the
> > count of dropped packets is slowly growing:
>
> (...)
>
> >             RX packets:8801144 errors:0 dropped:1824 overruns:0 frame:0
>
> (...)
>
> > This issue is present in the firmware nonfree from stable, and
> > backports, and the using the latest source compiled from broadcom
> > 3.122n. I have changed cables and switch ports. There is another other
> > gigabit nic on the switch is nVidia Corporation MCP77 and this has no
> > errors or dropped packets.
>
> So you have tested with almost all of the possibilities (you've discarded
> a hardware issue by replacing the patch cord and using a different switch
> port and you've discarded a software/driver problem by installing a
> different kernel and the latest available broadcom driver) yet still you
> don't see a noticeable improvement on this, right? Then it can be
> something  specific to your setup/environment... I would start with the
> ouput of "ethtool eth0" and "ethtool -k eth0" just in case.
>
> Have you noted an increment of packets being dropped when the system is
> running a concrete task or process that can exhaust the available memory?
> I ask this because Google suggest that dropped packates can be related to
> low memory situations :-?
>
> > Anyone else seeing this? How to progress?  .. should I log a debian bug,
> > or just go buy an Intel card? Or ? :-)
>
> In workstations and servers I always try to have at least a couple of
> different NIC cards (from different manufacturers and models) just to
> prevent these situations, because if you think about it, what's a server
> with no network connection? Nowadays, close to nothing; a toaster is even
> more useful :-)
>
> Greetings,

Strangest thing. The dropped packets stopped at 3000 odd. After reboots (this machine is shutdown overnight) the number 
of dropped packets seems to stop incrementing at 20 or 30 or so even after several gigabytes of transfer.

Performance however is fine with the backports kernel:

$ cat /proc/version
Linux version 3.2.0-0.bpo.2-amd64 (Debian 3.2.20-1~bpo60+1) (debian-kernel@lists.debian.org) (gcc version 4.4.5 (Debian 
4.4.5-8) ) #1 SMP Fri Jun 29 20:42:29 UTC 2012

$ sudo ethtool eth0
Settings for eth0:
	Supported ports: [ TP ]
	Supported link modes:   10baseT/Half 10baseT/Full
	                        100baseT/Half 100baseT/Full
	                        1000baseT/Half 1000baseT/Full
	Supports auto-negotiation: Yes
	Advertised link modes:  10baseT/Half 10baseT/Full
	                        100baseT/Half 100baseT/Full
	                        1000baseT/Half 1000baseT/Full
	Advertised pause frame use: Symmetric
	Advertised auto-negotiation: Yes
	Speed: 1000Mb/s
	Duplex: Full
	Port: Twisted Pair
	PHYAD: 1
	Transceiver: internal
	Auto-negotiation: on
	MDI-X: Unknown
	Supports Wake-on: g
	Wake-on: g
	Current message level: 0x000000ff (255)
	Link detected: yes

$ sudo ethtool -k eth0
Offload parameters for eth0:
rx-checksumming: on
tx-checksumming: on
scatter-gather: on
tcp-segmentation-offload: on
udp-fragmentation-offload: off
generic-segmentation-offload: on
generic-receive-offload: on
large-receive-offload: off
ntuple-filters: off
receive-hashing: off

$ sudo ethtool -i eth0
driver: tg3
version: 3.121
firmware-version: 5722-v3.07, ASFIPMI v6.02
bus-info: 0000:11:00.0

eth0      Link encap:Ethernet  HWaddr 00:21:5a:d3:d0:0c
          inet addr:192.168.2.10  Bcast:192.168.2.255  Mask:255.255.255.0
          inet6 addr: fe80::221:5aff:fed3:d00c/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:123154 errors:0 *dropped:26* overruns:0 frame:0
          TX packets:131936 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:15913111 (15.1 MiB)  TX bytes:107913843 (102.9 MiB)
          Interrupt:19



--
"Confidence is what you have before you understand a problem" - Woody Allen