On Tue, 2011-06-28 at 15:13 +0200, Michal Pokrywka wrote: > Package: linux-2.6 > Version: 2.6.39-2 > Severity: important > Tags: upstream > > > I had working HFSC configuration on gateway machine with kernel > 2.6.26-21~bpo40+1. Now I switched to new machine and kernel 2.6.39-2 with the > same configuration script. Shaping works but after a couple of minutes > houndreds of warnings per second fills dmesg severly slowing whole system > (kernel log attached below). This is the warning in hfsc_schedule_watchdog(). Patrick, Michal, it looks like you made some changes in this area since 2.6.32 (the last version reported as OK). Can you see how this can happen? Ben. > My configuration on eth0 involves SFQ queues connected to HFSC leafs, and one > RED queue connected to one HFSC leaf. Similar configuration runs on ifb0. > All incoming to eth0 traffic is directed to ifb0 using egress action mirred. > Also there is some hashtable filtering based in IP addresses. > > When I erased all queues and put for testing purposes only one HFSC queue on > eth0 with one default class I noticed that the problem exists no more. > So I connected one SFQ queue to this single HFSC class and problem still > not occurs. > > Then I did the same thing on ifb0 and redirected traffic from eth0 egress > and noticed still no warnings in dmesg. > > Connecting RED to HFSC leaf also produces no warnings. > > Switching in udev eth0->eth1 and eth1->eth0 (to make HFSC work with different > ethernet driver: forcedeth->tg3) does not solve the problem. > > Maybe someone can suggest what else could be done to trace the problem. > > Here are the HFSC script that triggers the bug (without repeating stuff). > It is a bit complicated but it works with different users on three different > machines (different ethernet devices) with 2.6.32 kernel. > It should work on every machine which has 192.168.20.0/24 network on eth0 > and internet access on eth1. > > -- CUT HERE (hfsc.sh begins) -- > modprobe ifb numifbs=1 > ### DOWNLOAD ### > tc qdisc del dev eth0 root 2>&1 >/dev/null > tc qdisc add dev eth0 root handle 1: hfsc default 4 > tc class add dev eth0 parent 1: classid 1:1 hfsc ls rate 100000kbit ul rate 100000kbit # main eth0 download > > tc class add dev eth0 parent 1:1 classid 1:2 hfsc ls rate 3891kbit ul rate 3891kbit # main internet download > tc class add dev eth0 parent 1:2 classid 1:4 hfsc ls rate 50kbit ul rate 100kbit # bulk class > tc class add dev eth0 parent 1:2 classid 1:5 hfsc ls rate 200kbit ul rate 1000kbit # wireless class > tc qdisc add dev eth0 parent 1:4 handle 4: sfq perturb 5 quantum 1920 # bulk qdisc > tc qdisc add dev eth0 parent 1:5 handle 5: sfq perturb 5 quantum 1920 # wifi qdisc > > tc class add dev eth0 parent 1:1 classid 1:3 hfsc ls rate 94904kbit ul rate 94904kbit # local class > tc qdisc add dev eth0 parent 1:3 handle 3: red min 75000 max 225000 probability 0.02 limit 675000 avpkt 1000 burst 120 bandwidth 94904 > > tc filter add dev eth0 parent 1: protocol ip prio 3 u32 match ip src 192.168.20.0/24 match ip dst 62.***.***.88/29 flowid 1:3 > tc filter add dev eth0 parent 1: protocol ip prio 3 u32 match ip src 192.168.20.0/24 match ip dst 62.***.***.96/28 flowid 1:3 > tc filter add dev eth0 parent 1: protocol ip prio 3 u32 match ip dst 192.168.20.0/24 match ip src 62.***.***.88/29 flowid 1:3 > tc filter add dev eth0 parent 1: protocol ip prio 3 u32 match ip dst 192.168.20.0/24 match ip src 62.***.***.96/28 flowid 1:3 > tc filter add dev eth0 parent 1: protocol ip prio 3 u32 match ip src 192.168.20.91 match ip dst 192.168.20.0/24 flowid 1:3 > > tc filter add dev eth0 parent 1: protocol ip prio 3 u32 match ip dst 10.0.0.0/8 flowid 1:5 > > tc filter add dev eth0 parent 1: prio 5 handle 100: protocol ip u32 divisor 256 > tc filter add dev eth0 protocol ip parent 1: prio 5 u32 ht 800:: match ip dst 192.168.20.0/24 hashkey mask 0x000000ff at 16 link 100: > > # example_user / 192.168.20.16 > tc class add dev eth0 parent 1:2 classid 1:0016 hfsc ls m1 400kbit d 4s m2 100kbit ul rate 3696kbit > tc class add dev eth0 parent 1:0016 classid 1:1016 hfsc rt rate 336bit ls m1 400kbit d 4s m2 100kbit ul rate 3696kbit # ack > tc qdisc add dev eth0 parent 1:1016 handle 1016: sfq perturb 10 quantum 1920 > tc class add dev eth0 parent 1:0016 classid 1:2016 hfsc rt rate 94566bit ls m1 400kbit d 4s m2 100kbit ul rate 3696kbit # data > tc qdisc add dev eth0 parent 1:2016 handle 2016: sfq perturb 10 quantum 1920 > > tc filter add dev eth0 parent 1: protocol ip prio 5 u32 ht 100:10 match ip protocol 6 0xff match u8 0x10 0xff at nexthdr+33 match u16 0x0000 0xffc0 at 2 flowid 1:1016 # ack > tc filter add dev eth0 parent 1: protocol ip prio 10 u32 ht 100:10 match u32 0 0 flowid 1:2016 # data > > (... more users ...) > > ### UPLOAD ### > ifconfig ifb0 up > > tc qdisc del dev ifb0 root 2>&1 >/dev/null > tc qdisc add dev ifb0 root handle 1: hfsc default 4 > tc class add dev ifb0 parent 1: classid 1:1 hfsc ls rate 100000kbit ul rate 100000kbit # main eth0 upload > tc class add dev ifb0 parent 1:1 classid 1:2 hfsc ls rate 460kbit ul rate 460kbit # main internet upload > > tc class add dev ifb0 parent 1:2 classid 1:4 hfsc ls rate 50kbit ul rate 10kbit # bulk class > tc class add dev ifb0 parent 1:2 classid 1:5 hfsc ls rate 200kbit ul rate 100kbit # wireless class > tc qdisc add dev ifb0 parent 1:4 handle 4: sfq perturb 5 quantum 1920 # bulk qdisc > tc qdisc add dev ifb0 parent 1:5 handle 5: sfq perturb 5 quantum 1920 # wifi qdisc > > tc class add dev ifb0 parent 1:1 classid 1:3 hfsc ls rate 94904kbit ul rate 94904kbit # local class > tc qdisc add dev ifb0 parent 1:3 handle 3: red min 75000 max 225000 probability 0.02 limit 675000 avpkt 1000 burst 120 bandwidth 94904 > > tc filter add dev ifb0 parent 1: protocol ip prio 2 u32 match ip src 192.168.20.0/24 match ip dst 62.***.***.88/29 flowid 1:3 > tc filter add dev ifb0 parent 1: protocol ip prio 2 u32 match ip src 192.168.20.0/24 match ip dst 62.***.***.96/28 flowid 1:3 > tc filter add dev ifb0 parent 1: protocol ip prio 2 u32 match ip dst 192.168.20.0/24 match ip src 62.***.***.88/29 flowid 1:3 > tc filter add dev ifb0 parent 1: protocol ip prio 2 u32 match ip dst 192.168.20.0/24 match ip src 62.***.***.96/28 flowid 1:3 > tc filter add dev ifb0 parent 1: protocol ip prio 2 u32 match ip dst 192.168.20.91 match ip src 192.168.20.0/24 flowid 1:3 > tc filter add dev ifb0 parent 1: protocol ip prio 2 u32 match ip src 192.168.20.91 match ip dst 192.168.20.0/24 flowid 1:3 > > tc filter add dev ifb0 parent 1: protocol ip prio 3 u32 match ip src 10.0.0.0/8 flowid 1:5 > tc filter add dev ifb0 parent 1: prio 5 handle 100: protocol ip u32 divisor 256 > > # example_user / 192.168.20.16 > tc class add dev ifb0 parent 1:2 classid 1:0016 hfsc ls m1 400kbit d 4s m2 100kbit ul rate 437kbit > tc class add dev ifb0 parent 1:0016 classid 1:1016 hfsc rt rate 2847bit ls m1 400kbit d 4s m2 100kbit ul rate 437kbit # ack > tc qdisc add dev ifb0 parent 1:1016 handle 1016: sfq perturb 10 quantum 1920 > tc class add dev ifb0 parent 1:0016 classid 1:2016 hfsc rt rate 8372bit ls m1 400kbit d 4s m2 100kbit ul rate 437kbit # data > tc qdisc add dev ifb0 parent 1:2016 handle 2016: sfq perturb 10 quantum 1920 > > tc filter add dev ifb0 parent 1: protocol ip prio 5 u32 ht 100:10 match ip protocol 6 0xff match u8 0x10 0xff at nexthdr+33 match u16 0x0000 0xffc0 at 2 flowid 1:1016 # ack > tc filter add dev ifb0 parent 1: protocol ip prio 10 u32 ht 100:10 match u32 0 0 flowid 1:2016 # data > > (... more users ...) > > tc qdisc del dev eth0 ingress 2>&1 >/dev/null > tc qdisc add dev eth0 ingress > > tc filter add dev eth0 parent ffff: protocol ip prio 40 u32 match ip src 192.168.20.0/24 flowid 1: action mirred egress redirect dev ifb0 > tc filter add dev eth0 parent ffff: protocol ip prio 40 u32 match ip src 10.0.0.0/8 flowid 1: action mirred egress redirect dev ifb0 > > # Local to hashtable > tc filter add dev ifb0 protocol ip parent 1: prio 40 u32 ht 800:: match ip src 192.168.20.0/24 hashkey mask 0x000000ff at 12 link 100: > > -- CUT HERE (hfsc.sh ends) -- > > -- Package-specific info: > ** Version: > Linux version 2.6.39-2-amd64 (Debian 2.6.39-2) (ben@decadent.org.uk) (gcc version 4.4.6 (Debian 4.4.6-3) ) #1 SMP Wed Jun 8 11:01:04 UTC 2011 > > ** Command line: > BOOT_IMAGE=/boot/vmlinuz-2.6.39-2-amd64 root=UUID=b9cc0f29-98ad-4034-ba84-167769b10cd6 ro quiet > > ** Tainted: W (512) > * Taint on warning. > > ** Kernel log: > [ 4522.772401] ------------[ cut here ]------------ > [ 4522.772405] WARNING: at /build/buildd-linux-2.6_2.6.39-2-amd64-kuqdRa/linux-2.6-2.6.39/debian/build/source_amd64_none/net/sched/sch_hfsc.c:1427 hfsc_dequeue+0x155/0x28b [sch_hfsc]() > [ 4522.772410] Hardware name: S2865 > [ 4522.772412] Modules linked in: act_mirred sch_ingress cls_u32 sch_red sch_sfq sch_hfsc ifb xt_tcpudp xt_state iptable_filter iptable_nat ip_tables x_tables nf_nat_ftp nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_conntrack_ftp nf_conntrack xfs dme1737 hwmon_vid loop snd_pcm snd_timer amd64_edac_mod snd soundcore snd_page_alloc k8temp edac_core serio_raw edac_mce_amd pcspkr evdev joydev parport_pc parport i2c_nforce2 processor button i2c_core ext4 mbcache jbd2 crc16 raid1 md_mod sg usbhid hid sr_mod sd_mod crc_t10dif cdrom ohci_hcd ata_generic pata_amd sata_nv libata ehci_hcd tg3 scsi_mod thermal usbcore floppy fan thermal_sys libphy forcedeth [last unloaded: scsi_wait_scan] > [ 4522.772453] Pid: 3, comm: ksoftirqd/0 Tainted: G W 2.6.39-2-amd64 #1 > [ 4522.772455] Call Trace: > [ 4522.772459] [<ffffffff810458b4>] ? warn_slowpath_common+0x78/0x8c > [ 4522.772463] [<ffffffffa039ab3f>] ? hfsc_dequeue+0x155/0x28b [sch_hfsc] > [ 4522.772467] [<ffffffff81291962>] ? __qdisc_run+0x93/0x11a > [ 4522.772471] [<ffffffff8127696f>] ? net_tx_action+0x10f/0x186 > [ 4522.772475] [<ffffffff8104b4cf>] ? __do_softirq+0xc3/0x19e > [ 4522.772479] [<ffffffff8104b626>] ? run_ksoftirqd+0x7c/0x122 > [ 4522.772483] [<ffffffff8104b5aa>] ? __do_softirq+0x19e/0x19e > [ 4522.772486] [<ffffffff8104b5aa>] ? __do_softirq+0x19e/0x19e > [ 4522.772490] [<ffffffff8105ef05>] ? kthread+0x7a/0x82 > [ 4522.772494] [<ffffffff81339ee4>] ? kernel_thread_helper+0x4/0x10 > [ 4522.772498] [<ffffffff8105ee8b>] ? kthread_worker_fn+0x147/0x147 > [ 4522.772502] [<ffffffff81339ee0>] ? gs_change+0x13/0x13 > [ 4522.772505] ---[ end trace 9a0810cc4dbd421e ]--- [...] -- Ben Hutchings If at first you don't succeed, you're doing about average.
Attachment:
signature.asc
Description: This is a digitally signed message part