[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Bug#536236: marked as done (kernel: [56585.121000] BUG: soft lockup - CPU#1 stuck for 61s! [pdflush:210])

Your message dated Sun, 30 May 2010 19:11:31 +0200
with message-id <20100530171130.GA2453@galadriel.inutil.org>
and subject line Re: kernel: [56585.121000] BUG: soft lockup - CPU#1 stuck for 61s! [pdflush:210]
has caused the Debian Bug report #536236,
regarding kernel: [56585.121000] BUG: soft lockup - CPU#1 stuck for 61s! [pdflush:210]
to be marked as done.

This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
Bug report if necessary, and/or fix the problem forthwith.

(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact owner@bugs.debian.org
immediately.)


-- 
536236: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=536236
Debian Bug Tracking System
Contact owner@bugs.debian.org with problems
--- Begin Message --- 
Package: linux-image-2.6.30-1-amd64
Severity: critical
Justification: breaks the whole system

I am running this kernel from the lenny-backports so it's named a little different. I WAS running this exact package name and got these errors on another machine. I decided to use the backports for lenny, so I uninstalled it and install 2.6.30-bpo.1-amd64. I was getting this error with 2.6.30-bpo.1-amd64 and 2.6.30-1-amd64, so I thought it was time to file a bug.

NOTE: I AM NOT USING THE FSCACHE built into this new kernel on this box. I was on another box with this kernel but also received these same errors.

I am running a web server with Apache 2.2.11, PHP 5.2.9 with 4 NFS shares. Here are my NFS mounts from fstab:


10.0.0.1:/export/home           /home   nfs     rw,rsize=8192,wsize=8192,intr,noatime           0       0
10.0.0.1:/export/sites/content  /var/www        nfs     rw,rsize=32768,wsize=32768,intr,noatime         0       0
10.0.0.1:/export/sites/content/sessions /var/phpsessions nfs     rw,rsize=32768,wsize=32768,intr,noatime             0       0
10.0.0.1:/export/sites/conf     /etc/apache2/sites-available    nfs     rw                      0       0

Heere is what syslog has to say.

Jul  8 06:26:25 web02 kernel: [56585.121000] BUG: soft lockup - CPU#1 stuck for 61s! [pdflush:210]
Jul  8 06:26:25 web02 kernel: [56585.121002] Modules linked in: ip6table_filter ip6_tables iptable_raw xt_comment xt_recent xt_policy ipt_ULOG ipt_REJECT ipt_REDIRECT ipt_NETMAP ipt_MASQUERADE ipt_LOG ipt_ECN ipt_ecn ipt_CLUSTERIP ipt_ah ipt_addrtype nf_nat_tftp nf_nat_snmp_basic nf_nat_sip nf_nat_pptp nf_nat_proto_gre nf_nat_irc nf_nat_h323 nf_nat_ftp nf_nat_amanda ts_kmp nf_conntrack_amanda nf_conntrack_tftp nf_conntrack_sip nf_conntrack_proto_sctp nf_conntrack_pptp nf_conntrack_proto_gre nf_conntrack_netlink nf_conntrack_netbios_ns nf_conntrack_irc nf_conntrack_h323 nf_conntrack_ftp xt_tcpmss xt_pkttype xt_physdev xt_owner xt_NFQUEUE xt_NFLOG nfnetlink_log xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_iprange xt_helper xt_hashlimit xt_DSCP xt_dscp xt_dccp xt_conntrack xt_CONNMARK xt_connmark xt_CLASSIFY xt_tcpudp xt_state iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_conntrack iptable_mangle nfnetlink iptable_filter ip_tables x_tables nfs lockd fscache
  nfs_acl auth_rpcgss sunrpc tcp_bic
Jul  8 06:26:25 web02 kernel: loop evdev dcdbas i2c_i801 pcspkr psmouse serio_raw i2c_core rng_core snd_pcm snd_timer snd soundcore snd_page_alloc iTCO_wdt shpchp pci_hotplug button processor i3000_edac edac_core jfs nls_base raid1 md_mod sd_mod crc_t10dif piix ide_pci_generic ide_core ata_piix ata_generic libata scsi_mod ehci_hcd tg3 libphy uhci_hcd thermal fan thermal_sys
Jul  8 06:26:25 web02 kernel: [56585.121002] CPU 1:
Jul  8 06:26:25 web02 kernel: [56585.121002] Modules linked in: ip6table_filter ip6_tables iptable_raw xt_comment xt_recent xt_policy ipt_ULOG ipt_REJECT ipt_REDIRECT ipt_NETMAP ipt_MASQUERADE ipt_LOG ipt_ECN ipt_ecn ipt_CLUSTERIP ipt_ah ipt_addrtype nf_nat_tftp nf_nat_snmp_basic nf_nat_sip nf_nat_pptp nf_nat_proto_gre nf_nat_irc nf_nat_h323 nf_nat_ftp nf_nat_amanda ts_kmp nf_conntrack_amanda nf_conntrack_tftp nf_conntrack_sip nf_conntrack_proto_sctp nf_conntrack_pptp nf_conntrack_proto_gre nf_conntrack_netlink nf_conntrack_netbios_ns nf_conntrack_irc nf_conntrack_h323 nf_conntrack_ftp xt_tcpmss xt_pkttype xt_physdev xt_owner xt_NFQUEUE xt_NFLOG nfnetlink_log xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_iprange xt_helper xt_hashlimit xt_DSCP xt_dscp xt_dccp xt_conntrack xt_CONNMARK xt_connmark xt_CLASSIFY xt_tcpudp xt_state iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_conntrack iptable_mangle nfnetlink iptable_filter ip_tables x_tables nfs lockd fscache
  nfs_acl auth_rpcgss sunrpc tcp_bic
Jul  8 06:26:25 web02 kernel: loop evdev dcdbas i2c_i801 pcspkr psmouse serio_raw i2c_core rng_core snd_pcm snd_timer snd soundcore snd_page_alloc iTCO_wdt shpchp pci_hotplug button processor i3000_edac edac_core jfs nls_base raid1 md_mod sd_mod crc_t10dif piix ide_pci_generic ide_core ata_piix ata_generic libata scsi_mod ehci_hcd tg3 libphy uhci_hcd thermal fan thermal_sys
Jul  8 06:26:25 web02 kernel: [56585.121002] Pid: 210, comm: pdflush Tainted: G    B      2.6.30-bpo.1-amd64 #1 PowerEdge 850
Jul  8 06:26:25 web02 kernel: [56585.121002] RIP: 0010:[<ffffffff802919f6>]  [<ffffffff802919f6>] find_get_pages_tag+0x6b/0xdd
Jul  8 06:26:25 web02 kernel: [56585.121002] RSP: 0018:ffff88010d1c7b10  EFLAGS: 00000297
Jul  8 06:26:25 web02 kernel: [56585.121002] RAX: ffffffffffffffff RBX: ffff88010d1c7be0 RCX: 0000000000000000
Jul  8 06:26:25 web02 kernel: [56585.121002] RDX: 0000000000040000 RSI: ffffe200032e05d0 RDI: ffffe200032e05c8
Jul  8 06:26:25 web02 kernel: [56585.121002] RBP: ffffffff802105ce R08: ffff88010d1c7ad0 R09: 0000000000000000
Jul  8 06:26:25 web02 kernel: [56585.121002] R10: ffff88010c4962c0 R11: 0000000000040000 R12: ffff88010d1c7be8
Jul  8 06:26:25 web02 kernel: [56585.121002] R13: 0000000d00000206 R14: ffff88010d1c7ad0 R15: 0000000000000010
Jul  8 06:26:25 web02 kernel: [56585.121002] FS:  0000000000000000(0000) GS:ffff880028037000(0000) knlGS:0000000000000000
Jul  8 06:26:25 web02 kernel: [56585.121002] CS:  0010 DS: 0018 ES: 0018 CR0: 000000008005003b
Jul  8 06:26:25 web02 kernel: [56585.121002] CR2: 00007fdaf571c015 CR3: 0000000000201000 CR4: 00000000000006e0
Jul  8 06:26:25 web02 kernel: [56585.121002] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Jul  8 06:26:25 web02 kernel: [56585.121002] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Jul  8 06:26:25 web02 kernel: [56585.121002] Call Trace:
Jul  8 06:26:25 web02 kernel: [56585.121002]  [<ffffffff80299a7a>] ? pagevec_lookup_tag+0x1a/0x21
Jul  8 06:26:25 web02 kernel: [56585.121002]  [<ffffffff802982c0>] ? write_cache_pages+0x162/0x322
Jul  8 06:26:25 web02 kernel: [56585.121002]  [<ffffffffa029e688>] ? nfs_writepages_callback+0x0/0x20 [nfs]
Jul  8 06:26:25 web02 kernel: [56585.121002]  [<ffffffff802355ff>] ? __wake_up+0x30/0x44
Jul  8 06:26:25 web02 kernel: [56585.121002]  [<ffffffffa0164c64>] ? txCommit+0xc19/0xc7e [jfs]
Jul  8 06:26:25 web02 kernel: [56585.121002]  [<ffffffffa029e647>] ? nfs_writepages+0xef/0x130 [nfs]
Jul  8 06:26:25 web02 kernel: [56585.121002]  [<ffffffffa029f84f>] ? nfs_flush_one+0x0/0xce [nfs]
Jul  8 06:26:25 web02 kernel: [56585.121002]  [<ffffffffa01652bc>] ? txEnd+0x61/0x138 [jfs]
Jul  8 06:26:25 web02 kernel: [56585.121002]  [<ffffffff802984bc>] ? do_writepages+0x20/0x2d
Jul  8 06:26:25 web02 kernel: [56585.121002]  [<ffffffff802d8b55>] ? __writeback_single_inode+0x18e/0x398
Jul  8 06:26:25 web02 kernel: [56585.121002]  [<ffffffff80235f3b>] ? dequeue_entity+0xf/0x11f
Jul  8 06:26:25 web02 kernel: [56585.121002]  [<ffffffff802d91b4>] ? generic_sync_sb_inodes+0x2a7/0x43d
Jul  8 06:26:25 web02 kernel: [56585.121002]  [<ffffffff802d94fc>] ? writeback_inodes+0x7e/0xce
Jul  8 06:26:25 web02 kernel: [56585.121002]  [<ffffffff80298606>] ? wb_kupdate+0xaa/0x121
Jul  8 06:26:25 web02 kernel: [56585.121002]  [<ffffffff8029926f>] ? pdflush+0x159/0x23b
Jul  8 06:26:25 web02 kernel: [56585.121002]  [<ffffffff8029855c>] ? wb_kupdate+0x0/0x121
Jul  8 06:26:25 web02 kernel: [56585.121002]  [<ffffffff80299116>] ? pdflush+0x0/0x23b
Jul  8 06:26:25 web02 kernel: [56585.121002]  [<ffffffff8025431e>] ? kthread+0x54/0x80
Jul  8 06:26:25 web02 kernel: [56585.121002]  [<ffffffff80210aca>] ? child_rip+0xa/0x20
Jul  8 06:26:25 web02 kernel: [56585.121002]  [<ffffffff802542ca>] ? kthread+0x0/0x80
Jul  8 06:26:25 web02 kernel: [56585.121002]  [<ffffffff80210ac0>] ? child_rip+0x0/0x20


Then it repeats again.

Jul  8 06:27:30 web02 kernel: [56650.621000] BUG: soft lockup - CPU#1 stuck for 61s! [pdflush:210]
Jul  8 06:27:30 web02 kernel: [56650.621002] Modules linked in: ip6table_filter ip6_tables iptable_raw xt_comment xt_recent xt_policy ipt_ULOG ipt_REJECT ipt_REDIRECT ipt_NETMAP ipt_MASQUERADE ipt_LOG ipt_ECN ipt_ecn ipt_CLUSTERIP ipt_ah ipt_addrtype nf_nat_tftp nf_nat_snmp_basic nf_nat_sip nf_nat_pptp nf_nat_proto_gre nf_nat_irc nf_nat_h323 nf_nat_ftp nf_nat_amanda ts_kmp nf_conntrack_amanda nf_conntrack_tftp nf_conntrack_sip nf_conntrack_proto_sctp nf_conntrack_pptp nf_conntrack_proto_gre nf_conntrack_netlink nf_conntrack_netbios_ns nf_conntrack_irc nf_conntrack_h323 nf_conntrack_ftp xt_tcpmss xt_pkttype xt_physdev xt_owner xt_NFQUEUE xt_NFLOG nfnetlink_log xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_iprange xt_helper xt_hashlimit xt_DSCP xt_dscp xt_dccp xt_conntrack xt_CONNMARK xt_connmark xt_CLASSIFY xt_tcpudp xt_state iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_conntrack iptable_mangle nfnetlink iptable_filter ip_tables x_tables nfs lockd fscache
  nfs_acl auth_rpcgss sunrpc tcp_bic
Jul  8 06:27:30 web02 kernel: loop evdev dcdbas i2c_i801 pcspkr psmouse serio_raw i2c_core rng_core snd_pcm snd_timer snd soundcore snd_page_alloc iTCO_wdt shpchp pci_hotplug button processor i3000_edac edac_core jfs nls_base raid1 md_mod sd_mod crc_t10dif piix ide_pci_generic ide_core ata_piix ata_generic libata scsi_mod ehci_hcd tg3 libphy uhci_hcd thermal fan thermal_sys
Jul  8 06:27:30 web02 kernel: [56650.621002] CPU 1:
Jul  8 06:27:30 web02 kernel: [56650.621002] Modules linked in: ip6table_filter ip6_tables iptable_raw xt_comment xt_recent xt_policy ipt_ULOG ipt_REJECT ipt_REDIRECT ipt_NETMAP ipt_MASQUERADE ipt_LOG ipt_ECN ipt_ecn ipt_CLUSTERIP ipt_ah ipt_addrtype nf_nat_tftp nf_nat_snmp_basic nf_nat_sip nf_nat_pptp nf_nat_proto_gre nf_nat_irc nf_nat_h323 nf_nat_ftp nf_nat_amanda ts_kmp nf_conntrack_amanda nf_conntrack_tftp nf_conntrack_sip nf_conntrack_proto_sctp nf_conntrack_pptp nf_conntrack_proto_gre nf_conntrack_netlink nf_conntrack_netbios_ns nf_conntrack_irc nf_conntrack_h323 nf_conntrack_ftp xt_tcpmss xt_pkttype xt_physdev xt_owner xt_NFQUEUE xt_NFLOG nfnetlink_log xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_iprange xt_helper xt_hashlimit xt_DSCP xt_dscp xt_dccp xt_conntrack xt_CONNMARK xt_connmark xt_CLASSIFY xt_tcpudp xt_state iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_conntrack iptable_mangle nfnetlink iptable_filter ip_tables x_tables nfs lockd fscache
  nfs_acl auth_rpcgss sunrpc tcp_bic
Jul  8 06:27:30 web02 kernel: loop evdev dcdbas i2c_i801 pcspkr psmouse serio_raw i2c_core rng_core snd_pcm snd_timer snd soundcore snd_page_alloc iTCO_wdt shpchp pci_hotplug button processor i3000_edac edac_core jfs nls_base raid1 md_mod sd_mod crc_t10dif piix ide_pci_generic ide_core ata_piix ata_generic libata scsi_mod ehci_hcd tg3 libphy uhci_hcd thermal fan thermal_sys
Jul  8 06:27:30 web02 kernel: [56650.621002] Pid: 210, comm: pdflush Tainted: G    B      2.6.30-bpo.1-amd64 #1 PowerEdge 850
Jul  8 06:27:30 web02 kernel: [56650.621002] RIP: 0010:[<ffffffff802919f6>]  [<ffffffff802919f6>] find_get_pages_tag+0x6b/0xdd
Jul  8 06:27:30 web02 kernel: [56650.621002] RSP: 0018:ffff88010d1c7b10  EFLAGS: 00000297
Jul  8 06:27:30 web02 kernel: [56650.621002] RAX: ffffffffffffffff RBX: ffff88010d1c7be0 RCX: 0000000000000000
Jul  8 06:27:30 web02 kernel: [56650.621002] RDX: 0000000000040000 RSI: ffffe200032e05d0 RDI: ffffe200032e05c8
Jul  8 06:27:30 web02 kernel: [56650.621002] RBP: ffffffff802105ce R08: ffff88010d1c7ad0 R09: 0000000000000000
Jul  8 06:27:30 web02 kernel: [56650.621002] R10: ffff88010c4962c0 R11: 0000000000040000 R12: ffff88010d1c7be8
Jul  8 06:27:30 web02 kernel: [56650.621002] R13: 0000000d00000206 R14: ffff88010d1c7ad0 R15: 0000000000000010
Jul  8 06:27:30 web02 kernel: [56650.621002] FS:  0000000000000000(0000) GS:ffff880028037000(0000) knlGS:0000000000000000
Jul  8 06:27:30 web02 kernel: [56650.621002] CS:  0010 DS: 0018 ES: 0018 CR0: 000000008005003b
Jul  8 06:27:30 web02 kernel: [56650.621002] CR2: 00007fdaf571c015 CR3: 0000000000201000 CR4: 00000000000006e0
Jul  8 06:27:30 web02 kernel: [56650.621002] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Jul  8 06:27:30 web02 kernel: [56650.621002] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Jul  8 06:27:30 web02 kernel: [56650.621002] Call Trace:
Jul  8 06:27:30 web02 kernel: [56650.621002]  [<ffffffff80299a7a>] ? pagevec_lookup_tag+0x1a/0x21
Jul  8 06:27:30 web02 kernel: [56650.621002]  [<ffffffff802982c0>] ? write_cache_pages+0x162/0x322
Jul  8 06:27:30 web02 kernel: [56650.621002]  [<ffffffffa029e688>] ? nfs_writepages_callback+0x0/0x20 [nfs]
Jul  8 06:27:30 web02 kernel: [56650.621002]  [<ffffffff802355ff>] ? __wake_up+0x30/0x44
Jul  8 06:27:30 web02 kernel: [56650.621002]  [<ffffffffa0164c64>] ? txCommit+0xc19/0xc7e [jfs]
Jul  8 06:27:30 web02 kernel: [56650.621002]  [<ffffffffa029e647>] ? nfs_writepages+0xef/0x130 [nfs]
Jul  8 06:27:30 web02 kernel: [56650.621002]  [<ffffffffa029f84f>] ? nfs_flush_one+0x0/0xce [nfs]
Jul  8 06:27:30 web02 kernel: [56650.621002]  [<ffffffffa01652bc>] ? txEnd+0x61/0x138 [jfs]
Jul  8 06:27:30 web02 kernel: [56650.621002]  [<ffffffff802984bc>] ? do_writepages+0x20/0x2d
Jul  8 06:27:30 web02 kernel: [56650.621002]  [<ffffffff802d8b55>] ? __writeback_single_inode+0x18e/0x398
Jul  8 06:27:30 web02 kernel: [56650.621002]  [<ffffffff80235f3b>] ? dequeue_entity+0xf/0x11f
Jul  8 06:27:30 web02 kernel: [56650.621002]  [<ffffffff802d91b4>] ? generic_sync_sb_inodes+0x2a7/0x43d
Jul  8 06:27:30 web02 kernel: [56650.621002]  [<ffffffff802d94fc>] ? writeback_inodes+0x7e/0xce
Jul  8 06:27:30 web02 kernel: [56650.621002]  [<ffffffff80298606>] ? wb_kupdate+0xaa/0x121
Jul  8 06:27:30 web02 kernel: [56650.621002]  [<ffffffff8029926f>] ? pdflush+0x159/0x23b
Jul  8 06:27:30 web02 kernel: [56650.621002]  [<ffffffff8029855c>] ? wb_kupdate+0x0/0x121
Jul  8 06:27:30 web02 kernel: [56650.621002]  [<ffffffff80299116>] ? pdflush+0x0/0x23b
Jul  8 06:27:30 web02 kernel: [56650.621002]  [<ffffffff8025431e>] ? kthread+0x54/0x80
Jul  8 06:27:30 web02 kernel: [56650.621002]  [<ffffffff80210aca>] ? child_rip+0xa/0x20
Jul  8 06:27:30 web02 kernel: [56650.621002]  [<ffffffff802542ca>] ? kthread+0x0/0x80
Jul  8 06:27:30 web02 kernel: [56650.624003]  [<ffffffff80210ac0>] ? child_rip+0x0/0x20

And again...

Jul  8 06:28:36 web02 kernel: [56716.117000] BUG: soft lockup - CPU#1 stuck for 61s! [pdflush:210]
Jul  8 06:28:36 web02 kernel: [56716.117001] Modules linked in: ip6table_filter ip6_tables iptable_raw xt_comment xt_recent xt_policy ipt_ULOG ipt_REJECT ipt_REDIRECT ipt_NETMAP ipt_MASQUERADE ipt_LOG ipt_ECN ipt_ecn ipt_CLUSTERIP ipt_ah ipt_addrtype nf_nat_tftp nf_nat_snmp_basic nf_nat_sip nf_nat_pptp nf_nat_proto_gre nf_nat_irc nf_nat_h323 nf_nat_ftp nf_nat_amanda ts_kmp nf_conntrack_amanda nf_conntrack_tftp nf_conntrack_sip nf_conntrack_proto_sctp nf_conntrack_pptp nf_conntrack_proto_gre nf_conntrack_netlink nf_conntrack_netbios_ns nf_conntrack_irc nf_conntrack_h323 nf_conntrack_ftp xt_tcpmss xt_pkttype xt_physdev xt_owner xt_NFQUEUE xt_NFLOG nfnetlink_log xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_iprange xt_helper xt_hashlimit xt_DSCP xt_dscp xt_dccp xt_conntrack xt_CONNMARK xt_connmark xt_CLASSIFY xt_tcpudp xt_state iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_conntrack iptable_mangle nfnetlink iptable_filter ip_tables x_tables nfs lockd fscache
  nfs_acl auth_rpcgss sunrpc tcp_bic
Jul  8 06:28:36 web02 kernel: loop evdev dcdbas i2c_i801 pcspkr psmouse serio_raw i2c_core rng_core snd_pcm snd_timer snd soundcore snd_page_alloc iTCO_wdt shpchp pci_hotplug button processor i3000_edac edac_core jfs nls_base raid1 md_mod sd_mod crc_t10dif piix ide_pci_generic ide_core ata_piix ata_generic libata scsi_mod ehci_hcd tg3 libphy uhci_hcd thermal fan thermal_sys
Jul  8 06:28:36 web02 kernel: [56716.117001] CPU 1:
Jul  8 06:28:36 web02 kernel: [56716.117001] Modules linked in: ip6table_filter ip6_tables iptable_raw xt_comment xt_recent xt_policy ipt_ULOG ipt_REJECT ipt_REDIRECT ipt_NETMAP ipt_MASQUERADE ipt_LOG ipt_ECN ipt_ecn ipt_CLUSTERIP ipt_ah ipt_addrtype nf_nat_tftp nf_nat_snmp_basic nf_nat_sip nf_nat_pptp nf_nat_proto_gre nf_nat_irc nf_nat_h323 nf_nat_ftp nf_nat_amanda ts_kmp nf_conntrack_amanda nf_conntrack_tftp nf_conntrack_sip nf_conntrack_proto_sctp nf_conntrack_pptp nf_conntrack_proto_gre nf_conntrack_netlink nf_conntrack_netbios_ns nf_conntrack_irc nf_conntrack_h323 nf_conntrack_ftp xt_tcpmss xt_pkttype xt_physdev xt_owner xt_NFQUEUE xt_NFLOG nfnetlink_log xt_multiport xt_MARK xt_mark xt_mac xt_limit xt_length xt_iprange xt_helper xt_hashlimit xt_DSCP xt_dscp xt_dccp xt_conntrack xt_CONNMARK xt_connmark xt_CLASSIFY xt_tcpudp xt_state iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_conntrack iptable_mangle nfnetlink iptable_filter ip_tables x_tables nfs lockd fscache
  nfs_acl auth_rpcgss sunrpc tcp_bic
Jul  8 06:28:36 web02 kernel: loop evdev dcdbas i2c_i801 pcspkr psmouse serio_raw i2c_core rng_core snd_pcm snd_timer snd soundcore snd_page_alloc iTCO_wdt shpchp pci_hotplug button processor i3000_edac edac_core jfs nls_base raid1 md_mod sd_mod crc_t10dif piix ide_pci_generic ide_core ata_piix ata_generic libata scsi_mod ehci_hcd tg3 libphy uhci_hcd thermal fan thermal_sys
Jul  8 06:28:36 web02 kernel: [56716.117001] Pid: 210, comm: pdflush Tainted: G    B      2.6.30-bpo.1-amd64 #1 PowerEdge 850
Jul  8 06:28:36 web02 kernel: [56716.117001] RIP: 0010:[<ffffffff802919f6>]  [<ffffffff802919f6>] find_get_pages_tag+0x6b/0xdd
Jul  8 06:28:36 web02 kernel: [56716.117001] RSP: 0018:ffff88010d1c7b10  EFLAGS: 00000297
Jul  8 06:28:36 web02 kernel: [56716.117001] RAX: ffffffffffffffff RBX: ffff88010d1c7be0 RCX: 0000000000000000
Jul  8 06:28:36 web02 kernel: [56716.117001] RDX: 0000000000040000 RSI: ffffe200032e05d0 RDI: ffffe200032e05c8
Jul  8 06:28:36 web02 kernel: [56716.117001] RBP: ffffffff802105ce R08: ffff88010d1c7ad0 R09: 0000000000000000
Jul  8 06:28:36 web02 kernel: [56716.117001] R10: ffff88010c4962c0 R11: 0000000000040000 R12: ffff88010d1c7be8
Jul  8 06:28:36 web02 kernel: [56716.117001] R13: 0000000d00000206 R14: ffff88010d1c7ad0 R15: 0000000000000010
Jul  8 06:28:36 web02 kernel: [56716.117001] FS:  0000000000000000(0000) GS:ffff880028037000(0000) knlGS:0000000000000000
Jul  8 06:28:36 web02 kernel: [56716.117001] CS:  0010 DS: 0018 ES: 0018 CR0: 000000008005003b
Jul  8 06:28:36 web02 kernel: [56716.117001] CR2: 00007fdaf571c015 CR3: 0000000000201000 CR4: 00000000000006e0
Jul  8 06:28:36 web02 kernel: [56716.117001] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Jul  8 06:28:36 web02 kernel: [56716.117001] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Jul  8 06:28:36 web02 kernel: [56716.117001] Call Trace:
Jul  8 06:28:36 web02 kernel: [56716.117001]  [<ffffffff80299a7a>] ? pagevec_lookup_tag+0x1a/0x21
Jul  8 06:28:36 web02 kernel: [56716.117001]  [<ffffffff802982c0>] ? write_cache_pages+0x162/0x322
Jul  8 06:28:36 web02 kernel: [56716.117001]  [<ffffffffa029e688>] ? nfs_writepages_callback+0x0/0x20 [nfs]
Jul  8 06:28:36 web02 kernel: [56716.117001]  [<ffffffff802355ff>] ? __wake_up+0x30/0x44
Jul  8 06:28:36 web02 kernel: [56716.117001]  [<ffffffffa0164c64>] ? txCommit+0xc19/0xc7e [jfs]
Jul  8 06:28:36 web02 kernel: [56716.117001]  [<ffffffffa029e647>] ? nfs_writepages+0xef/0x130 [nfs]
Jul  8 06:28:36 web02 kernel: [56716.117001]  [<ffffffffa029f84f>] ? nfs_flush_one+0x0/0xce [nfs]
Jul  8 06:28:36 web02 kernel: [56716.117001]  [<ffffffffa01652bc>] ? txEnd+0x61/0x138 [jfs]
Jul  8 06:28:36 web02 kernel: [56716.117001]  [<ffffffff802984bc>] ? do_writepages+0x20/0x2d
Jul  8 06:28:36 web02 kernel: [56716.117001]  [<ffffffff802d8b55>] ? __writeback_single_inode+0x18e/0x398
Jul  8 06:28:36 web02 kernel: [56716.117001]  [<ffffffff80235f3b>] ? dequeue_entity+0xf/0x11f
Jul  8 06:28:36 web02 kernel: [56716.117001]  [<ffffffff802d91b4>] ? generic_sync_sb_inodes+0x2a7/0x43d
Jul  8 06:28:36 web02 kernel: [56716.117001]  [<ffffffff802d94fc>] ? writeback_inodes+0x7e/0xce
Jul  8 06:28:36 web02 kernel: [56716.117001]  [<ffffffff80298606>] ? wb_kupdate+0xaa/0x121
Jul  8 06:28:36 web02 kernel: [56716.117001]  [<ffffffff8029926f>] ? pdflush+0x159/0x23b
Jul  8 06:28:36 web02 kernel: [56716.117001]  [<ffffffff8029855c>] ? wb_kupdate+0x0/0x121
Jul  8 06:28:36 web02 kernel: [56716.117001]  [<ffffffff80299116>] ? pdflush+0x0/0x23b
Jul  8 06:28:36 web02 kernel: [56716.117001]  [<ffffffff8025431e>] ? kthread+0x54/0x80
Jul  8 06:28:36 web02 kernel: [56716.117001]  [<ffffffff80210aca>] ? child_rip+0xa/0x20
Jul  8 06:28:36 web02 kernel: [56716.117001]  [<ffffffff802542ca>] ? kthread+0x0/0x80
Jul  8 06:28:36 web02 kernel: [56716.117001]  [<ffffffff80210ac0>] ? child_rip+0x0/0x20


And then 10 more times or so...

The system becomes unresponsive via SSh, Apache, etc. The only way to get back into this machine is to hard power off and then back on via the power button.


Anything else you need from me or want me to check?

-- System Information:
Debian Release: 5.0.2
  APT prefers stable
  APT policy: (500, 'stable')
Architecture: amd64 (x86_64)

Kernel: Linux 2.6.30-bpo.1-amd64 (SMP w/2 CPU cores)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/bash

--- End Message ---
--- Begin Message --- 
On Sun, May 30, 2010 at 08:50:37AM -0400, Matt Jamison wrote:
> You can go ahead and close this case, as I won't be able to help you troubleshoot this system. This was a production web server and since we were having this problem, we moved to Squid and used it's caching abilities to do what we needed. 

Thanks, closing the bug then.

Cheers,
        Moritz

--- End Message ---
Reply to: