[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Bug#800445: nf_conntrack: table full, dropping packet



Control: tag -1 upstream fixed-upstream patch moreinfo

On Tue, 2015-09-29 at 08:20 -0400, anomie@users.sourceforge.net wrote:
> Package: linux-image-4.2.0-1-amd64
> Version: 4.2.1-2
> 
> After booting the kernel image provided by this package and attempting
> to connect to the Internet, network access doesn't actually work. The
> message "nf_conntrack: table full, dropping packet" is repeatedly
> logged.
> 
> Comparing the contents of the various /proc/sys conntrack files between
> a working 4.1 kernel (from linux-image-4.1.0-2-amd64) and the broken
> 4.2, the only difference I see is that nf_conntrack_count has a value
> "-5". Yes, negative 5. /proc/net/stat/nf_conntrack's "entries" column
> matches this with a value of fffffffb. /proc/net/nf_conntrack is empty
> on the 4.2 kernel, while it has a handful of expected entries on 4.1.
> 
> I have iptables rules set up by shorewall on this machine, including
> configuration to forward/masq traffic on the interface used by vde2, in
> case that helps reproduce this. Removing all the iptables rules and
> removing the nf_conntrack_ipv4 module (and everything that depends on
> it, of course) stops the error. Disabling shorewall at boot allows
> network functionality and starting shorewall later didn't immediately
> cause the problem, but in some experimentation after the system locked
> up.

I think I found the fix for this, but please can you test the attached
patch?  (Instructions for building a patched kernel package are at 
<https://kernel-handbook.alioth.debian.org/ch-common-tasks.html#s-common-official>.)

Ben.

-- 
Ben Hutchings
compatible: Gracefully accepts erroneous data from any source
From: Daniel Borkmann <daniel@iogearbox.net>
Date: Mon, 31 Aug 2015 19:11:02 +0200
Subject: netfilter: conntrack: use nf_ct_tmpl_free in CT/synproxy error paths
Origin: https://git.kernel.org/linus/9cf94eab8b309e8bcc78b41dd1561c75b537dd0b
Bug-Debian: https://bugs.debian.org/800445

Commit 0838aa7fcfcd ("netfilter: fix netns dependencies with conntrack
templates") migrated templates to the new allocator api, but forgot to
update error paths for them in CT and synproxy to use nf_ct_tmpl_free()
instead of nf_conntrack_free().

Due to that, memory is being freed into the wrong kmemcache, but also
we drop the per net reference count of ct objects causing an imbalance.

In Brad's case, this leads to a wrap-around of net->ct.count and thus
lets __nf_conntrack_alloc() refuse to create a new ct object:

  [   10.340913] xt_addrtype: ipv6 does not support BROADCAST matching
  [   10.810168] nf_conntrack: table full, dropping packet
  [   11.917416] r8169 0000:07:00.0 eth0: link up
  [   11.917438] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
  [   12.815902] nf_conntrack: table full, dropping packet
  [   15.688561] nf_conntrack: table full, dropping packet
  [   15.689365] nf_conntrack: table full, dropping packet
  [   15.690169] nf_conntrack: table full, dropping packet
  [   15.690967] nf_conntrack: table full, dropping packet
  [...]

With slab debugging, it also reports the wrong kmemcache (kmalloc-512 vs.
nf_conntrack_ffffffff81ce75c0) and reports poison overwrites, etc. Thus,
to fix the problem, export and use nf_ct_tmpl_free() instead.

Fixes: 0838aa7fcfcd ("netfilter: fix netns dependencies with conntrack templates")
Reported-by: Brad Jackson <bjackson0971@gmail.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
---
 include/net/netfilter/nf_conntrack.h | 1 +
 net/netfilter/nf_conntrack_core.c    | 3 ++-
 net/netfilter/nf_synproxy_core.c     | 2 +-
 net/netfilter/xt_CT.c                | 2 +-
 4 files changed, 5 insertions(+), 3 deletions(-)

diff --git a/include/net/netfilter/nf_conntrack.h b/include/net/netfilter/nf_conntrack.h
index 37cd391..4023c4c 100644
--- a/include/net/netfilter/nf_conntrack.h
+++ b/include/net/netfilter/nf_conntrack.h
@@ -292,6 +292,7 @@ extern unsigned int nf_conntrack_hash_rnd;
 void init_nf_conntrack_hash_rnd(void);
 
 struct nf_conn *nf_ct_tmpl_alloc(struct net *net, u16 zone, gfp_t flags);
+void nf_ct_tmpl_free(struct nf_conn *tmpl);
 
 #define NF_CT_STAT_INC(net, count)	  __this_cpu_inc((net)->ct.stat->count)
 #define NF_CT_STAT_INC_ATOMIC(net, count) this_cpu_inc((net)->ct.stat->count)
diff --git a/net/netfilter/nf_conntrack_core.c b/net/netfilter/nf_conntrack_core.c
index 3c20d02..0625a42 100644
--- a/net/netfilter/nf_conntrack_core.c
+++ b/net/netfilter/nf_conntrack_core.c
@@ -320,12 +320,13 @@ out_free:
 }
 EXPORT_SYMBOL_GPL(nf_ct_tmpl_alloc);
 
-static void nf_ct_tmpl_free(struct nf_conn *tmpl)
+void nf_ct_tmpl_free(struct nf_conn *tmpl)
 {
 	nf_ct_ext_destroy(tmpl);
 	nf_ct_ext_free(tmpl);
 	kfree(tmpl);
 }
+EXPORT_SYMBOL_GPL(nf_ct_tmpl_free);
 
 static void
 destroy_conntrack(struct nf_conntrack *nfct)
diff --git a/net/netfilter/nf_synproxy_core.c b/net/netfilter/nf_synproxy_core.c
index d7f1685..d6ee8f8 100644
--- a/net/netfilter/nf_synproxy_core.c
+++ b/net/netfilter/nf_synproxy_core.c
@@ -378,7 +378,7 @@ static int __net_init synproxy_net_init(struct net *net)
 err3:
 	free_percpu(snet->stats);
 err2:
-	nf_conntrack_free(ct);
+	nf_ct_tmpl_free(ct);
 err1:
 	return err;
 }
diff --git a/net/netfilter/xt_CT.c b/net/netfilter/xt_CT.c
index 43ddeee..f3377ce 100644
--- a/net/netfilter/xt_CT.c
+++ b/net/netfilter/xt_CT.c
@@ -233,7 +233,7 @@ out:
 	return 0;
 
 err3:
-	nf_conntrack_free(ct);
+	nf_ct_tmpl_free(ct);
 err2:
 	nf_ct_l3proto_module_put(par->family);
 err1:

Attachment: signature.asc
Description: This is a digitally signed message part


Reply to: