Bug#719993: linux-image-3.2.0-4-s390x: hangs hard after a few hours
Package: src:linux
Version: 3.2.46-1
Severity: grave
Forwarded: http://www.mail-archive.com/linux-390@vm.marist.edu/msg63844.html
The s390x kernel provided in Wheezy (both for s390 and s390x) hangs hard
after a few hours of uptime. After investigation, it has been determined
to be due to the following commit:
commit 4f37a68cdaf6dea833cfdded2a3e0c47c0f006da
Author: Martin Schwidefsky <schwidefsky@de.ibm.com>
Date: Tue Aug 23 15:29:44 2011 +0200
s390: Use direct ktime path for s390 clockevent device
The clock comparator on s390 uses the same format as the TOD clock.
If the value in the clock comparator is smaller than the current TOD
value an interrupt is pending. Use the CLOCK_EVT_FEAT_KTIME feature
to get the unmodified ktime of the next clockevent expiration and
use it to program the clock comparator without querying the TOD clock.
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
Cc: john stultz <johnstul@us.ibm.com>
Link: http://lkml.kernel.org/r/20110823133143.153017933@de.ibm.com
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
While the problem is being worked on, the best is probably to revert
this commit. This is what the attached patch does.
-- Package-specific info:
** Version:
Linux version 3.2.0-4-s390x (debian-kernel@lists.debian.org) (gcc version 4.6.3 (Debian 4.6.3-14) ) #1 SMP Debian 3.2.46-1
** Command line:
ro vmpoff="LOGOFF" root=/dev/dasda1 enable=ccw-0.0.0120 BOOT_IMAGE=0
** Not tainted
** Kernel log:
[ 0.000000] Initializing cgroup subsys cpuset
[ 0.000000] Initializing cgroup subsys cpu
[ 0.000000] Linux version 3.2.0-4-s390x (debian-kernel@lists.debian.org) (gcc version 4.6.3 (Debian 4.6.3-14) ) #1 SMP Debian 3.2.46-1
[ 0.000000] setup: Linux is running natively in 64-bit mode
[ 0.000000] cpu: The CPU configuration topology of the machine is: 0 0 0 0 0 2 / 1
[ 0.000000] Zone PFN ranges:
[ 0.000000] DMA 0x00000000 -> 0x00080000
[ 0.000000] Normal empty
[ 0.000000] Movable zone start PFN for each node
[ 0.000000] early_node_map[1] active PFN ranges
[ 0.000000] 0: 0x00000000 -> 0x00020000
[ 0.000000] On node 0 totalpages: 131072
[ 0.000000] DMA zone: 1792 pages used for memmap
[ 0.000000] DMA zone: 0 pages reserved
[ 0.000000] DMA zone: 129280 pages, LIFO batch:31
[ 0.000000] PERCPU: Embedded 11 pages/cpu @00000000012f5000 s15872 r8192 d20992 u45056
[ 0.000000] pcpu-alloc: s15872 r8192 d20992 u45056 alloc=11*4096
[ 0.000000] pcpu-alloc: [0] 00 [0] 01 [0] 02 [0] 03 [0] 04 [0] 05 [0] 06 [0] 07
[ 0.000000] pcpu-alloc: [0] 08 [0] 09 [0] 10 [0] 11 [0] 12 [0] 13 [0] 14 [0] 15
[ 0.000000] pcpu-alloc: [0] 16 [0] 17 [0] 18 [0] 19 [0] 20 [0] 21 [0] 22 [0] 23
[ 0.000000] pcpu-alloc: [0] 24 [0] 25 [0] 26 [0] 27 [0] 28 [0] 29 [0] 30 [0] 31
[ 0.000000] Built 1 zonelists in Zone order, mobility grouping on. Total pages: 129280
[ 0.000000] Kernel command line: ro vmpoff="LOGOFF" root=/dev/dasda1 enable=ccw-0.0.0120 BOOT_IMAGE=0
[ 0.000000] PID hash table entries: 2048 (order: 2, 16384 bytes)
[ 0.000000] Dentry cache hash table entries: 65536 (order: 7, 524288 bytes)
[ 0.000000] Inode-cache hash table entries: 32768 (order: 6, 262144 bytes)
[ 0.000000] Memory: 502720k/524288k available (3845k kernel code, 0k reserved, 2071k data, 228k init)
[ 0.000000] Write protected kernel read-only data: 0x11200 - 0x570fff
[ 0.000000] Hierarchical RCU implementation.
[ 0.000000] CONFIG_RCU_FANOUT set to non-default value of 32
[ 0.000000] RCU dyntick-idle grace-period acceleration is enabled.
[ 0.000000] console [ttyS0] enabled
[ 0.822121] Calibrating delay loop (skipped)... 3777.00 BogoMIPS preset
[ 0.822193] pid_max: default: 32768 minimum: 301
[ 0.823189] Security Framework initialized
[ 0.823276] AppArmor: AppArmor disabled by boot time parameter
[ 0.823830] Mount-cache hash table entries: 256
[ 0.826067] Initializing cgroup subsys cpuacct
[ 0.826244] Initializing cgroup subsys memory
[ 0.826436] Initializing cgroup subsys devices
[ 0.826540] Initializing cgroup subsys freezer
[ 0.826631] Initializing cgroup subsys net_cls
[ 0.826721] Initializing cgroup subsys blkio
[ 0.826853] Initializing cgroup subsys perf_event
[ 0.827802] cpu: 2 configured CPUs, 0 standby CPUs
[ 0.833026] Brought up 2 CPUs
[ 0.840344] devtmpfs: initialized
[ 0.850631] print_constraints: dummy:
[ 0.853330] NET: Registered protocol family 16
[ 0.871646] bio: create slab <bio-0> at 0
[ 1.196411] NET: Registered protocol family 2
[ 1.197650] IP route cache hash table entries: 4096 (order: 3, 32768 bytes)
[ 1.202055] TCP established hash table entries: 16384 (order: 6, 262144 bytes)
[ 1.204478] TCP bind hash table entries: 16384 (order: 6, 262144 bytes)
[ 1.207157] TCP: Hash tables configured (established 16384 bind 16384)
[ 1.207255] TCP reno registered
[ 1.207356] UDP hash table entries: 256 (order: 1, 8192 bytes)
[ 1.207542] UDP-Lite hash table entries: 256 (order: 1, 8192 bytes)
[ 1.209936] NET: Registered protocol family 1
[ 1.210824] Unpacking initramfs...
[ 5.169733] Freeing initrd memory: 5018k freed
[ 5.178775] audit: initializing netlink socket (disabled)
[ 5.178967] type=2000 audit(1376605775.711:1): initialized
[ 5.182711] HugeTLB registered 1 MB page size, pre-allocated 0 pages
[ 5.187510] VFS: Disk quotas dquot_6.5.2
[ 5.188676] Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[ 5.190118] msgmni has been set to 991
[ 5.193904] alg: No test for stdrng (krng)
[ 5.194487] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 254)
[ 5.194629] io scheduler noop registered
[ 5.194726] io scheduler deadline registered
[ 5.195212] io scheduler cfq registered (default)
[ 5.196621] hvc_iucv: The z/VM IUCV HVC device driver cannot be used without z/VM
[ 5.199129] cio: Channel measurement facility initialized using format basic (mode autodetected)
[ 5.214599] TCP cubic registered
[ 5.215539] NET: Registered protocol family 10
[ 5.226815] Mobile IPv6
[ 5.226890] NET: Registered protocol family 17
[ 5.227002] Registering the dns_resolver key type
[ 5.229837] PM: Hibernation image not present or could not be loaded.
[ 5.229945] registered taskstats version 1
[ 5.236092] Initializing network drop monitor service
[ 5.237241] Freeing unused kernel memory: 228k freed
[ 5.512379] udevd[51]: starting version 175
[ 5.756319] dasd-eckd 0.0.0120: New DASD 3390/0C (CU 3990/02) with 32760 cylinders, 15 heads, 224 sectors
[ 5.903605] dasd-eckd 0.0.0120: DASD with 4 KB/block, 23587200 KB total size, 48 KB/track, compatible disk layout
[ 5.905962] dasda:VOL1/ LIN120: dasda1 dasda2
[ 6.621367] EXT4-fs (dasda1): mounted filesystem with ordered data mode. Opts: (null)
[ 10.717869] udevd[224]: starting version 175
[ 11.568157] ctcm: CTCM driver initialized
[ 11.613125] lcs: Loading LCS driver
[ 12.259099] net ctc0: setup OK : r/w = ch-0.0.0a00/ch-0.0.0a01, protocol : 0
[ 15.378358] ioctl32(fgconsole:468): Unknown cmd fd(3) cmd(00005603){t:'V';sz:0} arg(7faf814e) on /dev/console
[ 16.186473] Adding 1219148k swap on /dev/dasda2. Priority:-1 extents:1 across:1219148k
[ 16.573775] EXT4-fs (dasda1): re-mounted. Opts: (null)
[ 17.891993] EXT4-fs (dasda1): re-mounted. Opts: errors=remount-ro
[ 33.554168] net ctc0: Connected with remote side
[ 34.643037] IPv6 over IPv4 tunneling driver
[ 39.261122] ioctl32(fgconsole:1625): Unknown cmd fd(3) cmd(00005603){t:'V';sz:0} arg(7fdd0d0e) on /dev/console
[ 1951.060067] hrtimer: interrupt took 10545003 ns
** Model information
processor 0: version = 00, identification = 002623, machine = 3090
processor 1: version = 00, identification = 102623, machine = 3090
** Loaded modules:
sit
tunnel4
lcs
ctcm
ccwgroup
fsm
ext4
crc16
jbd2
mbcache
dasd_eckd_mod
dasd_mod
** PCI devices:
not available
** USB devices:
not available
-- System Information:
Debian Release: 7.1
APT prefers stable-updates
APT policy: (500, 'stable-updates'), (500, 'stable')
Architecture: s390 (s390x)
Kernel: Linux 3.2.0-4-s390x (SMP w/2 CPU cores)
Locale: LANG=fr_FR.UTF-8, LC_CTYPE=fr_FR.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/bash
Versions of packages linux-image-3.2.0-4-s390x depends on:
ii debconf [debconf-2.0] 1.5.49
ii initramfs-tools [linux-initramfs-tool] 0.109.1
ii kmod 9-3
ii linux-base 3.5
ii module-init-tools 9-3
Versions of packages linux-image-3.2.0-4-s390x recommends:
pn firmware-linux-free <none>
Versions of packages linux-image-3.2.0-4-s390x suggests:
pn debian-kernel-handbook <none>
pn linux-doc-3.2 <none>
ii s390-tools 1.16.0-2
Versions of packages linux-image-3.2.0-4-s390x is related to:
pn firmware-atheros <none>
pn firmware-bnx2 <none>
pn firmware-bnx2x <none>
pn firmware-brcm80211 <none>
pn firmware-intelwimax <none>
pn firmware-ipw2x00 <none>
pn firmware-ivtv <none>
pn firmware-iwlwifi <none>
pn firmware-libertas <none>
pn firmware-linux <none>
pn firmware-linux-nonfree <none>
pn firmware-myricom <none>
pn firmware-netxen <none>
pn firmware-qlogic <none>
pn firmware-ralink <none>
pn firmware-realtek <none>
pn xen-hypervisor <none>
-- debconf information excluded
From: Aurelien Jarno <aurelien@aurel32.net>
Date: Fri Aug 16 07:01:50 2013 +0200
Subject: Revert "s390: Use direct ktime path for s390 clockevent device"
Forwarded: no (upstream is working on a real fix)
This reverts commit 4f37a68cdaf6dea833cfdded2a3e0c47c0f006da.
---
arch/s390/kernel/time.c | 19 ++++---------------
1 file changed, 4 insertions(+), 15 deletions(-)
diff --git a/arch/s390/kernel/time.c b/arch/s390/kernel/time.c
--- a/arch/s390/kernel/time.c
+++ b/arch/s390/kernel/time.c
@@ -110,20 +110,10 @@
set_clock_comparator(S390_lowcore.clock_comparator);
}
-static int s390_next_ktime(ktime_t expires,
+static int s390_next_event(unsigned long delta,
struct clock_event_device *evt)
{
- struct timespec ts;
- u64 nsecs;
-
- ts.tv_sec = ts.tv_nsec = 0;
- monotonic_to_bootbased(&ts);
- nsecs = ktime_to_ns(ktime_add(timespec_to_ktime(ts), expires));
- do_div(nsecs, 125);
- S390_lowcore.clock_comparator = sched_clock_base_cc + (nsecs << 9);
- /* Program the maximum value if we have an overflow (== year 2042) */
- if (unlikely(S390_lowcore.clock_comparator < sched_clock_base_cc))
- S390_lowcore.clock_comparator = -1ULL;
+ S390_lowcore.clock_comparator = get_clock() + delta;
set_clock_comparator(S390_lowcore.clock_comparator);
return 0;
}
@@ -148,15 +138,14 @@
cpu = smp_processor_id();
cd = &per_cpu(comparators, cpu);
cd->name = "comparator";
- cd->features = CLOCK_EVT_FEAT_ONESHOT |
- CLOCK_EVT_FEAT_KTIME;
+ cd->features = CLOCK_EVT_FEAT_ONESHOT;
cd->mult = 16777;
cd->shift = 12;
cd->min_delta_ns = 1;
cd->max_delta_ns = LONG_MAX;
cd->rating = 400;
cd->cpumask = cpumask_of(cpu);
- cd->set_next_ktime = s390_next_ktime;
+ cd->set_next_event = s390_next_event;
cd->set_mode = s390_set_mode;
clockevents_register_device(cd);
Reply to: