[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Bug#702494: marked as done (malformed messages/gibberish in logs)

Your message dated Mon, 18 Oct 2021 14:13:39 +0200 (CEST)
with message-id <ef4c786-8a91-fe7a-f94-ed41aa9954a3@sourcepole.ch>
and subject line Re: Bug#702494: malformed messages/gibberish in logs
has caused the Debian Bug report #702494,
regarding malformed messages/gibberish in logs
to be marked as done.

This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
Bug report if necessary, and/or fix the problem forthwith.

(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact owner@bugs.debian.org
immediately.)


-- 
702494: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=702494
Debian Bug Tracking System
Contact owner@bugs.debian.org with problems
--- Begin Message --- 
Package: rsyslog
Version: 5.8.11-2
Severity: important

On two of our machines we are seeing that rsyslog is filling
up the log with undecipherable gibberish:


From /var/log/kern.log (other logs are also affected such as */syslog):


  Mar  7 03:29:02 brisi kernel: [236975.910574] EXT3-fs (dm-12): recovery complete
  Mar  7 03:37:06 brisi kernel: 9]X- m2 nnnjn<2596Xfd2mtfseiord e<6>[237459.93863] ET-s(m1) sn nenljra96]trnepeiufn d12>7.6 3p_a:lnnecie1
  Mar  7 03:37:06 brisi kernel: [492xr_ndirei3
  Mar  7 03:41:58 brisi kernel: 39]_nnegfed2>50X 2o st>56X 2c e<>[237712.63]jrlst.ottaes55E3sd1:ot lyewhrr tme15 ulti m elsn.34e3p_negfed9 rhnioe eee
  Mar  7 03:45:07 brisi kernel: >3526Xs- eptam

The corresponding output from dmesg:

  [236975.910574] EXT3-fs (dm-12): recovery complete
  [236975.984204] EXT3-fs (dm-12): mounted filesystem with ordered data mode
  [237050.760055] kjournald starting.  Commit interval 5 seconds
  [237050.789205] EXT3-fs (dm-12): using internal journal
  [237050.790426] EXT3-fs (dm-12): mounted filesystem with ordered data mode
  [237459.946661] kjournald starting.  Commit interval 5 seconds
  [237459.953863] EXT3-fs (dm-12): using internal journal
  [237459.965601] ext3_orphan_cleanup: deleting unreferenced inode 311354
  [237459.966597] ext3_orphan_cleanup: deleting unreferenced inode 311342
  [237459.966614] ext3_orphan_cleanup: deleting unreferenced inode 311340
  [237459.966621] ext3_orphan_cleanup: deleting unreferenced inode 311338
  [237459.967046] ext3_orphan_cleanup: deleting unreferenced inode 311298
  [237459.967056] EXT3-fs (dm-12): 5 orphan inodes deleted
  [237459.967662] EXT3-fs (dm-12): recovery complete
  [237460.037427] EXT3-fs (dm-12): mounted filesystem with ordered data mode
  [237712.565835] kjournald starting.  Commit interval 5 seconds
  [237712.596382] EXT3-fs (dm-12): using internal journal
  [237712.596995] EXT3-fs (dm-12): mounted filesystem with ordered data mode

Note that some of the timestamps in the kern.log and in dmesg correspond.

One could think that this is a result of the filesystem breaking, but that
is not the case. The above fsck's are from lvm snapshots that are (in theory) not interfering with the "real" FS.
Additionaly we are seing the same symptoms (gibberish in the logs) on another machine that doesn't do fscks on lvm snapshots, that is the gibberish occursare aparently random times there. 

This problem has also been reported to the rsyslog mailing list:
  http://lists.adiscon.net/pipermail/rsyslog/2013-March/031832.html
We have also tested this with rsyslog 7.2.5-1 from experimental (as seen inthe ML thread above).
It's weird that this problem hasn't been reported elsewhere yet, since we're seeing it on two different, independently set up machines. 
*t

--- End Message ---
--- Begin Message --- 
Hi everybody,

On Sat, 16 Oct 2021, Salvatore Bonaccorso wrote:


On Wed, Aug 26, 2015 at 02:43:04AM +0100, Ben Hutchings wrote:

I've seen this occasionally on various kernel versions in Debian and
also on SLE.

Last time I looked at one of these cases, the log messages weren't
complete gibberish, but only every second character was present.  In
this case the pattern of dropped characters is less regular.  Matching
up syslog and dmesg (with . for dropped characters):

...........9..]..X..-.. ..m..2.. ...n...n...n...j...n..
[237050.789205] EXT3-fs (dm-12): using internal journal

.....5...9...6...X...f...d...2...m...t...f...s...e...i...o...r...d... ...e
[237050.790426] EXT3-fs (dm-12): mounted filesystem with ordered data mode

[237459.9.3863] E.T.-.s.(.m.1.). .s.n. .n.e.n.l.j.r.a..
[237459.953863] EXT3-fs (dm-12): using internal journal

......9...6...]...........................t...................
[237459.946661] kjournald starting.  Commit interval 5 seconds

......................r...............................n................
[237459.965601] ext3_orphan_cleanup: deleting unreferenced inode 311354

................e......p......e...........i...u...f...n... ...d....1...
[237459.966597] ext3_orphan_cleanup: deleting unreferenced inode 311342

...7.......6... ...3...p..._...a...:...l...n...n...e...c...i...e...1...
[237459.966614] ext3_orphan_cleanup: deleting unreferenced inode 311340

[...4...9...2....x....r...._....n....d....i.....r....e.....i.....3.....
[237459.966621] ext3_orphan_cleanup: deleting unreferenced inode 311338

..3.....9.....]....._.....n.....n.....e.....g.....f.....e.....d........
[237459.967046] ext3_orphan_cleanup: deleting unreferenced inode 311298

.....5.....0.....X..... .....2.....o..... .....s.....t..
[237459.967056] EXT3-fs (dm-12): 5 orphan inodes deleted

.....5.....6.....X..... .....2.....c..... .....e..
[237459.967662] EXT3-fs (dm-12): recovery complete

[237712..6..3.]..j..r..l..s...t.......o...t...t...a.....e....s
[237712.565835] kjournald starting.  Commit interval 5 seconds

........5....5..E..3..s..d..1..:..o..t.. ..l..y..e..w..h..r..r.. ..t..m..e
[237712.596995] EXT3-fs (dm-12): mounted filesystem with ordered data mode


is this still something which people do observe or should we close
this bug?
Herewith I'm closing this bug. The bug is for old kernels that I am not running any more and I guess if the bug hasn't been fixed for those it won't any more. 

Please reopen this bug if anybody deems fit to do so.
Thanks a lot for triage Salvatore (and sorry for not coming back on the PM, I've fallen off the internet for a while and I will come back on the PM :-/ ).
Thanks Ben for caring so well for the kernels <3 <3, and to the others for helping with this bug! 
*t

--- End Message ---
Reply to: