[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: weird symptom - possible infection

Hi Johannes,
Yeah... I didn't see anything in the log to explain the crash either - I posted it in response to someone else's request. Power is the other thing I thought of too - though in this case, I'm in a datacenter, and there are a couple of other people's boxes on the same UPC - with no reported events at the time my box crashed. 

Sigh....  Don't you hate unexplained behavior.

Thanks,

Miles

Johannes Wiedersich wrote:

Miles Fidelman wrote:
ok, another look and I do find some suspicious stuff -- I've been having a number of people try to crack the machine for a while, but (I thought) to no avail 

from auth.log on both machines:
a whole slew of these, and similar entries with different user names (both 8/7 and 8/21 logs) Aug 7 08:49:07 server2 sshd[11271]: Illegal user diamond from ::ffff:60.28.24.84 Aug 7 08:49:10 server2 sshd[11273]: Illegal user heaven from ::ffff:60.28.24.84 Aug 7 08:49:12 server2 sshd[11275]: Illegal user guadalupe from ::ffff:60.28.24.84
This means that someone was unsuccessfully trying to log in via ssh. Nothing to worry, it happens all the time... If you don't want to permit remote ssh logins, you could disable sshd or set up a firewall. 


and these (only the earlier log):
Aug 7 06:48:02 server2 sshd[6567]: Address 66.132.182.28 maps to goldcrownresor t.com, but this does not map back to the address - POSSIBLE BREAKIN ATTEMPT! Aug 7 06:48:03 server2 sshd[6569]: Address 66.132.182.28 maps to goldcrownresor t.com, but this does not map back to the address - POSSIBLE BREAKIN ATTEMPT! Aug 7 06:48:03 server2 sshd[6571]: Address 66.132.182.28 maps to goldcrownresor t.com, but this does not map back to the address - POSSIBLE BREAKIN ATTEMPT! Aug 7 06:48:04 server2 sshd[6573]: Address 66.132.182.28 maps to goldcrownresor t.com, but this does not map back to the address - POSSIBLE BREAKIN ATTEMPT! 

Don't know about these, but it is warning of an attempt only.


BUT... the events stopped a couple of hours before the reboot

from auth.log on 1st server, today:
Aug 21 11:50:01 server1 CRON[27533]: (pam_unix) session closed for user root
This simply means that cron is doing something as root. This is normal behaviour.
Aug 21 11:55:01 server1 CRON[27540]: (pam_unix) session opened for user root by 
(uid=0)
Aug 21 11:55:01 server1 CRON[27540]: (pam_unix) session closed for user root Aug 21 12:00:01 server1 CRON[27550]: (pam_unix) session opened for user root by 
(uid=0)
Aug 21 12:00:01 server1 CRON[27550]: (pam_unix) session closed for user root Aug 21 12:02:01 server1 CRON[27556]: (pam_unix) session opened for user logcheck 
by (uid=0)
Aug 21 12:02:04 server1 CRON[27556]: (pam_unix) session closed for user logcheck 
Aug 21 12:05:43 server1 sshd[3166]: Server listening on :: port 22.
Aug 21 12:05:44 server1 perl: (pam_unix) authentication failure; logname= uid=0 
euid=0 tty= ruser= rhost=  user=root


I don't know about this, but if it is an login attempt, it failed.


Aug 21 12:05:46 server1 webmin[3187]: Webmin starting
Aug 21 12:05:46 server1 CRON[3296]: (pam_unix) session opened for user logcheck 
by (uid=0)
Aug 21 12:05:50 server1 CRON[3296]: (pam_unix) session closed for user logcheck Aug 21 12:09:01 server1 CRON[4163]: (pam_unix) session opened for user root by ( 
uid=0)

the two lines that caught my eye are:
Aug 21 12:05:43 server1 sshd[3166]: Server listening on :: port 22.
Are you sure that you blocked your sshd? This is the default configuration, if you have package ssh installed.
Aug 21 12:05:44 server1 perl: (pam_unix) authentication failure; logname= uid=0 
euid=0 tty= ruser= rhost=  user=root
This could be a perl script running on your machine. Check cron et al. for this. 

[snip]


syslog from the less built-up machine:
Aug 21 10:17:01 server2 /USR/SBIN/CRON[1534]: (root) CMD ( run-parts --report 
/etc/cron.hourly)
Aug 21 10:46:05 server2 -- MARK --
Aug 21 11:06:05 server2 -- MARK --
Aug 21 11:17:01 server2 /USR/SBIN/CRON[1538]: (root) CMD ( run-parts --report 
/etc/cron.hourly)
Aug 21 11:46:05 server2 -- MARK --


Your server crashed sometime after above and rebooted at time below.


Aug 21 12:06:08 server2 syslogd 1.4.1#17: restart.
Aug 21 12:06:08 server2 kernel: klogd 1.4.1#17, log source = /proc/kmsg started. 
Aug 21 12:06:08 server2 kernel: Inspecting /boot/System.map-2.6.8-2-386
Aug 21 12:06:08 server2 kernel: Loaded 28183 symbols from /boot/System.map-2.6.8 
-2-386.
Aug 21 12:06:08 server2 kernel: Symbols match kernel version 2.6.8.
Aug 21 12:06:08 server2 kernel: No module symbols loaded - kernel modules not en 
abled.
Aug 21 12:06:08 server2 kernel: \_SB_.PCI0.PEX1._PRT]
Aug 21 12:06:08 server2 kernel: ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.HU 


Any further thoughts?


Maybe your UPS are faulty? Just a thought.
I never really figured this out, but I had a similar problem with our webserver. It was the only machine that more or less regularly (every 2 to 4 weeks) suddenly rebooted without any indications in syslog, just like yours. I first also thought about different other reasons for this unpleasant behaviour. At last resort, I just connected it to 'ordinary' power and removed the UPS. It's never been down since then, except for kernel updates. 

Johannes
Reply to: