Re: Have I been hacked?
Am Dienstag, 6. Januar 2015, 20:04:56 schrieb Danny:
> Hi guys,
Hi Danny!
> A while ago I posted a question about SFTP (I think the thread name was
> "SFTP Question") about attacks I got against my server after syslog warned
> me about an attempted breakin.
>
> Consequently I installed fail2ban and did a few other things to let me sleep
> better at night.
If someone has already introduced it is too late for fail2ban.
> However, prior to this breakin, in early December 2014, I noticed my network
> behaving strangely especially through wireless connections. I have Debian
> that acts as a gateway (wlan0->br0->eth0). wlan0 is the pickup for the
> internal network that gets bridged to eth0 which then goes through the
> router to the internet. What I noticed was that wireless connections would
> break down quickly, bind9 would fail to resolve (even on wired connections)
> and pages would load slow. In general it was chaos.
>
> Under the impression that it was a hardware failure, I changed the wlan0
> adapter. Still it was the same. So I bought a more expensive one, and still
> no change. I changed eth0 with an expensive one and still it was the same.
> I bought 2 new Netgear ADSL routers but the chaos was still there.
>
> wlan0, br0 and eth0 just didn't want to work together no more. Eventually I
> stopped all bootup scripts and processes trying to isolate the problem. And
> guess what, I found the culprit.
>
> Here it is:
> ##########################################################
> -rwxr-xr-x 1 root root 648K Dec 11 17:17 /boot/dippqejwvf
> ##########################################################
>
> This file got booted up and caused all the havoc. I moved it to a secure
> place and now it seems that all gremlins have gone away. The date on this
> file is 11 Dec 2014, right about the time my troubles started. I think that
> those Chinese guys got into my system even before syslog warned me a few
> days later.
Okay, if you already made sure that this file has been executed, do the
following:
- Make a backup of the server to a place you can be sure no one executes any
files from. (If need be from a filesystem mounted with noexec.)
- *Wipe* your server and *reinstall* from scratch. In case you need to restore
some data after a *clean* OS installation, look very carefully at the data
before restoring it. Especially if the data is executable in some form or is
used by other executables and can influence their behavior. A bunch of png or
jpeg files that are really what they claim to be should be quite safe, but PHP
files on a webserver: Reinstall the PHP application from scratch. In the most
recent version. Probably select another PHP application if its not maintained
on a regular base.
Thats about it.
Just removing a *single* suspicious file is likely not enough to *clean* your
system. A good malware is likely to install itself into mutiple places and
hides its presence, so what you may have found is just some left over of the
malware installation process.
> However, I have a few other weird looking files in the /boot directory. Can
> you guys please have a look at them and tell me if they are normal or not.
>
> #########################################################
> drwxr-xr-x 3 root root 4.0K Jan 6 19:35 .
> drwxr-xr-x 24 root root 4.0K Jan 3 17:23 ..
> -rwxr-xr-x 1 root root 648K Jan 6 19:03 aknaykocbs
> -rwxr-xr-x 1 root root 648K Jan 1 11:34 bxerzoalfk
> -rw-r--r-- 1 root root 157K Dec 10 18:57 config-3.16.0-0.bpo.4-686-pae
> -rw-r--r-- 1 root root 132K Dec 8 00:36 config-3.2.0-4-686-pae
> -rwxr-xr-x 1 root root 648K Dec 20 08:04 cwpgfmvkrk
> -rwxr-xr-x 1 root root 648K Dec 30 22:41 czhlgmsgzh
> -rwxr-xr-x 1 root root 648K Dec 30 20:03 dkseypedtx
> -rwxr-xr-x 1 root root 648K Jan 3 15:14 esijfkmwnd
> -rwxr-xr-x 1 root root 648K Dec 27 14:49 fndswijgdk
> -rwxr-xr-x 1 root root 0 Dec 20 08:14 gbwokvqoch
> drwxr-xr-x 3 root root 12K Jan 3 17:23 grub
> -rwxr-xr-x 1 root root 648K Jan 5 07:28 gyimenpwnt
> -rwxr-xr-x 1 root root 648K Dec 31 17:49 hjmmvaxfzq
> -rwxr-xr-x 1 root root 648K Dec 15 21:25 hutaslspbf
> -rw-r--r-- 1 root root 14M Jan 3 17:25 initrd.img-3.16.0-0.bpo.4-686-pae
> -rw-r--r-- 1 root root 11M Jan 2 22:01 initrd.img-3.2.0-4-686-pae
> -rwxr-xr-x 1 root root 648K Jan 2 18:47 isrgzlchmx
> -rwxr-xr-x 1 root root 648K Dec 27 14:56 izytxsbskq
> -rwxr-xr-x 1 root root 648K Jan 5 18:40 kvvcqvddix
> -rwxr-xr-x 1 root root 648K Jan 1 11:19 ryrfvxjggh
> -rwxr-xr-x 1 root root 0 Jan 5 19:08 sgopxfsiac
> -rw-r--r-- 1 root root 2.0M Dec 10 18:57 System.map-3.16.0-0.bpo.4-686-pae
> -rw-r--r-- 1 root root 1.6M Dec 8 00:36 System.map-3.2.0-4-686-pae
> -rwxr-xr-x 1 root root 648K Dec 30 20:40 ttqssdikcn
> -rwxr-xr-x 1 root root 0 Dec 26 17:11 utxlhlmnix
> -rwxr-xr-x 1 root root 0 Dec 12 07:29 vdqepbezvg
> -rw-r--r-- 1 root root 2.9M Dec 10 18:56 vmlinuz-3.16.0-0.bpo.4-686-pae
> -rw-r--r-- 1 root root 2.6M Dec 8 00:35 vmlinuz-3.2.0-4-686-pae
> -rwxr-xr-x 1 root root 648K Dec 31 17:30 wevzubbsgn
> -rwxr-xr-x 1 root root 648K Jan 1 09:46 xjeemjyuly
> -rwxr-xr-x 1 root root 648K Jan 1 17:10 zfmpizunja
> -rwxr-xr-x 1 root root 648K Jan 1 10:00 zkdjlvhuui
> -rwxr-xr-x 1 root root 0 Dec 30 22:32 zpaqgbuxvr
> ########################################################
>
> What bothers me is that the "other" files are all the same size (648k) as
> the suspected file I removed and they are very recent additions to the
> /boot directory.
These files are not supposed to be there. They are executable as well.
These *may* be some temporary files, but I am not aware of any *standard*
mechanism in Debian that would create such kind of files in /boot (instead of
in a suitable directory for temporary files).
- I´d run file -k and strings and probably hexdump on one of the files and
probably also a rootkit checker on it to find more about it.
- I´d also grep -ir for the files in at least /etc and /boot to check whether
they are referenced elsewhere. Its important to find out whether they are
executed in some startscript.
- I´d also check *all* crontabs whether they reference any of these files.
- I´d check for any unusual network connections with netstat -anp or similar
tools.
- I´d also check loaded kernel modules with lsmod and process list, but
processes may hide themselves from view.
- There are likely other ideas on what to do to find out more about the
situation. Consider this as a incomplete first list.
These checks may *all* fail if done from the running system as that if there
is a rootkit or other malware running, it may fool you. So I´d likely do all
checks that can be done this way from a live distro like GRML.
But even then:
You can never be sure you found all occurences of the malware. So I repeat my
recommendation:
In case you found any traces of a malware:
*Wipe* your server and *reinstall* from scratch.
Ciao,
--
Martin 'Helios' Steigerwald - http://www.Lichtvoll.de
GPG: 03B0 0D6C 0040 0710 4AFA B82F 991B EAAC A599 84C7
Reply to: