Re: suspicious netstat ouput
On Fri, 20 Apr 2001, Jonathan Freiermuth wrote:
> I got the following output from "netstat -elpn" on my firewall (kernel 2.4.2, iptables).
> /-(root@cerberus)-(166/ttyS0)-(17:56:42:Friday Apr 20)-
> ROOT : netstat -elpn
> Active Internet connections (only servers)
> Proto Recv-Q Send-Q Local Address Foreign Address State User Inode PID/Program name
> tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 0 1229 427/sshd
> tcp 0 0 0.0.0.0:25 0.0.0.0:* LISTEN 0 1542 487/sendmail: accep
> udp 0 0 0.0.0.0:1112 0.0.0.0:* 0 127022 16024/send-mail
> Active UNIX domain sockets (only servers)
> Proto RefCnt Flags Type State I-Node PID/Program name Path
> unix 2 [ ACC ] STREAM LISTENING 125202 15009/pump /var/run/pump.sock
> What's up with the send-mail process listening on port 1112 ? That
> looks really bad to me. A few seconds later the process is gone. Further
> netstat command only show sshd and sendmail.
My first thought would be some kind of trojan or backdoor listening on
port 1112 udp. With a name like "send-mail" a less aware/paranoid admin
would simply ignore it. Keeping this in mind no binary on the system
should be trusted at this point. Any program you use for investigation
should be reinstalled from a known clean source.
> Then I did a "find / -inum 127022" but there is no file with that
> inode. Uh oh. That can't be good either. The firewall runs an old redhat
> 6.2 install (haven't converted everything to debian, but I'm working on
> it!) with most everything turned off, as seen from the netstat output.
A faster way to find out what is using that port would be lsof. For
lsof -i udp:1112
Would show any process listening for udp on that port, along with the
PID. Once you have a PID you can check out /proc/PID for yourself and see
where the offending program resides. Why not use ps? That's normally one
of the first binaries to be replaced by a rootkit. pstree seems to be
> My iptables rules log and then drop everything by default, with ssh and
> mail rerouted to a server on the internal LAN using NAT. The following
> lines show up in my logfiles ( "UNKNOWN CONNECTION ATTEMPT" is a prefix
> added by my iptables rule).
> Apr 20 17:41:28 cerberus kernel: UNKNOWN CONNECTION ATTEMPT IN=eth0
> OUT= MAC=[snip] SRC=18.104.22.168 DST=22.214.171.124 LEN=69 TOS=0x00
> PREC=0x00 TTL=247 ID=27145 DF PROTO=UDP SPT=53 DPT=1112 LEN=49
> Apr 20 17:41:45 cerberus kernel: UNKNOWN CONNECTION ATTEMPT IN=eth0
> OUT= MAC=[snip] SRC=126.96.36.199 DST=188.8.131.52 LEN=69 TOS=0x00
> PREC=0x00 TTL=247 ID=7141 DF PROTO=UDP SPT=53 DPT=1112 LEN=49
> The SRC= addresses in the above are valid RoadRunner DNS servers. They
> are the ones I use.
Do you log any OUTGOING packets? This "send-mail" process could be making
DNS queries and listening for responses .. or something more
malicious. Obviously something had to be sent out to warrant a response
from a nameserver. Perhaps you could increase the log level and see what
you pickup that is outbound. A program like snort can log the full
packets, so you can see exactly what is being sent/received.
I know rpm has some verify functions that will help pick out binaries that
have been modified from their original state. Maybe it's just time to
wipe the system clean and install Debian ;)
> Jonathan Freiermuth