Re: Linux machine hit by ransomware
On 7/6/25 19:47, Rick Macdonald wrote:
I apologize for the length of this question.
After running Debian for nearly 30 years (and other distros prior to
that), my Linux server has been hit by a ransomware attack about 11 days
ago.
I would power off all computers on your network. Only boot them when
they are known good (e.g. re-imaged or OS reinstall).
Change passwords and generate new user SSH keys (with passwords) on all
local hosts.
Get or make uninfected bootable media that you can use to boot
computers, install, and/or reinstall as needed.
Document your work thoroughly. Use the console and script(1) whenever
possible. Use a version control system for any and all files you touch.
I have backups, so nothing important has been lost at this point.
Do you have a multiple backups? If not, get more disks and do so.
Consider archiving your last good backup to write-once optical media
(e.g. CD-R, DVD-R, BD-R, DL, XL, etc.).
You are going to need backups while you recover.
However, I can't figure out how it got in, how it works, if there are
executables on my computer that need to be cleaned, etc. I believe I
have been able to stop the attack, by simply fixing permissions on
directories and files.
However, that obviously doesn't remove or block
the attack from my machine.
When I search for this malware on the web, I find Windows-specific
discussions. If I'm unable to learn what to do from the folks here,
suggestions about where to go for information and help would be most
welcome.
Here's what I have observed and done, which might have some clues:
- I first noticed it because of the rattling of the hard drive and the
hard drive activity light on solid.
- Looking at iotop and top, I expected to see some process pegging the
CPU and the disk I/O, but nothing seemed to stand out. I may have seen a
Chromium thread doing a lot of I/O, but not for long.
- I unplugged the network ethernet cable and it stopped. Later that day,
I reconnected it and it started up again, but it seemed like it wasn't
until an hour or three later. Then I unplugged it again.
- At first I thought it was related to my media servers, Plex and Kodi,
because the only files that I found to be encrypted were videos, audio
files, and image files. Then I found 1 encrypted file that was
different: my procmail rules file. This lead me to notice that all of
the encrypted files had "other" write permissions (666, 777). These were
pretty much all old files from various sources. For example, photos from
up to 20 years ago from other people's cameras, etc.
- Because I suspected Kodi, I powered off the 3 android boxes I have in
the house that run Kodi to access my server (using MariaDB and smb). I
haven't yet turned on any of these boxes again.
- The attack left a text file in every directory where it encrypted
files, with the name
"5a067ee9_3a53aaff_1aedfa64___READ_THIS___5a067ee9_3a53aaff_1aedfa64.txt",
with owner/group "nobody/nogroup". I've quoted the ransom file text below.
- No files outside of my home directory have been touched. I believe
that only files writable by "other" were encrypted. After encryption,
the files have a timestamp of the time of encryption, and are still
owned by me. The encrypted files have names like
"0H1JsqXEw5.fse_5a067ee9_3a53aaff_1aedfa64", where the characters after
the dot (the extension, so to speak) are always the same.
- I have found and changed the permissions of every file and directory
(except for /tmp) writable by "other". When I connect the ethernet
network cable now, there seems to be no further encrypting by the
malware. I check this by the lack of disk activity, and using the find
command to search for files newer than the time I last connected to the
network, I run "updatedb" and "locate" for filenames containing
"READ_THIS" and "fse_". I disconnect the network overnight though, just
in case.
- I eventually realized that some files that appeared to be encrypted
had not been renamed. I don't know what to think about this, other than
maybe the malware program doesn't rename file until a directory is
completed, and I disconnected the network cable while it appeared to be
active.
- During all this, there was a power outage. After that, one Windows PC
that belonged to my mother has not been powered back on. I think I've
read this such malware can jump from Windows to Linux.
Some thoughts:
I read that files created by NFS or smb can be owned by nobody/nogroup.
The 2 running process owned by nobody are /usr/bin/memcached and /usr/
sbin/smbd. The remote kodi boxes access the server files using smb.
I don't know what it means that only files owned by me have been hit,
but only files with 777/666 permissions. Given that the new files are
created by nobody, it seems like they aren't able to actually log into
my account?
The ransomeware notification file:
***REDACTED***
Print the ransomware file, take it to the police, file a complaint, and
get a police report number.
Making changes to the infected disks will make it harder to figure out
what happened. It is best to remove the infected disks, clone them to
working disks, and investigate the clones. And, you may want a third
set of disks when it is time to rebuild the server.
Have you configured your Internet gateway (firewall pinholes, port
forwarding) to allow WAN incoming packets and to forward the packets to
the server or some other internal host?
Please boot live media in the server, open a root terminal, mount the
server file systems under /mnt/server/, run the following commands (I
have assumed your username is "rick"; please substitute the correct
name), and copy/paste the console session into your reply. Document any
changes that you have made since the attack:
# egrep '^/dev' /mnt/server/etc/fstab
# grep nobody /mnt/server//etc/passwd
# grep rick /mnt/server//etc/passwd
# grep nogroup /mnt/server/etc/group
# grep sudo /mnt/server/etc/group
# grep rick /mnt/server/etc/group
# egrep '^#?PasswordAuthentication' /mnt/server/etc/ssh/sshd_config
# find /mnt/server/ -name '*___READ_THIS___*' -print0 2>/dev/null |
xargs -r -0 ls -l
# find /mnt/server/ -name '*___READ_THIS___*' -print0 2>/dev/null |
xargs -r -0 dirname -z | xargs -r -0 ls -ld
Please provide `ls -l` listings for some example malware encrypted files
and for the directories that contain them. Document changes.
David
Reply to: