Re: broken system after srm -r -d /tmp/.* (user login and several services not working)
On 2008-07-02 11:33 +0200, Jonas Meurer wrote:
> I've broken my debian/unstable system by executing as root the command
> # srm -r -d /tmp/.*
Ouch. That seems a good command to run before you sell your hard
disk. ;-)
> I aborted the command after something between 20 and 40 seconds, but
> since then, my system behaves strange:
Small wonder.
> If I try to login as normal user on the console, I get the error
> "Unable to cd to '/home/user'" and the login aborts.
>
> Login as root works without any issues.
>
> When I try to su to a normal user from root, I get: "Cannot execute
> /bin/bash: Permission denied." The permissions for /bin/bash are ok:
> -rwxr-xr-x 1 root root 797784 2008-05-12 19:00 /bin/bash
It's possible that the PAM modules or their configuration became
corrupt.
> I even tried to add a new user with 'adduser username', which seemed
> to work without problems, but if I try to login/su to that new user,
> the same errors occur. I already checked for /etc/passwd, /etc/group and
> /etc/pam.d being available.
>
> Additionally several services fail to start, please see below for exact
> error messages.
>
> Next, I rebooted to see whether the system still was able to boot at
> all. And indeed, the system booted without issues, except that the
> services mentioned below didn't start, and the normal user login still
> didn't work as well.
That's really surprising, I would not have expected the system to come
up at all.
> So I started to try to recover the system (I don't have system backups,
> I only backup /home):
That's bad, you should at least backup /etc. But you know already...
> First, I did a 'dpkg-reconfigure' for all installed packages, but as that
> didn't change anything, I reinstalled all installed packages:
> # packages=$(dpkg -l | grep "^i" |awk '{print $2}')
> # apt-get -o Dpkg::Options::="--force-confmiss" --reinstall install $packages
Good idea.
> That one succeeded for all packages except the ones that contain the
> services which don't start anymore, and obviosly the reinstall didn't
> fix that.
>
> So now, i've a system which has all packages reinstalled with the dpkg
> option to restore removed configfiles, but still user login and serveral
> services fail.
>
> Do you have any suggestions how to go on with debugging the issue? Maybe
> you even discovered similar issues in the past?
>
> It would be really painful to setup a clean new system and configure it
> the way I like, as this system is already several years old and has
> serveral individual configurations.
>
> Now the exact error messages of the services that do fail to start:
>
> # /etc/init.d/exim4 start
> Starting MTA:2008-07-01 20:49:06 Exim configuration error in line 662 of /var/lib/exim4/config.autogenerated.tmp:
> user mail was not found
Could you post the contents of your /etc/passwd? Even if that file
exists, it may not contain all users.
> exim: could not open panic log - aborting: see message(s) above
> Invalid new configfile /var/lib/exim4/config.autogenerated.tmp, not installing
> /var/lib/exim4/config.autogenerated.tmp to /var/lib/exim4/config.autogenerated
>
> # /etc/init.d/gdm start
> Starting GNOME Display Manager: gdmgdm[16973]: WARNING: gdm_config_parse: Authdir /var/lib/gdm does not exist. Aborting.
> gdm_config_parse: Authdir /var/lib/gdm does not exist. Aborting.
Probably you need to recreate the missing directory.
> # /etc/init.d/mysql start
> Starting MySQL database server: mysqld . . . . . . . . . . . . . . failed!
>
> # tail -n5 /var/log/syslog
> Jul 1 22:55:25 jonas /etc/init.d/mysql[17210]: 0 processes alive and '/usr/bin/mysqladmin --defaults-file=/etc/mysql/debian.cnf ping' resulted in
> Jul 1 22:55:25 jonas /etc/init.d/mysql[17210]: /usr/bin/mysqladmin: connect to server at 'localhost' failed
> Jul 1 22:55:25 jonas /etc/init.d/mysql[17210]: error: 'Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)'
> Jul 1 22:55:25 jonas /etc/init.d/mysql[17210]: Check that mysqld is running and that the socket: '/var/run/mysqld/mysqld.sock' exists!
> Jul 1 22:55:25 jonas /etc/init.d/mysql[17210]:
Does /var/run exist at all? Its absence could explain the failure to
start services that want to put files there. OTOH, it is included in
base-files, so if you reinstalled that package, it should be there.
But subdirectories like exim4 or mysqld could still be missing.
> So I rebooted to see whether everything was ok, and indeed the system
> booted without issues, only a few services failed to start. Among others
> these are avahi, hal, exim4, privoxy, mysql and gdm. But I already
> mentioned that above.
>
> If you have any suggestions, please tell me. I would be really glad to
> save my system.
One idea would be to install the cruft package and look for missing
files. Also, examine /etc for recent files: one file or the other might
be truncated by the interrupted srm command.
And please post a listing of /etc/pam.d.
Sven
Reply to: