Re: Some services cannot start at boot time because /run is not initialized
- To: debian-user@lists.debian.org
- Subject: Re: Some services cannot start at boot time because /run is not initialized
- From: Dmitry Katsubo <dma_k@mail.ru>
- Date: Thu, 22 Apr 2021 14:26:29 +0200
- Message-id: <[🔎] 4bf8a1ef-d39b-f46d-4c27-8d6a2f4d9a30@mail.ru>
- In-reply-to: <837ef752-ebb6-c0aa-4a37-d6cf0257e7ae@mail.ru>
- References: <ceb9fddf-6258-e9d0-a7ed-b96b3a2e971c@mail.ru> <E1jY32r-0002C9-Ge@enotuniq.net> <2dc3e7ec-88c0-4bdc-9d7e-c82d84ae4c3d@darac.org.uk> <317c1f81-0a19-63cb-7792-c7603290ed40@mail.ru> <837ef752-ebb6-c0aa-4a37-d6cf0257e7ae@mail.ru>
Dear Debian community,
I have noticed that all failed services were missing some directories under /run directory. I checked the service which is supposed to create them:
* systemd-tmpfiles-setup.service - Create Volatile Files and Directories
Loaded: loaded (/lib/systemd/system/systemd-tmpfiles-setup.service; static; vendor preset: enabled)
Active: inactive (dead)
Docs: man:tmpfiles.d(5)
man:systemd-tmpfiles(8)
systemd[1]: sysinit.target: Found ordering cycle on systemd-tmpfiles-setup.service/start
systemd[1]: sysinit.target: Found dependency on local-fs.target/start
systemd[1]: sysinit.target: Found dependency on zram-setup@zram1.service/start
systemd[1]: sysinit.target: Found dependency on sysinit.target/start
systemd[1]: sysinit.target: Job systemd-tmpfiles-setup.service/start deleted to break ordering cycle starting with sysinit.target/start
and it looks like there is problem in zram-setup@zram1.service which I configured like that:
[Unit]
Requires=dev-%i.device
After=dev-%i.device
Before=local-fs.target
[Install]
WantedBy=local-fs.target
# systemctl show -p Requires,Wants,Requisite,BindsTo,PartOf,Before,After local-fs.target
Requires=home.mount -.mount var.mount
Requisite=
Wants=systemd-fsck-root.service zram-setup@zram0.service zram-setup@zram1.service systemd-remount-fs.service
BindsTo=
PartOf=
Before=binfmt-support.service sysinit.target systemd-machine-id-commit.service systemd-tmpfiles-setup.service networking.service systemd-tmpfiles-clean.service console-setup.service
After=run-user-1000.mount zram-setup@zram0.service root.mount -.mount tmp.mount zram-setup@zram1.service systemd-fsck-root.service systemd-remount-fs.service home.mount var.mount local-fs-pre.target
Even though I don't see any conflict with dependencies, it is confusing systemd.
Any idea concerning how to fix that is welcomed.
On 2020-06-29 01:37, Dmitry Katsubo wrote:
> Dear Debian community,
>
> I hit the similar problem but this time with /run folder. Few services have
> failed to start:
>
> # systemctl status php7.0-fpm.service
> Jun 24 23:09:48 debian php-fpm7.0[893]: [24-Jun-2020 23:09:48] ERROR: unable to bind listening socket for address '/run/php/php7.0-fpm.sock': No such file or directory (2)
> Jun 24 23:09:48 debian php-fpm7.0[893]: [24-Jun-2020 23:09:48] ERROR: FPM initialization failed
> Jun 24 23:09:48 debian systemd[1]: php7.0-fpm.service: Main process exited, code=exited, status=78/CONFIG
> Jun 24 23:09:48 debian systemd[1]: php7.0-fpm.service: Failed with result 'exit-code'.
> Jun 24 23:09:48 debian systemd[1]: Failed to start The PHP 7.0 FastCGI Process Manager.
>
> # systemctl status motioneye.service
> Jun 24 23:09:47 debian systemd[1]: Started motionEye Server.
> Jun 24 23:09:48 debian meyectl[895]: INFO: hello! this is motionEye server 0.41
> Jun 24 23:09:48 debian meyectl[895]: CRITICAL: pid directory "/run/motioneye" does not exist or is not writable
> Jun 24 23:09:48 debian systemd[1]: motioneye.service: Main process exited, code=exited, status=255/EXCEPTION
> Jun 24 23:09:48 debian systemd[1]: motioneye.service: Failed with result 'exit-code'.
>
> # cat /etc/tmpfiles.d/php.conf
> d /run/php/sessions 1733 root root
>
> # cat /etc/tmpfiles.d/motioneye.conf
> d /run/motioneye 0750 motion motion
>
> Just after the boot I have inspected /run folder. It was created/mounted
> correctly and there have been a lot of files/directories there. I suspect that
> all services that have created the necessary directory under /run were able to
> start normally. Few of them which relied on existence of specific directory,
> have failed to started. After I have replayed the corresponding instructions for
> tmpfiles.d, the services have started normally.
>
> I have a feeling that systemd-tmpfiles was executed before /run was mounted.
>
> Needless to note that the problem is not persistent: sometimes OS boots without
> a single failed service.
>
> How can I debug the problem?
>
> Thank you!
>
> On 2020-05-18 02:39, Dmitry Katsubo wrote:
>> On 2020-05-11 20:11, Darac Marjal wrote:
>>> On 11/05/2020 08:40, Reco wrote:
>>>> Hi.
>>>>
>>>> On Mon, May 11, 2020 at 09:33:59AM +0200, Dmitry Katsubo wrote:
>>>>
>>>>> root@debian:~ # systemctl status binfmt-support
>>>>> * binfmt-support.service - Enable support for additional executable binary formats
>>>>> Loaded: loaded (/lib/systemd/system/binfmt-support.service; enabled; vendor preset: enabled)
>>>>> Active: failed (Result: exit-code) since Sun 2020-05-10 21:54:27 CEST; 10h ago
>>>>> Docs: man:update-binfmts(8)
>>>>> Process: 353 ExecStart=/usr/sbin/update-binfmts --enable (code=exited, status=2)
>>>>> Main PID: 353 (code=exited, status=2)
>>>>>
>>>>> May 10 21:54:27 debian update-binfmts[353]: update-binfmts: unable to open /var/lib/binfmts: No such file or directory
>>>>>
>>>>> Any help is appreciated.
>>>> This should help your problem:
>>>>
>>>> mkdir /etc/systemd/system/binfmt-support.service.d
>>>>
>>>> cat > /etc/systemd/system/binfmt-support.service.d/override.conf << EOF
>>>> [Unit]
>>>> RequiresMountsFor=/var
>>>> EOF
>>>
>>> As another alternative, one can run "systemctl edit
>>> binfmt-support.service", which will create the intervening folders and
>>> files for you, and reload the daemon if the editor exits with success.
>>
>> Thanks for suggestion! I have tried the advise and it actually worked
>> (I will keep an eye on that because one reboot may not be representative).
>> I wonder nevertheless what is the problem with this specific unit? It has
>> dependency on local-fs.target which in turn should mount /var. So what
>> exactly went wrong?
--
With best regards,
Dmitry
Reply to: