Bug#1120743: apt.systemd.daily update fails because it runs before network interfaces are ready
On Sat, Nov 15, 2025 at 10:20:07AM -0800, Petter H wrote:
> Package: apt
> Version: 3.0.3
>
> (Likely affects multiple versions)
>
> unattended-upgrades was not working (I let it try and fail for over a month
> while there were known updates available, it never worked). The problem was
> with /usr/lib/apt/apt.systemd.daily.
>
> 'apt.systemd.daily update' was invoked, by systemd, after system wake from
> suspend, before the network was available. Even though it was unsuccessful,
> it still touched the timestamp file indicating it had already ran that day.
> Subsequent attempts to run would simply be skipped, since apt.systemd.daily
> checks the timestamp file, and aborts if the stamp file indicates it has
> already ran that day.
>
> The update failing because the script was run before network interfaces were
> up, after a suspend/wake, was apparently a known issue, as a partial work
> around was already included in apt-daily.service.
>
> apt-daily.service includes the line:
> ExecStartPre=-/usr/lib/apt/apt-helper wait-online
>
> But, 'apt-helper wait-online' does nothing when using ifupdown. You can
> down the interface entirely, and 'apt-helper wait-online' will still,
> immediately, return with a zero exit code, the apt.systemd.daily script will
> attempt, unsuccessfully, to do the apt update, but the script will set the
> flag, as if it was successful, so no further attempts will run that same
> day.
>
> Per this Ubuntu bug report where it appears 'apt-helper wait-online' was
> introduced, there was no attempt to make it work for anything but systemd
> networkd and network manager.
>
> https://bugs.launchpad.net/ubuntu/+source/apt/+bug/1699850
This is wrong, we also support connman.
>
> Per that bug report, 'apt-helper wait-online' waits up to 60 minutes for the
> network to become available. The following hacky line, added to
> apt-daily.service, accomplishes the same, and works regardless of how
> network interfaces are managed.
>
> # try once per minute for 60 minutes to see if network ready, if yes,
> continue
> ExecStartPre=/bin/bash -c "for ((i=1; i<60; i++)); do nc -z -w 1
> deb.debian.org 80 2>/dev/null && break || sleep 60; done"
This is not an acceptable way of waiting for online connectiveness.
>
> This work around gets apt periodic's update to work again, so
> unattended-upgrades works again; unattended-upgrades has picked up updates
> each of the 3 days since putting this hacky work-around into place; while
> failing to work each of the 40 days prior to putting the work-around hack
> into place.
>
> It seems there are multiple bugs:
>
> 1. apt periodic update should not touch the "stamp" file if there was an
> error that prevented it from performing the update.
It works correctly, touching the stamp file for the run, but not the
succesful stamp.
>
> 2. 'apt-helper wait-online' should work for all supported network management
> tools, including ifupdown.
If ifupdown provides active network monitoring and a wait-online command
it can be added, otherwise you're on your own.
At some point when we have an apt daemon it may monitor netlink events
and then try to check for updates when it may be online, but it's a bit
off still.
>
> 3. if systemd had sane behavior, 'apt-helper wait-online', and my similar
> hack, would not be necessary, in the first place.
Unfortunately the network-online.target is not dynamic, yes.
--
debian developer - deb.li/jak | jak-linux.org - free software dev
ubuntu core developer i speak de, en
Reply to: