Hi all,
Since 2017 I have been using the script below (please ignore the
hackyness) to filter out messages from my apt upgrade logs (mailed via
unattended-upgrades) that are indicators of success or "normally"
happen and highlight messages that are indicators of failure or are
weird in some way that I should know about.
This script is a workaround for the fact that I don't want to waste
time reading "unpacking foo" over and over again. I could of course
just ignore all apt output but then I would miss upgrade failures,
warnings from postinsts, adequate and how-can-i-help. Even if I did
read all the output I could still miss those amongst the many "normal"
messages. I believe this also applies to many Debian users and probably
more Debian contributors.
For many years it was appropriate to do Debian updates interactively in
case of failure so warnings etc were often seen, but these days Debian
testing is almost always reliable enough to be upgraded unattended and
people doing that are likely missing some things that they could report
as bugs or fix on their systems.
Obviously the script I use is a suboptimal solution to this problem
since it means I maintain the "normalness" of each message in a central
location instead of distributing it over the packages that are
outputting each of the messages.
Does anyone agree that this is a problem worth solving?
Does it seem OK to ask maintainers to maintain "normalness" info?
Any thoughts on how to communicate "normalness" to apt/dpkg/etc?
Other thoughts?
#!/bin/sh
cat "$1" |
sed -n -e '/^Packages that were upgraded:$/{ :a' -e 'n; /^ /ba; }; p;' | \
sed -n -e '/^\[master [a-f0-9]\+\] committing changes in .etc /{ :a' -e 'n; /^ /ba; }; p;' | \
sed -n -e '/^Restarting services\.\.\.$/{ :a' -e 'n; /^ /ba; }; p;' | \
sed -n -e '/^Services being skipped:$/{ :a' -e 'n; /^ /ba; }; p;' | \
sed -n -e '/^Service restarts being deferred:$/{ :a' -e 'n; /^ /ba; }; p;' | \
sed -n -e '/^The following packages will be upgraded:$/{ :a' -e 'n; /^ /ba; }; p;' | \
sed -n -e '/^The following additional packages will be installed:$/{ :a' -e 'n; /^ /ba; }; p;' | \
sed -n -e '/^The following NEW packages will be installed:$/{ :a' -e 'n; /^ /ba; }; p;' | \
sed -n -e '/^The following packages will be REINSTALLED:$/{ :a' -e 'n; /^ /ba; }; p;' | \
sed -n -e '/^User sessions running outdated binaries:$/{ :a' -e 'n; /^ /ba; }; p;' | \
sed -e '/^Building format(s) --all\.$/,+1d' | \
sed -e '/^From /,/^$/d' | \
sed -e '/^-------- Uninstall Beginning --------$/,/^DKMS: uninstall completed\.$/d' | \
sed -e '/^Pending kernel upgrade!$/,/^so you should consider rebooting\. \[Return\]$/d' | \
sed -e '/^Restarting services possibly affected by the upgrade:$/,/^Services restarted successfully\.$/d' | \
GREP_COLORS='mt=:sl=01;31' \
grep --color -vPC10 \
"$(printf '%s' '
^$|
^Preconfiguring packages \.\.\.|
^\(Reading database \.\.\.|
^Reading package lists\.\.\. Done$|
^Building dependency tree *$|
^Reading state information\.\.\. Done$|
^Calculating upgrade\.\.\. Done$|
^Preparing to unpack \.\.\./[^ ]+\.deb ...$|
^Unpacking [^ ]+ \([^)]+\) \.\.\.$|
^Unpacking [^ ]+ \([^)]+\) over \([^)]+\) \.\.\.$|
^Replacing files in old package [^ ]+ \([^)]+\) \.\.\.$|
^Processing triggers for [^ ]+ \([^)]+\) \.\.\.$|
^Setting up [^ ]+ \([^)]+\) \.\.\.$|
^Selecting previously unselected package .*\.$|
^Removing [^ ]+ \([^)]+\) \.\.\.$|
^Replaced by files in installed package [^ ]+ \([^)]+\) \.\.\.$|
^De-configuring [^ ]+ \([^)]+\)(, to allow removal of [^ ]+ \([^)]+\))? \.\.\.$|
^[^ ]+ is already the newest version \([^)]+\)\.$|
^\d+ upgraded, \d+ newly installed, \d+ to remove and \d+ not upgraded\.$|
^\d+ packages upgraded, \d+ newly installed, \d+ reinstalled, \d+ to remove and \d+ not upgraded\.$|
^Need to get [\d,]+ [kMG]?B/\d+ [kMG]?B of archives\.$|
^Need to get [\d,]+ [kMG]?B of archives\. After unpacking [\d,]+ [kMG]?B will be used\.$|
^After this operation, [\d,]+ [kMG]?B of additional disk space will be used\.$|
^Get: *\d+ https?://[^/]*/debian.*$|
^Fetched [\d,]+ [kMG]?B in \d+min \d+s \([\d,]+ [kMG]?B/s\).*$|
^Fetched [\d,]+ [kMG]?B in \d+s \([\d,]+ [kMG]?B/s\).*$|
^Extracting templates from packages: \d+%$|
^Leaving .diversion of .+ to .+ by [^ ]+$|
^Removing .diversion of [^ ]+ to [^ ]+ by [^ ]+$|
^Adding .diversion of [^ ]+ to [^ ]+ by [^ ]+$|
^Performing actions\.\.\.$|
^Press Return to continue, .q. followed by Return to quit\.$|
^Installing new version of config file .* \.\.\.$|
^Removing obsolete conffile .* \.\.\.$|
^Replacing config file [^ ]+ with new version$|
^Creating config file [^ ]+ with new version$|
^Purging configuration files for [^ ]+ \([^ )]+\) \.\.\.$|
^update-alternatives: using [^ ]+ to provide [^ ]+ \([^ )]+\) in auto mode$|
^update-alternatives: renaming [^ ]+ link from [^ ]+ to [^ ]+$|
^apt-listchanges: Do you want to continue\? \[Y/n\] .*$|
^Do you want to continue\? \[Y/n\] .*$|
^apt-listchanges: Reading changelogs\.\.\.$|
^Reading changelogs\.\.\. Done$|
^apt-listchanges: Mailing [^:]+: apt-listchanges: news for .*$|
^apt-listchanges: Mailing [^:]+: apt-listchanges: changelogs for .*$|
^how-can-i-help: Unable to resolve your \$HOME directory - cannot continue\.$|
^====== How can you help\? \(doc: https://wiki\.debian\.org/how-can-i-help \) ======$|
^----- Show old opportunities as well as new ones: how-can-i-help --old -----$|
^Scanning processes\.\.\..*$|
^Scanning candidates\.\.\..*$|
^Scanning processor microcode\.\.\..*$|
^Scanning linux images\.\.\..*$|
^Running kernel seems to be up-to-date\.$|
^The processor microcode seems to be up-to-date\.$|
^No services need to be restarted\.$|
^No containers need to be restarted\.$|
^No user sessions are running outdated binaries\.$|
^Log started: [0-9]{4}-[0-9]{2}-[0-9]{2} [0-9]{2}:[0-9]{2}:[0-9]{2}$|
^Log ended: [0-9]{4}-[0-9]{2}-[0-9]{2} [0-9]{2}:[0-9]{2}:[0-9]{2}$|
^Unattended upgrade returned: True$|
^Unattended upgrade result: All upgrades installed *$|
^Calling .*apt-get.*-qq.*changelog.*=.* to retrieve changelog$|
^Allowed origins are: .*$|
^Packages that were upgraded:$|
^Package installation log:$|
^Packages (that )?were successfully auto-removed(: )?$|
^Packages that are kept back: $|
^All upgrades installed$|
^Writing dpkg log to .?/var/log/unattended-upgrades/unattended-upgrades-dpkg\.log.?$|
^Packages that will be upgraded: |
^Allowed origins are: \[.origin=Debian., .origin=Debian.\]$|
^Starting unattended upgrades script$|
^Initial whitelist(|ed packages): $|
^Initial blacklist( |ed packages): $|
^Unattended-upgrades log:$|
^update-initramfs: deferring update \(trigger activated\)$|
^update-initramfs: Generating /boot/initrd\.img-.*$|
^update-initramfs: Deleting /boot/initrd\.img-.*$|
^/etc/kernel/post(inst|rm).d/initramfs-tools:$|
^/etc/kernel/post(inst|rm).d/zz-update-grub:$|
^Generating grub configuration file \.\.\.$|
^Found (background|linux|initrd) image: .*$|
^No grub-efi\* package found, using 4/3 as default grub background ratio$|
^Installing for [^ ]+ platform\.|$
^Installation finished\. No error reported\.$|
^done$|
^Searching for services which depend on erlang and should be (stopped|started)\.\.\. ?none found\.$|
^Killing epmd\.\.\. *it is not running\.$|
^Updating PPD files for [^ ]+ \.\.\.$|
^No Gutenprint PPD files to update\.$|
^Creating/updating [^ ]+ user account\.\.\.$|
^Adding user [^ ]+ to group [^ ]+$|
^usermod: no changes$|
^addgroup: The group [^ ]+ already exists as a system group\. Exiting\.$|
^Updating symlinks in /usr/lib/ccache \.\.\.$|
^locales-all installed, skipping locales generation$|
^Updating GnuTLS DH parameter file$|
^Checking for services that may need to be restarted\.\.\.$|
^Checking init scripts\.\.\.$|
^[^ ]+\.(service|socket|target) is a disabled or a static unit( not running)?, not starting it\.$|
^Created symlink /etc/systemd/(user|system)/[^ ]+\.socket → (/usr)?/lib/systemd/(user|system)/[^ ]+\.socket\.$|
^Created symlink /etc/systemd/(user|system)/[^ ]+\.service → (/usr)?/lib/systemd/(user|system)/[^ ]+\.service\.$|
^Created symlink /etc/systemd/(user|system)/[^ ]+\.timer → (/usr)?/lib/systemd/(user|system)/[^ ]+\.timer\.$|
^Synchronizing state of [^ ]+\.service with SysV service script with /lib/systemd/systemd-sysv-install\.$
^Executing: /lib/systemd/systemd-sysv-install enable |
^Skipping starting [^ ]+\.service because [^ ]+ is running$|
^tl-paper: setting paper size for dvips to a4: /var/lib/texmf/dvips/config/config-paper\.ps$|
^tl-paper: setting paper size for dvipdfmx to a4: /var/lib/texmf/dvipdfmx/dvipdfmx-paper\.cfg$|
^tl-paper: setting paper size for xdvi to a4: /var/lib/texmf/xdvi/XDvi-paper$|
^tl-paper: setting paper size for pdftex to a4: /var/lib/texmf/tex/generic/config/pdftexconfig\.tex$|
^Running (updmap-sys|mktexlsr)\. This may take some time\.\.\. done\.$|
^Running mktexlsr /var/lib/texmf \.\.\. done\.$|
^Building format\(s\) --all\.$|
^ This may take some time\.\.\. done\.$|
^Running mktexlsr\. This may take some time\. \.\.\. done\.$|
^texlive-base is not ready, delaying updmap-sys call$|
^update-language: texlive-base not installed and configured, doing nothing!$|
^texlive-base is not ready, skipping fmtutil --all call$|
^System not running in EFI mode, not installing to EFI system partition.*$|
^EFI variables are not supported on this system\.$|
^Current default time zone: .*$|
^Local time is now:.*$|
^Universal Time is now:.*$|
^Run .dpkg-reconfigure tzdata. if you wish to change it\.$|
^AppStream cache update completed successfully\.$|
^aspell-autobuildhash: processing: [a-z]+ \[[-_0-9A-Za-z]+]\.$|
^ispell-autobuildhash: Processing [^ ]+ dict\.$|
^/etc/kernel/prerm.d/dkms:$|
^dkms: removing: .*$|
^intel-microcode: microcode will be updated at next boot$|
^Setcap worked! gst-ptp-helper is not suid!$|
^Installing Kismet with suid-root capture helper \.\.\.$|
^Updating database of manual pages \.\.\.$|
^Auto packing the repository in background for optimum performance\.$|
^See "git help gc" for manual housekeeping\.$|
^Regenerating fonts cache\.\.\. done\.$|
^patching file /usr/.*$|
^Reversed \(or previously applied\) patch detected! Skipping patch\.$|
^[0-9]+ out of [0-9]+ hunks? ignored$|
^running pypy (pre-|post-)?rtupdate hooks for [0-9.]+$|
^Checking smb\.conf( with testparm)?$|
^Load smb config files from /etc/samba/smb\.conf$|
^Loaded services file OK\.$|
^Server role: ROLE_STANDALONE$|
^Done$|
^Restarting virtual private network daemon\.:\.$|
^\*\* initializing cache\. This may take a while \*\*$|
^Running sa-compile \(may take a long time\)$|
^Reloading AppArmor profiles *$|
^\* Installing [0-9]+ assembl(y|ies) from [^ ]+ into Mono$|
^update-perl-sax-parsers: (Unr|R)egistering Perl SAX parser [^ ]+ with priority [0-9.]+\.\.\.$|
^update-perl-sax-parsers: Updating overall Perl SAX parser modules info file\.\.\.$|
^Sucessfully set capabilities for [^ ]+$|
' sed '/^#/d' | tr -d '\t\n')"
--
bye,
pabs
https://wiki.debian.org/PaulWise
Attachment:
signature.asc
Description: This is a digitally signed message part