Bug#989280: apt-mirror-setup: setting protocol=https in expert mode triggers failures

[Cyril Brulebois <kibi@debian.org>]

Package: apt-mirror-setup
Version: 1:0.163
Severity: important
X-Debbugs-Cc: Sylvain Tgz <tarjaizaid@gmail.com>

Hi,

This bug report is a summary of a conversation started by Sylvain Tgz
(thanks!) on the debian-boot@ mailing list:
  https://lists.debian.org/debian-boot/2021/05/msg00197.html

I'm merging bits from two replies of mine below, cutting and reshuffling
some paragraphs…
  https://lists.debian.org/debian-boot/2021/05/msg00220.html
  https://lists.debian.org/debian-boot/2021/05/msg00221.html

> I try again without mirror/protocol=https and the issue reappears
> 
> To reproduce the issue :
> - Start Debian netinst
> - Use Graphical expert installation mode
> - At the step " configure the package manager" chose https.
> - Use default option
> Thie issue should be reproductible.
> 
> Could you try ?

Yes, I can definitely reproduce the issue, and I think I have figured
out what happens. I didn't think too hard about how to fix it yet,
though.


<live-debugging>

You're using netinst, so debootstrap uses the ISO to install the base
system. Also, unless one has forced mirror/protocol=https at this stage,
there's no chance for the user to tell debootstrap https is desired.
This means that this code path doesn't help:
  https://salsa.debian.org/installer-team/debootstrap/-/blob/master/scripts/debian-common#L30-41

and we end up without ca-certificates in /target (where the installed
system is being built).

The error message in apt comes from:

   // Credential setup
   std::string fileinfo = Owner->ConfigFind("CaInfo", "");
   if (fileinfo.empty())
   {
      // No CaInfo specified, use system trust store.
→     err = gnutls_certificate_set_x509_system_trust(tlsFd->credentials);
→     if (err == 0)
→        Owner->Warning("No system certificates available. Try installing ca-certificates.");
      else if (err < 0)
      {
         _error->Error("Could not load system TLS certificates: %s", gnutls_strerror(err));
         return ResultState::FATAL_ERROR;
      }

A quick strace shows the following file (missing in the ca-certificates
udeb, and therefore in my manual copy into /target) is desired:

    /etc/ssl/certs/ca-certificates.crt

And finally, concatenating all certificates into that single file seems
to make `debconf-apt-progress` happy, so maybe we would just have to
create the directory and ship that particular file there to avoid an
installation failure, and I would expect ca-certificates to just
re-regenerate that file upon installation/upgrade, so that might not
break anything (even if not really clean)?

</live-debugging>


Now, where to go from here… I think I would call it a bug for apt-setup
not to notice we're about to use https and not do anything about it…
Maybe it could install ca-certificates from the currently configured
source (if any) in that case. As far as I can see from this test run,
using the netinst image like you did, /target/etc/apt/sources.list still
lists the NETINST image as a cdrom: source, so that should be doable.
With other installation images (e.g. netbooting) that might not work…

Maybe something like this?

    if https is configured
      if ca-certificates is installed in target
         be happy
      else
         if it can be installed
           install it, be happy
         else
           duplicate /etc/ssl/certs from / into /target
           queue the installation of ca-certificates

The “duplicate” part doesn't feel right but it might be better than fail
miserably if ca-certificates cannot be installed properly. Of course
that could create a discrepancy if ca-certificates-udeb (deployed in the
installer image) and ca-certificates get out of sync, and I'm not sure
how ca-certificates will react to manually-deployed certificates… Also,
as detailed in the live-debugging section, we would only have to ship
ca-certificates.crt in under /target, but that's an implementation
detail (in current apt and libraries it uses) that might change over
time… Maybe duplicate /etc/ssl/certs entirely, and generate the .crt
file by concatenating all files there, to maximize our chances?

Regarding the “queue” part, I *think* there's a way to achieve that but
I'm not sure that's true or how it would be done. If there's no generic
way to achieve this, it could still be done in a finish-install script
(apt-setup already has finish-install.d/10apt-cdrom-setup).

It is very likely this bug affects buster as well, and we might want to
get the fix into buster.

Since this can only (I think, feel free to prove me wrong) be triggered
in expert mode, I would probably not block #987441 with it (or do that
so that we don't lose track, while not actually blocking 11.0 with it).


Cheers,
-- 
Cyril Brulebois (kibi@debian.org)            <https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant