Bug#774523: Mirror/dist cache sub-directories
Package: live-build
Version: 5.0~a1-1
Severity: wishlist
A possible improvement for the future, submitting now ahead of time.
(Can be ignored for the time being).
Taking a que from the current installer_debian-installer cache, which
uses URLs (with forward slashes replaced with underscores) in the
filenames, to separate out files from different locations, such as daily
builds of installer image files from normal, my new almost complete bug
#718225 solution caches files in a similar way, using the URL to build
sub directories that closely mirror the URL path.
There are advantages to this, making the cache more suitable for
building images of different distributions, and when building for a
derivative distribution, it ensures that files from parent and
derivative mirrors are separated. Also, should a user mix up the parent
and derivative config parameters, and then correct them without flushing
the cache, there's no problem, the correct files are retrieved from the
cache.
I actually don't particularly like the base of the URL being used in
this way because although it works, it looks really messy. Furthermore,
simply switching from say one debian mirror to another means perfectly
good cached files are ignored, which isn't desirable. So I'm considering
alternatives for how this could be improved in the future (LB v5.x).
One would be simply sub-directories of 'parent' and 'derivative' or
'parent{-|.}mirror' and 'derivative{-|.}mirror'. This obviously breaks
the latter advantage of the user specifying the URLs the wrong way
around and fixing without a cache flush though (not a big deal, but good
to allow if possible). It also requires ensuring that 'parent'
attributes are used throughout the code when not doing a derivative
build, if this is not the case already, but such consistency is
desirable anyhow.
An alternative could be to have a simple label to go alongside each of
parent and derivative mirrors, i.e. 'debian', 'ubuntu',
'progress-linux', etc., which can be used for sub-directory names (e.g.
cache/packages.bootstrap/debian).
How such labels are supplied is a tricky question however. It would be
over the top I think for new config params to be added. Hard coding a
grep against the url to determine which of progress-linux/ubuntu/debian
might work, but further works against opening up use of LB on more
derivative distributions (something I personally am supportive of
improving, I don't know about maintainer).
The best approach might be to completely revise the config format in the
planned switch to python in v5.x+. Instead of taking the user's config
as a set of parameters, read them from a file (as is done with the full
config files in config/ after processing auto/config and other params).
This would provide a more simple opportunity for the user to supply
labels. Furthermore, much more of the 'mode' customisations could
possibly be captured in such a config file, making the code neater,
simpler, and more open to other debian derivatives.
Reply to: