Re: apt-get wrapper for maintaining Partial Mirrors
Joseph Rawson <umeboshi3@gmail.com> writes:
> There is another application that will help with the dependencies. It's
> called germinate, and it will take a short list of packages and a list of
> repositories and build a bunch of different lists of packages and their
> dependencies. Germinate will also determine build dependencies for those
> packages and recursively build a list of builddeps and the builddeps'
> builddeps.
>
> I have thought of making an application that would get germinate and reprepro
> to work together to help build a decent partial mirror that had the correct
> set of packages, but the process was a bit time consuming. It's been a while
Was it that bad? It only needs to run 4 times a day when the mirror
push comes in.
> since I've worked on this, since my temporary solution to the problem was to
> buy a larger hard drive. Currently, I have a full mirror that I keep
> updated, and a repository of locally built packages next to it. I'm not
> really happy with this solution, as it uses too much disk space and I'm
> downloading packages that will never be used, but it's given me time to
> tackle more important problems.
>
> Before writing any code, I would recommend taking a look at both reprepro and
> germinate, as each of these applications is good at solving half of the
> problems you describe. I think that an ideal solution would be to write a
> frontend program that takes a list of packages and upstream repositories,
> feeds them into germinate, obtains the result from germinate, parse those
> results and build a reprepro configuration from that, then get reprepro to
> fetch the appropriate packages.
Combining germinate and reprepro is the right thing to do. Or reprepro
and a new filter instead of germinate. But don't rewrite reprepro.
Given a little bit of care when writing the reprepro config this can
be completly done as part of the filtering. There is no need for a
seperate run that scanns all upstream repositories as long as you can
define a partial order between them, i.e. contrib needs things from
main but main never from contrib. That would also have the benefit
that you only need to process those packages files that have changed.
> I would be happy to help with this, as I could use such an application, and I
> already have a meager bit of python code that parses the output of germinate
> (germinate uses a wiki-type markup in it's output files). I stopped working
> on the code since I bought a new hard drive, since I just used the extra
> space to solve the problem for me, but I can bring it back to life, as I
> would desire to use a more correct solution.
Urgs, that sucks. It should take a Packages/Sources style input and
output the same format.
Maybe rewriting it using libapt would be better than wrapping germinate.
MfG
Goswin
Reply to: