On Thursday 18 June 2009 02:46:42 Goswin von Brederlow wrote: > Joseph Rawson <umeboshi3@gmail.com> writes: > > There is another application that will help with the dependencies. It's > > called germinate, and it will take a short list of packages and a list of > > repositories and build a bunch of different lists of packages and their > > dependencies. Germinate will also determine build dependencies for those > > packages and recursively build a list of builddeps and the builddeps' > > builddeps. > > > > I have thought of making an application that would get germinate and > > reprepro to work together to help build a decent partial mirror that had > > the correct set of packages, but the process was a bit time consuming. > > It's been a while > > Was it that bad? It only needs to run 4 times a day when the mirror > push comes in. > It wasn't the running that was time consuming, but the writing of all the code to seed germinate, then try and use the results for reprepro. I'm sorry if I wasn't clear on which part was consuming time. > > since I've worked on this, since my temporary solution to the problem was > > to buy a larger hard drive. Currently, I have a full mirror that I keep > > updated, and a repository of locally built packages next to it. I'm not > > really happy with this solution, as it uses too much disk space and I'm > > downloading packages that will never be used, but it's given me time to > > tackle more important problems. > > > > Before writing any code, I would recommend taking a look at both reprepro > > and germinate, as each of these applications is good at solving half of > > the problems you describe. I think that an ideal solution would be to > > write a frontend program that takes a list of packages and upstream > > repositories, feeds them into germinate, obtains the result from > > germinate, parse those results and build a reprepro configuration from > > that, then get reprepro to fetch the appropriate packages. > > Combining germinate and reprepro is the right thing to do. Or reprepro > and a new filter instead of germinate. But don't rewrite reprepro. I never intended to rewrite reprepro. It does it's job very well. It's not reprepro's job to resolve dependencies, nor should it be, as a dependency could lie in an entirely different repository. I do think that since each program has it's specific area of responsibility, that a program that glues them together would be appropriate, and help from reinventing wheels when it's not necessary. > > Given a little bit of care when writing the reprepro config this can > be completly done as part of the filtering. There is no need for a > seperate run that scanns all upstream repositories as long as you can > define a partial order between them, i.e. contrib needs things from > main but main never from contrib. That would also have the benefit > that you only need to process those packages files that have changed. > > > I would be happy to help with this, as I could use such an application, > > and I already have a meager bit of python code that parses the output of > > germinate (germinate uses a wiki-type markup in it's output files). I > > stopped working on the code since I bought a new hard drive, since I just > > used the extra space to solve the problem for me, but I can bring it back > > to life, as I would desire to use a more correct solution. > > Urgs, that sucks. It should take a Packages/Sources style input and > output the same format. > I don't like the output either, but I haven't taken much time to dig into the germinate code very much. > Maybe rewriting it using libapt would be better than wrapping germinate. Germinate uses libapt. It imports apt_pkg from the python-apt package, which is a python binding to libapt, AFAIK. It might be easier to just add '/usr/lib/germinate' to the sys.path and control the Germinator object directly, bypassing the way that the package lists are output from germinate. Germinate does have an advantage in that it can recursively add the builddeps for a package list, making a list for a partial, self-building mirror. BTW, the subject of this thread is "apt-get wrapper for maintaining Partial Mirrors". The solution I'm proposing is "a simple tool for maintaining Partial Mirrors" (which could possibly be wrapped by apt-get later). I think that just pursuing an "apt-get wrapper" leads to some complications that could be avoided by creating the "partial mirror tool" first, then looking at wrapping it later. One complication might be "how do handle apt-get remove", and another might be "how to handle sid libraries that disappear from official repository, yet local machines must have them". > > MfG > Goswin -- Thanks: Joseph Rawson
Attachment:
signature.asc
Description: This is a digitally signed message part.