On Tuesday 09 June 2009 13:14:53 sanket agarwal wrote: > Hi all, > > We all know that there are various distro's that build around Debian. > I had an idea in mind whereby the task of making mirrors for personal > distributions can be automated. This can be stated as: if a person > wants to keep a customised set of packages for usage with the > distribution, the tool should be able to develop dependencies, fetch > packages, generate appropriate documentation and then create the > corresponding directory structure in the target mirror! The task can > be extended to include packages which are currently not under one of > the standard mirrors! > > I think the tool can have immense utility in helping people automate > the task of mantaining the repositories. Suggestions, positive and > negative are invited. > > I have not included the impl details as I would first like to evaluate > the idea at a feasibility and utility level. I have been working on this idea myself for quite a while, but I haven't messed with the problem recently. I was using reprepro to maintain partial mirrors, but it required using the output from "dpkg --get-selections" from almost every machine that I needed to mirror packages for. The reprepro program is excellent for making partial mirrors, but it has a drawback in that it doesn't help resolve dependencies. This means that you can't just make a short list of packages and easily build a partial mirror that contains those packages and their dependencies, rather you have to install a machine with those packages and use the list of packages from that machine with reprepro to get a decent mirror. There is another application that will help with the dependencies. It's called germinate, and it will take a short list of packages and a list of repositories and build a bunch of different lists of packages and their dependencies. Germinate will also determine build dependencies for those packages and recursively build a list of builddeps and the builddeps' builddeps. I have thought of making an application that would get germinate and reprepro to work together to help build a decent partial mirror that had the correct set of packages, but the process was a bit time consuming. It's been a while since I've worked on this, since my temporary solution to the problem was to buy a larger hard drive. Currently, I have a full mirror that I keep updated, and a repository of locally built packages next to it. I'm not really happy with this solution, as it uses too much disk space and I'm downloading packages that will never be used, but it's given me time to tackle more important problems. Before writing any code, I would recommend taking a look at both reprepro and germinate, as each of these applications is good at solving half of the problems you describe. I think that an ideal solution would be to write a frontend program that takes a list of packages and upstream repositories, feeds them into germinate, obtains the result from germinate, parse those results and build a reprepro configuration from that, then get reprepro to fetch the appropriate packages. I would be happy to help with this, as I could use such an application, and I already have a meager bit of python code that parses the output of germinate (germinate uses a wiki-type markup in it's output files). I stopped working on the code since I bought a new hard drive, since I just used the extra space to solve the problem for me, but I can bring it back to life, as I would desire to use a more correct solution. -- Thanks: Joseph Rawson
Attachment:
signature.asc
Description: This is a digitally signed message part.