[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: apt-get wrapper for maintaining Partial Mirrors



On Friday 19 June 2009 12:57:25 Goswin von Brederlow wrote:
> Joseph Rawson <umeboshi3@gmail.com> writes:
> > On Friday 19 June 2009 00:27:06 Goswin von Brederlow wrote:
> >> Joseph Rawson <umeboshi3@gmail.com> writes:
> >> If so then you can configure a post invoke hook in apt that will copy
> >> the dpkg status file of the host to the server [as status.$(hostname)]
> >> and then use those on the server to generate the filter for
> >> reprepro. I think I still have a script for that somewhere but it is
> >> easy enough to rewrite.
> >
> > That's good for binaries, but I don't know about the source.  It wasn't
> > long ago that I noticed a problem with reprepro not obtaining the
> > corresponding source packages when you use a filter list taken
> > from  "dpkg --get-selections".  I remember that the source for jigdo
> > wasn't in my partial mirror, because there were no binaries named
> > "jigdo", rather "jigdo-file" and "jigdo-lite".  Since there were no
> > sources with that name, the jigdo source was never mirrored on my partial
> > mirror.  I don't know if that behavior has been fixed now, since there is
> > now a binary named jigdo, instead of jigdo-lite.
>
> My filter first converted the packages listed in the status file(s) to
> source package names (packages with different name have a "Source:"
> entry) and then output those for sources.
>
> > Also, it's more difficult for the local repository to determine the
> > difference between the automatically selected and manually selected
> > packages in this type of setup, since you would be sending a longer list
> > of "manually selected packages", instead of distinguishing which ones are
> > actually selected.  I guess that it doesn't matter much, as a package
> > would only be removed from the repository once it's not listed on any of
> > the lists.  There were times when I didn't want certain packages to be
> > removed from the repository, regardless of whether they were installed or
> > not, so I used to run xxdiff on the packages files, so the newer ones
> > were added.
>
> Same problem here. Esspecially build-depends. There where a lot of
> packages I only needed inside my build chroots and only for the time
> of the build. So they never showed up on the mirror. Then I just
> resized the mirror partition and mirrored all debs.
>
That was my ultimate solution to the problem.  I bought one of the new 
terabyte usb external drives and just mirrored the whole repository.  I had 
been satisfied to just call the problem solved at that point, but this thread 
resparked my interest in obtaining a better solution.  Before I bought the 
hard drive, I was seriously looking into getting germinate and reprepro 
working together, but once I bought the drive, I just set it all aside.  
Still, this external drive isn't portable, and my small portable drive is 
only 80G (which is more than enough for a partial mirror of source, i386, and 
amd64), so I do still need to solve the problem.  Besides, a month after I 
bought the drive, I discovered that I have a monthly cap on my transfers so 
it would be better, all around, to stop mirroring the complete repository.

> > In my way of thinking, I'm not looking to merge upstream repositories
> > together in one repository.  Besides, there are already tools, such as
> > apt-move that would be better for this job.  Long ago, apt-move was the
> > primary tool that I used to keep a local repository, and it worked pretty
> > well, as long as all the machines that were using it were on the same
> > release.
> >
> > I have found that reprepro is the absolute best tool for maintaining a
> > debian mirror.  The only problem I have with it is when I want to
> > maintain a partial mirror, and I don't want a merged repository, is that
> > I have to spread the packages lists to different places, and when you
> > start adding machines, you start adding more lists to the configuration,
> > when it would probably be better to maintain a set of "master" lists that
> > are generated from the many lists that come from the machines.
>
> Or have a proxy that adds packages that are requested.
When I woke up this morning, I was thinking that it might be interesting to 
have an apt method that talks directly to reprepro.  It's just a vague idea 
now, but I'll give it some more thought later.

>
> MfG
>         Goswin



-- 
Thanks:
Joseph Rawson

Attachment: signature.asc
Description: This is a digitally signed message part.


Reply to: