[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: apt-get wrapper for maintaining Partial Mirrors



On Sunday 21 June 2009 03:33:33 Goswin von Brederlow wrote:
<snip>
> > The Release could be signed using an rsign method with the machine(s)
> > that manage the repository, or it could be done locally on the server
> > using gpg-agent, or an unencrypted private key, depending on how the
> > administrator prefers to manage it.
>
> The simplest implementation would be a tiny proxy applet that, when a
> deb file is requested, checks if the file is in the local
> archive. If it is then send it. If not then request file from
> upstream and pipe it to apt (no latency) and a tempfile. When the
> download has finished then reprepro --include suite deb. Doing the
> same for source is a little more tricky as you needs the dsc and
> related files as a group.
>
I don't understand the tempfile part.  Otherwise, that's a better idea, since 
my idea depended on running reprepro update, then sending the appropriate 
debs.
> >> Optional the apt proxy could prefetch package versions but for me that
> >> wouldn't be a high priority.
> >>
> >> Nice would be that it fetches sources along with binaries. When I find
> >> a bug in some software while traveling I would hate to not have the
> >> source available to fix it. But then it also needs to fetch
> >> Build-depends and their depends. So that would complicate matters a
> >> lot.
> >
> > I mentioned that part above.
> >
> >> MfG
> >>         Goswin
> >
> > Overall, I think that reprepro does a good job of maintaining a local
> > repository, and we shouldn't reimplement what it does.  Reprepro also
> > seems flexible enough to implement most of the backend with simple
> > commands and options.  I've never tried to implement a new apt-method
> > before, so I think that would take a bit more research from me.
>
> I totally agree that reprepro as the cache/storage backend would be
> great use of existing software.
>
This is where I'm starting the code.  Since regardless of how the partial 
mirror(s) will be managed, we agree that using reprepro as the backend is the 
best choice,  I decided to start making a "frontend" or more 
appropriately "middle-layer" for this.  Making this part simple enough to use  
with the most likely used configuration, while keeping the option to be 
almost as flexible as reprepro is has been a quite bit of work and thought.

I have been working from the assumption that the local repository won't be a 
merged repository, but will be a set of partial mirrors.  By this I mean 
that "debian.org" doesn't have to be merged with "backports.org", 
but "sid/debian.org" may be in the same repository as "lenny/debian.org" 
(although even this could be separate, even if not recommended).  What I'm 
saying is that I'm trying to allow either separate or merged repositories to 
be used where they make the most sense.

> The problem I have with it being an apt method is that the apt method
> runs on a different host than the reprepro. That would require ssh
> logins from all participating clients or something to alter the
> reprepro filter.

I didn't stop to think about authentication, but I agree that it adds another 
level of work.  I took a bit of time to try and read up on how apt transport 
methods work, but I didn't get very far.  The only two transport methods that 
are available now are https and debtorrent.  Both of those are written in C, 
which I'm not very good at using.

I think that I'm just going to work on the basics of controlling reprepro, and 
adding/merging/removing filterlists, and when I'm satisfied that's working 
properly it'll be easier to decide how to control/manage it.  I think that it 
will be better to work in that direction first, since it will be needed 
anyway.

I have a small amount of code that I've started on.  It doesn't do anything 
yet, but create the distribution and updates files in the conf/ 
directory(ies).  I also have a bit of code to help merge filterlists, but I 
don't have any code that actually creates the lists and uses them in the 
reprepro config.  Once I figure out where to upload the code, I'll let you 
know.

-- 
Thanks:
Joseph Rawson

Attachment: signature.asc
Description: This is a digitally signed message part.


Reply to: