partial Debian mirroring - Was: dpkg rewrite in Java
Goswin von Brederlow wrote:
> On Sun, 25 Apr 2004, Tobias Hertkorn wrote:
>
>> Hi community!
>>
>> I am currently starting to rewrite parts of the dpkg functionality
>> in Java (For startes the dpkg --info part and a package list
>> parser). I need this for my senior programming project
>> apt-got, which is a partial mirror application that focuses
>> as a subproject on Debian archives.
>
> What kind of partial mirroring?
>
> I ask because hopefully any partial mirroring should be
> possible with debmirror2 and it would be wastefull to
> implement yet another mirroring script.
Hi Goswin,
apt-got[1] is the first subproject of a modular mirroring application
mainly written in Java (The apache module of the engine is written in
C). It does not focus on Debian archives, but it was reasonable to
start with a Debian repository module - because that is the part I
need most for my home and the Universitaet Bayreuth. Modules for
mirroring other kinds of repositories (e.g. ftp://www.kernel.org/pub/)
are planned.
I did some research before I started this project (Jan 2004) and I
came across debmirror. But debmirror did, as far as I could tell,
waste more overall bandwidth than save for my particular setup (10
machines - client and server setups - different archs - mix of stable
and testing, and some pinned unstable). This setup calls for a proxy
solution, but all proxy scripts out there had some (major) flaws that
I set out to solve in apt-got.
Apt-got will - in the first instance - be a repository proxy. That
means, it seems as if all packages are available in the local
repository, but packages are loaded from the remote repository on
client request only. (And then locally stored to further increase
delivery speed on later requests). But configurable pseudo-AI
algorithms are planned (eg prefetch the newest version of packages
that are frequently requested). My goal is to put as little stress on
the remote repository as possible. That means for example using
project/trace/ timestamps to predict possible updates, etc.
I did not know about debmirror2 up to now. Will it be written in perl
as debmirror is now? Will it support (optinal) acting as proxy - that
means, populating it on the fly? How far along is it?
As a sidenote: apt-got is my Computer Science Capstone[2] - so I
better have a reasonably stable and functional module by June 5. :-D
After that date I am allowed to and want to accept co-developers. This
will be fairly easy, as the project is already sourceforge based[3].
Greets,
T
[1] http://hacktor.fs.uni-bayreuth.de/apt-got/
[2] http://www.cs.ucsb.edu/~holl/CS189B/
[3] http://sourceforge.net/projects/apt-got/
Reply to: