Re: Debian i386 mirror

[Olaf Meeuwissen <olaf@epkowa.co.jp>]

"Gary Hennigan" <glhenni@sandia.gov> writes:

> I only mirror the 386 portions of potato, US and non-US, and my
> archive is about 2G.

I guess you don't do the sources then.

> I had a major problem setting up a potato mirror using any type of
> ftp-mirroring utility (like "mirror"). When I first started there were
> a lot of links in potato to slink and the ftp server on most of the
> debian mirror sites refused to flatten these symbolic links. If I had
> continued trying to use "mirror" I would've had to copy both potato
> and slink, both binary-all and binary-i386, in order to have a
> functional mirror. 
> 
> Instead I started using a http mirroring utility. I use w3mir but
> there are quite a few around that will do the job. The advantage is
> that there aren't any symbolic links and I can mirror just the
> binary-i386 directory structure and not worry about.

Mirroring both binary-all and binary-i386 is no problem, is it?  You
now have everything that's in binary-all in binary-i386 anyway.  The
symlinks to slink might be more of a problem.  What are you going to
do when you decide you want to mirror woody?

In the (unlikely) case you want to add an architecture to your mirror
you might want to consider mirroring both binary-all and binary-i386.
If your mirroring tool (or proxy!) flattens symlinks for you (even if
you really don't want it to), first do binary-all, then create links
for each architecture to binary-all and finally mirror architectures
you want.  The links are easily created with perl like so (please be
nice, it's my first perl program :-):

  #! /usr/bin/perl

  my($source,$target) = @ARGV;
  treelink($source,$target);

  # Duplicates the directory tree starting at $source as $target and
  # populates $target with symbolic links to all files in the $source
  # tree.  This results in a symlink mirror of the $source in $target.

  sub treelink {
      my($source,$target) = @_;
      (-d $target) || mkdir($target,0755);
      chdir($target);
      $source = "../$source";
      foreach $base (<$source/*>) {
          $base =~ s,.*/,,;
          if (-d "$source/$base") {
              treelink("$source/$base","$base");
          } else {
              symlink("$source/$base","$base");
          }
      }
      chdir("..");
  }

> I imagine most, if not all the links from potato to slink are gone
> now, but if you, for example, wanted to mirror just woody I suspect
> you'd be in the same situation, ie., having to ftp mirror both woody
> and potato, with the binary-i386 and binary-all subdirectories. Plus
> http has it's other advantages like still being easily accessible for
> those of us behind firewalls.

I just wished my http and ftp proxies didn't insist on pretty printing
directory listings.  All info about symlinks are gone, plus sizes are
in kb and times in hh:mm:ss.  Apart from all the html yuck, that is!
This wrecks havoc with just about any mirroring tool I've looked at.

-- 
Olaf Meeuwissen       Epson Kowa Corporation, Research and Development