Re: too many CD ISOs
Raphael Hertzog wrote:
Reason: it's much more effective for them when users download an ISO than
when they use jigdo. For example the ftp.fr.debian.org admin explained me
that they were limited by the disk seek and not by the bandwith used by
the users.
Right and wrong. I am not the admin of ftp.fr.debian.org (Aurélien is, I
am mostly a backup) which does not host (AFAIK) CD ISOs but the admin of
ftp.free.fr (AKA ftp.proxad.net) which hosts them (and many other
things). Both of them are hosted on a french ISP network. As one of the
purpose is to deliver the contents as fast as possible (we have around 2
millions of ADSL users with uncapped connections), we try to bypass most
of the bottlenecks.
The server is a dual Opteron X2 with 16 GB of RAM, 24 SATA disks on
3x3Ware (8 disks are setup in RAID5, the rest of them in JBOD and then
soft mirrored) and two aggregated Gbps links. On a regular day, there is
between 700 to 1000 simultaneous connexions for a daily traffic from 4
to 6 TB, evening peak is around 800Mbps to 1 Gbps. Historic peak was
with around 3K connexions, 2 GBps for a few hours and 9.5 TB delivered
in a day.
Richard Atterer wrote:
> That's interesting - I hadn't heard of this kind of problem before!
> :-/
Sooner or later, you will hear more and more often about this kind of
problem. Have a look to http://lwn.net/Articles/190222/ to get a brief
introduction (just read "the problem" part) about disks limitations.
Roughly, the problem is disks have less and less IOs/s per storage size
(ie between 2006 and 2013, disks capacity should grow by 16 times, their
bandwidth by 5 times and seek time by 1.2 times). Unless you are ready
to scale with the number of disks, you will try to reduce the number of
disks seeks (and thus try to read more and more data per seek).
16 GB of RAM is huge but with 2.5 TB of hosted data, disk IO load was
huge too (some high traffic mirrors like Mandriva or Fedora were on 4
disks RAID1 volume to keep up with disk load).
So, in our case, we had to :
- force pre-reading in a smart way (cf fadvise patch in
ftp://213.228.1.105/home/vsftpd/ ), using 1 MB chunk allowed us to move
some high-traffic mirror from 4 disks RAID1 volume to 2 disks with a
lower IO load.
- stop doing RAID5 volumes (excepted for low traffic mirrors) to reduce
disks seeks induced by stripping (64 KB on 3Ware cards).
But we also had to disabled resume ability (some "download accelerators"
are using very small chunks and most of them are mis-using server
ressources).
So, as far as I am concerned, unless one has to download less than 650
packages to build up his 650 MB CD ISO, I prefer ISOs downloads to
jidgo... :-)
François
Reply to: