Re: Hardware trouble ries.debian.org - ftpmaster.debian.org / release.d.o services disabled
Replying to three in one.
>> Well, this would mean:
>> a) double ftpmaster. Just as a rough number, the new machine that is
>> currently "in progress" has an estimated cost of 20000 Dollar (if you
>> take prices from HP Website. This is not what it will cost in the
>> end, but it shows what category of stuff we have to get)
>> b) Have the double space at our hoster.
>> c) Run it with heartbeat, drbd and all that.
>> Point c) is actually easy enough, even though im not DSA and can't decide
>> for them to do it. But technically it would be a working setup, provided
>> b) works out, as you really want a *FAST* connection between the
>> two. Which means local.
> Is there any way to build a distributed service instead of relying on
> one central machine (or two machines sitting next to each other)?
Not currently, no. The actions in the archive can not split in a way
this would make sense to place on various machines. That is, not those
that are the important ones here (upload processing, mirrortree
updating, that parts all around it).
> I don't know exactly what services are involved, but typically and
> generally, when I deploy a server infrastructure, I try to setup (at
> least) two machines at different geographical places that each can
> provide the entire service.
...
> The complexity in getting a heartbeat, drbd, etc solution to work can
> easily eat up any downtime savings you plan to get.
It is actually a very easy setup and I am running multiple of them. The
complexity is mainly in the double rackspace, double internet
connection, *FAST* inter-machine connection, not in the bit drbd stuff.
On 12071 March 1977, Guus Sliepen wrote:
> What is it that ftp-master does that can only run on one computer at a time? If
> most of the services it provides could be distributed, you could spread the load to
> multiple machines, and get redundancy at the same time.
Thank you, we never thought of that...
On 12071 March 1977, Thomas Koch wrote:
> Joerg Jaspert:
> <SNIP>
>> The only trouble this setup has is that you have a pretty huge expensive
>> machine always on and running, but not actually doing stuff for
>> 99.999999999999% of the time.
> </SNIP>
> Hadoop is now in Debian:
> http://packages.qa.debian.org/h/hadoop.htmlHadoop is an Open Source
> implementation of Google's File System, MapReduce and
> BigTable (HBase, not yet packaged).
> The idea behind Google's infrastructure and therefor Hadoop is: Have many
> cheap comodity servers that together form a powerful cluster. Each node of the
> cluster is redundant and can be replaced without downtime.
> I believe, but can't know for sure, that everything what FTP-Master does,
> could be implemented on top of hadoop.
> However it means for sure a lot of work and many hardcore sysadmins will feel
> very uncomfortable to use Java, the language Hadoop is written in.
> I'm planning to give a presentation of hadoop at the DebConf in Bosnia and
> maybe then we may discuss, if hadoop should have a place in Debian's
> infrastructure. - For now I'm happy, if somebody became curious. :-)
The idea behind hadoop and stuff is certainly a nice one. Yet, I do
believe it has an entirely different target and can not easily be
adopted to the tasks ftp-master does. There simply are things you can
not, in a sensible way, make distributed.
On 12071 March 1977, Obey Arthur Liu wrote:
> Isn't there /some/ python/jython support ?
> Would you co-mentor such a project as part of a Summer of Code project
> ? Do you know someone who would ?
> It need not be ftpmaster. There are probably other critical debian
> infrastructure which could use this.
I do not think changing dak and all it does to hadoop and distributed is
possible within gsoc. Nor within ten of it.
I wont block any who tries though, so have fun. :)
--
bye, Joerg
<andreasj> Also diese neuen Spam-Mails muten an wie Blog-Posts von Clint Adams
<andreasj> irgendwie ist es eine Geschichte, aber ich versteh sie nicht
Reply to: