Re: Hadoop in Debian, was:Re: Hardware trouble ries.debian.org
On Wed, Mar 31, 2010 at 2:42 PM, Thomas Koch <thomas@koch.ro> wrote:
> Joerg Jaspert:
> <SNIP>
>> The only trouble this setup has is that you have a pretty huge expensive
>> machine always on and running, but not actually doing stuff for
>> 99.999999999999% of the time.
> </SNIP>
>
> Hadoop is now in Debian: http://packages.qa.debian.org/h/hadoop.html
> Hadoop is an Open Source implementation of Google's File System, MapReduce and
> BigTable (HBase, not yet packaged).
>
> The idea behind Google's infrastructure and therefor Hadoop is: Have many
> cheap comodity servers that together form a powerful cluster. Each node of the
> cluster is redundant and can be replaced without downtime.
>
> I believe, but can't know for sure, that everything what FTP-Master does,
> could be implemented on top of hadoop.
> However it means for sure a lot of work and many hardcore sysadmins will feel
> very uncomfortable to use Java, the language Hadoop is written in.
Isn't there /some/ python/jython support ?
Would you co-mentor such a project as part of a Summer of Code project
? Do you know someone who would ?
It need not be ftpmaster. There are probably other critical debian
infrastructure which could use this.
> I'm planning to give a presentation of hadoop at the DebConf in Bosnia and
> maybe then we may discuss, if hadoop should have a place in Debian's
> infrastructure. - For now I'm happy, if somebody became curious. :-)
>
> http://en.wikipedia.org/wiki/Hadoop
>
> Best regards,
>
> Thomas Koch, http://www.koch.ro
Cheers
Arthur
Reply to: