Re: more about Zookeeper
Zookeeper is independent of hadoop except by heritage.
Hadoop should be excellent as a rendering system for maps. Map-reduce
is used extensively by Google for similar operations.
The basic idea is that you scan across all objects emitting a copy of
the object tagged with the tiles that object intersects. In the
reduce phase, the framework has collected all objects that intersect a
single map tile together so that rendering that tile is relatively
simple.
To render only changed tiles is similar. First you do the same
process as above to get a list of all tiles with changes. Then you
scan all objects emitting copies as before, but this time limited to
those tiles that have changes.
For very small numbers of changes, it may be faster to keep an index
and retrieve the involved objects using random access. Once the
fraction of objects involved in changed tiles reaches about 0.1 to 1%
of all objects, however, it will be much faster to simply scan all
objects since you will have sequential disk access.
(but this should really be another thread)
On Sun, Jun 12, 2011 at 1:00 PM, Mike Dupont
<jamesmikedupont@googlemail.com> wrote:
> Thanks, I would like to try it out. I am currently experimenting with
> kestrel for creating a distributed openstreetmap rendering system.
> http://osmopenlayers.blogspot.com/2011/06/get-started-with-renderbot-net.html
>
> I have looked at hadoop before, but I dont know if it will be good for a p2p
> network.
>
> mike
>
> On Sun, Jun 12, 2011 at 12:51 PM, Ted Dunning <ted.dunning@gmail.com> wrote:
>>
>> Perhaps this is what you need: http://packages.debian.org/sid/zookeeper
>>
>> On Sun, Jun 12, 2011 at 12:49 PM, Ted Dunning <ted.dunning@gmail.com>
>> wrote:
>> > Debian packages?
>> >
>> > Or the original code itself?
>> >
>> > On Sun, Jun 12, 2011 at 12:36 PM, Mike Dupont
>> > <jamesmikedupont@googlemail.com> wrote:
>> >> Hi,
>> >> this sounds interesting, where can I find packages to test?
>> >> mike
>> >>
>> >> On Sun, Jun 12, 2011 at 12:07 PM, Ted Dunning <ted.dunning@gmail.com>
>> >> wrote:
>> >>>
>> >>> Zookeeper
>> >>
>> >>
>> >> --
>> >> James Michael DuPont
>> >> Member of Free Libre Open Source Software Kosova and Albania flossk.org
>> >> flossal.org
>> >>
>> >
>
>
>
> --
> James Michael DuPont
> Member of Free Libre Open Source Software Kosova and Albania flossk.org
> flossal.org
>
Reply to: