[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

[gopher] Re: New Gopher Wayback Machine Bot

On Wed, Oct 12, 2005 at 11:28:26PM -0500, John Goerzen wrote:
> On Wed, Oct 12, 2005 at 09:07:52PM -0500, Benn Newman wrote:
> I will be doing more work tomorrow to make it be more round-robin so
> as to not put much load on any one server at once.

And this bit is now done as well.  It will make one connection to
every server in its "to-spider" list before it ever connects to one

I'll also keep an eye on it and make sure nobody gets pounded at the
very end.

-- John

Reply to: