Re: FW: Accessing snapshot.debian.org packages
On 05/07/2019 18:16, Paul wrote:
On 05/07/2019 10:27 weasel@debian.org wrote,
Don't overload the DB :)
So I guess pretty much every request to the machine readable interface
hits the database. How about if I did something like, make a request,
time how long it takes, wait 4 times that before making the next
request? Does that seem a reasonable place to start to avoid
breaking/abusing the system? No parallel requests, just a single
thread, with appropriate retries and backoff in the event of failure.
Hi, just a note to say I finally got around to doing something about
this. I have a single thread, making all requests. The requests which
hit the database, measure the time taken and wait 4 times longer before
continuing. The requests which fetch the actual files don't include any
delay but they are all part of the same thread so they aren't running
constantly one after another.
The traffic is coming from 2a05:d018:9e8:2100:b079:a49f:6b7a:407b.
If you have any concerns, or need me to stop, please let me know.
Regards,
Paul
Reply to: