[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

webcrawl to cache dynamic pages

I am considering how to crawl a site which is dynamically generated,
and create a static version of all generated pages (or selected
generated pages).  I guess it would be simplest to start with an
existing crawler, and bolt on some code. Or, alternatively, write a
script (perl, I fear) to modify the cache built by a crawler. 

The idea is to allow a static ecommerce site to be generated from any
database-generated shopping cart system.

Any advice where to begin?



Reply to: