webcrawl to cache dynamic pages
I am considering how to crawl a site which is dynamically generated,
and create a static version of all generated pages (or selected
generated pages). I guess it would be simplest to start with an
existing crawler, and bolt on some code. Or, alternatively, write a
script (perl, I fear) to modify the cache built by a crawler.
The idea is to allow a static ecommerce site to be generated from any
database-generated shopping cart system.
Any advice where to begin?