Re: [Offtopic]: Compress Downloads on the fly

[David Wright <deblis@lionunicorn.co.uk>]

On Sat 28 Jan 2017 at 20:24:39 (+0100), Michael Luecke wrote:
> On 01/28/2017 08:10 PM, Rainer Dorsch wrote:
> >I would like to download a large csv file from a JavaScript intensive web page. My client system has not enough storage for the file, but if I could compress the file on the fly, things would work, since the file has a huge compression ratio.
> 
> If you could fetch the URL, then use wget or curl to do this:
> 
> wget -O- http://www.example.com | bzip2 >your.file.bz2
> curl http://www.example.com | bzip2 >your.file.bz2

Yes, that's very straightforward. However, the problem often
arises that wget (at least) can't download a file that the
browser can. I suspect the phrase "JavaScript intensive web page"
is meant to convey that situation. So further advice would
be welcome.

(I always try to fetch files with wget, if only for timestamps.)

Cheers,
David.