[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Downloading all information in JSON format



Hi Grant,

On Donnerstag, 7. Januar 2016, Grant Murphy wrote:
> I'm trying to build a tool that monitors security issues across a
> number of different sources. One of which was the Debian security
> tracker.

cool!
 
> I had hoped to periodically poll this url:
> https://security-tracker.debian.org/tracker/data/json for changes and
> update my local copy as required. After a bit of looking I was
> surprised to see that:
> 
>   - The download size is ~30mb (Acceptable if the below measures had been
> taken) 
> - The file has not had whitespace removed (will reduce file size by
> about 5mb) 

that sounds like a useful addition now. IIRC it was done on purpose to make 
the json more readable during development of it…

> - The file is not compressed (gzip will reduce __total__ file
> size to ~2mb) 

iirc it's compressed by apache on the fly…

> - It is generated per request

true + currently by design… (not saying this cannot be changed…)

> - The 'If-Modified-Since' header is not honored so the complete file
> will need to be re-downloaded every time to check if it has changed.
> 
> 
> So here are my questions:
> 
>   - Is there a better way to access this information than periodically
> polling this URL

no

(well you could poll the same data from svn instead…)

>   - Can we at the very least get some form of compressed version of this
> data? - If nobody has time to fix this how can I contribute?

svn.debian.org/svn/secure-testing has the code and in there it's 
bin/tracker_service.py which has a function page_json…


cheers,
	Holger

Attachment: signature.asc
Description: This is a digitally signed message part.


Reply to: