[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Bug#563765: ITP: python-whisper -- fast, reliable storage of numeric data over time.

Package: wnpp
Severity: wishlist
Owner: Elliot Murphy <elliot@ubuntu.com>

This bug report is about my intent to package python-whisper,
which is a part of the graphite project:

* Package name    : python-whisper
  Version         : 0.9.5
  Upstream Author : Chris Davis <chrismd@gmail.com>
* URL             : http://pypi.python.org/pypi/whisper/
* License         : Apache-2.0
  Programming Lang: Python
  Description     : fast, reliable storage of numeric data over time.
 Whisper is a fixed size database, similar in design to RRD.
 RRD is great, and initially Graphite did use RRD for storage..
 Over time though, we ran into several issues inherent to RRD's design.
 1. RRD can't take updates for a timestamp prior to its most recent update.
 So for example, if you miss an update for some reason you have no simple
 way of back-filling your RRD file by telling rrdtool to apply an update
 to the past. Whisper does not have this limitation, and this makes
 importing historical data into Graphite way way easier.
 2. At the time whisper was written, RRD did not support compacting multiple
 updates into a single operation. This feature is critical to Graphite's
 3. RRD doesn't like irregular updates. If you update an RRD but don't follow
 up another update soon, your original update will be lost. This is the
 straw that broke the camel's back, since Graphite is used for various
 operational metrics, some of which do not occur regularly (randomly
 occuring errors for instance) we started to notice that Graphite
 sometimes wouldn't display data points which we knew existed because
 we'd received alarms on them from other tools. The problem turned out
 to be that RRD was dropping the data points because they were
 irregular. Whisper had to be written to ensure that all data was
 reliably stored and accessible.

Reply to: