Re: looking for a file format for time series data other software can easily consume
On Tue, Oct 31, 2017 at 12:21 PM, Ben Caradoc-Davies <ben@transient.nz> wrote:
> On 31/10/17 19:33, Dan Hitt wrote:
>>
>> I plan to produce some time series data and i would like it to be in a
>> form that makes it easy or effortless for pre-existing programs to
>> plot or even process further.
>
>
> NetCDF is very popular for scientific applications. NetCDF-4 uses HDF5 (also
> mentioned in this thread) for storage, NetCDF-3 has its own storage format:
> https://www.unidata.ucar.edu/software/netcdf/
>
> Many open source libraries and tools.
>
> Kind regards,
>
> --
> Ben Caradoc-Davies <ben@transient.nz>
> Director
> Transient Software Limited <http://transient.nz/>
> New Zealand
>
Thanks Ben, and also deloptes, Darac, and Henning.
deloptes --- thanks for your advice to base what i write on the
graphing tool i settle on. That's pretty reasonable, especially given
what i said. I'm still undecided about the tool right now though.
Thanks Darac for recommending a RRD. Using something old and
established is what i prefer. The package by Oetiker (rrdtool) looks
good and reminds me of mrtg which i used some years back at the
suggestion of a colleague (Sam S), and which i had forgotten. My data
is not really circular, although i may end up using it depending on
how everything else shakes out.
Thanks Henning for the suggestion of checking out HDF(5). I had run
across that in my googling, and was a little intimidated as it seems
to be a very big project, but after reading Ben's suggestion for using
NetCDF i may actually be secretly using it.
The NetCDF does seem to do what i want. As far as i can tell, it
allows you to set metadata at the file level as well as attributes
attached to specific arrays in your data. It can be installed in
debian (packages libnetcdf-dev, netcdf-doc, netcdf-bin). The api
doesn't look too complicated, at least if i understand it correctly.
The packages come with some simple example programs. And the 'file'
command recognizes the data that it produces. And there's some kind
of mailing list. So if i can make a go of it, i think that's what
i'll be doing.
And if i can't, well, then thanks everybody for the other suggestions,
which i'll try out more thoroughly.
I sure appreciate all the good advice!
dan
Reply to: