[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: pycdf vs. scipy.io.netcdf vs. python-netcdf4


So I forgot about python-netcdf and pupynere.

That adds up to 3 APIs from 5 packages I know of:

* pyCDF (circa 2007):
* Scientific.IO.NetCDF        : from python-netcdf or pupynere
* scipy.io.netcdf                : from scipy, or python-netcdf4

python-netcdf appears to work with netcdf4 these days,  or rather, it uses netcdf4 libraries
and so will _read_ these files, but doesn't give an API to manage them (eg. compression facilities, parallelism, etc)
to my knowledge.

The point is that we _don't_ want 2 APIs and 5 packages, but rather to converge on one.
A package that I'm working on passes back a CDF object (originally from pyCDF, planned now
to be a netcdf_file object from scipy.io.netcdf) to the user; ie. it assumes the underlying netcdf package
I'm using is pyCDF. We're in danger of having code that uses multiple netcdf libraries.

NetCDF is morphing from a simple file format to being a way of handling large scientific data (especially using
HDF5 underneath, as in netcdf4). The files can easily be 10-100 GB in size, and with netcdf using the opendap library
underneath to read the contents as-needed over a network, its important to be able to pass netcdf objects around
without reading the contents of the underlying file unless necessary: ie abstractions that involve hiding netcdf
by reading in the file into memory are not an option.

Is it possible to standardize on one python netcdf implementation, based on the latest netcdf4 libraries?

[note: I'm having some difficulties posting to debian lists at the moment. If this mail does not appear
on debian-science, can you please repost on my behalf? thanks]


On 2012-04-18 20:56, Arnaldo Russo wrote:
Hi Alaistair,

I think that pupynere [1] could be interesting for your research packages to manipulate netCDF4.
It only depends on numpy and obviously of python.

Arnaldo D'Amaral Pereira Granja Russo
Lab. de Estudos dos Oceanos e Clima
Instituto de Oceanografia - FURG

2012/4/18 Alastair McKinstry <alastair.mckinstry@ichec.ie>

I'm looking at the problem of netcdf-wrappers for python. Unfortunately
there seem to be multiple:
* pyCDF
* scipy (scipy.io.netcdf)
* python-netCDF4

Of which only scipy is in Debian. pyCDF appears to have last been
updated in 2007. python-netcdf4
seems be 'compatible' with scipy, in that it is based on netcdf4 rather
than netcdf3, but otherwise
follows the same API. ( I don't believe the underlying objects are

I'm proposing to package python-netCDF4 for Debian, as some code depends
on it. Does anyone know
of plans to merge these projects?


Alastair McKinstry ,  Computational Scientist <alastair.mckinstry@ichec.ie>
ICHEC, Room 301, IT Building NUI Galway,  Galway , Ireland
tel: +353 91 495946

To UNSUBSCRIBE, email to debian-science-request@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 4F8ED860.5040804@ichec.ie" target="_blank">http://lists.debian.org/4F8ED860.5040804@ichec.ie

Alastair McKinstry  , <alastair@sceal.ie> , <mckinstry@debian.org>    http://blog.sceal.ie

Anyone who believes exponential growth can go on forever in a finite world
is either a madman or an economist - Kenneth Boulter, Economist.

Reply to: