Re: file systems for linux clusters...
On Sun, 15 Apr 2001, Rob Latham wrote:
> i would not be so quick to dismiss PVFS. The 'P' stands for parallel,
> after all. today, PVFS and GFS solve very different problems. they
> might one day become more equivalent, but right now GFS is for storage
> networks, and pvfs is for parallel IO.
>
Yes, but I didn't know it had the ability to have you run code on disks
of nodes of the same database file. That sure is the capability required
for making passes over a database (say for running a parallel data mining
algorithm) in parallel.
> As you alluded, MPICH and LAM both come with support for ROMIO
> (MPI-IO. it's in v2 of the MPI spec ). As i understand it, you have
> to configure romio to work with a parallell file system of some sort.
> NFS works with romio (poorly, but works). The other romio options are
> 'unix' ( intended for something like an SGI running CXFS ), cray's
> PVS, some other 'big iron' file systems, and PVFS.
>
Can't you just use ROM I/O with normal local filesystems at each node?
That would be the one with minimum system overhead. Confusing. Have to
look at those ROMIO manuals again, I'll be doing something similar real
soon now. :{
> build your ROMIO libraries with support for pvfs ( a matter of
> specifying the right flag to 'configure' ), and there you go: message
> passing codes with access to a parallell file system.
>
> Eray, i haven't used ROMIO without pvfs support: is there a way to
> use it without pvfs or nfs as the underlying file system?
>
At this point I am not sure, I have to check the docs. But of course
using ROMIO or implementing your I/O with custom message passing code
seems to be the only viable option.
Thanks,
__
Eray
Reply to: