Re: OT: Collecting data in text files

[Erik Steffl <steffl@bigfoot.com>]

Paul M Foster wrote:
> 
> On Mon, Aug 27, 2001 at 12:29:17PM -0700, Abner Gershon wrote:
> 
> > I have am currently dual booting to debian and would
> > like to change to rely more on debian and related
> > applications.
> >
> > I currently collect lists of information on many
> > topics such as restaurants, phone lists, file folders,
> > slides using a database program, Lotus approach. Since
> > I am the only user of this information I thought I
> > could record this information in text files using vim
> > and then use gawk and sed to access this data.
> >
> > I have done a little reading on how these programs
> > work and would like to know if it is generally better
> > to seperate columns with spaces or tabs for this
> > purpose and if this would be the most appropriate
> > technique for collecting and organizing such data.
> >
> > I realize this does not pertain directly to the debian
> > os and would appreciate direction to more appropriate
> > mailing list if anyone knows one.
> >
> 
> Best answer is probably some other database, like PostgreSQL. However,
> if you're going to store in a text file, use tab as the delimiter. The
> reason being that any embedded spaces within the fields themselves will
> trip up your parsing of the files, unless you use something not likely
> to occur in your fields, like the tab character.
> 
> And if you have typical tasks you will always do with these files,
> consider using Perl rather than sed and awk. Perl will do everything sed
> and awk will do, and more.

  my suggestion is to use some other character that:

  - is definitely not going to occur in data (space is usually not good)
  - is easy to see (if you plan to edit the file, tab is not good)
  - is not going to be mangled by editor (tab is not good!)

  for general text data pick something like:

  ,;:|

  on input check data for delimiter, if found, panic (error message, do
not process data etc., depends on application)

	erik