kamaraju kusumanchi wrote: > Bob Proulx wrote: > > Are you aware of SQLite? > > I am still exploring all the suggestions given by others. But SQLite looks > very promising. There is a Perl DBI Interface to SQLite which might be what > I am after. Using Perl with DBI with an SQLite database works very well. It is definitely one of the robust combinations. > > Seriously though what do you want to do that can't be done easily with awk? > > I do use awk and have some one-liners based on that. But the problem comes > when you have to delete/insert a new column into existing data. Ah... That is an important note. > If you delete, say field_5 column, you have to change the index in awk > scripts for all the subsequent fields. I thought a better approach might be > to refer to the columns by their header names, so I thought a database > might be appropriate. I think you might like "Data munging with Perl, DBI, SQLite" quite well for this then. But it is a different mental paradigm from the command line data filter. As a program it will attach to the database and then you can loop over the data there. That would give you the ability to do whatever you wish to the data. But as other people have mentioned as soon as you use a database the data is no longer in simple text files. The database itself is binary. You access the data through any of a number of programmatic interfaces. It is possible to load plain text data into the database. It is possible to dump the database out as plain text files. That may be sufficient for you. You could cause all of that to be a filter. Start with plain text. Load it. Process it. Dumpt it out as plain text at the end. The extent to which this is lossless depends upon whether you have established relationships between tables in the database or not. Since SQLite is a relational database. You could have a field in one table called item_id and have it refer to another record in another table. That is the typical type of relationship. That isn't easily communicated in the type of plain text files you have described so far so I assume you do not have any such relationships in your data. But you know your data so you can decide that for yourself if what you are doing fits doing it that way or not. Bob
Attachment:
signature.asc
Description: Digital signature