Mag Gam wrote:
For instance, country/2005/01/01/foo.txt
...
For instance, grep "something" country/2005/??/01/foo.txt It gives an instant result. Thats how we are using it and we love it.
I would suspect it's two-fold.1. Your data is already separated and well organized down to country/year/day/month and listed in foo.txt, which keeps each foo.txt fairly slim with only what is pertinent to that country/year/day/month.
2. The query you're typing is limited to that country/year/day/month.That said, take foo.txt from all country/year/day/month and combine them all into a single file. You'll see that your query takes longer to produce results.
Also, I wouldn't go so far as to say your data never contains duplicates. I'm sure that once you combine foo.txt files from all country/year/day/month, you'll find duplicate entries of something, and that's where an DBMS can be beneficial.