Creating sample data for testing
I am working on a new benchmark program and I need to create large amounts of
sample data (say 10M entries which comprise 3G of data). I am considering
writing a programming language to generate sample data files, it would have
basic variable assignment, loops, and the ability to call pre-defined
functions (no facility to write your own functions).
Below is a sample of code needed to generate an LDIF file for input to an
LDAP server. The "[0-9]" type notation means to generate multiple values of
a variable (or multiple copies of an entry depending on whether it's in the
variable declaration or entry data part). @file_name means to read a line
from a text file, $R means a random number, $PWGEN() is a function to
generate a "password" from the variable name using some basic hashing
function. This is designed so that a program to test password validation can
know the password for an account without needing to lookup a table. The
$MD5() function is to generate an MD5 hash of the password (I also plan to
support CRYPT, BLOWFISH, etc).
Now the question is, has someone done something like this before well enough
to save me having to do it? If not can anyone offer suggestions on
implementation? I anticipate that this will be easiest to write in Perl, but
as I am not such a good Perl programmer I may code it in C++ anyway unless
someone offers some Perl assistance.
dn: ou=people1, dc=coker, dc=com, dc=au
objectClass: organizationalUnit
ou: people1
dn: ou=people2, dc=coker, dc=com, dc=au
objectClass: organizationalUnit
ou: people2
$1=test[[0-9][a-z]
$2=2000..64000
$3=100+$2%1000
dn: uid=$1, ou=people[12], dc=coker, dc=com, dc=au
objectClass: posixAccount
objectClass: shadowAccount
uid: $1
uidNumber: $2
gidNumber: $3
homeDirectory: /home/$1
loginShell: /bin/false
cn: @names_file[$R]
userPassword: {crypt}$MD5($PWGEN($1))
--
http://www.coker.com.au/bonnie++/ Bonnie++ hard drive benchmark
http://www.coker.com.au/postal/ Postal SMTP/POP benchmark
http://www.coker.com.au/projects.html Projects I am working on
http://www.coker.com.au/~russell/ My home page
Reply to: