Re: Fast Random Data Generation (Was: Re: Unidentified subject!)

[Jeffrey Walton <noloader@gmail.com>]

On Mon, Feb 12, 2024 at 3:02 PM Linux-Fan <Ma_Sys.ma@web.de> wrote:
>
> David Christensen writes:
>
> > On 2/11/24 02:26, Linux-Fan wrote:
> >> I wrote a program to automatically generate random bytes in multiple threads:
> >> https://masysma.net/32/big4.xhtml
> >>
> >> Before knowing about `fio` this way my way to benchmark SSDs :)
> >>
> >> Example:
> >>
> >> | $ big4 -b /dev/null 100 GiB
> >> | Ma_Sys.ma Big 4.0.2, Copyright (c) 2014, 2019, 2020 Ma_Sys.ma.
> >> | For further info send an e-mail to Ma_Sys.ma@web.de.
>
> [...]
>
> >> | 99.97% +8426 MiB 7813 MiB/s 102368/102400 MiB
> >> | Wrote 102400 MiB in 13 s @ 7812.023 MiB/s
> >
> >
> > What algorithm did you implement?
>
> I copied the algorithm from here:
> https://www.javamex.com/tutorials/random_numbers/numerical_recipes.shtml
>
> I found it during the development of another application where I needed a
> lot of random data for simulation purposes :)

A PRNG for a simulation has different requirements than a PRNG for
cryptographic purposes. A simulation usually needs numbers fast from a
uniform distribution. Simulations can use predictable numbers. Often a
Linear Congurential Generator (LCG) will do just fine even though they
were broken about 35 years ago. See Also see Joan Boyer's Inferring
Sequences Produced by Pseudo-Random Number Generators,
<https://asterix.cs.gsu.edu/crypto/p129-boyar.pdf>.

A cryptographic application will have more stringent requirements. A
cryptographic generator may (will?) take longer to generate a number,
the numbers need to come from a uniform distribution, and the numbers
need to be prediction resistant. You can read about the cryptographic
qualities of random numbers in NIST SP800-90 and friends.

> My implementation code is here:
> https://github.com/m7a/bo-big/blob/master/latest/Big4.java
>
> If I were to do it again today, I'd probably switch to any of these PRNGS:
>
>  * https://burtleburtle.net/bob/rand/smallprng.html
>  * https://www.pcg-random.org/
>
> >> Secure Random can be obtained from OpenSSL:
> >>
> >> | $ time for i in `seq 1 100`; do openssl rand -out /dev/null $((1024 * 1024
> >> * 1024)); done
> >> |
> >> | real    0m49.288s
> >> | user    0m44.710s
> >> | sys    0m4.579s
> >>
> >> Effectively 2078 MiB/s (quite OK for single-threaded operation). It is not
> >> designed to generate large amounts of random data as the size is limited by
> >> integer range...

Jeff