[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Fast Random Data Generation (Was: Re: Unidentified subject!)

David Christensen writes:


On 2/11/24 00:11, Thomas Schmitt wrote:


[...]


Increase block size:

2024-02-11 01:18:51 dpchrist@laalaa ~
$ dd if=/dev/urandom of=/dev/null bs=1M count=1K
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 3.62874 s, 296 MB/s


Here (Intel Xeon W-2295)

| $ dd if=/dev/urandom of=/dev/null bs=1M count=1K
| 1024+0 records in
| 1024+0 records out
| 1073741824 bytes (1.1 GB, 1.0 GiB) copied, 2.15018 s, 499 MB/s


Concurrency:

threads	throughput
1	296 MB/s
2	285+286=571 MB/s
3	271+264+266=801 MB/s
4	249+250+241+262=1,002 MB/s
5	225+214+210+224+225=1,098 MB/s
6	223+199+199+204+213+205=1,243 MB/s
7	191+209+210+204+213+201+197=1,425 MB/s
8	205+198+180+195+205+184+184+189=1,540 MB/s


I wrote a program to automatically generate random bytes in multiple threads:
https://masysma.net/32/big4.xhtml

Before knowing about `fio` this way my way to benchmark SSDs :)

Example:

| $ big4 -b /dev/null 100 GiB
| Ma_Sys.ma Big 4.0.2, Copyright (c) 2014, 2019, 2020 Ma_Sys.ma.
| For further info send an e-mail to Ma_Sys.ma@web.de.
| | 0.00% +0 MiB 0 MiB/s 0/102400 MiB 
| 3.48% +3562 MiB 3255 MiB/s 3562/102400 MiB
| 11.06% +7764 MiB 5407 MiB/s 11329/102400 MiB
| 19.31% +8436 MiB 6387 MiB/s 19768/102400 MiB
| 27.71% +8605 MiB 6928 MiB/s 28378/102400 MiB
| 35.16% +7616 MiB 7062 MiB/s 35999/102400 MiB
| 42.58% +7595 MiB 7150 MiB/s 43598/102400 MiB
| 50.12% +7720 MiB 7230 MiB/s 51321/102400 MiB
| 58.57% +8648 MiB 7405 MiB/s 59975/102400 MiB
| 66.96% +8588 MiB 7535 MiB/s 68569/102400 MiB
| 75.11% +8343 MiB 7615 MiB/s 76916/102400 MiB
| 83.38% +8463 MiB 7691 MiB/s 85383/102400 MiB
| 91.74% +8551 MiB 7762 MiB/s 93937/102400 MiB
| 99.97% +8426 MiB 7813 MiB/s 102368/102400 MiB
| | Wrote 102400 MiB in 13 s @ 7812.023 MiB/s 

[...]

Secure Random can be obtained from OpenSSL:

| $ time for i in `seq 1 100`; do openssl rand -out /dev/null $((1024 * 1024 * 1024)); done
|
| real	0m49.288s
| user	0m44.710s
| sys	0m4.579s
Effectively 2078 MiB/s (quite OK for single-threaded operation). It is not designed to generate large amounts of random data as the size is limited by integer range... 

HTH
Linux-Fan

öö

Attachment: pgpOF7Km8vUjO.pgp
Description: PGP signature

Reply to: