Re: dm-crypt/LUKS performance
On 11/16/2014 04:04 PM, David Fuchs wrote:
I am setting up a system with an Intel octo-core Avoton, which has AES-NI
support.
... The drive in question is a Samsung 840 pro SSD,
Which Debian release? Kernel? Motherboard make/ model? CPU model?
RAM module(s) make/ model? SSD exact model? Defaults? Customizations?
That SSD appears to have hardware encryption. So, why dm-crypt?
http://www.samsung.com/global/business/semiconductor/minisite/SSD/uk/html/about/SSD840Pro.html
After doing some crude benchmarking tests with dd, I am surprised
about the huge performance penalty that full-disk encryption apparently has
on read/write throughput.
I've also struggled with getting meaningful results from dd and other
tools -- there's a lot of complexity going on inside modern x86 systems.
Anandtech has some good articles about SSD benchmarking, with
discussion of the challenges involved and their custom benchmarking
software:
http://www.anandtech.com/SearchResults?q=ssd
I also see some titles about a firmware bug and read performance
problems with Samsung 840 drives. Does that affect you?
cryptsetup luksFormat -c aes-xts-plain64 --hash sha512 --iter-time 2000
--use-random -s 512 /dev/sdd6
I assume those are non-default option values. Predicting and measuring
how each of those are affected by AES-NI would be a non-trivial task.
You might want to try some benchmarks using the defaults first, and then
go from there.
Results w/o encryption:
# dd bs=1M count=256 if=/dev/zero of=/dev/sdd6 conv=fdatasync
256+0 records in
256+0 records out
268435456 bytes (268 MB) copied, 0.990527 s, 271 MB/s
# dd bs=1M count=512 if=/dev/sdd6 of=/dev/null
512+0 records in
512+0 records out
536870912 bytes (537 MB) copied, 1.15489 s, 465 MB/s
Results with encryption:
# dd bs=1M count=512 if=/dev/zero of=/dev/mapper/test conv=fdatasync
512+0 records in
512+0 records out
536870912 bytes (537 MB) copied, 3.26955 s, 164 MB/s
# dd bs=1M count=512 if=/dev/mapper/test of=/dev/null
512+0 records in
512+0 records out
536870912 bytes (537 MB) copied, 4.66179 s, 115 MB/s
Did you do a secure erase on the SSD prior to the write tests? SSD's
can write to fresh blocks faster than they can reclaim old blocks, erase
them, and then write.
Your first write test is 256 MB, but all the other tests are 512 MB.
Why the unequal size?
Did you clear the kernel block device memory buffers after writing and
before reading?
A stream of all zeros (or all ones) is easily compressed. It would be
safer to benchmark with random numbers. Does your CPU have Secure Key?
What's the throughput of your /dev/urandom? If it's not faster than
your SSD interface (SATA3 600 MB/s?), do you have another SSD that you
could put a huge random number file (or partition) on to use as a source?
What about CPU frequency governors? Cores available? Hyperthreading
enabled/ disabled? Concurrency/ scheduling? System loading, including
interrupts (such as network interfaces)?
1.0 to 4.7 seconds is pretty brief, and I assume you have more RAM than
512 MB. You might want to try 1x, 2x, 3x, etc., RAM size and see what
happens -- e.g. get past caching/ buffering and get into steady-state.
Also, it would be useful to get measurements along the way -- see dd(1)
and USR1 signals. Script it all to get repeatability.
David
Reply to: