[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: [Nbd] deadlock in nbd?



Mike Snitzer wrote:
On 10/22/07, Eric Gerlach <egerlach@...135...> wrote:
Mike Snitzer wrote:
On 10/22/07, Eric Gerlach <egerlach@...135...> wrote:
Peter Daum wrote:
Hi,

Mike Snitzer <snitzer@...17...>  wrote:
 > Which IO scheduler are you using?  Please have a look at this redhat
bug; turns out that there is definitely an issue with cfq (not
exclussive to redhat):
https://bugzilla.redhat.com/show_bug.cgi?id=241540
 >
But if you're seeing this on stock kernel.org I'd imagine your just
using anticipatory.
... I'll do some more thorough testing to be sure that this was indeed all of it,
but at first sight it looks like this is also what is striking me - On all machines
but one (accidentally also the only one without interface bonding) I had "cfq"
as the default IO scheduler. After settting the scheduler for /deb/nbd0 to "noop",
so far it is running without a problem ...
Ha ha!  It works!  Thanks a lot for the pointer.  For future reference
to other people who may not want to go on a search, the secret password is:

echo noop > /sys/block/nbdX/queue/scheduler (replace X as necessary).

I have only one further question:  Is it safe to change the scheduler
while the device is mounted?  I'd like to put that command in my
rc.local, but it will only run after the device is mounted.
Actually it was pointed out (later in the redhat bugziila, by David
Miller) that using the noop scheduler for nbd really isn't too smart.
That is because the IO requests aren't coalesced before they're sent
out over the wire via nbd.  As a result nbd becomes more inefficient.
Using cfq is fine provided you patch the kernel (like redhat has
done).  I've been using deadline with good success.

And its perfectly safe to change the IO scheduler while the device is
mounted.  But as I said, you really do want to use one of the
schedulers.  So just changing the system default via elevator=<blah>
on the Linux kernel command line may be sufficient.
It seems that there aren't any problems using the other schedulers
(anticipatory or deadline), so should I just pick one of those? (for the
time being until I can patch my kernel)

YMMV, but deadline has proven the most performant for my workloads.
You should benchmark both and see which is better.

Deadline has proven to be the best for me (better by about 10%, as I recall, over the others) -- it makes sense, because as Mike points out above you need the request coalescing (noop doesn't do that) but any other scheduling overhead is just a waste, because nbd-server is going to end up going through the scheduler on the other machine anyway

--
Paul



Reply to: