[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: dvd+rw-tools update [6.0, DVD-R DL]



scdbackup@gmx.net wrote:

Hi,

How come that the time granularity of the backup processing chain
does not get finer as the systems get faster ?
What do you understand by time granularity?

I see a fifo as a method to smoothen out peaks and gaps in a
input function and to bring the output nearer to the input
average.
The intensity of peaks and gaps resp. the deviation from the
average can be characterized by the time span in which one
may expect that those irregularities compensate each other.
The product of this time span and the average speed determines
the size which is needed for an effective buffer.
This time span is what i mean with "time granularity".

If the overall system gets faster, i would expect this characteristic timespan to get shorter. But it seems to
stay within the range of several seconds.
Since the average speed grew and the time staid more or
less the same, the fifo size had to grow.

The only logical explanation is that the characteristics
of the input function have changed while the system
became faster.
I think the problem is that they haven't changed... disks are larger, CPUs faster, latency less, but as drive sizes have jumped by two orders of magnitude (4GB=>400GB for ~$150US) seek times have not gotten faster by more than about 2:1 over what was used almost a decade ago. So people have far more small files, needing far more seeks to get the data, and the ability to get to those files is simply not up to the job.

Faster hardware isn't cost effective, I have servers using 15k RPM drives with LVD320 interface and ~4ms seek, and each drive cost more than a complete discount computer! RAID helps, overlapping file open and data read (or batching them) helps, but in truth the low cost of large disk and better filesystems which don't get unusably slow with thousands of small files have caused many people to HAVE a ton of small files, which really hurt when doing backup, or grepping through all of them, or similar.


Other views which lead me to the same result:

My considerations about the benefits and effectivity of a fifo
always dealt with relative speeds of input and output. Never
with absolute speed.
Thus one would expect that if both input and output speed
increase in the same proportion, then the effectivity should
stay the same. But it obviously doesn't.

A simple thought experiment:
Imagine a movie of an old backup run which is shown at double
speed. The report messages about buffer size and buffer fill would
stay exactly the same.
If everything would have gotten faster by technical progress then
we would have exactly that highspeed movie situation. But we haven't.


But oneproblem is that current drives have less internal RAM compared to
older drives if you take the drive speed into account.

True. But were the drive buffers sufficient 4 years ago ?
If they weren't very effective in the past then their relative
diminishment would not be significant now.
I'm still riddling.
What effect did change the shape of our input functions ?

You have my thoughts, I have a limited number of tests to support my conclusions, feel free to present data supporting a better (more easily improved) scenario.

--
bill davidsen <davidsen@tmr.com>
 CTO TMR Associates, Inc
 Doing interesting things with small computers since 1979



Reply to: