[CentOS] Optimum Block Size to use

[Michael Hennebry]

On Wed, 19 Aug 2015, Jatin Davey wrote:

> On 8/19/2015 1:54 PM, John Hodrien wrote:
>> On Wed, 19 Aug 2015, Jatin Davey wrote:

>>> We use CentOS 6.6 for our application. I have profiled the application and 
>>> find that we have a heavy requirement in terms of Disk writes. On an 
>>> average when our application operates at a certain load i can observe that 
>>> the disk writes / second is around 2 Mbps (Average).
>> 
>> Initial thought is, do you really care?  2Mbps is peanuts, so personally

> The application workload is lot of small writes written and flushed.

Is your application running fast enough?
If so, I echo: do you really care?

Let's suppose your application is not running fast enough.
Is the disk drive a bottleneck?
If not, you need to fix someting else.
Let's suppose the disk drive is a bottleneck.
Are your writes sequential?
If so, I'd expect that drive internal caching
would favor large block sizes, e.g. 2K.
Since that is what you have, I expect your writes are not sequential.
Find a way to make them sequential.
The "and flush" is what will make that hard.
Instead of writing X to location j in the main file,
write (j, X) to the next sequential location to a cache file.
When the cache has enough data,
do an in-memory stable sort and start writing to the main file.
Clear the cache file.
Unless the cache file is larege enough,
I expect that this will largely duplicate
what the disk drive does internally.

It might be simpler just to get a faster disk drive.

-- 
Michael   hennebry at web.cs.ndsu.NoDak.edu
"Sorry but your password must contain an uppercase letter, a number,
a haiku, a gang sign, a heiroglyph, and the blood of a virgin."
                                                              --  someeecards