On Wed, 19 Aug 2015, Jatin Davey wrote:
On 8/19/2015 1:54 PM, John Hodrien wrote:
On Wed, 19 Aug 2015, Jatin Davey wrote:
We use CentOS 6.6 for our application. I have profiled the application and find that we have a heavy requirement in terms of Disk writes. On an average when our application operates at a certain load i can observe that the disk writes / second is around 2 Mbps (Average).
Initial thought is, do you really care? 2Mbps is peanuts, so personally
The application workload is lot of small writes written and flushed.
Is your application running fast enough? If so, I echo: do you really care?
Let's suppose your application is not running fast enough. Is the disk drive a bottleneck? If not, you need to fix someting else. Let's suppose the disk drive is a bottleneck. Are your writes sequential? If so, I'd expect that drive internal caching would favor large block sizes, e.g. 2K. Since that is what you have, I expect your writes are not sequential. Find a way to make them sequential. The "and flush" is what will make that hard. Instead of writing X to location j in the main file, write (j, X) to the next sequential location to a cache file. When the cache has enough data, do an in-memory stable sort and start writing to the main file. Clear the cache file. Unless the cache file is larege enough, I expect that this will largely duplicate what the disk drive does internally.
It might be simpler just to get a faster disk drive.