[CentOS] strict memory
John R Pierce
pierce at hogranch.com
Thu Oct 16 17:42:52 UTC 2008
Mag Gam wrote:
> Hi John:
>
> Well, we run a lot of statistical analysis and our code loads a lot of
> data into a vector for fast calculations. I am not sure how else to do
> these calculations fast without loading it into memory. Thats why we
> have to do it this way.
>
well, if you got several processes that each need 32GB in a 64GB
machine, you're gonna end up swapping.
the traditional way of doing this sort of thing on limited memory
machines was to take a sequential pass through the data, calculating the
statistics on the fly. I know that kind of thing is very difficult for
some algorithms (FFT's are notorious for being unfriendly to sequential
processing), but for many algorithms, a few sequential passes of
calculations can be /faster/ than random access and swapping when theres
memory/process contention.
More information about the CentOS
mailing list