On Apr 30, 2007, at 2:33 PM, Ashley M. Kirchner wrote:
> So, anyone going to make sense out of that please?
Sure. If you're doing this sequentially:
for file in *.jpg; do
convert $file -thumbnail 200x200 > `basename $file .jpg`.th.jpg
done
it's never going to be using more than one processor. Some of the
time it's using processor 0, sometimes processor 5. (Keep in mind
that those percentages are averaged over some unit of time. As one of
my coworkers is fond of saying, at a given instant there's no such
thing as a 50% busy CPU - it's either 100% or 0% busy.) The processor
0 and 5 user and system numbers add up to a bit less than 100%.
You could do something like this instead:
$ cat > GNUmakefile <<EOF
SOURCES = $(wildcard *.jpg)
THUMBNAILS = $(SOURCES:%.jpg=%.th.jpg)
.PHONY: thumbnails
thumbnails: $(THUMBNAILS)
%.th.jpg: %.jpg
convert $< -thumbnail 200x200 > $@ || (rm $@; false)
.PHONY: clean
clean:
rm -f *.th.jpg
EOF
(note that those indentations have to be tabs, not spaces)
$ make clean
$ time make -j1
$ make clean
$ time make -j8
I'd expect the first timed make to take about eight minutes and the
second timed make to take somewhere between one and two minutes.
Now, going back to your first post:
> This problem is causing one of our web sites to time out because
> it's trying to process hundreds of image files and generate
> thumbnails, and it's taking forever to do that. So I'm starting at
> the bottom of the pile here, hardware. If it turns out the
> hardware is fine, and there's nothing else that can be done to
> speed it up, then I'll move forward to other possible culprits,
> such as the routines within the site scripts themselves...
You're not trying to generate thumbnails on every hit, are you? Even
with all the might of eight processors and this RAID array, you won't
get this task to complete in the .1 sec that usability experts say is
the maximum acceptable page load time. You're going to have to
precompute them. Or if you're doing it on upload and that's timing
out, then redirect the browser to a progress bar page before
converting or something.
--
Scott Lamb <http://www.slamb.org/>