[CentOS] Cron script crashing server...

Tue Oct 4 10:34:51 UTC 2005
Ian mu <mu.llamas at gmail.com>

Hiya, redid the crontab file and still crashed, had the strace running on it
and compared the run without cron to the run from cron. It does actually run
the stats script so was misleading before, so I'm guessing previously just
hadn't got as far as outputting it to file or something.
 There's loads of stuff in there, so just highlighting the end where there
seems to be a difference in case anything springs to mind...
 I don't want anyone to go through the pain of spending time to debug
everything naturally (not my script and thousands of lines in it anyway), it
was really just in case something popped out at them obvious.
 Thanks for the replies, Ian.
  Working none cron strace for script up to the point the cron equivalent
dies.
 3045 read(4, ",\n \'*\' => \'ae181a\',\n "..., 4096) = 4096
3045 brk(0) = 0x9fe9000
3045 brk(0xa00a000) = 0xa00a000
3045 read(4, ">[0] eq \'COMT\' and $r >= $floor;"..., 4096) = 4096
3045 read(4, "\n}\n\n### MODS ###################"..., 4096) = 3988
3045 brk(0) = 0xa00a000
3045 brk(0xa02b000) = 0xa02b000
3045 read(4, "", 4096) = 0
3045 close(4) = 0
3045 open("", O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such file or directory)
>>Via cron never reaches here
3045 read(3, "T:ALL]\";\n$snipe_head{ \'GARAND_SC"..., 4096) = 4096
  Crashing server from cron strace, 10297 is the script, 10296 is a mrtg
running from a different account which runs fine whatever...
 10297 read(4, ",\n \'*\' => \'ae181a\',\n "..., 4096) = 4096
10296 read(4, " \'0\' x 6\n# creates deep recursio"..., 4096) = 4096
10297 read(4, ">[0] eq \'COMT\' and $r >= $floor;"..., 4096) = 4096
10297 brk(0) = 0x870a000
10297 brk(0x872b000) = 0x872b000
10296 read(4, "|| $x->is_zero();\n\n my $sign = "..., 4096) = 4096
10296 stat64("/usr/bin/../lib/mrtg2/integer.pmc", 0xbfff9ba0) = -1 ENOENT
(No such file or directory)
10296 open("/usr/bin/../lib/mrtg2/integer.pm", O_RDONLY|O_LARGEFILE) = -1
ENOENT (No such file or directory)
10296 stat64("/usr/bin/integer.pmc", 0xbfff9ba0) = -1 ENOENT (No such file
or directory)
10296 open("/usr/bin/integer.pm", O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such
file or directory)
10296 stat64("/usr/lib/perl5/5.8.0/i386-linux-thread-multi/integer.pmc",
0xbfff9ba0) = -1 ENOENT (No such file or directory)
10296 open("/usr/lib/perl5/5.8.0/i386-linux-thread-multi/integer.pm",
O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such file or directory)
10296 stat64("/usr/lib/perl5/5.8.0/integer.pmc", 0xbfff9ba0) = -1 ENOENT (No
such file or directory)
10296 open("/usr/lib/perl5/5.8.0/integer.pm", O_RDONLY|O_LARGEFILE) = 5
10296 ioctl(5, SNDCTL_TMR_TIMEBASE or TCGETS, 0xbfff99a8) = -1 ENOTTY
(Inappropriate ioctl for device)
10296 _llseek(5, 0, [0], SEEK_CUR) = 0
10297 read(4, <unfinished ...>
10296 read(5, <unfinished ...>
10297 <... read resumed> "\n}\n\n### MODS ###################"..., 4096) =
3988
10296 <... read resumed> "package integer;\n\nour $VERSION ="..., 4096) =
3266
10296 read(5, "", 4096) = 0
10296 close(5) = 0
10296 brk(0) = 0x9229000
10296 brk(0x924a000) = 0x924a000
10297 read(4, "", 4096) = 0
10297 close(4) = 0
10296 read(4, "the nth decimal digit, negative "..., 4096) = 4096
10297 open("", O_RDONLY|O_LARGEFILE <unfinished ...>
10296 read(4, "norm(), but $x\n\n my $x = shift;"..., 4096) = 4096
<strace stopped here>


 On 10/3/05, Ian mu <mu.llamas at gmail.com> wrote:
>
> Hiya, never thought of stracing cron itself, will try that tomorrow (is a
> bit too busy to risk it going down atm).
>  Thanks, Ian
>
>  On 10/3/05, Mark Belanger <mark_belanger at ltx.com> wrote:
> >
> > Ian mu wrote:
> > > Ok, weirder, thanks for the advice by the way, I guess its isolated it
> > > somewhat and not where I thought it would be....added the strace, but
> > > looks like not needed as never gets that far...
> >
> >
> > The next thing I would do is to attach strace to cron just before
> > it kicks off. Make sure to capture the output - there will be a lot
> > of it. strace -p PIDofCron -f -o/path/somefile
> >
> > You might also look at the cron file with an octal editor to make
> > sure there are no funny characters in it. Or just create a brand
> > new one.
> >
> > I would also look carefully each directory in the path and
> > the contents of them as well i.e.
> >
> > ls -ld /home
> > ls -la /home
> > ls -ld /home/statsman
> > ls -la /home/statsman
> > ls -ld /home/statsman/stats
> > ls -la /home/statsman/stats
> >
> > -Mark
> >
> > >
> > > Piped the cron commandline to a file crondebug as mark suggested, its
> > > created at cron run time, but empty.
> > > 36 13 * * *
> > > >/home/statsman/stats/crondebug
> > >
> > > For the first line of the run_stats.sh (tried also with #!/bin/bash)
> > > file it calls I added
> > >
> > > echo "stage 1" >> /home/statsman/stats/gendebug
> > >
> > > Run it manually from console it echos stage 1, (also echo stage 2 I
> > > added, then after that the perl script is invoked) fine.
> > >
> > > Run from cron it never even does the echo "stage 1" to the file (or at
> > > least doesn't create it).
> > >
> > > So it looks like its either dying on execing the file and not getting
> > > any further as it never writes the first line of the simple bash
> > script
> > > or not even trying to call the file? (file gendebug isn't even
> > created).
> > >
> > > More puzzled at this moment than I thought (nothing in root mail from
> > > cron either btw, and rest of crontab seems ok, I leave the line in
> > just
> > > comment out the line with a single # and it doesn't crash ).
> > >
> > > Baffled :), thanks again, Ian
> > >
> > >
> > >
> > >
> > > On 10/3/05, *Ingimar Robertsson* <iar at skyrr.is <mailto:iar at skyrr.is>>
> > > wrote:
> > >
> > > You could also check what environment variables the perl and shell
> > > scripts
> > > depend upon. Perhaps they are missing in the cron environment (Cron
> > > jobs
> > > don't get the whole shell environment, which might explain why it
> > > does work
> > > from the shell)
> > >
> > >
> > > On Mon, Oct 03, 2005 at 07:39:48AM -0400, Mark Belanger wrote:
> > > > Ian mu wrote:
> > > > > Basically I have a perl script that runs once, first thing every
> > > > > morning, gathering some stats (currently turned off due to
> > > problem).
> > > > > Crontab -l shows the following...
> > > > >
> > > > > 00 6 * * * nice /home/statsman/stats/run_stats.sh >/dev/null
> > > 2>&1 (.sh
> > > > > file basically sources with full path a config with some vars,
> > > then runs
> > > > > the perl script).
> > > >
> > > > The first thing I would do to put some echos in run_stats.sh
> > > > and change > /dev/null to > /path/somefile
> > > >
> > > > If you're getting as far as the perl script, put some print
> > > > statements in it.
> > > >
> > > > You might try doing an strace from cron:
> > > >
> > > > 00 6 * * * strace -f -oOutputFile /home/statsman/stats/run_stats.sh
> > > > >/path/somefile 2>&1
> > > >
> > > > -Mark
> > > >
> > > > _______________________________________________
> > > > CentOS mailing list
> > > > CentOS at centos.org <mailto:CentOS at centos.org>
> > > > http://lists.centos.org/mailman/listinfo/centos
> > >
> > >
> > > Bestu kveðjur,
> > >
> > > Ingimar
> > >
> > > --
> > > Ingimar Róbertsson, kerfisstjóri, RHCE EMAIL: iar at skyrr.is
> > > <mailto:iar at skyrr.is >
> > > Skýrr hf TEL: +354-5695100
> > > Ármúli 2, 108 Reykjavík FAX: +354-5695251
> > > http://www.skyrr.is/legal/disclaimer.txt
> > > _______________________________________________
> > > CentOS mailing list
> > > CentOS at centos.org <mailto: CentOS at centos.org>
> > > http://lists.centos.org/mailman/listinfo/centos
> > >
> > >
> > >
> > >
> > ------------------------------------------------------------------------
> > >
> > > _______________________________________________
> > > CentOS mailing list
> > > CentOS at centos.org
> > > http://lists.centos.org/mailman/listinfo/centos
> >
> >
> > --
> > Mark Belanger
> > LTX Corporation
> > _______________________________________________
> > CentOS mailing list
> > CentOS at centos.org
> > http://lists.centos.org/mailman/listinfo/centos
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.centos.org/pipermail/centos/attachments/20051004/34cd2ac9/attachment-0004.html>