Hi All,
I need to write a script that I will manually start (or a cron job in future) but I need it to do a number of things in order one after another. How do i do that so everything gets dont as the steps depend on each other.
Example:
cd /system_backups/
tar cvf apache-conf.tar /etc/httpd/conf/* gzip -v9 apache-conf.tar
tar cvf apache-data.tar /var/www/* gzip -v9 apache-data.tar
then last step... tar cvf <current_date>-system_backup.tar <all> the gzip files above gzip -v9 <current_date>-system_backup.tar
scp <current_date>-system_backup.tar.gz user@10.0.0.1:/. etc...etc....
My questions: 1. How do I execute each statement and make sure subsequent statements are not executed until the previous is done?
2. How do I error check so if a step fails the script stops?
3. Since I run an SMTP Server on this box can I e-mail myself from bash the nightly results?
4. when I want to run the scp to send over the file to another machine for safety, how can I have it know the password to the machine I am scp'ing to?
My mind is going crazy sort of with the things that I could do to protect myself in case of a system failure and making restoring easier.
Can anyone provide insight for me?
-Jason
Jason Todd Slack-Moehrle <mailinglists@...> writes:
Hi All,
I need to write a script that I will manually start (or a cron job in future) but I need it to do a number of things in order one after another. How do i do that so everything gets dont as the steps depend on each other.
Example:
cd /system_backups/
tar cvf apache-conf.tar /etc/httpd/conf/* gzip -v9 apache-conf.tar
tar cvf apache-data.tar /var/www/* gzip -v9 apache-data.tar
then last step... tar cvf <current_date>-system_backup.tar <all> the gzip files above gzip -v9 <current_date>-system_backup.tar
scp <current_date>-system_backup.tar.gz user@...:/. etc...etc....
My questions:
- How do I execute each statement and make sure subsequent statements
are not executed until the previous is done?
by using &&
example cd /system_backups/ && \ tar cvf apache-conf.tar /etc/httpd/conf/* && \ gzip -v9 apache-conf.tar
- How do I error check so if a step fails the script stops?
I am afraid I do not have enough knowledge to help you in the above question
- Since I run an SMTP Server on this box can I e-mail myself from
bash the nightly results?
Yes
- when I want to run the scp to send over the file to another machine
for safety, how can I have it know the password to the machine I am scp'ing to?
You cant
You will have to ssh-keygen
Thanks and Regards
Rajagopal
On Thu, May 07, 2009 at 10:12:59PM -0700, Jason Todd Slack-Moehrle wrote:
Hi All,
I need to write a script that I will manually start (or a cron job in future) but I need it to do a number of things in order one after another. How do i do that so everything gets dont as the steps depend on each other.
Example:
cd /system_backups/
tar cvf apache-conf.tar /etc/httpd/conf/* gzip -v9 apache-conf.tar
tar cvf apache-data.tar /var/www/* gzip -v9 apache-data.tar
then last step... tar cvf <current_date>-system_backup.tar <all> the gzip files above gzip -v9 <current_date>-system_backup.tar
scp <current_date>-system_backup.tar.gz user@10.0.0.1:/. etc...etc....
My questions:
- How do I execute each statement and make sure subsequent statements
are not executed until the previous is done?
This will happen on its own unless you intentionally force a command into the background with &.
- How do I error check so if a step fails the script stops?
You can use && or || or check the status of $?. For example:
tar cvfz apache-conf.tar.gz /etc/httpd/conf/*
if [ $? != 0 ]; then echo "Error creating tar file." exit 255 fi
$? contains the exit code of the previously executed command.
Or something like...
tar cvfz apache-conf.tar.gz /etc/httpd/conf/* || \ { echo "Problem creating tar file."; exit 255; }
The first method is probably a little more clear.
- Since I run an SMTP Server on this box can I e-mail myself from
bash the nightly results?
Sure. If you call it from cron, anything on stdout will be emailed to the user running the job. Alternately you can call the "mail" command. You can also surround several commands and pipe their output to a command:
( echo "Test" echo "Test 2" ) | mail email@email.com
- when I want to run the scp to send over the file to another machine
for safety, how can I have it know the password to the machine I am scp'ing to?
You'll want to set up ssh keys for this.
My mind is going crazy sort of with the things that I could do to protect myself in case of a system failure and making restoring easier.
Can anyone provide insight for me?
-Jason
Ray
On Thu, May 07, 2009 at 10:12:59PM -0700, Jason Todd Slack-Moehrle wrote:
I need to write a script that I will manually start (or a cron job in future) but I need it to do a number of things in order one after another. How do i do that so everything gets dont as the steps depend on each other.
My questions:
- How do I execute each statement and make sure subsequent statements
are not executed until the previous is done?
Use the '&&' (logical "and") operator to pipeline the commands:
cd /foo/bar && tar cvf /path/to/tarball
This will only execute the "tar" command if the previous "cd" succeeded.
- How do I error check so if a step fails the script stops?
Most commands set a return status upon completion or error; this value is returned to the shell in the "$?" variable.
You could write the above as something like:
cd /foo/bar if [ $? -ne 0 ] then echo "cd failed ($?) - bailing out" exit 1 fi tar cvf /path/to/tarball if [ $? -ne 0 ] then echo "tar failed ($?) - bailing out" exit 2 fi
This checks the return status of each command, checks to see if it's not 0 (0 is "success") and if it is not alerts the user and exits with it's own error status.
- Since I run an SMTP Server on this box can I e-mail myself from
bash the nightly results?
Yep. "echo error message" | mail -s "script failure" user@dom.ain
Will mail the specified user a message containing "error message" with a subject of "script failure".
- when I want to run the scp to send over the file to another machine
for safety, how can I have it know the password to the machine I am scp'ing to?
You do this with public key authentication. "man ssh-keygen" for information on how to create a keypair.
My mind is going crazy sort of with the things that I could do to protect myself in case of a system failure and making restoring easier.
We've all been there.
Can anyone provide insight for me?
Test everything. On regular schedules. If you have the hardware available simulate a disaster condition and restore your backups to identical hardware to prove (to yourself and to management) that your disaster recovery steps actually do work.
John
Jason Todd Slack-Moehrle wrote:
Hi All,
I need to write a script that I will manually start (or a cron job in future) but I need it to do a number of things in order one after another. How do i do that so everything gets dont as the steps depend on each other.
A few commands can be stuck together in the cron command. Separating them with ';'' makes them execute sequentially regardless of status; separating with '&&' will continue only if the previous command succeeds. For more than a few, put the commands in a script file
Example:
cd /system_backups/
tar cvf apache-conf.tar /etc/httpd/conf/* gzip -v9 apache-conf.tar
I'd: cd /etc/httpd/conf && tar -czvf /system_backups/apache-conf.tar.gz instead.
tar cvf apache-data.tar /var/www/* gzip -v9 apache-data.tar
then last step... tar cvf <current_date>-system_backup.tar <all> the gzip files above gzip -v9 <current_date>-system_backup.tar
scp <current_date>-system_backup.tar.gz user@10.0.0.1:/. etc...etc....
My questions:
- How do I execute each statement and make sure subsequent statements
are not executed until the previous is done?
Commands in a script execute sequentially unless you explicitly use an & to put them in the background.
- How do I error check so if a step fails the script stops?
Each command will return a status in the $? variable that you can check. The shell philosophy is that there are lots of ways to fail but usually only one way to succeed, so the value 0 represents success, anything else is failure. You can use the 'test' operator (usually written as [, but read the test man page to see what it does), or just: command1 && command2 (runs command2 only if command1 succeeds) command1 || command2 (runs command2 only if command1 fails - you can exit this way)
- Since I run an SMTP Server on this box can I e-mail myself from
bash the nightly results?
Yes, cron will automatically mail any output to the user that owns the cron job. Just add an alias for this local user (probably root) to make it go where you want.
- when I want to run the scp to send over the file to another machine
for safety, how can I have it know the password to the machine I am scp'ing to?
Use ssh keys instead.
My mind is going crazy sort of with the things that I could do to protect myself in case of a system failure and making restoring easier.
Can anyone provide insight for me?
You'd probably be better off running something like backuppc (http://backuppc.sourceforge.net/) instead of rolling your own. And you might want some kind of version control system like subversion for your content and most important configuration files.
-- Les Mikesell lesmikesell@gmail.com
Jason Todd Slack-Moehrle wrote:
Hi All,
I need to write a script that I will manually start (or a cron job in future) but I need it to do a number of things in order one after another. How do i do that so everything gets dont as the steps depend on each other.
Example:
cd /system_backups/
CD = $(date +%Y-%m-%d)
if ! tar cvf apache-conf.tar /etc/httpd/conf/* && gzip -v9 apache-conf.tar; then echo "FATAL ERROR 1" exit elif if ! tar cvf apache-data.tar /var/www/* && gzip -v9 apache-data.tar; then echo "FATAL ERROR 2" exit elif if ! tar cvf $CD-system_backup.tar apache-*.tar.gz && gzip -v9 $CD-system_backup.tar; then echo "FATAL ERR 3" exit elif scp $CD-system_backup.tar.gz user@10.0.0.1:/.
if this is run in a cronjob, any output will be mailed to the user who owns the cronjob. this user's email can be aliased via /etc/aliases to another account if needed.
I need to write a script that I will manually start (or a cron job in future) but I need it to do a number of things in order one after another. How do i do that so everything gets dont as the steps depend on each other.
Example:
cd /system_backups/
# timestamp in YYYY-MM-DD format TIME_STAMP="$(date +%Y-%m-%d)"
# ip address of remote host to scp files to REMOTE_IP="10.0.0.X"
# comma separated list of users to receive notification alerts $AUDIENCE="user1@example.com,user2@example.com" cd /system_backups
tar cvf apache-conf.tar /etc/httpd/conf/* gzip -v9 apache-conf.tar
tar cvfz apache-conf.tar.gz /etc/httpd/conf if [[ $? -ne 0 ]] then echo "Failed" | /bin/mail -s "Apache configuration directory backup failed" # no more execution since previous step did not succeed exit -1 fi
tar cvf apache-data.tar /var/www/* gzip -v9 apache-data.tar
tar cvfz apache-data.tar.gz /var/www if [[ $? -ne 0 ]] then echo "Failed" | /bin/mail -s "Apache data directory backup failed" exit -1 fi
then last step... tar cvf <current_date>-system_backup.tar <all> the gzip files above gzip -v9 <current_date>-system_backup.tar
tar cvfz ${TIME_STAMP}-system-backup.tar.gz apache-conf.tar.gz apache-data.tar.gz if [[ $? -ne 0 ]] then echo "Failed" | /bin/mail -s "Apache system backup failed" exit -1 fi
scp <current_date>-system_backup.tar.gz user@10.0.0.1:/.
scp ${TIME_STAMP}-system-backup.tar.gz user@10.0.0.1:/ if [[ $? -ne 0 ]] then echo "Failed" | /bin/mail -s "Apache scp transfer failed" exit -1 else echo "Success" | /bin/mail -s "Apache system backup success" $AUDIENCE exit 0 fi
My questions:
- How do I execute each statement and make sure subsequent statements
are not executed until the previous is done?
if [[ $? -ne 0 ]] statement does that for you
- How do I error check so if a step fails the script stops?
exit X
- when I want to run the scp to send over the file to another machine
for safety, how can I have it know the password to the machine I am scp'ing to?
You need to setup password-less login first Try http://blogs.translucentcode.org/mick/archives/000230.html
Can anyone provide insight for me?
This is a very basic script. You can use functions for error checking and make the code smaller etc.
-Jason
Hope this helps.
On Thu, May 07, 2009 at 10:12:59PM -0700, Jason Todd Slack-Moehrle wrote:
Hi All,
I need to write a script that I will manually start (or a cron job in future) but I need it to do a number of things in order one after another. How do i do that so everything gets dont as the steps depend on each other.
In the script, before any command, add the line: set -e
That'll cause the shell to abort if a command fails.
On Thu, May 7, 2009 at 10:12 PM, Jason Todd Slack-Moehrle mailinglists@mailnewsrss.com wrote:
Hi All,
I need to write a script that I will manually start (or a cron job in future) but I need it to do a number of things in order one after another. How do i do that so everything gets dont as the steps depend on each other.
If you're asking these kinds of questions, a basic primer on how to use bash, and shell scripting in general, seems to be in order. It never hurts to have good manuals around if you are not thoroughly familiar with a particular paradigm, and it makes for handy reference in the future. You will (and did) get lots of help here, but you'll run into more and mroe of this as you go, and these kinds of issues are frequently covered in detail in a good manual.
FTR, no, I don't use a shell manual any more. I've been writing shell and make scripts for over 20 years, and I still occasionally use the man page for details that elude me from time to time. Man has been a great tool for me, for this.
HTH
mhr
On Fri, 8 May 2009 09:58:46 -0700 MHR wrote:
If you're asking these kinds of questions, a basic primer on how to use bash, and shell scripting in general, seems to be in order. It never hurts to have good manuals around if you are not thoroughly familiar with a particular paradigm, and it makes for handy reference in the future. You will (and did) get lots of help here, but you'll run into more and mroe of this as you go, and these kinds of issues are frequently covered in detail in a good manual.
O'Reilly book:
Learning the Bash Shell - Newham & Rosenblatt
ISBN 978-0-596-00965-6
The ABS guide http://tldp.org/LDP/abs/html/ and the GNU bash manual http://www.gnu.org/software/bash/manual/ should help
On 08 May 2009, at 6:58 PM, MHR wrote:
On Thu, May 7, 2009 at 10:12 PM, Jason Todd Slack-Moehrle mailinglists@mailnewsrss.com wrote:
Hi All,
I need to write a script that I will manually start (or a cron job in future) but I need it to do a number of things in order one after another. How do i do that so everything gets dont as the steps depend on each other.
If you're asking these kinds of questions, a basic primer on how to use bash, and shell scripting in general, seems to be in order. It never hurts to have good manuals around if you are not thoroughly familiar with a particular paradigm, and it makes for handy reference in the future. You will (and did) get lots of help here, but you'll run into more and mroe of this as you go, and these kinds of issues are frequently covered in detail in a good manual.
FTR, no, I don't use a shell manual any more. I've been writing shell and make scripts for over 20 years, and I still occasionally use the man page for details that elude me from time to time. Man has been a great tool for me, for this.
HTH
mhr _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
On Fri, May 8, 2009 at 1:12 AM, Jason Todd Slack-Moehrle mailinglists@mailnewsrss.com wrote:
I need to write a script that I will manually start (or a cron job in future) but I need it to do a number of things in order one after another. How do i do that so everything gets dont as the steps depend on each other.
Example:
cd /system_backups/
tar cvf apache-conf.tar /etc/httpd/conf/* gzip -v9 apache-conf.tar
tar cvf apache-data.tar /var/www/* gzip -v9 apache-data.tar
then last step... tar cvf <current_date>-system_backup.tar <all> the gzip files above gzip -v9 <current_date>-system_backup.tar
scp <current_date>-system_backup.tar.gz user@10.0.0.1:/. etc...etc....
My questions:
- How do I execute each statement and make sure subsequent statements
are not executed until the previous is done?
- How do I error check so if a step fails the script stops?
After each command you can check the return code. For example:
touch /tmp/foo RESULT_CODE=$?
A non-zero error code will signify that some sort of error has occurred, so your script can check for these. You can then use the error code in a case statement and either print an error message then halt, or do something to resolve.
- Since I run an SMTP Server on this box can I e-mail myself from
bash the nightly results?
Yes, you can use the mail/mailx commands to send yourself an email. Here's somethign that I use:
#!/bin/sh # # $ID: $
LOG_RECIPIENTS="klowe@rccl.com" LOG_PATH=/usr/local/logs
TODAY=`${DATE} -u +%Y%m%d` SCRIPT_NAME=`basename $0|sed -e 's/.sh//'` SCRIPT_DESCRIPTION="Replace Me With Description" LOG_FILE=${LOG_PATH}/${SCRIPT_NAME}_${TODAY}.log MACHINE_NAME=`hostname`
log_header(){ echo ----------------------------------------------------- echo ${SCRIPT_DESCRIPTION} on ${MACHINE_NAME} echo SCRIPT: $0 echo ----------------------------------------------------- echo echo Script started on `date` echo }
log_footer(){ echo echo Script completed on `date` echo echo ----------------------------------------------------- echo ${SCRIPT_DESCRIPTION} on ${MACHINE_NAME} echo ----------------------------------------------------- }
mail_logs(){ for recipient in ${LOG_RECIPIENTS}; do cat ${LOG_FILE} | ${MAILX} -s "${SCRIPT_DESCRIPTION} for ${TODAY}" ${recipient} done }
exec 1>${LOG_FILE} 2>&1 log_header log_footer mail_logs
- when I want to run the scp to send over the file to another machine
for safety, how can I have it know the password to the machine I am scp'ing to?
Look up "ssh passwordless authentication" on how to generate a public/private key and install.