Hey guys,
I'm a bit of a beginner (understatement!) with shell scripting and seek help! I am setting up our new squid proxy. Its working a treat and squidGuard is the icing on the cake. But, I am trying to write a shell script to search through our black list category's for squidGuard and remove the parsed value;
Scenario: /some/directory/where/blacklist/is/stored contains about 40-50 folders called, adult, gambling, banking, warez etc. There is a folder for each blocking category (and in each folder is two files, urls and domains, standard stuff for web filtering!)
I have a script to search through /some/directory/where/blacklist/is/stored and look at each text file trying to find someblockedsite.com and remove it. This is as far as I have got:
machine:/blacklistdir# sh ./find_files "blockedsite.com"
find_files is as follows:
#!/bin/bash rm -f ./found_files touch ./found_files find . -exec grep -q "$1" '{}' ; -print >> ./found_files i=1 while [ $i -le `wc -l ./found_files` ] ; do
grep -iv $1 $2 > $2.new ####This is where I am stuck, I have put $2 but I want to be reading each line of text from found_files? rm $2 mv $2.new $2
done
But I am totally stuck! My questions to this awesome list are: can someone help my with my script? And, if so, can you explain to me how you have achieved your solution?
Thanks a lot for reading guys n gals its greatly appreciated.
Regards,
James.
-----BEGIN GEEK CODE BLOCK----- Version: 3.1 GIT/MU/U dpu s: a--> C++>$ U+> L++> B-> P+> E?> W+++>$ N K W++ O M++>$ V- PS+++ PE++ Y+ PGP t 5 X+ R- tv+ b+> DI D+++ G+ e(+++++) h--(++) r++ z++ ------END GEEK CODE BLOCK------
For those interested here is an updated version of my script but still no luck :(
#!/bin/bash rm -f ./found_files touch ./found_files find . -exec grep -q "$1" '{}' ; -print >> ./found_files i=1 while [ $i -le `wc -l found_files` ] ; do line=`head -$i found_files | tail -1` grep -iv "$1" $line > $line.new rm $line mv $line.new $line i=`expr $i + 1` done
2008/12/23 James Bensley jwbensley@gmail.com:
Hey guys,
I'm a bit of a beginner (understatement!) with shell scripting and seek help! I am setting up our new squid proxy. Its working a treat and squidGuard is the icing on the cake. But, I am trying to write a shell script to search through our black list category's for squidGuard and remove the parsed value;
Scenario: /some/directory/where/blacklist/is/stored contains about 40-50 folders called, adult, gambling, banking, warez etc. There is a folder for each blocking category (and in each folder is two files, urls and domains, standard stuff for web filtering!)
I have a script to search through /some/directory/where/blacklist/is/stored and look at each text file trying to find someblockedsite.com and remove it. This is as far as I have got:
machine:/blacklistdir# sh ./find_files "blockedsite.com"
find_files is as follows:
#!/bin/bash rm -f ./found_files touch ./found_files find . -exec grep -q "$1" '{}' ; -print >> ./found_files i=1 while [ $i -le `wc -l ./found_files` ] ; do
grep -iv $1 $2 > $2.new ####This is where I am stuck, I have put $2 but I want to be reading each line of text from found_files? rm $2 mv $2.new $2
done
But I am totally stuck! My questions to this awesome list are: can someone help my with my script? And, if so, can you explain to me how you have achieved your solution?
Thanks a lot for reading guys n gals its greatly appreciated.
Regards,
James.
-----BEGIN GEEK CODE BLOCK----- Version: 3.1 GIT/MU/U dpu s: a--> C++>$ U+> L++> B-> P+> E?> W+++>$ N K W++ O M++>$ V- PS+++ PE++ Y+ PGP t 5 X+ R- tv+ b+> DI D+++ G+ e(+++++) h--(++) r++ z++ ------END GEEK CODE BLOCK------
James Bensley schrieb:
For those interested here is an updated version of my script but still no luck :(
#!/bin/bash rm -f ./found_files touch ./found_files find . -exec grep -q "$1" '{}' ; -print >> ./found_files i=1 while [ $i -le `wc -l found_files` ] ; do line=`head -$i found_files | tail -1` grep -iv "$1" $line > $line.new rm $line mv $line.new $line i=`expr $i + 1` done
How about following:
#/bin/sh
DIRECTORY="/root.dir.path.of.the.blacklists" PATTERN=$1
if [ "${PATTERN}" = "" ]; then echo "parameter missing - stop" exit 1 fi
if [ ! -d "${DIRECTORY}" ]; then echo "directory missing - stop" exit 1 fi
while read LINE; do echo "pattern found in ${LINE}: erasing [${PATTERN}]" sed -i "s/.*${PATTERN}.*//; /^$/d" ${LINE} done < <(grep -rl "${PATTERN}" ${DIRECTORY})
exit 0
Greetings
Alexander
Thanks all for your prompt replies. I have picked the best bits of what I received and make my own script from that.
Thanks a lot guys, a good lesson in shell scripting today! Greatly appreciated ;)
Regards,
James.
-----BEGIN GEEK CODE BLOCK----- Version: 3.1 GIT/MU/U dpu s: a--> C++>$ U+> L++> B-> P+> E?> W+++>$ N K W++ O M++>$ V- PS+++ PE++ Y+ PGP t 5 X+ R- tv+ b+> DI D+++ G+ e(+++++) h--(++) r++ z++ ------END GEEK CODE BLOCK------
On 2008-12-23, 12:15 GMT, James Bensley wrote:
find . -exec grep -q "$1" '{}' ; -print >> ./found_files
I think you can have only one action (either -exec or -print), but not sure about it. Anyway, my first instinct when things are getting to be more complicated is to use while cycle, like this:
find . |while read FILE ; do if grep "$FILE" '{}' >/dev/null 2>&1 ; then echo "$FILE" >> ./found_files fi done
Matěj
On Tue, Dec 23, 2008 at 4:34 PM, Matej Cepl mcepl@redhat.com wrote:
On 2008-12-23, 12:15 GMT, James Bensley wrote:
find . -exec grep -q "$1" '{}' ; -print >> ./found_files
I think you can have only one action (either -exec or -print), but not sure about it. Anyway, my first instinct when things are getting to be more complicated is to use while cycle, like this:
find . |while read FILE ; do if grep "$FILE" '{}' >/dev/null 2>&1 ; then echo "$FILE" >> ./found_files fi done
I don't think the '{}' construct for 'find' will work on the other side of a pipe....
mhr
On 2008-12-24, 01:22 GMT, MHR wrote:
On Tue, Dec 23, 2008 at 4:34 PM, Matej Cepl wrote:
On 2008-12-23, 12:15 GMT, James Bensley wrote:
find . -exec grep -q "$1" '{}' ; -print >> ./found_files
I think you can have only one action (either -exec or -print), but not sure about it. Anyway, my first instinct when things are getting to be more complicated is to use while cycle, like this:
find . |while read FILE ; do if grep "$FILE" '{}' >/dev/null 2>&1 ; then echo "$FILE" >> ./found_files fi done
I don't think the '{}' construct for 'find' will work on the other side of a pipe....
Of course, that's what I get from copying original text. Should read:
find . |while read FILE ; do if grep "$FILE" "$1" >/dev/null 2>&1 ; then echo "$FILE" >> ./found_files fi done
Matěj Cepl
On Wed, 2008-12-24 at 01:34 +0100, Matej Cepl wrote:
On 2008-12-23, 12:15 GMT, James Bensley wrote:
find . -exec grep -q "$1" '{}' ; -print >> ./found_files
I think you can have only one action (either -exec or -print), but not sure about it. Anyway, my first instinct when things are
Ought to work. That's the purpose of the ';' IIRC. Here's a similar test.
$ \
find .mozilla -name 'plugin*' -exec ls -ld {} ; -print
-rw------- 1 wild-bill hardtolove 1817 Feb 29 2008 .mozilla/pluginreg.dat- .mozilla/pluginreg.dat- drwxr-xr-x 2 wild-bill wild-bill 4096 Apr 2 2008 .mozilla/plugins .mozilla/plugins -rw------- 1 wild-bill hardtolove 7662 Jun 14 2008 .mozilla/firefox/pluginreg.dat- .mozilla/firefox/pluginreg.dat- -rw------- 1 wild-bill wild-bill 5225 Nov 27 07:18 .mozilla/firefox/iqa6tz9r.default/pluginreg.dat- .mozilla/firefox/iqa6tz9r.default/pluginreg.dat- -rw------- 1 wild-bill wild-bill 5817 Dec 7 08:45 .mozilla/firefox/iqa6tz9r.default/pluginreg.dat-2 .mozilla/firefox/iqa6tz9r.default/pluginreg.dat-2 -rw------- 1 wild-bill wild-bill 5817 Dec 7 08:45 .mozilla/firefox/iqa6tz9r.default/pluginreg.dat .mozilla/firefox/iqa6tz9r.default/pluginreg.dat
getting to be more complicated is to use while cycle, like this:
<snip>
Scenario: /some/directory/where/blacklist/is/stored contains about 40-50 folders called, adult, gambling, banking, warez etc. There is a folder for each blocking category (and in each folder is two files, urls and domains, standard stuff for web filtering!)
I have a script to search through /some/directory/where/blacklist/is/stored and look at each text file trying to find someblockedsite.com and remove it. This is as far as I have got:
machine:/blacklistdir# sh ./find_files "blockedsite.com"
find_files is as follows:
#!/bin/bash rm -f ./found_files touch ./found_files find . -exec grep -q "$1" '{}' ; -print >> ./found_files i=1 while [ $i -le `wc -l ./found_files` ] ; do
grep -iv $1 $2 > $2.new ####This is where I am stuck, I have put$2 but I want to be reading each line of text from found_files? rm $2 mv $2.new $2
done
Something like this?
find . -exec grep -q "$1" '{}' ; -print | while read BLOCKFILE; do grep -iv "$1" $BLOCKFILE > $BLOCKFILE.new mv -f $BLOCKFILE.new $BLOCKFILE done
-
Alexander Dalloz -
James Bensley -
John Doe -
Matej Cepl -
MHR -
William L. Maltby