[CentOS] Optimizing grep, sort, uniq for speed
m.roth at 5-cent.us
m.roth at 5-cent.us
Thu Jun 28 19:37:46 UTC 2012
Sean Carolan wrote:
> Thank you Mark and Gordon. Since the hostnames I needed to collect
> are in the same field, at least in the lines of the file that are
> important. I ended up using suggestions from both of you, the code is
> like this now. The egrep is there to make sure whatever is in the 9th
> field looks like a domain name.
>
> for host in $(awk '{ print $9 }' ${TMPDIR}/* | egrep
> "[-\.0-9a-z][-\.0-9a-z]*.com" | sort -u); do
> HOSTS+=("$host")
> done
>
*sigh*
awk is not "cut". What you want is
awk '{if (/[-\.0-9a-z][-\.0-9a-z]*.com/) { print $9;}}' | sort -u
No grep needed; awk looks for what you want *first* this way.
mark, who learned awk in the very early nineties, writing
100-200 line awk scripts....
More information about the CentOS
mailing list