a little out of my comfort zone and have practically gotten what I want but awk seems determined to send a message via std error which is problematic and annoying. Basically trying to get a list of virtual host names from nginx config files like this:
$ awk -F" " '/./ { if ( match ( "^server_name$", $2 ) ) print $1 }' /opt/nginx/sites/*.conf \ | grep -v server_name | grep -v ';' | grep -v '}'
and the list of virtual host names is perfect except that I get this line first...
awk: (FILENAME=/opt/nginx/sites/ids.conf FNR=55) fatal: Unmatched ( or (: /($host/
and I can't seem to dismiss it and I definitely don't want it in my data scrape.
Anyone know how to prevent this? (yes, there are lines in the file that say 'if ($host = something) {'
On Thu, Dec 6, 2012 at 2:07 PM, Craig White craig.white@ttiltd.com wrote:
a little out of my comfort zone and have practically gotten what I want but awk seems determined to send a message via std error which is problematic and annoying. Basically trying to get a list of virtual host names from nginx config files like this:
$ awk -F" " '/./ { if ( match ( "^server_name$", $2 ) ) print $1 }' /opt/nginx/sites/*.conf \ | grep -v server_name | grep -v ';' | grep -v '}'
and the list of virtual host names is perfect except that I get this line first...
awk: (FILENAME=/opt/nginx/sites/ids.conf FNR=55) fatal: Unmatched ( or (: /($host/
and I can't seem to dismiss it and I definitely don't want it in my data scrape.
Anyone know how to prevent this? (yes, there are lines in the file that say 'if ($host = something) {'
I'd start with perl instead of awk and come up with something that didn't need the greps to clean it up. But, if all you want is to discard stderr, won't sticking a 2>/dev/null before your first pipe take care of that courtesy of shell redirection?
You rang?
Craig White wrote:
a little out of my comfort zone and have practically gotten what I want but awk seems determined to send a message via std error which is problematic and annoying. Basically trying to get a list of virtual host names from nginx config files like this:
$ awk -F" " '/./ { if ( match ( "^server_name$", $2 ) ) print $1 }' /opt/nginx/sites/*.conf \ | grep -v server_name | grep -v ';' | grep -v '}'
Why are you doing all that piping and grepping? And the -F" " confuses me...oh, I see. First, whitespace is the default field separator in awk. Then, are you asking if there's a line with a "." in it, or just any non-whitespace? If the latter... mmm, I see, you *really* don't understand awk.
awk -f '{if ( $1 ~ /server_name/ ) {\ server = $2;\ gsub(/;|}/,"",server);\ print server; } }' <snip> mark
Definitely have little to no understanding of awk but…
/./ suppresses empty lines (records in awk speak)
the gsub looks interesting but your code just tosses syntax errors
and yes Les, the >2 /dev/null definitely redirected the awk squawk to where it belonged
Craig
On Dec 6, 2012, at 1:34 PM, m.roth@5-cent.us wrote:
You rang?
Craig White wrote:
a little out of my comfort zone and have practically gotten what I want but awk seems determined to send a message via std error which is problematic and annoying. Basically trying to get a list of virtual host names from nginx config files like this:
$ awk -F" " '/./ { if ( match ( "^server_name$", $2 ) ) print $1 }' /opt/nginx/sites/*.conf \ | grep -v server_name | grep -v ';' | grep -v '}'
Why are you doing all that piping and grepping? And the -F" " confuses me...oh, I see. First, whitespace is the default field separator in awk. Then, are you asking if there's a line with a "." in it, or just any non-whitespace? If the latter... mmm, I see, you *really* don't understand awk.
awk -f '{if ( $1 ~ /server_name/ ) {\ server = $2;\ gsub(/;|}/,"",server);\ print server; } }'
<snip> mark
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Please stop top posting, Craig.
Craig White wrote:
On Dec 6, 2012, at 1:34 PM, m.roth@5-cent.us wrote:
You rang?
Craig White wrote:
a little out of my comfort zone and have practically gotten what I want but awk seems determined to send a message via std error which is problematic and annoying. Basically trying to get a list of virtual host names from nginx config files like this:
$ awk -F" " '/./ { if ( match ( "^server_name$", $2 ) ) print $1 }' /opt/nginx/sites/*.conf \ | grep -v server_name | grep -v ';' | grep -v '}'
Why are you doing all that piping and grepping? And the -F" " confuses me...oh, I see. First, whitespace is the default field separator in awk. Then, are you asking if there's a line with a "." in it, or just any non-whitespace? If the latter... mmm, I see, you *really* don't understand awk.
awk -f '{if ( $1 ~ /server_name/ ) {\ server = $2;\ gsub(/;|}/,"",server);\ print server; } }'
<snip> mark
Definitely have little to no understanding of awk but…
/./ suppresses empty lines (records in awk speak)
Oh. Never used it. Wrote a *lot* of *long* awk scripts over the years. But it doesn't matter - looking for $1 to be == server_name will only pick those lines.
the gsub looks interesting but your code just tosses syntax errors
I see I didn't out \ on the lines, which I wrote that way only to make it more readable.
and yes Les, the >2 /dev/null definitely redirected the awk squawk to where it belonged
Ok, I just d/l an nginx.conf file from http://wiki.nginx.org/FullExample and ran the following script on it: { if ( $1 ~ /server_name$/ ) { server = $2; gsub(/;|}/,"",server); print server; } }
and my o/p was $ awk -f nginx.awk nginx.conf domain1.com domain2.com big.server.com
mark
On Dec 6, 2012, at 1:59 PM, m.roth@5-cent.us wrote:
Please stop top posting, Craig.
Craig White wrote:
On Dec 6, 2012, at 1:34 PM, m.roth@5-cent.us wrote:
You rang?
Craig White wrote:
a little out of my comfort zone and have practically gotten what I want but awk seems determined to send a message via std error which is problematic and annoying. Basically trying to get a list of virtual host names from nginx config files like this:
$ awk -F" " '/./ { if ( match ( "^server_name$", $2 ) ) print $1 }' /opt/nginx/sites/*.conf \ | grep -v server_name | grep -v ';' | grep -v '}'
Why are you doing all that piping and grepping? And the -F" " confuses me...oh, I see. First, whitespace is the default field separator in awk. Then, are you asking if there's a line with a "." in it, or just any non-whitespace? If the latter... mmm, I see, you *really* don't understand awk.
awk -f '{if ( $1 ~ /server_name/ ) {\ server = $2;\ gsub(/;|}/,"",server);\ print server; } }'
<snip> mark
Definitely have little to no understanding of awk but…
/./ suppresses empty lines (records in awk speak)
Oh. Never used it. Wrote a *lot* of *long* awk scripts over the years. But it doesn't matter - looking for $1 to be == server_name will only pick those lines.
the gsub looks interesting but your code just tosses syntax errors
I see I didn't out \ on the lines, which I wrote that way only to make it more readable.
and yes Les, the >2 /dev/null definitely redirected the awk squawk to where it belonged
Ok, I just d/l an nginx.conf file from http://wiki.nginx.org/FullExample and ran the following script on it: { if ( $1 ~ /server_name$/ ) { server = $2; gsub(/;|}/,"",server); print server; } }
and my o/p was $ awk -f nginx.awk nginx.conf domain1.com domain2.com big.server.com
---- not that I was looking for someone to write it for me but that works only when the nginx.conf looks like
server_name domain1.com domain2.com big.server.com;
which I actually didn't need to use awk to parse as I already handled those instances just fine with grep/sed
but I have some conf files which look like
server_name { domain1.com domain2.com big.server.com } ;
and that forced me into looking at alternative methods - hence awk
but your program gives me the following output…
$ awk -f nginx.awk /opt/nginx/sites/ids.conf
$
Craig
Craig White wrote:
On Dec 6, 2012, at 1:59 PM, m.roth@5-cent.us wrote:
Craig White wrote:
On Dec 6, 2012, at 1:34 PM, m.roth@5-cent.us wrote:
You rang?
Craig White wrote:
<snip>
Definitely have little to no understanding of awk but…
Ok, I just d/l an nginx.conf file from http://wiki.nginx.org/FullExample and ran the following script on it: { if ( $1 ~ /server_name$/ ) { server = $2; gsub(/;|}/,"",server); print server; } }
and my o/p was $ awk -f nginx.awk nginx.conf domain1.com domain2.com big.server.com
not that I was looking for someone to write it for me but that works only
I do awk for *fun*.... <g>
when the nginx.conf looks like
server_name domain1.com domain2.com big.server.com;
which I actually didn't need to use awk to parse as I already handled those instances just fine with grep/sed
but I have some conf files which look like
server_name { domain1.com domain2.com big.server.com } ;
and that forced me into looking at alternative methods - hence awk
but your program gives me the following output…
<snip> Of course it didn't work. I've never worked with nginx, so I could only base it on what I found. With a file like that, I'd write { if ( found == 1 && NF == 1 ) { if ( $1 ~ /}/ ) { found = 0; } else { print $1; } } else { if ( $1 ~ /server_name$/ && $2 ~ /{/ ) { found = 1; } } }
mark
On Thu, Dec 6, 2012 at 3:48 PM, Craig White craig.white@ttiltd.com wrote:
not that I was looking for someone to write it for me but that works only when the nginx.conf looks like
server_name domain1.com domain2.com big.server.com;
which I actually didn't need to use awk to parse as I already handled those instances just fine with grep/sed
but I have some conf files which look like
server_name { domain1.com domain2.com big.server.com } ;
and that forced me into looking at alternative methods - hence awk
It's kind of hard to write a generic parser in regexps, but something like this in perl should catch most of the likely layouts:
while (<>) { chomp(); next if (m/^\s*#/); #comment if ($enclosed) { if (m/}/) {$enclosed = 0;} # end found tr/};//d; #remove push @servers, split(); #anything else on line next; } else { next unless m/server_name/; if (m/{/) { $enclosed = 1;} if (m/}/) {$enclosed = 0;} #on same line? s/server_name//; tr/{};//d ; push @servers,split(); } } foreach (sort(@servers)) { print "$_\n";}
From: Craig White craig.white@ttiltd.com
but I have some conf files which look like
server_name { domain1.com domain2.com big.server.com } ;
What about something like: grep -P "^server_name {\n[^}]*" nginx.conf | grep -v "{|}"
JD
Why are you doing all that piping and grepping? And the -F" " confuses me...oh, I see. First, whitespace is the default field separator in awk. Then, are you asking if there's a line with a "." in it, or just any non-whitespace? If the latter... mmm, I see, you *really* don't understand awk.
Ok Mark very nice of you to help Craig. He does not claim to be an expert in "awk" or even competent ---> Which is obviously why he is asking for help in the first place. No need for the sarcasm and to belittle the poster. Remember lots of people looking for help will be directed to this answer and your help could be much appreciated....
Regards,
Steve
On Fri, Dec 7, 2012 at 2:23 AM, Steve Brooks steveb@mcs.st-and.ac.ukwrote:
Why are you doing all that piping and grepping? And the -F" " confuses me...oh, I see. First, whitespace is the default field separator in awk. Then, are you asking if there's a line with a "." in it, or just any non-whitespace? If the latter... mmm, I see, you *really* don't
understand
awk.
Ok Mark very nice of you to help Craig. He does not claim to be an expert in "awk" or even competent ---> Which is obviously why he is asking for help in the first place. No need for the sarcasm and to belittle the poster. Remember lots of people looking for help will be directed to this answer and your help could be much appreciated....
Regards,
Steve _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Hello there!
The best Help for awk beginners is to implement it with other tools for now if it's urgent, and debug your syntax and regex code.
Start here: http://www.unix.com/man-page/linux/1/sed/
You may also want to try and print awk to tty, start by that to debug your code until you find the solution
http://www.staff.science.uu.nl/~oostr102/docs/nawk/nawk_92.html
your main issue was, that you are asking about nginx.conf , which is not part of the stock installation, and your file can look like anything depending on custom configuration.
On 12/06/12 19:23, Steve Brooks wrote:
Why are you doing all that piping and grepping? And the -F" " confuses me...oh, I see. First, whitespace is the default field separator in awk. Then, are you asking if there's a line with a "." in it, or just any non-whitespace? If the latter... mmm, I see, you *really* don't understand awk.
Ok Mark very nice of you to help Craig. He does not claim to be an expert in "awk" or even competent ---> Which is obviously why he is asking for help in the first place. No need for the sarcasm and to belittle the poster. Remember lots of people looking for help will be directed to this answer and your help could be much appreciated....
Steve, first of all, this would have been more appropriate to email me offlist, unless you intend to try to do to me what you accuse me of doing to Craig.
Which I didn't, nor was I being sarcastic or belittling.
mark "awk hacker who doesn't get *NEAR* enough programming in these days"
On Fri, 7 Dec 2012, mark wrote:
On 12/06/12 19:23, Steve Brooks wrote:
Why are you doing all that piping and grepping? And the -F" " confuses me...oh, I see. First, whitespace is the default field separator in awk. Then, are you asking if there's a line with a "." in it, or just any non-whitespace? If the latter... mmm, I see, you *really* don't understand awk.
Ok Mark very nice of you to help Craig. He does not claim to be an expert in "awk" or even competent ---> Which is obviously why he is asking for help in the first place. No need for the sarcasm and to belittle the poster. Remember lots of people looking for help will be directed to this answer and your help could be much appreciated....
Steve, first of all, this would have been more appropriate to email me offlist, unless you intend to try to do to me what you accuse me of doing to Craig.
Sorry Mark you are right I should not have sent it to the list, apologies to you and to the list for that mistake. Wasn't having a good day and should have gone straight to bed instead of trying to do more work! Things always look different in the morning.
Regards,
Steve
-
Craig White -
John Doe -
Les Mikesell -
m.roth＠5-cent.us -
mark
-
Steve Brooks -
yonatan pingle