16 Jan
2010
16 Jan
'10
11:33 p.m.
On Sat, 2010-01-16 at 14:18 -0800, R-Elists wrote:
quite frankly, there are many orgs out there that dont follow this anyways,right?
Since robots.txt is a "suggestion" and .htaccess is actually enforced, I use a simple robots.txt like this:
User-agent: * Disallow:
and put the bad guys into .htaccess.
--
MELVILLE THEATRE ~ Melville Sask ~ http://www.melvilletheatre.com