[CentOS-docs] robots.txt

Thu Aug 6 12:14:01 UTC 2009
Ralph Angenendt <ra+centos at br-online.de>

Marcus Moeller wrote:
> I have again noticed that the wiki does not really show up in search
> results and wonder if it has any impact that robots.txt on
> wiki.centos.org is empty.
> Perhaps it should at least contain User-agent: * ?

I wonder why they ship a robots.txt at all ...



-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: <http://lists.centos.org/pipermail/centos-docs/attachments/20090806/e4598784/attachment-0002.sig>