ViewVC Help
View File
|
Revision Log
|
Show Annotations
|
Download File
|
View Changeset
|
Root Listing
root
/
www
/
trunk
/
robots.txt
Revision:
593
Committed:
Fri Apr 17 12:06:52 2015 UTC
(9 years ago) by
laffer1
Content type:
text/plain
File size:
190 byte(s)
Log Message:
dont hide css or js because googlebot makes wrong assumptions about mobile for their new filtering.
File Contents
#
Content
1
User-agent: *
2
Disallow: /cgi-bin/
3
Disallow: /shadowbox/
4
Disallow: /assets/
5
Disallow: /magus/auth/
6
Crawl-delay: 5
7
Sitemap:
http://www.midnightbsd.org/sitemap.xml.gz
8
Host: www.midnightbsd.org