| Revision: | 667 |
| Committed: | Fri Oct 16 02:32:32 2015 UTC (8 years, 6 months ago) by laffer1 |
| Content type: | text/plain |
| File size: | 807 byte(s) |
| Log Message: | don't index opengrok |
| # | Content |
|---|---|
| 1 | User-agent: * |
| 2 | Disallow: /bin/ |
| 3 | Disallow: /cgi-bin/ |
| 4 | Disallow: /html5lightbox/index.html |
| 5 | Disallow: /html5lightbox/skins/index.html |
| 6 | Disallow: /assets/ |
| 7 | Disallow: /magus/auth/ |
| 8 | Disallow: /ftp/MidnightBSD/distfiles/*.asc |
| 9 | Disallow: /ftp/MidnightBSD/distfiles/*.sig |
| 10 | Disallow: /opengrok/ |
| 11 | Crawl-delay: 5 |
| 12 | Sitemap: http://www.midnightbsd.org/sitemap.xml |
| 13 | Sitemap: http://www.midnightbsd.org/sitemap1.xml |
| 14 | Sitemap: http://www.midnightbsd.org/sitemap2.xml |
| 15 | Sitemap: http://www.midnightbsd.org/sitemap3.xml |
| 16 | Sitemap: http://www.midnightbsd.org/sitemap4.xml |
| 17 | Sitemap: http://www.midnightbsd.org/sitemap5.xml |
| 18 | Sitemap: http://www.midnightbsd.org/sitemap6.xml |
| 19 | Sitemap: http://www.midnightbsd.org/sitemap7.xml |
| 20 | Sitemap: http://www.midnightbsd.org/sitemap8.xml |
| 21 | Sitemap: http://www.midnightbsd.org/sitemap9.xml |
| 22 | Host: www.midnightbsd.org |