[Midnightbsd-cvs] www [666] trunk/robots.txt: add sitemaps

[laffer1 at midnightbsd.org]

Revision: 666
          http://svnweb.midnightbsd.org/www/?rev=666
Author:   laffer1
Date:     2015-10-11 11:48:32 -0400 (Sun, 11 Oct 2015)
Log Message:
-----------
add sitemaps

Modified Paths:
--------------
    trunk/robots.txt

Modified: trunk/robots.txt
===================================================================
--- trunk/robots.txt	2015-10-11 15:36:16 UTC (rev 665)
+++ trunk/robots.txt	2015-10-11 15:48:32 UTC (rev 666)
@@ -7,5 +7,14 @@
 Disallow: /ftp/MidnightBSD/distfiles/*.asc
 Disallow: /ftp/MidnightBSD/distfiles/*.sig
 Crawl-delay: 5
-Sitemap: http://www.midnightbsd.org/sitemap.xml.gz
+Sitemap: http://www.midnightbsd.org/sitemap.xml
+Sitemap: http://www.midnightbsd.org/sitemap1.xml
+Sitemap: http://www.midnightbsd.org/sitemap2.xml
+Sitemap: http://www.midnightbsd.org/sitemap3.xml
+Sitemap: http://www.midnightbsd.org/sitemap4.xml
+Sitemap: http://www.midnightbsd.org/sitemap5.xml
+Sitemap: http://www.midnightbsd.org/sitemap6.xml
+Sitemap: http://www.midnightbsd.org/sitemap7.xml
+Sitemap: http://www.midnightbsd.org/sitemap8.xml
+Sitemap: http://www.midnightbsd.org/sitemap9.xml
 Host: www.midnightbsd.org