# "Any empty [Disallow] value, indicates that all URLs can be retrieved. # At least one Disallow field needs to be present in a record." User-agent: Fasterfox Disallow: / User-Agent: OmniExplorer_Bot Disallow: / User-agent: * Disallow: /ads Disallow: /captcha Disallow: /comments Disallow: /css Disallow: /dating/profile Disallow: /email Disallow: /gateway/ui/service.html Disallow: /ibp Disallow: /im Disallow: /js # Disallow: /jobs/job/ads_apply # Disallow: /jobs/tracking/link_redirect.html Disallow: /login Disallow: /maintainer Disallow: /misc Disallow: /new_index.html Disallow: /nfo Disallow: /notes Disallow: /points Disallow: /professional/user_search Disallow: /relationships Disallow: /restricted Disallow: /social/user_search Disallow: /user_search Disallow: /videos Disallow: */ajax/ Disallow: */services/ Disallow: *?indexable= Disallow: *&indexable= Disallow: *?sort= Disallow: *&sort= Disallow: *?xrds= Disallow: *&xrds= Allow: /email/newsletter.html User-Agent: Mediapartners-Google Disallow: Sitemap: http://www.blackplanet.com/sitemap.html Sitemap: http://www.blackplanet.com/dating/sitemap.html Sitemap: http://www.blackplanet.com/forums/sitemap.html Sitemap: http://www.blackplanet.com/groups/sitemap.html Sitemap: http://www.blackplanet.com/jobs/sitemap.html Sitemap: http://www.blackplanet.com/news/sitemap.html Sitemap: http://www.blackplanet.com/professional/groups/sitemap.html Sitemap: http://www.blackplanet.com/your_page/sitemap.html