Robots.txt and Supplemental Results

Message Topic Search Topic Options Post Reply Create New Topic Printable Version Translate Topic

   Here are some of the area's I have disallowed in my robots.  These are areas that either I did not feel SE needed access to and somehow they showed up in my logs.  Or they were the cause of contentless pages getting indexed, or causing duplicate content problems with google.

User-agent: *

Disallow: /ProductCart/calendar/
Disallow: /ProductCart/htmleditor/
Disallow: /ProductCart/ii/
Disallow: /ProductCart/pcadmin/
Disallow: /ProductCart/UPSLicense/
Disallow: /ProductCart/setup/
Disallow: /ProductCart/pc/New
Disallow: /ProductCart/pc/images/
Disallow: /ProductCart/pc/catalog/
Disallow: /ProductCart/pc/advSrca.asp
Disallow: /ProductCart/pc/Affiliate
Disallow: /ProductCart/pc/advSearch_h.asp?
Disallow: /ProductCart/pc/prv_allreviews.asp?
Disallow: /ProductCart/pc/contact.asp
Disallow: /ProductCart/pc/checkout.asp
Disallow: /ProductCart/pc/custPref.asp
Disallow: /ProductCart/pc/custva.asp
Disallow: /ProductCart/pc/Custva.asp?
Disallow: /ProductCart/pc/custwl.asp
Disallow: /ProductCart/pc/default.asp
Disallow: /ProductCart/pc/tellafriend.asp?

Author	Message Topic Search Topic Options Post Reply Create New Topic Printable Version Translate Topic
watercrazed Members Profile Send Private Message Find Members Posts Add to Buddy List Groupie Joined: 31-December-2005 Location: United States Status: Offline Points: 0	Post Options Post Reply Quote watercrazed Report Post Thanks(0) Quote Reply Topic: Robots.txt and Supplemental Results Posted: 28-March-2006 at 7:09pm
	Here are some of the area's I have disallowed in my robots. These are areas that either I did not feel SE needed access to and somehow they showed up in my logs. Or they were the cause of contentless pages getting indexed, or causing duplicate content problems with google. User-agent: * Disallow: /ProductCart/calendar/ Disallow: /ProductCart/htmleditor/ Disallow: /ProductCart/ii/ Disallow: /ProductCart/pcadmin/ Disallow: /ProductCart/UPSLicense/ Disallow: /ProductCart/setup/ Disallow: /ProductCart/pc/New Disallow: /ProductCart/pc/images/ Disallow: /ProductCart/pc/catalog/ Disallow: /ProductCart/pc/advSrca.asp Disallow: /ProductCart/pc/Affiliate Disallow: /ProductCart/pc/advSearch_h.asp? Disallow: /ProductCart/pc/prv_allreviews.asp? Disallow: /ProductCart/pc/contact.asp Disallow: /ProductCart/pc/checkout.asp Disallow: /ProductCart/pc/custPref.asp Disallow: /ProductCart/pc/custva.asp Disallow: /ProductCart/pc/Custva.asp? Disallow: /ProductCart/pc/custwl.asp Disallow: /ProductCart/pc/default.asp Disallow: /ProductCart/pc/tellafriend.asp?
	John massagers, heat therapy, buckwheat pillows and more