ProductCart Shopping Cart Software Forums : Format Of Robots.txt

ProductCart Shopping Cart Software Forums : Format Of Robots.txt https://forum.productcart.com/ Copyright (c) 2006-2013 Web Wiz Forums - All Rights Reserved. Fri, 10 Apr 2026 18:59:15 +0000 Sun, 15 Mar 2009 07:14:27 +0000 http://blogs.law.harvard.edu/tech/rss Web Wiz Forums 12.04 360 https://forum.productcart.com/RSS_post_feed.asp?TID=688 <![CDATA[ProductCart Shopping Cart Software Forums]]> https://forum.productcart.com/forum_images/pc_logo_50.png https://forum.productcart.com/ <![CDATA[Format Of Robots.txt : We do not believe this is necessary....]]> https://forum.productcart.com/format-of-robots-txt_topic688_post9263.html#9263 Author: ProductCart
Subject: 688
Posted: 15-March-2009 at 7:14am

We do not believe this is necessary. A search engine spider will only locate and index pages that you (or the shopping cart itself) are linking to, and those are only the storefront pages.

We don't see any reason why some of the storefront pages should be excluded from the index. For example, why would you want to prevent the search pages from being indexed? A hardcoded link to a price-range search, for instance, could certainly be indexed. And why preventing product reviews from being found and indexed? Product reviews, which are text-heavy, typically rank well in search engines.

We honestly don't see any reason for doing this. If there are some that we are not seeing, certainly discuss them in this thread.

Even if you decide to use robots.txt, the renamed "pcadmin" folder should absolutely not be included in the robots.txt file: if you do so, you are making its path known to everyone. Hiding the path to the ProductCart Control Panel is a simple and strong security measure (beyond the Control Panel's built-in authentication system), and it should be implemented on all ProductCart-powered stores.]]> Sun, 15 Mar 2009 07:14:27 +0000 https://forum.productcart.com/format-of-robots-txt_topic688_post9263.html#9263 <![CDATA[Format Of Robots.txt : I would like to revive this question...]]> https://forum.productcart.com/format-of-robots-txt_topic688_post9255.html#9255 Author: carstone
Subject: 688
Posted: 14-March-2009 at 7:04pm

I would like to revive this question again, because I think there has been folder structure changes since this was last discussed. What is a good robots.txt file for PC. Also, should we NOT list some folders, like our ADMIN folder, in the robots.txt file because it shows a savvy surfer where the sensitive folders are??? I can go to anyone's site and type www.domain.com/robots.txt and learn a lot about their websites structure. Is that a real problem?]]> Sat, 14 Mar 2009 19:04:12 +0000 https://forum.productcart.com/format-of-robots-txt_topic688_post9255.html#9255 <![CDATA[Format Of Robots.txt : Thanks for sharing this John! I...]]> https://forum.productcart.com/format-of-robots-txt_topic688_post2182.html#2182 Author: Stuck
Subject: 688
Posted: 18-April-2007 at 9:54pm

Thanks for sharing this John!

I am just now creating our robots.txt file and this was a big time saver.

]]> Wed, 18 Apr 2007 21:54:39 +0000 https://forum.productcart.com/format-of-robots-txt_topic688_post2182.html#2182 <![CDATA[Format Of Robots.txt : This is what I use, what about...]]> https://forum.productcart.com/format-of-robots-txt_topic688_post2129.html#2129 Author: watercrazed
Subject: 688
Posted: 01-April-2007 at 8:05pm

This is what I use, what about the rest of you?

anything I am missing?

Disallow: /ProductCart/calendar/
Disallow: /ProductCart/cartdata/
Disallow: /ProductCart/htmleditor/
Disallow: /ProductCart/pcadmin/
Disallow: /ProductCart/UPSLicense/
Disallow: /ProductCart/database/
Disallow: /ProductCart/setup/
Disallow: /ProductCart/pc/New
Disallow: /ProductCart/pc/catalog/
Disallow: /ProductCart/pc/advSrca.asp
Disallow: /ProductCart/pc/Affiliate
Disallow: /ProductCart/pc/advSearch_h.asp?
Disallow: /ProductCart/pc/allreviews.asp?
Disallow: /ProductCart/pc/checkout.asp
Disallow: /ProductCart/pc/custPref.asp
Disallow: /ProductCart/pc/custva.asp
Disallow: /ProductCart/pc/Custva.asp?
Disallow: /ProductCart/pc/custwl.asp
Disallow: /ProductCart/pc/default.asp
Disallow: /ProductCart/pc/tellafriend.asp?
Disallow: /ProductCart/pc/ViewCart.asp
Disallow: /productcart/pc/viewCat_h.asp?ProdSort

]]> Sun, 01 Apr 2007 20:05:50 +0000 https://forum.productcart.com/format-of-robots-txt_topic688_post2129.html#2129 <![CDATA[Format Of Robots.txt : The Web Robots PageNoone's...]]> https://forum.productcart.com/format-of-robots-txt_topic688_post2038.html#2038 Author: NWilliams
Subject: 688
Posted: 10-March-2007 at 7:09pm

The Web Robots Page

Noone's going to do it for you, so the best solution is learn how it works. You'll find syntax information on the "Robots Exclusion" page.
]]> Sat, 10 Mar 2007 19:09:19 +0000 https://forum.productcart.com/format-of-robots-txt_topic688_post2038.html#2038 <![CDATA[Format Of Robots.txt : How do you setup a robots.txt...]]> https://forum.productcart.com/format-of-robots-txt_topic688_post2026.html#2026 Author: cart
Subject: 688
Posted: 08-March-2007 at 7:39am

How do you setup a robots.txt to correctly spider only the productcart files necessary for good page ranks but to ignore the other asp pages?

]]> Thu, 08 Mar 2007 07:39:47 +0000 https://forum.productcart.com/format-of-robots-txt_topic688_post2026.html#2026