Making Content Pages NON-Spiderable

Message Topic Search Topic Options Post Reply Create New Topic Printable Version Translate Topic

   We have set up our store to where when you go to the home page you get a screen that has no navigation and a login area. 

We have it set up for our dealers to be the only ones with access. Our dealers will be signed up manually through us, so they will have their user name and password ONLY through us. 

My question is: How do I make sure that the entire website is password protected? I don't want search engines to pick up on content URLs and I don't want dealers to have the ability to copy and paste the urls and send links to people. 

I have already put a robots.txt file in the root telling search engines not to spider, but I have read in several places that this doesn't always work for all search engines. 

Any ideas? Thank you in advance!

Author	Message Topic Search Topic Options Post Reply Create New Topic Printable Version Translate Topic
allensp Members Profile Send Private Message Find Members Posts Add to Buddy List Newbie Joined: 27-July-2007 Location: United States Status: Offline Points: 0	Post Options Post Reply Quote allensp Report Post Thanks(0) Quote Reply Topic: Making Content Pages NON-Spiderable Posted: 16-June-2008 at 9:54am
	We have set up our store to where when you go to the home page you get a screen that has no navigation and a login area. We have it set up for our dealers to be the only ones with access. Our dealers will be signed up manually through us, so they will have their user name and password ONLY through us. My question is: How do I make sure that the entire website is password protected? I don't want search engines to pick up on content URLs and I don't want dealers to have the ability to copy and paste the urls and send links to people. I have already put a robots.txt file in the root telling search engines not to spider, but I have read in several places that this doesn't always work for all search engines. Any ideas? Thank you in advance!

netprofits Members Profile Send Private Message Find Members Posts Add to Buddy List Certified ProductCart Developers Joined: 05-January-2006 Location: United States Status: Offline Points: 22	Post Options Post Reply Quote netprofits Report Post Thanks(0) Quote Reply Posted: 16-June-2008 at 10:08am
	The easiest solution would be to add code to the header.asp file that checks if a visitor is logged in and if not, redirect back to the login page. Since the header.asp file is included in all Product Cart pages, it would always perform the check and no one could view any page without being logged in. The code for that would be something like the following: if session("idCustomer")="0" or session("idCustomer")="" then response.redirect("default.asp") Additionally, Product Cart allows both retail and wholesale customers. You can define that when creating the customer account. You can turn of ordering for retail customers and you can also leave the retail price for products blank or 0. Then, even if someone gets to a product page, they could see products but no pricing unless they were logged in as a wholesale customer. See the User's Guide section on Managing Customers for complete details.
	NetProfits Internet Consulting Certified ProductCart Developer Our Site

allensp Members Profile Send Private Message Find Members Posts Add to Buddy List Newbie Joined: 27-July-2007 Location: United States Status: Offline Points: 0	Post Options Post Reply Quote allensp Report Post Thanks(0) Quote Reply Posted: 16-June-2008 at 10:25am
	We don't want outside customers to see any of the products - so I like the code idea you gave. Also, I saw an option in the product cart control panel to turn on SSL at login. If I had a dedicated SSL and turn it on at login - and my homepage is the login screen - would that protect the site from spiders? I am not a genius at coding, so is that the exact code I need that you gave me? If so, is there a certain place on header.asp that I need to put it? Does it need to go right after the <head> tag? Here is the beginning of my code: <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd"> <html> <head> <link type="text/css" rel="stylesheet" href="pcStorefront.css" /> <%Response.Buffer=True%> <% Set conlayout=Server.CreateObject("ADODB.Connection") conlayout.Open scDSN Set RSlayout = conlayout.Execute("Select * From layout Where layout.ID=2") Set rsIconObj = conlayout.Execute("Select * From icons WHERE id=1") %> <title>Rugged CCTV Shopping Cart</title> The rest is just divs and styles I appreciate your help so much!

netprofits Members Profile Send Private Message Find Members Posts Add to Buddy List Certified ProductCart Developers Joined: 05-January-2006 Location: United States Status: Offline Points: 22	Post Options Post Reply Quote netprofits Report Post Thanks(0) Quote Reply Posted: 16-June-2008 at 10:42am
	Try putting the following code as the very first line of header.asp: <%if session("idCustomer")="0" or session("idCustomer")="" then response.redirect("default.asp")%> You can change default.asp to whatever the name of your home page is within the PC folder, i.e. home.asp is the most common. default.asp just does a redirect to the page you define in the control panel.
	NetProfits Internet Consulting Certified ProductCart Developer Our Site

allensp Members Profile Send Private Message Find Members Posts Add to Buddy List Newbie Joined: 27-July-2007 Location: United States Status: Offline Points: 0	Post Options Post Reply Quote allensp Report Post Thanks(0) Quote Reply Posted: 16-June-2008 at 11:46am
	I am sorry but when you say first line do you mean even before this: <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd"> or after this: <link type="text/css" rel="stylesheet" href="pcStorefront.css" /> I have tried both ways and every time I try to login it acts like it is going somewhere - and then it quits. I'll keep trying though - it might be my server.

allensp Members Profile Send Private Message Find Members Posts Add to Buddy List Newbie Joined: 27-July-2007 Location: United States Status: Offline Points: 0	Post Options Post Reply Quote allensp Report Post Thanks(0) Quote Reply Posted: 16-June-2008 at 12:04pm
	I might also be doing something else wrong - this is how I have it setup: I have an index.html that is designed with just a banner image, no navigation and has the login form from product cart in the content section. When a customer logs in it takes them to productcart/pc/CustPref.asp, with all their personal info. When I add the code that you gave me to the header.asp file it messes up where the customers go when logged in. The html file does not have any include files in it at all - so it should not be connected to the header.asp file. I made another html page for non-members to be re-directed to that I put in place of the "default" page in the code. When I put the code in the header.asp file, and a customer logs in the are also taken to the non-member page. What could I be doing wrong? I appreciate your help so much!

Matt Members Profile Send Private Message Find Members Posts Add to Buddy List Moderator Group Joined: 20-July-2006 Location: United States Status: Offline Points: 73	Post Options Post Reply Quote Matt Report Post Thanks(0) Quote Reply Posted: 16-June-2008 at 12:12pm
	It doesn't matter where you put it as long as it is within <% %>. Make sure the script is not included in the page you redirecting to, otherwise you will create a continuous loop. Edited by Matt - 16-June-2008 at 12:16pm

Matt Members Profile Send Private Message Find Members Posts Add to Buddy List Moderator Group Joined: 20-July-2006 Location: United States Status: Offline Points: 73	Post Options Post Reply Quote Matt Report Post Thanks(0) Quote Reply Posted: 16-June-2008 at 12:18pm
	Ok. What is happening is that the customer is not getting logged in. They are being redirected before the login script executes.

allensp Members Profile Send Private Message Find Members Posts Add to Buddy List Newbie Joined: 27-July-2007 Location: United States Status: Offline Points: 0	Post Options Post Reply Quote allensp Report Post Thanks(0) Quote Reply Posted: 16-June-2008 at 12:24pm
	Where you trying to link to something? It is saying it is on your hard drive... I think I am doing everything correct...maybe I am just missing something that is right in front of my face. This is the code on my header.asp file: <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd"> <html> <head> <link type="text/css" rel="stylesheet" href="pcStorefront.css" /> <%if session("idCustomer")="0" or session("idCustomer")="" then response.redirect("not-member.html")%> <%Response.Buffer=True%> <% Set conlayout=Server.CreateObject("ADODB.Connection") conlayout.Open scDSN Set RSlayout = conlayout.Execute("Select * From layout Where layout.ID=2") Set rsIconObj = conlayout.Execute("Select * From icons WHERE id=1") %> <title>Rugged CCTV Shopping Cart</title> The divs.... This is the code on my not-member.html file: <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html xmlns="http://www.w3.org/1999/xhtml"> <head> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <title>Untitled Document</title> </head> <body> <p>You are not a member - please contact us if you would like to become one. <br /> <br /> You can shop our retail shop at <a href="www.rugged-cctv.comwww.rugged-cctv.com">www.rugged-cctv.com</a></p> <p>Thank you, Rugged CCTV</p> </body> </html> Finally, this is the code on my index.html file: <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html xmlns="http://www.w3.org/1999/xhtml"> <head> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <title>Untitled Document</title> <style type="text/css"> Then a bunch of styles and divs.... <form action="/productcart/pc/checkout.asp?cmode=1" method="post" name="login" id="login"> User Name <input type="text" name="LoginEmail" /> Password <input type="password" name="LoginPassword" /> <input name="PassWordExists" type="hidden" value="YES" /> <a href="JavaScript:document.login.submit();"> <input type="submit" name="Submit" id="Submit" value="Submit" /> </a> <input type="hidden" name="SubmitCO.y" value="1" /> </form> Thanks agan for your help.

Matt Members Profile Send Private Message Find Members Posts Add to Buddy List Moderator Group Joined: 20-July-2006 Location: United States Status: Offline Points: 73	Post Options Post Reply Quote Matt Report Post Thanks(0) Quote Reply Posted: 16-June-2008 at 12:31pm
	The problem is that checkout.asp does not log anyone in because the header file redirects them before the page loads. A quick workaround is to use a different header for checkout.asp that does not contain the redirect script.