Recording 404's - Is it possible?

The reason that 404.asp is showing is because that is the page that is used by the SEO mod to determine which page your navigation is pointing to.  That page then performs a redirect to viewprd.asp, while causing the URL to make it appear you are on a product, category or content page.

 

 

 

However, here's the rub.  If that were the route one took, you are looking at additional load on the database (maybe not so much of a problem), and potentially massive growth - which could become an issue. 

 

If this were the route that one was to take, then you better be prepared to roll those entries into summary records at some point because in a busy store that's going to grow your database at a phenomenal pace - and potentially increasing hosting costs.   If you consider the load in terms of visitors that a site may receive, including humans, search engine spiders/bots, and stuff like ScanAlert, this becomes a lot of entries.

 

So at that point I'd almost want to write those records in the format of a typical web server log file, and work out a way to cause some off the shelf stats application to read/summarize/report on the data - and of course to clear the logs with regularity.

 

Thoughts on that?

 

Greg - are you saying that the 404.asp page is called upon every request?  After viewing the comments in the code and added code to record to a .txt file, I see entries are recorded in the text file even when I click valid pages.  So maybe I don't have a problem at all, that the reason 404.asp is the top url is beacuse of what you mentioned in your previous post.

 

Here is what appears in the .txt file when I record Request.ServerVariables("PATH_INFO")

 

Matt - I have a google webmaster account... would you see that info under Diagnostics>> Crawl errors?  I've checked that as well and nothing is showing up as an error.

 

Please let me know if I'm understanding everything correctly...  Thanks Guys!