A web application I was hired to fix was experiencing a number of unhandled exceptions. After adding error logging to capture the exceptions for review, I noticed that one of the exceptions was triggered every time a web crawler hit the site. I added a robots.txt file to the