# tcm.com Robots.txt # Robot Exclusion File for main www -- /robots.txt # Author: James Hanna # Last Updated : Dec 30th 2008 User-Agent: * Disallow: /cgi-bin/ Disallow: /images/ Disallow: /includes/ Disallow: index-x.html Disallow: index-xx.html #Block All Images Disallow: *.gif Disallow: *.jpg Disallow: /*.gif$ Disallow: /*.jpg$ # IF YOU DO NOT WISH TO HAVE THE GOOGLE IMAGE BOT SCAN YOUR DOMAIN FOR IMAGES # THEN YOU CAN INCLUDE THE FOLLOWING IN YOUR ROBOTS FILE. # I FOUND THAT MY BANDWIDTH USAGE DROPPED BY A MASSIVE AMOUNT AFTER I GOT RID # OF THE GOOGLE IMAGE BOT. ALL I HAD WAS IMAGE HUNTERS STEALING PRODUCT SHOTS # AND NOT EVEN BROWSING THE SITE. User-agent: Googlebot-Image Disallow: /