PDA

View Full Version : Age Restricted Category



michael_s
04-16-2009, 12:12 PM
I've noticed that when using this contribution the spiders like Googlebot would not be allowed our website restricted category and are redirected with a 302.
so I've created an spider detect script to allow the spiders to the restricted categorie.

find inside application_top.php this line:

$restricted_category = (AGE_PAGE_CATEGORY);


en replace with extra function spider detect

function spiderDetect() {
$agentArray = array("ArchitextSpider", "Googlebot", "TeomaAgent",
"Zyborg", "Gulliver", "Architext spider", "FAST-WebCrawler",
"Slurp", "Ask Jeeves", "ia_archiver", "Scooter", "Mercator",
"[email protected]", "Crawler", "InfoSeek Sidewinder",
"almaden.ibm.com", "appie 1.1", "augurfind", "baiduspider",
"bannana_bot", "bdcindexer", "docomo", "frooglebot", "geobot",
"henrythemiragorobot", "sidewinder", "lachesis", "moget/1.0",
"nationaldirectory-webspider", "naverrobot", "ncsa beta",
"netresearchserver", "ng/1.0", "osis-project", "polybot",
"pompos", "seventwentyfour", "steeler/1.3", "szukacz",
"teoma", "turnitinbot", "vagabondo", "zao/0", "zyborg/1.0",
"Lycos_Spider_(T-Rex)", "Lycos_Spider_Beta2(T-Rex)",
"Fluffy the Spider", "Ultraseek", "MantraAgent","Moget",
"T-H-U-N-D-E-R-S-T-O-N-E", "MuscatFerret", "VoilaBot",
"Sleek Spider", "KIT_Fireball", "WISEnut", "WebCrawler",
"asterias2.0", "suchtop-bot", "YahooSeeker", "ai_archiver",
"Jetbot"
);

$theAgent = $_SERVER["HTTP_USER_AGENT"];

for($i=0;$i