View Full Version : MSNBot?

07-27-2004, 05:48 PM
I have had this spider on my site for last 17 hours...started at 1:59AM and is still going. IP address is which when I looked up in WHOIS found out it belongs to MSN. I have the spider on in admin and forced cookies. All other bots etc are in and out very quickly. This one has pulled 4300+ pages so far and over 135MB bandwidth. Is this normal? Any thoughts?

07-27-2004, 09:11 PM
After 21 hours, 5000+ pages and 150+MB, I emailed Microsoft and questioned the situation. Within a few minutes, I got an email back appologizing and a couple minutes later the bot stopped. What I don't know is whether the bot was really digging that deep or whether it was caught in some loop on the site. Hopefully Microsoft will answer...if they do I will pass it along so we all know.

07-28-2004, 07:11 AM
Not normal, but if you are using session ID's instead of cookies, the spider could have been caught spawning all kinds of new sessions, which it thinks are unique pages...

07-28-2004, 07:21 AM
correct settings then for cookies...forced cookies true? and spider true? should have stop the looping