Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

What happened in mid-July last year? A massive spike in page views, it seems, for what looks like about 2 weeks.


A very aggressive crawler.


Hi,

  Perhaps you should post the stats, after subtracting out the bot visits, from the stats. 

  For our daily stats we do that. Also (internally) report the bot visit counts. For example yesterday we had: 

    Bot name 	No. of visits
       User-Agent=Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html),	95000*
       User-Agent=Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm),	700*
* - the last few digits have been rounded off

Edit: For display




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: