Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Pick an obscure UA substring like MSIE 3.0 or HP-UX. Preemptively 403 these User Agents, (you'll create your own list). Later in the week you can circle back and distill these 403s down to problematic ASNs. Whack moles as necessary.


I've tracked bots that were stuck in a loop no legitimate user would ever get stuck in (basically by circularly following links long past the point of any results). I also decided to filter out what were bots for sure, and it was over a million unique IPs.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: