Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I have a personal site on a shared host that was recently suspended for excessive CPU use caused by incessant AI bot crawling. I don't have a problem with crawling per se, there's just so many and they're so bad at it.


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: