Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Make some of your content, which is invisible to normal users, so exceptionally toxic to the bots that they will begin to avoid your site by choice.


How can doing this cannot affect your SEO?


If SEO's a concern, then there's nothing you can do about the LLM bit. Either put the website behind a login, or deal with robots sucking everything up.

Pre-LLMs, there were already many websites showing scraped versions of other people's sites, and ranking higher in search results.


Serve different content to non search engine bots


I often read that doing that was a big no-no as Google would flag your website as cheating. But that may be a SEO legend.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: