Hacker News
new
|
past
|
comments
|
ask
|
show
|
jobs
|
submit
login
wareya
on March 30, 2015
|
parent
|
context
|
favorite
| on:
Inventing Favicon.ico
The whole point of robots.txt is that there are pages which people may hit that bots can't. What are you on?
Consider applying for YC's Fall 2026 batch!
Applications
are open till July 27.
Guidelines
|
FAQ
|
Lists
|
API
|
Security
|
Legal
|
Apply to YC
|
Contact
Search:
