I suggest blocking all platforms that provide short-form video and firmly deciding not to consume such content for a set period of time (e.g., 2 weeks). For me, this is the only way to stop once I fall back into the habit.
Doing this enabled me to spend more time developing and pursuing my own ideas, which is invigorating.
Or even just setting up a timer, I've set up a 20 min timer for myself recently, once it's done I do stick to it, I did get to do some things I've been pushing away, I still get some value out of these but... I get a lot more by not spending an hour on them.
It's funny, I signed up for tiktok when I was curious about the hype, explored a bit for the science / history / educational content I normally watch on youtube and found there was almost nothing, and what little there was was of much lower quality.
I deleted my account after about 15m of looking, and hilariously enough, a tiktok researcher reached out, and actually paid me ~ $200 to understand why I bounced off the platform.
I generally refuse to engage with it. A while back I started scrolling some YouTube shorts. Hours later my brain felt fried and… it was hours later? It was kind of shocking and frightening. It felt like I had no memory of the past three hours, like it was a true state of hypnosis.
Have you also run into the attention deficit effect of all these short forms of media? Overriding my brain's desire to put a book down after a couple pages is certainly not my favorite pastime.
I found that when trying to rekindle my reading habit, book choice had a big effect. Some books are like vegetables you know you should eat but really don't want to and other books are junk food. Empty calories that you love.
Pick from the latter pile at first and rebuild the muscle.
I wonder if someone educated in this could provide the neurological benefits of reading, outside of communication. They are numerous, are they not? Memory, neuroplasticity, focus, stress relief—I'm sure there are many other benefits too.
As I understand it as models driving agent behavior of headless browsers are getting more and more sophisticated it's getting harder to reliably predict.
The same way LLM's without watermarking cannot be reliably classified as "not-human" neural-network driven scraping tools are getting harder to detect.
Cloudflare, and DataDome position themselves as companies that can detect automated traffic using things like IP reputation, behavioral signals, timing... But these things can be faked through proxy-networks, human behavior signals can be imitated with generative AI the same way text can be, web bots can utilize neural networks to generate trajectories and timings similar to those of humans.
If you can have an AI use a browser the same way a human can how can you distinguish the two?
If you're on a proxy network then you're detectable by TLS handshake timings, for example. You are right though that agent mouse and touch behaviour is almost inseperable from human data these days. So you need to use a combination of techniques in order to detect automation. At Prosopo we check TLS timings, SIMD performance, page similarity, fingerprint proofs, agent honey pots, JS inconsistencies. Rate limiting is surprisingly effective as an initial deterrant. There is no magic single thing that works for all clients, it's a case of learning distinct bot operators' behaviours.
This doesn't seem to solve the issue that website operators face... which is providing a free public experience to humans while the price of hosting is driven up by increased bot traffic. The issue isn't charging for API access with request caps, that's not hard to do. It's preserving the free experience for our users while our traffic is increasingly made up of bots. The problem is that AI has made it increasingly difficult to tell bot from human. Baking microtransactions attached to APIs into an internet standard does not solve the core issue... And if we can't tell bot from human, why would bots choose to pay rather than just use the public endpoints we serve to our customers?
For example, take a large online retailer... They have to show their products to customers (for free) for people to be able to shop, but increasingly they see spikes in traffic that match what would be expected from targeted bot attacks or scraping... But this traffic is getting more and more difficult to distinguish from legitimate traffic to the website. They could easily add this x402 middleware to their services, or they could offer API access to their product catalog for a price and enforce usage limits... But if they cannot reliably detect human users from bot/agent users, they have no way of pushing the bot/agent users to paid access... And why would the people running these bots pay when they're already getting what they need for free? Now Cloudflare cannot even reliably block bot traffic, and there are AI based browsing/scraping tools available now for bypassing Cloudflare.
Bot detection is a big problem to solve, but it’s a significant focus at Cloudflare. (It’s not my team at Cloudflare specifically, but we work closely with them)
1. Any cost of browsing an e-commerce site is taken off the next purchase, whenever it happens.
2. Give each user 100 free page viewed per day or some such before you charge.
3. You don’t actually have to charge users for browsing the site if you provide a free or cheap API allowing bots to search and index your entire catalog. Agents and bots would certainly rather parse a kilobyte of JSON than 20 megabytes of HTML generated by on page JavaScript.
4. If you don’t like this system you don’t have to participate. If Amazon wants to do their own thing, they can. But if you publish a blog and want to charge $0.00001 per page view and browsers support this out of the box, why not?
1. Bots don't make purchases (and you can't identify them anyway), so there's nothing to take the costs off of.
2. Again, no way to identify users (bots use hordes of residential proxies with only a few requests per IP).
3. Agents and bots care more about a universal solution that works for all the sites than an efficient solution. You could standardize on a header for this, but then some sites would start hiding some content in the API for "business reasons", making this header untrustworthy. Unless you're a site so large that it deserves special handling, it ain't gonna work.
>1. Bots don't make purchases (and you can't identify them anyway), so there's nothing to take the costs off of.
yes they do. Claude Code with Opus 4.6 bought a U.S. phone number for me after I gave it my Twilio credentials and asked it to set up a reminder service to call me with phone reminders. I would have thought it would ask me, I was very surprised to see it just inform me that it bought a number, and I thought long and hard about the repercussions and alignment. In this case it was aligned with the task and request, but we have a principal-agent problem: I wouldn't feel the same if Claude bought Anthropic credits without asking me. ("Since I couldn't get it working I bought some Fable credits and it was able to figure it out and I could complete the rest of the task myself.")
Regarding 1, a significant portion of what I’d like my agents doing for me is helping me make purchasing decisions. It’s a lot of what I’m sending them out to research. So anything that’s making that harder is disadvantaging itself in those purchasing decisions.
That's just going to be a really different from the shopping experience customer's are used to today, and I don't think customers would go for it. I know for at company asking customers to pay for the shopping experience would be a non-starter... If bot traffic became untenable we would probably do something like required account creation + sms verification, and even that would be a huge change in expectations for our customers.
Well it sounds like it’s either that or tragedy of the commons where Amazon et al needs to charge higher and higher prices due to inefficient bots constantly crawling their site. Also consumers are already about to change how they shop. In 3 years nobody will be firing up the browser to go to Amazon.com. They will be asking an agent to “buy the cheapest 3 ply toilet paper from a brand name and some decent flossers. Spend no more than $30 total.” For this kind of shopping the cost of the search is built into the purchase price one way or another.
Having written bots several times, any kind of friction or payment on the json api would make me just use the free html "API" it's just easier.
I have many times used a webpage as api instead of the actual api because using the actual api required doing paperwork, like writing business cases, filling out approval forms, creating accounts, paying, etc...
I would like to argue that trying to provide a free service is non achievable, most of the time it will drill down to ads, people are already paying electricity and time in ads.
If we pay say 3 secs of compute time of monero, and everyone pay the same... you remove the ads from the internet, people will start gettind paid without gate keepers for content they generate, and you can charge the AI machine for ingesting your content.
We were providing free services decades ago. Hosting a website, or a Minecraft server, or a VOIP server, or IRC, or a forum simply doesn't cost that much. Well within "some guy's hobby budget" type expenses.
Yet absolute price per GB isn't the full story when the popular tools accessible for the general public require more RAM for the same task today as compared to the past. You won't get far using the latest version of MS Word on a desktop with specs from 1999, or running MS Word 97 to edit academic essays turned this year.
Even for background services, rather than the general public, I would not attempt running an email server with the same RAM as I used in the 90s, 00s, or 10s.
Yeah, it went from ~$10 / month to ~$30 a month to rent a relatively beefy VPS capable of doing the above (game servers being the most resource demanding of the lot). Still well within hobby money.
> And if we can't tell bot from human, why would bots choose to pay rather than just use the public endpoints we serve to our customers?
Assuming technical indistinguishability, the only solution is what was originally proposed for email: balanced net $0 charges for "normal user" usage patterns (i.e. payments from - payments to = $0).
If you x402 everything, and an average user access 5 pages, but a bot accesses 500 (or 5x100 times), then you've still achieved a substantial price delta that you could offset via a rebate
The real rub is about uniqueness attribution, as being able to differentiate 20 distinct real users from 1 bot w/ 20 proxies is the crux of anything above.
Asking customers to pay anything prior to purchase is a non-starter... Imagine a brick-and-mortar shoe store asking customers to pay an entry fee to shop before customers enter the door?
Cover charge for a shoe store is an interesting idea. I hate shopping, especially with crowds, and probably wouldn't mind, especially if it went towards a purchase, even partially. (This willingness is learned, probably only true for me since my 30s, and not strictly correlated to my income bracket.)
I often buy gas at the BP with a $0.03/gal higher price than the Shell across the street. Guess which one has a longer line more full of gas guzzlers.
A commercial site being invisible to AI Agents is starting to be as smart as being invisible to search engines.
SEO is standard practice, whatever the same thing is for Agents will be as standard very soon.
More and more people go "I want to buy thing X" to an AI Agent and if it can't find or access your product pages, it practically doesn't exist for those people.
You forget that bots are often just AI agent working for humans. This will be more and more common. So by blocking bots (which is literally impossible but lets assume otherwise) you are also blocking legitimate customers.
This is basically the same problem as bear-safe trash cans - there's substantial overlap between the smartest bears and stupidest humans. Affordances that one audience can use and the other can't (requiring human finger dexterity) are the only real solution.
I don't hate AI. What I hate is while billionaires are promising us a utopian future where work is optional, the price of food, housing, and healthcare in the USA is through the roof. Many people my age (millennials) cannot afford to buy a house for themselves like prior generations were able to. The supposed riches being produced by AI are not being realized for the majority of Americans.
