I grew up in a small rural town where math teachers were rare and good ones were nonexistent. I still remember watching Sal Khan's videos on calculus after struggling to get any grip on it from my frustratingly uninterested teacher who was forced into the position after our actual teacher retired. After I saw them I came back with a fire in me because for the first time he exposed me to the beauty of calculus.

This guy is completely wrong about Khan not inspiring students to care. Sometimes all students need is to see what caring looks like to ignite it in themselves. You can feel how much he cares just by watching him because the video is the product of all that caring, and he does it for millions of people.

But thank you to this blog author for being brave enough to call out Khan's failures, I am sure it'll help education much more than Khan did.

No good deed goes unpunished.

Excellent point, well expressed.

(Also, IMHO the sarcasm in your final sentence detracts from your comment.)

They have some significant advantages compared to (just) a classroom: (1) They set a quality floor – sometimes a teacher can be really incomprehensible, or mean, or whatever, but Khan's videos are consistently okay. (2) They are available from anywhere, without need for special social/technical infrastructure, like enrollment in school or access to a university library. (3) They are self paced, which means they can better accommodate the various interest level, motivation, and speed of different students. (A typical classroom also has some advantages: it gives social setting, a consistent schedule/structure, and human feedback.)

I'm also disappointed though, because it seems like, given production resources, there would be potential to make widely available materials that were much better than average lectures. With significant production effort (a team of subject experts and content creators doing a lot of dedicated work on each topic) I think it would be possible to make something truly great, but Khan's videos fall so far short of my imagination. For any particular topic, there are typically other freely available resources which I think are better.

I've also been disappointed because many people have hyped the site as being extraordinarily amazing, but when I have looked at it, what is there doesn't match the hype.

Its better than average quality classroom teacher in my experience.

And the point is this didn’t exist online at the time. His resources were strictly more. I imagine its still hard to find a single free resource that matches its breadth and quality.

Do you have few examples?

There are? I know an OCW course includes "lecture notes" pdfs. Does it have the lectures too?

For example, the course "distributed systems" is available on OCW as 6.824, from spring 2006: https://ocw.mit.edu/courses/6-824-distributed-computer-syste...

MIT has renumbered the course, but it's still around: https://pdos.csail.mit.edu/6.824/ . Don't be fooled by the URL; it's now 6.5840.

And a series of lectures is available too: https://www.youtube.com/@6.824/videos . They're from spring 2020. They link to the course website for spring 2020, which is great. But it doesn't link to them, and OCW doesn't know about any of this, as far as I can tell. The 6.824 lecture series is pretty hard to discover, too, because it's uploaded by an account dedicated solely to that one course, which isn't related to any other MIT lecture videos.

If AI tends to be something used mainly in ideation and development, which is how a lot of people use it today, then once consumer hardware gets good enough you could see a bunch of the current data centre workloads move onto consumer devices.

But if AI starts being used more in repeatable, operational workloads I think it makes sense to have significant cloud infrastructure for it. TBH I haven't seen much of this, and I've been skeptical about people using agents for much of anything when it can be done with just software. But we are starting to see more of this kind of workload, like the taggable Claude in your slack etc that people seem to really love.

Anthropic in particular has developed this almost Orwellian like veil of condescending rhetoric that on the surface suggests they're looking out for you while underneath they're taking actions that suggest they do not trust you, Mr/Mrs Ordinary Person. All the safety rhetoric, never supporting open weight models, the lockdowns on harnesses outside claude code, etc.

Anthropic if you care about public good, do something to empower people. Release an OSS model. Open source Claude Code. Open source some inference tooling or something. Just give people anything except your words.

Is this what kids call rage bait these days?

If an ASI model is 100% aligned to user intent then you only need one person on earth to prompt "kill everyone" for an extinction-level invent.

There's no logical way around this. The model either has to ignore the person at the helm or we have to do a multi-national abort before RSI. Anthropic is trying #1.

― Terry Pratchett, Thief of Time

"Please run N tasks doing X for Y duration, I want to test something"

"Hmm but that would waste tokens, we shouldn't do this test"

What the hell type of product is that? I wasn't asking, I was ordering it to do something, if Anthropic can't get their model to do as I say then I'll just go use someone else's.

Because the reasons outlined in the article, plus many others. Concentration of power, unemployment, rising energy/living costs, stealing of digital artifacts owned by people.

I myself acknowledge that AI is useful. But due to the reasons above, I will be anti AI for the most part.

Yes, it doesn't matter if AI manages to cure cancer. Due to the reasons above, I will be anti AI. I suspect a lot of other people feel the same way.

https://www.forbes.com/sites/antoniopequenoiv/2026/06/10/ant...

https://www.washingtonpost.com/technology/2026/03/04/anthrop...

And we’ve seen unprofitability not being an issue for a lot of companies. Markets function in funny days nowadays.

This line of reasoning is really very silly. I can be looking out for my young child by choosing not to put them behind the driving wheel of my car.

There are both stupid and malicious people in the world. If you have a tool that you believe may be genuinely dangerous in the hands of the wrong person, it is not the smart thing to do to give the public unrestricted access to that tool. Do you think it would be sensible to give every individual on Earth a nuclear weapon to do whatever they want with?

Curing cancer sounds insane, but it's also a research problem, not a societal level coordination problem. And one AI has already proved to help with breakthroughs (alphafold). IMO it makes sense for them to shoot for something like that as proof of AI's beneficial sides.

This is not true. There are two companies at the center of AI direction: Anthropic and OpenAI. If there were anyone on the plant who has the ability to influence our direction then it would be Dario Amodei.

Acknowledging the grievances is a good step, but it's not enough. There needs to be a clear explanation of actions to address them, a plan to enact those actions, and commitments with consequences in failure of those actions. Tell people how you're going to make them more employable and effective and needed. Tell people how your datacenters will be carbon neutral. Tell people how financial actions resulting in a frothy market will be coming to an end. He and Sam Altman alone have this power and their inaction says everything we need to know about their intent.

Hoping that they solve those huge problems themselves is basically hoping that they will have more power.

It would be like expecting Craigslist to rescue journalism. The amount of power needed to disrupt an industry is nowhere near what's needed to fix it.

Possibly, but not necessarily in a better direction. And that's the whole problem here. They should all go watch Phantasia.

If you don't tackle the societal level problems, nobody will care about research breakthroughs.

I don't think anyone will care about the largely made up issues about electricity prices or water use in case AI literally cures cancer.

His body was healthy, aside from the bullet in his skull.

I just don't buy the AI labs approach to this stuff. Like, unless we can basically simulate the entirety of human biology, I don't really see how LLMs can make progress here. Maths is different as it doesn't require a real-world interface, and programming already (by definition) can be simulated on a computer.

Without that, I can't see much (if any) progress being made on domains like biology.

Drug discovery is similar AFAIK. The space of possibilities is even larger than protein folding, but it's structurally similar enough that I think AI will help to make progress on the discovery side. Actually getting the drug tested and approved is another matter though for sure.

For one, the question for Anthropic is whether LLMs, specifically, not AI techniques more generally, can help significantly with cancer research. And here, all experience so far is that LLMs only really work when they can easily automatically verify their own outputs and self correct - such as in math (using automatic proof verifiers) or programming (using compilers and unit tests).

The second problem is that biological research speed is highly dependent on slow biological processes, such as cultures and long term studies. In programming, if an LLM could provide excellent insights and research suggestions 100x faster than a human, it would speed up the work roughly 100x. But in biology, it would only speed up the total work by a small amount - as any insight, even if absolutely brilliant and spot on, would still require months and years of actual experimentation.

If I was Dario, and I was after a cynical sales grab to go with the "actually curing cancer!" spiel, I'd probably aim for a drug repurposing strategy. I'd use the above approach to find existing drugs that might target known pathways that drive incurable cancers. I'd only screen compounds with extensive safety data and easy delivery mechanisms, and from the in silico hits, I'd throw a tonne of money at rapidly experimentally screening all those candidates in parallel, and then rapidly push those that worked into clinical trials. I'd assume that, with a small but non-negligible prior, and the money to push through hundreds and hundreds of candidate compounds through at once, I'd have a reasonable chance of getting one drug through to a "Claude Cured Cancer!" show-stopper headline.

But I think even then, with all of Anthropic's wealth, you'd need 2 years minimum to move from initial screening targets to a Phase 3 trial. And it would be an almost criminal waste of research funding to get there -- the dollars for discoveries ratio would be appallingly bad.

(If I was Dario, and I actually wanted to cure some forms of cancer? I'd just use my obscene profits to fund actual cancer research, step back, and let the researchers get on with it.)

I do agree that this kind of targeted approach makes sense.

However, discovery is not really the issue here. Running the clinical trials (1/2/3) is much much more difficult, and consumes basically all of the time in drug development, so even if LLMs perfectly automate this, the speedup will not be particularly large.

This is not correct, Alphafold predicts crystal structures of proteins. It is a known issue in the field that these folding models do not generate ensembles of protein conformations like the ensembles generated by DE Shaw Research (who built a super computer to simulate proteins). And these folding models fail when we don't have crystal structures, so they are certainly not generalizing. A great paper on the subject: https://www.biorxiv.org/content/10.1101/2025.02.03.636309v1

Drug discovery is significantly more complicated than protein folding. Small molecules with similar chemistries can adopt novel binding poses, can end up binding to off targets (PXR, hERG, etc) or simply never make it to the protein due to solubility or permeability.

I will always defer to Pat Walters who seems like the most careful and sane person in this field: https://patwalters.github.io/

Curing cancer is a societal problem. We have lots of cures for common diseases but resource allocation means people don't actually receive the treatment they need. For example, prohibitively expensive gene therapies, or HIV treatments in developing countries. A disease may have a "cure" but if people who need it don't receive it then from their perspective it may as well not exist.

But something tells me either they can't or they won't, so no trust will be built.

Anthropic is literally creating the bubble. It is not beyond their scope of influence, it is literally what they are consciously achieving.

As for peoples jobs, same actually applies. Anthropic is selling itself on dream of replacing jobs, even or especially where they are well aware AI does not perform that well. They are actively trying to replace people quickly before management notices it does not work well.

And also, they can influence how much their data centers contribute to global warming.

One word that few wealthy people ever hear, is “No.” It has a pretty significant effect on their worldview. Even the most reasonable, well-informed, well-intentioned, wealthy folks can have their thinking affected.

When every silly, should-be-smothered-in-the-crib idea gets enthusiastically endorsed by your entourage, it’s easy to lose the ability to self-regulate. I’ve watched it happen, numerous times, as acquaintances and friends have become more successful.

Obsequious LLMs are leveling the field. Less wealthy folks now have the chance to lose their ability to self-regulate, just like rich folks.

In the meanwhile, we can all publicly see how the the billionaires and trillionaires, behave and settle their priorities exactly, in the cartoonish way you are dismissing.

- The incredible insecurity and constant need for personal validation.

- The absurd and obsessive pursue of further wealth when it would be temporally impossible to even spend 1% of their current capital.

- The extreme level of cowardice, where an auto plant worker, can call out the powers to be, the pedophile protectors that they are. At the same time, only Jensen Huang did not submit itself, to the humiliation of standing behind face and front at the presidential inauguration...

That shows profound lack of imagination. Mark Shuttleworth's founding of Canonical was a choice to expend his wealth to try to popularize Linux. So was Gabe Newell's long running efforts with the Steam Machine (dating back to the original one in 2015) and Proton that only started bearing fruit in recent years. These were business decisions you say? At the level of wealth we're talking about, the two are intertwined.

gabe is not a philanthropy who want to see linux gaming happen no matter what. he's a reasonable man who saw a threat to his business when every operating system started making their own store and windows was directly trying to eat his lunch starting from window 7 and going at it the strongest in window 8 when they had their dedicated to game store directly pinned in the windows bar menu.

If you want philanthropy (or perhaps not), Gabe Newell has $1 billion worth of yachts and research ships.

Really, they're probably more displeased about the Valve near-monopoly as a marketplace for Windows games than displeased about MS.

The expectation of a need for decorum against all evidence almost seems like a quaint remnant of the aristocracy at this point. There's no advantage to being a nice or sensible person if you're a billionaire. People talk about "business" like it's still the 1800s and billionaires are the factory owners. The real economy (i.e. anything involving any pretense of being an exchange of goods and services in any shape or form) is now a minute fraction of the global economy. Everything else is finance. And if the nature of the finance "industry" wasn't obvious enough the US has given up all pretense to the point that members of the US government now intentionally manipulate online betting "markets" directly.

You may have needed some table manners to be able to run a tincan factory. You can let your entire ass hang out on TV 24/7 and still be a billionaire today. And people will still cheer you on like you're the inventor of sliced bread.

The big factor that works against military victory anywhere is that the military is left to visibly identify themselves as a means of projecting influence, whereas militants can seamlessly blend in with the population. And trying to punish the population at large to weed out the militants mostly just tends to backfire and create even more militants.

There is even modern precedent for this like the Algerian War where the people of Algeria managed to kick out the French even with the French engaging in widescale massacres, torture, and all other sorts of fun stuff that explains how anti-Western powers are so comfortably gaining influence in Africa today.

If one doesn't believe there was a conspiracy, then the assassination attempt on Trump would exemplify this - one guy with a rifle nearly single-handedly killed one of the most guarded men alive. Your average military logistics support doesn't have anything remotely like a team of secret service and police covering every single angle of attack in one well secured location.

Sure, but that being a failure mode is a design choice. Look at Iran: we've seen how many decapitation strikes now? At this point there shouldn't even anyone be left that was in the original chain of command but clearly they're chugging along quite nicely because they decentralized military operations exactly for a scenario like this, knowing anyone appearing on the US' or Israel's radar too prominently was one bad day away from needing replacement.

> This is how Afghanistan, with basic weapons and religious extremists running around in sandals, managed to defeat not only the US but also the USSR.

Not quite. The US managed to "defeat" Afghanistan. The problem was just that nation building was never really on the menu. The US was extremely hands-off during the occupation because their priority was making sure more US soldiers don't come home toes-first when the war had officially been declared won than to build any kind of legacy. This also led to massive subversion of the supposedly US-backed government. There were very few incentives in supporting the US more than necessary not to get shot at, especially after it became clear the US government had no interest in rewarding the loyalty of most of the people who had risked their lives as translators and negotiators, let alone supporting the people who volunteered to join the new army or police. That's why when the US left the supposed new Afghan state just dropped its weapons and ran away and why the Taliban could pretty much just take over without much resistance.

A good counter-example is Iraq where not only did the US have to do much less work to do (because Iraq already had the concept of a powerful central government rather than effectively being a coalition of often loosely connected tribal communities), they also engaged in a lot more active nation building and had reasons to maintain a long-term presence.

If you want to see how bottle rockets vs modern hardware can also play out, look at whatever the IDF is doing at the moment. Granted, the US military being deployed against US citizens would probably show a lot more hesitance and restraint, but again that's a design choice, not an inherent property.

> And trying to punish the population at large to weed out the militants mostly just tends to backfire and create even more militants.

That assumes militants are a problem, not a calculated risk. Long-term social instability is manageable if you don't rely on popular approval as long as it doesn't disrupt supply chains enough to directly threaten the military's ability to operate. Israel for example arguably flourished for decades while having to continuously deal with militants and responding with large scale violence.

As for Afghanistan I think people don't realize how much fighting was still going on over there. The Taliban forces weren't just camping out in the mountains and defending, they were actively striking all US forces and US proxy forces. In total hundreds of thousands were killed. The Taliban killed thousands of US soldiers, wounded tens of thousands, had a significant minority of land under their undisputed control, were actively contesting control of much more of the country, and rapidly took control of the country once the US left. And yeah nation building was 100% on the menu. There was a friggin George Floyd mural in Kabul lol. But the US was never able to get the country under control, and things were trending worse more than better.

Iraq was much easier for one simple reason - it's a heavily divided country which Saddam (who was himself part of a minority Sunni population) was able to stabilize only with an iron fist. This made much of the country hate him. He was also extremely paranoid, probably in part because of this, and didn't trust his own armed forces which led to direct efforts to limit their power/capabilities, installing commanders based on loyalty over competence, and so on. So it was pretty much an ideal target for a decapitation strike. But even in that ideal scenario we were left waving a victory from flag within tiny little heavily fortified green zones, because stepping outside of them was highly dangerous.

And I don't think Israel is showing anything productive at all. Hamas has comparable numbers today as when the war started, Israel's leader has a warrant for his arrest for genocide, and the future of Israel is probably more uncertain than it's ever been in its entire existence because they've isolated themselves from basically everybody except Trump who will be gone in a couple of years.

Maybe UC Berkeley is good enough for you?

"Upper Class More Likely to Be Scofflaws Due to Greed, Study Finds" - https://vcresearch.berkeley.edu/news/upper-class-more-likely...

"Does money make you mean?" - https://youtu.be/bJ8Kq1wucsk?t=311

"Wealth and the inflated self: class, entitlement, and narcissism" - https://pubmed.ncbi.nlm.nih.gov/23963971/

"Multiple studies show that drivers of expensive, luxury cars are significantly less likely to yield to pedestrians at crosswalks and more prone to committing traffic violations compared to drivers of budget-friendly vehicles" - https://www.ralphaschwartzpc.com/blog/study-luxury-car-drive...

"Polish millionaire apologizes after snatching signed hat from child at US Open" - https://abc7ny.com/post/video-goes-viral-polish-millionaire-...

Who is naive here?

Btw, I opened the very first "scientific proof" you offered, and it happens to be a story with no links or data about one cool student's deep studies where people who believe stealing isn't so bad steal more often, and that's why upper class is bad-bad. So how do I know you didn't even try to read it before posting?

“The increased unethical tendencies of upper-class individuals are driven, in part, by their more favorable attitudes toward greed,” said Paul Piff, a doctoral student in psychology at UC Berkeley and lead author of the paper published today (Monday, Feb. 27) in the journal Proceedings of the National Academy of Sciences..."

"The rich are different: Unravelling the perceived and self-reported personality profiles of high-net-worth individuals" - https://bpspsychub.onlinelibrary.wiley.com/doi/10.1111/bjop....

"Tax Evasion and Inequality"- https://www.aeaweb.org/articles?id=10.1257%2Faer.20172043

"...Drawing on a unique dataset of leaked customer lists from offshore financial institutions matched to administrative wealth records in Scandinavia, we show that offshore tax evasion is highly concentrated among the rich. The skewed distribution of offshore wealth implies high rates of tax evasion at the top: we find that the 0.01 percent richest households evade about 25 percent of their taxes. By contrast, tax evasion detected in stratified random tax audits is less than 5 percent throughout the distribution..."

"The personality traits of self-made and inherited millionaires" - https://www.nature.com/articles/s41599-022-01099-3

Traits they exhibit plenty more, than the general population: "The “Why” and “How” of Narcissism: A Process Model of Narcissistic Status Pursuit" - https://pmc.ncbi.nlm.nih.gov/articles/PMC6970445/

https://youtu.be/bJ8Kq1wucsk?t=310

https://youtu.be/bJ8Kq1wucsk?t=371

Bankman-Fried — FTX -> misuse of billions in customer money for investments, influence, political contributions and personal interests despite enormous wealth.

Bill Hwang — Archegos -> extreme risk-taking, market manipulation and deception that ultimately imposed billions in losses on banks.

Alex Mashinsky — Celsius -> personal profit from token sales while customers were misled and ultimately left unable to access billions in assets.

Bernie Madoff -> massive Ponzi fraud sustained for years despite already having wealth, status and an elite financial reputation.

R. Allen Stanford — Stanford Financial -> diverted billions from investors to finance his businesses and lifestyle despite extraordinary existing wealth.

Leona Helmsley — Helmsley Hotels -> billionaire convicted of tax evasion and fraud involving personal luxury expenses. The sentencing judge explicitly described the conduct as motivated by "naked greed" and an arrogant belief that she was above the law.

Elizabeth Holmes — Theranos -> maintained sweeping false claims to investors while building a company valued in the billions, resulting in hundreds of millions invested on false premises.

Trevor Milton — Nikola -> repeatedly exaggerated his company's technology and achievements to stimulate investor demand and support its stock price.

Karl Sebastian Greenwood — OneCoin -> helped sell a fictitious cryptocurrency to millions of victims who invested more than $4 billion while he personally received hundreds of millions.

Miles Guo — GTV / investment schemes -> former billionaire convicted of using lies and misrepresentations to extract more than $1 billion from followers

Do Kwon — Terraform Labs -> deception surrounding a huge crypto ecosystem that culminated in roughly $40 billion in losses

Joseph Lewis — Tavistock Group -> billionaire who admitted abusing confidential corporate information by tipping friends, employees and romantic partners, while his company separately admitted securities fraud involving concealed share ownership.

Elon Musk — X -> Changed the platform algorithm after Biden posts outperformed his, massively increasing exposure of Musk own posts.

Jeff Bezos — Venice -> turned his 2025 wedding into a three-day billionaire spectacle that prompted protests over the privatization and commodification of the city.

Mark Zuckerberg — Meta -> commissioned and publicly displayed a giant Roman-style statue of his wife while simultaneously building one of America most extraordinary private compounds.

Bryan Johnson — Blueprint -> turned his own body into a multimillion dollar anti-aging project, including receiving plasma from his teenage son in an unsuccessful rejuvenation experiment.

Larry Ellison — Oracle -> built a personal real-estate empire including ownership of 99% of a Hawaiian island

Then you have the Narcissistic in Chief: https://www.independent.co.uk/news/world/americas/us-politic...

I can think of loads of things I’d do with hundreds of millions of dollars of disposable income. Make large buildings that look like how I think buildings should look, fund scientific research in areas I’m personally interested in but have little aptitude for, find artists that align with my tastes and give them the ability to realise their visions, contribute to political organisations and charities that align with my values, et cetera. Those all are basically infinite money sinks.

In the past, the ultra-wealthy did do all of these things, funding loss-making research and expeditions, monumental architecture, philanthropic organisation and artistic works. The problem might be that you think billionaires shouldn’t do some or all of these things, but that is a very different position than that they can’t do any of these things.

I don’t think the GP meant that it’s impossible to dispose of a large amount of money. But it is effectively impossible to spend down a certain level of accumulated wealth by purchasing goods and services that any human being or their family could require to maintain even lavish standards of living.

The examples you gave are conversions of wealth to power, using money to reshape the world. Which, I think, is kind of the crux of the issue. Perhaps you were alluding to this at the end of your comment.

Skirt taxes to give a billion dollars to prospective college kids through your own organization? Nice gesture, but still a greedy fuck who refuses to give up control.

Pay a billion in taxes that ends up mostly going to heavy administrative bloat? Give that guy sainthood.

The point is less that these people are surrounded by "yes-men" but more that wealth (especially when measured in billions) is power and with sufficient power it becomes easy to forego any question of consent, let alone of whether consent is coerced or not. Remember that power is ultimately about the ability to enact violence and violence can take many forms, most of which are perfectly legal (because the legal system itself exists to regulate how, by who and against whom violence can be used).

You tend to hear "no" a lot less when you always point a gun at the head of the persons you're asking. Note that "wealth" isn't the only way to get there but a certain level of it is usually necessary to get to the point where other options become available - and some of the ways are a lot riskier in the long term (cf. Epstein).

Side note: this is also why I hate the pseudo-intellectual counter argument against "billionaires" of "that doesn't mean they have billions of dollars sitting in a bank account" - it's like arguing that De Beers didn't benefit much from holding a quasi-monopoly on natural diamonds because the diamonds would be devalued if they flooded the market with the ones they had intentionally kept off the market to drive up value: beyond a certain amount money ceases to be about liquidity and starts being about leverage. Unless you happen to be dealing with lower level bureaucracy in Russia, the most efficient way to use wealth to your advantage isn't to just hand people stacks of dollar bills.

When you start having 'people'. ("Have your people call my people.")

A broader discussion on the spectrum of wealth:

* https://ofdollarsanddata.com/the-wealth-ladder/

* https://ofdollarsanddata.com/the-ideal-level-of-wealth/

Also power. You see the same thing happen with managers that dismiss criticism, and have the power to make it stick.

The reason that the ultrawealthy behave like toddlers is that toddlers are, relatively speaking, ultrawealthy: all their needs are met without any effort on their part, and so many of their desires are fulfilled simply by expressing those desires out loud that any impediment or refusal is obviously enemy action.

For example poor people who have never thought about rising out of it - eg about 50% of kids in my Brooklyn public highschool had parents who didn't give a shit if the kids studied or not. Completely oblivious to how the world works - meanwhile the other 50% wa immigrants who pushed their kids and those kids are now in the 1%.

In general I think what's more telling than your level is your journey. Someone born rich maybe mirrors what you described (I don't know people like that) but the few centi-millionaires and billionaires I "know" (ie worked for and dealt with in that context) have encountered plenty of "no".

When you are building a company, you are going to get a lot of no. No I won't buy, no I won't work for you, no I won't invest in you. In fact I would say a universal attribute of someone who has "made it" is having ample of experience getting "no" and dealing with that fact property. That's true even like at the level that plenty oh HN readers are - a successful faang employee and the like.

For what its worth - I generally find that orienting to what some other group is like "rich people are like x etc" is a tell-tale of not focusing on what's within ones sphere of control and knew life. Any brain cell I spend fantasizing about someone else's imagined behavior is a brain cell not dedicated to engaging soberly with my own reality.

    For example poor people who have never thought about rising out of it...

My personal experience is that the required difficulties of my life have decreased in direct proportion to my income, leaving mainly the self-imposed difficulties. It's not hard to extrapolate that line a little further to billionaires.

De facto some parents look at that as a gift and "force" their kids to take advantage of it. That's why the valedictorian etc is usually an immigrant kid not a rich"native" kid.

Meanwhile plenty of parents seemingly have never considered the opportunity in front of them. Content to let their kids not study and do stupid crime.

To say it simply: not everyone values education the same degree - or a all. That's all I am saying here - you'd think a poor family would grab to the opportunity to rise out through education but it's obvious that for many many many people this has never crossed their mind. If you had not encountered this in your own life I find that odd.

It's been a while, but I remember changing a setting on mine, so it is less sycophantic.

Mine says "I don't know," frequently, and also suggests against ideas.

But it also confidently states complete garbage, much more frequently. I have to stay on my toes.

LLMs, even frontier models by major providers, still have no reliable internalised way to assess the accuracy of their output and they have, do and will continue to for the foreseeable future, lead people down paths they shouldn't [1], partly by their architecture and the limits of the technology, partly by an intent for maximising retention.

For what it's worth on people, I have, both in politics and business, unfortunately made the painful discovery that, whether intentional (because the powerful person in question cannot or doesn't want to deal with different opinions to their own) or unintentionally (because those with sycophantic tendencies simply managed to manipulate themselves into their inner circle over years and became trusted), there are people in (financial, political or other types of) power which are surrounded by few willing to tell them when they are wrong and even fewer that are actually listened to if push comes to shove, which often affects the personality and mental health of said powerful people negatively, to the detriment of society, their family, their employees, etc.

Members of the media are actually complicity. I very much disagree that the media presents obscenely powerful people in too negative a light, more the opposite. Am very firm that, to retain access to the rich, powerful and famous, there is far too much sane-washing of utterly ridiculous, unacceptable, harmful and/or dangerous behaviour. Sometimes the person in questions own health and safety are put at risk, because neither the people around them, nor public opinion or reporting treat their behaviour in the way appropriate. Sometimes this again leads to harm for the public, their family, those working under them, etc.

What'd get someone with less power or in a lower tax bracket ridiculed or even sectioned is often reported as "eccentricities", just being "passionate about a topic" and trying to do the "marketing rounds".

[0] https://artificialanalysis.ai/?omniscience=omniscience-hallu...

[1] https://arxiv.org/html/2602.19141v1

This is false. Perhaps you meant they have insufficient methods, or imperfect methods, but asserting none at all is facile and your links do not say that at all.

If your believes were true, this actual quote, pulled from a recent sonnet conversation, would be impossible:

“The XT60’s 15A limit is unsuitable for an appliance that… no, it’s the XT30 that is rated for 15A, XT60 is rated for 30A continuous. For a 20A appliance, XT60 will be fine.”

I did say "no reliable internalised way to assess the accuracy of their output", which is something very specific. If that Sonnet output is reasoning traces, there are many issues with using that as a source:

For one, back and forth reasoning does often correlate with less, not more accurate overall outputs in evaluations and one example could never seriously be extrapolated to be considered "reliable", i.e. happening consistently and dependably.

Secondly, self-correction is also not self-verification in regard to model output, revisions not necessarily mean internal accuracy assessment by themselves (again, over-revisioning has lead LLMs in my and even public evals like the one linked above to step away from accurate information written in their reasoning traces but discounted in the final output (if we must use anecdotal examples like your Sonnet quote)).

Then there is the fact that, unless that reasoning trace (if it indeed is one) was copied from a months old chat history, Anthropic has obfuscated their reasoning traces so this output is (if it isn't an ancient history you dug up) from the obfuscation model in between and not reflective of the actual models reasoning. So even for anecdotal evidence, this can likely not be used (unless again, you went for December 2025 history). If this was not reasoning traces (I am fairly confident it is having spend a long time reading Anthropic summarisations vs actual reasoning traces when the switch was on-going and you could get both for a limited time, what you shared reads very much like obfuscated rather than pre-obfuscation reasoning output), that still leaves this as anecdotal, one time, possibly erroneous (did this back and forth even prevent an error in the final output) and there are more issues still with just using that quote as evidence, this is simply unsuitable as a source in any situation.

Here are some papers I read lately, all published in 2026 and using the current crop of models which were what led me to make that specific statement. LLMs currently have no reliable internalised way to assess the accuracy of their output, at least as far as the literature is concerned:

> Even state-of-the-art models struggle to reliably discriminate between data uncertainty and model uncertainty.

Beyond “I Don’t Know”: Evaluating LLM Self-Awareness in Discriminating Data and Model Uncertainty [0]

> LLMs cannot reliably revise their own errors without an external signal.

> The same models that confidently catch and repair errors in external content routinely fail to identify identical errors in their own reasoning traces.

The Self-Correction Illusion: Role Relabeling Gates Explicit Error Flagging in Large Language Models [1]

Simply, as of today, LLMs cannot reliably translate whatever internal signals they have into accurate self-assessment of their outputs. Even in papers that show limited, edge case capability, this often breaks with minimal prompt and/or task changes (happy to link those too, I just need to get to my Mac where I have the PDFs), so it is not reliable.

If you got a paper that shows that I am false, that there is reliable, internal self-verification of a models output (even a small research LLM not yet publicly released), I'd be happy to read it and retract my original statement.

[0] https://arxiv.org/html/2604.17293

[1] https://arxiv.org/html/2606.05976

I found another sign of that is the way LLMs answer with a professional, business-like tone even if the request is completely bananas. It's what a concierge or butler would do, but not an actual close friend.

How does a fan work: Swish swish swish swish

Where do these clouds come from: Points to a far away direction in the sky and says they come from there.

Who does all these roads, trees and environment belong to? It all belongs to me. Obviously.

They have an answer ready for every question you throw at them and they will answer it with absolute certainty. I will have to wait and see at what age does the concept of "I don't know" develop.

ChatGPT, which is usually quite reliable, refuses to answer me because law.

In the end, an extraction should work any plant material depending on potency, skill and available equipment.

After the extraction you can evaporate the ethanol(carefully since it’s highly flammable) and increase potency.

A mix of own research(basically emulating others) combined with strong models we can do a lot more than we can do ourselves.

Granted, I've only been using the free version but I've been getting significantly more mileage out of those from ChatGPT and Claude. I guess it might be a feature that Gemini is more often obviously wrong from the start (e.g. by giving sources that don't support its claims whatsoever) but considering this is the AI from the company that had become synonymous with the concept of trying to find information on the Internet, that's pretty damn pathetic.

Even though they will get chemistry right more often than me, I still wouldn't want to ingest the result of it walking me through that on a drug, psychoactive or otherwise.

Any given answer might be right, but I'm not a trained chemist and don't know how to safely test things.

That's false. The LLM will only answer competently if it was trained on that data; and if it has enough data to make the correct connections between your question and the "correct" answer.

In the case of this article they're specifically saying the LLM has limited training.

How would you know if it didn't?

A prudent strategy, but I'm not sure how prevalent it is in the general population. (Or even if people do look at multipole "sources", they're in a self-reinforcing echo chamber that may reject contradictory information.)

I have actually gotten "hey i don't think this is a good idea, here's why" as feedback from at least Opus. It WILL still do it if I just demand stupidity (and hell i've been right, which is another topic entirely) but it has given me more confidence this can be a useful tool in the right spots.

That said I probably don't need the top tiers (metrics at least confirm that) and I'm guessing that's specifically because I was working in coding. Most were worded in a "is this a good idea" framing which probably helped, but at least once I said 'lets use this library/method" and it gave a decent argument on why that was basically redundant without prompting.

I still struggle to see the price point panning out.

Over the last 3 years I've seen projects where I thought, pretty obviously that's a bad idea. But, because LLMs don't say no and can just be pushed to build it anyway, the people building them might never learn that or learn why.

It's nice to be able to have a quick prototype or mvp. But if we never hit friction or something not working out, we never learn or have to come up with a creative solution.

Now, the LLM might seem incredibly intelligent (relatively speaking) and also creative but let's not forget that all is based on its training data. I simply don't believe it can ever be omniscient or that the companies training it are careful enough when doing so.

Lacan

https://ecole-lacanienne.net/wp-content/uploads/2016/04/1966...

I think this is one potential path to machinic subjectivity, or a machine phenomenology. To fully replace the human, we don't just want to give the machine some nebulous notion of "agency", we want it to possess this degree of Being as subject. If Lacan's right, perhaps we're closer to this than we might think. The machine already has language in a very Lacanian sense (what I've been calling a machinic linguistic unconscious), the subject just needs something extra to emerge where meaning breaks down. This will be the Lacanian split subject, one not fully present to itself, and allow desire already present in the language mappings within the model to provide immanent causal force.

Until that happens, we'll still need at least one human on the planet to retain his full faculties, to give the global compute infra its telos. Once that threshold is crossed, then that'll be the moment of our final displacement.

If someone comes to me and asks a general question I can easily say no. But if I go up to for example a librarian and ask them where to find book N, then I would expect them to either know where it is, or how to find it.

If instead I asked them what the weather was going to be tomorrow, then I don't know would again be a reasonable response.

So for me the line becomes a search engine problem where no just means "there are no pages for this search result", but translated into LLM.

I think instead of Yes/No I'd rather want some probabilities such as, "This response is N% accurate based on these research metrics", or "M% accurate based on the latest research on topic O at date P" etc.

Reinforcement learning seems to be making these models more difficult to control since while it attempts to control some behaviors, it has also recently been shown to result in models that pursue long-term goals and promised rewards in general (outside of the goals reinforced during training), overriding human preferences.

https://alignment.openai.com/measuring-reward-seeking/

The ability of animals to co-exist in a dynamic balance, not to destroy their own species, directly or indirectly (by destroying the ecosystem) is something that has come about by millions of years of co-evolution, and is enabled by having a brain complex enough to allow these evolutionary lessons to be encoded in their DNA and control the phenotype in fundamental ways.

An LLM has none of this. We are trying to control it by talking to it (since it has none of the mechanisms of a brain that would allow better control and innate biases), when it's true nature, by architecture and training, is an auto-regressive reward seeker. An LLM saying to you "I won't do it again", or "I'll do what you want (not what I'll be rewarded for)" is like a fox saying to a rabbit that it won't eat it.

Regardless the frontier model considered, we're certainly in a "know-it-all" era.

Maybe the sort of introspective prompt-response is difficult to implement when it could limit/contaminate future improvement. I speculate it's easier to correct a "confidently incorrect" model than a "I don't know" model. A confidently incorrect model response >=0% correct over a 0% correct (I don't know).

Maybe "I don't know" is a model cognito hazard of sorts when many queries can lead back to the response. Maybe future Turing tests will use this sort of introspective evaluation. Who knows? I don't :)

Not only that, but also:

1) The training data, consisting of things like WikiPedia, textbooks, and overly confident posters on Reddit and Stack Overflow, is going to be massively deficient in people saying "I don't know" when they in fact don't.

2) Given 10 training samples, 9 saying "The answer to X is Y", and 1 saying "I don't know", how should the LLM respond? The opposite could also (less likely) happen with 9 training samples saying "I don't know", and 1 saying "The answer is Y". In either case the LLM as a statistical predictor will go with the majority, oblivious to whether that is right or wrong.

3) An LLM doesn't respond with what IT "knows", but rather with what the majority of the people reflected in the training data assert they know (or don't). Sometimes, if it occurs to you, you may be able to separate the wheat from the chaff by asking the LLM to role play an expert, or the vox pop, and it may turn out that the lone expert saying "I don't know" is the right answer, not the 9 confident fools.

4) An LLM objectively doesn't KNOW anything, since it hasn't experienced/verified anything first hand. It has only "read" things, and has no way to reconcile this against reality, nor much idea what to trust or not, other than by the context of the training sample. It has no idea what came from where (Textbook vs Twitter), since it's not told this, nor has any mechanism of storing metadata. In contrast, if you ask a smart human about something they've never experienced first hand, then they may reply "well, I've read conflicting reports ...", or "I don't know, although I've heard that ...".

Luckily the consensus in the LLM's training data is mostly right, so regurgitating it mostly/often correct, but going off-script to "reason" about how to prevent the cheese from sliding of your pizza (early LLMs would suggest glue) reveals how fragile this is.

Nope. Emergent behavior exists and at this point dominates LLM behavior. Most of the stuff LLMs say they never learned (they are, always, imitating many different sources at the same time)

... which imho is exactly what humans do.

Word prediction with some randomness can lead to emergent behavior, depending on the specifics (complexity and scale) of the used prediction logic.

And this very claim is "simply" the result of a neurotranmitter-moderated Natrium - Potassium ion cascade across a semipermeable membrane.

Yes, we have all seen the math theorems being proven... just higher processing power at the service of the same algorithmic and conceptual patterns? [1]

I am sure the next version of Opus or GPT, if given only fifth grade knowledge, will somehow be able to build all the mathematics necessary to solve the problem on its own... right? Right?

[1] - "AI Isn’t Outthinking Mathematicians. It’s Out-Remembering Them" - https://davidepiffer.com/p/ai-isnt-outthinking-mathematician...

It is possible to create (subjective) reasoning traces like https://huggingface.co/datasets/Bachstelze/ethical_coconot_6...

And train or adapt a model to it: https://huggingface.co/Bachstelze/olmo-7b-ethical-reasoning-...

This is just a little proof of concept, though it is maybe the direction you are looking for?!

I do agree that it's a problem but the root cause is the fundamental limitations of current gen LLMs, it's not an alignment problem.

This means not giving an answer is not a technically possible option. Best you can do is force it to output a magic "stop speaking" token, but this is a vastly different training problem than getting it to not know something.

People naively expect LLM outputs to have some sort of confidence value when predicting, but the technology just doesn't work that way.

Or when it comes back with 50% I know, let's talk about this a bit more maybe I add more context and such.

Granted I don't think LLM word math does anything but mostly just output the numbers that make the word salad so maybe that doesn't exist.

You're asking a lot from extremely fancy auto complete...

It's a bit annoying honestly. I'm always very careful to be incredibly neutral on the direction of a request, and I'd say 10% are knocked back on on valid grounds, which is great.

On occasion I accidentally say "let's do this" and it blindly goes and does it - I spent 2 days undoing something I built that was just a truly awful idea, because I accidentally phrased it lightly as a request, not a discussion!

Nowadays I often prompt like "I heard there is also this different direction, what do you think about that?"

Another thing I do is asking the agent to make a decision matrix for choices. It's useful to discuss, give feedback on, and signals that it's a discussion, not a request for a particular direction.

It's then also easy to say: create a prototype for multiple directions so I can compare the solutions.

That way I choose the problem, I choose the solution, but the agent can help me discover solutions, make tradeoffs visible, and implement solutions.

You can get just as good information by asking its thoughts for and against some issue.

That doesn't force it to stop being sycophantic; in fact it actually exploits sycophancy to give you what you want.

There's also a second aspect to it, just in terms of RLHF mechanisms. If you've ever experimented with VLA models (i.e. vision input + text task = robotic arm motion output), they tend to need all the training examples of the robotic arm being motionless removed entirely, otherwise the model simply learns that staying still is rewarded and proceeds to never do anything at all. You successfully train the laziest bot in the universe. I wouldn't be surprised if something similar happens to LLMs if reinforcement learning is involved in the instruct tuning process. If no is a valid answer, why ever do anything?

It's relevant to AI safety. If you have a diversity of outputs, the AI will agree to hack the bank 0.1% of the time regardless. If you have a uniformity of outputs, in most contexts the AI will hack the bank 0% of the time, but in certain odd contexts, all AIs will work together to hack the bank 100% of the time.

There's a famous Socrates quite about wisdom: I know that I know nothing.

Opus 4.7 would flat out refuse to follow instructions to the point where it was just too frustrating to use.

I've had refusals for GPT 5.5 before as well (not because of a ToS violation, it just refused to take conversations in directions it felt were in bad taste)

LLMs, incidentally, respond in a similar pattern in my experience.