The "global stop order" is just generally perceived as an impossible coordination problem. So instead we see a mix of labs voluntarily putting in guardrails and regulatory efforts (which are not only aimed at hypothetical super-AIs of the future). Of course labs are also in a competitive race. And I actually think that it does make sense that the richest companies in the most dominant positions would in a better position to worry about safety than a startup that is just trying to survive at all. And just in general, it seems reasonable that the fewer companies have access to dangerous tech the better. This isn't really about some highly speculative future tech either -- current models already pose lots of risks, and the pace of model improvement is something wildly unprecedented. Whether or not you call it ASI, the capabilities we will have two years from now are hard to even imagine properly. Also, I don't think the issues that you are highlighting are all ones that Anthropic would dismiss as second-tier. In particular, mass unemployment from AI is how we will deal with a massive devaluation of human labor is one of the most serious concerns. And about other issues, reasonable people may differ. I'm more worried about biorisk than environmental damage, for example, but clearly we should be keeping an eye on both. Serious risks and problems, just because they aren't already harming people today, are not just a distraction.
I'll concede that a lot (most?) of the problems are not technically the responsibility of the AI labs to address, and it wouldn't entirely be their fault for our government failing to get ahead of the problem. Mass unemployment, for example, is nearly 100% a political problem.
That being said, I can't help but experience a bit of Deja Vu over arguments like those around biorisk. I've seen the same exact things said in the early 2000s over widespread access to broadband and Google. When the anarchist cookbook spread around online and everyone was super paranoid about democratized terrorism, and we had big regulatory pushes for ISP level censorship and user tracking. Telecoms frequently argued that only they can keep the web safe, with strict and expensive regulations that naturally only those large heavily capitalized companies can afford to go through. Like the early internet and search, its just another way to lower the latency required for a human to find already existing public data
Well, very little of that played out. Turns out the math, for now, is the same, and information retrieval doesn't directly correlate to democratized weaponization. In 2001, a bad actor still needed a physical lab, precursor chemicals, etc to build a physical threat. Those same exact physical constraints exist today. The software cannot yet cross the digital-to-physical divide.
Keep an eye on the risk, by all means, but I don't see it yet as justification to cement a monopoly or oligopoly, nor do I see it as a reason to prioritize a risk of information availability over the climate and environmental risks that are far more likely to end the species.
If you have a sizeable bucket of money, it's so, so easy to get folks so distracted by (or invested in) movie plot threats that they totally fail to (or have a "plausible" excuse to fail to) notice the actual, lasting harm that you're doing to society at scales both small and large.
If Anthropic had pushed hard and nonstop since their founding to ensure that all LLM companies in the world were legally bound to stop all LLM development the minute any one of them called for a halt to work, then I'd give their claims about safety some credit. They've been screaming about "safety" and "alignment" for years, but -because LLMs are impossible to secure against code injection- their products are fundamentally unsafe and always have been... I just don't trust their claims about a commitment to actual safety.
My read on their recent calls for a global "stop work" emergency cord is that they're very soon to (if they haven't already) reach a point where they will not be able to produce products that are sufficiently improved over the previous versions to justify the level of investment required for their development.
My prediction is that Anthropic and OpenAI will get serious barriers to entry of new competitors enshrined in Federal law, they will call for a "pause" or a "slowdown" in new research for "safety" reasons, and the US will attempt to engage in economic warfare with any countries that don't agree to force their domestic LLM companies to stop working on those LLMs.
> the capabilities we will have two years from now are hard to even imagine properly.
unless the bitter pill is gone, extraordinarily not this. The capabilities will be limited by the training data we can create to pull information and patterns from
and then we will still be limited by compute, space, and power
mass devaluing of labour isnt particularly believable when everyones predicting that all the big labs are gonna go under trying to subsidize tokens.
The original topic was Anthropic's guardrails, which were meant in part to stop China from using Anthropic's models to bootstrap their own. I take it the logic of the comment was that pulling attention to Anthropic's stance on regulation is switching to the topic. But for what it's worth, I also think that people are way to quick to assume that strong regulations would only help China and thereby hurt safety. There are many reasons why the opposite may be true:
- reducing demand for Chinese models reduces the incentive for Chinese companies to make them
- if US companies can't use Chinese models, they won't have an incentive to help their development
- China may enact similar regulations if the US leads, either out of concern for US safety or for commercial reasons
Also, I think some similar things can be said about AI safety measures in China aside from regulation. Currently, the US leads in model safeguards, but it isn't like China has zero interest in AI safety. Even if the US and China are rivals, there are many points of common interest (biorisk and "sci-fi" scenarios like an AI takeover, to name just two).
I don't subscribe to the belief that regulations in the US will lead to China advancing further.
But I also don't buy into the "China bad" narrative that gets frequently spread in online circles and in political circles. Its the cold war all over again, but this time its China instead of the Soviet Union.
Regardless of that, the regulations being proposed by Anthropic recently are not focused on the current issues which is my problem with all the hype marketing around hypothetical AGI/ASI. What is being proposed to be put in place will further cement the current frontier labs in their marketing leading position, and work to block new entrants, and open source competitors. That is the problem.
The other problem is none of them are talking about the real, difficult issues we are experiencing right now in the present. We don't need to talk about a sci-fi future scenario to recognize that LLMs have already caused and are causing harm in the real world. "We should probably regulate future frontier models" does nothing to help the current issues.
Wake me up when Anthropic says "The government should immediately stop us from hoovering up data and selling it back to the public. They should immediately stop us and others from enabling misinformation at scale that is already negatively effecting our democratic process. They should immediately stop us from building out new data centers until we have a large scale switch to renewables in the country, shore up the grids, or force us to generate our own power only with renewables" so on and so forth. Notice how any time the labs propose regulations, its only for a future hypothetical super intelligent model. Its never about their current operational liabilities.
It's important to separate objections to utilitarianism from the obvious fact that it can very be hard to correctly apply the utilitarian calculus. It's partly because of this difficulty that most classical utilitarians thought that people should generally follow commonsense morality and not try to directly apply the utilitarian calculus (which then led to the charge of paternalism and teaching one morality to the masses and another to a supposed elite).
But there are also people who just oppose utilitarianism, like G.E.M. Anscombe. For instance, in https://integrityproject.org/wp-content/uploads/2015/07/mr_t..., she seems to grant that dropping the nuclear bombs on Japan was probably good from a utilitarian perspective (because it saved lives overall) and also to grant that bombing campaigns that necessarily entail massive civilian deaths (including, apparently, area bombing German cities) are morally permissible but still to argue that dropping the nuclear bombs was impermissible because it constituted murder ("intentionally" killing the innocent). But this kind of distinction, which I think is what actual anti-utilitarianism must come to, is hard to even consistently maintain, and I suppose many HN readers would find the effort quixotic.
The first half of your answer presupposes some platonic utilitarian calculus that, if it were applied correctly, would yield moral outcomes. This is very hard to believe. If I look at notable/well-known examples of EA-affiliated people, it is hard to skip by members such as SBF. Did he correctly apply the utilitarian calculus?
It is relatively easy to take the proceeds of a massive fraud, buy a relatively small (as a percentage of the fraud) $ amount of mosquito nets, and save more lives than the lives impacted by your massive theft. Is this a correct application of the utilitarian calculus? What sort of data would we need a priori to do this calculation "correctly"? Do you think he had a careful estimate of the suicide rate of victims of ponzi schemes before perpetuating the fraud, or would any suicide rate have made the decision net [pun intended] moral, as any such victim of fraud would lead to >> 1 net purchased (so you would almost always net save lives).
The above is of course snarky. It is also a best-effort way of analyzing a notable utilitarian's actions. I do not think it would be difficult at all to use this type of argument to argue that SBF's actions net raised utility in the world. If only we all would become fraudsters, then we could truly live in Omelas --- a notable utilitarian paradise.
Yeah, I didn't mean to downplay how hard it is to apply the utilitarian calculus or even to suppose that the bare doctrine of utilitarianism resolves questions about what the ultimate good we should be trying to maximize is. I basically agree that utilitarianism is not a complete recipe for how to live. I just think that it probably gives the correct answer in cases where we can see clearly how to apply it because I'm skeptical of theories like Anscombe's. Which is to say that utilitarianism is a big tent.
Now if we look at EA, the basic tenet of EA seems obvious -- basically just utilitarianism. And from what I've seen, in practice also, EA is a pretty big tent. I don't know the specifics of SBF's case, but I think essentially no one thinks that he acted correctly. I don't know how many mosquito nets he bought, but I agree that if he bought enough, it might be that he net raised utility, and if that is so, it's something to be thankful for. But it doesn't make him some kind of utilitarian saint unless he couldn't have done even more good by some other course of action that wouldn't have hurt the ponzi scheme victims and brought opprobrium on the whole EA movement
This kind of reasoning leads you to reasoning that if he was an ineffective fraudster, it would be less moral, as he would have bought less mosquito nets. So it’s not only moral to do fraud, but you most extremely competently do fraud.
I think this being a reasonable utilitarian point to make is not a point in utilitarianism’s favor.
Even people who say they are deontologists often slip back into utilitarian arguments when they're not careful — for example, when arguing Kant's categorical imperative against lying, they slip into talking about the local benefits vs. overall harms.
The real gap, as you've said, is more about overconfidence in one's utilitarian calculus for distal vs. proximal moral outcomes. An average Joe is likely to give a lot more weight to the moral outcomes that rely on local information and affect his friends and family. The characterization of an "EA" — whether fair or not — is that they're much more likely to use a lot more explicit moral calculus and attempt to correct for proximal vs. distal biases.
In a way its very similar to Sowell's arguments about the informational economics of a distributed market vs. a central planner.
Had a similar journey recently. Started with Stack Auth, found it unusable in production due to extremely hard rate limits and bad performance even when not rate limited. Switched to WorkOS AuthKit, which works much better and supports useful enterprise features. But inclined to BetterAuth for new projects.
- Syncing external auth provider state with your user state is a bug center. It helps to keep as little state as possible in the auth provider, but there is still some.
- Refreshing JWT access tokens every few minutes is another bug center and honestly there is no need to do this if you control your own auth.
- WorkOS does not have a complete API. It is built on the assumption that you have one product per billing account and a fixed number of environments (staging, production, and they can give you another one if you ask support). You have to whitelist redirect and other URLs in the dashboard, and there doesn't seem to be an easy way for agents to do it.
Outsourcing auth does not make much sense IMO. The less you can split your state over multiple services the fewer problems you will have. Sometimes it is inevitable, like for payments, or if you need specialized databases for performance reasons. But for auth there is really no good reason if good libraries are available. To people who say that using a service will help you get started faster, none of the problems I hit with auth services had to do with having high scale -- most of them hit before I even launched.
We're working on multi-app support. The large majority of our customers only have 1 app (ChatGPT, Claude, Cursor, etc.) but this isn't the case for developers building lots of side projects.
Also working on shipping an agent-friendly Dashboard. Stay tuned :)
Would love to hear any more feedback: mg@workos.com
> Outsourcing auth does not make much sense IMO. The less you can split your state over multiple services the fewer problems you will have.
I agree with the general principle. Fewer moving pieces make for more stable applications ("choose boring technology"[0]).
However, I was wondering what you do when you have more than one application that the same userbase wants to access. I can see 3 options:
1. make them register/have credentials for each application (not a great user experience)
2. use a standalone auth server and deal with the increased complexity
3. pick one of your applications to 'own auth' and have the other applications delegate to it. congrats, you've just invented a standalone auth server that is coupled to one of your apps
I think depending on how much integration you want, either option 1 or treating the multiple applications as a single application that can do multiple things could be good options. Also, option 2 could still avoid having to worry about your auth service going down or rate limiting you. And you could avoid JWT headaches.
Perhaps there is also an option 4, which is option 1 plus a shared user information database for things like Stripe account, profile picture, etc. That is more complex obviously, but it would still solve the issues I had with WorkOS. In particular, I think it would mostly solve "syncing external auth provider state with your user state is a bug center." In particular, the awkwardness around sequencing of account creation and deletion would largely go away because that would be managed as in 1, and the extra shared information would be just that. (But maybe you would want to delete it if the user deletes all their accounts?) And you wouldn't be forced to use webhooks to get updates to shared user state. You could put it in a shared Convex database, for instance, or use some other solution of your choosing.
Yeah, I see two common patterns with people using an external auth server.
1. store everything user related in there, rely on APIs or syncing if the application needs user attributes
2. store nearly nothing in there, just credentials, MFA, account recovery info and anything else related to authentication. All profile data is stored in each app
I found that agents can already figure out how to do everything in CloudFlare as long as they have an API token with the right privileges. They are smart enough to figure out how to use the API. But the friction spot is that I need to make the API token by hand and basically add permission to do everything they might ever need to do. It would be nice to have a smoother way to handle the permissions so I can let them do everything they need to do frequently without asking for permission yet still make it easy to approve actions that are only needed occasionally, without having to guess in advance which permissions fall into which category
