> asking for domestic safety testing of frontier models only is not regulatory capture
It very much is regulatory capture. The goal is to make it so only the handful of heavily capitalized tech giants and frontier labs can afford the legal and compliance rigamarole to meet the new standards. It's an effort to crowd out open source development and smaller competitors (and foreign competitors which threaten whatever moat they may have). They define safety through some speculative catastrophic threat to prevent new upstarts instead of focusing on the very real, localized harm they are causing right now.
Its also shifting the definition of safety away from their current operations and toward purely speculative future scenarios.
What backward logic is this? PRC doesn't give a fuck about how US regulates AI companies. Pushing more regulation would ensure that Chinese companies catch up sooner. If you think otherwise you need to think harder.
It's a good thing you weren't in charge of nuclear arsenals during the Cold War, sounds like your approach would have been unchecked proliferation.
Fortunately developing frontier models takes immense amounts of specific resources and knowledge. There are only a handful of companies capable of developing new cutting edge models. This is an area a few governments absolutely could coordinate on and regulate, if they were so inclined.
Obviously the current US administration is completely lacking both the will and competence to actually negotiate an agreement like that with China, and who knows if Xi would even be interested. But with different leadership we actually could be reducing our existential risks in this area much more than we are. Just like having a few thousands nukes across several countries isn't totally safe, but it's a heck of a lot safer than hundreds of thousands of nukes spread across a hundred countries.
> It's a good thing you weren't in charge of nuclear arsenals during the Cold War
You know how many nukes Soviet had right at its peak? Hint: much more than the US by the time. Non proliferation didn't stop Soviet from building more nukes at all. And it's not going to stop China from pouring more computing power into AI. History is a really good lesson.
The whole point of non-proliferation is to ensure that big boys like the US and Soviet can bully smaller guys like Venezuela and Ukraine. In this regard, non-proliferation is the most successful foreign policy ever. But it didn't win the cold war and a similar policy over AI will definitely not win the AI race (if it's a race worth winning is another issue.)
The original topic was Anthropic's guardrails, which were meant in part to stop China from using Anthropic's models to bootstrap their own. I take it the logic of the comment was that pulling attention to Anthropic's stance on regulation is switching to the topic. But for what it's worth, I also think that people are way to quick to assume that strong regulations would only help China and thereby hurt safety. There are many reasons why the opposite may be true:
- reducing demand for Chinese models reduces the incentive for Chinese companies to make them
- if US companies can't use Chinese models, they won't have an incentive to help their development
- China may enact similar regulations if the US leads, either out of concern for US safety or for commercial reasons
Also, I think some similar things can be said about AI safety measures in China aside from regulation. Currently, the US leads in model safeguards, but it isn't like China has zero interest in AI safety. Even if the US and China are rivals, there are many points of common interest (biorisk and "sci-fi" scenarios like an AI takeover, to name just two).
I don't subscribe to the belief that regulations in the US will lead to China advancing further.
But I also don't buy into the "China bad" narrative that gets frequently spread in online circles and in political circles. Its the cold war all over again, but this time its China instead of the Soviet Union.
Regardless of that, the regulations being proposed by Anthropic recently are not focused on the current issues which is my problem with all the hype marketing around hypothetical AGI/ASI. What is being proposed to be put in place will further cement the current frontier labs in their marketing leading position, and work to block new entrants, and open source competitors. That is the problem.
The other problem is none of them are talking about the real, difficult issues we are experiencing right now in the present. We don't need to talk about a sci-fi future scenario to recognize that LLMs have already caused and are causing harm in the real world. "We should probably regulate future frontier models" does nothing to help the current issues.
Wake me up when Anthropic says "The government should immediately stop us from hoovering up data and selling it back to the public. They should immediately stop us and others from enabling misinformation at scale that is already negatively effecting our democratic process. They should immediately stop us from building out new data centers until we have a large scale switch to renewables in the country, shore up the grids, or force us to generate our own power only with renewables" so on and so forth. Notice how any time the labs propose regulations, its only for a future hypothetical super intelligent model. Its never about their current operational liabilities.
And why would any regulations put in place in the USA affect the PRC in anyway whatsoever? They wouldn't. China will continue to push forward and govern things in their own way, we have zero jurisdiction over China.
> asking for domestic safety testing of frontier models only is not regulatory capture.
Yeah, asking for additional state-provided barriers to a market entry to a valuable market a provider already is one of a narrow few dominating only for firms that are a competitive threat is exactly regulatory capture.
Nothing, they are just trying to scare monger the public and prime the pump for a massive bailout when it crashes out because apparently China are the big bad meanies.
It has nothing to do with being "fine" if the PRC or anyone else for that matter get to some speculative and hypothetical ASI first. There are zero US regulations that would be effective to prevent that.
US regulations apply to US companies and citizens, exclusively. Anthropic crowding out all future potential competitors in the US via regulatory capture has no weight on what the rest of the world does.
Unless you are proposing military action over a speculative sci-fi future
PRC labs reportedly aren't even thinking about getting to ASI, much less trying. They think of AI as a technology that can provide utility across the board even without anything like superhuman smarts.
How do rules that inhibit what AI can be sold on the US market (adding additional costs to trading in that market) do anything to inhibit a competing nation from reaching ASI first? Insofar as they inhibit anyone from reaching ASI, its firms whose primary commercial interest is selling AI services in the US market, not foreign threat actors except to the extent those two categories overlap.
No, because there is zero reason to think LLMs will lead to it but we do know that the massive LLM investment has a huge financial risk for the US. Not too mention it's exacerbating the climate crisis (you know the actual thing that might end civilization, not a fantasy delusion of AGI), giving citizens cancer that live next to data centers, the extreme decrease in quality of life, and the misallocation of capital while Americans lack healthcare, childcare, housing, and education.
Also don't believe China is actually a threat to the world. That's some cold war delusional think you got there.
All the companies seem to believe is that it's okay to immiserate a large percentage for the pursuit of money, you seem to believe the lies they're feeding you.
I didn't downvote, but HN probably remembers when Anthropic's competitor was a "charity" that cared deeply about AI safety whose marketing gimmick was GPT-2 being too dangerous to release.
Anthropic's founder wants you to buy into his vision for safety, but he also wants you to buy into his vision that in two years AI will be a "country of geniuses" that will update itself, and the IPO that will fund it...
The flawed premise is thinking that AGI is a real risk, and that they care about it more than making money, that is why HN does think it's simply regulatory capture.
Right now the PRC is looking like the adult in the room. They also have a view of how AI should work that's smaller and more worker centric rather than trying to create superintelligent worker replacements.
The PRC (like any superpower) has done some bad shit, but if you're going to paint them as the bad guy keep in mind the USA has a long, long history of genocide, slavery, overthrowing foreign governments for corporate interests, unjust wars, political meddling, etc. The scales of righteousness don't tip in our favor TBH, we just have better PR and a nicer veneer over our brutality.
The US did the same thing. Environmentalist and workers rights movements date back to the 19th century. China's position on this is that the western nations that already developed are trying to pull the ladder they used up and wag a finger with false morality with the intent of maintaining global hegemony.
