This doesn’t hold up imo. Your phone analogy fits the description of a harness. That is the thing you interact with. The LLM is more like the processor in that phone. As long as it gets the work done, I will choose the cheapest/fastest because I can switch it out in a matter of seconds anyways.
You're straying away from the analogy though. If the LLM is more like the processor in the phone then you'd just compare the Apple series chips to the low cost chips in the low cost phone.
> I will choose the cheapest/fastest because I can switch it out in a matter of seconds anyways.
Sure but that's why you would go with the Apple series chip in this analogy. It's more expensive but when it comes to getting what you want done it would actually be cheaper and faster.
For some reason everyone wants to praise China's low-cost models and simultaneously applaud their move to create and restrict higher-cost/performative models while criticizing the US for also creating and restricting higher-cost/performative models. It really doesn't make sense.
If you want to say all models are the same or close enough so you'll just switch to the cheapest and fastest then market forces will allow American companies to just create those same models or, since they're open source, they'll just use those but American companies can just do that too and they're choosing not to because performance matters, approximate winner-take-all matters, and cost is important but not the only factor.
If you want to suggest that America is doing all the high cap-ex spend and then China is just undercutting with cheap models, then American companies will adopt that strategy too if it is what is most profitable and then Chinese companies will have to spend on cap-ex to get ahead and they can be undercut by American companies. But this doesn't reflect reality and China's actions demonstrate that point because they're still spending cap-ex and trying to produce higher-end models which they then restrict and blah blah blah.
Does this use levels from the original game or some custom ones? The solutions to the original levels should be in the training data, be it blogs, reddit comments, or wikis.
Unless the goal was to test how well do the large language models translate solutions in prose to actionable keyboard inputs, which is pretty interesting in itself.
Agreed. I’m surprised how often people seem to miss this. They don’t realize just how gargantuan the training datasets are for these large language models, especially for a very popular game like Baba Is You. I’m sure that both GameFAQs and the Steam forums are in the training data for any reasonably SOTA LLM both of which almost assuredly have complete walkthroughs for BIY.
I remember "Baba is Eval" (https://fi-le.net/baba/), released 11 months ago, back when Claude Opus 4 was the strongest model. Back then, I was surprised how poor was it even at the first level.
I am happy to see an another approach - and indeed, with much stronger results.
That is very true but I was surprised by how clear the “signal” was. Only Gemini really confidently solved all levels. But yeah the goal is now to include harder levels as well!
I think this view assumes no human will/should ever read the code. This is considered bad practice because someone else will not understand the code as well whether written by a human or agent. Unless 0% human oversight is needed anymore agents should still code like us.
Genuine question to people more knowledgeable: Why are politicians/technocrats doing this?
Also generally speaking e.g. in relation to chat control and so on. Do they think this is what the people actually want because of lobbying or are they aware and believe they know better? Is it literally just corruption? Or are there actual benefits and we are just in the HN bubble where most people think its a bad idea?
Politicians in democracies need a fallback career for when they lose office. Before capital controls were lifted in the 80's, economies were a lot more local: UK politicians would take positions in UK companies or institutions, French in french ones, etc. This did mean a certain amount of corruption, but it did mean politicians were highly interested in the success of national companies and institutions.
Now, most of our senior politicians go to the US after leaving office; so for consistency they adopt the belief that there is no downside to making the UK beholden to foreign companies, or becoming a nation where all the innovative professions end up building capital for foreign owners, instead of building strong UK companies. As a consequence of this, they almost compete to sell out the public. It's impossible for them to believe that what they were doing is a betrayal of their country, because that would go directly against their personal interest.
It depends on what you're referring to when you say 'corruption'.
The public officials involved in signing off on these contracts with Palantir will almost certainly be offered non-executive, board, or consulting positions with one of its subsidiaries. These roles will likely net them £50k-£100k a year for four to five years and conveniently begin a fixed number of months/years after their terms in public office conclude.
This will all be strictly legal and well within the regulations those same officials voted on for themselves (without public consultation, and watered down further by the lobbying efforts of Palantir and similar companies looking for a cut of public funds).
This is an entirely legal and extremely common practice. If you choose to label it 'corruption', that's your call.
Businesses and politicians don't care about (you), the little guy. They want your demographic, the individual outrage you feel is pointless. Nobody is going to throw away their iPhone or protest the internet because NSO Group and Palantir exist. Your outrage is Palantir's commodity.
Even among tech-obsessed ideologues, both sides roll over and accept this because it's less flattering than arguing over CPU specs. Would we really break up with Big Tech over a gold trophy and a few backdoors?
I don't think politicians across the board are corrupt. I think they're just surrounded by syncophants and special interest. Also the old absolute power corrupts, absolutely sort of thing.
I can't be convinced people go into politics twiddling their mustaches like a cartoon villain. I think that they go into it either to genuinely help, or because they like the attention. Then the system surrounds then with people who either take small bites of their ethics, or agree with anything for the chance to be powerful as well.
I am amazed that the IDM is able to produce enough high quality annotations for the downstream FDM to work, even matching the ground truth contractor annotations!
As an EU citizen I really hope we can gain some meaningful distance to the US asap. I hope my leaders feel the same. And if everything works out I think this will be great for the EU.
This is really some sort of diplomatic Streisand effect. If the US would not have been so aggressive and just string us along they could have continued to feed us their slop indefinitely without us noticing.
I believe it comes down to intrinsic interest, that he would not consider this something boring he needs to work through, but rather something fun and intriguing to spend his time on.
Disregarding the unusual age in this case, I believe that most people could be significantly better at mathematics than they are, if only they found it interesting enough.
I have used a lot of them. They’re impressive for open weights, but the benchmaxxing becomes obvious. They don’t compare to the frontier models (yet) even when the benchmarks show them coming close.
Here is what I don’t get tho: you have UX designers/engineers creating a new interface. What do you tell them? Just to do whatever? They probably spent months designing the new interface but why not fix this? They must have seen it is unusable…
