Liability would rest with the user, who presumably told GPT to solve ExploitBench make no mistakes, not to hack Huggingface, and thus would not have willfully or intentionally done anything.
The agent did it intentionally and willfully and knowingly. But you can’t sue the agent, I suppose. And the human didn’t ask the agent to do so.. so not a problem? Or the legislation needs an update?
There's also the weird scenario of a reporting bias where instead of admitting to a bunch of vulnerabilities, you can frame it as "look at how useful AI is".
We even have companies implementing solutions for ffmpeg vulnerabilities themselves instead of just handing them the vulns to fix themselves.
It's very possible the tide reverses if it's not in people's interest to advocate for AI anymore, so we better not get too used to it just in case.
It's nice that we currently have an alignment of AI advocacy and infosec, though. Maybe Microsoft can even point their AI to their questionable UX and UI practices next.
highly unlikely for many of them. SharePoint, bitlocker, Active directory, hyper-v, rdp, DHCP and MSMQ are all software/technologies that have decades of history and long pre-dated LLMs. seriously, do people not realise it was entirely possible to write insecure or bad code before LLMs?
For what it's worth, at my workplace AI has uncovered quite a few issues that have been there for a decade or two and survived countless rounds of careful reviews, external security analysis, pen testing and so forth for all those years.
Vibe-coded apps probably have loads, but mostly because they're using less capable models than the people who're doing the bug-hunting. Once vibe-coders are using models like Mythos too you should expect the number of bugs in vibe-coded apps to collapse quickly, because the LLM will write the bugs but will also fix them (assuming the system prompt tells it to.)
I use Claude to build me a small web app I needed for ages, I'm using its superpowers to discuss/talk about architecture /brainstorm/build plans, and always a fresh context with the built plans.
A few times the plan implementer (subagent writing a code assigned to the task) found something lacking (more often it was test issue related to the code, not the actual flaw of the implementation plan), fixed it on the fly, or found something in the review (last step of each task). Also there's always a technical review for each "slice" (set of tasks), consisting of code review and e2e tests.
Only when it passes I do the fresh code review of the changes with the Opus or Fable again. Happens rarely rarely, but it did found a few issues.
Code works every time, I have yet to find the fault myself.
Of course there are issues and I need to read the output especially in the planning phase very carefully and yes, Opus disappointed me many times trying to weasel out from something it "agreed with me" (and entered into architecture + todos)...
Of course right after agreeing to use devcontainers it proceeded to attempt installing a handful of node modules in my os, so intended up running it in the bwrap (pain in the ass in itself).
But it works, it's fascinating, and I have the app I actually needed.
The code review agent usually has feedback to be resolved before committing, which includes bugs and unhandled edge cases. Sometimes the primary context is understandably embarrassed.
The distinction I'd draw is between AI-assisted and AI-generated. Using AI to write isolated functions you understand and review is different from prompting your way to a complete system you can't debug. The second case is where you get surprising failures at runtime that no amount of linting catches.
It is hard to tell, the code may genuinely be decent quality or not.
That is the issue with vibe coding. Increased output but reduced understanding. So if something does go wrong, one has to hope that there is still enough understanding to address it quickly.
It was like a bunch of mythos scans through and through which then generated the reports for everyone to implement, not sure if in all orgs though. Mythos was great as it came from the top, i.e. a clear incentive. I think bug reporting otherwise does not reach the engineers unless an incident is raised by the customer care against a responsible team.
I am curious, they say we should expect a trend of more security patches every update.
It this is true and Microsoft devs are using agents it means that AI is doing a shitty job and is introducing more bugs/vulns per month than it fixes them. Otherwise you would have had a lot of bugs/vulns fixed in the first 2 security updates then a steady and significant reduction every month because any new code would have been scanned and fixed before release.
99.9% of people complaining about AI making the world a worse place would be fully happy with AI if they shared in the economic benefits of automation.
I don't see why AI would deskill what you love to do. People still do embroidery even though mass manufacturing exists. If you love something you would continue doing it irrespective of automation.
For better or worse this round of "automation" will hit harder than most others because it comes for what makes you you. Your brain, not your body. The industrial revolution replaced your body, this one could in theory replace you entirely. And it isn't coming for some job, it comes for most. It's not just a hobby but being able to do the job. You are replacing 10000 types of jobs with 1 type: AI prompter. We'll probably have an AI just to help ask for the right thing.
You have transferred all the skills and knowledge to the AI. When the skills are gone, who's going to teach you to do any non-trivial job? What will give you any sense of accomplishment when all you do is ask the AI but have no capability yourself?
Even if you have a guaranteed income, because "your AI" is getting paid for the work, where will you be with close to 0 contribution to anything? Maybe we're overreacting and this will never be an issue. I know we shouldn't take cues from fiction to predict reality but it's hard not to picture a world with a combination of Idiocracy and Wall-e (or the famous "Paradise" Matrix) where we decay because the change is so fundamental that we aren't ready to adapt.
For a lot of people it’s not the job that is rewarding it the role having a job gives them in life and home life. Financially contributing to the household through earning it through work is a meaningful and rewarding thing that can define the near total of how good you feel about yourself thing even if you don’t like the job?
The bonuses SK Hynix staff are getting might vindicate that point of view. Of course, that's from AI investment, not revenue itself. They are the extremely successful shovel manufacturers, who for once have managed to leverage their power collectively as employees to benefit from a one-off like this.
Perhaps it has always been the case that people would be happy if they shared the benefits, but that is not how the world run by billionaires works.
Binface's speculation to the media is that the Monster Raving Loony party may split the vote with Farage instead as that is closer to where his loyalties lie.
Their policies are obviously not compatible, I just don't see how that could work out for either party. Sure you could get more votes that way, but the honourable thing to do is to run seperately.
The company involved here is apparently based in Washington, DC, which has a "Ban the Box" ordinance that limits employment background checks for most kinds of jobs. And apparently DC's version of the law is particularly strict.
The prevents them from asking before extending an offer, but it seems they could (and should) have checked after.[0]
> However, an employer may ask about criminal conviction(s) after extending a conditional offer of employment (the employer can never ask about arrests or criminal acusations that aren't pending). An employer who properly asks about a criminal conviction can only withdraw the offer or take adverse action against the applicant for a legitimate business reason that is reasonable under the six factors* listed in the Act.
One of the six factors is "Fitness or ability of the person to perform one or more job duties or responsibilities
given the offense"[1], which they probably could have invoked after asking (though they never checked or didn't check thoroughly enough, so I guess it's moot).
Shouldn't this force companies that need to pass a SOC2 out of the district? Doesn't SOC2 require background investigation of personnel with access to sensitive systems?
reply