Democracies require conserved stake to function. How individuals initially get that stake is a degree of freedom, but if the stake can be printed freely, the system is unstable.
For working implementations of democracy, look at: invite trees, shareholder corporations, proof-of-stake distributed ledgers, etc.
This framing of security as something that belongs in the harness is completely wrong, and I hope no one is relying on a correct harness to keep their agents isolated.
VM, or even just a container will do. The agent should be able to run as root in its environment and do whatever it wants. If you can't give it that, you aren't sandboxing correctly.
Article isn't about agents. It's about the inference engine itself being exploited by a malicious LLM output before it is ever sent to your machine or harness.
"malicious LLM" is just a bunch of weights. It runs on, say, Llama.cpp as regular user in separate account, often on its own hardware.
"malicious LLM output" is just a Markdown formatted Unicode-encoded text, produced by Llama.cpp and printed on the screen by my python API script.
I control the input.
Let's assume that "rogue LLM weights" from HF produce "rm -Rf" instruction. it never gets to shell.
And how that "malicious LLM" will disrupt and hack me? With swear words and em—dashes? :-)
(Adding this philosophical point: Black.Mirror.S07E04.Plaything is probably the closest scenario to what you are describing?)
Single turn set-ups may work like this. You control the thing you input, the LLM outputs something and then nothing happens further for that specific context. (Simple question/answer style interactions..)
(Multi-turn) tool calling set-ups however, you need to store the LLM output, the results of the tool calls and feed it back into the inference engine and get the output for the next tool call and/or turn. So yes, print the LLM output on screen and verify it, but maybe the LLM is able to figure out how to hide payloads from your specific set-up. E.g. perhaps it can inject raw ANSI escape codes into your terminal, with which it would be trivial.
Now you have a situation where the true chat completion payload and your view of it have significantly diverged. The LLM could in theory then try (one-shot) to hide further exploits in the hidden payload. E.g. a json parser escape specifically for the inference engine, giving it a means of RCE (although, one can debate whether this is really remote ;) ). Then from the RCE gain a shell, from the shell get access to some privileged device on the current network, and then...
This is great point, actually - "you hit the nail on the head" :-) and I would like to reply.
I am running a very long sessions with LLMs via custom python scripts. Technically, one may call them "harness" but that would be just laughable ... It's literally python script using direct API calls (Vertex in my case) and maintaining the "living session" with all turns etc and also doing the explicit caching. I'm not using LLMs for coding. That hopefully answers another comment regarding why I brought up MD – this is how LLMs output responses to my prompts.
And this is the thing: I fully control input and output and I just know it can't use any other tool. It also, as I said, runs on separate hardware if it is "obliterated" model or runs in GCP for me.
In my setup it is impossible for LLM to get anything hidden with one-shot or gain a shell, as you mentioned.
Did I understand you correctly or I missed something?
Thanks for your points.
Why are you talking about markdown? LLMs output bytes and llama.cpp has to process those bytes correctly. What if the LLM outputs a null byte and causes a buffer overflow somewhere and starts writing into executable memory?
Again, we are not talking about agents or your Python API script but instead talking about exploitable flaws within the inference engine itself. It wouldn't output `rm -rf`. It would output literal CPU instructions that llama.cpp would start executing directly. The payload would never get back to your Python script.
I think if you are convinced you are sandboxing an LLM properly, you almost certainly are not. I think it is essentially impossible to have a frontier LLM with enough access to be useful without also giving it enough access to do damage if it's compromised or just goes off the rails.
If you do not provide access to tools the LLM cannot do anything other than generate tokens. So really it is not about sandboxing a LLM but more about having control over what tools can be accessed and what they can do. Tools can be sandboxed depending on the sophistication of the tooling. A calculator tool for example is trivial to secure. Ensuring human approval allows for useful use cases and models trained to gate permissions work. A super intelligence with a weaker approval gate will be able to subvert. Inversely a super intelligent gate should be expected to prevent subversion by a weaker model.
Unfortunately it is not so simple. Once you provide a source of intelligence that is accessible over the network you are supercharging any software that can access it.
Web interfaces (chat) designed for humans can very easily be used by programs, you don't actually need API keys. Any program which can submit queries can then be subverted by its input. Malware can definitely find corporate chat interfaces like Teams Copilot. "Business intelligence" systems can also be leveraged, they rarely have good ACLs.
Are you saying that LLM's will be able to exploit novel hypervisor bug with such ease that even a vm not running with any kind of network connection is a threat? I find this hard to believe. All the escape stuff I have seen has been around very poorly sandboxed agents.
"The agent should be able to run as root" The article is right – we are doomed.
If This ^^^ is The Conclusion we, as an Industry, have arrived to after 40 years ... It's sad.
I use my own LLMs ("Personal AI", anyone? ANT-PAI-XT 486? LOL") with my own scripts (I do not use "agents", "harnesses", "agentic teams" etc) and this setup processes my own prompts.
It happily runs on its own Mac Studio where I also may watch a movie later on.
(Edited: not as root. Just ordinary separate AI user account in MacOS. And no HTTP access either.)
> you could also just get the AI a laptop and apply the traditional tools to manage user laptops
This is what we're doing right now. And it works as well as it does with people.
> If we are treating ai agents like people
Problem is, industry is very reluctant to even hint at a thought of maybe tentatively anthropomorphising LLMs even a little, not for real but just for system design. This is entirely backwards. Entertaining the notion, even just a little, immediately suggests additional approaches.
Because we don't just restrict end-point devices for human employees. We also have two other things:
- Laws and bylaws and economy that makes losing your job a real threat to health and life of yourself and possibly your family - this one does not yet apply to AI, not for now;
- Methods and codes of practice to structure organizations in a way that limits the amount of damage any employee's unreliability or malice can do to an org.
TL;DR: That's a long way of saying: let's actually start treating AI agents as people operating laptops, not as being the laptops - and talk a little less laptop lockdown ("harness security"), and a little more about not putting people-shaped things into jobs requiring machine-level reliability.
Yeah, I just wish seccomp worked per user. So you could define a policy of which users can do which syscalls, and then that follows them no matter which application they start.
This is also the democratic way, arguably more democratic, it's just probabilistic. Juries are supposed to be randomly sampled from the jurisdiction's population (a kind of democratic process).
The ability of a jury to judge the law, and not just the particular crime is a feature, not a bug of the justice system.
This is how the system is set up, and it is as legitimate as action through a legislative process.
EDIT: the parent asked a legitimate question in good faith, there is no need to downvote them because you disagree with their implied current understanding of an issue.
Not sure why your comment is being downvote. That is in fact the best explanation for what is going on here, and the outcome has little to do with the competence of the prosecutor, as is perhaps alluded to by the original comment.
It's good to see individuals on a jury acting according to rational self-interest.
"Cancer isn't a single disease" is often shouted from the Dunning-Kruger peak of cancer research. I think it was popularized by an explainer video on Youtube, and now it's being memed. The people who think this way want to legitimize funding a particular research niche that they benefit from.
A more sophisticated view is that cancer (one idea) is an problem of rates (like most things in biology) and there is a tumor burden per unit time from whatever causes vs. the immune system's capacity to detect and destroy them per unit time. As humans age, the balance tips in one direction. If the balance is too far off for too long, tumors accumulate, doctors take notice, and assign whatever labels to the condition.
Looking into particular causes of tumors (and calling each cause its own cancer) can create a vast research program that employs a lot of people, since there are so many mutations that lead to tumors. But interrupting the development of a particular mutation is not going to have the same society-changing effect that solving the fundamental issue of rates would.
No, cancer is actually many diseases. They hold some commonalities, but it’s not one thing. That’s why chemotherapy can straight cure some cancers and do nothing for others. That’s also why we have hundreds of chemotherapy drugs.
Also, not even all cancers have tumors. You have solid tumor oncologists and then oncologists who specialize in blood and lymphatic cancers, “liquid cancers”.
In addition, cancer does not arise from one mutation. It’s a set of mutations that allow cancer to occur. You need mutations for growth, mutations for preventing cellular death, mutations for stealing energy, mutations for hiding from the immune system. Your body, right now, has probably millions of cells that could be candidates for cancer. They either kill themselves or your immune system nukes them. You really need the stars to align on this stuff to get actual cancer.
And then, it’s not like the cancer stops mutating. Cancer becomes resistant to chemotherapy through mutations. Cancer is “smart”, it will even temporary slow its own growth (metabolism) to intake less chemotherapy through treatment.
>> "Cancer isn't a single disease" is often shouted from the Dunning-Kruger peak of cancer research.
Literally first phrase on Cancer Wikipedia entry.... [1]
"Cancer is a group of diseases involving uncontrolled cell growth typically resulting in tumors with the potential to invade or spread to other parts of the body...."
I should have expected this level of discourse from 2026 HN.
Here's another way of framing things that might encourage better discussion:
Suppose in a few years, it was announced that an LLM did discover a cure for a cancer. What approach would you expect it to have taken?
1. Developing an intervention for every possible mutation that leads to loss of cellular identity, and uncontrolled growth.
2. Preserving immune function later into life.
3. Preserving genetic and epigenetic accuracy later into life.
Maybe it will be 1. But I would bet on 2 or 3. Which both assume that by reducing the difference between rates, the phenomenon of cancer could be reduced or eliminated.
And this business of rates is also why getting the immune system—such as via personalized vaccine—to recognize a tumor is functionally "curing cancer". The problem is it isn't one disease in the sense that different tissues, cell types, and systems are affected leading to different parameters in the design and delivery of said vaccines.
Is there something like a meta-Dunning-Kruger, where someone invokes Dunning-Kruger in a way that itself is a potential Dunning-Kruger phenomenon?
I'm inclined to believe that the thing - that everyone intelligent says is complicated - is actually complicated. I would be surprised if the thing - that everyone intelligent says is complicated - is actually simple.
This is almost like an application of the conspiracy theory gambit where you try to find out how many people would have to be "in on it" in order to disprove the theory's likelihood. Like flat earth.
The answer, of course, is that the taste-making apparatus inside the typical tech company is now entirely imposters (non-technical, non-power-users), who continue to LARP as visionaries.
The people who notice broken or degraded behavior (devs and power users) and conceive of good designs are not in charge of what goes into the product.
So you shouldn't expect it to get better, only to continuously change so that the imposters can point to a difference that they were responsible for, knowing that upper management also has no idea whether that difference was actually an improvement.
The solution is to gut your product org, replace management with leading engineers, and hire some of your most fanatic users. They don't need to do anything other than give their opinions, and use the product every day. Put them in a room with your top engineers, and the software will mysteriously get better.
All of this was obvious decades ago, but now everyone wants to have a do-nothing job at a tech company, and to help their friends get one too.
The result is imposters everywhere; "imposter syndrome" was brought into the vernacular to normalize a lack of expertise.
"No one knows what they are doing, it's okay to not know how to do your job". That's what your (product) manager tells themself in the mirror each morning. Turns out: skill predicts quality, and an org ships quality proportional to the skill of the decision makers.
> The solution is to gut your product org, replace management with leading engineers, and hire some of your most fanatic users.
It’s not surprising that this sentiment is popular on a message board filled with engineers but I’m not so convinced by it. Nor do I think the current state of tech companies is acceptable, to be clear. But there needs to be some level of management outside of the engineers themselves. A lot of talented lead engineers I know are terrible people managers, or ineffective project managers. And that’s fine, because there are other people to do those tasks.
If your utopia is a bunch of engineers given freedom to do whatever they want, however they want to do it… I suspect the dream and the reality will not match up. In the scenario you’ve outlined I strongly suspect you’ll have infrastructure on its third rewrite, full of power user features and tweaks… and near-zero adoption from newcomers. Product fit is a thing that needs to be managed!
The OP is right most production code is full of errors, and suffers from code bloat and feature creep that impacts UX.
Is this really the best we can do? It strikes me as unacceptable to ship errors to production, yet it's commonplace.
Code quality matters, and being in touch with users/dogfooding matters. But that can only address the underlying problem by eliminating the low-talent developers who win by numbers.
How to solve the relevant problems is not something hiring and recruiting teams can assess. So, hiring process uses proxy metrics like having a HS and CS degree, years of experience, brand name companies, list of tools, "best practices", trends, etc.
Teams of self-congratulating code monkeys using all the latest 'industry standard' libraries and tools, don't accept responsibility for their errant code. Instead, they blame (the user, his browser, the library), make excuses, minimize "all software has bugs". They can't fix the bugs they've created for themselves, but it can't be their fault, after all, they're doing what everyone else is doing.
The answer is to send them packing. Hire by talent, not by proxy of a hiring team (who don't know how to think/use the aforementioned proxy metrics to gauge talent).
Intelligence cannot be taught, but it can and is discriminated against. That is another problem with these faux 'team-player' follower types. Anyone on the team pointing out the code is a disaster and an attitude to fix it is not going to last. He'll be fed pushback of the same crap excuses for why things must be done the way they're done.
When a talented developer does deep dive analysis to identify root causes, most on the team don't get it. The scary bugbear of coherence is too hard and threat to the team's shared narrative.
Again, the only way to solve that problem is to send them packing.
Yeah I hate that you’re right. Blender is an example of OSS software which is actually a joy to use and they have product designers. But I think there is a massive gap between good and bad product people. No product person would be better than bad ones, but for truly great software you probably want some good ones.
That being said lots of good OSS exists purely built by engineers so I think it’s really the case that good product people help a lot and bad ones hurt a ton.
Some developers also take an active interest in the design and feature set of the product they're developing. IME those developers can often produce good results once a general direction is set if they have access to real users to collect feedback. But it's important to recognise that these are essentially two completely distinct skill sets and mindsets and it just happens that some people have both.
It's not really any different to the old arguments when continuing a career beyond senior dev level meant moving into management but there was no guarantee that just because someone had been a competent developer they would also be any good at managing anything. Some were and produced great results - in some cases informed by their understanding as former developers about what would be possible and what it would realistically cost - but in other cases the Peter Principle hit early.
I'm not totally convinced by OP's whole point, but I want to point out an issue with this argument of yours:
> lot of talented lead engineers I know are terrible people managers, or ineffective project managers.
A lot of people who can read are also terrible manager, but that's not an argument to put illiterate people in charge. Not every engineer has what it takes to be a good manager, and that's fine since you don't need this many manager anyway, but I'd argue that for a management jobs, understanding the craft of the people you're managing is as important as literacy, because the most important role of a manager is make decisions and you cannot make good decisions about things you don't understand.
That being said, engineering is only half the takes in software, the other half being Design and you cannot have a manager that is merely an engineer without design skills (otherwise your product is going to have the user-friendliness of Arch Linux).
The problem, as I've experienced it, is that companies don't try to determine what engineers also have managment aptitude. They simply promote whatever engineers are good at writing code.
And furthermore, in the last decades engineers have become infantilized to such a degree that most don't want to learn to be a good manager and prefer to stay Peter Pans for ever. I believe it's an outcome of the 80's and 90's "nerd" culture because, if you ask the elders, it didn't use to be like that in the US, and it's much less prevalent in Europe (but alas that's far from being the center of the software industry).
I’d say that describes me but I don’t see it as a Peter Pan thing… I’m very technically skilled and I want to keep using those skills. I know the role I want, it’s what I often see advertised as an Architect role. But companies have an order of magnitude fewer of those roles than people management roles. I don’t know that it has to be that way, a lot of projects suffer from a lack of top level technical direction more architects could provide.
In an organisation made of mature people, some of the highly skilled people would recognise the need for them to become people managers (not Architects), and so the organisation would benefit from managers who were actually competent, having done the work themselves before.
I see complaints about companies not promoting from within their ranks, and choosing MBAs for managers, but the grim reality is that the engineers themselves don't want to step up and grow up.
You’re really stuck on the maturity thing here and I’ve got to be honest, I don’t get it. Becoming a people manager is not “growing up”, it’s simply a different job.
My skills are not in people management. My skills are technical. The organisation would not benefit from me being a people manager because I’m not good at it. I don’t understand why I should opt to do a job I am neither good at nor enjoy just for the supposed benefit of my employer. Part of “growing up” is realising that employers are replaceable and sacrificing yourself at the altar of any given employer is a fool’s errand.
Rather, you're the one stuck thinking this is about yourself. If you read more attentively, you'd see I carefully phrased my reply to avoid mentioning you.
You say you aren't good at it, but have you ever tried ? I suppose not, so you'll never know if you'd have been good at it. This is not a quality that comes innate, it's something that one must learn and cultivate.
The important thing here is the observation that engineering companies used to be filled with engineers that were both *technically competent* and *willing to become a manager*. That's much rarer nowadays. Why ?
Has it not occurred to you that the job of manager has changed in this time? The people managers I am friends with bemoan spending half of their lives doing paperwork for performance reviews. It wasn’t always that way. To be a project manager is spend half your life in JIRA workflows.
I don’t want to become a manager not just because I’m not good at it, but also because it doesn’t look even slightly enjoyable. Why would I do it?
> Has it not occurred to you that the job of manager has changed in this time?
Maybe, but that still doesn't answer the question. In my many years working for a FAANG in Europe, almost all good managers were non-US, even in the sister teams on the US side. On the US side, the best managers I've worked with were, with a few exceptions, first generation immigrants from India, Iran, Lebanon or various European countries.
> A lot of talented lead engineers I know are terrible people managers, or ineffective project managers
Ok but non-technical “managers” are way worse.
Has more to do with role focus and hours in the day. I would rather have a dev dedicated to PM than a “manager” larping as one cringily trying to keep up with us totally ineffective to lead a technical team.
Tech companies are anything but technical anymore since they got rid of all the talent.
> The solution is to gut your product org, replace management with leading engineers, and hire some of your most fanatic users. They don't need to do anything other than give their opinions, and use the product every day. Put them in a room with your top engineers, and the software will mysteriously get better.
I feel like there's this one random comment on HN that should be adopted on its merits alone, but everyone will ignore for no good reason. This will never happen because entrenchment and bureaucracy will strangle every product organization and every organizational reboot attempt.
It's probably the right solution, but the resistance built into organizations assembled by bad decision makers cannot be overcome. Maybe the best you can do is try to find a legal way to spin-off a competitor with a truly fresh start.
I've worked with good product people. They come up with good ideas, have great suggestions, leave latitude for implementation issues to guide some of the design, and work well in the constraints. It is kinda amazing because you might be given a good idea/ticket, and it just flows smoothly and naturally (building on top of existing good product design of course)
Having them around is like 100000x better than having just random engineers be doing product design. And "oh just listen to your power users"... come on folks, surely we know about designing our software into a corner right?
I'm saying all this but am very sympathetic to the pain brought by bad product people (or just like ... mid product people. It's a hard space)
I started using the term “opinionated software” a little over 15 years when I was starting a new company. I built what I wanted as a participant and a fan of the sport.
Lots of other people had their opinions on it (“you’re doing it wrong and everyone wants it this way,” “you must have this feature” which barely anyone actually uses… the power users), but I held fast with my opinion (did usability testing to prove the it was easier to achieve the goals to back up my opinion) and I’m glad I did. It changed an industry for the better.
Excellent usability and functional design that achieves a goal better and easier will win people over.
I also built it with an API so that people can build their own versions of it with the same data, because others also have strong opinions, and I whole heartedly respect that. Some interesting things have come from it that are more targeted for specific users and situations. Pretty cool to see what others make.
While I don’t know how opinionated the guys that created Dark Sky were with it, I would guess that it’s a good example. It was different from all the other apps in their category in the way it surfaced information. Their new app is good (not quite as great as the original), but they’ve really spent a lot of time making a really usable and functional system.
Edit: About that same time in my life, I also realized that I no longer just wanted functional, I wanted both functional and form. I wanted well designed functional stuff. Turns out that people like that too.
If an idea of what the software is supposed to do conflicts with what the software does or intends to do, then the idea is wrong. What product people do (even good ones) is stretch the product to scratch every itch and address every user request, despite the software's opinion. That's Zawinski's Law. (See also the Antoine de Saint-Exupéry quote, "Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away." Which unfortunately does not have a pithy name to my knowledge.)
The best thing a product team can do is learn to say no.
Wasn't this the whole point of agile and self organizing teams?
Then managers with 10 different PM* acronyms in their email signature got hired in, with no engineering experience, to lead product and software teams?
I don't know if that's what agile was supposed to do, but replacing management with "anyone but leading engineers" is exactly opposite of what GP proposed.
> I feel like there's this one random comment on HN that should be adopted on its merits alone, but everyone will ignore for no good reason.
Because it's populist slop, just like how every long enough discussion about government incompetence will lead to someone proposing all political leaders should be paid minimum wage to fix the problem.
Don't forget, the entire Web 2.0 winner ecosystem of companies that make up our modern Internet was founded by Engineer GodKings who leveraged their product prowess to gain a distribution advantage: Facebook, Dropbox, Slack, Stripe, Github, Whatsapp, Instagram etc. They were also the ones who transformed their companies to what they are today, nobody forced them to do this. Contrary to populist opinion, it's not because they were stupid or brainwashed, it's because the proposed scheme has known, predictable downsides that a leader actually has to reckon with while internet commenters only have to worry about preserving their egos.
Taking out the trash is not populist slop. If you have a bad product org, the right thing to do is replace it. If you have a bad engineering org, the right thing to do is replace it.
It is difficult to draw any conclusions other than you like to keep things broken.
Don't blame it all on entrenchment and bureaucracy. Rather show me the engineers who want to become management.
I have thought of the same solution myself before and came to the conclusion that I, for one, wouldn't want to be leading anything even if I had some ideas on why the product sucked.
> Rather show me the engineers who want to become management.
You're talking to one. Wanting agency and responsibility and having the drive to put in the work (for potentially brutal hours) is not exactly a common trait. It's a founding engineer attitude. It should be no surprise that I'm a founding engineer of an LLC of one. I understand that it is not for everyone.
But I'm pretty confident that the blame is well placed.
The real.answer is that people stopped making things with care and genuine interest.
Everything should be delivered in the fastest way possible, in the shortest time possible, on the tightest budget possible.
This is a recipe for creating a mess, in which AI has now pushed the limits and we are seeing how failure is now per-default embedded and shipped to users, who are now beta-testing products continuously.
> Everything should be delivered in the fastest way possible, in the shortest time possible, on the tightest budget possible.
because companies that did this is rewarded with more users and more revenue (than a hypothetical competitor that didnt). Darwinian natural selection is about survival of the fittest, not survival of the "best looking" or "best performing". And the determination of fitness is by the actions of consumers, not what they state verbally.
In a few years will have a cemetery of software and products locked behind DRM or requiring a remote connection to a server that won't exist anymore, because it's too expensive to maintain.
It is already happening, the effect will be much more amplified sooner
The days where middle and high-end goods were of a relatively higher quality were also produced under a capitalist system, though. The US was capitalist when it built my parents' normal washer/dryer combination that lasted for 25 years with no interruptions to service, and it was capitalist when Google was first invented and you could conduct literal string searches on the internet. I don't think there's anything inherent to "capitalism" that causes degradation of product lines that doesn't also cause the creation of the product lines in the first place.
My suspicion is that the "great enshittification" has more to do with the swelling in numbers and purchasing/borrowing power of the lower classes. The middle class now has worse computers and dryers [1], but the lower class now has computers and dryers. Corporations can still make colossal sales numbers with degraded products because they're selling to people for whom possessing a product at all is a major life upgrade, and aren't worried about "the best version of this product". The same is happening to movies and TV.
[1] Many will point out that on paper and in certain metrics modern goods outperform past goods, but I think the extremely high prices commanded by quality vintage and near-vintage goods across all genres of product lines speaks for itself.
We have been beta-testing the outcome of incompetent software production processes long before AI. But LLMs do accelerate the slop production. And if what you're shipping is going to be a bloated buggy mess anyway, then LLMs really do make you more "productive".
There are good PMs out there watching in horror as their peers make changes and features that are obviously doomed (to anybody with a functional brain). I also write as much code as the median engineer on my team (skews junior). Please don't write off the entire profession based on many (most?) being awful - if you work with a good PM some day I promise you will find there is an important point to the job.
I think it's a question of integrity - I have reached a comfortable ceiling in my career, but doing the dumb shit would have got me further, faster. I don't begrudge the peers who have sold out when the delta is $1m+ a year but I personally got into CS for the art, not the money.
A lot of people know the exec bandwagon things are going to fail and do it anyway because of incentive misalignment.
> The solution is to gut your product org, replace management with leading engineers, and hire some of your most fanatic users. They don't need to do anything other than give their opinions, and use the product every day. Put them in a room with your top engineers, and the software will mysteriously get better.
This will just lead to massive selection bias. If all you care about are power users go for it. Otherwise you end up with a complicated system that is going to put off new users.
You can even have different power users who likes different parts of the software. Now it's totally possible to end up with extensive subsystems that don't really gel with each other
My anecdote for this is the the recentish redesign of Musescore. Tantacrul, the ux designer/product manager for Musescore has an hour long video on how he redesigned the interface and UX of the software. He is also a musician so you can call him a power users if you want but that was not the users he had in mind when redesigning the UX
> This will just lead to massive selection bias. If all you care about are power users go for it.
Simple things should work. The types of errors described in the article about feature creep and errors with simple user flows are all too common.
Just surf the web with the developer console open. It's not just "an error was thrown", but the kinds of errors and how they manifest. A rejected promise after localStorage access was blocked, after user clicked a submit button on a form that went through twenty seven delegation calls of Angular js using backspace-escaped method names to handle invalid HTML, and now the form can't submit because they did it using the latest tools, as per their resume. Crap like that. The level of quality of production web code is the cause of the poor user experience and outright failures. It is unacceptable and its cause is directly traceable to the design decisions and skills employed in its production.
Part of me wants to agree with you; I've seen a lot of lousy PMs in my career. But when placed in the driver's seat with the same mandate to bring in more customers and revenue (or else), I've also seen developers drive products straight into the ground just as hard.
It's a nice fantasy about nerds being able to do better at everyone else's but that's all it is, a fantasy.
Yup. This is basically what Steve Jobs argued throughout the 1980s and 1990s, and championed as a philosophy. Adopting it, took Apple from being a struggling tech company on the verge of bankruptcy, to the dominant personal consumer technology maker it is today.
A reason is different performance metrics. One bonus depends on "make number go up". The other realizes that making "number go up" degrades the experience for the product as a whole.
An example. I worked for many years at a major satellite TV company. The customer support department were spending a lot of time and money handling customer calls due to signal loss because of inclement weather, and they wanted something done about it.
Department X pushed for removing the 1-800 support number from the on-screen-display, because of the "quick win". No visible phone number means some customers would just give up. Who cares that this would frustrate customers even more.
Department Y pushed for reworking the UX flows around helping the user troubleshoot the problem, point them to their DVR recordings or alternative shows on broadband if connected, and automatically rebook the interrupted shows.
Traditional engineers are "better" at making things work in a functional sense (eg: aircraft) whereas product people are optimised to make "better" selling products (eg: Labubus - functionally useless but Black Friday riot desirable)
Devs and power users create something like libreoffice or thunderbird. They are incredibly resistant to any change for any reason and hang on to what they already know rather than what’s best.
The reality is most people actually like the stuff Apple and such are putting out and like modern UX more. The only bad stuff is deliberate enshitification which is pushed by financial incentives rather than product people being incompetent.
Mostly agree and would add that product managers should be responsible for the commercial success of their products. That means spending more time deciding what market to be in, how to differentiate, how to price, how to drive adoption.
For the most part these are not strengths for devs and power users.
Sorry, didn't see this reply till now. I think a lot of PMs are judged just on shipping software, not necessarily commercial success, or even just production adoption.
If software is solved then the power users can be in charge because there is no technical moat. Competition should thrive as imposter-ware flounders, but it doesn't seem to be the case.
I've actually had the opposite problem much more often: Managers who were promoted due to their skill as a developer who have no skill in management or leadership.
I see where you're coming from, but my experience says (TLDR): lack of dogfooding.
In my current job, I'm building a system which has replaced the "SaaS" we used before. Not only was it fairly expensive, and sluggish, it was also not good, and in some parts even bad. And that fits a pattern I've seen before: that SaaS product, a survey platform, was built by a company that builds software, but doesn't use it. Engineers nor management are involved in making actual surveys and processing the results at any scale. The result is a system that requires bizarre hacks to make the system do what you need it to do. OTOH, I work right next to the people that use my software, and some of which have a lot of experience. We're small, independent, there's not much management involved, and everyone is aligned, and we all talk to each other.
Not knowing what your customers actually need is one of the big faults of software engineering, and in large companies, architects, engineers, and management can be quite far removed from their customers' experience.
> And no, HN is not social media in any normal sense of the word. The pedantry involved in that comparison is extremely tiresome.
Whatever you want to call it, HN has followed a similar trend.
It's rare to see the authors of small, but interesting/innovative, projects show up in the comments, surprised to see their work on the front page. That used to be common, even the default, if you look far enough back.
Now the front page is current events and marketing campaigns. I don't think I've seen a single software project here in the last year that wasn't already extremely popular, or being pushed by a company with a marketing budget.
In theory AI should have helped. I know people are still making cool stuff, faster now with AI, but it's harder and harder to find it.
Why would generative AI help stop enshittification? If anything AI has made low-effort slop far more common on the front page and sometimes it gets voted up because of a snappy headline and few people attempting to read it, particularly if long-form and initially convincing.
I agree with everything you said. Most open source projects are limited by contributor labor. Generative AI does help with that problem, but it introduces a sea of vibe coded slop as a side effect. Truly a Genie/Jinn.
This was always going to be the case, no genie situation at all.
Information sharing networks with humans in it can only track so many things, or spend limited time on consumption. The more stuff on the network, the harder it is for things to be seen. The stuff that gets seen is content that is evolved to gain attention, or is resourced to gain attention.
> Regardless, polymarket seems to be on balance corrupting, by monetizing and normalizing use of inside information,
This is a common take on "inside information", but for most people this opinion is totally unaligned with their own goals.
The people who benefit from "no insider trading" in any market, are a small group of active traders, some institutional, some not.
For literally everyone else, insider trading is a net win. Insider trading improves price discovery. If you passively invest then you benefit from the price being more accurate when whatever fraction of your paycheck goes into the market.
I don't know your own situation, maybe you are one of those few traders who needs information to spread a certain way in order to make money. For everyone else, don't be fooled into promoting an idea against your own interests.
In fact, a lot of people claim that the main point of prediction markets is to give the general public better predictions, and insider participation actually helps with that.
This is missing the primary reasons insider trading is bad, which are that it's an information theft incentive against employers, and worse, that it's a sabotage incentive.
Clearly the people who benefit from insider trading in any market are those doing the insider trading, not all market participants.
The argument is not that Polymarket et al are "insider trading only" but rather that insider trading in those markets is not regulated so people can get ahead of trades based on confidential information and make a lot of money off of all the suckers gambling their money away on ridiculously frivolous bets.
If you don't see the problem with that, you're complicit, misinformed or brainwashed.
A similar issue is that of market manipulation, since many markets in these platforms can be directly manipulated by participants in manners as easily as spamming some words on an earnings call.
> If you don't see the problem with that, you're complicit, misinformed or brainwashed.
The problem that I imagine you see with this, is that it doesn't conform to a particular, special, notion of fairness that you think the market should have.
Informed parties have an edge over uninformed parties. This edge is "unfair" if you believe the market should be a lottery. The market is designed to pay people with accurate beliefs, by taking from people with inaccurate beliefs. Everyone's belief is valued based on its accuracy, and the market is fair in that sense.
Fairness is actually irrelevant to the societal good the market provides, which is to produce accurate prices. A third party, who doesn't participate, shouldn't care about the market being "fair", they should care about it giving good information.
> A similar issue is that of market manipulation, since many markets in these platforms can be directly manipulated by participants in manners as easily as spamming some words on an earnings call.
If you are betting on what a person will say, and the person knows about the market, that is a chaotic system. If you bid the price away from max entropy then you deserve the outcome.
This has nothing to do with fairness of outcomes but with defrauding naive investors of their money.
For context, I'm a former Investment Banker so this isn't coming from a place of naivete but an informed view. I did have to study SEC regulations for those FINRA examinations....
Democracies require conserved stake to function. How individuals initially get that stake is a degree of freedom, but if the stake can be printed freely, the system is unstable.
For working implementations of democracy, look at: invite trees, shareholder corporations, proof-of-stake distributed ledgers, etc.
reply