What exactly makes it "obvious" that this is written by AI? I could totally believe that AI was used to generate parts of it, but I really don't get the sense that the whole thing was written that way. I've seen way worse examples on this site.
As software engineers we are constantly told that we need to heavily use these tools for our daily work. So is it surprising that software engineers use the same tools as writing aids? Using AI does not mean no human effort was involved.
Subheading and dot point spam, low density writing (the opposite of standard technical english), including useless detail (like enumerating stats on Shopify's scale), using contrastive parallelism, and other llm-isms. Even if it's not AI it's bad writing done by someone who has picked up AI's worst ticks.
For example this subheading:
> "The real bottleneck: connections, not CPU"
That's two AI smells. AI likes to say vacuous punchy statements like "the final takeaway" or "here's the rub". Then contrastive parallelism "connections, not CPU". Contrastive parallelism should scarcely exist in technical writing, regardless of whether it's AI generated.
I had never heard the phrase "contrastive parallelism" before and I love it. I mean, I can't stand how often LLMs do it, but having a word for my annoyance is nice.
I really wish that slop writing was disincentivized in whatever RLHF they do. I don’t want to read the weird LinkedIn pop-sci tone for the rest of my life in such amounts.
I wonder if the average reader is also getting annoyed like this or whether they just don’t care - especially seeing what seems to get upvoted on your run of the mill social media sites. They probably collectively shape things more than I do.
I don’t know how you stop the slop writing. What the LLM writes has the voice of whatever has been RLHF’d into the weights. There will be a voice. It will have ticks.
They can change it but it will be there.
I do like that I have a term for that “this, not that” phrasing that’s nails on the chalkboard for me after several years of reading slop.
The LinkedIn “techbro” writings are the worst. They talk like a cryptobro “APPLE JUST PUBLISHED A CODE REPOSITORY ON THEIR HAND CRAFTED LLM ANYONE CAN USE THIS TO COMPETE WITH OPENAI AND ANTHROPIC” and its some random simple LLM on github that handles very little compared to OpenAI and Anthropic.
That's two AI smells. AI likes to say vacuous punchy statements like "the final takeaway" or "here's the rub". Then contrastive parallelism "connections, not CPU". Contrastive parallelism should scarcely exist in technical writing, regardless of whether it's AI generated.
Aren't the LLMs trained on a massive corpus of human written texts? If that stands, then they are doing what they were asked, kind of? I am also not a fan of the fluff words that Claude pollutes the context with, it's a bit much, I could live 30% less but it doesn't want to read its instructions in the .md
If the author wanted, they could ask their "AI" to write the article as a caveman, no? I, simply for fun, slop coded up a skill so that I can ask Claude to reply as Samual L. Jackson, or Kramer, etc.
I also wonder if there is an "AI language barrier", likely not in this particular article, but let's say there was a published article by a researcher whose native language isn't English. What happens with the translation? I realize I am asking naive things :)
There are distinctive patterns of language use that can be powerful when used sparingly. LLMs trained on a massive copus of human communication, picked effective patterns, and overuses those patterns to the point that it feels both artificial and underwhelming.
I wonder if the human training process amplified this effect. I like to imagine there's an OpenAI RLHF trainer in Kenya whose name we'll never know who really likes this style of writing and is solely responsible for LLM voice. I know that's not how it works, but it would make for a fun story.
And you know what? That's not annoying--that's brave.
>Aren't the LLMs trained on a massive corpus of human written texts? If that stands, then they are doing what they were asked, kind of?
I think you might be interested in reading in training data generation, training, and post training papers/articles. I think you might be surprised at how much intervention there is on some of these levels.
Just? With all the Claudisms it is hard to distill what really happened. My guess:
- Our oversell protection was a gross hack that broke ACID.
- MySQL has a new feature that allows us to remove parts of the gross hack.
- Question: To what extent does the hack still exist?
Instead you get garbage like "The answer is often in the plumbing, not the engine." and "Crucially, this wasn't about making reservations fast. It was about making them safe neighbors."
But this crap is what Lütke wants, so they deliver.
Taken verbatim "But the hardest lesson wasn't about database design. It was discovering that the real bottleneck wasn’t what we were observing and measuring. "
Because the post is just rambling without a clear intent or direction. Why do you need "oversell protection" if you have "transactions". Isn't the whole point of a "transaction" that it handles concurrency and disk failures?
I work with Shopify often and the whole oversell protection thing is like a huge joke.
Oversell happens because Shopify doesn't decrement inventory until payment is confirmed. And they apparently would rather die than change that invariant.
It's funny the organizational rules that get written into a company's DNA.
I could see a historical moment where Shopify, sans that invariant, massively fucked up inventory counts from uncompleted transactions... then had to unwind all that when a bunch failed to clear payment.
Would a retail business want the inventory to decrement just because users put items into carts that get abandoned? Seems like that would really mess things up more in the long run.
I know this would add complexity, but I wonder if they could train a model to predict the odds of a given cart being abandoned. I'm a data scientist. I bet there are at least a few situations where you could be confident that someone was going to complete a sale.
In every AI-written post, we have the script: some top post complaining it's AI-written, someone cluelessly saying they don't see it, and another person pointing out the claudisms. Sometimes someone posts a link to pangram.
I think we need the moderation team to set up some policy. In which direction idk.
Well, one thing to remember: AI learned from us (See what I did, there?).
The training these LLMs got, was from endless human-slop, on sites like LinkedIn, and marketing copy, everywhere. In fact, don't be surprised, if we start learning from AI; reversing the process.
But I think that it's only a matter of time, before almost everything will be at least touched by AI.
I posted this, yesterday[0]. It wasn't a particularly popular comment, but I stand by it.
I read your linked comment and I'm bummed out because I agree.
The thing is that I LIKE writing code. It's fun.
I don't LIKE coding with an LLM. It isn't fun.
We can't go back to a place where the professional code is all hand written. But at the same time I'm sad and tired. The joy has been utterly sapped and I don't think I'll be able to do this as a career for much longer.
The other day, I was working with Polars and decided to go old school with everything. I asked my questions on udm14.com and wrote the code in Notepad++. It was invigorating. Reminded me why I started coding in the first place.
Well, I am sorry to hear that, but it hasn’t been my experience.
That may have something to do with the way that I use an LLM.
I use it as a “pair partner,” not as an agent.
Most of the code in my apps is mine, but I do ask the LLM to take care of some of the “overhead” tasks, such as minor utility functions.
In fact, I just got done, ripping out a bunch of code the LLM wrote, in my current app, and replacing it with my own work. It seems that I can’t really ask the LLM to do much beyond function level, without it going sideways.
I simply can’t understand people shipping entire, vibe-coded apps. If I let the LLM write my whole app, it would be hot garbage.
Yeah, people act like this style is new, when it absolutely is not. There's this weird romanticism of the time pre-LLMs where people imply all writing/code was perfect, and only now it's slop.
It's likely because no one could be bothered to respond. There's plenty there, but it's not their job to fetch it for you.
People on the Internet (especially here), seem to think that it's OK to make rude demands of complete strangers, and expect them to spend considerable time, meeting them.
I'd rather some rando think they "won the Internet," than spend a bunch of my time, working for free, just to satisfy them. I have better things to do with my time, and I don't really have a lot of investment in what complete strangers think of me.
One of the things that I learned, about the time I lost my baby teeth, is that making rude demands of people, instead of working with them, is likely to result in exactly the opposite.
Chances are good, that you learned this, as well, which might explain the rude demands. They are unlikely to be met, so the matter stays open.
I apologize for my inferior communication skills. It appears as if I am unable to make the point clear. It has nothing at all to do with the matter at hand, and everything to do with communication style.
Your clearly doing it wrong, on the internet your supposed to make a positive claim such as; No blog exists from the pre-llm days that include the llm-isms. And then you wait for the internet trolls to come out of the woodwork and prove you wrong.
You dont ask people on the internet to provide you with some kinda proof because nobody can be bothered, you dont even ask them to prove you wrong because they cant be bothered. You have to confidently and boldly assert your false claim and that will enrage some poor autists somewhere into doing what you want.
> But enjoy your intellectual wheelchair, which for some reason you feel the need to advertise after 40 years of programming. I'm sure it is just in the best interest of future generations ...
There are some claudeisms here (it's not x, it's y) but it does read like they did at least one human pass on it. Or maybe this is opus 5 writing which is a bit less distinct idk. It doesn't feel as obviously generated slop as some of the stuff posted here with every other sentence a fragment and load bearing and all that nonsense
Using AI is a pretty good indicator that the reader will need to exert more effort than the writer did. Unless the reader also decides to use AI to summarize it, but then what's the point?
Anyone who thinks people work for anything other than money is delusional. Anyone who works for "the mission" is an idiot - no company or billionaire running a company cares about you. They will let you go in a heart beat.
I’m aware of this (consciously) but I would really do my job if it was unpaid, in fact: many do (and did) in the form of side projects and volunteering and FOSS.
I was always weirded out by the “only in IT for the money” crowd, which is a lot more than the people in it for passion these days it seems, and I might be an extreme case, but consider that while I run an IRC community- I do not write open source software.
I do a ton of side projects that personally interest me. I’m mainlining hard technical problems with zero need for permissions or waiting on bureaucracy or jira tickets. If a decision needs to be made, I don’t have to wait until next week when all of the “stakeholders” are available to talk about it. I figure it out myself.
If my day job was a side project I wouldn’t work on it at all.
I disagree, while money is a really important factor, that's not the final. Personally, I've changed to lower paid job to work on something i am more interested in.
I agree. We all make these kinds of trade-offs sometimes. We trade some amount of money for peace of mind, more interesting work, better colleagues, a healthier environment, and so on.
But I never buy the "mission" story, or the "we’re a family" line. We’re not a family, and I’m not on a mission.
I’m fine with the word "vision." I’ll be part of your vision if I find it interesting or if I believe in it too. But I’m not on a mission to achieve it.
To me, "mission" implies sacrifice and renunciation - whether that means sacrificing your lifestyle, your time, or your family. I’m not doing that.
I'm fine if leaders talk about mission and vision as long as it's to inspire and not in lieu of money. The right amount of money is table stakes. The problems start when leaders begin replacing money with mission, and people buy it.
I know a lot of people didn't like the harshness of the scene in Mad Men where Don tells Peggy, that's what the money is for, but people would be good to remember it. I tell people who work for me if someone offers you a lot more money than I can, take it!
He is right, when a startup becomes successful the hiring pool becomes different.
For example, you won't see people who came for politics-sake before real money is on the table.
This can kill the culture and make the actual people that were shipping unhappy
People can work for passion, some times even contrary to their own self interests, especially when you're starting your career. You hopefully grow out of it when you become established because companies don't pay large salaries for passion.
I've not seen smart professionals put passion before money unless it's their own company, and that's becaue they expect to earn quite a lot in the future, so it's usually money first, and that's OK because you need it to live comfortably.
I turned down a better paid, lower stress role recently where I would have been working with good people I liked and had worked with before.
Main reason was a sense of mission. If I screwed up in that role, a billionaire would have been a tiny bit poorer but literally nobody else would care. My current role involves critical global infrastructure, everyone would notice if we screwed up.
Having said that, money is still very important. I am pretty well remunerated - more would be nice but I'm not hurting.
> Anyone who thinks people work for anything other than money is delusional.
I work so that I am given food, shelter, entertainment, etc. at some point in the future in exchange for what I have given. Money is just the IOU that records the promise that I will receive those things later. I'm sure there is someone out there who is driven by the desire to collect IOUs and nothing more, but everyone (else but me)?
There will be be plenty of people, they just wouldn't be in Silicon Valley. Or possibly anywhere in the USA.
$800k/year in the UK is "Buy nice house in commuter belt in one year without bothering with a mortgage, three years if you want somewhere or something fancy but still not aristocratic, ten years if you do want aristocratic".
This sounds a little dystopian. How confident are they that this system is reliable? Any downtime could literally result in deaths. I would be highly skeptical of such a system being put in place in my city, not to mention the private company being paid to provide these services
I can’t even get through the banks automated live agent in a non-emergency. This is just going to be a disaster. I still have memories of McDonalds trying this and the insanity the order became was hilarious.
The central premise of this post is basically not true: working conditions and wages have pretty consistently improved over the past 200 years. The cases where it didn't were generally offset by the rest of society benefiting from increased automation and productivity. Where the current revolution will lead remains unclear, but if the past is any indication things will probably continue to improve for most people.
I can accept the premise that condiitions have improved, but not that it was consistent. It seems to me that it was anything but; more like extremely spiky. great depression, postwar economic miracle. Massive labor movements rose and fell, general strikes... Plus most countries have had several wars + revolutions in the last 200 years, including two world wars. Not all of those are to do with technology, but I also dont think technology wasnt indirectly responsible for the conditions that created some of them
The problem with these discussions is that it’s hard to control for the horrendously lower living standards of pre-industrial life where things like antibiotics and flushing toilets weren’t around.
Certainly, many people still lack access to those things. But also, the median person essentially lives better than the monarchs did 400 years ago when it comes to some of the more important things in life.
Sometimes, whether or not your power distance from your employer is high is less relevant than all the other important stuff in your life.
not only inconsistent over time, but also inconsistently distributed over people. rich people got richer (at least lately) and the less rich didn't get much richer (if at all)
Such a good comment, I would also add how much our perception of technology lifting conditions is conflated with the increased ability of governments to steer the economy to smooth things over.
All of those problems had nothing to do with technology and everything to do with economic mismanagement. And the commies were the worst at it. The US never had famines even at the worst of the great depression.
Economic mismanagement assumes there is economic management, and this is a very recent development. I would wager that we still do not know how to manage economies well in the long run - case in point US deficit and wealth inequality.
The US Department of Labor measures wages, and adjusted for inflation the hourly wage is below what it was in 1973. It is even lower if you are only counting non-management roles then and now. Wages have not gotten better over the past half century they have gotten worse.
Workers now work more hours pee year to try to catch up to what they made back then, so working conditions are worse now as well.
This corresponds with the increasing share of the economy consumed by the government.
> the hourly wage
This is a misleading statistic. The actual statistic is "total employee compensation", which includes the benefits packages, the (so-called) employer contribution to SS, 401k employer contributions, etc.
> The US Department of Labor measures wages, and adjusted for inflation the hourly wage is below what it was in 1973. It is even lower if you are only counting non-management roles then and now. Wages have not gotten better over the past half century they have gotten worse.
This is a flat out lie. Comp/GDP is a side note that does not negate the fact that inflation adjusted wages are up.
The part of compensation includes healthcare, because we have no choice, it's a lock in - there's no marketplace to choose your healthcare, unless you go ACA which is a bad idea in many states - and as a lock-in the out of pocket costs have skyrocketed. To borrow your wording, google 'out of pocket healthcare costs annual trends.'
I think you're conflating productivity growth with distribution. Technology increased wealth, but whether workers shared in those gains depended heavily on bargaining power and institutions.
The Robber Barons were able to r>g their empires just fine in gold-backed currency.
The Cantillon Pump is only one of several major "rich get richer" mechanisms in the economy. It is not the most important, and it does not consistently run in the same direction (wages can inflate faster than assets).
It can be caused by deficit spending. It can also be caused by supply shocks. For example, if you restrict building new housing, the price of housing will increase, both rents and house prices. If you tariff imports, a basket including those goods will rise in price to the degree those imports are not substituable. If you start a war in the Middle East, oil prices will rise, and increased energy costs can feed into lots of different things, raising prices.
Sigh. It is always the result of deficit spending.
Bob earns $20 every day. He buys 5 eggs $2/ea and 5 apples $2/ea every day. Now, due to supply shocks, egg prices double. He still has only $20, and so he now buys fewer eggs and fewer apples.
What happens when he buys fewer apples? The price of apples goes down, due to the Law of Supply and Demand. There is no inflation.
What happens when this economy is flooded with dollars? The Law of Supply and Demand again, meaning the value of each dollar drops. That means the dollar price of eggs and apples rise, as well as his wages.
Oil prices rising means people have less money to spend meaning prices of other things drop.
Another way to look at it is the US had zero net inflation from 1800-1914. From 1914 to today a dollar is worth 3 cents of a 1914 dollar. That isn't due to supply shocks, and there certainly were plenty of supply shocks before 1914.
1914 is when the Fed was created and empowered to print money with no backing.
People had to put up with deflationary shocks and yet robber barons built their empires all the same. Hard money doesn't stop r>g, it doesn't stop expected returns from growing increasingly burdensome, and it doesn't stop assets from appreciating relative to wages. It doesn't actually deliver the things we care about.
It delivers plenty of things we don't care for, though. History provides the most lurid examples, but we have modern ones too. In the US, Clinton balanced the budget and the macroeconomic consequences broke something very important (audience participation: what was it? Hint: we call them "dual deficits" for a reason). We quickly took our finger off the stove, though, so it was only a lesson for the observant. Germany on the other hand kept its debt brake in place, which mechanically suppressed investment (macro 101 quiz time again: why?) with staggering consequences, leading to one of the most underinvested economies in Europe and almost completely shutting them out of the digital revolution, in which I privately suspect they'd have otherwise participated fabulously. In any case, had Clinton installed a debt brake, that would have been us. HackerNews, YCombinator, and all the ZIRP babies around these parts would have been among the most affected. The Magnificent 7 would not have all been in the US.
So no, deficits aren't the root of all evil and the appropriate deficit is considerably north of 0. That said, it's south of where we have it. Interest rates are the gauge. Is money being pushed in (low rates) or pulled in (high rates)? We are transitioning from the former regime to the latter regime, partly due to imperial retreat, partly due to crisis-level spending in a time of no crisis which is wildly irresponsible. The US is clearly headed for a debt crisis. But the answer is belt-tightening and either gentle-repression-over-time like we did after WWII (fat chance) or an inflation spike followed by a rate hike (probably several) until the bond market is happy again. It's going to be bumpy, but not as bumpy as the hard money counterfactual.
Louis Brandeis would like a word. In particular: no, the fact that tech advances and economies of scale dropped oil prices at the same time Rockefeller was building his monopoly does not make that monopoly harmless or good or justify it into perpetuity.
More importantly, after Ronald Reagan and Robert Bork brought us back to robber-baron era antitrust policy, we've seen robber-baron level corporate consolidation and corporate profits. We've exceeded them in most regards, in fact. It has been terrible for consumers and excellent for shareholders.
Printing money without backing is not deficit spending!
We're all well aware of the money supply, M1 / M2 / M3, velocity of money, etc. It's important for money to not get tight when the economy slows down, and it's important to tighten up the amount of money when the velocity increases again. But this is neither deficit spending nor printing money without backing!
Inflation is can come from a social psychological phenomenon around expectations. If the cost of living goes up in a surprising way, you feel the pinch. You want more money. Maybe your employer gives you a raise, and raises prices to compensate, reasoning that other things are also going up in price. The risk is this creates a self-propagating loop (which in turn does to a degree depend on accommodating institutions that don't restrict monetary supply).
You can trigger inflation through loose monetary policy, sure, and you can dampen the cycle by restricting the supply of money, and that's been the main function of independent central banks for most of the latter 20th and 21st century.
But inflation is literally just increase in the cost of living. It's a symptom, not a disease. It admits multiple explanations.
You're missing the trees for the forest. The Industrial Revolution was terrible for the people who were working during it. Their jobs got eliminated and they only sorta sometimes were able to retrain for the new jobs, and the working conditions in those new jobs were atrocious, exactly because they were new enough that we didn't yet have safety regulations for them.
It was only their children who eventually saw any improvement in working/living conditions, and it was only because of all the absolutely awful things that happened to their parents. Safety regulations are written in blood. It only looks like consistent improvement if you zoom out to a level that obscures intra-generation experiences. At the level of the actual human beings subjected to the turbulence, it was awful.
I’ve seen a similar thing with globalization. Macro scale, it’s great. Much number go up.
Zoom in, though, and it also impoverished my hometown and many others like it. A whole generation saw their job prospects gutted, and had no realistic options for a career shift because it’s really hard to get an entry level position when you’re middle aged; doubly so when there’s a glut of middle aged people looking for jobs.
And waving away the pain those people experience with an appeal to large scale, intergenerational, historical trends is simply heartless.
Awful compared to what though? To our modern perspectives yes, but for subsistence farmers working in the fields 16 hours a day and rarely far away from starvation, life in a factory might not seem so bad. After all, wages were far worse before the factories and wage growth was almost non-existent.
Although that's true, is 200 years ago really the baseline we want to use for judging working conditions?
Two hundred years ago meant widespread child labor, harsh factory conditions, little worker protection, and slavery or serfdom. Add to it the onset of Industrial Revolution.
Modern studies indicate people in the dark ages worked 6 hours a day: Ironically a better baseline!
I don't believe that "6 hours a day" is an answer any historian would give when asked how much workers in any period in history worked.
There have been indications that there were approximately 150 days per year which were most active for medieval farmers, but if I had a time machine I wouldn't stay there very long if my desire was to experience good working conditions.
> working conditions and wages have pretty consistently improved over the past 200 years
Really depends on the group of people you are including. In the US the middle class has been eviscerated to the point where raising a family on a single income is not a realistic goal for most people.
>to the point where raising a family on a single income is not a realistic goal for most people.
Was this ever true? During the 1950s, (the perceived heyday of American middle class wealth) the female labor force participation rate was 35%[1], which was close half of the male labor force participation rate.
1) Based on your numbers, 65% of women were not working at that time.
2) That 35% number was likely even lower for married women.
3) Racism was still a systemic problem and was bad. Black men were excluded from the GI bill for example.
4) In spite of that, the median American household needed only 2x their salary to buy the median house. That number is 5x today, and after taxes the ratios are 2.2x then vs 6x today, when two adults are often working rather than one.
5) In short, Americans are working much harder today.
>4) In spite of that, the median American household needed only 2x their salary to buy the median house. That number is 5x today, and after taxes the ratios are 2.2x then vs 6x today, when two adults are often working rather than one.
Sources for these numbers? Also, housing is only a fraction of your costs.
A housing development nearby my home was built in the early 1960s (I think). If you look closely at the homes, they are all of two designs, made into four by mirror images.
But it's pretty clear that over time the homes were remodeled, gut remodeled, extended, garages added, and so on, so they are significantly improved over the original. I've been in a couple, and the hallways are pretty narrow and everything is very cheaply built.
If you want to see what they were like originally, see the old Bob Hope movie "Bachelor in Paradise", filmed at a just-built suburban community.
"Narrow and cheaply built" isn't enough to make houses affordable in an age where crack dens with busted utilities go for over a million dollars. The ability of material to become cheap is capped far below the ability of location to become expensive.
google sez: "The median home sale price in Seattle is approximately $853,000 to $879,000, depending on the specific neighborhood mix and recent monthly data. Listing prices sit slightly lower around $750,000 to $775,000, placing Seattle among the highest-priced major metropolitan housing markets in the United States."
I assume a crack den with busted utilities would go for less.
I've lived in this area my adult life. Pretty much all the open spaces and lots have been infilled with houses over time. Prices go down the further away from the business core. Being within commuting distance to Microsoft comes with a big boost in price.
True, YIMBY can only do so much. The market serves wealth-weighted people not people-weighted people, so when inequality cooks to the point that the whims of the wealthy outweigh the needs of the poor, the latter gets the boot when the two are in conflict.
The reason the 'middle class' is smaller is because more families have become "upper middle class". All the data shows this.
You can argue that housing is too much or that inflation hasn't benefited the right things, but to say that the middle class is shrining just doesn't match reality.
The only way one can claim that the middle class isn’t shrinking is to have a floating definition of the middle class.
If you lock it in any way to something like “home ownership and simple vacations once a year” (which was absolutely possible in the 90s), it has plummeted.
The data that deflate with CPI? The metric notorious for laundering forced substitution into voluntary substitution? The American Dream is to Owner's Imputed Rent a house?
I love BLS, they do a lot of thankless work very well, but if you give someone else control of the inflation basket you are letting them pick any slope for these graphs that they want. Don't do that. It's the economic equivalent of handing your passport to an employer, talking about your net worth in public, or signing a blank check. Just don't. Yeah, we're all busy, you probably don't have time to construct a basket more sophisticated than "gold" or "housing" -- but even though those both have big problems, they still aren't as bad long-term as CPI. Or maybe you disagree and think the CPI basket and hedonic adjustment methodology compounds better for your purposes, but if so you should be able to roughly account for the dramatic difference and articulate why.
> working conditions and wages have pretty consistently improved over the past 200 years
They've improved, but not consistently on human timescales.
Being a worker in the pre-labor-movement industrial revolution was by all accounts horrendous. Things have, so far, trended better over time but they can be bad for whole generations and also past results do not guarantee future results.
> The cases where it didn't were generally offset by the rest of society benefiting from increased automation and productivity.
I don't think we can sweep the worst parts of it under the rug because the bad impacts are only happening to "the poors". And anyway when it comes to the current tech revolution, it is likely that almost all of us are going to be "the poors".
> Being a worker in the pre-labor-movement industrial revolution was by all accounts horrendous.
Average height and life expectancy improved steadily in the US throughout the 19th century.
The pre-Revolutionary Americans tended to live short and backbreaking lives.
Take a tour through Fort Williams. There is a collection of clothing from the Revolutionary War. The baby clothes look like doll clothes, and the adult clothes look like they were for boys. Then look at the clothing in the Gettysburg museums. The people were taller then, but still about the size of boys.
> Take a tour through Fort Williams. There is a collection of clothing from the Revolutionary War. The baby clothes look like doll clothes, and the adult clothes look like they were for boys. Then look at the clothing in the Gettysburg museums. The people were taller then, but still about the size of boys.
I don't think that's a correct interpretation, see the explanation here [0]. Not quite the same situation, but likely a similar case. Are you sure the "adult" clothes weren't made for young boys or teens, or women?
I don't think your statement about short lives in colonial America is correct either, they had a low life expectancy like everyone else, but mostly due to a lot of childbirth deaths and young infant deaths. Surviving childhood gave you a good shot at making it to your 70s, which has been true for a long time pretty much everywhere.
> Are you sure the "adult" clothes weren't made for young boys or teens, or women?
Yes. I read the informative cards at the display. Some were for officers. George Washington was a giant in his day. Look at the short beds and low ceilings. Watch your head walking through a doorway!
Tour a navy ship of the time.
If you like, look up historical US statistics on height.
BTW, I saw Henry VIII's armor in London. Looked like a boy's armor.
As for the short colonial lives, I read an account by bone archaeologists who examined the bones of colonials and noted that they suffered much damage from overloading the bones and their age at death was usually under 40.
Just think about the hard work in making your cabin with a saw and an axe. Then a barn. Then fences. Then about the endless amount of firewood you have to chop every day. Then using a horse-drawn plow. And dig a well by hand. And so on.
> I don't think we can sweep the worst parts of it under the rug because the bad impacts are only happening to "the poors".
We can and we will.
If you read HN for any period of time, you'll probably read at least 5 apologia for the harms caused by prior technological change for every one critique or expression of sympathy. After all, we're well-indoctrinated software engineers, and most of the rest are temporarily embarrassed billionaires. We know what we're supposed to think.
> And anyway when it comes to the current tech revolution, it is likely that almost all of us are going to be "the poors".
Not me! I'm 100x AI engineer who will certainly remain comfy an secure. LLMs are awesome! It's all those luddite doomer jerks who cling to their old ways of understanding how things work and manual coding that are going to be poor, and they deserve it.
> Not me! I'm 100x AI engineer who will certainly remain comfy an secure. LLMs are awesome! It's all those luddite doomer jerks who cling to their old ways of understanding how things work and manual coding that are going to be poor, and they deserve it.
We will see in 5 years ! I think you highly overestimate llms
Funnily enough, the parent made the point that people probably overestimate their position in the future pecking order, and did so sarcastically, which you seemed to have completely missed (probably due to being ESL), something even a current LLM would easily "understand", thus proving his point.
Yeah no. It's only since 1950 that conditions improved for everyone. Before then everything from sewage, to coal smoke, to chronically low wages made life worse for everyone involved. The most miserable people to live in history were the British working class in the mid/late 19th century.
Early in the Industrial Revolution people voluntarily moved from the countryside to the cities despite all that, so apparently a lot of them thought it was an improvement. Some of the cities were proud of the soot, since it showed that they were participating in Progress.
You can see some of this repeating in China. Migrant workers move to the cities because, while what they can earn is low, it is a lot higher than subsistence farming in the countryside. "The Last Train Home" (documentary) follows such a family, although to understand the tragedy in the documentary you need to understand that Mao created the hukou system to prevent exactly this, which means that their children can only receive state education in the countryside.
> Early in the Industrial Revolution people voluntarily moved from the countryside to the cities despite all that, so apparently a lot of them thought it was an improvement
The industrial revolution took many of the countryside jobs to the city. You cannot stay in a place that no longer has jobs
Before 200 years ago we lived as hunter gatherers, and we did it sustainably for millions of years, longer than we've been human.
200 years is nothing. It's nothing, unless that is, we try to think about what the mess we're currently making will look like 200 years from now. Then 200 years from now is a terrifying prospect. It has to be sustainable. If it isn't sustainable it's a joke.
With subsidization from the Chinese government they will probably be equal to or better than the models here. I mean, have you looked at the author list of any given AI paper published within, say, the past 5 years? I wouldn't be surprised if half or more AI researches are from China.
You mean from the private investors? It seems the labs on both sides of the ocean are quite negative in their profitability right now due to the competitiveness. Though Anthropic claims they will have a profitable quarter this year (despite the huge build-out), so their margins on API costs are likely quite decent.
Maybe when I found out you can use it to run terminal commands, spin up and take down dev environments, and even run other LLMs. Suddenly 90% of the difficulty of onboarding to new repos disappeared overnight and a lot of heavily CLI-based workflows became trivial to automate. Never again do I want to spend hours manually sorting out Python dependencies.
In the book the rules are a bit fuzzy. It's told in first person so it isn't really explained so much as related to the reader from a character who is also confused about how the time loop works. Although who knows, maybe there's some grand explanation in the books that haven't been translated yet.
It's a good read, I can't really predict where the author will go with this after Book IV.
Albertan/Western Canadian identity is totally a thing, and has been around for a lot longer than this latest round of separatist sentiment. The west has been griping about unfair treatment from the federal government for over a century now, so 1) this isn't primarily driven by foreign interference and 2) it's not coming out of nowhere.
Whether it's a good idea is a different question. I doubt most Albertans want to be independent. I also think being a landlocked country with a resource economy means that you will always be subject to outside control, whether that be parliament in Ottawa or corporate offices in Dallas. It remains unclear if being independent will solve the issue of Alberta being land-locked.
Former Albertan here. Alberta even griped about unfair treatment when their conservative party had a majority in Ottawa for almost a decade. It’s just what people have learned to say.
A national identity needs to stand for something. It can't just be defined as 'we hate Ontario and the federal libs and want to become an American gas station'.
> The Wild Rose province isn’t like Scotland, Quebec, or Catalonia. Everyone can more or less agree these are “stateless nations.” Maybe you think those nations should have their own state, maybe you don’t, but they follow a well-worn political pattern. Albertan separatists, on the other hand, are trying to create a state with no nation. That does not follow a well-worn political pattern. Nobody has ever done this.
> The idea that Quebec is a nation is not divisive in Quebec, not an idea that separatists think is super and federalists think is dumb. It is basically a matter of consensus among political actors there. What they disagree on is whether the Quebec nation is better off inside or outside of the Canadian confederation.
> Who sees that in Alberta? When has any elected deputy of Alberta’s legislature, let alone literally every single one of them, loudly and publicly affirmed that on behalf of their constituents they perceive Alberta as a nation? Are there any historical instances whatsoever of outside observers seeing Albertans as a nation that would compare to such seminal documents as the Durham Report?
> Even separatist leaders use the word “nation” sparingly. The Alberta Prosperity Project’s manifesto, The Value of Freedom: A Draft Fully Costed Fiscal Plan for an Independent Alberta, lives up to its title by speaking in exclusively financial terms; even then it can only refer to Alberta as a nation using somewhat sideways language. When it states that “a sovereign Alberta could become one of the lowest taxed and regulated nations in the world, rivalling jurisdictions similar to Dubai and Monaco,” it could just as easily substitute “state” for “nation.”
> It’s clear that Alberta (the place where I have lived longest in my life) is not like Ontario, any more than the Maritimes are (I also lived eight years in Dartmouth). These places are all homes to distinct cultures. But in none of those three places do we find sustained instances of diverse groups of both insiders and outsiders clearly referring to them as repositories of a national identity other than “Canadian.”
I've been saying it for a while not but "independence" is a distraction and not the end goal here. The inevitable outcome would be annexation by the US.
A huge amount of academic research into ''western alienation'' has been, and continues to be, researched at Canadian universities. The concept is bedrock to studies of Canadian history and political science.
I understand this. I lived there, I heard the "Onterrible" jokes and wore them with grace.
The concept of an independent Alberta as an identity is a fringe matter, not equivalent with generalized notions of alienation and grievances related to equivalence within confederation on a policy level.
If you don't think it's a thing then you're either not from here, or haven't been paying attention. The average Canadian's opinion of Alberta is also very telling, with most of the rest of the country seeming to despise the province, or think it's some sort of regressive backwater.
Since you decided to take things in a personal direction, yes. I have lived and worked in Alberta. I have had family in Alberta. I have friends in Alberta. My partner is from the west, and we visit regularly.
That's some bad karma, pretending you can read someone like that and attempting to beat them down with your ignorance and then claim to be a victim.
Now now, when was I claiming to be a victim? Let's not be dramatic now.
Also, which Alberta? Edmonton? Cardston? Fort McMurray? Lloydminster? Grande Prairie? Canmore? I guarantee you will find varying levels of Albertan identity and many different perspectives in each. Albertan identity is not a mirage.
Claiming the rest of the country is against you is playing the victim. And which Alberta? That sounds like "What kind of Albertan were you". Which is starting to sound like real foul stuff.
You're being incredibly silly with your arguments. If you talk to anyone in the country outside of Alberta you're very likely to hear a negative tone when talking about them, especially if they are liberal. Our governments have very much fostered a hostile relationship with Alberta and has done very little to address their concerns. Anybody surprised by Alberta wanting out has had their head in the sand.
I don't think them leaving the country is the right solution, but this is what happens when people feel ignored for a long time, they go with the nuclear option of leaving. It's very clear that a lot of people in Alberta feel mistreated, and the governments should be working to hear their concerns and make changes. But sadly they seem to do the opposite and ignore them and continue to make negative remarks about them which furthers the problem.
In fact their behaviour is similar to the dismissive behaviour you have been showing in these replies to the other user.
No, it's not proving anything of the sort. You're trying to claim that the average canadian despises alberta, and that's simply not a thing. It is in fact invented whole-cloth.
According to this poll (https://www.cbc.ca/news/canada/calgary/poll-canadians-living...), nearly half of Canadians think that:
1) Alberta is not a welcoming place
2) Albertans don't care about other Canadians
3) Alberta is not a place they would feel comfortable living
And noticeably, the opinions of the Albertans are generally different from the rest of the country! How curious for a place without an identity of its own, as you claim.
I've never experienced this myself despite living here for years, working in oil/resource/agriculture-adjacent sectors most of my life, and spending a few years in Ontario. It is, in my experience, purely CPC/UCP propaganda. I certainly wouldn't have spent january pushing edmontonians out of their parking lots if I despised them lol
I am aware of recent political movements, yes. Like the Western Independence Party.
However, they failed to even get enough signatures to properly form.
Their platform is to "basically remove Alberta from confederation" (the party founder's words). But note: there was no Alberta before confederation.
Alberta business owners having a beef with Ottawa leadership is not the same as a common and foundational identity across Alberta that desires independence. That latter notion is in the extreme minority. Fringe stuff. For instance, the support between the WIP (and aligned groups) is similar to the support for the province's Communist parties.
I was getting dangerously close to my weekly Claude Code limit last night so I had Claude set up Qwen3.6 with llama.cpp and OpenCode. Honestly it's a great (free!) alternative to Claude Code--certainly more than good enough for a lot of smaller less complex tasks. I'm excited to try this new version. The fact that open-source models are so close to the frontier is very impressive.
Out of interest, what machine and model are you running it on?
I tried the qwen3.6-27b Q6_k GUFF in llama.cpp and LM Studio on my M2 MacBook Pro 32GB machine last week, and I barely get a token a second with either.
What sort of speed should I be expecting?
I tried some of the Llama 3 34b (nous-capybara?) models two years ago with llama.cpp, and I seem to remember getting a few tokens a second then, so not sure if I've got something completely mis-configured, or I just have unreasonable expectations.
Or maybe qwen 3.x is slower for some reason? (Is it mixture of experts?)
I'm not expecting it to be instant, but what I'm currently seeing is not really usable.
- A 35B "Mixture of Experts" model, which activates only 3B parameters for each token.
For your hardware, I strongly recommend `unsloth/Qwen3.6-35B-A3B-GGUF:Q4_K_M`. I have an M1 Max with 32GB VRAM from 2021 that can read at ~300-500 tokens/sec and write at ~30 tokens/sec with llama-cpp's default settings, which is plenty fast. The 27B model can read ~70tok/sec and write ~5tok/sec.
The 35B MoE model technically takes slightly more memory but is much faster because it's doing 1/9th the work. It's not quite as "smart", but it's comparable.
I played around with local LLMs on my M4 Max 64GB this weekend and this is exactly what I found. I put Opus 4.7 "head to head" on the same task as Qwen 3.6 and a few other local models. The 35B did not perform well IME - it needed a lot of handholding and even then the final result did not work until a few more tweaks, while Claude one shot the task. The 27B was much better and also one shot the task, but took about ~55min as opposed to about ~15min for Claude. The 27B is probably something that I could happily run for many use cases if I had some faster hardware... the main problem there seems to be that at larger context sizes, prompt decoding can take several minutes.
This matches my experience too. The little a3b model is quite capable for its size class, as is the 27B model, but it’s still an order of magnitude less effective than Claude on the “effectiveness / time” curve
These are dynamic quants, and they're basically just an indication of how far away from the desired quant it is allowed to go to achieve the goal. Generally, unsloth's toolchain moves quants up, rarely down.
* _0 and _1 do not use K quant and scales 32x32 blocks according to the original (B)F16 values; _0 scales the block using the original max and min values. _1 does this per row instead of per block.
* K quants do something similar, but now splits blocks into subblocks inside a superblock where the superblock has min/max scaling, but the subblocks also have scaling in the range of the superblock's scaling and are stored using less bits.
* K's M, L, XL are just how aggressively the subblocks and their scaling factors are chosen. Generally, it puts a max on how far you can deviate from the chosen quant to maintain the desired quality, but also gives them a bigger budget to perform that excursion in. XL most aggressively tries to preserve the intended quality, while S does the least.
* Dynamic quant on top of this scales entire layers, full of blocks, according to how much they effect various measurements (such as KLD and perplexity).
That said, there is no reason K_S is even produced by anyone, same with Q_0, Q_1, and I_NL. People should no longer be using those. M only is meaningful if you're trying to restrict the upper bounds: K_XL can reach BF16 for some weights, but rarely; people think this has a speed implication for hardware that has native 8bit in their tensor units (but it doesn't).
Unless you're specifically trying to cure a problem, stick with K_XL.
You seem to understand this stuff pretty well, any recommendations on resources (blogs, YouTube channels, whatever) for software engineers that want to keep up with this stuff on this kind of level?
A lot of the content about AI out there is kind of produced to the lowest common denominator. Basically a never ending scheme of get rich quick/passive income kinds of AI content.
Unsloth’s guides on getting various models running are great starting-off points for the “practicioner’s side” of things. Note that they include settings for llama-cpp, ollama, and other runtimes in addition to their own “unsloth studio” (their product seems like overkill imo)
If you’re curious about what a particular switch does, clone the llama-cpp repository to your computer and try asking your favorite pet rock prompts like “This is llama-cpp. Can you look at what the -ctk parameter does and explain to me?” Giving Claude/codex/whatever access to the actual code goes a long way, but it is just one opinion.
If you’d like to learn how transformer-based language modeling works in detail, I suggest starting with chapter 0 or 1 of https://arena-chapter0-fundamentals.streamlit.app/ depending on your skill level, then use that to work your way to reading research papers.
Graduate students who study these topics are generally as annoyed by the “get rich quick” style of advertising as you are, so the deeper you go toward academic research the quieter those voices tend to get, mercifully. That said, this is balanced by the unfortunate fact that top labs have strong posturing signals they try to send, so it can be hard to see which preprints actually have good ideas, which are trying to promote their group’s tech instead of doing science out of curiosity, and which have authors who’ve innocently deluded themselves into overfitting their own pet projects. Read widely but adversarially, test everything but hold fast to the good stuff, etc etc
Hey some of us are on hardware (gfx906 based Radeon MI50s with 32GB of stupidly fast VRAM and basically no compute) that inference significantly faster with Q_0 and Q_1 quants
Its not amazing at compute (yet is a member of the GCN family, which I have been a fan of since its inception) and ended up being too expensive for perf/$ and perf/watt.
The only thing it did was make Nvidia rush Series 10 out the door and make it too good. Nvidia has been unable to live up to the gen-to-gen uplift Series 10 did, all because AMD made Nvidia blink.
Basically, you're 2 gens too early. CDNA2/gfx90a is the minimum you need to get any meaningful performance out of inference, or maybe CDNA1/gfx908 if you really don't need to quantize at all.
BTW, I did suggest this elsewhere in this HN story, but have you tried just disabling KV quant entirely? That is a huge speed uplift for compute-poor users.
Also, llama.cpp's support for gfx906 is probably never going to as good as it is for other cards, and good ROCm support for cards before they rebooted the driver/stack team is probably never going to materialize. I don't see the point in hanging onto them.
Like, if I was in your place, replacing it with even a 9060xt, with half the RAM, would be a step up. They go for $450. People have been building dedicated inference machines with these and they've been amazing, just throwing in 3 or 4 in, and scaling VRAM to meet needs.
But as models are starting to pack more information into less bits, some weights are just going to end up becoming super important and very sensitive to quant. So, I'd just move down a Q size, and continue with K_XL. Like, I'm betting Q3_K_XL will beat Q4_K_M on any given model in real world testing, even though its ~20% smaller, but perform worse on benchmaxxing.
The only exception I could think of is quantizing small models, like, my testing on Gemma E2B/E4B and Qwen 3.5 9B, quantizing at all was super noticeable... they can't spread the error across more weights.
Good news (at least for me), 24GB of VRAM is enough to store either of those in BF16 and then a ton of room for F16/F16 KV cache.
on my M1 Max, MTP consistently lowers my performance! I’ve tried both llama-cpp’s recently landed MTP support (cloned and built Tuesday) as well as one of the other forks a few weeks ago. Suspect nobody’s done a comparison on hardware like mine.
I recommend sticking with the dense models for both Qwen and Gemma.
On testing I've done on same-quant apples to apples, with F16/F16 (ie, unquantized) kv cache, 35B-A3B underperforms against 27B on anything even remotely complex. But yes, 35B-A3B can be like 3-4x faster on my hardware.
By Qwen's own admission, on any meaningful benchmark (ie, ones that involve logic, math, or tool calling), 27B performs like 122B-10B and 397B-A17B, but 35B-A3B is somewhere between 27B dense and 9B dense.
Also, MTP recently got merged in, so I'd suggest downloading Qwen 3.6 MTP (I assume you get it from unsloth) and updating your copy of llama.cpp, and adding `--spec-type draft-mtp --spec-draft-n-max 2` to your arguments.
Also, I recommend not quantizing kv cache, and if you do, only quantize v. Lowering model quant while also lowering context size to fit F16/F16 or F16/Q8_0 massively improves model performance for thinking models. Also, quantizing cache, either k or v, decreases speed by a lot on some hardware.
I have a 24gb 7900xtx, so I can fit >32k F16/F16 context with Qwen3.6-27B, but use unsloth's Q3_K_XL. This performs better than Q(4,5,6)_K_XL with v quantized.
Edit: Oh, and since I mentioned Gemma 4, my testing mirrors my Qwen 3.5/3.6 experiences, 26B-A4B performs worse than 31B, but is also way faster. llama.cpp doesn't support Gemma 4's MTP style yet, so both could get even faster.
I tried the qwen3.6-27b Q6_k GUFF in llama.cpp
and LM Studio on my M2 MacBook Pro 32GB machine
last week, and I barely get a token a second with either.
The fact that it was this slow makes me suspect it's a matter of insufficient free RAM. The entire model needs to fit into RAM (and stay there the entire time) for acceptable performance.
(not sure of exact diagnosis/fix, but definitely look in that direction if you're still having this issue when you give it another shot)
Also, there are two stages - prompt processing, and token generation. Prompt processing is notoriously slow on Apple Silicon unfortunately. If you have large context (which includes system prompts, lots of tools loaded by a harness like Claude Code, OpenCode, etc) it can take minutes for prompt processing before you see the first output token. On the bright side, the tokens are cached between turns, so subsequent turns won't be so bad.
You are using Q6 6 bit quantization; on my 32G MacMini I use Q4 and it is faster but when I use it with OpenCode, I set up a task and go outside to walk for ten minutes. Smart, capable, and slow. Still, I love using local models.
The 27B model is dense, so is relatively slow. The 35B-A3B model is marginally weaker but being MoE is much faster - like ~4-8x faster in basic benchmarks on my M1 Max.
For comparison, I just ran a couple of quick benchmarks (default settings) with llama-bench:
Qwen3.6-35B-A3B at Q6_K_XL gave 858 t/s pp512 (prompt processing) and 43 t/s tg128 (token generation).
Qwen3.6-27B at Q4_K_XL gave 103 t/s pp512 and 8 t/s tg128.
Have you tried enabling MTP? Those numbers are similar to what I was getting on my Strix Halo box, but configuring/enabling MTP doubled the TG speed of the 27B model (18-20 t/s now).
Thanks - I’m in the process. I’ve tried briefly, but so far it appears marginally slower. (Noting that llama-bench doesn’t support MTP yet so you’re reduced to running different prompts and eyeballing the log.)
So I’m assuming I’ve done something wrong along the way, but I’ve not had time yet to explore it.
27B is the dense one. Try the Qwen3.6-35B-A3B variants for the MoE release. That's what I'm running on a Framework Desktop and I get ~50 tok/s plus or minus a few. The dense one is similarly slow for me -- not sure what to expect on your hardware from the MoE but it should probably be much faster.
Which exact model are you using? And with which parameters and quant? And on what hardware? Are you using any specific MCPs or other tools to optimize performance like context-mode or dynamic context pruning? I’ve used local models a reasonable amount before but I’m just starting out with opencode. Haven’t had great results yet but really want this to work for simpler tasks. My opencode newly installed is also having iterm on 100% cpu in idle. :/
I'm running Qwen3.6:27b Q4 KM on a 4090 and similarly fast CPU and I think 32GB of RAM. Make sure the context window is set to be big enough otherwise the conversation will keep compacting. No special MCP tools set up yet. Qwen is able to do web search out-of-the-box although I think it is getting blocked by anti-bot firewalls--I still need to figure out if I can fix that.
here's a simple setup to get you started on an Apple M1 Max from 2021 with 32GB VRAM. it will download 20GB of models to `~/.cache/huggingface/hub`, which you can delete when you're done.
/Users/gcr/llama.cpp/build/bin/llama-server
-hf unsloth/Qwen3.6-35B-A3B-GGUF:Q4_K_M
--no-mmproj-offload
--fit on
-c 65536 # edit to taste
--reasoning on --chat-template-kwargs '{"preserve_thinking": true}'
--sleep-idle-seconds 90 # very aggressive: purge model from vram after this long
-ctk q8_0 -ctv q8_0 # Optional. Lower memory use, but lower speed. Omit if you can.
I don't recommend ollama or lm-studio. Ollama's in the process of switching from their llama-cpp backend anyway, but their new go framework frequently OOMs and crashes on my hardware. I also don't recommend MLX-based inference backends on this hardware; I've found them to consistently reduce performance, contrary to what I've read online. I've tried all the llama-cpp metal forks, but right now, MTP, TurboQuant, MLX, etc etc etc are too new and just slow things down. It's all dust in the wind still.
For agent harnesses, opencode is okay, as is pi or even Zed's built in agent panel. Claude code "works" with ANTHROPIC_BASE_URL=http://localhost:8080/v1, but is very chatty (the default system prompt burns 20k tokens). Crush (from the charm-bracelet folks) is particularly nice when starting out. I've personally converged on pi-agent under an otherwise-mostly-default setup. You can ask qwen to customize pi or write you an extension which helps a little.
You'll need to add `http://localhost:8080/v1` as an OpenAI-compatible model provider in your coding harness with any API key (doesn't matter) and any model identifier (doesn't matter with llama-cpp).
Note that pi doesn't have permissions. Everything is permitted. The hundred hungry ghosts you've trapped in a jar WILL find a way to delete your home folder someday. That's what Man gets for summoning demons without casting a circle of protection first. Flying too close to the sun etc etc etc
Take backups and then go have fun. Hope this helps.
I have a 5070TI (16gb VRAM) with 32GB system ram and a 16 core AMD cpu. I am considering buying a second used videocard, probably the same model, but not for months yet. This hardware setup is new-for-me in that a buddy gave me most of it and I bought the TI card.
Are there any resources to help me figure out how to best optimize my runtime paramaters for a given model, based on a given task, similar to what you've shown?
I've been a little... irritated? that hooking vscode up to my company LLM subscription seems so much more out-of-the-box capiable than what I can get to work. My assumption at the moment is that I need to create a lot of... I think they're called harnesses? agents? workflows? integrations? (not sure) by hand. Is that accurate?
Right now I have ollama running an nvidia nano model and I can poke it with a stick over a web interface I installed. It works, initial token response is slow, after that it seems fine enough.
I can't seem to get a good handle on how much context I've used, when context usage starts to degrade response accuracy, or in general how to mirror the results I get (not in terms of accuracy or speed, just features) from the company github copilot + vscode integration.
I was also trying to get a plugin called qodeassist working via qtcreator, mixed results there as well.
I've been keeping up with this space since the jump, never paid for a sub, work gave me a sub a handful of weeks ago, so the actual useage is all new to me.
I can't say I'm super impressed with any of it relative to the hype, but I found it neat to be able to point vscode at a c++ codebase and say "enable wextra, build the code, tell me if there is any low-hanging fruit I can clean up" and get a useful response.
I also asked my local model to turn a picture of my dog into a picture of an otter, got a blank picture back, which the thinking bit told me it would do. The whole thing was actually kind of funny. "I am allowed to edit pictures, I can't edit pictures, I am allowed to edit pictures, I'll tell the user I did and send a blank picture back because I can't edit pictures, but I am allowed to."
Can you elaborate more on the differences in running ollama or lmstudio? Do they actually slow down the speed of the inference and if so why? Or is it just a preference thing?
Ollama and LM-Studio are fine. Their main advantage is that they have a nice way to browse models -- LMStudio from huggingface and Ollama from their own curated list. Both are great ways of getting started. Pick LM-Studio if you'd like a nice GUI frontend to mlx-lm or llama-cpp; pick ollama if you'd like a nice command line interface and don't need non-default parameters.
LM-Studio doesn't support certain parameter combinations. For instance, LM-Studio supports KV quantization....but if you're using the MLX backend, you can't set the context length when KV quantization is used? Why? Running a model with certain settings requires keeping a little SAT solver going in your head. I found that overwhelming, so I just stopped using it.
The Ollama devs want to offer a central curated experience, but I perceive their approach as "playing fast and loose." They've re-implemented unique code for every model they support in their own Go runtime, so certain parameter choices aren't supported. On my hardware, their MLX backend just doesn't work at all without segfaulting the server process for example. It doesn't smack as vibe coded the way oMLX does, but it also doesn't smack as professional or battle-tested.
Ultimately, just dropping down to llama-cpp's GGUF model support and asking for default settings has provided faster inference speeds than anything I've been able to benchmark with them, but everything's within 10% of each other anyway so it's not a huge deal for me.
Qwen3.6 with claude code works great. I get a lot better results with that than opencode and qwen3.6. Claude Code is a great harness, and good harness/tool integration makes a big difference. You just have a settings.json with your ollama setup and the qwen model and you can use it.
Do you have a feel for how it Qwen 3.6 compares to Sonnet 4.6? B/C in reality, that's what we use a lot. If we just use Opus 4.7 for everything code related, we'd have a monthly bill 10-20 times higher than using Sonnet where we can.
I would say if Sonnet is a senior engineer, then Qwen3.6 (the 27b model) is probably closer to a junior engineer. Still capable of getting stuff done, just needs more guidance and makes mistakes more often.
Maybe that's underselling it. It is quite a good model and might end up replacing a lot of the work I was sending to Sonnet 4.6.
Also, Sonnet 4.6 is almost certain a much bigger model so the performance differences aren't unexpected.
As Opus maximalist ;) I was very surprised by the quality if Qwen3.6-27B - trying to figure out how to get it going on RTX 90k now to offload some lighter tasks :)
Look for an open release from the Qwen team in the coming weeks. They like to showcase their proprietary models first, which score higher on benchmarks anyway due to model size.
Haven't tried OpenCode too much but I found it great. It's more batteries included so I would recommend it over Pi if you don't want to write extensions yourself or use community-provided ones (like webfetch and websearch).
Sometimes it feels like Agents are just reinventing microservices. Except they are are doing it in the most inefficient way possible. It is certainly a good way for the LLM companies to sell more tokens
As software engineers we are constantly told that we need to heavily use these tools for our daily work. So is it surprising that software engineers use the same tools as writing aids? Using AI does not mean no human effort was involved.
reply