Also, the dark pattern where it shows the interface and lets you enter a prompt/set settings, but then pops up the 'create account' dialog when you press submit is pretty annoying.
>"Massgrave, the group behind Microsoft Activation Scripts, has noted that Windows setup sends hardware info to Microsoft and receives identifiers back that are later used for Store access and licensing. Blocking GDID assignment breaks both activation and UWP apps."
I think probably it sends the gdid back to Microsoft as part of telemetry/updates/MS account periodic re-auth, which lets them know the current IP address, and then once the government has Microsoft's logs and the various target websites' logs, they can correlate based on IP address. I don't think it's actually sending the gdid to the web sites. Maybe.
This serves to further illustrate that nobody should be using Windows for anything that involves the need for privacy. And doubly, triply, and morefold so, nobody should ever sign a Windows machine into a MS account for any reason.
The article is not about "should people be allowed to buy this product because it's potentially dangerous/addictive/etc" but "Should the company be allowed to sell this product because it consists of acetaminophen plus two useless ingredients and is basically a scam".
This is the Platonic ideal of an LLM-generated blog post; it's pages long, it uses a lot of flowery language, and communicates almost zero actual information. I can't disagree with the premise, because the premise melts away when you try to pin it down, like cotton candy washed in a stream by a raccoon.
Congratulations, person who wrote the prompt that caused a machine to generate this article, you have truly achieved some sort of perfect something.
I agree; the situation "young people know all about computers" was maybe a twenty-five year phenomenon; before that, there weren't enough computers for it to be generally true, and at some point you no longer needed to know all about computers to successfully use a computer for the sorts of things the general population does.
"Young people know all about computers" was never true either. It was maybe true for people like you and me, the type of people who now frequent places like HN. But for the vast majority of people, computers were never all that interesting.
>"Young people know all about computers" was never true either.
That's true, but only because not all young people owned or used computers. I was born in the mid 70s and got my first computer as a young child. Back then you had to "know all about computers" to use the computer and get it to function. These days, every kid grows up surrounded by computers, with a computer in their pocket, and very few have even an inkling of how they work.
Thus "young people who use computers know all about computers" used to be a true statement, but no longer is.
It wouldn't have been true even if all young people in our generation would have had all the access to computers they would've liked. Most people just don't like tinkering with technology and that's fine!
The whole argument is pretty dumb anyway. Just try replacing "computers" with any other technology and you see how weird it starts sounding.
- "People use roads every day but almost nobody even knows how to make asphalt from scratch these days."
- "People use electricity every day but almost nobody even knows the basic equations for electric generators."
- "Children these days use refrigerators only as consumers, they don't even understand the differences between HCFC and A3 type refrigerants..."
Computer programmers despairing about how few children understand computers has strong vibes from https://xkcd.com/2501/ IMO.
The issue with this is that we don't know how it works. Generally speaking, we know how the level of abstraction that we were born with works. We might have some understanding of one or two previous levels, but that decreases the farther down you go. We might understand the next level, and some of the next after that, but eventually people will be making things that we don't have the context to understand without having to unlearn a lot of what we know now.
I'm old enough to see this process in action; I used to be young and in possession of esoteric knowledge that made me infinitely in demand and now most of the things that young people have esoteric knowledge about is things that I don't particularly care about, and I'm left with a lot of finely honed skills to solve problems that have mostly been abstracted away.
I do think there's something to be said for the big difference in the current abstraction layer jump - the loss of determinism. It is meaningfully different.
There's a big difference between "deterministic + noise" non-determinism and "intelligent agent" non-determinism. Only the former can be statistically modeled to characterize and work around the noise with any reliability. If you measure hallucination rate for one prompt it tells you nothing about hallucination rate for another prompt.
> There's a big difference between "deterministic + noise" non-determinism and "intelligent agent" non-determinism
I think I agree, but could you expand?
My point is humans have made progress on understanding systems without clean determinism. Genetics and cosmology is technically deterministic, practically speaking, but I don’t think that’s accessible to pre-modern societies. Instead, trends and tendencies were observed, acted and capitalized on.
One big difference is the former doesn't "know" it's being experimented on. Gregor Mendel was able to grow large numbers of pea plants and identify patterns in their inheritance because the non-determinism followed consistent statistical laws. A plant can't suddenly decide to follow different laws of genetics because it saw a biologist.
On the other hand, an LLM "knows" what a benchmark is, and is capable of detecting benchmark-like scenarios. A prompt that appears reliable in testing may fail unexpected in the real world. In this way, an LLM is like a stock market (also composed of intelligent agents): the act of experimenting on it changes its properties. Such systems are never reliably predictable.
In engineering you have tolerances to deal with non-determinism. ”Within these bounds” and ”given these assumptions then…” is the foundation of building something _on top of_ those things. LLMs are the same, it relies on heavily exact Turing machines as input but its output is entirely unstable. Even if you can get determinism it will never get anything resembling ”bounds” out of the box. That makes it a poor foundation for building on top of. Ie it’s not a screwdriver, it’s the monkey who’s holding it.
I do not understand the need to argue that monkeys are better than screwdrivers at screwing. Just let the monkey be the best version of a monkey.
I would argue that management is a better discipline to pull from for employing LLMs; It is better equipped to deal with non-determinism and going completely off the rails.
So now instead of improving the tools of biology so we can actually understand it deeply - we increase complexity of IT so we have to rely on muddy, side-effecty tools of biology to try to infer some of the properties of the systems we made. That's depressing.
I feel crushing sadness when I think about how humans made these wonderful deterministic machines that we can understand and manipulate to do our bidding. Within a scant few generations, though, we've managed to effectively turn them into "magic boxes" that, soon, we'll only be able to poke and prod at when they don't do what we want.
I think it's safe to say that computing should be mostly deterministic. I know compilers use heuristics that are stochastic, but come on. Imagine if you were given a layer of abstraction that randomly flipped bits from time to time... That's not an abstraction, it's random programming.
> Imagine if you were given a layer of abstraction that randomly flipped bits from time to time... That's not an abstraction, it's random programming.
Computing has never really been deterministic all the way down. Storage gets corrupted, RAM has soft errors, networks drop/reorder packets, schedulers race, caches go stale, distributed systems partition, query planners change plans, ...
Obviously those become tolerable because we've developed layers of contracts and understanding the bounds around them. Error correcting codes, checksums, retries, consensus, idempotency, false-positive rates, SLAs, etc.
If the abstraction is “delegate this task to a junior engineer, analyst, lawyer, designer, or support rep,” you dont expect deterministic behavior. There's review, constraints, escalation, checklists, tests, and accountability.
People use the word "determinism" when they really mean something akin to "linearity", i.e. the predictability of a change in input on a change in output. Compilers for example are both deterministic and "more linear" in the sense that I can tell what the output will look like given a change in input (yes yes optimizations violate this to a small degree). LLMs can be made totally deterministic, but a seemingly insignificant change in input can create a drastic change in output, which is the characterstic we don't want.
The generally used term that I'm familiar with for this is "referential transparency", that given the same inputs you'll get the same outputs every time. LLMs can be deterministic (referentially transparent) but almost none are, i.e. when they are given the exact same input they do not return the exact same output.
I would say it is unambiguously defined by the problem you're solving. For example a spellchecker is inherently probabilistic so you shouldn't need hard coded rules here like we did for years. A "bug" then would be considered "weak accuracy" not "crash" or "incorrect behavior", as it might be the case in any other layer.
They should add a term for it, maybe "embarrassingly nondeterministic"
> I would say [whether nondeterminism is bad] is unambiguously defined by the problem you're solving.
Sure. You gave an example where it can work. Another example is something like a PCB or chip layout engine.
That particular domain (layout) is NP-complete. You'll never have an exhaustive brute force search for the optimal layout, but...
You can subsequently easily check whether the produced layout meets all your acceptance criteria or not.
Another example that successfully utilizes non-determinism for good outcomes is the application of genetic algorithms to things like antenna design.
This works because (a) you have a relatively cheap fitness test; and (b) as with real evolution, the mix of combining working designs and randomly introducing mutations often eventually produces outstanding results.
Presumably, if you applied genetic algorithm techniques to, e.g., creating your LLM prompts, you could also get good results, but that would probably quickly get expensive in terms of tokens.
So we're left with people just semi-randomly modifying prompts in order to try to tweak results.
When it works, it can be amazing. When it doesn't work, it's like a brick wall.
I like your "embarrassingly nondeterministic" term, but I somewhat disagree with:
> A "bug" then would be considered "weak accuracy" not "crash" or "incorrect behavior",
When a lawyer asks an LLM for citations of cases that support his position, he is arguably doing something stupid, because embedding an assertion such as "Show me cases which support X" is just asking for hallucinatory trouble with many current LLMs.
Nonetheless, I submit that hallucinations are, by definition, "incorrect behavior" and not merely "weak accuracy."
Now, nondeterminism could certainly be useful to the lawyer, in that it could help an LLM make connections that LexisNexis might not have in their database. So asking an LLM for help with legal issues is theoretically not an insane thing to do, but the results need to be checked very carefully.
I think I made an implicit distinction there that almost always get overlooked in these discussions, between runtime inference (when you run a transformer in your product) and offline inference (where you use AI in any ways anytime before that).
Chip design, etc would go to the latter category which is actually a huge space for innovation regardless of how you do it, while I was focusing more on the former which I think a lot of discussion seems to be concerned about.
Now if you are "running" your "generated code" that's where people will have strong opinions because it conflates the two completely different processes in the worst way possible.
It's kind of a useful distinction, but the categories are really blurry.
For example, the layout software vendors provide a solution that both uses non-determinism in their final product, and also checks the results against whatever criteria you give it.
If it tells you it will work, it's probably not lying, and if it tells you it won't work, it's almost certainly not lying.
So in your own creative process of building a board, you can rely on that.
Many LLMs don't have this property at all, and honestly, it really doesn't matter whether you are using the LLM to craft a legal document or to craft code; if you don't check the output yourself, you're possibly in for a world of hurt.
> Now if you are "running" your "generated code" that's where people will have strong opinions because it conflates the two completely different processes in the worst way possible.
I think a primary driver of the attractive nuisance here is that, because the execution of the resultant code is itself a deterministic process (even if the process that produced that code it isn't) people think that they can tame the process, by some combination of automated testing and looped automated reprompting.
At some level, they may not be wrong -- computer chips themselves are built on top of stochastic atomic processes, and google famously proved in its early days that it could have reliable processes on top of shitty commodity hardware.
But one key difference is that the nondeterminism in the atomic processes or in the non-ECC memory of the white box computers is uncorrelated.
And another key difference is that, unlike the layout software vendors, so far the LLM vendors seem unable or unwilling to properly self-check their own outputs.
To me, this is kind of the canary in the coal mine. You would think that the LLM vendors have every incentive to weed out bad results, that they, more than anybody else, have the understanding of root causes and probabilities that a particular output might be bad, and have more than sufficient resources to fix this, if it's not an intractable problem.
So Occam's Razor says that (currently) this is still an intractable problem.
In 50 years, Gen Z will be retired, and they're going to be complaining: "The new generations don't know how anything works! They don't know how to enter in a prompt or optimize token usage. They don't even remember what keyboards are! They just Neropath into the UltraMind and synthesize facts into their neurons without even putting them to words or reading anything. We have lost something critical!"
I agree, but having abstractions allow us to learn things in a system that are broadly applied.
This is out the door for AI generated systems. Unless extreme care is taken there is is no consistency in these new-found codebases. New paradigms are introduced left and right, because in the eye of the LLM, and the prompter, they worked.
It didn't matter that the same pattern was repeated 37 times in a slightly different manner.
So your knowledge now is no longer portable.
It used to be that you could look at code, and ask why? And usually (not always) the answer was something like: 'I tried x, y, and z, and those didn't work', either by past experience or current experimentation. (we could argue if the current experimentation should be documented).
But for LLMs they put in something complex for no other reason than that it does what is requested. Reading a string from a valid source byte by byte until you hit \0 is valid, and it'll work. But if you take a step back at, read the API docs of what you're consuming, and then consider: if the API says X, why am I testing whether that's correct?
I'm not sure that's true - you may be over generalizing
Probably not many here know the processes needed to turn sand into silicon and the expertise needed for hand grinding fancy lenses used in lithography. But we do know those things are needed and the approximate philosophical concepts behind these needs.
I think the danger comes when or if we fully automate a lot of the low level infrastructure tasks to the point where future generations do not even have the conceptual framework for how the tech they use is created
Exactly, I find this sort of nostalgizing to be unhelpful. I am not _that_ old but I did cut my teeth on C++ and learned about manual memory management, programming language design and had a few (even at the time) esoteric courses about assembly language. I am not assembly fluent and have no desire to be - but getting an understanding of how higher level code compiles into lower level assembly/machine statements is invaluable to get a sense of how performance can be manipulated and the how impactful it can be to do "silly" things like unroll loops to minimize the number of instructions needed for some higher level operation.
Issac Newton/Bernard of Chartres said we stand on the shoulders of giants and that allows us to see further afield but it also loses the detail of the ground beneath us - modern folks will no longer have the broad expertise that someone struggling through building a computer from transistors would gain through overcoming that[1], but we should learn about it academically to garner the bits of knowledge that remain salient. It is incredibly useful when experts in these highly specific fields have the communication skills and knowledge to share what is relevant to others without overwhelming them[2] with detail.
1. Real modern consumer grade computers are far too complex to understand fully without dedicating your life (or a significant portion of it) solely to that. It is more pragmatic to allow specialization and small realm expertise instead.
2. Of course, if you want to geek out and learn the minutiae that's awesome - but we can only geek so much in our limited time here.
> Generally speaking, we know how the level of abstraction that we were born with works
I'd say it's generally true that the majority of jobs of an era deal with a similar level of abstraction, and that's why most people stay on it. However, I frame this as being born with technical debt, and it's my obligation as an engineer to understand what the previous generations have built, and where it makes sense for me to work, directionally.
I think it's more that "Some human some time has known how it works", not that "Any given human knows how it (all) works.".
But yea this glosses over a bit trial-and-error designs and, so to speak, "genetic optimization" kinds of designs where we just try random stuff and say "Hey, this works. Not sure why, but it works.".
There was a period of a century or more where steam engine design parameters were determined by experiment and gut feel. The physics of fluid dynamics required to explain why a design is optimal just did not exist yet.
There's a lot of places in history where engineering far outpaced the science required to explain it.
I disagree. For example, the Wright brothers did air foil engineering empirically without full knowledge of the underlying fluid mechanics. Early and innovative engineering will often be less based on existing scientific or mathematical knowledge, while still using some of the methods. It's still absolutely defined as engineering.
It was always engineering. You're making the assumption that their theories are not rooted in anything coherent at all, but almost always this is not the case. Usually a "what if we..." thought is based on other knowledge, and there's absolutely nothing wrong with trial and error... this is what research and experimentation, areas of science, are all about.
Of course it is. The science comes from reproducing what you tried and getting the same results.
You may not have a good enough model for this observation to allow you to predict the results of other starting conditions, but it’s a model nonetheless, one that narrowly applies to the conditions you tried.
The level of abstraction when I started was that you had to solder the components (resistors, capacitors, chips - well IC holders, chips were expensive) to the PCB board - that was "building your computer", none of this nancy-boy rubbish of "plugging stuff together" with "slots that can only take one type of card".
I'm fairly certain I know how it works. Being a physicist helps with the even-lower-level-details if you want to start talking about transistor doping, or electrical circuit theory, for example
Exactly, I'm over 50 and I remember all the complaints about script kiddies who looked at windows bat files as opposed to all the 'real programmers' who knew C and Assembly and used VIM and linux (which is still going strong)
A (sad/funny) anecdote. Someone told me earlier this week "Not using a coding agent today is like using Vim for development instead of an IDE, you're just lagging behind".
As someone who's been exclusively using Vim for my development (and can definitely integrate it with AI workflows), that's just an insanely silly opinion. But I guess it shows how the next generation thinks about these tools that they've heard of but never actually bothered to learn.
Script kiddies didn't look at the .bat files. They ran them without a clue how they worked. There was some respect for the people who wrote those scripts but none for users who couldn't write them
I expect 20 years from now there will be people reminiscing "remember claude.md? Back then we knew how it all worked. We had deep control. We counted tokens, chose the model..."
> The issue with this is that we don't know how it works. Generally speaking, we know how the level of abstraction that we were born with works.
What? Definitely not. I went to university and my first two years were subjects where I had to understand really deep levels of abstractions. I had to build logic gates, I had to work with hardware, wires, etc. I didnt see the point back then (I never used any of that professionally). The same about algorithms, databases, and a lot of things. But now I find it valuable and thankful that my professors (and whoever designed the career) considered important topics that I had to lear.
Yes, thats exactly what uni is for.. To learn all those previous centuries of stuff before you can either start contributing to the corpus yourself, or go get a job (Where you will pick up more immediately useful stuff)
We really dont understand how AI is working, even the earliest "genetic algorithms" could be incomprehensible, but computer systems in general, they're not really that complicated.. its like an audio mixing desk..it looks insanely complicated until you realise it's just the same few knobs repeated many times for many channels. High level languages, compilers, assembly, machine code, nand, mosfets. A single person really can understand it all.
You started with logic gates. How much EE did you do, or the actual physics that makes the transistor possible? Those are the previous (deeper) levels that people had to know before they got abstracted away.
When I took software focused computer engineering around 2010, we still had courses that took us all the way down to transistors and even the physics of P and N junctions and how that applies to CMOS. (And even some basic analog electronics.)
Did I end up an expert at those layers? Of course not, but I know the basics and I know enough that if I need to I know where to start learning more. Just like I wasn't a C++ or hard realtime expert after university either, but now a decade and a half later I am pretty good at those (and a bunch of other skills that ended up relevant to my line of work).
Basically, none of the layers are "magic" to me. Even if I don't know the details of it, I know the general principle and I know I could learn more if I need it.
(I think you naturally end up an expert at the layer(s) you work in, and the knowledge tapers off as you go down (or up) the stack. For example, I know a fair bit about how the CPU works (cache coherency, pipeline stalls etc), I can passably read x86 assembly, etc. Because they affect the layer I work at (hard realtime systems C++ and now also Rust). I know far less about web dev than hardware.)
Similar experience in the 90s, but we don't really know the intricacies of doping silicone, or smelting metal to make the pins. And what about mining it?
I think the last time people knew how things were made was in preindustrial societies because they had to build everything themselves (whatever little things they had)
I partially disagree. I know the basic high level concept of all those. Would I be able to reproduce state of the art results on my own? Obviously not.
But a core part of the engineer or scientist mindset is curiosity for the sake of curiosity. Just the fact that I don't know something is enough reason for me to poke at it or otherwise learn more. Same reason I still take apart broken electronics as an adult and try to find the fault (and sometimes even repair it).
By the way, mining silicon is particularly easy: it's basically sand. The difficult part is purifying it, especially to the levels needed for modern nm scale chips.
A more useful question than making high end silicon would be: could you with reasonable tooling reproduce basic electric components? I'm talking things like light bulbs, resistors, generators, perhaps capacitors even? Just the basics crappy versions, not modern highly optimised surface mount components. And I think the anwer is yes (for me personally) if I had access to metal wire and sheet stock and industrial revolution era tools.
I was once in a museum in Bolzano, Italy. And they had the ice man mummy there (we call him Ötzi). He died like 5000years ago. And his "axe" was made of copper from some mine very far away. So even this guy probably did not know how everything he had was made.
There is a theory that he was probably very rich. So maybe less rich people were more in touch with what they owned. But still I found it fascinating that even so far back people relied on technology and materials that they didn't really know about.
I have similar experience with the poster, and the way I read it is, "from the things I build here are some examples". I did learn about advanced physics topics that enable transistors, and even did some experiments, but for fundamental stuff you "don't build stuff".
Did I do all physics or all electronic circuit design or all software stacks? Definitely not. But I spent 3 years learning (and building) about lots of stuff.
The one thing I appreciated about my Computer Engineering undergrad - and it took me a few years to fully appreciate it - is that yes, we did cover those levels.
The first two years were shared with Electrical Engineering. The second two years started to specialize towards Computer Engineering topics.
* Physics and chemistry.
* Circuits.
* Transistors.
* Logic gates.
* FPGAs.
* Assembly.
* Compilers.
* CPU and hardware design.
* Operating systems.
* Networking layers.
* Programming languages.
* Computer graphics.
Did I master all of the above - absolutely not. I loved some of them, struggled with others. Generally the cut-off for how my brain works is logic gates, I was never strong at the levels below that.
But we did cover them, and I could honestly say I had at least a rough understanding and mental map of everything that happens inside a computer from the point where it's plugged into an outlet, to the point where pixels show up on the screen.
> You started with logic gates. How much EE did you do, or the actual physics that makes the transistor possible?
As someone with an EE degree, in first year studies we had physics classes on quantum topics that talked about electron tunnelling and its applicability to transistors:
how did you make the transistors? made your own vacuum chambers?
i had to make logic gates and so on, but i wouldnt say i really learnt it, even if back in highschool i learned all the different things a 555 timer can do
Please get used to this sort of depressive, absurd and out of touch tone from HNers, it’s literally all they do now. Don’t bother calling people here hackers anymore, they have checked out emotionally and spiritually.
What's absurd is thinking because you took a logic class and made a flip flop 30 years ago that that is the ground floor and that it means you "understand how it all works". You're not building a CPU from logic gates and you don't know how it works. If I put you or OP in a room for a year I highly doubt you could build an 8 bit Atari-like CPU from scratch. I worked with wires and logic too but I'm not arrogant enough to say that I know how it all works.
Don't you think there is a difference from "knowing how it works" and "reproducing every aspect of it at the level of the state of the art" ?
Also, your example seems flawed if you restrict to a certain product. Can I build a compiler from scratch? Yes. Can I reproduce in a year a compiler with LLVM/GCC performance level? No. Can I build a compiler from scratch in a year from a room if I need to starting mining from metals, building transistors, then building the first assembler and then implementing the compiler? You can imagine the answer.
Not really, at least not in this discussion. An assembly programmer gets logic gates, a C programmer gets assembly, etc etc. And each one can say to the one above "well you don't really get it if you can't bang out some meaningful work". Otherwise idk what we're even talking about here. Having a vague understanding of something isn't "knowing how it works".
Yeah, but if we use your criteria, people have to invent their own atoms out of individual electrons, protons, neutrons, quarks, etc before you'd admit that "hey, they did build a compiler from scratch".
I think a fair bar for "from scratch" is stopping at the point they need an expensive fab process (or how else will they reproduce the workings of semiconductors?)
> If I put you or OP in a room for a year I highly doubt you could build an 8 bit Atari-like CPU from scratch.
I believe I can, depending on what "from scratch" means. With nothing but transistors, resistors, inductors, coils, capacitors, I can probably do apretty poor general-purpose CPU. Maybe something like a stack-based ISA or, if I had more time, an accumulator-based processor like a 6502, but with an 8-bit bus.
If you're going to ask me to create transistors themselves, well I believe that needs specialised equipment.
The parent post's argument can be boiled down to "You don't know absolutely everything therefore it's fine to know little or nothing." It's the Chewbacca defense of AI boosterism.
That was one the major criticism I have of my CS degree's program. Perhaps the knowledge I wish for is not truly CS in the mathematical/theoretical perspective, but I believe it would have been absolutely valuable to learn.
Kroger is one example. Most other stores in Texas had the same depressed skeleton crew feel, except for higher end options that cost 3x normal groceries.
Same thing for Bucee’s compared to normal gas stations.
Bucee’s popularity exploded by asking “what if we paid someone to clean the bathrooms at a gas station” and following the logical chain of thought from there.
People like spending money at businesses that aren’t depressing or gross to be in.
Their gas is priced competitively. Everything is hard to compare because it's usually not even available aty other gas stations. It's like half of a Bass Pro Shops tacked on to a bizarre bbq restaurant that lacks seating. Comparing it to 7-11 or a Sinclair or Conoco gas station makes little sense. It's closer to a Love's Travel Center, and in which case their pricing holds up.
Perhaps interesting, my Publix experiences are almost always… not great. The staff seems aged (don’t hate me, I’m also aged!) - but they seem more on the apathetic and miserable side of the spectrum. I always heard how well they treat their employees so I assume it is some kind of retail trap where the job is too good to give up but never good enough to be truly good?
Kroger can and does run nice stores. Kroger Signature, Harris Teeter are two I've been in relatively recently.
You're also only going to find those in the same kinds of places you'd find a Trader Joe's or Costco - only in the nicest parts of town unless it's very densely populated or a small but rich enclave.
reply