One interesting thing of (accidentally) starting with assembly is that you mostly can’t see magic at all, but instead you see learning magicians explaining computers in all sorts of funny ways.
My first “pc” was a clone of zx spectrum and all I had was a built-in BASIC and then some assembler on a cassette. Both went with “books” on how to use them, together with all of the unlimited time you have when you’re a kid.
This transferred to my first PC and eventually I learned how FAT, DOS, BIOS work, how to make a TSR and fool around B8000/A0000, first steps with 386+. It also helped that my granddad was an impulse electronics engineer and taught me how actual gates work and how computers count, sum and select numbers. He also had access to multiple books on hardware. I knew it all down to the silicon.
Other people had all sorts of magical ideas on how computers work. Special “hidden system areas”, “graphics card does X”, “computers multiply by addition”, etc etc. It’s a human thing that if we don’t understand something, our mind tries to yadda yadda it.
And the more you yadda yadda, the less chances it leaves that you’ll actually learn it. I tend to fight with these half-baked autogenerated explanations and try to dig down to how it really works. For no particular reason, that’s just what I like to do. It leaves a mark on how you work though.
This (roughly) came up in a chat just yesterday. Someone posted a screenshot of a web browser using a font face with no glyphs or something, or I assume this is what causes this display of the codepoints(1). Reminded me of the kind of amateur magic lessons I see regularly:
So, people on the internet who feel like they are some kind of computer wizards will try to school each other online about how computers work. They will go on, like, Quora and ask weird questions like "what is one programming language every real hacker should know?" And someone will come along and write an incredibly definitive-sounding and confident essay about how a real hacker has to know the language of binary, because everything that's happening in a computer is all just binary, 1s and 0s. They heavily imply (at the least) that a real hacker regularly peruses binary values. Not like how you might use a hex editor. Rather, as if wannabe hackers read source, but real hackers read the raw binary. Some kind of M-x Butterfly shit. Assembly language is rarely mentioned.
I don't blame them for it, it's just kind of funny. I can kind of imagine that the way low-level things are glossed over for the sake of explaining high-level things has reasonably left some people with the impression that binary is somehow expressive and that people who truly know computers are fluent in it. And I probably had stranger conceptions than that on my own journey.
Anyway, this screenshot is what they must imagine a turbohacker's computer looks like all the time
Ah, this is the reason I avoid internet squares and chats filled with those who started but didn’t go deep. I just cringe to almost fainting everytime I read something there and my ears burn. Programmer’s humor sections are especially triggering. Surprisingly, HN is not that programmy at all (there’s lots of tech but no code nor “hacker secrets”), so it’s relatively nice to be in.
Lol, now for some reason I'm wondering what you think of the old "how to shoot yourself in the foot" / "how to determine which programming language you're using" jokes.
I guess one actually realistic version of the fully-jacked-in-hacker fantasy is network guys who are in so deep that they can read bin/hex numbers fluently without consciously "converting", etc. This I know of, and from there, I can conceive of someone who has become familiar with the binary representations of certain instructions? Or something? But I guess it would have to be a very specific (and probably read-only) situation. And, unless I'm missing something, it wouldn't amount to a deeper or more enlightened understanding of anything meaningful to software concerns
So-so, honestly. Was okay at the time. Sorry for not sharing it, but that’s how I feel.
Btw, reading and calculating hex is not that hard. In Hiew (and other hex/asm binary file editors that showed hex together with mnemonics) you quickly learn that e.g. B8 xx xx is “mov ax, imm” and C9/CB is a return. You can actually spot these in an 8-bit hex view, and yes, code has a pretty distinct feel in hex and ascii/exts. Zips and data don’t look like code.
For me hex code was read-only, but I remember my granddad entering programs through literally 8 on-off switches plus some more on a pre-sinclair “computer”. I could input hex values. He could input D0-D7 and #CLK itself.
That said there’s not much “hacker” in it. It’s just a knowledge induced by a relatively low level tooling and the times. We did what we did out of interest, not for social attitude. So yeah.
I agree, hallucination is a completely different phenomena from inadvertently filling in knowledge gaps.
Hallucinating to me, is not a one off effect, but a dynamic phenomena. When our sensory processing/interpreting continues iterating, but wanders away from, or separates from, actual sensory input.
Dreams being a functional example. Drugs that cause our sensory systems to be disrupted or overwhelmed by unusual internal signals, being another example.
LLM’s operate in discrete steps. So that “interpretation continues iterating” is a very good description of what’s actually happening.
There’s a little uncertainty in the process, so sometimes it will pick the wrong symbol at the wrong time and the entire process just spirals into nonsense. But it’s semi lucid nonsense like a dream or hallucination not line noise.
The confidently stating the wrong thing bit is arguably a different though related problem. There’s making up a citations that don’t exist and there’s inserting song lyrics where a citation should be.
Confabulation: Inadvertent pseudo-memory fill in, within a generally reasonable stable context representation.
Hallucination: Context representation becomes unstable, resulting in open ended drifting and morphing into incoherence.
The scale of error and recursion of error in the latter make the effects quite different.
You could be right, perhaps sometimes the initial causes might not be so different. But I would be surprised if that was true, given there doesn't seem to be much of a middle ground between the wide differences in scale and downstream effect.
You could be right that it’s a fundamental difference in what’s going on, but each word is a new iteration/token and most errors are more than a single token.
I’m assuming it’s like a converging vs diverging series in how well the LLM recovers from problems. Where the boundary between the two states is arbitrarily small, but OpenAI etc has fine tuned the system so you generally see the converging / confabulation types of errors even if the difference is just a slight change in the scale of error.
> I’m assuming it’s like a converging vs diverging series in how well the LLM recovers from problems.
That's a really good point.
Similar causes diverging in effect, a phase change or not, depending on whether a critical feedback threshold is hit or not. That could certainly be a factor.
With increasing doses of some drugs, humans do progress from being just a bit wonky, to full on lost in space!
But I think confabulation might be on a different continuum too. We fill in recalled memories with made up details all the time. Our memory storage is lossy, associative and overlapping. Our recall is always an imperfect combination of actual incidence memory filled out by similar memories.
We must confabulate to recall. Statistically, sometimes we over confabulate.
Hallucinate seems like a good word for image-producing neural nets (which is probably where it’s use originated?). Confabulate might be a better word for talkative LLMs, and less good for images.
I wonder if any neuroscientists or psychologists agree about humans and neural nets being similar this way. It’s less than unlikely that the mechanisms are the same between people and LLMs saying not true things. Aside from there being a wide variety of reasons people fabricate untrue things, we already know the mechanisms for neural net hallucinations and confabulations. It’s a non-self-aware machine designed to output tokens or pixels or whatever, it will turn the crank and spit out something with no concept of whether it’s true or not at all times. People, on the other hand, are often using emotion to drive what they say (and often without knowing it). People will sometimes rationalize their confabulations, sometimes say untrue things based on belief, sometimes say things driven by fear or embarrassment, sometimes lie because they have ulterior motives. None of these things apply to neural nets, so the similarity between human and NN confabulations seems at best limited to superficial summaries, no?
I was talking about the tendency to purely imagine things in low knowledge situation, but (iiuc) agree that this is a thing with our minds as well. Mine can skip a whole page of text if an earlier paragraph made it actively think. After that it’s sure we read it all. I have to detect that and rewind properly.
My first “pc” was a clone of zx spectrum and all I had was a built-in BASIC and then some assembler on a cassette. Both went with “books” on how to use them, together with all of the unlimited time you have when you’re a kid.
This transferred to my first PC and eventually I learned how FAT, DOS, BIOS work, how to make a TSR and fool around B8000/A0000, first steps with 386+. It also helped that my granddad was an impulse electronics engineer and taught me how actual gates work and how computers count, sum and select numbers. He also had access to multiple books on hardware. I knew it all down to the silicon.
Other people had all sorts of magical ideas on how computers work. Special “hidden system areas”, “graphics card does X”, “computers multiply by addition”, etc etc. It’s a human thing that if we don’t understand something, our mind tries to yadda yadda it.
And the more you yadda yadda, the less chances it leaves that you’ll actually learn it. I tend to fight with these half-baked autogenerated explanations and try to dig down to how it really works. For no particular reason, that’s just what I like to do. It leaves a mark on how you work though.