Therefore Einstein couldn't have had the idea of relativity because it wasn't in his training data.
Joking aside, the reality is that a pre-Einstein in 1850 couldn't have had the idea of relativity because Maxwell wasn't in his training data.
And a pre-Maxwell couldn't have had the idea of electromagnetism in 1820 because Faraday wasn't in his training data.
What we call "new ideas" are actually just deduced from the available training data, which includes all the previous ideas and all experimental results. AI can do the exact same thing (I assert). Perhaps not today's AI, but some AI, probably not too far in the future.
How I see it, is if the human brain does lossy compression/prediction of the natural world that learns from its "training set" (sensory inputs) and we have been able to come up with new ideas, then it seems like AI would be able to as well.
Once MP3s were invented, I had the idea for the Apple IPOD; but obviously I didn't have a giant manufacturing wing, the ability to make small hard drives, or anything else.
I don't think Apple invented the ipod anymore than I invented it; LLMs likely would have also come to the same conclusion about an ipod like device.
Original ideas either dont exist or have a functionally irrelevent definition in comparison with inputing tokens to LLMs to get novel ideas out.
Imagine you're curve-fitting a bunch of data points on, say, the orbital motion of planets and asteroids. You get tons and tons of data on these orbital motions, and then put it into a huge black-box optimization algorithm that compresses the heck out of it. It compresess and compresses until it can't find a more compact representation, no matter how much more effort it applies. The output is a function, where you can provide an initial condition, and it gives you the predicted future orbital position at any requested time.
Of course, one thing you get out of this is a great curve-fit for your existing data, which you can interpolate to find the position of any observed planet at any desired time.
But could this function also succeed in predicting the orbital motion of objects that aren't in the dataset? If I spot a new comet, and put it into the compressed function, would I get an accurate prediction of its orbital motion, even though that object wasn't in the training data?
The answer is "it depends, but probably yes". Newton's laws of orbital motion turn out to be simple compared to the size of the training data. So if the black-box compression has done a good job, it might output that function, or a close approximation of it. With a sufficient quantity of sufficiently accurate data, it might even improve on it; random errors can't be compressed, but where the deviations between observations and Newton's law turn out not to be random but rather the influence of an unobserved gravity source, or general relativity, the black-box algorithm will likely capture that as well.
A lot of people seem to think of the training process as curve-fitting data (the "stochastic parrot" model), but I think of it more as "solving an inverse problem to approximate the unknown source that generated the training data". Machine learning has proven to be quite good at solving inverse problems, and this is just a very abstract one of them.
(A forward-problem is something like solving for the electric fields from a set of charged particles; an inverse problem, https://en.wikipedia.org/wiki/Inverse_problem, is one where you have data on the electric fields at various points and want to reconstruct the arrangement of charged particles that produced it. Or more generally, you have sampled data on the output of an unknown process, and want to reconstruct the process that produced the data).
The inverse-problem-solving happens at the ML training step. The language model itself, that comes out of that, is solving the forward-problem: it has a generative-process baked in and now it's generating new data from it. But if the training process has done a good job of compression, it will certainly be able to generate valid new ideas that aren't in the training set, because the inverse model has solved for the underlying features of the real process that generated the training data, and those features can generate additional valid outputs that it wasn't trained on.
> A lot of people seem to think of the training process as curve-fitting data (the "stochastic parrot" model), but I think of it more as "solving an inverse problem to approximate the unknown source that generated the training data"
I just wanted to confirm your underlying point here: training a model isn’t about finding a function that fits the observed data (even though that’s the outcome) but instead finding an approximation of the unknown source that generated the source data in the first place.
In the case of LLMs an approximation not of a human but of the sum of humanity that produced the training copora?
Yes, exactly. And it's the compression that makes this happen. If it was just curve-fitting then the LLM really would act like a stochastic parrot, and it wouldn't generalize; also the model would also need to be much much bigger. As it is, the models are very big but they're still tiny compared to the dataset that they compress, so the compression process forces them to approximate a solution to the inverse problem.
And yeah, that means the base model training process is solving the inverse problem of finding an approximation for the processes that generate all human records and also anything else in the dataset with a compressible pattern to discover (weather data, etc). So it's not surprisng that some kind of world model emerges out of this.
except that it's really not. It's about finding a function to fit the data. And "generalizing" is only when the question is jsut barely outside their training data. Like a truncated taylor series, is still somewhat accurate just outside the region it was designed for, But I couldn't say a truncated taylor series generalizes over the function it's approximating.
I urge you to reconsider your beliefs. You are missing something important because you are thinking in terms of low-dimensional statistics. Deep learning doesn't just fit data, it finds features (abstractions) of the data.
I asked the original commenter to confirm my understanding of what they were saying.
I find it a fascinating alternative view to what is largely well understood (your counter-point).
The thing that stood out is the comment that it’s an approximation of the original data generator (humanity). Early approximations were poor (GPT 2-3, to an extent GPT-4).
I’m not so sure I can reject the hypothesis that such an approximation can be found.
Hard in "hard science" doesn't mean hard as in difficulty, it means hard as in not soft. Soft sciences are difficult to explore since they aren't rigid, they move around as you prod at them etc, you can't get a good grasp of its shape since they are so soft.
Physics on the other hand is hard as in unyielding. It is easy to figure out boundaries of physics and map out what is and isn't true, and the few cases were we made a mistake everyone can agree a mistake was made and that formulas needs to be updated since physics is so extremely hard that even a tiny error will get noticed.
Hope that clears it up, physics isn't hard as in difficult, its hard as in rigid. And psychology is soft, not easy.
So, your statement doesn't make sense at all in this discussion, they just said psychology has soft traits, and then you say "ok, so its hard since its soft!". No, soft is difficult, not hard.
So for example, physics is like describing the shape of a metal spoon, and psychology is like describing the shape of a pillow. You can see how describing the shape of the pillow is massively more difficult, because its not fixed, so you have to come up with a language to describe all the ways it can deform and how that would work.
"Hard" in hard science means rigorous methodology and experimentally verifiable predictions. Psychology achieves neither, ergo it is a soft science, or more precisely, opinion masquerading as fact.
I appreciate this push-back. I keep a running list of words like this...content, product, device, experience, space, vehicle, resource, etc. I can't put my finger on why, but all of these make me distrust the speaker.
Yeah I have a similar list. Mine also includes "journey". As soon as a word gets adopted by MBAs to abstract away the real world, it gets added to my list. They're words that project the idea of a smart person but only to stupid people.
I know a few people who unironically call themselves that. They sound awesome, but scratch beneath the surface of what they say... there's nothing there.
I saw a news article recently where the headline was something about someone's "cancer journey". Maybe I'm old - only 51! - but I thought it was kind of a tasteless way to put it. That said, language is a malleable thing and no one needs my approval.
It’s a conflict of interest. When people organically create things online with no incentive except the love, you can give them a lot of trust. Once someone goes from that to manufacturing “content” (or even worse goes from nothing to “content”) there’s major incentive to either shill for some product for compensation, or to generate outlandish things that get more clicks, views, likes, subscribes. All of this is antithetical to creating about a subject because you enjoy the subject.
From about 1998-2008 I ran one of the largest sites on the internet for a particular niche. Zero advertisers, no ads, no shady links, just content about a subject I loved. I worked hard to make sure it was accurate and up to date. Now someone creating the same site would be loads of referral links or paid ads, so no trust whatsoever in what’s being presented.
I get a similar ick when I hear someone say "I enjoy consuming this type of content." They're not selling anything, but they are still adopting the soulless language. Like, if it still exists after you interact with it, you haven't consumed anything. Second, do you really see yourself this way? Why not just say "I love to read travel blogs"?
All these words are too broad is why. It allows someone to say something without really knowing what they mean. But also they may know what they mean, so you are not sure!
Device is a handy one though you often want to say "computer, laptop, tablet or phone" in one go. We could call them all computers but that would confuse people I think.
Yeah exactly, It is meant to mean/sound something in a way which could impress people or get any desirable outcome but in reality it means nothing or sometimes even opposite in a way similar to double-speak, recent layoffs and how they are worded seems to be another example.
These words are sometimes like schrodingers cat and this is also what I have observed that within recent layoffs, its really hard to accurately find what the real intent behind it was within some companies and there is no wide consensus.
Perhaps this is their true purpose, to hide the real intent behind things and perhaps (to give the benefit of doubt?)
I don't know how common it is but I can't stand "consume" for "eat/drink". Especially when used in the first person. Why would anybody say that about what they eat?
A few days ago I was watching a video history of a storied company which eventually disappeared. Which one doesn't really matter, as the story is repeated so many times.
By the 1990s, it had largely withdrawn from its earlier manufacturing role to "branding" products made largely in Taiwan. Its manufacturing "partner" learned the design and the business, and eventually took over the marketing role itself. The new (foreign) brand exists, and is relatively well-known, the century-plus original ... was sold in bankruptcy auction in the early aughts and ... surprise to me, actually continues on as a "brand".
What I've noticed as well are two common trends:
Many well-known brands, particularly in appliances, sell products which are all but certainly produced by whitebox manufacturers in China, with the "brand" literally an adhesive label slapped on at the end. There's not even enough trust in the relationship to weld or stamp the name onto the goods. Often times those badges will fall off after a few months....
Increasingly I see goods with no manufacturer indication at all. Ceramics, electronics, household goods, and the like. There may have been some sort of adhesive label, but there's no permanent branding or marking that will remain on the device.
I'm hardly the most brand-aware or brand-loyal person out there, but I did come of age in an environment where the marketing philosophy was "brand is king". After decades of rebadging goods from independent manufacturers, even the pretense of branding's significance no longer seems to exist, in many cases.
"Divisions" and "product lines" were the terms used at the beginning of mass production. One of the innovations of Alfred P. Sloan at General Motors was the idea of a division-based corporation, or the M-form company, contrasting with the earlier U-form (unitary) business.
If you're not an English speaker, it's even more aggravating when speakers of your language replace common words with English ones to sound more sophisticated/corporate. So instead of using the native word for "knowledge", they'd use "know-how". (Of course using more common words like "knowledge" or "skill" are not good enough, you need something that sounds more sophisticated to the foreign ear, like "know-how".)
Someone walks out of a magic store holding a coin.
They propose a bet. If they flip it 100 times and the proportion of heads is within [0.4, 0.6], you win $100. If it's not, you pay $100. Do you take that bet?
Explanation: absent the magic store scenario, a `rational' person would take the bet. Your prior belief is that most coins are roughly unbiased. Given that they walked out of a magic store, you now have additional information. Maybe the coin is a trick coin. In that case, your belief that the coin is unbiased should be weaker, even if you don't know which direction the coin is biased in.
This illustrates two things: one, additional information (magic store) can update your beliefs. Two, a strong prior and a weak prior, in this case about the coin's bias, can lead to materially different decisions.
The bet does not really matter, the central question is whether they have a fair coin or if they are trying to me in some way. Even without the magic store, I would be very suspicious of anyone approaching random people with an offer like that.
So I would certainly consider it likely, that they are trying to trick me. But the probability I would assign to this, would still be rooted in some frequency, somewhere under the hood I would try to estimate the possible situations leading to such an offer and in which fraction of them I will be tricked.
If I am doing a good job with that, then repeatedly being in this situation should result in me getting tricked with the probability I cooked up. If I am bad at figuring out the possible states and their probabilities, then I they will not match.
The key operation of Bayesian inference is integrating information. This can be from the same source (an additional coin flip, for example) or from different sources (coin flips, plus auxiliary knowledge about where the coin came from, or the person's motives).
Calculated frequency is a point-estimate of bias. A Bayesian estimate is a distribution of belief over possible values of the bias, integrating all available information.
You could do a two stage process, a blinded text review and an unblinded review that can only be failed if you have a vigorous lack of knowledge about your work.
I highly doubt anything like that will be implemented though.
My own preference would be to build educational experiences on three pillars:
1. experiences. Intuition comes from experiences, and IMO an under-appreciated amount of 'education' is building strong intuitions. Experiences can include project work (including struggling!), travel & reading (what it's like to be someone else), sports and music (what it's like to build skills over time and work as a team).
2. practice. So much of what we can do - from language to mathematics - is a composition of rote behaviors, responses, and habits. It's impossible to become skilled without practice.
3. building habits of mind. This includes scientific thinking, applying mental models (I like this list here: https://fs.blog/mental-models/), pro-social behavior (listening, conversing). Much of science & math is having an available set of mental models, understanding how/where to apply them, and recognizing when a new one is needed.
My preference would be for traditional subjects to be taught with these firmly in mind: when thinking about biology, for example, what are the rote skills that must be learned? What intuitions should students achieve, and what experiences will enable them? What habits of mind produce an orientation, attitude, or set of thought processes conducive to practicing the science and art of biology?
I've been coding for 50 years. When I write code, I think it is great work. About five years later, I realize it was crap. This is true of all the code I write.
So, about five years later is the right time for refactoring.
P.S. It takes about five years to forget what you thought you were doing with that code, and see the reality of what you wrote.
The use of fossil fuels equates directly to higher standard of living, military power, wealth, prosperity, and advanced economies. As well, transportation is heavily dependent on fossil fuels. "Exiting" fossil fuels means either nothing, or it means impoverishing your people.
The use of energy certainly equates directly to a higher standard of living. Oil seems like an implementation detail, with benefits as well as costs. Why not consider other implementation options?
Not long ago this was linked to coal and cancer. And even higher rates of cancer and lung disease correlated with higher standards of living. Should we start advocating for the return to coal? Maybe transplanting cancers to cause better prosperity?
> "Exiting" fossil fuels means either nothing, or it means impoverishing your people.
Utter bullshit. Exiting fossil fuels means prosperity for the people in the near future (the next generation). Staying on fossil fuels means stagnation and decay.
Don't believe me? I welcome you to visit West Virginia. Or pretty much any former coal-mining region, for that matter. Almost all of them are a depressing sight.
Aren't the former coal-mining regions badly impoverished today because we dramatically cut back our usage of their primary economic product?
I'm not sure that supports your point. I don't think they are stagnating and decaying because they want to keep mining coal specifically, it's just that the coal miners and their next generation don't have any capital to found cool innovative startups, and not enough people with capital have any incentive to go there and make job-creating ventures to employ them.
It absolutely supports my point. Countries and regions that stopped using or depending on coal early are now doing better than regions that are still clinging to it.
Yes, in the past coal was useful, and having access to coal was linked to prosperity. Oil is associated with prosperity now. But the writing is on the wall for oil.
And speaking of the current meeting, I don't know a single example of a country that decided to buck the trend and got rich by selling coal when the world started switching to oil/gas.
This is not intuitive to me. It seems like a "new idea" is something that (almost by definition) isn't in the training set. Can you elaborate a bit?
Edit: but perhaps a good model could arise from training, which would be a good idea in the sense that parsimonious ideas are good scientific ideas.