Gives you the basics on LLM internals in about 90 minutes and includes an already built model in JavaScript that you can step through in browser devtools to get as detailed as you want.
The future of programming and IT tech - If you need a database, you just say to the model "you're a database". Or "you're a CRM", "you're a Doom game", "you're a Word processor", etc.
It was "network is the computer", now it will be "model is the computer", and the model will be like one large ("multi tenant" - it will know on its own how/when to separate tenants' data and when to analyze it all together) model living on tens/hundreds of millions of nodes in AWS ... the AWS itself will be just that model.
And when the datacenter staff show up for work every morning the AI will them “You are employee #5378. Today you are a janitor. You will…” going off on a long list of hyper precise instructions for them to follow, like a human prompt.
TFA is missing a host of many a popular isolation techniques like Isolates, Code Interp / Binary Translators [0], Enclaves, Exclaves, Domains/Worlds, (RISC V) SEEs, TEEs, SEs, HSMs, pKVMs ...
I'm not a fan of the database lookup analogy either.
The analogy I prefer when teaching attention is celestial mechanics. Tokens are like planets in (latent) space. The attention mechanism is like a kind of "gravity" where each token is influencing each other, pushing and pulling each other around in (latent) space to refine their meaning. But instead of "distance" and "mass", this gravity is proportional to semantic inter-relatedness and instead of physical space this is occurring in a latent space.
Basically a boid simulation where a swarm of birds can collectively solve MNIST. The goal is not some new SOTA architecture, it is to find the right trade-off where the system already exhibits complex emergent behavior while the swarming rules are still simple.
It is currently abandoned due to a serious lack of free time (*), but I would consider collaborating with anyone willing to put in some effort.
I don’t recall if there was ever a difference between “abort” and “fail.” I could choose to abort the operation, or tell it … to fail? That this is a failure?
Makes sense. Maybe I ran across a proper use a time or two back then and just don’t remember. But the two being the same was the overwhelming experience.
This was a final project for a graphics class where we used WebGL a lot. Also I was just more familiar with OpenGL and haven't looked that much into webGPU
Since its in Cyrillic you should perhaps use a translation service. There are some screens showing results, though as I was really on a tight deadline, and its not a PHD but masters thesis, I decided to not go into in-depth evaluation of the proposed methodology against SPIDER (https://yale-lily.github.io/spider). Even though you can find the simplifed GBNF grammar, also some of the outputs. The grammar, interestingly it benefits/exploits a bug in llama.cpp which allows some sort of recursively-chained rules. Bibliography is in English, but really - there is so much written on the topic, by no means comprehensive.
Sadly no open inference engine (at time of writing) was both good enough in beam search, and grammars, so this whole things needs to perhaps be redone in pytorch.
If I find myself in a position to do this for commercial goals, I'd also explore the possibility of having human-catered SQLs against the particular schema, in order to guide the model better. And then do RAG on the DB for more context. Note: I'm already doing E/R model reduction to the minimal connected graph which includes all entities of particular interest to the present query.
And finally, since you got that far - the real real problem with restricting LLM output with grammars is the tokenization. Because all parsers work reading one char at a time, and tokens are very often few chars, so the parser in a way needs to be able to "lookahead", which it normally does not. I believe OpenAI wrote they realized this also, but I can't really find the article atm.
Gives you the basics on LLM internals in about 90 minutes and includes an already built model in JavaScript that you can step through in browser devtools to get as detailed as you want.