I don't see why it makes a difference for this purpose that you're replaying network packets or controller inputs or any other interface to the game engine.
The important thing is that there is some well-defined interface. I guess designing for networked multiplayer does probably necessitate that, but if the engine isn't deterministic it still isn't going to work.
There was a twitter thread years ago (which appears to be long gone) about how the SNES Pilot Wings pre-game demo was just a recording of controller inputs. For cartridges manufactured later in the game's life, a plane in the demo crashes rather than landing gracefully, due to a revised version of a chip in the cartridge. The inputs for the demo were never re-recorded, so the behaviour was off.
It does make quite a big difference. The network packets received from the server in Quake will tell you exactly what state the game is in at any point in time. They contain information about the position and state of every entity and their motion, compressed via delta encoding. That means there's very little room for misinterpretation on the client side that would lead to de-sync issues. In fact clients have quite a lot of freedom in how they want to represent said game state, and can for example add animation interpolation to smoothen things out.
The example you mention of demo playback de-syncing when the circumstances slightly change, that is exactly what you get when you only record inputs from the player. Doom actually did this too for its networking model and demo playback system. That relies much more on the engine being deterministic and the runtime environment behaving consistently, because each client that replays those inputs has to run the exact same game simulation, in order for the resulting game states to match.
Look into dead reckoning vs lock step for networking. Lockstep requires determinism at the simulation layer, dead reckoning can be much more tolerant of differences and latency. Quake and most action games tend to be dead reckoning (with more modern ones including time rewind and some other neat tricks).
Very common that replay/demo uses the network stack of it's present in a game.
I used to be a professional sailor, and love finding nautical terminology in programming. At sea dead reckoning is navigating using the speed and direction of the ship, and adding tide and wind to calculate a fix based on the last known position. The term dates back to the 1600s.
It is fun to point at a chart and confidently state “We’re here! I reckon...”
There's a book I read a while back named "Longitude" that maps the storied quest in science to improve upon dead reckoning by devising greater and greater accuracy in time pieces used on ships. Iirc it was a fun read if anyone else finds that sort of thing interesting (as I do.)
It's a great read! A story of how the scientific elite stalled progress because the right answer wasn't the one they hoped it would be, and didn't come from the sort of person they thought it should.
If you get the chance, you can see some of Harrison's chronometers at the Royal Observatory in London, though I don't know if they're always on display.
I'll add a recommendation for Sextant by David Barrie.
An interesting thing about the a lockstep solution which only considers inputs is that any RNG required in the game must be generated from the input history somehow. This could lead to players being able to manipulate their luck with extremely precise inputs.
The other interesting trick is you need a separate RNG for visual only affects such as particles than the one you use for the physics simulation. Depending on the game during replays, you could position the camera differently and then particle effects would render differently depend, depending on what’s on screen. Obviously that shouldn’t affect the way objects decide to break during the physics simulation.
Typical deterministic game engines will do this, send it to every machine as part of the initial game state, and also check the seed across machines on every simulation frame (or periodically) to detect desyncs.
That could lead to other subtle problems elsewhere though, because it requires synchronizing the seed. If you can't do that, it could lead to problems. E.g. when comparing offline speedruns where everyone would have a different seed. Then some players could have more luck than others even with the same inputs, which would be unfair. (Though I can't think of anything else at the moment.)
DonHopkins on Feb 16, 2022 | parent | context | favorite | on: Don't use text pixelation to redact sensitive info...
When I implemented the pixelation censorship effect in The Sims 1, I actually injected some random noise every frame, so it made the pixels shimmer, even when time was paused. That helped make it less obvious that it wasn't actually censoring penises, boobs, vaginas, and assholes, because the Sims were actually more like smooth Barbie dolls or GI-Joes with no actual naughty bits to censor, and the players knowing that would have embarrassed the poor Sims.
The pixelized naughty bits censorship effect was more intended to cover up the humiliating fact that The Sims were not anatomically correct, for the benefit of The Sims own feelings and modesty, by implying that they were "fully functional" and had something to hide, not to prevent actual players from being shocked and offended and having heart attacks by being exposed to racy obscene visuals, because their actual junk that was censored was quite G-rated. (Or rather caste-rated.)
But when we later developed The Sims Online based on the original The Sims 1 code, its use of pseudo random numbers initially caused the parallel simulations that were running in lockstep on the client and headless server to diverge (causing terribly subtle hard-to-track-down bugs), because the headless server wasn't rendering the randomized pixelization effect but the client was, so we had to fix the client to use a separate user interface pseudo random number generator that didn't have any effect on the simulation's deterministic pseudo random number generator.
[4/6] The Sims 1 Beta clip ♦ "Dana takes a shower, Michael seeks relief" ♦ March 1999:
(You can see the shimmering while Michael holds still while taking a dump. This is an early pre-release so he doesn't actually take his pants off, so he's really just sitting down on the toilet and pooping his pants. Thank God that's censored! I think we may have actually shipped with that "bug", since there was no separate texture or mesh for the pants to swap out, and they could only be fully nude or fully clothed, so that bug was too hard to fix, closed as "works as designed", and they just had to crap in their pants.)
The other nasty bug involving pixelization that we did manage to fix before shipping, but that I unfortunately didn't save any video of, involved the maid NPC, who was originally programmed by a really brilliant summer intern, but had a few quirks:
A Sim would need to go potty, and walk into the bathroom, pixelate their body, and sit down on the toilet, then proceed to have a nice leisurely bowel movement in their trousers. In the process, the toilet would suddenly become dirty and clogged, which attracted the maid into the bathroom (this was before "privacy" was implemented).
She would then stroll over to toilet, whip out a plunger from "hammerspace" [1], and thrust it into the toilet between the pooping Sim's legs, and proceed to move it up and down vigorously by its wooden handle. The "Unnecessary Censorship" [2] strongly implied that the maid was performing a manual act of digital sex work. That little bug required quite a lot of SimAntics [3] programming to fix!
if its deterministic lockstep, then all you need to do is record inputs and replay the inputs, since the engine itself is guaranteed to behave the same. If it's client/server and non-deterministic, then you need to record the entire state of the system at every step (which you'll naturally receive from the server) to replay. The main difference would be in how large a replay file would get; and more dynamism naturally implies more information to record. Large unit quantities in e.g. an RTS behaves more sanely with deterministic replay.
the other negative with deterministic input-based replay is what you've said -- if the engine deviates in any manner, the replay becomes invalidated. You'd have to probably ship with every version of the engine, and the replay just runs on the relevant release. Just replaying and re-recording the inputs on the new version wouldn't do anything, because the outcome behavior would inevitably out of sync with the original.
I'm also not sure how one would support scrubbing, except by also having inverse operations defined for every action or by fully-capturing state at various snapshots and replaying forward at like 10x speed.
> I don't see why it makes a difference for this purpose that you're replaying network packets or controller input
Building a simulation that has perfect determinism is incredibly time consuming. Incredibly. Especially one that is identical across platforms, chipsets, and architectures.
Deterministic simulation replay also breaks anytime you change the simulation. Which is kind of obvious. But quite meaningful.
In any case, I’ve shipped games that use both solutions. And let me tell you, deterministic simulation from input is an order of magnitude more effort to build, test, and maintain!
There was a recent ruling that LLM output is inherently public domain (presumably unless it infringes some existing copyright). In which case it's not possible to use them to "reimplement everything we can as copyleft".
it's more complicated, the ruling was that AI can't be an author and the thing in question is (de-facto) public domain because it has no author in context of the "dev" claim it was fully build by AI
but AI assisted code has an author and claiming it's AI assisted even if it is fully AI build is trivial (if you don't make it public that you didn't do anything)
also some countries have laws which treat it like a tool in the sense that the one who used it is the author by default AFIK
Finland too. Fundamentally it is putting food safety over profits. Eggs being salmonella free is based on regular testing and culling infected flocks. It is a process that need constant work.
Have you ever had a chatbot solve your problem? I don't think this has ever happened to me.
As a reasonably technical user capable of using search, the only way this could really happen is if there was no web/app interface for something I wanted to do, but there was a chatbot/AI interface for it.
Perhaps companies will decide to go chatbot-first for these things, and perhaps customers will prefer that. But I doubt it to be honest - do people really want to use a fuzzy-logic CLI instead of a graphical interface? If not, why won't companies just get AI to implement the functionality in their other UIs?
Actually, I have, Amazon has an excellent one. I had a few exchanges with it, and it initiated a refund for me, it was much quicker than a normal customer service call.
Outside of customer service, I'm working on a website that has a huge amount of complexity to it, and would require a much larger interface than normal people would have patience for. So instead, those complex facets are exposed to an LLM as tools it can call, as appropriate based on a discussion with the user, and it can discuss the options with the user to help solve the UI discoverability problem.
I don't know yet if it's a good idea, but it does potentially solve one of the big issues with complex products - they can provide a simple interface to extreme complexity without overwhelming the user with an incredibly complex interface and demanding that they spend the time to learn it. Normally, designers handled this instead by just dumbing down every consumer facing product, and I'd love to see how users respond to this other setup.
I'm happy that LLMs are encouraging people to add discoverable APIs to their products. Do you think you can make the endpoints public, so they can be used for automation without the LLM in the way?
If you need an LLM spin to convince management, maybe you can say something about "bring your own agent" and "openclaw", or something else along those lines?
Yep, I’m developing the direct agent access api in parallel as a first class option, seems like the human ui isn’t going to be so necessary going forward, though a little curation/thought on how to use it is still helpful, rather than an agent having to come up with all the ideas itself. I’ve spun off one of the datasets I’ve pulled as an independent x402 api already, plan to do more of those.
What I mean is that I want to be able to build my own UIs and CLIs against open, published APIs. I don't care about the agent, it's an annoyance. The main use of it is convincing people who want to keep the API proprietary that they should instead open it.
I did think about this use-case as I was typing my first message.
I can see it working for complex products, for functionality I only want to use once in a blue moon. If it's something I'm doing regularly, I'd rather the LLM just tell me which submenu to find it in, or what command to type.
Yeah true, might be a good idea to have the full UI and then just have the agent slowly “drive” it for the user, so they can follow along and learn, for when they want to move faster than dealing with a chatbot. Though I think speech to text improves chatbot use speed significantly.
Amazon's robot did replace the package that vanished. I don't believe it ever understood that I had a delivery photograph showing two packages but found only one on my porch. But I doubt a human would have cared, either--cheap item, nobody's going to worry about how it happened. (Although I would like to know--wind is remotely possible but the front porch has an eddy that brings stuff, it doesn't take stuff.)
Your docs are probably read many more times than they are written. It might be cheaper and quicker to produce them at 90% quality, but surely the important metric is how much time it saves or costs your readers?
In the same way it's better that adults are the recipients of the harms of smoking, drinking or gambling. It's still not desirable, but societies have settled upon thresholds for when people have some capacity to take responsibility for their choices.
Not saying those thresholds are always right and should definitely apply in this case, but it surely isn't an alien or non-obvious concept.
Others have suggested "bullshit". A bullshitter does not care (and may not know) whether what they say is truth or fiction. A bullshitter's goal is just to be listened to and seem convincing.
There was a twitter thread years ago (which appears to be long gone) about how the SNES Pilot Wings pre-game demo was just a recording of controller inputs. For cartridges manufactured later in the game's life, a plane in the demo crashes rather than landing gracefully, due to a revised version of a chip in the cartridge. The inputs for the demo were never re-recorded, so the behaviour was off.
reply