Dependent types is one possible direction. Not sure when a language with dependent types will arise which will be useful for making real programs.
Agda is the most mature dependently typed programming languae (having been around since the 90s – it is basically Haskell on steroids), but has a more proof-assistant flavor than an actual programming language flavor. Opus & Fable write Agda quite well, so LLMs can understand dependent types.
I recently came across the existence of Penrose string diagram after curiously and naively researching the existence of index free notation for linear algebra. This seems to be a very interesting paper in the very same category of things I'd love to study and learn about but probably won't ever find the time!
You will possibly like Needham's "Visual Differential Geometry and Forms" and even if you don't have much time to study it, it's quite beautiful to look at from time to time.
Is a paper that publishes a 0.01% improvement of something at the cost of 5 times more power really an improvement? I believe that every single computer science measurement metric should have Joules or Watts in the denominator. If you are training a model I want to see performance per total energy consumed. If you are measuring inference accuracy, measure PER WATT.
I've always been a bit confused by the apparent tendency of the computer science field to mostly ignore energy and power. We are too often satisfied with the idea that software and programs exist in a perfect whiteboard world of xkcd 505 abstract compute.
open source ai labs care a lot about inference speed. that translates to energy and e waste (gpus that work for less time take longer to wear out). training power is another thing and thats where we see a lot of duplicate work we could fix by making it mandatory to release weights for all models above some total power limit.
if you want to look at the real waste of power just open up some electron app. no good reason why we still use it for new apps in 2026 when gpui and avalonia and tauri are all options
GPUI is still very hard to build things upon the last time (from my limited experimentation) I checked but I wish the team @Zed luck for the GPUI project as I am definitely fascinated by it and its certainly an interesting project for sure!
I work on differentiable geometric optics with PyTorch. Seeing a list like this is really illustrative of the power that PyTorch provides when you start considering it like a general purpose GPU-enabled state of the art numerical optimization framework.
One thing I wonder is why no one has made a fork of PyTorch yet that removes all the API surface that doesn't produce GPU friendly code. Make dtype and device arg mandatory without defaults, remove in place operations that trigger a CPU sync, etc. This would increase confidence that written code will run on the GPU and pass torch.export() on the first try.
It's useful to be able to run models/code on the CPU or split between CPU and GPU, especially for models that cannot fit into the GPU's VRAM, if you are running multiple models, or if you have training data that you need to move between CPU and GPU.
Seems like you could write a simple source code checker program to check all of that. Making an extra library just for some (user hostile) tweaks seems like overkill.
Thanks! Arbitrary precision arithmetic is definitely something I'd like to learn more about, yeah. Haven't had time to study it so much yet unfortunately.
I don't handle it, ahah. You are right that if you take any classical numerical computing algorithm and replace the floating point reals by interval unions, most of the time the number of intervals in the unions in each of your variables will grow very fast. This is one of the problems of unions and as far as I'm aware it's a topic of active academic research.
Yeah it's super interesting. Like you said, I learned that the IEEE 754 spec actually requires that complete implementations of floating point numbers expose a way to programmatically choose the rounding mode. As far as I know only C allows you to do that, and even then it depends on hardware support. For JS I had to use ugly typedarray casts. Which kinda only accidentally work due to endianess. But technically there should be an API for it!
There's other unused stuff in IEEE 754 like that: the inexact bit or signaling NaNs!
It's possible to support that but it makes the code very very much more complicated. I've decided early on to not support it. Would be a cool addition though!
reply