/code-review in Claude Code spawns a lot of sub-agents (counted like 8 once), each looking at the code from some certain aspect (like correctness, maintainability, duplication, testing, etc). It eats tokens like crazy doing that, but also covers quite a lot. The default code review in Codex does far less (feels like it's only correctness) and doesn't uses subagents. Actually I made a skill for Codex that does a review closer to what Claude does by default, but using like 4-5 agents and some being cheaper models/less than xhigh reasoning. I'm getting pretty nice reviews with that that cover more than just correctness.
MSVC (Microsoft's C++ compiler) had an pretty advanced inter-procedural (LTO) way of doing devirtualization, but it was so buggy and slow that it eventually got disabled. It was trying to prove that a pointer can only target a certain class all the time (or maybe a couple), but things get really messy with typical C/C++ code and even worse once you have DLLs which may inject new derived classes.
Sadly for them, Nvidia didn't stay still in the meantime and created the next generation of CUDA, CuTile for Python and soon for C++, through CUDA Tile IR (using a similar compiler stack based on MLIR).
Event though it's not portable, it will likely have far greater usage than Mojo just by being heavely promoted by Nvidia, integrated in dev tools and working alongside existing CUDA code.
Tile IR was more likely a response to the threat of Triton rather than Mojo, at least from the pov of how easy is to write a decently performing LLM kernel.
And for not staying behind, Intel and AMD are doing similar efforts, and then we have the whole CPython JIT finally happening after so many attempts.
Not to mention efforts like GraalPy and PyPy.
And all these efforts work today in Windows, which is quite relevant in companies where that is the assigned device to most employees, even if the servers run Linux distros.
I keep wondering if this isn't going to be another Swift for Tensorflow kind of outcome.
People keep mistaking Mojo as good syntax for writing GPU code, and so imagine Nvidia's Python frameworks already do that. But... would CuTile work on AMD GPUs and Apple Silicon? Whatever Nvidia does will still have vendor lock-in.
Indeed, but Intel and AMD are also upping their Python JIT game, and in the end Mojo code isn't portable anyway.
You always need to touch the hardware/platform APIs at some level, because even if the same code executes the same, the observed performance, or in the case of GPUs the numeric accuracy has visible side effects.
It is portable in that you can write code to target multiple platforms in the same codebase. Mojo has powerful compile-time metaprogramming that allows you to tell the compiler how to specialise using a compile-time conditional, e.g. https://github.com/modular/modular/blob/9b9fc007378f16148cfa...
Of course, this won't be necessary in most cases if you're building on top of abstractions provided by Modular.
You don't get this choice using vendor-specific libraries; you're locked into this or that.
Yes you do, you get PyTorch or whatever else, built on top of those vendor-specific libraries.
That is the thing with Mojo, when it arrives as 1.0, the LLM progress and the investment that is being done in GPU JITs for Python, make it largely irrelevant for large scale adoption.
Sure some customers might stay around, and keep Modular going, the gold question is how many.
Have you ever wondered how much work would have been saved by the Pytorch team if they could have used just Cuda for all the platforms they support? If they didn't have to write compatibility abstractions or layers, and instead just focused on the problem of training neural networks? What if all the primitives they used from Cuda and cuDNN worked just as well on AMD GPUs, Apple GPUs, and probably Google's TPUs as they did on Nvidia GPUs?
Mojo and Modular's Max platform would do to heterogeneous compute what LLVM did to programming language development. People who dismiss the real value offering here know nothing. Modular have already raised $350m+ from industry giants (including Nvidia and Google) to solve this, and I believe they will.
Yes, because one of my hobbies was graphics programming for a long time, and I keep observing all the time how FOSS folks misunderstand the games industry, what gets talked at GDC and IGDA events, isn't one API to rule them all.
> What if all the primitives they used from Cuda and cuDNN worked just as well on AMD GPUs, Apple GPUs, and probably Google's TPUs as they did on Nvidia GPUs?
Why should they? CUDA is a GPGPU paradigm, AMD/Apple/Intel all ship diverse raster-focused hardware, and TPUs are a systolic array. How much can you realistically expect to abstract with unified primitives? How much performance do you perceive to be left on the table with native CUDA-based implimentations?
Pytorch's abstractions answer this by ignoring raster hardware conventions entirely. The underlying ATen library is basically a CUDA wrapper, which is not much of a surprise since nobody else is willing to standardize a better alternative. We learned as much when OpenCL died, and now that Khronos is riding into the sunset it's unlikely we'll even see that level of paltry early-2010s cooperation. Mojo really should have taken Vulkan's lessons to heart; you need stakeholders to succeed, simply "disrupting" the proprietary status quo is a recipe for coming dead last in adoption rates.
> People who dismiss the real value offering here know nothing.
So explain the value, then. This is not an "optimize this IR for MIPS and x86" problem, the Lattner Fairy can't shoehorn shaders into every CUDA Compute Capability to make raster GPUs a viable GPGPU platform. If you followed Geohot's gradual descent into (sadly, quite literal) insanity then this would have been glaringly obvious from the offset. Tinygrad has an IR, industry-scale support, multiplat deployment, and it's still a dumpster fire. The project exacerbated all of the issues in ROCm and Metal, without contributing to any form of upstream cooperation between CUDA's competitors. If Mojo goes the same route with a more ambitious goal, they'll end up entrenching CUDA and obsoleting themselves. As much as people hate to admit it, CUDA is less of a software moat and more of a hardware one.
> Why should they? CUDA is a GPGPU paradigm, AMD/Apple/Intel all ship diverse raster-focused hardware, and TPUs are a systolic array. How much can you realistically expect to abstract with unified primitives?
Ah, it seems impossible to you. These are very different hardwares... It is hard enough to make compatibility among different hardwares of the same vendor. Very difficult to imagine building primitives for hardwares with completely different memory layouts.
> How much performance do you perceive to be left on the table with native CUDA-based implimentations?
Zero is the idea. And I wasn't saying there should be a native cuda-based implementation, I'm asking you to imagine how much easier everything would have been if Cuda was cross-platform without any performance or ergonomic penalties.
Mojo is a foundational step here. The big HOW is powerful parametric programming. So much information could be passed during compile time which the compiler uses to specialize.
> Ah, it seems impossible to you. These are very different hardwares...
In effect, they are completely different hardware. The only thing any of them have in common is rasterization primitives, so unless you're focused on render workloads you're nearly better off software-accelerating the language for CPUs instead. As a point of comparison, go look at early ray tracing implementations on GPUs that have no dedicated RT blocks or hardware-accelerated denoising. They were oftentimes slower than software-accelerating the same thing on a cheaper CPU.
> I'm asking you to imagine how much easier everything would have been if Cuda was cross-platform
...you are aware of what CUDA actually does, right? Mojo is not a cross-platform version of it. At risk of repeating myself, creating an OpenCL-style library without a Khronos-style consortium does not address the problem. CUDA is a hardware solution, you need to define standards around GPGPU programming because that's what Nvidia does internally. Ignoring industry-wide standardization is the express bet that Nvidia and Google are making, investing in Mojo. Without any hardware stakeholders, Mojo's only opportunity is to become a DirectML/ONNX-style middleware like @pjmlp suggested. Spoiler: that's not a super disruptive or successful goal. Certainly not an LLVM scale opportunity.
> So much information could be passed during compile time which the compiler uses to specialize.
Like what? None of this is a rhetorical question, explain to me what information would help bring GPUs and TPUs up to parity with CUDA. What information are you thinking of that isn't currently expressed in SPIR-V and MLIR? What optimizations do you have in mind?
> In effect, they are completely different hardware. The only thing any of them have in common is rasterization primitives
They can also do matrix multiplication, which is the relevant thing here.
Mojo is not an OpenCL-style library; it is a programming language, and it doesn't really abstract much. However, it is designed to support compilation to exotic hardware. Cuda CPP can also be thought of as a language; however, the compilation model and core primitives are fundamentally tied to Nvidia hardware.
> Like what? None of this is a rhetorical question, explain to me what information would help bring GPUs and TPUs up to parity with CUDA.
My understanding from speaking with a few Tile IR devs on dates is that its primary motivation was providing better portability for programming tensor cores than PTX offers. Nobody ever told me they saw it as a response to anything other than customer feedback.
Maybe OP was thinking about compilers "cracking" certain SPEC benchmarks: implementing exactly the optimization needed to boost a benchmark quite a lot, but that opt. probably won't apply to any other code out there (usually it's so targeted and risky with general C/C++ code that intentionally it doesn't work on anything else). That happened a couple of times over the years, I know about the Intel compiler cases for ex. I can certainly see LLM providers adding tricks that help a certain class of benchmarks, but doesn't help much for anything else.
They won't, Windows isn't even compiled with AVX2 extensions on, even though every CPU in the last 10y+ supports it, for the fear of it running on some machine that doesn't have AVX2. The whole "CPU unsupported" thing is from the marketing side, trying to push purchase of new hardware, not from the actual devs.
I'm not 100% sure, but I think the main reason the new Win11 apps like Notepad, File Explorer, Task Manager have a slow UI and the piece-by-piece drawing issues is because they combine UWP (the "new" tech) with the old Win32 controls, they are not a from-scratch rewrite. There seems to be some big overhead in using the two different UI frameworks together.
There was an attempt to make a Chrome OS competitor that had the entire UI rewritten in UWP, and when that got canceled it seems MS saved the new start menu + taskbar and bolted it on top of Win10, and for Explorer and other apps made this UWP+Win32 abomination. Actually when you profile Explorer you can also see DirectUI running, which is I believe another UI framework from the Office org. this time (maybe it's related to OneDrive).
Btw, apps written in C# using WPF can be surprisingly fast, so it's not really a problem of .NET managed vs. native apps. For ex. may favorite Git GUI (git-fork.com) is C#/WPF.
Thanks for the reply! I'm in WA, the moonlighting IP law seems very similar to CA. Both my contract and current HR guidance state that any invention is mine if done outside work hours, not using their hardware AND doesn't compete with the company and harms it's business interests - the vague part, especially with a company that has it's fingers in dozens of different software.
The app itself is similar to a module of a much larger product the company sells, but that product is far from being a money-maker for the company (may just break even). People are certainly not buying the whole product because of that module, it's a nice thing to have. There are several other companies that do sell stand-alone apps similar to mine though, there is a market for it.
The other concern was whether starting now the work based on the MIT open-sourced version of my app creates a clean slate for me, since the previous work was done in part during work hours on their PCs. Open-sourcing the app through my employer was the only way not to lose it completely in case I switch companies, and have a chance to try a startup based on it.
Some advice I've got was that forking the OSS version should be done by a friend (not in the company) and I should work on it in the background and officially join later, after I quit - that also feels sketchy.
How is performance compared to Graphviz?
There are some large compiler Control flow graphs with 500+ nodes where Graphviz really starts to take a long time to build a layout (dozens of seconds or worse). Profiling dot a bit, saw that most time was spent doing some DFS, and a lot of the data structs seem to be linked lists, the opposite of cache friendly.
reply