I love your gradual approach pretty much. It sounds like gradual typing but not just the typing part.
I used to make many tools with libclang Python bindings to automate some chores of refactoring. I don't remember if one could expand the macros using it, and I was told that lexer and preprocessor are messed together in Clang. So it should be quite hard to extend the existing framework.
I would definitely go into this direction, but for now it just looks like some final boss, let me finish some early quests.
> Build embedded or core OS code with it. things that do MMIO, DMA interactions, low level IO in kernel code or firmware (more embedded).
I have a friend currently writing a GC in C and I was making it a project to test around my ideas (where problems with generic containers, compile-time eval and static reflection kicked in so early). But these you mentioned sound more important to me, I have more friends working on RISC-V projects, I should find some inspirations from their projects. Thanks for the mention!
> Do you have an opinion on the dotnet version of generics?
I'm not familiar with dotnet languages, but I have much experience implementing generics, especially how to avoid getting the poor C++ templates. Also, I believe traits (not that of C++) and typeclasses (Rust/Haskell) are definitely needed, and it's already proven too, to pair with the generics feature.
> Ceph.
My former job was to write a distributed object storage in Go and it was heavily inspired by Ceph. But subset of the design could make much benefits for a smaller company, since the initial idea to write a new one was from the pain of doing rebalancing in Ceph... oh you want to know the solution? Manual partitioning and migration by our SRE/Ops team lol. It's unexpectedly effective.
> On the other hand I think we are in a local maxima with programming languages and type systems.
I think I gotcha. Oh no.
> But I haven't finished fleshing it out and proving the type system, so I really can't share it yet.
Any profile I could follow to wait for this to happen some day? I've made several pen-and-paper attempts on some problems you just mentioned, and further ones would be "strict aliasing", "pointer provenance" and many which appeared in all the dicussions here. It feels like you've already done many stuff, but I can't find your profile anywhere.
I found it might be possible to tackle "strict aliasing" and "pointer provenance" with a type system and I would head down to it early. The approach might sound like Rust's `MaybeUninit` but I didn't think much about it.
I've already implemented procedural metaprogramming in a JS dialect of mine [1], it's also trivial to use it to implement compile-time format strings. I would improve the whole experience in this new C-like language.
Again, very very practical ideas here. Great thanks!
> But the only thing that really took off was effort to change things at the very base level rather than patch issues.
Exactly, that's the most important takeaway I got from all the discussions here: I will be patching issues while people are flooding in for better general approaches.
I might shift much of my direction, but lucky that there are many "lessons" (like ooc) to learn.
I got stuck at how to tackle "new language doesn't compile with a standard C compiler" for many times, but my solution is much worse than yours: Like LuaJIT, they left one unreadable "minilua" C file [1] to bootstrap some stuff, we could have a source-code version of the "new C" compiler, compile things twice. That sounds bad.
For languages with a very advanced type system that compiles to C, I could only think of Koka [2], which translates the "algebraic effect and handlers" code into pure C, achieving pure C generators, coroutines and async/await without the support of setjmp/setcontext. But the generated C code is unreadable, I would definitely think about how to handle the readability and debugging issue with sidecar files.
Oh I heard about it and oops just hate myself for forgetting it.
And the first sentence [1] of its pointer type introduction, exactly says everything I said here...
> [...] greatest motivation behind ATS is to make it employed to construct safe and reliable programs running in OS kernel.
Also found this interesting Reddit thread [2]. Time to bring some old gems back with good ergonomics now, it won't be that hard. Hold my beer for a while.
I also learned that in LLVM IR, they have the implicit null checks [1] to replace the guard with just a signal handler, so it won't hurt the branch predictor too much.
So I believe there are many options upon "debug" and "release" profiles for the bound and null checking here. Very good design space.
[1]: https://people.eecs.berkeley.edu/~necula/Papers/deputy-esop0...