There are some pretty bad usability problems with most async APIs too.
One is they make your functions colored; async functions world best with other async functions while normal blocking functions work best with other blocking functions.
They also introduce a lot of noise; putting async/await everywhere doesn't tell you anything interesting.
Considering a normal-sized Linux server can handle a million threads without much trouble, it really seems like misplaced effort.
In the Java world, project Loom[1] is hopefully going to end this situation of async code that is hard to use with blocking code. They introduce a concept called Virtual Threads (previously called Fibers, but they are still looking for the perfect name). This will allow for seamless interoperability between blocking and non-blocking code as everything in Java runs on a Thread and Virtual Threads are just a specialization of the concept that doesn't boil down to OS threads.
I haven't used it yet, so I can only repeat the advertising copy, but nevertheless wanted to give some perspective from other ecosystems.
Go “solved the problem” at the expense of being unable to interface with C libraries without a big performance penalty (and that's you'll have Rust in Chromium and not Go)
This is definitely a good thing. All computations should me "marked" as total or effectful with various possible effects (blocking, async, nondeterministic, possibly non-terminating etc etc).
Reasoning about your program is hard when each computation is a blackbox possibly containing any side-effects which could cause unpredictable changes in the control flow and result in a completely incomprehensible way.
Async/await thing is indeed least sound and ergonomic way of doing this. Monads with monad transformers are a bit better. Algebraic effects are the best in terms of composability, ergonomics and mental overhead, but not here yet (though OCaml may be soon become the first industrial-grade language incorporating them [1])
> This is definitely a good thing. All computations should me "marked" as total or effectful with various possible effects (blocking, async, nondeterministic, possibly non-terminating etc etc).
Marking functions for side-effects would be a good thing but it isn't what function coloring means in this context.
An async function and a normal function are semantically the same, they just have different syntaxes and you can't easily call one from another.
They can be both be either blocking or non-blocking, especially if they take other functions as arguments.
They're not really the same: you know that an async function may potentially suspend and have other code run before its completion, while a normal function (in the absence of threads and signals) is guaranteed to run atomically, at least as far as your process's memory space is concerned. You also know that the only points at which an async function may suspend are an 'await' statement, and so data invariants that don't cross await statements or other async function calls can be reasoned about as if you had purely sequential code.
That's precisely the coloring that makes async useful. Without it you need to explicitly protect all shared data with mutexes or other synchronization primitives. 40+ years of threaded programming has shown that programmers cannot generally be trusted to get this right, and this in an area ripe with bugs.
Rust won't magically make your threaded code blocks right. It can just provide you tools to ensure that memory won't leak. It's just 1/10 of the solution.
Rust solves threading-specific problems. Yes it's partial, but other problems happen in both threaded code and async code, so that's not a reason to choose one over another.
> An async function and a normal function are semantically the same
But they are not, AsyncIO and BlockingIO are different side effects, thus you have different types of computation, that's exactly what I'm talking about. In languages with monads or algebraic effects these would have different types.
You don't say that Lists and Arrays are semantically the same, despite being similar sequential collections, they still have separate types for a reason. Though it's good to be able to abstract over them.
And in languages with monads we can parametrize over various effect types by using tagless final approach, which allows us to write computations which could be interpreted in contexts of various effects (in this case, Async and Sync), just as we parametrize containers with types of content (in [1] there is an example of how we can parametrize computation over various async implementations), but still these are different effects.
I don't think considering the blocking strategy an effect is very useful. Even in async context in complex enough applications functions can call other functions including your own, so async is not enough to guarantee reentrancy.
I do agree that parametrizing over the blocking strategy is a great idea, but languages that simply provide an async syntactic marker don't necessarily allow that, and if your language is powerful enough you do not need the annotation in the first place.
> One is they make your functions colored; async functions world best with other async functions while normal blocking functions work best with other blocking functions.
I'm skimming your link trying to understand the design. It sounds like there is a global flag for whether the whole program is in evented or blocking mode?
> during compile-time, it’s possible to inspect if the overall program is in evented mode or not, and properly designed code might decide to move to a threaded model when in blocking mode, for example.
Sounds like a horribly complex special case. I'd far rather just have higher-kinded types and be able to write sometimes-async code using normal polymorphism (like I do in Scala all the time).
Yep, I was mind blown when I started using futures years ago with this coloring. But then I realized it’s all about types and monoids and applicatives and it started to get clear why I just can’t get the value out of a promise.
So that would mean a lot of memory for 1 million threads, 2TB of RAM. But you can change the default. With a 64k stack you'd use up ~68GB of RAM, which doesn't seem like a lot for 1 million threads and 1 million requests happening at the same time.
Also worth noting that the entire stack isn't allocated at once so 1 million threads would be using 2TB/68GB of virtual address space, not 2TB/68GB of physical memory.
That is indeed a very important fact to keep in mind! Thread stack sizes have been a problem with 32 bit systems where you quickly run out of virtual memory because the adress space is not large enough. With 64 bit that is not a problem anymore.
That is also the maximum stack size which shouldn't normally be reached. You'll have to be careful how you use memory when you're handling a million clients, either as async or threaded.
The point is that each stack need to be big enough for the worst case. That means it does not really scales to start many thousands of threads. While the futures themselves used in async code can be kept relatively small, as they only need to contain the state needed while awaiting.
In theory a smart enough OS (or runtime) should be able to reclaim any memory beyond the stack pointer (plus redzone) at any time without preserving its content and shrink back the stack. Because of signal handlers that memory is to be considered volatile anyway.
It might not be worth doing it in practice, but it is something to keep on mind.
Most servers support at least 128GB; it isn't even very expensive. And if you want to handle a million concurrent users you also need to consider CPU and latency, so for most real-world workloads the memory probably won't even be your bottleneck.
How much does a 68GB cost on the cloud per day? Also, you don't have 1 million cores so quite a bit of your daily server costs will be eaten up by the OS running context switching code.
The difference, at least in the way this is built in Rust, is that when you create a task, you get a single allocation that's exactly sized. There's no resizing, which means that you aren't getting stacks that are too big or too small, with all of the other runtime shenanigans that that entails.
That said, the future has the size of the biggest state that need to be kept across await. The future might be slightly oversized, but still order of magnitude smaller than a perfectly sized stack.
I don't get what's so bad about "colored" function. That color is just about the return type of the function.
How do you return an error from a function that does not return a Result? You must call unwrap (panic) or change the colour of your function by changing the return type to Result and fix all the caller.
Similarly, if you want to use a future from a non-async function, you either call `block_on(...)`, or you change the return type to a future by marking the function async.
I don't think it is that bad. That's just the way explicitly typed programming languages work.
One is they make your functions colored; async functions world best with other async functions while normal blocking functions work best with other blocking functions.
They also introduce a lot of noise; putting async/await everywhere doesn't tell you anything interesting.
Considering a normal-sized Linux server can handle a million threads without much trouble, it really seems like misplaced effort.