? what if you need two different versions of cuda toolkit on same machine?

I may be wrong, but I don’t think you would want that - it should match with your driver version. Weird stuff happens with minor version mismatches

Probably true, but it usually works anyway. And a lot of ML code depends on exact versions of the CUDA libraries, so you don't get a choice.