Is there something like a sharpe ratio (which essentially figures out how much return was generated vs risk taken compared to a "risk free" benchmark) for AI?
Essentially, the AI's performance under test is the return rate, the risk free rate is some arbitrary benchmark (a human for example), and the risk taken is the "bandwidth" of the algorithm -- bandwidth being the AI's ability to adapt to changing conditions.
An AI that composes excellent poetry, for example, might have a high rate of return (it composes excellent poetry) but a low bandwidth (the English language does not change rapidly) while an AI chatbot might have a low rate of return (it is sometimes incoherent) but a high bandwidth (it's very adaptable to different patterns of speech).
Such a ratio would allow you to compare the performance of different approaches (ML& variants, rule based, etc) in disparate fields.
Does anything like this exist?