At this point, hardware crypto engines are a "must have" feature for many purchasers. Their desire for performance outweighs concerns about malicious actors backdooring the crypto engine.

And if the crypto engine is compromised, how little/great of a leap is it to believe there is microcode to backdoor a general OS or crypto library?

These aren't unusual features in SoCs. Of course if you're concerned about backdoors you don't have to invoke these engines, at least with a software implementation you can control the implementation.

Forgive me if I'm being dense but what harm would embedded crypto and compression engines do? You don't have to use them right?

There's little room for tampering with crypto, the only room is for tampering with the RNGs.

As far as I know all crypto algorithms are deterministic based on the key/IV and the data.

While that's true and required to successfully decrypt most algorithms, it is also true that there are more types of tampering one can do than changing the output ciphertext. Usually involving storing the key or leaking data somehow.

Assuming they've already compromised the crypto bits of the chip there's nothing to gain in avoiding them since the non-crypto bits could just as well have the same compromises. Might as well just take the time/energy savings.

Tampering with the RNG probably provides the best value for an attacker, and is harder to detect.

That is a pretty great thing, as far as I am concerned. Custom ASICs for routine server tasks that would otherwise clog up a general purpose core? I'll take it.