That's why you pair it with a command line switch:
--color=auto (default, isatty + termcap)
--color=always
--color=never
Now if we could only agree on a standard to do this using environment variables... Unfortunately, the most widely known "standard" [0], is still very rarely supported.
That's a pretty niche case, you can force it by either allocating a pseudo tty or using LD_PRELOAD with an isatty() shim that returns true.
I only ever had to do it once (using LD_PRELOAD) because there was an insane vendor binary blob that would return "SUCCESS" in green/red depending on if it passed or failed. When isatty() returned false, it was impossible to tell if it passed or failed, but the software was pure garbage.
I don't entirely agree. When I do want colour (generally because I've taken the time to configure the program suitably for it), then I usually still want the colour when I pipe its output through a pager. `diff` is the canonical example.
Edit: Another comment¹ points out pipetty² as a workaround, so I learned something useful today.