I agree with your sentiment: to analyze these amounts of documents you need computers. And I understand the importance of parallelism, GPU computing can be way faster than CPUs for certain tasks.
In a sense, and correct me if I'm wrong, I get the feeling that the whole ML is just starting now because we have faster computers. To analyze such amounts of text and make some sense of it, you'd need a massive "database" or "fitted model" to decipher what the legalese means. Only recently has this been possible to create (thanks to Moore's law). It's not like there is a breakthrough in mathematics (statistics) right? The theory probably has been around for ages but is only now becoming practical.
> It's not like there is a breakthrough in mathematics (statistics) right? The theory probably has been around for ages but is only now becoming practical.
That's absolutely correct. But to pooh pooh it on those grounds is like pooh-poohing the fourier transform around the 70s (which theory had been around for 200 years) and only had then "become practical" with the advent of transistor-based computer. Now the FT and relatives are used in everything from audio processing and compression, image filters, statistical analysis, pretty much every form of scientific spectroscopy, MRIs, etc. If you had generally made investments in "FT-related technologies" you'd be doing well.
Only if you consider techniques like Word2Vec[1] as a non-breakthough.
No one knew you could do it, but yes, it build upon previous work. I was working in the field before and after and if anyone had asked me if one could represent all human languages in only 300 dimensions, and have vector composition be meaningful I'd have laughed at them.
Take using back-propagation to train deep neural networks. People had shown it worked in 1 or 2 layer networks, but despite years of work no one had been able to train anything deep enough to be useful. Then Krizhevsky, Sutskever and Hinton won ImageNet[2], proved it was possible and kicked off this whole ML craziness.
Neither of them is exactly because of more powerful computers, nor magical math breakthroughs. It was more lots of hard work by researchers trying many combinations of techniques until something worked.
These techniques, combined with huge volumes of data and - yes - more powerful computers are what have made the difference.
In a sense, and correct me if I'm wrong, I get the feeling that the whole ML is just starting now because we have faster computers. To analyze such amounts of text and make some sense of it, you'd need a massive "database" or "fitted model" to decipher what the legalese means. Only recently has this been possible to create (thanks to Moore's law). It's not like there is a breakthrough in mathematics (statistics) right? The theory probably has been around for ages but is only now becoming practical.