This seems like a sort of unwinnable arms race. Can't the people who work on generative text models use this classifier as a feedback mechanism so that their output doesn't flag it? I'm not an AI expert, but I believe this is even the core mechanism behind Generative Adversarial Networks.
Detectors can be a black box "pay $5 per detection" type service.
That way, you can't fire thousands of texts at it to retrain your generative net.
Plagiarism detectors in schools and universities work the same. In fact, some plagiarism detection companies now offer the same software to students to allow them to pay some money to pre-scan their classwork to see if it will be detected...
$5 is way too high of a price to use regularly. In any case, if it's only available to education institutions, teachers and grad students are poor enough to sell access to it to people on the dark web for the right price.
Make a model to detect cheating. Market it as "a custom built and unique model to detect cheating; able to catch cheating that other models miss!" It's all 100% true. Market and profit.
Language Models produce a high probability sequence of words given history (or an approximation of it). This is the only paradigm that we know works for language synthesis.
What the creators of this page did is turn that into its head, and use exactly that reasoning to identify candidate passages as computer generated, exactly because they have access to those probabilities, so it's not a viable approach to improving the language model directly.
With ChatGPT however, we have 2 models working , a language model, and a ranking model. The ranking model is trained to order the results of the language model to look better to humans. The suggested approach could be used to help fit the model by ranking lower probability sequences higher, but this comes at the cost of increased computation time by generating many more sequences, and constructing incoherent output.
> This is the only paradigm that we know works for language synthesis.
No, it's the easiest paradigm we know works for language synthesis. The other way to synthesize language is to understand what you're saying. This is "old-school" AI (we wouldn't even call it AI now), done with if statements, expert systems, and queries of a robust, structured data model. The bullshitting capabilities of neural networks have skyrocketed so far as to dwarf the "expert system" approach, but it's still there, slowly getting better, and still the right choice for many situations.
What I'm excited about is combining the capabilities of both. Right now there's a huge gap between the two.
You're right, that's the core mechanims of GANs. The current state of the art models aren't using a GAN structure, but it's plausible that they achieve state of the art numbers in the future
