Traditional LLMs would be better suited (ironically) for fact checking, eg they check for citations, then go to follow the links and see if it matches the text. They’re also much better at “checking” for sanity than actually writing it out. An obviously this would just be a first pass for a person to quickly confirm.
Would be funny if the Wikimedia Foundation developed an AI that is trained on all the major AI models, and uses that to detect AI slop.
They are constantly changing, but one could probably get pretty far focusing on ChatGPT (which is what most “lazy” authors use).
And there are already efforts in this domain from the community, see the “slop” profiles in EQ bench:
https://eqbench.com/creative_writing.html
Traditional LLMs would be better suited (ironically) for fact checking, eg they check for citations, then go to follow the links and see if it matches the text. They’re also much better at “checking” for sanity than actually writing it out. An obviously this would just be a first pass for a person to quickly confirm.