Why Is It Important to Detect AI-Generated Text?
The GenAI era has exposed a multitude of problems that come along with the advent of automated content creation. Namely, AI-created texts can be used to:
- Infringe copyright.
- Generate malicious, infected code.
- Orchestrate various types of elaborate scams.
- Spread hate speech and perpetuate stereotypes.
- Automate production of propaganda, fake news, and misinformation.
- Forfeit scientific writing, watering down the quality of the academic publications.
At the same time, detecting such content is still a challenge. Detectors prove to be vulnerable to text modifications — grammatical mistakes made on purpose or through paraphrasing — while also falsely identifying human writing as AI-tailored. According to OpenAI, AI detectors are not reliable and can even erroneously target non-native English speakers.
AI-Generated Text Detectors — How Accurate Are They?
There are a few key issues that undermine the accuracy of AI detection:
- AI vs. Human discrepancy. At some point, the difference between generative and human writing may become imperceptible as Large Language Models keep evolving. GPT-4 is already capable of mimicking human writing in various styles and genres.
- Watermarking. While it’s possible to insert a token inside a text that signals AI usage, it still can be bypassed. This can be done by hacking, using a custom-trained neural model, or by employing a ready-made generator available in the darknet.
- Training. GenAI is extensively trained on human writing — this is why it can be mis-identified as an average human writer. Training datasets with synthesized samples also exist now, but the initial and crucial impact was produced by the “human pen.”
- Modifications. Prompt optimization, recursive paraphrasing, Substitution-based In-Context example Optimization (SICO), and other techniques can make AI text virtually undetectable.
Additionally, it is assumed by Sadasivan, et al. that detecting AI-text is mathematically impossible.
Main Problems of AI-Generated Text Detectors
Generally, several issues have been identified in relation to AI text detection:
- Limited Accuracy
It is reported that, on average, a free AI text detector shows a 26% accuracy. Other tools are prone to giving false positives/negatives. As a study showed, half of all detectors misidentified AI-written text as human.
- AI Sophistication
As mentioned earlier, GenAI can evolve up to the point it cannot be told apart from a human. It can learn to use writing patterns that are intrinsic to famous writers, such as Shakespeare, Poe, Wilde, Twain, and others.
- High Costs and Resource Limitations
Detecting AI writing can also be hampered by lack of resources, such as computational power, money, and other factors. The top-performing detectors charge rather high subscription fees.
- Content Obfuscation (Human+AI Hybrid Content)
As reported, hybrid texts — featuring both human and AI input — will become one of the major issues in the foreseeable future. The human writing will mask generated content, making it hard to expose.
- Lack of Expertise
GenAI content may feature inappropriate, offensive, false, and misleading materials. Text detectors aren’t capable of tracking down such writing, neither can they provide fact-checking or source validating.
- Lack of clear definition of generated text
It is unclear to what extent a text can be labelled as “generated.” A part of it could be produced with GenAI — e.g. for artistic purposes — and the rest could be original. Then it’d be hard to define the piece as “generated”.
- Lack of standards and regulations
Currently, there are no strict standards or regulations — similar to those outlined by ISO — dedicated to generated writing.
Notable Examples of AI-Text Detectors’ Underperformance
Apart from OpenAI discontinuing its Text Classifier, there are more cases that show the ineptitude of text detectors. One incident demonstrated how ZeroGPT attested a generated piece as genuine human writing. Another peculiar case involves Turnitin’s AI-text detector mistakenly labelling a student’s essay as generated, even though it was an original work.
As the future of AI-Generated text unfolds, it's uncertain how this could affect content on the internet, business publications, and even academic writing. To read on about the potential dangers of GenAI text in scientific writing specifically, read our next article here.
Antispoofing
