AI Detectors Falter: How Minor LLM Polishing Leads to False Accusations in Arabic Content
By Saleh Almohaimeed, Saad Almohaimeed, Mousa Jari, Khaled A. Alobaid, Fahad Alotaibi
Published on November 24, 2025| Vol. 1, Issue No. 1
Content Source
This is a curated briefing. The original article was published on cs.CL updates on arXiv.org.
Summary
This research highlights a critical flaw in current AI detection models: their inability to accurately classify human-authored articles that have undergone minor polishing by large language models (LLMs), particularly in Arabic. While efforts have been made to address this in English, this paper fills a gap by creating two Arabic datasets. The first dataset helped evaluate 14 LLMs and commercial detectors on their ability to distinguish between human and AI-generated content. The second, Ar-APT, comprises 400 human-authored articles subtly polished by 10 LLMs across four settings, totaling 16,400 samples. When the 8 best-performing models were tested on Ar-APT, results showed a significant drop in accuracy. For instance, Claude-4 Sonnet's accuracy plummeted from 83.51% to 57.63% for LLaMA-3 polished articles, and Originality.AI's impressive 92% accuracy fell drastically to 12% for articles polished by Mistral or Gemma-3, leading to widespread misattribution of human work to AI.
Why It Matters
This finding is profoundly concerning for professionals across the AI industry, education, and content creation. Firstly, it signals a credibility crisis for AI detection technologies. If leading commercial and LLM-based detectors cannot distinguish between original human work and human work slightly enhanced by AI, their claims of accurately identifying AI-generated content become unreliable. This undermines trust, particularly in contexts like academic integrity, journalism, and content authenticity verification, where false accusations of AI plagiarism can have severe professional and reputational consequences.
Secondly, it exposes a critical gap in linguistic and cultural inclusivity within AI development. The specific focus on Arabic, and the observation that this challenge is under-explored compared to English, underscores the risk of developing AI solutions that are biased or ineffective outside of dominant languages. It highlights the urgent need for more diverse datasets and research in non-English contexts to ensure AI tools are equitable and robust globally.
Finally, this research forces a re-evaluation of human-AI collaboration paradigms. As LLMs become indispensable tools for productivity, writers, researchers, and professionals increasingly use them for drafting, editing, and polishing. If even minor AI-driven refinement of human-authored text triggers false positives, it creates an impossible dilemma: either forego efficiency gains from AI or risk being falsely accused of AI plagiarism. This discourages beneficial human-AI symbiosis and pushes users towards an unsustainable "AI-or-nothing" approach. The industry must move beyond simplistic binary detection to develop more nuanced, context-aware systems that can recognize and accommodate the legitimate integration of AI assistance in human creative processes, rather than penalizing it.