"Majority of individuals are susceptible to being deceived by AI-generated visuals. Believe you possess superior discernment?"
In a recent study conducted by the Microsoft AI for Good Lab, it was found that humans struggle to identify images generated by artificial intelligence (AI), with success rates hovering around 58% to 62%. This is only slightly better than random guessing, as a 50% accuracy rate would imply.
The study, which utilised a "Real or Not" quiz created by Microsoft in August 2024, did not find a significant difference in success rates between identifying human faces and landscapes. Participants achieved around 65% accuracy for human faces and 59% for nature scenes.
Several factors contribute to this modest human ability. The type of image content plays a significant role, as humans are better at recognising AI synthesis in human faces due to our innate proficiency in facial recognition. However, as AI models become more sophisticated, creating increasingly photorealistic images, human detection becomes more difficult.
Individual differences, such as visual literacy, familiarity with AI artifacts, and experience with digital images, can also affect detection skills. Some AI models leave subtle cues or inconsistencies, but these require trained or attentive observers to notice reliably. Cognitive bias may also play a role, as people may rely on intuition or conventional cues that AI increasingly mimics or avoids, reducing effective recognition.
In contrast, automated AI detection tools greatly outperform humans. These tools, such as the "AI Or Not" tool, achieve accuracies above 90% in controlled settings by analysing known AI patterns and forensic features in images.
Emerging research aims to improve explainability in AI image detection by combining human annotations with machine learning to align model reasoning with human intuition. This could potentially enhance future detection approaches.
The study also found that several AI image generators, including the Generative Adversarial Network (GAN), had high failure rates. In some cases, there were clear signs of AI use, but in many cases, it was difficult to tell if an image was real or fake.
The images that caused the lowest success rate included elements that looked unnatural but were genuine, such as unusual lighting conditions. The researchers attribute the higher success rate in identifying humans to our high ability to identify faces, as our brains are wired to recognise faces.
Approximately 287,000 image evaluations took place, and over 12,500 people participated in the study. The study concludes that generative AI is evolving rapidly, and new or updated generators are producing increasingly realistic output.
Interestingly, when participants tried to identify images that were altered with or created by AI, the average success rate dropped to 47%. This suggests that even with advancements in AI technology, there is still a need for human oversight and scrutiny in image analysis.
For those who are interested, the Real or Not quiz, used as a basis for the study, is still live and available for anyone to take.
- The study conducted by the Microsoft AI for Good Lab found that the "AI Or Not" tool, a tool designed for AI detection, greatly outperforms humans, achieving accuracies above 90% in controlled settings.
- Individuals with higher visual literacy, familiarity with AI artifacts, and experience with digital images may show better detection skills in recognizing AI-generated images compared to those with less experience.
- The study also found that AI image generators like the Generative Adversarial Network (GAN) had high failure rates, with many images being difficult to distinguish from real images.
- Microsoft's "Real or Not" quiz, utilized in the study, is still live and available for anyone to take, providing an opportunity for users to test their ability to identify AI-generated images.
