Experiment: Can humans recognize AI-generated images?

Computer-generated images have been around for decades. People use computers to make digital art and animations for movies, TV shows and video games. However, just like making a physical painting with a paintbrush, these images take a lot of time and human effort to produce.

The rise of artificial intelligence has drastically reduced the amount of time and effort it takes to create computer-generated images. New programs and websites can generate images using a text prompt from the user, such as “a picture of a tiger walking through a grassy field.” These programs can automatically generate artwork and photo-realistic images that can be difficult to tell apart from “real” photos and artwork. Can you tell which picture at the top of this story is real and which is AI-generated?

While AI-generated images might seem fun or harmless, they can also cause problems. People can use them to create deepfakes, or fake images of things that did not really happen. Some people might see the picture and believe that it is real. While fake news and fake images have been around for a long time, AI tools can make it easier and faster to produce this misleading content.

Can people tell the difference between real pictures and AI-generated pictures? How hard is it to spot “fake” pictures? In this science project you will find out!

Terms and concepts

• Computer-generated image
• Artificial intelligence
• Deepfake

Questions

• What are some uses for AI-generated images?
• What are some potential problems caused by AI-generated images?

Resources

• Axios (2023, July 31). How to spot deepfakes created by AI image generators. Retrieved September 14, 2023.
• Bond, S. (2023, June 13). AI-generated images are everywhere. Here’s how to spot them. NPR. Retrieved September 14, 2023.

Materials and equipment

• Internet access
• Digital camera or smartphone
• Optional: printer
• Volunteers
• Lab notebook

Experimental procedure

1. Decide on a topic or theme for your pictures. For example, you could use pictures of animals, flowers, scenery, vehicles or people.
2. Take at least 10 real pictures of the object/topic you decided on or find pictures online.
   1. Make sure you label or organize the images so you do not lose track of them later. (As an example, you could put all the real pictures in a folder on your computer.)
   2. If you are finding the pictures online, make sure they are from a legitimate source and you know they are real pictures. (See references in the Bibliography for tips.)
3. Find or make at least 10 AI-generated images of the same object/topic.
   1. You can search online for an “AI image generator” and you will find many options available. Some services might be built into major search engines. Others might have their own websites. Also note that some services might be free, or allow you to generate a limited number of images for free. But others might require a paid subscription.
   2. Follow the instructions for the website or program you decide to use to enter a prompt and generate an image. Since you will be comparing them to real images, make sure you generate “realistic” photos and not images that look like paintings or drawings.
   3. Save the images. Again, make sure you keep track of which images are real and which are AI-generated.
4. Prepare all the images for viewing by your volunteers. For example, you can label them 1 through 20 and put them in a random order in a different folder on your computer. Or you could print them. Make sure you keep track of which images are real and which are AI-generated. But this information should not be visible to your volunteers.
5. Prepare a data table like Table 1. In the second column, write whether each image is real or AI-generated.

Image number	Real or AI generated	Volunteer  1	Volunteer 2	Volunteer 3	Volunteer “real” responses	Volunteer “AI” responses	% of volunteers correct

6. One at a time, show each picture to a volunteer. Ask them whether they think the picture is real or AI-generated. Record their response in your data table.
7. Repeat the process for each volunteer.
8. For each image, add up the number of volunteers who said the image was real. Enter this value in your data table.
9. For each image, add up the number of volunteers who said the image was AI-generated. Enter this response in your data table.
10. Calculate the percentage of volunteers who correctly identified whether each individual image was real or AI-generated. Enter the percentage in your data table.
11. Create another data table like Table 2.

		Volunteer responses
		Real	AI-generated
Actual image	Real
	AI-generated

12. Analyze your data.
    • Overall, how good were your volunteers at correctly identifying real images as real?
    • Overall, how good were your volunteers at correctly identifying AI-generated images as AI-generated?
    • Are there large differences in your results between individual pictures? Were some pictures harder for your volunteers to correctly identify than others? Looking at the pictures, why do you think this occurred?

Variations

• Repeat the experiment with artwork instead of pictures. Can your volunteers tell the difference between real artwork and AI-generated art?
• Do the experiment with two groups of volunteers: a control group and a group that you have trained to spot AI-generated images. (See some of the references in the Bibliography.) Can people with training do a better job correctly identifying the images?
• Try the experiment with different categories of images/objects. Are some things easier for people to recognize than others? For example, what about pictures of inanimate objects vs. pictures of living things? What about pictures of “regular” people vs. famous people like politicians or actors?
• Compare different AI image generation websites or services. Are some better than others at producing convincing images?
• Do an experiment to find out if people can recognize AI-generated text instead of images.
• Can you produce fake news articles that include both images and text about real people or events? Run the experiment with both real and fake news articles. Can people tell which is which?

This activity is brought to you in partnership with Science Buddies. Find the original activity on the Science Buddies website.