REALITY CHECK: Everyone thinks they can spot AI photos. Here is a game that tests if that's true.
8mon 27d ago by piefed.social/u/Ek-Hou-Van-Braai in Technology@programming.dev from realitycheckk.com
The game is simple:
Two photos side by side
One's made by a human, one's made with AI
Pick the AI one
See if you're right (plus the source/prompt)
Is this site just going to use the results to train AI to generate better fake imagery?
I don't think much training can be done on 20 pictures
This is literally us giving them training data for AI to identify what pictures look like AI...
Which in turn is training data to help AI make better pictures
I didn't zoom in or anything like that, but I also feel like I was playing against "Dev traps" instead of unbiased pairs if that makes sense.
🤖 Reality Check #1 15/20 🔥 Perfect Streak: 9
I can spot AI images 75% of the time. 🧠 Can you beat me? realitycheckk.com
🤖 Reality Check #1 20/20 🔥 Perfect Streak: 20
I can spot AI images 100% of the time. 🧠
Most of the AI generated images looked too clean and smooth. It made me appreciate the real images more.
Yeah, the quickest way to tell was to look for things that AI wouldn't include, like brown tips on flower petals or broken sticks that have the right fracture pattern and don't just fade away. AI tends to generate things that are very averaged/ideal looking. The textures of ai images also look wavy with like a 10-20pixel periodicity when zoomed in. The noise from real cameras is much more speckly.
I also got 20/20, but only because it was an A/B comparison. If asked AI or real on each image indlividually, I'm not sure I would get much above 65%.
70%. I noticed that the more I tried to dissect it, the worse I was. I was far more accurate taking a quick glance at both and making a decision after like 5 seconds.
Just like multiple choice quizzes, go with your first guess if unsure.
Interesting, I was kinda the opposite. I got 80% looking very carefully at each pair.
Reality Check #1 19/20
This thing messed me up:
They both looked like they could be AI - but to me it looks like that one has an AI artifact
Meh, the block just got pushed vertical, but it is weird, agreed. The one on the left doesn't strike me as natural. Ice neither forms nor breaks up like it's been put through a blender.
This broke my streak…
Same
Here's the thing: you should expect some things that aren't average in reality. Sure, most ice won't do what you're looking at, but some will. The left image all looks the same across the image. There's nothing that is non-average looking.
That’s the one got me too.
That's one of the problems with AI images, anything that looks weird is now dismissed as AI even if the photos are century old. A colleague of mine said crabeater seal teeth are definitely AI, for instance.
crabeater seal skull
So the hardest part was that a lot of the pairings both looked like AI and every 50/50 guess I made was wrong. Many real images that I got wrong looked like staged professional shots with a ton of digital manipulation afterward.
Final Accuracy: 60%
AI image generation is in this weird place between professional staged shots and amateur photography and is probably there because the training set includes both.
Same, I was looking for inconsistencies in lighting, black/white points, etc, and I think in several cases I picked an image as AI based on details that were a result of photo manipulation and not "real". I got the same score as you in the end here.
Kinda interesting, somewhat invalidated what I thought this was testing, but it's an interesting test nonetheless. I may retake this in a few days now that I know it's comparing AI to photoshop and not "reality". Would certainly need a different strategy.
🤖 Reality Check #1 16/20
🔥 Perfect Streak: 12
I can spot AI images 80% of the time. 🧠
It's almost always, "this looks too clean/perfect."
Those images are very AI-easy IMO. No crowds or complex images.
85% ... and a lot of these pictures were just landscapes. If there we more pictures with people, animals or other objects, this would be even easier.
Nice try but I'm not doing QA checks for generative AI slop.
It seems like a reply to the wrong point.
I'm not saying most people can tell the difference. If they can now, they won't soon. My issues with AI are about training on people's work and then killing their jobs. Also destroying the environment, and other issues.
fuck that UI. tapping to select and zoom below is odd. I messed up a few and exited.
The swipe to move to the next slide was so similar to the swipe backwards, I exited twice and gave up.
Skill issue /s
🤖 Reality Check #1 18/20 🔥 Perfect Streak: 7
I can spot AI images 90% of the time. 🧠 Can you beat me? realitycheckk.com
The two I got wrong were both very artsy and abstract. Anytime there is anything with detail in the image either check it's super fuzzy and dreamlike or it's way too sharp. I looked at each image for no more than a few secs, I feel with say a minute per image you could easily get close to 100%.
🤖 Reality Check #1 16/20
🔥 Perfect Streak: 7
I can spot AI images 80% of the time. 🧠
Can you beat me? realitycheckk.com
Quite interesting
🤖 Reality Check #1 19/20 🔥 Perfect Streak: 18
I can spot AI images 95% of the time. 🧠 Can you beat me? realitycheckk.com
The one pic with the pink flower and water bubbles under water really put me off
Yep that got me too. Those lily pads looked very unlikely
Plot twist: they were all AI generated.
That's what I would do if I would create such a test. But I would add the option "can not decide" to keep it a bit more fair.
85%! I've gotten pretty good at this.
🤖 Reality Check #1 17/20
🔥 Perfect Streak: 8
🤖 Reality Check #1 19/20
🔥 Perfect Streak: 14
Damn, so close
FUCK AI
🤖 Reality Check #1 18/20 🔥 Perfect Streak: 10
I can spot AI images 90% of the time. 🧠 Can you beat me? realitycheckk.com
Am dumb?
🤖 12/20 🔥 Perfect Streak: 4
I can spot AI images 60% of the time. 🧠
Got up to 11 myself before I lost it.
80%
🤖 Reality Check #1 18/20 🔥 Perfect Streak: 9
I can spot AI images 90% of the time. 🧠 Can you beat me? realitycheckk.com
- Reality Check #1 18/20
- Perfect Streak: 13
I can spot AI images 90% of the time.
I got 70%, Which is not that much better than random chance (50%) Scary stuff
I only ot 50% lol
75%
At first I was analyzing both and picking the wrong one (one of the human ones had a tree that looked like early AI lmao), eventually I just went off on instinct and ended up being right on 90% of them. I guess the details were fine-tuned, but there's still something off about them.
The problem with this is that it's mostly nature scenes. Once images of people are used, it's much easier to detect slop.
I got 14/20. Once I stopped over thinking it it was easier. Sometimes I found myself picking the ones I thought looked "more realistic" because I kept getting them wrong and if worked better. Something something hyperreal.
I got 90% and most of my guesses we against the pic that looked too perfect. Hyperreal indeed!
🤖 Reality Check #1 16/20 🔥 Perfect Streak: 7
I can spot AI images 80% of the time. 🧠 Can you beat me? realitycheckk.com
Someone should make a study about this.
I miss clicked once by mistake but otherwise this is my score. Pretty interesting. Though if I wasn't given a side by side to pick just one I'm not sure I would have guessed as many of them. When it's just two it's pretty easy to get lucky. I'd like to see something like this with more than two at once and told X amount are AI and see how well we do. My bet is we do worse.
🤖 Reality Check #1 16/20 🔥 Perfect Streak: 12
I can spot AI images 80% of the time. 🧠 Can you beat me? realitycheckk.com
You could also do one at a time and have to guess on it's own
Oh great point! I'm sure I would have done worse that way as well. Particularly if it didn't give me feedback until the end.
🤖 Reality Check #1 18/20 🔥 Perfect Streak: 9
I can spot AI images 90% of the time. 🧠 Can you beat me? realitycheckk.com
Granted I accidentally refreshed half way through so maybe my score should be 17/20
The images are pretty good but a lot of them still have the "too polished" look.
Also wish the text to copy doesn't look so AI itself lol
Just me not reading any directions till 3 in, I thought I was picking the real one at first
🤖 Reality Check #1 14/20
🔥 Perfect Streak: 8
I can spot AI images 70% of the time. 🧠
Can you beat me? realitycheckk.com
The problem with this test is that it all depends on the model for the AI pictures. There are models now that specifically aim to have imperfections to emulate real pictures, unlike these
20/20.
The nature shots were harder than shit that had man-made things or people in them. But also, the resolution of each pairing being dramatically different kinda gave a pattern to which one would be which.
🤖 Reality Check #1 20/20 🔥 Perfect Streak: 20
...some of those looked much more impressive than I expected, though.
It's a lot easier if you zoom in and check fine patterns.
We are cooked.
80%, better than I expected for myself. It does help that they are showing similar images side by side.
ezpz
🤖 Reality Check #1 20/20
🔥 Perfect Streak: 20
I can spot AI images 100% of the time. 🧠
Can you beat me? realitycheckk.com
It looks too perfect and kinda slimy so that's how I know.
🤖 Reality Check #1 15/20 🔥 Perfect Streak: 4
I can spot AI images 75% of the time. 🧠 Can you beat me? realitycheckk.com
The stars ones beat me every time. How can that many stars be a real photo?? Long exposure blah blah blah
No I don't
Half of them would go unnoticed on a random article.
🤖 Reality Check #1 13/20 🔥 Perfect Streak: 10
I can spot AI images 65% of the time. 🧠 Can you beat me? realitycheckk.com
I can clearly do it sometimes, but the other people in this thread got me beat.
78%, not bad. Anything with no people or animals are always harder to spot, especially with how small the pictures are. I want to try one with with bigger pictures and people prominently featured.
🤖 Reality Check #1 20/20 🔥 Perfect Streak: 20
I can spot AI images 100% of the time. 🧠 Can you beat me? realitycheckk.com
75% I found the landscape photos much more difficult.
🤖 Reality Check #1 14/20 🔥 Perfect Streak: 5
I can spot AI images 70% of the time. 🧠 Can you beat me? realitycheckk.com
The ones I got wrong I rushed and selected my first instinct without looking them over for at least ten seconds.
🤖 Reality Check #1 19/20
🔥 Perfect Streak: 18
I can spot AI images 95% of the time. 🧠
Ahh, almost! I did have to zoom into most photos though so a lot of them could pass on quick glance.
🤖 Reality Check #1 17/20 🔥 Perfect Streak: 8
I can spot AI images 85% of the time. 🧠 Can you beat me? realitycheckk.com