The Blind Smartphone Camera Test Winners!

(upbeat music) (logo humming) - Alright, everyone's been waiting patiently for this. Once again this year, we've done our

blind

smartphone

camera

test

, scientific style. We always think we know what the best

smartphone

camera

s are, but do we really know? What if we wanted an objective answer directly from real-world data? Literally every time I review a phone, there are some people in the comments section who say, "Oh, the iPhone photos are the only good ones," or "I only like the Pixel photos," or "Samsung is the only one that takes good photos." portrait mode." But what if we put that to the

test

?

What if we took a bunch of the best smartphone cameras on the planet and took them out into the real world and took the exact same photo with each and every one of them, and then we would remove all their labels, and then we would have you, the public, vote on which of the photos you like the most? That should reveal the best current smartphone camera. So that's exactly what we did this year. 20 smartphones, we update all their software, charge them all and bring their batteries to 100% Then we go out and take the same three photos with each one A daytime photo, a low light photo and a portrait mode photo Yes,.

It's surprisingly difficult to stay completely still while taking 20 identical photos in a row, but I did it for you. Then we had to import all the photos from each smartphone camera - harder than you might think - to strip them of all their EXIF ​​data. then upload them to a site we created that allows us to put them side by side. for you to vote face to face millions and millions and millions of times, which you did. That should give us a statistically significant winner and loser, we'll get to that. And this year's results were a bit interesting.

I almost didn't make this video, but I can tell that people have been waiting, the competitive tension is high, and the actual results may surprise some of you a little. So let's get into it. So all the data our site spits out comes in these text files with comparison ratings for each smartphone combination. In fact, we code them with letters of the alphabet that you can see, from A to T. But that little number, that one here, is the most important one. That's the Elo rating. You know, the system that ranks chess players or tennis players, basically anything, head to head.

With enough matchup information, we can rank all of these letters based on who they beat and who they lost to, and create a power ranking, and that will give us our

winners

. So the question you probably all want to know is: who were the

winners

? And then we'll go category by category because, like I said, there were three different scenarios, daytime, low light and portrait mode, so let's go one by one. So during the day, for normal daylight photos, we had it set up with just me sitting in front of this window here, but there's a lot in this photo.

The brightest light comes from inside the room, but window light is actually a good test of dynamic range. And then there's also a variety of colors, there's my skin tone, there's the orange chair next to the blue pillow, a few other things. This is intentional, there is no single variable that can dominate this test. So we try to frame each phone exactly the same way, from exactly the same place, holding one of them as a reference so we can get as close as possible. And it's all completely automatic, we just open the camera app, make sure the lens is clean and just press the shutter button.

We don't even touch to focus, we just point and shoot. It takes about six minutes to check all 20 phones. Like I said, it's a little hard to sit still for six minutes straight, but we did it. So for this normal daylight photography situation, the winner with the highest Elo rating is the Pixel 7A. That's pretty impressive. So this is the photo that you guys, for the most part, voted as the winner in most of your matchups. Pretty neutral photo, to be honest, not too bright or too dark, and pretty excellent dynamic range. And it's not a coincidence either because the second highest Elo rating came from the Pixel Fold.

Then the Fold took a very similar photo. And then finishing on the podium was the OnePlus Open, one of those high-end foldable phones. So I will say that my favorite photo and the one that really won when I took the

blind

test was this one. This is the fourth highest Elo rating, this is the OnePlus 11. It's definitely a little bit more contrasty, but a little bit safer at achieving exposure and having dynamic range. This is definitely interesting though. Many people pointed out that the sun behind me looked a little different in each photo. It could have been setting up so the background was a little different, which is kind of true, but it wasn't late enough for that to really affect the photo.

I think the fact that the 7A is the only one with this little lens flare is really just a coincidence. And then, a fun fact: Last with the absolute lowest Elo rating for this daytime photo is the iPhone 15 Pro. It's not a terrible photo. I mean, you can't really get a horrible photo in normal lighting for most of these smartphones, but when you compare it to the others, it's mostly just the darkest. And then when you compare it to these much brighter photos, people just choose the brighter one almost every time. Interestingly, though, you can absolutely overexpose for this competition, as the second-lowest Elo rating belongs to the Galaxy S23 Ultra for producing this strangely overexposed masterpiece.

So here are all the photos along with their Elo ratings. Feel free to pause it if you want to make more observations, but let's move on to the night photos. So in low light, low light is a much more challenging shot for a smartphone camera, especially with a tiny sensor and tiny optics trying to capture as much light as possible. But computational photography has come a long way, and even on this fairly dark rooftop where the only light source was about 40 feet away from me, these cameras managed to do a good job. And if you're ready for the plot twist, the highest overall Elo rating for the low light photo comes from, the last one for the daylight photo, is again the iPhone 15 Pro, with another totally reasonable photo.

Now, following very closely in second place was the Pixel 8 Pro. And, in fact, the Pixel 7A is in third place. And to be honest, I totally agree with all of this. It's starting to get ridiculous how much some of the other smartphone cameras started upping the light, like literally the minute you get out of the top four, you're straight into bad HDR territory. This is the fifth place photo of the S23 Ultra, which is definitely too bright. And also, you can see this little HDR halo effect starting to appear around my head. And then if you scroll down to Oppo Find X6 Pro, oh my god, what happened?

Like these cameras are going crazy. It seems like in this one they literally sloppily painted over me and just increased the exposure. Matter of fact. (scratching pencil) Yeah, see? That's exactly what it sounds like. The Zenfone also did this along with a few others. And honestly, I think the OnePlus Open not only made me brighter, but it also identified the sky and made it darker, which at that point starts to seem ridiculous. I mean, real life was nothing like this. Here are all the night photos and their Elo ratings. Pause it if you need to. And now we can move on to the last but not least category, the portrait mode.

So portrait mode turns out to be the most difficult test to run, like this one, because everyone does portrait mode a little differently. If you just open the camera app and switch to portrait mode, some of them do 2X, some of them are 3X, some say 1X, but they're actually like 1.5X, it's a little strange. So we tried to keep it simple for this test and just open the camera app, switch to portrait mode, and then move your feet forward or back to try to match the focal length with all of these shots. It worked a little, and they all have different levels of bokeh too, some are adjustable, some are not.

But like I said, we're just going with the default, point-and-shoot that most people will get when they do it. And the winner in portrait mode, point and shoot with the highest Elo rating is the Pixel 8 Pro. This was followed pretty closely by the Samsung Z Fold 5, which I actually think had a better cutout. And then the third highest Elo here was the iPhone 15 Pro, which I think had the most natural looking blur. But basically all of these podium finishers have relatively subtle portrait modes, nothing too dramatic and then pretty good detail and overall balance just in the rest of the photo.

I say this because the losers here in this particular category look nothing like this. (laughs) They were some of the strangest and strangest photos of the entire competition. And the loser with the absolute lowest Elo rating for this category, and really in any category, was the Sony Xperia 1 V with this photo, with that, is that the photo? Well, this seems like a mistake. In fact, we all thought it was a mistake when we filmed it, like, "This can't be right, right?" We clean the lens, we pick it up again, it does it again. And then he did it again and again.

I guess for some reason this $1,200 Android flagship phone in its auto portrait mode just couldn't handle this lighting, it had some weird issues. Anyway, here are all the portrait mode photos and their Elo ratings. You are welcome. I would like to give out some prizes. Now that we have all of our matchup data and all of our voting and all of our Elo ratings for every person you voted for, with over 20 million votes in total, there's some pretty interesting stuff here. So we have our winners and losers in each of the three categories, which is great, but if you average it all out, we have the highest overall average Elo rating.

We will call this the People's Champion. And with the highest average of 1660, that's pretty competitive. It was Pixel 7A. So this one won the daytime photo, as you probably remember, but it also came in third place in the night photo and fourth out of the 20 in the portrait mode photo. So that's pretty strong. But here's the kicker, the second highest average Elo rating was the Pixel 8 Pro, and the third highest average Elo rating, again, these are your votes, is the Pixel Fold. So Pixel, Pixel, Pixel, it's a perfect podium for Pixel. So here you can see the complete list from top to bottom.

The three Pixels at the top, surprisingly in reverse price order. (laughs) And then at the back for full Elo, there are the Sony Xperia 1 V and the Xiaomi 13 Ultra. Then I also had to bring back the Bang for the Buck award, which I did last year as well, it's basically the majority of your votes divided by MSRP at launch. Basically, most votes per dollar. And then your winner, again, will be the Pixel 7A. This was a phone that launched around $500. He did amazing in the tests. But in second place in votes per dollar is the Nothing Phone 2.

It's a pretty good performing phone and was actually launched as an upper mid-range price. So this is a good phone for the money. And then at 3A, 3B, side by side below these two, you have the OnePlus 11 and the Zenfone 10. And then, impressively, Sony is actually not last in this category, it's expensive and it had a horrible performance. but it's no worse than a super expensive foldable phone, the overall average Z Flip5 performed worse for the dollar, the more you know. So what did we learn from all this? Well, I'll leave you with this. First of all, the Pixels killed it, they achieved the one-two-three, they are an excellent investment.

It was strange that they had a reverse price order, but that's okay. But of course, a smartphone camera is also much more than just a staged photograph of a seated person. We learned a little about how it picks up my skin tones. There are a lot of other things you might be interested in, from the autofocus speed to the user interface, the actual file format, how editable it is, and videos, as if there are so many more. But at the end of the day, I think we still weren't surprised that the slightly brighter photos outperformed the slightly less bright photos.

And maybe we'll run it again next year and learn something new. Anyway, thanks for watching this one. See you next time. Peace. (upbeat music)