Samsung rigs benchmark scores to make Galaxy Note 3 seem faster

Well, well, well. Samsung thought it'd get away with a little bit of cheating, but it didn't take long to figure out that Samsung tried and failed to pull the wool right over our eyes. It turns out that the new Galaxy Note 3 is designed specifically with benchmarks in mind so that scores are inflated to make the device look faster than it actually is.

Discovered by Ars Technica, the Galaxy Note 3 kicks into high-gear whenever a benchmark app is launched and in fact will not let the CPU idle whatsoever. This high performance mode allows the phablet to run up to 20 percent faster, but just while the benchmark app is running. Once the test is complete and the app is closed, Samsung's device will return to its regular state and return to normal speeds. The clear benefit to doing this is to make the Galaxy Note 3 look faster than what it is to reviewers, technology enthusiasts and, indirectly, customers.

The Note 3 has a 2.3GHz Snapdragon 800 processor and so does the LG Optimus G2. Ars Technica knew something was up when Samsung's handset was scoring far better than LG's, yet the two feature the exact same processor. Monitoring CPU usage led to the realization that the Note 3 got an unfair boost in apps like Geekbench. In fact, using a renamed version called Stealthbench deactivated the power boost.

Samsung, you've been caught red-handed.

Source: Ars Technica | Image via Ars Technica

Report a problem with article
Previous Story

NVIDIA launching GeForce GTX Battlebox brand for high-end desktop PCs

Next Story

Valve trademarks Half-Life 3 name; development team revealed

233 Comments

View more comments

stevan said,

Actually it isn't, it's somewhat faster but not a long. Not as much as the benchmark cheat made it seem. Look at the chart, then calculate the speed increases in % and you'll see.


It's still quite faster according to the picture in the article. This is the point.

I'm also gonna write down the same thing as i wrote down in the same post as you quoted from me as you most likely ignored it or didn't see it, but here it is: The point is that Samsung have optimized their Samsung ROM's to run better, in the same way as Samsung have optimized their Samsung ROM's to have longer batterylife on a TouchWiz ROM than you can get on a stock Google Edition ROM.

Go look on GSMArena if you don't believe me.

Exynos said,

It's still quite faster according to the picture in the article. This is the point.

I'm also gonna write down the same thing as i wrote down in the same post as you quoted from me as you most likely ignored it or didn't see it, but here it is: The point is that Samsung have optimized their Samsung ROM's to run better, in the same way as Samsung have optimized their Samsung ROM's to have longer batterylife on a TouchWiz ROM than you can get on a stock Google Edition ROM.

Go look on GSMArena if you don't believe me.

You mind replying me with some proof that Apple does the same thing?

They optimized their code to kick the processor to 100% when benchmark app is running. The fact that the benchmark app gets speeds from the cpu and gpu and those speeds are not available to many other applications can be called cheating.

I'm not blaming you for defending Samsung, it's just so obvious they cheated and so many different tech sites are writing about it.

stevan said,

You mind replying me with some proof that Apple does the same thing?

They optimized their code to kick the processor to 100% when benchmark app is running. The fact that the benchmark app gets speeds from the cpu and gpu and those speeds are not available to many other applications can be called cheating.

I'm not blaming you for defending Samsung, it's just so obvious they cheated and so many different tech sites are writing about it.


Because it's obvious that everyone does this, then YOU can rather find evidences that they don't do it?

Again, why shouldn't Apple do it when everyone else is doing it?

Exynos said,

Because it's obvious that everyone does this, then YOU can rather find evidences that they don't do it?

Again, why shouldn't Apple do it when everyone else is doing it?

Apple wasn't caught doing it. Apple also doesn't brag about benchmarks, they compare their hardware in keynotes to their previous gen and that's pretty much it. Others run the hardware for them.

You're defending this pretty bad even though a lot of users are disagreeing with you and a LOT of tech sites are saying opposite of you as well. Must make you right?

Samsung got caught cheating, giving the benchmark app access to resources other apps couldn't get. It's fair and simple.

stevan said,

Apple wasn't caught doing it. Apple also doesn't brag about benchmarks, they compare their hardware in keynotes to their previous gen and that's pretty much it. Others run the hardware for them.

You're defending this pretty bad even though a lot of users are disagreeing with you and a LOT of tech sites are saying opposite of you as well. Must make you right?

Samsung got caught cheating, giving the benchmark app access to resources other apps couldn't get. It's fair and simple.


Because no one have been able to look into their code in iOS 7 yet.

Does that makes sense to you?

No, Samsung gave benchmark apps access to use the full potential out of the hardware at stock speeds where other apps can do the same if they actually require such power. But yeah, most of the apps out to Android today doesn't even needs 40-50% of the full potential of the Galaxy Note 3 to run perfectly, so again, i don't see why they should run at 100% to just waste battery?

Edited by Exynos, Oct 2 2013, 1:31pm :

alwaysonacoffebreak said,
So when Samsung does it it's not okay but when Intel has done this for years no one even cares?

Funny.

I didn't know Intel made smartphones...

stevan said,

I didn't know Intel made smartphones...


This isn't about smartphones, it's about how benchmarks apps are made and how they are used.

Exynos said,

This isn't about smartphones, it's about how benchmarks apps are made and how they are used.
What all the Apple fans and Samsung haters are all up in arms about is this fact:

In every model of the iPhone, the CPU was significantly under-clocked, likely to save battery power. That is why in the first 3 iPhone models, they used the exact same CPU, and simply clocked it up. The original CPU Apple used in the first 3 models weree capable of running at over 1GHz. So why did Apple only run them at 400Mhz, 612 and 800? IMO, it was because they were to lazy/cheap to simply make the battery bigger.

Look at today's hardware. What is different? Nothing. Its faster, its smaller and they all suck more juice but simply do it more efficiently. So how do you overcome more speed? You make the battery bigger. If you took a battery out of an iPhone 3G and placed it in today's iPhone 5S, it would drain in likely 1/2 the time as the battery does now.

The way I see this article is simple. Samsung allowed for its phone to run 20% faster in benchmark score so that there top device beats the competition. As we can see, the hardware is capable of more but was likely limited to same on power consumption or similar. It proves the Exynos/Snapdragon is a very powerful platform even if we as typical consumers will never use all of the power.

HipHopSinceFriday said,
What all the Apple fans and Samsung haters are all up in arms about is this fact:

In every model of the iPhone, the CPU was significantly under-clocked, likely to save battery power. That is why in the first 3 iPhone models, they used the exact same CPU, and simply clocked it up. The original CPU Apple used in the first 3 models weree capable of running at over 1GHz. So why did Apple only run them at 400Mhz, 612 and 800? IMO, it was because they were to lazy/cheap to simply make the battery bigger.

Look at today's hardware. What is different? Nothing. Its faster, its smaller and they all suck more juice but simply do it more efficiently. So how do you overcome more speed? You make the battery bigger. If you took a battery out of an iPhone 3G and placed it in today's iPhone 5S, it would drain in likely 1/2 the time as the battery does now.

The way I see this article is simple. Samsung allowed for its phone to run 20% faster in benchmark score so that there top device beats the competition. As we can see, the hardware is capable of more but was likely limited to same on power consumption or similar. It proves the Exynos/Snapdragon is a very powerful platform even if we as typical consumers will never use all of the power.

So they should call it the exynos benchmark rather than the Note 3 benchmark, if it is underclocked in the Note 3.

I presume you're saying that the iPhone 3GS used the same processor as the iPhone and iPhone 3G, but was simply clocked faster? If so, you never saw Apple showing benchmarks saying that the original iPhone was as fast as the 3GS did you?


who even cares about benchmarks? I have never bought a phone by looking at numbers. If I like it, i buy it. Benchmarks dont mean anything.

Question - Is it not a fact that benchmarking software is designed to push a device/computer hardware to excessive limits where a normal user isn't going to reach anyways?

So in this case, Samsung added a little boost by allowing the CPU to run faster during the tests.

Which is equal to the following fact. A computer modder will usually install software that will allow them to push hardware (overclock) to speeds no typical user will use anyway.

Fact - even if you consider it cheating it isn't. The fact is it shows that the hardware is more powerful than what the typical benchmarks shows and that it has been capped for good reason. Like battery-life, or to prevent excessive over-heating or the like.

I don't see the problem.
The articles heading is bogus. Samsung didn't rig the score, they rigged the phone to get a better score. That is a huge difference. he writer did this just to start a flame war with over-sensationalized titles to draw the lovers and the haters. Becoming like Engadget.

Edited by HipHopSinceFriday, Oct 2 2013, 3:52pm :

you guys are crazy when you don't see the problem with this--or maybe you just aren't aware of what is going on.

When benchmarking, the idea is to be as systematic as possible. you should in fact benchmark all sorts of possible scenarios.

The most usefull benchmark is the "reality" benchmark, which is how anything will perform under real circumstances.

when your test case manipulates how it will behave based on the application being executed, and if this application is the benchmark, then this is for obvious reasons is unwanted because if the other test cases don't do the same, you cannot compare results.

it's as simple as that. if you try to argue otherwise you are suffering from some sort of serious brain or judgement deficiencies and maybe should stop to think about what you are saying.

if you still insist think about it:

that fancy galaxy note will have a useless program running wasting your resources executing code just to make sure that if a benchmark app is running all priority is given to it.

Read the facts - http://www.slashgear.com/galax...igans-accusations-01299884/

Read the facts - http://www.slashgear.com/galax...igans-accusations-01299884/

I think if people were not biased and read these 2 articles with a open mind, they would see Samsung did nothing. They didn't cheat. They did what everyone else did, and the phone does what it was designed to do. When apps that are designed to push the hardware harder are loaded, the hardware runs faster which is what it is suppose to do.

In the GS4 article, they stated as a fact, the GPU is designed to run up to 533Mhz and when a full screen app is loaded like games or benchmarking software, the GPU ran at 533Mhz like it was suppose too.

That is not cheating and this phone isn't cheating either. The fact is where the above devices had similar hardware, the Galaxy Note III simply performed better under the same conditions. Get over it!

HipHopSinceFriday said,
Read the facts - http://www.slashgear.com/galax...igans-accusations-01299884/

Read the facts - http://www.slashgear.com/galax...igans-accusations-01299884/

I think if people were not biased and read these 2 articles with a open mind, they would see Samsung did nothing. They didn't cheat. They did what everyone else did, and the phone does what it was designed to do. When apps that are designed to push the hardware harder are loaded, the hardware runs faster which is what it is suppose to do.

In the GS4 article, they stated as a fact, the GPU is designed to run up to 533Mhz and when a full screen app is loaded like games or benchmarking software, the GPU ran at 533Mhz like it was suppose too.

That is not cheating and this phone isn't cheating either. The fact is where the above devices had similar hardware, the Galaxy Note III simply performed better under the same conditions. Get over it!

Samsung denies benchmark boosting, yet a renamed benchmark achieves less performance.

Why lie if they've done nothing wrong?

they changed the name of the benchmark application and the result was different.
this is the fact.

and the fact that others do it doesn't make it ok: _No One Should_ it is deceitful to us as customers.

seamus2a2 said,
they changed the name of the benchmark application and the result was different.
this is the fact.

and the fact that others do it doesn't make it ok: _No One Should_ it is deceitful to us as customers.

All benchmark scores are deceitful. Fact, you will NEVER achieve those number in normal usage so it doesn't even matter.

seamus2a2 said,
they changed the name of the benchmark application and the result was different.
this is the fact.

and the fact that others do it doesn't make it ok: _No One Should_ it is deceitful to us as customers.


Because the CPU and GPU will throttle the speeds on them if they don't allows the benchmark apps to run at full speeds, witch makes the whole benchmarking thing a waste. And that's why the script takes cares of that so the benchmark apps overrun the speed throttling.

When you run a benchmark app on a Windows computer, do you think the speed on the CPU and GPU are throttling then?

Do you think my 3.7 GHz Turbo Speed CPU will throttle down to 1.8 GHz under a benchmarking run?

I looked at the test in more detail and here are the facts. In all the benchmarking tests on the phones, in the case of the Galaxy Note III, all 4 cores were running at 2.3Ghz which is the max. However, even though they were all running at a full 2.3Ghz, they were not all pushed to 100%.

In the case of the LG G2, all 4 cores were not pushed to the full 2.3Ghz. It looks likes like, one core ran at full speed while the other cores were idle. ON another test, the same LG phone ran the cores at the full 2.3Ghz, but not at 100%.

In fact if you look at the facts of this article and the pictures http://www.slashgear.com/galax...igans-accusations-01299884/ none of the CPU cores on any of the test ever ran at 100% even if an when they cores were running at full clock.

What has been shown as fact is, on the Galaxy Note III, the phone calls up a java file which simply tells the phone to run all 4 cores at 2.3Ghz. Even though they are running at 2.3Ghz each, they are not all running at 100%. In fact in the test the combine percentage was roughly 4% of the total available power in one of the tests.

So my question is, how is this misleading? The facts shows the cores didn't even overclock, they were simply running at full clock speed.

Any of you who have a Windows PC can just open the task manager to see all your CPU cores. If you fire up an application like Photoshop, 3 or my 4 cores ramped up to full speed than they dropped. If you use your PC to convert a video file from one type to another, I have have render 4 files at one time as the software will force 1 core to each file. In this case, in the rendering, all 4 cores are at full 2.4Ghz and they are all at 100%. This is real world usage based on an application.

All Samsung did was tell the phone, when this benchmark tool is ran, run all 4 cores at a full 2.3Ghz. The fact that a typical user will never do this is because none of the apps they use are designed to run the cores at full speed. If an app ran the cores at full speed all the time simply because they are there, it would be a complete waste of battery and power. Benchmark tools typical run the hardware at the fastest possible speed for the task. Samsung simply made sure that under such conditions the hardware was running at full speed. The fact I will never reach full speed does matter.

If I buy a Ferrari F40, then I know the full speed of the care is 240mph. That doesn't mean when I get in may car and drive to the grocery store, I am going to do 240mph every time. What it does mean is that if the situation is right on an open highway with no cops or a track, hitting 240 mph won't be a problem.

The Note III calling up the Java app, is equal to me simply pressing the accelerator hardware and shifting to the highest gear when needed. How is that cheating when I am simply pushing my car to the speed it is capable of?

HipHopSinceFriday said,
I looked at the test in more detail and here are the facts. In all the benchmarking tests on the phones, in the case of the Galaxy Note III, all 4 cores were running at 2.3Ghz which is the max. However, even though they were all running at a full 2.3Ghz, they were not all pushed to 100%.

In the case of the LG G2, all 4 cores were not pushed to the full 2.3Ghz. It looks likes like, one core ran at full speed while the other cores were idle. ON another test, the same LG phone ran the cores at the full 2.3Ghz, but not at 100%.

In fact if you look at the facts of this article and the pictures http://www.slashgear.com/galax...igans-accusations-01299884/ none of the CPU cores on any of the test ever ran at 100% even if an when they cores were running at full clock.

What has been shown as fact is, on the Galaxy Note III, the phone calls up a java file which simply tells the phone to run all 4 cores at 2.3Ghz. Even though they are running at 2.3Ghz each, they are not all running at 100%. In fact in the test the combine percentage was roughly 4% of the total available power in one of the tests.

So my question is, how is this misleading? The facts shows the cores didn't even overclock, they were simply running at full clock speed.

Any of you who have a Windows PC can just open the task manager to see all your CPU cores. If you fire up an application like Photoshop, 3 or my 4 cores ramped up to full speed than they dropped. If you use your PC to convert a video file from one type to another, I have have render 4 files at one time as the software will force 1 core to each file. In this case, in the rendering, all 4 cores are at full 2.4Ghz and they are all at 100%. This is real world usage based on an application.

All Samsung did was tell the phone, when this benchmark tool is ran, run all 4 cores at a full 2.3Ghz. The fact that a typical user will never do this is because none of the apps they use are designed to run the cores at full speed. If an app ran the cores at full speed all the time simply because they are there, it would be a complete waste of battery and power. Benchmark tools typical run the hardware at the fastest possible speed for the task. Samsung simply made sure that under such conditions the hardware was running at full speed. The fact I will never reach full speed does matter.

If I buy a Ferrari F40, then I know the full speed of the care is 240mph. That doesn't mean when I get in may car and drive to the grocery store, I am going to do 240mph every time. What it does mean is that if the situation is right on an open highway with no cops or a track, hitting 240 mph won't be a problem.

The Note III calling up the Java app, is equal to me simply pressing the accelerator hardware and shifting to the highest gear when needed. How is that cheating when I am simply pushing my car to the speed it is capable of?


Bingo, you have understood it. +1 to you.

The fact that alot of the crybabies / Samsung haters in here doesn't even understands this makes the whoile thing a comedy at best.

bjc4ever said,
Lol, how much time have you spent arguing the same point over and over again? Don't you have anything better to do?

I'm just making sure someone gets the whole point as someone clearly haven't done yet here.

exynos, your defense of the point is ludicrous. you make some valid arguments, but you are being highly polarized in your view point. it's as if you are the person that had the brilliant idea of putting in the code on the phone to do the benchmark fudging.

if you say a benchmark is a tool to compare two equipments, then you want the two equipments to be comparable. no arguments. stop making excuses or looking to justify anything. period.

you would be pretty ****ed off if porsche for a benchmark run gave you the 400hp but on your everyday drive to work only gave you 300hp. as a paying client you have everyright to not have been misled.

(by the way car manufacturers fuel benchmarks are one of your "everyone does it" and still not morally right from everyone's point of view)

Commenting is disabled on this article.