Jump to content



Photo

Samsung caught cheating on phone specs


  • Please log in to reply
82 replies to this topic

#1 Secular Humanist

Secular Humanist

    Neowinian Senior

  • Joined: 17-June 09

Posted 31 July 2013 - 13:51

Samsung caught optimizing its phones to score higher on benchmarks

 

It can be tricky figuring out which mobile device to buy. Some get longer battery life than the competition, while others offer more storage, memory, or special features like hands-free operation.

But some folks like to get the model with the fastest processor — and it turns out that may be a little harder to determine than you’d think.

That’s because while there are a series of apps that let you benchmark your phone or tablet’s performance, manufacturers have started to optimize their devices specifically to score well on those tests.It’s sort of like teachers instructing kids about everything they need to know to pass a test. While it boosts the chances that they’ll ace the exam, it’s not clear if they’re learning anything that will help them in the real world.

Likewise, just because a phone scores well on the Antutu, GLBenchmark, or Quadrant benchmark doesn’t necessarily mean it’s going to be faster in everyday use than a model that achieved a lower score — because it’s not always transparent to the user just how that score was earned.

Some manufacturers have been accused of running fake, or modified benchmarking software in order to inflate their scores.

Now the folks at AnandTech have discovered that Samsung takes a different approach: The Samsung Galaxy S4 smartphone runs the CPU and GPU at full blast whenever you’re running common benchmarking apps.

When you close the app, odds are that your hardware will run more slowly. The result is that the score doesn’t provide a realistic indicator of real-world performance and if you try to compare your score with that of another device, you won’t really know for certain why one is the fastest.

You can find more details at AnandTech, but in a nutshell, there are two versions of the Galaxy S4. Whether you have a model with a Samsung Exynos 5 Octa processor or a Qualcomm Snapdragon 600 chip, the CPU will run at its highest speed while a benchmarking app is open.

With the Exynos model, the GPU will also run at 532 MHz. What’s strange about that is that while the graphics processor can handle that speed, the only time it ever seems to go that high is when you’re using benchmarking apps. When you’re playing games (which tend to be among the most resource intensive apps), it tends to top out at 480 MHz.

In other words – theoretically the Galaxy S4 is capable of performing just as well in other apps as it does in benchmarks. It just probably won’t.

 

 

 

http://liliputing.co...benchmarks.html

 

I love my note 2, but this is pretty scummy of a company.




#2 +Nik L

Nik L

    Where's my pants?

  • Tech Issues Solved: 2
  • Joined: 14-January 03

Posted 31 July 2013 - 13:57

manufacturers have started to optimize their devices specifically to score well on those tests.

 

As they have been for about 5 years now.

 

The Samsung Galaxy S4 smartphone runs the CPU and GPU at full blast whenever you’re running common benchmarking apps.

 

The benchmark is to determine what they are capable of, not what they can do when stepped down.  As such, not a problem.

 

When you close the app, odds are that your hardware will run more slowly.

 

Because it doesn't need to run at 100% to send a text.

 

the only time it ever seems to go that high

 

Seems... So a non-conclusive statement there...



#3 Draconian Guppy

Draconian Guppy

    LippyZillaD Council

  • Tech Issues Solved: 3
  • Joined: 22-August 04
  • Location: Neowin

Posted 31 July 2013 - 13:59

samsung already denied and explained this:

 

http://www.techradar...results-1170026

 

 

Which is pretty normal behavior in processors if you ask me, downclock when not in use, upclock when juice is required, how is that cheating?



#4 Knive Party

Knive Party

    To Death

  • Joined: 25-June 07
  • OS: Windows 8.1 update 3
  • Phone: Galaxy S2 CM

Posted 31 July 2013 - 14:09

it 's to save battery life, the S4 cpu and gpu doesn't need to run 100% max frequencies for it to the "fastest experience" all the time. Scummy? Really? Come on.... stop leaping to stupid conclusions OP.



#5 Geoffrey B.

Geoffrey B.

    LittleNeutrino

  • Tech Issues Solved: 11
  • Joined: 25-July 05
  • Location: Ohio
  • OS: Windows 7 Ultimate
  • Phone: Nokia Lumia 928 WP8.10.14203.306

Posted 31 July 2013 - 14:17

it 's to save battery life, the S4 cpu and gpu doesn't need to run 100% max frequencies for it to the "fastest experience" all the time. Scummy? Really? Come on.... stop leaping to stupid conclusions OP.

Yeah i was thinking the same thing, I do not see any problem with them cranking the governor on the phone all the way up during a benchmark because that shows what the phone is capable of doing. during normal use we would not want it to be cranked all the way up.



#6 OP Secular Humanist

Secular Humanist

    Neowinian Senior

  • Joined: 17-June 09

Posted 31 July 2013 - 14:25

it 's to save battery life, the S4 cpu and gpu doesn't need to run 100% max frequencies for it to the "fastest experience" all the time. Scummy? Really? Come on.... stop leaping to stupid conclusions OP.

its what i belive, sorry if it hurts your feelings. I don't know if i am inturpretting it incorrectly or if others are. they way i am reading it is fabricated results. I am sure you also believe what you believe too.



#7 Draconian Guppy

Draconian Guppy

    LippyZillaD Council

  • Tech Issues Solved: 3
  • Joined: 22-August 04
  • Location: Neowin

Posted 31 July 2013 - 14:29

its what i belive, sorry if it hurts your feelings. I am sure you also believe what you believe too.

You do realize this happens everywhere?

 

CPUS, GPUs, do you own intel/amd  nvidia or ati?  This is pretty NORMAL BEHAVIOR     Intel used to call this "speedstep" and "powernow and cool and quiet", I think they're both called "turbo something".


http://en.wikipedia.org/wiki/SpeedStep

 

http://en.wikipedia....tel_Turbo_Boost

 

http://en.wikipedia.org/wiki/PowerNow!



#8 vcfan

vcfan

    Doing the Humpty Dance

  • Tech Issues Solved: 3
  • Joined: 12-June 11

Posted 31 July 2013 - 14:29

benchmarking apps are stupid. real application performance benchmarking is the only way a comparison should be made.



#9 Steven P.

Steven P.

    aka Neobond

  • Tech Issues Solved: 88
  • Joined: 09-July 01
  • Location: Neowin HQ

Posted 31 July 2013 - 14:33

I thought they all did it? Nokia was caught out promoting their stabilizer while filming on the move too :p



#10 Stetson

Stetson

    Neowinian Senior

  • Joined: 04-October 06

Posted 31 July 2013 - 14:34

The issue is not that the CPU/GPU do not run at full speed all the time. Of course they don't.

 

The issue is that the maximum GPU clock even when being pushed by a graphics intensive app like a full screen game is not the same as it is when a benchmark app on the 'booster' list gets launched.

 

That's disingenuous of the phone's maximum real world performance. 

 

Forcing the CPU to full clock speed while a benchmark app is launched also means that the benchmark app doesn't get to see the real world performance of the speed stepping system. If there were delays or problems with the speed stepping you wouldn't be able to see them from the benchmarks because of this trick.



#11 thechronic

thechronic

    Neowinian Senior

  • Joined: 21-July 05
  • Location: UK

Posted 31 July 2013 - 14:34

the way i am reading it is fabricated results

 

See, actually the device performs at it's absolute best during a benchmark - showing what it is actually capable of achieving. That is what a benchmark is for. The question is not why do this for benchmarks, but rather why not do this for resource intensive games/apps? as the article appears to state the GPU clocks at a lower speed outside of Benchmark apps. - Probably, because some resources are being allocated for the Operating System e.t.c 

Conclusion? The Benchmark represents what the hardware is capable of achieving but does not reflect real life scenarios as less hardware will be available to other apps e.t.c due to resource sharing.



#12 OP Secular Humanist

Secular Humanist

    Neowinian Senior

  • Joined: 17-June 09

Posted 31 July 2013 - 14:38

You do realize this happens everywhere?

 

CPUS, GPUs, do you own intel/amd  nvidia or ati?  This is pretty NORMAL BEHAVIOR     Intel used to call this "speedstep" and "powernow and cool and quiet", I think they're both called "turbo something".


http://en.wikipedia.org/wiki/SpeedStep

 

http://en.wikipedia....tel_Turbo_Boost

 

http://en.wikipedia.org/wiki/PowerNow!

from what i read it is not done this way, You may not know what "speed step" and "power now" and "cool and quiet" actually do. Yes they take the actual attainable TOP performance of 100% and knock it down to save power, BUT the difference is the tests that Samsung have rigged the S4 to excel at ARE NOT ATTAINABLE by any program.... that's the error of there results... I think i am reading it correctly.


See, actually the device performs at it's absolute best during a benchmark - showing what it is actually capable of achieving. That is what a benchmark is for. The question is not why do this for benchmarks, but rather why not do this for resource intensive games/apps? as the article appears to state the GPU clocks at a lower speed outside of Benchmark apps. - Probably, because some resources are being allocated for the Operating System e.t.c 

Conclusion? The Benchmark represents what the hardware is capable of achieving but does not reflect real life scenarios as less hardware will be available to other apps e.t.c due to resource sharing.

I could be wrong, but if this top performance is unattainable, then what good is the info? To me its useless data then.



#13 adrynalyne

adrynalyne

    Neowinian Senior

  • Tech Issues Solved: 1
  • Joined: 29-November 09

Posted 31 July 2013 - 14:39

I thought they all did it? Nokia was caught out promoting their stabilizer while filming on the move too :p

They do.  HTC has been doing it for years.



#14 Draconian Guppy

Draconian Guppy

    LippyZillaD Council

  • Tech Issues Solved: 3
  • Joined: 22-August 04
  • Location: Neowin

Posted 31 July 2013 - 14:40

 


I could be wrong, but if this top performance is unattainable, then what good is the info? To me its useless data then.

from my first response:

 

 

 

In response to the accusation that it has been jimmyingGalaxy S4 benchmark results just to make its handset look good, Samsung says nuh-uh guv'nor.

The company's statement explains that whatever GPU frequencies "BenchmarkBooster" rigged up on theExynos 5 variant are "not intended to improve certain benchmark results" and the same maximum settings can just as easily be used to optimise performance on a users' handset based on what they're using it for.

That's as may be, Samsung, but using phrases like "BenchmarkBooster" in your code is still a bit suspicious in our book. It seems pretty clear that, regardless of whether these maxed out settings have real world use, Samsung was also hoping to beef up its benchmark results.

Now for the science

The full statement reads: "Under ordinary conditions, the Galaxy S4 has been designed to allow a maximum GPU frequency of 533MHz.

"However, the maximum GPU frequency is lowered to 480MHz for certain gaming apps that may cause an overload, when they are used for a prolonged period of time in full-screen mode.

"Meanwhile, a maximum GPU frequency of 533MHz is applicable for running apps that are usually used in full-screen mode, such as the S Browser, Gallery, Camera, Video Player and certain benchmarking apps, which also demand substantial performance.

"The maximum GPU frequencies for the Galaxy S4 have been varied to provide optimal user experience for our customers, and were not intended to improve certain benchmark results. We remain committed to providing our customers with the best possible user experience."



#15 adrynalyne

adrynalyne

    Neowinian Senior

  • Tech Issues Solved: 1
  • Joined: 29-November 09

Posted 31 July 2013 - 14:47

To be fair, it is Android.

 

Virtually all the apps are used in full screen unless you have an app that mods things to be windowed.  So why wouldn't that apply to all of them ;)