Does iOS Crash More Than Android? A Data Dive

[techbeck]

Ever wonder why certain mobile apps you use crash so much?

It turns out there are many possible reasons. And it can vary particularly depending on whether you are using an

Apple iOS device such as an iPhone or iPad, or an Android device.

One of the reasons for crashes is the proliferation of mobile operating systems on iOS and Android. As Apple and

Google have released more new operating systems, each with multiple updates, app developers face more operating systems to test apps on. In data that mobile app monitoring startup Crittercism compiled for crashes between December 1 and 15, there were at least 23 different iOS operating systems that crashed and 33 Android operating systems that crashed. (See the graphs above.) Note that the graphs that separate out Android and iOS show these number of operating systems and the graph that combines both iOS and Android shows less?22 iOS and 17 Android.

The largest proportion of crashes from both iOS and Android platforms were on iOS 5.01 with 28.64% of overall crashes (in a normalized data set). That makes sense since iOS 5 was still relatively new at that time and many apps still need to work out the kinks with the new OS. But there are also older iOS versions that have a significant proportion of crashes. For example, iOS 4.2.10 had 12.64% of crashes, iOS 4.3.3 had 10.66% and iOS 4.1 had 8.24%. One other point that this made clear to me is that many people apparently take their time updating their iPhone software or never update it at all.

The data raises two main questions for me: why do these operating systems crash so much, and does iOS crash more than Android? On the first question of why apps crash, the reasons are many, says Crittercism CEO Andrew Levy. This can be due to hardware issues, such as the use of location or GPS services or cameras; it could be due to the Internet connection, that is, how a phone connects to 3G or WiFi, or that the device is not connected to the Internet at a certain moment, or that something happens during the switch between 3G and WiFi. There could also be issues with language support on certain devices. There can also be memory problems if an app uses too much memory.

Problems can also occur with third-party services that developers use in their apps, from analytics to advertising systems. For example, there were reports that Apple?s iAds system gave some developers problems if they did not adhere to certain standards. ?It can be a mix of both hardware and software issues that developers may or may not be responding to,? Levy says.

In addition, developers also constantly create new updates to their apps to create new features or fix bugs. But again, people often don?t update their apps?just as they don?t update their operating system. (Android, unlike iOS, allows users to auto-update their apps, which can eliminate some of the problems.) So developers often test all previous versions of their apps with each version of the different operating systems. ?The permutations go on forever,? Levy says. ?That?s a large reason for creating our platform.? Particularly with a new OS, developers have to test their app to make sure it still works on the new OS. Often they will seek to test their apps in a test environment, but often that isn?t possible.

The Apple iOS operating system crashes accounted for more of the crashes in Crittercism?s data than did Android-based phones, as mentioned above. In the pie graph ?Crashes by OS Version Normalized? you can see that iOS accounted for close to three-fourths of the crashes, with Android making up the rest. But is that just because Crittercism has more iOS phones in its network? Crittercism parsed some data to answer that question.

Crittercism analyzed a total of more than 214 million app launches from November and December 2011 from apps that use its service (see graph at top of this article). There were about 3 times more app launches for iOS that Crittercism analyzed, about 162 million to 52 million. But the analysis examined crashes as a percentage of each app launch, so this data takes out the issue of there being more iOS than Android apps. In other words for each iOS app and each Android app how often percentage-wise do they crash?

In the top quartile of apps, Android apps crashed 0.15% of the time they launched, while top quartile iOS apps crashed 0.51% of the time. In the second quartile of apps, Android apps crashed 0.73% of the time and iOS apps crashed 1.47% of their launches. In the third quartile of apps, Android apps crashed 2.97% of the time, while iOS apps crashed 3.66% of the time.

So what does all this data mean? On a basic level, you can see that iOS apps crashed more than Android apps during this time period. But Crittercism?s Levy cautions that this doesn?t necessarily mean that overall iOS apps crash more than Android apps. That?s because Apple had recently released a new version iOS 5 in October. Android?s new Ice Cream Sandwich operating system (Android 4.0), meanwhile, had not been widely released on phones yet at the time of this study. ?I expect as Ice Cream Sandwich just launched and the new Nexus S phone launched (during the study), we?ll expect the same situation to occur (with Android) as what happened (with iOS),? Levy says.

Still, the data shows that apps on iOS did crash substantially more than Android apps. Anecdotally, I know that certain apps I use on my iPhone crash and they crash often. Will that change as Ice Cream Sandwich rolls out and as more developers improve on iOS 5? We?ll see.

One other thing about the data: in the best apps, that is, the top quartile, the apps crashed much less than in the third quartile. Levy emphasized that that shows the difference that developers can make with their apps by analyzing the data and improving their apps. Splitting up the data by quartiles also removes apps with massive user bases that can skew the averages.

My own point: when you get to the top apps, Android?s lower crash rate than iOS makes less of a difference because they are both well below 1%. However, there was a bigger difference between iOS and Android crashes in the top quartile of apps than in the third quartile. In other words, the best apps in Android crashed about one third as many times as the best iOS apps, while the second best quartile Android apps crashed about half as much as comparable iOS apps, and in the quartile, the difference between the two operatings systems was even less. So the very top Android apps are achieving a crash rate that, at least in this time period, the best iOS apps can?t match. Why that is, I?m not entirely clear.

However, Android, it should be noted, allows developers to push updates faster than Apple. With Android developers can just send an update to its code, which can show up almost in real-time. But for iOS it can takes days or a week for an update to show up. That means there can be more crashes while those updates are waiting to happen. Whereas with Android, presumably if developers know there?s a bug they can immediately fix it.

One final piece of interesting data from Crittercism: The performance of apps is not only different on various operating systems but also on different devices. About 74.41% of the iOS crashes Crittercism tracked were on the iPhone, 14.81% were on the iPod Touch, and 10.72% were on the iPad.

Crittercism, which is backed by Google Ventures, Kleiner Perkins Caufield & Byers, AngelPad, AOL Ventures, Opus Capital and Shasta Ventures, provides crash reporting to app developers. The company provides developers with a wide range of data besides the mobile device and operating system, including, for example, the length of time between when an app is loaded and when it crashes, or how a user is holding a phone?portrait or landscape?when it crashes. It can also help evaluate whether an app?s own code has caused it to crash or whether a third party service SDK being used is causing the problem. This kind of data is important for helping developers plug all the holes in the landscape of operating systems. Clients using Crittercism include:

Aston Martin Explore

,

Bullet Time

and

Hipster

.

Protecting apps from crashes is not only important for app developers, who by definition live and die by the ability of their apps to work smoothly. Increasingly, many types of companies rely on apps to run their businesses, even if there isn?t a large consumer user base using the apps. Banks have mobile apps that enable check deposits, sales people use apps in the field, and so on. ?It?s about protecting your brand,? says Crittercism cofounder Rob Kwok. ?More and more business-critical functions are moving to apps.?

Crittercism is signing up a number of large customers to its platform as a result. The company has already ramped way up from the 214 million app launches that it monitored in this data from November and December. If you?re hoping to end those annoying crashes on your phone or tablet, that should be welcome news.

http://www.forbes.co...id-a-data-dive/

[still1]

so iOS have more crash than Android??? wow that's amazing...

I had like 3 crashes on my android in the last 3 months. maybe because i have the nexus brand.

[Muhammad Farrukh]

My friend has a 3GS, and it hasn't crashed sice last August.

My brother had an Omnia 7, and it didn't crashed for over a month

My brother had a Galaxy S, and it crashed more than once every month

[MarkusDarkus]

hmm. Don't think I've experienced more than a couple crashes on my Desire HD but I thought iOS didn't crash at all for some reason.

[techbeck]

so iOS have more crash than Android??? wow that's amazing...

I had like 3 crashes on my android in the last 3 months. maybe because i have the nexus brand.

I have the Galaxy Nexus for almost 2 months. Hasnt crashed once.

My friend has a 3GS, and it hasn't crashed sice last August.

My brother had an Omnia 7, and it didn't crashed for over a month

My brother had a Galaxy S, and it crashed more than once every month

My Galaxy S Vibrant never crashed unless I was trying out a different ROM.

I think crashes will vary on user/device and depending on what app is being used or what is being done with the phone. I used to have a myTouch 3g a few years ago and that think crashed all the time because I was constantly modding it and applying different hacks, themes, and ROMs. The stock was better but that was back when Android was first starting out.

[Muhammad Farrukh]

My brother never installed or played with custom ROMS, because he doesn't know how to.

He just makes calls, download games and apps, and browse, thats all.

[FloatingFatMan]

You can't really blame any particular OS for crashing when 95% of the time, the crash is being caused by a poorly written app, not the OS itself. I doubt very much that many of these "measured" crashes were just OS based, for either brand.

[Vice]

I've only ever had the iMessage app on my iPhone crash once. All other crashes (and they are extremely rare) are from 3rd party apps by other developers. To me this indicates that it's the developers at fault and not the operating system. At-least my entire phone has never crashed only an app that is instant to relaunch. I don't think I've ever had my phone itself crash.

[still1]

My friend has a 3GS, and it hasn't crashed sice last August.

My brother had an Omnia 7, and it didn't crashed for over a month

My brother had a Galaxy S, and it crashed more than once every month

I am going to take the millions of apps/devices crash statistics rather than the statistics from 1 persons device/ app.

so yeah, Android have less crash.,

[Belazor]

On my iPad 2, Safari and the App Store crashes frequently (iOS 5.0.1). Also, the App Store randomly removes the Update All button forcing me to remove it from the recently opened apps tray and relaunch it before the button returns.

[Pink Floyd Veteran]

dunno where they take their numbers. I had all iPhones except the first iPhone because it wasnt available in Canada. Now on the 4S and my wife has my old 4 and I can't remember the last time an app crashed

[Denis W. Veteran]

In addition to crashes, sometimes Apple's own apps behave weird until you force a close. For instance, one time Mail was stuck on small fonts even though it was on the default 'medium' font setting. Another time, with a RSS app, the Core Animations were messed up, with all UI elements warping in from the left.

But for crashes, culprits are usually the Settings and the Messages apps.

I took in my phone to a Genius Bar to get it inspected (battery issues), and the first thing they pointed out were the number of app crashes logged on the device.

hmm. Don't think I've experienced more than a couple crashes on my Desire HD but I thought iOS didn't crash at all for some reason.

The OS itself almost never needs to be rebooted, but use an iOS device long enough and you should come across at least one app that kicks you to SpringBoard without warning.

[Salty Wagyu]

Had quite a few crashes with certain ROMs (Cyanogenmod, Redux), sometimes every day in fact. But since putting a stock-based ROM such as Oxygen on it 7 months ago I've not had a single crash.

[DrJohnSmitherson]

Apps on my ipod touch 4g crash all the time. Such a pain in the ass

[Hell-In-A-Handbasket]

Have an iPad 1, when i have multitouch navigation enabled, crash's id say 4+ times a day minimum.

But since day before yesterday i reloaded it and havent enabled multitouch navigation and crashed i think once so far. So id belive those statistics,,,

But if a developer doesnt test their crap and just shoves it out there, its bound to happen

[grik]

i own an iPad2 for 6 months +- and i just have to reboot it sometimes 1 every 2 weeks or so. I dont know what you guys do with these things but i also own a iPhone4s for 3 months and only had 3 crashes that i can remmember.

[Pygmy_Hippo]

Since I upgraded my iPad 1 to ios5 all it ever does is crash. It's not exotic programs either, stuff like Safari will crash several times a session. To be honest i've never had an OS crash so much - people used to rag on the likes of Windows ME but mine was solid as a rock, my iPad is just flaky.

Come to think of it - I have 3 iPods (different generations) that are all software screwed in some way, constant restarts making them all unusable. Maybe my house just doesn't like Apple stuff :(

[Scorbing]

http://www.forbes.co...id-a-data-dive/

I don't know where they got that data from or who made it up, but I have NEVER had iOS 5 crash on me...ever. When I had my Android HTC phone it was a nightmare. Constant force-closes, crashes, freezing, slow downs. I dumped it and went back to iOS (iPhone) and I haven't looked back. And no, I am not an Apple fan-boy, but the truth is the truth. iOS and iOS appz simply work better so I am sorry but I do not believe those charts at all. Seems someone is trying to make Android look better.

[Beyond Godlike]

My friends apps on his iphone crash repeatedly. I get my share of crashes due to my running beta custom roms n everything on my android, but never did i on stock!

[Scorbing]

My friends apps on his iphone crash repeatedly. I get my share of crashes due to my running beta custom roms n everything on my android, but never did i on stock!

Lucky you because my Android stock stuff crashed constantly.

[+Nik Louch Subscriber²]

My friend has a 3GS, and it hasn't crashed sice last August

So in that time it hasn't once shelled out of an app?

In that time it hasn't once not started an app after clicking the icon?

Those are crashes, and I don't believe you!

[Astra.Xtreme]

Haven't had a single crash on my 4S. And I've had one maybe two crashes of Safari on my iPad 2. I'm pretty surprised at such high percentages.

[Scorbing]

So in that time it hasn't once shelled out of an app?

In that time it hasn't once not started an app after clicking the icon?

Those are crashes, and I don't believe you!

Why don't you believe him? I haven't had a single crash on my iPhone 4S since I've had it and I bought it when it came out and updated it to latest iOS.

[techbeck]

I don't know where they got that data from or who made it up, but I have NEVER had iOS 5 crash on me...ever. When I had my Android HTC phone it was a nightmare. Constant force-closes, crashes, freezing, slow downs. I dumped it and went back to iOS (iPhone) and I haven't looked back. And no, I am not an Apple fan-boy, but the truth is the truth. iOS and iOS appz simply work better so I am sorry but I do not believe those charts at all. Seems someone is trying to make Android look better.

So since you have no crashes, the data is made up? I have had no crashes on my Galaxy Nexus either but like the iPhone, each user experience differs from another.

[thommcg]

More here on Crittercism;

http://www.critterci...m/features.html

Don't forget, this is only based on Apps that Crittercism tracks.