Study finds open source is slightly better than closed source software

The open source code quality has improved, according to the new Scan Open Source Integrity Report from Coverity. The study compares more than 37 million lines of open source code against 300 million lines of closed source code from a sample of unknown users of the company products, and finds that when the codebase is on par the open source software contains fewer defects than its proprietary counterpart.

Coverity test platform is able to scan code hunting for known defects, and its latest upgrade let it find even more bugs. According to data contained in the scan report, for an average size of 7.5 million lines of code, closed source software includes 0.64 defects per thousand lines.

In comparison, open source software contains 832,000 lines of code on average and 0.45 defects per thousand lines. The study found that when closed and open source projects are almost the same size, they are basically on par as for defects in the code with the open source side performing slightly better on that regard.

Coverity states that the seven million lines of Linux code, for example, have just 0.62 defects per thousand lines and can be considered – together with PHP 5.3 and PostgreSQL 9.1 – among the best quality (open source) software the industry has to offer right now.

During the latest years, Coverity’s Zack Samocha noted, open source has improved in code quality a lot and  “The line between open source and proprietary software will continue to blur over time as open source is further cemented in the modern software supply chain”.

Report a problem with article
Previous Story

Doom 4 cancellation rumors denied by Bethesda Softworks

Next Story

Windows Phone not affected by smartphone exploit

44 Comments

Commenting is disabled on this article.

better code or not.. linux still blows because there isn't just one distro. there is no consistancy. You programming a KDE app and your risk it not working on gnome. You could code for straight X11 but risk your program being bland. The distros are so varied it's almost impossible to distribute a program without the code and compiling it your self. I'd love to see grandma do a
./configure
make

and then deal with the errors of not having the right libaries... or the right version of the libraries to make the damn thing work.

i'll go closed source. IT JUST WORKS.

I'm not sure how they're comparing the two. Where are they getting the closed-source code to scan and compare?

correct me if I'm wrong, but wouldn't the "defects" in opensource software (assuming, of course, that defect would mean a security vulnerability) count more because it's easier for an attacker to discover them, and thus exploit them? my 2 cents ;D

PHP could be one of the best open source software, but its totally useless. I had to work with PHP and it was horrible and it only brought me a depression with suicidal ideas. On the other hand ASP.NET is just wonderful piece of work that "just works"!

When I was web coding, I started learning in ASP. When ASP.NET came out, I nearly pulled all my hair out over it. I just didn't like the approach. I picked up PHP and for me it was better.

Palpatine said,
PHP could be one of the best open source software, but its totally useless. I had to work with PHP and it was horrible and it only brought me a depression with suicidal ideas. On the other hand ASP.NET is just wonderful piece of work that "just works"!

Agree... using code behind is the best way to develop web applications

Wow, 37 million with 300 million... and so, if i compare 37 microwave ovens with 300 the latter group will have more warranty issues, WOW!

At least PRETEND that the study is impartial.

Opensource is a bit better because the source code is open and any developer can contribute to fixing the program if it has bugs and stuff compare to closed source.. Close source is better in terms of security since not everyone can access to it. But with close source people have to wait for days for the updates to happen compare to opensource it is like instant or like within days.

Chica Ami said,
Opensource is a bit better because the source code is open and any developer can contribute to fixing the program if it has bugs and stuff compare to closed source.. Close source is better in terms of security since not everyone can access to it. But with close source people have to wait for days for the updates to happen compare to opensource it is like instant or like within days.

So what operating system does Secunia rate as most secure because of how fast vulnerabilites are addressed and patches release?

Hm... Seems like a jaded study. Shouldn't they at least pretend they're not and compare the same number of lines of code at the very least?

Since they cannot disclose the "closed source" source, this report is worthless.
The open source sample is probably composed of the most well-known open-source apps; you'd expect their code quality to be very high. Everyone can know what you did on a FOSS project...you'd better write good code.
On the other hand, it's easier to write bad code if nobody will know you did it. Or if your company just doesn't care about bad code as long as it works (e.g. crapware preinstalled on computers, or kernel-mode drivers which don't need to be in kernel mode...).

Aethec said,
Since they cannot disclose the "closed source" source, this report is worthless.
The open source sample is probably composed of the most well-known open-source apps; you'd expect their code quality to be very high. Everyone can know what you did on a FOSS project...you'd better write good code.
On the other hand, it's easier to write bad code if nobody will know you did it. Or if your company just doesn't care about bad code as long as it works (e.g. crapware preinstalled on computers, or kernel-mode drivers which don't need to be in kernel mode...).

When we are under pressure to deliver something then the coding standards and best practices no longer matter. You have a deadline to meet and the last thing you care is if you missed a comment here or there. Open-source projects never run with deadlines and to be fair neither do Microsoft projects (mentioning M$ for the sake of argument here). When you look at agencies an consultancies then yes, cheap labour to make it more profitable = bad skill => bad code and a budget which is 100% over to fix it. Plus the pressure to deliver really fast.

Riva said,
Open-source projects never run with deadlines and to be fair neither do Microsoft projects...

Many open source projects have deadlines. Open source does not imply random people sitting on couches coding when they feel like it.

Meph said,
Define "defect".

Defect is a term used to define three categories of flaws in software:
1) Crashing, locking, incorrect result
2) Not meeting the original requirement
3) Defect due to requirements analysis gap (meaning the original requirements have nothing to do with the end users reality)

Riva said,

Defect is a term used to define three categories of flaws in software:
1) Crashing, locking, incorrect result
2) Not meeting the original requirement
3) Defect due to requirements analysis gap (meaning the original requirements have nothing to do with the end users reality)

Oh, I see. Thanks.

A software application that scans for defects... OK, I can see how looking for certain coding styles or patterns may be recognised, but for every flaw, there are countless ways around them. So simply stating "this makes use of this flaw" doesn't mean it is flawed, unless you, as a human, go through it and see "oh, wait, it has this over here to counter that". Now I'm sure you can fill in a list of different methods of patterns to look for these fixes, but a lot of them can be so heavily bespoke that they don't fall into any of these patterns, yet still work in a very stable manner.

Personally, I don't agree with the analysis.

To bad we have to pay to get the full report.
I call this BS, there's no way to know what they compared. I find it hard to believe that they have found 10 more closed source than the opened one.

By the looks of it it seems they've compared the biggest open source softwares... to presumably smaller, more unknown closed source solutions. I can hardly see an reputable company just giving them their closed source to look through.

and yet most look like they came form the 90s. I know looks arnt everything but looks do drive a product to sell for average users. An average user whoisn't a techie may look at a software that was made in 2011 and think it is old because the developers chose to use out dated icons

In terms of quality I've found PDF readers, disc burners and media players to be of good quality. But as you say the icons used look ancient in some, skins are available but first impressions are crucial. Heck on many of their webpages the screenshots are taken in XP for a release done this year. And it would be nice of these projects to add jump list support for Win 7-8.

CMG_90 said,
and yet most look like they came form the 90s. I know looks arnt everything but looks do drive a product to sell for average users. An average user whoisn't a techie may look at a software that was made in 2011 and think it is old because the developers chose to use out dated icons

Looks may / may not be essential for making a sale, however the vast majority of open-source software the end user uses is free so therefore no sale occur.

Sadelwo said,
In terms of quality I've found PDF readers, disc burners and media players to be of good quality. But as you say the icons used look ancient in some, skins are available but first impressions are crucial. Heck on many of their webpages the screenshots are taken in XP for a release done this year. And it would be nice of these projects to add jump list support for Win 7-8.

True. Many of the web pages were created with in 5 minuntes.

thealexweb said,

Looks may / may not be essential for making a sale, however the vast majority of open-source software the end user uses is free so therefore no sale occur.

It isn't about creating sales, it is about usability issues that are far deeper and more flawed than just the dated appearance, in addition to performance issues.

A lot of OSS developers spend too much time in their niche, and don't stop to see what the rest of the world is doing and why. I still run into Linux zealots that haven't seriously used Windows since their computer came with Windows 98.

When you don't at least give what you consider the 'competition' respect enough to 'see' what they are doing, and explore why and how their stuff possibly could be doing something better, you eventually fall generations behind.

In teaching, it is painful when encountering hard core OSS users that are still trapped in the coding technologies of the 1980s that they 'believe' are 'the only/right way' to do things.

My generation shoved hard to move away from dated and obsolete coding methodologies and sadly OSS has made them popular again.

Not sure what kind of hardware you're running, but Linux works on my 3,5,1 year old PCs in my home. Chrome for web and there hasn't been a program that Windows has that Linux doesn't have. Except for PC games, but the same can be said for Mac and everyone that has a Mac loves Macs

Cute James said,

Adobe Creative Suite?
MS Office?

He probably means there isn't an equivalent for, and will promptly state that GIMP and Libre Office are completely equivalent, if not superior.

Memnochxx said,

He probably means there isn't an equivalent for, and will promptly state that GIMP and Libre Office are completely equivalent, if not superior.

At which point we will all laugh and move on.

Memnochxx said,

He probably means there isn't an equivalent for, and will promptly state that GIMP and Libre Office are completely equivalent, if not superior.

In 1991, yes they would have been almost equivalent.

I hope you weren't really being serious... If you were, I apologize for my jest and hope you someday recover from whatever brain injury you incurred.

thenetavenger said,
I hope you weren't really being serious... If you were, I apologize for my jest and hope you someday recover from whatever brain injury you incurred.

I can safely say that he was being sarcastic.