Canonical Developer Criticizes Linux Mint's Security


Recommended Posts

Original article:
 

While Linux Mint is derived from Ubuntu's package-set, a Canonical developer has criticized the popular Ubuntu derivative for its handling of packaging upgrades that could leave the system in a vulnerable state.

Ubuntu Linux developer Oliver Grawert had originally pointed out that security updates from Ubuntu don't necessarily get down to Linux Mint users since changes from X.Org, the kernel, Firefox, the boot-loader, and other core components are blocked from being automatically upgraded. Linux Mint doesn't send down updates for some Ubuntu packages automatically due to having their own customizations, etc.

The list of update rules can be found via GitHub as pointed out by Oliver in a follow-up post. Grawert explained, "this is the list of packages it will never update, instead of just integrating changes properly with the packagaes in the ubuntu archive they instead suppress doing (security) updates at all for them. i would say forcefully keeping a vulnerable kernel browser or xorg in place instead of allowing the provided security updates to be installer makes it a vulnerable system, yes. i personally wouldn't do online banking with it ;)"

Another Ubuntu developer, Benjamin Kerensa, added, "It is unclear why Linux Mint disables all of their security updates although to some degree they have tried to justify their disabling of kernel updates by suggesting that such updates could make a system unstable and that normal users shouldn't get these kinds of updates. Anyways it is something that might be better researched on their forums since people have asked a few times over the past couple years, and they probably have a better idea than us. I can say that it took them many months to get a fixed version of Firefox packaged while Ubuntu and Debian had already had security fixes in their package. This puts Linux Mint users at risk and is one of the key reasons I never suggest Linux Mint to anyone as an alternative to Ubuntu."

 

Source: http://www.phoronix.com/scan.php?page=news_item&px=MTUxNzY

 

 

Linux Mint's response:

 

 

I hear a Canonical dev was more opinionated than knowledgeable and the press blew what he said out of proportion. I wouldn?t mind too much, if we weren?t finding ourselves answering questions from panicked users rather than working on what matters right now (i.e. Mint 16 RC). So I?ll be brief.

 

About package updates:

  • We explained in 2007 what the shortcomings were with the way Ubuntu recommends their users to blindly apply all available updates. We explained the problems associated with regressions and we implemented a solution we?re very happy with.
  • Anybody running Mint can launch Update Manager -> Edit -> Preferences and enable level 4 and 5 updates, thus making their Linux Mint as ?Secure? and ?Unstable? as Ubuntu.

About Firefox updates:

  • Linux Mint uses the same Firefox package as Ubuntu from the same repository. Firefox is a level 2 update so every Mint user receives it by default.
  • LMDE, which is not based on Ubuntu, uses its own Firefox package. We?ve been slow in updating it by the past in LMDE (and that?s probably what confused the Canonical developer) but we took action and automated that. Firefox 25 was released on the 29th of October and updated in LMDE on the 30th.

I personally talked to the legal dept. at Canonical (for other reasons, they?re telling us we need a license to use their binary packages) and it is clear they are confused about LMDE and Mint. They don?t know what repositories we?re using and they don?t know what we?re doing. We?re 2 years younger than them and they have no idea how many users we have (they use http://stats.wikimedia.org/archive/squid_reports/2013-10/SquidReportOperatingSystems.htm but don?t realize our user agent is ?Ubuntu? since the days of Firefox 4 ? Mint 9 if I remember correctly).

 

I don?t really mind what people at Canonical understand or do not understand about us. I understand why the press and media sell controversy. I just really don?t want to waste time with this.

 

Source: http://segfault.linuxmint.com/2013/11/answering-controversy-stability-vs-security-is-something-you-configure/

 

 

Although Canonical has made some very public mistakes recently (some very recently), I actually have to agree with the Canonical developer's criticisms on this one. I had no idea that Linux Mint imposed such a foolish security policy. Admittedly I have been critical of Linux Mint in the past for perceived technical debt, but this seems like an undeniable, blindly simple blunder. Furthermore, the project's response does nothing to allay my concerns. Is there something I missed, or is Linux Mint really justified in their current security policy?

Link to comment
Share on other sites

Their response is quite clear on the matter to me.

They don't automatically apply new patches before they're tested to be stable, but users who want to be fully patched can be with a single setting change(I can understand how this confuses Ubuntu, since they don't have a single easily changeable system setting any more...)

I wonder if Ubuntu LTS also applies these patches right away...

Link to comment
Share on other sites

Lots of distros do the same thing - I'm not really sure why he's calling Mint out on it. It should be down to systems admins/users to choose which updates they wish to install. Heck, I'm pretty sure that if you install Windows 8, unless you explicitly tell it to, it won't automatically install updates either. Mac OS does the same thing - it asks. It doesn't just assume that you want every update by default.

 

The package/dependency system *should*, in theory, just allow you to update whatever like Canonical do - however, in practice, blindly applying updates has a nasty habit of breaking things or causing issues (i.e. if a dev states he needs v=1.0 rather than v>=1.0).

 

TBH, as someone who developed for Ubuntu for a while - I really do not like how Canonical operates. Ubuntu is basically a way for Canonical to sell their corporate services these days.

Link to comment
Share on other sites

this reminds me so much of the debates if the close/minimize buttons should be on the left or right side of a window :rofl:

Link to comment
Share on other sites

Lots of distros do the same thing - I'm not really sure why he's calling Mint out on it. It should be down to systems admins/users to choose which updates they wish to install. Heck, I'm pretty sure that if you install Windows 8, unless you explicitly tell it to, it won't automatically install updates either. Mac OS does the same thing - it asks. It doesn't just assume that you want every update by default.

I understand that some system administrators prefer to test updates first. Given Microsoft's spotty history of security updates breaking things, sometimes that policy makes sense. The stand-alone security updates Apple releases for OS X very rarely, if ever, cause more problems, but the massive combo updates they are so fond of releasing sometimes do. Therefore I can definitely why even OS X system administrators like to extensively test updates first. However the situation is very different with stable Linux distributions. The whole point of a stable distribution like RHEL, Debian, or to a lesser extent Ubuntu, Fedora, SUSE, and others is to provide that buffer against upstream, and thereby absolute stability. Unlike with Windows or OS X, if you don't trust your Linux distribution to not break things in updates to their stable releases, you are welcome to choose a more suitable distribution. For example, I use the latest Debian stable release on my server. I trust Debian implicitly. I install every update as soon as it is released without hesitation. I have complete confidence that nothing will break, because that is the whole point of Debian doing stable releases.

 

The package/dependency system *should*, in theory, just allow you to update whatever like Canonical do - however, in practice, blindly applying updates has a nasty habit of breaking things or causing issues (i.e. if a dev states he needs v=1.0 rather than v>=1.0).

Building on my previous point, the dependency issues you describe will never happen in stable Ubuntu releases as long as you are exclusively using software from the official Ubuntu repositories. Although Ubuntu releases are not truly frozen like Debian Stable (Firefox and a handful of other important user-facing packages receive version updates for the lifetime of each Ubuntu release), Canonical takes care of the dependency issues involved in backporting newer software. If anything in the archive breaks when they update one of its dependencies, they take care of that before the updates are pushed to users.

In any case, the problem being discussed here is Linux Mint's security, not miscellaneous bug fixes. It is perfectly within their rights to block bug fixes by default, no matter how trivial they may be. However security updates are another matter entirely. Blocking security updates with no alternative seems like negligence. Unlike in a corporation where system administrators do this on a regular basis, like you pointed out, Linux Mint has no right to make this decision for their users. People use Linux Mint with the expectation of security. As it stands, the project is not addressing that expectation in a timely fashion, which is almost as bad as not addressing it at all. Oliver Grawert's proof of this seems pretty irrefutable and damning.

 

TBH, as someone who developed for Ubuntu for a while - I really do not like how Canonical operates. Ubuntu is basically a way for Canonical to sell their corporate services these days.

While I agree with you to an extent, I don't discount the sincerity or commitment of the Ubuntu Security Team. Based on the fact that Canonical uses Ubuntu to sell their corporate services, it appears to be in their best interest to keep Ubuntu secure and stable, in direct opposition to Linux Mint's claims. Linux Mint deciding that they don't like the way Ubuntu does things is an understandable sentiment, but the project either needs to provide a reasonable alternative or just put up with Canonical's policies. Especially now that the Internet is all-but ubiquitous, security is more important than ever.

Link to comment
Share on other sites

I understand that some system administrators prefer to test updates first. Given Microsoft's spotty history of security updates breaking things, sometimes that policy makes sense. The stand-alone security updates Apple releases for OS X very rarely, if ever, cause more problems, but the massive combo updates they are so fond of releasing sometimes do. Therefore I can definitely why even OS X system administrators like to extensively test updates first. However the situation is very different with stable Linux distributions. The whole point of a stable distribution like RHEL, Debian, or to a lesser extent Ubuntu, Fedora, SUSE, and others is to provide that buffer against upstream, and thereby absolute stability. Unlike with Windows or OS X, if you don't trust your Linux distribution to not break things in updates to their stable releases, you are welcome to choose a more suitable distribution. For example, I use the latest Debian stable release on my server. I trust Debian implicitly. I install every update as soon as it is released without hesitation. I have complete confidence that nothing will break, because that is the whole point of Debian doing stable releases.

 

 

what.... linux is in no way inherently more stable than windows. and yo don't know if these patches cause instability in the wild until it has been tested. again, there's literally an unlimited amount of hardware combinations, then add in the software and driver combos. 

Link to comment
Share on other sites

what.... linux is in no way inherently more stable than windows. and yo don't know if these patches cause instability in the wild until it has been tested. again, there's literally an unlimited amount of hardware combinations, then add in the software and driver combos. 

 

I am not claiming that Linux is more stable or secure than Windows. However unlike Windows, most Linux distributions are not as tightly coupled. Therefore security updates outside of the kernel and firmware have a negligible chance of instigating hardware-related issues. Also unlike Windows, I am free to download the source code, look at the change log, and see exactly what changed for each and every update if I so desire. While I certainly do not expect every user to do so, it is a perfectly reasonable expectation for a distribution like Linux Mint to take this step and fix any issues they find before pushing the updates if they do not completely trust Canonical. If the project withholds updates, those updates still aren't being tested on a wide range of hardware. Even if that really is a key concern, which I would debate, withhold updates still does nothing to mitigate the core discrepancy while simultaneously exasperating the security issue.

Link to comment
Share on other sites

The way things are going with Ubuntu & decisions by Canonical as of late - I can really see LMDE becoming the mainstream Mint flavour in the future (especially over the whole Mir debacle).

Link to comment
Share on other sites

I am not claiming that Linux is more stable or secure than Windows. However unlike Windows, most Linux distributions are not as tightly coupled. Therefore security updates outside of the kernel and firmware have a negligible chance of instigating hardware-related issues. Also unlike Windows, I am free to download the source code, look at the change log, and see exactly what changed for each and every update if I so desire. While I certainly do not expect every user to do so, it is a perfectly reasonable expectation for a distribution like Linux Mint to take this step and fix any issues they find before pushing the updates if they do not completely trust Canonical. If the project withholds updates, those updates still aren't being tested on a wide range of hardware. Even if that really is a key concern, which I would debate, withhold updates still does nothing to mitigate the core discrepancy while simultaneously exasperating the security issue.

 

if looking at the source code revealed all bugs, and let you see how interaction with thousands of other lines of code, apps, hardware and drivers cause it to react. then there would be no bugs and stability issues in any OS though ;)

Link to comment
Share on other sites

if looking at the source code revealed all bugs, and let you see how interaction with thousands of other lines of code, apps, hardware and drivers cause it to react. then there would be no bugs and stability issues in any OS though ;)

 

I am not claiming that either. There is no way to completely eliminate bugs; otherwise there would be no need for software updates or a dedicated security team. On the other hand, updates to a stable Linux distribution tend to be relatively small. Between the changelog and diff of the source-level changes, I can maintain a relatively high degree of certainty that any given update introduces no major bugs (as long as I can read and understand the code). Also if there is something that I think needs to be fixed, I can revert the offending changes or patch the software myself. While no software is perfect, access to the source code affords a much greater degree of control - and therefore confidence - than opaque binary-only updates, especially if those updates come from an organization with a checkered support history.

Link to comment
Share on other sites

Although Canonical has made some very public mistakes recently (some very recently), I actually have to agree with the Canonical developer's criticisms on this one. I had no idea that Linux Mint imposed such a foolish security policy. Admittedly I have been critical of Linux Mint in the past for perceived technical debt, but this seems like an undeniable, blindly simple blunder. Furthermore, the project's response does nothing to allay my concerns. Is there something I missed, or is Linux Mint really justified in their current security policy?

I personally talked to the legal dept. at Canonical (for other reasons, they?re telling us we need a license to use their binary packages) and it is clear they are confused about LMDE and Mint.

I get what Canonical's saying with their complaint, it does make sense as there's a fair number of CVE bulletins that get issued for those components just like for everything else.. if Mint's merging the updates themselves that's one thing, but if not, that's a problem, especially if it's an ugly vulnerability.. just as an extreme over-dramatic worst-case example, that SSL glitch that lead to rootkits on a few high profile servers. I'd want that fix ASAFP personally.  Out of curiosity though, why would Mint need a special license to use Canonical's binary packages? Isn't that all GPL'd?

 

Oh hay, grats on going green by the way.

Link to comment
Share on other sites

I am not claiming that either. There is no way to completely eliminate bugs; otherwise there would be no need for software updates or a dedicated security team. On the other hand, updates to a stable Linux distribution tend to be relatively small. Between the changelog and diff of the source-level changes, I can maintain a relatively high degree of certainty that any given update introduces no major bugs (as long as I can read and understand the code). Also if there is something that I think needs to be fixed, I can revert the offending changes or patch the software myself. While no software is perfect, access to the source code affords a much greater degree of control - and therefore confidence - than opaque binary-only updates, especially if those updates come from an organization with a checkered support history.

 

And this is different from windows how ? stable and I can revert any change that causes an issue.

 

Either way the point is that the updates aren't added because Mint don't consider them safe for use, and anyone using ubuntu can attest to that. and if you do want all the updates you can change the setting. 

Link to comment
Share on other sites

Security.

 

this seems like an issue that the folks who develop on the debian branch need to address together. anyone makes an update that is for the betterment of the whole, pass it upstream to be added in. from my reading, hoping I'm NOT mistaken, developers of the different flavors of the same distro aren't working nearly hard enough to bring some collective stabilization. Canonical is a big player nd they have some issues to get ironed out

Link to comment
Share on other sites

I get what Canonical's saying with their complaint, it does make sense as there's a fair number of CVE bulletins that get issued for those components just like for everything else.. if Mint's merging the updates themselves that's one thing, but if not, that's a problem, especially if it's an ugly vulnerability.. just as an extreme over-dramatic worst-case example, that SSL glitch that lead to rootkits on a few high profile servers. I'd want that fix ASAFP personally.

That is the problem as I see it too. It's not that Linux Mint is merging the security updates themselves, they are completely ignoring them for "stability" purposes. Linux Mint does not maintain a complete copy of the Ubuntu repository with their own modifications. Instead, Linux Mint uses the Ubuntu repositories directly for most packages, and merely maintain their own repository for software they package themselves (similar in concept to using a PPA to distribute their packages like the Elementary OS Team). Therefore the policy this controversy is over stays some updates from the Ubuntu repository (which Linux Mint is using directly) so they are not installed by Linux Mint users (by default).

 

Out of curiosity though, why would Mint need a special license to use Canonical's binary packages? Isn't that all GPL'd?

As far as I known, they don't. It could be that Canonical made a mistake again, or it could simply be a case of miscommunication. I am more inclined to believe the latter than the former. In the words of Napoleon Bonaparte, "Never ascribe to malice that which is adequately explained by incompetence." The concept is similar, and I think it applies with equal gravitas.

 

Oh hay, grats on going green by the way.

Thanks! I'm still learning, but hopefully I can use my new position help Neowin continue to be a fantastic community.

Link to comment
Share on other sites

I understand what they guys are saying, but at the same time I kinda feel bad for them picking on Mint. Especially that they won't recommend Mint as an alternative to Ubuntu or for banking as they mentioned, I think that was a bit mean-spirited from Canonical. Maybe i'm wrong.

Link to comment
Share on other sites

It is important to note that although the response was officially on behalf of Linux Mint, the Ubuntu developers who levied the original claims were not speaking on behalf of Canonical. People have their own opinions. The aforementioned Ubuntu developers were merely expressing theirs. Also, I would prefer to debate this issue on its merits. There is no need to disparage Canonical, Linux Mint, or any of the individuals involved.

Link to comment
Share on other sites

And this is different from windows how ? stable and I can revert any change that causes an issue.

While you can revert updates in Windows, that is your only recourse. You cannot bisect a bug or fix it yourself. You also cannot be sure exactly what changed, further limiting your analysis of the problem and potential mitigations. The Windows XP svchost issue warwagon posted about recently is a perfect example. The 10 page thread it spawned with complicated mitigation suggestions and no clear is solution after several months, despite the scope of the issue and high level of attention it received, indicates that it is clearly more than a transient mirage.

However, we are getting off topic. I did not mean to spark a debate about Windows or Microsoft's update practices with my comment about its update history. I was merely using it as an example, a point of comparison. My experience with "Microsoft's spotty history of security updates" is clearly subjective. I meant no ill will. I merely used it as a nonessential observation to support my point.

 

Either way the point is that the updates aren't added because Mint don't consider them safe for use, and anyone using ubuntu can attest to that. and if you do want all the updates you can change the setting.

Although Linux Mint may not consider all Ubuntu updates safe for use, it is equally as subjective to claim that all Ubuntu users have the same experience. If that were the case, I imagine that Ubuntu would not be able to sustain their userbase. I never incurred any stability problems when I ran Ubuntu, and my friends who still use it regularly don't have that that type of issue either.

The reason this is such a big issue, is not that Linux Mint forbids its users from receiving the blocked updates, it is that they don't allow it by default. To quote Matt Hartley, the co-host of the Linux Action Show, "Defaults rule the world."

 

this seems like an issue that the folks who develop on the debian branch need to address together. anyone makes an update that is for the betterment of the whole, pass it upstream to be added in. from my reading, hoping I'm NOT mistaken, developers of the different flavors of the same distro aren't working nearly hard enough to bring some collective stabilization. Canonical is a big player nd they have some issues to get ironed out

Although there are indeed issues that Debian derivatives need to consider, security of the main archive is not normally among them. Most derivatives use the Debian repository for their target release directly, and merely supplement it with packages from their own repository. Therefore as long as they don't set unconventional priorities like Linux Mint apparently did, they will receive security updates directly from Debian. Each derivative only needs to keep on top of updates to the packages in their own repository. In this way derivatives are encouraged to contribute back to Debian, and since it is generally within their best interest to do so, many do. Ubuntu is the exception rather than the rule in this regard. Canonical does maintain a complete repository for Ubuntu which is completely independent of Debian. However, Canonical has the resources and expertise to track updates and keep on top of patches like few others do, especially on the scale of Ubuntu.

Link to comment
Share on other sites

I think the Ubuntu guys do have a point (to an extent), but then so do the Mint guys. There's always going to be a lead time between receiving a new patch and integrating it in those cases where the Mint team make customizations, and therein is the security risk. Realistically there's probably never going to be an exploit that is used successfully within the day or two between the Ubuntu fix and the Mint fix. But, even so, the risk is stil there. Cases where there is a large lead time (I'm surprised that there was ever a "several month" discrepancy between Ubuntu and Mint updates) are indeed a problem IMO, but reading what Mint have said that is no longer a problem.

I'm siding with the Mint team on the whole "fixes should be immediately integrated" issue though. Blindly accepting patches into downstream is never a good idea, even for security fixes. The patch itself may require modifications, so patching without a review is possibly going to undo someone's work, or worse, introduce regressions.

To be honest, I think the Ubuntu devs need a bit of a kick up the arse. They seem to be running their mouths off a lot about things that they don't really understand, which inevitably causes embarassment. I'm inclined to say that it's a case of stupidity rather than malice, but people thinking otherwise would be forgiven for thinking that Canonical seem to be throwing around a lot of FUD these days, even if not in an official capacity.

  • Like 2
Link to comment
Share on other sites

But... but... Linux doesn't need security, unlike Windoze!

 

/s

 

TBH, I don't think Canonical is exactly thorough in their security, either. :/ 

Link to comment
Share on other sites

TBH, as someone who developed for Ubuntu for a while - I really do not like how Canonical operates. Ubuntu is basically a way for Canonical to sell their corporate services these days.

 

Point taken, but Canonical is a for-profit company, so they do exist to make money. I tend to prefer donating to non-profits such as SPI (www.spi-inc.org/), who help support Debian and other projects. It just feels right.

Link to comment
Share on other sites

Here we go ... I've seen them Mint fanbois clashing with Ubuntu's on a forum (forgot wich exactly) , a true breed war . They are both right , the average Joe won't look for Level 4-5 clearance/update points and Ubuntu won't fix their **** very soon. Instead of uniting and solving problems together , they just throw meatballs on each other - thats the problem with Linux : too fragmented.

Link to comment
Share on other sites

This topic is now closed to further replies.
  • Recently Browsing   0 members

    • No registered users viewing this page.