The mythical "twice as efficient" application
04 November 2012
I watched several talks from Build 2012 this week – fascinating event. I was pleasantly surprised to see the people behind MonoGame and Unity there, as well as Brandon Bray’s talk on the future of .NET. For all the fear-mongering there has been around Microsoft abandoning .NET in favor of Javascript, C++, or both, the future is in fact looking brighter than ever for .NET. Yes, Hejlsberg is still working on C#! (phew!)
There was much talk about C++ as well, as Microsoft is investing heavily in supporting C++ and improving its compilers and tools for that language. Herb Sutter gave a very interesting talk about the future of C++, devoid of any of the dubious statements about .NET he made last year.
But then, there was this interview (I can't find it anymore) with the guy who did all the interviews and whose name I can’t remember at the moment, and Herb Sutter started to talk about .NET. A lot. Here are some excerpts (not nearly everything) of what he said about C++ vs .NET (the quotes are long, but at least you can’t say I’m quoting out of context):
My main problem with Sutter’s statements, and the reason I decided to write this, is this “twice as efficient application” he keeps referring to. Just what is it? Is it the average C++ application compared to the average C# application? Is it some utopian scenario, given access to the very best developers and infinite development time? Is it implied that by simply choosing C++ as your implementation language rather than C#, your application will be twice as efficient? It’s a vague, caricatural and misleading picture, yet he keeps repeating this as if it bore some profound truth.
He then goes on to say “if you want efficiency, consider an efficient language” – that is, in context, consider C++ instead of C#. So what does that make C#, an inefficient language? I guess if C++ gives you “twice as efficient” apps, then sure, C# must be pretty darn inefficient. Maybe that's not what he meant, but the way he put it makes it hard to think otherwise.
Except that’s wrong. If anything, given the same development time, C++ gives you less efficient apps, because C++ makes it entirely up to you to make things efficient (and makes it very difficult at that), while .NET provides you with a bunch of services and libraries that are already well-optimized. Chances are .NET’s GC is way faster than whatever manual technique you’re using to allocate and free memory in C++ - unless you really know what you’re doing and you’re optimizing very specifically for the application you’re building. That’s why Frostbite 2 is coded in C++ - and that’s also why you don’t need C++ for your typical Windows Store or even Desktop application.
C# and C++ are in fact very similar at a low level. They’re both imperative, C-based languages that compile to very efficient native code. They support much of the same structures which do very much the same thing. Here’s a rather striking example: this is code that compiles as both C# and C++, and its corresponding assembly for both languages at their highest respective levels of optimization. Can you tell which is which?
Given up yet? The first one is C++ and the second is C# . This kind of tight loop is the domain where C++ excels, and yet this simple code illustrates that even there, and in common cases, C# can yield the same results, while offering the guarantee of defined behavior in case of out-of-bounds array access (here, the optimizer eliminated the bounds check through static analysis).
Yes, use the right tool for the job – to me, that means don’t use a complicated, unsafe, poorly tooled language unless you have very specific needs only that language can satisfy. No, that’s certainly not “very often” – most applications certainly don’t need highly tuned custom memory allocation schemes, or to take advantage of every platform-specific CPU instruction. And besides these cases, I don’t see what C++ really gives you to offset a massive decrease in productivity and the corresponding effect on the team’s morale.
Lastly I’d like to address this blanket statement – that C++ is “THE portable language, always has been”. That depends on what your definition of portable is. If by portability, you mean the ability to use the same language on many different platforms, then sure, C++ is one of the most portable languages. A more useful definition of portability, at least from my perspective as a developer, is the ability to re-use the same code with little to no change on many different platforms – and here, C++’s low level nature and barebones runtime runs contrary to this goal. The very point of using C++ is getting closer to the machine – but the closer your code is to the machine, the more machine-specific, and thus, less re-usable it becomes. If you’re going to use C++ for what it excels at, that is, making platform-specific optimizations, then you’ll have to redo that work for every machine you’re targeting. And that’s putting aside the various discrepancies between compilers since, as Herb Sutter said himself, there is no standards-compliant implementation of C++, and the varying quality of standard library implementations.
To conclude, I’m happy to see Microsoft invest in a variety of languages, including C++ which will continue being relevant and necessary. Many existing and upcoming applications will require the level of power and control only C++ provides, and Microsoft's commitment to push this language forward across all their platforms is great news. That said, besides dubious statements such as Herb Sutter’s here, I find the level of emphasis put on C++ out of place and worrying. For games in particular, Microsoft’s current advice is simply misleading. Their “Developing games” for Windows Store documentation starts with the following: “If you want to use advanced game development technologies, such as DirectX Graphics and Gaming, you must know Windows and C++ programming” (emphasis mine). That’s simply not true; with SharpDX providing an up-to-date, lightweight wrapper over the entire DirectX 11 API, all .NET languages can be used to develop Windows Store games – and it’s certainly a favorable option over C++ for everything but the most demanding games. Also, while XNA seems abandoned for the time being, MonoGame makes it very easy to port existing XNA games to the Windows Store. “Using the right tool for the job” begins with being aware of the existing tools.
There was much talk about C++ as well, as Microsoft is investing heavily in supporting C++ and improving its compilers and tools for that language. Herb Sutter gave a very interesting talk about the future of C++, devoid of any of the dubious statements about .NET he made last year.
But then, there was this interview (I can't find it anymore) with the guy who did all the interviews and whose name I can’t remember at the moment, and Herb Sutter started to talk about .NET. A lot. Here are some excerpts (not nearly everything) of what he said about C++ vs .NET (the quotes are long, but at least you can’t say I’m quoting out of context):
Quote
“Windows Phone, for example, now supports C++, which means C++ is supported on every major mobile OS, every major desktop operating system, every major server operating system, it is THE portable language, always has been. Along with C, but it gives you the abstraction, and still the efficiency, which is its big differentiator from C.”
“(…) One size doesn’t fit all. If you have your toolbox, you ask the question “why do you have a screwdriver and a saw, and not just a hammer? You should just have hammers!” That would be silly. The same thing is true of programming languages. They’re each designed for specific kinds of usages. Use the right tool for the right part of your job – very often that’s C++. “
“What’s being pointed out is, the pendulum has swung too far, we’re now getting more balance. We don’t want to get unbalanced the other way, and now say “oh you don’t really need all that managed stuff”. Of course you do, there’s great benefits to that, you should use that tool. But the balance is to, especially where you care about power efficiency, getting the most performance per watt, getting those performance per transistor on mobile devices, and having a twice as efficient app, run twice as long on battery. Have a data center where there’s a twice as efficient app, runs with half the cost of your data center. And remember, the server cost in power is 88% percent of the cost of running a data center. You run an app that’s twice as efficient, it is directly almost twice as cost-effective. Costs half as much. That is really important.”
“And there is still a place where you want to optimize for programmer productivity. [If] programmer time is your biggest cost, like you’re developing an internal IT application, or for some other reason programmer time’s your big cost, you should optimize for that. That’s what many languages are good for. If reach is your biggest thing, that’s where a web-based solution is going to play well. If you want local apps, (…) and you want efficiency, then consider an efficient language. Because C++ has always been about efficient abstractions. To get classes and templates, generics, all that kind of stuff, but having it compiled away to almost nothing. You don’t pay for what you don’t use. So, it’s one of your tools. “
“(…) more and more, especially in mobile and in data centers, yes C++ is being used more and more. Because you want your twice as efficient application to consume twice as less resources and run twice as long on the same battery. People love that. And that’s why you’re seeing native code. And you can write the rest of your application in some other language, often the framework you use will be implemented in native code, or use native code for at least the compute-intensive parts. But C++ is getting so easy to use, modern C++, which is not like it was a few years ago, is getting so easy to use that often you can just write the whole thing in C++ and you can whatever tool you want to… the right tool for the job.”
“(…) One size doesn’t fit all. If you have your toolbox, you ask the question “why do you have a screwdriver and a saw, and not just a hammer? You should just have hammers!” That would be silly. The same thing is true of programming languages. They’re each designed for specific kinds of usages. Use the right tool for the right part of your job – very often that’s C++. “
“What’s being pointed out is, the pendulum has swung too far, we’re now getting more balance. We don’t want to get unbalanced the other way, and now say “oh you don’t really need all that managed stuff”. Of course you do, there’s great benefits to that, you should use that tool. But the balance is to, especially where you care about power efficiency, getting the most performance per watt, getting those performance per transistor on mobile devices, and having a twice as efficient app, run twice as long on battery. Have a data center where there’s a twice as efficient app, runs with half the cost of your data center. And remember, the server cost in power is 88% percent of the cost of running a data center. You run an app that’s twice as efficient, it is directly almost twice as cost-effective. Costs half as much. That is really important.”
“And there is still a place where you want to optimize for programmer productivity. [If] programmer time is your biggest cost, like you’re developing an internal IT application, or for some other reason programmer time’s your big cost, you should optimize for that. That’s what many languages are good for. If reach is your biggest thing, that’s where a web-based solution is going to play well. If you want local apps, (…) and you want efficiency, then consider an efficient language. Because C++ has always been about efficient abstractions. To get classes and templates, generics, all that kind of stuff, but having it compiled away to almost nothing. You don’t pay for what you don’t use. So, it’s one of your tools. “
“(…) more and more, especially in mobile and in data centers, yes C++ is being used more and more. Because you want your twice as efficient application to consume twice as less resources and run twice as long on the same battery. People love that. And that’s why you’re seeing native code. And you can write the rest of your application in some other language, often the framework you use will be implemented in native code, or use native code for at least the compute-intensive parts. But C++ is getting so easy to use, modern C++, which is not like it was a few years ago, is getting so easy to use that often you can just write the whole thing in C++ and you can whatever tool you want to… the right tool for the job.”
My main problem with Sutter’s statements, and the reason I decided to write this, is this “twice as efficient application” he keeps referring to. Just what is it? Is it the average C++ application compared to the average C# application? Is it some utopian scenario, given access to the very best developers and infinite development time? Is it implied that by simply choosing C++ as your implementation language rather than C#, your application will be twice as efficient? It’s a vague, caricatural and misleading picture, yet he keeps repeating this as if it bore some profound truth.
He then goes on to say “if you want efficiency, consider an efficient language” – that is, in context, consider C++ instead of C#. So what does that make C#, an inefficient language? I guess if C++ gives you “twice as efficient” apps, then sure, C# must be pretty darn inefficient. Maybe that's not what he meant, but the way he put it makes it hard to think otherwise.
Except that’s wrong. If anything, given the same development time, C++ gives you less efficient apps, because C++ makes it entirely up to you to make things efficient (and makes it very difficult at that), while .NET provides you with a bunch of services and libraries that are already well-optimized. Chances are .NET’s GC is way faster than whatever manual technique you’re using to allocate and free memory in C++ - unless you really know what you’re doing and you’re optimizing very specifically for the application you’re building. That’s why Frostbite 2 is coded in C++ - and that’s also why you don’t need C++ for your typical Windows Store or even Desktop application.
C# and C++ are in fact very similar at a low level. They’re both imperative, C-based languages that compile to very efficient native code. They support much of the same structures which do very much the same thing. Here’s a rather striking example: this is code that compiles as both C# and C++, and its corresponding assembly for both languages at their highest respective levels of optimization. Can you tell which is which?
for (int i = 0; i < size; ++i) {
if (i % modulus == 0) {
arr[i] = i * i;
}
}
// for (int i = 0; i < size; ++i) {
00F91319 xor ecx,ecx
00F9131B test edi,edi
00F9131D jle 0F9133Fh
// if (i % modulus == 0) {
00F9131F mov esi,dword ptr [esp+10h]
00F91323 mov eax,ecx
00F91325 cdq
00F91326 idiv eax,dword ptr [esp+1Ch]
00F9132A test edx,edx
00F9132C jne 0F91336h
// arr[i] = i * i;
00F9132E mov eax,ecx
00F91330 imul eax,ecx
00F91333 mov dword ptr [esi+ecx*4],eax
// for (int i = 0; i < size; ++i) {
00F91336 inc ecx
00F91337 cmp ecx,edi
00F91339 jl 0F91323h
// for (int i = 0; i < size; ++i) {
0000000d xor esi,esi
0000000f test ebx,ebx
00000011 jle 0000002A
// if (i % modulus == 0) {
00000013 mov eax,esi
00000015 cdq
00000016 idiv eax,ecx
00000018 test edx,edx
0000001a jne 00000025
// arr[i] = i * i;
0000001c mov eax,esi
0000001e imul eax,esi
00000021 mov dword ptr [edi+esi*4+8],eax
// for (int i = 0; i < size; ++i) {
00000025 inc esi
00000026 cmp esi,ebx
00000028 jl 00000013
Given up yet? The first one is C++ and the second is C# . This kind of tight loop is the domain where C++ excels, and yet this simple code illustrates that even there, and in common cases, C# can yield the same results, while offering the guarantee of defined behavior in case of out-of-bounds array access (here, the optimizer eliminated the bounds check through static analysis).
Yes, use the right tool for the job – to me, that means don’t use a complicated, unsafe, poorly tooled language unless you have very specific needs only that language can satisfy. No, that’s certainly not “very often” – most applications certainly don’t need highly tuned custom memory allocation schemes, or to take advantage of every platform-specific CPU instruction. And besides these cases, I don’t see what C++ really gives you to offset a massive decrease in productivity and the corresponding effect on the team’s morale.
Lastly I’d like to address this blanket statement – that C++ is “THE portable language, always has been”. That depends on what your definition of portable is. If by portability, you mean the ability to use the same language on many different platforms, then sure, C++ is one of the most portable languages. A more useful definition of portability, at least from my perspective as a developer, is the ability to re-use the same code with little to no change on many different platforms – and here, C++’s low level nature and barebones runtime runs contrary to this goal. The very point of using C++ is getting closer to the machine – but the closer your code is to the machine, the more machine-specific, and thus, less re-usable it becomes. If you’re going to use C++ for what it excels at, that is, making platform-specific optimizations, then you’ll have to redo that work for every machine you’re targeting. And that’s putting aside the various discrepancies between compilers since, as Herb Sutter said himself, there is no standards-compliant implementation of C++, and the varying quality of standard library implementations.
To conclude, I’m happy to see Microsoft invest in a variety of languages, including C++ which will continue being relevant and necessary. Many existing and upcoming applications will require the level of power and control only C++ provides, and Microsoft's commitment to push this language forward across all their platforms is great news. That said, besides dubious statements such as Herb Sutter’s here, I find the level of emphasis put on C++ out of place and worrying. For games in particular, Microsoft’s current advice is simply misleading. Their “Developing games” for Windows Store documentation starts with the following: “If you want to use advanced game development technologies, such as DirectX Graphics and Gaming, you must know Windows and C++ programming” (emphasis mine). That’s simply not true; with SharpDX providing an up-to-date, lightweight wrapper over the entire DirectX 11 API, all .NET languages can be used to develop Windows Store games – and it’s certainly a favorable option over C++ for everything but the most demanding games. Also, while XNA seems abandoned for the time being, MonoGame makes it very easy to port existing XNA games to the Windows Store. “Using the right tool for the job” begins with being aware of the existing tools.
0
, your post went under my radar with the keyword "SharpDX", and as the subject of your post also resonates with my personal experience (I even wrote a post recently "
Still, I have some pending concerns with C# JIT, the lack of static AOT, the lack of SIMD or most of a time a code a but less eficient when it comes to more complex loop with floating point calculations. The fact is that CLR team has not been working a lot for the past few years to provide a better optimized code in C# (or options to generate it), and this is an issue when you want to entirely leverage on C# and you want to achieve almost a 1-1 perf against C++. Unlike H.Sutter, I strongly believe that with a tightly optimized JIT or AOT, some other allocator scheme integrated into the language (other than pure GC), more aggressive unsafe, a proper SIMD integration, C# generated code would be on par or way ahead C++.
For example, we had a simple algorithm with a tight consuming loop that would require CPU cache pre-fetch to access next elements of an array. Without cache pre-prefetch, the loop was running around 1.3ms, with pre-fetch, it was running around 0.5ms (and in a game, this kind of factor is quite significative). The problem in C# is that it is almost impossible, when working with stock managed arrays to perform some cache-prefetch (using directly specific CPU instructions), as the JIT, no matter what you are trying to do, is always trying to access the element of an array even if you are not using it (using by ref for example), so It is not possible to pre-fetch elements into a managed array (unless going through some nasty direct access of the managed array).
Of course, these cases are not common for a casual C# or even C++ developer, but when it comes to optimization, C# is a bit more tricky to optimize as we have less opportunities or more restrictions on the kind of optim we can perform.
Anyway, this whole "Going Native C++" trend is wrong, the futre is absolutely not in such a cluttered language (and I believe that software industry and innovation has more suffered from C++ pitfalls than its supposed greatness). Would love to see how far they have been with projects like Midori and their C# like language fully integrated into the OS, rumors where saying that It was damn efficient...