• 0

FLOAT and DOUBLE in Visual Studio is garbage?


Question

I am having some problems with visual studio 2005 professional,

I am declaring a simple double to 0.7 but in runtime it is 0.699999999999996 which is not CLOSE ENOUGH precision for my program...

I was wondering if this is a normal thing, or is it a bug in visual studio or is my hardware broken...?

check the image for a break/debug right after a few doubles have been allocated, you can see my code and you can see whats in mem..

the following info is to try and give a bigger picture of my system:

OS Name Microsoft? Windows Vista? Ultimate

Version 6.0.6000 Build 6000

System Manufacturer FUJITSU SIEMENS

System Model Amilo Mx43y Series

System Type X86-based PC

Processor Intel® Pentium® M processor 1.73GHz, 1733 Mhz, 1 Core(s), 1 Logical Processor(s)

BIOS Version/Date American Megatrends Inc. 1.10C, 05/10/2005

SMBIOS Version 2.3

Hardware Abstraction Layer Version = "6.0.6000.16386"

Total Physical Memory 2,046.81 MB

Microsoft Visual Studio 2005

Version 8.0.50727.867 (vsvista.050727-8600)

Microsoft .NET Framework

Version 2.0.50727

Installed Edition: Professional

Microsoft Visual C++ 2005 77626-009-0000007-41812

Microsoft Visual C++ 2005

Microsoft Visual Studio 2005 Professional Edition - ENU Service Pack 1 (KB926601)

Update for Microsoft Visual Studio 2005 Professional Edition - ENU (KB932232)

post-2527-1183128508_thumb.jpg

14 answers to this question

Recommended Posts

  • 0

Uh. I recommend reading up on the differences between basic data types....If you need absolutely perfect precision, you'll need some kind of BCD-type to store your numbers.

A double is a 64bit floating point number, which is only accurate to like 18ish (Maybe less?) decimal points. After that it does some rounding.

Also, you're programming in c++, Visual Studio is just an IDE, and kinda irrelevent to the question. (Just a tip)

Anyway.

Try using a long double.

Edited by MioTheGreat
  • 0
  noroom said:
That's what Double.Epsilon is for.

That's .NET.

Anyway, I take back what I said above, a long double and a double are the same thing w/ msvc++. They don't have extended precision doubles (Apparently, according to an MSDN blog, it might come in VC9).

You'll have to roll your own or find a sample online with a higher precision float, or a BCD (Which would be slower than the other option, but much more precise)

  • 0

in this case I just needed 0.7 as a nice number, when I saw that it wasnt what I specified I got worried that something might be broken somewhere, I remember reading about how NASA couldnt use the pentium cpu because it calculated numbers wrong... so I got a bit worried.. ;)

thanks for your replies, it really helped me to understand how things work. still I find it weird that I specify 0.7 but it doesnt store it like that...

I tried the same thing in C# and 0.7 is 0.7 in there...

  • 0

Yeah, this one is easy to understand. In fact, it happens very often and very easily if you aren't careful.

What's going on? Consider the value 0.5. That is easily representable in binary because binary uses powers of two. In the case of fractional parts, it works the same way as in decimal:

0.5 = 5 * 10^(-1) = 5 * 1/10 = 5/10 = 1/2

0.1b = 1 * 2^(-1) = 1 * 1/2 = 1/2

Obviously, for negative powers of 2, it is easy to store:

0.25 = 2 * 10^(-1) + 5 * 10^(-2) = 2/10 + 5/100 = 20/100 + 5/100 = 25/100 = 1/4

0.01b = 0 * 2^(-1) + 1 * 2^(-2) = 0 + 1 * 1/4 = 0 + 1/4 = 1/4

However, what about 1/5 (0.2)?

0.2 = 2 * 10^(-1) = 2 * 1/10 = 2/10 = 1/5

0.00110011...b = 1 * 2^(-3) + 1 * 2^(-4) + 1 * 2^(-7) + 1 * 2^(-8) + .... = 1/5

As you can see, unlike in decimal, 1/5 written in binary is a repeating value, just as 1/3 is in decimal.

You are also probably wondering how I got that value, right? The method that I've found easiest to understand is displayed here:

 x = 1 / 5
 y = "."

 x = x * 2 //multiply the value by the number base
 If x >= 1.0 Then // if x > 1 then
	 x = x - 1.0  // subtract 1 from x
	 y = y + "1" // and add a form of "carry" to a string containing the binary
 Else
	 y = y + "0"
 End If
 // repeat the multiplication process until x = 0
 // or until you find a recognizable repetition in y
 // or until the string exceeds the precision of the data type you are using

To illustrate, I'll use 1/5 as an example as if the above pseudocode actually ran and the comment about repeating the multiplication was implemented as some form of loop already:

1/5 * 2 = 2/5

2/5 >= 1 ?? No - ".0"

2/5 * 2 = 4/5

4/5 >= 1 ? No - ".00"

4/5 * 2 = 8/5

8/5 >= 1 ? Yes - ".001" // 8/5 - 1 = 3/5

3/5 * 2 = 6/5

6/5 >= 1 ? Yes - ".0011" // 6/5 - 1 = 1/5

We're back to 1/5, which is obviously where we started. Therefore, we can deduce that 1/5 repeats the 0011 combination.

Note that the above method actually works for any base:

// Hexadecimal

1/5 * 16 = 16/5

16/5 >= 1 ? Yes - ".3" // 16/5 - int(16/5) = 1/5

Since 1/5 ended up being 1/5 again, we have no need for testing again because we know that it will end up being the same thing. But why the .3 this time? Simply put, when you integer divide 16 by 5, you get 3. That value is the value that you need to use for the decimal. Also note the following way to check:

every group of 4 binary digits (bits) is equal to one hexadecimal digit (sometimes known as a hexit or hex bit). In other words, look at the following from the example:

.00110011

notice the 1s? Split them up!

. 0011 0011

Now convert them to hexadecimal:

. 3 3

The result for each group is 3, just like when the hexadecimal conversion was done.

This is probably confusing to you now. However, if you practice the conversion a couple of times, it is not that difficult.

Just remember that you check to see if the fraction is greater than or equal to 1 since the fraction should be less than 1 (if it is more than one, only part of the value is a fraction ;)). If it is greater than or equal to 1, you need to subtract the integer part of the value and you also need to add the integer part of the value to a string containing the values to give you your fraction in whatever base you want. Otherwise, you add a 0 to that string.

  • 0

Try this out:

http://www.alphaworks.ibm.com/tech/decnumb...usplus/download

Or, you can switch over to Managed C++ and use the System.Decimal class which is built in. (You might be using managed c++ for all I know, I can't tell based on what you've told us)

Edited by MioTheGreat
  • 0

You could also try using GMP (the GNU Multi-Precision library). There is a static build and a dynamic build for Visual C++ here as well as the source code to GMP and patches for the source code apparently -

http://cs.nyu.edu/exact/core/gmp/

Using System.Decimal would probably be the best solution since it should be a relatively similar syntax (rather than learning a whole new language by using a 3rd-party library).

This topic is now closed to further replies.
  • Recently Browsing   0 members

    • No registered users viewing this page.
  • Posts

    • Par for the course. I asked a question a few days ago and was threatened to be reported. 😂
    • Ummmm Are you kidding? Do think these books being sold are disclosing they are AI written?
    • i'm just commenting because of this madness. you simply asked "what crowd" which i too am genuinely curious about, only to receive a response in the form of a link that directs to an analysis of the audited financial statements with the accompanying notes. then you say that the guy who wrote it is stupid, which whatever, that's not an argument being discussed atm, only to receive a response from Arceles saying "i don't deal with people whose first response is an ad homenim". jesus. this is like making a claim, and then saying "i don't deal with people who speak in a certain way or swear so i'm not gonna answer you hah!" (said in a nasaly voice, not trying to depict you Arceles). then focus on the argument instead of the explanation begins... "so what don't you like about the guy (lunduke)" followed by "he just likes to insult people" and the explanation for the "crowd" being referred to was never even established. so a request for an explanation about the crowd turned into an argument about "why do you think lunduke is an idiot". wowza.
    • Sony lays off 30% of staff from Days Gone developer Bend Studio by Pulasthi Ariyasinghe Another wave of layoffs has hit the game developer space, and this time, it's a first-party Sony studio that's been affected. Following a report by Bloomberg's Jason Schreier, Bend Studio has confirmed that it is letting go of "incredibly talented teammates" as it begins work on a new game project. "Today, we said goodbye to some incredibly talented teammates as we transition to our next project," said Bend Studio in a social media post today. "We're deeply thankful for their contributions as they've shaped who we are, and their impact will always be part of our story." Bend Studio is most well-known for its 2019-released open-world zombie adventure Days Gone, which even received a remaster just a few months ago. Prior to that, the studio had been responsible for the classic Syphon Filter series while also developing several PlayStation Portable and Vita games like Resistance: Retribution and Uncharted: Golden Abyss, respectively. "This is a difficult moment for our team, but we hold immense respect for everyone who got us here," the company added. "As we move forward, we remain committed to building the future of Bend Studio with creativity, passion, and innovation in the titles we craft." While Sony did not detail just how many staff have been affected by this latest decision, Jason Schreier revealed that 30% of the studio is being laid off. This amounts to around 40 people, according to the reporter. Earlier this year, Sony canceled two live service games that were in development at Bend Studio and Bluepoint Games. It was never revealed what this mystery game was supposed to be. "Bend and Bluepoint are highly accomplished teams who are valued members of the PlayStation Studios family," Sony said at the time. It's unclear if Bluepoint, which had been developing a God of War live service experience, will soon be hit by its own wave of layoffs too.
    • KataLib 4.5.3.0 by Razvan Serea KataLib is more than just a music player — it's a complete audio suite designed for music lovers and creators alike. It combines a powerful audio player, a flexible metadata editor, a capable audio converter, and a music library manager into one streamlined application. Core Features: Audio Player Enjoy seamless playback of virtually any audio format or even streaming video files. DJ Mode lets you mix tracks with manual or automatic crossfades. You can also load and save WinAmp-style playlists for quick access to your favorite sets. Audio Converter Convert between a wide range of audio formats effortlessly. Trim or normalize your output automatically, and even extract audio from streaming video sources. Ideal for preparing files for different devices or platforms. Metadata Editor View and edit ID3v2 tags and other metadata. Batch edit multiple files at once, and fetch missing information directly from the MusicBrainz database. You can also apply or update album art with ease. Music Library Manager Organize your entire audio collection, search across tracks instantly, and download cover images from the internet — or use your own custom artwork. KataLib makes it easy to keep your library tidy and enriched with useful info. Supported Formats: KataLib supports a wide range of both lossy and lossless audio formats: Input: OPUS, AAC, FLAC, M4A, MP3, MP4, MPC, APE, AIF, MKV, AVI, MOV, FLV, WEBM, Ogg Vorbis, WAV, WAVPack, WMA Output: OPUS, FLAC, M4A, MP3, Ogg Vorbis, WAV Under the hood, KataLib uses the trusted FFmpeg engine for audio conversion and media playback, ensuring compatibility with virtually all mainstream media formats. Download: KataLib 4.5.3.0 | 64.5 MB (Open Source) Links: KataLib Home Page | Github | Screenshot Get alerted to all of our Software updates on Twitter at @NeowinSoftware
  • Recent Achievements

    • First Post
      K Dorman earned a badge
      First Post
    • Reacting Well
      rshit earned a badge
      Reacting Well
    • Reacting Well
      Alan- earned a badge
      Reacting Well
    • Week One Done
      IAMFLUXX earned a badge
      Week One Done
    • One Month Later
      Æhund earned a badge
      One Month Later
  • Popular Contributors

    1. 1
      +primortal
      545
    2. 2
      ATLien_0
      268
    3. 3
      +FloatingFatMan
      207
    4. 4
      +Edouard
      203
    5. 5
      snowy owl
      140
  • Tell a friend

    Love Neowin? Tell a friend!