[VB.NET]Truncate without rounding

[GSDragoon]

So lets say I have a number, 16.95. I want it truncated to one decimal place, but NO ROUNDING. I want 16.9 for the final result.

So far I'm using:

Dim first As Int32 = Math.Floor(tp)
Dim second As Int32 = Int((tp - first) * 10)
Dim TPs As String = first & "." & second

It works for the following example, but not every time. I think only when there's already one decimal place. Another example:

tp = 16.7

first = 16

tp - first = 6.9999885559082 <---- do not understand

TP is a single, if that matters. I'm having a problem getting that one digit in the tenths place.

[rpgfan]

Dim tp As Single = 16.95

'Fix simply truncates the decimal whereas Math.Floor always rounds down
'  (e.g. Math.Floor(-16.95) = -17, but Fix(-16.95) = -16)
Dim first As Int32 = Fix(tp)
Dim second As Int32 = Int((tp - first) * 10)
Dim TPs As String = first & "." & second

tp = 16.95

first = 16

second = Int((16.95 - 16) * 10) = Int((.95) * 10) = Int(9.5) = 9

TPs = 16.9

For the record, it could be floating-point rounding errors. You might try using a Double rather than a Single.

Fix truncates. Int rounds down to the nearest integer less than or equal to itself, same as Math.floor IIRC. Other things like CInt and Round all use Banker's Rounding, which is a bit odd for some people... If you have a fractional part of .5 or less, the result rounds toward the nearest even integer. Otherwise, the result is rounded to the nearest integer.

Examples:

Using CInt on a value in the range -0.5 to 0.5 results in 0.

Using CInt on a value in the range -1.499999... to -0.500...001 or 0.500...001 to 1.499999... results in -1 or 1, depending on the sign.

Using CInt on a value in the range -2.5 to -1.5 or 1.5 to 2.5 results in -2 or 2, depending on the sign.

Using CInt on a value in the range -3.499999... to -2.500...001 or 2.500...001 to 3.499999... results in -3 or 3, depending on the sign.

Simply put, Banker's Rounding relies on that extra minute fractional piece to round to the appropriate integer. This is why I use Fix for everything - I don't have any issues with negative v.s. positive values like I would with Int or Math.floor or Math.ceiling or whatever. I also don't have rounding issues like with Round and CInt. Everything is predictable, in other words. Of course, the built-in Math.Round does a wonderful job of rounding "the normal way" already, so I suppose my reinventing the wheel using Fix is rather pointless. ^_^

Edited June 15, 2008 by rpgfan

[GSDragoon]

Fix gives me the same results and changing it to a double just gives me 0.69999999999999999999999 instead of 0.7

[ramesees]

Without knowing exactly what you wish to use the final output for, why not convert the 16.95 to a string ("16.95") and then simply return a substring from index 0 up until the first number after the decimal point ?

[Rob Veteran]

Truncate to 1 decimal place, no rounding:

Dim decimalNumber As Double = 16.95
Dim truncatedNumber As Double = Math.Truncate(decimalNumber * 10) / 10

Should work. Math.Truncate() gets the integral part of the number (the bit before the decimal point). Therefore, if you multiply that by 10, then truncate, and then divide after, you'll get the one decimal place without any rounding.

[Andre S. Veteran]

  GSDragoon said:

So lets say I have a number, 16.95. I want it truncated to one decimal place, but NO ROUNDING. I want 16.9 for the final result.

tp = 16.7

first = 16

tp - first = 6.9999885559082 <---- do not understand

Single and double precision numbers are inherently approximative. It's very dangerous with such numbers, for instance, to compare 7 with 7 for equality: one could be 6.999999999 and the other 7.00000000001.

If you need exactitude, use integers only. Define a type that holds the number as an integer and a power of 10. But in most cases you should be fine with the built-in floats, doubles and decimals.

[GSDragoon]

Thanks for the help everyone, I didn't know about them being approximates. Math.Truncate(decimalNumber * 10) / 10 worked for me. I know I could have manipulated a string, but now I've learned something new.