Rounding in VB Scripts

[JScannell]

I have seen several situations when the subtraction of two identical numbers has resulted an extremely small number instead of 0. I've seen -7.10543E-15, for example, when it should have been 0.

I have had to fix the issue in ASP by changing my VBasic code to be:
newNumber = Round ( Number1 - Number2, 2 )

I also have seen this in client-side javaScripts in the past and had to fix it by code like (assume numOldBalance and numAmountPaid are the same value):
numNewBal = numOldBalance - numAmountPaid;
numNewBal = Math.round ( numNewBal * 100 ) / 100;


Has anyone else come across these kind of problems?

Thanks in advance,

Jerry Scannell

 

[johnwm]

It's fundamental to the nature of floating point arithmetic. The normal method of dealing with it is to use scaled integers rather than floating point when you're adding and subtracting currency, or to use 'less than' for zero comparisons

If a - b = 0 Then ' fails for many FP ops
If Abs(a - b) < .00001 Then ' avoids FP problem

___________________________________________________________
If you want the best response to a question, please check out FAQ222-2244 first.
'If we're supposed to work in Hex, why have we only got A fingers?'
Drive a Steam Roller
Steam Engine Prints