Weird JavaScript Internal Math Error

[dtqbterror]

Anyone ever seen this issue with JavaScript in IE.

alert(2355.20 * 100.0)

Using this specific (2355.20) number the value returned is actually.

235519.99999999997

http://www.chiefscoalition.com

 

[trollacious]

This has been quite normal for a couple decades now with floating point numbers in many programming languages. If you want, you can use

Math.round();

to round to the nearest integer, which in this case would give you the exact result you want.

Lee

 

[mmerlinn]

Has always been a problem with computers. And until someone figures out a way to make a 5-way electronic switch, it will always remain a problem.

Since the computer uses 2-way switches (ON and OFF states only), ANY computation in ANY number base except 2, 4, 8, 16, 32, etc REQUIRES that all numbers and computations be rounded by the computer. Since rounding a number is almost never the same as the number that is being rounded, computational errors occur in almost every computation.

These errors have been mostly side-stepped by doing the computations in high precision, then lopping off a few bits at the end of the number to eliminate the rounding errors.

However, as you have found out, this process does NOT eliminate all rounding errors, just the vast majority.


mmerlinn

"Political correctness is the BADGE of a COWARD!"