Tek-Tips is the largest IT community on the Internet today!

Members share and learn making Tek-Tips Forums the best source of peer-reviewed technical information on the Internet!

  • Congratulations wOOdy-Soft on being selected by the Tek-Tips community for having the most helpful posts in the forums last week. Way to Go!

math errors adding 0000000001 to the end of a number? 1

Status
Not open for further replies.

ripple

Programmer
Jul 17, 2001
6
US
um, hey all, this is really confusing me. does anyone know of any common errors or bugs in the javascript engine itself that could be mucking with numbers? maybe it is internet explorer's javascript that is at fault?

hmmm.

i've narrowed the bug down to the following code at the bottom of the post.

my question is, why does it return 0.60000000001 after multiplying 6 * 0.1 ??? it returns 0.5 when you change intminutes to 5, and have it multiply 5 * 0.1.

i wasnt able to get around the error by detecting the strange numbers and subtracting 0.00000001 or whatever, and also i could not convert it to a string and clip it with substr or substring (it returned something like 1~E68e90.8734 or an equally strange response)

this is very obnoxious. please help!

----------------

<script language=&quot;Javascript&quot;>

intminutes = 6
inthours = intminutes / 60;

cost = (intminutes * 0.1);

amsg = cost;

alert(amsg);

</script>
 
Is this what you are trying to do?

<SCRIPT LANGUAGE=&quot;JavaScript&quot;>
<!--
// formatCurrency() converts a number to a proper
// Dolar format $0.00
function formatCurrency(num) {
num = num.toString();
if(isNaN(num))
num = &quot;0&quot;;

sign = (num == (num = Math.abs(num)));
num = Math.floor(num*100+0.50000000001);
cents = num%100;
num = Math.floor(num/100).toString();
if(cents<10)
cents = &quot;0&quot; + cents;

for (var i = 0; i < Math.floor((num.length-(1+i))/3); i++)
num = num.substring(0,num.length-(4*i+3))+','+

num.substring(num.length-(4*i+3));
return (((sign)?'':'-') + '$' + num + '.' + cents);
}


var vCostPerHour = 15.75; // Cost Per Hour
var intminutes = 6; // The number of minutes it took
var inthours = intminutes / 60; // Convert the minutes to hours
var cost = formatCurrency(vCostPerHour * inthours); // Formulate the cost
var amsg = cost;

// Ouput the Dolar amount
document.write(amsg);

//-->
</SCRIPT> - tleish
 
hummm yes sort of. except i have to calculate it based on a non linear pricing model, which requires me to process the hours and the minutes seperately. but really, isnt the code i posted just fine?? its just some simple math and javascript doesnt output the right answer!!
 
I don't uderstand either what this line of code is or why you did it this way:

cost = (intminutes * 0.1); - tleish
 
there is a charge of 10 cents per minute... was it weird to use the ( ) parentheses ? i think it is that way because i copy and pasted it from somewhere else.

anyway that is not all the javascript on the page, just a clipping i made that shows the malfunction.
 
The reason you are getting the strange looking number isn't a bug, it has to do with the way math is done in computers. You should be able to use the floor or ceil methods to get around it. Tracy Dryden
tracy@bydisn.com

Meddle not in the affairs of dragons,
For you are crunchy, and good with mustard.
 
Status
Not open for further replies.

Part and Inventory Search

Sponsor

Back
Top