Precision problems

[tfstom]

I have 2 variables both set up as Singles.

otHoursWorked = 0
tempOTPremium = 8.07

These are displayed correctly everywhere. After this command, however:

otHoursWorked = otHoursWorked + tempOTPremium

I get the following:

8.06999969482422

What is happening here? I have all kinds of Singles added and subracted together and I don't get this problem - Only with this instruction and variable.

This is about the 4th time I hit this instruction with problems at all.

What could be happening here?

Thanks,

Tom.

 

[percent]

Private Sub Command1_Click()
Dim otHoursWorked As Single
Dim tempOTPremium As Single

otHoursWorked = 0
tempOTPremium = 8.07

otHoursWorked = otHoursWorked + tempOTPremium
MsgBox otHoursWorked

End Sub

I just tried this and it returns 8.07 you must have more code - that you are not including - please list all code that has to do with these variables and maybe you could receive some more help


%, 2004

 

[tfstom]

Sorry, my mistake.

otHoursWorked was a Double and tempOTPremium was a Single.

When I changed the tempOTPremium to a Double, it worked.

Why is that?

Thanks,

Tom.

 

[ArtieChoke]

Doubles have more precision than singles, so it can represent a number more accurately. If you set a double variable to 8.07 and print it, you will get 8.07, so I suspect it's something to do with the way the conversion from single to double is handled. If you print CDbl(CSng(8.07)), you will get the long number you show above.

"I think we're all Bozos on this bus!" - Firesign Theatre