Tek-Tips is the largest IT community on the Internet today!

Members share and learn making Tek-Tips Forums the best source of peer-reviewed technical information on the Internet!

  • Congratulations Rhinorhino on being selected by the Tek-Tips community for having the most helpful posts in the forums last week. Way to Go!

Rounding off error 1

Status
Not open for further replies.

Rixn

Programmer
Joined
Jun 4, 2005
Messages
9
Location
SE
When I calculate in the VB editor I get round off errors.

I have the following:

Excel cells:
E3 is =CalcResult(C3;F3;B3) (Sending the values to the VB editor)
C3 is changeable, 1 to 6
CF is changeable, 1 to 6
B3 contains the value I want to calculate

In VB Editor:
Function CalcResult(Arg1 As Long, Arg2 As Long, Arg3 As Long)
If Arg1 = 1 And Arg2 = 2 Then
CalcResult = Arg3 * 10
ElseIf Arg1 = 1 And Arg2 = 3 Then
CalcResult = Arg3 * 20
....
....
End If
End Function

When Arg3 is multiplied with 100 I will not get the exact answer. I. e. when Arg3 = 23,45 it should return 234,5 but I get the result 230.

1) What is wrong and how should I correct it?
2) Will the fact that I use comma "," in Excel and dot "." in the VB editor affect the calculation?

/Rixn
 
Replace this:
Arg3 As Long
By this:
Arg3 As Double

Hope This Helps, PH.
Want to get great answers to your Tek-Tips questions? Have a look at FAQ219-2884 or FAQ181-2886
 
Status
Not open for further replies.

Part and Inventory Search

Sponsor

Back
Top