Tek-Tips is the largest IT community on the Internet today!

Members share and learn making Tek-Tips Forums the best source of peer-reviewed technical information on the Internet!

  • Congratulations Chriss Miller on being selected by the Tek-Tips community for having the most helpful posts in the forums last week. Way to Go!

functions rounding for some reason

Status
Not open for further replies.

drewdaman

Programmer
Aug 5, 2003
302
CA
hello..

i have the following function:
Code:
Function w(ByVal diameter As Long) As Long
    w = (diameter / 2 * 10 * 2.54) * (diameter / 2 * 10 * 2.54) * 3.14
End Function

when i call this function with an argument 12 it returns 72929 instead of 72928.8864. ie it is rounding up for some reason. i'm absolutely sure that the variable i set equal to the function (when i call it) is declared as a long.

any ideas why this is happening?

thanks!
 

You need to define the function result as Double instead of Long:
Code:
Function w(ByVal diameter As Long) As [b] [COLOR=green]Double[/color][/b]
 
hmm... so long can only hold whole numbers? i have used them with decimal places before...

(incidentally, it did work when i changed the return type to double AND the var i use to store the result in as double- so THANKS!)
 

Must have been some other language. Directly from the Help file:
[tt]
Long (long integer) variables are stored as signed 32-bit (4-byte) numbers ranging in value from -2,147,483,648 to 2,147,483,647.[/tt]

 
Status
Not open for further replies.

Part and Inventory Search

Sponsor

Back
Top