Tek-Tips is the largest IT community on the Internet today!

Members share and learn making Tek-Tips Forums the best source of peer-reviewed technical information on the Internet!

  • Congratulations Wanet Telecoms Ltd on being selected by the Tek-Tips community for having the most helpful posts in the forums last week. Way to Go!

Float: decimal point in wrong place

Status
Not open for further replies.

Gweener

IS-IT--Management
Jul 22, 2003
53
GB
I'm performing some analysis on some data. I'm summarising data using variables and then performing calculation on the variables.

@var1 int
@var2 int
@var3 float

calculation: @var3 = cast @var1 as float/cast @var2 as float

the values (in this instance) are 6/45 (var1 & 2 respectively).

The result is var3 = 13.33334

I want (need) it to be 0.133334 (i.e. correct)

Can anyone shed any light??
 
DECLARE @V1 int, @V2 int, @v3 float
SET @V1 = 6
SET @V2 = 45
SET @V3 = cast(@V1 AS float) / cast(@V2 AS float)
SELECT @V3
works on my system and so does:
Set @V3=1.*@V1/@V2
-Karl
 
your syntax works on my system too Karl. But the var1 and var2 (in my far more complext query) are the result of a select statement rather then a straight forward setting of a value.

As a side issue, i'm trying to put the query into DTS, to update a table, but am getting an "invalid pointer" error.......any idea?
 
OK - i'm being stupid.

The decimal place is in the right place, but it's to the power (has the E-2) at the end.

I'm declaring the variable as float (4) to try and limit the number of decimal places used, but to no avail. Am I missing something fundamental here?
 
Status
Not open for further replies.

Part and Inventory Search

Sponsor

Back
Top