calculated values in a SQL statement are wacky 1

[link9]

Hello all --

Ok, I'm stumped...

I have a table called customerCount that I want to create a view on... basically, any unit has a customer count, and by this function, I am to determine how many surveys they need to complete in order to have valid data...

Table customerCount has but two columns, unit & customerCount. RDBMS is SQL Server 7.0

Here's the SQL Statement:

SELECT unit,
 (.25 / POWER((.05 / 1.55), 2) + (.25 / cast(customerCount as float))) AS target,
 customerCount
FROM customerCount

Now here's the vb statement that will give me what I want:

(.25 / (.05/1.44)^2 + (.25 / customerCount))

Now, no matter whether their customerCount is 50 or 5000, I always get 240 -- with varying floating point values like:
240.1547
240.1549

etc.... They are all 240.15xxxxxxxx and so on...

Can anyone see what I have done to muck that statement up? I'm going blind over here.

Thanks, as always.
Paul Prewett

 

[link9]

please ignore up there where I have 1.55 instead of 1.44 -- it should be 1.44, but has no bearing on the problem.

 

[Jerrycurl]

I would guess that your formula is not correct.
.25 / (.05/1.44)^2 equals 240.xxx

.25 divided by any whole number will be less than 1, so it will just add to the decimal portion of your result.

(.25 / customerCount) contributes an insignificant amount to your result. Right?

 

[link9]

Correct -- just found my error. thanks.

 

[tlbroadbent]

Your SQL statment gives the same result as the VB code. Are you sure of the VB statement? Terry Broadbent
Please review faq183-874.

"The greatest obstacle to discovery is not ignorance -- it is the illusion of knowledge." - Daniel J Boorstin

 

[link9]

No, like Jerrycurl pointed out, it was the formula. The problem is now resolved.

Long day yesterday.