how to avoid E-x ??? 1

[link9]

Hello all --

I have a view that has a whole lotta 1's and 0's in it.

I then have a view that runs an AVG() on each of those columns, and from time to time, I'm getting:

9.7014925373134331E-2

or some number like that. I see these every now and then, and usually come up with some "on-the-fly" way to take care of it, but this is a very large project where I'm having to crunch alot of data at run-time for people, and so error checking (technically, that's not an error, but you get my meaning) needs to be kept to a minimum.

So the question is:

Is there a sure-fire way to get rid of those types of numbers forever?

An example of the type of calculation that's generating it:
AVG(CAST(q2z AS FLOAT)) AS q2z,

SQL Server 7.0

Thanks for any input!
Paul Prewett

 

[tlbroadbent]

One way to get rid of the types of numbers you mention is to use Decimal or Numeric data types rather than Float or Real. Float and Real are approximate data types and will always produce approximate results. Terry L. Broadbent
FAQ183-874 contains tips for posting questions in these forums.
NOTE: Reference to the FAQ is not directed at any individual.

 

[link9]

Good, then. I'll remember that. Coming from a procedural programming background, I have trouble keeping up with the many many many different data types in databases.

And to think that the first time someone mentioned a "long" and an "int", I thought, "How stupid. A number is a number". ;-)

As it turns out, these numbers are being churned through another function in vbScript, formatPercent() before they are output to the user, and so these super long numbers aren't going to cause a problem after all.

But thanks for the tip!


paul