Contact US

Log In

Come Join Us!

Are you a
Computer / IT professional?
Join Tek-Tips Forums!
  • Talk With Other Members
  • Be Notified Of Responses
    To Your Posts
  • Keyword Search
  • One-Click Access To Your
    Favorite Forums
  • Automated Signatures
    On Your Posts
  • Best Of All, It's Free!

*Tek-Tips's functionality depends on members receiving e-mail. By joining you are opting in to receive e-mail.

Posting Guidelines

Promoting, selling, recruiting, coursework and thesis posting is forbidden.

Students Click Here

format problem

format problem

format problem

I have a a string with value '7.0445'.
It is placed in a real variable (Ci) and appears as

7.044500 in the debugger.

When I try and write it to a string (STR) using F8.3

It comes out as  7.044 (NOT 7.045)

However if I make the variable = to 7.044510
Then the value placed in the string is '7.045'

Why could this be?  
Why, in the first instance, is the REAL not being rounded?


RE: format problem

Looks like the rounding is done correctly!
The accountants and scientists do not like to round up automatically at 5, because it will SYSTEMATICALLY gives a number too big on the average.  So if the remainder to be rounded is exactly 0.5 unit, it rounds to the next even digit.

To test it on your compiler, try displaying
7.0445 and 7.0455
If the above scheme is used, it should give 7.044 and 7.046 respectively.
However, 7.044501 and 7.045501 will give 7.045 and 7.046 respectively, as they are over 0.5 unit and the above rule no longer applies.

RE: format problem

Some addition: think about floating point arithmetics. You have not exactly 7.0445 or 7.044510 or what else with decimal fractions on binary floats of modern computers...
Comp floats are not math reals...

RE: format problem

Thanks.  I actually was thinking it had something to do with the floating point arithmetic...

But was not sure why that would be.

Anyway, thanks for the insight.


Red Flag This Post

Please let us know here why this post is inappropriate. Reasons such as off-topic, duplicates, flames, illegal, vulgar, or students posting their homework.

Red Flag Submitted

Thank you for helping keep Tek-Tips Forums free from inappropriate posts.
The Tek-Tips staff will check this out and take appropriate action.

Reply To This Thread

Posting in the Tek-Tips forums is a member-only feature.

Click Here to join Tek-Tips and talk with other members! Already a Member? Login

Close Box

Join Tek-Tips® Today!

Join your peers on the Internet's largest technical computer professional community.
It's easy to join and it's free.

Here's Why Members Love Tek-Tips Forums:

Register now while it's still free!

Already a member? Close this window and log in.

Join Us             Close