×
INTELLIGENT WORK FORUMS
FOR COMPUTER PROFESSIONALS

Contact US

Log In

Come Join Us!

Are you a
Computer / IT professional?
Join Tek-Tips Forums!
  • Talk With Other Members
  • Be Notified Of Responses
    To Your Posts
  • Keyword Search
  • One-Click Access To Your
    Favorite Forums
  • Automated Signatures
    On Your Posts
  • Best Of All, It's Free!

*Tek-Tips's functionality depends on members receiving e-mail. By joining you are opting in to receive e-mail.

Posting Guidelines

Promoting, selling, recruiting, coursework and thesis posting is forbidden.

Students Click Here

real number decimal prob.?

real number decimal prob.?

real number decimal prob.?

(OP)
In our postscript program we have prices that sum up in a total value. Each price is also printed.

When the total and the price is summed we get a automatic round if the value has more than 9999 and two decimals.


/print_adprice
{
    /adprice exch def
    adprice 10 string cvs dup
    /total_price total_price adprice add def
    adtext.cfont fontchange
    stringwidth pop adprice.right exch sub adtext.curtop m
    show
} bind def


31220.01 print_adprice
this call produces in our form: 31220.0

If we change the value to this:
3122.01 print_adprice
this call produces in our form: 3122.01
and that is what we was expecting whith the value 31220.01


The problem is when the values are shown as strings there is only one decimal and when the values are summed the decimal is rounded to ceil, so .01 == .1?

the total_price has incorrect value because of the rounding, i dont know postscript that well and dont know if the error comes when the values are passed and converted to strings or if threre is a limit in the real datatype or mabye ghostscript cant handle the values????

Weve tried to use whole integers and when we sould print them divide with 100 to shift the decimal, but we get the same result as above.

Any remark is welcome.
Best regards
/Olof Lager.


"Some thoughts has a certain sound..."

RE: real number decimal prob.?

I hate this about PostScript. While you can have numbers accurate to about 8 decimal places, each interpreter can do what it wants with regard to precision when performing calcuations with those numbers.

Meaning, you can start with accurate numbers, but as soon as you want to DO ANYTHING WITH THEM, including "add" and "cvs" or even "==", the accuracy defenestrates (that's Latin for "flies out the window").

Keep in mind that PostScript usually uses numbers to specify tints of color, or positions on a page, where such precision as "0.01" doesn't come into play, or if it did, would be visually insignificant.

The bottom line: do your calculations in the program that produces your PostScript, and only output the final, computed numbers as strings in the PostScript.

Final Note: precision does indeed vary from one PostScript interpreter to another. For example, Acrobat Distiller is horrible and doesn't retain much precision in calculations. PSAlter (a PostScript debugger) does a very good job in retaining precision. GhostScript does not do well.

Thomas D. Greer
http://www.tgreer.com

Providing PostScript & PDF
Training, Development & Consulting

Red Flag This Post

Please let us know here why this post is inappropriate. Reasons such as off-topic, duplicates, flames, illegal, vulgar, or students posting their homework.

Red Flag Submitted

Thank you for helping keep Tek-Tips Forums free from inappropriate posts.
The Tek-Tips staff will check this out and take appropriate action.

Reply To This Thread

Posting in the Tek-Tips forums is a member-only feature.

Click Here to join Tek-Tips and talk with other members! Already a Member? Login


Close Box

Join Tek-Tips® Today!

Join your peers on the Internet's largest technical computer professional community.
It's easy to join and it's free.

Here's Why Members Love Tek-Tips Forums:

Register now while it's still free!

Already a member? Close this window and log in.

Join Us             Close