# real number decimal prob.? 1

Status
Not open for further replies.

#### Dragonsoft

##### Programmer
In our postscript program we have prices that sum up in a total value. Each price is also printed.

When the total and the price is summed we get a automatic round if the value has more than 9999 and two decimals.

{
show
} bind def

this call produces in our form: 31220.0

If we change the value to this:
this call produces in our form: 3122.01
and that is what we was expecting whith the value 31220.01

The problem is when the values are shown as strings there is only one decimal and when the values are summed the decimal is rounded to ceil, so .01 == .1?

the total_price has incorrect value because of the rounding, i dont know postscript that well and dont know if the error comes when the values are passed and converted to strings or if threre is a limit in the real datatype or mabye ghostscript cant handle the values????

Weve tried to use whole integers and when we sould print them divide with 100 to shift the decimal, but we get the same result as above.

Any remark is welcome.
Best regards
/Olof Lager.

&quot;Some thoughts has a certain sound...&quot;

I hate this about PostScript. While you can have numbers accurate to about 8 decimal places, each interpreter can do what it wants with regard to precision when performing calcuations with those numbers.

Meaning, you can start with accurate numbers, but as soon as you want to DO ANYTHING WITH THEM, including "add" and "cvs" or even "==", the accuracy defenestrates (that's Latin for "flies out the window").

Keep in mind that PostScript usually uses numbers to specify tints of color, or positions on a page, where such precision as "0.01" doesn't come into play, or if it did, would be visually insignificant.

The bottom line: do your calculations in the program that produces your PostScript, and only output the final, computed numbers as strings in the PostScript.

Final Note: precision does indeed vary from one PostScript interpreter to another. For example, Acrobat Distiller is horrible and doesn't retain much precision in calculations. PSAlter (a PostScript debugger) does a very good job in retaining precision. GhostScript does not do well.

Thomas D. Greer

Providing PostScript & PDF
Training, Development & Consulting

Status
Not open for further replies.

Replies
0
Views
64
Replies
0
Views
27
Replies
0
Views
41
Replies
2
Views
31
Replies
0
Views
22