|
|
|
|
Purpose
|
PRECISION specifies the rounding factor for the result of an arithmetic operation.
|
|
Syntax
|
PRECISION (rounding-factor)
|
|
Operator
|
rounding-factor is any valid numeric expression with an integer value between –n and n inclusive where n is the maximum number of digits depending on the data type as follows:
Numeric (N) – 12 on ‘C’ systems, 15 on Mainframe.
Long Numeric (L) – 20 on ‘C’ systems, 15 on Mainframe.
This specifies rounding to the nearest 10-(rounding-factor).
|
|
Remarks
|
The rounding factor for calculations is set to a default of six places to the right of the decimal.
Once specified via a PRECISION statement, rounding is in effect for the duration of the logic routine or until another PRECISION statement is executed.
Within Global Logic, a precision of six decimal places is used unless a PRECISION statement within the logic changes it to some other value. The precision in use before entry to Global Logic is restored on exit from the logic, whether or not the Global Logic contains a PRECISION statement.
|
|
Example
|
Assume #A = 1.25 and #B = 2.75
First Logic routine:
#C = #A * #B #C is set to 3.4375 (default rounding is to 6 decimal places) PRECISION(2)
#D = #A * #B #D is set to 3.44 (rounded to the nearest 10-2) PRECISION (3)
#E = #A * #B #E is set to 3.438 (rounded to the nearest 10-3)
#F = #D - #C #F is set to 0.003 (PRECISION (3) is still in effect) PRECISION (0)
#G = #A * #B #G is set to 3 (rounded to the nearest 100 decimal places)
#H = #C - #G #H is set to 0 (PRECISION (0) is still in effect)
Second logic routine executed:
#I = #C - #G #I is set to 0.4375 (default rounding is 6 decimal places) PRECISION (-2) #H = #A * 1000 #H is set to 1300 (rounded to the nearest 10—2 = 102)
|
