Setting Output Precision of Distance Units

Model Reference Manual

General Commands : Setting Working Units and Tolerances : Setting Output Precision of Distance Units

Note:

The user must not change the precision if customising the AVEVA Applicationware. Only use this command if writing a completely new application.

Keywords:

PRECISION

Description:

Output dimensions, in the current units, are normally rounded to the following degrees of precision:


Millimetres	To the nearest mm (e.g. 316)
Inches	To the nearest 0.1 inch (e.g. 38.8)
Feet and inches	To the nearest 1/32 inch (e.g. 3’7.15/32)

Output for setting variables defaults to the following, higher, precisions:


Millimetres	To two decimal places (e.g. 316.25)
Inches	To two decimal places (e.g. 38.75)
Feet and inches	To the nearest 1/64 inch (e.g. 3’7.31/64)

The user may control the output precision of distances in some modules by using the PRECISION command. Which allows the user to specify the number of decimal places for MM or INCH output, or the largest fraction denominator for FINCH output. Precision levels for general output and for the setting of variables are specified separately.

The following command is used to specify the output precision of distance values.

PRECision opt1 DIstance opt2 value opt3 (value2 opt4)

PRECision opt1 opt2 value opt3 (value2 opt4)

To specify the precision of all other values, including pure numerical values the following command is used:

PRECision opt1 GENeral opt2 value opt3 (value2 opt4)

To reinstate default precision (as at programme initialisation) the following command is used:

PRECision opt1 [DISTance|GENeral| ] DEFault

The default initial precisions for both GENERAL and DISTANCE values are set 5 digits and 4 decimal places for OUTPUT values, and 6 digits and 6 decimal places for VARIABLE values.

Where the command options (opt1, opt2, opt3 and (optionally) opt4) are as follows:

opt1 defines how the specified precision is to be applied. It is one of the following:


OUTput	For general dimensions
VARiables	For setting variables

If opt1 is omitted:

•

Both Output and Variables are set if opt2 is DEFAULT

•

Output only is set if opt2 is TO

opt2 defines whether the user is changing to a newly-defined precision or back to the default settings. It is one of the following:

TO
DEFault

If opt2 is omitted, TO is assumed.

opt3, which only applies if opt2 is TO, qualifies the number specified by value. The available choices are:


DECi or DECIMAls	specifying number of decimal places - for decimal values
DPls	specifying number of decimal places - for decimal values
DIGits	specifying number of significant digits - for decimal values

THs As	specifying number of 'ths' fractions as in 64ths - for FINCH values
NDs As	specifying number of 'nds' fractions as in 32nds - for FINCH values

If opt3 is omitted, value is assumed to apply as follows:

•

If current distance units are MM or INCH: value is Decimals

•

If current distance units are FINCH: value is THs or NDs

The FINCH precision is only accepted and set if the current working DISTANCE or BORE units are FINCH. Otherwise the command is ignored.

If opt3 is present and set to decimals then a second value (value2) with an optional fourth option (DIGITS - but this is assumed) may be specified. This is used to be able to set both the maximum number of decimal places in the number and the maximum number of significant digits to be output, as in:

PRECI VAR DISTANCE 6 DECIMALS 8 DIGITS.

If both DPLS and DIGITS are set then they constrain each other so neither of the values is exceeded unless the number is of size greater than 10**DIGITS, for example, more digits are required to represent the number with no decimals than DIGITS has specified.

It is possible to unset a precision by using the string UNSET in place of the number of decimal places or digits (value and value2). This allows the other constraint to control the output of the value in all cases without regard any limitation of the other. For example, with no limitation on digits there will always be the required number of decimal places regardless of the size of number.

If both digits and places are unset the precision reverts back to default precisions.

Default initial precisions set 5 digits and 4 decimal places for OUTPUT values, and 6 digits and 6 decimal places for VARIABLE values.

OUTPUT to DATAL files uses its own precision setting that the user has no control over

VARIABLE precision applies to the default formatting of numbers for real to string conversions of PML variables.

OUTPUT precision applies to the precision applied to normal query output including GetAsString calls from .NET.


Examples:
PREC OUT DI DEF	Set default precision for output
PREC VAR DEF	Set default precision for variables
PREC DI DEF	Set default precision for output and variables
PREC DI 1 DP	Set output to 1 decimal place (MM or INCH units)
PREC OUT DI 4 DEC	Set output to 4 decimal places (MM or INCH units)
PREC VAR DI 3 DP	Set variables to 3 decimal places (MM or INCH units)
PREC DI 64THS	Set output to 1/64 inch precision (FINCH units)