This extension is an interface to the basic calculator function for Arbitrary Precision Mathematics.

# 1 Overcoming Javascript numeric precision issues

Floating-point numbers are represented as binary (base 2) fractions. Regrettably, most decimal fractions cannot be represented exactly as binary fractions. The decimal floating-point numbers you enter are only approximated by the binary floating-point numbers actually stored in the machine. That being said, you'll see that floating-point arithmetic is NOT 100% accurate.

Copy
0.2 + 0.1
0.30000000000000004

0.3 - 0.1
0.19999999999999998

1111.11+1111.11+1111.11+1111.11+1111.11
5555.549999999999

To solve this problem, we need an alternative to Javascript numbers. That's why we have a BigDecimal Number Object and library, to address this issue. It provides arbitrary-precision decimal arithmetic.

## 1.1 Decimal data type

Primitive numeric types are useful for storing single values in memory. But when dealing with calculation using double and float types, there is a problems with the rounding. It happens because memory representation doesn't map exactly to the value.

To solve this issues you can use immutable, arbitrary-precision signed decimal numbers. A BigDecimal consists of an arbitrary precision integer unscaled value and a 32-bit integer scale. If zero or positive, the scale is the number of digits to the right of the decimal point. If negative, the unscaled value of the number is multiplied by ten to the power of the negation of the scale. The value of the number represented by the BigDecimal is therefore (unscaledValue × 10-scale). The BigDecimal class provides operations for arithmetic, scale manipulation, rounding, comparison, hashing, and format conversion. The toString() method provides a canonical representation of a BigDecimal.

## 1.2 Decimals created by application

In the following example we can see two implementations of the factorial function.

• First uses the native JavaScript numbers. We can see data precision is lost whe number increases.
• Second used multiplications using Ax.math.bc.mul function that takes two numnbers and returns a Decimal data type as result.

Copy
<script>
function factorialDouble(num)
{
return (num === 0) ? 1 : num * factorialDouble( num - 1 )
}

function factorialDecimal(num)
{
return (num === 0) ? 1 : Ax.math.bc.mul(num, factorialDecimal( num - 1 ))
}

console.log(Ax.text.String.format("%3s %24s %s", "Num", "double", "Decimal"));
for (var idx = 1; idx <= 50; idx++)
console.log(Ax.text.String.format("%3s %24s %s", idx, factorialDouble(idx), factorialDecimal(idx)));
</script>

# 2 Decimal returned from database

Databases have a special type called DECIMAL to perform abitrary precision calculations. When returning data of such type from database to application, data is mapped to class BigDecimal.

## 2.1 Getting count(*)

A typical operation that returns a BigDecimal in most databases is when selecting a COUNT(*).

• executeGet, automatically detects for BigDecimal and converts to int/long
• executeGetInt, returns an int
• executeGetLong, returns a long

Notice that the previous functions prevents the need to use BigDecimal comparision to test for COUNT(*).

## 2.2 Dealing with bigdecimals

Copy
<script>
var row = Ax.db.executeQuery(
SELECT CAST(0.02 AS DECIMAL) a, CAST(0.03 AS DECIMAL) b
FROM systables
WHERE tabid = 1
).toOne();

// A: using javascript double precision aritmetic
console.log("Double precision");
console.log(0.02 - 0.03);
console.log(row.a - row.b);

// B: using BigDecimal aritmetic
console.log("BigDecimal precision");
console.log(new Ax.sql.Decimal("0.02").subtract(new Ax.sql.Decimal("0.03")));
console.log(row.a.subtract(row.b));
</script>

Notice that Decimal types returned from database must be operated using its own function (add, subtract, div, mul, etc). If not, they are converted to double before operating.

# 3 BC functions

The Ax.math.bc package contains a group of functions to perform BigDecimal operations.

Return Method Description
Creation
BigDecimal of(String number) Translates the string representation of a BigDecimal into a BigDecimal. The string representation consists of an optional sign, '+' ( '\u002B') or '-' ('\u002D'), followed by a sequence of zero or more decimal digits ("the integer"), optionally followed by a fraction, optionally followed by an exponent.
BigDecimal of(Number number) Translates a double into a BigDecimal which is the exact decimal representation of the double's binary floating-point value. The scale of the returned BigDecimal is the smallest value such that (10scale × val) is an integer.
BigDecimal add(Number n1, Number n2, ...) Returns a BigDecimal whose value is (this + augend), and whose scale is max(this.scale(), augend.scale()).
BigDecimal sub(Number n1, Number n2, ...) Returns a BigDecimal whose value is (this - subtrahend), and whose scale is max(this.scale(), subtrahend.scale()).
BigDecimal mul(Number n1, Number n2, ...) Returns a BigDecimal whose value is (this × multiplicand), and whose scale is (this.scale() + multiplicand.scale()).
BigDecimal div(Number n1, Number n2) Returns a BigDecimal whose value is (this / divisor), and whose preferred scale is (this.scale() - divisor.scale()); if the exact quotient cannot be represented (because it has a non-terminating decimal expansion) an ArithmeticException is thrown.
BigDecimal div(Number n1, Number n2, int scale) Returns a BigDecimal whose value is (this / divisor), and whose preferred scale is (this.scale() - divisor.scale()); if the exact quotient cannot be represented (because it has a non-terminating decimal expansion) an ArithmeticException is thrown.
BigDecimal div(Number n1, Number n2, int scale, String roundingMode) Returns a BigDecimal whose value is (this / divisor), and whose preferred scale is (this.scale() - divisor.scale()); if the exact quotient cannot be represented (because it has a non-terminating decimal expansion) an ArithmeticException is thrown.
abs, pow, round, scale
BigDecimal abs(Number n1) Returns a BigDecimal whose value is the absolute value of this BigDecimal, and whose scale is this.scale().
BigDecimal pow(Number n1, int n2) Returns a BigDecimal whose value is (thisn), The power is computed exactly, to unlimited precision.
BigDecimal round(Number n1, int precision) Returns a BigDecimal rounded according to the MathContext settings. If the precision setting is 0 then no rounding takes place
BigDecimal scale(Number n1, int scale, int roundingMode) Returns a BigDecimal whose scale is the specified value, and whose value is numerically equal to this BigDecimal's. Throws an ArithmeticException if this is not possible.

This call is typically used to increase the scale, in which case it is guaranteed that there exists a BigDecimal of the specified scale and the correct value. The call can also be used to reduce the scale if the caller knows that the BigDecimal has sufficiently many zeros at the end of its fractional part (i.e., factors of ten in its integer value) to allow for the rescaling without changing its value.

int getScale(Number n1) Returns the scale of this BigDecimal. If zero or positive, the scale is the number of digits to the right of the decimal point. If negative, the unscaled value of the number is multiplied by ten to the power of the negation of the scale. For example, a scale of -3 means the unscaled value is multiplied by 1000.
comparision
boolean isZero(Number n1) Returns a comparision between the number and BigDecimal.ZERO.
boolean equals(Number n1, Number n2) Compares this BigDecimal with the specified Object for equality. Unlike compareTo, this method considers two BigDecimal objects equal only if they are equal in value and scale (thus 2.0 is not equal to 2.00 when compared by this method).
int compareTo(Number n1, Number n2) Compares this BigDecimal with the specified BigDecimal. Two BigDecimal objects that are equal in value but have a different scale (like 2.0 and 2.00) are considered equal by this method. This method is provided in preference to individual methods for each of the six boolean comparison operators (<, ==, >, >=, !=, <=). The suggested idiom for performing these comparisons is: (x.compareTo(y) <op> 0), where <op> is one of the six comparison operators.
Copy
<script>

console.log(Ax.math.bc.mul(2, 2));
console.log(Ax.math.bc.mul(2, 2, 3));

</script>

## 3.1 Comparing numbers

Be carefull when comparing BigDecimal to a javascript number. You can use

• equals - takes scale into account
• compareTo - does not take scale but returns 0, 1 or -1 accoding if number is equal, greater or lesser
• isZero - compares if number is zero

##### equals
Expression Result
Ax.math.bc.equals(new Ax.math.BigDecimal(0), 0.00) false
Ax.math.bc.equals(new Ax.math.BigDecimal(0.0), 0.00) false
Ax.math.bc.equals(new Ax.math.BigDecimal(0.00), 0.00) false
Ax.math.bc.equals(new Ax.math.BigDecimal(0), 0) true
Ax.math.bc.equals(new Ax.math.BigDecimal(0.00), new Ax.math.BigDecimal(0.00)) true

Why the first cases fails ? You are passing 0.00 as right side element. This value, a javascript double, is passed to the equals function and it creates a BigDecimal using the double toString() representation that will generate it.

Unlike compareTo, equals considers two BigDecimal objects equal only if they are equal in value and scale (thus 2.0 is not equal to 2.00 when compared by this method).

##### compareTo
Expression Result
Ax.math.bc.compareTo(new Ax.math.BigDecimal(0), 0.00) 0
Ax.math.bc.compareTo(new Ax.math.BigDecimal(0.0), 0.00) 0
Ax.math.bc.compareTo(new Ax.math.BigDecimal(0.00), 0.00) 0
Ax.math.bc.compareTo(new Ax.math.BigDecimal(1), 0.00) 1
Ax.math.bc.compareTo(new Ax.math.BigDecimal(0), 1.00) -1
##### isZero
Expression Result
return Ax.math.bc.isZero(new Ax.math.BigDecimal(0)) true
return Ax.math.bc.isZero(new Ax.math.BigDecimal(0.00)) true

# 4 BigDecimal tutorial

## 4.1 Primer on Financial Issues

Currency calculations require precision to a specific degree, such as two digits after the decimal for most currencies. They also require a specific type of rounding behavior, such as always rounding up in the case of taxes.

### 4.1.1 Tax example

For example, suppose we have a product which costs 10.00 in a given currency and the local sales tax is 0.0825, or 8.25%. If we work it out on paper, the tax amount is,

Copy
10.00 * 0.0825 = 0.825

Because our precision for the currency is two digits after the decimal, we need to round the 0.825 figure. Also, because this is a tax, it is good practice to always round up to the next highest cent. That way when the accounts are balanced at the end of the day, we never find ourselves underpaying taxes.

Copy
0.825 -> 0.83

And so the total we charge to the customer is 10.83 in the local currency and pay 0.83 to the tax collector. Note that if we sold 1000 of these, we would have overpaid the collector by this much,

Copy
1000 * (0.83 - 0.825) = 5.00

### 4.1.2 Quantity example

Another important issue is where to do the rounding in a given computation. Suppose we sold Liquid Nitrogen at 0.528361 per liter. A customer comes in and buys 100.00 liters, so we write out the total price,

Copy
100.0 * 0.528361 = 52.8361

Because this isn't a tax, we can round this either up or down at our discretion. Suppose we round according to standard rounding rules: If the next significant digit is less than 5, then round down. Otherwise round up. This gives us a figure of 52.84 for the final price.

Now suppose we want to give a promotional discount of 5% off the entire purchase. Do we apply this discount on the 52.8361 figure or the 52.84 figure? What's the difference?

Copy
Calculation 1: 52.8361 * 0.95 = 50.194295 = 50.19
Calculation 2: 52.84 * 0.95 = 50.198 = 50.20

Note that we rounded the final figure by using the standard rounding rule.

See how there's a difference of one cent between the two figures? The old code never bothered to consider rounding, so it always did computations as in Calculation 1. But in the new code we always round before applying promotions, taxes, and so on, just like in Calculation 2. This is one of the main reasons for the one cent error.

## 4.2 Introducing BigDecimal

From the examples in the previous section, it should be clear that we need two things:

• Ability to specify a scale, which represents the number of digits after the decimal place
• Ability to specify a rounding method

### 4.2.1 Creating a BigDecimal

You can get a BigDecimal from a double, however it is a good idea to use the string constructor. Let's see why.

Copy
<script>
console.log(Ax.math.bc.of("1.4"));
console.log(Ax.math.bc.of(1.4));
</script>

So we need to be carefull when using Javascript numbers in our operations if they are related to the previous issues.

### 4.2.2 Rounding and Scaling

To set the number of digits after the decimal, use the setScale(scale) method. However, it is good practice to also specify the rounding mode along with the scale by using setScale(scale, roundingMode). The rounding mode specifies how to round the number.

Why do we also want to specify the rounding mode? Let's use the BD of 1.5 from above as an example,

Copy
<script>
var bd = Ax.math.bc.of(1.4);
bd.setScale(1); // This is an incorrect syntax of setScale()
</script>

It throws the exception because it does not know how to round 1.399999 .... So it is a good idea to always use setScale(scale, roundingMode).

There are eight choices for rounding mode:

Copy
<script>
console.log(Ax.math.bc.RoundingMode);
console.log("------------------------------");

var bd = Ax.math.bc.of(1.4);
console.log("Number original =" + bd);
console.log("Number scaled  4=" + bd.setScale(4, "HALF_UP"));
console.log("Number scaled 18=" + bd.setScale(18, "HALF_UP"));
</script>
Copy
<script>
function round(bd, rounding)
{
var res = bd.setScale(2, rounding);
console.log(Ax.text.String.format("%6s %6s %s", bd, res,rounding));
return res;
}

for (var mode of Ax.math.bc.RoundingMode) {
if (mode == Ax.math.bc.RoundingMode.UNNECESSARY)
continue;
round(Ax.math.bc.of("0.333"), mode);
round(Ax.math.bc.of("-0.333"), mode);
}
</script>

The following table shows results of scaling a value to tow deciml places on each mode

Mode Description 0.333 -0.333
CEILING Ceiling function 0.34 -0.33
UP .. 0.34 -0.34
DOWN Round towards zero 0.33 -0.33
FLOOR Floor function 0.33 -0.34
HALF_UP Round up if decimal >= .5 0.33 -0.33
HALF_DOWN Round up if decimal > .5 0.33 -0.33
HALF_EVEN Round half even will round as normal. However, when the rounding digit is 5, it will round down if the digit to the left of the 5 is even and up otherwise 0.33 -0.33
UNNECESSARY When you need to use one of the methods that requires input of a rounding mode but you know the result won't need to be rounded. RoundingNeccesary RoundingNeccesary

## 4.3 BigDecimal Immutability

BigDecimal numbers are immutable. What that means is that if you create a new BD with value "2.00", that object will remain "2.00" and can never be changed.

So how do we do math then?

The methods .add(), .multiply(), and so on all return a new BigDecimal instance with new value containing the result. For example, when you want to keep a running total of the order amount,

Copy
<script>
var amount = new Ax.math.BigDecimal("2");
var other  = new Ax.math.BigDecimal("3");
console.log(amount);
</script>

## 4.4 Arithmetic and precission

Some considerations about precission should be taken when using BigDecimal arithmetic operations.

• In divide operation (div), by default value returned has a minimum scale of 16 decimals. If operands has no scale defined (for example using strings or float native classes), a scale of 16 decimals is used to avoid exception in operations returning period decimal.
• In multiply operation (mul) scales of both operads are added to get the result decimal precission.
Copy
<script>
var div_floats = Ax.math.bc.div(1,  3);
console.log(div_floats);
var div_scaled = Ax.math.bc.div(new Ax.math.BigDecimal(1).setScale(4),  new Ax.math.BigDecimal(3).setScale(4));
console.log(div_scaled);
var mul_scaled = Ax.math.bc.mul("0",  new Ax.math.BigDecimal(2.15).setScale(8));
console.log(mul_scaled);
</script>

You cannot know in advance the real precission of float numbers. That's why it's recommended to use always create bigdecimals from string representation of numbers.

Copy
<script>
var d1 = Ax.math.bc.of(1.23);
var d2 = Ax.math.bc.of(2.15);
console.log("d1=" + d1);
console.log("d2=" + d2);

var mul_float = Ax.math.bc.mul(d1,  d2);
console.log("result=" + mul_float);

console.log("============================");

var d3 = Ax.math.bc.of("1.23");
var d4 = Ax.math.bc.of("2.15");
console.log("d3=" + d3);
console.log("d4=" + d4);

var mul_scaled = Ax.math.bc.mul(d3,  d4);
console.log("result=" + mul_scaled);

</script>

# 5 Nashor vs Graal script engines

Nashorn and Graal have some differences when treating BigDecimal comparisons.

• Nashorn, succesfully compares BigDecimal to int values
• Graal, fails to compare BigDecimal to int values
Operation Nashorn Graal
BigDecimal(0) == 0 true false
BigDecimal(1) == 1 true false

It's better to use always BigDecimal comparison functions to ensure application runs in both engines.