Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Accurate floating point arithmetic in JavaScript

I am creating number spinner widget in JavaScript to essentially mimic the number field in webkit.

When you change the number, it needs to check to see if the value is not only within the accepted range, but also that it's in step:

<input type="number" min="0" max="100" step="1" />

If a user enters 5.5 the field will truncate this to the closest step lower than the value, which in this case is 5.

For a step of 2, if the user entered 5.5, the result would be 4.

The equation I was planning on using for this calculation looks like this:

...code...
_checkStep: function (val) {
    val ret,
        diff,
        moddedDiff;
    diff = val - this._min;
    moddedDiff = diff % this._step;
    ret = val - moddedDiff;
    return ret;
},
//set elsewhere
_min,
_step,
...more code...

Although for testing purposes, you could simply use this:

function checkStep(val, min, step) {
    var ret,
        diff,
        moddedDiff;
    diff = val - min;
    moddedDiff = diff % step;
    ret = val - moddedDiff;
    return ret;
}

This works great for integers and larger values, however I've run into issues with decimals due to how JavaScript handles floating point numbers.

For example:

checkStep(0.5, 0, 0.1) //returns 0.4, 0.5 is expected

In analyzing each line, it turns out that 0.5 % 0.1 in JavaScript returns 0.09999999999999998.

What can be done to make this function more accurate*?


*accurate being that it works for increments of 0.01 and up.

like image 761
zzzzBov Avatar asked Sep 21 '26 04:09

zzzzBov


1 Answers

You could try making sure step is greater than 1 (by repeatedly multiplying by 10), then do your modulus, then scale back down to original. For example:

function checkStep(val, min, step) {
  var ret,
    diff,
    moddedDiff;
  var pows = 0;
  while( step < 1 ) { // make sure step is > 1
    step *= 10;
    val *= 10;
    min *= 10;
    pows++;
  }
  diff = val - min;
  moddedDiff = diff % step;
  ret = val - moddedDiff;
  return ret / Math.pow( 10, pows );
}

This works for the examples you provided, but I can't guarantee it will work for everything. See the jsfiddle here:

http://jsfiddle.net/bCTL6/2/

like image 156
Ethan Brown Avatar answered Sep 22 '26 18:09

Ethan Brown



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!