Round-off error in rateRange control in prefs window

[Sean M. Pappalardo]

While tuning a MIDI controller's pitch range LEDs, I discovered that the rateRange" control introduces round-off error somewhere. When I had the controller set the range from 0.01 through 0.05 it was fine, but 0.06 had a repeating decimal, causing the LED equals checks to be skipped. (I worked around it in the script by rounding off to two places, but this should probably be checked in the engine incase it's messing anything else up.)

[Phillip Whelan (pwhelan)]

I tried this out in my working copy of trunk.

This is the script snippet I added to midi-mapping-script.js:

var rateRanges = [0.3, 0.6, 0.7, 0.8, 0.9];

for (var i = 0; i < rateRanges.length; i++) {
    engine.setValue("[Channel1]","rateRange", rateRanges[i]);
    print("rateRange=" + engine.getValue("[Channel1]","rateRange"));
}

And here is the results I got:

rateRange=0.3
rateRange=0.6
rateRange=0.7
rateRange=0.8
rateRange=0.9

I also added a slot hooked to rateRanges valueChanged signal and it spit this out:

Debug: [Main]: DEBUG VALUE: 0.3
Debug: [Main]: DEBUG VALUE: 0.6
Debug: [Main]: DEBUG VALUE: 0.7
Debug: [Main]: DEBUG VALUE: 0.8
Debug: [Main]: DEBUG VALUE: 0.9

[Phillip Whelan (pwhelan)]

I also tried one negative power of ten lower and these are the results:

rateRange=0.03
rateRange=0.06
rateRange=0.07
rateRange=0.08
rateRange=0.09

Debug: [Main]: DEBUG VALUE: 0.03
Debug: [Main]: DEBUG VALUE: 0.06
Debug: [Main]: DEBUG VALUE: 0.07
Debug: [Main]: DEBUG VALUE: 0.08
Debug: [Main]: DEBUG VALUE: 0.09

[Sean M. Pappalardo (pegasus-renegadetech)]

The problem appears to be in the preferences dialog. Setting the range to 8% for example then polling the value from MIDI script reveals it to be 0.079999... and this happens with most of the selections in the dialog.

[Phillip Whelan (pwhelan)]

I've seen it before but I cannot confirm it is still happening, at least not with that snippet I sent you.

I tried resetting the value from preferences. I still do not get that rounded off value. Perhaps I should turn on debugging again in my BPMSlider script (which is where I've seen the problem before).

[RJ Skerry-Ryan (rryan)]

0.6 is not a valid double precision number, so this is not something we can fix. Marking invalid :).

[Swiftb0y (swiftb0y)]

Mixxx now uses GitHub for bug tracking. This bug has been migrated to:
https://github.com/mixxxdj/mixxx/issues/5356