Round-off error in rateRange control in prefs window

Bug #533477 reported by Sean M. Pappalardo on 2010-03-06
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Mixxx
Low
Sean M. Pappalardo

Bug Description

While tuning a MIDI controller's pitch range LEDs, I discovered that the rateRange" control introduces round-off error somewhere. When I had the controller set the range from 0.01 through 0.05 it was fine, but 0.06 had a repeating decimal, causing the LED equals checks to be skipped. (I worked around it in the script by rounding off to two places, but this should probably be checked in the engine incase it's messing anything else up.)

Phillip Whelan (pwhelan) wrote :

I tried this out in my working copy of trunk.

This is the script snippet I added to midi-mapping-script.js:

var rateRanges = [0.3, 0.6, 0.7, 0.8, 0.9];

for (var i = 0; i < rateRanges.length; i++) {
    engine.setValue("[Channel1]","rateRange", rateRanges[i]);
    print("rateRange=" + engine.getValue("[Channel1]","rateRange"));
}

And here is the results I got:

rateRange=0.3
rateRange=0.6
rateRange=0.7
rateRange=0.8
rateRange=0.9

I also added a slot hooked to rateRanges valueChanged signal and it spit this out:

Debug: [Main]: DEBUG VALUE: 0.3
Debug: [Main]: DEBUG VALUE: 0.6
Debug: [Main]: DEBUG VALUE: 0.7
Debug: [Main]: DEBUG VALUE: 0.8
Debug: [Main]: DEBUG VALUE: 0.9

Phillip Whelan (pwhelan) wrote :

I also tried one negative power of ten lower and these are the results:

rateRange=0.03
rateRange=0.06
rateRange=0.07
rateRange=0.08
rateRange=0.09

Debug: [Main]: DEBUG VALUE: 0.03
Debug: [Main]: DEBUG VALUE: 0.06
Debug: [Main]: DEBUG VALUE: 0.07
Debug: [Main]: DEBUG VALUE: 0.08
Debug: [Main]: DEBUG VALUE: 0.09

The problem appears to be in the preferences dialog. Setting the range to 8% for example then polling the value from MIDI script reveals it to be 0.079999... and this happens with most of the selections in the dialog.

summary: - Round-off error in rateRange control
+ Round-off error in rateRange control in prefs window
Phillip Whelan (pwhelan) wrote :

I've seen it before but I cannot confirm it is still happening, at least not with that snippet I sent you.

I tried resetting the value from preferences. I still do not get that rounded off value. Perhaps I should turn on debugging again in my BPMSlider script (which is where I've seen the problem before).

Changed in mixxx:
assignee: nobody → Pegasus (pegasus-renegadetech)
importance: Undecided → Low
milestone: none → 1.8.0
RJ Skerry-Ryan (rryan) wrote :

0.6 is not a valid double precision number, so this is not something we can fix. Marking invalid :).

Changed in mixxx:
status: Confirmed → Invalid
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers