How can I get rid of these floating point errors?
-
Occasionally I am getting values like this when doing math:
How can I limit this to a decimal place? -
@VirtualVirgin
Math.round()
,Math.floor()
,Math.ceil()
,Engine.doubleToString()
-
@d-healey
In this case I am already using Math.ceil(), Math.floor() and Math.round()
then I am only multiplying and diving by powers of 10 to get a rounded decimal:// gives a correct return for proper round (JUCE has somehow broken the rounding math) inline function roundFix(value) { local remainder = Math.fmod(value, 1.0); local roundedValue; remainder >= 0.5 ? roundedValue = Math.ceil(value) : roundedValue = Math.floor(value); return Math.round(roundedValue); }; // round to a decimal place inline function roundToDecimal(value, decimalPlace) { if (decimalPlace == 0) return roundFix(value); local multiplicationFactor = Math.pow(10, decimalPlace); local multipliedValue = value * multiplicationFactor; local roundedMultipliedValue = roundFix(multipliedValue); local result = roundedMultipliedValue / multiplicationFactor; return result; }; // test decimal rounding const decimalRoundTest = roundToDecimal(78.678, 2); Console.print("decimalRoundTest: " + decimalRoundTest);
I can't use Math.ceil(), Math.floor() and Math.round() on the end return value because I need it to keep the decimal places.
If I use Engine.doubleToString() then I don't see a way to get the value back. If I use .getIntValue() I lose the decimal places.
-
@VirtualVirgin Why do you want less precision?