FFT Analyser Path - Need help drawing the magnitude to height

ustk

@HISEnberg I've had a look at your code, is there a particular reason you are building the path manually instead of using the DisplayBuffer.createPath() method?

mapping the magnitude (or peak value) onto an oscilloscope or path.

I'm not sure I understand. But I might have done weird things lol

HISEnberg

@ustk Thanks for taking a look! In this particular case I am attempting to create a subpath so that I can ignore painting/drawing any values that are already displayed. I wasn't able to do this with DisplayBuffer.createPath, thus the subPath.

Long story short is I have one project with a very large FFT display and users are reporting a lot of UI lag, so I suspect this is the culprit. I just cracked the magnitude issue however! This is the current draft of the code (though still a mess from working with AI on this, I am in the process of cleaning it up):

To be honest I don't actually see much performance benefit to this method so far so this might be an effort in vain. The only other real benefits of this script is it will allow you to define the bins (I know you can do this already), but also assign different colour gradients, etc.

Content.makeFrontInterface(400, 400);
 
namespace MinimalFftAnalyser
{
    // Core components
    const var Analyser1 = Synth.getEffect("Analyser1");
    const var source = Synth.getDisplayBufferSource("Analyser1");
    const var fftTimer = Engine.createTimerObject();
    const var pnl_Fft = Content.getComponent("pnl_Fft0");
    
    // Display buffer
    pnl_Fft.data.buffer = source.getDisplayBuffer(0);
    pnl_Fft.data.buffer.setActive(true);
    pnl_Fft.data.path = Content.createPath();
    
    // Processing buffers
    const buff = Buffer.create(pnl_Fft.data.buffer.getReadBuffer().length);
    reg lastPoints = [];

    // Configuration constants
    const THRESHOLD = 1.0;   
    const SIGNAL_DECAY = 0.1; 
    const SILENCE_THRESHOLD = 0.05; 
    const DISPLAY_BINS = 4098;  // Increase/Decrease for more detail
    const HEIGHT_SCALE = 0.5;  
    const CURVE = 0.99;
    
    // Frequency mapping parameters
    const MIN_FREQ = 20;       // Hz - lowest frequency to display
    const MAX_FREQ = 20000;    // Hz - highest frequency to display
    const SAMPLE_RATE = Engine.getSampleRate(); // Hz - adjust to match your system
    Console.print(SAMPLE_RATE);
    
    // Amplitude response settings
    const AMPLITUDE_DECAY = 0.95;     // Higher values = slower decay
    const AMPLITUDE_ATTACK = 0.3;     // Higher values = faster attack
    const MAX_MAGNITUDE = 0.1;        // Lowered to make visualization more sensitive
    const MIN_DISPLAY_DB = -60;       // Minimum dB level to display (lower = more sensitive)
    
    const button = Content.getComponent("Button1");
    
    // Fixed scaling factor - adjust for desired sensitivity
    const scaleFactor = 4.0;   // Increased for better visibility
    
    // State variables
    reg signalLevel = 0.0;
    reg smoothedMagnitudes = [];      // Array to store smoothed magnitude values
    
    // Button callback for enabling/disabling the FFT
    inline function onButton1Control(component, value)
    {
        if (value == 1)
            fftTimer.startTimer(30);
        else
            fftTimer.stopTimer();
    }
    
    button.setControlCallback(onButton1Control);
    
    // Initialize the FFT system
    inline function initialize()
    {
        pnl_Fft.setPaintRoutine(fftPaintRoutine);
        updateFFT();
        fftTimer.setTimerCallback(updateFFT);
        fftTimer.startTimer(30);
    }
    
    // Main paint routine for the FFT panel
    inline function fftPaintRoutine(g)
    {
        local bounds = this.getLocalBounds(0);
        local w = bounds[2];
        local h = bounds[3];
        local path = this.data.path;
        
        g.setColour(Colours.withAlpha(0xFFFFFFFF, signalLevel));
        g.fillPath(path, [0, 0, w, h]);
    }
    
    // Converts linear magnitude to decibels with a minimum threshold
    inline function linearToDb(linearValue)
    {
        if (linearValue < 0.000001) 
            return MIN_DISPLAY_DB;
            
        local dbValue = 20.0 * Math.log10(linearValue);
        return Math.max(dbValue, MIN_DISPLAY_DB);
    }
    
    // Maps decibels to display height with a better curve
    inline function dbToDisplayHeight(db, height)
    {
        local normalized = (db - MIN_DISPLAY_DB) / (-MIN_DISPLAY_DB);
        normalized = Math.max(0, Math.min(1, normalized));
        normalized = Math.pow(normalized, CURVE);
        
        return height/2 - (normalized * height/2 * HEIGHT_SCALE);
    }
    
    // Detects signal level from buffer data with improved decaying
    inline function detectSignalLevel()
    {
        local magnitude = buff.getMagnitude(0, buff.length);
        
        // Use slower decay for a more natural fade out
        signalLevel = Math.max(magnitude * scaleFactor, signalLevel * SIGNAL_DECAY);
        signalLevel = Math.min(1.0, signalLevel);
        
        return signalLevel;
    }
    
    // Handles silence or very low signal
    inline function handleSilence(path, w, h)
    {
        // Reset smoothed magnitudes during silence
        smoothedMagnitudes = [];
        
        for (i = 0; i < lastPoints.length; i++)
            lastPoints[i] = h/2;
        
        path.lineTo(w, h/2);
        path.lineTo(w, h/2);
        path.lineTo(0, h/2);
        path.closeSubPath();
        
        return path;
    }
    
    // Creates the FFT path with optimized points and improved frequency distribution
    inline function createFilteredPath()
    {
        local bounds = pnl_Fft.getLocalBounds(0);
        local w = bounds[2];
        local h = bounds[3];
        
        local path = Content.createPath();
        path.startNewSubPath(0, h/2);
        
        local actualBins = Math.min(DISPLAY_BINS, buff.length);
        
        detectSignalLevel();
        
        if (lastPoints.length != actualBins)
        {
            lastPoints = [];
            for (i = 0; i < actualBins; i++)
                lastPoints[i] = h/2;
        }
        
        // Initialize smoothedMagnitudes if needed
        if (smoothedMagnitudes.length != actualBins)
        {
            smoothedMagnitudes = [];
            for (i = 0; i < actualBins; i++)
                smoothedMagnitudes[i] = 0.0;
        }
        
        if (signalLevel < SILENCE_THRESHOLD)
            return handleSilence(path, w, h);
        
        // Calculate the proper bin-to-frequency mapping
        local totalBins = buff.length;
        local binFrequencies = [];
        
        // Pre-calculate bin frequencies
        for (i = 0; i < actualBins; i++)
        {
            local freq = (i / totalBins) * (SAMPLE_RATE / 2);
            binFrequencies[i] = freq;
        }
        
        // Now draw the path using the frequency mapping
        for (i = 0; i < actualBins; i++)
        {
            local freq = binFrequencies[i];
            
            // Skip if frequency is out of our visible range
            if (freq < MIN_FREQ || freq > MAX_FREQ)
                continue;
                
            // Logarithmic mapping of frequency to x position
            local logPos = Math.log(freq / MIN_FREQ) / Math.log(MAX_FREQ / MIN_FREQ);
            local x = logPos * w;
            
            // Get current bin magnitude
            local magnitude = Math.abs(buff[i]);
            
            // Apply envelope smoothing for each bin
            // Fast attack, slow decay
            if (magnitude > smoothedMagnitudes[i])
                smoothedMagnitudes[i] = smoothedMagnitudes[i] * (1 - AMPLITUDE_ATTACK) + magnitude * AMPLITUDE_ATTACK;
            else
                smoothedMagnitudes[i] = smoothedMagnitudes[i] * AMPLITUDE_DECAY;
            
            // Convert to dB for better visualization
            local dbValue = linearToDb(smoothedMagnitudes[i]);
            
            // Map dB to display height
            local y = dbToDisplayHeight(dbValue, h);
            
            // Ensure valid range
            y = Math.max(0, Math.min(h, y));
            
            if (Math.abs(y - lastPoints[i]) >= THRESHOLD)
            {
                path.lineTo(x, y);
                lastPoints[i] = y;
            }
            else
            {
                path.lineTo(x, lastPoints[i]);
            }
        }
        
        path.lineTo(w, lastPoints[actualBins-1]);
        path.lineTo(w, h/2);
        path.lineTo(0, h/2);
        path.closeSubPath();
        
        return path;
    }
    
    // Main update function called by the timer
    inline function updateFFT()
    {
        pnl_Fft.data.buffer.copyReadBuffer(buff);
        pnl_Fft.data.path = createFilteredPath();
        pnl_Fft.repaint();
    }
    
    // Initialize everything
    initialize();
}

ustk

@HISEnberg Have you checked the perf with the new profiler? This would help finding the bottleneck

HISEnberg

@ustk Precisely what I am using, it's been super helpful and I really appreciate Christoph's work on this! Unfortunatley it's a really strange issue. Generally (90% of cases) the plugin runs completley fine, and there really is nothing super complex about it. However for about 10% of users there is some serious lag and even crashes their DAW and I can't put my finger on whats causing this (I haven't been able to recreate the issue). So I am trying to minimize the imapct of any UI/Script callbacks and paint routines

ustk

@HISEnberg Maybe you could just try with the stock createPath to see if it's stable and if the issue is about UI drawing / message thread

Is your interface script deferred?

Long ago I had a lot of crashes doing that kind of things. It's now stable and I reckon I added extra security here and there.

Also when working with a buffer, I tend to create a copy to work with instead of the original. Afraid that another thread want access at the same time (I don't always know if the writeLock is set, so...)

HISEnberg

@ustk Ah great tips thank you!

My OG script is using the createPath method. It performs well (generally better than the HISE stock display with the floatingTile).

The interface script deferred, does that just mean the Synth.deferCallbacks(true), or are you referring to the specific script?

Interesting suggestion about copying the buffers. I would have thought there was more overhead this way but I could see how this is more safe for threading.

ustk

@HISEnberg said in FFT Analyser Path - Need help drawing the magnitude to height:

does that just mean the Synth.deferCallbacks(true), or are you referring to the specific script?

Interface script should always be deferred. Included scripts are just part of interface so they inherit from it.
Scripts that are dealing with midi/audio shouldn't be deferred

oskarsh

@HISEnberg said in FFT Analyser Path - Need help drawing the magnitude to height:

@ustk Precisely what I am using, it's been super helpful and I really appreciate Christoph's work on this! Unfortunatley it's a really strange issue. Generally (90% of cases) the plugin runs completley fine, and there really is nothing super complex about it. However for about 10% of users there is some serious lag and even crashes their DAW and I can't put my finger on whats causing this (I haven't been able to recreate the issue). So I am trying to minimize the imapct of any UI/Script callbacks and paint routines

I am running into this same issue with my new Plugin update. It generally works well but for some Windows users it would lag to the point it would not be possible to use.

I've tested with the few users with the issue to get the FFT performance up to speed with no luck so far!

@HISEnberg make sure to use IPP or FFTW3 on windows, this should boost the performance significantly. However it did not seemed to help here.

I tested to have
no IPP -> Performs BAD
IPP -> Performans BAD
FFTW3 -> Performans BAD
Synth deferred callbacks -> no change

This only happens to around 10% of Windows Users tho. MacOS and Linux is completly fine.

oskarsh

@ustk @HISEnberg

I am using a quite complex graphics but MacOS and even my cheap Windows system are working quite nicely.

Main Light-min.png

Not sure how to get the FFT performance up for these couple of Windows users and actually what is happening on their system. They use Win 11 Ableton 12. I also of that bug and the crashing of the DAW.

@Christoph-Hart maybe you can help here or share some insights.

HISEnberg

@oskarsh Ah very interesting you are coming across this issue as well!

Ya I am finding that the IPP is boosting performance but isn't solving the issue for these handful of users. It's frustrating since 10% is too many user reports to ignore (maybe 5% would have been acceptable).

I had the same exact experience as you. I booted up a 12 year old laptop that runs on Windows 8 (I thought it was dead) and ran my plugin without any problems. I also had another user who has nearly the exact same Windows setup as me (Lenovo Ideapad Slim7, Ryzen 7, AMD Radeon Graphics Card) reporting this issue.

Maybe you will have better luck with this. I created a profiling script which records 1-5 seconds of the user interacting with the plugin. Afterwards it will just dump the snippet onto the users desktop. You can import the profiling session and get a snapshot of how the plugin is performing on the users system (time it takes for paint routines, processing on the different threads, control callbacks, etc.) No one sent me the file back but you could possibly ask one of your users to try this and get some insight into what aspect of the FFT is causing the plugin to choke.

Christoph has actually already given me some advice here about what to do. The one thing I haven't tried which might solve this is setting your FFT script to a background task. I am still wrapping my head around how threading works in HISE (and audio plugins more generally), but I could see how an FFT which is communicating between the audio thread and the UI/Scripting threads could be unsafe if the pointers on the JUCE end aren't setup correctly. I think this is something that the Synth.deferCallbacks should be addressing but possibly not in this particular case.

So if anyone has a snippet on how to use the background task API and can share I would really appreciate it!

ustk

@HISEnberg Perhaps something like this:

HiseSnippet 1642.3ocuX0saaaCElxNJowKcnEXWLrqD5UtCot1yoosXXHt1ItKXMIFwYAcXnHiVh1hvRjZRT0wan.CX2zWgd01qQuYuA6p8BrGg9FrcHorsjkSyhQWLRaLO+wuygmyGYamPtMIJhGhLJcx3.BxXCytiYB2VtXJCs+tHiO0rc6SNaDU31DaObPHOl4bBNZHp43.bTDwAYXT7oRqMVeEj5y61oI1CyrIyDgPmxo1jmQ8ohYR6z3anddswNjSn9ordqF6ayYs3d7X.YEMqhBf8FOfbHVZVASzWiibQFetYsG8Xa7V1OX6Zjs5ssyCc1tm8irq2GW24wO.WW80sqWuNxX08bnBdXWAVPhPFqzj6LtqKeDSuAmRin87HxE0PcgcVKtM2yQlhRonVtTOmNSJYQHHJclU.KpKfeh4ATG5T4yJj2RovZlGoKfFExBuhYfWszvqZJ3s.HYjBRqngzsM6ZGRCDyzHwyGYtOSPB6igyozPQaKpveWzrEGrfIp3iGRZGBKl5Q4sqVcSK3ut6WVBrJh6QpX6QvgkAAkfCuHg0KwgVPuiNdset0WYoZspLfH1qeehsn7c.0VSzemrd1uunCyCbZBH.2Zw8C3LXQ46nUOmO61ra5cYWZTfGdbyXX2B6B8R.tysko81B7FhQNeKWMigGmXnztiIXmDixhklO8Dvn8XCnLnzDRfy0rCPkAKjtDRFXEfEtoRUs4c.gpfBezoakHhPNoD1B640ChV49wLaAkyJe2R+bo0gHVwFTcDK0V4BAyoL.UY7fv8J3OSBm.GpCX45U06z8uuUK0taIbIZbQYpu2aZHKQYdPVYMYysRBtJij3HIcfJTpLIYOAaeFG.YSYjhfJ6lVe+8psoE7CzQcuZu.9cEIXVOwgPR.LBIjUh0KsdHQDGxfrHaUoizji4wB.WyJJCTnYPk9.KyS77JqYThpLxkJHx3MP5qV5Tk87fDUuYCp3DhGoPuLk1TUOjYPB3qUQB+ZJ39pjSpbelVk3rC4BxQImUPFXMup98WnNYaQH2yS1is.05Cv2iikYw98HgaB8kdwjoFB7CYIcLuXRmzbh15tzTFxY6ynhiBHrKhoDkzZCe6a2eWr.KYpRjA1EPBETIDL1k7R3dBMu05l6RhFJ3AJaSF8AlbgR6MmvpgYDODE3ttgotg.c9rqR9tFimsvowHpivcpfe37FtD5.2TWIQafxSfBT2bmXOrHKet7VuDEP8NCIpjnjEQEiSeq3GLR9+qP71lcnBa2EiwBK.ivox+GXL4pwaZp48mAvULa+7qg6AM06eIyID+pM+iMSeWPFXbXrelK.jCGFFPRkb4nwqMmMmEHYnBImv6.1WNB6G3QNFf+lV8731C6R+IR9gy.M5ZJsnrsKlAMwQKyL7p+mKX0d+mWZ1yAGfEgTX.xDpA56LakfNYMnfbTVutpbsrkoKg4nV7OvmDk0lVv.k0lnLEKvgDwHd3P0IUx2QFqIOV1vLRUjOq+40P.qMejb1mlz.CGNJYc3diCb4LpsTj1hIH8I9.8rXBbgmJdBl5I636FGAbTNGw5BFqdepQwSgSWUfWyDt2oBjHENj6.pVsM1FJrikz+xQIIsGz2RBqXOs+MCVyz.odnZF0xnN4Is5BGfZeEiF7fVihonA0cuxUvrYA8FsezoRsvMmnSks.x7QESo4ESfr1TdjMbgmjJ.kKGBIURodQPNk5qBjkpfF+ymPtO27y7Rj4Eygb9jVi8ftaG3o1J2mztzAGBfRn8e0oqfypCnrjpvLpqCvmOmra0nqfDHG.SSC2cHYjttjVprzspjDkfxEkcI8wwdhb6H58CosZjCRu9ulGR+9adyayCoF+p+eti9U5sCI+XLzsNdNb8KCWLt.4WBtd2N4KUUuhkpMT3x5X4TIJW7WbA6c6ft5mg.HthmgOEXBu1gDvzcgPZMyNt3nbXBgVx9pODXZU0qWlAo3+329r6+h2dwkoTLLlx2YkkZ4llXF1abDohTUdZEsKWS7Ix.eo7IoSn0LU+aWylRaX5KesuVS9LZhOWRNUHSNkgPy35nySGfknySB2lb9Per5V5k5Y6WKuovGaGxOyV+LIYc8FJIP6DScPst4Ax0V0Pub9r0m5POy1Nanx43WrrNVeYcbqk0wGrrNt8x53CWVGezk6n7+urmDK395m9ACDc1SOOarGCCupUMNh9W.KKnpjB

Here it is only the path generation that is handled in the bg, not the painting. But I reckon the painting already happens in the bg as it is not in sync with the message thread (I might be wrong though...)

Don't forget to handle the BG status and/or shouldAbort cases if needed

FFT Analyser Path - Need help drawing the magnitude to height

16

1.7k

11.8k

102.4k