Shader vs paint routine performance

ustk

I am currently drawing signals from arrays onto a panel via paint routine. Because the signals are approx 1sec in length, this means there are more than 40000 points to draw. But in the for loop I take 1 of every 10 samples so in reality I "only" have 4000 points to draw. If on a static interface this is not a big deal, when scrolling panels in the viewport it becomes laggy as hell.

The question is, does drawing signals with a shader can substantially remove the lag behavior?

My instinct tells me yes, but I'd prefer to be sure before learning how to draw a signal array in a shader, which is certainly quite difficult.

Side question, if OpenGL is removed/not supported anymore on mac, do shaders still work?

Christoph Hart

There are a few limitations of OpenGL's shader language that prevent this use case (I think the max number of items in an uniform array is 1024.

However drawing even 4000 points with 30+ fps is definitely a bit too much for the Path renderer (I spent quite a few time in there while redesigning the sampler workspace waveform preview). Can you upload a screenshot of your paint routine? Maybe there's a clever way to optimize it without changing the entire architecture to OpenGL (which comes with its own set of problems).

ustk

@Christoph-Hart Yeah that is obvious that trying to draw as many points is not the best thing. But when you need to draw a signal, there's not much to do. Except as I said just taking one sample every X sample to lighten the process a wee bit... I tried path and direct drawing with the same laggy results. I also tried to either draw it straight away or all positive samples first then all negative ones to make a closed shape. But in the end, 4000 points stay 4000 points...

I'll post a snippet ;)

Christoph Hart

Unless you have an interface with 8000px width, you are wasting CPU time with stuff that you won't display if you create a path from 4000 data points.

One thing you could do is to implement the "viewport" yourself and then just draw the section that is visible. You hardly need more than half of the pixel width that you want to display, so you should be able to narrow it down to about 300-500 data points per paint routine. That's what happens with the sampler waveform preview (although C++ gives you much better tools to achieve this than doing it manually in HiseScript so expect to get into a world of joy here).

So if you have a signal with 44100 samples and you've zoomed in by 10x, then check the scrollbar position (might have to be a custom panel too). If it's at 50%, you'll take the sample range 22050 - 26460 and then downsample it to half of the pixel width and pass that to the path creation function. The performance of creating the paths on the fly will be much better than creating one gigantic path and then rendering it with different viewport settings naively.

ustk

@Christoph-Hart I understand, but I don't zoom, I show the 1sec length.
the viewport is for a list of child panels, where I have a 1sec signal to draw in each. No zoom stuff here, just full signal...
Though I understand it's not beneficial to draw 4000 points in a 700px length panel... I'll have a look at how to reduce this amount a bit more, but my previous attempts showed that it makes the signal ugly when I reduce this amount too drastically...

d.healey

@ustk Why do you need to draw the signal manually? Can't you use the built in audio analyser?

ustk

@d-healey That's an idea I had, but these are no wave files... I record an input signal into an array (I tried buffer but had some technical difficulties regarding the length not being dynamic)
then I do 2 things on these signals:

display on the interface as a scrolling list
and compute some statistical values between them

So I don't know if it's possible to draw them as a waveform like any other audio file loaded in a module...
Also, child panels only accept child panels, no other components are allowed to create a scrolling list with children

Christoph Hart

@ustk said in Shader vs paint routine performance:

but my previous attempts showed that it makes the signal ugly when I reduce this amount too drastically...

Well that's another problem, but hammering 4000 data points into 700 pixels is definitely the wrong implementation way. I would recommend thinking the other way around: you have 700 pixels that need to be drawn in a non ugly fashion, so how can you create 700 (or even 350) data points that represent the data in the best way? There are multiple tools that you have here:

Trying different downsampling techniques
Trying to fetch the best average value for the downsampled range (Math.min and Math.max are your friend here)...
Drawing the upper waveform line and then the lower bottom separately - I noticed that the path rendering performance gets super slow when there is a lot of up and downs, so creating a upper edge and then a lower edge that you just fill will speed up things significantly.

You might extract some inspiration from the C++ source of the HISE thumbnail path calculation here:

HISE/hi_tools/hi_standalone_components/SampleDisplayComponent.cpp at 2cb415c671298f627cf61f91541ace26a730df5a · christophhart/HISE

The open source framework for sample based instruments - HISE/hi_tools/hi_standalone_components/SampleDisplayComponent.cpp at 2cb415c671298f627cf61f91541ace26a730df5a · christophhart/HISE

GitHub (github.com)

although it's rather convoluted and ugly code :)

d.healey

@ustk said in Shader vs paint routine performance:

but these are no wave files...

But there is sound output?

ustk

@Christoph-Hart Thanks I'll try all of these.
drawing the upper part and then the lower one is what I do already. So adding a downsampling algorithm should do.
Fetching the best value though I don't know, but that shouldn't be too complicated...

@d-healey

But there is sound output?

No, just statistical computation

d.healey

@ustk said in Shader vs paint routine performance:

No, just statistical computation

Aha now I get it.

Christoph Hart

Alright here you go (it took me way too long for this example lol).

HiseSnippet 1832.3ocsXs2aabbD+nrYRHSCPLP9.L0+iIkooHSTRaiaZT0qThFqPXJamf.ifU2Mjbita2q2tmnIJBPy237MHclcuWTkx0QIQ+gD2cmG+l2bzzLcHZL5rfVcNecJFz5Ozd1Zkc4QKERUvjiCZcu1Swr45rDgJDOGM1fCWmJLFLJnUq67ELYs5b2.2O+zmenHloq9pffmqkg3WJSj15amdv+TFGepHBOWlzf58OXRnVcjNVmSP5NsGEjJBuTr.OSvjsS6fVu0IQRqNalUXQSPq6dnNZ8rk5UJO8OWZjWDi7gwAyHA4u9TcbDiX91fiVJiilVZ5l.RJSqcD2w6H9f1OQFIqtu1g79tGfZNZ5OZsylv6Na.uwMg2nFvaKPpUCHcWOjtW6YgYxTa8KLdd21STVJDIH2dSn3oMXme9caejlnPYGlHtDOMiNTwQuOYznA.8q9Ota281CdA9f3X36yMVHLCIa.DvE4ymiYvJocIcxfTDJBzyAiNWE0kNQDekHCLvmAG5ncnm2d6u+3JImFKVChLlGunrKkFvpIAh.4fnyHrRbEx4Z.wen03ILRxxjvOKl4Y3+JGUgRz.8jVhBBugKEpEnSBkuulQHeQDFJVKUKpjc+BHajpSIhIPOZ3n8cf7OBGmmjrldZgRD68.RspKwVOIS3iAI7WAmYQe7gOre2+c2NDpDQQrqQp7lP2NluU9RhgwCGA6BOQXWNjdjjwtkpsO73tcp30nSPPoklRVeHipOrj2LgJRmzqeEKoowjyDP0UXrNsjoc8JbOvoHxpF8WfGBioOLlX8GHKb2cgyHW8J7AYXg4QdFAnDxqPHkzEjabWQdUiHIMFMCgc2qaiv7YLsSEJLlLvxLqEn8HcRpVQG5c+ZRtOo3ZVU96IsTyoOUgujsutz0FcLNLMSVIIOvJiF2uIYFqHydHEuWlHxtjkv+2XkRCQT8ny5HSc.PkBYbQIjV2nqamJrNjnBOW2SN.Xur2SVqec5FpuassOzf1oTWC6S04jeF6MOWExVPuENrrfIv2qqm+OlgqVJsnONeMOwrXJv0ygp9vylPUHpHjdZA6O5by9CRMyoZDmCtxlF3p93n1WpCEwGxUkldi5uontlo0o6O3pkqShnjjbC0hv.jSaMDIrBHUSv0v2shpqEbzS9JL1bsrnYjTsu9rnZRbg7FMZ7O7ZRipnXXXLJxbAFmVO+oSN9DhsTQlAotf8LCiQ0BRT60.RLPdgLhkUQ2quQmCIxEKs.dEp.oh0mw2xoJah59PcszYttZIZsqIWcSMRNE52Un9QjjkJN4BJSL.RuNT7bQbN1SOeNkgL.7Hb.jmRYnkowOSEpSRHSeqMRcULwZ8kFPNGVSnORqdfk9MPQJm.b9nMJE.gkK74WyPadlhR38X3kkcdNm0kuma43gLz0cNDYLHXOqsr2qexQ2NwbRlCm94C0CJbLSEWlAvlVKmu4Y6J1W36RWhhyvUPBZWpiFv1FmGpPLhAPZdbrS2w7.XK7OlL6D3S6WJrL2fhOqAV3X8TTb4S4Wpaw5yvgB+.RygHXyxcIkEvBlByTEI5RII.1sSAotfD74dM8sieI7oEebzK2nyQYYsKJL85M3t45YWdrhRmXPSVhqJ9IhEJoMOxY.aM62E6PJikS3LzjCSdFm9Jr04KTnUDuRr1.gH+EDHOJM3woLQLMaJZXCQ6PFEHlkegqpiBOCfGUBrMpA2Jou4TR+pvB9BMAV5qPrvWXUTzo4bOFnT.NLmi6tW8AhKbs1tooB7bVeMoqrpFH0872rjjtvMQNgll6YbfWXvi.YepsZVN1uXXqOIZgFtf9NraAfb5SI.GVgvRo4kNiw+VArezuwXctH1z.ru.qJhBi0lhjCehghR2iPWisKn1UT8OFsQZFQeYLaywyWe7Xilr+ZFOdyUQznwmdKFMVYL2pQizfwpd3Z0YZK9UpdNqfbtv0eZ97s9FOJKSSt1rs9LuoT1qiwdp7jKvrA9FRUDRqOr4NIsu4cRZtxTnexZCB0pITWluJEU2zhTAEiioO8rIGSec.dQlh6H5nBRqjgPqiwqnsB8q0zo8wn4RxkRK3UM9On0aacu9dkK8v4LAxHdE05uiUvqZt7355CSNXEO.u5h+yOdvRjGfWSx2bfEekszTdiTbc16MpX4uPE++tlGsfoNhaSr4Vm7p1EOPg8MV0iWmSYj10MWE+2rUQeSg38ZOUZCWtcLtyVvHkb76AFKVf+8ZeBMeOzVCv619zu92ms0C78vVP8ayjTlQ6yxSlQcrBQR6JJcgK6ZsCWf3OOhOydfYTuJ2gel9o3ww74VEONt7wfDQXl96B8E67+hf2wcCgIk6+NRm1OgOCiCbM.H9ZSiNCRjQxuKLjM+GQ4aammO7VvyGcK3Y+aAOe7sfmO4Vvye5Vvye90xC+OJ5uma0I9xA5hom35J1p0IJAkY4xBC9uTMEYiA

I've added a few convenient methods to the buffer class, so you need to pull the latest commits.

This is the code. It uses one naive path rendering using all sample points and then the smart downsampled solution where it looks for a upper and lower value for each pixel.

Content.makeFrontInterface(600, 600);

// We'll just create a buffer with a second of sound
const var s = Buffer.create(44100);

// play around with this to see how the waveform reacts with different
// frequencies (it will change the frequency of the decaying waveform)
const sinFreq = 0.04;

//! Dummy signal creation
for(i = 0; i < 44100; i++)
{
	// add a sine wave
	s[i] = 1.0 * Math.sin(i * sinFreq) ;
	
	// add some noise
	s[i] += 0.2 * Math.random();
	
	// apply a envelope
	s[i] *= 1.0 / (i * 0.009 + 10.01);
}

/** Now we're creating a naive path using all samples. */

const var NaivePanel = Content.getComponent("NaivePanel");
const var naivePath = Content.createPath();

Console.print("Naive path creation");

Console.startBenchmark();
for(i = 0; i < 44100; i++)
{
	// no downsampling, terrible performance
	naivePath.lineTo(i, s[i]);
}

Console.stopBenchmark();

NaivePanel.setPaintRoutine(function(g)
{
	g.setColour(Colours.white);
	
	Console.print("Slow (naive) UI rendering");
	Console.startBenchmark();
	g.fillPath(naivePath, this.getLocalBounds(0));
	Console.stopBenchmark();
	
});

/** Now we'll use as many data points as we have pixels */

const var SmartPanel = Content.getComponent("SmartPanel");

const var smartPath = Content.createPath();
smartPath.clear();

var STRIDE = parseInt(s.length / SmartPanel.getWidth());

// You might even increase the downsample factor to smooth the waveform
//STRIDE *= 1.3;

inline function getSmartValue(offset, length, upper)
{
	// Uncomment this to see how the path looks if you don't do any
	// smart downsampling at all
	//return s[offset];
	
	// This will create a reference to a part of the buffer
	local thisBuffer = Buffer.referTo(s, offset, length);
	local value = 0.0;
	
	// New method, you'll need to pull the latest HISE :)
	local range = thisBuffer.getPeakRange();
	
	// Now we return either the highest or lowest value
	return upper ? range[1] : range[0];
}

Console.print("Smart Path creation");
Console.startBenchmark();

var normPeak = s.getMagnitude();

smartPath.clear();

// These lines ensure that the path is always centered and normalised.
smartPath.startNewSubPath(0.0, -normPeak);
smartPath.startNewSubPath(0.0, normPeak);
smartPath.startNewSubPath(0.0, 0.0);

// Go through the sample once and calculate the upper bounds
for(i = 0; i < 44100; i += STRIDE)
{
	smartPath.lineTo(i, getSmartValue(i, Math.min(STRIDE, 44100 - i), true));
}

// Now go back and calculate the lower bounds.
for(i = 44100 - STRIDE; i >= 0; i -= STRIDE)
{
	smartPath.lineTo(i, getSmartValue(i, Math.min(STRIDE, 44100 - i), false));
}

// We need to close that path in order to be filled
smartPath.closeSubPath();

Console.stopBenchmark();

SmartPanel.setPaintRoutine(function(g)
{
	g.setColour(Colours.white);
	Console.print("Smart Path UI Rendering");
	Console.startBenchmark();
	g.fillPath(smartPath, this.getLocalBounds(0));
	Console.stopBenchmark();
})

Debug Output:

Interface: Naive path creation
Interface: Benchmark Result: 121.742 ms
Interface: Smart Path creation
Interface: Benchmark Result: 24.656 ms
Interface: Slow (naive) UI rendering
Interface: Benchmark Result: 2.642 ms
Interface: Smart Path UI Rendering
Interface: Benchmark Result: 0.060 ms

You get a 50x performance gain like this :)

ustk

@Christoph-Hart said in Shader vs paint routine performance:

You get a 50x performance gain like this

:smiling_face_with_heart-eyes:

ustk

@Christoph-Hart That is very cool, I'll try a cherry-pick then because for this project I am using a modified version of develop

d.healey

@Christoph-Hart said in Shader vs paint routine performance:

I've added a few convenient methods to the buffer class, so you need to pull the latest commits.

I'm not seeing them in github.

ustk

@d-healey @Christoph-Hart Yeah strange, only the name of the set method changed...

EDIT: but it works, it was just a redundant function name... Thanks Chris!

ustk

@Christoph-Hart Why did you need to add the getPeakRange function? This is just a [max, min]array right? So easily doable with hise script?

Christoph Hart

But FASTER!!!

Shader vs paint routine performance

HISE/hi_tools/hi_standalone_components/SampleDisplayComponent.cpp at 2cb415c671298f627cf61f91541ace26a730df5a · christophhart/HISE

27

1.8k

12.0k

104.2k