Video as background eats CPU (obviously)

[hisefilo]

Hi people! I'm wishing to add video loop to a plugin. Already made it. But I get a 90% of CPU usage.

This is how I made it (A panel with a png strip and a timer moving the strip every 40ms)
Is there a better way to do it?

Content.makeFrontInterface(1024, 680);

const var Panel1 = Content.getComponent("Panel1");


Panel1.loadImage("{PROJECT_FOLDER}Sequence0200.png", "filmstrip");
Panel1.setImage("filmstrip", 0, 0);	

Panel1.setTimerCallback(function()
{
	Panel1.setValue(1 + (Panel1.getValue() % 214));
	Console.print("Count is:" + Panel1.getValue() );
	
	Panel1.setImage("filmstrip", 0, Panel1.getValue() * 680.0);	
});

Panel1.startTimer(40);

[Sawer]

@hisefilo That's a banger! How did you convert the video into a film strip with such quality?

[Christoph Hart]

You need to be aware that this will increase the plugin size and the memory usage by a unreasonable amount:

680*214*4Bytes = 568kB / frame * 25fps = 14MB/s

so the CPU usage is only one problem (and yes it's completely unoptimised to throw in this many pixels at this frequency).

Can you use a Lottie animation? You're vector based, but this would bring it down to a few KB.

[hisefilo]

@Sawer Rendered to png frames (with any video editor). Then Strip generator
https://www.wavesfactory.com/blog/posts/strip-generator/

[hisefilo]

@Christoph-Hart Sadly it's pixel-based. The current film strip size is 300MB :anxious_face_with_sweat:

[hisefilo]

@Christoph-Hart Maybe https://docs.juce.com/master/classVideoComponent.html

??

[hisefilo]

Maybe a gif? it reduces size 10x and does not need the loop at 25fps
Any way to make a gif run?

[d.healey]

@hisefilo said in Video as background eats CPU (obviously):

Maybe a gif? it reduces size 10x

It will still be full bitmap when loaded into RAM though, no?

[hisefilo]

@d-healey Yes, but only 60mb for this quality

[A Former User]

@hisefilo I think you’d probably be much better off using Lottie or OpenGL!

Not that this would be without issues, but more performant!

[hisefilo]

@UrsBollhalder That would be great! Any quick tip on how to add a video or frames within ScriptShader?

Just found this snippet but I guess will be a long way to figure it out.

const var Panel1 = Content.getComponent("Panel1");
const var shader = Content.createShader("shader");

//shader.setBlendFunc(true, shader.GL_SRC_ALPHA , shader.GL_ONE);


Panel1.setPaintRoutine(function(g)
{
	g.applyShader(shader, [0, 0, this.getWidth(), this.getHeight()]);
});


const var Knob1 = Content.getComponent("Knob1");
inline function onKnob1Control(component, value)
{
	shader.setUniformData("myValue", value);
	shader.
};

Content.getComponent("Knob1").setControlCallback(onKnob1Control);

[A Former User]

@hisefilo I was thinking more along the lines of creating that sort of landscape and fog with shader code all along, instead of using mentioned video. I'm not 100% sure, but I don't think that'd be even possible.

Doing shader code is not for the faint of heart in my experience... But there are many talented people out there:

(www.shadertoy.com)

Maybe you can find something landscape-like on shadertoy and then you can contact the creator to discuss possible licensing...

[hisefilo]

@UrsBollhalder Yes! that's a good idea. Will see if I can find something!
Thanks bro

[Christoph Hart]

Actually the problem with most "landscapy" shaders on shadertoy is that they use some kind of textures, which isn't supported in HISE yet (you can only load fragment shaders in HISE which don't have the ability to load textures).

[hisefilo]

@Christoph-Hart Ohhh. I see.
Then... what if I do some kind of layered transparent pngs? and move them onNoteOn or whatever? (timers will kill cpu)

This are 8 layers (moons, terrain, etc)

[Christoph Hart]

Yes that might sound like the best solution - and shuffling around 8 images at 25fps shouldn't kill the CPU.

[Lindon]

@hisefilo - you could try independently scaling the layers - the "disney" technique...

[hisefilo]

@Lindon LOL true. HISE becomes a 2D animation framework

[Christoph Hart]

@hisefilo Don't say it too loud or @ulrik will recreate The Jungle Book with ScriptPanels..

[hisefilo]

@Christoph-Hart LOL