added scriptable FFT

Christoph Hart

@iamlamprey said in added scriptable FFT:

I think FFTs can also be used for time-stretching stuff right?

The FFT is maybe the most important algorithm in DSP but let's keep it realistic guys. This implementation works within the scripting layer and any attempt for realtime usage will at some point run against performance issues: As soon as you need to operate on the actual sample data, it will burn the CPU as @dustbro has shown with its infamous CPU Screamer example.

At some point in the future I will add the FFT to the SNEX API, with an emphasis on realtime usage, but until then just relax and enjoy the possibility of sorting a sample set based on its harmonic spectrum :)

A Former User

@christoph-hart Of course! :) keep up the great work

d.healey

Could this be used to compare dynamic layers and create an aet type thing?

Dan Korneff

@christoph-hart said in added scriptable FFT:

it will burn the CPU

Wait till you see my analog tape dsp :grinning_squinting_face:

Christoph Hart

@d-healey Yes that's more in the ballpark. However it would require an inverse FFT so that you can do (in pseudo code):

for(chunk in signal)
{
    forwardTransform(chunk);
    doSomeStuffInFrequencyDomain();
    inverseTransform(chunk);
}

However I gotta admit I have never used the inverse transform (all FFT application I did until now only needed to analyse the frequency domain) so maybe I need a bit input of someone who is a bit more proficient in this area. A quick googling revealed this website which contains some basic information, but let me know if you know better resources:

Overlap-Add Decomposition | Spectral Audio Signal Processing

(www.dsprelated.com)

ustk

@christoph-hart as for a non-real time decimation of a buffer, it should work right? Just in order to get something better than than one sample every X samples…
This is not related to displaying the buffer, which works great btw thanks to you :)

Christoph Hart

@ustk said in added scriptable FFT:

Just in order to get something better than than one sample every X samples…

Ah, you're still in the waveform drawing stuff? Not sure if the FFT is seriously overkill for this task. Have you tried other interpolation algorithms? The next best thing is linear interpolation which should be good enough (it's good enough for the sample playback lol).

ustk

@christoph-hart no no as I said this is not for drawing, this one is for computational stuff.
Actually I am taking a sample every X samples to decimate before a correlation function, and it works. But I'd like a more precise decimation model…

What I found online always uses an fft, but the formulas are too complex for me to translate into code. Although I'm sure it's a trivial operation, I'm just not used to fft…

I imagine this requires an inverse fft too though

d.healey

This post is deleted!

d.healey

What about a resynthesis module? :D

A Former User

@d-healey with the results I got from DDSP I'm surprised more developers aren't looking into resynthesis more... we'd never have to sample a round robin again

d.healey

@iamlamprey It would be nice if we never had to sample anything again.

Christoph Hart

@iamlamprey ~~What's DDSP?~~ Nevermind it's more googleable than I thought.

A Former User

@christoph-hart Yep :) The style-transfer is particularly appealing, I wouldn't be surprised if you can legally train it with other people's sample libraries since it's 100% synthesis.

Not a lawyer but ;)

d.healey

@Christoph-Hart I thought I'd have a go at this fft thing. Following your instructions above and the docs I load a sample from the sampler into a buffer. If I uncomment the process line in this script HISE will just crash.

const var Sampler1 = Synth.getSampler("Sampler1");

reg sound = Sampler1.createSelection(".*")[0];

const buf = sound.loadIntoBufferArray();

const fft = Engine.createFFT();
fft.setEnableSpectrum2D(true);
fft.setWindowType(fft.BlackmanHarris);

fft.setMagnitudeFunction(function(data, offset)
{
	var max = 0.0;
	
	for(s in data)
	{
		max = Math.max(max, s);
	}
	
	Console.print("The max value at " + offset + " is " + Engine.getDecibelsForGainFactor(max));
	
}, true);

fft.prepare(1024, 2);

//fft.process(buf);

d.healey

@d-healey Bump

Orvillain

@d-healey Did you ever resolve this Dave?

d.healey

@Orvillain Not as far as I recall

Orvillain

@d-healey

I'm not running into a crash, primarily because I'm not enabling the 2D Spectrum.

But a very simple example like this:

Content.makeFrontInterface(600, 600);

const var AudioLoopPlayer1 = Synth.getAudioSampleProcessor("Audio Loop Player1");
const var audioFileObj = AudioLoopPlayer1.getAudioFile(0);
const var sampleRate = audioFileObj.getSampleRate();
const var audioData = audioFileObj.getContent(); // [channel][samples]
const var fft = Engine.createFFT();
fft.prepare(1024, 1); // 1 channel, 1024 sample size
fft.setWindowType(fft.Hann);
fft.setOverlap(0.75);

fft.setMagnitudeFunction(function(data, offset)
{
    Console.print("Frame starting at sample: " + offset);
}, false);

audioFileObj.setContentCallback(function()
{
    if (audioData.length == 0 || audioData[0].length == 0)
    {
        Console.print("No audio loaded.");
        return;
    }

    var buffer = audioData[0]; // Mono: use first channel
    fft.process([buffer]); // Wrap in array of channels
});

Just isn't working. I get nothing printed in the console. It is as if the setMagnitudeFunction is not being executed.

I have IPP enabled for Hise.

@Christoph-Hart any thoughts????

full snippet:

HiseSnippet 1333.3ocyWs0SabDEdWLaTw8hZZ4g93HTevzhb7xcHBU.ClfZLXgoIUBPQi2cV6oLdlUyLKD2F9O1eJ8o9Z6Y1K16FrHNtIQ0Ojry49bNm46bnkT3QTJgzxt74CBIV1egS6Abcu58vTt0wGXY+UNMwJMQhRHs+fPrRQ7srsKcjgf8byZE+6u9o8wLL2iLhjk0KDTOxyo8o5QTas6OSYrFXex4z94jd0cO1SvqKXhHHdJ4TyJD6cMtK4DrQrYbrdFV0yx9GbVNni2xAarxlta4t0Fqtkq6Za1Y0ZK64u1Vqrx5q3Gr7l302DT5QG5S0BYaMVSTfQ2W3OncOws7DG7Bph1gQLGbsZCdNgrU8dTleqrjixxx1o0nTUojT07NMo9zgzGkx95XFnQZjOoYOyCERtuGgjctPZ1jP5wNs8jzP8HNl34ycNlCUv.LTaxGJIxZMyeWxot.jfqq1GeMogDNLTiJqWq1RH3eV7okKC0GkFcCVh1KxmJdtPD1hgGPjtncPwMNU6Rzw7Zi6GxHCCjJKDSEYTAkpyBfIGYQrgeCJibZmeCr1a6fgF1HRkZETUE6qyfDJnXd6XTp8PlUtu+N.qwiQmzzAn.5IOAcgWOLmSXWcQhiTWkyLAAZv.Gx6R4jpdRB3mFMN23JfS0PIIDKIUbqs7pKgbSrmKJ0f.EfdZ3iTzemDqjhneIk6Kt0TTqXn7LP7TKBLO8FhjgCqTq5FqYpJojah6xo5HeRiHtmlJ3UBx9vGtlKgDAAfXKV9OJifevcTIXDHDovMcgFR3UFRowRMk2Eg0og01nEP+XlpOs7cKgBvLEw32BYM0vrVcLi0Ad4Nx8YtjFfpLLsWkQ3c08P6rCpF5MuYT83hZWkm2hwplXfwD2mHRzDwD.hheUSSUlnRhNRxSNeW43+yTx5DED..Z6TvkwUllBtXaTjhfBnRn.mVmh0LobF2NW4hDSbUR87kRbHhxQXoDO.RUYpoJemonklEPB9IBM4zjzwbkuqL5sYEDLVdl7pTvXD4XYaPQkOjhU3Q86PjKAWdVDYnf.pQQnnGMYPQdIk4bBJ3GCcdmFRRO2Pv7MPLluuOvkUZeB70ubrI6avxRoAxERfNPS3Xe.4FX5QBx1bNGPTWqEgv7i6A6A.tB+HFVWDE1LeJkAjOJ.8Yf23JpdP94WuGPy0dPn4IMDerSKp1q23iwYFSLBYpOFwX5.suz4Pnq1SOJ.m0owuNsSulb2+sYteHlOIYv07N2enwCrAxeNoafDNwaf.yz7fxCIeaRbM6bIfvA.k4ckILOjig6uedxmIDZyy67a.kLTpsAsELeAoI.5tpnELXrIKAU04nsu7.3QRGwqu7nHJXfynAA0ExvH0kMoWSTWdvwUp4tX0aw2X02jdbrM1nO90lOQcbrXPbF65Ldr3.2OiucohE4YmLbghO2xI+X1cY1OVqSM6j0P9NPn9+ODSgs+lKKFaSMsUGxugvfGQww32..mA3HlNiZwWPlIdg8DbpWwlPsj1sKQlO1G6EZOsFdHLhx76dFgQvpbc6e+tOG1MBKe62QeT2Ddr0quyIIbQlm8n+K0sOosYO3XhRSzXhO3w6zOx3CSnblHxrmZSLznZ.1NIpea.01iTOc2KCX1LlkMRNWKCqqMg6Ge3efeoLcMmsSY5lw7ShO5i8jhW4krnloW9yho.2adLf+bve9MbF4ZEu7V9Zbe.U7UddEM08Tb4oUwUlVEWcZUbsoUw0mVE2XZUby2shlgV6EoE8SdxZY0r0gwa6ZaObIA6RV+KClD4JA

Orvillain

@Orvillain

Even a minimal version like this doesn't work:

const var testBuffer = [];

for (i = 0; i < 2048; i++)
    testBuffer.push(Math.sin(i * 0.1));

const var testFFT = Engine.createFFT();
testFFT.prepare(1024, 1);
testFFT.setWindowType(testFFT.Hann);
testFFT.setOverlap(0.5);

testFFT.setMagnitudeFunction(function(data, offset)
{
    Console.print("OK - frame at: " + offset);
}, false);

testFFT.process([testBuffer]);

fft.process() is either silently failing, or not being called properly because of some kind of HISE setup thing, is my suspicion.

added scriptable FFT

Overlap-Add Decomposition | Spectral Audio Signal Processing

22

1.8k

12.2k

106.1k