added scriptable FFT

d.healey

What about a resynthesis module? :D

A Former User

@d-healey with the results I got from DDSP I'm surprised more developers aren't looking into resynthesis more... we'd never have to sample a round robin again

d.healey

@iamlamprey It would be nice if we never had to sample anything again.

Christoph Hart

@iamlamprey ~~What's DDSP?~~ Nevermind it's more googleable than I thought.

A Former User

@christoph-hart Yep :) The style-transfer is particularly appealing, I wouldn't be surprised if you can legally train it with other people's sample libraries since it's 100% synthesis.

Not a lawyer but ;)

d.healey

@Christoph-Hart I thought I'd have a go at this fft thing. Following your instructions above and the docs I load a sample from the sampler into a buffer. If I uncomment the process line in this script HISE will just crash.

const var Sampler1 = Synth.getSampler("Sampler1");

reg sound = Sampler1.createSelection(".*")[0];

const buf = sound.loadIntoBufferArray();

const fft = Engine.createFFT();
fft.setEnableSpectrum2D(true);
fft.setWindowType(fft.BlackmanHarris);

fft.setMagnitudeFunction(function(data, offset)
{
	var max = 0.0;
	
	for(s in data)
	{
		max = Math.max(max, s);
	}
	
	Console.print("The max value at " + offset + " is " + Engine.getDecibelsForGainFactor(max));
	
}, true);

fft.prepare(1024, 2);

//fft.process(buf);

d.healey

@d-healey Bump

Orvillain

@d-healey Did you ever resolve this Dave?

d.healey

@Orvillain Not as far as I recall

Orvillain

@d-healey

I'm not running into a crash, primarily because I'm not enabling the 2D Spectrum.

But a very simple example like this:

Content.makeFrontInterface(600, 600);

const var AudioLoopPlayer1 = Synth.getAudioSampleProcessor("Audio Loop Player1");
const var audioFileObj = AudioLoopPlayer1.getAudioFile(0);
const var sampleRate = audioFileObj.getSampleRate();
const var audioData = audioFileObj.getContent(); // [channel][samples]
const var fft = Engine.createFFT();
fft.prepare(1024, 1); // 1 channel, 1024 sample size
fft.setWindowType(fft.Hann);
fft.setOverlap(0.75);

fft.setMagnitudeFunction(function(data, offset)
{
    Console.print("Frame starting at sample: " + offset);
}, false);

audioFileObj.setContentCallback(function()
{
    if (audioData.length == 0 || audioData[0].length == 0)
    {
        Console.print("No audio loaded.");
        return;
    }

    var buffer = audioData[0]; // Mono: use first channel
    fft.process([buffer]); // Wrap in array of channels
});

Just isn't working. I get nothing printed in the console. It is as if the setMagnitudeFunction is not being executed.

I have IPP enabled for Hise.

@Christoph-Hart any thoughts????

full snippet:

HiseSnippet 1333.3ocyWs0SabDEdWLaTw8hZZ4g93HTevzhb7xcHBU.ClfZLXgoIUBPQi2cV6oLdlUyLKD2F9O1eJ8o9Z6Y1K16FrHNtIQ0Ojry49bNm46bnkT3QTJgzxt74CBIV1egS6Abcu58vTt0wGXY+UNMwJMQhRHs+fPrRQ7srsKcjgf8byZE+6u9o8wLL2iLhjk0KDTOxyo8o5QTas6OSYrFXex4z94jd0cO1SvqKXhHHdJ4TyJD6cMtK4DrQrYbrdFV0yx9GbVNni2xAarxlta4t0Fqtkq6Za1Y0ZK64u1Vqrx5q3Gr7l302DT5QG5S0BYaMVSTfQ2W3OncOws7DG7Bph1gQLGbsZCdNgrU8dTleqrjixxx1o0nTUojT07NMo9zgzGkx95XFnQZjOoYOyCERtuGgjctPZ1jP5wNs8jzP8HNl34ycNlCUv.LTaxGJIxZMyeWxot.jfqq1GeMogDNLTiJqWq1RH3eV7okKC0GkFcCVh1KxmJdtPD1hgGPjtncPwMNU6Rzw7Zi6GxHCCjJKDSEYTAkpyBfIGYQrgeCJibZmeCr1a6fgF1HRkZETUE6qyfDJnXd6XTp8PlUtu+N.qwiQmzzAn.5IOAcgWOLmSXWcQhiTWkyLAAZv.Gx6R4jpdRB3mFMN23JfS0PIIDKIUbqs7pKgbSrmKJ0f.EfdZ3iTzemDqjhneIk6Kt0TTqXn7LP7TKBLO8FhjgCqTq5FqYpJojah6xo5HeRiHtmlJ3UBx9vGtlKgDAAfXKV9OJifevcTIXDHDovMcgFR3UFRowRMk2Eg0og01nEP+XlpOs7cKgBvLEw32BYM0vrVcLi0Ad4Nx8YtjFfpLLsWkQ3c08P6rCpF5MuYT83hZWkm2hwplXfwD2mHRzDwD.hheUSSUlnRhNRxSNeW43+yTx5DED..Z6TvkwUllBtXaTjhfBnRn.mVmh0LobF2NW4hDSbUR87kRbHhxQXoDO.RUYpoJemonklEPB9IBM4zjzwbkuqL5sYEDLVdl7pTvXD4XYaPQkOjhU3Q86PjKAWdVDYnf.pQQnnGMYPQdIk4bBJ3GCcdmFRRO2Pv7MPLluuOvkUZeB70ubrI6avxRoAxERfNPS3Xe.4FX5QBx1bNGPTWqEgv7i6A6A.tB+HFVWDE1LeJkAjOJ.8Yf23JpdP94WuGPy0dPn4IMDerSKp1q23iwYFSLBYpOFwX5.suz4Pnq1SOJ.m0owuNsSulb2+sYteHlOIYv07N2enwCrAxeNoafDNwaf.yz7fxCIeaRbM6bIfvA.k4ckILOjig6uedxmIDZyy67a.kLTpsAsELeAoI.5tpnELXrIKAU04nsu7.3QRGwqu7nHJXfynAA0ExvH0kMoWSTWdvwUp4tX0aw2X02jdbrM1nO90lOQcbrXPbF65Ldr3.2OiucohE4YmLbghO2xI+X1cY1OVqSM6j0P9NPn9+ODSgs+lKKFaSMsUGxugvfGQww32..mA3HlNiZwWPlIdg8DbpWwlPsj1sKQlO1G6EZOsFdHLhx76dFgQvpbc6e+tOG1MBKe62QeT2Ddr0quyIIbQlm8n+K0sOosYO3XhRSzXhO3w6zOx3CSnblHxrmZSLznZ.1NIpea.01iTOc2KCX1LlkMRNWKCqqMg6Ge3efeoLcMmsSY5lw7ShO5i8jhW4krnloW9yho.2adLf+bve9MbF4ZEu7V9Zbe.U7UddEM08Tb4oUwUlVEWcZUbsoUw0mVE2XZUby2shlgV6EoE8SdxZY0r0gwa6ZaObIA6RV+KClD4JA

Orvillain

@Orvillain

Even a minimal version like this doesn't work:

const var testBuffer = [];

for (i = 0; i < 2048; i++)
    testBuffer.push(Math.sin(i * 0.1));

const var testFFT = Engine.createFFT();
testFFT.prepare(1024, 1);
testFFT.setWindowType(testFFT.Hann);
testFFT.setOverlap(0.5);

testFFT.setMagnitudeFunction(function(data, offset)
{
    Console.print("OK - frame at: " + offset);
}, false);

testFFT.process([testBuffer]);

fft.process() is either silently failing, or not being called properly because of some kind of HISE setup thing, is my suspicion.

Orvillain

Okay, I rejigged Chris's example above. This actually works:

// Create a dummy signal with two sine waves
const var signal = Buffer.create(4096);

reg uptime = 0.0;

for(s in signal)
{
    s = 0.3 * Math.sin(uptime);
    s += 0.7 * Math.sin(uptime * 2.0);
    uptime += 0.1;
}

// Create a FFT object. If you right click on it in the 
// script watch table and view the popup you'll see a spectrogram
// for the data passed in
const var fft = Engine.createFFT();

// creates a spectrogram image (required for 
// the debug popup but might be used later to draw
// on a panel. */
fft.setEnableSpectrum2D(true);

// Set the window type for the processing
fft.setWindowType(fft.BlackmanHarris);

// Give the fft a function that will be called for 
// each signal chunk. `data` will contain either a 
// buffer or an array of buffers containing the transformed
// FFT signal and offset will contain the index of the first
// sample in the chunk (here it will be 0, 1024, 2048 and 3072
// because the processing size is 1024
fft.setMagnitudeFunction(

(function(data, offset)
{
	var max = 0.0;

	for(s in data)
	{
		max = Math.max(max, s);
	}
	
	Console.print("The max value at " + offset + " is " + Engine.getDecibelsForGainFactor(max));
}), true);

fft.prepare(1024, 1);

// Process the buffer with the function above.
fft.process(signal);

Prepare needs to be called AFTER setting the magnitude function. Not before.

Orvillain

Another thing to bear in mind - it seems to me that you cannot run an FFT over a typical array. It HAS to be a buffer.

added scriptable FFT

21

1.8k

12.1k

104.9k