Spectral Morphing in HISE

Casmat

Hey!

I've found that HISE supports spectral morphing, but after looking for countless hours, I couldn't find any code examples or help that shows it use within HISE. Has anyone used it and can share a couple lines to get me up and running? In my use case, I have three samplers and I am looking to use a xy pad that can morph the three sources from the three samplers in a simple xy pad, I was able to do it with crossfading, but I want to see how it'll sound with spectral morphing. In return, I could share some simple fx modules made in scriptnode?

Thanks!

d.healey

@Casmat said in Spectral Morphing in HISE:

I've found that HISE supports spectral morphing,

Tell me more...

Casmat

@d-healey with access to the FFT buckets for processing, couldn't one accomplish spectral morphing?

d.healey

@Casmat No idea, but if you figure it out let me know! Might also be possible using Loris: http://www.cerlsoundgroup.org/Kelly/soundmorphing.html

Casmat

@Christoph-Hart?

Casmat

bump!!!!

clevername27

@d-healey @Casmat Yes. Crossfade/interpolate between them. For a less manual approach, see Loris.

Christoph Hart

I've found that HISE supports spectral morphing

Where did you get that information? There are a few tools in HISE that are somewhat related to the subject (Loris, some FFT tools), but there's definitely not a readymade spectral morphing algorithm in HISE.

with access to the FFT buckets for processing, couldn't one accomplish spectral morphing?

What's "access to the FFT buckets"? Also an FFT won't get you all the way to spectral morphing it's like saying "Now that we've invented electricity, where are the self-driving cars"?

Casmat

@Christoph-Hart ahh I see, I guess I misread something!

DanH

Digging this up in case any advancements have been made in the spectral morphing area?!!

Could Max MSP do it?

HISEnberg

@DanH I don't think you will make it far in Max MSP if your intention is to bring it into HISE. RNBO's fft performs poorly, and Gen~ doesn't really have a viable fft option. Max itself has some good options for spectral processing but nothing exportable to C++

mmprod

@HISEnberg what would you use for fft processing if RNBO is not a viable option? and I’ve also heard that Faust is not optimal for this either

HISEnberg

@mmprod It's on my bucket list to do a deep dive into the Loris library, which I think has the solution. I have only scratched the surface here, but it is seriously impressive what it can do. My understanding with Loris is that it can't be used for live processing (though I could very well be mistaken on this).

RNBO, unless they have updated their FFT objects, would be my go to but from experience I know it is not useable. Otherwise the task probably involves some external C++ which is not a simple feat. I wonder if @griffinboy , the GOAT for external C++ implementation has looked into it?

griffinboy

@HISEnberg

spectral? Haha it's on my list but I've never done it before. I'm very interested in linking up neural networks to spectral synthesis in the future but I've not got an external node for that yet : )

Most of my finished nodes are for analog modelling, filters, compressors, etc.

clevername27

@Christoph-Hart said in Spectral Morphing in HISE:

"Now that we've invented electricity, where are the self-driving cars"?

Are you Elon Musk?

aaronventure

@clevername27 It's funny how even on this forum I cannot get away from hearing about this fucker.

clevername27

@aaronventure Right?

"OK, I found HISE on the internet. Why isn't my plugin finished?" — Phoney Stark

hisefilo

@Christoph-Hart

Got stuck on this:

lorisManager.process(file, "morph", ??????????

I'm implementing dilate without issues

lorisManager.process(file, "dilate", dilatePoints);

Is "morph" available as "dilate" and "shiftPitch" are?

Hope you can shed some light on this topic :)

clevername27

@hisefilo @aaronventure I'm thinking you can just use the FFT object in HISE, in real-time. Anything more than crossfading between the amplitudes may be too much, though. Use straight C or C++ for any sort of real DSP.

Take DFTs of the source and destination. Measure the amplitude (i.e., magnitude) difference between each of the buckets. Crossfade between them. There are some phase issues, but given your application, I they may not matter.

~~I haven't looked at HISE's object, but assuming it just takes in audio stream, and returns buckets, we're only talking about a few lines of code in HISEscript.~~ Just ensure that you start and finish the matrices at the same time. I can go into more detail.

I have taken a look—yes, it's just a few lines a code to do what you want. Preallocate your buffers; it looks like prepare will do that. The setMagnitudeFunction call looks like it lets you define a single function to process the bins, which is helpful in your case, because ostensibly, you want to do the same thing to all of them. I'm not clear what process does. Depending on what you want to do, the windowing function doesn't matter. You may wish to play around with the weighting of the bins, as the ear responds differently to different frequencies. Again, I'm not sure what you want to achieve, perceptually, but see if you perform your tasks without worrying about phase—you need to process the bins as quickly as possible, as HISEscript is not optimal for this. Use setEnableInverseFFT to reconstruct the signal.

All that being said, I would suggest listening to examples of spectral morphing. Much of the time, it's indistinguishable from simply crossfading the summed audio signals.

Start with what you want to achieve, perceptually, and work backwards from there.

ustk

@clevername27 said in Spectral Morphing in HISE:

All that being said, I would suggest listening to examples of spectral morphing. Much of the time, it's indistinguishable from simply crossfading the summed audio signals.

That's what I was thinking, if you crossfade between to FFTs, that'll be the same as crossfading gains in time domain.

Also, let say you do not apply the same weight to different bins. Wouldn't this be equivalent to filtering both signals before applying a crossfade in the time domain?

There are some phase issues...

Yeah I've seen this here and there when starting to play with magnitude ins... But in that case, wouldn't crossfading the phase bins at the same time as magnitude ones prevent any issues?

Spectral Morphing in HISE

47

1.6k

11.1k

96.9k