The need for Spectral Processing and AI in HISE

resonant

I think this has been asked a few times before, but is there any progress on Spectral Processing?

Scriptnode really needs this. Many plugins currently on the market can perform Eq, Compressor and other operations at a much more advanced level using Spectral processing.

In addition to spectral processing, AI is also absolutely necessary.

It is absolutely necessary for Hise to add these systems in the near future in order to avoid remaining old technology and to be integrated with current technology.

d.healey

@resonant said in The need for Spectral Processing and AI in HISE:

AI is also absolutely necessary.

What do you mean by AI?

Fortune

@d-healey said in The need for Spectral Processing and AI in HISE:

@resonant said in The need for Spectral Processing and AI in HISE:

AI is also absolutely necessary.

What do you mean by AI?

I think AI is the Artificial Intelligence.

Yes nowadays we hear Artificial Intelligence and Spectral Processing Plugins more and more everyday. And Hise should integrate these tech asap.

And +1000 for this @Christoph-Hart

Casmat

@Fortune @resonant The thing is, for most people, the algorithm/data sets would need to be tailored since each plugin may have a different use for AI. It's just very broad unless there's a framework that could do something general (pytorch/tensorflow?), but that would take quite some effort from my understanding..

Fortune

@Casmat At least with machine learning integration, developers might have something to customize to their domain. Or as you said, a general approach.

Many things have been added to Hise lately. Unlike AI, Spectral Processing can at least be added, which can be integrated more easily.

aaronventure

Machine learning is mostly done in python. I'm not sure how well that would work in audio for real-time tasks.

You can translate it to C++, but then you can already do that without HISE, and simply import the code into HISE.

Christoph Hart

What do you want to do with AI? If you‘ve got a valid use case, then you can train your models in python and add whatever deep learning library you need yourself - this is the least of your problems.

My guess is that whenever you ask for AI to be added to HISE you have no clue how it works so you wouldn‘t be able to use whatever generic building blocks I might add to the framework anyway. At some point in the future I might add a few tools to execute neural networks but I don‘t see this a critical or mainstream feature at all.

I‘m happy to work with any developer who starts working on an AI feature and realizes that some stuff in HISE could be changed to make the process smoother, but a general request for „add AI to HISE in order to not get left behind“ is complete FOMO and rather naive.

With spectral processing it‘s similar - what do you want to do here? Make your own timestretching algorithm? Make a vocoder, simulate MP3 codec artifacts? In each of those examples the bulk of the work has to be done with raw number crunching from the FFT output so you need to do this in C++ which you can do already (and the complexity of understanding and creating these algorithms vastly exceed the complexity of writing C++) There‘s a limited amount of generic building blocks that will make things easier here (FFT functions, windowing etc) and they are already available on the C++ level.