AI with Hise?
-
I don't know if this has been asked before.
Will HISE integrate with Artificial Intelligence (AI) technology or Machine Learning in the future?
Every day we see many fx & instrument plugins integrating with AI. Integration into this is absolutely mandatory for the future tech compatibility. Is there any plan regarding this?
-
Integrate in what way? What kind of AI?
-
It's possible already with third-party-nodes and RTNeural
-
@d-healey AI can be used in a wide range of areas.
-
@iamlamprey I mean standard integration like FAUST. Is there any libraries for it with a proper licensing?
-
@harris-rosendahl What do you mean by Proper Licensing? I think RTNeural is compatible with HISE's licensing: https://github.com/jatinchowdhury18/RTNeural/blob/main/LICENSE
There's nothing that comes packaged with HISE, but the third-party dsp nodes let you include any C++ library you want with some extra effort, RTN is the simplest to get up and running, but AI audio in general is still pretty limited aside from a few edge-cases like distortion pedals or singing-synths
-
@iamlamprey I think AI should be included standard in Hise, just like scriptnode system.
AI audio in general is still pretty limited aside from a few edge-cases like distortion pedals or singing-synths
Check the above link, it's the way much more than that now :)
-
@harris-rosendahl yeah i'm familiar with a fair few of those plugins, unfortunately ai audio is pretty gimmicky and a lot of the current tools use quite a bit of traditional processing to make the ai-part more powerful, things like the MIDI pattern generators would be pretty simple to make I think as they can call inference whenever you want and dont need any actual audio buffers
realtime audio processing purely with ai is still in its infancy because processing audio data necessitates long(very long) term dependencies which massively complicates things, it's simple (relative) to predict the stock market using ai since a year is only 365 days, but 1 second of audio is 44100 data-points at a minimum, and most samples are much longer than 1 second. there's also things like Phase and stereo-processing which make it even more difficult
there's an LA-2A clone by Christian Steinmetz and co that sounds pretty good, but generating a large receptive field like that used 30-40% of my cpu in a compiled plugin, so it's basically: stick to simple offline inferrence (like a lot of those plugins you linked) or experience a perpetual migraine
