Simple ML neural network
-
@iamlamprey No libtorch has GPU support. But I just discovered TorchStudio, that's precisely the kind of GUI wrapper I needed to avoid the frustrating Python first steps stage...
-
@Christoph-Hart Alright, the first experimental integration is pushed. You can now load neural networks using RTNeural and either inference it using the scripting API or run it as realtime effect using the
math.neuralnode. I've created an example project with some hello world stuff and a roundtrip tutorial for getting started with TorchStudio:hise_tutorial/NeuralNetworkExample at master · christophhart/hise_tutorial
The Tutorial project for HISE. Contribute to christophhart/hise_tutorial development by creating an account on GitHub.
GitHub (github.com)
This is far from being production ready but it should be good enough for playing around and let me know what features might be interesting to add.
-
@Christoph-Hart oh shit! There goes my weekend plans. R.I.P my marriage
-
@Christoph-Hart Awesome. My knowledge of Neural Networks is close to 0 but this is a good opportunity to learn something new. Thank you genius!
-
checking this out now, very exciting stuff. I also appreciate you skipping over MNIST in the roundtrip example
-
@Christoph-Hart Thanks for this
-
- added RTNeural & neural network API · christophhart/HISE@a295c6c
The open source framework for sample based instruments - - added RTNeural & neural network API · christophhart/HISE@a295c6c
GitHub (github.com)
:beaming_face_with_smiling_eyes:
-
@Christoph-Hart said in Simple ML neural network:
A very powerful boost for this functionality could be the Loris library, I can imagine that having an array of highly precise time-varying gain values associated to the harmonic index is a much better input data than these few pixels from a spectrogram
Have you had a chance to play around with this particular use-case? Having the neural node handle the greyishbox modelling sounds a lot more streamlined than fine tuning an additive synth
-
@iamlamprey nope, I‘ve suspended my journey into ML until I have a real use case for it :)
If anyone is using this stuff I‘m happy to implement new features or fix issues, but now that the „hello world“ is implemented I expect it to grow with actual projects and their requirement.
-
@Christoph-Hart yeah fair enough, i'll keep noodling on this additive synth but i'll definitely be trying the neural & loris pairing at some point
-
@Christoph-Hart do you happen to have the training/dataloader for the sine model handy? the repo only has tanh
i'm currently migrating the example over to pure torch because torchstudio keeps uninstalling my local packages and in general is kinda gross
-
@d-healey is there a Rhapsody update coming soon? I just tried loading the example as a rhapsody library and (duh) it broke everything, i assume the next version of Rhapsody will be built with the latest version of HISE?
-
@iamlamprey said in Simple ML neural network:
is there a Rhapsody update coming soon?
Probably next month.
@iamlamprey said in Simple ML neural network:
i assume the next version of Rhapsody will be built with the latest version of HISE?
Yup
-
@d-healey i am excited to ignite everyone's CPUs
-
@iamlamprey I thought the ML stuff is only in scriptnode?
-
@d-healey uncompiled networks work in Rhapsody, the snex and expr stuff doesn't though
-
@iamlamprey Aha ok that's good to know
-
This post is deleted! -
This post is deleted! -
@Christoph-Hart said in Simple ML neural network:
@iamlamprey nope, I‘ve suspended my journey into ML until I have a real use case for it :)
If anyone is using this stuff I‘m happy to implement new features or fix issues, but now that the „hello world“ is implemented I expect it to grow with actual projects and their requirement.
Well speak of the devil and he will appear.... I now have a customer who would like to make Amp sims using Neural Net ML....
So I've spent some time with the research papers, the youTube Videos and the blogs. It would seem I might need a statetful LSTM model - but I could easily be wrong.
My understanding of how I(we) might approach this is very very poor, but I think there any number of ways to get to the json code that would be needed to "make it work", but wow do I have a bunch of questions.... here's some of them:
- is this even possible in HISE? (I mean the playback not the modelling or learning)
- Can I "just" use (say) this stuff : https://www.youtube.com/watch?v=xkrqF0D8pfQn and take the .json output and load it into RTNeural and expect it to "sort of" work?
- If not what's the best way to get from a bunch of guitar recordings to a set of .json files that I can then load into math.neural?
- I assume all these Learned models are in fact static snapshots of dynamic processes, so every time the user changes the distortion control on the plugin I will need to load another model - if so how practical is that?
