Simple ML neural network

[hisefilo]

@Christoph-Hart WOW!!! Nice surprise,
I can think of ML HISE playing the same role it did for non JUCE C++ experts in dsp developing arena. DDSP or RAVE also will be nice to have onboard.
We all are waiting for your news!!

[Christoph Hart]

Alright maybe I‘ll hold off with the HISE network training part and dust off my Python skillset, the pipeline is just to advanced to not use it for model building.

DDSP and stuff is nice but I think I need to add „conventional“ neural networks first, then we can expand on that (basically fast forwarding the last 30 years of development in this area lol).

[A Former User]

@Christoph-Hart Jatin (the guy who made RTNeural) is super friendly and knowledgeable, i'm sure he'd be happy to answer any questions you have about recurrent models and such if you haven't touched base with him already

i don't think training inside HISE would be particularly useful since it would be limited to the CPU anyway right?

[Christoph Hart]

@iamlamprey No libtorch has GPU support. But I just discovered TorchStudio, that's precisely the kind of GUI wrapper I needed to avoid the frustrating Python first steps stage...

[Christoph Hart]

@Christoph-Hart Alright, the first experimental integration is pushed. You can now load neural networks using RTNeural and either inference it using the scripting API or run it as realtime effect using the math.neural node. I've created an example project with some hello world stuff and a roundtrip tutorial for getting started with TorchStudio:

hise_tutorial/NeuralNetworkExample at master · christophhart/hise_tutorial

The Tutorial project for HISE. Contribute to christophhart/hise_tutorial development by creating an account on GitHub.

GitHub (github.com)

This is far from being production ready but it should be good enough for playing around and let me know what features might be interesting to add.

[Dan Korneff]

@Christoph-Hart oh shit! There goes my weekend plans. R.I.P my marriage

[Matt_SF]

@Christoph-Hart Awesome. My knowledge of Neural Networks is close to 0 but this is a good opportunity to learn something new. Thank you genius!

[A Former User]

checking this out now, very exciting stuff. I also appreciate you skipping over MNIST in the roundtrip example

[orange]

@Christoph-Hart Thanks for this

[hisefilo]

- added RTNeural & neural network API · christophhart/HISE@a295c6c

The open source framework for sample based instruments - - added RTNeural & neural network API · christophhart/HISE@a295c6c

GitHub (github.com)

:beaming_face_with_smiling_eyes:

[A Former User]

@Christoph-Hart said in Simple ML neural network:

A very powerful boost for this functionality could be the Loris library, I can imagine that having an array of highly precise time-varying gain values associated to the harmonic index is a much better input data than these few pixels from a spectrogram

Have you had a chance to play around with this particular use-case? Having the neural node handle the greyishbox modelling sounds a lot more streamlined than fine tuning an additive synth

[Christoph Hart]

@iamlamprey nope, I‘ve suspended my journey into ML until I have a real use case for it :)

If anyone is using this stuff I‘m happy to implement new features or fix issues, but now that the „hello world“ is implemented I expect it to grow with actual projects and their requirement.

[A Former User]

@Christoph-Hart yeah fair enough, i'll keep noodling on this additive synth but i'll definitely be trying the neural & loris pairing at some point

[A Former User]

@Christoph-Hart do you happen to have the training/dataloader for the sine model handy? the repo only has tanh

i'm currently migrating the example over to pure torch because torchstudio keeps uninstalling my local packages and in general is kinda gross

[A Former User]

@d-healey is there a Rhapsody update coming soon? I just tried loading the example as a rhapsody library and (duh) it broke everything, i assume the next version of Rhapsody will be built with the latest version of HISE?

[d.healey]

@iamlamprey said in Simple ML neural network:

is there a Rhapsody update coming soon?

Probably next month.

@iamlamprey said in Simple ML neural network:

i assume the next version of Rhapsody will be built with the latest version of HISE?

Yup

[A Former User]

@d-healey i am excited to ignite everyone's CPUs

[d.healey]

@iamlamprey I thought the ML stuff is only in scriptnode?

[A Former User]

@d-healey uncompiled networks work in Rhapsody, the snex and expr stuff doesn't though

[d.healey]

@iamlamprey Aha ok that's good to know