Simple ML neural network

[Dan Korneff]

@Christoph-Hart The structure does look like TensorFlow, but the script is using torch libraries to create the model:

torch
torch.optim
torch.cuda
torch.optim.Adam
torch.optim

I was using this example code to load the model:

const var pt = Engine.createNeuralNetwork("PytorchNetwork");;

// Load the model layout & weights that were exported as JSON
const var modelJSON = pythonRoot.getChildFile("model_keras.json").loadAsObject();

// Load the model & weights:
pt.loadPytorchModel(modelJSON);

when I load the model into the Neural node, HISE checks out.

I've run through the MNIST dataset tutorial a couple times to get a basic idea of how TorchStudio works, but I'm not sure how to adapt the scripts to work there.

[ccbl]

I asked Jatin to have a quick look through the thread to see if he could see any issues, he just had this to say.

"Hmmm, it seems to me that the model JSON file that is being loaded into the "Neural Node" is structured as a TensorFlow-style JSON file, but it's being loaded with the HISE's loadPytorchModel() method? I don't really know what the Neural Node does internally, so idk how much I can help beyond that."

[Dan Korneff]

Using loadTensorFlowModel() was indeed the solution. I'll try to make some tutorials on training and loading models this weekend.

[Lindon]

@Dan-Korneff that would be very cool. Do these models include the ability to define parameters(like tone controls on an amp)? Or are they static snap shots?

[Dan Korneff]

@Lindon said in Simple ML neural network:

@Dan-Korneff that would be very cool. Do these models include the ability to define parameters(like tone controls on an amp)? Or are they static snap shots?

The training scripts can create parameterized models, but I've only tested a static model so far.
Once I get my sea legs I'll pull in @Christoph-Hart to figure out multiple parameters.

[aaronventure]

@Dan-Korneff man this sounds great, thanks.

how are you finding the results so far comparing the capture and the model output?

[Dan Korneff]

@aaronventure I haven't gotten that far yet. Still in the "does this even work" stage

[griffinboy]

@ccbl

Chowdhury? I'd love to talk to him! His dsp is very inspiring

[scottmire]

Apologies if this has already been discussed....but I would think this would be low hanging fruit:

GitHub - Tr3m/nam-juce: A JUCE implementation of the Neural Amp Modeler Plugin

A JUCE implementation of the Neural Amp Modeler Plugin - Tr3m/nam-juce

GitHub (github.com)

[aaronventure]

@scottmire keep in mind that this is GPLv3 licensed

[ccbl]

@Dan-Korneff What sample rate were you using? I'd love to train at 96khz or even 192khz for aliasing reduction reasons. My ultimate plan is to stack several smaller models of individual components together sandwiched between regular DSP so I think reducing aliasing should be important in this case.

[ccbl]

@griffinboy Here's the link to their discord channel for RTNeural

https://discord.gg/enmpURqR

[Dan Korneff]

@ccbl I'm only experimenting with 48K at the moment. Feel free to look at my repository scripts to figure out the sample rate stuff. I'm taking baby steps with this stuff while I finish up other projects.

[scottmire]

@aaronventure NAM itself is MIT licensed and this is simply a JUCE implementation of the NAM Player....so, I have no idea how they could enforce a GPLv3 license. But...I'm definitely no expert.

[d.healey]

@scottmire MIT is a weak license so you can take MIT code and relicense it pretty much however you like. If you are releasing a GPL project then all code in that project needs to be GPL. So the developer of nam-JUCE has relicensed NAM as GPL within their project.

[scottmire]

@d-healey Ahhh got it. Thanks for the clarification.

[orange]

@Dan-Korneff said in Simple ML neural network:

Using loadTensorFlowModel() was indeed the solution. I'll try to make some tutorials on training and loading models this weekend.

@Dan-Korneff @Christoph-Hart
Is there any progress? We look forward to using this neural model in a guitar amp simulation :)

[Christoph Hart]

@orange Jatin said in the GitHub issue that he's thinking about adding the wavenet model to RTNeural, when that's the case, I'll resume the work to support NAM models.

[Dan Korneff]

@orange I had to take a couple days vacation over here. Back in the office Monday :)

[Orvillain]

@Christoph-Hart Looks like Jatin has made some progress with this:
https://github.com/jatinchowdhury18/RTNeural/issues/143#issuecomment-2472915024

So does this bring us closer to being able to load NAM models into RTNeural inside Hise???