Simple ML neural network
-
@Lindon id say they can be about as accurate as a decent whitebox model, 8.5-9.0/10
-
@iamlamprey In that case I'd go with the IR which takes a few seconds to make - I have a script that can process 1000s of files to create loads of IRs in one go.
@iamlamprey said in Simple ML neural network:
rather use multiple lightweight networks and fade between them with a Gain node or something
Can do the same with multiple IRs
-
@iamlamprey said in Simple ML neural network:
@Lindon id say they can be about as accurate as a decent whitebox model, 8.5-9.0/10
so to get this clear in my head - the ML version is no better at the dynamic processing of guitar input that a set of IRs is?
..and by "dynamic" I mean different volume levels of input as the instrument is played...quietly (for those AOR ballads) or loudly...(thrashing it)
-
@Lindon it really depends on the implementation of the model and the data, having a robust model trained on a lot of varying data (different amp settings, different players etc) will sound great, and probably beat IRs and blackbox models
that being said, a dedicated whitebox model recreating each component of the signal path is still probably the best method for physical accuracy, i don't think a test comparing a proper neural net vs a proper physical model has ever been done, for obvious reasons ($$$)
there's still a lot of progress to be made with neural audio, so who knows how things will look in a year or two
-
@iamlamprey said in Simple ML neural network:
@Lindon it really depends on the implementation of the model and the data, having a robust model trained on a lot of varying data (different amp settings, different players etc) will sound great, and probably beat IRs and blackbox models
that being said, a dedicated whitebox model recreating each component of the signal path is still probably the best method for physical accuracy, i don't think a test comparing a proper neural net vs a proper physical model has ever been done, for obvious reasons ($$$)
there's still a lot of progress to be made with neural audio, so who knows how things will look in a year or two
Yes you are probably right, I was however thinking the amount of effort to build a neural net based amp sim would possibly greater than that to build a "nebula like" , "dynamic" IR model...
-
@iamlamprey - so does a "well formed/well trained" neural net model perform with the non-linearities of a real amp or not? I guess I'm back at "is ML even worth it right now?" as a question.
-
@Lindon here's an example of an effective implementation of NN:
https://mcomunita.github.io/gcn-tfilm_page/
Christian's repos are also public (or were last time i checked), they're more complex models compared to a regular LSTM, often utilizing multi-resolution STFT loss & other advanced things like dilated convolutions
I believe RTNeural has convolution dilation already implemented, so something like this is possible, albeit probably difficult
-
@iamlamprey said in Simple ML neural network:
@Lindon here's an example of an effective implementation of NN:
https://mcomunita.github.io/gcn-tfilm_page/
Christian's repos are also public (or were last time i checked), they're more complex models compared to a regular LSTM, often utilizing multi-resolution STFT loss & other advanced things like dilated convolutions
I believe RTNeural has convolution dilation already implemented, so something like this is possible, albeit probably difficult
Thanks -listening to the examples was interesting. Its clear the LSTM approach has some weaknesses at low gain inputs...but the approach outlined in the paper seems to deal with these much better...but as you say its probably considerably more difficult to achieve -especially as I have no idea what STFT is or dilated convolutions or how to build a model using them.
Hmm, this is getting more complex the more I look at it.
-
@Lindon said in Simple ML neural network:
-especially as I have no idea what STFT is
Short-Time Fourier Transform, basically a series of FFT's across the length of the signal to see how the frequency response changes over time, "multi-resolution" STFT means to create multiple STFT's of the same signal using different settings, which helps solve the frequency-time resolution issue
or dilated convolutions
if i'm remembering correctly, dilated convolutions basically refers to a layer in the network "skipping" a certain number of values and relying on other layers to process them, this means the resolution for that single layer might be lower, and therefore less computationally intensive which means you can have a larger receptive field (how far back in time the model can look)
Green: No dilation
Red: Skips 1
Blue: Skips 2or how to build a model using them
both Torch & Tensorflow have built-in dilation parameters for layers:
tf.keras.layers.Conv1D(dilation_rate=1)
torch.nn.Conv1d(dilation=1))
Hmm, this is getting more complex the more I look at it.
Yep, the cool part of ML is how automated it is, the uncool part is the massive amount of prerequisite work needed to be able to create robust models that do all the work for you
if you're wondering if I think you should do ML or old school blackbox modelling for your specific project, I'd lean towards the latter, especially if you already have experience in the field, even hiring someone to consult on an ML project would be tricky because the field is very new and there's still a lot of problems to be solved
-
@iamlamprey yep I am beginning to see the magnitude of the problem space,
This is an interesting interview;
https://www.youtube.com/watch?v=WLTzbEKTxhk (starts at min 12)
For those who dont know Neural DSP are the new kids on the block and I guess are the reason for all the fuss about neural net based guitar amp sims.
I looked up how they were funded, they got E5.9 Million in 3 Round of funding... and a lot of them arrived at work day 1 with PhDs in this field, and it still took them from 2018 to 2022 to get anything out the door...so to be in their league -= a lot of money + a lot of research+ a lot of time.
-
@Lindon Let me know if you want my IR script :D
-
@d-healey said in Simple ML neural network:
@Lindon Let me know if you want my IR script :D
My that made me laugh out loud.....