Oh I'm one of those science nerds....
Neural Amp Modeler v0.2 (open-sourced amp modeler)
- Thread starter chameleon101663
- Start date
KingsXJJ
Member
- Messages
- 2,017
All good my man!
I code/experiment a lot with tube amp modeling for embedded and VST, but I have no experience with neural stuff or ML. In traditional DSP, you basically do virtual modelling of the analog circuit. You program for various tube characteristics, DC offset, phase, filters, etc etc... Pretty long and tedious process (at least for me), but a lot of fun.
How does the modeling code work for neural? I imagine you don't have to do all those steps.
How does the modeling code work for neural? I imagine you don't have to do all those steps.
sdatkinson
Member
- Messages
- 1
Hi, everyone! I'm sdatkinson on GitHub/RunawayThumbtack on YouTube, the creator of NAM.
I thought I'd create an account & drop in on this thread to add to some of the discussion & answer a few of the questions/comments.
@chameleon101663 thanks for sharing & thanks for the kind words!
Since then, prefixing things with "neural" has become somewhat of a trope in my line of work when you're representing some unknown (e.g. an amp) with a neural net and fitting it to data. If anything, the name was meant as a play on "NAMM"
@PLysander
@tjontheroad
@Orvillain
@Quad4
The training code will support it in the next version...It's already merged into the development branch. I need a weekend here & I'll get the plugin side implemented too
@mbenigni
@LaXu - thanks for the very detailed thoughts!
@Fatty McAtty
@Calaban
@ricepr
(Still had to do the plugin in C++ anyways.)
@aro
I thought I'd create an account & drop in on this thread to add to some of the discussion & answer a few of the questions/comments.
@chameleon101663 thanks for sharing & thanks for the kind words!
It's already out & being developed more or less "in public"...have a look!
- https://github.com/sdatkinson/neural-amp-modeler - the training code
- https://github.com/sdatkinson/iPlug2 <- puts it into a plugin. You'll specifically want to open the solution under `Examples/NAM` (this is my first plugin, so I just took one of the examples and made the changes I needed to get the net in there & minimal anything else)
So this might be a good excuse to explain my background & choice of name. My most recent line of work has been in ML research. My interests are mainly in developing new approaches to fuse ML/AI with natural science applications. The "neural" prefix comes from a now-famous work called Neural Ordinary Differential Equations from which there are literally thousands of follow-on works, but it was relatively new when I first did version 0.1:
Since then, prefixing things with "neural" has become somewhat of a trope in my line of work when you're representing some unknown (e.g. an amp) with a neural net and fitting it to data. If anything, the name was meant as a play on "NAMM"
@PLysander
NDSP has published a paper that is likely very similar to the modeling they used; I have a few guesses what the QC does for captures, and it's not quite what I'm doing here.
@tjontheroad
Nope--all mine
@Orvillain
This is very true. My "go-to" dataset I created has a bunch of different guitars, pickups, playing articulations & dynamics, chords, single notes, etc basically going chromatically up and down the fretboard. Fortunately, data is super-cheap for this--record for a couple minutes and you get millions of data points!
Let me know how it goes...I haven't gotten the chance to try one yet myself
@Quad4
Soon!
The training code will support it in the next version...It's already merged into the development branch. I need a weekend here & I'll get the plugin side implemented too Oh, it does that fine. The data has me playing over the clean-to-crunch dynamic transition, so the model learns it. What's really crazy is if you juice the input and hit it with a huge signal. I put a "x100" on the input of the Deluxe Reverb model in the iPlug repo and the resulting overdrive sounded amazing (and frankly amazed me that it extrapolated to that kind of signal so well!) Can't attest that it's exactly like the real thing though because I don't have any 20dB clean boosts hanging around!
@mbenigni
There's a lot of ways to do this. One common way is to instead of having a single number be the input (your mono DI), make it a list of number where the other numbers are the knob settings. But there are other ways as well. What you don't want to do though is take the (weighted) average of several models because then you've got N times more CPU to run it.
@LaXu - thanks for the very detailed thoughts!
That's the thing--there...isn't any, really. I started this (and am still doing it) as a for-fun project, and decided to basically start with the simplest approach I could think of--things anyone could have tried in the early 2010's--and it...just worked! With another minor tweak (using a ConvNet), it basically gave me the results in the videos. Combined with how nice modern ML stuff like PyTorch is, it's super-accessible to do this now.
Here ya go (open the .sln file in Visual Studio; don't have a Mac to experiment w/)
I've now got an RTX 3070 Ti, but did these w/ my 2070. But you'd be fine with a bottom-barrel GPU on the cloud--this is a tiny model (by necessity for real-time!) so you really don't need anything modern for this.
Almost certainly--there's a paper that NDSP published a few years back (full disclosure: I read it--but only after I had my "solution" just to see for fun how ours compared. Theirs is a bit more complex...but when I coded theirs up, it did worse...) and others have done a straight-code of the paper (e.g. https://github.com/GuitarML/SmartGuitarAmp)
This is basically true; I'd quibble that it's not a "short cut"--it's just not necessary to go through that trouble with this application because the data are dirt-cheap. (I use more elaborate approaches in my day job, but you basically always start with the "straightforward" answer in case it works.) In the past, things like SPICE were the main toolkit, and they're still useful, but this is just another "tool" to model.
I'll have a video for this soon
In short: yep. I don't personally buy the "feel" thing--either it's the same wave form or it's not, and this is...really close (see the "mono test" in the QC comparison). I got below my own threshold of perception back w/ v0.1, so I've honestly just been flying by numbers since.
I've got an idea I'm pretty confident will work (quite quickly, though I'm going to avoid claiming how fast until I do it & I'm sure so I don't eat my words). But "fast" fits may fall under the "special sauce" category
@Fatty McAtty
True--my brain has been completely smoothed out from Python
@Calaban
The last vid shows it w/ my Deluxe Reverb
@ricepr
There's always one SciML-er around these days...hey there! In this application, you might not get much speedup from Julia--you can still vectorize & keep the GPU fed well enough with big minibatches w/ PyTorch (whose kernel calls are async) that it was worth the easier development (IMO) to keep to Python...but Julia's great for shoving MLPs into classical computation for sure
(Still had to do the plugin in C++ anyways.)Actually, that might not be true here--guitar amps are pretty simple circuits (though I don't do RT-SPICE stuff myself so I couldn't say for certain...might have if I picked this up a decade ago!)
@aro
Basically, yep. I run the signal through the amp, export the WAV, point the trainer at the new file, then hit go and grab it in the morning
nwinkler
Member
- Messages
- 151
In layman's terms, a neural network is trained on a set of input data and expected output results. The more (different) data you feed in, the better the training is going to be.
For calculating BMI (body mass index) for example, you'd provide a person's height and weight (and potentially gender, age) as the input values and then the pre-calculated BMI value as the expected result as training data. Once you have trained the model with enough data so that it's calibrated, you can feed just the input values in and it will determine an output value with some level of accuracy. This works quite well, but of course depends on the quality of your training data. This way, you won't have to program the actual calculation (BMI or amp output), but the neural network model will figure that out on its own.
Under the hood, a simplified description of a neural model is a set of formulas with multiple input parameters and some constant values. During training, the constant values are adjusted until the training data produces the expected training results. It's actually a bit more complex like that, but it's best to think of it as a set of formulas with calibrated values that produce an expected output value for a set of input parameters. Depending on the type of model, they also support interpolation of values, which is what's needed for the amp simulation.
Typical applications are either "classification" (a fixed set of possible output values, e.g. handwriting - this is the letter A), or "regression" (the BMI example from above, where an output value is interpolated and you don't have a finite set of well-defined output values).
Given that big companies like Google, Microsoft, or Amazon have pre-trained models for things like face recognition, voice recognition, etc., I would imagine that a model that interpolates an amp's behavior should be absolutely feasible to do - it's a great application of this technology. The biggest hurdle is likely going to be usability from an end user perspective.
kyolic
Member
- Messages
- 1,045
Hi mate welcome to TGP. Any plans for releasing it as a VST so that we non-coder guitarists can enjoy it too?
Seanspeed
Member
- Messages
- 400
It's open source, so it's gonna be up to somebody who knows how to make such a program to utilize it like that.
PLysander
Member
- Messages
- 550
Just realized i didn't do this on my previous comment, but: congratulations
This is excellent work!
UnnoTed
Member
- Messages
- 68
Have you tried running it on iOS or raspberry pi?
Last edited:
- Messages
- 4,542
Hi, everyone! I'm sdatkinson on GitHub/RunawayThumbtack on YouTube, the creator of NAM.
I thought I'd create an account & drop in on this thread to add to some of the discussion & answer a few of the questions/comments.
@chameleon101663 thanks for sharing & thanks for the kind words!
It's already out & being developed more or less "in public"...have a look!
@phil_m
- https://github.com/sdatkinson/neural-amp-modeler - the training code
- https://github.com/sdatkinson/iPlug2 <- puts it into a plugin. You'll specifically want to open the solution under `Examples/NAM` (this is my first plugin, so I just took one of the examples and made the changes I needed to get the net in there & minimal anything else)
So this might be a good excuse to explain my background & choice of name. My most recent line of work has been in ML research. My interests are mainly in developing new approaches to fuse ML/AI with natural science applications. The "neural" prefix comes from a now-famous work called Neural Ordinary Differential Equations from which there are literally thousands of follow-on works, but it was relatively new when I first did version 0.1:
Since then, prefixing things with "neural" has become somewhat of a trope in my line of work when you're representing some unknown (e.g. an amp) with a neural net and fitting it to data. If anything, the name was meant as a play on "NAMM"
@PLysander
NDSP has published a paper that is likely very similar to the modeling they used; I have a few guesses what the QC does for captures, and it's not quite what I'm doing here.
@tjontheroad
Nope--all mine
@Orvillain
This is very true. My "go-to" dataset I created has a bunch of different guitars, pickups, playing articulations & dynamics, chords, single notes, etc basically going chromatically up and down the fretboard. Fortunately, data is super-cheap for this--record for a couple minutes and you get millions of data points!
Let me know how it goes...I haven't gotten the chance to try one yet myself
@Quad4
Soon!
Oh, it does that fine. The data has me playing over the clean-to-crunch dynamic transition, so the model learns it. What's really crazy is if you juice the input and hit it with a huge signal. I put a "x100" on the input of the Deluxe Reverb model in the iPlug repo and the resulting overdrive sounded amazing (and frankly amazed me that it extrapolated to that kind of signal so well!) Can't attest that it's exactly like the real thing though because I don't have any 20dB clean boosts hanging around!
@mbenigni
There's a lot of ways to do this. One common way is to instead of having a single number be the input (your mono DI), make it a list of number where the other numbers are the knob settings. But there are other ways as well. What you don't want to do though is take the (weighted) average of several models because then you've got N times more CPU to run it.
@LaXu - thanks for the very detailed thoughts!
That's the thing--there...isn't any, really. I started this (and am still doing it) as a for-fun project, and decided to basically start with the simplest approach I could think of--things anyone could have tried in the early 2010's--and it...just worked! With another minor tweak (using a ConvNet), it basically gave me the results in the videos. Combined with how nice modern ML stuff like PyTorch is, it's super-accessible to do this now.
Here ya go (open the .sln file in Visual Studio; don't have a Mac to experiment w/)
I've now got an RTX 3070 Ti, but did these w/ my 2070. But you'd be fine with a bottom-barrel GPU on the cloud--this is a tiny model (by necessity for real-time!) so you really don't need anything modern for this.
Almost certainly--there's a paper that NDSP published a few years back (full disclosure: I read it--but only after I had my "solution" just to see for fun how ours compared. Theirs is a bit more complex...but when I coded theirs up, it did worse...) and others have done a straight-code of the paper (e.g. https://github.com/GuitarML/SmartGuitarAmp)
This is basically true; I'd quibble that it's not a "short cut"--it's just not necessary to go through that trouble with this application because the data are dirt-cheap. (I use more elaborate approaches in my day job, but you basically always start with the "straightforward" answer in case it works.) In the past, things like SPICE were the main toolkit, and they're still useful, but this is just another "tool" to model.
I'll have a video for this soon
In short: yep. I don't personally buy the "feel" thing--either it's the same wave form or it's not, and this is...really close (see the "mono test" in the QC comparison). I got below my own threshold of perception back w/ v0.1, so I've honestly just been flying by numbers since.
I've got an idea I'm pretty confident will work (quite quickly, though I'm going to avoid claiming how fast until I do it & I'm sure so I don't eat my words). But "fast" fits may fall under the "special sauce" category
@Fatty McAtty
True--my brain has been completely smoothed out from Python
@Calaban
The last vid shows it w/ my Deluxe Reverb
@ricepr
There's always one SciML-er around these days...hey there! In this application, you might not get much speedup from Julia--you can still vectorize & keep the GPU fed well enough with big minibatches w/ PyTorch (whose kernel calls are async) that it was worth the easier development (IMO) to keep to Python...but Julia's great for shoving MLPs into classical computation for sure
Actually, that might not be true here--guitar amps are pretty simple circuits (though I don't do RT-SPICE stuff myself so I couldn't say for certain...might have if I picked this up a decade ago!)
@aro
Basically, yep. I run the signal through the amp, export the WAV, point the trainer at the new file, then hit go and grab it in the morning
Great first post! Very impressive work indeed, even though a lot of it is over my head, lol.
Welcome!
I'm genuinely wondering... Why does anyone still use the "traditional" DSP approach when creating plug-ins or embedded modeled amps? What are the shortcomings of the neural approach, if any?
From what I see, plug in companies are still hiring DSP engineers. Why bother, why not ML instead? It seems like it would be more efficient.
The neural approach seems to eliminate entirely the electrical engineering part of analog circuits modeling, which I find to be the time-consuming part (and I'm still learning). It takes a lot of experimenting and DSP math. The actual coding is the easier part of the entire process.
I was aware that ML started being used in our audio space and I always figured I'll look into it, some day. Now I'm wondering if I shouldn't do that instead of going down the "traditional" SPICE and DSP path. I am a lot more comfortable in the IT world vs. electrical engineering.
the-trooper
Member
- Messages
- 500
Calaban
Member
- Messages
- 3,063
The last vid shows it w/ my Deluxe Reverb
I’m having trouble finding clean tones or a Deluxe in any of the videos.
Maybe I’m old and don’t know how to work the interwebz like you kids.
Would you please post the video again?
2dor
Member
- Messages
- 123
@sdatkinson I listened to all the clips you published - killer thing you got there. Look forward to having a skinny VST to try it out.
chameleon101663
Member
- Messages
- 432
It’s the first video in the first post of this thread.
@sdatkinson do you think, your trained models could run on a device like the MOD Dwarf?
It would be definitely a quick way to have NAM running in a stomp box, plus the MOD ecosystem is open source, too.
It would be definitely a quick way to have NAM running in a stomp box, plus the MOD ecosystem is open source, too.
Trending Topics
-
-
Bands with no original members left, but still on the road
- Started by brcampbe01
- Replies: 166
-
-
-
If you could point your finger at ONE album that represents the real JCM800 tones at their finest , what would it be ?- Started by Telejester
- Replies: 251