Vibe Coding Custom Nodes

[scottmire]

I apologize if this has been discussed before....

Has anyone had any luck in vibe coding new nodes (SNEX or C++)? If so, could you share anything that might either talk me out of even trying or, better still, point me in the right direction?

Chances are I would mostly be porting existing code into custom scriptnodes. There might be cases where I try to vibe code something from scratch...

Any advise would be greatly appreciated.

[dannytaurus]

@scottmire What experience do you have so far with agentic coding?

[griffinboy]

@scottmire

TLDR At Bottom

It's pretty rough in my opinion.
(But also, take my opinion with a grain of salt, I get a bit serious when it comes to DSP).

I'd say it depends what kind of quality (and efficiency) you are after.

AI can totally make a "working" C++ node very easy, but it won't be that efficient and will have all kinds of artefacts, and there are things that are straight up impossible to develop without investing in additional equipment.
So it comes down to what kind of effect/synth you're wanting to develop, and how much of a purist you are!

(again, this is my opinion) The main issue with Agentic Coding (and especially vibecoding) is that it works by asking the AI to deliver an outcome. And you're then at the mercy of whatever method the AI decides to use, to fill in the gaps in your prompt.

Vagueness and lack of knowledge can be a problem, for example, ask it to create a waveshaper distortion effect. Sure, it will write one, and it'll probably be smart enough to suggest that it needs to be oversampled to reduce aliasing, or use a modern antialiasing technique such as ADAA.
Well, if that's all bundled implicitly into the vague prompt of "make a waveshaper distortion" then it will probably not devote much effort to deriving a good quality oversampler, it probably won't realize that a quick and dirty oversampler can audibly roll off your high end or give you ripples in the frequency response. And there is the tradeoff of latency to consider if you want to go linear phase... And designing the coefficients for an oversampler is an entire research task in its own right. Techniques like ADAA have tradeoffs depending on how exactly this waveshaper is going to be used in your final plugin. For example if this is in a plugin that already has latency, you could probably use a higher quality oversampler and compute it gradually over time, using the latency that the other parts of the plugin are creating as extra time to do processing, massively reducing CPU and allowing you to do a much higher quality result.

But since the prompt was just "create a waveshaper distortion" the AI will likely just throw together the most quick and dirty method, and then hand that back and say "there you go, the waveshaper is done!". And you'll be none the wiser.

For some people this is good enough.
But you can imagine how worse this gets for actually complex DSP.
If we take analog modelling as an example, the current leading industry methods involve measuring DC and other electrical measurements from real real circuits under different conditions in order to get the datasets you need to replicate how components behave when they are actually inside a specific circuit, and to be able to compare your emulation to the real thing and find out why it differs.
But an Agent, if asked to analog model a piece of hardware, is unlikely to tell you anything about this process, and will just "imagine" the measurements and then give you back a program full of made up assumptions, claiming this is a good analog model.

But the thing is, the program will totally work. And lots of people are happy with this, so really it depends.
If you're a hobbyist, it's probably going to give you a result better than you could've made without the AI.
Plus, the "simple" approach the AI takes means it's quite easy to debug, and you won't get challenging cross platform issues or with different hardware not supporting the approach.
And heck, if we are talking about purely creative sound design, then it doesn't matter: maybe that "roll off" in the highs and ripple, are part of "the sound" and people love it.

So there are advantages. But the likelihood for "quality" DSP or low-artefacts is very low.
Since it just skips right to the end result instead of doing the months of research required to produce the kind of work a professional engineer would deliver.

You can tell, I get frustrated with AI, but I'm actually not against using it for audio coding. I just wish it was better suited for the task, because I'm constantly arguing with it and having to give up and do it by hand

---

TLDR:
If you're writing stuff from scratch, you might run into problems down the line.
And you might need to end up learning all the DSP theory anyway, just so that you can tell the AI exactly what to do, or complain at it.

On the other hand if you're just porting existing DSP to C++ Scriptnode Nodes, that will work excellently as long as you make sure to tell the AI agent to check that the CPU performance and Numerical Outcome of the DSP is the same as the original code. AI can still make transcription mistakes and end up taking shortcuts that remove optimizations "by accident". DSP is pretty fragile math, subtle changes can end up creating a different sound.

If you're going to do this, I recommend Codex:
https://chatgpt.com/codex/?utm_source=google&utm_medium=paid_search&c_id=24184341331&c_agid=198291088223&c_crid=822139703727&c_kwid=kwd-111182835&c_ims=&c_pms=9208073&c_nw=g&c_dvc=c&utm_campaign=24184341331&gad_source=1&gad_campaignid=24184341331&gbraid=0AAAAA-I0E5dTcdT4kjWBldRGjwm0ADIX7&gclid=Cj0KCQjwxKfWBhDkARIsAMKwkNbDY-zY7xKGSKfYxHXKVElvVZLUsP64_A6UMpZV2h8trCJ-l6ZoeIEaArGXEALw_wcB

[scottmire]

Thank you so much for the response!

So here's the situation....

I think I'm a pretty damn good programmer, but I'm not really a DSP guy (I know enough to get by). I spent the 4 years at my last job working with Will Pirkle, so we pretty much had the DSP covered...LOL. Now, I'm trying to do as much as I can on my own without bothering too many people . I figured that maybe the solution would be to leverage AI to fill in the holes in my tool kit. I've tried SOOOO many AI plugin development platforms (Amorph, APC, Vybecoding VST, etc, etc, etc) and they all seem half baked. So, I'm back to HISE (which I love) but, to build some of the things I need to build, I feel like I'm going to need some custom scriptnodes. I probably just need to get in there and start the old tried and true way, but I was hoping I could save some time. I think what's been missing in my vide coding attempts has been that I need go to lower level in my prompts and specify actual DSP tasks that I'm trying to get solved.

[Sifres]

@scottmire Vibe coding SNEX is very possible. Some recommendations;

1. Create a Claude skill. There are some common pitfalls that you will run into because Snex is a C++ like language. So get familiar with the skill creator and start by giving it the empty template of a snex node + documentation (and forum!) link + keep in mind that the misnaming between node title in de code and the node title in the network is a common tripwire with a not so logical error naming. Also tell it to keep previous versions of the DSP and make a new artifact any time (so you can rollback). Update the skill anytime you work out more pitfalls in SNEX handling DSP.

2. Ideate first. Keep the modules as small as you can keep them and keep in mind that SNEX nodes dont accept 16+ parameters. Also you can use less sophisticated but still smart models. I use Opus mainly. Tell it to research DSP and become familiar with DSP research papers. Use AI as a sparring partner in this phase. The more specific the sparring the better (ie. dont try to prompt 'Create a buchla for me' but rather 'Do some deep research on what makes the buchla sound unique, dive into the circuitry as well as the synthesis'.) Tell it to make a sophisticated prompt, referencing any DSP it needs, for the smarter model. You dont want to use this sparring conversation to escalate to DSP work!!! Because that will exponentionally increase token size with all kind of context from the research phase.

3. Use complex and high reasoning newer models like Fable or Astra for the DSP building phase. Especcialy Claude is quite adept in DSP work and will actually run python scripts in its background to check it's own DSP math. You dont have to grasp all math, but basic DSP knowledge is recommended. Also use all kind of scopes that Hise offers, to look for weird phasing. The newer models are actually really great in DSP work. Just costly.

4. Debug again using the smaller models Sonet and as an escalation use Opus. But Sonet will be fine for looking up syntax errors or doing a quick forum check to see if someone came across the same error.

[HISEnberg]

@scottmire I believe you are on the most fruitful track if you want to leverage LLMs for custom DSP development in Scriptnode. AI is still not at a point where it can function as a replacement for DSP knowledge, but it can be a very helpful tool for filling in certain gaps. AI models have a tendency to cut corners and get to a solution as fast as possible rather than considering if it is really the right solution in a specific context (I think griffinboy's point covers this quite well).

Personally I like to start with research articles and open source github repos and either read or skim them to gather context about a particular topic. Identify which solutions work in your context, ask here about it or on other forums, and for the aspects which you find difficult to understand you can possibly consult an LLM (pinch of salt as always).

That being said, letting an AI autonomously work on a particular task and being able to verify the output is the most critical part. Specific harnesses and tests you can run and trust are quite critical here. This is true whether it's vibecoded or human written. I rely heavily on tools like HART. Pluginval is useful for validating your plugin, and PluginDoctor is very helpful for visualizing and measuring your results.

I also believe the hise-cli can construct scriptnode networks in realtime though I haven't really tried it as I prefer creating Thirdparty C++ networks.

[griffinboy]

@scottmire said in Vibe Coding Custom Nodes:

Now, I'm trying to do as much as I can on my own without bothering too many people

Feel free to bother me, I could talk about DSP all day. It's already the thing I do day and night.