@DanH might want to run HISE in the debug version and see where it gets stuck, looks like the code generator is stuck in an endless loop (there are a few node layouts that might cause indefinite recursion).

@dannytaurus yes.

@hisefilo I mean you're kind of limited to this exact pipeline at the moment so if you want to play around with some hello world type stuff I would recommend a simple classification system that detects different waveforms (like static sine / saw / noise) spectrograms.

Once you get that going, we can think about other use cases and how to expand the integration towards other input / output scenarios.

@Morphoice Yeah, VST2 as a plugin API is dead in the water but this just redirects the VST2 API calls to scriptnode so it's not actually using the VST2 SDK - it basically mimics the API so that you can reuse code that relies on the VST2 SDK.

how do I find it?

It should show up in the project.xxx node list. Have you created the C++ template first so that it registers the node file properly?

@Morphoice There's this repo with some glue code but I haven't used it for quite a while:

GitHub - christoph-hart/vst2scriptnode: A simple boilerplate file to wrap headless VST 2.4 plugins into scriptnode nodes

A simple boilerplate file to wrap headless VST 2.4 plugins into scriptnode nodes - christoph-hart/vst2scriptnode

GitHub (github.com)

Let me know if it still works.

@hisefilo it's currently only used by the processFFTSpectrum() method which directly grabs the spectrum from the FFT object. The workflow for this is:

1. Create the FFT object, set the spectrogram properties suitable to your task
2. Run the FFT on your training data, then dump the spectrums as images
3. Train your model with these spectrograms (I've been using TorchStudio for that).
4. Export the model as ONNX.
5. Load it back into HISE
6. Use the FFT with the same properties to create spectrograms of the user input and feed that into the processFFTSpectrum() method.
7. It outputs an array of float numbers that you can use for classification or something else.

This is a very narrow use case but I've been using that to train on samples to detect the release trigger point and some basic classification of drum samples with moderate success.

I don't think you can safely hook into the threading model of HISE with its dedicated loading / scripting threads as they don't work across DLL boundaries, but you can always spawn your own thread if you want to perform heavyweight tasks.

I would just use the juce::Thread class to spawn a thread that performs the operation. Just make sure you are synchronizing the data access correctly so you don't end up with race conditions.

FIY, I added a FileSystem.Music constant which brings you directly to the folder.

@d-healey just tried to delete it. Didn't work.

@tomekslesicki what's the old tensor-flow method?

There are two neural engines available in HISE:

1. RTNeural which is focused on realtime processing of audio data. It comes with a limited set of layer types predominantly used by neural networks that process realtime audio (so eg. there's no use of adding a transformer or LLM type neural network support as this would not run in realtime. You can load neural networks into this engine using two formats (PyTorch & TensorFlow) with the respective API methods.
2. ONNX runtime is a more general purpose network inference engine that can be used to run almost any kind of network as the framework is basically feature complete. Of course the integration into HISE is very spotty (I just used it for spectrogram analysis so far), plus you need to build a separate DLL and ship it with your project because the framework is very heavyweight so I wouldn't want to include it in the default compilation process.

@d-healey said in How to access the default music folder with filesystem?:

I don't think you can rely on a Music folder existing

Sure, in fact that's one of the only "sandbox-safe" locations for music apps on macOS (eg. GarageBand can only access the sample content from there).

I'm using this as default location for all my sample-based plugins on macOS.

@d-healey yeah but it has a very limited set of layers so you can‘t expect it to load any model. The ONNX runtime in HISE is basically feature complete and should be able to run any model that you can export into this format.

@hisefilo check out ONNX for this, I‘ve been using it to run non realtime models in HISE for stuff like spectrogram analysis etc.

@JulesV whats X and Y?

@aaronventure can‘t you just bind a function that you call from HTML when the DOM has loaded?

@VirtualVirgin you‘re confusing JSON and Javascript. Javascript can define objects with functions as values but JSON can‘t. If you‘re storing and object somewhere it uses the JSON format.

What you need to do is to define your functions in a Javascript object in your script and then just store the function name in the JSON that you export.

@VirtualVirgin you can‘t serialize a function, JSON is a data only format.

@Orvillain yes, but you can't access the buffer of an external file to implement eg. a granulator (or simple sample player) directly in Faust.

@Orvillain BTW, here's a SNEX code that implements a very basic pitch-shifting algorithm, but it also uses the internal container types & index algorithms as a best practice on how to use them in SNEX:

hise_tutorial/PitchShifting/DspNetworks/CodeLibrary/snex_node/PitchShifter.h at master · christophhart/hise_tutorial

The Tutorial project for HISE. Contribute to christophhart/hise_tutorial development by creating an account on GitHub.

GitHub (github.com)

other than to say it is more complicated than just doing it directly in scriptnode.

yeah that's the entire reason of existence of scriptnode, to make it easier than writing code :)

But I've updated the docs for the delay nodes to reflect that information.