ScriptNode Compilation Workflow and What to Do About It
-
Unfortunately I was not able to make it to the meet, so I decided to convert my notes about Scriptnode into a post here. On a second thought that’s maybe even better, as this way it’ll stay here easily searchable and visible.
I wanted to talk about the current ScriptNode workflow with regard to compiling of networks and/or nodes.
ScriptNode evolved a lot in the short time that I’ve been here, but the compilation workflow seems to be stuck in a time where it was a smaller scale add-on to HISE.
In its current state, the compilation workflow is incompatible with large projects, and by that I mean any network with a lot of interconnect and repetitive node chains.
I’ll examine three examples:
- Math Expression node
- A node chain template
- Faust
The Expression Node
I’ll use it as the most basic example of a problem that the current compilation workflow introduces. This is a very important node that could play a very significant role in almost any network, but I find myself avoiding it at all costs because it requires network compilation in order to work in the final plugin.
This is problematic because network compilation is not just freeze-in-place like a track in a DAW. You need to enable compilation for the network, run the compilation process, restart HISE, load your project, remove your network from the chain, add in a hardcoded FX module, and add your network there. You cannot leave your ScriptFX network in the chain, even if it’s bypassed, because the exported plugin will refuse to scan in a number of DAWs, and there’s no warning in the exporter for this. Even if it didn’t, that’s a whole lot of additional clutter.
This is fine if you need a small modulator that you set up and forget. It’s not fine for big networks where you are likely to be making changes a few dozen times in the future. It’s also problematic if you’re using that network in multiple places in your entire project tree, because in order to make changes, you need to now load in the scriptFX network in all the spots, and every time you make a change, you still need to reload in order for the change to be reflected everywhere, which in this case means restarting HISE.
A good workflow in big projects requires a lot of testing, which means a lot of plugin exports. Every time some major change is made to the project, I export it to make sure it still works both as a plugin and functionality-wise. If the plugin is conceived to be used many times in a standard composition/mixing session, that needs to be tested, too, which makes the plugin version of HISE unsuitable for the task (but also because it doesn’t have a FAUST version).
The Node Chains
There are cases where you need to have a node chain repeated a number of times across the network. This is the same as a function in code. HISE has node templates, but the major problem with it is that if you make a change in one instance, it’s not reflected in another. This could be beneficial, true, but in order to have the changes reflected in all instances, you need to do that whole thing in a separate network. This clutters the network list as the only way to differentiate between these “function” networks and networks to be used as modules in the main tree is to settle on a naming scheme and name the networks accordingly.
Once you have them all over your main network (loaded from the Project tab in the Node Browser), if you make any changes to any of them, the network requires a reload (unload the network, then load it again) and if you’re using them in multiple networks, it’s easier to just restart HISE. God forbid you used an Expression/SNEX node in there which mandates that you first need to compile the function network.
Faust
When working with Faust, you cannot just leave the faust node in your network and compile it, expecting the network to work. The compiled faust node needs to be used in order to export the plugin. This is a bit problematic because in the Projects tab in the Node Browser, there’s no real indication that this is a compiled faust node, and not a compiled network. A simple naming error can take you places. If you’re going to be using a compiled faust node as part of a bigger network, you need a separate network where you’ll be compiling the faust.dsp files, then unloading that network and reloading the main network where you’re using the compiled faust node. The compiled faust node by itself currently has a lot of bugs, which I posted on the Github issue tracker.
What to do about all of this?
I’d recommend to always look to Unreal Blueprint when in doubt about UX changes. That thing has been in development for the past 12+ years and has received contributions from dozens, if not hundreds of people. It's incredibly smooth and easy to work with. Unreal Blueprint has 2 types of “node chains”: macros and functions. Macros are project-wide, and functions are local to the blueprint where they were created.
A blueprint is the same thing as a network in HISE. When creating a function, a new window pops up and you build a node chain like you do in the base blueprint window. You can define function inputs and their types, as well as function outputs and their types. The function then appears in the sidebar while editing the main blueprint, and you can either add it from the search/browser or drag it from the sidebar. Any changes you make to the function itself will be instantly reflected in all the functions in the blueprint.
This would somewhat solve the Node Chains issue.
About Faust and SNEX, if compilation itself cannot be added to the export process automatically, maybe there should be a freeze-in-place option. Right click, freeze the node, it instantly becomes available in the projects tab in the Node Browser. If you unfreeze any instance, make changes to it, then freeze it again, all other frozen instances should automatically update. For faust this can be tracked based on the .dsp file, for SNEX it could be tracked using the file as well. No idea how it would be tracked for the expression nodes. Maybe they’d be assigned another type of ID on freeze.
I wrote about expression nodes requiring more inputs (maybe dynamically like the xfader node) as their strongest trait is that the expression is visible and instantly editable.
-
@aaronventure With the repeated node chains, can't you compile it as a separate node and then it will show up in the "project" tab and I believe that will keep all instances in sync.
It's been a while since I've used scriptnode but there used to be a thing that if you were using a compiled node it would show up with a snowflake icon, and you could click it to switch between the compiled and uncompiled networks. I just checked and that doesn't seem to be a thing any more, or I'm missing a step.
-
@d-healey There's also talk in the docs about being able to export a node (like a chain) to cpp, which you can then store in a location like the script library, which all projects can then pull from. There are some options when right clicking but it does nothing. That would be cool as well - the equivalent of macros in Unreal Blueprint, or even better (because they can be shared between multiple projects.
ScriptNode also needs like a reload button that doesn't require closing HISE to pull in the new nodes or refresh the existing ones if they're an embedded network and a change was made there.
