ScriptNode Compilation Workflow and What to Do About It
-
@Christoph-Hart please do fix the silent_killer not being passed through multiple polyphonic scriptnodefx all the way to the synthesizer group ;)))
a simplification of the workflow would sure be nice, but I have somehow gotten used to it, there's probably a lot of smaller bugs and glitches that can be far more annoying and could use some love, before you revisit the entire scriptnode system/workflow ;)
e.g. adding a parameter in hise still messing up all the cables sometimes, even if you click "rebuild faust parameters"... often enough only loading another faust script into the node and reloading the original one helps. I would document them if I could accurately reproduce them but it seems like it happens at random. ;/yesterday I had to create a Hardcoded Polyphonic ScriptFX node 8 times before it finally decided to remember the network I had assigned to it after saving and reopening the project ;)) I'm sure if we could get this a little more rock solid more people wouldn't be "I avoid compiling networks at all cost" ;))) I feel them lol
A couple of less alert popups after compilation, at the start of hise and during the save would be great though. By know I know I have to close and reopen HISE, still don't want to initialize the defaults on startup and am really sure I want to save the XML if I hit CMD+S and am even sure I want to overwrite it, though that question is probably legit although not standard on apps ;) I do this after every line I write, just out of fear HISE will crash on me when I hit compile and my line will be lost ;)) So that's a lot of confirmations a minute and I am such a lazy bastard as it is ;)))))
-
@Christoph-Hart also please let us color all nodes and the main container aswell as having the background of them also tinted ever so slightly, it makes things so much more neat and clear :-p
-
@Christoph-Hart said in ScriptNode Compilation Workflow and What to Do About It:
multiple inputs / outputs for expression nodes will create the question on when you want to do the recalculation - when the first parameter changes, when any of them changes, and to which outputs do you send it to?
No need for multiple outputs, just inputs, like the xfader node, so that the expression can then reference input1, input2 etc in the calculation. I honestly can't think of a scenario where more than 4 was ever needed, but if you already have the system worked out in the xfader node, it can't hurt, right?
@Christoph-Hart said in ScriptNode Compilation Workflow and What to Do About It:
when I'm back working on scriptnode
Will you at least be fixing the breaking bugs like the Faust channels thing and the compare-into-soft-bypass? That (along with ScriptNode Direct Access which you'll find the Feature Requests forum) are the only real barriers to my progress, everything else I can work around until you get to it (even the soft-bypass thing can be worked around but I can't imagine that being hard to fix).
-
@aaronventure yup the bugfixes are on my immediate todo list.
-
Cotinued from the Github discussion on Faust vbargraph: https://github.com/christophhart/HISE/issues/619#issuecomment-2602609622
This means that the Faust node needs to be within the network with the sole purpose of providing a public mod for the Faust node. Also, there can only be one public mod per network, and it cannot be named. It would be cool if it took the name of the publicmod node and wrote that instead of "drag to modulation target" (perhaps it could switch to that text on mouse hover).
Also, the compilation results in two additions to the project tab
The .dsp file itself (here named "test" and the compiled network.In the project tab, there's no tag or anything to tell us which is which. There's also no way to exclude the .dsp file from appearing here, even though it's (theoretically) useless without us having access to its vbargraph output.
This again forces us to use a naming scheme where we designate faust effects like fst_ or something and networks with ntw_ or net_.
Perhaps you could add a category for Faust in the browser, and another for networks.
-
@aaronventure or add a "private" flag to the faust C++ nodes so they don't show up in the browser and only the ones you wrapped show up?
-
@Christoph-Hart that's an easy way to hide them, yes. It would still be nice to see at a glance which effects are Faust. Does making another tab for the compiled .dsp files sound so bad? I guess you could add an F icon next to them in the projects tab, but this just seems a lot cleaner. Perhaps the tab only shows up once there are compiled faust effects to show?
-
@aaronventure that sounds super useful. Where to are those faust nodes compiled anyways? are they in the DLL or can they be easily moved to e.g. another project once compiled? that would open up new ways to build a whole assortment of nodes and tools and share them with each other without the need to share the source code and compile
-
Also, there's a need for a "Clean Network Compilation Directories" option which will clear out all compilation data.
If you compile a network of Faust .dsp programs, then remove some (or all) of them, then want to compile again, the compiled Faust .dsp programs will show up in the to-be-compiled list just because they were previously compiled.
This can lead to errors where you might accidentally use them (they're still available in the Projects tab) even though the original nodes don't exist anywhere, nor is their .dsp file present. But if you then pull the project back from the remote repo (given that you're not committing the compiled dylib), you can end up with a major whoopsie.
Given how the compiler writes the .cpp files for the faust .dsp programs into DspNetworks/ThirdParty, how should that be managed in gitignore? Isn't that also the directory where you put your own third party cpp files? How to handle manual cleaning?
-
@aaronventure There's already a special tool for this: Export -> Clean DSP Network files.
-
I see you already pushed a few commits regarding this. Nice.
I pulled the latest one, 0a39f3d4dfcceac85a0eb0e61051cd731b9fc2a5, and found a lot of my networks to be broken. I figured out it's due to nested networks being removed; or at least refusing to load as nodes, because if I compile an older commit, like 56023e4dff900aae469bd5c040c5a48686a52a39 from Jan 21, they load back up just fine.
There's no way to add nested networks anymore and the entire Project tab is missing from the Node Browser.
There's no mention of this in any of the commit titles so I don't know whether it's a regression or just work in progress.
-
@aaronventure yeah I might have been a bit too ambitious with the cleanup.
What is gone now is the ability to freeze / unfreeze nodes as well as to run the compiled root network from a script FX - this was a complete mess on the implementation side and probably caused the script FX to crash in a compiled plugin. I will add a tool that allows you to automatically convert a script FX into a hardcoded FX module as this is a major gap in the development UX right now but this makes everything more predictable and stable in the long run.
I also removed the ability to load uncompiled networks as a node (so nesting of uncompiled networks is not possible anymore). However I probably need to roll back this change as it was too intrusive.
-
@Christoph-Hart said in ScriptNode Compilation Workflow and What to Do About It:
I also removed the ability to load uncompiled networks as a node (so nesting of uncompiled networks is not possible anymore). However I probably need to roll back this change as it was too intrusive.
If you wanna introduce this back as a "function" (read again the last section of the first post where I mention Unreal) where it's available only in that network and the edit dialogue for it opens in another window and any changes are reflected in all instances of it, now is a good chance.
-
I figured out it's due to nested networks being removed; or at least refusing to load as nodes.
There's no way to add nested networks anymore and the entire Project tab is missing from the Node Browser.
Alright, that was only happening if you don't have a compiled DLL present - I've fixed that now to show dummy nodes that you can click on in order to compile the DLL - this should also work without restarting HISE now.
Feel free to check it out, I've also spent a few hours changing the UX of the entire compile process to be within the HISE window now so it won't spawn a console and block the UI.
-
I will add a tool that allows you to automatically convert a script FX into a hardcoded FX module
Alright that's also implemented:
This operation is also performed whenever you export your project through the new compilation dialog so it's guaranteed to never use a ScriptFX with a compiled network in a compiled plugin!
-
@Christoph-Hart Amazing work this will be such a time saver!
-
@Christoph-Hart awesome, thank you Christoph!
-
@Christoph-Hart Ahh that's right, now that you removed the need to restart HISE, networks marked as Allow Compilation can be compiled prior to the exporting process.
Is this what this dialog does? Are networks marked as Allow compilation now automatically compiled on export, i.e. there's no need for swapping them for hardcoded anymore just for the export and I can work on them, click export, test my plugin, then come back and the network is still editable?
-
Are networks marked as Allow compilation now automatically compiled on export,
Nope, but almost. On compilation the ScriptFX modules are replaced with their hardcoded counterpart so we're 80% there. What's missing is that if there is no DLL present at all (or the DLL was never exported manually once before), it won't create the C++ files for the networks that will be included in the plugin export.
I've added this step now too in the compilation process although I'm hitting a few issues with hardcoded FX not being initialised correctly if there is no DLL present.
I'll sort this out and then commit it, but this will make it very convenient - you actually do not have to bother at all about compiling scriptnode networks anymore and it will be all taken care of at plugin export - unless you want to compile the DLLs before to see that state during development.
-
@Christoph-Hart Awesome!
Let me run down the original post and see if I understood you correctly.
1. Individual SNEX Nodes and single-instance Faust .dsp programs
This is now solved because the networks get compiled automatically at export.
2. Function-like Node Chains and Nested Networks
You mentioned this could be behind collapsible node chains (lock thing). I mentioned functions in Unreal Blueprint. Also, if I got it right, networks cannot be nested unless they're compiled first. How does this play with a workflow where we need to compile just that network? To be clear, I'm talking about editable network chains that are used repeatedly across networks, which, when edited, update appropriately everywhere.
3. Faust Specifically
While single-instance Faust nodes need no compilation anymore, requiring a single .dsp Faust program to be used across multiple places in a network (or networks) will still pose challenges:
- when the .dsp file itself is changed, are all Faust nodes sporting that .dsp file automatically updated across all networks or do we have to manually hit the refresh button?
- why are parameters not being rebuilt on every compile? I found that there's zero downside to doing it other than the second long visual glitch, because the cable connections always get restored properly. It's just an unnecessary extra step that you always have to do when you make parameter changes, and it always does recompilation again anyway. The node craps itself anyway (all the parameters go away) if you have a code error, and the rebuilding happens then. Why not have it happen all the time and remove this manual step from the workflow altogether?
Sometimes Faust is a good alternative to making a long node chain, so this is in fact also the other side of #2 problem in this post, it's just done in Faust.
