Scriptnode breaking changes

[Christoph Hart]

Hi everybody,

as you might know, I am currently rewriting big chunks of the scriptnode framework in order to make it compatible with SNEX JIT compilation.

While I'm at it, I've found a few things that I will have to redesign that will not be backwards compatible. I'll try to avoid this as much as possible, but I thought I'll drop a list of all features that will be changed and need some adjustments in order to work with the new scriptnode version. I know there are a few people already working with scriptnode, so if any of you is relying deeply on one of the concepts below, let me know and I'll try to think about a migration strategy

Everything about SNEX

this one is pretty obvious, but the core.jit node will not work in the newer versions because the language has been replaced by a far more powerful C++ subset that can be fed with the output of the C++ generator in order to JIT compile the scriptnode patches. This way everybody benefits from the power of JIT compilation and not just the people who are using the core.jit node.

The API for writing custom nodes will resemble the actual C++ API (some of the C++ nodes in HISE will actually be JIT compiled by the SNEX compiler if it yields a performance gain).

Parameter Add / Multiply modulation

the OpType property in the connections is being removed and replaced with a dedicated node called core.pma (PMA stands for Parameter Multiply & Add) which is an intermediate node that can be used to modulate parameters:

The reason for this is that the old implementation was messy and complicates the C++ output. Also I find the new workflow more intuitive (on the screenshot you see the multiply-result in red and the delta that is added in blue)

Hardcoded nodes

The C++ exporter needs to be completely rewritten in order to be supported by SNEX which means that every scriptnode patch that was converted to a C++ file will not work anymore. Luckily the C++ code contains the scriptnode patch as Base64 snippet, so if all goes well, you can just take that snippet, load the original patch again and then export it with the new C++ generator once it's ready.

Send / Receive nodes

I am not 100% sure about that one yet, but it might be possible that I have to rewrite the way that feedback loops are implemented so that the routing.send/ routing.receive nodes will be affected. Let me know if anyone heavily relies on these nodes, then I'll try to think about a way how to use the old system.

[ustk]

@Christoph-Hart Whouhou!

SNEX

Will it be straight forward to recreate the old JIT node in the new C++ API?
I'm afraid this will be too complicated now I'm just getting my mark with jit...

PMA

It promises to be a good addition, even if I never used add/multiply in a connexion...
In fact, I realize that I've made my own conversion with nodes, not thinking I could use that feature...
I'm sure it could have saved my life sometimes, so I'll think about using this in the future ;)
So thanks for that!

Hardcoded nodes

I'm just getting my marks with JIT nodes, I'm afraid the C++ API would be too hard for me now...
What I'm worried about is re-importing my old graphs, and make my core.jit to work again with the new SNEX
Actually, the different core.jit I have are not exported as C++ anyway, simply because this currently doesn't work with jit nodes in the current version...
So what will be the procedure for re-importing old graphs? (that don't contain custom exported nodes)

Send/Receive

I use them a lot, but currently not as feedback loops, but generally just to send some modulations or signals here and there in the graph...
I don't mind replacing them by hand but, how the transition will be implemented when importing old graphs that contain deprecated send/receive?
Also, will it be possible to send to more than one place (one routing.send to several routing.receive)? Would be cool ;)

And finally the crucial question

Do you have any rough idea on when this will be available? in weeks? maybe 2 months? more 5 months?
Because I'm really waiting for the new version AND some bug fixes for releasing my FXs (one is ready for a long time now...)
I know it should be hard to tell but I need to have a rough idea so I can manage my time to focus on certain things more than others...

In the waiting, I can tell I have really good times using scriptnode!

[Christoph Hart]

Will it be straight forward to recreate the old JIT node in the new C++ API?

Kind of. There will still be the same callbacks as in the old jit node, but it will be wrapped into some boilerplate code around the SNEX code that will make it a valid C++ class. So

void process(block data, int channelIndex)
{

}

void processFrame(block data)
{
    for(auto& s: data)
    {
        s = something();
    }
}

// reset(), prepare() and handleEvent() are also still there

becomes

struct MyProcessor
{
    static const int NumChannels = 2;

    void processFrame(ProcessData<NumChannels>& data)
    {
        for(auto& ch: data)
        {
            for(auto& s: data.toChannelData(ch))
            {
                s = something();
            }
        }
    }

    void processFrame(block data)
    {

    }
};

You might notice the ProcessData<NumChannels> argument - this is called templates in C++ and allows you to do very elegant compile-time code generation - in this case the compiler knows that the channel amount is always 2 and will unroll the outer loop.
Also you can use the scriptnode container structures inside SNEX with the same syntax:

container::chain<MyProcessor, MyProcessor> chain;

chain.process(data);

which has the same effect as if you would have two MyProcessor nodes in a container.chain node (and this is how the C++ code generator and SNEX will work together - it will basically create one big class that mirrors the scriptnode patch using this concept).

So what will be the procedure for re-importing old graphs?

You need to take the base64 snippet that is being embedded into the Cpp node and import that from the clipboard (you have to prepend ScriptNode so that it will be detected as scriptnode patch), then just reexport it with the new generator:

juce::String instance::getSnippetText() const
{
    return "123.blablabla";
}

so in this case copy 123.blablabla to the clipboard, paste it into a text editor, prepend ScriptNode, then select everything and copy it to the clipboard again. Now if you try to add a node in scriptnode, it will show it as first entry...

Also, will it be possible to send to more than one place (one routing.send to several routing.receive)?

Funny, I didn't read that, but implemented this functionality today. You can now also drag and drop the connections just like any other connection in scriptnode (and it shows a cable). As far as I can tell, it won't break the old patches, so we're good here...

Do you have any rough idea on when this will be available?

Hard to say, it's a LOT of stuff left to do, but I would say that there will be a working prototype within the next two months, and some more time will pass until it can be confidently deployed in actual projects.

[ustk]

@Christoph-Hart Everything sounds very promising!!!

So the same goes for processSample I imagine?
I hope for some more example in the doc for each callback then :)

[Christoph Hart]

The processSample callback was removed, because there is no need for it anymore. But you can simply do something like this:

void processFrame(span<float, 2>& frameData)
{
    for(auto& s: frameData)
        s = processSample(s);
}

float processSample(float input)
{
    // your old code
}

The compiler will be smart enough to inline that function call so there is absolutely no overhead for wrapping the DSP signal into the processSample function.

And yes, there will be more examples, but good thing is, the HISE C++ source code of the existing nodes is also one big example because the API is almost identical :)

[AxiomCrux]

@Christoph-Hart said in Scriptnode breaking changes:

core.pma (PMA stands for Parameter Multiply & Add)

That node looks really interesting! I am very excited about this Scriptnode feature. I have been raving about it to my friends after my initial tests earlier today.

I will for sure be reporting any bugs I come across in an effort to help get this to a stable state :D

Also, regarding the naming 'PMA':

I was just imagining myself scrolling through the node list trying to find this basic and most essential and common node required for dsp modulation and I would never think to check a node called core.pma. I think it might be worth considering a more obvious name for the node. a title more obvious to its function, such as "core.scale" or "core.*+" if that's possible. Or even "core.map" ? Just my initial thoughts.