Passing Large Data Objects into ScriptProcessors

[Lindon]

@d-healey said in Passing Large Data Objects into ScriptProcessors:

@Lindon said in Passing Large Data Objects into ScriptProcessors:

Ok so @Christoph-Hart - before I raise a feature request for this - do you know of a way to do this?

I think a dual control approach is the right paradigm here. Maybe start with a much simpler arp to test the concept

I think this just doesnt scale....it will be nearly unmanageable - globals are better than this - in the case of thousands of controls....

[Lindon]

@d-healey said in Passing Large Data Objects into ScriptProcessors:

@Lindon said in Passing Large Data Objects into ScriptProcessors:

@d-healey said in Passing Large Data Objects into ScriptProcessors:

@Lindon Correct, but, now that I think about it, I believe you can set a panel's value to an object. I don't know if this works via setAttribute but worth trying.

tried already - cant get it to work...

Got a snippet?

its nearly exactly the same as the one I posted before but here you go:

HiseSnippet 1113.3oc6WEtaaaCDlxNrHVqqXEaO.DB6GxXYt1accConXN1IYvXKoFyYACXnsfVh1hKRjBTTsyHH.6EXuO6wZuAaGkjskcbxBTSB1OlLfMH4c79ti799jGpjdrjDoBY03jYwLj0CwilIzA8CnbAZv9HqGgOhlnYJR9T8lESSRX9HKq5emYBqFagxd9qusGMjJ7XKmBgNUx8X+.OhqWN6vteOOL7PpO6DdTIqeZ2AdRQeYnLEvScbaTL06L5T1wTiY0vHqGbfOWKUizTMKAYsUOo+rQAx2Ixs+TdBebHyLnCZDrQ4SenLz2fXyrn9A7P+gyy6DDxBObYUnddU3SvGw84KleY03ixVfrzix0CqZWG75TFds2H7ZuI3YUBdakCuGiG4o3w5kqXv1GfGHfCpIT3HnLrxsEU6Opg6KAKD5VQzyXGpfAK7v8YsauCA9p4yssgigDM4sTEYs3zg7BR18iVSY5UJQtN4ltrzzwwrUbQHWvHSREdZtTPjhdoZsTzw.EkLz0SFEKE.n1ABXXJqo841MdxS7kjDYDSGvmJHALEinkj3TMAlIgvEvuLRLUvBmOX83a1Ex4N64rqSOmKranXSIQyFZxf0RJSxrmVq3iS0LWSEnAftDYHqUrhKztN7HhPpIfcZtXJgRTrI.lfK6jeMEJUTRrLgaRvccHelILlMI+ykhVR4n4LzjCcb1YARy77hmaO+rBBZ+40HWmhpmSSy1TTC6SCCGCsJtqWbgspTg+Xol8RgaVA19Bax5KMYxFWqXuBYpMtroGVccN5JRiFyTkOdMFBWlWsaAe0cKkal8xKKkLTJFH35WFyDWUKNpnVZ5lJPEXpNqW5QE8R4UNDGZYZfKJinLHWlhC8SC1mpoy2HXOg3DyTZtIEr1m8VfuKuIsAdeVxYZYblsEGfPVeCB6usHjceU2YKF76i6htLY.PII8SCo5U4oLjyEK.GGqPHXZ5Ev00Ykyr2axqMV3ugv8w3gbsWvlwasMfW3.3tFuERAeH9fISXd5kfcK7g+7sKu++FT19JzleXFgNLho5banLGeiUlspeWnDu0ckRb8a1Ix6i56Giuj.3FUg+SKLYo9ZN6OnIsY19BwAfGuvxV9.6SK5IAFYnWP9EfLkXSZ3PgLUocH6Rbn4DsN1jKdE3mMoBRvjygMcUMPsx7VBygA.ySMl51roAbfbE41Quh7+BV+GPvpVIAqGnK3AKt0at.jE0sw4WFJIWAOqPR+NtuNX4DC5Fv3SCzkYVPUVaD85RZinO8dTarxzI2i5g2JXr5ZfUO7+nL07R1GQgWQFNrwGmFMBzc7XPzEv8MSmmUMSOR931lwlJvHlvOaveCOEK1wL1pXwNyW7dIFQTOk7Md4cuFojsylAxaQ1+msA7GqgwjEMsXb6VsQQfp1a77Lk3OGnF1rOeQE74KqfOOsB97UUvmmUAe95J3y2bs9Xdgh8R0xn7VNXhgGjQdZYcffB2dytoi9Gv5WkbL

[aaronventure]

@Lindon said in Passing Large Data Objects into ScriptProcessors:

globals = bad code.

How so?

@d-healey said in Passing Large Data Objects into ScriptProcessors:

Yeah I avoid globals too. They make code less reusable/portable. Where possible i try to make everything standalone.

How are they worse than latching onto a referenced parameter via setAttribute?

You can either write all the large data to a global and have it just be there for the arp to grab, or you can store the arp controls references into a global, change their values from anywhere based on your data output and have a broadcaster fire on control value change if you need precision regarding when the controls need to do things.

[Lindon]

@aaronventure said in Passing Large Data Objects into ScriptProcessors:

You can either write all the large data to a global and have it just be there for the arp to grab, or you can store the arp controls references into a global, change their values from anywhere based on your data output and have a broadcaster fire on control value change if you need precision regarding when the controls need to do things.

Ok so this isnt MVC in any way I understand it. To be clear there are NO controls in my ScriptProcessor - its all code in there - it uses a large data array to fire notes in an arp - the "controls" are a single large panel in my main interface which uses a paint routine to drawn all the controls the user interacts with - so the UI is separated from the processing...

So what we are discussing here is how we separate out the presentation of the ARP to the user from the "engine" that does the work...and how we might pass the large "blob" of data that this presentation generates..

[aaronventure]

@Lindon you mentioned you can't figure out how to pass an array to a panel.

Your array should be stored in a variable on its own, and then you should pass that variable to the panel via setValue(variable).

[Lindon]

@aaronventure said in Passing Large Data Objects into ScriptProcessors:

@Lindon you mentioned you can't figure out how to pass an array to a panel.

Your array should be stored in a variable on its own, and then you should pass that variable to the panel via setValue(variable).

no thats not quite what I said.. I said I cant pass an array to a panel in a ScriptProcessor - check out either of the snippets I posted.

[aaronventure]

@Lindon

Weird, can't get setAttribute to work here either.

Globals work fine.

HiseSnippet 1241.3oc6XEnaaaCDkxNpHVqqXEae.2DFvrQBbry55FZPwbiSRgwZRMlyBFPQWAsDsEWjHEnnRlWP.1m79C1NJIGK63zEXjLrMTAACSd7t6w63cOZ2WI8XIIREwp1wShYDqGZOXhPGzMfxEjd6Qrdj8gzDMSA4Ss6jXZRBymXYU8klIrpsFI64O9tcogTgGa1TDxIRtG6U7Htd1r86787vvCn9ri4QkV8S5zySJ5JCkoHdpZ2hDS8NkNlcD0rrJ1DqGruOWKUCzTMKgXs1tR+ICBjmKxW+I7D9vPlYPax.zP4SefLz2fXyrjtA7P+9S22IDhkc+YQgp4QgOy9PtO+p4mEM9jLAvLMJGOrpLO7pNG7ZWFdsJAuk.IqRPZsbH8X6AdJdrdlDCd9H6dBL4Lhhg8xPIesjJaT0tqDWgP2LhdJ6.EN3JMp+zVs1DvOZriiCF5SzvYTErfeZCOGxNSzbLSOWXota9RmENZ6ZLkihMF7zJTwV3PtHjKXvnTgmlKEfTrapVKEsMHSICq6IihkBDiah9OLk0v4BmZ7QP8OGMRCXqsfXE6LTN3QCQaMFMAvEbMLj4QSSXfNfAwTAKDPLljIiSC4+FyG7wvelbOSRzoFZ5ZF7ESmDJo9HFuv8EtOycW2K2AEgun+RXZswOZIPgwgxgzPPwFwTL77MvS.FMYxlvDYJ7KoIFbI9RM5J.wzhXC+Z.MwHeHiI.OECOc3CSXZzW8MftcSzemX140KPUicfbrj8doC9AhpTQHFky7Zj7r7ccA3nCMSHUFDGfvbSfJ7MKP.I7n3vIkguNOnLjEJOuI5nBaLdogD70Y5QHL31cZtptaQVzsgA9E4xtXBZHV0Vewjb1QLsZiM1oz4fijZ1qE0yx2NW5.KJZznkJqvjgL0REa5pndeJVWjFMjoJeZyrPrTa95W6at9sb6Eu7nSoEJE8vS.uNlItolNjhPpoVu.U3R0YU5OpnROO.R3XAcM6hnIICxka5R9wd6Q0zoFBsI5mXlRyMaAq8XmgcfyagTydOVxoZYb1ZKxi3t9V31e8JW14sclb0feeXGx0aUgMIk9ogT87cNMzEEBvzwbsqLsjDIb8jx6r6r1o2VH9X69bsWvxwXkkfQLneefwBRnO1d+QiXd5Y.bM6C9o6GFmxte8a3l.OLiJAGwTsuKtGP7s9d.VUuO38W6+v79ep80ndWJ++fJ1NEs2yoZvl6KuadtXC+8raBjM21ueU1Nmxuv7M8wdQMoGGXXOeN7Fr0J3.0boXrQocgmAtz71ttNvkucGGT5Jb+.3Bzn4teF243qHOQg3KpdhLj0LVwMn8kYgAD.vFfVYt9SAjGO0.MZjo5B58JIdqisLSpulxaunxHaIb2vVBefm7eA7jUKwS9.cQa4hBRyAfLuttc9IoRrj3yb7Dmy80AylnWm.FebftbSOxJSIS94RTxjuXYV5FA81F6XaU.37u7AJ8+WPo+CxTyOh4PpVwMI4iRiFfTpdLz6BL0apbspXpwxG2ZZ9e.S3mM3OwmBgsMisJD1dpv+Q7QD0SIemWd0ugkb8rYv8sH6OFnl8glwv0J5IjHjz9cddyapqo31qphe0pp3SVUE+5UUwmtpJ9Mqphe6euhl6T8hTsLJuzjPNr+9YMosr1WPwS4YUDj+BL5aEgJ

[d.healey]

@aaronventure said in Passing Large Data Objects into ScriptProcessors:

How are they worse than latching onto a referenced parameter via setAttribute?

With Global both scripts must be aware of the global variables. Using set attribute is no different than interacting with any other HISE module. The module doesn't need to be aware of the place (or places) it is being controlled from. It is self contained and can operate independently of the UI script.

[d.healey]

@Lindon said in Passing Large Data Objects into ScriptProcessors:

I think this just doesnt scale....it will be nearly unmanageable

Given Christoph's lack of responsiveness lately I suspect he is busy with some project and you may have a wait for your feature request.

I would flip around the way you are doing things. Currently you have a GUI that you want to pass to a processing script. Instead I would build a standalone arp script and then figure out how to build a GUI that can communicate with it.

I don't know the details of what you are doing but the simplest solution to me seems like a few sliderpacks.

You say there would be 20 sets of controls, and I assume the scale issue would arise if you wanted to add more sets in the future. You obviously don't want to end up with 1000s of sliderpacks. So we need to find a solution to this.

I don't know what your 20 controls are, but assuming each can be represented with a single slider pack, we add 2 slider packs to our arp script. 1 is a visible sliderpack that we interact with and has 20 sliders, the other is hidden, and is used for data storage and can have as many sliders as necessary to store all the data we need, giving us unlimited "virtual" sets.

Then we have a knob to switch between our virtual sets. Pulling values from the big slider pack and displaying them on our smaller slider pack. I can write this for you if you like, it won't take very long at all.

Now on your main UI script you need to add a hidden 20 slider sliderpack, and a knob. You connect these up via processor/parameter ID to the ones in the arp script. You can continue to use your data blob as you currently do in the UI script, and forward that data to the arp script which will do the work. The arp script can also be used independently of the UI script, making it suitable for other projects with different requirements.

Edit: I just saw you said "128 steps, each step has around 20 attributes". So that definitely is more complex... maybe this is a time to use the global object and then take a shower.

[Christoph Hart]

Given Christoph's lack of responsiveness lately I suspect he is busy with some project and you may have a wait for your feature request.

Lol I haven't looked at the forum for 4 hours...

Without understanding exactly what you're trying to do, I would recommend using one of those:

https://docs.hise.audio/scripting/scripting-api/fixobjectarray/index.html

as a global variable in the Interface onInit array, then create a fixed size array of 128 elements with whatever data structure you need for your arp.

[d.healey]

@Christoph-Hart said in Passing Large Data Objects into ScriptProcessors:

Lol I haven't looked at the forum for 4 hours...

Lol I meant that we hadn't had an update in a week! That's practically glacial for HISE, also you're usually busy with theatre work around this time, no?

[Christoph Hart]

@d-healey ah yeah, no theatre this year, but I have been working on a new feature (a multipage dialog system that will be used inside HISE as well as a floating tile on compiled plugins).

[d.healey]

@Christoph-Hart said in Passing Large Data Objects into ScriptProcessors:

(a multipage dialog system that will be used inside HISE as well as a floating tile on compiled plugins).

Oh yes I saw the commits for the broadcaster wizard. Looks like it will be very helpful.

[Lindon]

@d-healey said in Passing Large Data Objects into ScriptProcessors:

Edit: I just saw you said "128 steps, each step has around 20 attributes". So that definitely is more complex... maybe this is a time to use the global object and then take a shower.

That seems to be the plan.....just using Christophs suggestion for fixedobject array.

[Lindon]

@Lindon said in Passing Large Data Objects into ScriptProcessors:

@d-healey said in Passing Large Data Objects into ScriptProcessors:

Edit: I just saw you said "128 steps, each step has around 20 attributes". So that definitely is more complex... maybe this is a time to use the global object and then take a shower.

That seems to be the plan.....just using Christophs suggestion for fixedobject array.

I predict that at some point in the future someone is going to ask for a way to pass a large data blob into a ScriptProcessor..... :-)

[Christoph Hart]

@Lindon said in Passing Large Data Objects into ScriptProcessors:

I predict that at some point in the future someone is going to ask for a way to pass a large data blob into a ScriptProcessor..... :-)

And I predict that I will respond to it in the same way. Some things can be changed, but this is not the case here, the API needs to be as fast as possible so I can't switch it to a var value.

[Lindon]

@Christoph-Hart said in Passing Large Data Objects into ScriptProcessors:

@Lindon said in Passing Large Data Objects into ScriptProcessors:

I predict that at some point in the future someone is going to ask for a way to pass a large data blob into a ScriptProcessor..... :-)

And I predict that I will respond to it in the same way. Some things can be changed, but this is not the case here, the API needs to be as fast as possible so I can't switch it to a var value.

So maybe an alternative, not using scriptProcessor.setAttribute but say scriptProcessor.setPanelData(PanelId, DataBlob)