Variable Persistence

Christoph Hart

Alright, the GUI glitch is still there, but this snippet has the correct functionality:

HiseSnippet 1387.3ocuW0uaaaCDmJMBc1adXcXO.b9elRqmhURZSW6BVyGMqdqM0XNHXCAAMLRTwbUlzPhJotE4cZOZ6MX6NRIK4D2t5ArYDmDdeP9i+t6HO1OUExyxToDmO8vIi4DmOycvDod3tCYBIo2dDmO28ErLMOkZEsyjwrrLdDww4V+HJvowxDym+7G1gkvjg7JQDxQJQH+4hQBckzSexOKRR1mEwOTLpl0a7jdgJ4tpDUNfma41kLlE9Z147CXnYK4RtPvuLi3z0c807SecvYucae3yF60e8r9ax886+Le+Cx24x6Yja+riuOILOMkK0GAtSbbc9K3ii6SiDZU5.MSyg4zcGUzjACUWJsK8QhLwYIbbP.Y.fIq3dRfJhYg7ZltuJIBIDzRxtCEIQ8Ko0LBwY49Uj7srj7W49BQjXp7Jx9KLJnUdTmtcVZVHu7LPN38A4J30cdvyoF7V1Bu63NHLULVWoAw1WVH88fNqRxR921lA4yhhNTsepRp8zo47Udbql6BCf3feFW+Lt37gZuf62EUzp4ErT5nIamlxlP2hd7IfvhgG28DTRPG5Zcnq2gtQG586PePG5lcnOrC865PC5V27fEy70VLyWewLeiRy6BZp86q+yIFR.R+yzTjJFyj7Dv0RFCHy9nHu1FMssdgTmYLxndsuvl..JiYIY7qoMicAumreJGFAlTFSZ0b06R2SI+FMUHEZpdHmxLggg7TtQ+p8hQwobpHiNlmlIxPPQiXZF3jwEKfg+SRSTrHp.mNspZ5Z0TDS89ZOKhNmqOhkjy8VwOgKOWOjt0VaQykQ7XgjGsxJsZ9tVMaDqRodHeH.tn6ig+78kYIE9Axt28.qafl2nj1EHse8UBjB62FWAeAZMSkv8GmBfzqsMs64.t4QsQR4Jbee2Us69AIhH37u9vgQTXGEIxFm.la18pXZUntd.zncl3mcZvYwqMpsVLb0UoG+SCd4AFuNYl5DnXCXbsfmAkRiPil5Mrioz1WJhzCa+H5lAc6XDLzTYARBVuPxLQ9GQKkZvy1iT4Ri0qUHVyGOP7VNHpcPafJJA3p0QHl0Tl1DgQYHynJeYuBJ5Bj3ynw.xqQSXJicwwy2a07CDjAquVbFCyHFPtwxnaq6AoMuwSzolWPrtVbLNWFpEJIUIOPo4uT5sRy20rQyqZdcMwwySEFORUII7z4oEuEK8C3lmLezY7zNV5nXO.L5gPogIM4R35P5YbJDffi24Qk0TSmAZHKI4LCYgN964PRlwTJiFpFOASCsEgovUk3k0vLTPFfKXgmECPUlsr35UL.yaPGVfz.+BKSYTrp71FOQ0KTLqgcoVf3VCSUZCaw5M1A37LcC..c1s9oFTdJRALZUQ2TNDODC8PfqdwLHU1C9Lt5S+MUNXNDBOSyJhEFMzKDL54ka.ywJaq8Vo9IOvVvXJd.jY+dCysYAEmDcEbu7r2p27i6V8xL4WNlWX3zL3RQnQUYt0kZxXmiYUhBsm.UqKGkrGb8P0xcyVIHEGagMUTLgfoZSKEsJao.yiHBniga6ZxoHF5f37Gt3wKG+ObyZG5+GFL8Z4fZWZGTW3ImPH2ruIn6MUTdBSOaKcXaxEJfyElooIrcRYlPOodazKPedcmaedyM37QB2631WnCGNe7tzbvKDs+uFuEcM2x8oww7PcEXW1c+e8eaKxK.T9EUtVHO+ELcp3MvSENHez.3MJgb.IRHGFe9fyRXAfcbWbLxLC3xntku3nPY.N1oPYPoRxHVXp5Ug1xFru7OwHAvjz75mFvyvfwzfxBFD2ifGK7pvPjH9V.4y2i0VXOVeg8XiE1i6uvd7fE1iMWXOd3GvC7kYamqUirkEDxeCmbrLqB

Basically you made two mistakes:

The persistent values of interface widgets are initialized after the onInit callback. Internally this happens when you hit Compile:

Save all interface values which are set to saveInPreset
Clear everything
Recompile the script
Restore the interface values with the previously saved ones
Run the onControl callback for every stored interface widget

So it makes no sense trying to restore the data in the onInit callback as it will be merciless overwritten by the previously saved ones. Instead you must use the onControl callback of the panel (the value parameter will be the saved object, so you can directly assign it to myArray.

The control callback of a SliderPack has the changed index as value parameter.
You can use SliderPack.getSliderIndexAt() to obtain the actual value (if it would be the other way around, there would be no way to get the index of the changed slider, which is useful for stuff like arpeggiators etc.).
I admit the number parameter is extremely poorly named (I originally passed the index of the interface widget and not a reference to it and changing it breaks older presets).

d.healey

Thanks for the detailed response, it's good to know we can use one panel for many variables. I'll check out the patch and play around some more with persistent values :)

d.healey

Based on your pointers I've put together this little module, the only problem is it seems that my public functions can't see my private variables. Is this a HISE thing or some daft JS mistake I've made? I'm thinking I can probably streamline this more and get rid of the persistentStorage object but for now I'd be happy if I can just get it working.

var persistentData = (function() {

	//Persistent data must be stored in a GUI widget
	var persistentStorage = {}; //Object used to read and write data to persistent panel

	var persistentPanel = Content.addPanel("persistentPanel", 0, 0); //Hidden GUI widget for storing data
	persistentPanel.set("visible", false);
	persistentPanel.set("saveInPreset", true);

	return 
	{
		get: function(key)
		{
			persistentStorage = persistentPanel.getValue();
			return persistentStorage[key];
		},

		set: function(key, value)
		{
			persistentStorage[key] = value;
			persistentPanel.setValue(persistentStorage);
		}
	};

})();

Christoph Hart

Your code seems to be a bit too complicated - too many brackets :)

But you can wrap this up into a class like function like this:

/** A wrapper object around a hidden panel.
*
*	Create it using `new PersistentData()` and call its `set` and `get` methods.
*/
function PersistentData()
{
	this.panel = Content.addPanel("persistentData", 0, 0);
	this.panel.set("visible", false);
	this.panel.set("saveInPreset", true);
	this.panel.setValue({}); // make sure the panel's value is a object
	
	this.set = function(key, value)
	{
		this.panel.getValue()[key] = value;
	};
	
	this.get = function(key)
	{
		return this.panel.getValue()[key];
	};
};

// Example usage
var data = new PersistentData();
data.set("test", 5);
Console.print(data.get("test"));

Notes:

notice the usage of the this keyword, understanding it is crucial for every object oriented approach within Javascript (it will also fix your public / private problem).
you can access the object directly from Panel.getValue(), no need for a temporary object
make sure you use the new operator when creating the object.
this class should only be created once (because the identifier of the panel is hardcoded). If you need more objects, pass a string as argument and use it for the panel name.

d.healey

Thanks again Christoph - you always have a good solution :)

d.healey

I've found a small issue with the solution. Check out this example:

function myModule()
{
	this.chicken = "Bird";

	this.getChicken = function()
	{
	    return this.chicken;
	};
}

var module = new myModule();
module.chicken = "Dog";
Console.print(module.chicken);

chicken should be a private member that the user doesn't have direct access to but using the above could makes everything public. In my previous example I was using an iife (hence the extra brackets) which would usually allow private and public members but it wasn't working as I expected in HISE.

The other solution I've tried is to declare chicken as var chicken = "Bird" which should make it accessible from within the getChicken function but not from outside of myModule. Is there a solution that would provide both private and public members?

Christoph Hart

Hmm, do you really need this level of encapsulation? TBH I think Javascript isn't designed for this language paradigm and the solutions are kind of hacky.

For >200.000 line projects written in C++ encapsulation is a must but for scripts it seems a bit over the top.

Why do you bother about this (is there a special use case that I am not seeing that definetily needs private members?

d.healey

I can certainly move on without it, I'm just being pedantic really because I like encapsulation. My main purpose for this is for the framework I'm putting together so I can keep all my namespaces separate and safe. I think as long as I put comments to make sure the classes are used as intended everything should be fine.

Christoph Hart

This advice gives the most encapsulation for the least complicated implementation IMHO.

Possibly the best known way to achieve some form of privacy in JavaScript is to simply prefix all “private” members with “” or something similar. It’s very comfortable to use since it doesn’t add much (or any) complexity to your code, and requires no additional operations or memory at runtime. The obvious downside is that it doesn’t provide any real privacy at all, it only marks the members as “don’t touch that” for any developers working with the code. It’s what I do most of the time, though there is a minor variation that I have been trying out: Instead of prefixing the actual properties, I create a single property called “” which is an object that holds all the other “private” properties. (Let’s call that Method 1b.)

(taken from here)

d.healey

I like that idea, nice and simple. Thanks for bringing it to my attention, I shall use it!

Christoph Hart

Alright the glitch is fixed (I didn't find out the exact reason why it did that, but it should not occur again)...

d.healey

Excellent!

d.healey

I remember getting on with this before but I'm working on a little module now and the persistent storage doesn't seem to be working for me. Would you mind taking a look when you have the time? This module is an articulation switcher that seeks out containers that have a midiMuter with the same name as the container and counts them as articulations which can be muted or unmuted.

HiseSnippet 2683.3oc6astbaabEFTRH0j1HMIi+Qm9qsb5jP5KTjV4Vkpq0U6xwVwbDk8zY73IcEvRxMBDfC1ERVwidm5iPej7aP64rKtRAJAYSKmKVw1ivhy4re645GHX5E3ayDB+.iJWe+SlvLpbCy9m3IGs0HJ2yn61FU9DSISHM17jITgf4XToxhOBuWkpKYn94MOXSpK0ylktjgwy841rmvGykoq1a8GycceH0gsOebFo+506Z66skuqeHfiEMaaLgZeHcH6GnnXKXZbDmcrvnRayUtWqfC6bvOuQqje1W8ui1czeSzGWda7xdJALrCCBXdxmCpaTwrx+C9oh4NNboePeIENWvpa56bR+Q9G6o25myE7CbY3EcL5CXRu7C8ccvCO96c8jrfATaVF0L1ZD20oWr6TX.VtWpycQsy8ll6xc3Iqm5j+L0MHoZj0cWYg7PdobPtSVH2tPH2o.3YjAcKoQG.BemPWpLOxvnczM39d4PF5I7Db4IYyF9fB2O2rGWZOpX7tPA3EBTuuwaTv2xbmACX1xTvtj4C+WWsQ5qMih7qatkumDthELOpz+2ktRuxhm2ItTkierz6ikd+JtzqpoHTfkderv6sKbbIfWkylJ+4l8sC3Sjo2Aw1WDs5LPm9lFK7W9ylKeqaUibKx9boKaUxFARtcT5Oo+wX8.KfbTmVsg+6mDnjaDJG4GrJYa5QbGx+jQcYmfquMbPWkrR6ka+sKeu1c9NbMnVhOfybld8m.QdOAH9i58jiVgbWxHobhX0kW93iOt0PuvV9ACW1UKjX4gSbu6Jv9y7ZMRN1s1sVtVsAgd1JP1iEH3Bn1TB.f1nYsWWqpbDWzZB0i4RtOAGL.2sE0woGtTi5SxoR86PZC+o4ZY0qkfIaT+HcPBjX.0UvJRDA8HVWudAL3JPNYPXAh8bpaHqwqOs4ZjkWlLldHiHBCXD4HFQIzWIHGgxP3BBk3evOA050pFYFv.vwH9.23P1I2QKcyZUgCa18ZX7d07EfXuDTSIH.nSWKwdCOi8hLT.SFF3Qls8z1A9SM3Xryqnim3xHgBfoasinADAjuC+NXbO1wmIv.ZAkvBIQIpR2fm.B.hq5nfaVWGbkF06qucckC6Q.dijm3ALpEYric7T+YYoDZAo1JQGAg6j0XzLI+BvXu3knJc2VP7GjpE3enRxHHtCgpwXGfwg.eZxwi7ELR2sIbOa2PGlPEdSUCgdlMSozrPM1XYWTfTTq50nVSg5LVxAbuZzVKfMj.ApnB2zyvFttDn3mOzi4fBPDQRnzHjZaWrnp6nRgzRF87HpDfG8rtYONdgak5WyU0sk+3C72z+UYBFhzhtbl3wyT0GCfVer.U67MEo7yzGiBU+Yar0VfhqzNRQ.+87mfwZ1TPG1iQpHmJUaVQ9wwAHQsA9AjFbXiauFgS96oA1VtLugxQvp291MAO6NT6Qjc6tcWcXDaToz0FxXxmHmHcxp5BT9.Ric7FBKzZLUE+1iMj8pFIa4K3u7Nj5MpStMwF9a8l0ah1p6POenciZuUxJzGEGeuuRFmuVT5ZU01p1WLKCLO49POEetSSRxBjWepJwwwIpyEAOVY7Z95TTZP.8DzbQZ1RW2jI8O2f1bmplw6Plx.bWvdU42JtTn2t3tnUqlqtt0jPwnF1IFLUYnzEzWgRLrmUqn9Nfsxkrf4XckrwQlKIiJ0lNA9Sbfg9fpmBMOqcZtLuG2mP8bHXxYhjmMgpy89dcNDlwnJRR12njgHW2O3KUejCM3MyiGXeRABGCgsIOHtjIwX02lKnv3Nm5MIqdl6xyaxbXFOVICj88Pb7TO8nXL0ISGoVQtFQicgfLLr.wtB2giOfEznoJecCDwx.9vgr.nKjM008.p8g5h.k6wAqYvvbRohJlybYigZ+37VGcJFNAcCIXuCfqZ.c.5nNKXVDghs6xFoO3DhCa.MzUVKI0u0gBzoMKHC15g9gPfjh3VUYhppfvLwPaEFdlGdWTvSyjhLsubv.syDtWlag4gA9ttHFhc0YfX5siAJdF5P9xujTnTQC5I+C.ZHxf.bZWGbPPgwtynuxcreRrSkmjxsIdvBVn4kywGGyJW38BbtkL.WUYDcHVOD79mquIajVcvRh0Q3ozQaU3dVwb7QeBRh3.AcqxyP2ZVTzsJhit0rHoa8NyR2BnoaUyZ1D0sp8Zq2Vp5VkfqtUIIqaMWXqCVw5B4qCRfm4xvX2BoZmwnEPZO1ZWLs8Hig+05LT2sJC2cTw4D6cq4J8cqqT96VyMB74cnoT3st7b3sJOIdqbr30YC.Odq4.Qdq2Il7Vu0T40Gh4DYdq2Z175tYklOebs6UMidbOUa77jSu0UModq4Iqdq2AZ8ntmhcWglqSkHVH2dqYStWm.8tRu258B+9nSWAT70nddRxOtxnD7.SSle23Ahms2dp9Jc0vnTz+TRd5zYNEQ4W4bm9tYY8m58mu79y1c58Oy+zvXIi4Wj2trQ8pVyE9+ZqDgoxmBnyAN2LgnGD37xCZ3oBxoLLQbnYFDcqDdh1TfeSDOuUIpMO5J7v1nN5lqq3CW8f.F8v0z0EJ0x0ebUktQrHhYrd2N34aqnEOSHS2XKgcsdzSjIdITzc6NM0ac03FSIxlWTLNABOSX93H3koeThx4wZeINBU.nT6SgztwTANDB6cmevVFjlwH47ghDe3czoup38DGraZ5iwDG.lE7wit9.fGzhPdAfJh62bFVpLxy7d1pUtWjksl5VFA8855wkOcBK5EzEOD4LqLXP1kRKBytppxn.whWpvWlYDcRCiJKEINHpL5ErF8Vzvmexf6fucu7O6ogx6l88WZLsQ9rHiDyUUYmaj9MSPLsMdyCJqMRIM+1ZiOwD6XUf1e7k4+6vWl+M0P4KL6Kgp8wbugQOz9GxWoeu.lqO0oO+myH1+c8MCA2UP9U+Oqq1pMFCSiysWQGi8XSXTID8yAu816QA9gSlVq27.Utx9A.8SvSj8FO0C+RxIyZjsB7EhAvwQYKQ1a0KLXH5qxririf597qAzxfPuGznA+56UoBrjF06Rmn91BN8WwuqUtuZCf5p1WYxS1jOLiowUxWDmQzji093yHjY2N+jpJexud9FSDAuqal7oyD8ckH45hR6md.3hkCWm67mEltk8mF0xdyPoD5UgMr+TSt5g7iV5hG+TfM9ilC3upuLz9Pbxpn.ibk08ewx0B6BBg+gXL1mio063cDzx.VQ2xeaMy9jUuPPBLzgZ9zUt456AzAApXoK8WW+IvraZP9dIWpyWmxe9NuXvexTCWB1Gk7aiIwK969IwmmqCpn0eP8vtGH+Edb9FlpAhDbR+GJntmenDlfuKEdv6WAyQgQs8A5E1rrCbW.6cputMdMBn9LOm1weE6itYmjoyvM6DeSC6XSAmsneW87AUVj4o930PBltvbe2rozBfDCBIX7tg90gLFGIuD54f05SOho6vnwyLYprK2tmO3LwOzSzMr1Uyg92J6wXpcf+OZqmbhgvqoVA7jdp+e0np4t30jNwiJwjN78C7i11X2g6BocEqw8tzZrxkViu9Rqw2bo03auzZ7cWZM99yQCjK1FgR+w5YEFF+e8fXD6B

Christoph Hart

I think the problem is that you are storing the initial object in the onInit callback which gets overwritten by any value in the onControl callback of the ScriptPanel.

A better solution is to use a dedicated variable for the object and think of the panel as "restorer" (not as actual container for data). This script might explain this idea:

/** The restorer.
*
*    This panel will restore the storage object in its control callback. If the value is not an object, it won't do anything but use the given storage variable as value (see the onControl callback)
*/
const var restorer = Content.addPanel("restorer", 0, 0);
restorer.set("saveInPreset", true);
restorer.set("visible", false);

/** The data object. This will be overwritten by the onControl callback of the restorer */
reg storage = {};

/** Checks if a given variable is an object. */
inline function isObject(variable)
{
    return typeof(variable) == "object";
};

const var Knob = Content.addKnob("Knob", 0, 0);

// Make the knob non persistent for demonstrating purposes
Knob.set("saveInPreset", false);


inline function onKnobControl(component, value)
{
    // Just use the storage object (ignore the restorer)
    storage.a = value;
};

Knob.setControlCallback(onKnobControl);

function onNoteOn() {}
function onNoteOff() {}
function onController() {}
function onTimer() {}
function onControl(number, value)
{
    if(number == restorer)
    {
        // Check in the callback if the value is an object
        // If not, set the value to the storage object
        
        if(isObject(value))
        {
            // Copy the object from the restorer to the storage variable
            storage = value;
            Console.print("Restoring storage");
            
            Console.print("The restored knob value is: " + storage.a);
        }
        else    
        {    
            restorer.setValue(storage);
            Console.print("Initialising restorer");
        }
    }
}

d.healey

Thanks Christoph, I think I get it (again). This is what happens when I stop coding in HISE for a month I forget everything

d.healey

What's the advantage of using an inline function for the control callback over putting it in the usual onControl callback?

Christoph Hart

It might be a bit faster with big scripts because it doesn't need to resolve the big switch statement. Also you can assign the functions dynamically but in the end it's just a matter of taste :)

Lindon

@Christoph-Hart So if I :

this.panel.set("saveInPreset", false);

in your function - do i get to persist a piece of data but NOT store it in any preset?

d.healey

@Lindon It's been a long time since I used this since I haven't needed to persist any data that wasn't stored already in another control. What's your specific use case? it will probably turn out there is a more straightforward solution.

dxmachina

Hi @d-healey @Christoph-Hart ,

Reading through these solutions for data persistence and wanted to ask about scale. Would this still be the approach if you were storing 1,000,000 points of data? And before you ask "is it really necessary" let's assume it is. :)

Similarly, is there any solution for serialization directly to/from disk to avoid storing that quantity of data in every preset (when such data is shared among multiple presets?

Thanks!

Variable Persistence

35

1.8k

10.4k

90.9k