Multiple instruments, single project or multiple projects?

Christoph Hart

It works well for decaying samples (with ~60% size which is equal to FLAC but the decompression is way faster because the algorithm is much simpler).

For sustain samples it's not that good though (80-90%) but maybe I can improve this.

For general audio files it's horrible - when compressing a mastered music track you still end up with 99% file size.

I'll devote a new blog entry to this once it's done where I'll explai how it works.

Christoph Hart

I am using three compression algorithms, each with different decoding / compression ratios:

the "Block" algorithm is a simple bit depth compressor without any processing
the "Delta" algorithm detects a cycle in the waveform and stores delta values until the cycle is invalidated
the "Diff" algorithm uses 4x downsampling + bit depth reduction for the error signal (basically you take each 4th sample, draw a line between them and store the deviation of the second and third sample).

I made some more tests & first decoding benchmarks:

=====================================================
FLAC ratio:	33.1% file size
Block ratio:	74.2% file size
Delta ratio:	66.8% file size
Diff ratio:	57.3% file size
=====================================================
PCM speed:	11842.3x realtime
FLAC speed:	1705.3 x realtime
Block speed:	10086.3 x realtime
Delta speed:	8175.8 x realtime
Diff speed:     tba.

The performance limit is uncompressed PCM decoding, which works at about 12000x realtime speed. While offering a far better compression ratio, FLAC's decompression speed is far behind the others (14% of PCM speed). The Block algorithm comes pretty close (and in a streaming context it might surpass the uncompressed PCM performance because it has to load fewer bytes from the disk). I didn't implement the Diff decompressor yet, but I expect it to yield something like 6000 - 7000x realtime speed, which will be the bang for the buck algorithm.

Another speed advantage will come from using memory mapped files for this (the FLAC reader in JUCE doesn't support memory mapping).

This format will also be used for the preload buffers so it'll drop the RAM usage significantly (currently the preload buffers are stored as 32bit floats!)

d.healey

This looks awesome! well done. The drop in RAM usage will be very welcome but what is the effect on CPU?

Christoph Hart

Well if you can decompress 8000 voices in realtime the CPU usage for each voice is 0,0125%

d.healey

Is the compression tool part of HISE or is it a separate app?

Christoph Hart

It will be of course embedded into HISE but I am using the seperate tool for testing & tuning the algorithms until it gets to a stable state.

Christoph Hart

Alright, the crashs should be fixed (it was the same reason that caused the REAPER crash so it's two for one :)

d.healey

Brilliant, I'll give it a try later today

d.healey

Good news! The crash is fixed, not so good news, found a new bug :)

When using the last snippet I posted:

HiseSnippet 1184.3ocsV8+SabCE2GvM0KaTsNs+.73mBprzj1xV0XnUR.1xJDh5QQUpppy4NGN2dwNxmu.YH9mX+kt+C1d19tbWBWYDjVDBc980O989X6WeoHfljHjHmu7zoioHmux0eJWE0Ihv3nt6ibdn6wjDEUhshZOcLIIgFhbbV8W0Bb7VCY982+RaRLgGPKDgPmIXAziXiXpBo+wKeEKN9PRH8T1nRV+7W1MPv6HhEo.dV0sIZLI3Sjyo8HZyVwEMgQuHA4zz8YOsg7SsF7m603y9axYGz3iu3T8mnfTojxUmAtibbc9G3mi6AgLkP5qHJJDy0ZKBm5GItfaS8YrD1fXpdQKjOfIq3CEwg5Mu9aTmHVbX+7RXBBhR+hB5p1B525dLKjMSdQg8qMJvEdTtz5rxsAuV2S34TBdqYg2ib8CjrwpBMZr8MYR+LnypDsxe459jmzsWmidy9G3Wql96tmVqifqfpciDp52nryiT02t4l6nUi6Hov9ASvIvdC5r3wDNMdKbDKjhYpsvDdHNgLQu.y3XUDXhjBQpFPMRT3IDYtu80th2EmmNRXnQT8MJavFagaB+A4urTM1puwDaQDLYHINgVoMZvzk22fAvPkLkt3d47Xw.RLVL3iz.E.YhBeAPvwQPyvrClPhSoI3gBoYo.9GrKL0vjF0xbOMgJsoYehh.6qqt1lGepx3Vlc4UNRBjkvyoJa3mEcnTFR41BaCs+cGMVHUDt5m.8rDK13BENHVvoVDYf9V3AoJbRDQp2UR5PJbnIf9c2npblNg0mGv5hRQKZfhejfDNe2ocpRI302vnYVEs01P+oktAMm69ZRPktazLy8bmK4MOcjg5atHKABh4iFPop2bZpOWNClyk28dHj5RZcFrp4NXF9mWHvfrG+3MqcUMuRt9N16KmvByaOsKOjdYcFjzq080CeSuNm18jdlyMc16niZuWmW4WaXJOPwDbrf2SnnmvqaRA3yhZFNrJU5BlTDGSkUoUegq7VbqNrEGPkaY4TF6XCwYRw6t6bm81rlGn2CnXulNNlDPKSSyNMbASEYoX.UanTLxrPScxOW64cCluI463UyCJTKj+LhUVpuSMHOioKzjZ.4C1JTy8qKPkA0PSxCxtmmMyEjdMUqZXook2CXciTCts.ToWpO.aQ5+EvtFt6e9WNpc2d4HvdPqjgBdWNScxX5r0VBYtjlnYDwxhJHfkkZHdUXVgna9bEJ6vO7v0pYlCeoLOasd9yVZhHhEpmSoL2DY3PHmG3969mz6pqsAqbDdXVDrWqXBw5tktZJOBdeQ9rL2sPT55oJBwMe.FFCPDlFSTyOafd1pLEvIz4d8sqtpjvTSKO60RLvPyJGXnUUcf6HbejaelJHpZ7tRE3EZo+ei2rwuV28fgCgahJ.6ZtG9166rVKATdsHUw3meLQIYWBScBu83CC1FPAjvAFZhdRzUzrJ65l505JiOkG1LeL0LkszqcxT1JWIZDIPJ9PfkVpGv6AFI.l3lQl8fY2g03V4LQMtGASc9gf.cg36AjWsGOco83YKsGOeo8X6k1ieXo83GWZOdws3gdD+8RUhQ1iEHz+B3sqkq.

If I load this from clipboard and then add a container to the Master container, and then try and add a sampler (or any other sub-module) to the new container it doesn't do anything. If I then delete the container and click the save button on my script I get an error that the container I deleted can't be found.

Christoph Hart

Yes this is because scripts get recompiled if you add a sound generator, which interferes with the possibility of adding sound generators via scripting.

This is a rather annoying design flaw and I have to think about a clever solution (perhaps just removing the possibility to restore containers might be enough).

d.healey

Would that affect the contents of containers too?

Christoph Hart

If you restore a container, it will delete every sound generator in it and add the new ones which triggers a global script recompile (I am not 100% sure why it does that, but I think I added this for a good reason).

d.healey

Ah I see, well taking it out shouldn't really be an issue then because it's really the contents of the container that needs restoring, although I can think of a few edge cases ;)

d.healey

I just noticed the "save current state as new user preset" option, when did this appear? Does it also save the state of samplers and sample maps?

Christoph Hart

It simply stores every control on the main interface (so basically it's the same as using the preset browser to save a new preset). So you need a control that restores the samplers and sample maps

d.healey

Aha ok, same as before then :)

d.healey

I'm working with the storeAllControlsAsPreset and restoreAllControlsFromPreset functions but I'm getting a crash as soon as I try to restore. Any ideas?

/**
 * Title: Preset Hander v1.0.0.js
 * Author: David Healey
 * Date: 02/07/2017
 * Modified: 03/07/2017
 * License: GPLv3 - https://www.gnu.org/licenses/gpl-3.0.en.html
*/

//INIT
const var save = Content.addButton("Save", 0, 0);
const var load = Content.addButton("Load", 150, 0);

//FUNCTIONS
function savePreset()
{
	Content.storeAllControlsAsPreset("test");
}

function loadPreset()
{
	Content.restoreAllControlsFromPreset("test");
}

//CALLBACKS
function onNoteOn()
{
}

function onNoteOff()
{
}

function onController()
{
}

function onTimer()
{
}

function onControl(number, value)
{
	switch(number)
	{
		case save:
			number.setValue(0);
			savePreset();
		break;

		case load:
			number.setValue(0);
			loadPreset();
		break;
	}
}

Christoph Hart

You are storing the widget that calls the restore function in the preset. This causes an endless loop because it will load the preset on the onControl callback, fire the onControl callback, load the preset on the onControl callback, fire the onControl callback, load the preset on the onControl callback, fire the onControl callback, load the preset on the onControl callback, fire the onControl callback, load the preset on the onControl callback, fire the onControl callback, load the preset on the onControl callback, fire the onControl callback, load the preset on the onControl callback, fire the onControl callback, load the preset on the onControl callback :)
This can be solved by setting saveInPreset to false for both buttons. (I thought about adding a safety measure that prevents this loop, but there was no easy solution and this is the kind of error that you'll only do once.

Also you need to give it a file extension (the HISE default extension for user presets is .preset) - and the number widget is not defined, which causes a script error.

d.healey

Thanks Christoph, that makes total sense - you are right, I will not make that mistake twice :p

d.healey

After I saved a user preset with Content.storeAllControlsAsPreset("Bank 1/Category 1/test.preset"); I then click to load it from the preset browser and I get the prompt "This User Preset Was Built With a Previous Version. Do you want to update it?" - am I missing a setting somewhere?

Also is there a way to get the ID of the current script (self)? I want to save the state of all scripts but I can't reference the script I'm in, my current solution is to ignore midi processors with a particular ID but I have to hard code that ID to be the same as the MIDI processor and I would like it to be more generic if it could just know its own ID.

Multiple instruments, single project or multiple projects?

12

1.9k

10.7k

93.0k