Dynamically load instruments over the Internet?

[bthj]

Hi

Is it possible to build an app / plugin with HISE where the samples are loaded dynamically over the Internet from a server?

• that is, could in interface be developed with HISE where the user can browse lists of different instruments, where the lists themselves are dynamically loaded and when selected, the instrument sample file content would be downloaded (and preferably cached) along with other metadata content, such as Sample Maps, ADSR config, etc.?

I'm just starting to look into HISE and before diving further in, I'd like to make sure it's the right tool for such development. The Server documentation seems to indicate this should be possible, but I still lack an overall picture of HISE's capabilities and whether it is intended for such development?

[d.healey]

@bthj Yes, you can download files with HISE.

[bthj]

Thanks @d-healey .

Is it possible to perform (web) authentication first? - like popping up a login window (web form) for obtaining credentials (like a JWT) for use in subsequent download calls? (I do see the possibility of setting a http header, just wondering if it is possible to obtain the data from an external login flow)

[d.healey]

@bthj Yes. You might find these threads interesting - https://forum.hise.audio/topic/4967/woocommerce-linked-licensing-system-for-your-plugins, https://forum.hise.audio/topic/3850/get-data-from-woocommerce-via-server-api

[bthj]

Thanks again, @d-healey - will be diving in :)

[d.healey]

@bthj I have some videos on Patreon about downloading files which you might find useful, nothing about the JWT stuff though.

[bthj]

@d-healey Great! I'll look into those as well.

[bthj]

Now I have written some code towards this functionality - a milestone for me, as this is the first thing I've actually done with HISE.  Really enjoyable getting to know this project and community.

What I've done so far is:

• hardcode a list of sample map IDs, to be dynamically fetched via some API endpoint
• when not available locally, download the corresponding samples monolith HLAC to the Samples folder
• then fetch the sample map Base64 encoded and load it into the Sampler

The task in the last bullet is done every time a sample map is selected (for now via combobox), which wastes a bit of network resources (and I have to choose Tools -> Clear all Samplemaps, so they won't be saved in the archive/xml).  It would be good to save the sample map in the filesystem, like the HLAC monoliths, and maybe just download it as xml rather than base64 encoded, but there doesn't seem to be a Special Location for the SampleMaps folder.  Is there some way to obtain a filesystem reference to that folder, as a target for Server.downloadFile?

Here's the code I've written so far:

// sampler
const var Sampler = Synth.getChildSynth("Sampler").asSampler();

// sample maps array
const var sampleMaps = [
  "fg_01FFRFAGFF9M9R8D6AWHTX7XB9",
  "fg_01FNAAJA8CXA8CNAXAGVWJN0HA"
];  // TODO: fetch samples list from some API endpoint, instead of this hardcode

// Internet communication
Server.setBaseURL("https://storage.googleapis.com");

// combo box
const var cmbSampleMap = Content.getComponent("cmbSampleMap");
cmbSampleMap.set("items", sampleMaps.join("\n"));


// sample map selection

// - sample map downloaded as base64 and loaded into a Sampler
inline function fetchAndLoadSampleMap( sampleMapId ) {
	local sampleMapBase64FileName = sampleMapId + ".txt";
	Server.callWithGET( 
		"/synth_is_samples/map/" + sampleMapBase64FileName, 
		{}, 
		function(status, sampleMapBase64) {
			Console.print("sample response: " + sampleMapBase64);
			Sampler.loadSampleMapFromBase64( sampleMapBase64 );
		}
	);
}

// - combo box callback with conditional  
// samples monolith download and  storeage in filesystem
const var samplesFolder = FileSystem.getFolder(FileSystem.Samples);
inline function oncmbSampleMapControl(component, value)
{
	var sampleMapId = sampleMaps[value-1];
	
	// ensure samples monolith is available
	local samplesFileName = sampleMapId + ".ch1";
	local samplesFile = samplesFolder.getChildFile( samplesFileName );
	Console.print("samplesFile: " + samplesFile.isFile());
	if( ! samplesFile.isFile() ) { // we need to download the file
		Server.downloadFile(
			"/synth_is_samples/hlac/" + samplesFileName, 
			{}, 
			samplesFile, 
			function() {
		    	var message = "";
		    	message += Engine.doubleToString(
		    		this.data.numDownloaded / 1024.0 / 1024.0, 1
		    	);
				message += "MB / " + Engine.doubleToString(
					this.data.numTotal / 1024.0 / 1024.0, 1
				) + "MB";
		    
		    Console.print(message);
			
			if( this.data.finished && this.data.success ) {
				Console.print( "Samples loaded" );
				fetchAndLoadSampleMap( sampleMapId );
			}
		});
	} else { // samples monolith is present, so we can go ahead and load the sample map
		fetchAndLoadSampleMap( sampleMapId );
	}
};

• and a snippet of the project:

HiseSnippet 2568.3oc4YrsaabbcojWCSZ2Tmf7PeoESYKBnfk4MKqKV0wh5BsUrtAQ5KAoFBi1cH4Ts6NK1YVIwFHfh9Wj25mP+AJPds+M4On8blcWtConTUDRBRPIfL4bl4Lm62FePjvgIkhHqBE6NLjYU3A1cFFnFrw.JOvZ6MsJ76rOidJ6DtmK6HI0OzicjOM7HOA0kGz2Z8ggToj4ZUnvruDwoPw6Xo+7cuXcpGMvgkCxx5sBtCaGtOWkC8f0dM2yqM0k0k6ab5EVaaGQvFBOQLveyZW2Jj5bBsOaOJdrYrsJb2sb4JQTGEUwjVEty5B2gcFHNKH47ukK4G6wvEMr5.WTB31BPXbSfZsw.PzNHSOHsrJXePtVY1DsxmZuK2kOBdt14g5MH4XXpOJLy3r2riwdMLYu5Fr2TXoBFrzcRXoO1tiSDOTkuCxO22d6.EKpGET6lrRxYsl4ece6MDvIBTU8omvZGAKFgQkEqWedRy50ma0RkpUijXtiJAVAohbJMhzIAB44DsWR09LklY0qpTNc6xyUkJS+ckz6p69at+yHC.CHQIfuOkQnADd.WwoddCINdL39SnHAbv.bRugp5sRVrKMrxXLGdTIgFEQGZvlxrCKANEOb1UA76nKZGtTgW1WUhPJ2q+Q0azt8gsa8x1sWY2UNb4MWr06dU22uz6Wekxymej8Z05KZs7FuG9auVuu0Ke669h8p+pVkK8gUIjQxYOlxYPJaHId.kH8hD9DovmQZcv1DVfanfGnlGTAREi5RD8HpAbInZhbcDtLsPpMMALEwQ36GGvcnJtHnTGVzofzHYp0oR1aNbmJkGnTgxmUqlD72fPjp8Eh9fdKjKqB3VNUoA+7XA4Xw4FJKG+iGoS.0Ul6AZZE9gh.XQkxlGBuLy0HeToLWw7kkm2P2W8u.RXkx+4fxycIaFQx7XNZggG3wCXjdwA50I5tVAt6.YXxs54261tj4HecohdBGpWNXTSr3Bs4d5zCffXhviHkqpNWUd0RES0c.tduiqF7xs5VgTpXwx0jnO7QbYZVNYMfMqUFP8JHw7HZe8E5uxX9JRHTOVN+j3nY3hEAcqT3wpFFwQkZp5HhIA0rj8LxTnFn4.Dy7e8L0IPvqexgpLIVDMZWTpH78EktTjgTmADCiQgoyPvCzGM3IfqX.LgXR3ZlzLIBL8APulHgWEmLWl4Ax4EylqDH3FAfiI.Ux8U9J8oebiOfLdohiEDC1OCqY9Q0mDtaVfLNhMJZyWDH7.KKABlnmR4dTHG63tKxqwMwYPCzM4RGezQSUdix8gaV4RWLJGS0bqOhooVCnJW+UELRoHuWExuep6hd9XVlyXj.FyESk5BkSPsJj8.rNbTTy7wy1RiK5FMEm7AdTmZSvLi7tybuKZrYBfQN7Id1D3i1n4CkhfzOfxpLpES1HC3idNYqf9fWDvYwfQoqniBTM8qjcvhXFvptTEsZPr+lorOHm0HMp2bgp0G8i4IMxPJIBwjHk2cc3fnPckjq3DzpqPAl6qfLEKNG5Zr65iDozuF2.mxAI7C9GZGyoROnXmb.HLe1mY.UF6fkuIoYHlvmgjVREpgnUDkIoR6MIIo9jWf4AvedAg4IYItOSKTIDRCoiakBz8xApN2WPnCvZSz.WxHeLiZz2X9.xAoCUybZOixgJhhHBkfIhOFZK3YDRVMLWgirJngXUowtbQMot4Evxk+qGCU1RWERCXd03AtryqNP468GfxQocmlc2HsmdtWPJRhEmVGBsjooWwnxKfRX2jZiX0vzrgajR9JSMWIVSTWUbiXnnsO+uxzp2SXCOV.8.Pd7UpO3H9tw5.vZJn63Z9wRtyig55ZMPIPyVgCwf0WkvI+IRilKA+3QORmLFbJ4jOG1C8CgceNYgFyUB8lQGvz.FPFdMaXRq2U3ySpedyls0elaUs+O9OWn+WzqJEcc33UdCKsTc8GPtSv9hjFCwldbXXiNRsBHhF.gwbnBSbDIzKFtNxYf4UbFohbnuOSEMTma+LF1N0b4cWlS5cDvlHCzUrIWF5QGV4IqL2pFEu1SnX6CouPUx3tF6lDFiFY7P6E6eL1FqtRJYxanWujqXh8RMw59emx13bNQWGhUBzT0rHJdPn4+wmnv9pmnvbfGmD+ViCJB1FZ8d+PVvUMFjUpyNN8QJWAGUom83goydrA1N45hys3vPFejsoStklyMmFz5MauIjwK69fqFHWHKRwQIovlrSgQCSlson8lL4IJQHLk2n.MqB26lS8yyo7mu1PyYJ08oZU341WaK+kt1t8s7o.ArKLCbimwcUCFQf+1IqMfw6OPYReKCY3ApzAJSjg1PlJLeVWnxpVN901lfZXJHVVFBxWt13D9a+6Whve4Zo54DkZI6WmlZw539YSU+.6EZt7BKs3JOY45VnpIai6auvSatxxKr7SZX.uYBFqrvJKtTyUdpk1fNyGJXiA+kg6+cHSU9YjFXwSDVZ329NJXjujXIX6dTHqgdeHREPC.kb7WwSVAorzqSxM9xHZ3.tibLL2j0iF6oZEFBiGhuy.rqJJNYy0g1ZNAtpCwAlvau5hqn+r7x0WZk5K0Pepth9vPR6BSaM1MiizCi3G.0UPVQC6MR1agAVDQF7xHpAa11ip5nF5MwMcvVYJ8IgCgvQpbpzLC9VAtFzdwRWXoXmqLeEDqK+X.OzFDhXfEF+sIvGjIcCH8xXOH.NYYfjqFZFh9C1CVbSYwO19.Nz.wz4wYlBOBYO9wfGSelmek8V85AV4bF7N1se+ONuoiI4+sIj+SrgFTYTeH3OscDMOTzNa0O.O1V3M9w1NHhomVCZLI+Xe6ZqGCpnnwg9OWSSpV9h3fwnUJieHChQUXbV9l2asCO7kQh3vIw56dg1mnaDDAiuvnwF6Gv.UsxTF2HRHk8.wQeWRysNHNpOpqxgbHCFKRNNLHmTZ7FVOsPA.DDNilWtytTXfgwx.qoR8tTsIdF6j0MlXcyIV+jIVuvDqe5DqWbh0KkudTENzw.eAzIZH3t2rFBfK3.rsYiStNued4SMj+GOS5c+Y2yj9KfDii8RtEy3wNbTwuUvoPDG.A4wOwNs9VFzFiwk6BStENPDvcF2+F7W62mEYx6SJP+iu4a92unkRAgW4G6SW6PlGCl1IGzebscftooQiG198RWz368qZOU60uwNgcIXJKxubKnM6+2TP65TQeTZ4LhtCneFZCefsNqKAKR9SI6cnHF66OqniMTYpCTM1gYVeBG4HcccbMxDcXAt5E+G3S5lMxJlga1HaSKmrqBjmzeq6guvrr.rHituKOnLomoKpDp4irDTv4mDtzm5DINxIYpSjWumFBvkA5+q+JZuKtlzHaJSa65Uqa4C0XNxwAcxeLTxZ53z7VfyStE3rvs.mmdKvYwaANKcKvY4qEGr7dqXXTsjjd.fC1JwwpvVibrl05+hLTr5H

[d.healey]

@bthj Since the samples maps are predefined anyway why not just include them with the plugin instead of downloading them?

[bthj]

@d-healey The aim is not to have the sample maps predefined, but generated mechanically.

For a bit of a context: I'm exploring those possibilities for the synth.is project, where you can evolve sounds[1], render them as sample packs and currently download them along with programmatically generated SFZs[2]. Many things broken there and I aim to continue working on this research project in the coming years, but all are very much welcome to play with this (I know the interactive evolution part is broken, which is a regression resulting from work on quality diversity search for sounds - I welcome requests for accelerated fixes : )

Just as SFZs can currently be rendered and downloaded, it would be nice to be able to obtain those sounds directly in a plugin.

[1] https://youtu.be/dm2cgPqQVP0
[2] https://youtu.be/3eLo8qORiMQ
(YouTube links here as inlining above was causing problems)

[bthj]

The approach I have currently implemented is maybe ok - sample maps are not large so redundant downloading isn't a big problem - but if it is possible to persist them in the plugin folder structure, then it would be interesting to try that out as well.

[d.healey]

@bthj said in Dynamically load instruments over the Internet?:

The approach I have currently implemented is maybe ok - sample maps are not large so redundant downloading isn't a big problem - but if it is possible to persist them in the plugin folder structure, then it would be interesting to try that out as well.

You could write the string to a file.

[bthj]

@d-healey Wouldn't it be best placed in the SampleMaps folder? - then I'm wondering how to get a filesystem handle on that folder.

[bthj]

Tried something like:

samplesFolder.getParentDirectory().getChildFile("SampleMaps").getChildFile(sampleMapFileName);

which works, so I'll probably go that route, if it isn't some kind of an anti-pattern?

[d.healey]

@bthj said in Dynamically load instruments over the Internet?:

@d-healey Wouldn't it be best placed in the SampleMaps folder?

There is no sample maps folder once your project is compiled to a plugin, all the sample maps are embedded in the binary.

[bthj]

@d-healey Thanks for the insight. Then it makes sense that there is no Special Location FileSystem constant for that folder. I'll just download the sample maps to the Samples folder, alongside the monoliths, Base64 encoded for loading with Sampler.loadSampleMapFromBase64.

[bthj]

The above procedure of downloading the files to FileSystem.Samples works fine, until running on iOS, where the Samples folder seems to be part of the read-only app bundle. So downloading to FileSystem.Documents might be a better idea, but after loading the sample map, the Sampler seems to be fixed on searching for the files in FileSystem.Samples.

Is there some way to instruct Sampler to look for the (monolith) samples in FileSystem.Documents?

[bthj]

Made a couple of changes which solve this issue on iOS, discussed here:

Dynamically download samples on iOS · Issue #265 · christophhart/HISE

Downloading samples into the FileSystem.Samples directory doesn't work on an iOS device (though it works on a simulator), where the Samples folder seems to be part of the read-only app bundle. A couple of changes solve this particular is...

GitHub (github.com)