How to make dynamic expansion Artwork accessible for Webview?

[Straticah]

Hey there i have been building a custom Webview Preset browser for a multisampler.

Each expansion comes with an Image - i have troubles displaying it in the sandboxed webview aswell as passing image data from HISE into it.

Currently I move new expansion artworks with a script to the root folder (which will not work with a compiled VST since expansions are dynamic) AppData folders are not accessible to webview i believe.

Any tipps on how to approach this?

[Christoph Hart]

@Straticah yeah, I haven't implemented it yet, but it was pretty simple to add:

const var WebView1 = Content.getComponent("WebView1");
WebView1.set("x", 0);
WebView1.set("y", 0);
WebView1.set("width", 600);
WebView1.set("height", 600);

WebView1.setHtmlContent("
<style>
body { background: #888; }
</style>
<p>Hallo</p>
<img src=\"{EXP::noicenoicenoice}hise.png\"/>
");

Compile the latest commit, create an expansion called noicenoicenoice and slap the hise logo in there, then load this script and it should display the image. I haven't tested it in a compiled plugin but since it uses the default HISE image loading mechanism this should work without issues. Let me know if there are any hiccups / problems.

[HISEnberg]

@Straticah That's one sexy Preset Browser!

I'm no Webview expert but I think you have a few options. The Webview docs even cover this a little bit (under Resource Management).

I'm seeing three options:

1. Embed Images in the project: This happens automatically in HISE but doesn't sound like a good option since you would have to embed all of the expansions images before compiling the plugin (i.e. its static).

2. Manual distribution to AppData folder: In this case you would actually copy and distribute your webview files and expansion images to the AppData folder, neither of which would be embedded in the plugin.

3. Websocket: This is probably the best alternative but a little more complex. You can setup a connection to communicate the data. You would need to encode the images (Base64) and a websocket for it to communicate and send over. I've actually never tried this but it should theoretically work!

[HISEnberg]

@Straticah I am curious how is this preset browser holding up otherwise?

I'm considering rolling my own system to get around some limitations I am experiencing, particularly in terms of layout and user interaction. I gave it a shot with CSS a few weeks ago but I don't think the CSS integration is stable enough just yet.

Preset Browser in CSS

Full code and snippet are at the bottom. I've been abusing CSS lately and wanted to share some findings/bugs. I am really loving CSS for the transformation a...

Forum (forum.hise.audio)

[Straticah]

@HISEnberg Thank you! i am trying step 3 right now using HISEs File.loadAsBase64String() to convert but my webview has troubles to read it so i assume its different to what it is expecting. When using in web context.

Option 2 did not work (eventho i would prefer it) because Webview can not read system paths.

Will explore (docs) more it seems not as easy as i thought :) Websocket seems overkill for loading 400KB of artwork data - but might be the only valid option if no simpler approach works.

[Straticah]

@HISEnberg Interesting :)

Webview has been amazing so far for creating browser functionality using the HISE preset browser webview example. Feels like night and day compared to usual VST UIs.

Only Issue is the loading of external files so far since it can not access these for security reasons.

[d.healey]

@Straticah Are you using a custom expansion system or the HISE expansions?

[Straticah]

@d-healey I use the HISE expansion System.

Hise Side:

// Build path to Images/artwork.png in the expansion folder
var artworkFile = rootFolder.getChildFile("Images").getChildFile("artwork.png");

// Copy the artwork file to the webview folder for guaranteed access
var sourceArtwork = rootFolder.getChildFile("Images").getChildFile("artwork.png");
var webviewArtwork = webroot.getChildFile(expansionFolders[i] + "_artwork.png");

// Copy the file using the correct copy() method
Console.print("Copying artwork to webview folder...");
sourceArtwork.copy(webviewArtwork);
Console.print("Successfully copied artwork to: " + webviewArtwork.toString(0));

// Use the local copy in the webview folder
coverArtPath = expansionFolders[i] + "_artwork.png";
Console.print("Using local artwork: " + coverArtPath);

Problem was webview has no access to files outside of root - which on the other hand is essential for using external expansions.

[d.healey]

@Straticah For each expansion you can include an image called "Icon.png" in the expansion's Images folder. This should be available without having to load the expansion or doing any special handling. I don't know about accessing it in the webview though.

[Straticah]

@d-healey Yes this is the same approach i am using.

Expansions/Images/artwork.png

I have no problems displaying any artworks in HISEs preset browser - but HISE is not able to pass images into Webview. It only exposes the image path which is not enough.

System paths can not be read by webview. Because lt is not able to access any folders outside of root - for security reasons.

[d.healey]

@Straticah Are you able to use the FileSystem API to copy the images to where they are accessible?

[Straticah]

@d-healey this would be inside the VST which is not possible since its compiled.

= any other place would be outside of root.

[d.healey]

@Straticah Aha I get it. What about pulling from a server?

[Straticah]

@d-healey I guess that is what i am going for if local url with a Base64 string fails.
I just thought there has to be an easier solution if i can send so much stuff via API already.

Also i have no test so far if web based hosting in VST plugins is cached reliably, but will explore :)

[Oli Ullmann]

@Straticah
I can't help you with your problem but your browser looks great!

[Straticah]

@Oli-Ullmann

[HISEnberg]

@Straticah You've certainly peaked my interest with a webview based preset browser, so I'll test some things out and let you know if I find a solution to the image distribution issue.

[Straticah]

@HISEnberg currently i load via url which i try to integrate into expansion_info.xml without breaking it and bypassing any caching errors (that i currently experience in build).

https://prototype-audio-public.s3.eu-west-1.amazonaws.com/Expansions/Immersive/artwork.png

But @oskarsh suggested shipping base64 encoded artworks in expansion instead.

[Christoph Hart]

@Straticah there's actually a system built into the webview wrapper that allows you to create virtual assets from the embedded images in a plugin - this is a very common use case so you shouldn't have to hack around loading images from a server every time.

It's been a while that I've implemented it so I have to look into it again, but it should be possible to reference image data using the normal HISE expansion wildcards in your HTML:

<img src="{EXP::MyExpansion}thumbnail.png">

I haven't tested this with expansions though, but I can take a look how to make it work.

[Straticah]

@Christoph-Hart oh this would be very helpful?! :)

[Straticah]

@Christoph-Hart I was not able to get it to work with my Expansions so far. If you have an example or rough script this would be very helpful :)