keypresscallback object has a "isFocusChange" property

@ustk OSC doesn't work. You need to write to a file and read it on a timer. It's dirty as hell but it works.

Named pipe or some other memory-based solution would be much better, though, and a lot faster (no filesystem access needed).

@d-healey it gets destroyed completely, so even the memory gets cleared

@Christoph-Hart said in Webview! || Whats going on here??:

I've heard multiple complaints from developers getting into trouble with multiple instances or bugs that arise when people load in different plugins that all use a webview.

Big if true, but also an issue that should then be fixed by DAW devs or JUCE devs or Webview2/Webkit devs. The potential is too insane to ignore. Though I've never seen two Webview based desktop apps interact with each other negatively, which means this could be a thing because they're within the same process/host, which might then put it into the DAW developers' court.

No doubt in some cases your users will have to be using the latest version of the DAW. That's the price. It's also a question of where HISE's Webview component is with tech, since HISE is still on JUCE 6.

Also, if your plugin has a skeumorphic interface with little or no animations or you're generally just using filmstrips, it's simpler and faster to just use HISE.

If it's not complex and you don't need performant animations, it's also faster and simpler to use HISE, and it's even possible to do a responsive design now with the changeable plugin resolution.

HISE is uniquely positioned here because of hot reloading. Actually developing a complex interface in JUCE is not super fun. Melatonin inspector helps, but man...

@Chazrox it's just a frameless browser instance that loads a html file. When you connect to a website with a normal browser app, it loads the same bundle, it's just served by a server. Here it's shipped in your dll. Put the bundle in the Images directory and point the Webview component towards it.

The idea of Webview in general is to leverage the undoubtedly most mature frontend technology on the planet - the web platform technologies - to develop an interface for your desktop app.

The Web tech is so far ahead of the native JUCE UI stuff in terms of what it can do, how well it runs and how easy it is to work with it. As a bonus, you also get skills that directly translate elsewhere outside HISE and even audio.

Once you know this, the only missing thing will be how to point CloudFlare Pages or Github Pages to your github repo (it's like two clicks) and voila, you have a static website. Then if you learn Firebase or Supabase for example, you can ship a whole SSR web app with user accounts, database etc.

@dannytaurus Don't have anything released yet but this is what you do in HISE

const var Interface = Content.addWebView("Interface", 0, 0);
Interface.set("width", 1600);
Interface.set("height", 900);

Interface.set("enableCache", true);
Interface.set("enableCache", false); // comment this out before export

Interface.set("enablePersistence", true);
Interface.reset();

Interface.set("saveInPreset", true);

// Point to index.html
var index = FileSystem.getFolder(FileSystem.AudioFiles).getParentDirectory().getChildFile("Images/Interface/build/index.html");
Interface.setIndexFile(index);

Interface.setConsumedKeyPresses("all");

//=====================================================================

Interface.bindCallback("DOMContentLoaded", function(args)
{
	// Push persisted state into the interface. Simply call this when the DOM loads in your index.html, 
});

//=====================================================================

// WebView component callback
// Only executes on init when persistent data is restored
inline function onInterfaceControl(component, value)
{
	//Init stuff, recall config.ini, push persisted state back into the interface...
};

Content.getComponent("Interface").setControlCallback(onInterfaceControl);

Your app.html while developing the interface is mostly empty anyway since you do everything in +layout.svelte and the individual components. But it does have

	<script>
		// Define a fallback for development mode
		if (typeof window.DOMContentLoaded === 'undefined') {
			window.DOMContentLoaded = function () {

			};
		}

		document.addEventListener('DOMContentLoaded', function () {
			// Call the function that will be defined in the HISE app
			DOMContentLoaded();
		});
	</script>

So you open the interface, it finishes loading and calls DOMContentLoaded, which is defined in HISE, and in HISE you simply take the value of the webview component (or wherever you're storing the state) and call a function in your interface https://docs.hise.dev/scripting/scripting-api/scriptwebview/index.html#callfunction.

In your interface, you take the object that this function receives and reset your state.

For every interaction, you're managing your Svelte state anyway. In your appState.svelte or whatever you call it, you simply define a callback which says when the state changes, it calls a function "storeToHISE" or whatever you call it, which again you define in HISE, which just receives the json and stores it in the webviewcomponent or in any other component via setValue() so that it can persist.

@Christoph-Hart I made the Webview component itself saveInPreset and am storing the whole state as an object on every UI interaction.

When I close the interface and open it, the DOM finishes loading, I call a function in HISE which then feeds this back into the interface.

The state management in the web app itself is while it's running: full reactivity, store subscriptions etc.

@Chazrox I built a full UI using Webview. This is the future IMHO.

I used Svelte 5 + SvelteKit and developed a "static Single Page App".

Svelte 5 + SvelteKit gave me a built in router and state management, no virtual DOM, tree shaking and compiling to simple html+css+java script. The Webview in general gives me screen refresh rate performance with no hiccups whatsoever, responsive design, full accessibility and accessibility checklist + warnings, performance monitor, profiling (Chrome), GPU rendering if needed.

The build folder is in the Images folder of a HISE project and the Webview component's path simply points to the html file.

The work flow is the standard webdev workflow: npm run dev to launch a server and see the changes in real-time, npm run build to compile.

Then if you one day want to move from HISE, you're not locked in. Take your webview GUI and go anywhere you please: JUCE, iPlug2, Cmajor...

@Lurch said in Neural Amp Modeler (NAM) in HISE:

EDIT: Ignore the Const Var at the end there, copied by accident, doing a few things at once.

this is funny because you could've used the edit to remove it

@JulesV i haven't gotten around to actually implementing this, but it should work, instead of just reading the file as object, i would just paste the data in there

@Chazrox Console.print(trace(yourArray)) ;

@rglides You're gonna run into agreeing issues even when running a local open source model.

It does not have the training data for HISE, for the docs, likely for the interface screenshots or videos of it etc.

Even Context7 availability is limited by the tool calling ability, and output quality quickly degrades with context size (except on the latest frontier models).

The real benchmark would be to try and use something like this https://github.com/e-p-armstrong/augmentoolkit to add training data to a good open source LLM like Devstral. Then feed it the docs, the entire forum content, transcripts of David's videos etc.

You would then try to use it with a strong system prompt that discourages glazing and agreeing.

@d-healey how so? aren't you just loading it from the computer in that case?

EDIT: Ah, the IPP license clarifies this.

@d-healey if hise resorts to using the Ipp that's gettable via nuget or any of the ones that can be be found with direct links on the Intel website, then you could add it to the installer? There would then have to be an option that it's not included in the plug in, but found in the pc.

Kind of like video games of old used to pack C++ resists and direct X versions.

@Gab you can use a Webview and get the best performance that way.

@VirtualVirgin

Right click -> Find all occurences. It will look in all included files.

If you want to search even the non-included ones, use VSCode, open the Scripts directory in a workspace and use Ctrl+Shift+F (or H for replace) to search and/or replace in all of them.

@marcrex Ship ollama or a similar layer with your plugin, and a model of your choice. For this you probably want a good tool calling model, like Devstral. You will need the super light version. Unsloth has a super small quant https://huggingface.co/unsloth/Devstral-Small-2505-GGUF/blob/main/Devstral-Small-2505-UD-IQ1_S.gguf but I have no idea how well it runs. Check Gemma 1b as well.

Your system prompt needs to contain the functions that it has available for use and what each of them does.

Define these functions in HISE.

You'll run it by mounting the model, if not mounted, on plugin start, using BackgroundTask. These small models should be fast to mount anyway, so best to have it unload itself automatically after an hour of not using it.

The user can then write an instruction and the model is fed system prompt + instruction via BackgroundTask. The system prompt details what it can do, the instruction tells it what to do. You want to explicitly tell it to return a "function call" in a clearly specified format (for example doThis(arg1, arg2)) and for that to be the only return, no other text.

You should then validate the return of the BackgroundTask call, by checking that it contains a function by using string operations, and if it validates, you would call eval(return). eval() will attempt to execute the string within the code. In this case it's a function that you already defined, so it will run.

In the end, the brunt of the work is you designing the tools (functions) that the model can call.

This is a very simple example, if you want to chain tool calls, you'll need to feed the output of the function back into the model along with the previous prompt so that it can continue doing the work if needed.

If you want an example of how to use eval(), I did a cool experiment here https://forum.hise.audio/topic/9086/hise-s-best-kept-secret-dynamic-plugins

These are the limits of my current knowledge on the subject, some of these workflows may already be outdated, but hopefully I gave you a good starting point to go and explore.

Before you download ollama and start fiddling with local inference, you can try it out instantly via OpenRouter. https://openrouter.ai/mistralai/devstral-small:free

This free endpoint will of course use any data you send for training, but you're not gonna be sending any sensitive info, just your system prompt any whatever the user types in. For a simpler implementation (though obviously not as rock-solid) you can offer the user to enter their OpenRouter API and use the free endpoint that way, or provide your own via cloud functions, at which point it becomes a service. However, this way everyone can run the proper 24B model; on macBooks with 32+ GB of RAM it runs fine, but on Windows, VRAM isn't super easy to come by.

@DanH if you need context

@DanH the same was you edited in SuspendOnSilence

@DanH it is if your network features reverb, delays etc