HISE Transformation to the new age

[Christoph Hart]

Let's discuss potential changes to the HISE framework here.

Read this post for an introduction of my perspective on AI at the moment:

https://forum.hise.audio/topic/14390/ai-roadmap

[David Healey]

@Christoph-Hart

You mentioned removing the script editor, how would we view the our scripts, would we be expected to use an external editor?

[Oli Ullmann]

@David-Healey
@Christoph-Hart
I was just about to ask that. Wouldn't it make sense to keep the code editor so that changes can be made to the AI-generated code?

[Christoph Hart]

The script editor won‘t go away (maybe that was formulated a bit too drastic), but the focus of where you write the code will shift most likely to external editors as they need to operate on the same source of truth as the AI agent.

That was just an example though and this entire discussion is a moving target, once we lock in on a tight workflow things become more clear. Also I don‘t plan to rip everything out of HISE that works, it is just my current gut feeling that editing scripts inside of HISE will become a thing of the past soon.

[David Healey]

Another concern is a lot of newcomers to HISE already find it difficult to get started. Adding on to this that they need to understand and trust AI agents and subscribe to something like Claude risks making HISE seem even more complicated.

Of course for experienced coders and those already using AI tools it will make it more appealing - but the people I see wanting to start using HISE are mostly musicians not programmers.

[Oli Ullmann]

@Christoph-Hart
Wouldn't AI integration, as is the case with Visual Studio Code, be ideal?

https://youtu.be/PyUuLYJLhUA?si=av2DZ7sQdqIDya0y

[Christoph Hart]

@David-Healey luckily we can leverage that an AI agent and HISE newcomers have kind of the same knowledge point so improving the robustness of the LLM output will also remove friction points for human developers.

Just one example: I was testing some agent guidelines with a very small model to ensure that it gets basic tasks right. It always got local vs. var wrong so instead of me updating the guidelines to scream at the LLM to do this correctly I just realized that there is no scenario where you actuallyhave to choose between local and var. it‘s basically one or the other, so I just removed that entirely so if you define a var statement in an inline function the parser just silently converts it to local.

That increased the chances of a small LLM oneshotting prompts to about 80% and will also remove the friction point for new developers.

[HISEnberg]

@Christoph-Hart Yup I just wrote what a pain it is to always correct the variable type. There's small behaviours I notice they do which I suppose we can start to keep a list of, and you can adopt the parser to correct for the obvious? One thing I notice is they always mess up the x/y order for something like this:

g.drawLine(0, 100, 50, 100, 1.2); // x1,x2,y1,y2,linewidth

I believe in Javascript (could be wrong) it is usually x1, y1, x2, y2? I notice an AI will always make this mistake...

[David Healey]

@Christoph-Hart said in HISE Transformation to the new age:

I just removed that entirely so if you define a var statement in an inline function the parser just silently converts it to local.

So in the final script will the var appear as a local or is it completely hidden from view?

[Christoph Hart]

Hidden from view and resolved internally in the parser:

inline function dudel()
{
    // valid code now:
    var iAmSoSorryDave = "Why did I need to explain this to users for 5 years"?;
}

local alsoValid = "allTheTimeSpentExplainingThis...";

There's literally no field in the 2x2 matrix of valid input vs. scenarios that has more than one valid syntax so all this did was to make it complicated for both LLM and new user.

Type	Inline / callback	function / root
local	Works	compile error
var	compile error	Works

[David Healey]

@Christoph-Hart said in HISE Transformation to the new age:

var iAmSoSorryDave = "Why did I need to explain this to users for 5 years"?;

Hmm I'm not sure how I feel about it - if we answer that question there is a real answer - it's important to educate newbies about the concept of scope. Experienced devs already know better, so this is really just of benefit to newbies who will remain ignorant and AI agents. Do we really want more ignorant scripters... maybe improving the error message to be more educational would have been another route, no going back now

[Christoph Hart]

local is still here to stay, obviously otherwise every single script in HISE would break, so you can still emphasise this distinction if you think it adds value to the semantics of a script when teaching human noobs. The LLMs don't need that guidance though :)

[dannytaurus]

@David-Healey said in HISE Transformation to the new age:

Do we really want more ignorant scripters

Like it or not, this is the way everything is going. At least, what I believe you mean by 'ignorant'. I feel the same.

It's a hard pill to swallow that all the years we've spent learning to code amounts to little when AI can produce excellent results from fairly simple prompts.

[David Healey]

@dannytaurus said in HISE Transformation to the new age:

At least, what I believe you mean by 'ignorant'.

I mean users who don't understand the code they are "writing" because the inner workings are being hidden from them.

@dannytaurus said in HISE Transformation to the new age:

AI can produce excellent results from fairly simple prompts.

But an ignorant scripted doesn't know what excellent results are. I think the focus here is too much on people who know what they're doing. I see a lot more threads popping up asking me to fix AI slop :p

But I'm happy to be wrong and see HISE become easier for people to use.

[dannytaurus]

@David-Healey said in HISE Transformation to the new age:

I mean users who don't understand the code they are "writing" because the inner workings are being hidden from them.

Yes, but it's all just abstractions all the way down to the metal. JUCE is an abstraction of pure C++ APIs. HISE is an abstraction of JUCE. And AI agents coding will be an abstraction of HISE.

One could argue that HISE scripters are ignorant of the inner C++ workings of the plugin they're building.

I see a lot more threads popping up asking me to fix AI slop

This is very much the start. Give it time.

[David Healey]

@dannytaurus said in HISE Transformation to the new age:

This is very much the start. Give it time.

This is what people have been telling me for years :)

[dannytaurus]

@David-Healey said in HISE Transformation to the new age:

This is what people have been telling me for years :)

Nah, this is the start. The last few years were just getting LLM to get the language working.

[David Healey]

@dannytaurus said in HISE Transformation to the new age:

Nah, this is the start.

Tell me that again in 5 years

I'm not quite as cynical as I seem, I use AI all the time, I'm just cautious.

I also am concerned about AI inbreeding which is a real problem with limited solutions at the moment.

[dannytaurus]

@David-Healey said in HISE Transformation to the new age:

Tell me that again in 5 years

I'm marking my calendar

[Christoph Hart]

@David-Healey I don't want to be condescending, but have you tried using an AI agent with the latest models? Not like the local LLMs that you can run on your GPU and not one-shotting generic prompts into a ChatGPT window? You literally sound like me 2 weeks ago and I would have agreed 100% with you then and 0% with you now :)

I see a lot more threads popping up asking me to fix AI slop

Look it at this way: you're just one part of a very slow and ineffective Ralph loop.