AI Roadmap

[Christoph Hart]

Hi everybody,

this might be my longest post in the entire history of this forum, but drastic times require drastic measures. So...

The last two weeks I've been reevaluating the newest progress in AI technology and testing the "new" (lol, welcome to the future) workflow of agentic coding. I was burned in the past with getting duped by AI hype stuff where it failed even the slightest complexity increase from a hello world toy request, so I thought this would be just another case of AI bros trying to collect the next round of venture capital to keep the servers running.

Well, here that's not the case (happy to explain why if anyone's interested) and the implication this will have on software development in general and the future of HISE in particular are completely transformative. So first of all I want to apologize to all the people here in the forum (notably @dannytaurus and @clevername27) which tried to point me to agentic coding which I've dismissed as AI slop (again: burned before).

Now I've spend the last week evaluating this in different use cases:

• scripting in HISE
• codereviewing complex HISE projects
• web design vibecoding funk
• deep architecture redesign in C++
• feature changes in C++
• bug fixing & debugging in C++
• new feature development in C++
• documentation building

to get a in-depth look into what the current state of the art models excel in and where they still need human guidance and knowledge. So the good news first: I'm pretty confident that we will still need developers for the foreseeable future that guide the AI to implement projects (so your job is safe), but the way that we will achieve this goal of "writing" software will never be the same again (so mine is not). If I could press a button and make the entire AI thing and its implications it has on society, politics and ecology disappear, I would not skip a heartbeat and smash that button like the subscribe button to David Healeys Youtube channel, but it's out in the world and it will stay, so it's adapt or die. One probable outcome of this is that the black magic art of plugin development will be democratized to a point that will completely devalue mediocre work in that field, but this is nothing that we can impact in any way.

So the fact that the way we write software going forward has changed forever, it obviously has severe implication on the entire relevance of the HISE framework. From its days of just David and me in this forum until today, HISE acted as a bridge to enable people to implement their audio plugin projects without knowing C++. That nice USP is gone out of the window and it's inevitable that there will be alternative tools that achieve that same goal. While the current projects that are popping up which lets you vibecode a plugin (see the iPlug3 thread) are far from being a full competitor to what a experienced HISE developer can create with the HISE framework, it would be a super ignorant thing to just ignore all that and keep HISE on the "pre-agentic AI" course where we explain new developers what the diffference between local and var is.

I'm currently reviewing the entire feature scope of HISE and its user flow / benefits that it provides to developers in order to evaluate which parts will be completely irrelevant going forward and which parts will bring additional benefits to the Agentic AI workflow that might ensure the relevance of the framework in this new era. My current assessment (which might change over the next days / weeks) is that having a mature codebase (and community!) like HISE that gives you the guardrails of a structured project layout, high-level modules and a tight feedback loop for the agent to iterate fast might be enough to justify its existence and make it a viable option as a framework in the next years to come. Obviously I'm totally biased at the moment, because the alternative is just rolling over and give up.

In this phase of my assessment I'll try to be as open as possible and also would like to start a discussion about this entire topic as I'm really curious about your feedback on this matter. So currently Dominik and I are working on these things in parallel (because they heavily influence each other):

Documentation Redesign

We'll be doing a full redesign of the docs. My initial experiences of letting the latest coding models explore the HISE codebase and come up with factually correct and comprehensive documentation just by looking at the source code is one of the most mindblowing experiences I had so far - if you have a few tokens left on your Claude account let it explore the HISE codebase and explain everything related to eg. ExternalData to you. This is an easy win where everybody will benefit from even if we wake up in three months and all that AI thing is a fever dream of the past because the VC capital dried up and nobody will pay 9000USD/month for a Claude Max subscription to cover the actual costs. We'll structure the documentation redesign with both AI agents and human readers in mind, but with the realistic assumption that there will not too many people reading docs in a few months. I think going forward having a non-complete / deprecated and clunky documentation like the HISE docs are right now is completely unacceptable - one could argue that it was unacceptable in the past too, but here at HISE we don't care about the past, we care about the future.

AI Agent Hackathon

I'm toying around with AI agents and how to integrate HISE into a agentic coding workflow. I can imagine that having a fast and tight development loop where the agent can compile scripts, evaluate the outcome and test it will bring a significant advantage over the current agentic AI workflow where you have to compile C++ and load a plugin, then test it. If this assumption is true, then HISE will have a significant head start and we all can happily ride into the sunset together. The "gold standard" for this is web development and the toolchain that a browser gives the AI: once you've watched it one-shot a next.js webserver application and debug it live, it's pretty jarring. In this scenario, HISE would be running next to the coding client but you would spend most of the time instructing the client and just interact with HISE to play around with it or occasionally inspect some script / debug it. So having a working version of your project available to test / inspect and analyse will still be valuable, but I can imagine that the actual UI of HISE (as in the window you're looking right now if you open HISE) will drastically change to accomodate to that workflow. Note that I'm a super noob when it comes to agentic coding so I'm happy to discuss your insights you gained when doing agentic coding regarding context window management, general best practices and all that. This is really a completely new skill to acquire as a software developer so let's do this together - most resources you can find are for web-developers and if I watch another video where a web developer shows me how to implement a shopping cart with a detailed todo planning list I'll puke, so being able to discuss things related to AI audio software development will be very important.

Transform the HISE codebase

There is a slight chance that the combination of my coding style which I would summarize as:

• thinking very long about architectural design of the framework to ensure its robust
• be super critical about performance relevant parts of the codebase
• then crank out undocumented code that only I understand
• half ass the UI to be a non-intuitive grey mess

is a match made in heaven with the "skills" that AI coding brings to the table:

• completing what I've started before my brain moved onto the next shiny thing. Eg. getting that CSS parser from 80% standard compliancy and bug-filled entry points (missing ID selectors, etc) to 99% compliancy should be a 1-2 day job.
• do the footwork of writing proper unit tests and code documentation, then do code reviews to spot any typos I made.
• design a UI that people can use intuitively
• fix regressions and analyse the codebase for gunk and duplicated stuff that I don't use myself daily and thus don't get the attention it deserves (eg. the entire sampler workflow)

So if the stars align, this will bring a code quality / usability boost to the HISE codebase in the next months that should give it enough headstart to be one of the front-runners in the new age.

Future-proof the framework

What parts of HISE are deprecated (spoiler alert: probably the code editor) and where to focus the attention on. Here I'm heavily relying on your feedback because this will lead to decisions that will change the way you will work with HISE so I wouldn't want to do this before discussing things thoroughly. I'm currently in the acceptance stage of grief for most of my darling features in HISE, but I'm also looking forward to remove large weird chunks of the codebase (like the entire markdown doc generator system, such an abomination of a doc system lol).

Anyways, let me know what you think the impact this will have on you as a HISE (?) developer. There will definitely lots of tribal knowledge that you've acquired over the years working with HISE which will get irrelevant pretty soon, but what would you expect from a framework like HISE in the new age? What would you say would the core-skills be that should be part of the toolkit of anyone working with HISE?

You can see that I've made a new category in this forum "AI Discussion" which this is the sticky starter thread alongside a few other posts that I tried to sort so that the different discussions don't end up in one big fat post, so feel free to start any conversation about this topic in there:

https://forum.hise.audio/topic/14391/agentic-coding-workflows
https://forum.hise.audio/topic/14392/hise-transformation-to-the-new-age

TLDR: just tried claude code, we're all fucked lol.