Full Expansions again....MASSIVE problems.....Warning swearing in the message....

[HISEnberg]

@dannytaurus

Agreed the Expansion system deserves quite a bit more swearing. I think it could use a pretty serious overhaul especially on the UX side when developing.

[dannytaurus]

@Lindon Is Engine.setCurrentExpansion useful here? What's the difference?

https://docs.hise.dev/scripting/scripting-api/engine/index.html#setcurrentexpansion

[dannytaurus]

@HISEnberg Yes, indeed. I'm seeing so many issues here lately. One of my upcoming projects will use expansions so I'm invested in helping to sort it out

[HISEnberg]

@Lindon Given that Rhapsody uses it (and I see some people posting issues with it actually, is this the same issue?), would david's system be an option for you here?

[dannytaurus]

@Lindon CLAUDE SAYS:

I traced it through the Full Instrument Expansion machinery and there's a genuine bug. Here's what's happening.

The mechanism

For "Full" expansions, restoring the base player after unload is handled by FullInstrumentExpansion::DefaultHandler (ScriptExpansion.cpp:2704). It stores a defaultPreset value tree and listens for expansion changes:

• expansion loaded (e != nullptr) → marks defaultIsLoaded = false
• expansion unloaded (e == nullptr) → reloads the stored defaultPreset

That stored default is (re)captured by setNewDefault(), which is called from the pointer overload only when transitioning from base:

// ExpansionHandler.cpp:363
void ExpansionHandler::setCurrentExpansion(Expansion* e, NotificationType notifyListeners)
  {
        if (currentExpansion != e)
        {
                if (currentExpansion == nullptr)   // <-- "we are leaving the base state, snapshot it"
                        FullInstrumentExpansion::setNewDefault(mc, mc->getMainSynthChain()-exportAsValueTree());
                ...

The intent: snapshot the base player state the first time you leave it, so unload can return there.

The bug

The name-based overload sabotages that guard:

// ExpansionHandler.cpp:340
bool ExpansionHandler::setCurrentExpansion(const String& expansionName)
  {
        if (currentExpansion != nullptr && expansionName.isEmpty()) { ... return true; }

        currentExpansion = nullptr;   // <-- BUG: blindly nulls the pointer, no notification

        for (auto e : expansionList)
                if (e->getProperty(ExpansionIds::Name) == expansionName)
                {
                        setCurrentExpansion(e);   // pointer overload now sees currentExpansion == nullptr
                        return true;
                }
        ...

Trace the user's exact sequence (load exp1 → load exp2 directly → unload):

1. Load exp1: pointer overload sees currentExpansion == nullptr, snapshots the base as default. Correct. exp1 loads.
2. Load exp2 directly: name overload does currentExpansion = nullptr; without notifying, so the chain is still exp1's content. It then calls the
   pointer overload, which now sees nullptr and calls setNewDefault(...) snapshotting the current chain = exp1. The saved "base default" is now silently overwritten with exp1's preset. exp2 loads.
3. Unload exp2 (setCurrentExpansion("")): DefaultHandler restores defaultPreset, which is now exp1's tree — but exp1 is no longer the active expansion, so its sample/resource references resolve against the wrong (or no) expansion. Result: the corrupted "nothing loaded, nothing working" UI.

That's why load-then-load looks fine — the corruption is silent and only bites at unload time. Lindon's instinct ("it unloads to the previously loaded expansion rather than the base") is essentially right.

Immediate workaround (no core change, script-only)

Always unload to base before switching expansions, instead of switching directly:

eh.setCurrentExpansion("");      // restores the real base default, defaultIsLoaded = true
eh.setCurrentExpansion("exp2");  // now snapshots base (correct) before loading exp2

Because step 1 here restores the chain to the genuine base before the next load, setNewDefault captures the correct default and unload works. The bug only triggers on a direct expansion→expansion switch.

The proper fix

Delete line 349 (currentExpansion = nullptr;) in the name overload and let the pointer overload manage the transition. With it gone, an exp1→exp2 switch enters the pointer overload with currentExpansion == exp1, so the == nullptr guard correctly skips setNewDefault and the base default is preserved.

Bonus: reloading the same expansion by name becomes a proper no-op (the currentExpansion != e guard), instead of re-snapshotting.

The only behavior change: passing an unknown name would no longer implicitly unload the current expansion (it'd leave it loaded and return false), which is arguably more correct — but worth noting since the empty-string unload path is handled separately above it anyway.

Does that align with what you're seeing?

[Lindon]

@dannytaurus said in Full Expansions again....MASSIVE problems.....Warning swearing in the message....:

@Lindon So you're manually unloading each expansion before loading the next one? Or are you relying on a HISE mechanism to unload/load?

I've tried both ways ...neither work. I suspect calling eh.setExapnsion("") from within the expansion itself(whihc is what i would need to do) is def. a no go... and myexperience is this is the case...it no fixy the problemy...

[Lindon]

@dannytaurus said in Full Expansions again....MASSIVE problems.....Warning swearing in the message....:

Load exp1: pointer overload sees currentExpansion == nullptr, snapshots the base as default. Correct. exp1 loads.
Load exp2 directly: name overload does currentExpansion = nullptr; without notifying, so the chain is still exp1's content. It then calls the
pointer overload, which now sees nullptr and calls setNewDefault(...) snapshotting the current chain = exp1. The saved "base default" is now silently overwritten with exp1's preset. exp2 loads.
Unload exp2 (setCurrentExpansion("")): DefaultHandler restores defaultPreset, which is now exp1's tree — but exp1 is no longer the active expansion, so its sample/resource references resolve against the wrong (or no) expansion. Result: the corrupted "nothing loaded, nothing working" UI.

Almost exactly this yes.....

[dannytaurus]

@Lindon Maybe you need the real bug fix in HISE then, rather than the manual unload/load steps in your scripts.

[Lindon]

@HISEnberg said in Full Expansions again....MASSIVE problems.....Warning swearing in the message....:

@Lindon Given that Rhapsody uses it (and I see some people posting issues with it actually, is this the same issue?), would david's system be an option for you here?

I think Rhapsody only lets you load an expansion and then return to the player to load the next one, so the stack doesnt get corrupted

[dannytaurus]

@Lindon From David's recent discussion about an unload/back button, I think you're right.

However, my expansion-based project will need to work like yours - freely swapping expansions from a visible grid/list of available ones.

[Lindon]

@dannytaurus said in Full Expansions again....MASSIVE problems.....Warning swearing in the message....:

@Lindon Is Engine.setCurrentExpansion useful here? What's the difference?

https://docs.hise.dev/scripting/scripting-api/engine/index.html#setcurrentexpansion

I have no idea what the difference is between Engine and ExpansionHandler versions of this call...

[dannytaurus]

@Lindon I asked Claude, it says they both eventually call the same function but the Engine version is just a convenience where you don't have to created an ExpansionHandler first to load. You can just load with a String.

[Lindon]

@dannytaurus said in Full Expansions again....MASSIVE problems.....Warning swearing in the message....:

@Lindon From David's recent discussion about an unload/back button, I think you're right.

However, my expansion-based project will need to work like yours - freely swapping expansions from a visible grid/list of available ones.

So working on a work around ---- where I use a file to Request the load of an expansion:

The player checks the request file every time it loads....if the request == NONE it just sets itself up as nomal... but if it == anything else - it loads the requested expansion...

So the expansions themselves now no longer load other expansions, they just make a request and unload themselves...back to the Player who handles the load of the new expansion...

its not great, in fact its a cludge, and Im about 20% of the way thru building it...

[dannytaurus]

@Lindon said:

So the expansions themselves now no longer load other expansions,
its not great, in fact its a cludge,

I would say this is a good thing. Feels to me like the player should always handle expansion loading and unloading, not the expansions themselves.

[Lindon]

@dannytaurus well good or not..it works....

[David Healey]

@Lindon said in Full Expansions again....MASSIVE problems.....Warning swearing in the message....:

I think Rhapsody only lets you load an expansion and then return to the player to load the next one, so the stack doesnt get corrupted

Correct. I believe this is how Christoph intended it to be used originally - I think he posted an example somewhere years ago that I was building off.

[Lindon]

@David-Healey yeah I'm sure that was the original intent, but if we all could be bothered to look back at my previous post about how I really didn't like expansions taking the full interface he pretty explicitly say it should be easy to build a "loader" widget and copy it into all the expansions, so I think I'm on safe ground here assuming that the loader should work as outlined above...

[David Healey]

@Lindon

I've made a minimal project which confirms the issue you've described.

FullExpansionsTest.zip

This might be a clue

I'm seeing if I can find a solution.

[David Healey]

@Lindon Here's the fix, just two lines need changing:

https://github.com/christophhart/HISE/pull/983

It is related the default state being saved when loading the first expansion.