MIDI Processor Inconsistent/Processing Speed Issue?

aaronventure

Christoph Hart

@aaronventure Alright, it's pushed to the current working branch new_dispatcher (which should be considered stable unless you're on Linux, then it won't compile :)

https://github.com/christophhart/HISE/commit/05b9f1f474bbf7c0cf0a2850da2b44cbcaadd46e

and:

https://docs.hise.audio/scripting/scripting-api/sampler/index.html#setactivegroupforeventid

aaronventure

@Christoph-Hart oh shit look at that doc entry!

aaronventure

@Christoph-Hart I'm ready to test this, but the plugin version of HISE fails to build. Standalone builds fine.

Windows 11. All commits since this one. Reverted back to develop just to check, develop builds fine.

1>F:\HISE\HISE Source\tools\SDK\perfetto\perfetto.h(338,1): fatal  error C1189: #error:  Perfetto is exploring a switch to C++17 in v34 (Feb 2023). During this transitionary period, we are throwing an error when compiling Perfetto with a standard less than C++17. Please reach out to perfetto-dev@googlegroups.com if you have objections or thoughts on this move. To continue compiling this release of Perfetto with C++11/14, specify the define PERFETTO_ALLOW_SUB_CPP17. *Note*: this define *will* stop working in v34 (Feb 2023). (compiling source file ..\..\JuceLibraryCode\include_hi_core.cpp)
1>F:\HISE\HISE Source\tools\SDK\perfetto\perfetto.h(338,1): fatal  error C1189: #error:  Perfetto is exploring a switch to C++17 in v34 (Feb 2023). During this transitionary period, we are throwing an error when compiling Perfetto with a standard less than C++17. Please reach out to perfetto-dev@googlegroups.com if you have objections or thoughts on this move. To continue compiling this release of Perfetto with C++11/14, specify the define PERFETTO_ALLOW_SUB_CPP17. *Note*: this define *will* stop working in v34 (Feb 2023). (compiling source file ..\..\Source\PluginProcessor.cpp)
1>F:\HISE\HISE Source\tools\SDK\perfetto\perfetto.h(338,1): fatal  error C1189: #error:  Perfetto is exploring a switch to C++17 in v34 (Feb 2023). During this transitionary period, we are throwing an error when compiling Perfetto with a standard less than C++17. Please reach out to perfetto-dev@googlegroups.com if you have objections or thoughts on this move. To continue compiling this release of Perfetto with C++11/14, specify the define PERFETTO_ALLOW_SUB_CPP17. *Note*: this define *will* stop working in v34 (Feb 2023). (compiling source file ..\..\JuceLibraryCode\include_hi_snex.cpp)
1>F:\HISE\HISE Source\tools\SDK\perfetto\perfetto.h(338,1): fatal  error C1189: #error:  Perfetto is exploring a switch to C++17 in v34 (Feb 2023). During this transitionary period, we are throwing an error when compiling Perfetto with a standard less than C++17. Please reach out to perfetto-dev@googlegroups.com if you have objections or thoughts on this move. To continue compiling this release of Perfetto with C++11/14, specify the define PERFETTO_ALLOW_SUB_CPP17. *Note*: this define *will* stop working in v34 (Feb 2023). (compiling source file ..\..\JuceLibraryCode\include_hi_snex_62.cpp)
1>F:\HISE\HISE Source\tools\SDK\perfetto\perfetto.h(338,1): fatal  error C1189: #error:  Perfetto is exploring a switch to C++17 in v34 (Feb 2023). During this transitionary period, we are throwing an error when compiling Perfetto with a standard less than C++17. Please reach out to perfetto-dev@googlegroups.com if you have objections or thoughts on this move. To continue compiling this release of Perfetto with C++11/14, specify the define PERFETTO_ALLOW_SUB_CPP17. *Note*: this define *will* stop working in v34 (Feb 2023). (compiling source file ..\..\JuceLibraryCode\include_hi_streaming.cpp)
1>F:\HISE\HISE Source\tools\SDK\perfetto\perfetto.h(338,1): fatal  error C1189: #error:  Perfetto is exploring a switch to C++17 in v34 (Feb 2023). During this transitionary period, we are throwing an error when compiling Perfetto with a standard less than C++17. Please reach out to perfetto-dev@googlegroups.com if you have objections or thoughts on this move. To continue compiling this release of Perfetto with C++11/14, specify the define PERFETTO_ALLOW_SUB_CPP17. *Note*: this define *will* stop working in v34 (Feb 2023). (compiling source file ..\..\JuceLibraryCode\include_hi_scripting_01.cpp)
1>F:\HISE\HISE Source\tools\SDK\perfetto\perfetto.h(338,1): fatal  error C1189: #error:  Perfetto is exploring a switch to C++17 in v34 (Feb 2023). During this transitionary period, we are throwing an error when compiling Perfetto with a standard less than C++17. Please reach out to perfetto-dev@googlegroups.com if you have objections or thoughts on this move. To continue compiling this release of Perfetto with C++11/14, specify the define PERFETTO_ALLOW_SUB_CPP17. *Note*: this define *will* stop working in v34 (Feb 2023). (compiling source file ..\..\JuceLibraryCode\include_hi_scripting_03.cpp)
1>F:\HISE\HISE Source\tools\SDK\perfetto\perfetto.h(338,1): fatal  error C1189: #error:  Perfetto is exploring a switch to C++17 in v34 (Feb 2023). During this transitionary period, we are throwing an error when compiling Perfetto with a standard less than C++17. Please reach out to perfetto-dev@googlegroups.com if you have objections or thoughts on this move. To continue compiling this release of Perfetto with C++11/14, specify the define PERFETTO_ALLOW_SUB_CPP17. *Note*: this define *will* stop working in v34 (Feb 2023). (compiling source file ..\..\JuceLibraryCode\include_hi_tools.cpp)
1>Done building project "HISE_SharedCode.vcxproj" -- FAILED.
2>------ Rebuild All started: Project: HISE_VST, Configuration: Release x64 ------
2>include_juce_audio_plugin_client_VST2.cpp
2>C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Microsoft\VC\v170\Microsoft.CppCommon.targets(1096,5): error MSB6006: "link.exe" exited with code 1181.
2>LINK : fatal error LNK1181: cannot open input file 'HISE x64.lib'
2>Done building project "HISE_VST.vcxproj" -- FAILED.

Testing in the standalone, it seems to work. I can't get it it "drop" a note. Going back and forth between the new and the old method, I get wrong active group 3/20 times with the old one, and so far none with the new one.

Great!

d.healey

@aaronventure Did you try cleaning the build folder between building the standalone and the plugin?

aaronventure

@d-healey I always do.

Tried it again just in case I was maybe tripping earlier but no, clearing and building fails.

aaronventure

Everything working great in the latest commit.

aaronventure

@Christoph-Hart Posting here because I think it's related.

I wanna introduce humanization to my instrument. In theory, this is dead simple: call the delay method for a random number of samples within a limited range.

The issue: storing event IDs and properly noting them off, as well as preventing execution of the play() method in the first place in case the note was shorter than the set humanized delay.

The noteOn callback will always execute in its entirety before the noteOff callback (for the same key). if there's a Message.delay() in there, the noteOn still executes and it seems like it stores any Synth.play() calls into a queue.

So I thought about reverting my dismemberment of the original Play Logic script and stuffing it all back there (since the new setActiveGroupForEventId() solves the issue there), and did a quick test.

I call Console.print() in noteOn and noteOff of the Play Logic, which resides in the master container, and then in the Guitar Event Handler which is a Script Processor for the Sampler. The Guitar Event Handler features a Message.delay() call in the loop that executes its <5 voices (multitracking).

The actual MIDI note is incredibly short. It's released way before the delayed events begin playing. Even so, the noteOns get executed first, and then noteOffs.

This makes any isKeyDown checks (whether using the built in Synth class method or filling my own array in noteOn/noteOff) impossible, because even though the note was released at the time the delayed note is scheduled to launch, the scheduling of it was executed on the physical noteOn.

The noteOff happens before the scheduled play() execution, there's no event ID to noteOff and the scheduled voice will then play forever (or until the sample/envelope runs out, but you get it).

I'm not sure if timers are a solution because this would require a crazy resolution of 1ms or shorter (and I've already been yelled at by HISE for timer timing before).

Christoph Hart

@aaronventure why don't you delay the note-off too?

aaronventure

@Christoph-Hart because even if I delay the entire noteOff by humanize.get("max") * 2, so around 100ms (which introduces noticeable floatiness to performance), I can still reproduce the hanging notes by playing a very short note on a keyboard (I don't have to try and make artificially small nanosecond long MIDI events in the piano roll).

I use the humanize control for all multitracks beyond the first, so that even as you play, the sound starts coming out (the first voice plays) on your time with the others having a small random delay, so it "feels" tight.

In Kontakt, the solution is to simply call wait() and it delays the execution of the callback at that line. I My understanding is that HISE works differently so that's not exactly an option here.

aaronventure

@Christoph-Hart I made some progress by childing the sampler 2 levels. The event handler is now at the top, the humanization happens second, and at sampler level I check if the key is down.

But there's another problem here:

The multitracking I do is based on neighbouring zones. So the actual NoteNumber of these events differs from the key that is pressed.

I already have the tuning numbers for each event so I can use these correct the check to make sure I'm checking for the correct key.

The issue comes with playing multiple notes at the same time (on the same string). Due to execution order (same timestamp event, container will execute twice before the child gets to play), the second note's IDs will overwrite the first in the ID array. This is not normally an issue because for the notes on the same string I execute on note-off before playing them (and overwriting their IDs).

However, due to humanization the note are not yet playing and by the time the check for ignoring the event in AntiHang comes, the tuning numbers are already different (overwritten) so it runs the same 5 tuning numbers for both notes (10 events in total).

The execution order of script processors prevents any solid event-related information hand-down down the hierarchy. I can create a massive array and store information into indexes based on event IDs, but I'm not sure how ideal that is memory wise. The solution then gets highly specific with each individual project and refactoring, as well as keeping track of the whole setup between projects, becomes a mess.

Is there a chance the message object could get an empty property (or 16) where we could store data? Something like Message.getCustom(index) and Message.setCustom(index, value). This would be highly beneficial to any complex project. Kontakt has EVENT_PAR_CUSTOM with 16 value slots and it works really well for passing down event-related data.

~~HISEScript isn't typed so maybe just a single property, and we can then put an array there or whatever (would that be accesed by Message.getCustom()[6], after being set as Message.setCustom(array)?)~~ Actually this is dynamic and realtime+dynamic=nono, so maybe the last idea of 16 slots is better.

dxmachina

@aaronventure said in MIDI Processor Inconsistent/Processing Speed Issue?:

Is there a chance the message object could get an empty property (or 16) where we could store data? Something like Message.getCustom(index) and Message.setCustom(index, value). This would be highly beneficial to any complex project. Kontakt has EVENT_PAR_CUSTOM with 16 value slots and it works really well for passing down event-related data.

Yeah, I agree. I've been on the verge of putting in a feature request for this many times. Usually in these situations I have found it's just been my HISE ignorance, but custom event params are not something I've totally found a replacement for (and would often seem to solve problems elegantly).

Christoph Hart

I can't change the Message object - it can't exceed 16 bytes because this assumption is being used all across HISE from fast copy operations to passing the data to JIT compiled SNEX code.

What I could do is to implement a storage system that is basically an array with the event IDs, but that's nothing that couldn't be implemented on a scripting level too:

namespace CustomMessage
{
const var NUM_SLOTS = 512;
const var NUM_PER_SLOT = 16;

if(!isDefined(GLOBAL_STORAGE))
{
    global GLOBAL_STORAGE = [];
    GLOBAL_STORAGE.reserve(512);

    for(i = 0; i < NUM_SLOTS; i++)
    {
        GLOBAL_STORAGE[i] = [];
        for(j = 0; j < NUM_PER_SLOT; j++)
            GLOBAL_STORAGE[i][j] = 0;
    }
}

inline function setCustomValue(index, value)
{
    GLOBAL_STORAGE[Message.getEventId() % NUM_SLOTS][index] = value;
}

inline function getCustomValue(index)
{
    return GLOBAL_STORAGE[Message.getEventId() % NUM_SLOTS][index];
}
}

The interesting part is probably the truncation by the modulo operator which keeps the memory usage in check and can be adapted for each use case (512 is a very generous value to start with, usually you don't have this much events active at the same time).

dxmachina

@Christoph-Hart Aha... ok, that's more or less what I've been trying (though I'm learning from your optimization here -- nice). Are you suggesting we use this as is, or would you implement into the engine?

Christoph Hart

@dxmachina There's not too much benefit from implementing it in C++ - maybe the % operator gets optimized to a bit mask for power-of-two modulos, but if it's implemented in HiseScript you can adapt it more easily to your specific use case, but from a performance perspective both are O(1) operations, so the performance is pretty similar.

aaronventure

@Christoph-Hart This works really well. I thought about a global array but couldn't think of a good way to be getting around the memory issue. Are event IDs just incremental ints for each new note on? If so, modulo is the obvious solution there, thanks. 512 is plenty indeed.

Copying this thing into an external script and including it everywhere you want to use it is the simplest implementation.

This is so incredibly useful you should pin it somewhere.

I added two more functions:

    inline function setCustomValueForId(id, index, value)
    {
        GLOBAL_STORAGE[id % NUM_SLOTS][index] = value;
    }

    inline function getCustomValueForId(id, index)
    {
        return GLOBAL_STORAGE[id % NUM_SLOTS][index];
    }

Christoph Hart

Are event IDs just incremental ints for each new note on

To be fully precise, they are unsigned 16 bit integers, so they go from 0 to 65536 (they are part of the message object so every bit saved is a good bit).

Now what happens if you play more than 65536 notes? Either Y2K will happen 24 years too late, or it starts at zero again :)

This is so incredibly useful you should pin it somewhere.

Might be a good spot here:

https://docs.hise.audio/tutorials/recipes/event-processing/index.html

aaronventure

@Christoph-Hart

This solves the same timestamp issue but there was a still issue with two notes being played one next to another with a time interval big enough so as to avoid having two same timestamps, but small enough to be under the humanization window. This would cause ID overwrites and checking became problematic. I think I have accounted for all cases but will continue testing.

You may be thinking that this is highly specific to my project and, that's true. Any other complex project will have its own ultimate solution to prevent note hangings depending on how it's handling note ons and its usage of Synth.play().

Note hanging is in 99% of cases a timing issue where a noteOff doesn't execute on the proper event. For systems where you can pause code execution (Kontakt) this is trivial and a simple check can prevent this, once you're aware of how it all works (and many devs still aren't because there isn't a concrete example in the manual or a PSA on "How to avoid note hanging once and for all"). In HISE, where you have this queue-like thing (code executes and the note gets stored into the queue, even though conditions may change by the time it's supposed to play), it can get interesting.

So I'll propose a method for dealing with any complex situations where note hangings might occur without having to test for and eliminate all possible edge cases as well as having to create complex hierarchical setups where refactoring and keeping track of things becomes troublesome.

Synth.playNoteWithStartOffsetIf (channel, number, velocity, offset, condition). Whenever the time for execution comes due to possible usage of note delaying, the note will only play if the condition is true. Usually that condition would be to check whether a key is still being held, but could be anything else like checking if a variable (or array) still contains its ID etc.)

Let me know if you think this makes any sense.

Christoph Hart

@aaronventure would a Synth.cancelNote(eventId) function also work? You can call this in your note off callback and it will then look in all queued upcoming events and remove the ones with matching event ID.

function onNoteOn()
{
    Message.delayEvent(44100);
}

function onNoteOff()
{
    // if you play a note < 1 second, this will cancel the future event.
    Synth.cancelNote(Message.getEventId());
}

This is easier to implement and I don't have to drag that condition function around (the condition parameter in your suggestion must be a function object so if we try storing this from a realtime thread, our friend the AudioThreadGuard says no.

aaronventure

@Christoph-Hart said in MIDI Processor Inconsistent/Processing Speed Issue?:

(the condition parameter in your suggestion must be a function object so if we try storing this from a realtime thread, our friend the AudioThreadGuard says no.

Can't it just take true/false like a bunch of other methods? I mean e.g. Message.ignoreEvent(bool) takes a boolean, this should be the same (I have no clue how it works it the backend so I'm just throwing stuff at the wall.

The thing with Synth.cancelNote(eventId) is you again have to query various conditions. What's the physical note that caused this ID to be triggered? It's not necessarily the one in Message.getNoteNumber(). So here we go storing data in our custom array and handing it down the line again, while writing logic for the edge cases. It's barely a step less than adding another layer in hierarchy and culling the events at the lowermost script processor (which is what I'm doing now).

The idea with Synth.playNoteWithStartOffsetIf() was that you write the condition for the note to start and then you just continue writing your algorithms. The conditions for the note to start after already scheduling it will in most cases only be about whether a certain key is pressed down (because the assumption is that you're already doing all the logic before calling Synth.play() in the first place).

The idea of it is to have one last safety check before the note fires off at a point where you have no control over it (the code after calling the method gets executed anyway, but the playNote gets delayed with Message.delay() ). It would be the endgame anti-hang solution so that each processor doesn't need its own child processor with project-specific culling logic.

Basically a "go to that house, knock on the door and if no one's home, come back" kinda thing, give the order and forget about it.

I can understand if this is somehow hard to implement. I don't need this immediately today or tomorrow, I can deal with complex shit because I'm masochistic like that (I'm here after all, aren't I?). But long term it would be a good feature because it enables rock-solid note management for any project that has any kind of time-specific note management (I remember Mike on VI saying he gave up on HISE because it didn't have wait() like Kontakt -- my opinion is that you can achieve everything with Message.delay() but the queue introduces complexity to follow-up logic both because of event inheritance, script processor execution order and (the lack of, for noobs like me until today) custom event information, whereas in Kontakt you're just thinking in a linear timeline, for better or worse.

MIDI Processor Inconsistent/Processing Speed Issue?

18

1.8k

10.3k

90.3k