MIDI Processor Inconsistent/Processing Speed Issue?

Christoph Hart

ok, hear me out: forget everything I wrote above. What if I simply solve that problem at the root so you don't have to do anything?

For each note off I just need to iterate the queue of pending note ons and remove the ones that match the event ID and have a bigger timestamp (the second condition is important because it allows super fast notes where a note-on and note-off occurs within the same buffer). This removes basically every scenario where a script creates a note-on that is scheduled after its note-off.

It comes with a tiny performance overhead because it has to do a nested loop (once for the incoming notes and once for each queued event), so while it has a theoretical complexity of O(n^2), it most likely won't matter too much because the number of events can be expected to stay reasonably low. On the other hand I could just disable this functionality by default and add a API call Synth.setCancelFutureNotes(true) that enables this for each sound generator that has a script fiddling with note delays (so all you need to do is to call this once in each onInit callback of a script that is about to mess with the event timestamps).

aaronventure

@Christoph-Hart Alright, that sounds even better than what I proposed.

So it would check which noteOfById calls were made and remove these from the queue, if I got that right?

And the enabling call sounds good without adding overhead by default.

@Christoph-Hart said in MIDI Processor Inconsistent/Processing Speed Issue?:

But I agree about the attractiveness of having a wait function and it has been one of the first things I tried to hack into the JS engine back in 2013/2014 when I started the HISE development. The problem is that I can‘t suspend the function execution and then pick it up with the exact state for each variable but since my brain has grown since 2014 maybe I find a smart solution when I think about it now…

Yeah it does have some cool uses too, some of which are like a cult secret. You can dynamically create timers by having a while loop with wait() in each iteration which puts a callback on hold until conditions are met. Once you realize that playing an artificial note and immediately doing a noteOff for its ID in the very next line creates a release callback with that same ID, you effectively get the option of infinite dynamic async callbacks. You can then call stopWait() with callback/event IDs and cancel the waiting elsewhere, etc.

Other times it can get a bit messy but with great power comes great responsibility.

Christoph Hart

So it would check which noteOfById calls were made and remove these from the queue, if I got that right?

Yes. Whenever a note-off message is about to be processed, it looks in the queue if there is a pending note on with a bigger timestamp and then just removes that so it won't happen. Whether the note-off was created artificially or is an actual key event doesn't matter.

I've pushed the change (without the new API call so it's now enabled by default). Feel free to check if that solves your issues.

aaronventure

@Christoph-Hart Alright, some observations.

So this new implementation that you added requires a specific eventID storage paradigm if you're ignoring the physical events and creating your own.

Due to the possibility of two notes with the same timestamp resulting in two noteOns happening before noteOffs, this will not prevent note hanging if you're overwriting note IDs is any way other than for the same note number.

So you NEED per-note ID storage, even for monophonic instruments.

If doing MPE, it's gonna require another dimension for the channels in case of multiple cases of same note number on different channels.

Christoph Hart

@aaronventure well if you spawn off multiple events from a single note then you definitely have to keep track of them and call note-off in the respective note-off callback of the message. I don't think this is a design flaw, but just a reasonable bookkeeping task (again, think of it as RAII: the one who created it (key down) is responsible for the cleanup (key up).

I can offer to add a few more data containers that make this a bit more convenient, but there are a few of those already:

https://docs.hise.audio/scripting/scripting-api/midilist/index.html
https://docs.hise.audio/scripting/scripting-api/unorderedstack/index.html

What this new method brings to the table is that you don't have to care about the order of on / off messages anymore due to possible timestamp manipulations, as long as the event ID matches you're fine now - I agree that this is messy and should not be the burden of the script developer.

However I realized that this didn't apply to artificial events (because they are not processed by the safe check as it was). I've extended the robustness of this method (it performs this check now at the API call that adds the note-off) and now I think it should be fine.

I've added the docs including an example that should demonstrate the "proper" way of handling artificial note creation.

Now if that is still not convenient for you, I think the next stop would be an API call that automatically handles this logic. I vaguely remember that you could pass in -1 somewhere in KSP and it would "tie" the note to the note off for the current note-on message and the HiseScript equivalent would be:

// adds a note on that will be automatically stopped at the note-off message with the given source event ID
Synth.attachNote(int sourceEventId, int noteNumber, int velocity, int timestamp);

Lindon

@Christoph-Hart said in MIDI Processor Inconsistent/Processing Speed Issue?:

Now if that is still not convenient for you, I think the next stop would be an API call that automatically handles this logic. I vaguely remember that you could pass in -1 somewhere in KSP and it would "tie" the note to the note off for the current note-on message and the HiseScript equivalent would be:
// adds a note on that will be automatically stopped at the note-off message with the given source event ID
Synth.attachNote(int sourceEventId, int noteNumber, int velocity, int timestamp);

Wouldnt this be nicer (and more flexible) like this:

Synth.attachNote(int sourceEventId, int targetEventId);

Christoph Hart

@Lindon Ah you mean first you call addNoteOff() (or whatever), then "link" the returned event ID of the new note to the sourceId? Yes that's even better.

aaronventure

@Christoph-Hart said in MIDI Processor Inconsistent/Processing Speed Issue?:

@aaronventure well if you spawn off multiple events from a single note then you definitely have to keep track of them and call note-off in the respective note-off callback of the message. I don't think this is a design flaw, but just a reasonable bookkeeping task (again, think of it as RAII: the one who created it (key down) is responsible for the cleanup (key up).

I think you misunderstood me, artificial events are no longer an issue for me after you showed me how to pass down event ID-related info down the chain and keep it tight.

The issue is the one from the very first post in this thread: two notes in a chord, not perfectly on grid, but same timestamp. MIDI Processors get processed in parallel instead of serially.

It invalidates any logic and scripting that wants to extend beyond not even a single scripting processor, but a single callback (e.g. setting flags for whatever down the line). Logic that isn't organized into per-note, 128-size arrays (and even more for MPE) gets overwritten by the second parallel event. If there's a note launch there and that's all you need, fine. But if there's anything down the line that you're hoping will successfully refer to the logic here, unless the data is stored into per-note ( x per channel for MPE) arrays, it gets overwritten.

This is why being able to store data along with MIDI events is crucial. If the per-note event ID array model is not a good fit for your project for any reason and you want to play artificial notes, you're fucked without being able to pass down event-related data.

So my observation was just about the importance of storing event IDs in a per-note (x per channel for MPE) array for any project dealing with artificial notes, unless you want to deal with writing a custom note filter(like I have). The queue removal is great and eliminates the need for a custom note filter, but it doesn't work if a simple chord causes parallel note-on execution that overwrites your artificial note ID unless you have proper storage (per-note array, x per-channel for MPE).

aaronventure

@Christoph-Hart But the note ID for the note off is already the same as the one for the corresponding physical note on. Correct me if I'm wrong, I think Lindon meant that in Note On, you would play your artificial note, storing its ID into a variable, then call

Synth.attachNote(artificialNoteId, Message.getEventId);

Then when you release the physical key, it notes off all the artificial notes you attached this way as well without you having to call the noteOff method and worry about passing in the correct IDs.

Together with your queue culling implementation, that's one hell of a cocktail.

Christoph Hart

I think Lindon meant that in Note On, you would play your artificial note, storing its ID into a variable, then call Synth.attachNote()

Yes I think I meant the same. I'll just go ahead and implement it, then we can discuss what I did wrong :)

If the per-note event ID array model is not a good fit for your project for any reason

That's the entire point of having an ID attached to an event, there should be no use case where the event ID can't be used for note identification - it's far more robust than note-number or MIDI channel (read MPE) identification, so there is absolutely no reason to switch to another data model (except you have a very narrow use case and can trim some overhead).

The issue you had at the beginning of the post was an outlier because of how the internals of the sampler module work(ed) and I'm sure there are other edge cases like this lurking in the shadows but in general you should always rely on the ID for, well, IDentifying any event.

it doesn't work if a simple chord causes parallel note-on execution that overwrites your artificial note ID unless you have proper storage (per-note array, x per-channel for MPE).

Have you checked out the snippet from the docs I posted (the chord script)? This will still work with messages at the same timestamp.

MIDI Processor Inconsistent/Processing Speed Issue?

20

1.9k

10.7k

93.0k