Thinking more about it, it is probably my script-specific that I can't use this on noteOff.
This is probably because within noteOn, an artificial note is created to kill an existing note lasting note already present in noteOn callback. I think that's where the mismatch happens.