Piano Roll UI Component?

[HISEnberg]

Has anyone tried implementing a piano roll for the UI of a HISE project before? Personally I am almost always inclined to just use the DAWs piano roll for creating and editing MIDI notes, but I am curious if this has been done in HISE. I imagine using a panel's child panels and data would be the best way to go about doing this but it isn't an easy task. This could also make for a nice feature request if there is interest, but I thought of asking the forum before doing a deep dive into the topic.

[clevername27]

@HISEnberg That seems reasonable, yeah—especially since you can store meta-data about each note inside each panel/note. I could offer more if I knew why you wanted to offer this inside a plugin? (Like fxPansion's BFD has an internal sequencer to make use of their groove stuff.) There may be some stuff in HISE for using SliderPacks for step sequencers, and perhaps it would make sense to create a front end for that (depending on what you ultimately want to do).

[HISEnberg]

@clevername27 My initial reason was a sequencer, but even that has it's limitations (always being tempo synced, for instance). My inquiry was for more general use cases. I imagine the most flexible system might make use of the Keyboard Floating Tile (communicate the Piano Roll panel data to the keyboard to trigger midi events), but this seems fraught with issues as well. The cleanest solution would be a built in UI component (probably under the floating tile), but I doubt @Christoph-Hart would take an interest in implementing this unless there was a general consensus that the HISE community wants it.

[HISEnberg]

Perhaps we could just lean on some open framework as well. I am not really sure how to add UI components to the HISE source code but here are a couple MIT licensed JUCE Piano Roll on Github:

GitHub - Sjhunt93/Piano-Roll-Editor: A simple piano roll editor written in c++ using the JUCE framework for graphics

A simple piano roll editor written in c++ using the JUCE framework for graphics - Sjhunt93/Piano-Roll-Editor

GitHub (github.com)

GitHub - johndpope/PianoRoller: This is a piano roll VST that will allow musicians to use a piano roll sequencer inside their DAW to control midi instruments. It supports multiple presets and can be controlled through DAW automation. It was built in the Juce framework.

This is a piano roll VST that will allow musicians to use a piano roll sequencer inside their DAW to control midi instruments. It supports multiple presets and can be controlled through DAW automation. It was built in the Juce framework. - johndpope/PianoRoller

GitHub (github.com)

[clevername27]

@HISEnberg Is it just the rectangle-editor you want from the piano roll, or do you also want the meta data fields (e.g. velocity) drawn as editable spikes below each note (in the footer)?

[HISEnberg]

@clevername27 my immediate objective would be pitch, velocity, note length. Extra MIDI data in the future would be nice but I think this is a pretty reasonable starting place. So in this case there would need to be a footer for the velocity.

[clevername27]

@HISEnberg Would an Event List work for you?

[d.healey]

@HISEnberg Have you seen this? https://docs.hise.dev/scripting/scripting-api/midiplayer/index.html#getnoterectanglelist

[HISEnberg]

@d-healey Oh amazing this looks like exactly what I need. If I pursue this further I will repost here with a piano roll for everyone to use!

[d.healey]

@HISEnberg The main issue I see is with interaction. Probably the easiest solution is to use child panels, but I'm not sure if you'll hit some issues with potentially 1000s of child panels representing the notes.

I actually started building my own sequencer (notation based) and ran into this issue so put it aside.

[ustk]

@d-healey @HISEnberg @clevername27 I wouldn't go with child panels here. I've made an editor ages ago and, to me, just storing the note object in a big object like {note:x, vel:x, length:x, anything;x...} and drawing rectangle in the main panel is nice. Then it's a matter of mouse detection in the note area.
The only drawback I have seen since I played a lot with both solutions (child panels and objects) is that the object solution requires you to loop through all notes in the object before deciding if you're hovering one. If you have hundreds of notes, this can add some significant overhead.

But child panels can be a nice solution too, we just have to be sure moving it around and changing the length can be handled reliably

[ustk]

@d-healey @HISEnberg @clevername27 Actually I am making one with child panels and it works nicely
Still a few things to add and I'll share a basic MIDI editor (to be improved)

[d.healey]

@ustk Do you think there will be any issues with lots of notes?

[ustk]

@d-healey It is just that the overhead of the object solution is the fact that it requires you to loop through all notes, while the panel solution can overtask the UI since child panels still are components (and I know from a past experience that too many components can make the interface laggy)
Now, which solution is the less greedy, I don't know...

[ustk]

Alright so here's a prototype of a MIDI editor using child panels.

• The notes are created in an array of MessageHolders
• Updating the notes doesn't work yet because of an ID tracking issue, if someone as a better idea that'll be nice
• There are still a few bugs and many improvements/additions can be made depending on the use case

As for now, I can't find much more time to finalise it, so, here's the snippet which I hope is a good starting point

HiseSnippet 5641.3oc26jrbiibkjU2z1R1db6H7o4TZEw3fpDKRBRJtTkqoKt.tHwMwEIQ0QGc.BjfDk.AHA.IEkcEy7IL+By7k3y91bZtNm7o4f+C548R.P.PQoRtpxWL6tnHx78d4Ke64B5ZnKRMM0MBE9vAalSCE9WDo+FMqokmJnnEpQkPg+lHsTjTZqaQ4kTr.HKsYtfoIUJT3veUMDpvG70gXe9qeaIAUAMQpWSgBcothHsoxLEKuV69tyUTUqJHQGnLyGzYdWCQcsx5p5KAN5qhjLzbAwaElPaKff8hHgpKXNMT3WFIk7XwTx4Rmmq.WgbYJvwcZ9wYRlRT5zBoSmMsjbp7BYyCH8SrY69VBVTSfnkzk1zep9ZM6A3RESkwpT7AtP8gQ1t4PkmpnJ00U7XFJT3utqmv5qrEV+FlrYa66JzHdX3WnE9EOEKw82.K8BerzO0U+wXIUgMTCFq7OEoUiJMH1svEfQJuzvfpY0mtXIMfV6u9sNcMv.j+9auot9bdMAf8j72LR8w.n8mSC1Q3u5KzbMDNsppnRw4cX2G.nw4HXXTQvRHzK9+95HExjLt1frmmu7kkaN3hZwutT5MipT474UiGWi8+rOr+pIqUWpzcoZMd8clZWVU71ZKKLdbNK822XcwSUWe98mlJ9kbkt58mlMkFOeqSpnu3702l9l2WqXN566cZZAk7zhqtNdN93um1M+hZEtux4xoU0rzSUqv64pUuS4Uk2T9pyJLt7fbMzZUo6cRMlLRtYuLzdWung93EBW0MmdsUMyXJc5kqSMJYNJO8NC0yWJZlbA+IEaITH0zEpWuoS5xS6F+jK6d4n5SKMoxUSoYNQc0YkazslXcE0Ei0SOXP2M8GqubY5h0xK1bHU6L5MhwummOiV8dERNH46i23Fs92JVTpC+4Kydwr1JxoiWucg5iG2KG20k2rrStFK5XsNqN0r9zIiimr280K29lBylJlTtcugF4xYUtv6aro6FYtKuXYi7zUyJZrli6jjo6uN6jSOYxEzAUKWrGcy59S5Nqp3FM0BqqJjKSwSKPE4alejDkNavhgmWnYItTyeOOUcpImQuqJdqQy05YaLc56ue0x4YRWQZyJqKTZNY0r3qtRpa8bSjue0U8ReR7ViJUV89lc0KlqmR5TsGd6YCi28pZ0pUs40UybwFdk6FrTtz4UZIWu0zgqFkj+hNhmZjZV6YY3F1Z7oF0JYnO8hYIkU3EKrbsYoty3WNZ5okyIurb50c6HtPVQkeT1a0u5lytrpk7hazjZLuyhEwyWKYmlF2yq0JeY4BJ2TYxD00wiWZP77EMTkGMesp7fI2sd3fKttd0I02ToTubE6FOdu5MWI259pYybR9Mv+VZYjscs6EDoBcjMiuIemYKmXZQqLdvlqNwJSlqV2YwERKRYIuQS+d9I83xMbg4ra5Ula53IzgbMNK8I22cSGUfLWoNd70Kp1Nc5wJsm1ejxDk4iu8RsZWTkan5hqqeUtUqMxVbzjTMyWmax3dWdwhKSdS+JWK041bm0KSqKNq6v5SzWrYdkx4pYLM+vRVWuT75YRq6z1XRKqloGsYZsjxhiT34RyqdWQ0Qit87RoFPuxjezJsFktp3U55mcBU+pxuWNY40mb57yt.lJ2IU91Su9diyt4ViUY4qsToXxV7kp2WXZt0B8RVuTu7qWVe3zxBVWLNc2MyEKksbm0hMUlJ0otB2fwWr9h5J2G2ZPxzquT9R9LWe28EGbYwFcOeUEcf372pzo1cxCFmYxH8ZUNQ49FCxIrPpQqUmYZMlWVd07bky1PIeol2LePIyNM51k+p0me9IMJT8lVWTd8ssU6pOcyIkpyWRluc03s6U8z9sdulZ+bIGHHWXXMJs6hyKJu39qh2PuZp5W2oX80oMMAK+6GWuSggitRdo9fBZsRsYRm04mKW5rzcF1c5lbitOd23ixloTUyoUm0nzBkgajmKzzhtZ8nIWTYLUbEe+p2Ijur5hxSsRM3lkZqGQKuVLdHLBnclw+qHIFZRMLSLwfNIQEcwkyfn2lI9gt85TqWwVsZztVhenVmdEa+C050oyk78STWwjRtr+fWQpKnIsgTwX4LyWQ5pulZP5oKdK4Uj05F2lXaj2D0zMDzpYnquhZ9XX8JI5LcRTKHaDIIYFf6wwgugf1Xt99JSzDrVZfIlaubVIAH1ta5h+Tp20VelBzOlOyMGx27tJTs80LlJBRqXX4uZC67SARAMfNatN1v+ci+yuI7659tP9xZ901YM+0Q5KZnL2xqGLsxOORCMKpgL3OFHuoMrgdw+ywQJqCPnYEelvszpFvCawHZ9jIiQxlL4wu4P.JScUZbQUpfQTngCS7xD+i5mCO7.VEGs6LfmvWowfN8HIHcaTrcGRuNMad3AEWZMU230jg8Gb9gG.fWr8vhMe8gG7JRE8kP4.jxpJf8mkNQPRJgDUkZQIBDMnHTDHrXTShfAkHorRQhJQTzHVSMnTxbvdv70GnRksHG7p+UxLvT8vC19ALDk.xi8rhppKpXswWuFJSlZilzRCAKEcMbzJ2pBP+UTfvvfPIyDlnQsTLmw5bPulrAwtOjCQ11fpB3uhlPXLn3WBMNW2TwlhGd.HYHi3GPpNrc4AM5z1YtubtDTjja0XDyo5KUkHxJZRdzd7FBcEXvQZTIFY7RKBTJkibhHnAsxdfzBLhgxlqqqJQM9VvabJcCSfIrzReFvZhBppaHlTKD9jGibUog05y3ihDH7wR6I.wPeoljh1DB0vP2ff149lpRFJxVXulpnrCHofIYMkIQHJH2YnbO3UvFMjKHyMzAE7LhBREAKhnN3cpgAqX72bUv0QBYJDX1fCONW4NpZLhoNQhJpLSPkLlJX0W4dpOFBjvWiHNFz.FK0tkJwlVMZAw.ujuEe6ANSOPTwLPg74Thpt.N8vN3uCbkkHSLTjHSoLaAfbboxwlq.+Acx3OzbSfboBc8bcCKWZVOwkj600mgHQYE851CSToAApA+f4JBZf8gtpJIJHfVyzw.KqAqRBwbp.H4.Mo9bzZAaAmgrInoEFT9b5lw5BFRjp.qix9A3z.MOQkJNk+G4PKPjyDj29I+gTgubyh8Jhdc8+LnygvxTMsHqDLXlRNKth7VBawywmPs7VEVzi7u.rivn+dXaanzUSEv0MWBfcYWuhnGsEhcvDcw5JXM0GhhFfWAqQLGiK.wgXCEAm20Q+tjwOMFA956iQfemx82IimLEqCN1WOGby+YfK62o9LvM+dw0mrwfZBwFdRoiGHeJxmmA1OgDxGmhwwp3jrA30uKShLw7RHA15bN4mHjLIhl5kbwO83XvOSEi8bB1ytflHEjTfC6C9ZfeD41hHGCQtcPLCCwLruBfHLE1hJa9Y21VzYXmmgcd1W6fcJ+nmxE+TaI.CetrLBvk096cHAW.ZvskHbaoBiHosm7oSY+8NDIY1.TAeLlWOL578ugQmr1xhrYr+dvgfxdBQa4L6pNN3sjSS9F61fFpamm.ZcqeJ5+Z2bzig.9tX5fyXrjWxA.BYraAS2vxjAMkbKPfYgouV1ltyqISHd+sCzqgYqdKQVP0jBVV9Ls5yewP91k4QypuOfQWolEKe9Of0m0G6LUrLwxEqPLNNDrO2Hr04a1ku21Za9rhxBbBVxrFa2fLnZGpnopnQIxK0DYdLA5M5wG9GN7.YnJknhXSXZ5.ZE6MURPip1TwDzNPxxCXPF2fhQdfB3m0U.22KLBwGNbGp+FaYCPHbkLlVByl2PqO7MrznGvY6EpnPMJMz5dGiQgxOcJjwDzBN8.FLt552fUuXPgEKoQ30l.TGmCNTpptwEKgZcoFLSknLxvXZWdDM5fUXMlZ.v1U2bz9XwG.TzMLdyYXagA3jU00MhFc+12uhrgYju0UvgG9LMhbV1P2hs4aBiQWAEnj2d5KsvIvmgAk2j.J8kQUGhF0UpDcBS.LINTcnZQU0nIuSVli84XlFAlXnXzzwRS4sbugn76gZaUL8KYR3WjPTN4DvXCn6AJxjn97+hKhUGqnYFU4egKEyhDgBFeSrJ.bivYbPxb3+gbvAN7VOpnEjyIFY2QFjWJuzaz8.3JEIL2WLeZqumQwO.+6vGNloXeXPLItjgv5lnfJ4dHXvwK3SXzdjFevQ3wrTO7IFL+hUmHgtBP+7gxK2gMR3.Lx.OZW6QdwxSaqaYL35ovR0IigvzOjKSy9rGtDf9Yyj.rOBOx54IXwOb7a9h6a0ReoIrf6RewbqXDrLrhO7.A77qXKak4agt.rmhKwVsOaw9NtGX5I1xkfDc6M9jMhaXVkHvVlczb.dOwa8mj7asW4b76vIMKv1LcIG5cWrsgcOl7ZW.8OHxx3nvFsSdjAaKEbQaEUEQJax2X6fAkJxiTNJ7CblU112GVh.NkiwndL6AKFh7w19MeQz56NjeNp6cSj7bmNdY8vZc8LZ.78RMi4YOPKtjfkPbWKA7uds5pxw+Fn0s5HYYu1YJAnY3uPiLOqfzl44ayVrsP3s9x15OGKXA5powQ9XxK8mpdKAFADXOgjixDJmvgnEHaoClqwn.OywVV1+fCDG4a1zPCc.idD37ou10Ez7nXjifLYDuFrEwXJPmckBKL45XL1OlMqDaGtjEZLPdXWR7XYQY9y9CfZ+Gy3qUrlVTc9TgssLwfR0bB6wTNSgpwL.OVHrG3NxRg3jGhk5ydagvLfBZSToaS91DEkkvdMwJ6g3lohm7XGuOLrihYEpLvnRQ8FKS7nI8x7hNthfhXWH.GFxQXUhGQ9i+QxucGd8XfYCLa.1duTvdkiG4C7wpKo.ztOhaw9D6XHG3xO24ZSsW1ADSO1n4tImHP6ju4kIiCox.DeX6Yy5ToAN1B3RDtKPZo.T3gYq99s79Sq8EeNZ7GSkK3oa2S4KOtoljfwsfBZyyyZCSj+Ls1Rcps411xI1WIUEXebHNzUUcbKdJpII.kbzRWSG7Uy5YxZy.EUUl.lrCn2Y4SIuB2QzG2v+HQHQiAU5H+l+OpMju5NQoMaSRb2KlXa2pIbjbFjc04GuUSPgEhRdrAyw5Ofyl2doTzfhVa91bkmX.YFGAw86R989B95XkBgG2E22ryL0iJw1gabmUeXaHP2xkbh98zE6vlj6Zm8VmxKXO4nbP4f8982GqXYKLhyjpnuVCrM+s9Kk409W8+aH.a8nGRAjlXpvJEv.jDseyFcSTqWiJG6LjJlrgqn6gTrcXsLTww0lz3QbrXoflEHXLwcjlsC4QsO0C2S5PcCpzgbFj4fHzjs46tlc9BVpXVztlge2uyuX3Xbfla.YbDsHBSvEDYQjEfulwj23XwNQS6CM.OofYB2hG7.6XBb3AL+EQbp.lBkcfEaOnin3d2ap6rM5X7d7HdPnDDsOUFa6QuRSui76IQ20T5kDVHOVxB+JVGCbGmIl40V2fOB4vXsOJ41F9dKIeLHcbrdfS3u0uP9v8hKFcikSzWfBGy.rvbaM4VoySmkAzp6QYum4mg0kT0.Y0rqOyIwwdLXdKdpNz2DrWcCEPWKn10tvMWAbzityNt29AcT.P27nfdkSMY6n27kywSVAUVXHLoil5lOp3ZaDWavrWyCcsybfszjYB2scg1ObhdhiqCjbXx0Xsk9Cc7sOXwMLvBt.mjNwOYBamJiBFJ.zj9npsF7.Otz8W9WJkSa9GoS7Whr6v4MeG8Vxinf7OEGsmErwZe2ZTcG.FM8Wc6VNyYX2yVXr+7KGrikKaQCOxJScnMlvg6M6fGaIKO1hTc3ti2CRvJZ9HXARJOSYVk6G4v6eHPXnmzu0mwX.GRWyQ14vF09mX0oQ2iu7q7qwRXWHTL7zSe1riuhC74Yr1uOA2dcJr8SC3U3WXZ5IZfE272nXNPHEXLV6a1f+K3g1y.e+AHXIxFN+3GM7lyV3uCZA2hDa4BC88sk0eb9wA043h1XX3TYiyF9p8EY2aKVoBg+R91CHC536XH9htgC1ajhlu8TfIDcV9r8Y8E3VPvls5rh177ci5aiF75uLTBAtSDbdMgWSn.O6ZoD02lR308kNdVQ81Dhi8sOC5LCtOFuJK+QXVa.Bxs1swX2T9aXO7qrrMO4pghOeo4zn5rowNsw.8KfkwvtUJNf+uK1D6X3aaYjHg6UuZtAZweDJMeM4HvQ1+VLcBzxejvtLYjA8CzuyN9g8yC0P9vdAUoc2fNOPea04fHaP8F8Iv++va9iyNIOCOrJWwhuiJXV7.4ZfhFAone6.rvYjpW0o24MZWKFoe8NCaVgLnWwxmSJMxwQrQE+Ukn8wJIY1SZiaGM4Cr8l7y2l.WPBL0GL.X+9eIsH.MupxbbIRkrzvMmzPWM516aTLxJA0kTlYRvyRk0NXtyt2d64FY3ipGcr8B7Yzd6x.22.y1C2Gxgtm964ZieRNb6oD+1.WYfuiAAtgKtmY7acOh4Wt8jkgd2tWg6bN0tqDIg6YNyLX8.wWlhGUb3aJ7XhiGNKQwgOw.Zg2w4.bAaJxtcIKu29bHlJ5XrmtQCXimBwnZLuJ+BZDvPg+5fu0B+jm2asfns7wGf5ZMzTr5Lm57bUL7tj8u2y63fi.FeyNb3P.TK1MT8ex4FpVZokktVH7B09Ki3yLKDaJ3+duF5wHQeUEHGiCI7oZ1kD+l2EZXC16UQnvgcXMfcmiW0LTpDtBckhH09FzdPjJTyaszmCBusFGgB+ysG5eo68qE2ge1H+yir0JKzcdi4n2sw+Crp1BENR3qeQnP12IO7o+ivgBEbOtCE9WEIvdbGRwhNy8MI5fexe4u7W9QjjdslBa9O+m+yrlGqa.hjdBRJKM8echsaGkO9aMjuI3O8Ypd7li+6B9mis8li2Dx+b72.OYQuCkgQ1twMAF5e1yTs5Mz+IC+Cck2AqmhhzmcEo6uQSLDTjsG.odmoEcdvY+e8asuCstqsxqmidmel6WX47FRYyb9urhLV7WEweSbOlMv+6+16br7rMyNLh68eLz3IdpWIIocTuXq+3O9i+y6QoCMy.Fktd.6paseQhlDNxe3PB4HXzXwHO50Dtbw4xjJ+o4fuKDC6rhh4bUgMcDsDV4jfFfiUGOq+l5qA7QTSwdtth6i4XOWdook9rZFBympHZF.0JTYgkpVEmOmJXfuRcPu39ev5rjJXgCjpGlC3HbGqyZuwx4ymLWgj43XPMPexDUZKPEGfx3ESzojQjUXsMzjdIUD7F8wKaGMnyppP5CqMp6Pot7tJicamUJk2njxscnDJeic1C+PnsV+CB625uJ60D7A2S+uIBLevqlbvW1N70PzoCzjz+c0GuO9Zfo5l.uUZO+2.ujO8ak1yjE+0Q5pXINc+73K1COho.96.O57dK9KivKKCJbOF7qiT85O0WRwOxvae1fSf0van.d4Q.Gk9fOmH0wRvD02u.ipZ+bR7YTBzmpIwd.cXc5jCeNrSmbtcBgsDMz+AQ6Dd3aH3Oi0BvSZr2OlChzBelvsaNtPgv2PkePTLHod.ho9TQL8mJhY9TQ7zOUDy9ohXtOUDy+wQDCWUz4kV.baBEBhhvBQGN712C0veUn+eje9sKB

[d.healey]

@ustk Looks nice

[HISEnberg]

@ustk Wow seriously impressive work here! Excited to dig into this.