Help Needed: Detecting Audio Signal in Display Buffer for Oscilloscope Animation

Mighty23

I'm working on creating an oscilloscope visualization in HISE. The basic display is working, but I need help with a specific feature: detecting when there's actually audio signal present in the display buffer.
Here's what I'm trying to achieve:

When there's no audio signal, the oscilloscope lines should stay static
When audio is playing, the lines should have a gentle floating/drifting animation in addition to showing the waveform

Questions:

Is there a way to check if the display buffer contains actual audio signal?
Can we access the raw buffer values or path data to determine signal presence?
or... what I need to achieve a floating animation only when the sound is coming?

Any help or pointers would be greatly appreciated!

HiseSnippet 2128.3oc2Xs0aiabElz1bSrRRQSwll9PeXfQKJcrWshx21Tmzr9l10n9hpkWuAvvXwHxQRCL0LDjirsZvBzG6On9CnO1G6+i9R+GzdNyPJRJo3rQHa5lxGj3blys4blyY9F1LV5yRRjwV1UNeXDyx9CcZMTn5sWOJWXc39V1+LmioIJVLwPZ2gQzjDVfks87OCIXu3BV5m+8WsKMjJ7Y4jrrtPx8YGw6yU4Ta9z+HOLrAMfcNueAtW+oG5KE6ICkC.+YdmZVQT+qocYmPQ1lyw54zjdV1elymuwFdzfNa0waMuOucPGe1ZaUay02pNcimzoyV07Vesfs5rN3jO3f.tRF2RQUrDK6E1UFLrUO4sBiAtfmvaGxvAdVs.KaH2PFFfKQjp0d83gAMyBTIVV1NMyCayaBaOz4Xd.eD87v2OWOAIWhhAP64J6dyWx87J5d0J3dSwkrK3RKXboO1okeLORkOC5OefygBHa1gB4ohthgWq4N+8c1SBbHTU6Sul0HFFLRB2MqUaUB7yxaWoBjqRTjanwjfHxWRzaap1ko1mmDERGt6fNcXwsfbIH2ROmFG3KgfpYyTiu1aIPG4pHtMnhfnIj2sVI118EMZbvYu5nCN4Ym+bPhM7pWb5nXYDKVwYIvbeSEB7rjQOGwDcU8V52WVCqZX4jA8gDlPvBS.N7p75sqD2tZBScFWz0HeyQZ1M2HkbsldfMyBcvxXOY+Ho.F3tTywVrm7hie0K24hCZb5YG2BjZyhSdK8FVGYbeicwExkWscEbFETs.CqAwdfAhKRiqI.+8Ek0JPZkUVNMFLlJujeEHErB8iYv9tlTUO2ldnOejzmFtqbfHHAB7qRtDx1kCXjG8jp.Q3mqVkTqJlcdcEs20Vq7CEAr6LNIRT1oCDFScZvFv.rlOdOZXXan31sy.guhKEtY9JJkfcK5TS3jZFvGUOdxz72QLbowvi48jUHYzejWcbcf+d0p45EiveFws3ZYEh2x.sZUqqYCVv3eSMtVPLLBmtLLBTN7LoE9sik+zBMJ74l91Jj5arIxbo0kgYsyuBDpq5UvG0gpXVDzOR4hYqkGkIZhzNSNPwEr7DQ2hYBJD7AyO83swJcq1A5luSXnas6Zznl9oXP5wOFN4f4eMg2g3OHNFpHHQX1sGMgPGDvkj.phNxjo7jtA39BwaORFPU6n0zWVTbziScVbcr7kqcE4OfaZ81tjWk6MhgjdLd2dpb2+6WkVlC0llvNcTxiSdTYYdbcbK05aWRlavlJP7cjb4JImwQu.9sa1xtn0SC4mJBGRnAAjNgRJjf6R5KwzKJmpGKl86xB9OhLjQ6QDRE4VY70IUIewakmRd3XqUXW6wXBKgKbSqAqUcCX2NW+1SvnkWs7nvqmLdj2sYGyd1Kwb9kXaJ8KdWApqrYSmodFKqMIKWUNE4CGAGOJ+nct6v86dd470EKsLfXRqH1Xis1BzbAgWtH6AwTceB2IaSuZgkzpj5oh857hXEM1zP00ayk2NqFlHEmHUrSgtpU9lJKBcnIiOUmNScN77qXYXHnvoMswT2ifthA8ayhWEhVgCXiXDvjTFniy2NPmh3v7MGmVfQo3PAWcZDS7sgNyJ8LX3sWb39PmEDcTJs7SwAZ6ytAvlZvJsny9rjqUxHMuoGbCnGU5Y+nLjTT.hfEGvKsfSSOq6xgt9ad5vhCtkGn.fpN16OmkkoiBN5uoQ+NAPM.hnLXPHUUF2HhtNcBHFWBrFBHSjvUCKh99GLvjuot3G6zjq76Mcebto3iPl3sgOlBA+ibN.pa7U4N3BNM952N3sKZ9O0X9Ov4kP8qFEr13ehdLVOSdFSvhwXm28b8o+4a50mhdiu9zo9JvENOlJRhjILuhZtEqO+bXadRIpiIQ8oJQIp6yTCDkUsgTFWOcs0l+uighFPnvq3F1LhEz2CeJTiURYv3R16XdVY2e4e8m9Ge0ABJjvZwfUdvoI9PfAiyEsRyAgIrWhEjdSmb8hjw6o.4P+RaZm+9tDo2LdIxG7N2kH+IPqoR2ycwLerEueTH6.wMrPn+t1G+EPC8NzAgpLpkK8NVJjQ8jBdoD8YLULuaWVo8OScAsiRA2do391yXgL.uVwSANBvUSig3DaFiEdeuuy+TyW+JGi6Rv9Eje5djx7+e8QJ1yUv9NF6WIE3AXSz3+ZmXVe4ML88wHswCWX.rx9Q7vReLm4NYP+ReEEDwksM1C27Udr+qN4f2hvaGFyNW1D32Mgh0RmAKlUIsCk9W2h+mYSh3Kx3q6hb35m9wSlEfgO3GnpAKyEY6B2iHFOhvAhAlO9T1m1AiAyg3CMiqgiMmqIBzC9OvS5jdiBXvjdYSV.Z4ILk91RXYX56V1uGlj9Pm8ShxHAWKVd6d5LjY6rYQAYD8f2yAtMZ0ZF1ZJCGl0SxddiPYN+N8g6wpxVAOmlbNkGhY5VCR.rvAmJZALqAO.aIsOQ2y4AMn9PncHd4BrzBQSC6iYwU8Gsetj2luEBiwlu5ZIFP8lAvvD7.2ruFp7bNfT37lPhwALe+Xo9bY3Wr4bIe5W5PEzvgIrpk3YZ9wXp46xSvvWH6tL7+iUMLNEKDtZ.6NiAO.1oF.GcqEFUG.BIFbBEKqMQ1kHtmYzcmwQCQkqizGlbQ5kKst.qBv72LcGkozM8SbF8EVIoeu9zlFOzYxu8ZoNXi19VJUONnm2sKLeiQt.vGe29DvO0AfwVmnOFj7+lyA+wHg8igM5S8ikux2b.DVR99ZJv5Vnaornyw3Xhm0Mlxw7fYe.t8q78KqpIDr9rJ3ZypfqOqBtwrJ3lypfaMqB9juaAwaCsy.kruo1.tUXyCLcqsMWHTWlX8eATMR+1.

const var dp = Synth.getDisplayBufferSource("HardcodedMasterFX1");
const var rb = dp.getDisplayBuffer(0);
const var BUFFER_LENGTH = 512;
const var properties = {
    "BufferLength": BUFFER_LENGTH,
    "NumChannels": 1
};
rb.setRingBufferProperties(properties);
const var P1 = Content.getComponent("P1");
const var NUM_WAVEFORMS = 6;
const var waveformBuffers = [];
var time = 0;

for (var i = 0; i < NUM_WAVEFORMS; i++) {
    waveformBuffers[i] = rb.createPath(P1.getLocalBounds(0), [0, BUFFER_LENGTH, -8.0, 8.0], 0.0);
}

var bufferIndex = 0;
var offset = 0;

P1.setTimerCallback(function() {
    var newPath = rb.createPath(
        this.getLocalBounds(0), 
        [offset, BUFFER_LENGTH + offset, -12.0, 12.0],
        time * (bufferIndex + 1) * 0.2
    );
    
    waveformBuffers[bufferIndex] = newPath;
    bufferIndex = (bufferIndex + 1) % NUM_WAVEFORMS;
    offset = (offset + 256) % BUFFER_LENGTH;
    time += 0.1;
    
    this.repaint();
});

P1.setPaintRoutine(function(g) {
    var area = this.getLocalBounds(0);
    g.fillAll(0xFF000000);
    
    // Check if current path has audio data
    var currentPath = waveformBuffers[bufferIndex];
    var hasAudio = currentPath.getBounds(area)[3] > 0.01; // Check if path has any height
    
    for (var i = 0; i < NUM_WAVEFORMS; i++) {
        var baseOffset = (i - NUM_WAVEFORMS/2) * 4;
        var verticalOffset = baseOffset;
        
        if (hasAudio) {
            // Only add floating motion if there's audio - yeah not works. <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
            verticalOffset += Math.sin(time * 0.5 + i * 0.8) * 10;
        }
        
        var offsetArea = [area[0], area[1] + verticalOffset, area[2], area[3] + verticalOffset];
        var colorOffset = i * 0x000011;
        g.setColour(0xFF005577 + colorOffset);
        g.drawPath(waveformBuffers[i], offsetArea, 2);
    }
});

P1.startTimer(16);

ustk

@Mighty23 Why not a detector directly on the audio path, and when a threshold is reached you can authorise the paint routine do do it's job

ustk

@Mighty23 Actually you can copy a display buffer to a standard buffer, so from here it's easy to get the RMS value and take action above a threshold:

HiseSnippet 1779.3oc0WstaaaCEVJIps1qcXcqac+TnX+vYHyvxw1IYCaM2raCZbhQbVZAJJJnknsIBEo.EcR7F5+2i1.1KQeD1av1ghRQRNNYFdaYqFvFlGdt7wCO2XGA2EGFxEFlEOdb.1v79VcGyjC2YHhvL1aWCyO1pMJThE1ZRaON.EFh8LLMW7YJBlEVxH5yu+zsQTDyEmRxv3DNwEuOwmHSo1YyWPnzVHO7wD+LbWay8b4rc3T9H.OKZUwH.4dJZ.9.jhsErLdNJbng4WaUodCLdsU664gVcUuFMZrw5aTaCGT+J0P8bZrQsp0Vq55q02v7NM8HRtnqDIwgfR2l6Mt6P94LsANgDR5QwpENFcAKqIaryPB0qShyIzvvzpSpqZQsq5QVsIdjKom5x9jnMrSkHqSybg7PZobPxIKjpjARs3TOkBTrLE3YlAdKog2Cs55JHAxzcTX6yholhNmbvSuqwBcWxZGNShYxx9nSwsDvh8f0h9HWboFUprhM7yxeWQ3CbqEJsOCIr8Br+d6n.nxCvxcIgATz3sG0uOVzEtUAIeRr4a8Jmm.RmQzdfndAWQtRQFIkuNHFl5.7l.OPfc39AbFrnzSzaqTcQAdfsToUshJ6JvfKuTcmpQvVyY4PrTEFJ1AQo8f3sR8GwbkDNqzxE+4hE75U1kGL9HLxKFOxdf3EJVPNjDV1CIQkO7EfQj8TH4n1c2GeFlVZY6evtIa.ggUjUYJsPtv0dKtXWrKoGlFV5aZTobkLJSfC.9jk.RuSAwDDJQBMFKUMwkmA8cTxbDejDrUJ3GDg9Ak6CoZaQokzoUgk6Qgyn1lCTRqoWJ2g4o1Ib2mKvgxABLlY+sWR0CINUf8Rzhm.cdSJkDDh05ANv6ycQzs4iXdg.nWdE6Z5SpFPIb+550gvH3aUHbZBHTsNXQmJuYJR4Tc1DqXARe6bGskKV.7JSf4W2.TyFf5VOJn9MWB1bdnq3.iQ0QXWIbPTp.95nRLpoMeAAVNRvf+8tq5ntVidi1bRS53TOwl00GYHvIIFvlyNfKwGFGIW7cEsmbq98m5dpTKAmRgHtossNV7FDrDajeOrXEHgkNBeIiPoo709rt9ZeYKG6pyzyvHmsGiHOL.yttJhFwkGf+8i6sKb6qJRFSC3K.KjDEDL2EeFzhRWxrf0t3vSk7fHdiqo.MQjQ69fjBppTOCBTt7dV5zPiKR6hQ1bb1EmS7jPOKKyeEVNDSFLTlrx3p0rgNGbuQTjLe6DU4i3M.+bt50pZxrPhbb1FwS1iYwYqGyT7hyHDenUGhzc3zw3BSAivsw+FXLty7CrZBkpcko.bIqVu5FaCO2lOWq26nsew3HEvlJieeqzld4.wAi72ZjGg2hPiRGLMgiCPLWCvK2Ht0r4uXklxEn5YHvGy6.7WJD4GPwGAmnUr6Q4tm1k7S3qlmFnA71JNJ4NDwfX3v4Ic9NyrO7lmtxP29ZPajTPfbIKvGnmWXmXzo7AKnxp0qqnVqhh5hYdQK9C3S7lNW5vfMcR1LSAgCvxy4hSit7h+ug4c02TgQN421+B.9fuWEJadWKnOc4JFPeT94p5Bj3fb3JLhVGNcbvPNi3pHo4HA5a4CcAkI3Glf8XDgpxJ5NJDpe4cHqKvbzXypnxC3dv+tqdXgwcPphGOzRUADBkUCwbYHcNrlFUo7v5Alywf5ZoCR.aHiBoffOvRICaqcl.v8iJ3AiZq2W6hzPR+X.dnKzAB9EJgZjCkegEhgniCwkywSt38HbMgRtYbjL2K3aVLSc6HlfX8KRJsOQJyjTLTUg7vWjXhlP.sGn3HwU1wHiyYxEYLaT1tZ0XkFitE1K7D0tv7NFmnxOT2sFay4m5ihhwlq9OYKp8X8lej0KQmgiFtN59+yiVCSn4a+LLDZnpH6bsuOaie68y56yBl42mcnqDfvwBDKLfGhcxp4tXexwPCzvbTmPhpSUhbT2EFhhkW0ZR43R4JfQq8yv281LgX0rDgN14TFrNmlZSxzL+8OsICAkv5hgSt2gQgsJ+b1lscFQCwuT0l2Iqfojqlk7yQBO3NzMWqvEm4mDlsNZ9oBxv+TZRcm+W+70O.lDJWa9BIXrKQUDpICdwmpTlBieJjC2GMhJSnlOmrMmwSZUjFAbDFZ7MX.NWf0TOPaIkvaARo7nMOBSwnvL4oe0l6CsJPh1Q0smKegyr6Kto6quzRCWaUgD6+N2a2pgY23zrKNSSy9ONdm+IaukyPg5me3bS+XKnldU6nqa6+6uuuMFC91vF9HWA+st5WOnJXbuHJv4lEMAXAq1p01NFmomXJ0w5CsddqqadUcEAqNuBt57JXs4Uv5yqfMlWAWadEb8+ZAUSFr0HI2Wmm.iK0oodFZS8jRQoLF+IvkXmZv

Mighty23

@ustk said in Help Needed: Detecting Audio Signal in Display Buffer for Oscilloscope Animation:

Actually you can copy a display buffer to a standard buffer, so from here it's easy to get the RMS value and do action above a threshold

Thank you Thank you Thank you, thanks to your example I was able to solve it immediately.

@Christoph-Hart Can this become part of the documentation? Could it be useful as a UI Snippet?

HiseSnippet 2265.3oc2Xs0aiabElx1bSrRRQRwlj9PeXvhV.5XsZEkusotsw2jVaTeQ0xq2.XXXLhbjzDSwgfbjsUBVf9X+A0e.8w9X+ezW5+f1yYFRQNRJabLRR2TtXk4Ly4bly84aXqXgGKIQDaUp7YihXVkde61iBk82sOkGZcvdVkdW6VwrDlzZmQQzjDluUoRy+Bb0RKtfk54e+E6PCngdr7orrNWv8XGxGvk4y1Zq+DOHnI0mcFePApWcqC7Dg6JBDCAMYd6ZVQTuqo8XGSQxly1ZeZReqRel8mu1ZtT+taz0cE2OuieWO1JaTa8U2nNcsm2s6F0bWcE+M5tJnjOpgOWJhaKoRVhUoE1Q3Opcewsg5M3bdBuS.CG3Z0F1Y8zMEA9nIhyZsaedfeqLWThkUI6V4Nr40NrGaeD2mOd9bG2GpVfjyQQGXo4LUu4MTO2hpWsBp2LToREToEzpzGY21KlGIyWA0m2y9fPIKtKEhSEUEMsVyUYQ6cE.EgxpCnWyZFCCFygy50pUg.+rzlkK+rmQ1cXhTLfb31MKCQtDI4FZLwOh7GHpzmp8Xx83IQAzQ6LraWVbaHxBR4I6Si88DfK9HZBH5leo6S.IlKh3NfH7ilhemZFjsyKa1rwoWcXiiewY6Cbrla8hKGEKhXwRNKAV6aJSfmmnkygrvdx9O42YJgJZRNd3.H7EFxBR.JbK+5MKG2oJj6eJOrml+VikrS9lXnZsbg8LyQBlwthAQhPXfySZMgwd7KO5pWs84MZdxoG0F3Z8hKdK8FVWQ7.89hFxEWVbcIaPjdMXI8KU8hYP5jigsYriv9b0Y6eZi16exg6A70HrGOjgpIVQ2j5AojMEw6w73c.ufySWuVUvySfH919eEDyIx9L3+PGg9PkBglPBYLHbVtrRmfpZPp01TMpiRoNHzmcW9jhtcAGZ939zjsG5yEs48BoAv7coAILHICrchCRAWQL7meuoCClZ4kWJM7Ng25B9k.WPvS6QZQk8cZ4h14gBOZvNhgg9IPNUExEPZsYt.4oOuJLI7ykUHJyu7qKWF3FzarsU7tzffNP+ImtCC8jbQnSlVfEFhnQDepjR5FCEHn6xWmHm5OHRAgphdhXZ73YQyEyB.syaX.EEqRjnM.h7TF0OsRHOvCZFRxTtvbJPKFj5graXAfV9GMS..uLxOuKwwTFY1SpM8xHvfXJaIyOStgmLjFv+5bMEev3UH6VzeOk+eLQ3irOOYVgCChtPmrLQ.hrLIa9m5VGCU3uWVwT9Xl3mQbJlCtLwcIXtZUqOlzTOH9L9kISkJHBLoJ07xYzLMe5c72NQZ6XFGWJ3j91xj5qsNxfg8lyfxnVFpFp5pm70kGq4JGZLKBJikNXJK1nVmz1Bm6TwPITpmmy1KKHiwLJDlvDmYFUz6Uupcgyt2NHvQeNcR0NAPQvR2qjnueEyEyl5PSXmL1QwIOchtlOiTGCpqt4TbdC1bFriwbmKpTctPJdy.AjIG1iPC4CT4zDQXvHzrnnIQ3IDelj4IgdcF6j4t.QmifripI7PmzbvZUWChrb0aOG0U2ZSr+48E2VGGt.iGWf8eTu3dIH.yMJck5Yjrxzjb4ztDOHxEO1enTo6pAOttlz1CSazgYmZ20rYsZqs1Fa.6PAArzjr3GSUUFNS2KtRAyqBDvxY80o4wiSWkzXcWVG20WZyrrUHXbrPxNAZ0V9aJuHzPlL4Rc6Ny0vShiEAAf.m0x5s5MvnS3vAcXwU.+WvP1XBArVl.3r+1AvUDeomFXPABEgGDxkmDwB+1PcZkhl.d6kGrGbzBh5Kctb7Hvb6wtAvbqw.tn8drjqkhHEsoPP.TwR0pePFBQJ.1whC3.Wvtkq0c4Px+MaMp3fa49R..tco8lyxpOi2quDG82Tn5mB.J.8U3imhYhGFwXjt.3iM.gh.MCS3xQEuUwOXfjuup3GY2hK85OacbtYniPj3GCcL8pEefcCnBxSlqfKX27K+w4dDE29OUu8um8qfJYEdd0l+wpwpS+eAKjEi9N22v0B+m22qEFcuuV3IdRPENKlFlDIRXtEkba1.9YPZdhwrSvQ8Yxgwr6wjCCMEsdpLp1ZkUl+uitB.o7.2hIrYSVPdOdKnFyPXvXi86HdVY2e4e8m+GeQiPJDvZy.K2+jDOvwf94h6Rqg.B4WgEjtyd55EmFuwEDC8LRZm+Mc4X2G3kiezacWN9mAslLt+9hY5Xa9fn.ViP.1NzeWoi+RngdW5v.Y1rlkdGIBEQ8Egbi.8oLYLuWOlQ9yLMnskR.MWw71SYAL.wTwSANDPPRiA+D6A5Kb+d+sLlY75WYqUWB1uf7y2iTl++qORozbE1ea89WNE3Arm3l+qsiYCD2vT27fzAObgAfLGDwCL9HUyc7vAFeOHDwUoRXOb8Wupze0NG7VDdOnX1YhV.8NITrV5TvXpP5DH7ttM+qYSi3KRqq6fT33k9YfdH.CezOPUCV5qr0CtOQLdDgM3CzeFsrORE5ClCwGpGWCGqOWKzWM3+.OoK5N1gAK5lsXAnkGyj2JhuVUFl9tUo2ACRuu8dIQYSAW.Tb6tpHjNcVaTPDQM3crqUE9mlrVhfQY8jJMuloLke6Av0KkYVv9zjyn7.LR2dXBfE1+jv1.wJvCPJYoiU8bdj9KTMBulAVZgnog7X7KeMNe1PaySgPer9qIaP.J2L.FZmGnlCTPkmyF3BWW6RzJ.tC+Bag5bY3Wr4rgN8I1T3puiRXUMnYV5wDh46RSP2W.6tL7+STML4LVHbUe1c5MrAjo5CGcqXFEG.BIFTBIKqMQ1kHdCqn5NiiFgBW4oOH47zqbZcNVEfwuGzcTlQ2zO1d72Jln+XwjzlFO1d5uhrQGrwouFg5IA871cg48F4B.e7s6S.+Ta.Fach5XPx+aNG7mh.1OE6w.pWr3JO8APXI46plAr6PUKkEsOBGSbstQWNl6LG.vsuxyyTTSwX8GJiq7PYb0GJiq8PYb8GJia7PY74e2Lh2FZ6gRg9aOh2JrUCc25R5KDpJSr9uvGVOTj

ustk

@Mighty23 the methods are already explained in the doc:

https://docs.hise.dev/scripting/scripting-api/displaybuffer/index.html#copyreadbuffer

https://docs.hise.dev/scripting/scripting-api/buffer/index.html#getrmslevel

HISEnberg

@Mighty23 nice work and thank you for sharing the progress you made with this! :)

Help Needed: Detecting Audio Signal in Display Buffer for Oscilloscope Animation

7

1.9k

10.7k

93.2k