HISE Logo Forum
    • Categories
    • Register
    • Login

    added scriptable FFT

    Scheduled Pinned Locked Moved Scripting
    28 Posts 6 Posters 2.2k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • ?
      A Former User @d.healey
      last edited by

      @d-healey with the results I got from DDSP I'm surprised more developers aren't looking into resynthesis more... we'd never have to sample a round robin again 😂

      d.healeyD Christoph HartC 2 Replies Last reply Reply Quote 2
      • d.healeyD
        d.healey @A Former User
        last edited by

        @iamlamprey It would be nice if we never had to sample anything again.

        Libre Wave - Freedom respecting instruments and effects
        My Patreon - HISE tutorials
        YouTube Channel - Public HISE tutorials

        1 Reply Last reply Reply Quote 2
        • Christoph HartC
          Christoph Hart @A Former User
          last edited by Christoph Hart

          @iamlamprey What's DDSP? Nevermind it's more googleable than I thought.

          ? 1 Reply Last reply Reply Quote 1
          • ?
            A Former User @Christoph Hart
            last edited by

            @christoph-hart Yep :) The style-transfer is particularly appealing, I wouldn't be surprised if you can legally train it with other people's sample libraries since it's 100% synthesis.

            Not a lawyer but ;)

            1 Reply Last reply Reply Quote 0
            • d.healeyD
              d.healey
              last edited by

              @Christoph-Hart I thought I'd have a go at this fft thing. Following your instructions above and the docs I load a sample from the sampler into a buffer. If I uncomment the process line in this script HISE will just crash.

              const var Sampler1 = Synth.getSampler("Sampler1");
              
              reg sound = Sampler1.createSelection(".*")[0];
              
              const buf = sound.loadIntoBufferArray();
              
              const fft = Engine.createFFT();
              fft.setEnableSpectrum2D(true);
              fft.setWindowType(fft.BlackmanHarris);
              
              fft.setMagnitudeFunction(function(data, offset)
              {
              	var max = 0.0;
              	
              	for(s in data)
              	{
              		max = Math.max(max, s);
              	}
              	
              	Console.print("The max value at " + offset + " is " + Engine.getDecibelsForGainFactor(max));
              	
              }, true);
              
              fft.prepare(1024, 2);
              
              //fft.process(buf);
              

              Libre Wave - Freedom respecting instruments and effects
              My Patreon - HISE tutorials
              YouTube Channel - Public HISE tutorials

              d.healeyD 1 Reply Last reply Reply Quote 0
              • d.healeyD
                d.healey @d.healey
                last edited by

                @d-healey Bump

                Libre Wave - Freedom respecting instruments and effects
                My Patreon - HISE tutorials
                YouTube Channel - Public HISE tutorials

                OrvillainO 1 Reply Last reply Reply Quote 0
                • OrvillainO
                  Orvillain @d.healey
                  last edited by

                  @d-healey Did you ever resolve this Dave?

                  d.healeyD 1 Reply Last reply Reply Quote 0
                  • d.healeyD
                    d.healey @Orvillain
                    last edited by

                    @Orvillain Not as far as I recall

                    Libre Wave - Freedom respecting instruments and effects
                    My Patreon - HISE tutorials
                    YouTube Channel - Public HISE tutorials

                    OrvillainO 1 Reply Last reply Reply Quote 0
                    • OrvillainO
                      Orvillain @d.healey
                      last edited by

                      @d-healey

                      I'm not running into a crash, primarily because I'm not enabling the 2D Spectrum.

                      But a very simple example like this:

                      Content.makeFrontInterface(600, 600);
                      
                      const var AudioLoopPlayer1 = Synth.getAudioSampleProcessor("Audio Loop Player1");
                      const var audioFileObj = AudioLoopPlayer1.getAudioFile(0);
                      const var sampleRate = audioFileObj.getSampleRate();
                      const var audioData = audioFileObj.getContent(); // [channel][samples]
                      const var fft = Engine.createFFT();
                      fft.prepare(1024, 1); // 1 channel, 1024 sample size
                      fft.setWindowType(fft.Hann);
                      fft.setOverlap(0.75);
                      
                      fft.setMagnitudeFunction(function(data, offset)
                      {
                          Console.print("Frame starting at sample: " + offset);
                      }, false);
                      
                      audioFileObj.setContentCallback(function()
                      {
                          if (audioData.length == 0 || audioData[0].length == 0)
                          {
                              Console.print("No audio loaded.");
                              return;
                          }
                      
                          var buffer = audioData[0]; // Mono: use first channel
                          fft.process([buffer]); // Wrap in array of channels
                      });
                      
                      

                      Just isn't working. I get nothing printed in the console. It is as if the setMagnitudeFunction is not being executed.

                      I have IPP enabled for Hise.

                      @Christoph-Hart any thoughts????

                      full snippet:

                      HiseSnippet 1333.3ocyWs0SabDEdWLaTw8hZZ4g93HTevzhb7xcHBU.ClfZLXgoIUBPQi2cV6oLdlUyLKD2F9O1eJ8o9Z6Y1K16FrHNtIQ0Ojry49bNm46bnkT3QTJgzxt74CBIV1egS6Abcu58vTt0wGXY+UNMwJMQhRHs+fPrRQ7srsKcjgf8byZE+6u9o8wLL2iLhjk0KDTOxyo8o5QTas6OSYrFXex4z94jd0cO1SvqKXhHHdJ4TyJD6cMtK4DrQrYbrdFV0yx9GbVNni2xAarxlta4t0Fqtkq6Za1Y0ZK64u1Vqrx5q3Gr7l302DT5QG5S0BYaMVSTfQ2W3OncOws7DG7Bph1gQLGbsZCdNgrU8dTleqrjixxx1o0nTUojT07NMo9zgzGkx95XFnQZjOoYOyCERtuGgjctPZ1jP5wNs8jzP8HNl34ycNlCUv.LTaxGJIxZMyeWxot.jfqq1GeMogDNLTiJqWq1RH3eV7okKC0GkFcCVh1KxmJdtPD1hgGPjtncPwMNU6Rzw7Zi6GxHCCjJKDSEYTAkpyBfIGYQrgeCJibZmeCr1a6fgF1HRkZETUE6qyfDJnXd6XTp8PlUtu+N.qwiQmzzAn.5IOAcgWOLmSXWcQhiTWkyLAAZv.Gx6R4jpdRB3mFMN23JfS0PIIDKIUbqs7pKgbSrmKJ0f.EfdZ3iTzemDqjhneIk6Kt0TTqXn7LP7TKBLO8FhjgCqTq5FqYpJojah6xo5HeRiHtmlJ3UBx9vGtlKgDAAfXKV9OJifevcTIXDHDovMcgFR3UFRowRMk2Eg0og01nEP+XlpOs7cKgBvLEw32BYM0vrVcLi0Ad4Nx8YtjFfpLLsWkQ3c08P6rCpF5MuYT83hZWkm2hwplXfwD2mHRzDwD.hheUSSUlnRhNRxSNeW43+yTx5DED..Z6TvkwUllBtXaTjhfBnRn.mVmh0LobF2NW4hDSbUR87kRbHhxQXoDO.RUYpoJemonklEPB9IBM4zjzwbkuqL5sYEDLVdl7pTvXD4XYaPQkOjhU3Q86PjKAWdVDYnf.pQQnnGMYPQdIk4bBJ3GCcdmFRRO2Pv7MPLluuOvkUZeB70ubrI6avxRoAxERfNPS3Xe.4FX5QBx1bNGPTWqEgv7i6A6A.tB+HFVWDE1LeJkAjOJ.8Yf23JpdP94WuGPy0dPn4IMDerSKp1q23iwYFSLBYpOFwX5.suz4Pnq1SOJ.m0owuNsSulb2+sYteHlOIYv07N2enwCrAxeNoafDNwaf.yz7fxCIeaRbM6bIfvA.k4ckILOjig6uedxmIDZyy67a.kLTpsAsELeAoI.5tpnELXrIKAU04nsu7.3QRGwqu7nHJXfynAA0ExvH0kMoWSTWdvwUp4tX0aw2X02jdbrM1nO90lOQcbrXPbF65Ldr3.2OiucohE4YmLbghO2xI+X1cY1OVqSM6j0P9NPn9+ODSgs+lKKFaSMsUGxugvfGQww32..mA3HlNiZwWPlIdg8DbpWwlPsj1sKQlO1G6EZOsFdHLhx76dFgQvpbc6e+tOG1MBKe62QeT2Ddr0quyIIbQlm8n+K0sOosYO3XhRSzXhO3w6zOx3CSnblHxrmZSLznZ.1NIpea.01iTOc2KCX1LlkMRNWKCqqMg6Ge3efeoLcMmsSY5lw7ShO5i8jhW4krnloW9yho.2adLf+bve9MbF4ZEu7V9Zbe.U7UddEM08Tb4oUwUlVEWcZUbsoUw0mVE2XZUby2shlgV6EoE8SdxZY0r0gwa6ZaObIA6RV+KClD4JA
                      
                      OrvillainO 1 Reply Last reply Reply Quote 0
                      • OrvillainO
                        Orvillain @Orvillain
                        last edited by

                        @Orvillain

                        Even a minimal version like this doesn't work:

                        const var testBuffer = [];
                        
                        for (i = 0; i < 2048; i++)
                            testBuffer.push(Math.sin(i * 0.1));
                        
                        const var testFFT = Engine.createFFT();
                        testFFT.prepare(1024, 1);
                        testFFT.setWindowType(testFFT.Hann);
                        testFFT.setOverlap(0.5);
                        
                        testFFT.setMagnitudeFunction(function(data, offset)
                        {
                            Console.print("OK - frame at: " + offset);
                        }, false);
                        
                        testFFT.process([testBuffer]);
                        
                        

                        fft.process() is either silently failing, or not being called properly because of some kind of HISE setup thing, is my suspicion.

                        1 Reply Last reply Reply Quote 0
                        • OrvillainO
                          Orvillain
                          last edited by

                          Okay, I rejigged Chris's example above. This actually works:

                          // Create a dummy signal with two sine waves
                          const var signal = Buffer.create(4096);
                          
                          reg uptime = 0.0;
                          
                          for(s in signal)
                          {
                              s = 0.3 * Math.sin(uptime);
                              s += 0.7 * Math.sin(uptime * 2.0);
                              uptime += 0.1;
                          }
                          
                          // Create a FFT object. If you right click on it in the 
                          // script watch table and view the popup you'll see a spectrogram
                          // for the data passed in
                          const var fft = Engine.createFFT();
                          
                          // creates a spectrogram image (required for 
                          // the debug popup but might be used later to draw
                          // on a panel. */
                          fft.setEnableSpectrum2D(true);
                          
                          // Set the window type for the processing
                          fft.setWindowType(fft.BlackmanHarris);
                          
                          // Give the fft a function that will be called for 
                          // each signal chunk. `data` will contain either a 
                          // buffer or an array of buffers containing the transformed
                          // FFT signal and offset will contain the index of the first
                          // sample in the chunk (here it will be 0, 1024, 2048 and 3072
                          // because the processing size is 1024
                          fft.setMagnitudeFunction(
                          
                          (function(data, offset)
                          {
                          	var max = 0.0;
                          
                          	for(s in data)
                          	{
                          		max = Math.max(max, s);
                          	}
                          	
                          	Console.print("The max value at " + offset + " is " + Engine.getDecibelsForGainFactor(max));
                          }), true);
                          
                          fft.prepare(1024, 1);
                          
                          // Process the buffer with the function above.
                          fft.process(signal);
                          
                          

                          Prepare needs to be called AFTER setting the magnitude function. Not before.

                          OrvillainO 1 Reply Last reply Reply Quote 1
                          • OrvillainO
                            Orvillain @Orvillain
                            last edited by

                            Another thing to bear in mind - it seems to me that you cannot run an FFT over a typical array. It HAS to be a buffer.

                            1 Reply Last reply Reply Quote 0
                            • First post
                              Last post

                            19

                            Online

                            1.8k

                            Users

                            12.0k

                            Topics

                            104.8k

                            Posts