HISE Logo Forum
    • Categories
    • Register
    • Login

    added scriptable FFT

    Scheduled Pinned Locked Moved Scripting
    28 Posts 6 Posters 2.2k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • d.healeyD
      d.healey @Christoph Hart
      last edited by

      This post is deleted!
      1 Reply Last reply Reply Quote 0
      • d.healeyD
        d.healey
        last edited by

        What about a resynthesis module? :D

        ? 1 Reply Last reply Reply Quote 1
        • ?
          A Former User @d.healey
          last edited by

          @d-healey with the results I got from DDSP I'm surprised more developers aren't looking into resynthesis more... we'd never have to sample a round robin again 😂

          d.healeyD Christoph HartC 2 Replies Last reply Reply Quote 2
          • d.healeyD
            d.healey @A Former User
            last edited by

            @iamlamprey It would be nice if we never had to sample anything again.

            1 Reply Last reply Reply Quote 2
            • Christoph HartC
              Christoph Hart @A Former User
              last edited by Christoph Hart

              @iamlamprey What's DDSP? Nevermind it's more googleable than I thought.

              ? 1 Reply Last reply Reply Quote 1
              • ?
                A Former User @Christoph Hart
                last edited by

                @christoph-hart Yep :) The style-transfer is particularly appealing, I wouldn't be surprised if you can legally train it with other people's sample libraries since it's 100% synthesis.

                Not a lawyer but ;)

                1 Reply Last reply Reply Quote 0
                • d.healeyD
                  d.healey
                  last edited by

                  @Christoph-Hart I thought I'd have a go at this fft thing. Following your instructions above and the docs I load a sample from the sampler into a buffer. If I uncomment the process line in this script HISE will just crash.

                  const var Sampler1 = Synth.getSampler("Sampler1");
                  
                  reg sound = Sampler1.createSelection(".*")[0];
                  
                  const buf = sound.loadIntoBufferArray();
                  
                  const fft = Engine.createFFT();
                  fft.setEnableSpectrum2D(true);
                  fft.setWindowType(fft.BlackmanHarris);
                  
                  fft.setMagnitudeFunction(function(data, offset)
                  {
                  	var max = 0.0;
                  	
                  	for(s in data)
                  	{
                  		max = Math.max(max, s);
                  	}
                  	
                  	Console.print("The max value at " + offset + " is " + Engine.getDecibelsForGainFactor(max));
                  	
                  }, true);
                  
                  fft.prepare(1024, 2);
                  
                  //fft.process(buf);
                  
                  d.healeyD 1 Reply Last reply Reply Quote 0
                  • d.healeyD
                    d.healey @d.healey
                    last edited by

                    @d-healey Bump

                    OrvillainO 1 Reply Last reply Reply Quote 0
                    • OrvillainO
                      Orvillain @d.healey
                      last edited by

                      @d-healey Did you ever resolve this Dave?

                      d.healeyD 1 Reply Last reply Reply Quote 0
                      • d.healeyD
                        d.healey @Orvillain
                        last edited by

                        @Orvillain Not as far as I recall

                        OrvillainO 1 Reply Last reply Reply Quote 0
                        • OrvillainO
                          Orvillain @d.healey
                          last edited by

                          @d-healey

                          I'm not running into a crash, primarily because I'm not enabling the 2D Spectrum.

                          But a very simple example like this:

                          Content.makeFrontInterface(600, 600);
                          
                          const var AudioLoopPlayer1 = Synth.getAudioSampleProcessor("Audio Loop Player1");
                          const var audioFileObj = AudioLoopPlayer1.getAudioFile(0);
                          const var sampleRate = audioFileObj.getSampleRate();
                          const var audioData = audioFileObj.getContent(); // [channel][samples]
                          const var fft = Engine.createFFT();
                          fft.prepare(1024, 1); // 1 channel, 1024 sample size
                          fft.setWindowType(fft.Hann);
                          fft.setOverlap(0.75);
                          
                          fft.setMagnitudeFunction(function(data, offset)
                          {
                              Console.print("Frame starting at sample: " + offset);
                          }, false);
                          
                          audioFileObj.setContentCallback(function()
                          {
                              if (audioData.length == 0 || audioData[0].length == 0)
                              {
                                  Console.print("No audio loaded.");
                                  return;
                              }
                          
                              var buffer = audioData[0]; // Mono: use first channel
                              fft.process([buffer]); // Wrap in array of channels
                          });
                          
                          

                          Just isn't working. I get nothing printed in the console. It is as if the setMagnitudeFunction is not being executed.

                          I have IPP enabled for Hise.

                          @Christoph-Hart any thoughts????

                          full snippet:

                          HiseSnippet 1333.3ocyWs0SabDEdWLaTw8hZZ4g93HTevzhb7xcHBU.ClfZLXgoIUBPQi2cV6oLdlUyLKD2F9O1eJ8o9Z6Y1K16FrHNtIQ0Ojry49bNm46bnkT3QTJgzxt74CBIV1egS6Abcu58vTt0wGXY+UNMwJMQhRHs+fPrRQ7srsKcjgf8byZE+6u9o8wLL2iLhjk0KDTOxyo8o5QTas6OSYrFXex4z94jd0cO1SvqKXhHHdJ4TyJD6cMtK4DrQrYbrdFV0yx9GbVNni2xAarxlta4t0Fqtkq6Za1Y0ZK64u1Vqrx5q3Gr7l302DT5QG5S0BYaMVSTfQ2W3OncOws7DG7Bph1gQLGbsZCdNgrU8dTleqrjixxx1o0nTUojT07NMo9zgzGkx95XFnQZjOoYOyCERtuGgjctPZ1jP5wNs8jzP8HNl34ycNlCUv.LTaxGJIxZMyeWxot.jfqq1GeMogDNLTiJqWq1RH3eV7okKC0GkFcCVh1KxmJdtPD1hgGPjtncPwMNU6Rzw7Zi6GxHCCjJKDSEYTAkpyBfIGYQrgeCJibZmeCr1a6fgF1HRkZETUE6qyfDJnXd6XTp8PlUtu+N.qwiQmzzAn.5IOAcgWOLmSXWcQhiTWkyLAAZv.Gx6R4jpdRB3mFMN23JfS0PIIDKIUbqs7pKgbSrmKJ0f.EfdZ3iTzemDqjhneIk6Kt0TTqXn7LP7TKBLO8FhjgCqTq5FqYpJojah6xo5HeRiHtmlJ3UBx9vGtlKgDAAfXKV9OJifevcTIXDHDovMcgFR3UFRowRMk2Eg0og01nEP+XlpOs7cKgBvLEw32BYM0vrVcLi0Ad4Nx8YtjFfpLLsWkQ3c08P6rCpF5MuYT83hZWkm2hwplXfwD2mHRzDwD.hheUSSUlnRhNRxSNeW43+yTx5DED..Z6TvkwUllBtXaTjhfBnRn.mVmh0LobF2NW4hDSbUR87kRbHhxQXoDO.RUYpoJemonklEPB9IBM4zjzwbkuqL5sYEDLVdl7pTvXD4XYaPQkOjhU3Q86PjKAWdVDYnf.pQQnnGMYPQdIk4bBJ3GCcdmFRRO2Pv7MPLluuOvkUZeB70ubrI6avxRoAxERfNPS3Xe.4FX5QBx1bNGPTWqEgv7i6A6A.tB+HFVWDE1LeJkAjOJ.8Yf23JpdP94WuGPy0dPn4IMDerSKp1q23iwYFSLBYpOFwX5.suz4Pnq1SOJ.m0owuNsSulb2+sYteHlOIYv07N2enwCrAxeNoafDNwaf.yz7fxCIeaRbM6bIfvA.k4ckILOjig6uedxmIDZyy67a.kLTpsAsELeAoI.5tpnELXrIKAU04nsu7.3QRGwqu7nHJXfynAA0ExvH0kMoWSTWdvwUp4tX0aw2X02jdbrM1nO90lOQcbrXPbF65Ldr3.2OiucohE4YmLbghO2xI+X1cY1OVqSM6j0P9NPn9+ODSgs+lKKFaSMsUGxugvfGQww32..mA3HlNiZwWPlIdg8DbpWwlPsj1sKQlO1G6EZOsFdHLhx76dFgQvpbc6e+tOG1MBKe62QeT2Ddr0quyIIbQlm8n+K0sOosYO3XhRSzXhO3w6zOx3CSnblHxrmZSLznZ.1NIpea.01iTOc2KCX1LlkMRNWKCqqMg6Ge3efeoLcMmsSY5lw7ShO5i8jhW4krnloW9yho.2adLf+bve9MbF4ZEu7V9Zbe.U7UddEM08Tb4oUwUlVEWcZUbsoUw0mVE2XZUby2shlgV6EoE8SdxZY0r0gwa6ZaObIA6RV+KClD4JA
                          
                          OrvillainO 1 Reply Last reply Reply Quote 0
                          • OrvillainO
                            Orvillain @Orvillain
                            last edited by

                            @Orvillain

                            Even a minimal version like this doesn't work:

                            const var testBuffer = [];
                            
                            for (i = 0; i < 2048; i++)
                                testBuffer.push(Math.sin(i * 0.1));
                            
                            const var testFFT = Engine.createFFT();
                            testFFT.prepare(1024, 1);
                            testFFT.setWindowType(testFFT.Hann);
                            testFFT.setOverlap(0.5);
                            
                            testFFT.setMagnitudeFunction(function(data, offset)
                            {
                                Console.print("OK - frame at: " + offset);
                            }, false);
                            
                            testFFT.process([testBuffer]);
                            
                            

                            fft.process() is either silently failing, or not being called properly because of some kind of HISE setup thing, is my suspicion.

                            1 Reply Last reply Reply Quote 0
                            • OrvillainO
                              Orvillain
                              last edited by

                              Okay, I rejigged Chris's example above. This actually works:

                              // Create a dummy signal with two sine waves
                              const var signal = Buffer.create(4096);
                              
                              reg uptime = 0.0;
                              
                              for(s in signal)
                              {
                                  s = 0.3 * Math.sin(uptime);
                                  s += 0.7 * Math.sin(uptime * 2.0);
                                  uptime += 0.1;
                              }
                              
                              // Create a FFT object. If you right click on it in the 
                              // script watch table and view the popup you'll see a spectrogram
                              // for the data passed in
                              const var fft = Engine.createFFT();
                              
                              // creates a spectrogram image (required for 
                              // the debug popup but might be used later to draw
                              // on a panel. */
                              fft.setEnableSpectrum2D(true);
                              
                              // Set the window type for the processing
                              fft.setWindowType(fft.BlackmanHarris);
                              
                              // Give the fft a function that will be called for 
                              // each signal chunk. `data` will contain either a 
                              // buffer or an array of buffers containing the transformed
                              // FFT signal and offset will contain the index of the first
                              // sample in the chunk (here it will be 0, 1024, 2048 and 3072
                              // because the processing size is 1024
                              fft.setMagnitudeFunction(
                              
                              (function(data, offset)
                              {
                              	var max = 0.0;
                              
                              	for(s in data)
                              	{
                              		max = Math.max(max, s);
                              	}
                              	
                              	Console.print("The max value at " + offset + " is " + Engine.getDecibelsForGainFactor(max));
                              }), true);
                              
                              fft.prepare(1024, 1);
                              
                              // Process the buffer with the function above.
                              fft.process(signal);
                              
                              

                              Prepare needs to be called AFTER setting the magnitude function. Not before.

                              OrvillainO 1 Reply Last reply Reply Quote 1
                              • OrvillainO
                                Orvillain @Orvillain
                                last edited by

                                Another thing to bear in mind - it seems to me that you cannot run an FFT over a typical array. It HAS to be a buffer.

                                1 Reply Last reply Reply Quote 0
                                • First post
                                  Last post

                                19

                                Online

                                1.8k

                                Users

                                12.0k

                                Topics

                                104.8k

                                Posts