Testing before release
-
@David-Healey @cassettedeath 100% agree with this - get it tested by real users.
Even if you test on all OS, in all major DAWs, you'll still mostly be testing the happy paths.
Let other users stress test it for you and give some real-world feedback.
-
Yeah! I’ve had a few user testers. A couple of meetings afterward were super useful and have resulted in UX changes.
I think the documentation and especially video walkthroughs of the installation process will be essential.
A lot of testers I’ve sent it to haven’t registered their serials (as it’s not really all that official yet/maybe they’re just super busy)
Do you still think it’s worth testing on the different Mac OS and DAWs?
-
@cassettedeath said in Testing before release:
Do you still think it’s worth testing on the different Mac OS and DAWs?
Definitely. It's your first line of confirmation that at least it loads and mostly functions in every DAW.
I did my own stupid version of VMs by having a bunch of 40GB SSD, each with a different Mac operating system back to High Sierra, with all major DAWs installed on each one.
Then I booted my MacBook from one SSD at a time and test in all DAWs.
Having other real users test it is a different kind of testing. That's more like stress testing and finding edge cases in specific OS version + DAW version combos. Something you'd never likely find on your own. Too many combinations even for VMs to handle!
-
Thanks for the info!
I’ll probably try make one audio demo for the website in each DAW/OS combination… And then spam the automation etc
A heap of work but I’d rather sort out the bugs now.And then hopefully make some cash to pay some reliable user testers in the future!
-
Is there an affordable way to test all the different DAWS without having to spend all out on the pro versions?
I currently have logic and bitwig, and then some versions of cubase and ableton lite for testing, and of course reaper.
But the other daws I dont own. Do you use demo versions on each operating system to test?
The amount of permutations with vst, audio unit, DAW and operating system is overwhelming to say the least.
I use pluginval as well but im not sure if thats completely realiable.
-
@jeffd My approach was very manual. And I was only testing on Mac. I'm revising my testing plan right now to include Windows.
The only Windows testing my plugins had before was paying users!
But it was pretty much what you said: Logic, some free/cheap 8-track version of Bitwig, Ableton Live Lite, Reaper and FL Studio. I don't think I had a version of Cubase but there was a lite version you can get by buying some cheap hardware, bot sure if that's still a thing.
This was my original testing matrix from 2022, but I didn't keep up this level of scrutiny for long!
-
Im on 2021 MacBook Pro / Sequoia 15 / M1, Abelton & Logic if you need help testing on this side.
-
@jeffd said in Testing before release:
Do you use demo versions on each operating system to test?
I use demo/trial versions when needed. But I don't test all DAWs. I test Pro-Tools for AAX. Reaper, Cubase, FL Studio, StudioOne, Qtractor, Ardour, Logic.
-
@David-Healey said in Testing before release:
Qtractor, Ardour,
wow, ive never even heard of these two.
-
@jeffd said in Testing before release:
wow, ive never even heard of these two.
Unless you're releasing for Linux then don't worry about them
-
@David-Healey ah ok.
im guessing after ableton, logic, cubase, and reaper--
fl studio and studio one are the next 2 you should test with.and looks like a fl studio trial can test thirdparty plugins, but studio one you woudd need a pro license or subscription. But I havent tried those 2 so I could be wrong.
and of course pro tools if you are doing AAX.
