Virtual Machine solutions for cross-platform development
-
I was thinking of a solution to the difficulties of building the latest HISE version and an app export of one´s instruments on all 3 platforms (win, linux, mac). It took extensive digging to just install the Mac version for me, even though I am an advanced mac user.
The solution I´ve been thinking of is a virtual machine environment, like a macOS or Linux / Windows made with VirtualBox. It´s free and one can install an os from a ISO file and compress the completed and pre-installed operating system´s files using 7zip for small sharing sizes.
(this will just be the pros/cons I can think of and my thoughts on easier dev down the line, using VMs. Feel free to punish my notes :) )
Virtual Machines / Virtual Operatingsystems / VMs
If you are not familiar with VMs (virtual machines, which most of you probably are) the host operating system is the machine that VirtualBox is running on. The guest operating system is the virtual one running in a window displaying the entire operating system as an app on your computer.
As a virtual operating system can be set to have a minimal size and only use the amount actually used by the guest OS. That way, once the required software has been installed (IPP, xcode/MS Visual Studio/Linux equivalent, VST/AU libs) and HISE has been built, the VM can then be shut down and compressed into a .7zip document and posted to a place on the internet where it will remain for the longest time possible. Either to a torrent network or to a free account of a sync app that do not delete inactive files. Such as sync.com.
Pros:
- anyone could spin up HISE in a newer version than he/she has on his normal machine, without having to upgrade or change files
- anyone could test HISE out in a proper environment, discarding audio drivers (which are normally the issue in VMs)
- anyone could build HISE in ANY of the 3 platforms that it is supposed to be able to be built on, without the extensive knowledge of handling... say a linux environment coming from Windows (which would most likely be quite a journey)
- testing HISE builds in revisions would be easier, given you got enough space on your SSD (or HDD :/) to keep revisions and versions
- exchanging builds of an entire build environment is possible and usable to all, with the proper documentation.
- when working with others, a developer could build on all systems for a client to test out directly
Cons:
- getting into Virtual Machines could prove to be more lengthy than building HISE natively, for some people.
- a virtual machine does not constitute a complete computer in terms of audio input/output for real-time testing
- installing a virtual machine is not always straight forward and there are many formats available
- downloading torrent files from other builds online would be a risk in general use, as malware could be installed. (for existing torrents that are not from this community)
So, I know lots of you probably already use either a virtualized environment or tripple-boot your systems or have separate ones on either different hard drives or systems entirely. But for a lone gunman with nothing but one laptop and even a single partitioned system form Apple stock-style, that could prove to be impossible.
I use virtual operating systems along side the partitioning on my machines to run multiple versions of macOS, but I have only once been able to install a Linux virtual machine. I guess settings matter when installing a VM but the point here is to bypass as many of the unnecessary steps in that as possible, by making one basic file, a fresh OS for each platform, for all to use and then build on if or when you need a new build or try out your work in a specific version of windows, mac or Linux.
Moreover, this is not something one person will be able to maintain, it would just be a growing number of verifiable builds down the line, as new updates come all the time to all platforms.
Licensing and stuff
Your Macintosh computer comes with your own license that is downloaded from your own mac from your apple account, making that available to all would be risky for the person in mind, the account owner. I do not think, however, that the DMG file inside the macOS installer retains any info of who you are as the originating account holder. The OS is most likely entirely "retail fresh" if you make a bootable installer using some tool, or the terminal. And thus the OS could be delivered as that same DMG to the community for use with a VM of your choice or using a pre-packaged VDI made in VirtualBox. I´m sure that this could be found on other forums as a trustable starting point.
Apple do not allow you to download older versions of macOS if you did not click on the download link in App Store application during the time when it was shipping or was in use and development, so unless that was the case there is no legal way for you to download that older version form Apple itself. Insane, I know. But if you did you can re-download it again as an .app installation file and test your product out in earlier versions of macOS and something like Logic Pro 10.3 or below.
Linux Distributions
These are things I know little about, apart from the Linux Manjaro Gnome I just installed on an older PC a few days ago. I know that most Linux distros share a lot, but that there are major releases and distros that are more pivotal in their content directions, like Suse, Debian, Ubuntu (is Debian?
). Well you see, where would a Mac-owner begin? Or a Windows dev? A preset environment would no doubt be very usable for a build for me and here´s where the licensing is already a non-issue.Anyone in possession of a VM for a working build environment for a semi-new to new Linux environment is more than welcome to PM me.
Any thoughts?
-
I don't quite understand. Are you suggesting that someone should setup a virtual machine with 3 OS for anyone to connect to build their plugins?
-
@d-healey said in Virtual Machine solutions for cross-platform development:
I don't quite understand. Are you suggesting that someone should setup a virtual machine with 3 OS for anyone to connect to build their plugins?
@d-healey No, that´s not something I think exists or is possible to make. (a multi-boot virtual machine) And I do not mean to spin one up that anyone could just Terminal into or connect to. (Although that would be amazing)
I am suggesting that perhaps sharing of virtual machines could be beneficial. Anyone who uses say MacOS or Windows has their normal machine would be able to just spin up a version of Linux that has been pre-installed with the proper software installed in already. To save time. Not a combo, just different VMs.
-
Might hit a problem with Apple Developer IDs. And PACE credentials if exporting AAX.
-
@d-healey said in Virtual Machine solutions for cross-platform development:
Might hit a problem with Apple Developer IDs. And PACE credentials if exporting AAX.
Good point. I am unfamiliar with PACE or exporting for the different formats still, so perhaps that could persist. But the operating system itself and the basic HISE files and pre-built environment would be there from the start. Apple dev id could perhaps be added after, not be added upon installation. The download from Apple Developers is probably not signed or tagged as to what user downloaded it, just that it´s available for a logged in user only.
I guess the most complex environment could still be Linux, where a VM would save some headaches for less than experts, like me. Of course, on Linux there might already be VMs out there for distros, made by other forums. Will check that. Also checking your videos about compiling on Linux, of course.
Also, the version is an issue when supplying someone else with HISE in a prepared package like that. There are only so many actual releases so a VM could be used for a year or so before it would become irrelevant to use with the latest version.
The VM would just be a good starting point, from which one could update both the guest OS and HISE.
-
I thought about doing this a couple years back. @Christoph-Hart convinced me otherwise due to instability .
@Christoph-Hart said in known compatible MAC operating systems:
VirtualBox will 100% not work correctly. Even if it loads, there are some many things (like the native UI rending or core audio drivers) that will make it a terrible experience.
-
@dustbro Okay, to use it for a development system is not advisable. I am merely suggesting using VMs as a way to spin up the other two systems (the ones you are not using or have as a native computer) to render the final app on those systems. Never to use as your development OS.
For example:
Scenario 1:
- I use mac and I have a physical MacBook
- when I finish and want to render the instrument I got on my mac in my development project, I make a zip of the project folder and transfer it via shared folders to my VM, or use DropBox to automatically do so.
- in my VM (windows or linux), I run the render script to finish the instrument, never once needing to start hise itself.
Scenario 2:
- I use Linux as my main computer operating system
- when I finish my instrument project I do the same as in scenario 1 but using one VM for Windows and another one for macOS.
Done, in theory. I will now have the instrument rendered on all 3 platforms without having to leave one single computer.
As @Christoph-Hart mentions, using it on mac in a virtualized environment is a "terrible experience", referring to "UI rending or core audio drivers", meaning when running HISE as an .app in the design of an instrument. But if we use HISE on the other systems (the VM based systems) only to build the end-result (our vst-plugins for that guest OS) we won´t have problems with drivers as we would while designing in that same VM. That´s the part I am guessing that @Christoph-Hart was referring to.
@d-healey is using nothing but a terminal command in the video "Tinkering with the HISE source code" (video below, at 11:00) to compile HISE on Linux. Is it possible to build our entire already designed project file through such a command as well? In case of yes, could we also do that on other systems?
-
Give it a try and let us know what happens. I ended up building a dual boot Win10/hackintosh, which is working great so far.
Both OS share a "work drive" formatted exFAT so I can work with the same data.Dan Korneff - Producer / Mixer / Audio Nerd
-
@dustbro said in Virtual Machine solutions for cross-platform development:
Give it a try and let us know what happens. I ended up building a dual boot Win10/hackintosh, which is working great so far.
Both OS share a "work drive" formatted exFAT so I can work with the same data.and I just followed Christophs advice and bought a second hand mac mini...very cheap
-
Yes, having a dedicated machine for each OS is the most stable option.
Is it possible to build our entire already designed project file through such a command as well? In case of yes, could we also do that on other systems?
Yes, you can use HISE as command line tool and create a shell script that compiles all versions and builds an installer automatically. This is required for any automated build process, so you can run CI tools (eg. Jenkins).
I make a zip of the project folder and transfer it via shared folders to my VM, or use DropBox to automatically do so.
NONONONONONO :) Use Git.
-
@Christoph-Hart said in Virtual Machine solutions for cross-platform development:
Yes, you can use HISE as command line tool and create a shell script that compiles all versions and builds an installer automatically. This is required for any automated build process, so you can run CI tools (eg. Jenkins).
Okay, I have not used such CI tools before but Jenkins (which I just saw a nice introduction to) seems great. However also more time to get into more tech I don´t already use. But If I was to do that, how would you set that up in detail? (Assuming you have 3 physical machines, for simplicity sake)
Bare in mind, I don´t use git or have ever used it, I do know the terminal and shell script (bash on mac/linux).
If I got it correctly:
- use git to commit/sync? to all machines, which are also running git in their command line environments
- then use Jenkins on my main machine to push the command to build the latest committed code on the other machines, Jenkins (which is also installed on all other machines) would execute and build the latest shell-script on each machine.
Right?
(also, since shell-script does not natively run on windows, what´s the tool for windows to cover or handle that?)
-
@dustbro said in Virtual Machine solutions for cross-platform development:
Give it a try and let us know what happens. I ended up building a dual boot Win10/hackintosh, which is working great so far.
Both OS share a "work drive" formatted exFAT so I can work with the same data.Okay, seems like a great option, using a hack is much easier today than just a few years ago. So you do not build at all for Linux right now?
@Lindon said in Virtual Machine solutions for cross-platform development:
and I just followed Christophs advice and bought a second hand mac mini...very cheap
Mac minis are a great option and they will be sufficient for both windows and macOS. If you got money for only one, get a mac and dual-boot/bootcamp. An older mac will ensure backwards compatibility as a mac can only be used with the OS version it was shipped with and up. So a brand new mac = nothing but Catalina and up. A pretty big "cons" for a new Apple computer.
-
Mac minis are a great option and they will be sufficient for both windows and macOS
An GNU/Linux. I've set mine up with a triple boot.
If I got it correctly:
use git to commit/sync? to all machines, which are also running git in their command line environmentsI wouldn't use git as a syncing tool (I did, but not any more). I use git to manage code versions and I keep everything up to date on my main system. I then use network file sharing so that the Mac Mini has access to the HISE project files on my main system.
This means I don't need to manage multiple versions of HISE or my projects, everything is on the main system.
-
@d-healey said in Virtual Machine solutions for cross-platform development:
An GNU/Linux. I've set mine up with a triple boot.
Cool, nice to hear it can be done on Linux as well.
I then use network file sharing so that the Mac Mini has access to the HISE project files on my main system.
This means I don't need to manage multiple versions of HISE or my projects, everything is on the main system.So the networking lets you just get in and copy the files over to the system you´re working on, or you actually run it from the original place?
And the HISE files are not specific to any OS, I guess, so the whole HISE source folder is just read in place then? From the original machine?
-
@d-healey said in Virtual Machine solutions for cross-platform development:
This means I don't need to manage multiple versions of HISE or my projects, everything is on the main system.
And do you build it by opening HISE as an application and running the build via the app on each system or use a terminal script on all machines? (just curious)
-
So the networking lets you just get in and copy the files over to the system you´re working on, or you actually run it from the original place?
And the HISE files are not specific to any OS, I guess, so the whole HISE source folder is just read in place then? From the original machine?I run the files from the main system. I of course still have to build HISE on each OS first, but these executables all reside on the main system too.
And do you build it by opening HISE as an application and running the build via the app on each system or use a terminal script on all machines? (just curious)
I use build scripts, much faster, and I can just leave it to build everything while I get on with other stuff.
-
I am running a Jenkins server on a Mac Mini that is connected to a Windows PC as slave machine in my private network. They automatically pull the latest state from GitHub and launch the build scripts, then upload the signed installers to Dropbox when they are ready.
The setup took me at least two annoying weeks to implement, but it's the most convenient and bulletproof workflow.
-
@d-healey said in Virtual Machine solutions for cross-platform development:
I use build scripts, much faster, and I can just leave it to build everything while I get on with other stuff.
@Christoph-Hart said in Virtual Machine solutions for cross-platform development:
I am running a Jenkins server on a Mac Mini that is connected to a Windows PC as slave machine in my private network. They automatically pull the latest state from GitHub and launch the build scripts...
I found @Christoph-Hart´s build scripts in the example projects. Lot´s of good work already done
Are you both using those?What language are they written in? Is it windows shell script / batch script?
Or are you using Linux/Mac bash/shell-script and, on Windows, running that in git-bash or something to run bash on windows?
-
@andioak said in Virtual Machine solutions for cross-platform development:
I found @Christoph-Hart´s build scripts in the example projects. Lot´s of good work already done
Are you both using those?Mine are heavily modified from scripts Christoph provided me with a few years ago, so they are probably similar to Christoph's example scripts.
What language are they written in? Is it windows shell script / batch script?
Or are you using Linux/Mac bash/shell-script and, on Windows, running that in git-bash or something to run bash on windows?Bash on GNU/Linux and MacOS. Batch on Windows.
-
Building and testing are two different beasts. Graphics are rarely supported or not using virtual or headless build systems. A Mac will run all the OS's you need if you have a legitimate licence for windows and Linux is of course open source. Can recommend an SSD for quickly booting from system to system. Some CLI's will compile for a fee but you will need a system or users to test builds on multiple hosts.
