Mac Installer issue that effects (I think) all of us

Dan Korneff

Hey Guys,
This is my long winded post about the annoyances of creating an installer for MAC. Hoping someone has some insight on what I could do.
The Short version, I've created a plugin that requires files to be deposited both in ~/Library and /Library on a MAC.

Now the long story.
I'm working on an FX plugin where the audio resources are too large to embed. So I'm forced to create an installer that deposits plugins to their respective folders, and the AudioResources.dat file to a folder within ~/Library. Since Packages was highly recommended on this forum, I decided to use it to create my installer.
3 Separate Raw Packages were created for vst/aax/au. A distribution package, created with instructions on the whitebox site for installing to user library, contains the AudioResources.dat file. Raw Packages were then imported to the distribution package, and I built the installer.

Roadblock #1

When the installer runs, it not only installs the AudioResources.dat file to ~/Library, but the plugins as well. Not good since the plugins need to be in /Library. After researching the Whitebox website, I found a note stating that:

You can not install items both in the current user's home directory and outside it. This is not supported by the distribution script format.

So... that's pretty annoying. Next ideas was to just create 2 separate installers.

Roadblock #2

I created a distribution package for the AudioResources.dat using directions from their website. For testing purposes, I created a folder called XYZ within Application Support and selected 1 file to deposit.

Create installer.gif

When I run the installer, it deposits the file to /Library NOT ~/Library

run installer.gif

After some trial and error, I found the cause. When you create a sub folder for your destination, the installer adds a page called Destination Select where it gives you the option to change the install location. If you hit the Install button without first clicking on the Change Install Location button, the installer will place the file in /Library.
If you click on Change Install Location first, you then see an error message which forces you to select "install for me only" as the location... this will finally install the files into ~/Library !

run installer with directory.gif

I bet there will be zero chance that the end user would figure this out, so I contacted the developer about this issue, and here's the response I got:

No there is no way to work around this annoying bug in Installer.app that Apple has not fixed since it has been reported years ago.
I filed again a bug report about this issue a month ago just as reminder to the trillion-dollar company. No feedback at all from Apple.

Since this appears to be a system wide issue traced back to the installer app on MAC, how the heck is everyone dealing with this?
How do you make an installer that updates user presets? How do you make an installer for plugins that do not have embedded resources?

ustk

@dustbro I don't understand what going on on your side, because it works here...
Have a try with the test I've made in this post:
https://forum.hise.audio/topic/2166/update-user-presets-on-mac/7

LeeC

@dustbro Installing files to ~/Library works fine for me at the moment - even when creating a new sub folder to house these files as part of the installer.

alt text

I just tried the test installer that @ustk made and this also installs fine to
~/Library/Application Support

Is there any chance you can send over a test installer so we can try it out on our systems?

Dan Korneff

I appreciate the replies.
@ustk I just tried your installer. Same thing happens on my end. It installs to /Library

I'm noticing that during the install, it's asking for my password and installs to /Library, but when I navigate to the "install for me only" option, no password is needed and installs to ~/Library.
Is this leading to a permissions setting causing the issue?

Here's my test installer that installs a text file to ~/Library/Application Support/xyz/
test.pkg.zip
Does this work for you guys?

ustk

@dustbro It works straight on my end... I have no idea what's going on with your mac, what's your OS version?

Dan Korneff

@ustk I'm on 10.13.6 here

ustk

@dustbro same...

Dan Korneff

@ustk is your user account Administrator, Standard, or Managed?

Christoph Hart

I solved that problem ages ago (actually with the first commercial product that was shipped with HISE) by using a post-build script that copies the files from the global to the user domain.

Set the global domain in the installer (so that it installs to /Library/)
Copy the files you want have in the user's app data folder to an arbitrary accessible location
Run the .sh script as post-install script that copies the files to ~/Library/Application Support.

I just looked up the shell script and the magic ingredient was rsync (because it will keep the user presets at the location if they are newer or something like that):

rsync -avurpE "/SomeSourceFolderThatIsAccessibleFromGlobalDomain" "~/Library/Application\ Support/My\ Company/My\ Product/"

rm -rf "/SomeSourceFolderThatIsAccessibleFromGlobalDomain"

Disclaimer: That shell script is not tested, but should give you a direction. I probably fucked up the usage of quotes or whitespaces as I never understood how that works in bash scripts...

d.healey

I probably fucked up the usage of quotes or whitespaces as I never understood how that works in bash scripts...

I'm pretty sure you don't need the escape characters if it's in quotes.

Dan Korneff

@Christoph-Hart I'm giving this script a go. It appears to work nicely on it's own, but when I add it to my installer as a postscript, the installer fails

Ever run into this before?

Christoph Hart

Check the macOS Console for errors it should spit something out (but it‘s very verbose so be prepared to scroll around)

Dan Korneff

@Christoph-Hart Here's what I'm seeing:

2020-01-08 07:49:07-05 Dans-Mac-mini installd[1365]: PackageKit: *** Couldn't posix_spawn: error 8
2020-01-08 07:49:08-05 Dans-Mac-mini installd[1365]: PackageKit: Failed task attributes: {
	    NSFileCreationDate = "2020-01-08 12:47:28 +0000";
	    NSFileExtensionHidden = 0;
	    NSFileGroupOwnerAccountID = 0;
	    NSFileGroupOwnerAccountName = wheel;
	    NSFileHFSCreatorCode = 0;
	    NSFileHFSTypeCode = 0;
	    NSFileModificationDate = "2020-01-08 12:47:28 +0000";
	    NSFileOwnerAccountID = 0;
	    NSFileOwnerAccountName = root;
	    NSFilePosixPermissions = 493;
	    NSFileReferenceCount = 1;
	    NSFileSize = 58;
	    NSFileSystemFileNumber = 690250;
	    NSFileSystemNumber = 16777220;
	    NSFileType = NSFileTypeRegular;
	}

It looks like posix_spawn: error 8 may be a permissions issue? Once I create the .sh file, I'm running:

chmod a+wx

to give the script it's permissions. :person_shrugging:
This is on my newly installed Catalina rig. Wouldn't be a notarizing issue, would it?

Dan Korneff

Still struggling with this one... can't get a post install script to work in a Whitebox PKG.
Here's the full error I'm getting:

2020-01-24 18:25:03-05 Dans-Mac-mini Installer[40398]: @(#)PROGRAM:Install  PROJECT:Install-1000
2020-01-24 18:25:03-05 Dans-Mac-mini Installer[40398]: @(#)PROGRAM:Installer  PROJECT:Installer-965.1
2020-01-24 18:25:03-05 Dans-Mac-mini Installer[40398]: Hardware: Macmini4,1 @ 2.40 GHz (x 2), 16384 MB RAM
2020-01-24 18:25:03-05 Dans-Mac-mini Installer[40398]: Running OS Build: Mac OS X 10.15.2 (19C57)
2020-01-24 18:25:03-05 Dans-Mac-mini Installer[40398]: Env: USER=dan
2020-01-24 18:25:03-05 Dans-Mac-mini Installer[40398]: Env: LOGNAME=dan
2020-01-24 18:25:03-05 Dans-Mac-mini Installer[40398]: Env: PATH=/usr/bin:/bin:/usr/sbin:/sbin
2020-01-24 18:25:03-05 Dans-Mac-mini Installer[40398]: Env: SSH_AUTH_SOCK=/private/tmp/com.apple.launchd.f5vjJkAmgb/Listeners
2020-01-24 18:25:03-05 Dans-Mac-mini Installer[40398]: Env: HOME=/Users/dan
2020-01-24 18:25:03-05 Dans-Mac-mini Installer[40398]: Env: SHELL=/bin/zsh
2020-01-24 18:25:03-05 Dans-Mac-mini Installer[40398]: Env: __CF_USER_TEXT_ENCODING=0x1F5:0x0:0x0
2020-01-24 18:25:03-05 Dans-Mac-mini Installer[40398]: Env: TMPDIR=/var/folders/6k/pjx9t2q93mg8b0jyjs2y24sw0000gn/T/
2020-01-24 18:25:03-05 Dans-Mac-mini Installer[40398]: Env: XPC_SERVICE_NAME=com.apple.installer.548
2020-01-24 18:25:03-05 Dans-Mac-mini Installer[40398]: Env: XPC_FLAGS=0x0
2020-01-24 18:25:04-05 Dans-Mac-mini Installer[40398]: test  Installation Log
2020-01-24 18:25:04-05 Dans-Mac-mini Installer[40398]: Opened from: /Users/dan/Desktop/test/test/build/test.pkg
2020-01-24 18:25:04-05 Dans-Mac-mini Installer[40398]: Package Authoring Error: <background_scaling> has an unsupported MIME type: X-NSObject/NSNumber
2020-01-24 18:25:04-05 Dans-Mac-mini Installer[40398]: Package Authoring Error: <background_alignment> has an unsupported MIME type: X-NSObject/NSNumber
2020-01-24 18:25:04-05 Dans-Mac-mini Installer[40398]: Failed to load specified background image
2020-01-24 18:25:05-05 Dans-Mac-mini Installer[40398]: Referenced component packages (1) trustLevel=100
2020-01-24 18:25:18-05 Dans-Mac-mini Installer[40398]: Administrator authorization granted.
2020-01-24 18:25:18-05 Dans-Mac-mini Installer[40398]: ================================================================================
2020-01-24 18:25:18-05 Dans-Mac-mini Installer[40398]: User picked Standard Install
2020-01-24 18:25:18-05 Dans-Mac-mini Installer[40398]: Choices selected for installation:
2020-01-24 18:25:18-05 Dans-Mac-mini Installer[40398]: 	Install: "test"
2020-01-24 18:25:18-05 Dans-Mac-mini Installer[40398]: 		test.pkg : com.mygreatcompany.pkg.test : 1.0
2020-01-24 18:25:18-05 Dans-Mac-mini Installer[40398]: ================================================================================
2020-01-24 18:25:18-05 Dans-Mac-mini Installer[40398]: It took 0.00 seconds to summarize the package selections.
2020-01-24 18:25:19-05 Dans-Mac-mini Installer[40398]: -[IFPKGDerivedDocument sortedPackageLocations]: result = (
	    "file://localhost"
	)
2020-01-24 18:25:19-05 Dans-Mac-mini Installer[40398]: -[IFDInstallController(Private) _buildInstallPlanReturningError:]: location = file://localhost
2020-01-24 18:25:19-05 Dans-Mac-mini Installer[40398]: -[IFDInstallController(Private) _buildInstallPlanReturningError:]: file://localhost/Users/dan/Desktop/test/test/build/test.pkg
2020-01-24 18:25:19-05 Dans-Mac-mini Installer[40398]: Set authorization level to root for session
2020-01-24 18:25:19-05 Dans-Mac-mini Installer[40398]: Will use PK session
2020-01-24 18:25:19-05 Dans-Mac-mini Installer[40398]: Using authorization level of root for IFPKInstallElement
2020-01-24 18:25:19-05 Dans-Mac-mini Installer[40398]: Starting installation:
2020-01-24 18:25:19-05 Dans-Mac-mini Installer[40398]: Configuring volume "Hard Drive"
2020-01-24 18:25:19-05 Dans-Mac-mini Installer[40398]: Preparing disk for local booted install.
2020-01-24 18:25:19-05 Dans-Mac-mini Installer[40398]: Free space on "Hard Drive": 244.58 GB (244576600064 bytes).
2020-01-24 18:25:19-05 Dans-Mac-mini Installer[40398]: Create temporary directory "/var/folders/6k/pjx9t2q93mg8b0jyjs2y24sw0000gn/T//Install.40398fiFQ0a"
2020-01-24 18:25:19-05 Dans-Mac-mini Installer[40398]: IFPKInstallElement (1 packages)
2020-01-24 18:25:19-05 Dans-Mac-mini installd[1528]: PackageKit: Adding client PKInstallDaemonClient pid=40398, uid=501 (/System/Library/CoreServices/Installer.app/Contents/MacOS/Installer)
2020-01-24 18:25:20-05 Dans-Mac-mini Installer[40398]: PackageKit: Enqueuing install with framework-specified quality of service (utility)
2020-01-24 18:25:20-05 Dans-Mac-mini installd[1528]: PackageKit: ----- Begin install -----
2020-01-24 18:25:20-05 Dans-Mac-mini installd[1528]: PackageKit: request=PKInstallRequest <1 packages, destination=/>
2020-01-24 18:25:20-05 Dans-Mac-mini installd[1528]: PackageKit: packages=(
	    "PKLeopardPackage <id=com.mygreatcompany.pkg.test, version=1.0, url=file://localhost/Users/dan/Desktop/test/test/build/test.pkg>"
	)
2020-01-24 18:25:20-05 Dans-Mac-mini installd[1528]: PackageKit: Set reponsibility for install to 40398
2020-01-24 18:25:20-05 Dans-Mac-mini installd[1528]: PackageKit: Extracting file://localhost/Users/dan/Desktop/test/test/build/test.pkg (destination=/Library/InstallerSandboxes/.PKInstallSandboxManager/4E0FB4FE-6A1F-4232-9082-EF19CD75F8EF.activeSandbox/Root, uid=0)
2020-01-24 18:25:20-05 Dans-Mac-mini installd[1528]: PackageKit: prevent user idle system sleep
2020-01-24 18:25:20-05 Dans-Mac-mini installd[1528]: PackageKit: suspending backupd
2020-01-24 18:25:21-05 Dans-Mac-mini installd[1528]: PackageKit: Using trashcan path /var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/PKInstallSandboxTrash/4E0FB4FE-6A1F-4232-9082-EF19CD75F8EF.sandboxTrash for sandbox /Library/InstallerSandboxes/.PKInstallSandboxManager/4E0FB4FE-6A1F-4232-9082-EF19CD75F8EF.activeSandbox
2020-01-24 18:25:21-05 Dans-Mac-mini installd[1528]: PackageKit: Shoving /Library/InstallerSandboxes/.PKInstallSandboxManager/4E0FB4FE-6A1F-4232-9082-EF19CD75F8EF.activeSandbox/Root (0 items) to /
2020-01-24 18:25:21-05 Dans-Mac-mini installd[1528]: PackageKit: Executing script "./postinstall" in /private/tmp/PKInstallSandbox.it0Hz8/Scripts/com.mygreatcompany.pkg.test.aWq9Cy
2020-01-24 18:25:21-05 Dans-Mac-mini installd[1528]: PackageKit: *** Couldn't posix_spawn: error 8
2020-01-24 18:25:21-05 Dans-Mac-mini installd[1528]: PackageKit: Failed task attributes: {
	    NSFileCreationDate = "2020-01-24 23:23:25 +0000";
	    NSFileExtensionHidden = 0;
	    NSFileGroupOwnerAccountID = 0;
	    NSFileGroupOwnerAccountName = wheel;
	    NSFileHFSCreatorCode = 0;
	    NSFileHFSTypeCode = 0;
	    NSFileModificationDate = "2020-01-24 23:23:25 +0000";
	    NSFileOwnerAccountID = 0;
	    NSFileOwnerAccountName = root;
	    NSFilePosixPermissions = 493;
	    NSFileReferenceCount = 1;
	    NSFileSize = 53;
	    NSFileSystemFileNumber = 1581080;
	    NSFileSystemNumber = 16777220;
	    NSFileType = NSFileTypeRegular;
	}
2020-01-24 18:25:21-05 Dans-Mac-mini install_monitor[40406]: Temporarily excluding: /Applications, /Library, /System, /bin, /private, /sbin, /usr
2020-01-24 18:25:21-05 Dans-Mac-mini install_monitor[40406]: Re-included: /Applications, /Library, /System, /bin, /private, /sbin, /usr
2020-01-24 18:25:21-05 Dans-Mac-mini installd[1528]: PackageKit: releasing backupd
2020-01-24 18:25:21-05 Dans-Mac-mini installd[1528]: PackageKit: allow user idle system sleep
2020-01-24 18:25:21-05 Dans-Mac-mini installd[1528]: PackageKit: Cleared responsibility for install from 40398.
2020-01-24 18:25:21-05 Dans-Mac-mini installd[1528]: PackageKit: Cleared permissions on Installer.app
2020-01-24 18:25:21-05 Dans-Mac-mini installd[1528]: PackageKit: Install Failed: Error Domain=PKInstallErrorDomain Code=112 "An error occurred while running scripts from the package “test.pkg”." UserInfo={NSFilePath=./postinstall, NSURL=file://localhost/Users/dan/Desktop/test/test/build/test.pkg, PKInstallPackageIdentifier=com.mygreatcompany.pkg.test, NSLocalizedDescription=An error occurred while running scripts from the package “test.pkg”.} {
	    NSFilePath = "./postinstall";
	    NSLocalizedDescription = "An error occurred while running scripts from the package \U201ctest.pkg\U201d.";
	    NSURL = "file://localhost/Users/dan/Desktop/test/test/build/test.pkg";
	    PKInstallPackageIdentifier = "com.mygreatcompany.pkg.test";
	}
2020-01-24 18:25:21-05 Dans-Mac-mini installd[1528]: PackageKit: Running idle tasks
2020-01-24 18:25:21-05 Dans-Mac-mini Installer[40398]: install:didFailWithError:Error Domain=PKInstallErrorDomain Code=112 "An error occurred while running scripts from the package “test.pkg”." UserInfo={NSFilePath=./postinstall, NSURL=file://localhost/Users/dan/Desktop/test/test/build/test.pkg, PKInstallPackageIdentifier=com.mygreatcompany.pkg.test, NSLocalizedDescription=An error occurred while running scripts from the package “test.pkg”.}
2020-01-24 18:25:21-05 Dans-Mac-mini installd[1528]: PackageKit: Removing client PKInstallDaemonClient pid=40398, uid=501 (/System/Library/CoreServices/Installer.app/Contents/MacOS/Installer)
2020-01-24 18:25:21-05 Dans-Mac-mini installd[1528]: PackageKit: Done with sandbox removals
2020-01-24 18:25:22-05 Dans-Mac-mini Installer[40398]: Install failed: The Installer encountered an error that caused the installation to fail. Contact the software manufacturer for assistance.
2020-01-24 18:25:22-05 Dans-Mac-mini Installer[40398]: IFDInstallController A0AB60 state = 8
2020-01-24 18:25:22-05 Dans-Mac-mini Installer[40398]: Displaying 'Install Failed' UI.
2020-01-24 18:25:22-05 Dans-Mac-mini Installer[40398]: 'Install Failed' UI displayed message:'The Installer encountered an error that caused the installation to fail. Contact the software manufacturer for assistance.'.
2020-01-24 18:25:24-05 Dans-Mac-mini Installer[40398]: Package Removal: Package is not in downloads directory. Not offering removal.
2020-01-24 18:25:24-05 Dans-Mac-mini Installer[40398]: Package Removal: Package cannot be removed.

Does this mean anything to anyone here?

Dan Korneff

Ok! Finally sorted out.
Apparently, you can't use ~/ in a pre or post-installation script of a package. I had to use $HOME instead.

Casey Kolb

Thank you for this thread! Super useful.

@dustbro Do you mind posting your final shell script? I haven't quite cracked it yet on my system.

Dan Korneff

Here's what I got going on. My installer places all of the extra files (Markdown, AudioRecources.dat, ImageResources.dat, user manual, etc...) into the system Library/Application Support folder, then I use a post-install script to move the data into the user Library/Application Support folder.
I've added comments above each line.

#!/bin/sh

// this line moves the System Library folder to the User Library folder:
/usr/bin/rsync -avurpE --remove-source-files /Library/Application\ Support/Company/Product/ $HOME/Library/Application\ Support/Company/Product/

// this line deletes the System Library folder after the files are moved:
find /Library/Application\ Support/Company/ -type d -empty -delete

// determine logged in user:
loggedInUser=`/bin/ls -l /dev/console | /usr/bin/awk '{ print $3 }'`

//This line copies my user manual from the User Library to Desktop for easy access:
/usr/bin/rsync -avurpE  $HOME/Library/Application\ Support/Company/Product/User\ Manual/ /Users/$loggedInUser/Desktop/

exit 0

Casey Kolb

@dustbro Amazing! Thank you

Casey Kolb

@dustbro I'm having one serious issue with this approach and I wonder if you've encountered this as well. When I rsync the application folder like above, if the folder doesn't already exist (which will be the case for any user downloading the plugin for the first time) then it will keep the permissions set by Packages for the temp folder which has some pretty whack results and makes the folder read-only for the user. Only the system will be able to read/write.

This is super deadly because it removes the ability to save presets or create new folders in the preset browser. It also disables the changing of the Sample Folder directory because the entire plugin folder becomes read-only. I've tried a bunch of different things in the bash script to fix this but nothing seems to be doing the trick.

Dan Korneff

@Lunacy-Audio have you checked another system yet? I'm currently using this method successfully.
The most common problem I've seen is the inability to remove the system Company folder after sync due to permissions issues.

Mac Installer issue that effects (I think) all of us

52

1.7k

11.7k

101.8k