CPU usage during compilation

[d.healey]

This might be a solution worth investigating for multi-threaded linking - https://lld.llvm.org/

[d.healey]

I found what was causing my CPU issues. The CPU was power throttling. I've increased the power limits and now I'm getting much better clock speed when compiling.

[d.healey]

I just tried lld, seems it might be a bit faster at linking but I'd need to do some proper benchmarks to know for certain.

In case anyone else (or future Dave) is thinking of using lld (or gold) for linking here is what you need to do:

Put the following in the Extra Linker Flags box in Projucer

For lld:
-fuse-ld=lld

With lld you need to use Clang instead of GCC. So the make command will look something like: make CXX=clang++ CONFIG=Release -j 6

For gold:
-fuse-ld=gold
-Wl,--threads
-Wl,--thread-count=6

[d.healey]

I've tried both of these linkers so far and have had no success in improving compile times.

For some reason, which I can't work out, clang + lld will compile and link but the linking is still only single threaded. @Christoph-Hart Any ideas?

With gold the compiling via GCC works but the linking fails with an error - I haven't dug into this one much more because it seems lld is the better of the two.

[Christoph Hart]

I think the newest script node version compiles a little bit faster because the template system is a little bit less complex, but in my experience the only way to really speed up compile times is to get a better system :)

[d.healey]

@Christoph-Hart Compile time is fine, it's the linking that's the problem, it only uses a single thread.

[d.healey]

Finally solved the linker threading thing.

Add this to the Extra Linker Flags section -flto=auto :D

[ustk]

@d-healey Nice one! Is that linux only?

EDIT: oh yes it's linux... I wonder if I can speed up the process on mac too...

[d.healey]

@ustk I think Xcode already does multi-threaded linking.

[yall]

@ustk mac je met 1h pour un vst. Windows 4 minutes. Ça rend ouf

[Matt_SF]

@yall pareil. 1/2 an hour on my mac (Intel i7) , 5mins on pc

[d.healey]

How many threads are you guys using for your builds?

[oskarsh]

@d-healey

I am using all threads -2

I am building using the batchCompile

make CONFIG=Release AR=gcc-ar -j`nproc --ignore=2`

[Christoph Hart]

@d-healey said in CPU usage during compilation:

How many threads are you guys using for your builds?

Yes. :)

[d.healey]

@oskarsh I was asking the Mac guys who are saying their builds are slow. :)

[Matt_SF]

@d-healey Just 1 thread here. Do you know if there's a way to change that ?

[d.healey]

@Matt_SF That explains it. There is definitely a way but I can't remember how. Google will help you here :)

[Christoph Hart]

@d-healey I think the problem is that on macOS the linker (ld) is only single threaded (at least here) and you can't replace it with another linker easily.

[Christoph Hart]

-flto=auto

Setting the link time optimization to auto is not the solution as it might just deactivate it (depending what auto means) and sure then you decrease the build time but you end up with a bigger and less optimized binary. The link time optimization (lto) is the step that takes the most time which is why I deactivated it on the CI test build of the demo project so that the macOS test suite runs as fast as the Windows one.

[d.healey]

@Christoph-Hart

Setting the link time optimization to auto is not the solution

Works here on Debian. If that option is available on Mac you can replace auto with the number of threads you'd like to use.