Let’s talk about build systems.

My job is to program video games targeted to iOS and Android devices. The platform-specific sections are written in Objective-C or Java, respectively, and the parts common to both platforms, that is 99% of the application, are written in C++. The main interest in doing everything in C++ is that the developers can also build the game for their workstations, either running Linux or OSX, and test their modifications without paying the price of an emulator or the transfer to a mobile device.

The inconvenient is that we have to handle builds for four platforms.

In order to compile an application for iOS one must use an Xcode project, even if libraries can be compiled as usual on the command line. It means that we must maintain an Xcode project file or have a tool to generate it.

On Android’s side, compiling C++ code is done with a tool called ndk-build, which is actually an interface to GNU Make. Consequently the project files are obviously makefiles but using a set of variables and functions of some kind of framework in Make’s language. There again, we have to maintain these makefiles for the project, or we can use a tool to generate them.

For Linux and OSX the compilation is done with more classical tools but obviously with neither the iOS or Android project files. Again, we have to keep these files up to date.

Unless there exist a tool to generate them all…

Some build systems

The problem of handling the compilation of a project has been encountered well before you were born. That’s why there’s a single method to do it today, being a consensus since several decades.

Ha! Ha! Just kidding!

I can list a dozen tools from memory. Let’s see how they describe themselves on their websites and don’t forget to be fair.

Make

GNU Make is a tool which controls the generation of executables and other non-source files of a program from the program’s source files.

This is a quote from the description as one can read it on GNU Make’s website but the first release of Make was in April 1976. It predates the GNU project by seven years.

The base principle of Make is quite simple, we give it a Makefile file in which are listed recipes: “here is the file I want to create, it depends of these other files, and there are the commands to run to build it”. With its integrated dependency mechanism the tool handles the creation of the targets in the expected order.

I still write small Makefiles sometime but it’s quickly painful. For example, if a binary app is created from a file a.cpp which depends on a file b.hpp which includes itself a file c.hpp, then I must list these three files in the dependencies of app in order for the compilation to be done when one of them is modified. It’s something you do once, then you search a tool to automatically list the dependencies.

Another case I find difficult to handle is the dependency to the Makefile itself. For example, if I add an option in the linker arguments, it does not trigger the link of the targets by default. Indeed, no dependency of the target has changed! One would suggest to list the Makefile in the dependencies but then it will trigger a new build every time the Makefile is modified, for any reason.

Autotools

[The first goal of the Autotools] is to simplify the development of portable programs. The system permits the developer to concentrate on writing the program, simplifying many details of portability across Unix and even Windows systems, and permitting the developer to describe how to build the program using simple rules rather than complex Makefiles.

If you have ever used the Autotools you are certainly laughing by reading the above quote. This tool suite which describes itself as simple and portable is actually the most complex build system ever created and is portable only among Unix likes, carefully avoiding Windows, the most deployed operating system in the last thirty years.

Undoubtedly revolutionary at its conception, this tool aged very badly. I had to compile third-party software on Windows with the Autotools in multiple occasions and it was, to put it simply, just like hell. I also tried to use it for my own projects and I could not even complete the tutorial. Since then I am convinced that these tools are made for developers who love to sweat.

SCons

SCons is an Open Source software construction tool — that is, a next-generation build tool. Think of SCons as an improved, cross-platform substitute for the classic Make utility with integrated functionality similar to autoconf/automake and compiler caches such as ccache. In short, SCons is an easier, more reliable and faster way to build software.

Welcome to the future!

The build files of SCons are written directly in Python, which is quite a nice thing since it allows the developer to describe his project in a good programming language with a lot of available libraries.

I have compiled some projects using SCons like fifteen years ago and I think that I have never seen a tool so slow, ever. And it’s not only Python’s fault…

I must state that the default behavior of the tool is to detect the modifications in source files by computing an MD5 hash of the files, to avoid the recompilation of files when only its date has changed. When you see the cost of an MD5 I find this choice quite questionable. This behavior can be modified but even when using the classical timestamp comparison of the files and applying all available tricks to make the build faster, it’s still extremely slow.

Premake

Powerfully simple build configuration.

Describe your software project just once, using Premake’s simple and easy to read syntax, and build it everywhere.

Generate project files for Visual Studio, GNU Make, Xcode, Code::Blocks, and more across Windows, Mac OS X, and Linux. Use the full featured Lua scripting engine to make build configuration tasks a breeze.

Wow, it’s awesome! This is just the fourth in the list and it already seems to solve everything.

With Premake the build scripts are written in Lua and, just like SCons, it allows the developer to “program” his build and to take advantage of existing libraries.

On paper it seems quite great, in practice it does not work as expected. As it turns out Premake’s syntax is declarative and does not mix well with the procedural aspect of Lua. For example, if I write

project "P"

if false then
   files
   {
      "file.cpp"
   }
end

One will certainly think that the file file.cpp will not be part of project P, when actually it is. The if does not change anything here. It becomes difficult to program in these conditions.

We’ve been using Premake since almost four years at work. The choice was made in its favor even if the tool was still in alpha because it could generate the files for Xcode and ndk-build with two independently developed plugins, added to the generation of the usual Makefiles. Also it seemed easy to hack, which is reassuring.

Nowadays I try to replace it whenever possible.

Among the recurring problems the most painful is certainly the message error: (null), with no other information, that the tool displays when an error has slipped in the scripts. Good luck in debugging this. I’m also very fond of the message type 'premake5 --help' for help which is written in my console when I have a typo on the command line. Here again, there’s absolutely no useful information and we have to scan the command line to find the error. Another concern: there’s no way to add link properties for a specific library. It’s annoying when you need a -Wl,--whole-archive.

The development of Premake itself seems very labored. Four years after we started using it at work it’s still in alpha, with the most recent release dated from one year ago. The Xcode module has been merged with the main project but it’s OSX-only. We had to reapply all our patches for iOS. As for the NDK module it does not work anymore due to modifications in Premake (hey, it’s an alpha…). Once more, we had to apply our patches again.

There are a lot of contributors to the project, even big ones, but each one seems to go in its own direction with no overall common goal. For example there are two generators for Makefiles, named gmake and gmake2 (I am impatient about yagmake). There are features that work only with Visual Studio, other stuff that used to work four years ago and does not anymore. It looks like your typical perfection-obsessed project that actually doesn’t do anything correctly. In short, the product does not match the pitch.

CMake

CMake is an open-source, cross-platform family of tools designed to build, test and package software. CMake is used to control the software compilation process using simple platform and compiler independent configuration files, and generate native makefiles and workspaces that can be used in the compiler environment of your choice.

CMake is my favorite build tool of the last fifteen years.

This says it all.

Well, okay, I’ll explain. CMake reads the projects to build from CMakeLists.txt files written in a language of its own. From there it generates some Makefiles (or equivalent project files for Xcode or Visual Studio for example) that can be used to build the project.

The main things that initially made me appreciate this tool were that it is quite fast (even though it is not the fastest) and that it handles perfectly the rules for rebuilding the targets. For example, if I add a argument for the linker, then the link step will be executed. If I modify a CMakeLists.txt file and then run make without cmake, then CMake will automatically update the generated files. I can also easily define the folders for the headers, or compile options and other parameters on a per-target basis, with fine-grained visibility of these settings for the depending projects.

The tool is quite well conceived and very popular in the C++ world.

The main criticism against CMake is its language, especially when the version 2 of the software was around. Before version 3 the scripts were excessively verbose and ALL CAPS. Writing or reading a script was like being yelled at during the whole file. Today these problems are solved and people only complain about having to learn one more specific language to use it (unlike with SCons and Premake for example). Personally I see no difficulty here, it’s a simple macro-based language with very convenient mechanisms to name and group the functions’ arguments.

As usual the description of the configuration files as “platform and compiler independent” is questionable since there is everything you need to slip system commands in the build.

One problem I had with recent versions of CMake is in the project export features. Indeed, there is a command install(EXPORTS) which creates a CMake configuration file to include the target as a dependency in third-party projects. Unfortunately this command exports by default the absolute paths of the dependencies so you’ll have to hack around it to export the dependencies correctly (by wrapping them in imported targets for example).

Another issue is that CMake generates a lot of intermediate files. It would not be a problem if the general tendency was not to build from the root of the source’s tree. In a perfect world the tool would refuse to configure the project in-source. For my most recent projects I now place the CMakeLists.txt files in a tree outside the main sources and I ensure that it can be built from any directory.

Ninja

Ninja is a small build system with a focus on speed. It differs from other build systems in two major respects: it is designed to have its input files generated by a higher-level build system, and it is designed to run builds as fast as possible.

Oh great, here’s another self-claimed fast tool, this is exactly what we needed. Moreover when we look at SCons and Premake who also claimed to be the fastest, we can only be confident. That being said, contrary to SCons and Premake, Ninja is not a generator of build scripts. It’s more like Make.

I have never actively used Ninja but I’ve read that it is really faster than Make when there is nothing to do (i.e. when all your targets are up to date).

I did try it with CMake on iscool::core to measure the gap with Make on a full build:

$ time make -j 4
real 0m29,490s
user 1m36,778s
sys 0m5,436s

$ time ninja -j 4
real 0m28,451s
user 1m39,555s
sys 0m4,701s

Well… I don’t see any winner here. I suppose the project is too small. Also, this project uses compilation units (a.k.a. unity builds) which is also an efficient working solution to make the build faster.

Meson

Meson is an open source build system meant to be both extremely fast, and, even more importantly, as user friendly as possible.

I’m losing hope. Here we have, again, a tool claiming to be fast and still no tool pretending to work correctly.

Meson is a generator that produces build files for Ninja or optionally for other build systems (Xcode, Visual Studio and maybe others). It is comparable to CMake and according to its own benchmark it is indeed the fastest build system in town.

I’ve never used Meson but some people who’ve heard about it told me that (enable your robot voice now) “it’s new so it’s great so you should try it”.

Why not, but what I am actually looking for is a tool that can build my project for iOS, Android, OSX and Linux. Something that just works; speed is a secondary concern.

FASTbuild

FASTBuild is a high performance, open-source build system supporting highly scalable compilation, caching and network distribution.

/me crying

I have never used this tool either. The pitch is great but I do not see any interest compared with the other thousands similar tools.

Sharpmake

Sharpmake is an open-source C#-based solution for generating project definition files, such as Visual Studio projects and solutions, GNU makefiles, Xcode projects, etc.

This one is developed by Ubisoft initially as an internal tool then open-sourced in September 2017. It seems that there is no activity in the repository since October the same year. As its name suggests, its build scripts are written in C#.

According to the documentation it can also generate Makefiles and projects for Xcode. I had seriously considered this as a replacement for Premake but on one hand it is written in C#, so using it on Linux is a dead end, and on the other hand I can feel that everything unrelated to Windows and Visual Studio will be half-baked.

Maven

Apache Maven is a software project management and comprehension tool. Based on the concept of a project object model (POM), Maven can manage a project’s build, reporting and documentation from a central piece of information.

I don’t know about you but I can barely understand the description. Or maybe my English is not so good.

Maven is a tool from the Java world. I did use it on some occasions in third party projects and there is nothing I can complain about. It seems very efficient on the dependency management.

This is a very professional-looking tool which… Wait a minute… Hey! that’s why the description is so unintelligible!

The first release goes back to 2004 so it’s just naturally that the most popular language of the beginning of the century had been chosen for the build scripts, I’m obviously talking about 🎉 XML 🎉.

Ant

Apache Ant is a Java library and command-line tool whose mission is to drive processes described in build files as targets and extension points dependent upon each other.

Again, very professional.

The initial release of this tool had been published in 2000. The Ant files are some kind of Makefiles with a twist, they are XML-Makefiles. Needless to say, Ant is also a tool of the Java World.

Gradle

Accelerate developer productivity

From mobile apps to microservices, from small startups to big enterprises, Gradle helps teams build, automate and deliver better software, faster.

I’ll let you guess which language is targeted by this tool.

Gradle is the reference for building Android applications so it’s in this context that I had the opportunity to use it. Gradle scripts are written in Groovy, a language I’ve never used otherwise. I’m not fond of it but maybe it’s just because it’s far from how we do things in C++.

The most tiring thing with Gradle is the time spent in the compilation. For the Java part of our games, which does not contain a lot of code, it takes nearly one minute. To be fair, this is certainly caused by the multiple steps required by the Android plugin. Another problem is the difficulty to find a clear and easy to digest documentation. The official documentation accounts for 24% of the web [reference needed] which implies that any interrogation will require several hours of reading, and examples one can find on StackOverflow and various blogs are quite disparate.

How to choose

Twelve tools and still none of them seems to stand out :/ Maybe we should write a new tool to replace them all, something fast, scalable, but also adaptable for the most interprocedural deployment processes on decentralized hubs for the most professional among us, with scripts written in a popular language, like GOTO++.

Despite the broad choice I did not find anything to solve my problem: generating a project for iOS, Android, Linux and OSX with a single script. Our best bet is currently CMake but it seems to lack official iOS support (there is at least this toolchain we use to build our libraries) and I don’t think it will ever handle the Java part of the project.

Apart from the no-time-for-this-for-now-it-works excuse, we still have not replaced Premake with CMake because there are some subtleties in the generation of the Xcode project that I do not know if they can be done with CMake. I’m talking about the patches we had to apply on the generation of the pbxproj file in Premake:

First of all we still use the manual selection of the provisioning profile. It’s configured in the project by setting the property ProvisioningStyle = Manual in the attributes.TargetAttributes of the targets in the pbxproj.

Then there are the capabilities (push notifications and others). They must be listed in SystemCapabilities.

We also had to add the SKIP_INSTALL option on the libraries to avoid their inclusion in the final IPA.

Finally we had to label the frameworks as optional to allow the application to be launched on older devices which do not have the required version of iOS for the given features. For example, when ReplayKit has been introduced we needed the application to still be available on the previous versions of iOS. Then we test the availability of the framework at run time.

All these issues have been encountered from day to day and the plugin system of Premake, written in Lua, allowed us to find solutions quite efficiently. I do not know if it would have been so easy with CMake.

If you have a suggestion, please tell me.