This article discusses the Ninja build tool. Earthly ensures consistent builds across team environments with its reproducibility features. Check it out.
Ninja is a compact build system with a focus on fast incremental builds. It was originally developed by Evan Martin, a Google dev, partly in response to the needs of building large projects such as Google Chrome.
If you’re developing a software system and you require a rebuild every few minutes to test your latest feature or code block, then Ninja will only rebuild what you have just modified or added and nothing else—as opposed to Make, which would rebuild the whole project every single time.
This article will start by explaining build systems in a little more detail. It’ll then introduce Ninja and teach you how to use Ninja to build a simple C++ project.
What Is a Build System?
Software projects are usually composed of many files. And the process of compiling, linking, copying, structuring, testing, or more generally, processing these files into an executable program is called a build.
Build systems fall into two broad categories:
- Build generator: Software that takes as input a spec file written in a specially designed language (in many cases, Turing Complete programming languages) and generates a build file that tells build tools how to go about building a software package.
- Build tool: Software that takes in a build file previously generated by a build generator and then builds a software package. Ninja is an example of a build tool.
Next, let’s take a closer look at Ninja and when you may want to use it.
What Is Ninja?
Ninja is a fast build tool that can also be used as a build tool for other build generators. As mentioned, it was originally developed by Evan Martin, a Google dev, as a resource to speed up the building of projects such as Chrome. Since its inception, some notable projects built using Ninja include Chrome, Android, all Meson projects, Swift, and LLVM.
For a very interesting review and tech analysis of the Ninja build system, Ninja’s creator wrote a critical review article eight years after its original release.
Ninja differs from other build systems in two major ways:
- It’s designed to have its input files generated by a higher-level build system such as CMake or Meson, and
- It’s designed to run builds as fast as possible.
This philosophy even goes into the default arguments, which are designed to provide the best performance with little to no tinkering. For example, it builds things in parallel by default. This motivates developers to ensure that their code can be built in parallel, and any problems can be detected early in the development process.
Next, we’ll take a closer look at the strengths and weaknesses of Ninja as a build system.
Advantages and Limitations of Ninja
The main advantage of Ninja is its speed in incremental builds. It incentivizes developers to write code that can be built in parallel, using defaults that utilize the -jN
flag, which causes Ninja to build in parallel. Furthermore, Ninja doesn’t use a background daemon to constantly keep track of things in memory; it always starts its own binary from scratch and works without relying on any state. So developers will always have a realistic idea of build time without fancy background optimizations done by daemons, and it also makes Ninja very portable and simple.
According to benchmarks, Ninja performs as well as Make in a fresh build but outperforms it in an incremental build by what appears to be an exponentially increasing factor.
On the downside, Ninja can’t build projects without a build file, so it must always work with a build generator, such as CMake or Meson, the most popular build generators that work with Ninja. Problems can be introduced depending on the build generator used. For example, Make requires each file to be specified in the build file, making the process of writing these files extremely complicated, time consuming, and prone to errors. So, the key is to choose the right build generator that works well with Ninja.
When Should You Use Ninja?
Ninja works well for large projects with many files that need many incremental builds over a short time. If you’re already using Make, Meson, or CMake to generate build files and using Make to build them, Ninja is a plug-and-play replacement that will, at worst, keep the performance the same or, at best, improve it in an exponential manner in case of incremental builds.
Conversely, Ninja might not be a good choice if you want an end-to-end tool (build generator and tool in one) that has a high-level language to describe relationships between files and is also a build tool. In that case, something like Bazel might be better, but it’s often slower than Ninja and not as portable.
Implementing a Ninja Build
The following sections explain the different ways to install Ninja before going through step-by-step instructions for implementing a Ninja build.
How to Install Ninja
This section explains how to install Ninja on Linux, Mac, and Windows, and how to build it from source.
For a more thorough set of instructions for any specialized installation cases, please see the Ninja GitHub page, a very useful resource. For day-to-day use, the wiki page also includes a list of standard build patterns and build generators that work well with Ninja.
Installing Ninja on Linux
Depending on the Linux flavor, the installation process differs a bit:
- Arch:
pacman -S ninja
- Debian/Ubuntu:
apt-get install ninja-build
- Fedora:
dnf install ninja-build
- Gentoo:
emerge dev-util/ninja
- OpenSUSE:
zypper in ninja
- Alpine:
apk add ninja
Installing Ninja on MacOS
Ninja can be installed using either Homebrew or MacPorts with the following one-liners:
- Homebrew:
brew install ninja
- MacPorts:
port install ninja
Installing Ninja on Windows
Chocolatey or Scoop can be used to install Ninja with a one-liner on Windows:
- Chocolatey:
choco install ninja
- Scoop:
scoop install ninja
Installing Ninja via Package Managers
Ninja can also be installed via package managers, which generally provide more convenience when managing multiple other packages in addition to it:
- Conda:
conda install -c conda-forge ninja
- Pip:
python -m pip install ninja
- Spack:
spack install ninja
Building Ninja From Source
Users who don’t want to build Ninja with specialized flags can build it from source with the following instructions.
First, clone
and checkout
the Ninja repo:
git clone git://github.com/ninja-build/ninja.git && cd ninjagit checkout release
Then, build a basic Ninja binary and a set of files needed to build Ninja:
./configure.py --bootstrap
This will generate the Ninja binary and a build.ninja
file that can be used to build Ninja with itself. That is, the basic Ninja binary generated in the previous step can be used to build the particular configuration of Ninja required. A sort of “ninjaception” if you will.
Use the following code to build Ninja:
cmake -Bbuild-cmakecmake --build build-cmake
The Ninja binary will now be inside the build-cmake
directory (though the user can name this directory anything).
The following code will run the unit tests:
./build-cmake/ninja_test
Creating a Project With Ninja
To demonstrate how to use Ninja as well as showcase some of its strengths, this tutorial uses a sample project, which can be found in this GitHub repo.
This tutorial will show you how to create a simple from-scratch project and an incremental project to demonstrate the time-saving features of Ninja.
Prerequisites
The following are prerequisites to follow along:
- CMake is required to build the project. Instructions for CMake installation can be found on their official website.
- For Linux users, CMake can be installed with the single terminal command
sudo snap install cmake --classic
. It can be installed on macOS withbrew install cmake
. - Python is required to generate the sample files used in this tutorial. Ensure Python is installed.
Creating a Project From Scratch
To create a project from scratch, do the following:
- Clone the repository with
git clone https://github.com/AntreasAntoniou/ninja-tutorial.git
- Navigate to the project directory using
cd ninja-tutorial
- Navigate to the from-scratch project using
cd scratch
- Notice the two files inside this folder:
hello_world.cpp
andCMakeLists.txt
hello_world.cpp
is a simple C++ program that prints “Hello World” to the console:
// C++ program to display "Hello World"// Header file for input/output functions#include <iostream>using namespace std;// main() function: where the execution of program beginsint main(){ // Print "Hello World" cout << "Hello World"; return 0;}
CMakeLists.txt
is a CMake file that describes the project and how to build it:
cmake_minimum_required (VERSION 3.8)project(HelloWorld CXX)set(CMAKE_CXX_STANDARD 14)add_executable(HelloWorld hello_world.cpp)
Now use CMake to generate a build file for Ninja:
cmake -G Ninja
This should generate a build.ninja
file in the current directory, along with related configuration files.
The project can now be built with Ninja using the following command:
ninja
The following output should be generated:
[2/2] Linking CXX executable HelloWorld
With that, the Ninja project should be successfully built.
Creating and Building an Incremental Project
Now, go back to the root of the repository and navigate to the incremental project by running:
cd ..cd incremental
There are three files here: hello_world-template.cpp
, CMakeLists-template.txt
, and generate_project_files.py
.
The generate_project_files.py
file is a Python script that generates the C++ project files and the CMake file from the template files.
This script needs to be run twice: once to generate a 1000-file project and a second time to generate a 1001-file project. So the second project will be an incremental build of the first.
Generate the 1000-file project first:
python3 generate_project_files.py --num_files 1000
Now, use CMake to generate a build file for Ninja:
cmake -S sample_project -G Ninja
This should generate a build.ninja
file in the current directory, along with related configuration files.
Build the project with Ninja:
ninja
Next, emulate an incremental build by adding one more file to your sample project:
python3 generate_project_files.py --num_files 1001
Use CMake to generate a build file for Ninja:
cmake -S sample_project -G Ninja
This should generate a build.ninja
file in the current directory, along with related configuration files.
As before, build the project with Ninja:
ninja
In the second build, Ninja only builds the new file and not the entire project. This can be seen in the terminal output that shows how many files had to be processed, as well as the time taken for the build to complete.
On the local setup (Apple M1 Max, 16-inch) used in this tutorial, the first build took 35 seconds, and the second build took three seconds.
The following is a copy of the terminal output for the second build:
❯ cmake -S sample_project/ -G Ninja-- The CXX compiler identification is AppleClang 14.0.0.14000029-- Detecting CXX compiler ABI info-- Detecting CXX compiler ABI info - done-- Check for working CXX compiler: \ /Library/Developer/CommandLineTools/usr/bin/c++ - skipped-- Detecting CXX compile features-- Detecting CXX compile features - done-- Configuring done-- Generating done-- build files have been written to: \ /Users/helloworld/ninja-tutorial/incrementalninja-tutorial/incremental on main [!+?] via △ v3.24.2 via 🐍 v3.9.13 \on ☁️ took 4s❯ ninja[2000/2000] Linking CXX executable HelloWorld998ninja-tutorial/incremental on main [!+?] via 🐍 v3.9.13 on ☁️ took 5m54s❯ python generate_project_files.py --num_files 1001Doneninja-tutorial/incremental on main [!+?] via 🐍 v3.9.13 on ☁️❯ cmake -S sample_project/ -G Ninja-- Configuring done-- Generating done-- build files have been written to: \ /Users/helloworld/ninja-tutorial/incrementalninja-tutorial/incremental on main [!+?] via 🐍 v3.9.13 on ☁️ took 3s❯ ninja[2/2] Linking CXX executable HelloWorld1000
As you can see, Ninja is able to build the updated 1001-file project in only three seconds, compared to the 35 seconds it took to build its predecessor project consisting of 1000 files. Because Ninja had already built the 1000-file variant of the same project, it only had to add one more file to the build.
Conclusion
So, we’ve dived deep into build systems and Ninja, covering how to install and use Ninja for your C++ projects. Remember, tools like build systems are your best pals for automating software building, testing, and deployment. Ninja is just one awesome tool that can save you heaps of time.
And if you loved Ninja, you might want to take a peek at Earthly for more build automation improvements.
Happy coding!
Earthly Cloud: Consistent, Fast Builds, Any CI
Consistent, repeatable builds across all environments. Advanced caching for faster builds. Easy integration with any CI. 6,000 build minutes per month included.
Get Started Free