hugopl/devblog

23 Apr 2014

It's benchmark time!

I started to write my own build system for C++ years ago, at my own pace, one line at the time… and sometimes no lines at all. Last months I did work a bit more on it and nowadays I would consider it in a pretty good shape, so it’s time for a benchmark!

The motivations behind of benchmarking meique (BTW this is the project name) was the most pure act of human curiosity, mainly after read about the tup benchmak.

Meique is slower than tup, on the other hand it doesn’t need the “complicated” tup setup, run with suid, etc. So my comparison here is with 2 other contenders: CMake/make and CMake/ninja.

Metodology

The metodology is very similar to the one used by the tup benchmak, just the code generator differ, so there’s a dummy C project with 4 shared libraries, and a main.c that uses them all, the number of files per shared libraries increase in each round, first 10 then 100, 1000 and 10000. i.e. 41, 401, 4001 and a 40001 files project. On each round the following operations are done:

Except for clean build, all other operations are repeated 3 times and the mean is used as result, there are also some other details that you can check in the run-test.sh script.

The computer used was a 8 cores intel i7 machine with 16GB on a hardware enabled RAID 0, make was ran with -j9.

Results

For my surprise, meique is faster than CMake/make and as expected it’s slower than CMake/ninja. Since speed is a concern but not the meique main focus as it’s on ninja I consider the results very good.

Ninja isn’t showing in the results due to a bug, it can’t link a shared library with 10000 files due to the very long argument list, meique had this very same bug, now fixed.

1.95x faster on the 40K files project! And I don’t know why because clean build is the most simple task.



There are still much room for improvements on meique, so the next goal is simple: to be faster than a ninja! ;-)

comments powered by Disqus