Compiled binary path based on compiler options

Question

I have a large program and library written in C++ and built with make. About a dozen options that are set in the makefile turn into preprocessor directives that change the implementation (with ifdefs, etc.). Right now I'm brute-forcing my build process by make cleaning when I change these compiler options before I run the code. I'd like to set up the system so that the binary's name changes based on the options. But, I'm worried that I'm going to miss one or add one in the future and forget to change the name, etc. Is there a clean way to handle this problem?

Some options I've considered:

1. manually create a binary name, such as APP.#{OPT_1}.#{OPT_2}.#{OPT_3}.#{OPT_4} when I build, and then run that one

2. create a hash (e.g., SHA1) from all of the compiler flags (CXXFLAGS) and put that hash into the name of my binary, such as APP.#{SHA1(CXXFLAGS)}. This has the value of extensibility in the future.

Any better approaches / recommendations?

Answer 1

Any better approaches / recommendations?

If I understand correctly, your GNU Make build system can build several variants of your executable, differentiated by preprocessor macros that are defined (or not) in the compilation commands depending on conditions that are tested in your Makefile and/or on the arguments that you pass to make. And you want to be able to build any of these variants independently, without needing a make clean to remove the artifacts of the previous build, which might well have been a build of a different variant.

This is one of the basic needs of build systems. The conventional solution is not the one you're thinking about - to somehow encode differentiations into the name of the executable. That won't work anyway, unless you do the same thing with the names of the object files that are linked into the executable. If you don't, then when you switch from variant X to variant Y, a variant-X object file foo.o that is not older than foo.cpp, will not need to be recompiled, even if should be for variant-Y, and that variant-X foo.o will be linked into the variant Y executable, no matter what it is called.

The conventional solution is to differentiate, per variant, the place where the compiler will output the object files and correspondingly the place where the linker outputs the executable. No doubt all of the C/C++ IDEs you have ever used allow you to build either a debug variant or a release variant of your project, and they differentiate the debug object files and executable from the release object files and executables by generating them in different subdirectories of the project directory, e.g.

<projdir>/Debug/{obj|bin}
<projdir>/Release/{obj|bin}

or maybe:

<projdir>/obj/{debug|release}
<projdir>/bin/{debug|release}

This approach automatically encodes the variant of an object file or executable into its absolute pathname,e.g.

<projdir>/Debug/obj/foo.o
<projdir>/bin/release/prog

without any further ado, and the variants can be built independently.

It's straightforward to implement this scheme in a makefile. Most of the IDEs that use it do implement it in the makefiles that they generate behind the scenes. And it's also straightforward to extend the scheme to more variants than just debug and release (although whatever variants you want, you'll certainly want debug and release variants of those variants).

Here's an illustration for a toy program that we want to build in any of the variants that we get for combinations of two build-properties that we'll call TRAIT_A and TRAIT_B:

 | TRAIT_A | TRAIT_B |
 |---------|---------|
 |    Y    |    Y    |
 |---------|---------|
 |    Y    |    N    |
 |---------|---------|
 |    N    |    Y    |
 |---------|---------|
 |    N    |    N    |

And we want to be able to build any of those variants in debug mode or release mode. TRAIT_{A|B} might map directly to a preprocessor macro, or to an arbitrary combination of preprocessor flags, compiler options and/or linkage options.

Our program, prog, is built from just one source file:

main.cpp

#include <string>
#include <cstdlib>

int main(int atgc, char * argv[])
{
    std::string cmd{"readelf -p .GCC.command.line "};
    cmd += argv[0];
    return system(cmd.c_str());
}

And all it does is invoke readelf to dump the linkage section .GCC.command.line within its own executable. That linkage section only exists when we compile or link with the GCC option -frecord-gcc-switches. So purely for the purpose of the demo we'll always compile and link with that option. Here's a makefile that adopts one way of differentiating all the variants: object files are compiled in ./obj[/trait...]; executables are linked in ./bin[/trait...]:

Makefile

CXX = g++
CXXFLAGS := -frecord-gcc-switches
BINDIR := ./bin
OBJDIR := ./obj

ifdef RELEASE
ifdef DEBUG
$(error RELEASE and DEBUG are mutually exclusive)
endif
CPPFLAGS := -DNDEBUG
CXXFLAGS += -O3
BINDIR := $(BINDIR)/release
OBJDIR := $(OBJDIR)/release
endif

ifdef DEBUG
ifdef RELEASE
$(error RELEASE and DEBUG are mutually exclusive)
endif
CXXFLAGS += -O0 -g
BINDIR := $(BINDIR)/debug
OBJDIR := $(OBJDIR)/debug
endif

ifdef TRAIT_A
CPPFLAGS += -DTRAIT_A   # or whatever
BINDIR := $(BINDIR)/TRAIT_A
OBJDIR := $(OBJDIR)/TRAIT_A
endif

ifdef TRAIT_B
CPPFLAGS += -DTRAIT_B   # or whatever
BINDIR := $(BINDIR)/TRAIT_B
OBJDIR := $(OBJDIR)/TRAIT_B
endif

SRCS :=  main.cpp
OBJS := $(OBJDIR)/$(SRCS:.cpp=.o)
EXE := $(BINDIR)/prog

.PHONY: all clean

all: $(EXE)

$(EXE): $(OBJS) | $(BINDIR)
    $(CXX) $(CPPFLAGS) $(CXXFLAGS) -o $@ $(LDFLAGS) $^ $(LIBS)

$(OBJDIR)/%.o: %.cpp | $(OBJDIR)
    $(CXX) -c -o $@ $(CPPFLAGS) $(CXXFLAGS) $<

$(BINDIR) $(OBJDIR):
    mkdir -p $@

clean:
    $(RM) $(EXE) $(OBJS)

Now let's build, say, two variants in debug mode and two other variants in release mode, one after the other

$ make DEBUG=1 TRAIT_A=1
mkdir -p obj/debug/TRAIT_A
g++ -c -o obj/debug/TRAIT_A/main.o -DTRAIT_A     -frecord-gcc-switches -O0 -g main.cpp
mkdir -p bin/debug/TRAIT_A
g++ -DTRAIT_A    -frecord-gcc-switches -O0 -g -o bin/debug/TRAIT_A/prog  obj/debug/TRAIT_A/main.o

$ make DEBUG=1 TRAIT_B=1
mkdir -p obj/debug/TRAIT_B
g++ -c -o obj/debug/TRAIT_B/main.o -DTRAIT_B     -frecord-gcc-switches -O0 -g main.cpp
mkdir -p bin/debug/TRAIT_B
g++ -DTRAIT_B    -frecord-gcc-switches -O0 -g -o bin/debug/TRAIT_B/prog  obj/debug/TRAIT_B/main.o

$ make RELEASE=1 TRAIT_A=1 TRAIT_B=1
mkdir -p obj/release/TRAIT_A/TRAIT_B
g++ -c -o obj/release/TRAIT_A/TRAIT_B/main.o -DNDEBUG -DTRAIT_A      -DTRAIT_B   -frecord-gcc-switches -O3 main.cpp
mkdir -p bin/release/TRAIT_A/TRAIT_B
g++ -DNDEBUG -DTRAIT_A   -DTRAIT_B   -frecord-gcc-switches -O3 -o bin/release/TRAIT_A/TRAIT_B/prog  obj/release/TRAIT_A/TRAIT_B/main.o

$ make RELEASE=1
g++ -c -o obj/release/main.o -DNDEBUG -frecord-gcc-switches -O3 main.cpp
g++ -DNDEBUG -frecord-gcc-switches -O3 -o bin/release/prog  obj/release/main.o

That last one is the release variant with neither TRAIT_A nor TRAIT_B.

We've now built four versions of program prog in different ./bin[/...] subdirectories of the project, from different object files that are in different ./obj[/...] subdirectories, and those versions will all tell us how they were differently built. Running in the order we built them:-

$ bin/debug/TRAIT_A/prog

String dump of section '.GCC.command.line':
  [     0]  -imultiarch x86_64-linux-gnu
  [    1d]  -D_GNU_SOURCE
  [    2b]  -D TRAIT_A
  [    36]  main.cpp
  [    3f]  -mtune=generic
  [    4e]  -march=x86-64
  [    5c]  -auxbase-strip obj/debug/TRAIT_A/main.o
  [    84]  -g
  [    87]  -O0
  [    8b]  -frecord-gcc-switches
  [    a1]  -fstack-protector-strong
  [    ba]  -Wformat
  [    c3]  -Wformat-security

$ bin/debug/TRAIT_B/prog

String dump of section '.GCC.command.line':
  [     0]  -imultiarch x86_64-linux-gnu
  [    1d]  -D_GNU_SOURCE
  [    2b]  -D TRAIT_B
  [    36]  main.cpp
  [    3f]  -mtune=generic
  [    4e]  -march=x86-64
  [    5c]  -auxbase-strip obj/debug/TRAIT_B/main.o
  [    84]  -g
  [    87]  -O0
  [    8b]  -frecord-gcc-switches
  [    a1]  -fstack-protector-strong
  [    ba]  -Wformat
  [    c3]  -Wformat-security

$ bin/release/TRAIT_A/TRAIT_B/prog

String dump of section '.GCC.command.line':
  [     0]  -imultiarch x86_64-linux-gnu
  [    1d]  -D_GNU_SOURCE
  [    2b]  -D NDEBUG
  [    35]  -D TRAIT_A
  [    40]  -D TRAIT_B
  [    4b]  main.cpp
  [    54]  -mtune=generic
  [    63]  -march=x86-64
  [    71]  -auxbase-strip obj/release/TRAIT_A/TRAIT_B/main.o
  [    a3]  -O3
  [    a7]  -frecord-gcc-switches
  [    bd]  -fstack-protector-strong
  [    d6]  -Wformat
  [    df]  -Wformat-security

$ bin/release/prog

String dump of section '.GCC.command.line':
  [     0]  -imultiarch x86_64-linux-gnu
  [    1d]  -D_GNU_SOURCE
  [    2b]  -D NDEBUG
  [    35]  main.cpp
  [    3e]  -mtune=generic
  [    4d]  -march=x86-64
  [    5b]  -auxbase-strip obj/release/main.o
  [    7d]  -O3
  [    81]  -frecord-gcc-switches
  [    97]  -fstack-protector-strong
  [    b0]  -Wformat
  [    b9]  -Wformat-security

We can clean the first one:

$ make DEBUG=1 TRAIT_A=1 clean
rm -f ./bin/debug/TRAIT_A/prog ./obj/debug/TRAIT_A/main.o

And the last one:

$ make RELEASE=1 clean
rm -f ./bin/release/prog ./obj/release/main.o

The second and third are still there and up to date:

$ make DEBUG=1 TRAIT_B=1
make: Nothing to be done for 'all'.

$ make RELEASE=1 TRAIT_A=1 TRAIT_B=1
make: Nothing to be done for 'all'.

For the exercise, you might consider refining the makefile to let you build, or clean, all the variants at the same time. Or to default to DEBUG if RELEASE not is defined, or vice versa. Or to fail if no valid combination of traits is selected, for some definition of valid.

BTW, note that preprocessor options are conventionally assigned in the make variable CPPFLAGS, for either C or C++ compilation; C compiler options are assigned in CFLAGS and C++ compiler options in CXXFLAGS. GNU Make's built-in rules assume that you follow these conventions.