reduce stack size in debug session to catch endless recursion

Question

I have a algorithm which "sometimes" did not break the recursion and ends up in a stack overflow. Now I want to debug that situation. gdb points me directly to the line where I overwrite the stack boarder, but it is impossible to get a stacktrace because gdb needs endless time to unwind the stack and display it in ddd.

I now simply want to reduce the stack size to get the corruption much earlier. So I need a way to reduce to stack for my running task. There are no threads involved so it should be not such a big deal.

But I have no idea if gdb itself has an option to manipulte the stack (size) itself or I need to programmatic change the size at start of my prog or I can maybe reduce the size from the os command line ( linux ) after starting the debug session.

Maybe it is also possible the stop if the call tree reaches a maximum depth if gdb has such an option.

Answer 1

Try to use ulimit -s (stack_size) in shell where you run your program or you can do it programmatically Change stack size for a C++ application in Linux during compilation with GNU compiler

EDIT:

We can write simple nohup-like program, which send SIGSTOP to itself, and than execute it's arguments. Let's name it runpaused

//runpaused.c
#include <signal.h>
#include <unistd.h>

int main(int argc, char **argv)
{
    raise(SIGSTOP);
    return execvp(argv[1], argv+1);
}

Than, we can do next things: compile runpaused

$ gcc runpaused.c -o runpaused

limit stack for current shell

$ ulimit -s 32

run our program via runpaused, use sh as example

$ ./runpaused sh -c "echo Hello"

From other terminal: find it's PID

$ ps -ef | grep -v grep | grep runpaused

load gdb with our program

$ gdb $(which sh)

attach to PID

(gdb) attach PID

send signal SIGCONT from gdb

(gdb) signal SIGCONT

continue program execution from gdb

(gdb) continue

After several times of continuing, we get what we want.