How to handle timeouts when a process receives SIGSTOP and SIGCONT?

Question

I have some Python code which uses threading.Timer to implement a 60-second timeout for an operation.

The problem is that this code runs in a job-control environment where it may get pre-empted by a higher priority job. In this case it will be sent SIGSTOP, and then some time later, SIGCONT. I need a way to somehow notice that this has happened and reset the timeout: obviously the operation hasn't really timed out if it's been suspended for the whole 60 seconds.

I tried to add a signal handler for SIGCONT but this seems to get executed after the code provided to threading.Timer has been executed.

Is there some way to achieve this?

Answer 1

A fairly simple answer that occurred to me after posting this is to simply break up the timer into multiple sub-timers, e.g. having 10 6-second timers instead where each one starts the next one in a chain. That way, if I get suspended, I only lose one of the timers and still get most of the wait before timing out.

This is of course not foolproof, especially if I get repeatedly suspended and restarted, but it's easy to do and seems like it might be good enough.

Answer 2

You need to rethink what you're asking for; a timeout reflects elapsed time (wall time); you want to know the time used by your process.

Fortunately you can measure this with getrusage: http://docs.python.org/library/resource.html

You'll still need to set a timeout; when it returns, measure the increase in user or system time usage since the start of the operation and terminate the operation if it exceeds the limit, else reschedule the timeout appropriately.