what to log and what to ignore in Global.asax Application_Error()?

Question

Currently I have the following code snippet to handle errors in Global.asax Application_Error():

    var ex = server.GetLastError();
    server.ClearError();
    if (ex is HttpUnhandledException && ex.InnerException != null)
        ex = ex.InnerException;
    Trace.TraceError(ex.ToString());
    // send some output to the user that they should contact support etc

It catches all errors, including 404 errors for example (when the user types a nonexistent .aspx URL, or when the browser asks for stuff like favicon.ico and static file requests are set up to go through the pipeline). There are two problems with this: this should not be logged (the log is full of useless 404 errors) and the original HTTP code should be sent back (otherwise search bots may index error pages etc.).

How would you decide what to log and what to ignore here? Basically I want to include application/business errors, which indicate bugs, and ignore "normal" HTTP errors. I also don't just want to exclude 404 errors only, there is also 403 and who knows what else.

Answer 1

Your approach is good: you log unhandled errors. The point of that is to weed out bugs. Humans cannot avoid bugs but we can quickly fix them.

Start with logging all errors. Then blacklist individual errors or categories of errors that you learn are useless. Within a few iterations you'll have a fairly clean log.

Remove everything that is not actionable. For example, your website will have bots performing crazy requests and so on. No action necessary, so you can remove them from the log.

I recommend that you also log seemingly useless errors into a supplementary log file so that you can scroll through it from time to time to make sure that everything is alright.

You especially want to know the frequency of normally harmless errors. Three deadlocks per day might be cool with you, but 300 might be too much.

I'll contribute a sketch of what your global error handler might contain:

static bool IsExceptionIgnored(Exception exception)
{
    if (exception == null) throw new ArgumentNullException("exception");

    var nestedExceptions = exception.GetExceptionChain();
    return nestedExceptions.Any(ex =>
        ex is ViewStateException ||
        ex.Message.Contains("Timeout") ||
        ex.Message.StartsWith("Invalid viewstate") ||
        ex.Message.Contains("potentially dangerous") ||
        ex.Message.Contains("The remote host closed the connection") ||
        ex.Message.Contains("System.Web.UI.ViewStateException: Invalid viewstate") ||
        ex.Message.Contains("System.Web.Hosting.IIS7WorkerRequest.RaiseCommunicationError") ||
        ex.Message.Contains("0x80070032") ||
        (ex is HttpException && ((HttpException)ex).GetHttpCode() == 404) ||
        ex is ThreadAbortException);
}
public static IEnumerable<Exception> GetExceptionChain(this Exception exception)
{
    if (exception == null) throw new ArgumentNullException("exception");

    for (var current = exception; current != null; current = current.InnerException)
        yield return current;
}