Parse log4j and Java exception logs

Question

I have logs with errors in the following format:

2014-01-30 16:15:04:720 GMT [commandHandler-thread-3] ERROR com.example.Main 123-1234567-1234567 - Something bad happened.
java.lang.RuntimeException: Something bad happened.
        at ...
Caused by: java.lang.RuntimeException: ...
        at ...
        at ...
        ... 13 more
Caused by: java.lang.RuntimeException: org.hibernate.exception.ConstraintViolationException: Could not execute JDBC batch update
        at ...
        at ...
        ... 18 more
Caused by: org.hibernate.exception.ConstraintViolationException: Could not execute JDBC batch update
        at ...
        at ...
        ... 19 more
Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Duplicate entry '123-1234567-1234567-2014-01-31 06:52:11' for key 'PRIMARY'
        at ...
        at ...
        ... 32 more
2014-01-31 06:58:02:933 GMT ...

I want to parse it with grep, awk, sed, whatever and produce something like this:

<filename> 123-1234567-1234567 - Something bad happened: com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Duplicate entry '123-1234567-1234567-2014-01-31 06:52:11' for key 'PRIMARY'

So basically, I want to filter all the ERROR lines and the last 'caused by' line in that group (groups are delimited by the log4j dates). If there is no 'caused by' line, I can simply have

<filename> 123-1234567-1234567 - Something bad happened:

EDIT: I tried something like this:

grep "commandHandler.*ERROR\|^\S*Caused by"

but I don't want to get the 'caused by' lines not belonging to that particular exception.

Answer 1

This is what I have so far, still need to remove the "." in "happened." and remove "Caused by:". I have to go soon though, hope it helps so far. Im no AWK guru though!

 awk '{ 
 {for (x=1;x<=NF;x++)
    if ($x~"ERROR") {
    f++ 
    {if (c !~ f)  print "<"file">",a,b}
    a=$(x+2)" - "$(x+4)" "$(x+5)" "$(x+6)}
} 
{
   if (match($0,"Caused by:")) 
   b=$0
} 
{c=f;file=FILENAME}}
END {
print "<"file">",a,b}' javalogs* | sed 1d