How to Add DataDog trace ID in Logs using Spring Boot + Logback

Question

OK, I spent quiet some time figuring out how to configure stuff to have DataDog trace ID in logs but couldn't get it working. To be clear what I'm looking for is to see trace IDs in logs message, the same way that adding spring-cloud-starter-sleuth to the classpath, automatically configure Slf4j/Logback to show trace IDs in log messages.

Where I've started:

1. We've got a simple web spring boot application running as a Docker container deployed as an AWS Elastic BeansTalk, whose logs go to CloudWatch and we read them there.
2. We have DataDog as a Java agent (thus no dependencies in pom.xml)
3. We have SLF4J/Logback in our dependencies list.
4. There's no other related depndencies (like dd-trace-ot or any opertracing libs)

What I did so far:

1. I found on SO that adding opentracing-spring-cloud-starter will add log integration automatically. But I couldn't get it working.
2. On DD website, it says configuring the pattern is enough to see the IDs, but in our case it didn't work. (is it because we don't have our logs a JSON?). Also, adding dd-trace-ot didn't help.

Notes:

1. We can't switch to JSON logs.
2. We can't switch to any other library (e.g. Slueth).
3. We can't go away from CloudWatch.

Can someone tell me how exactly I need to configure the application to see trace IDs in log messages? Is there any documentation or samples I can look at?

Answer 1

Do you have the ability to add some parameters in the logs sent. From the documentation you should be able to inject the trace id into your logs in a way that Datadog will interpret them.

You can also look at a parser to extract the trace id and span id from the raw log. This documentation should help you out on that.

Answer 2

From the documentation, if you don't have JSON logs, you need to include dd.trace_id and dd.span_id in your formatter:

If your logs are raw formatted, update your formatter to include dd.trace_id and dd.span_id in your logger configuration:

<Pattern>"%d{yyyy-MM-dd HH:mm:ss} %-5p %c{1}:%L -
%X{dd.trace_id:-0} %X{ dd.span_id:-0} - %m%n"</Pattern> ```

So if you add %X{dd.trace_id:-0} %X{ dd.span_id:-0}, it should work.