Micrometer traceId is lost when switching threads in Kotlin's coroutines

Question

In a simple kotlin Spring Boot 3 application, with Micrometer Tracing, I was expecting the trace context to propagate between threads. However that seems not to be the case.

In this simple RestController we can verify that the traceId is lost after calling the kotlin delay method. In the second log statement, we are on a different Thread and the traceId is null.

import io.micrometer.tracing.Tracer
import kotlinx.coroutines.delay
import org.slf4j.Logger
import org.slf4j.LoggerFactory
import org.springframework.web.bind.annotation.GetMapping
import org.springframework.web.bind.annotation.RestController

@RestController
class TestController(
    val tracer: Tracer
) {

    private val logger: Logger = LoggerFactory.getLogger(this::class.java)

    @GetMapping("/test")
    suspend fun test() {
        logger.info("before: ${tracer.currentTraceContext().context()?.traceId()}")
        delay(50)
        logger.info("after:  ${tracer.currentTraceContext().context()?.traceId()}")
    }
}

I thought the Micrometer library would propagate the trace context when the thread changes. Please note that I'm including the context-propagation library.

dependencies {
    implementation("org.springframework.boot:spring-boot-starter-webflux")
    implementation("org.springframework.boot:spring-boot-starter-actuator")
    implementation("io.micrometer:micrometer-tracing-bridge-brave")
    implementation("org.jetbrains.kotlin:kotlin-reflect")
    implementation("org.springframework.cloud:spring-cloud-starter")
    implementation("org.jetbrains.kotlinx:kotlinx-coroutines-reactor")
    implementation("io.micrometer:context-propagation:1.0.2")
    testImplementation("org.springframework.boot:spring-boot-starter-test")

}

Am I missing something here?

Answer 1

Not perfect solutions, but they work on your minimum reproducible example.
Consider them as workarounds.

2 approaches:

1. Manually set context via withContext(observationRegistry.asContextElement()) for each endpoint:

@RestController
class TestController(
    val tracer: Tracer,
    val observationRegistry: ObservationRegistry
) {

    private val logger: Logger = LoggerFactory.getLogger(this::class.java)

    @GetMapping("/test")
    suspend fun test(): Unit = withContext(observationRegistry.asContextElement()) {
        log.info("Before delay, thread.name=${Thread.currentThread().name}, tracerId=${tracer.currentTraceContext().context()?.traceId()}")
        delay(100)
        log.info("After delay, thread.name=${Thread.currentThread().name}, tracerId=${tracer.currentTraceContext().context()?.traceId()}")
    }
}

2. Global config to propagate context for each request:

• add spring.reactor.context-propagation=auto to application.properties
• define the bean that propagates observability context to coroutines

@Bean
fun coObservabilityFilter(observationRegistry: ObservationRegistry): CoWebFilter =
    object : CoWebFilter() {
        override suspend fun filter(
            exchange: ServerWebExchange,
            chain: CoWebFilterChain
        ) {
            withContext(observationRegistry.asContextElement()) {
                chain.filter(exchange)
            }
        }
    }

The workarounds are taken from the following issues:

• https://github.com/micrometer-metrics/micrometer/issues/4754
• https://github.com/micrometer-metrics/tracing/issues/174

These issues are still open, so maybe there will be a more handy solution soon