Passing Serilog LogContext between processes

Question

Is it possible in Serilog get all properties from LogContext? Does LogContext support serialization/deserialization to pass context between processes?

Answer 1

There is no bulletproof way to pass LogContext state between processes. However there is a solution that will work in most cases (limitations are listed at the answer bottom).

LogContext is a static class that exposes following methods:

public static class LogContext
{
    public static ILogEventEnricher Clone();
    public static IDisposable Push(ILogEventEnricher enricher);
    public static IDisposable Push(params ILogEventEnricher[] enrichers);
    [Obsolete("Please use `LogContext.Push(properties)` instead.")]
    public static IDisposable PushProperties(params ILogEventEnricher[] properties);
    public static IDisposable PushProperty(string name, object value, bool destructureObjects = false);
}

The pair of Clone() and Push(ILogEventEnricher enricher) methods looks very promising but how to pass returned instance of ILogEventEnricher between processes?

Let's dig into LogContext source code. First of all we see that all Push variations alter private Enrichers property of ImmutableStack<ILogEventEnricher> type by adding new instance of ILogEventEnricher. Most frequently used method PushProperty(string name, object value, bool destructureObjects = false) adds instance of PropertyEnricher:

public static IDisposable PushProperty(string name, object value, bool destructureObjects = false)
{
    return Push(new PropertyEnricher(name, value, destructureObjects));
}

Clone() just returnes stack of enrichers wrapped into SafeAggregateEnricher:

public static ILogEventEnricher Clone()
{
    var stack = GetOrCreateEnricherStack();
    return new SafeAggregateEnricher(stack);
}

So we could pass the state of LogContext by extracting the values stored in enricher returned by Clone() method. ILogEventEnricher has the only method:

public interface ILogEventEnricher
{
    void Enrich(LogEvent logEvent, ILogEventPropertyFactory propertyFactory);
}

Enrichers affect LogEvent by adding instances of LogEventPropertyValue to its Properties dictionary. Unfortunatelly, there is no easy way to save state of event property object since LogEventPropertyValue is an abstract class with such descendants as ScalarValue, DictionaryValue, etc.

However we can use custom implementation of ILogEventPropertyFactory that will collect all created properties and expose them for transfer between processes. The drawback is that not all enrichers use propertyFactory. Some of them create properties directly like for example ThreadIdEnricher:

public void Enrich(LogEvent logEvent, ILogEventPropertyFactory propertyFactory)
{
    logEvent.AddPropertyIfAbsent(new LogEventProperty(ThreadIdPropertyName, new ScalarValue(Environment.CurrentManagedThreadId)));
}

However PropertyEnricher which is probably the most interesting for our case uses the factory:

public void Enrich(LogEvent logEvent, ILogEventPropertyFactory propertyFactory)
{
    if (logEvent == null) throw new ArgumentNullException(nameof(logEvent));
    if (propertyFactory == null) throw new ArgumentNullException(nameof(propertyFactory));
    var property = propertyFactory.CreateProperty(_name, _value, _destructureObjects);
    logEvent.AddPropertyIfAbsent(property);
}

Now the plan should be clear:

1. Create custom implementation of ILogEventPropertyFactory that collects all created properties.
2. Clone LogContext to get aggregate enricher.
3. Call Enrich on this enricher that will eventually call CreateProperty in our factory for each property in LogContext.
4. Serialize received properties and pass between processes.
5. Deserialize the properties on the other end and populate context.

Here is a code that implements these steps:

PropertyValue class:

public class PropertyValue
{
    public string Name { get; set; }

    public object Value { get; set; }

    public bool DestructureObjects { get; set; }

    public PropertyValue(string name, object value, bool destructureObjects)
    {
        Name = name;
        Value = value;
        DestructureObjects = destructureObjects;
    }
}

LogContextDump class:

public class LogContextDump
{
    public ICollection<PropertyValue> Properties { get; set; }

    public LogContextDump(IEnumerable<PropertyValue> properties)
    {
        Properties = new Collection<PropertyValue>(properties.ToList());
    }

    public IDisposable PopulateLogContext()
    {
        return LogContext.Push(Properties.Select(p => new PropertyEnricher(p.Name, p.Value, p.DestructureObjects) as ILogEventEnricher).ToArray());
    }
}

CaptureLogEventPropertyFactory class:

public class CaptureLogEventPropertyFactory : ILogEventPropertyFactory
{
    private readonly List<PropertyValue> values = new List<PropertyValue>();

    public LogEventProperty CreateProperty(string name, object value, bool destructureObjects = false)
    {
        values.Add(new PropertyValue(name, value, destructureObjects));
        return new LogEventProperty(name, new ScalarValue(value));
    }

    public LogContextDump Dump()
    {
        return new LogContextDump((values as IEnumerable<PropertyValue>).Reverse());
    }
}

LogContextSerializer class:

public static class LogContextSerializer
{
    public static LogContextDump Serialize()
    {
        var logContextEnricher = LogContext.Clone();
        var captureFactory = new CaptureLogEventPropertyFactory();
        logContextEnricher.Enrich(new LogEvent(DateTimeOffset.Now, LogEventLevel.Verbose, null, MessageTemplate.Empty, Enumerable.Empty<LogEventProperty>()), captureFactory);

        return captureFactory.Dump();
    }

    public static IDisposable Deserialize(LogContextDump contextDump)
    {
        return contextDump.PopulateLogContext();
    }
}

Usage sample:

string jsonData;
using (LogContext.PushProperty("property1", "SomeValue"))
using (LogContext.PushProperty("property2", 123))
{
    var dump = LogContextSerializer.Serialize();
    jsonData = JsonConvert.SerializeObject(dump);
}

//  Pass jsonData between processes

var restoredDump = JsonConvert.DeserializeObject<LogContextDump>(jsonData);
using (LogContextSerializer.Deserialize(restoredDump))
{
    //  LogContext is the same as when Clone() was called above
}

I used serialization to JSON here, however with such primitive types as LogContextDump and PropertyValue you could use any serialization mechanism you want.

As I've already said this solution has its drawbacks:

1. Restored LogContext is not 100% the same as the original one. Original LogContext could have different kinds of enrichers but restored context will have only instances of PropertyEnricher. It should not be a problem however if you use LogContext as a simple bag for the properties like in sample above.

2. This solution will not work if some of context enrichers create properties directly bypassing propertyFactory.

3. This solution will not work if some of added values have a type that could not be serialized. Value property in above PropertyValue has type of object. You could add properties of any type to LogContext but you should have a way to serialize their data for passing between processes. Above serialization/deserialization to JSON will work for simple types, but you have to adjust it if you add some complex values to LogContext.