Serilog with Elasticsearch sink, indexformat has no rolling date

Question

I use Serilog with the Elasticsearch sink to push my logs into Elastic, as well as my transactions in APM in a .net 5 application.

When the Logger is created, it use the current datetime as part of the indexformat (mslogs-yyyy.MM.dd) but as soon as the container is running, this is still the same index name that is used, regardless of the current time. For example if a container is started on 2021/06/25, the index is created as mslogs-2021.06.25 and it's kept for the whole lifetime of the container, so logs beeing produced on following days are still pushed in that index.

As my Ops put policies on Elk to remove old logs, based on index name (datetime part) fresh logs may be removed because they are put into an "old index".

Here is my Logger builder

    public static Logger GetELKLogger(IConfiguration config, string varEnv = "ElasticSearchLog")
    {
        var configuration = config.Get<Configuration>(varEnv);

        return new LoggerConfiguration()
        .ReadFrom.Configuration(config)
        .Enrich.WithElasticApmCorrelationInfo()
        .WriteTo.Elasticsearch(new ElasticsearchSinkOptions(new Uri(configuration.ElasticSearchLog))
        {
            AutoRegisterTemplate = true,
            IndexFormat = $"mslogs-{DateTime.UtcNow:yyyy.MM.dd}", // called once at startup !!
            DetectElasticsearchVersion = true,
            RegisterTemplateFailure = RegisterTemplateRecovery.IndexAnyway,
            AutoRegisterTemplateVersion = AutoRegisterTemplateVersion.ESv7,
            FailureCallback = e => Console.WriteLine($"Unable to submit event {e.RenderMessage()} to ElasticSearch. Full message : " + e.Serialize()),
            EmitEventFailure = EmitEventFailureHandling.WriteToSelfLog |
                                    EmitEventFailureHandling.WriteToFailureSink |
                                    EmitEventFailureHandling.RaiseCallback,
            BufferCleanPayload = (failingEvent, statuscode, exception) =>
            {
                dynamic e = JObject.Parse(failingEvent);
                return JsonConvert.SerializeObject(new Dictionary<string, object>()
                    {
                        { "action", "DeniedByElasticSearch"},
                        { "@timestamp",e["@timestamp"]},
                        { "level","Error"},
                        { "message","Error: "+e.message},
                        { "messageTemplate",e.messageTemplate},
                        { "failingStatusCode", statuscode},
                        { "failingException", exception}
                    });
            },
            CustomFormatter = new EcsTextFormatter()
        })
        .CreateLogger();
    }

and the logger creation

    public class Startup
    {
        public void ConfigureServices(IServiceCollection services)
        {
            // Logger
            Log.Logger = SharedLibraries.Logging.LoggerFactory.GetELKLogger(Configuration);

Should I force the logger instance to be recreated once a day to benefit from the DateTime.UtcNow updating ? Or is there any more elegant solution ?

Dependencies

    <TargetFramework>net5.0</TargetFramework>

    <PackageReference Include="Serilog" Version="2.10.0" />
    <PackageReference Include="Serilog.Extensions.Hosting" Version="4.1.2" />
    <PackageReference Include="Serilog.Extensions.Logging" Version="3.0.1" />
    <PackageReference Include="Serilog.Settings.Configuration" Version="3.1.0" />
    <PackageReference Include="Serilog.Sinks.Console" Version="3.1.1" />
    <PackageReference Include="Serilog.Sinks.ElasticSearch" Version="8.4.1" />
    <PackageReference Include="Serilog.Sinks.Seq" Version="5.0.1" />

    <PackageReference Include="Elastic.Apm.NetCoreAll" Version="1.11.0" />
    <PackageReference Include="Elastic.Apm.SerilogEnricher" Version="1.5.3" />
    <PackageReference Include="Elastic.CommonSchema.Serilog" Version="1.5.3" />

Answer 1

The IndexFormat property is a .NET format string over the event's timestamp (in UTC), so you shouldn't use $ to interpolate the value:

IndexFormat = "mslogs-{0:yyyy.MM.dd}"