Responding to 429 exceptions (throttling) using Castle.Windsor and Polly

Question

We are struggling with 429 HTTP exceptions (coming from SharePoint online or Microsoft Graph), and I want to leverage Polly and Castle.Windsor to handle this.

My code is (in excerpts)

Registration of Polly stuff in my Castle.Windsor container:

_container.Register(Component.For<IRepository>()
            .ImplementedBy<Repository>()
            .DependsOn(Dependency.OnValue<ImportSettings>(_importSettings))
            .Interceptors(InterceptorReference.ForKey("throttle")).Anywhere
            .LifestyleTransient());

 _container.Register(Component.For<WaitAndRetryInterceptor<WebException>>().LifeStyle.Singleton
    .Named("throttle"));

My Polly stuff:

 public class WaitAndRetryInterceptor<T> : IInterceptor where T : WebException
    {
        private readonly RetryPolicy _policy;

        public void Intercept(IInvocation invocation)
        {
            var dictionary = new Dictionary<string, object> {{"methodName", invocation.Method.Name}};

            _policy.Execute(invocation.Proceed, dictionary);
        }

        public WaitAndRetryInterceptor()
        {
            _policy =
                Policy
                    .Handle<T>()
                    .WaitAndRetry(new[]
                    {
                        TimeSpan.FromSeconds(16), TimeSpan.FromSeconds(32), TimeSpan.FromSeconds(64),
                        TimeSpan.FromSeconds(128), TimeSpan.FromSeconds(256), TimeSpan.FromSeconds(512)
                    });
        }
    }

So this implementation covers my need - but it is insanely conservative. So I have tried to implement direct support for the 429 exception thrown - and specifically support for the Reply-After header available from the server.

I found out from this https://github.com/App-vNext/Polly/issues/414, that I need to implement support for one of the overloads taking a sleepDurationProvider, but I am having issues getting my code right.

My implementation was this:

_policy =
  Policy
    .Handle<HttpRequestException>()
    .OrResult<HttpResponseMessage>(r => r.StatusCode == HttpStatusCode.Accepted) //needs to be changed to 429
    .WaitAndRetryAsync(
        retryCount: 3,
        sleepDurationProvider: (retryCount, response) =>
        {
            return getServerWaitDuration(response);
        })
;

The getServerWaitDurationsimply returns a TimeSpan

private TimeSpan getServerWaitDuration(DelegateResult<HttpResponseMessage> response)
{
    return TimeSpan.Zero;  //not actual code ;-)
}

The idea is that I will simply look at the headers of the response from the server and pass the timespan back to the sleepDurationProvider.

However - I am getting errors from the line where I configure the sleepDurationProvider. I am told that the (retryCount, response) is an 'incompatable anonymous function signature'

I feel that I am missing something obvious here. But why? How do I get access to the response object to extract the Retry-After duration?

Answer 1

You might look into the headers like this

public AsyncRetryPolicy<HttpResponseMessage> GetRetryPolicy()
        {
            return HttpPolicyExtensions
                .HandleTransientHttpError()
                .OrResult(msg => msg.StatusCode == System.Net.HttpStatusCode.TooManyRequests)
                .OrResult(r =>  r?.Headers?.RetryAfter != null)
                .WaitAndRetryAsync(
                    3,
                    sleepDurationProvider: (retryCount, response, context) =>
                    {
                        return response.Result.Headers.RetryAfter.Delta.Value;
                    },
                    onRetryAsync: (e, ts, i, ctx) => Task.CompletedTask
                );
        }