Elasticsearch Delete by Query Version Conflict

Question

I am using Elasticsearch version 5.6.10. I have a query that deletes records for a given agency, so they can later be updated by a nightly script.

The query is in elasticsearch-dsl and look like this:

def remove_employees_from_search(jurisdiction_slug, year):
    s = EmployeeDocument.search()
    s = s.filter('term', year=year)
    s = s.query('nested', path='jurisdiction', query=Q("term", **{'jurisdiction.slug': jurisdiction_slug}))
    response = s.delete()
    return response

The problem is I am getting a ConflictError exception when trying to delete the records via that function. I have read this occurs because the documents were different between the time the delete process started and executed. But I don't know how this can be, because nothing else is modifying the records during the delete process.

I am going to add s = s.params(conflicts='proceed') in order to silence the exception. But this is a band-aid as I do not understand why the delete is not processing as expected. Any ideas on how to troubleshoot this? A snapshot of the error is below:

ConflictError:TransportError(409,
u'{  
   "took":10,
   "timed_out":false,
   "total":55,
   "deleted":0,
   "batches":1,
   "version_conflicts":55,
   "noops":0,
   "retries":{  
      "bulk":0,
      "search":0
   },
   "throttled_millis":0,
   "requests_per_second":-1.0,
   "throttled_until_millis":0,
   "failures":[  
      {  
         "index":"employees",
         "type":"employee_document",
         "id":"24681043",
         "cause":{  
            "type":"version_conflict_engine_exception",
            "reason":"[employee_document][24681043]: version conflict, current version [5] is different than the one provided [4]",
            "index_uuid":"G1QPF-wcRUOCLhubdSpqYQ",
            "shard":"0",
            "index":"employees"
         },
         "status":409
      },
      {  
         "index":"employees",
         "type":"employee_document",
         "id":"24681063",
         "cause":{  
            "type":"version_conflict_engine_exception",
            "reason":"[employee_document][24681063]: version conflict, current version [5] is different than the one provided [4]",
            "index_uuid":"G1QPF-wcRUOCLhubdSpqYQ",
            "shard":"0",
            "index":"employees"
         },
         "status":409
      }

Answer 1

First, this is a question that was asked 2 years ago, so take my response with a grain of salt due to the time gap.

I am using the javascript API, but I would bet that the flags are similar. When you index or delete there is a refresh flag which allows you to force the index to have the result appear to search.

I am not an Elasticsearch guru, but the engine must perform some systematic maintenance on the indices and shards so that it moves the indices to a stable state. It's probably done over time, so you would not necessarily get an immediate state update. Furthermore, from personal experience, I have seen when delete does not seemingly remove the item from the index. It might mark it as "deleted", give the document a new version number, but it seems to "stick around" (probably until general maintenance sweeps run).

Here I am showing the js API for delete, but it is the same for index and some of the other calls.

client.delete({
  id: string,
  index: string,
  type: string,
  wait_for_active_shards: string,
  refresh: 'true' | 'false' | 'wait_for',
  routing: string,
  timeout: string,
  if_seq_no: number,
  if_primary_term: number,
  version: number,
  version_type: 'internal' | 'external' | 'external_gte' | 'force'
})

https://www.elastic.co/guide/en/elasticsearch/client/javascript-api/current/api-reference.html#_delete

refresh 'true' | 'false' | 'wait_for' - If true then refresh the affected shards to make this operation visible to search, if wait_for then wait for a refresh to make this operation visible to search, if false (the default) then do nothing with refreshes.

For additional reference, here is the page on Elasticsearch refresh info and what might be a fairly relevant blurb for you. https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-refresh.html

Use the refresh API to explicitly refresh one or more indices. If the request targets a data stream, it refreshes the stream’s backing indices. A refresh makes all operations performed on an index since the last refresh available for search.

By default, Elasticsearch periodically refreshes indices every second, but only on indices that have received one search request or more in the last 30 seconds. You can change this default interval using the index.refresh_interval setting.