Mysterious Filebeat 7 X-Pack issue using Docker image

Question

I've also posted this as a question on the official Elastic forum, but that doesn't seem super frequented.

https://discuss.elastic.co/t/x-pack-check-on-oss-docker-image/198521

At any rate, here's the query:

We're running a managed AWS Elasticsearch cluster — not ideal, but that's life — and run most the rest of our stuff with Kubernetes. We recently upgraded our cluster to Elasticsearch 7, so I wanted to upgrade the Filebeat service we have running on the Kubernetes nodes to capture logs.

I've specified image: docker.elastic.co/beats/filebeat-oss:7.3.1 in my daemon configuration, but I still see

Connection marked as failed because the onConnect callback failed:
request checking for ILM availability failed:
401 Unauthorized: {"Message":"Your request: '/_xpack' is not allowed."}

in the logs. Same thing when I've tried other 7.x images. A bug? Or something that's new in v7?

The license file is an Apache License, and the build when I do filebeat version inside the container is a4be71b90ce3e3b8213b616adfcd9e455513da45.

Answer 1

It turns out that starting in one of the 7.x versions they turned on index lifecycle management checks by default. ILM (index lifecycle management) is an X-Pack feature, so turning this on by default means that Filebeat will do an X-Pack check by default.

This can be fixed by adding setup.ilm.enabled: false to the Filebeat configuration. So, not a bug per se in the OSS Docker build.