AWS Batch:/usr/local/bin/python: cannot execute binary file

Question

I built an AWS Batch compute environment. I want to run a python script in jobs. Here is the docker file I'm using :

FROM python:slim
RUN apt-get update
RUN pip install boto3 matplotlib awscli
COPY runscript.py /
ENTRYPOINT ["/bin/bash"]

The command in my task definition is :

python /runscript.py

When I submit a job in AWS console I get this error in CloudWatch:

/usr/local/bin/python: /usr/local/bin/python: cannot execute binary file

And the job gets the status FAILED.

What is going wrong? I run the container locally and I can launch the script without any errors.

Answer 1

Delete your ENTRYPOINT line. But replace it with the CMD that says what the container is actually doing.

There are two parts to the main command that a Docker container runs, ENTRYPOINT and CMD; these are combined together into one command when the container starts. The command your container is running is probably something like

/bin/bash python /runscript.py

So bash finds a python in its $PATH (successfully), and tries to run it as a shell script (leading to that error).

You don't strictly need an ENTRYPOINT, and here it's causing trouble. Conversely there's a single thing you usually want the container to do, so you should just specify it in the Dockerfile.

# No ENTRYPOINT
CMD ["python", "/runscript.py"]

Answer 2

You can try with following docker file and task definition.

Docker File

FROM python:slim
RUN apt-get update
RUN pip install boto3 matplotlib awscli
COPY runscript.py /
CMD ["/bin/python"]

Task Definition

['/runscript.py']

By passing script name in task definition will give you flexibility to run any script while submitting a job. Please refer below example to submit a job and override task definition.

import boto3
session = boto3.Session()
batch_client = session.client('batch')

response = batch_client.submit_job(
            jobName=job_name,
            jobQueue=AWS_BATCH_JOB_QUEUE,
            jobDefinition=AWS_BATCH_JOB_DEFINITION,
            containerOverrides={
                'command': [
                    '/main.py'
                ]
            }
        )