Keras model is stuck at prediction level when running the server with gunicorn

Question

I am using keras model to predict something and used flask-restplus to make this as a REST service. so far everything is working fine when I run the server with python app.py command.

But I wanted to run this through gunicorn and nginx (gunicorn -b 0.0.0.0:5000 --error-logfile - -w 4 --access-logfile - --preload wsgi:app -t 900 --preload --reload), every configuration is correct since It was working fine with the normal endpoint but predict service is not working.

I have investigated and found that my code is stuck at prediction level p = model.predict(np.array(X))

can some one tell me what could be the issue.

Answer 1

Running TF2 alpha 0 with Keras, the issue has been the --preload flag for me. I suspect the session/graph being the issue. Not an expert though.