pyCUDA with Flask gives pycuda._driver.LogicError: cuModuleLoadDataEx

Question

I want to run a pyCUDA code on a flask server. The file runs correctly directly using python3 but fails when the corresponding function is called using flask.

Here is the relevant code:

cudaFlask.py:

import pycuda.autoinit
import pycuda.driver as drv
import numpy

from pycuda.compiler import SourceModule

def cudaTest():
    mod = SourceModule("""
        int x = 4;
    """)

    print ("done")
    return

if __name__ == "__main__":
    cudaTest()

server.py (only the part which calls the function):

@app.route('/bundle', methods=['POST'])
def bundle_edges():
    cudaTest()
    return "success"

On running python cudaFlask.py I get the output done as expected but on starting the server and doing POST request at website/bundle I get the following error on the flask console:

pycuda._driver.LogicError: cuModuleLoadDataEx failed: invalid device context - 

on the line mod = SourceModule...

Where am I going wrong? There is a similar question out there but it has not been answered yet.

Answer 1

Solved the issue with lazy loading in flask and making the context manually (i.e. without pycuda.autoinit in PyCUDA.

Refer this for lazy loading in flask.

My views.py file:

import numpy as np
import pycuda.driver as cuda
from pycuda.compiler import SourceModule

def index():
    cuda.init()
    device = cuda.Device(0) # enter your gpu id here
    ctx = device.make_context()

    mod = SourceModule("""
        int x = 4;
    """)

    ctx.pop() # very important

    print ("done")
    return "success"