Scikit-learn Multicore: "AttributeError: StdIn instance has no attribute 'close'"

Question

I'm using PyCharm to run a RandomForestRegressor from Scikit-learn. This works fine when executing the whole script:

clf = ensemble.RandomForestClassifier(n_estimators=100, n_jobs = 4, verbose=1, oob_score=True)
clf = clf.fit(x_train, y_train)

When I try to run the line from the console I get this error repeatedly:

AttributeError: StdIn instance has no attribute 'close'
  File "/usr/local/Cellar/python/2.7.6_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/process.py", line 249, in _bootstrap
    sys.stdin.close()

If I change the n_jobs parameter to n_jobs=1 I don't get an error, but it can't take advantage of multithreading.

I'm assuming this has to do with the fact that the PyCharm console is redirecting stdin, but I'm not sure how to get it to work. Any thoughts are appreciated - thanks!

Answer 1

Coming a little late to the party, but I encountered the same problem myself just now.

The workaround, suggested in the other answer, is to avoid the interactive interpreter of PyCharm. This, of course, works but does not satisfy me since I like to use the interactive interpreter very much.

I found a different workaround that solves the problem and allows the usage of the interactive interpreter. Before each creation of a new process via multiprocessing.Process(), the following code snippet has to be applied:

if not hasattr(sys.stdin, 'close'):
    def dummy_close():
        pass
    sys.stdin.close = dummy_close

Now, multiprocessing can call close() on sys.stdin just fine, and everything works smoothly.

Please note that it is not sufficient to attach the dummy close() once during a start-up phase. I found that it is necessary to do it before each call to multiprocessing.Process(); I didn't try it with process pools, though.