sklearn selectKbest: which variables were chosen?

Question

I'm trying to get sklearn to select the best k variables (for example k=1) for a linear regression. This works and I can get the R-squared, but it doesn't tell me which variables were the best. How can I find that out?

I have code of the following form (real variable list is much longer):

X=[]
for i in range(len(df)):
X.append([averageindegree[i],indeg3_sum[i],indeg5_sum[i],indeg10_sum[i])


training=[]
actual=[]
counter=0
for fold in range(500):
    X_train, X_test, y_train, y_test = crossval.train_test_split(X, y, test_size=0.3)
    clf = LinearRegression()
    #clf = RidgeCV()
    #clf = LogisticRegression()
    #clf=ElasticNetCV()

    b = fs.SelectKBest(fs.f_regression, k=1) #k is number of features.
    b.fit(X_train, y_train)
    #print b.get_params

    X_train = X_train[:, b.get_support()]
    X_test = X_test[:, b.get_support()]


    clf.fit(X_train,y_train)
    sc = clf.score(X_train, y_train)
    training.append(sc)
    #print "The training R-Squared for fold " + str(1) + " is " + str(round(sc*100,1))+"%"
    sc = clf.score(X_test, y_test)
    actual.append(sc)
    #print "The actual R-Squared for fold " + str(1) + " is " + str(round(sc*100,1))+"%"

Answer 1

You need to use get_support:

features_columns = [.......]
fs = SelectKBest(score_func=f_regression, k=5)
print zip(fs.get_support(),features_columns)