Time series forecasting with support vector regression

Question

I'm trying to perform a simple time series prediction using support vector regression.

I am trying to understand the answer provided here.

I adapted Tom's code to reflect the answer provided:

import numpy as np
from matplotlib import pyplot as plt
from sklearn.svm import SVR

X = np.arange(0,100)
Y = np.sin(X)

a = 0
b = 10
x = []
y = []
while b <= 100:
    x.append(Y[a:b])
    a += 1
    b += 1
b = 10

while b <= 90:
    y.append(Y[b])
    b += 1

svr_rbf = SVR(kernel='rbf', C=1e5, gamma=1e5)
y_rbf = svr_rbf.fit(x[:81], y).predict(x)

figure = plt.figure()
tick_plot = figure.add_subplot(1, 1, 1)
tick_plot.plot(X, Y, label='data', color='green', linestyle='-')
tick_plot.axvline(x=X[-10], alpha=0.2, color='gray')
tick_plot.plot(X[10:], y_rbf[:-1], label='data', color='blue', linestyle='--')
plt.show()

However, I still get the same behavior -- the prediction just returns the value from the last known step. Strangely, if I set the kernel to linear the result is much better. Why doesn't the rbf kernel prediction work as intended?

Thank you.

Answer 1

I understand this is an old question, but I will answer it as other people might benefit from the answer.

The values you are using for C and gamma are most likely the issue if your example works with a linear kernel and not with rbf. C and gamma are SVM parameters used for nonlinear kernel. For a goodexplanation of what C and gamma are intuitively, have a look here: http://scikit-learn.org/stable/auto_examples/svm/plot_rbf_parameters.html?

In order to predict the values of a sinusoid, try C = 1 and gamma = 0.1. It performs much better than with the values you have.