On-line regression On-line learning algorithms are not restricted to classification problems. The update rule for the kernel adatron algorithm also suggests a general methodology for creating on-line versions of the optimisations.
making the first update of the kernel adatron algorithm equivalent to αi ← αi + ∂W(α) ∂αi making it a simple gradient ascent algorithm augmented with corrections to ensure that the additional constraints are satisfied. If, for example, we apply this same approach to the linear ε-insensitive loss version of the support vector regression algorithm.
One of the advantages of Support Vector Machine, and Support Vector Regression as the part of it, is that it can be used to avoid difficulties of using linear functions in the high dimensional feature space and optimization problem is transformed into dual convex quadratic programmes. In regression case the loss function is used to penalize errors that are grater than threshold - . Such loss functions usually lead to the sparse representation of the decision rule, giving significant algorithmic and representational advantages.

Reference:
Kernel Methods for Pattern Analysis byJohn Shawe-Taylor & Nello Cristianini
http://kernelsvm.tripod.com/